Find Package

If the user sends an image, use your vision capabilities to read the pickup code from it. Look for patterns like:

• 取件码：5-2-1234
• 货架号：5 取件码：1234
• 格口：5-2-1234

Confirm the extracted code with the user before proceeding: "我识别到的取件码是 5-2-1234，对吗？"

Step 2: Get shelf photos (货架照片)

Once you have the confirmed pickup code, ask the user to take photos of the package shelves. Tell them:

"请拍一下货架的照片发给我，可以一次发多张～"

The user may send:

• A single photo of one shelf section
• Multiple photos covering different shelf sections

Step 3: Recognize and match (OCR + annotate)

For each shelf photo, use the one-stop script that runs OCR detection, matches pickup codes, and annotates the image:

python3 {baseDir}/scripts/find_package.py \
  --input /path/to/shelf_photo.jpg \
  --code "9347" \
  --output /tmp/find-package-result.jpg

For multiple pickup codes at once:

python3 {baseDir}/scripts/find_package.py \
  --input /path/to/shelf_photo.jpg \
  --code "9347,0295,1339" \
  --output /tmp/find-package-result.jpg

The script outputs JSON to stdout with match results:

{
  "total_codes": 3,
  "found": 2,
  "not_found": 1,
  "matched": [
    {"text": "9347", "bbox": [45, 280, 130, 340], "confidence": 0.95, "matched_code": "9347"},
    {"text": "0295", "bbox": [200, 310, 350, 360], "confidence": 0.92, "matched_code": "0295"}
  ],
  "unmatched_codes": ["1339"],
  "output_image": "/tmp/find-package-result.jpg"
}

If you need finer control, the two steps can be run separately:

# Step 1: OCR detect
python3 {baseDir}/scripts/ocr_detect.py \
  --input /path/to/shelf_photo.jpg \
  --output /tmp/detections.json

# Step 2: Annotate matches
python3 {baseDir}/scripts/annotate.py \
  --input /path/to/shelf_photo.jpg \
  --output /tmp/find-package-result.jpg \
  --detections /tmp/detections.json \
  --match "9347"

After getting the annotated image, send it back to the user via the message tool.

Step 4: Report results

When a package is found:

• Send the annotated photo with the red bounding box
• Say something like: "找到了！你的快递在这里，取件码 5-2-1234"

If the user has multiple pickup codes (they mentioned several or you detected multiple in the screenshot):

• Track which ones have been found and which are still missing
• After each shelf photo, report: "已找到 2/3 个快递，还有 1 个没找到（取件码：7-3-5678）"

When all packages are found:

• "全部找到了！祝取件顺利～"

When no match found after all photos:

• "在这些照片里没有找到你的快递，要不要再拍几张其他货架的照片？"

Important Notes

• Always communicate in Chinese — this feature is for Chinese delivery stations (驿站/快递柜)
• Be patient — users may be unfamiliar with taking clear shelf photos. If OCR is unclear, ask them to retake with better lighting or angle
• The pickup code format varies by delivery company. Common formats:
  • X-Y-ZZZZ (e.g., 5-2-1234)
  • Just numbers on a label
  • QR codes (if you can't read QR, tell the user to provide the text code instead)
• The red bounding boxes are drawn by the OCR engine with precise coordinates — do not guess coordinates yourself
• If the shelf photo is blurry or codes are unreadable, ask for a clearer photo rather than guessing

Sending Messages

Use the message tool with channel: "telegram":

{
  "action": "send",
  "channel": "telegram",
  "message": "请发一下你的取件码～可以直接打字，也可以截图给我"
}

Send with annotated image:

{
  "action": "send",
  "channel": "telegram",
  "message": "找到了！你的快递在红框标记的位置",
  "media": "file:///tmp/find-package-result.jpg"
}