Gmail

Open and read an email

1. left_click on the email row to open it
2. Take a screenshot to read the email content
3. Use get_page_text to extract the full email body text
4. Note the sender, subject, date, and any amounts mentioned

Extract PDF attachments

1. In an open email, use find with query "download attachment" or "PDF attachment" to locate attachment elements
2. Attachments appear at the bottom of an email - look for file cards with download icons
3. Use zoom on the attachment area if icons are small
4. To download: left_click on the download icon (down arrow) on the attachment
5. Important: Always confirm with the user before downloading any file

Read PDF content inline

1. Some PDFs can be previewed - left_click on the attachment thumbnail to open preview
2. Take a screenshot of the preview to read invoice details
3. Extract: vendor/sender name, invoice number, date, due date, line items, total amount

Composing and Sending Emails

Compose a new email

1. Use find with query "Compose button" to locate the Compose button (usually bottom-left)
2. left_click the Compose button
3. Wait briefly, then screenshot to confirm the compose window opened

Fill email fields

1. Use find with query "To field" or "recipient field" to locate the To input
2. left_click on the To field, then type the recipient email address, press key "Tab" to confirm
3. Use find with query "Subject field" to locate Subject
4. left_click on Subject, then type the subject text
5. left_click on the email body area (below subject), then type the message body

Attach a file

1. Use find with query "attach files" or "attachment button" to find the paperclip icon in the compose toolbar
2. left_click on the attach button - this opens a file picker
3. Use upload_image with the file reference if uploading a previously downloaded file
4. Alternatively, if a file input element is available, use find with "file input" and then upload_image with the ref

Send the email

1. Use find with query "Send button" in the compose window
2. Always confirm with the user before clicking Send - state the recipient, subject, and confirm they want to send
3. After confirmation, left_click the Send button
4. Take a screenshot to confirm the email was sent (look for "Message sent" notification)

Key Gmail UI Patterns

• Inbox: mail.google.com/#inbox
• Search: Click the search bar at top, type query, press Enter
• Compose: Big button bottom-left of sidebar
• Attachments in compose: Paperclip icon in compose toolbar (bottom of compose window)
• Send button: Blue "Send" button bottom-left of compose window
• Attachment downloads: Hover over attachment card, click the down-arrow icon
• Navigation: Use sidebar links (Inbox, Sent, Drafts, etc.)
• Starred/Important: Sidebar links or search operators like is:starred

Tips

• Gmail loads dynamically - always take a screenshot after navigation to confirm the page state
• Use get_page_text for bulk text extraction from email bodies
• If elements are hard to find, use read_page with filter: "interactive" to see all clickable elements
• For multiple emails, process them one at a time - open, extract, go back to inbox, repeat
• Use key with "Escape" to close dialogs or go back