Find Jobs

Read each source guide in sources/ before scraping:

Phase 1: Scrape All Sources

For each source, read the source guide and follow its scraping instructions. Collect a list of jobs with:

• company — company name
• role — job title
• location — job location
• url — direct application URL (strip tracking params)
• age — days since posted (0 = today)
• source — which source it came from

If a source fails (blocked, down, CAPTCHA), log the failure and continue with other sources.

Performance rule: Extract all jobs from a search results page in ONE eval call per query. Never click individual job cards — that's too slow. The source guides have the exact eval scripts.

Phase 2: Deduplicate

1. Normalize company names: lowercase, strip "Inc", "Ltd", "Corp", "LLC", etc.
2. Check against jobs.db — skip jobs discovered within the last 14 days (by URL or company+role match). Jobs older than 14 days are allowed to be re-discovered.
3. If same job found from multiple sources in this run, keep the one with the direct ATS URL.
4. Cross-check against jobditto.com applied jobs — skip if already applied within 14 days.

Phase 3: Filter

Remove jobs where:

• Role title contains 🛂 or 🇺🇸
• Role title or snippet mentions "no sponsorship", "US citizen", "clearance", "TS/SCI", "public trust"
• Role is contract/temp (unless user opts in)
• Role level is Senior, Staff, Principal, Lead, or Manager
• Experience requirement > 3 years (if detectable from snippet)

Phase 4: Rank (top 5)

Score each job. Priority order:

1. Recency (highest weight) — posted today: +10, yesterday: +8, 2 days: +6, 3 days: +4, older: +0
2. Skills match — +1 for each of Simon's skills in the role title or snippet: TypeScript, JavaScript, React, Next.js, Node.js, Express, FastAPI, Go, Python, PostgreSQL, MySQL, Docker, AWS, Terraform, Playwright
3. Company tier — known tech companies: +3, other companies: +0 (Tier list: Google, Meta, Amazon, Apple, Microsoft, Stripe, Airbnb, Netflix, Uber, Lyft, Coinbase, Robinhood, Databricks, Snowflake, Cloudflare, Vercel, Figma, Discord, Twitch, Reddit, Pinterest, Snap, Block, Palantir, Ramp, Brex, Scale AI, Anthropic, OpenAI)

Take top 10 by total score. If fewer than 10 pass filters, take what's available. (Target: 20 applications per day across 2 runs = 10 per run.)

Phase 5: Hand Off to Mass-Apply

If jobs found:

"Found {N} new matching jobs. Starting mass-apply..."
→ /mass-apply with the job URLs

If no jobs found:

"No new matching jobs found. Will check again at next scheduled run."

State Persistence (SQLite)

Database: ~/work/personal/jobs/.claude/jobs.db

Initialize if missing:

python3 ~/work/personal/jobs/.claude/scripts/init_jobs_db.py --base-path ~/work/personal/jobs

Tables

discovered_jobs — every job found from any source:

• company, company_normalized, role, location, url (unique)
• source, discovered_at, age_days
• skills_score, company_tier, total_score
• description_snippet, status (new/applied/skipped/rejected)

scrape_runs — log each cron run:

• started_at, completed_at, sources_checked, jobs_found, jobs_filtered, jobs_applied

scrape_state — per-source state (e.g., SimplifyJobs last SHA):

• source, key, value

Deduplication query

Only skip jobs discovered within the last 14 days. If a job was seen 14+ days ago, allow re-discovery — the position may still be open and worth re-applying.

SELECT 1 FROM discovered_jobs
WHERE (url = ? OR (company_normalized = ? AND role = ?))
  AND discovered_at > datetime('now', '-14 days')

After each run

-- Log the run
INSERT INTO scrape_runs (sources_checked, jobs_found, jobs_filtered, jobs_applied)
VALUES (?, ?, ?, ?);

-- Insert new jobs (ignore duplicates)
INSERT OR IGNORE INTO discovered_jobs (company, company_normalized, role, location, url, source, age_days, skills_score, company_tier, total_score, description_snippet)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?);

-- Update status for applied jobs
UPDATE discovered_jobs SET status = 'applied', applied_at = datetime('now') WHERE url = ?;

-- Update source state
INSERT OR REPLACE INTO scrape_state (source, key, value, updated_at)
VALUES ('simplify', 'last_sha', ?, datetime('now'));

Error Handling

• Source blocked/down → skip source, continue with others. Log which sources failed.
• LinkedIn session expired → Discord: "Log into LinkedIn in Chrome." Skip LinkedIn for this run.
• Glassdoor CAPTCHA → Discord notification, wait for solve, continue.
• < 5 jobs found → apply to however many passed filters. If 0, log and skip mass-apply.
• jobditto API down → can't deduplicate. Skip entire run, retry at next schedule.
• All sources fail → Discord: "Job search failed — all sources blocked. Check browser sessions."

Status Updates

• "Starting job search across {N} sources..."
• "SimplifyJobs: found {N} new listings"
• "LinkedIn: found {N} matching jobs"
• (per source)
• "After filtering: {N} unique jobs"
• "Top 5 matches: {list with company + role}"
• "Handing off to mass-apply..."