Analyzing Email Headers For Phishing

Common Pitfalls

• Relying on partial header snippets only.
  In observed runs, quick grep passes were useful, but full-header follow-up was needed to confirm auth failures and spoofing details.
• Missing data due to terminal output truncation.
  Combined dumps can be truncated; inspect individual files when output is clipped.
• Under-classifying because only one indicator is checked.
  Example pattern: messages with suspicious domains + bad relays + auth failures are clearer than any single signal alone.
• Schema mistakes despite correct analysis.
  Forgetting one email, duplicating filenames, or mismatching suspicious_count can fail tests even if reasoning is correct.

Verification Strategy

Run verification aligned to typical grader assertions:

1. File existence

   • Confirm /solution/phishing_analysis.json exists.

2. JSON validity

   • Run python3 -m json.tool /solution/phishing_analysis.json.

3. Required fields and types

   • Ensure keys exist and types are:
     • arrays for phishing_emails, legitimate_emails
     • integer for suspicious_count

4. Coverage and uniqueness

   • Check union of both arrays equals the full /emails file set.
   • Check no duplicates across or within arrays.

5. Count consistency

   • Verify suspicious_count == len(phishing_emails).

6. Classification confidence check

   • Re-open any borderline file and confirm SPF/DKIM/DMARC + identity alignment before finalizing.

References to Load On Demand

• RFC 7208 (SPF), RFC 6376 (DKIM), RFC 7489 (DMARC)
• Practical header triage commands:
  • Bulk key-field extraction loop
  • sed -n '1,/^$/p' for full header block