Skipclass

• data/video/ - Raw lecture recordings (mp4/mov/etc.)
• data/audio/ - Extracted audio files (wav)
• data/transcripts/ - Text transcripts from audio
• data/class_slides/ - Lecture slide PDFs
• data/notes/ - Final lecture notes and slide matches

Create missing folders before running scripts. Keep file names consistent across stages (video -> audio -> transcript) to simplify matching.

Scripts and Their Roles

• scripts/video2audio.py - Convert videos to audio with optional silence removal.
• scripts/audio2text.py - Transcribe audio using faster-whisper; supports chunking.
• scripts/find_matched_slides.py - Retrieve top-matching slide PDFs for a transcript.

Step-by-Step Workflow

1) Inspect and Prepare the Workspace

Check that data/ exists and has required subfolders. Create any missing folders.

Confirm that dependencies are available:

• ffmpeg and ffprobe on PATH.
• Python packages: faster-whisper, torch (if using Silero VAD), pymupdf, sentence-transformers.

2) Place Input Files

• Put lecture videos in data/video/.
• Put lecture slides (PDFs) in data/class_slides/.
• If using a pre-existing transcript, place it in data/transcripts/ (or data/rawtexts/ for legacy data).

Prefer consistent naming, e.g. week2_lecture.mp4 -> week2_lecture.wav -> week2_lecture.txt.

3) Run Video -> Audio

Convert lecture videos to audio files:

uv run scripts/video2audio.py \
  --input-dir data/video \
  --output-dir data/audio \
  --silence-backend silero

4) Run Audio -> Text

Transcribe audio into text:

uv run scripts/audio2text.py \
  --input-dir data/audio \
  --output-dir data/transcripts \
  --model base \
  --chunk-minutes 30 \
  --chunk-overlap-minutes 0.2 \
  --parallel-workers 2 \
  --language en \
  --device cpu \
  --compute-type int8

5) Verify Transcript Quality

Open the generated transcript in data/transcripts/ and scan for:

• Missing segments (check for long gaps or repeated phrases).
• Poor language detection (wrong language).
• Excessive silence removal (too short transcript).

If needed, re-run with adjusted parameters:

• Increase --min-silence-duration or lower --silence-threshold-db.
• Change --model to a larger Whisper model.
• Increase --chunk-minutes for long lectures.

6) Match Slides to Transcript

Use the slide matching script:

uv run scripts/find_matched_slides.py \
  --mode hybrid \
  --hybrid-weight 0.7 \
  --transcript data/transcripts/week2_lecture.txt \
  --slides-dir data/class_slides \
  --top-k 5

Record the results in data/notes/ for reference and use the top results as context when summarizing.

7) Generate Lecture Notes

Combine the transcript and the most relevant slides into a concise English note that follows the structure and tone of example_note.md. Keep the output short, factual, and actionable. Use markdown headings and bullet points where appropriate.

Required Output Structure

1. Lesson Recap

   • Primarily summarize the slides content.
   • Organize by sections/topics as in the slides (use numbered sections or subheadings if helpful).
   • Include key points, definitions, examples, and important formulas/processes from slides.

2. Key Takeaways By Teacher

   • Synthesize Lesson Recap with the teacher’s spoken emphasis from the transcript.
   • Extract higher-level insights, rationale, or cautions the teacher stressed.
   • Avoid repeating the Lesson Recap verbatim; focus on distilled, teacher-driven emphasis.

3. Important Notification

   • Exams/quiz arrangements.
   • Assignment details and deadlines (include explicit dates).
   • Project milestones or schedule changes.

4. TODO

   • Direct instructions for a student who skipped class.
   • Required readings, problem sets, or code tasks.
   • Mention how to verify understanding (e.g. "work through slide X").

5. Attendance Check

   • State whether attendance was checked.
   • If unknown, state "Not mentioned in recording."

8) Save the Final Output

Write the final notes to data/notes/<lecture_name>_notes.md. Optionally store slide matches to data/notes/<lecture_name>_slides.txt.

Notes Quality Rules

• Keep each section concise and scannable.
• Prefer concrete terms over generic summaries.
• Avoid fabricating deadlines or announcements; if not found, state "Not mentioned."
• Convert relative time phrases into absolute dates when possible.

Troubleshooting

• ffmpeg/ffprobe not found: Install ffmpeg and ensure it is on PATH.
• faster-whisper import error: Install faster-whisper and its dependencies.
• Slide matching slow: Reduce PDF count or use --mode keyword.
• Slide matching inaccurate: Use --mode hybrid and tune --hybrid-weight.

Additional Resources

Scripts

• scripts/video2audio.py - Video to audio conversion and silence removal.
• scripts/audio2text.py - Faster-whisper transcription with chunking.
• scripts/find_matched_slides.py - Slide matching with vector/keyword/hybrid retrieval.