Programmatic video pipeline (evaluation)
VideoLab
A local-first, scriptable video editing pipeline: transcript and scene analysis, a reviewable edit list, captions, animated overlays, GPU rendering and automated quality checks.
- Status
- Experimental evaluation build. Private.
- Source
- Source: private
The problem
Cutting talking-head footage by hand is repetitive: removing filler words, pauses and retakes, adding captions and titles, normalising audio, then checking the result.
The goal was to test how far a plain set of command-line steps can take that workflow while staying fully local, with nothing uploaded and a human-reviewable edit decision at the centre.
The approach
Each stage is a plain command-line script run in order, with no orchestration framework or background service.
Analysis transcribes the footage on the GPU with a Whisper model and detects scene cuts with FFmpeg. A reviewable edit decision list (a JSON file) records what is kept, what is removed and why.
Captions are generated as styled subtitles, overlays such as titles and lower thirds are rendered by an isolated Remotion project, and the final render uses FFmpeg with NVENC hardware encoding and loudness normalisation.
A QC stage checks the output: streams and codec, duration against the edit list, black or frozen frames, long silences, loudness and true peak, and caption timing against a fresh transcription.
Architecture
Text alternative: the flow in words
- Footage is transcribed on the GPU and scanned for scene cuts.
- The result becomes a reviewable edit decision list with reasons for every removal.
- Captions and overlays are produced from that list.
- FFmpeg renders the final video with NVENC and loudness normalisation.
- QC and visual QA produce a report. Nothing is corrected automatically.
Interactive demo
Demo with fictional data
Step through the pipeline, then try the frame QA check on a made-up frame. The contrast figure is a real WCAG calculation; no media is used.
Analyze
output: transcript + scene cuts
The footage is transcribed with word timings and scanned for scene cuts. Nothing is cut yet.
Key technical decisions
Reviewable edit list as the contract
Decisions live in a versioned JSON file with a reason for every removed segment, so a person can inspect and change them before rendering.
Fully local
No cloud GPUs and no footage uploads. A local voice model handles narration, offline.
Narration quality loop with a bound
Narration lines are transcribed and scored against the script. An optional corrector regenerates only failing lines, at most two rounds, reuses every other line byte-for-byte and never writes into the project folder.
Automated visual QA
Deterministic checks on what is actually on screen: clipping, overlap, safe-area margins, contrast against rendered pixels (WCAG thresholds), numbers against a source-of-truth file, and caption presence. Report only: nothing is silently corrected.
Safety and security notes
- Voice and media samples are not part of this site and are not described here.
- On-screen numbers can be tagged as illustrative or business facts, and unapproved business facts are flagged for confirmation before use.
Limits and honest caveats
- Evaluation only. Remotion licensing terms need to be confirmed before any commercial use of its output.