Daily single — testing guide

Daily single — testing guide

This page explains every kind of test used in the daily single video pipeline: what it checks, when to run it, and where results are saved. Written for operators and contributors — no need to read the Python source first.

Three layers

┌─────────────────────────────────────────────────────────────┐
│  Layer 1 — Unit tests (pytest, no video project required)   │
├─────────────────────────────────────────────────────────────┤
│  Layer 2 — Modular QA (validate-qa / video_qa stages)       │
├─────────────────────────────────────────────────────────────┤
│  Layer 3 — Legacy publish gates (validate-sync, validate-all)│
└─────────────────────────────────────────────────────────────┘

Run Layer 1 after code changes. Run Layer 2 after each pipeline phase. Run Layer 3 before upload — or rely on stage s10-final-composite, which runs the same checks.

Layer 1 — Unit tests (pytest)

Fast, offline tests. No API keys required for the core QA module tests.

conda activate test
cd /path/to/praisonaippt

# Minimal — video_qa module only
pytest tests/test_video_qa.py -q

# Full daily_single suite
pytest tests/test_daily_single_display_sync_unit.py \
       tests/test_daily_single_sync_validation.py \
       tests/test_daily_single_hook_montage.py \
       tests/test_daily_single_media_sync.py \
       tests/test_daily_single_visual_audit.py \
       tests/test_daily_single_youtube_quality.py \
       tests/test_daily_single_captions.py \
       tests/test_video_qa.py -q

!!! tip “When to run” Run the full pytest suite before merging changes to praisonaippt/daily_single/ or praisonaippt/video_qa/.

Layer 2 — Modular QA (validate-qa)

Module: praisonaippt/video_qa/
CLI: daily-single -p $PROJECT validate-qa or python -m praisonaippt.video_qa --project $PROJECT run

Each stage runs independently and writes a JSON report under merge/qa/. A rollup lives in merge/qa/summary.json.

When to run

daily-single -p $PROJECT validate-qa --when pre_build
daily-single -p $PROJECT validate-qa --when post_build

# Single stage debug
daily-single -p $PROJECT validate-qa s08-av-sync
python -m praisonaippt.video_qa --project $PROJECT list

Stage reference (what each test does)

Full stage config: Video QA.

Degradation (warn, not fail)

Some environments cannot run every check. The suite records flags in summary.json:

Set PRAISONAIPPT_QA_OFFLINE=1 in CI to skip API-dependent stages.

Layer 3 — Legacy publish gates

These pre-date the modular video_qa package but remain the authoritative publish bar. Stage s10 runs them automatically; you can also run them standalone.

validate-display

Maps every SRT cue to the visual shown at the cue midpoint.

Output: merge/display_sync_report.json

daily-single -p $PROJECT validate-display

validate-spoken-visual

Stricter full-video gate: montage fragments, slide windows (worst overlapping cue), chart/plain-language checks, transition samples at every image change, coverage, and plain-language rules.

Output: merge/spoken_visual_sync_report.json — require "ok": true before publish.

daily-single -p $PROJECT validate-spoken-visual

Cue-aligned rebuild (beat-06, beat-01 views): build-captions → assemble-beats → validate-display → validate-spoken-visual. Skill: .cursor/skills/daily-single-video-pipeline/spoken-visual-sync.md

validate-slide-quality / validate-engagement-assets / validate-viral-readiness

Professional and viral publish gates (trust-audit uses stricter thresholds via variant: trust-audit in beat-map).

Full matrix: .cursor/skills/daily-single-video-pipeline/scripts/run-publish-gate.sh

Unit tests:

pytest tests/test_cue_slide_sync.py tests/test_spoken_visual_sync.py \
       tests/test_slide_design_audit.py tests/test_engagement_audit.py \
       tests/test_viral_readiness.py tests/test_video_qa.py -q

validate-sync --runs 3

Runs the full spoken↔visual suite three times and requires identical results (idempotency).

Output: merge/sync_validation_report.json

daily-single -p $PROJECT validate-sync --runs 3

audit-visual

Samples merge/final.mp4 every 5 seconds (plus cue midpoints). Compares frames to planned assets.

Optional vision LLM (gpt-4o-mini) flags off-topic or generic B-roll.

Output: merge/visual_audit_report.json, frames in merge/visual_audit_frames/

daily-single -p $PROJECT audit-visual --interval 5
daily-single -p $PROJECT validate-visual-audit

validate-all

Single publish gate combining tools, output specs, media inventory, and all reports above.

Output: validation_report.json (project root)

daily-single -p $PROJECT validate-all

Recommended test workflow (full rebuild)

Use this checklist when building a video step by step:

PROJECT=examples/videos/<slug>

# 1 — After scripts + handoff
daily-single -p $PROJECT validate-qa --when pre_build

# 2 — After voice-over
daily-single -p $PROJECT validate-qa --when post_vo

# 3 — After HeyGen bookends
daily-single -p $PROJECT validate-qa --when pre_assemble

# 4 — After captions + assemble (cue-aligned order)
daily-single -p $PROJECT build-captions
daily-single -p $PROJECT assemble-beats
daily-single -p $PROJECT validate-display
daily-single -p $PROJECT validate-spoken-visual
pytest tests/test_cue_slide_sync.py tests/test_spoken_visual_sync.py -q

# 5 — Main modular gate
daily-single -p $PROJECT validate-qa --when post_build

# 6 — Confirm legacy gates (optional if s10 passed)
daily-single -p $PROJECT validate-all
daily-single -p $PROJECT validate-sync --runs 3

# 7 — After code changes only
pytest tests/test_video_qa.py tests/test_daily_single_sync_validation.py \
       tests/test_cue_slide_sync.py tests/test_spoken_visual_sync.py -q

Output files (where to look when something fails)

Related

• Daily single video pipeline
• Video QA stages
• Commands — daily single
• Spoken↔visual workflow skill