Singing Segment Detector

Automatically detect singing segments in live stream recordings and export timeline markers for DaVinci Resolve.

What It Does

4hr stream recording (.mp4/.mkv/.flv)
    │
    ▼  [ffmpeg: extract 16kHz mono audio]
    │
    ▼  [inaSpeechSegmenter: classify speech/music/noise]
    │
    ▼  [merge nearby music segments, filter short ones]
    │
    ├──→ .edl        (import as DaVinci Resolve timeline)
    ├──→ _markers.csv (import as DaVinci Resolve markers)
    ├──→ .csv         (human-readable segment list)
    ├──→ .json        (programmatic use)
    └──→ .bat         (optional: auto-cut with ffmpeg, no editing needed)

Key insight: inaSpeechSegmenter classifies singing voice as "music". So in a typical singing stream, talking = "speech", singing = "music". We filter for "music" segments and merge nearby ones (a song might have brief pauses between verses).

Setup (Windows with conda)

Prerequisites

• conda (Anaconda or Miniconda)
• ffmpeg in your PATH
• NVIDIA GPU recommended (RTX 2080 works great) but CPU also works

Step 1: Create conda environment

conda create -n singing-detector python=3.11 -y
conda activate singing-detector

Step 2: Install PyTorch with CUDA (for GPU acceleration)

# For RTX 2080 (CUDA 12.x)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

Step 3: Install TensorFlow (required by inaSpeechSegmenter)

pip install tensorflow[and-cuda]

Or CPU-only (slower but works):

pip install tensorflow

Step 4: Install inaSpeechSegmenter and dependencies

pip install inaSpeechSegmenter

Step 5: Verify installation

python -c "from inaSpeechSegmenter import Segmenter; print('OK!')"

Usage

Single file

conda activate singing-detector

# Basic usage
python detect_singing.py "D:\streams\2026-04-04_stream.mp4"

# Custom settings
python detect_singing.py "D:\streams\2026-04-04_stream.mp4" --gap 15 --min-duration 30

# Output to a specific directory
python detect_singing.py "D:\streams\2026-04-04_stream.mp4" -o "D:\singing_edits"

# Also generate an auto-cut ffmpeg script (no DaVinci needed)
python detect_singing.py "D:\streams\2026-04-04_stream.mp4" --auto-cut

# For 4K F-log footage at 29.97fps (your XT3 settings)
python detect_singing.py "D:\streams\2026-04-04_stream.mp4" --fps 29.97

Batch processing

# Process all videos in a folder
python batch_detect.py "D:\streams"

# Process new files only (skip already-done ones)
python batch_detect.py "D:\streams" --skip-existing

# Process and auto-generate singing-only videos
python batch_detect.py "D:\streams" --auto-cut -o "D:\singing_edits"

# Only process .flv files
python batch_detect.py "D:\streams" --pattern "*.flv"

Importing Results into DaVinci Resolve

Option A: Import EDL as Timeline (recommended)

1. Open your project in DaVinci Resolve
2. Import the original stream video into your Media Pool
3. Go to File → Import → Timeline
4. Select the _singing.edl file
5. DaVinci will create a new timeline with only the singing segments

Option B: Import Markers CSV

1. Create a timeline from your stream video
2. Right-click on the timeline → Timelines → Import Markers from CSV
3. Select the _markers.csv file
4. Blue markers will appear at each singing segment start/end

Option C: Auto-Cut (no DaVinci needed)

If you used --auto-cut, just run the generated .bat script:

D:\singing_edits\2026-04-04_stream_singing.bat

This uses ffmpeg to directly cut and concatenate all singing segments into a single _singing_only.mp4 file. Fastest option, but no manual review.

Tuning Parameters

Parameter	Default	Description
--gap	10s	Max gap between music segments to merge. Increase if singer pauses >10s between verses.
--min-duration	20s	Minimum segment length. Lower if singer does very short songs.
--padding	2s	Extra seconds before/after each segment. Helps catch song intro/outro.
--fps	29.97	Frame rate for timecode. Use 25 for PAL.

Recommended settings for typical singing streams

• Singer with lots of chatting between songs: --gap 10 --min-duration 30
• Singer with minimal breaks: --gap 5 --min-duration 20
• Singer who does short covers/snippets: --gap 8 --min-duration 15
• Conservative (catch everything): --gap 20 --min-duration 15 --padding 5

Troubleshooting

"Singing voice detected as speech"

This can happen if the singer talks over background music. Try --gap 15 to merge nearby segments.

"Too many false positives (non-singing music detected)"

If there's background music during chatting, increase --min-duration 45 to only keep longer segments (full songs).

"Segments cut off the beginning/end of songs"

Increase --padding 5 to add more buffer.

GPU not being used

Check that TensorFlow detects your GPU:

python -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"

Processing is very slow

• A 4-hour stream typically takes 5-15 minutes on GPU, 30-60 minutes on CPU.
• Make sure you're using GPU-enabled TensorFlow.
• Close other GPU-heavy applications during processing.

Output Files

For input stream_2026-04-04.mp4, you get:

File	Purpose
stream_2026-04-04_singing.edl	DaVinci Resolve timeline import
stream_2026-04-04_markers.csv	DaVinci Resolve marker import
stream_2026-04-04_singing.csv	Human-readable segment list
stream_2026-04-04_singing.json	Programmatic use
stream_2026-04-04_singing.bat	Auto-cut script (with --auto-cut)
stream_2026-04-04_raw_segments.csv	All segments for debugging (with --raw-segments)