youfu/sing_detect
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v4: batch processing layer over the CLI engine

Browse Source 
Wraps detect_singing to process whole folders of recordings at once: --recursive, --pattern, --output-dir, --auto-cut.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
youfufan
2026-01-04 10:00:00 +00:00
parent 91df8d3cec
commit 98a4d37162
3 changed files with 183 additions and 241 deletions
Download Patch File Download Diff File Expand all files Collapse all files

185
README.md
View File

@@ -1,185 +0,0 @@
# Singing Segment Detector
Automatically detect singing segments in live stream recordings and export timeline markers for DaVinci Resolve.
## What It Does
```
4hr stream recording (.mp4/.mkv/.flv)
│
▼ [ffmpeg: extract 16kHz mono audio]
│
▼ [inaSpeechSegmenter: classify speech/music/noise]
│
▼ [merge nearby music segments, filter short ones]
│
├──→ .edl (import as DaVinci Resolve timeline)
├──→ _markers.csv (import as DaVinci Resolve markers)
├──→ .csv (human-readable segment list)
├──→ .json (programmatic use)
└──→ .bat (optional: auto-cut with ffmpeg, no editing needed)
```
**Key insight**: `inaSpeechSegmenter` classifies **singing voice as "music"**. So in a typical singing stream, talking = "speech", singing = "music". We filter for "music" segments and merge nearby ones (a song might have brief pauses between verses).
## Setup (Windows with conda)
### Prerequisites
- **conda** (Anaconda or Miniconda)
- **ffmpeg** in your PATH
- **NVIDIA GPU** recommended (RTX 2080 works great) but CPU also works
### Step 1: Create conda environment
```powershell
conda create -n singing-detector python=3.11 -y
conda activate singing-detector
```
### Step 2: Install PyTorch with CUDA (for GPU acceleration)
```powershell
# For RTX 2080 (CUDA 12.x)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
```
### Step 3: Install TensorFlow (required by inaSpeechSegmenter)
```powershell
pip install tensorflow[and-cuda]
```
Or CPU-only (slower but works):
```powershell
pip install tensorflow
```
### Step 4: Install inaSpeechSegmenter and dependencies
```powershell
pip install inaSpeechSegmenter
```
### Step 5: Verify installation
```powershell
python -c "from inaSpeechSegmenter import Segmenter; print('OK!')"
```
## Usage
### Single file
```powershell
conda activate singing-detector
# Basic usage
python detect_singing.py "D:\streams\2026-04-04_stream.mp4"
# Custom settings
python detect_singing.py "D:\streams\2026-04-04_stream.mp4" --gap 15 --min-duration 30
# Output to a specific directory
python detect_singing.py "D:\streams\2026-04-04_stream.mp4" -o "D:\singing_edits"
# Also generate an auto-cut ffmpeg script (no DaVinci needed)
python detect_singing.py "D:\streams\2026-04-04_stream.mp4" --auto-cut
# For 4K F-log footage at 29.97fps (your XT3 settings)
python detect_singing.py "D:\streams\2026-04-04_stream.mp4" --fps 29.97
```
### Batch processing
```powershell
# Process all videos in a folder
python batch_detect.py "D:\streams"
# Process new files only (skip already-done ones)
python batch_detect.py "D:\streams" --skip-existing
# Process and auto-generate singing-only videos
python batch_detect.py "D:\streams" --auto-cut -o "D:\singing_edits"
# Only process .flv files
python batch_detect.py "D:\streams" --pattern "*.flv"
```
## Importing Results into DaVinci Resolve
### Option A: Import EDL as Timeline (recommended)
1. Open your project in DaVinci Resolve
2. Import the original stream video into your Media Pool
3. Go to **File → Import → Timeline**
4. Select the `_singing.edl` file
5. DaVinci will create a new timeline with only the singing segments
### Option B: Import Markers CSV
1. Create a timeline from your stream video
2. Right-click on the timeline → **Timelines → Import Markers from CSV**
3. Select the `_markers.csv` file
4. Blue markers will appear at each singing segment start/end
### Option C: Auto-Cut (no DaVinci needed)
If you used `--auto-cut`, just run the generated `.bat` script:
```powershell
D:\singing_edits\2026-04-04_stream_singing.bat
```
This uses ffmpeg to directly cut and concatenate all singing segments into a single `_singing_only.mp4` file. Fastest option, but no manual review.
## Tuning Parameters
| Parameter | Default | Description |
|-----------|---------|-------------|
| `--gap` | 10s | Max gap between music segments to merge. Increase if singer pauses >10s between verses. |
| `--min-duration` | 20s | Minimum segment length. Lower if singer does very short songs. |
| `--padding` | 2s | Extra seconds before/after each segment. Helps catch song intro/outro. |
| `--fps` | 29.97 | Frame rate for timecode. Use 25 for PAL. |
### Recommended settings for typical singing streams
- **Singer with lots of chatting between songs**: `--gap 10 --min-duration 30`
- **Singer with minimal breaks**: `--gap 5 --min-duration 20`
- **Singer who does short covers/snippets**: `--gap 8 --min-duration 15`
- **Conservative (catch everything)**: `--gap 20 --min-duration 15 --padding 5`
## Troubleshooting
### "Singing voice detected as speech"
This can happen if the singer talks over background music. Try `--gap 15` to merge nearby segments.
### "Too many false positives (non-singing music detected)"
If there's background music during chatting, increase `--min-duration 45` to only keep longer segments (full songs).
### "Segments cut off the beginning/end of songs"
Increase `--padding 5` to add more buffer.
### GPU not being used
Check that TensorFlow detects your GPU:
```python
python -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"
```
### Processing is very slow
- A 4-hour stream typically takes 5-15 minutes on GPU, 30-60 minutes on CPU.
- Make sure you're using GPU-enabled TensorFlow.
- Close other GPU-heavy applications during processing.
## Output Files
For input `stream_2026-04-04.mp4`, you get:
| File | Purpose |
|------|---------|
| `stream_2026-04-04_singing.edl` | DaVinci Resolve timeline import |
| `stream_2026-04-04_markers.csv` | DaVinci Resolve marker import |
| `stream_2026-04-04_singing.csv` | Human-readable segment list |
| `stream_2026-04-04_singing.json` | Programmatic use |
| `stream_2026-04-04_singing.bat` | Auto-cut script (with `--auto-cut`) |
| `stream_2026-04-04_raw_segments.csv` | All segments for debugging (with `--raw-segments`) |