v4: batch processing layer over the CLI engine
Wraps detect_singing to process whole folders of recordings at once: --recursive, --pattern, --output-dir, --auto-cut. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
185
README.md
185
README.md
@@ -1,185 +0,0 @@
|
|||||||
# Singing Segment Detector
|
|
||||||
|
|
||||||
Automatically detect singing segments in live stream recordings and export timeline markers for DaVinci Resolve.
|
|
||||||
|
|
||||||
## What It Does
|
|
||||||
|
|
||||||
```
|
|
||||||
4hr stream recording (.mp4/.mkv/.flv)
|
|
||||||
│
|
|
||||||
▼ [ffmpeg: extract 16kHz mono audio]
|
|
||||||
│
|
|
||||||
▼ [inaSpeechSegmenter: classify speech/music/noise]
|
|
||||||
│
|
|
||||||
▼ [merge nearby music segments, filter short ones]
|
|
||||||
│
|
|
||||||
├──→ .edl (import as DaVinci Resolve timeline)
|
|
||||||
├──→ _markers.csv (import as DaVinci Resolve markers)
|
|
||||||
├──→ .csv (human-readable segment list)
|
|
||||||
├──→ .json (programmatic use)
|
|
||||||
└──→ .bat (optional: auto-cut with ffmpeg, no editing needed)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Key insight**: `inaSpeechSegmenter` classifies **singing voice as "music"**. So in a typical singing stream, talking = "speech", singing = "music". We filter for "music" segments and merge nearby ones (a song might have brief pauses between verses).
|
|
||||||
|
|
||||||
## Setup (Windows with conda)
|
|
||||||
|
|
||||||
### Prerequisites
|
|
||||||
- **conda** (Anaconda or Miniconda)
|
|
||||||
- **ffmpeg** in your PATH
|
|
||||||
- **NVIDIA GPU** recommended (RTX 2080 works great) but CPU also works
|
|
||||||
|
|
||||||
### Step 1: Create conda environment
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
conda create -n singing-detector python=3.11 -y
|
|
||||||
conda activate singing-detector
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 2: Install PyTorch with CUDA (for GPU acceleration)
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
# For RTX 2080 (CUDA 12.x)
|
|
||||||
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 3: Install TensorFlow (required by inaSpeechSegmenter)
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
pip install tensorflow[and-cuda]
|
|
||||||
```
|
|
||||||
|
|
||||||
Or CPU-only (slower but works):
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
pip install tensorflow
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 4: Install inaSpeechSegmenter and dependencies
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
pip install inaSpeechSegmenter
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 5: Verify installation
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
python -c "from inaSpeechSegmenter import Segmenter; print('OK!')"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
### Single file
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
conda activate singing-detector
|
|
||||||
|
|
||||||
# Basic usage
|
|
||||||
python detect_singing.py "D:\streams\2026-04-04_stream.mp4"
|
|
||||||
|
|
||||||
# Custom settings
|
|
||||||
python detect_singing.py "D:\streams\2026-04-04_stream.mp4" --gap 15 --min-duration 30
|
|
||||||
|
|
||||||
# Output to a specific directory
|
|
||||||
python detect_singing.py "D:\streams\2026-04-04_stream.mp4" -o "D:\singing_edits"
|
|
||||||
|
|
||||||
# Also generate an auto-cut ffmpeg script (no DaVinci needed)
|
|
||||||
python detect_singing.py "D:\streams\2026-04-04_stream.mp4" --auto-cut
|
|
||||||
|
|
||||||
# For 4K F-log footage at 29.97fps (your XT3 settings)
|
|
||||||
python detect_singing.py "D:\streams\2026-04-04_stream.mp4" --fps 29.97
|
|
||||||
```
|
|
||||||
|
|
||||||
### Batch processing
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
# Process all videos in a folder
|
|
||||||
python batch_detect.py "D:\streams"
|
|
||||||
|
|
||||||
# Process new files only (skip already-done ones)
|
|
||||||
python batch_detect.py "D:\streams" --skip-existing
|
|
||||||
|
|
||||||
# Process and auto-generate singing-only videos
|
|
||||||
python batch_detect.py "D:\streams" --auto-cut -o "D:\singing_edits"
|
|
||||||
|
|
||||||
# Only process .flv files
|
|
||||||
python batch_detect.py "D:\streams" --pattern "*.flv"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Importing Results into DaVinci Resolve
|
|
||||||
|
|
||||||
### Option A: Import EDL as Timeline (recommended)
|
|
||||||
|
|
||||||
1. Open your project in DaVinci Resolve
|
|
||||||
2. Import the original stream video into your Media Pool
|
|
||||||
3. Go to **File → Import → Timeline**
|
|
||||||
4. Select the `_singing.edl` file
|
|
||||||
5. DaVinci will create a new timeline with only the singing segments
|
|
||||||
|
|
||||||
### Option B: Import Markers CSV
|
|
||||||
|
|
||||||
1. Create a timeline from your stream video
|
|
||||||
2. Right-click on the timeline → **Timelines → Import Markers from CSV**
|
|
||||||
3. Select the `_markers.csv` file
|
|
||||||
4. Blue markers will appear at each singing segment start/end
|
|
||||||
|
|
||||||
### Option C: Auto-Cut (no DaVinci needed)
|
|
||||||
|
|
||||||
If you used `--auto-cut`, just run the generated `.bat` script:
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
D:\singing_edits\2026-04-04_stream_singing.bat
|
|
||||||
```
|
|
||||||
|
|
||||||
This uses ffmpeg to directly cut and concatenate all singing segments into a single `_singing_only.mp4` file. Fastest option, but no manual review.
|
|
||||||
|
|
||||||
## Tuning Parameters
|
|
||||||
|
|
||||||
| Parameter | Default | Description |
|
|
||||||
|-----------|---------|-------------|
|
|
||||||
| `--gap` | 10s | Max gap between music segments to merge. Increase if singer pauses >10s between verses. |
|
|
||||||
| `--min-duration` | 20s | Minimum segment length. Lower if singer does very short songs. |
|
|
||||||
| `--padding` | 2s | Extra seconds before/after each segment. Helps catch song intro/outro. |
|
|
||||||
| `--fps` | 29.97 | Frame rate for timecode. Use 25 for PAL. |
|
|
||||||
|
|
||||||
### Recommended settings for typical singing streams
|
|
||||||
|
|
||||||
- **Singer with lots of chatting between songs**: `--gap 10 --min-duration 30`
|
|
||||||
- **Singer with minimal breaks**: `--gap 5 --min-duration 20`
|
|
||||||
- **Singer who does short covers/snippets**: `--gap 8 --min-duration 15`
|
|
||||||
- **Conservative (catch everything)**: `--gap 20 --min-duration 15 --padding 5`
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### "Singing voice detected as speech"
|
|
||||||
This can happen if the singer talks over background music. Try `--gap 15` to merge nearby segments.
|
|
||||||
|
|
||||||
### "Too many false positives (non-singing music detected)"
|
|
||||||
If there's background music during chatting, increase `--min-duration 45` to only keep longer segments (full songs).
|
|
||||||
|
|
||||||
### "Segments cut off the beginning/end of songs"
|
|
||||||
Increase `--padding 5` to add more buffer.
|
|
||||||
|
|
||||||
### GPU not being used
|
|
||||||
Check that TensorFlow detects your GPU:
|
|
||||||
```python
|
|
||||||
python -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"
|
|
||||||
```
|
|
||||||
|
|
||||||
### Processing is very slow
|
|
||||||
- A 4-hour stream typically takes 5-15 minutes on GPU, 30-60 minutes on CPU.
|
|
||||||
- Make sure you're using GPU-enabled TensorFlow.
|
|
||||||
- Close other GPU-heavy applications during processing.
|
|
||||||
|
|
||||||
## Output Files
|
|
||||||
|
|
||||||
For input `stream_2026-04-04.mp4`, you get:
|
|
||||||
|
|
||||||
| File | Purpose |
|
|
||||||
|------|---------|
|
|
||||||
| `stream_2026-04-04_singing.edl` | DaVinci Resolve timeline import |
|
|
||||||
| `stream_2026-04-04_markers.csv` | DaVinci Resolve marker import |
|
|
||||||
| `stream_2026-04-04_singing.csv` | Human-readable segment list |
|
|
||||||
| `stream_2026-04-04_singing.json` | Programmatic use |
|
|
||||||
| `stream_2026-04-04_singing.bat` | Auto-cut script (with `--auto-cut`) |
|
|
||||||
| `stream_2026-04-04_raw_segments.csv` | All segments for debugging (with `--raw-segments`) |
|
|
||||||
183
batch_detect.py
Normal file
183
batch_detect.py
Normal file
@@ -0,0 +1,183 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Batch Singing Detector
|
||||||
|
=======================
|
||||||
|
Process multiple stream recordings at once.
|
||||||
|
Point it at a folder of recordings and it will process each one.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
python batch_detect.py "D:\\streams"
|
||||||
|
python batch_detect.py "D:\\streams" --output-dir "D:\\singing_edits" --auto-cut
|
||||||
|
python batch_detect.py "D:\\streams" --pattern "*.mp4" --recursive
|
||||||
|
"""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
import glob
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
def find_videos(input_dir: str, pattern: str = "*", recursive: bool = False,
|
||||||
|
extensions: tuple = (".mp4", ".mkv", ".flv", ".ts", ".avi", ".mov", ".webm")) -> list:
|
||||||
|
"""Find all video files in directory."""
|
||||||
|
videos = []
|
||||||
|
|
||||||
|
if recursive:
|
||||||
|
for ext in extensions:
|
||||||
|
videos.extend(glob.glob(os.path.join(input_dir, "**", f"*{ext}"), recursive=True))
|
||||||
|
else:
|
||||||
|
for ext in extensions:
|
||||||
|
videos.extend(glob.glob(os.path.join(input_dir, f"*{ext}")))
|
||||||
|
|
||||||
|
# Filter by pattern if specified
|
||||||
|
if pattern != "*":
|
||||||
|
videos = [v for v in videos if Path(v).match(pattern)]
|
||||||
|
|
||||||
|
# Sort by modification time (newest first)
|
||||||
|
videos.sort(key=lambda x: os.path.getmtime(x), reverse=True)
|
||||||
|
|
||||||
|
return videos
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(
|
||||||
|
description="Batch process multiple stream recordings for singing detection.",
|
||||||
|
)
|
||||||
|
|
||||||
|
parser.add_argument("input_dir", help="Directory containing stream recordings")
|
||||||
|
parser.add_argument("--output-dir", "-o", default=None,
|
||||||
|
help="Output directory (default: same as input)")
|
||||||
|
parser.add_argument("--pattern", default="*",
|
||||||
|
help="Filename pattern to match (default: all video files)")
|
||||||
|
parser.add_argument("--recursive", "-r", action="store_true",
|
||||||
|
help="Search subdirectories recursively")
|
||||||
|
parser.add_argument("--gap", "-g", type=float, default=10.0,
|
||||||
|
help="Max gap to merge singing segments (default: 10s)")
|
||||||
|
parser.add_argument("--min-duration", "-m", type=float, default=20.0,
|
||||||
|
help="Minimum segment duration (default: 20s)")
|
||||||
|
parser.add_argument("--padding", "-p", type=float, default=2.0,
|
||||||
|
help="Padding before/after segments (default: 2s)")
|
||||||
|
parser.add_argument("--fps", type=float, default=29.97,
|
||||||
|
help="Frame rate (default: 29.97)")
|
||||||
|
parser.add_argument("--auto-cut", action="store_true",
|
||||||
|
help="Generate auto-cut ffmpeg scripts")
|
||||||
|
parser.add_argument("--skip-existing", action="store_true",
|
||||||
|
help="Skip files that already have output .edl files")
|
||||||
|
parser.add_argument("--limit", type=int, default=0,
|
||||||
|
help="Process only N files (0 = all)")
|
||||||
|
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
if not os.path.isdir(args.input_dir):
|
||||||
|
print(f"[ERROR] Directory not found: {args.input_dir}")
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
# Find videos
|
||||||
|
videos = find_videos(args.input_dir, args.pattern, args.recursive)
|
||||||
|
|
||||||
|
if not videos:
|
||||||
|
print(f"[INFO] No video files found in: {args.input_dir}")
|
||||||
|
sys.exit(0)
|
||||||
|
|
||||||
|
# Filter already-processed files
|
||||||
|
if args.skip_existing:
|
||||||
|
output_dir = args.output_dir or args.input_dir
|
||||||
|
unprocessed = []
|
||||||
|
for v in videos:
|
||||||
|
stem = Path(v).stem
|
||||||
|
edl = os.path.join(output_dir, f"{stem}_singing.edl")
|
||||||
|
if not os.path.exists(edl):
|
||||||
|
unprocessed.append(v)
|
||||||
|
else:
|
||||||
|
print(f"[SKIP] Already processed: {Path(v).name}")
|
||||||
|
videos = unprocessed
|
||||||
|
|
||||||
|
# Apply limit
|
||||||
|
if args.limit > 0:
|
||||||
|
videos = videos[:args.limit]
|
||||||
|
|
||||||
|
print(f"\n{'='*60}")
|
||||||
|
print(f" Batch Singing Detector")
|
||||||
|
print(f"{'='*60}")
|
||||||
|
print(f" Input: {args.input_dir}")
|
||||||
|
print(f" Videos: {len(videos)} to process")
|
||||||
|
print(f"{'='*60}\n")
|
||||||
|
|
||||||
|
# Process each video
|
||||||
|
# Import the main detect_singing module
|
||||||
|
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
from detect_singing import (
|
||||||
|
extract_audio, run_segmentation, merge_singing_segments,
|
||||||
|
export_edl, export_csv, export_davinci_markers_csv,
|
||||||
|
export_json, export_ffmpeg_concat, format_timecode,
|
||||||
|
)
|
||||||
|
|
||||||
|
results = []
|
||||||
|
for i, video_path in enumerate(videos, 1):
|
||||||
|
video_name = Path(video_path).name
|
||||||
|
print(f"\n{'─'*60}")
|
||||||
|
print(f" [{i}/{len(videos)}] Processing: {video_name}")
|
||||||
|
print(f"{'─'*60}")
|
||||||
|
|
||||||
|
try:
|
||||||
|
output_dir = args.output_dir or os.path.dirname(video_path)
|
||||||
|
os.makedirs(output_dir, exist_ok=True)
|
||||||
|
|
||||||
|
stem = Path(video_path).stem
|
||||||
|
wav_path = os.path.join(output_dir, f"{stem}_audio.wav")
|
||||||
|
|
||||||
|
# Step 1-3
|
||||||
|
extract_audio(video_path, wav_path)
|
||||||
|
raw_segments = run_segmentation(wav_path)
|
||||||
|
singing = merge_singing_segments(
|
||||||
|
raw_segments,
|
||||||
|
max_gap=args.gap,
|
||||||
|
min_duration=args.min_duration,
|
||||||
|
padding=args.padding,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Step 4: Export
|
||||||
|
if singing:
|
||||||
|
edl = os.path.join(output_dir, f"{stem}_singing.edl")
|
||||||
|
csv_f = os.path.join(output_dir, f"{stem}_singing.csv")
|
||||||
|
markers = os.path.join(output_dir, f"{stem}_markers.csv")
|
||||||
|
json_f = os.path.join(output_dir, f"{stem}_singing.json")
|
||||||
|
|
||||||
|
export_edl(singing, edl, fps=args.fps,
|
||||||
|
source_filename=Path(video_path).name)
|
||||||
|
export_csv(singing, csv_f)
|
||||||
|
export_davinci_markers_csv(singing, markers, fps=args.fps)
|
||||||
|
export_json(singing, json_f)
|
||||||
|
|
||||||
|
if args.auto_cut:
|
||||||
|
export_ffmpeg_concat(singing, video_path, json_f)
|
||||||
|
|
||||||
|
total_dur = sum(s["duration"] for s in singing)
|
||||||
|
results.append((video_name, len(singing), total_dur, "OK"))
|
||||||
|
else:
|
||||||
|
results.append((video_name, 0, 0, "No songs found"))
|
||||||
|
|
||||||
|
# Cleanup WAV
|
||||||
|
if os.path.exists(wav_path):
|
||||||
|
os.remove(wav_path)
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
print(f" [ERROR] {e}")
|
||||||
|
results.append((video_name, 0, 0, f"Error: {e}"))
|
||||||
|
|
||||||
|
# Summary
|
||||||
|
print(f"\n\n{'='*60}")
|
||||||
|
print(f" BATCH COMPLETE")
|
||||||
|
print(f"{'='*60}")
|
||||||
|
print(f" {'Video':<40s} {'Songs':>5s} {'Singing Time':>12s} Status")
|
||||||
|
print(f" {'─'*40} {'─'*5} {'─'*12} {'─'*15}")
|
||||||
|
for name, count, dur, status in results:
|
||||||
|
name_short = name[:38] + ".." if len(name) > 40 else name
|
||||||
|
print(f" {name_short:<40s} {count:>5d} {format_timecode(dur):>12s} {status}")
|
||||||
|
print(f"{'='*60}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -1,56 +0,0 @@
|
|||||||
@echo off
|
|
||||||
REM ============================================
|
|
||||||
REM Singing Detector - Quick Setup (Windows)
|
|
||||||
REM ============================================
|
|
||||||
REM Run this in Anaconda Prompt / PowerShell
|
|
||||||
REM Prerequisites: conda, ffmpeg in PATH
|
|
||||||
REM ============================================
|
|
||||||
|
|
||||||
echo.
|
|
||||||
echo ============================================
|
|
||||||
echo Singing Segment Detector - Setup
|
|
||||||
echo ============================================
|
|
||||||
echo.
|
|
||||||
|
|
||||||
REM Step 1: Create conda environment
|
|
||||||
echo [1/4] Creating conda environment (python 3.11)...
|
|
||||||
call conda create -n singing-detector python=3.11 -y
|
|
||||||
if %ERRORLEVEL% neq 0 (
|
|
||||||
echo ERROR: Failed to create conda environment.
|
|
||||||
pause
|
|
||||||
exit /b 1
|
|
||||||
)
|
|
||||||
|
|
||||||
REM Activate environment
|
|
||||||
call conda activate singing-detector
|
|
||||||
|
|
||||||
REM Step 2: Install TensorFlow with GPU support
|
|
||||||
echo.
|
|
||||||
echo [2/4] Installing TensorFlow (GPU)...
|
|
||||||
pip install tensorflow[and-cuda]
|
|
||||||
if %ERRORLEVEL% neq 0 (
|
|
||||||
echo WARNING: GPU TensorFlow install failed, trying CPU version...
|
|
||||||
pip install tensorflow
|
|
||||||
)
|
|
||||||
|
|
||||||
REM Step 3: Install inaSpeechSegmenter
|
|
||||||
echo.
|
|
||||||
echo [3/4] Installing inaSpeechSegmenter...
|
|
||||||
pip install inaSpeechSegmenter
|
|
||||||
|
|
||||||
REM Step 4: Verify
|
|
||||||
echo.
|
|
||||||
echo [4/4] Verifying installation...
|
|
||||||
python -c "from inaSpeechSegmenter import Segmenter; print('inaSpeechSegmenter OK!')"
|
|
||||||
python -c "import tensorflow as tf; gpus = tf.config.list_physical_devices('GPU'); print(f'TensorFlow GPUs: {gpus}' if gpus else 'TensorFlow: CPU only')"
|
|
||||||
|
|
||||||
echo.
|
|
||||||
echo ============================================
|
|
||||||
echo Setup complete!
|
|
||||||
echo ============================================
|
|
||||||
echo.
|
|
||||||
echo Usage:
|
|
||||||
echo conda activate singing-detector
|
|
||||||
echo python detect_singing.py "path\to\stream.mp4"
|
|
||||||
echo.
|
|
||||||
pause
|
|
||||||
Reference in New Issue
Block a user