diff --git a/README.md b/README.md deleted file mode 100644 index b4cf2cf..0000000 --- a/README.md +++ /dev/null @@ -1,185 +0,0 @@ -# Singing Segment Detector - -Automatically detect singing segments in live stream recordings and export timeline markers for DaVinci Resolve. - -## What It Does - -``` -4hr stream recording (.mp4/.mkv/.flv) - │ - ▼ [ffmpeg: extract 16kHz mono audio] - │ - ▼ [inaSpeechSegmenter: classify speech/music/noise] - │ - ▼ [merge nearby music segments, filter short ones] - │ - ├──→ .edl (import as DaVinci Resolve timeline) - ├──→ _markers.csv (import as DaVinci Resolve markers) - ├──→ .csv (human-readable segment list) - ├──→ .json (programmatic use) - └──→ .bat (optional: auto-cut with ffmpeg, no editing needed) -``` - -**Key insight**: `inaSpeechSegmenter` classifies **singing voice as "music"**. So in a typical singing stream, talking = "speech", singing = "music". We filter for "music" segments and merge nearby ones (a song might have brief pauses between verses). - -## Setup (Windows with conda) - -### Prerequisites -- **conda** (Anaconda or Miniconda) -- **ffmpeg** in your PATH -- **NVIDIA GPU** recommended (RTX 2080 works great) but CPU also works - -### Step 1: Create conda environment - -```powershell -conda create -n singing-detector python=3.11 -y -conda activate singing-detector -``` - -### Step 2: Install PyTorch with CUDA (for GPU acceleration) - -```powershell -# For RTX 2080 (CUDA 12.x) -pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 -``` - -### Step 3: Install TensorFlow (required by inaSpeechSegmenter) - -```powershell -pip install tensorflow[and-cuda] -``` - -Or CPU-only (slower but works): - -```powershell -pip install tensorflow -``` - -### Step 4: Install inaSpeechSegmenter and dependencies - -```powershell -pip install inaSpeechSegmenter -``` - -### Step 5: Verify installation - -```powershell -python -c "from inaSpeechSegmenter import Segmenter; print('OK!')" -``` - -## Usage - -### Single file - -```powershell -conda activate singing-detector - -# Basic usage -python detect_singing.py "D:\streams\2026-04-04_stream.mp4" - -# Custom settings -python detect_singing.py "D:\streams\2026-04-04_stream.mp4" --gap 15 --min-duration 30 - -# Output to a specific directory -python detect_singing.py "D:\streams\2026-04-04_stream.mp4" -o "D:\singing_edits" - -# Also generate an auto-cut ffmpeg script (no DaVinci needed) -python detect_singing.py "D:\streams\2026-04-04_stream.mp4" --auto-cut - -# For 4K F-log footage at 29.97fps (your XT3 settings) -python detect_singing.py "D:\streams\2026-04-04_stream.mp4" --fps 29.97 -``` - -### Batch processing - -```powershell -# Process all videos in a folder -python batch_detect.py "D:\streams" - -# Process new files only (skip already-done ones) -python batch_detect.py "D:\streams" --skip-existing - -# Process and auto-generate singing-only videos -python batch_detect.py "D:\streams" --auto-cut -o "D:\singing_edits" - -# Only process .flv files -python batch_detect.py "D:\streams" --pattern "*.flv" -``` - -## Importing Results into DaVinci Resolve - -### Option A: Import EDL as Timeline (recommended) - -1. Open your project in DaVinci Resolve -2. Import the original stream video into your Media Pool -3. Go to **File → Import → Timeline** -4. Select the `_singing.edl` file -5. DaVinci will create a new timeline with only the singing segments - -### Option B: Import Markers CSV - -1. Create a timeline from your stream video -2. Right-click on the timeline → **Timelines → Import Markers from CSV** -3. Select the `_markers.csv` file -4. Blue markers will appear at each singing segment start/end - -### Option C: Auto-Cut (no DaVinci needed) - -If you used `--auto-cut`, just run the generated `.bat` script: - -```powershell -D:\singing_edits\2026-04-04_stream_singing.bat -``` - -This uses ffmpeg to directly cut and concatenate all singing segments into a single `_singing_only.mp4` file. Fastest option, but no manual review. - -## Tuning Parameters - -| Parameter | Default | Description | -|-----------|---------|-------------| -| `--gap` | 10s | Max gap between music segments to merge. Increase if singer pauses >10s between verses. | -| `--min-duration` | 20s | Minimum segment length. Lower if singer does very short songs. | -| `--padding` | 2s | Extra seconds before/after each segment. Helps catch song intro/outro. | -| `--fps` | 29.97 | Frame rate for timecode. Use 25 for PAL. | - -### Recommended settings for typical singing streams - -- **Singer with lots of chatting between songs**: `--gap 10 --min-duration 30` -- **Singer with minimal breaks**: `--gap 5 --min-duration 20` -- **Singer who does short covers/snippets**: `--gap 8 --min-duration 15` -- **Conservative (catch everything)**: `--gap 20 --min-duration 15 --padding 5` - -## Troubleshooting - -### "Singing voice detected as speech" -This can happen if the singer talks over background music. Try `--gap 15` to merge nearby segments. - -### "Too many false positives (non-singing music detected)" -If there's background music during chatting, increase `--min-duration 45` to only keep longer segments (full songs). - -### "Segments cut off the beginning/end of songs" -Increase `--padding 5` to add more buffer. - -### GPU not being used -Check that TensorFlow detects your GPU: -```python -python -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))" -``` - -### Processing is very slow -- A 4-hour stream typically takes 5-15 minutes on GPU, 30-60 minutes on CPU. -- Make sure you're using GPU-enabled TensorFlow. -- Close other GPU-heavy applications during processing. - -## Output Files - -For input `stream_2026-04-04.mp4`, you get: - -| File | Purpose | -|------|---------| -| `stream_2026-04-04_singing.edl` | DaVinci Resolve timeline import | -| `stream_2026-04-04_markers.csv` | DaVinci Resolve marker import | -| `stream_2026-04-04_singing.csv` | Human-readable segment list | -| `stream_2026-04-04_singing.json` | Programmatic use | -| `stream_2026-04-04_singing.bat` | Auto-cut script (with `--auto-cut`) | -| `stream_2026-04-04_raw_segments.csv` | All segments for debugging (with `--raw-segments`) | diff --git a/batch_detect.py b/batch_detect.py new file mode 100644 index 0000000..1ed4a3e --- /dev/null +++ b/batch_detect.py @@ -0,0 +1,183 @@ +#!/usr/bin/env python3 +""" +Batch Singing Detector +======================= +Process multiple stream recordings at once. +Point it at a folder of recordings and it will process each one. + +Usage: + python batch_detect.py "D:\\streams" + python batch_detect.py "D:\\streams" --output-dir "D:\\singing_edits" --auto-cut + python batch_detect.py "D:\\streams" --pattern "*.mp4" --recursive +""" + +import argparse +import os +import sys +import time +import glob +from pathlib import Path + + +def find_videos(input_dir: str, pattern: str = "*", recursive: bool = False, + extensions: tuple = (".mp4", ".mkv", ".flv", ".ts", ".avi", ".mov", ".webm")) -> list: + """Find all video files in directory.""" + videos = [] + + if recursive: + for ext in extensions: + videos.extend(glob.glob(os.path.join(input_dir, "**", f"*{ext}"), recursive=True)) + else: + for ext in extensions: + videos.extend(glob.glob(os.path.join(input_dir, f"*{ext}"))) + + # Filter by pattern if specified + if pattern != "*": + videos = [v for v in videos if Path(v).match(pattern)] + + # Sort by modification time (newest first) + videos.sort(key=lambda x: os.path.getmtime(x), reverse=True) + + return videos + + +def main(): + parser = argparse.ArgumentParser( + description="Batch process multiple stream recordings for singing detection.", + ) + + parser.add_argument("input_dir", help="Directory containing stream recordings") + parser.add_argument("--output-dir", "-o", default=None, + help="Output directory (default: same as input)") + parser.add_argument("--pattern", default="*", + help="Filename pattern to match (default: all video files)") + parser.add_argument("--recursive", "-r", action="store_true", + help="Search subdirectories recursively") + parser.add_argument("--gap", "-g", type=float, default=10.0, + help="Max gap to merge singing segments (default: 10s)") + parser.add_argument("--min-duration", "-m", type=float, default=20.0, + help="Minimum segment duration (default: 20s)") + parser.add_argument("--padding", "-p", type=float, default=2.0, + help="Padding before/after segments (default: 2s)") + parser.add_argument("--fps", type=float, default=29.97, + help="Frame rate (default: 29.97)") + parser.add_argument("--auto-cut", action="store_true", + help="Generate auto-cut ffmpeg scripts") + parser.add_argument("--skip-existing", action="store_true", + help="Skip files that already have output .edl files") + parser.add_argument("--limit", type=int, default=0, + help="Process only N files (0 = all)") + + args = parser.parse_args() + + if not os.path.isdir(args.input_dir): + print(f"[ERROR] Directory not found: {args.input_dir}") + sys.exit(1) + + # Find videos + videos = find_videos(args.input_dir, args.pattern, args.recursive) + + if not videos: + print(f"[INFO] No video files found in: {args.input_dir}") + sys.exit(0) + + # Filter already-processed files + if args.skip_existing: + output_dir = args.output_dir or args.input_dir + unprocessed = [] + for v in videos: + stem = Path(v).stem + edl = os.path.join(output_dir, f"{stem}_singing.edl") + if not os.path.exists(edl): + unprocessed.append(v) + else: + print(f"[SKIP] Already processed: {Path(v).name}") + videos = unprocessed + + # Apply limit + if args.limit > 0: + videos = videos[:args.limit] + + print(f"\n{'='*60}") + print(f" Batch Singing Detector") + print(f"{'='*60}") + print(f" Input: {args.input_dir}") + print(f" Videos: {len(videos)} to process") + print(f"{'='*60}\n") + + # Process each video + # Import the main detect_singing module + sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) + from detect_singing import ( + extract_audio, run_segmentation, merge_singing_segments, + export_edl, export_csv, export_davinci_markers_csv, + export_json, export_ffmpeg_concat, format_timecode, + ) + + results = [] + for i, video_path in enumerate(videos, 1): + video_name = Path(video_path).name + print(f"\n{'─'*60}") + print(f" [{i}/{len(videos)}] Processing: {video_name}") + print(f"{'─'*60}") + + try: + output_dir = args.output_dir or os.path.dirname(video_path) + os.makedirs(output_dir, exist_ok=True) + + stem = Path(video_path).stem + wav_path = os.path.join(output_dir, f"{stem}_audio.wav") + + # Step 1-3 + extract_audio(video_path, wav_path) + raw_segments = run_segmentation(wav_path) + singing = merge_singing_segments( + raw_segments, + max_gap=args.gap, + min_duration=args.min_duration, + padding=args.padding, + ) + + # Step 4: Export + if singing: + edl = os.path.join(output_dir, f"{stem}_singing.edl") + csv_f = os.path.join(output_dir, f"{stem}_singing.csv") + markers = os.path.join(output_dir, f"{stem}_markers.csv") + json_f = os.path.join(output_dir, f"{stem}_singing.json") + + export_edl(singing, edl, fps=args.fps, + source_filename=Path(video_path).name) + export_csv(singing, csv_f) + export_davinci_markers_csv(singing, markers, fps=args.fps) + export_json(singing, json_f) + + if args.auto_cut: + export_ffmpeg_concat(singing, video_path, json_f) + + total_dur = sum(s["duration"] for s in singing) + results.append((video_name, len(singing), total_dur, "OK")) + else: + results.append((video_name, 0, 0, "No songs found")) + + # Cleanup WAV + if os.path.exists(wav_path): + os.remove(wav_path) + + except Exception as e: + print(f" [ERROR] {e}") + results.append((video_name, 0, 0, f"Error: {e}")) + + # Summary + print(f"\n\n{'='*60}") + print(f" BATCH COMPLETE") + print(f"{'='*60}") + print(f" {'Video':<40s} {'Songs':>5s} {'Singing Time':>12s} Status") + print(f" {'─'*40} {'─'*5} {'─'*12} {'─'*15}") + for name, count, dur, status in results: + name_short = name[:38] + ".." if len(name) > 40 else name + print(f" {name_short:<40s} {count:>5d} {format_timecode(dur):>12s} {status}") + print(f"{'='*60}") + + +if __name__ == "__main__": + main() diff --git a/setup_windows.bat b/setup_windows.bat deleted file mode 100644 index 64b895d..0000000 --- a/setup_windows.bat +++ /dev/null @@ -1,56 +0,0 @@ -@echo off -REM ============================================ -REM Singing Detector - Quick Setup (Windows) -REM ============================================ -REM Run this in Anaconda Prompt / PowerShell -REM Prerequisites: conda, ffmpeg in PATH -REM ============================================ - -echo. -echo ============================================ -echo Singing Segment Detector - Setup -echo ============================================ -echo. - -REM Step 1: Create conda environment -echo [1/4] Creating conda environment (python 3.11)... -call conda create -n singing-detector python=3.11 -y -if %ERRORLEVEL% neq 0 ( - echo ERROR: Failed to create conda environment. - pause - exit /b 1 -) - -REM Activate environment -call conda activate singing-detector - -REM Step 2: Install TensorFlow with GPU support -echo. -echo [2/4] Installing TensorFlow (GPU)... -pip install tensorflow[and-cuda] -if %ERRORLEVEL% neq 0 ( - echo WARNING: GPU TensorFlow install failed, trying CPU version... - pip install tensorflow -) - -REM Step 3: Install inaSpeechSegmenter -echo. -echo [3/4] Installing inaSpeechSegmenter... -pip install inaSpeechSegmenter - -REM Step 4: Verify -echo. -echo [4/4] Verifying installation... -python -c "from inaSpeechSegmenter import Segmenter; print('inaSpeechSegmenter OK!')" -python -c "import tensorflow as tf; gpus = tf.config.list_physical_devices('GPU'); print(f'TensorFlow GPUs: {gpus}' if gpus else 'TensorFlow: CPU only')" - -echo. -echo ============================================ -echo Setup complete! -echo ============================================ -echo. -echo Usage: -echo conda activate singing-detector -echo python detect_singing.py "path\to\stream.mp4" -echo. -pause