# Sing Detect

A fast, native-feeling desktop app that **finds singing segments in a stream
recording** and exports editor-friendly markers (DaVinci Resolve EDL / markers
CSV / CSV / JSON). Built with Tauri 2 (Rust backend + TypeScript UI) and a small
Python ML sidecar.

This is the detection-only spin-off of Stream Studio (the video-concat feature
lives in its own project now).

## How it works

```
 recording ──▶ [Rust ▸ ffmpeg ▸ 16 kHz mono WAV]
                                  │
              [Python sidecar ▸ inaSpeechSegmenter] ──json──▶ segments
                                  │
              [Rust ▸ EDL / markers / CSV / JSON exports]
```

| Layer | Tech | Why |
|-------|------|-----|
| UI | **WebView2** via **Tauri 2** | GPU-composited, tiny app, not Electron-sized. |
| Core | **Rust** | Audio extraction, drives the sidecar, writes exports. No console windows. |
| Audio | **ffmpeg / ffprobe** | 16 kHz mono WAV extraction. |
| ML | **Python 3.10 sidecar** | `inaSpeechSegmenter` (TensorFlow) classifies singing as "music"; TensorFlow needs Python 3.10. |

## Prerequisites

- **Node** (18+) and **Rust** (`rustup`, MSVC toolchain) — to build.
- **ffmpeg + ffprobe** on PATH (or beside the exe) — bundled by `pack.bat`.
- **Python 3.10** on the machine that runs detection. The app's built-in
  **"Test detection engine"** button installs the ML packages + downloads the
  model for you on first run (or run `sidecar\setup_sidecar.bat` manually).

## Run / build

```bat
dev.bat        REM hot-reload dev (UI + Rust)
build.bat      REM release MSI installer → src-tauri\target\release\bundle\msi\
pack.bat       REM STANDALONE portable folder → dist-portable\SingDetect\
```

`pack.bat` produces a copy-paste folder with `Sing Detect.exe`, ffmpeg, and the
Python `sidecar/` bundled. Copy it to any Windows PC; the only thing the target
needs is Python 3.10 (the in-app engine test handles the rest).

Detection auto-discovers `py -3.10`; override the interpreter with the
`STREAM_STUDIO_PYTHON` env var, or the script path with `STREAM_STUDIO_SIDECAR`.

## Project layout

```
sing_detect_v2/
├── index.html
├── src/                      # Frontend (TypeScript, no framework)
│   ├── main.ts               #   shell: single view, theme, drag-drop
│   ├── api.ts                #   typed bridge to Rust commands + events
│   ├── dom.ts / ui.ts        #   tiny DOM + toast/progress/song helpers
│   ├── styles.css            #   Fluent-ish design system (light/dark)
│   └── views/detect.ts       #   the detection view + engine setup card
├── src-tauri/
│   ├── src/
│   │   ├── lib.rs            #   commands, state, app wiring
│   │   ├── tools.rs          #   ffmpeg/ffprobe discovery + helpers
│   │   ├── detect.rs         #   drives the Python sidecar; env check/setup
│   │   └── export.rs         #   EDL / markers / CSV / JSON
│   ├── tauri.conf.json       #   window + bundles sidecar/ as a resource
│   └── capabilities/         #   dialog + opener permissions
└── sidecar/
    ├── detect.py             #   the ONLY Python — ML step only
    ├── requirements.txt
    └── setup_sidecar.bat
```

## Detection parameters

| Parameter | Default | When to change |
|-----------|---------|----------------|
| Merge gap | 10s | Raise if the singer pauses >10s mid-song |
| Min duration | 20s | Lower for short covers/snippets |
| Padding | 2s | Raise if songs get clipped at the edges |
| FPS | 29.97 | Match your video (25 PAL, 23.976 film) — affects timecode export only |

Singing is classified as **"music"** by inaSpeechSegmenter; talking is "speech".