Compare commits
3 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 4887981a2d | |||
| 765816e701 | |||
| 33c275a114 |
41
.gitignore
vendored
@@ -1,7 +1,40 @@
|
|||||||
|
# deps & caches
|
||||||
|
node_modules/
|
||||||
__pycache__/
|
__pycache__/
|
||||||
*.pyc
|
*.pyc
|
||||||
*.wav
|
|
||||||
*.mp4
|
|
||||||
.DS_Store
|
|
||||||
venv/
|
|
||||||
.venv/
|
.venv/
|
||||||
|
venv/
|
||||||
|
.pytest_cache/
|
||||||
|
.mypy_cache/
|
||||||
|
.ruff_cache/
|
||||||
|
# build outputs
|
||||||
|
target/
|
||||||
|
dist/
|
||||||
|
dist-portable/
|
||||||
|
build/
|
||||||
|
# logs & databases
|
||||||
|
*.log
|
||||||
|
*.db
|
||||||
|
*.sqlite
|
||||||
|
*.sqlite3
|
||||||
|
# secrets / credentials
|
||||||
|
.env
|
||||||
|
*.env
|
||||||
|
*cookie*
|
||||||
|
*credential*
|
||||||
|
credentials.json
|
||||||
|
token.json
|
||||||
|
*token.json
|
||||||
|
.cookies.json
|
||||||
|
google_cred/
|
||||||
|
*.pem
|
||||||
|
*.key
|
||||||
|
# model weights / large artifacts
|
||||||
|
*.onnx
|
||||||
|
*.pt
|
||||||
|
*.pth
|
||||||
|
*.ckpt
|
||||||
|
*.safetensors
|
||||||
|
# os cruft
|
||||||
|
.DS_Store
|
||||||
|
Thumbs.db
|
||||||
3
.vscode/extensions.json
vendored
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
{
|
||||||
|
"recommendations": ["tauri-apps.tauri-vscode", "rust-lang.rust-analyzer"]
|
||||||
|
}
|
||||||
52
CLAUDE.md
Normal file
@@ -0,0 +1,52 @@
|
|||||||
|
# Sing Detect — project context for Claude
|
||||||
|
|
||||||
|
This file is auto-loaded each session and travels with the folder, so it stays
|
||||||
|
valid even after the folder is renamed (`stream-studio` → `sing_detect_v2`).
|
||||||
|
|
||||||
|
## What this project is
|
||||||
|
A Windows desktop app (**Tauri 2**: Rust backend + TypeScript/WebView2 UI) that
|
||||||
|
**detects singing segments** in stream recordings and exports editor markers
|
||||||
|
(EDL / markers CSV / CSV / JSON). It is the **detection-only** spin-off of the
|
||||||
|
former "Stream Studio"; the video-concat feature was split into a **separate
|
||||||
|
project** (`video_concat_v5`, since moved out of this folder).
|
||||||
|
|
||||||
|
- Crate: `sing-detect` / lib `sing_detect_lib`. Product: **Sing Detect**.
|
||||||
|
- Detection runs a **Python 3.10 sidecar** (`sidecar/detect.py`,
|
||||||
|
inaSpeechSegmenter + TensorFlow). ffmpeg extracts a 16 kHz WAV first.
|
||||||
|
|
||||||
|
## Build / run (Windows)
|
||||||
|
- `dev.bat` — hot-reload dev.
|
||||||
|
- `build.bat` — MSI installer → `src-tauri\target\release\bundle\msi\`.
|
||||||
|
- `pack.bat` — **standalone portable** folder → `dist-portable\SingDetect\`
|
||||||
|
(bundles `Sing Detect.exe` + ffmpeg/ffprobe + the Python `sidecar/`).
|
||||||
|
- Requirements on the build machine: Node, Rust (`winget install Rustlang.Rustup`),
|
||||||
|
VS 2022 C++ tools, ffmpeg at `C:\ffmpeg\bin` (or on PATH).
|
||||||
|
|
||||||
|
## ⚠️ State left by the rename (READ FIRST if a build fails)
|
||||||
|
- `src-tauri\target\` and `src-tauri\gen\` were **intentionally deleted** before
|
||||||
|
the folder rename, because cargo bakes absolute paths into `target\` and a
|
||||||
|
build under the old path name fails with `os error 3 (系统找不到指定的路径)`
|
||||||
|
referencing the old folder. **The first build after rename is a full ~5–8 min
|
||||||
|
recompile** — this is expected, not an error.
|
||||||
|
- `node_modules\` and `dist-portable\SingDetect\` were kept (relocatable /
|
||||||
|
self-contained). The existing `Sing Detect.exe` in `dist-portable\` works
|
||||||
|
regardless of the folder name.
|
||||||
|
- Do **not** keep `target\` in a Nextcloud-synced location — it re-introduces the
|
||||||
|
stale-absolute-path build failure across machines.
|
||||||
|
|
||||||
|
## Detection environment (per-machine)
|
||||||
|
- Needs **Python 3.10** (TensorFlow has no 3.12+ support). The app's in-UI
|
||||||
|
**"Test detection engine"** button checks the env and auto-installs the ML
|
||||||
|
packages + downloads the model on first run (or run `sidecar\setup_sidecar.bat`).
|
||||||
|
- Setup uses **default international PyPI** and does no proxy manipulation — pip
|
||||||
|
and the model download just inherit the system network settings, so a VPN/proxy
|
||||||
|
(e.g. Clash) routes them normally. Keep pip reasonably current: the very old pip
|
||||||
|
21.2.3 bundled with Python **3.10.0** crashes through an HTTP proxy
|
||||||
|
(`check_hostname requires server_hostname`); installing **Python 3.10.11**
|
||||||
|
(ships a modern pip) or running `py -3.10 -m pip install -U pip` once avoids it.
|
||||||
|
|
||||||
|
## Notable backend behavior
|
||||||
|
- Long ops (`detect_songs`, `setup_detection`, `export_segments`) run via
|
||||||
|
`spawn_blocking` so the window stays responsive (not "未响应").
|
||||||
|
- `cancel` kills the whole **process tree** (`taskkill /F /T`), because
|
||||||
|
`py -3.10` spawns `python.exe` as a child — killing only `py` left it running.
|
||||||
85
README.md
Normal file
@@ -0,0 +1,85 @@
|
|||||||
|
# Sing Detect
|
||||||
|
|
||||||
|
A fast, native-feeling desktop app that **finds singing segments in a stream
|
||||||
|
recording** and exports editor-friendly markers (DaVinci Resolve EDL / markers
|
||||||
|
CSV / CSV / JSON). Built with Tauri 2 (Rust backend + TypeScript UI) and a small
|
||||||
|
Python ML sidecar.
|
||||||
|
|
||||||
|
This is the detection-only spin-off of Stream Studio (the video-concat feature
|
||||||
|
lives in its own project now).
|
||||||
|
|
||||||
|
## How it works
|
||||||
|
|
||||||
|
```
|
||||||
|
recording ──▶ [Rust ▸ ffmpeg ▸ 16 kHz mono WAV]
|
||||||
|
│
|
||||||
|
[Python sidecar ▸ inaSpeechSegmenter] ──json──▶ segments
|
||||||
|
│
|
||||||
|
[Rust ▸ EDL / markers / CSV / JSON exports]
|
||||||
|
```
|
||||||
|
|
||||||
|
| Layer | Tech | Why |
|
||||||
|
|-------|------|-----|
|
||||||
|
| UI | **WebView2** via **Tauri 2** | GPU-composited, tiny app, not Electron-sized. |
|
||||||
|
| Core | **Rust** | Audio extraction, drives the sidecar, writes exports. No console windows. |
|
||||||
|
| Audio | **ffmpeg / ffprobe** | 16 kHz mono WAV extraction. |
|
||||||
|
| ML | **Python 3.10 sidecar** | `inaSpeechSegmenter` (TensorFlow) classifies singing as "music"; TensorFlow needs Python 3.10. |
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
- **Node** (18+) and **Rust** (`rustup`, MSVC toolchain) — to build.
|
||||||
|
- **ffmpeg + ffprobe** on PATH (or beside the exe) — bundled by `pack.bat`.
|
||||||
|
- **Python 3.10** on the machine that runs detection. The app's built-in
|
||||||
|
**"Test detection engine"** button installs the ML packages + downloads the
|
||||||
|
model for you on first run (or run `sidecar\setup_sidecar.bat` manually).
|
||||||
|
|
||||||
|
## Run / build
|
||||||
|
|
||||||
|
```bat
|
||||||
|
dev.bat REM hot-reload dev (UI + Rust)
|
||||||
|
build.bat REM release MSI installer → src-tauri\target\release\bundle\msi\
|
||||||
|
pack.bat REM STANDALONE portable folder → dist-portable\SingDetect\
|
||||||
|
```
|
||||||
|
|
||||||
|
`pack.bat` produces a copy-paste folder with `Sing Detect.exe`, ffmpeg, and the
|
||||||
|
Python `sidecar/` bundled. Copy it to any Windows PC; the only thing the target
|
||||||
|
needs is Python 3.10 (the in-app engine test handles the rest).
|
||||||
|
|
||||||
|
Detection auto-discovers `py -3.10`; override the interpreter with the
|
||||||
|
`STREAM_STUDIO_PYTHON` env var, or the script path with `STREAM_STUDIO_SIDECAR`.
|
||||||
|
|
||||||
|
## Project layout
|
||||||
|
|
||||||
|
```
|
||||||
|
sing_detect_v2/
|
||||||
|
├── index.html
|
||||||
|
├── src/ # Frontend (TypeScript, no framework)
|
||||||
|
│ ├── main.ts # shell: single view, theme, drag-drop
|
||||||
|
│ ├── api.ts # typed bridge to Rust commands + events
|
||||||
|
│ ├── dom.ts / ui.ts # tiny DOM + toast/progress/song helpers
|
||||||
|
│ ├── styles.css # Fluent-ish design system (light/dark)
|
||||||
|
│ └── views/detect.ts # the detection view + engine setup card
|
||||||
|
├── src-tauri/
|
||||||
|
│ ├── src/
|
||||||
|
│ │ ├── lib.rs # commands, state, app wiring
|
||||||
|
│ │ ├── tools.rs # ffmpeg/ffprobe discovery + helpers
|
||||||
|
│ │ ├── detect.rs # drives the Python sidecar; env check/setup
|
||||||
|
│ │ └── export.rs # EDL / markers / CSV / JSON
|
||||||
|
│ ├── tauri.conf.json # window + bundles sidecar/ as a resource
|
||||||
|
│ └── capabilities/ # dialog + opener permissions
|
||||||
|
└── sidecar/
|
||||||
|
├── detect.py # the ONLY Python — ML step only
|
||||||
|
├── requirements.txt
|
||||||
|
└── setup_sidecar.bat
|
||||||
|
```
|
||||||
|
|
||||||
|
## Detection parameters
|
||||||
|
|
||||||
|
| Parameter | Default | When to change |
|
||||||
|
|-----------|---------|----------------|
|
||||||
|
| Merge gap | 10s | Raise if the singer pauses >10s mid-song |
|
||||||
|
| Min duration | 20s | Lower for short covers/snippets |
|
||||||
|
| Padding | 2s | Raise if songs get clipped at the edges |
|
||||||
|
| FPS | 29.97 | Match your video (25 PAL, 23.976 film) — affects timecode export only |
|
||||||
|
|
||||||
|
Singing is classified as **"music"** by inaSpeechSegmenter; talking is "speech".
|
||||||
15
build.bat
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
@echo off
|
||||||
|
REM Sing Detect - build a distributable Windows app (MSI installer).
|
||||||
|
REM Output: src-tauri\target\release\bundle\msi\
|
||||||
|
setlocal
|
||||||
|
set "PATH=%USERPROFILE%\.cargo\bin;%PATH%"
|
||||||
|
|
||||||
|
if not exist "node_modules" call npm install
|
||||||
|
|
||||||
|
echo Building release bundle (this takes a few minutes the first time)...
|
||||||
|
call npm run tauri build
|
||||||
|
|
||||||
|
echo.
|
||||||
|
echo Done. Find the installer under:
|
||||||
|
echo src-tauri\target\release\bundle\msi\
|
||||||
|
pause
|
||||||
@@ -1,188 +0,0 @@
|
|||||||
"""
|
|
||||||
Export singing segments to various formats:
|
|
||||||
- EDL (DaVinci Resolve timeline import)
|
|
||||||
- CSV (human-readable)
|
|
||||||
- DaVinci Resolve Markers CSV
|
|
||||||
- JSON (programmatic)
|
|
||||||
- ffmpeg concat script (auto-cut)
|
|
||||||
"""
|
|
||||||
|
|
||||||
import csv
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
from utils.formats import format_timecode, seconds_to_smpte
|
|
||||||
|
|
||||||
|
|
||||||
def export_edl(segments: list, output_path: str, fps: float = 29.97,
|
|
||||||
title: str = "Singing Segments", source_filename: str = None):
|
|
||||||
"""Export EDL file for DaVinci Resolve."""
|
|
||||||
if source_filename:
|
|
||||||
clip_name = source_filename
|
|
||||||
reel_name_safe = Path(source_filename).stem[:32].replace(" ", "_")
|
|
||||||
else:
|
|
||||||
clip_name = None
|
|
||||||
reel_name_safe = "001"
|
|
||||||
|
|
||||||
rec_offset = 3600.0
|
|
||||||
|
|
||||||
with open(output_path, "w", encoding="utf-8") as f:
|
|
||||||
f.write(f"TITLE: {title}\n")
|
|
||||||
f.write("FCM: NON-DROP FRAME\n\n")
|
|
||||||
|
|
||||||
current_rec_pos = rec_offset
|
|
||||||
for seg in segments:
|
|
||||||
edit_num = f"{seg['index']:03d}"
|
|
||||||
src_in = seconds_to_smpte(seg["start"], fps)
|
|
||||||
src_out = seconds_to_smpte(seg["end"], fps)
|
|
||||||
rec_in = seconds_to_smpte(current_rec_pos, fps)
|
|
||||||
rec_out = seconds_to_smpte(current_rec_pos + seg["duration"], fps)
|
|
||||||
|
|
||||||
f.write(f"{edit_num} {reel_name_safe} V C {src_in} {src_out} {rec_in} {rec_out}\n")
|
|
||||||
|
|
||||||
if clip_name:
|
|
||||||
f.write(f"* FROM CLIP NAME: {clip_name}\n")
|
|
||||||
|
|
||||||
f.write(f"* COMMENT: Song {seg['index']} - Duration {seg['duration']:.0f}s\n\n")
|
|
||||||
current_rec_pos += seg["duration"]
|
|
||||||
|
|
||||||
|
|
||||||
def export_csv(segments: list, output_path: str):
|
|
||||||
"""Export singing segments as CSV."""
|
|
||||||
with open(output_path, "w", newline="", encoding="utf-8") as f:
|
|
||||||
writer = csv.writer(f)
|
|
||||||
writer.writerow(["index", "start_sec", "end_sec", "duration_sec",
|
|
||||||
"start_timecode", "end_timecode"])
|
|
||||||
for seg in segments:
|
|
||||||
writer.writerow([
|
|
||||||
seg["index"],
|
|
||||||
f"{seg['start']:.2f}",
|
|
||||||
f"{seg['end']:.2f}",
|
|
||||||
f"{seg['duration']:.1f}",
|
|
||||||
format_timecode(seg["start"]),
|
|
||||||
format_timecode(seg["end"]),
|
|
||||||
])
|
|
||||||
|
|
||||||
|
|
||||||
def export_markers_csv(segments: list, output_path: str, fps: float = 29.97):
|
|
||||||
"""Export DaVinci Resolve Markers CSV."""
|
|
||||||
with open(output_path, "w", newline="", encoding="utf-8") as f:
|
|
||||||
writer = csv.writer(f)
|
|
||||||
writer.writerow(["#", "Color", "Name", "Start TC", "End TC", "Duration TC", "Notes"])
|
|
||||||
for seg in segments:
|
|
||||||
writer.writerow([
|
|
||||||
seg["index"],
|
|
||||||
"Blue",
|
|
||||||
f"Song {seg['index']}",
|
|
||||||
seconds_to_smpte(seg["start"], fps),
|
|
||||||
seconds_to_smpte(seg["end"], fps),
|
|
||||||
seconds_to_smpte(seg["duration"], fps),
|
|
||||||
f"Duration: {seg['duration']:.0f}s",
|
|
||||||
])
|
|
||||||
|
|
||||||
|
|
||||||
def export_json(segments: list, output_path: str):
|
|
||||||
"""Export as JSON."""
|
|
||||||
with open(output_path, "w", encoding="utf-8") as f:
|
|
||||||
json.dump(segments, f, indent=2, ensure_ascii=False)
|
|
||||||
|
|
||||||
|
|
||||||
def export_ffmpeg_script(segments: list, video_path: str, output_dir: str):
|
|
||||||
"""
|
|
||||||
Export a script that uses ffmpeg to cut and concatenate all
|
|
||||||
singing segments into a single video file.
|
|
||||||
"""
|
|
||||||
video_name = Path(video_path).stem
|
|
||||||
ext = Path(video_path).suffix
|
|
||||||
out_video = str(Path(output_dir) / f"{video_name}_singing_only{ext}")
|
|
||||||
|
|
||||||
is_windows = os.name == "nt" or sys.platform == "win32"
|
|
||||||
script_ext = ".bat" if is_windows else ".sh"
|
|
||||||
script_path = str(Path(output_dir) / f"{video_name}_autocut{script_ext}")
|
|
||||||
concat_list_path = str(Path(output_dir) / f"{video_name}_concat_list.txt")
|
|
||||||
|
|
||||||
lines = []
|
|
||||||
if is_windows:
|
|
||||||
lines.append("@echo off")
|
|
||||||
lines.append("REM Auto-generated ffmpeg script to extract singing segments")
|
|
||||||
lines.append(f'REM Source: {video_path}')
|
|
||||||
lines.append("")
|
|
||||||
else:
|
|
||||||
lines.append("#!/bin/bash")
|
|
||||||
lines.append("# Auto-generated ffmpeg script to extract singing segments")
|
|
||||||
lines.append(f'# Source: {video_path}')
|
|
||||||
lines.append("")
|
|
||||||
|
|
||||||
segment_files = []
|
|
||||||
for seg in segments:
|
|
||||||
seg_file = f"_seg_{seg['index']:03d}{ext}"
|
|
||||||
seg_path = str(Path(output_dir) / seg_file)
|
|
||||||
segment_files.append(seg_file)
|
|
||||||
|
|
||||||
cmd = (f'ffmpeg -y -ss {seg["start"]:.2f} -i "{video_path}" '
|
|
||||||
f'-t {seg["duration"]:.2f} -c copy "{seg_path}"')
|
|
||||||
lines.append(f"echo Extracting Song {seg['index']}...")
|
|
||||||
lines.append(cmd)
|
|
||||||
lines.append("")
|
|
||||||
|
|
||||||
lines.append("echo Creating concat list...")
|
|
||||||
if is_windows:
|
|
||||||
lines.append("(")
|
|
||||||
for sf in segment_files:
|
|
||||||
seg_path = str(Path(output_dir) / sf)
|
|
||||||
lines.append(f" echo file '{seg_path}'")
|
|
||||||
lines.append(f') > "{concat_list_path}"')
|
|
||||||
else:
|
|
||||||
for i, sf in enumerate(segment_files):
|
|
||||||
seg_path = str(Path(output_dir) / sf)
|
|
||||||
op = ">" if i == 0 else ">>"
|
|
||||||
lines.append(f"echo \"file '{seg_path}'\" {op} \"{concat_list_path}\"")
|
|
||||||
|
|
||||||
lines.append("")
|
|
||||||
lines.append("echo Concatenating all segments...")
|
|
||||||
lines.append(f'ffmpeg -y -f concat -safe 0 -i "{concat_list_path}" -c copy "{out_video}"')
|
|
||||||
lines.append("")
|
|
||||||
lines.append("echo Cleaning up temp files...")
|
|
||||||
|
|
||||||
for sf in segment_files:
|
|
||||||
seg_path = str(Path(output_dir) / sf)
|
|
||||||
if is_windows:
|
|
||||||
lines.append(f'del "{seg_path}"')
|
|
||||||
else:
|
|
||||||
lines.append(f'rm -f "{seg_path}"')
|
|
||||||
if is_windows:
|
|
||||||
lines.append(f'del "{concat_list_path}"')
|
|
||||||
else:
|
|
||||||
lines.append(f'rm -f "{concat_list_path}"')
|
|
||||||
|
|
||||||
lines.append("")
|
|
||||||
lines.append(f'echo Done! Output: {out_video}')
|
|
||||||
if is_windows:
|
|
||||||
lines.append("pause")
|
|
||||||
|
|
||||||
with open(script_path, "w", encoding="utf-8") as f:
|
|
||||||
f.write("\n".join(lines))
|
|
||||||
|
|
||||||
if not is_windows:
|
|
||||||
os.chmod(script_path, 0o755)
|
|
||||||
|
|
||||||
return script_path
|
|
||||||
|
|
||||||
|
|
||||||
def export_raw_segments(segments: list, output_path: str):
|
|
||||||
"""Export ALL raw segments (speech/music/noise) as CSV for debugging."""
|
|
||||||
with open(output_path, "w", newline="", encoding="utf-8") as f:
|
|
||||||
writer = csv.writer(f)
|
|
||||||
writer.writerow(["label", "start_sec", "end_sec", "duration_sec",
|
|
||||||
"start_timecode", "end_timecode"])
|
|
||||||
for label, start, end in segments:
|
|
||||||
writer.writerow([
|
|
||||||
label,
|
|
||||||
f"{start:.2f}",
|
|
||||||
f"{end:.2f}",
|
|
||||||
f"{end - start:.1f}",
|
|
||||||
format_timecode(start),
|
|
||||||
format_timecode(end),
|
|
||||||
])
|
|
||||||
@@ -1,99 +0,0 @@
|
|||||||
"""
|
|
||||||
ffmpeg / ffprobe discovery and probing helpers.
|
|
||||||
Shared by both concat and singing detection features.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import shutil
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
# ── Suppress console windows on Windows ──────────────────────────────────────
|
|
||||||
_NO_WINDOW: dict = {}
|
|
||||||
if sys.platform == "win32":
|
|
||||||
_NO_WINDOW["creationflags"] = subprocess.CREATE_NO_WINDOW
|
|
||||||
|
|
||||||
|
|
||||||
def app_dir() -> str:
|
|
||||||
if getattr(sys, "frozen", False):
|
|
||||||
return os.path.dirname(sys.executable)
|
|
||||||
return os.path.dirname(os.path.abspath(__file__))
|
|
||||||
|
|
||||||
|
|
||||||
def find_tool(name: str) -> str | None:
|
|
||||||
"""Locate ffmpeg / ffprobe next to the exe / script, then on PATH."""
|
|
||||||
base = app_dir()
|
|
||||||
for sub in ["_internal", "..", ""]:
|
|
||||||
for variant in [name + ".exe", name]:
|
|
||||||
p = os.path.join(base, sub, variant)
|
|
||||||
if os.path.exists(p):
|
|
||||||
return os.path.abspath(p)
|
|
||||||
return shutil.which(name + ".exe") or shutil.which(name)
|
|
||||||
|
|
||||||
|
|
||||||
def probe_duration(path: str, ffprobe_bin: str | None = None) -> float:
|
|
||||||
"""Get video/audio duration in seconds via ffprobe."""
|
|
||||||
ffprobe = ffprobe_bin or find_tool("ffprobe")
|
|
||||||
if not ffprobe:
|
|
||||||
return 0.0
|
|
||||||
try:
|
|
||||||
r = subprocess.run(
|
|
||||||
[ffprobe, "-v", "error",
|
|
||||||
"-show_entries", "format=duration",
|
|
||||||
"-of", "json", path],
|
|
||||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE,
|
|
||||||
text=True, timeout=20,
|
|
||||||
**_NO_WINDOW,
|
|
||||||
)
|
|
||||||
if r.returncode != 0:
|
|
||||||
return 0.0
|
|
||||||
data = json.loads(r.stdout or "{}")
|
|
||||||
return max(0.0, float(data.get("format", {}).get("duration", 0) or 0))
|
|
||||||
except Exception:
|
|
||||||
return 0.0
|
|
||||||
|
|
||||||
|
|
||||||
def extract_audio(video_path: str, output_wav: str,
|
|
||||||
ffmpeg_bin: str | None = None,
|
|
||||||
progress_cb=None) -> str:
|
|
||||||
"""
|
|
||||||
Extract audio from video file as 16kHz mono WAV.
|
|
||||||
Required format for inaSpeechSegmenter.
|
|
||||||
|
|
||||||
progress_cb: optional callable(message: str) for status updates.
|
|
||||||
"""
|
|
||||||
ffmpeg = ffmpeg_bin or find_tool("ffmpeg")
|
|
||||||
if not ffmpeg:
|
|
||||||
raise FileNotFoundError("ffmpeg not found. Install ffmpeg and add to PATH.")
|
|
||||||
|
|
||||||
if progress_cb:
|
|
||||||
progress_cb(f"Extracting audio from: {Path(video_path).name}")
|
|
||||||
|
|
||||||
cmd = [
|
|
||||||
ffmpeg,
|
|
||||||
"-i", video_path,
|
|
||||||
"-vn",
|
|
||||||
"-acodec", "pcm_s16le",
|
|
||||||
"-ar", "16000",
|
|
||||||
"-ac", "1",
|
|
||||||
"-y",
|
|
||||||
output_wav,
|
|
||||||
]
|
|
||||||
|
|
||||||
result = subprocess.run(
|
|
||||||
cmd,
|
|
||||||
capture_output=True,
|
|
||||||
text=True,
|
|
||||||
timeout=1800,
|
|
||||||
**_NO_WINDOW,
|
|
||||||
)
|
|
||||||
if result.returncode != 0:
|
|
||||||
raise RuntimeError(f"ffmpeg failed:\n{result.stderr[-500:]}")
|
|
||||||
|
|
||||||
if progress_cb:
|
|
||||||
size_mb = os.path.getsize(output_wav) / (1024 * 1024)
|
|
||||||
progress_cb(f"Audio extracted: {size_mb:.1f} MB")
|
|
||||||
|
|
||||||
return output_wav
|
|
||||||
@@ -1,118 +0,0 @@
|
|||||||
"""
|
|
||||||
Singing segment detection via inaSpeechSegmenter.
|
|
||||||
|
|
||||||
Pipeline:
|
|
||||||
1. Run inaSpeechSegmenter to classify speech/music/noise
|
|
||||||
2. Filter for 'music' segments (singing voice is classified as music)
|
|
||||||
3. Merge nearby segments, apply padding, filter by minimum duration
|
|
||||||
"""
|
|
||||||
|
|
||||||
import time
|
|
||||||
from utils.formats import format_timecode
|
|
||||||
|
|
||||||
|
|
||||||
def check_segmenter_available() -> tuple[bool, str]:
|
|
||||||
"""Check if inaSpeechSegmenter is installed."""
|
|
||||||
try:
|
|
||||||
from inaSpeechSegmenter import Segmenter # noqa: F401
|
|
||||||
return True, ""
|
|
||||||
except ImportError:
|
|
||||||
return False, (
|
|
||||||
"inaSpeechSegmenter is not installed.\n\n"
|
|
||||||
"Install it with:\n"
|
|
||||||
" pip install inaSpeechSegmenter\n\n"
|
|
||||||
"You also need TensorFlow:\n"
|
|
||||||
" pip install tensorflow[and-cuda] (GPU)\n"
|
|
||||||
" pip install tensorflow (CPU)"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def run_segmentation(wav_path: str, progress_cb=None) -> list:
|
|
||||||
"""
|
|
||||||
Run inaSpeechSegmenter on audio file.
|
|
||||||
Returns list of (label, start_sec, end_sec) tuples.
|
|
||||||
Labels: 'music', 'speech', 'noise', 'noEnergy'
|
|
||||||
"""
|
|
||||||
from inaSpeechSegmenter import Segmenter
|
|
||||||
|
|
||||||
if progress_cb:
|
|
||||||
progress_cb("Loading segmentation model…")
|
|
||||||
|
|
||||||
start_time = time.time()
|
|
||||||
seg = Segmenter(vad_engine='smn', detect_gender=False)
|
|
||||||
|
|
||||||
if progress_cb:
|
|
||||||
progress_cb("Running audio segmentation (this may take a while)…")
|
|
||||||
|
|
||||||
segments = seg(wav_path)
|
|
||||||
elapsed = time.time() - start_time
|
|
||||||
|
|
||||||
if progress_cb:
|
|
||||||
progress_cb(f"Segmentation complete in {elapsed:.1f}s — {len(segments)} raw segments")
|
|
||||||
|
|
||||||
return segments
|
|
||||||
|
|
||||||
|
|
||||||
def merge_singing_segments(
|
|
||||||
segments: list,
|
|
||||||
max_gap: float = 10.0,
|
|
||||||
min_duration: float = 20.0,
|
|
||||||
padding: float = 2.0,
|
|
||||||
) -> list:
|
|
||||||
"""
|
|
||||||
Filter for 'music' segments and merge nearby ones.
|
|
||||||
|
|
||||||
Returns list of dicts:
|
|
||||||
[{"index", "start", "end", "duration", "original_start", "original_end"}, ...]
|
|
||||||
"""
|
|
||||||
music_segs = [(start, end) for label, start, end in segments if label == "music"]
|
|
||||||
|
|
||||||
if not music_segs:
|
|
||||||
return []
|
|
||||||
|
|
||||||
music_segs.sort(key=lambda x: x[0])
|
|
||||||
|
|
||||||
# Merge segments with gaps smaller than max_gap
|
|
||||||
merged = []
|
|
||||||
current_start, current_end = music_segs[0]
|
|
||||||
|
|
||||||
for start, end in music_segs[1:]:
|
|
||||||
if start - current_end <= max_gap:
|
|
||||||
current_end = max(current_end, end)
|
|
||||||
else:
|
|
||||||
merged.append((current_start, current_end))
|
|
||||||
current_start, current_end = start, end
|
|
||||||
merged.append((current_start, current_end))
|
|
||||||
|
|
||||||
# Apply padding and minimum duration filter
|
|
||||||
results = []
|
|
||||||
idx = 1
|
|
||||||
for start, end in merged:
|
|
||||||
duration = end - start
|
|
||||||
if duration >= min_duration:
|
|
||||||
padded_start = max(0, start - padding)
|
|
||||||
padded_end = end + padding
|
|
||||||
results.append({
|
|
||||||
"index": idx,
|
|
||||||
"start": padded_start,
|
|
||||||
"end": padded_end,
|
|
||||||
"duration": padded_end - padded_start,
|
|
||||||
"original_start": start,
|
|
||||||
"original_end": end,
|
|
||||||
})
|
|
||||||
idx += 1
|
|
||||||
|
|
||||||
return results
|
|
||||||
|
|
||||||
|
|
||||||
def get_segment_stats(segments: list) -> dict:
|
|
||||||
"""Get summary statistics from raw segmentation output."""
|
|
||||||
type_counts = {}
|
|
||||||
type_durations = {}
|
|
||||||
for label, start, end in segments:
|
|
||||||
type_counts[label] = type_counts.get(label, 0) + 1
|
|
||||||
type_durations[label] = type_durations.get(label, 0) + (end - start)
|
|
||||||
return {
|
|
||||||
"counts": type_counts,
|
|
||||||
"durations": type_durations,
|
|
||||||
}
|
|
||||||
13
dev.bat
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
@echo off
|
||||||
|
REM Sing Detect - run in development (hot-reload UI + Rust).
|
||||||
|
REM Requires: Node, Rust (cargo), ffmpeg on PATH.
|
||||||
|
setlocal
|
||||||
|
set "PATH=%USERPROFILE%\.cargo\bin;%PATH%"
|
||||||
|
|
||||||
|
if not exist "node_modules" (
|
||||||
|
echo Installing npm dependencies...
|
||||||
|
call npm install
|
||||||
|
)
|
||||||
|
|
||||||
|
echo Starting Sing Detect (dev)...
|
||||||
|
call npm run tauri dev
|
||||||
14
index.html
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
<!doctype html>
|
||||||
|
<html lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8" />
|
||||||
|
<link rel="stylesheet" href="/src/styles.css" />
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||||
|
<title>Sing Detect</title>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<div id="app"></div>
|
||||||
|
<div class="toast-host" id="toasts"></div>
|
||||||
|
<script type="module" src="/src/main.ts"></script>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
54
pack.bat
Normal file
@@ -0,0 +1,54 @@
|
|||||||
|
@echo off
|
||||||
|
REM ============================================================
|
||||||
|
REM Sing Detect - build a PORTABLE folder for ANY Windows PC.
|
||||||
|
REM
|
||||||
|
REM Run this instead of build.bat when you want a copy-paste app.
|
||||||
|
REM Output: dist-portable\SingDetect\
|
||||||
|
REM Usage: copy that whole folder anywhere, double-click
|
||||||
|
REM "Sing Detect.exe". ffmpeg is bundled inside.
|
||||||
|
REM ============================================================
|
||||||
|
setlocal EnableDelayedExpansion
|
||||||
|
set "PATH=%USERPROFILE%\.cargo\bin;%PATH%"
|
||||||
|
cd /d "%~dp0"
|
||||||
|
|
||||||
|
REM --- 1. Locate ffmpeg.exe + ffprobe.exe --------------------
|
||||||
|
set "FFMPEG="
|
||||||
|
set "FFPROBE="
|
||||||
|
if exist "C:\ffmpeg\bin\ffmpeg.exe" set "FFMPEG=C:\ffmpeg\bin\ffmpeg.exe"
|
||||||
|
if exist "C:\ffmpeg\bin\ffprobe.exe" set "FFPROBE=C:\ffmpeg\bin\ffprobe.exe"
|
||||||
|
if not defined FFMPEG for /f "delims=" %%i in ('where ffmpeg 2^>nul') do if not defined FFMPEG set "FFMPEG=%%i"
|
||||||
|
if not defined FFPROBE for /f "delims=" %%i in ('where ffprobe 2^>nul') do if not defined FFPROBE set "FFPROBE=%%i"
|
||||||
|
|
||||||
|
if not defined FFMPEG ( echo [ERROR] ffmpeg.exe not found. Install ffmpeg or place it in C:\ffmpeg\bin & pause & exit /b 1 )
|
||||||
|
if not defined FFPROBE ( echo [ERROR] ffprobe.exe not found. Install ffmpeg or place it in C:\ffmpeg\bin & pause & exit /b 1 )
|
||||||
|
echo Using ffmpeg: !FFMPEG!
|
||||||
|
echo Using ffprobe: !FFPROBE!
|
||||||
|
|
||||||
|
REM --- 2. Build the release app ------------------------------
|
||||||
|
if not exist "node_modules" call npm install
|
||||||
|
echo Building release (this takes a few minutes the first time)...
|
||||||
|
call npm run tauri build
|
||||||
|
if errorlevel 1 ( echo [ERROR] Build failed. & pause & exit /b 1 )
|
||||||
|
|
||||||
|
REM --- 3. Assemble the portable folder ----------------------
|
||||||
|
set "OUT=dist-portable\SingDetect"
|
||||||
|
if exist "dist-portable" rmdir /s /q "dist-portable"
|
||||||
|
mkdir "%OUT%"
|
||||||
|
copy /y "src-tauri\target\release\sing-detect.exe" "%OUT%\Sing Detect.exe" >nul
|
||||||
|
copy /y "!FFMPEG!" "%OUT%\ffmpeg.exe" >nul
|
||||||
|
copy /y "!FFPROBE!" "%OUT%\ffprobe.exe" >nul
|
||||||
|
xcopy /e /i /y "sidecar" "%OUT%\sidecar" >nul
|
||||||
|
|
||||||
|
echo.
|
||||||
|
echo ============================================================
|
||||||
|
echo DONE. Your portable app is here:
|
||||||
|
echo %CD%\%OUT%
|
||||||
|
echo.
|
||||||
|
echo Copy that whole "SingDetect" folder to any Windows PC and
|
||||||
|
echo double-click "Sing Detect.exe". ffmpeg is included.
|
||||||
|
echo.
|
||||||
|
echo Detection needs Python 3.10 on the target PC. The app has a
|
||||||
|
echo built-in "Test detection engine" button that installs the
|
||||||
|
echo ML packages and model for you on first run.
|
||||||
|
echo ============================================================
|
||||||
|
pause
|
||||||
1371
package-lock.json
generated
Normal file
22
package.json
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
{
|
||||||
|
"name": "sing-detect",
|
||||||
|
"private": true,
|
||||||
|
"version": "2.0.0",
|
||||||
|
"type": "module",
|
||||||
|
"scripts": {
|
||||||
|
"dev": "vite",
|
||||||
|
"build": "tsc && vite build",
|
||||||
|
"preview": "vite preview",
|
||||||
|
"tauri": "tauri"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"@tauri-apps/api": "^2",
|
||||||
|
"@tauri-apps/plugin-opener": "^2",
|
||||||
|
"@tauri-apps/plugin-dialog": "^2"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@tauri-apps/cli": "^2",
|
||||||
|
"vite": "^6.0.3",
|
||||||
|
"typescript": "~5.6.2"
|
||||||
|
}
|
||||||
|
}
|
||||||
180
sidecar/detect.py
Normal file
@@ -0,0 +1,180 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Stream Studio — singing-detection sidecar.
|
||||||
|
|
||||||
|
This is the ONLY Python in the project. It exists solely because the ML model
|
||||||
|
(inaSpeechSegmenter, which classifies singing voice as "music") is TensorFlow-
|
||||||
|
based and has no native Rust/JS equivalent. Everything else — UI, ffmpeg
|
||||||
|
concat, audio extraction, file exports — is handled by the Tauri/Rust host.
|
||||||
|
|
||||||
|
Contract (line-delimited JSON on stdout):
|
||||||
|
{"type": "progress", "message": "..."}
|
||||||
|
{"type": "result", "segments": [{"index","start","end","duration"}...],
|
||||||
|
"stats": {"music": 123.4, "speech": 456.7, ...}}
|
||||||
|
{"type": "error", "message": "..."}
|
||||||
|
|
||||||
|
The host extracts a 16 kHz mono WAV and passes it via --wav, so this script
|
||||||
|
never touches ffmpeg.
|
||||||
|
|
||||||
|
Run with Python 3.10 (TensorFlow does not support 3.12+):
|
||||||
|
py -3.10 detect.py --wav audio.wav --gap 10 --min-duration 20 --padding 2
|
||||||
|
"""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
|
||||||
|
|
||||||
|
def emit(obj: dict) -> None:
|
||||||
|
"""Write one JSON record per line and flush immediately."""
|
||||||
|
sys.stdout.write(json.dumps(obj, ensure_ascii=False) + "\n")
|
||||||
|
sys.stdout.flush()
|
||||||
|
|
||||||
|
|
||||||
|
def progress(msg: str) -> None:
|
||||||
|
emit({"type": "progress", "message": msg})
|
||||||
|
|
||||||
|
|
||||||
|
def merge_singing_segments(segments, max_gap, min_duration, padding):
|
||||||
|
"""Filter for 'music' segments and merge nearby ones into songs."""
|
||||||
|
music = sorted((s, e) for label, s, e in segments if label == "music")
|
||||||
|
if not music:
|
||||||
|
return []
|
||||||
|
|
||||||
|
merged = []
|
||||||
|
cur_start, cur_end = music[0]
|
||||||
|
for s, e in music[1:]:
|
||||||
|
if s - cur_end <= max_gap:
|
||||||
|
cur_end = max(cur_end, e)
|
||||||
|
else:
|
||||||
|
merged.append((cur_start, cur_end))
|
||||||
|
cur_start, cur_end = s, e
|
||||||
|
merged.append((cur_start, cur_end))
|
||||||
|
|
||||||
|
results = []
|
||||||
|
idx = 1
|
||||||
|
for s, e in merged:
|
||||||
|
if (e - s) >= min_duration:
|
||||||
|
ps = max(0.0, s - padding)
|
||||||
|
pe = e + padding
|
||||||
|
results.append({
|
||||||
|
"index": idx,
|
||||||
|
"start": round(ps, 3),
|
||||||
|
"end": round(pe, 3),
|
||||||
|
"duration": round(pe - ps, 3),
|
||||||
|
})
|
||||||
|
idx += 1
|
||||||
|
return results
|
||||||
|
|
||||||
|
|
||||||
|
def _model_present() -> bool:
|
||||||
|
"""Best-effort check for cached model weights, without downloading."""
|
||||||
|
try:
|
||||||
|
import glob
|
||||||
|
import os
|
||||||
|
|
||||||
|
import inaSpeechSegmenter
|
||||||
|
|
||||||
|
roots = [
|
||||||
|
os.path.dirname(inaSpeechSegmenter.__file__),
|
||||||
|
os.path.join(os.path.expanduser("~"), ".keras"),
|
||||||
|
]
|
||||||
|
for root in roots:
|
||||||
|
for ext in ("*.hdf5", "*.h5", "*.keras", "*.pb"):
|
||||||
|
if glob.glob(os.path.join(root, "**", ext), recursive=True):
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def run_check() -> int:
|
||||||
|
"""Report whether the ML packages (and weights) are available."""
|
||||||
|
record = {"type": "check", "packages": False, "model": False, "detail": ""}
|
||||||
|
try:
|
||||||
|
import inaSpeechSegmenter # noqa: F401
|
||||||
|
import tensorflow # noqa: F401
|
||||||
|
record["packages"] = True
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
record["detail"] = f"Packages not installed ({type(exc).__name__})."
|
||||||
|
emit(record)
|
||||||
|
return 0
|
||||||
|
record["model"] = _model_present()
|
||||||
|
record["detail"] = (
|
||||||
|
"Ready — packages and model weights are installed."
|
||||||
|
if record["model"]
|
||||||
|
else "Packages installed; model weights will download on first run."
|
||||||
|
)
|
||||||
|
emit(record)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def run_warmup() -> int:
|
||||||
|
"""Instantiate the model once, forcing the weight download if needed."""
|
||||||
|
try:
|
||||||
|
progress("Loading detection model (downloading weights if needed)…")
|
||||||
|
from inaSpeechSegmenter import Segmenter
|
||||||
|
|
||||||
|
Segmenter(vad_engine="smn", detect_gender=False)
|
||||||
|
emit({"type": "done", "message": "Model ready."})
|
||||||
|
return 0
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
emit({"type": "error", "message": f"{type(exc).__name__}: {exc}"})
|
||||||
|
return 1
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
ap = argparse.ArgumentParser()
|
||||||
|
ap.add_argument("--wav")
|
||||||
|
ap.add_argument("--gap", type=float, default=10.0)
|
||||||
|
ap.add_argument("--min-duration", type=float, default=20.0)
|
||||||
|
ap.add_argument("--padding", type=float, default=2.0)
|
||||||
|
ap.add_argument("--check", action="store_true", help="probe environment, then exit")
|
||||||
|
ap.add_argument("--warmup", action="store_true", help="download/verify weights, then exit")
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
if args.check:
|
||||||
|
return run_check()
|
||||||
|
if args.warmup:
|
||||||
|
return run_warmup()
|
||||||
|
if not args.wav:
|
||||||
|
emit({"type": "error", "message": "--wav is required for detection."})
|
||||||
|
return 1
|
||||||
|
|
||||||
|
try:
|
||||||
|
progress("Loading detection model (first run downloads weights)…")
|
||||||
|
from inaSpeechSegmenter import Segmenter
|
||||||
|
except ImportError:
|
||||||
|
emit({
|
||||||
|
"type": "error",
|
||||||
|
"message": (
|
||||||
|
"inaSpeechSegmenter is not installed for this Python.\n"
|
||||||
|
"Install it with:\n"
|
||||||
|
" py -3.10 -m pip install inaSpeechSegmenter tensorflow"
|
||||||
|
),
|
||||||
|
})
|
||||||
|
return 1
|
||||||
|
|
||||||
|
try:
|
||||||
|
t0 = time.time()
|
||||||
|
seg = Segmenter(vad_engine="smn", detect_gender=False)
|
||||||
|
progress("Analysing audio — this can take a while on long streams…")
|
||||||
|
raw = seg(args.wav)
|
||||||
|
progress(f"Segmentation done in {time.time() - t0:.0f}s ({len(raw)} raw segments).")
|
||||||
|
|
||||||
|
stats = {}
|
||||||
|
for label, s, e in raw:
|
||||||
|
stats[label] = round(stats.get(label, 0.0) + (e - s), 1)
|
||||||
|
|
||||||
|
songs = merge_singing_segments(raw, args.gap, args.min_duration, args.padding)
|
||||||
|
progress(f"Found {len(songs)} singing segment(s) after merge + filter.")
|
||||||
|
emit({"type": "result", "segments": songs, "stats": stats})
|
||||||
|
return 0
|
||||||
|
except Exception as exc: # noqa: BLE001 — surface any model/runtime error to the host
|
||||||
|
emit({"type": "error", "message": f"{type(exc).__name__}: {exc}"})
|
||||||
|
return 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
12
sidecar/requirements.txt
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
# Stream Studio detection sidecar — install into a Python 3.10 environment.
|
||||||
|
# py -3.10 -m pip install -r requirements.txt
|
||||||
|
#
|
||||||
|
# TensorFlow does NOT support Python 3.12+, which is why the sidecar pins 3.10
|
||||||
|
# even though the rest of the toolchain can use newer Python.
|
||||||
|
|
||||||
|
# NumPy MUST stay <2: TensorFlow 2.11 is built against NumPy 1.x and crashes on
|
||||||
|
# NumPy 2.x with "_pywrap_bfloat16.TF_bfloat16_type() () -> handle". Keep this pin
|
||||||
|
# above tensorflow so the resolver installs the compatible NumPy.
|
||||||
|
numpy<2
|
||||||
|
inaSpeechSegmenter
|
||||||
|
tensorflow # use: tensorflow[and-cuda] for NVIDIA GPU acceleration
|
||||||
25
sidecar/setup_sidecar.bat
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
@echo off
|
||||||
|
REM Install the singing-detection ML dependencies into Python 3.10.
|
||||||
|
REM TensorFlow does not support Python 3.12+, so we pin 3.10 via the py launcher.
|
||||||
|
|
||||||
|
py -3.10 --version >nul 2>&1
|
||||||
|
if errorlevel 1 (
|
||||||
|
echo Python 3.10 not found via the "py" launcher.
|
||||||
|
echo Install Python 3.10 from https://www.python.org/downloads/release/python-31011/
|
||||||
|
echo Then re-run this script.
|
||||||
|
pause
|
||||||
|
exit /b 1
|
||||||
|
)
|
||||||
|
|
||||||
|
echo Installing inaSpeechSegmenter + TensorFlow into Python 3.10...
|
||||||
|
py -3.10 -m pip install --upgrade pip
|
||||||
|
py -3.10 -m pip install -r "%~dp0requirements.txt"
|
||||||
|
|
||||||
|
REM Re-pin NumPy last: TensorFlow can drag NumPy 2.x back in, which crashes TF
|
||||||
|
REM with "_pywrap_bfloat16.TF_bfloat16_type() () -> handle".
|
||||||
|
py -3.10 -m pip install "numpy<2"
|
||||||
|
|
||||||
|
echo.
|
||||||
|
echo Verifying...
|
||||||
|
py -3.10 -c "from inaSpeechSegmenter import Segmenter; print('inaSpeechSegmenter OK')"
|
||||||
|
pause
|
||||||
7
src-tauri/.gitignore
vendored
Normal file
@@ -0,0 +1,7 @@
|
|||||||
|
# Generated by Cargo
|
||||||
|
# will have compiled files and executables
|
||||||
|
/target/
|
||||||
|
|
||||||
|
# Generated by Tauri
|
||||||
|
# will have schema files for capabilities auto-completion
|
||||||
|
/gen/schemas
|
||||||
5045
src-tauri/Cargo.lock
generated
Normal file
33
src-tauri/Cargo.toml
Normal file
@@ -0,0 +1,33 @@
|
|||||||
|
[package]
|
||||||
|
name = "sing-detect"
|
||||||
|
version = "2.0.0"
|
||||||
|
description = "Sing Detect — find singing segments in stream recordings"
|
||||||
|
authors = ["youfu"]
|
||||||
|
edition = "2021"
|
||||||
|
|
||||||
|
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
|
||||||
|
|
||||||
|
[lib]
|
||||||
|
# The `_lib` suffix may seem redundant but it is necessary
|
||||||
|
# to make the lib name unique and wouldn't conflict with the bin name.
|
||||||
|
# This seems to be only an issue on Windows, see https://github.com/rust-lang/cargo/issues/8519
|
||||||
|
name = "sing_detect_lib"
|
||||||
|
crate-type = ["staticlib", "cdylib", "rlib"]
|
||||||
|
|
||||||
|
[build-dependencies]
|
||||||
|
tauri-build = { version = "2", features = [] }
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
tauri = { version = "2", features = [] }
|
||||||
|
tauri-plugin-opener = "2"
|
||||||
|
tauri-plugin-dialog = "2"
|
||||||
|
serde = { version = "1", features = ["derive"] }
|
||||||
|
serde_json = "1"
|
||||||
|
|
||||||
|
[profile.release]
|
||||||
|
opt-level = "s"
|
||||||
|
lto = true
|
||||||
|
codegen-units = 1
|
||||||
|
strip = true
|
||||||
|
panic = "abort"
|
||||||
|
|
||||||
3
src-tauri/build.rs
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
fn main() {
|
||||||
|
tauri_build::build()
|
||||||
|
}
|
||||||
15
src-tauri/capabilities/default.json
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
{
|
||||||
|
"$schema": "../gen/schemas/desktop-schema.json",
|
||||||
|
"identifier": "default",
|
||||||
|
"description": "Capability for the main window",
|
||||||
|
"windows": ["main"],
|
||||||
|
"permissions": [
|
||||||
|
"core:default",
|
||||||
|
"core:event:default",
|
||||||
|
"core:window:allow-start-dragging",
|
||||||
|
"opener:default",
|
||||||
|
"opener:allow-reveal-item-in-dir",
|
||||||
|
"dialog:default",
|
||||||
|
"dialog:allow-open"
|
||||||
|
]
|
||||||
|
}
|
||||||
BIN
src-tauri/icons/128x128.png
Normal file
|
After Width: | Height: | Size: 3.4 KiB |
BIN
src-tauri/icons/128x128@2x.png
Normal file
|
After Width: | Height: | Size: 6.8 KiB |
BIN
src-tauri/icons/32x32.png
Normal file
|
After Width: | Height: | Size: 974 B |
BIN
src-tauri/icons/Square107x107Logo.png
Normal file
|
After Width: | Height: | Size: 2.8 KiB |
BIN
src-tauri/icons/Square142x142Logo.png
Normal file
|
After Width: | Height: | Size: 3.8 KiB |
BIN
src-tauri/icons/Square150x150Logo.png
Normal file
|
After Width: | Height: | Size: 3.9 KiB |
BIN
src-tauri/icons/Square284x284Logo.png
Normal file
|
After Width: | Height: | Size: 7.6 KiB |
BIN
src-tauri/icons/Square30x30Logo.png
Normal file
|
After Width: | Height: | Size: 903 B |
BIN
src-tauri/icons/Square310x310Logo.png
Normal file
|
After Width: | Height: | Size: 8.4 KiB |
BIN
src-tauri/icons/Square44x44Logo.png
Normal file
|
After Width: | Height: | Size: 1.3 KiB |
BIN
src-tauri/icons/Square71x71Logo.png
Normal file
|
After Width: | Height: | Size: 2.0 KiB |
BIN
src-tauri/icons/Square89x89Logo.png
Normal file
|
After Width: | Height: | Size: 2.4 KiB |
BIN
src-tauri/icons/StoreLogo.png
Normal file
|
After Width: | Height: | Size: 1.5 KiB |
BIN
src-tauri/icons/icon.icns
Normal file
BIN
src-tauri/icons/icon.ico
Normal file
|
After Width: | Height: | Size: 85 KiB |
BIN
src-tauri/icons/icon.png
Normal file
|
After Width: | Height: | Size: 14 KiB |
380
src-tauri/src/detect.rs
Normal file
@@ -0,0 +1,380 @@
|
|||||||
|
//! Singing-segment detection.
|
||||||
|
//!
|
||||||
|
//! Rust owns the fast/IO parts (audio extraction via ffmpeg, progress events,
|
||||||
|
//! and all file exports). The Python sidecar owns *only* the ML step
|
||||||
|
//! (inaSpeechSegmenter + TensorFlow), which has no native equivalent.
|
||||||
|
//!
|
||||||
|
//! Protocol: the sidecar prints one JSON object per line to stdout:
|
||||||
|
//! {"type":"progress","message":"…"}
|
||||||
|
//! {"type":"result","segments":[{index,start,end,duration}, …],
|
||||||
|
//! "stats":{"music":123.4,"speech":456.7,…}}
|
||||||
|
//! {"type":"error","message":"…"}
|
||||||
|
|
||||||
|
use std::io::{BufRead, BufReader};
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
use std::process::{Command, Stdio};
|
||||||
|
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use serde_json::Value;
|
||||||
|
use tauri::{AppHandle, Emitter};
|
||||||
|
|
||||||
|
use crate::tools::hide_window;
|
||||||
|
use crate::AppState;
|
||||||
|
|
||||||
|
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||||
|
pub struct Segment {
|
||||||
|
pub index: u32,
|
||||||
|
pub start: f64,
|
||||||
|
pub end: f64,
|
||||||
|
pub duration: f64,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Clone, Serialize)]
|
||||||
|
pub struct DetectResult {
|
||||||
|
pub segments: Vec<Segment>,
|
||||||
|
pub stats: Value,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Clone, Serialize)]
|
||||||
|
struct DetectProgress {
|
||||||
|
message: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Environment readiness for the detection feature.
|
||||||
|
#[derive(Clone, Serialize)]
|
||||||
|
pub struct DetectionStatus {
|
||||||
|
pub python: bool,
|
||||||
|
pub packages: bool,
|
||||||
|
pub model: bool,
|
||||||
|
pub ready: bool,
|
||||||
|
pub detail: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Clone, Serialize)]
|
||||||
|
struct SetupProgress {
|
||||||
|
message: String,
|
||||||
|
pct: f64,
|
||||||
|
}
|
||||||
|
|
||||||
|
fn emit(app: &AppHandle, message: impl Into<String>) {
|
||||||
|
let _ = app.emit("detect-progress", DetectProgress { message: message.into() });
|
||||||
|
}
|
||||||
|
|
||||||
|
fn emit_setup(app: &AppHandle, message: impl Into<String>, pct: f64) {
|
||||||
|
let _ = app.emit("detect-setup-progress", SetupProgress { message: message.into(), pct });
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Probe whether Python 3.10 + the ML packages (and weights) are available.
|
||||||
|
/// Never installs or downloads anything — purely a read-only check.
|
||||||
|
pub fn check_env() -> DetectionStatus {
|
||||||
|
let not_ready = |python: bool, detail: &str| DetectionStatus {
|
||||||
|
python,
|
||||||
|
packages: false,
|
||||||
|
model: false,
|
||||||
|
ready: false,
|
||||||
|
detail: detail.into(),
|
||||||
|
};
|
||||||
|
|
||||||
|
let script = match sidecar_script() {
|
||||||
|
Some(s) => s,
|
||||||
|
None => return not_ready(false, "detect.py was not found next to the app."),
|
||||||
|
};
|
||||||
|
|
||||||
|
let mut cmd = python_command(&script);
|
||||||
|
cmd.arg("--check");
|
||||||
|
cmd.stdout(Stdio::piped()).stderr(Stdio::piped());
|
||||||
|
cmd.env("PYTHONIOENCODING", "utf-8");
|
||||||
|
hide_window(&mut cmd);
|
||||||
|
|
||||||
|
let out = match cmd.output() {
|
||||||
|
Ok(o) => o,
|
||||||
|
Err(_) => {
|
||||||
|
return not_ready(
|
||||||
|
false,
|
||||||
|
"Python 3.10 was not found. Install Python 3.10, then click Test again.",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
let stdout = String::from_utf8_lossy(&out.stdout);
|
||||||
|
for line in stdout.lines().rev() {
|
||||||
|
if let Ok(v) = serde_json::from_str::<Value>(line.trim()) {
|
||||||
|
if v.get("type").and_then(|t| t.as_str()) == Some("check") {
|
||||||
|
let packages = v.get("packages").and_then(|b| b.as_bool()).unwrap_or(false);
|
||||||
|
let model = v.get("model").and_then(|b| b.as_bool()).unwrap_or(false);
|
||||||
|
let detail = v.get("detail").and_then(|s| s.as_str()).unwrap_or("").to_string();
|
||||||
|
return DetectionStatus { python: true, packages, model, ready: packages, detail };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Python launcher ran but produced no status — most likely 3.10 isn't installed.
|
||||||
|
not_ready(
|
||||||
|
false,
|
||||||
|
"Python 3.10 was not found. Install Python 3.10, then click Test again.",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Stream a child command's output to the setup progress channel, holding the
|
||||||
|
/// bar at `pct`. Returns an error (with captured stderr) on non-zero exit.
|
||||||
|
fn run_streamed(app: &AppHandle, mut cmd: Command, pct: f64) -> Result<(), String> {
|
||||||
|
cmd.stdout(Stdio::piped()).stderr(Stdio::piped());
|
||||||
|
cmd.env("PYTHONUNBUFFERED", "1").env("PYTHONIOENCODING", "utf-8");
|
||||||
|
hide_window(&mut cmd);
|
||||||
|
|
||||||
|
let mut child = cmd.spawn().map_err(|e| format!("Could not start Python ({e})."))?;
|
||||||
|
let stdout = child.stdout.take();
|
||||||
|
let stderr = child.stderr.take();
|
||||||
|
|
||||||
|
if let Some(out) = stdout {
|
||||||
|
for line in BufReader::new(out).lines().map_while(Result::ok) {
|
||||||
|
let line = line.trim();
|
||||||
|
if !line.is_empty() {
|
||||||
|
// detect.py emits JSON; pip emits plain text — show whichever.
|
||||||
|
let msg = serde_json::from_str::<Value>(line)
|
||||||
|
.ok()
|
||||||
|
.and_then(|v| v.get("message").and_then(|m| m.as_str()).map(String::from))
|
||||||
|
.unwrap_or_else(|| line.to_string());
|
||||||
|
emit_setup(app, msg, pct);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let mut stderr_text = String::new();
|
||||||
|
if let Some(mut se) = stderr {
|
||||||
|
use std::io::Read;
|
||||||
|
let _ = se.read_to_string(&mut stderr_text);
|
||||||
|
}
|
||||||
|
let status = child.wait().map_err(|e| format!("process wait failed: {e}"))?;
|
||||||
|
if !status.success() {
|
||||||
|
let tail: String = stderr_text.chars().rev().take(900).collect::<String>().chars().rev().collect();
|
||||||
|
return Err(if tail.trim().is_empty() { "Step failed.".into() } else { tail });
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Install the detection ML stack into Python 3.10 and pre-download the model
|
||||||
|
/// weights, streaming progress to the UI. Requires Python 3.10 to be present.
|
||||||
|
pub fn setup_env(app: &AppHandle) -> Result<(), String> {
|
||||||
|
let script = sidecar_script().ok_or("detect.py was not found next to the app.")?;
|
||||||
|
|
||||||
|
// Preflight: confirm Python 3.10 exists before we try to install into it.
|
||||||
|
let mut ver = python_base();
|
||||||
|
ver.arg("--version");
|
||||||
|
ver.stdout(Stdio::piped()).stderr(Stdio::piped());
|
||||||
|
hide_window(&mut ver);
|
||||||
|
match ver.output() {
|
||||||
|
Ok(o) if o.status.success() => {}
|
||||||
|
_ => {
|
||||||
|
return Err(
|
||||||
|
"Python 3.10 was not found.\n\nInstall it from \
|
||||||
|
https://www.python.org/downloads/release/python-31011/ \
|
||||||
|
(or run: winget install Python.Python.3.10), then click Test again."
|
||||||
|
.into(),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
emit_setup(app, "Upgrading pip…", 0.05);
|
||||||
|
let mut pip_up = python_base();
|
||||||
|
pip_up.args(["-m", "pip", "install", "--upgrade", "pip"]);
|
||||||
|
run_streamed(app, pip_up, 0.05)?;
|
||||||
|
|
||||||
|
emit_setup(app, "Installing inaSpeechSegmenter + TensorFlow (large — a few minutes)…", 0.15);
|
||||||
|
let mut pip_main = python_base();
|
||||||
|
pip_main.args(["-m", "pip", "install", "inaSpeechSegmenter", "tensorflow"]);
|
||||||
|
run_streamed(app, pip_main, 0.15)?;
|
||||||
|
|
||||||
|
// TensorFlow can drag NumPy 2.x back in, which crashes it; pin <2 last.
|
||||||
|
emit_setup(app, "Pinning NumPy < 2…", 0.70);
|
||||||
|
let mut pip_np = python_base();
|
||||||
|
pip_np.args(["-m", "pip", "install", "numpy<2"]);
|
||||||
|
run_streamed(app, pip_np, 0.70)?;
|
||||||
|
|
||||||
|
emit_setup(app, "Downloading model weights…", 0.82);
|
||||||
|
let mut warm = python_command(&script);
|
||||||
|
warm.arg("--warmup");
|
||||||
|
run_streamed(app, warm, 0.82)?;
|
||||||
|
|
||||||
|
emit_setup(app, "Setup complete.", 1.0);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Locate the sidecar script: bundled resources first, dev tree as fallback.
|
||||||
|
fn sidecar_script() -> Option<PathBuf> {
|
||||||
|
if let Ok(p) = std::env::var("STREAM_STUDIO_SIDECAR") {
|
||||||
|
let pb = PathBuf::from(p);
|
||||||
|
if pb.exists() {
|
||||||
|
return Some(pb);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if let Ok(exe) = std::env::current_exe() {
|
||||||
|
if let Some(base) = exe.parent() {
|
||||||
|
for sub in ["sidecar", "resources/sidecar", "../sidecar"] {
|
||||||
|
let cand = base.join(sub).join("detect.py");
|
||||||
|
if cand.exists() {
|
||||||
|
return Some(cand);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Dev fallback: <crate>/../sidecar/detect.py
|
||||||
|
let dev = Path::new(env!("CARGO_MANIFEST_DIR")).join("../sidecar/detect.py");
|
||||||
|
if dev.exists() {
|
||||||
|
return Some(dev);
|
||||||
|
}
|
||||||
|
None
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Build the bare python invocation (no script/args). Prefers the Windows
|
||||||
|
/// `py -3.10` launcher (TensorFlow does not yet support 3.12+), then a custom
|
||||||
|
/// override, then plain `python`. Callers append a script or `-m pip …`.
|
||||||
|
fn python_base() -> Command {
|
||||||
|
if let Ok(custom) = std::env::var("STREAM_STUDIO_PYTHON") {
|
||||||
|
return Command::new(custom);
|
||||||
|
}
|
||||||
|
#[cfg(windows)]
|
||||||
|
{
|
||||||
|
// The py launcher lets us pin 3.10 regardless of the default install.
|
||||||
|
let mut c = Command::new("py");
|
||||||
|
c.args(["-3.10"]);
|
||||||
|
return c;
|
||||||
|
}
|
||||||
|
#[allow(unreachable_code)]
|
||||||
|
Command::new("python")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `python <script>` ready for detection args.
|
||||||
|
fn python_command(script: &Path) -> Command {
|
||||||
|
let mut c = python_base();
|
||||||
|
c.arg(script);
|
||||||
|
c
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Extract 16 kHz mono WAV (the format inaSpeechSegmenter expects).
|
||||||
|
fn extract_audio(app: &AppHandle, ffmpeg: &str, video: &str, wav: &Path) -> Result<(), String> {
|
||||||
|
emit(app, "Extracting audio (16 kHz mono)…");
|
||||||
|
let mut cmd = Command::new(ffmpeg);
|
||||||
|
cmd.args([
|
||||||
|
"-y", "-i", video,
|
||||||
|
"-vn", "-acodec", "pcm_s16le", "-ar", "16000", "-ac", "1",
|
||||||
|
"-hide_banner", "-loglevel", "error",
|
||||||
|
&wav.to_string_lossy(),
|
||||||
|
]);
|
||||||
|
cmd.stdout(Stdio::null()).stderr(Stdio::piped());
|
||||||
|
hide_window(&mut cmd);
|
||||||
|
let out = cmd.output().map_err(|e| format!("ffmpeg audio extract failed: {e}"))?;
|
||||||
|
if !out.status.success() {
|
||||||
|
return Err(format!("Audio extraction failed:\n{}", String::from_utf8_lossy(&out.stderr)));
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn run(
|
||||||
|
app: &AppHandle,
|
||||||
|
state: &AppState,
|
||||||
|
ffmpeg: &str,
|
||||||
|
video: String,
|
||||||
|
gap: f64,
|
||||||
|
min_duration: f64,
|
||||||
|
padding: f64,
|
||||||
|
) -> Result<DetectResult, String> {
|
||||||
|
state.cancel.store(false, std::sync::atomic::Ordering::SeqCst);
|
||||||
|
|
||||||
|
let script = sidecar_script().ok_or(
|
||||||
|
"Detection sidecar not found. Expected sidecar/detect.py next to the app.",
|
||||||
|
)?;
|
||||||
|
|
||||||
|
// Temp WAV beside the source video.
|
||||||
|
let wav = Path::new(&video)
|
||||||
|
.with_extension("")
|
||||||
|
.to_string_lossy()
|
||||||
|
.to_string();
|
||||||
|
let wav = PathBuf::from(format!("{wav}.stream_studio.wav"));
|
||||||
|
|
||||||
|
extract_audio(app, ffmpeg, &video, &wav)?;
|
||||||
|
|
||||||
|
let mut cmd = python_command(&script);
|
||||||
|
cmd.args([
|
||||||
|
"--wav", &wav.to_string_lossy(),
|
||||||
|
"--gap", &gap.to_string(),
|
||||||
|
"--min-duration", &min_duration.to_string(),
|
||||||
|
"--padding", &padding.to_string(),
|
||||||
|
]);
|
||||||
|
cmd.stdout(Stdio::piped()).stderr(Stdio::piped());
|
||||||
|
cmd.env("PYTHONUNBUFFERED", "1").env("PYTHONIOENCODING", "utf-8");
|
||||||
|
hide_window(&mut cmd);
|
||||||
|
|
||||||
|
emit(app, "Loading detection model…");
|
||||||
|
let mut child = cmd.spawn().map_err(|e| {
|
||||||
|
format!("Could not start Python sidecar ({e}). Is Python 3.10 + inaSpeechSegmenter installed?")
|
||||||
|
})?;
|
||||||
|
let stdout = child.stdout.take().ok_or("No sidecar stdout")?;
|
||||||
|
let stderr = child.stderr.take();
|
||||||
|
*state.proc.lock().unwrap() = Some(child);
|
||||||
|
|
||||||
|
let mut result: Option<DetectResult> = None;
|
||||||
|
let mut err_msg: Option<String> = None;
|
||||||
|
|
||||||
|
for line in BufReader::new(stdout).lines() {
|
||||||
|
let line = match line {
|
||||||
|
Ok(l) => l,
|
||||||
|
Err(_) => break,
|
||||||
|
};
|
||||||
|
let line = line.trim();
|
||||||
|
if line.is_empty() {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let parsed: Value = match serde_json::from_str(line) {
|
||||||
|
Ok(v) => v,
|
||||||
|
Err(_) => continue, // ignore stray non-JSON output
|
||||||
|
};
|
||||||
|
match parsed.get("type").and_then(|t| t.as_str()) {
|
||||||
|
Some("progress") => {
|
||||||
|
if let Some(m) = parsed.get("message").and_then(|m| m.as_str()) {
|
||||||
|
emit(app, m);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Some("result") => {
|
||||||
|
let segments: Vec<Segment> = serde_json::from_value(
|
||||||
|
parsed.get("segments").cloned().unwrap_or(Value::Array(vec![])),
|
||||||
|
)
|
||||||
|
.unwrap_or_default();
|
||||||
|
let stats = parsed.get("stats").cloned().unwrap_or(Value::Null);
|
||||||
|
result = Some(DetectResult { segments, stats });
|
||||||
|
}
|
||||||
|
Some("error") => {
|
||||||
|
err_msg = parsed
|
||||||
|
.get("message")
|
||||||
|
.and_then(|m| m.as_str())
|
||||||
|
.map(|s| s.to_string());
|
||||||
|
}
|
||||||
|
_ => {}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let mut child = state.proc.lock().unwrap().take().ok_or("Sidecar vanished")?;
|
||||||
|
let status = child.wait().map_err(|e| format!("sidecar wait failed: {e}"))?;
|
||||||
|
let mut stderr_text = String::new();
|
||||||
|
if let Some(mut se) = stderr {
|
||||||
|
use std::io::Read;
|
||||||
|
let _ = se.read_to_string(&mut stderr_text);
|
||||||
|
}
|
||||||
|
|
||||||
|
let _ = std::fs::remove_file(&wav);
|
||||||
|
|
||||||
|
if state.cancel.load(std::sync::atomic::Ordering::SeqCst) {
|
||||||
|
return Err("__cancelled__".into());
|
||||||
|
}
|
||||||
|
if let Some(m) = err_msg {
|
||||||
|
return Err(m);
|
||||||
|
}
|
||||||
|
if let Some(r) = result {
|
||||||
|
return Ok(r);
|
||||||
|
}
|
||||||
|
if !status.success() {
|
||||||
|
let tail: String = stderr_text.chars().rev().take(800).collect::<String>().chars().rev().collect();
|
||||||
|
return Err(format!("Detection failed.\n{tail}"));
|
||||||
|
}
|
||||||
|
Err("Detection produced no result.".into())
|
||||||
|
}
|
||||||
129
src-tauri/src/export.rs
Normal file
@@ -0,0 +1,129 @@
|
|||||||
|
//! Export detected singing segments to editor-friendly formats.
|
||||||
|
//! EDL + DaVinci marker CSV ported 1:1 from the proven Python implementation.
|
||||||
|
|
||||||
|
use std::fmt::Write as _;
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
|
||||||
|
use serde::Serialize;
|
||||||
|
|
||||||
|
use crate::detect::Segment;
|
||||||
|
|
||||||
|
#[derive(Clone, Serialize)]
|
||||||
|
pub struct ExportResult {
|
||||||
|
pub files: Vec<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
fn timecode(seconds: f64, fps: f64) -> String {
|
||||||
|
let total_frames = (seconds * fps) as i64;
|
||||||
|
let fps_i = fps.round() as i64;
|
||||||
|
let ff = total_frames % fps_i;
|
||||||
|
let total_seconds = total_frames / fps_i;
|
||||||
|
let ss = total_seconds % 60;
|
||||||
|
let total_minutes = total_seconds / 60;
|
||||||
|
let mm = total_minutes % 60;
|
||||||
|
let hh = total_minutes / 60;
|
||||||
|
format!("{hh:02}:{mm:02}:{ss:02}:{ff:02}")
|
||||||
|
}
|
||||||
|
|
||||||
|
fn hms(seconds: f64) -> String {
|
||||||
|
let s = seconds as i64;
|
||||||
|
format!("{:02}:{:02}:{:02}", s / 3600, (s % 3600) / 60, s % 60)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn write_edl(segs: &[Segment], path: &Path, fps: f64, title: &str, source: &str) -> std::io::Result<()> {
|
||||||
|
let reel = Path::new(source)
|
||||||
|
.file_stem()
|
||||||
|
.and_then(|s| s.to_str())
|
||||||
|
.unwrap_or("001")
|
||||||
|
.chars()
|
||||||
|
.take(32)
|
||||||
|
.collect::<String>()
|
||||||
|
.replace(' ', "_");
|
||||||
|
|
||||||
|
let mut out = String::new();
|
||||||
|
let _ = writeln!(out, "TITLE: {title}");
|
||||||
|
let _ = writeln!(out, "FCM: NON-DROP FRAME\n");
|
||||||
|
|
||||||
|
let mut rec_pos = 3600.0_f64; // start at 01:00:00:00
|
||||||
|
for s in segs {
|
||||||
|
let edit = format!("{:03}", s.index);
|
||||||
|
let src_in = timecode(s.start, fps);
|
||||||
|
let src_out = timecode(s.end, fps);
|
||||||
|
let rec_in = timecode(rec_pos, fps);
|
||||||
|
let rec_out = timecode(rec_pos + s.duration, fps);
|
||||||
|
let _ = writeln!(out, "{edit} {reel} V C {src_in} {src_out} {rec_in} {rec_out}");
|
||||||
|
let _ = writeln!(out, "* FROM CLIP NAME: {source}");
|
||||||
|
let _ = writeln!(out, "* COMMENT: Song {} - Duration {:.0}s\n", s.index, s.duration);
|
||||||
|
rec_pos += s.duration;
|
||||||
|
}
|
||||||
|
std::fs::write(path, out)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn write_markers_csv(segs: &[Segment], path: &Path, fps: f64) -> std::io::Result<()> {
|
||||||
|
let mut out = String::from("#,Color,Name,Start TC,End TC,Duration TC,Notes\n");
|
||||||
|
for s in segs {
|
||||||
|
let _ = writeln!(
|
||||||
|
out,
|
||||||
|
"{},Blue,Song {},{},{},{},Duration: {:.0}s",
|
||||||
|
s.index,
|
||||||
|
s.index,
|
||||||
|
timecode(s.start, fps),
|
||||||
|
timecode(s.end, fps),
|
||||||
|
timecode(s.duration, fps),
|
||||||
|
s.duration
|
||||||
|
);
|
||||||
|
}
|
||||||
|
std::fs::write(path, out)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn write_csv(segs: &[Segment], path: &Path) -> std::io::Result<()> {
|
||||||
|
let mut out = String::from("index,start_sec,end_sec,duration_sec,start_timecode,end_timecode\n");
|
||||||
|
for s in segs {
|
||||||
|
let _ = writeln!(
|
||||||
|
out,
|
||||||
|
"{},{:.2},{:.2},{:.1},{},{}",
|
||||||
|
s.index, s.start, s.end, s.duration, hms(s.start), hms(s.end)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
std::fs::write(path, out)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn write_json(segs: &[Segment], path: &Path) -> std::io::Result<()> {
|
||||||
|
let body = serde_json::to_string_pretty(segs).unwrap_or_else(|_| "[]".into());
|
||||||
|
std::fs::write(path, body)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Write all four formats next to (or into `out_dir` for) the source video.
|
||||||
|
pub fn export_all(
|
||||||
|
segments: &[Segment],
|
||||||
|
source_video: &str,
|
||||||
|
out_dir: Option<String>,
|
||||||
|
fps: f64,
|
||||||
|
) -> Result<ExportResult, String> {
|
||||||
|
let src = Path::new(source_video);
|
||||||
|
let stem = src.file_stem().and_then(|s| s.to_str()).unwrap_or("output").to_string();
|
||||||
|
let source_name = src.file_name().and_then(|s| s.to_str()).unwrap_or(source_video).to_string();
|
||||||
|
let dir: PathBuf = match out_dir {
|
||||||
|
Some(d) if !d.trim().is_empty() => PathBuf::from(d),
|
||||||
|
_ => src.parent().map(|p| p.to_path_buf()).unwrap_or_else(|| PathBuf::from(".")),
|
||||||
|
};
|
||||||
|
std::fs::create_dir_all(&dir).map_err(|e| format!("Cannot create output dir: {e}"))?;
|
||||||
|
|
||||||
|
let edl = dir.join(format!("{stem}_singing.edl"));
|
||||||
|
let csv = dir.join(format!("{stem}_singing.csv"));
|
||||||
|
let markers = dir.join(format!("{stem}_markers.csv"));
|
||||||
|
let json = dir.join(format!("{stem}_singing.json"));
|
||||||
|
|
||||||
|
write_edl(segments, &edl, fps, &format!("{stem} - Singing"), &source_name)
|
||||||
|
.map_err(|e| format!("EDL write failed: {e}"))?;
|
||||||
|
write_markers_csv(segments, &markers, fps).map_err(|e| format!("Markers write failed: {e}"))?;
|
||||||
|
write_csv(segments, &csv).map_err(|e| format!("CSV write failed: {e}"))?;
|
||||||
|
write_json(segments, &json).map_err(|e| format!("JSON write failed: {e}"))?;
|
||||||
|
|
||||||
|
Ok(ExportResult {
|
||||||
|
files: [edl, markers, csv, json]
|
||||||
|
.iter()
|
||||||
|
.map(|p| p.to_string_lossy().into_owned())
|
||||||
|
.collect(),
|
||||||
|
})
|
||||||
|
}
|
||||||
142
src-tauri/src/lib.rs
Normal file
@@ -0,0 +1,142 @@
|
|||||||
|
//! Sing Detect — Tauri backend.
|
||||||
|
//! Detects singing segments in a recording via a Python ML sidecar
|
||||||
|
//! (inaSpeechSegmenter + TensorFlow), with ffmpeg for audio extraction, and
|
||||||
|
//! exports editor-friendly markers (EDL / CSV / JSON).
|
||||||
|
|
||||||
|
mod detect;
|
||||||
|
mod export;
|
||||||
|
mod tools;
|
||||||
|
|
||||||
|
use std::process::Child;
|
||||||
|
use std::sync::atomic::AtomicBool;
|
||||||
|
use std::sync::Mutex;
|
||||||
|
|
||||||
|
use serde::Serialize;
|
||||||
|
use tauri::{AppHandle, Manager, State};
|
||||||
|
|
||||||
|
use detect::{DetectResult, Segment};
|
||||||
|
use export::ExportResult;
|
||||||
|
|
||||||
|
pub struct AppState {
|
||||||
|
pub proc: Mutex<Option<Child>>,
|
||||||
|
pub cancel: AtomicBool,
|
||||||
|
pub ffmpeg: Option<String>,
|
||||||
|
pub ffprobe: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl AppState {
|
||||||
|
fn new() -> Self {
|
||||||
|
Self {
|
||||||
|
proc: Mutex::new(None),
|
||||||
|
cancel: AtomicBool::new(false),
|
||||||
|
ffmpeg: tools::find_tool("ffmpeg"),
|
||||||
|
ffprobe: tools::find_tool("ffprobe"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Serialize)]
|
||||||
|
struct ToolsInfo {
|
||||||
|
ffmpeg: Option<String>,
|
||||||
|
ffprobe: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Commands ─────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
#[tauri::command]
|
||||||
|
fn get_tools(state: State<AppState>) -> ToolsInfo {
|
||||||
|
ToolsInfo {
|
||||||
|
ffmpeg: state.ffmpeg.clone(),
|
||||||
|
ffprobe: state.ffprobe.clone(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Probe whether the detection environment (Python 3.10 + ML packages) is
|
||||||
|
/// ready. Read-only — installs nothing.
|
||||||
|
#[tauri::command]
|
||||||
|
async fn check_detection() -> detect::DetectionStatus {
|
||||||
|
tauri::async_runtime::spawn_blocking(detect::check_env)
|
||||||
|
.await
|
||||||
|
.unwrap_or_else(|e| detect::DetectionStatus {
|
||||||
|
python: false,
|
||||||
|
packages: false,
|
||||||
|
model: false,
|
||||||
|
ready: false,
|
||||||
|
detail: format!("Environment check failed: {e}"),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Install the detection ML stack and pre-download the model weights,
|
||||||
|
/// streaming progress via the `detect-setup-progress` event.
|
||||||
|
#[tauri::command]
|
||||||
|
async fn setup_detection(app: AppHandle) -> Result<(), String> {
|
||||||
|
tauri::async_runtime::spawn_blocking(move || detect::setup_env(&app))
|
||||||
|
.await
|
||||||
|
.map_err(|e| format!("setup task failed: {e}"))?
|
||||||
|
}
|
||||||
|
|
||||||
|
// Detection runs the ML sidecar for a long time, so it MUST run off the main
|
||||||
|
// thread or the window goes "Not responding". `spawn_blocking` keeps the UI
|
||||||
|
// responsive while still resolving with the final result.
|
||||||
|
#[tauri::command]
|
||||||
|
async fn detect_songs(
|
||||||
|
app: AppHandle,
|
||||||
|
video: String,
|
||||||
|
gap: f64,
|
||||||
|
min_duration: f64,
|
||||||
|
padding: f64,
|
||||||
|
) -> Result<DetectResult, String> {
|
||||||
|
tauri::async_runtime::spawn_blocking(move || {
|
||||||
|
let state = app.state::<AppState>();
|
||||||
|
let ffmpeg = state.ffmpeg.clone().ok_or("ffmpeg not found.")?;
|
||||||
|
detect::run(&app, &state, &ffmpeg, video, gap, min_duration, padding)
|
||||||
|
})
|
||||||
|
.await
|
||||||
|
.map_err(|e| format!("detect task failed: {e}"))?
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tauri::command]
|
||||||
|
async fn export_segments(
|
||||||
|
segments: Vec<Segment>,
|
||||||
|
source_video: String,
|
||||||
|
out_dir: Option<String>,
|
||||||
|
fps: f64,
|
||||||
|
) -> Result<ExportResult, String> {
|
||||||
|
tauri::async_runtime::spawn_blocking(move || {
|
||||||
|
export::export_all(&segments, &source_video, out_dir, fps)
|
||||||
|
})
|
||||||
|
.await
|
||||||
|
.map_err(|e| format!("export task failed: {e}"))?
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tauri::command]
|
||||||
|
fn cancel(state: State<AppState>) {
|
||||||
|
state.cancel.store(true, std::sync::atomic::Ordering::SeqCst);
|
||||||
|
if let Some(child) = state.proc.lock().unwrap().as_mut() {
|
||||||
|
// Kill the whole tree: detection runs `py -3.10` → `python.exe`, and
|
||||||
|
// killing only `py` would leave the Python/TensorFlow process alive.
|
||||||
|
tools::kill_process_tree(child.id());
|
||||||
|
let _ = child.kill();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg_attr(mobile, tauri::mobile_entry_point)]
|
||||||
|
pub fn run() {
|
||||||
|
tauri::Builder::default()
|
||||||
|
.plugin(tauri_plugin_opener::init())
|
||||||
|
.plugin(tauri_plugin_dialog::init())
|
||||||
|
.setup(|app| {
|
||||||
|
app.manage(AppState::new());
|
||||||
|
Ok(())
|
||||||
|
})
|
||||||
|
.invoke_handler(tauri::generate_handler![
|
||||||
|
get_tools,
|
||||||
|
check_detection,
|
||||||
|
setup_detection,
|
||||||
|
detect_songs,
|
||||||
|
export_segments,
|
||||||
|
cancel,
|
||||||
|
])
|
||||||
|
.run(tauri::generate_context!())
|
||||||
|
.expect("error while running tauri application");
|
||||||
|
}
|
||||||
6
src-tauri/src/main.rs
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
// Prevents additional console window on Windows in release, DO NOT REMOVE!!
|
||||||
|
#![cfg_attr(not(debug_assertions), windows_subsystem = "windows")]
|
||||||
|
|
||||||
|
fn main() {
|
||||||
|
sing_detect_lib::run()
|
||||||
|
}
|
||||||
81
src-tauri/src/tools.rs
Normal file
@@ -0,0 +1,81 @@
|
|||||||
|
//! ffmpeg / ffprobe discovery, probing, and small shared helpers.
|
||||||
|
|
||||||
|
use std::path::PathBuf;
|
||||||
|
use std::process::Command;
|
||||||
|
|
||||||
|
/// On Windows, prevent child processes from flashing a console window.
|
||||||
|
#[cfg(windows)]
|
||||||
|
pub const CREATE_NO_WINDOW: u32 = 0x0800_0000;
|
||||||
|
|
||||||
|
/// Apply the no-window flag to a Command (no-op on non-Windows).
|
||||||
|
pub fn hide_window(cmd: &mut Command) {
|
||||||
|
#[cfg(windows)]
|
||||||
|
{
|
||||||
|
use std::os::windows::process::CommandExt;
|
||||||
|
cmd.creation_flags(CREATE_NO_WINDOW);
|
||||||
|
}
|
||||||
|
#[cfg(not(windows))]
|
||||||
|
{
|
||||||
|
let _ = cmd;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Forcibly kill a process *and all of its descendants*.
|
||||||
|
///
|
||||||
|
/// `Child::kill()` only signals the direct child. That is fine for ffmpeg
|
||||||
|
/// (a single process), but detection runs through the `py` launcher, which
|
||||||
|
/// spawns the real `python.exe` as a child — killing `py` would orphan the
|
||||||
|
/// TensorFlow process and the job would keep running. `taskkill /T` walks the
|
||||||
|
/// whole tree.
|
||||||
|
pub fn kill_process_tree(pid: u32) {
|
||||||
|
#[cfg(windows)]
|
||||||
|
{
|
||||||
|
let mut cmd = Command::new("taskkill");
|
||||||
|
cmd.args(["/F", "/T", "/PID", &pid.to_string()]);
|
||||||
|
hide_window(&mut cmd);
|
||||||
|
let _ = cmd.output();
|
||||||
|
}
|
||||||
|
#[cfg(not(windows))]
|
||||||
|
{
|
||||||
|
// Best-effort on Unix: kill the process group.
|
||||||
|
let _ = Command::new("pkill").args(["-TERM", "-P", &pid.to_string()]).output();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Directory the executable lives in (resources dir when bundled).
|
||||||
|
fn app_dir() -> Option<PathBuf> {
|
||||||
|
std::env::current_exe().ok()?.parent().map(|p| p.to_path_buf())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Locate ffmpeg / ffprobe: next to the exe, then on PATH.
|
||||||
|
pub fn find_tool(name: &str) -> Option<String> {
|
||||||
|
let exe = format!("{name}.exe");
|
||||||
|
if let Some(base) = app_dir() {
|
||||||
|
for sub in ["", "bin", "resources", "_internal", ".."] {
|
||||||
|
let cand = base.join(sub).join(&exe);
|
||||||
|
if cand.exists() {
|
||||||
|
return Some(cand.to_string_lossy().into_owned());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Fall back to PATH discovery via `where` semantics.
|
||||||
|
if which(&exe).is_some() {
|
||||||
|
return Some(exe);
|
||||||
|
}
|
||||||
|
if which(name).is_some() {
|
||||||
|
return Some(name.to_string());
|
||||||
|
}
|
||||||
|
None
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Minimal PATH lookup so we don't pull in an extra crate.
|
||||||
|
fn which(name: &str) -> Option<PathBuf> {
|
||||||
|
let path = std::env::var_os("PATH")?;
|
||||||
|
for dir in std::env::split_paths(&path) {
|
||||||
|
let full = dir.join(name);
|
||||||
|
if full.is_file() {
|
||||||
|
return Some(full);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
None
|
||||||
|
}
|
||||||
42
src-tauri/tauri.conf.json
Normal file
@@ -0,0 +1,42 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://schema.tauri.app/config/2",
|
||||||
|
"productName": "Sing Detect",
|
||||||
|
"version": "2.0.0",
|
||||||
|
"identifier": "com.youfu.sing-detect",
|
||||||
|
"build": {
|
||||||
|
"beforeDevCommand": "npm run dev",
|
||||||
|
"devUrl": "http://localhost:1420",
|
||||||
|
"beforeBuildCommand": "npm run build",
|
||||||
|
"frontendDist": "../dist"
|
||||||
|
},
|
||||||
|
"app": {
|
||||||
|
"withGlobalTauri": true,
|
||||||
|
"windows": [
|
||||||
|
{
|
||||||
|
"title": "Sing Detect",
|
||||||
|
"width": 960,
|
||||||
|
"height": 820,
|
||||||
|
"minWidth": 720,
|
||||||
|
"minHeight": 620,
|
||||||
|
"dragDropEnabled": true
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"security": {
|
||||||
|
"csp": null
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"bundle": {
|
||||||
|
"active": true,
|
||||||
|
"targets": ["msi"],
|
||||||
|
"resources": {
|
||||||
|
"../sidecar": "sidecar"
|
||||||
|
},
|
||||||
|
"icon": [
|
||||||
|
"icons/32x32.png",
|
||||||
|
"icons/128x128.png",
|
||||||
|
"icons/128x128@2x.png",
|
||||||
|
"icons/icon.icns",
|
||||||
|
"icons/icon.ico"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
43
src/api.ts
Normal file
@@ -0,0 +1,43 @@
|
|||||||
|
// Typed bridge to the Rust backend (detection-only build).
|
||||||
|
|
||||||
|
import { invoke } from "@tauri-apps/api/core";
|
||||||
|
import { listen, type UnlistenFn } from "@tauri-apps/api/event";
|
||||||
|
import { getCurrentWebview } from "@tauri-apps/api/webview";
|
||||||
|
import { open } from "@tauri-apps/plugin-dialog";
|
||||||
|
import { revealItemInDir } from "@tauri-apps/plugin-opener";
|
||||||
|
|
||||||
|
export interface Tools { ffmpeg: string | null; ffprobe: string | null; }
|
||||||
|
export interface Segment { index: number; start: number; end: number; duration: number; }
|
||||||
|
export interface DetectResult { segments: Segment[]; stats: Record<string, number>; }
|
||||||
|
export interface ExportResult { files: string[]; }
|
||||||
|
export interface DetectionStatus { python: boolean; packages: boolean; model: boolean; ready: boolean; detail: string; }
|
||||||
|
export interface SetupProgress { message: string; pct: number; }
|
||||||
|
|
||||||
|
export const getTools = () => invoke<Tools>("get_tools");
|
||||||
|
export const checkDetection = () => invoke<DetectionStatus>("check_detection");
|
||||||
|
export const setupDetection = () => invoke<void>("setup_detection");
|
||||||
|
export const detectSongs = (video: string, gap: number, minDuration: number, padding: number) =>
|
||||||
|
invoke<DetectResult>("detect_songs", { video, gap, minDuration, padding });
|
||||||
|
export const exportSegments = (segments: Segment[], sourceVideo: string, outDir: string | null, fps: number) =>
|
||||||
|
invoke<ExportResult>("export_segments", { segments, sourceVideo, outDir, fps });
|
||||||
|
export const cancel = () => invoke<void>("cancel");
|
||||||
|
|
||||||
|
export const onDetectProgress = (cb: (msg: string) => void): Promise<UnlistenFn> =>
|
||||||
|
listen<{ message: string }>("detect-progress", (e) => cb(e.payload.message));
|
||||||
|
export const onDetectSetupProgress = (cb: (p: SetupProgress) => void): Promise<UnlistenFn> =>
|
||||||
|
listen<SetupProgress>("detect-setup-progress", (e) => cb(e.payload));
|
||||||
|
|
||||||
|
export const onFilesDropped = (cb: (paths: string[]) => void): Promise<UnlistenFn> =>
|
||||||
|
getCurrentWebview().onDragDropEvent((e) => {
|
||||||
|
if (e.payload.type === "drop") cb(e.payload.paths);
|
||||||
|
});
|
||||||
|
|
||||||
|
export async function pickVideo(): Promise<string | null> {
|
||||||
|
const sel = await open({
|
||||||
|
multiple: false,
|
||||||
|
filters: [{ name: "Video", extensions: ["mp4", "mov", "mkv", "avi", "flv", "ts", "wmv", "m4v", "webm", "mts", "m2ts"] }],
|
||||||
|
});
|
||||||
|
return typeof sel === "string" ? sel : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const reveal = (path: string) => revealItemInDir(path);
|
||||||
6
src/assets/tauri.svg
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
<svg width="206" height="231" viewBox="0 0 206 231" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||||
|
<path d="M143.143 84C143.143 96.1503 133.293 106 121.143 106C108.992 106 99.1426 96.1503 99.1426 84C99.1426 71.8497 108.992 62 121.143 62C133.293 62 143.143 71.8497 143.143 84Z" fill="#FFC131"/>
|
||||||
|
<ellipse cx="84.1426" cy="147" rx="22" ry="22" transform="rotate(180 84.1426 147)" fill="#24C8DB"/>
|
||||||
|
<path fill-rule="evenodd" clip-rule="evenodd" d="M166.738 154.548C157.86 160.286 148.023 164.269 137.757 166.341C139.858 160.282 141 153.774 141 147C141 144.543 140.85 142.121 140.558 139.743C144.975 138.204 149.215 136.139 153.183 133.575C162.73 127.404 170.292 118.608 174.961 108.244C179.63 97.8797 181.207 86.3876 179.502 75.1487C177.798 63.9098 172.884 53.4021 165.352 44.8883C157.82 36.3744 147.99 30.2165 137.042 27.1546C126.095 24.0926 114.496 24.2568 103.64 27.6274C92.7839 30.998 83.1319 37.4317 75.8437 46.1553C74.9102 47.2727 74.0206 48.4216 73.176 49.5993C61.9292 50.8488 51.0363 54.0318 40.9629 58.9556C44.2417 48.4586 49.5653 38.6591 56.679 30.1442C67.0505 17.7298 80.7861 8.57426 96.2354 3.77762C111.685 -1.01901 128.19 -1.25267 143.769 3.10474C159.348 7.46215 173.337 16.2252 184.056 28.3411C194.775 40.457 201.767 55.4101 204.193 71.404C206.619 87.3978 204.374 103.752 197.73 118.501C191.086 133.25 180.324 145.767 166.738 154.548ZM41.9631 74.275L62.5557 76.8042C63.0459 72.813 63.9401 68.9018 65.2138 65.1274C57.0465 67.0016 49.2088 70.087 41.9631 74.275Z" fill="#FFC131"/>
|
||||||
|
<path fill-rule="evenodd" clip-rule="evenodd" d="M38.4045 76.4519C47.3493 70.6709 57.2677 66.6712 67.6171 64.6132C65.2774 70.9669 64 77.8343 64 85.0001C64 87.1434 64.1143 89.26 64.3371 91.3442C60.0093 92.8732 55.8533 94.9092 51.9599 97.4256C42.4128 103.596 34.8505 112.392 30.1816 122.756C25.5126 133.12 23.9357 144.612 25.6403 155.851C27.3449 167.09 32.2584 177.598 39.7906 186.112C47.3227 194.626 57.153 200.784 68.1003 203.846C79.0476 206.907 90.6462 206.743 101.502 203.373C112.359 200.002 122.011 193.568 129.299 184.845C130.237 183.722 131.131 182.567 131.979 181.383C143.235 180.114 154.132 176.91 164.205 171.962C160.929 182.49 155.596 192.319 148.464 200.856C138.092 213.27 124.357 222.426 108.907 227.222C93.458 232.019 76.9524 232.253 61.3736 227.895C45.7948 223.538 31.8055 214.775 21.0867 202.659C10.3679 190.543 3.37557 175.59 0.949823 159.596C-1.47592 143.602 0.768139 127.248 7.41237 112.499C14.0566 97.7497 24.8183 85.2327 38.4045 76.4519ZM163.062 156.711L163.062 156.711C162.954 156.773 162.846 156.835 162.738 156.897C162.846 156.835 162.954 156.773 163.062 156.711Z" fill="#24C8DB"/>
|
||||||
|
</svg>
|
||||||
|
After Width: | Height: | Size: 2.5 KiB |
25
src/assets/typescript.svg
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
<?xml version="1.0" standalone="no"?>
|
||||||
|
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 20010904//EN"
|
||||||
|
"http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
|
||||||
|
<svg version="1.0" xmlns="http://www.w3.org/2000/svg"
|
||||||
|
width="512.000000pt" height="512.000000pt" viewBox="0 0 512.000000 512.000000"
|
||||||
|
preserveAspectRatio="xMidYMid meet">
|
||||||
|
|
||||||
|
<g transform="translate(0.000000,512.000000) scale(0.100000,-0.100000)"
|
||||||
|
fill="#2D79C7" stroke="none">
|
||||||
|
<path d="M430 5109 c-130 -19 -248 -88 -325 -191 -53 -71 -83 -147 -96 -247
|
||||||
|
-6 -49 -9 -813 -7 -2166 l3 -2090 22 -65 c54 -159 170 -273 328 -323 l70 -22
|
||||||
|
2140 0 2140 0 66 23 c160 55 272 169 322 327 l22 70 0 2135 0 2135 -22 70
|
||||||
|
c-49 157 -155 265 -319 327 l-59 23 -2115 1 c-1163 1 -2140 -2 -2170 -7z
|
||||||
|
m3931 -2383 c48 -9 120 -26 160 -39 l74 -23 3 -237 c1 -130 0 -237 -2 -237 -3
|
||||||
|
0 -26 14 -53 30 -61 38 -197 84 -310 106 -110 20 -293 15 -368 -12 -111 -39
|
||||||
|
-175 -110 -175 -193 0 -110 97 -197 335 -300 140 -61 309 -146 375 -189 30
|
||||||
|
-20 87 -68 126 -107 119 -117 164 -234 164 -426 0 -310 -145 -518 -430 -613
|
||||||
|
-131 -43 -248 -59 -445 -60 -243 -1 -405 24 -577 90 l-68 26 0 242 c0 175 -3
|
||||||
|
245 -12 254 -9 9 -9 12 0 12 7 0 12 -4 12 -9 0 -17 139 -102 223 -138 136 -57
|
||||||
|
233 -77 382 -76 145 0 224 19 295 68 75 52 100 156 59 242 -41 84 -135 148
|
||||||
|
-374 253 -367 161 -522 300 -581 520 -23 86 -23 253 -1 337 73 275 312 448
|
||||||
|
682 492 109 13 401 6 506 -13z m-1391 -241 l0 -205 -320 0 -320 0 0 -915 0
|
||||||
|
-915 -255 0 -255 0 0 915 0 915 -320 0 -320 0 0 205 0 205 895 0 895 0 0 -205z"/>
|
||||||
|
</g>
|
||||||
|
</svg>
|
||||||
|
After Width: | Height: | Size: 1.4 KiB |
1
src/assets/vite.svg
Normal file
@@ -0,0 +1 @@
|
|||||||
|
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--logos" width="31.88" height="32" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 257"><defs><linearGradient id="IconifyId1813088fe1fbc01fb466" x1="-.828%" x2="57.636%" y1="7.652%" y2="78.411%"><stop offset="0%" stop-color="#41D1FF"></stop><stop offset="100%" stop-color="#BD34FE"></stop></linearGradient><linearGradient id="IconifyId1813088fe1fbc01fb467" x1="43.376%" x2="50.316%" y1="2.242%" y2="89.03%"><stop offset="0%" stop-color="#FFEA83"></stop><stop offset="8.333%" stop-color="#FFDD35"></stop><stop offset="100%" stop-color="#FFA800"></stop></linearGradient></defs><path fill="url(#IconifyId1813088fe1fbc01fb466)" d="M255.153 37.938L134.897 252.976c-2.483 4.44-8.862 4.466-11.382.048L.875 37.958c-2.746-4.814 1.371-10.646 6.827-9.67l120.385 21.517a6.537 6.537 0 0 0 2.322-.004l117.867-21.483c5.438-.991 9.574 4.796 6.877 9.62Z"></path><path fill="url(#IconifyId1813088fe1fbc01fb467)" d="M185.432.063L96.44 17.501a3.268 3.268 0 0 0-2.634 3.014l-5.474 92.456a3.268 3.268 0 0 0 3.997 3.378l24.777-5.718c2.318-.535 4.413 1.507 3.936 3.838l-7.361 36.047c-.495 2.426 1.782 4.5 4.151 3.78l15.304-4.649c2.372-.72 4.652 1.36 4.15 3.788l-11.698 56.621c-.732 3.542 3.979 5.473 5.943 2.437l1.313-2.028l72.516-144.72c1.215-2.423-.88-5.186-3.54-4.672l-25.505 4.922c-2.396.462-4.435-1.77-3.759-4.114l16.646-57.705c.677-2.35-1.37-4.583-3.769-4.113Z"></path></svg>
|
||||||
|
After Width: | Height: | Size: 1.5 KiB |
78
src/dom.ts
Normal file
@@ -0,0 +1,78 @@
|
|||||||
|
// Minimal DOM helpers — keeps the views declarative without a framework.
|
||||||
|
|
||||||
|
type Props = Record<string, unknown>;
|
||||||
|
type Child = Node | string | null | undefined | false;
|
||||||
|
|
||||||
|
export function h<K extends keyof HTMLElementTagNameMap>(
|
||||||
|
tag: K,
|
||||||
|
props: Props = {},
|
||||||
|
...children: Child[]
|
||||||
|
): HTMLElementTagNameMap[K] {
|
||||||
|
const el = document.createElement(tag);
|
||||||
|
for (const [k, v] of Object.entries(props)) {
|
||||||
|
if (v == null || v === false) continue;
|
||||||
|
if (k === "class") el.className = String(v);
|
||||||
|
else if (k === "html") el.innerHTML = String(v);
|
||||||
|
else if (k.startsWith("on") && typeof v === "function") {
|
||||||
|
el.addEventListener(k.slice(2).toLowerCase(), v as EventListener);
|
||||||
|
} else if (k === "dataset" && typeof v === "object") {
|
||||||
|
Object.assign(el.dataset, v);
|
||||||
|
} else if (k in el) {
|
||||||
|
// direct property (value, disabled, checked, textContent…)
|
||||||
|
(el as any)[k] = v;
|
||||||
|
} else {
|
||||||
|
el.setAttribute(k, String(v));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (const c of children.flat()) {
|
||||||
|
if (c == null || c === false) continue;
|
||||||
|
el.append(c instanceof Node ? c : document.createTextNode(String(c)));
|
||||||
|
}
|
||||||
|
return el;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const $ = <T extends Element = HTMLElement>(sel: string, root: ParentNode = document) =>
|
||||||
|
root.querySelector<T>(sel);
|
||||||
|
|
||||||
|
export function clear(el: Element) {
|
||||||
|
while (el.firstChild) el.removeChild(el.firstChild);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function basename(p: string): string {
|
||||||
|
return p.replace(/[/\\]+$/, "").split(/[/\\]/).pop() ?? p;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function stem(p: string): string {
|
||||||
|
const b = basename(p);
|
||||||
|
const i = b.lastIndexOf(".");
|
||||||
|
return i > 0 ? b.slice(0, i) : b;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function dirname(p: string): string {
|
||||||
|
const norm = p.replace(/[/\\]+$/, "");
|
||||||
|
const i = Math.max(norm.lastIndexOf("/"), norm.lastIndexOf("\\"));
|
||||||
|
return i >= 0 ? norm.slice(0, i) : "";
|
||||||
|
}
|
||||||
|
|
||||||
|
export function joinPath(dir: string, name: string): string {
|
||||||
|
if (!dir) return name;
|
||||||
|
const sep = dir.includes("\\") ? "\\" : "/";
|
||||||
|
return dir.replace(/[/\\]+$/, "") + sep + name;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function fmtDur(seconds: number): string {
|
||||||
|
if (seconds < 0) return "…";
|
||||||
|
if (!seconds) return "—";
|
||||||
|
const s = Math.round(seconds);
|
||||||
|
const h = Math.floor(s / 3600);
|
||||||
|
const m = Math.floor((s % 3600) / 60);
|
||||||
|
const sec = s % 60;
|
||||||
|
return h ? `${h}:${String(m).padStart(2, "0")}:${String(sec).padStart(2, "0")}`
|
||||||
|
: `${m}:${String(sec).padStart(2, "0")}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function fmtElapsed(s: number): string {
|
||||||
|
if (s < 60) return `${Math.round(s)}s`;
|
||||||
|
const m = Math.floor(s / 60);
|
||||||
|
return `${m}m ${String(Math.round(s % 60)).padStart(2, "0")}s`;
|
||||||
|
}
|
||||||
60
src/main.ts
Normal file
@@ -0,0 +1,60 @@
|
|||||||
|
// Sing Detect — app shell: title bar, theme, drag-drop.
|
||||||
|
|
||||||
|
import "./styles.css";
|
||||||
|
import { h } from "./dom";
|
||||||
|
import { toast } from "./ui";
|
||||||
|
import { getTools, onFilesDropped } from "./api";
|
||||||
|
import { buildDetect } from "./views/detect";
|
||||||
|
|
||||||
|
async function boot() {
|
||||||
|
// ── Theme (system default, persisted toggle) ────────────────────────────
|
||||||
|
const saved = localStorage.getItem("theme");
|
||||||
|
const prefersDark = matchMedia("(prefers-color-scheme: dark)").matches;
|
||||||
|
let theme = saved ?? (prefersDark ? "dark" : "light");
|
||||||
|
const applyTheme = () => document.documentElement.setAttribute("data-theme", theme);
|
||||||
|
applyTheme();
|
||||||
|
|
||||||
|
// ── Single view: detection ──────────────────────────────────────────────
|
||||||
|
const detect = buildDetect();
|
||||||
|
|
||||||
|
const themeBtn = h("button", { class: "icon-btn", title: "Toggle theme" },
|
||||||
|
theme === "dark" ? "☀" : "☾") as HTMLButtonElement;
|
||||||
|
themeBtn.addEventListener("click", () => {
|
||||||
|
theme = theme === "dark" ? "light" : "dark";
|
||||||
|
localStorage.setItem("theme", theme);
|
||||||
|
applyTheme();
|
||||||
|
themeBtn.textContent = theme === "dark" ? "☀" : "☾";
|
||||||
|
});
|
||||||
|
|
||||||
|
const app = document.getElementById("app")!;
|
||||||
|
app.append(
|
||||||
|
h("div", { class: "titlebar" },
|
||||||
|
h("div", { class: "brand" },
|
||||||
|
h("div", { class: "logo" }, "🎤"),
|
||||||
|
h("span", {}, "Sing Detect")),
|
||||||
|
h("div", { class: "spacer" }),
|
||||||
|
themeBtn,
|
||||||
|
),
|
||||||
|
h("div", { class: "content" }, detect.el),
|
||||||
|
);
|
||||||
|
|
||||||
|
// First-show hook: highlight the engine test + silent env probe.
|
||||||
|
detect.onShow?.();
|
||||||
|
|
||||||
|
// ── Drag-and-drop routes to the detection view ──────────────────────────
|
||||||
|
await onFilesDropped((paths) => detect.onDrop(paths));
|
||||||
|
|
||||||
|
// ── ffmpeg health check ─────────────────────────────────────────────────
|
||||||
|
try {
|
||||||
|
const tools = await getTools();
|
||||||
|
if (!tools.ffmpeg) {
|
||||||
|
const banner = h("div", { class: "banner", style: "margin:0 22px 12px" },
|
||||||
|
"⚠ ffmpeg was not found. Put ffmpeg.exe & ffprobe.exe next to the app or on your PATH — detection needs it to extract audio.");
|
||||||
|
app.querySelector(".content")!.before(banner);
|
||||||
|
}
|
||||||
|
} catch (e) {
|
||||||
|
toast(`Backend not ready: ${e}`, "err");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
boot();
|
||||||
349
src/styles.css
Normal file
@@ -0,0 +1,349 @@
|
|||||||
|
/* ============================================================================
|
||||||
|
Stream Studio — design system
|
||||||
|
Native-feeling on Windows 11: Segoe UI Variable, Fluent-ish surfaces,
|
||||||
|
GPU-friendly transitions, custom scrollbars. Light/dark via [data-theme].
|
||||||
|
========================================================================= */
|
||||||
|
|
||||||
|
:root {
|
||||||
|
--accent: #6d5efc;
|
||||||
|
--accent-2: #8b7bff;
|
||||||
|
--accent-ink: #ffffff;
|
||||||
|
|
||||||
|
--bg: #f4f5fb;
|
||||||
|
--bg-grad-1: #eef0fb;
|
||||||
|
--bg-grad-2: #f7f5ff;
|
||||||
|
--surface: #ffffff;
|
||||||
|
--surface-2: #f6f7fc;
|
||||||
|
--surface-hover: #eef0fa;
|
||||||
|
--border: #e4e6f0;
|
||||||
|
--border-strong: #d3d6e6;
|
||||||
|
|
||||||
|
--text: #1b1c28;
|
||||||
|
--text-dim: #5a5d70;
|
||||||
|
--text-faint: #8b8fa6;
|
||||||
|
|
||||||
|
--ok: #16a34a;
|
||||||
|
--warn: #f59e0b;
|
||||||
|
--danger: #e11d48;
|
||||||
|
|
||||||
|
--shadow-sm: 0 1px 2px rgba(20, 22, 50, 0.06), 0 1px 3px rgba(20, 22, 50, 0.05);
|
||||||
|
--shadow-md: 0 4px 16px rgba(20, 22, 50, 0.08), 0 2px 6px rgba(20, 22, 50, 0.05);
|
||||||
|
--shadow-lg: 0 18px 50px rgba(20, 22, 50, 0.16);
|
||||||
|
|
||||||
|
--radius: 12px;
|
||||||
|
--radius-lg: 16px;
|
||||||
|
--radius-sm: 8px;
|
||||||
|
|
||||||
|
--ease: cubic-bezier(0.22, 1, 0.36, 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
[data-theme="dark"] {
|
||||||
|
--accent: #8b7bff;
|
||||||
|
--accent-2: #a594ff;
|
||||||
|
--accent-ink: #14121f;
|
||||||
|
|
||||||
|
--bg: #0e0f17;
|
||||||
|
--bg-grad-1: #11111d;
|
||||||
|
--bg-grad-2: #161427;
|
||||||
|
--surface: #1a1b27;
|
||||||
|
--surface-2: #20212f;
|
||||||
|
--surface-hover: #262838;
|
||||||
|
--border: #2a2c3c;
|
||||||
|
--border-strong: #383b50;
|
||||||
|
|
||||||
|
--text: #eceefb;
|
||||||
|
--text-dim: #a6a9bf;
|
||||||
|
--text-faint: #71748c;
|
||||||
|
|
||||||
|
--shadow-sm: 0 1px 2px rgba(0, 0, 0, 0.3);
|
||||||
|
--shadow-md: 0 6px 22px rgba(0, 0, 0, 0.4);
|
||||||
|
--shadow-lg: 0 22px 60px rgba(0, 0, 0, 0.55);
|
||||||
|
}
|
||||||
|
|
||||||
|
* { box-sizing: border-box; margin: 0; padding: 0; }
|
||||||
|
|
||||||
|
html, body {
|
||||||
|
height: 100%;
|
||||||
|
overflow: hidden;
|
||||||
|
}
|
||||||
|
|
||||||
|
body {
|
||||||
|
font-family: "Segoe UI Variable Text", "Segoe UI", system-ui, -apple-system, sans-serif;
|
||||||
|
font-size: 14px;
|
||||||
|
color: var(--text);
|
||||||
|
background: radial-gradient(1200px 600px at 80% -10%, var(--bg-grad-2), transparent 60%),
|
||||||
|
radial-gradient(900px 500px at -10% 110%, var(--bg-grad-1), transparent 55%),
|
||||||
|
var(--bg);
|
||||||
|
-webkit-font-smoothing: antialiased;
|
||||||
|
text-rendering: optimizeLegibility;
|
||||||
|
user-select: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
button, input, select { font-family: inherit; font-size: inherit; color: inherit; }
|
||||||
|
:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; }
|
||||||
|
|
||||||
|
/* ── Scrollbars ───────────────────────────────────────────────────────── */
|
||||||
|
::-webkit-scrollbar { width: 10px; height: 10px; }
|
||||||
|
::-webkit-scrollbar-thumb {
|
||||||
|
background: var(--border-strong);
|
||||||
|
border-radius: 99px;
|
||||||
|
border: 3px solid transparent;
|
||||||
|
background-clip: content-box;
|
||||||
|
}
|
||||||
|
::-webkit-scrollbar-thumb:hover { background: var(--text-faint); background-clip: content-box; }
|
||||||
|
|
||||||
|
/* ── App shell ────────────────────────────────────────────────────────── */
|
||||||
|
#app { display: flex; flex-direction: column; height: 100vh; }
|
||||||
|
|
||||||
|
.titlebar {
|
||||||
|
display: flex; align-items: center; gap: 12px;
|
||||||
|
padding: 14px 22px;
|
||||||
|
-webkit-app-region: drag;
|
||||||
|
}
|
||||||
|
.titlebar .brand {
|
||||||
|
display: flex; align-items: center; gap: 10px;
|
||||||
|
font-weight: 650; font-size: 16px; letter-spacing: 0.2px;
|
||||||
|
}
|
||||||
|
.titlebar .logo {
|
||||||
|
width: 26px; height: 26px; border-radius: 8px;
|
||||||
|
background: linear-gradient(135deg, var(--accent), var(--accent-2));
|
||||||
|
display: grid; place-items: center; color: var(--accent-ink);
|
||||||
|
box-shadow: var(--shadow-sm);
|
||||||
|
font-size: 15px;
|
||||||
|
}
|
||||||
|
.titlebar .spacer { flex: 1; }
|
||||||
|
.icon-btn {
|
||||||
|
-webkit-app-region: no-drag;
|
||||||
|
width: 34px; height: 34px; border-radius: 9px;
|
||||||
|
border: 1px solid transparent; background: transparent;
|
||||||
|
color: var(--text-dim); cursor: pointer;
|
||||||
|
display: grid; place-items: center; font-size: 16px;
|
||||||
|
transition: background .18s var(--ease), color .18s var(--ease);
|
||||||
|
}
|
||||||
|
.icon-btn:hover { background: var(--surface-hover); color: var(--text); }
|
||||||
|
|
||||||
|
/* ── Tabs ─────────────────────────────────────────────────────────────── */
|
||||||
|
.tabs {
|
||||||
|
-webkit-app-region: no-drag;
|
||||||
|
display: flex; gap: 4px;
|
||||||
|
margin: 0 22px;
|
||||||
|
padding: 4px;
|
||||||
|
background: var(--surface-2);
|
||||||
|
border: 1px solid var(--border);
|
||||||
|
border-radius: 12px;
|
||||||
|
width: fit-content;
|
||||||
|
}
|
||||||
|
.tab {
|
||||||
|
display: flex; align-items: center; gap: 7px;
|
||||||
|
padding: 8px 16px; border-radius: 9px;
|
||||||
|
border: none; background: transparent; cursor: pointer;
|
||||||
|
color: var(--text-dim); font-weight: 550;
|
||||||
|
transition: color .18s var(--ease), background .18s var(--ease);
|
||||||
|
}
|
||||||
|
.tab:hover { color: var(--text); }
|
||||||
|
.tab.active {
|
||||||
|
color: var(--text);
|
||||||
|
background: var(--surface);
|
||||||
|
box-shadow: var(--shadow-sm);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Content area ─────────────────────────────────────────────────────── */
|
||||||
|
.content { flex: 1; overflow-y: auto; padding: 18px 22px 28px; }
|
||||||
|
.view { display: flex; flex-direction: column; gap: 16px; max-width: 880px; margin: 0 auto; }
|
||||||
|
.view[hidden] { display: none; }
|
||||||
|
|
||||||
|
.card {
|
||||||
|
background: var(--surface);
|
||||||
|
border: 1px solid var(--border);
|
||||||
|
border-radius: var(--radius-lg);
|
||||||
|
box-shadow: var(--shadow-sm);
|
||||||
|
padding: 18px;
|
||||||
|
}
|
||||||
|
.card h2 {
|
||||||
|
font-size: 13px; font-weight: 650; letter-spacing: .2px;
|
||||||
|
color: var(--text); margin-bottom: 14px;
|
||||||
|
display: flex; align-items: center; gap: 8px;
|
||||||
|
}
|
||||||
|
.card h2 .step {
|
||||||
|
width: 20px; height: 20px; border-radius: 6px;
|
||||||
|
background: var(--surface-2); border: 1px solid var(--border);
|
||||||
|
display: grid; place-items: center; font-size: 11px; color: var(--text-dim);
|
||||||
|
}
|
||||||
|
.muted { color: var(--text-dim); }
|
||||||
|
.faint { color: var(--text-faint); font-size: 12px; }
|
||||||
|
|
||||||
|
/* ── Buttons ──────────────────────────────────────────────────────────── */
|
||||||
|
.btn {
|
||||||
|
-webkit-app-region: no-drag;
|
||||||
|
display: inline-flex; align-items: center; gap: 8px;
|
||||||
|
padding: 9px 16px; border-radius: 10px;
|
||||||
|
border: 1px solid var(--border-strong); background: var(--surface);
|
||||||
|
color: var(--text); font-weight: 550; cursor: pointer;
|
||||||
|
transition: transform .12s var(--ease), background .18s var(--ease),
|
||||||
|
border-color .18s var(--ease), box-shadow .18s var(--ease), opacity .18s;
|
||||||
|
}
|
||||||
|
.btn:hover { background: var(--surface-hover); }
|
||||||
|
.btn:active { transform: translateY(1px) scale(0.99); }
|
||||||
|
.btn:disabled { opacity: .45; cursor: not-allowed; }
|
||||||
|
.btn.primary {
|
||||||
|
background: linear-gradient(135deg, var(--accent), var(--accent-2));
|
||||||
|
border-color: transparent; color: var(--accent-ink);
|
||||||
|
box-shadow: 0 6px 18px color-mix(in srgb, var(--accent) 40%, transparent);
|
||||||
|
}
|
||||||
|
.btn.primary:hover { filter: brightness(1.05); background: linear-gradient(135deg, var(--accent), var(--accent-2)); }
|
||||||
|
.btn.ghost { background: transparent; border-color: transparent; color: var(--text-dim); }
|
||||||
|
.btn.ghost:hover { background: var(--surface-hover); color: var(--text); }
|
||||||
|
.btn.danger:hover { border-color: var(--danger); color: var(--danger); }
|
||||||
|
.btn.lg { padding: 12px 22px; font-size: 15px; }
|
||||||
|
.btn.accent-soft {
|
||||||
|
background: color-mix(in srgb, var(--accent) 14%, var(--surface));
|
||||||
|
border-color: color-mix(in srgb, var(--accent) 30%, transparent);
|
||||||
|
color: var(--accent);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Inputs ───────────────────────────────────────────────────────────── */
|
||||||
|
.field { display: flex; flex-direction: column; gap: 6px; }
|
||||||
|
.field label { font-size: 12px; color: var(--text-dim); font-weight: 550; }
|
||||||
|
.input, select.input {
|
||||||
|
-webkit-app-region: no-drag;
|
||||||
|
width: 100%; padding: 9px 12px; border-radius: 10px;
|
||||||
|
border: 1px solid var(--border-strong); background: var(--surface-2);
|
||||||
|
color: var(--text); transition: border-color .18s var(--ease), background .18s var(--ease);
|
||||||
|
user-select: text;
|
||||||
|
}
|
||||||
|
.input:focus { border-color: var(--accent); background: var(--surface); }
|
||||||
|
.row { display: flex; gap: 10px; align-items: center; }
|
||||||
|
.row.wrap { flex-wrap: wrap; }
|
||||||
|
.grow { flex: 1; min-width: 0; }
|
||||||
|
|
||||||
|
/* ── Dropzone ─────────────────────────────────────────────────────────── */
|
||||||
|
.dropzone {
|
||||||
|
border: 1.5px dashed var(--border-strong); border-radius: var(--radius);
|
||||||
|
padding: 22px; text-align: center; color: var(--text-dim);
|
||||||
|
background: var(--surface-2);
|
||||||
|
transition: border-color .2s var(--ease), background .2s var(--ease), color .2s var(--ease);
|
||||||
|
}
|
||||||
|
.dropzone.hot {
|
||||||
|
border-color: var(--accent); color: var(--accent);
|
||||||
|
background: color-mix(in srgb, var(--accent) 8%, var(--surface));
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── File list ────────────────────────────────────────────────────────── */
|
||||||
|
.list { display: flex; flex-direction: column; gap: 6px; max-height: 280px; overflow-y: auto; padding-right: 4px; }
|
||||||
|
.list-head, .list-row {
|
||||||
|
display: grid; grid-template-columns: 30px 26px 1fr 72px 32px;
|
||||||
|
align-items: center; gap: 6px;
|
||||||
|
}
|
||||||
|
.list-head { padding: 4px 10px; font-size: 11px; color: var(--text-faint); font-weight: 650; }
|
||||||
|
.list-row {
|
||||||
|
padding: 9px 10px; border-radius: 10px;
|
||||||
|
background: var(--surface-2); border: 1px solid transparent;
|
||||||
|
transition: background .15s var(--ease), border-color .15s var(--ease);
|
||||||
|
}
|
||||||
|
.list-row:hover { background: var(--surface-hover); }
|
||||||
|
.list-row.active { border-color: color-mix(in srgb, var(--accent) 50%, transparent);
|
||||||
|
background: color-mix(in srgb, var(--accent) 10%, var(--surface)); }
|
||||||
|
.list-row .idx { color: var(--text-faint); font-size: 12px; text-align: center; }
|
||||||
|
.list-row .name { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||||
|
.list-row .dur { text-align: right; color: var(--text-dim); font-variant-numeric: tabular-nums; font-size: 12px; }
|
||||||
|
.list-row .x {
|
||||||
|
width: 24px; height: 24px; border-radius: 7px; border: none; cursor: pointer;
|
||||||
|
background: transparent; color: var(--text-faint);
|
||||||
|
transition: background .15s, color .15s;
|
||||||
|
}
|
||||||
|
.list-row .x:hover { background: color-mix(in srgb, var(--danger) 14%, transparent); color: var(--danger); }
|
||||||
|
.checkbox { width: 17px; height: 17px; accent-color: var(--accent); cursor: pointer; }
|
||||||
|
|
||||||
|
.toolbar { display: flex; gap: 8px; flex-wrap: wrap; }
|
||||||
|
|
||||||
|
/* ── Progress ─────────────────────────────────────────────────────────── */
|
||||||
|
.progress-wrap { display: flex; flex-direction: column; gap: 8px; }
|
||||||
|
.bar { height: 10px; border-radius: 99px; background: var(--surface-2);
|
||||||
|
overflow: hidden; border: 1px solid var(--border); }
|
||||||
|
.bar > i {
|
||||||
|
display: block; height: 100%; width: 0%;
|
||||||
|
background: linear-gradient(90deg, var(--accent), var(--accent-2));
|
||||||
|
border-radius: 99px; transition: width .25s var(--ease);
|
||||||
|
}
|
||||||
|
.bar.indeterminate > i {
|
||||||
|
width: 35% !important; border-radius: 99px;
|
||||||
|
animation: slide 1.1s var(--ease) infinite;
|
||||||
|
}
|
||||||
|
@keyframes slide { 0% { margin-left: -35%; } 100% { margin-left: 100%; } }
|
||||||
|
.progress-meta { display: flex; justify-content: space-between; font-size: 12px; }
|
||||||
|
.progress-meta .status { color: var(--text); font-weight: 550; }
|
||||||
|
.progress-meta .timing { color: var(--text-faint); font-variant-numeric: tabular-nums; }
|
||||||
|
|
||||||
|
/* ── Log ──────────────────────────────────────────────────────────────── */
|
||||||
|
.log {
|
||||||
|
margin-top: 10px; max-height: 150px; overflow-y: auto;
|
||||||
|
font-family: "Cascadia Code", "Consolas", monospace; font-size: 12px;
|
||||||
|
background: var(--surface-2); border: 1px solid var(--border);
|
||||||
|
border-radius: 10px; padding: 10px; color: var(--text-dim);
|
||||||
|
white-space: pre-wrap; user-select: text;
|
||||||
|
}
|
||||||
|
.log:empty { display: none; }
|
||||||
|
|
||||||
|
/* ── Songs result table ───────────────────────────────────────────────── */
|
||||||
|
.songs { display: flex; flex-direction: column; gap: 4px; }
|
||||||
|
.song {
|
||||||
|
display: grid; grid-template-columns: 46px 1fr auto;
|
||||||
|
gap: 10px; align-items: center;
|
||||||
|
padding: 8px 12px; border-radius: 9px; background: var(--surface-2);
|
||||||
|
}
|
||||||
|
.song .badge {
|
||||||
|
background: color-mix(in srgb, var(--accent) 16%, var(--surface));
|
||||||
|
color: var(--accent); border-radius: 6px; padding: 2px 0; text-align: center;
|
||||||
|
font-weight: 650; font-size: 12px;
|
||||||
|
}
|
||||||
|
.song .tc { color: var(--text-dim); font-variant-numeric: tabular-nums; }
|
||||||
|
.song .len { color: var(--text-faint); font-variant-numeric: tabular-nums; font-size: 12px; }
|
||||||
|
|
||||||
|
.pills { display: flex; gap: 6px; flex-wrap: wrap; }
|
||||||
|
.pill { font-size: 11px; padding: 3px 9px; border-radius: 99px;
|
||||||
|
background: var(--surface-2); border: 1px solid var(--border); color: var(--text-dim); }
|
||||||
|
|
||||||
|
/* ── Toast ────────────────────────────────────────────────────────────── */
|
||||||
|
.toast-host { position: fixed; bottom: 18px; left: 50%; transform: translateX(-50%);
|
||||||
|
display: flex; flex-direction: column; gap: 8px; z-index: 50; }
|
||||||
|
.toast {
|
||||||
|
padding: 11px 18px; border-radius: 12px; box-shadow: var(--shadow-lg);
|
||||||
|
background: var(--surface); border: 1px solid var(--border);
|
||||||
|
display: flex; align-items: center; gap: 10px; font-weight: 550;
|
||||||
|
animation: rise .3s var(--ease);
|
||||||
|
}
|
||||||
|
.toast.ok { border-color: color-mix(in srgb, var(--ok) 40%, var(--border)); }
|
||||||
|
.toast.err { border-color: color-mix(in srgb, var(--danger) 40%, var(--border)); }
|
||||||
|
@keyframes rise { from { opacity: 0; transform: translateY(10px); } to { opacity: 1; transform: none; } }
|
||||||
|
|
||||||
|
.banner {
|
||||||
|
padding: 10px 14px; border-radius: 10px; font-size: 13px;
|
||||||
|
background: color-mix(in srgb, var(--warn) 14%, var(--surface));
|
||||||
|
border: 1px solid color-mix(in srgb, var(--warn) 35%, transparent);
|
||||||
|
color: var(--text);
|
||||||
|
}
|
||||||
|
.hint { font-size: 12px; color: var(--text-faint); }
|
||||||
|
.rename-preview { font-size: 12px; color: var(--accent); min-height: 16px; }
|
||||||
|
|
||||||
|
/* Detection-engine environment status */
|
||||||
|
.env-status { display: flex; align-items: center; gap: 9px; }
|
||||||
|
.env-text { font-size: 13px; color: var(--text-dim); }
|
||||||
|
.env-dot {
|
||||||
|
width: 10px; height: 10px; border-radius: 50%; flex: none;
|
||||||
|
background: var(--text-faint); box-shadow: 0 0 0 3px color-mix(in srgb, var(--text-faint) 18%, transparent);
|
||||||
|
}
|
||||||
|
.env-dot.ok { background: var(--ok); box-shadow: 0 0 0 3px color-mix(in srgb, var(--ok) 22%, transparent); }
|
||||||
|
.env-dot.bad { background: var(--danger); box-shadow: 0 0 0 3px color-mix(in srgb, var(--danger) 22%, transparent); }
|
||||||
|
.env-dot.busy { background: var(--accent); animation: env-pulse 1s ease-in-out infinite; }
|
||||||
|
|
||||||
|
@keyframes env-pulse {
|
||||||
|
0%, 100% { opacity: 1; }
|
||||||
|
50% { opacity: .35; }
|
||||||
|
}
|
||||||
|
|
||||||
|
/* First-visit attention pulse on the engine-test button */
|
||||||
|
.btn.attention { animation: btn-attention 1.4s var(--ease) infinite; }
|
||||||
|
@keyframes btn-attention {
|
||||||
|
0%, 100% { box-shadow: 0 0 0 0 color-mix(in srgb, var(--accent) 55%, transparent); }
|
||||||
|
50% { box-shadow: 0 0 0 7px color-mix(in srgb, var(--accent) 0%, transparent); }
|
||||||
|
}
|
||||||
80
src/ui.ts
Normal file
@@ -0,0 +1,80 @@
|
|||||||
|
// Shared UI helpers: toasts and a reusable progress panel.
|
||||||
|
|
||||||
|
import { h, fmtElapsed, fmtDur } from "./dom";
|
||||||
|
|
||||||
|
export function toast(message: string, kind: "ok" | "err" | "" = "") {
|
||||||
|
const host = document.getElementById("toasts")!;
|
||||||
|
const el = h("div", { class: `toast ${kind}` },
|
||||||
|
h("span", {}, kind === "ok" ? "✓" : kind === "err" ? "⚠" : "•"),
|
||||||
|
h("span", {}, message),
|
||||||
|
);
|
||||||
|
host.append(el);
|
||||||
|
setTimeout(() => {
|
||||||
|
el.style.transition = "opacity .25s, transform .25s";
|
||||||
|
el.style.opacity = "0";
|
||||||
|
el.style.transform = "translateY(8px)";
|
||||||
|
setTimeout(() => el.remove(), 260);
|
||||||
|
}, kind === "err" ? 6000 : 3200);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A progress bar + status/timing line + collapsible log, returned as one unit. */
|
||||||
|
export function makeProgress() {
|
||||||
|
const fill = h("i");
|
||||||
|
const bar = h("div", { class: "bar" }, fill);
|
||||||
|
const status = h("span", { class: "status" }, "Ready");
|
||||||
|
const timing = h("span", { class: "timing" });
|
||||||
|
const log = h("div", { class: "log" });
|
||||||
|
const root = h("div", { class: "progress-wrap" },
|
||||||
|
bar,
|
||||||
|
h("div", { class: "progress-meta" }, status, timing),
|
||||||
|
log,
|
||||||
|
);
|
||||||
|
|
||||||
|
return {
|
||||||
|
root,
|
||||||
|
reset() {
|
||||||
|
bar.classList.remove("indeterminate");
|
||||||
|
fill.style.width = "0%";
|
||||||
|
status.textContent = "Ready";
|
||||||
|
timing.textContent = "";
|
||||||
|
log.textContent = "";
|
||||||
|
},
|
||||||
|
indeterminate(on: boolean, text?: string) {
|
||||||
|
bar.classList.toggle("indeterminate", on);
|
||||||
|
if (on) fill.style.width = "35%";
|
||||||
|
if (text) status.textContent = text;
|
||||||
|
},
|
||||||
|
set(pct: number, statusText: string, opts?: { elapsed?: number; eta?: number; speed?: number }) {
|
||||||
|
bar.classList.remove("indeterminate");
|
||||||
|
fill.style.width = `${Math.round(pct * 100)}%`;
|
||||||
|
status.textContent = statusText;
|
||||||
|
if (opts) {
|
||||||
|
const bits: string[] = [];
|
||||||
|
if (opts.elapsed != null) bits.push(`Elapsed ${fmtElapsed(opts.elapsed)}`);
|
||||||
|
if (opts.eta != null && opts.eta > 0) bits.push(`ETA ${fmtElapsed(opts.eta)}`);
|
||||||
|
if (opts.speed != null && opts.speed > 0) bits.push(`${opts.speed.toFixed(1)}×`);
|
||||||
|
timing.textContent = bits.join(" · ");
|
||||||
|
}
|
||||||
|
},
|
||||||
|
setTiming(text: string) { timing.textContent = text; },
|
||||||
|
appendLog(line: string) {
|
||||||
|
log.textContent += (log.textContent ? "\n" : "") + line;
|
||||||
|
log.scrollTop = log.scrollHeight;
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export type Progress = ReturnType<typeof makeProgress>;
|
||||||
|
|
||||||
|
/** Render a list of detected songs. */
|
||||||
|
export function renderSongs(container: HTMLElement, segments: { index: number; start: number; end: number; duration: number }[]) {
|
||||||
|
container.replaceChildren(
|
||||||
|
...segments.map((s) =>
|
||||||
|
h("div", { class: "song" },
|
||||||
|
h("div", { class: "badge" }, `#${s.index}`),
|
||||||
|
h("div", { class: "tc" }, `${fmtDur(s.start)} → ${fmtDur(s.end)}`),
|
||||||
|
h("div", { class: "len" }, `${Math.round(s.duration)}s`),
|
||||||
|
),
|
||||||
|
),
|
||||||
|
);
|
||||||
|
}
|
||||||
247
src/views/detect.ts
Normal file
@@ -0,0 +1,247 @@
|
|||||||
|
// Detect view — find singing segments in one video, export editor markers.
|
||||||
|
|
||||||
|
import { h, clear, basename, fmtDur } from "../dom";
|
||||||
|
import { makeProgress, toast, renderSongs } from "../ui";
|
||||||
|
import {
|
||||||
|
pickVideo, detectSongs, cancel, exportSegments, onDetectProgress, reveal,
|
||||||
|
checkDetection, setupDetection, onDetectSetupProgress,
|
||||||
|
type Segment,
|
||||||
|
} from "../api";
|
||||||
|
|
||||||
|
export interface View {
|
||||||
|
el: HTMLElement;
|
||||||
|
onDrop: (paths: string[]) => void;
|
||||||
|
setVideo: (p: string) => void;
|
||||||
|
onShow?: () => void;
|
||||||
|
}
|
||||||
|
|
||||||
|
const FPS = ["29.97", "25", "23.976", "30", "60"];
|
||||||
|
|
||||||
|
function numField(label: string, value: number, step = "1"): [HTMLElement, HTMLInputElement] {
|
||||||
|
const input = h("input", { class: "input", type: "number", value: String(value), step, min: "0" }) as HTMLInputElement;
|
||||||
|
return [h("div", { class: "field" }, h("label", {}, label), input), input];
|
||||||
|
}
|
||||||
|
|
||||||
|
export function buildDetect(): View {
|
||||||
|
let running = false;
|
||||||
|
let settingUp = false;
|
||||||
|
let envReady = false;
|
||||||
|
let firstShow = true;
|
||||||
|
let segments: Segment[] = [];
|
||||||
|
let videoPath = "";
|
||||||
|
|
||||||
|
// ── Detection engine (Python ML) environment ────────────────────────────
|
||||||
|
const envDot = h("span", { class: "env-dot unknown" });
|
||||||
|
const envText = h("span", { class: "env-text" }, "Not checked yet.");
|
||||||
|
const testBtn = h("button", { class: "btn primary" }, "🩺 Test detection engine") as HTMLButtonElement;
|
||||||
|
const envProgress = makeProgress();
|
||||||
|
envProgress.root.style.display = "none";
|
||||||
|
|
||||||
|
function setEnv(state: "unknown" | "ok" | "bad" | "busy", text: string) {
|
||||||
|
envDot.className = `env-dot ${state}`;
|
||||||
|
envText.textContent = text;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function runEnvCheck() {
|
||||||
|
if (running || settingUp) return;
|
||||||
|
testBtn.classList.remove("attention");
|
||||||
|
testBtn.disabled = true;
|
||||||
|
setEnv("busy", "Checking environment…");
|
||||||
|
envProgress.root.style.display = "none";
|
||||||
|
try {
|
||||||
|
const st = await checkDetection();
|
||||||
|
if (st.ready) {
|
||||||
|
envReady = true;
|
||||||
|
setEnv("ok", st.model
|
||||||
|
? "Ready ✓ — model weights cached."
|
||||||
|
: "Ready ✓ — weights will download on first detection.");
|
||||||
|
testBtn.textContent = "✓ Ready — re-check";
|
||||||
|
refreshDetectBtn();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (!st.python) {
|
||||||
|
envReady = false;
|
||||||
|
setEnv("bad", st.detail);
|
||||||
|
toast(st.detail, "err");
|
||||||
|
testBtn.textContent = "🩺 Test detection engine";
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
// Python is present but packages/weights are missing → install them.
|
||||||
|
settingUp = true;
|
||||||
|
setEnv("busy", "Setting up — installing packages & downloading model…");
|
||||||
|
envProgress.root.style.display = "";
|
||||||
|
envProgress.reset();
|
||||||
|
envProgress.indeterminate(true, "Starting setup…");
|
||||||
|
const un = await onDetectSetupProgress((p) => {
|
||||||
|
if (p.pct >= 0) envProgress.set(p.pct, p.message);
|
||||||
|
else envProgress.indeterminate(true, p.message);
|
||||||
|
envProgress.appendLog(p.message);
|
||||||
|
});
|
||||||
|
try {
|
||||||
|
await setupDetection();
|
||||||
|
envProgress.indeterminate(false);
|
||||||
|
envProgress.set(1, "Setup complete — click “Test again” to confirm.");
|
||||||
|
setEnv("unknown", "Setup finished. Click Test again to confirm.");
|
||||||
|
testBtn.textContent = "Test again";
|
||||||
|
toast("Detection environment installed. Click Test again to confirm.", "ok");
|
||||||
|
} finally {
|
||||||
|
un();
|
||||||
|
}
|
||||||
|
} catch (e) {
|
||||||
|
envReady = false;
|
||||||
|
envProgress.indeterminate(false);
|
||||||
|
envProgress.set(0, "Setup failed.");
|
||||||
|
setEnv("bad", "Setup failed — see message.");
|
||||||
|
toast(String(e), "err");
|
||||||
|
} finally {
|
||||||
|
settingUp = false;
|
||||||
|
testBtn.disabled = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const videoInput = h("input", { class: "input grow", placeholder: "Select a video…", readonly: true }) as HTMLInputElement;
|
||||||
|
const [gapF, gapI] = numField("Merge gap (s)", 10);
|
||||||
|
const [minF, minI] = numField("Min duration (s)", 20);
|
||||||
|
const [padF, padI] = numField("Padding (s)", 2);
|
||||||
|
const fpsSel = h("select", { class: "input" }, ...FPS.map((f) => h("option", { value: f }, f))) as HTMLSelectElement;
|
||||||
|
|
||||||
|
const detectBtn = h("button", { class: "btn primary lg", disabled: true }, "🎤 Detect Songs") as HTMLButtonElement;
|
||||||
|
const stopBtn = h("button", { class: "btn danger", disabled: true }, "■ Stop") as HTMLButtonElement;
|
||||||
|
const exportBtn = h("button", { class: "btn", disabled: true }, "⬇ Export EDL / CSV") as HTMLButtonElement;
|
||||||
|
const revealBtn = h("button", { class: "btn ghost", style: "display:none" }, "Show in folder") as HTMLButtonElement;
|
||||||
|
const progress = makeProgress();
|
||||||
|
const statsEl = h("div", { class: "pills" });
|
||||||
|
const songsEl = h("div", { class: "songs" });
|
||||||
|
let lastExportDir = "";
|
||||||
|
|
||||||
|
function refreshDetectBtn() {
|
||||||
|
detectBtn.disabled = running || settingUp || !videoPath || !envReady;
|
||||||
|
}
|
||||||
|
|
||||||
|
function setVideo(p: string) {
|
||||||
|
videoPath = p;
|
||||||
|
videoInput.value = basename(p);
|
||||||
|
videoInput.title = p;
|
||||||
|
refreshDetectBtn();
|
||||||
|
}
|
||||||
|
|
||||||
|
async function detect() {
|
||||||
|
if (!videoPath) return;
|
||||||
|
running = true;
|
||||||
|
detectBtn.disabled = true;
|
||||||
|
stopBtn.disabled = false;
|
||||||
|
exportBtn.disabled = true;
|
||||||
|
revealBtn.style.display = "none";
|
||||||
|
clear(songsEl); clear(statsEl);
|
||||||
|
progress.reset();
|
||||||
|
progress.indeterminate(true, "Starting detection…");
|
||||||
|
|
||||||
|
const un = await onDetectProgress((msg) => {
|
||||||
|
progress.indeterminate(true, msg);
|
||||||
|
progress.appendLog(msg);
|
||||||
|
});
|
||||||
|
|
||||||
|
try {
|
||||||
|
const res = await detectSongs(videoPath, +gapI.value, +minI.value, +padI.value);
|
||||||
|
segments = res.segments;
|
||||||
|
progress.indeterminate(false);
|
||||||
|
progress.set(1, `Found ${segments.length} singing segment(s).`);
|
||||||
|
renderSongs(songsEl, segments);
|
||||||
|
const totalSing = segments.reduce((a, s) => a + s.duration, 0);
|
||||||
|
statsEl.replaceChildren(
|
||||||
|
h("span", { class: "pill" }, `${segments.length} songs`),
|
||||||
|
h("span", { class: "pill" }, `singing ${fmtDur(totalSing)}`),
|
||||||
|
...Object.entries(res.stats || {}).map(([k, v]) =>
|
||||||
|
h("span", { class: "pill" }, `${k} ${fmtDur(v as number)}`)),
|
||||||
|
);
|
||||||
|
exportBtn.disabled = segments.length === 0;
|
||||||
|
if (!segments.length) toast("No singing detected — try a larger gap or smaller min duration.", "");
|
||||||
|
} catch (e) {
|
||||||
|
const msg = String(e);
|
||||||
|
progress.indeterminate(false);
|
||||||
|
if (msg.includes("__cancelled__")) progress.set(0, "Stopped by user.");
|
||||||
|
else { progress.set(0, "Detection failed."); toast(msg, "err"); }
|
||||||
|
} finally {
|
||||||
|
un();
|
||||||
|
running = false;
|
||||||
|
stopBtn.disabled = true;
|
||||||
|
refreshDetectBtn();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
exportBtn.addEventListener("click", async () => {
|
||||||
|
if (!segments.length) return;
|
||||||
|
try {
|
||||||
|
const res = await exportSegments(segments, videoPath, null, +fpsSel.value);
|
||||||
|
lastExportDir = res.files[0] ?? "";
|
||||||
|
toast(`Exported ${res.files.length} files`, "ok");
|
||||||
|
if (lastExportDir) revealBtn.style.display = "";
|
||||||
|
} catch (e) { toast(String(e), "err"); }
|
||||||
|
});
|
||||||
|
revealBtn.addEventListener("click", () => lastExportDir && reveal(lastExportDir));
|
||||||
|
detectBtn.addEventListener("click", detect);
|
||||||
|
stopBtn.addEventListener("click", () => cancel());
|
||||||
|
testBtn.addEventListener("click", runEnvCheck);
|
||||||
|
|
||||||
|
const el = h("div", { class: "view" },
|
||||||
|
h("div", { class: "card" },
|
||||||
|
h("h2", {}, h("span", { class: "step" }, "0"), "Detection Engine"),
|
||||||
|
h("div", { class: "row", style: "align-items:center;gap:12px" },
|
||||||
|
h("div", { class: "env-status grow" }, envDot, envText),
|
||||||
|
testBtn),
|
||||||
|
h("div", { class: "hint", style: "margin-top:8px" },
|
||||||
|
"Song detection uses a Python ML model. Run this once per computer — it checks for the required environment and installs/downloads anything missing."),
|
||||||
|
h("div", { style: "margin-top:10px" }, envProgress.root),
|
||||||
|
),
|
||||||
|
h("div", { class: "card" },
|
||||||
|
h("h2", {}, h("span", { class: "step" }, "1"), "Source Video"),
|
||||||
|
h("div", { class: "row" }, videoInput,
|
||||||
|
h("button", { class: "btn", onclick: async () => { const p = await pickVideo(); if (p) setVideo(p); } }, "Browse…")),
|
||||||
|
h("div", { class: "hint", style: "margin-top:8px" }, "Tip: drop a file here, or use a Concat output."),
|
||||||
|
),
|
||||||
|
h("div", { class: "card" },
|
||||||
|
h("h2", {}, h("span", { class: "step" }, "2"), "Detection Settings"),
|
||||||
|
h("div", { class: "row wrap" },
|
||||||
|
h("div", { class: "grow", style: "min-width:130px" }, gapF),
|
||||||
|
h("div", { class: "grow", style: "min-width:130px" }, minF),
|
||||||
|
h("div", { class: "grow", style: "min-width:130px" }, padF),
|
||||||
|
h("div", { class: "field", style: "min-width:120px" }, h("label", {}, "FPS (export)"), fpsSel)),
|
||||||
|
h("div", { class: "hint", style: "margin-top:8px" }, "Singing = \"music\". More chatting between songs → raise the gap. Short covers → lower the min duration."),
|
||||||
|
),
|
||||||
|
h("div", { class: "card" },
|
||||||
|
h("h2", {}, h("span", { class: "step" }, "3"), "Run"),
|
||||||
|
h("div", { class: "row", style: "margin-bottom:14px" }, detectBtn, stopBtn, exportBtn, revealBtn),
|
||||||
|
progress.root,
|
||||||
|
h("div", { style: "margin-top:12px" }, statsEl),
|
||||||
|
h("div", { style: "margin-top:10px" }, songsEl),
|
||||||
|
),
|
||||||
|
);
|
||||||
|
|
||||||
|
function onShow() {
|
||||||
|
if (!firstShow) return;
|
||||||
|
firstShow = false;
|
||||||
|
// Draw attention to the engine test the first time the user lands here…
|
||||||
|
testBtn.classList.add("attention");
|
||||||
|
// …and quietly probe in the background so an already-set-up machine
|
||||||
|
// flips straight to "Ready ✓" without the user doing anything.
|
||||||
|
checkDetection()
|
||||||
|
.then((st) => {
|
||||||
|
if (st.ready) {
|
||||||
|
envReady = true;
|
||||||
|
testBtn.classList.remove("attention");
|
||||||
|
setEnv("ok", st.model
|
||||||
|
? "Ready ✓ — model weights cached."
|
||||||
|
: "Ready ✓ — weights will download on first detection.");
|
||||||
|
testBtn.textContent = "✓ Ready — re-check";
|
||||||
|
refreshDetectBtn();
|
||||||
|
} else if (st.python) {
|
||||||
|
setEnv("bad", "Setup needed — click “Test detection engine”.");
|
||||||
|
} else {
|
||||||
|
setEnv("bad", st.detail);
|
||||||
|
}
|
||||||
|
})
|
||||||
|
.catch(() => { /* leave the highlight; user can click to find out */ });
|
||||||
|
}
|
||||||
|
|
||||||
|
return { el, onDrop: (paths) => { if (paths[0]) setVideo(paths[0]); }, setVideo, onShow };
|
||||||
|
}
|
||||||
23
tsconfig.json
Normal file
@@ -0,0 +1,23 @@
|
|||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"target": "ES2020",
|
||||||
|
"useDefineForClassFields": true,
|
||||||
|
"module": "ESNext",
|
||||||
|
"lib": ["ES2020", "DOM", "DOM.Iterable"],
|
||||||
|
"skipLibCheck": true,
|
||||||
|
|
||||||
|
/* Bundler mode */
|
||||||
|
"moduleResolution": "bundler",
|
||||||
|
"allowImportingTsExtensions": true,
|
||||||
|
"resolveJsonModule": true,
|
||||||
|
"isolatedModules": true,
|
||||||
|
"noEmit": true,
|
||||||
|
|
||||||
|
/* Linting */
|
||||||
|
"strict": true,
|
||||||
|
"noUnusedLocals": true,
|
||||||
|
"noUnusedParameters": true,
|
||||||
|
"noFallthroughCasesInSwitch": true
|
||||||
|
},
|
||||||
|
"include": ["src"]
|
||||||
|
}
|
||||||
505
ui/tab_detect.py
@@ -1,505 +0,0 @@
|
|||||||
"""
|
|
||||||
Tab 2: Song Detection
|
|
||||||
Detect singing segments in a video and export EDL/CSV/JSON for DaVinci Resolve.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import os
|
|
||||||
import threading
|
|
||||||
import tkinter as tk
|
|
||||||
from pathlib import Path
|
|
||||||
from tkinter import filedialog, messagebox
|
|
||||||
|
|
||||||
import customtkinter as ctk
|
|
||||||
|
|
||||||
from core.fftools import find_tool, extract_audio
|
|
||||||
from core.segmenter import check_segmenter_available, run_segmentation, merge_singing_segments, get_segment_stats
|
|
||||||
from core.exporters import export_edl, export_csv, export_markers_csv, export_json, export_ffmpeg_script
|
|
||||||
from utils.formats import format_timecode, fmt_dur, fmt_elapsed
|
|
||||||
from ui.theme import BTN_NEUTRAL, ACCENT_PURPLE, ACCENT_GREEN
|
|
||||||
|
|
||||||
|
|
||||||
class DetectTab:
|
|
||||||
"""Song detection tab."""
|
|
||||||
|
|
||||||
VIDEO_EXTS = {".mp4", ".mov", ".mkv", ".avi", ".flv",
|
|
||||||
".ts", ".wmv", ".m4v", ".webm", ".mts", ".m2ts"}
|
|
||||||
|
|
||||||
def __init__(self, parent_frame, app):
|
|
||||||
self.frame = parent_frame
|
|
||||||
self.app = app
|
|
||||||
self.ffmpeg = app.ffmpeg
|
|
||||||
|
|
||||||
# State
|
|
||||||
self.input_path_var = tk.StringVar(value="")
|
|
||||||
self.output_dir_var = tk.StringVar(value="")
|
|
||||||
self.gap_var = tk.DoubleVar(value=10.0)
|
|
||||||
self.min_dur_var = tk.DoubleVar(value=20.0)
|
|
||||||
self.padding_var = tk.DoubleVar(value=2.0)
|
|
||||||
self.fps_var = tk.DoubleVar(value=29.97)
|
|
||||||
self.autocut_var = tk.BooleanVar(value=False)
|
|
||||||
|
|
||||||
self._running = False
|
|
||||||
self._stop_req = False
|
|
||||||
|
|
||||||
# Results
|
|
||||||
self._singing_segments = []
|
|
||||||
self._raw_segments = []
|
|
||||||
|
|
||||||
self._build_ui()
|
|
||||||
self._check_deps()
|
|
||||||
|
|
||||||
# ══════════════════════════════════════════════════════════════════════════
|
|
||||||
# UI
|
|
||||||
# ══════════════════════════════════════════════════════════════════════════
|
|
||||||
|
|
||||||
def _build_ui(self):
|
|
||||||
scroll = ctk.CTkScrollableFrame(self.frame, corner_radius=12)
|
|
||||||
scroll.pack(fill="both", expand=True, padx=4, pady=4)
|
|
||||||
|
|
||||||
self._build_sec_input(scroll)
|
|
||||||
self._build_sec_params(scroll)
|
|
||||||
self._build_sec_run(scroll)
|
|
||||||
self._build_sec_results(scroll)
|
|
||||||
|
|
||||||
def _section(self, parent, title):
|
|
||||||
f = ctk.CTkFrame(parent, corner_radius=12)
|
|
||||||
hdr = ctk.CTkFrame(f, fg_color="transparent")
|
|
||||||
hdr.pack(fill="x", padx=14, pady=(10, 0))
|
|
||||||
ctk.CTkLabel(hdr, text=title,
|
|
||||||
font=ctk.CTkFont(size=13, weight="bold")).pack(side="left")
|
|
||||||
ctk.CTkFrame(f, height=1, fg_color=("gray78", "gray32")).pack(
|
|
||||||
fill="x", padx=14, pady=(6, 0))
|
|
||||||
return f
|
|
||||||
|
|
||||||
# ── Input ─────────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def _build_sec_input(self, parent):
|
|
||||||
sec = self._section(parent, "1 · Input Video")
|
|
||||||
sec.pack(fill="x", pady=(0, 12))
|
|
||||||
|
|
||||||
body = ctk.CTkFrame(sec, fg_color="transparent")
|
|
||||||
body.pack(fill="x", padx=14, pady=(10, 4))
|
|
||||||
|
|
||||||
r1 = ctk.CTkFrame(body, fg_color="transparent")
|
|
||||||
r1.pack(fill="x", pady=(0, 4))
|
|
||||||
ctk.CTkLabel(r1, text="Video file:", width=96, anchor="w").pack(side="left")
|
|
||||||
ctk.CTkEntry(r1, textvariable=self.input_path_var).pack(side="left", fill="x", expand=True)
|
|
||||||
ctk.CTkButton(r1, text="Browse…", width=100, command=self._pick_input,
|
|
||||||
**BTN_NEUTRAL).pack(side="left", padx=(8, 0))
|
|
||||||
|
|
||||||
# "Use Concat Output" button
|
|
||||||
self.use_concat_btn = ctk.CTkButton(
|
|
||||||
body, text="📎 Use Concat Output", width=200,
|
|
||||||
fg_color=ACCENT_PURPLE["fg"], text_color=ACCENT_PURPLE["text"],
|
|
||||||
hover_color=ACCENT_PURPLE["hover"],
|
|
||||||
command=self._use_concat_output)
|
|
||||||
self.use_concat_btn.pack(anchor="w", pady=(4, 4))
|
|
||||||
|
|
||||||
r2 = ctk.CTkFrame(body, fg_color="transparent")
|
|
||||||
r2.pack(fill="x", pady=(4, 0))
|
|
||||||
ctk.CTkLabel(r2, text="Output dir:", width=96, anchor="w").pack(side="left")
|
|
||||||
ctk.CTkEntry(r2, textvariable=self.output_dir_var).pack(side="left", fill="x", expand=True)
|
|
||||||
ctk.CTkButton(r2, text="Browse…", width=100, command=self._pick_output_dir,
|
|
||||||
**BTN_NEUTRAL).pack(side="left", padx=(8, 0))
|
|
||||||
|
|
||||||
ctk.CTkLabel(sec, text="Output defaults to same directory as input video.",
|
|
||||||
font=ctk.CTkFont(size=11), text_color=("gray50", "gray55")
|
|
||||||
).pack(anchor="w", padx=14, pady=(4, 10))
|
|
||||||
|
|
||||||
# ── Parameters ────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def _build_sec_params(self, parent):
|
|
||||||
sec = self._section(parent, "2 · Detection Parameters")
|
|
||||||
sec.pack(fill="x", pady=(0, 12))
|
|
||||||
|
|
||||||
body = ctk.CTkFrame(sec, fg_color="transparent")
|
|
||||||
body.pack(fill="x", padx=14, pady=(10, 10))
|
|
||||||
|
|
||||||
params = [
|
|
||||||
("Merge gap (s):", self.gap_var, 1, 60, "Max silence between song parts to merge"),
|
|
||||||
("Min duration (s):", self.min_dur_var, 5, 120, "Ignore segments shorter than this"),
|
|
||||||
("Padding (s):", self.padding_var, 0, 15, "Extra seconds before/after each song"),
|
|
||||||
("FPS:", self.fps_var, 23, 60, "Video frame rate for timecodes"),
|
|
||||||
]
|
|
||||||
|
|
||||||
for label_text, var, lo, hi, tooltip in params:
|
|
||||||
row = ctk.CTkFrame(body, fg_color="transparent")
|
|
||||||
row.pack(fill="x", pady=3)
|
|
||||||
|
|
||||||
ctk.CTkLabel(row, text=label_text, width=130, anchor="w",
|
|
||||||
font=ctk.CTkFont(size=12)).pack(side="left")
|
|
||||||
|
|
||||||
slider = ctk.CTkSlider(row, from_=lo, to=hi, variable=var,
|
|
||||||
width=200, number_of_steps=max(1, hi - lo))
|
|
||||||
slider.pack(side="left", padx=(0, 8))
|
|
||||||
|
|
||||||
val_lbl = ctk.CTkLabel(row, text=f"{var.get():.1f}", width=50, anchor="w",
|
|
||||||
font=ctk.CTkFont(size=12, weight="bold"))
|
|
||||||
val_lbl.pack(side="left")
|
|
||||||
|
|
||||||
ctk.CTkLabel(row, text=tooltip, font=ctk.CTkFont(size=11),
|
|
||||||
text_color=("gray50", "gray55")).pack(side="left", padx=(10, 0))
|
|
||||||
|
|
||||||
# Update label on slider move
|
|
||||||
var.trace_add("write", lambda *_, v=var, l=val_lbl: l.configure(text=f"{v.get():.1f}"))
|
|
||||||
|
|
||||||
# Auto-cut checkbox
|
|
||||||
ctk.CTkCheckBox(body, text="Generate auto-cut ffmpeg script (no DaVinci needed)",
|
|
||||||
variable=self.autocut_var).pack(anchor="w", pady=(8, 0))
|
|
||||||
|
|
||||||
# ── Run ───────────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def _build_sec_run(self, parent):
|
|
||||||
sec = self._section(parent, "3 · Detect")
|
|
||||||
sec.pack(fill="x", pady=(0, 12))
|
|
||||||
|
|
||||||
body = ctk.CTkFrame(sec, fg_color="transparent")
|
|
||||||
body.pack(fill="x", padx=14, pady=(12, 6))
|
|
||||||
|
|
||||||
btn_row = ctk.CTkFrame(body, fg_color="transparent")
|
|
||||||
btn_row.pack(fill="x", pady=(0, 10))
|
|
||||||
|
|
||||||
self.detect_btn = ctk.CTkButton(
|
|
||||||
btn_row, text="🎤 Start Detection", width=230, height=42,
|
|
||||||
font=ctk.CTkFont(size=14, weight="bold"),
|
|
||||||
fg_color=ACCENT_GREEN["fg"], hover_color=ACCENT_GREEN["hover"],
|
|
||||||
command=self.start_detect)
|
|
||||||
self.detect_btn.pack(side="left")
|
|
||||||
|
|
||||||
self.stop_btn = ctk.CTkButton(
|
|
||||||
btn_row, text="■ Stop", width=106, height=42,
|
|
||||||
fg_color=("gray80", "gray25"), text_color=("gray10", "gray90"),
|
|
||||||
hover_color=("#FCA5A5", "#7F1D1D"),
|
|
||||||
command=self._stop, state="disabled")
|
|
||||||
self.stop_btn.pack(side="left", padx=(12, 0))
|
|
||||||
|
|
||||||
# Dependency warning
|
|
||||||
self.dep_lbl = ctk.CTkLabel(body, text="", font=ctk.CTkFont(size=11),
|
|
||||||
text_color="#F97316", wraplength=600, anchor="w", justify="left")
|
|
||||||
self.dep_lbl.pack(fill="x", pady=(0, 4))
|
|
||||||
|
|
||||||
# Progress
|
|
||||||
self.progress = ctk.CTkProgressBar(body, height=16, corner_radius=8, mode="indeterminate")
|
|
||||||
self.progress.pack(fill="x", pady=(0, 6))
|
|
||||||
self.progress.stop()
|
|
||||||
self.progress.set(0)
|
|
||||||
|
|
||||||
self.status_lbl = ctk.CTkLabel(body, text="", anchor="w", font=ctk.CTkFont(size=12))
|
|
||||||
self.status_lbl.pack(fill="x")
|
|
||||||
|
|
||||||
ctk.CTkFrame(sec, height=6, fg_color="transparent").pack()
|
|
||||||
|
|
||||||
# ── Results ───────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def _build_sec_results(self, parent):
|
|
||||||
sec = self._section(parent, "4 · Results")
|
|
||||||
sec.pack(fill="x", pady=(0, 14))
|
|
||||||
|
|
||||||
self.results_body = ctk.CTkFrame(sec, fg_color="transparent")
|
|
||||||
self.results_body.pack(fill="x", padx=14, pady=(10, 10))
|
|
||||||
|
|
||||||
self.results_lbl = ctk.CTkLabel(
|
|
||||||
self.results_body, text="No results yet. Run detection first.",
|
|
||||||
font=ctk.CTkFont(size=12), text_color=("gray50", "gray55"))
|
|
||||||
self.results_lbl.pack(anchor="w")
|
|
||||||
|
|
||||||
# Song list (populated after detection)
|
|
||||||
self.song_list_frame = ctk.CTkFrame(self.results_body, fg_color="transparent")
|
|
||||||
|
|
||||||
# Export buttons (hidden until results)
|
|
||||||
self.export_frame = ctk.CTkFrame(sec, fg_color="transparent")
|
|
||||||
|
|
||||||
# ══════════════════════════════════════════════════════════════════════════
|
|
||||||
# Logic
|
|
||||||
# ══════════════════════════════════════════════════════════════════════════
|
|
||||||
|
|
||||||
def _check_deps(self):
|
|
||||||
available, msg = check_segmenter_available()
|
|
||||||
if not available:
|
|
||||||
self.dep_lbl.configure(text=f"⚠ {msg}")
|
|
||||||
self.detect_btn.configure(state="disabled")
|
|
||||||
else:
|
|
||||||
self.dep_lbl.configure(text="")
|
|
||||||
|
|
||||||
if not self.ffmpeg:
|
|
||||||
self.dep_lbl.configure(text="⚠ ffmpeg not found. Install ffmpeg and add to PATH.")
|
|
||||||
self.detect_btn.configure(state="disabled")
|
|
||||||
|
|
||||||
def _pick_input(self):
|
|
||||||
f = filedialog.askopenfilename(
|
|
||||||
title="Select video file",
|
|
||||||
filetypes=[("Video files", "*.mp4 *.mov *.mkv *.avi *.flv *.ts *.wmv *.m4v *.webm *.mts"),
|
|
||||||
("All files", "*.*")])
|
|
||||||
if f:
|
|
||||||
self.input_path_var.set(f)
|
|
||||||
if not self.output_dir_var.get().strip():
|
|
||||||
self.output_dir_var.set(str(Path(f).parent))
|
|
||||||
|
|
||||||
def _pick_output_dir(self):
|
|
||||||
d = filedialog.askdirectory(title="Choose output folder")
|
|
||||||
if d:
|
|
||||||
self.output_dir_var.set(d)
|
|
||||||
|
|
||||||
def _use_concat_output(self):
|
|
||||||
"""Fill input from the Concat tab's output path."""
|
|
||||||
if hasattr(self.app, 'concat_tab'):
|
|
||||||
path = self.app.concat_tab.get_last_output_path()
|
|
||||||
if path and Path(path).exists():
|
|
||||||
self.input_path_var.set(path)
|
|
||||||
if not self.output_dir_var.get().strip():
|
|
||||||
self.output_dir_var.set(str(Path(path).parent))
|
|
||||||
self._set_status(f"Loaded concat output: {Path(path).name}")
|
|
||||||
elif path:
|
|
||||||
self.input_path_var.set(path)
|
|
||||||
if not self.output_dir_var.get().strip():
|
|
||||||
self.output_dir_var.set(str(Path(path).parent))
|
|
||||||
self._set_status("Concat output path set (file not yet created — run concat first)")
|
|
||||||
else:
|
|
||||||
messagebox.showinfo("No output", "Set an output path in the Concat tab first.")
|
|
||||||
|
|
||||||
def _set_status(self, text, error=False):
|
|
||||||
color = "#DC2626" if error else ("gray10", "gray90")
|
|
||||||
self.status_lbl.configure(text=text, text_color=color)
|
|
||||||
|
|
||||||
def _stop(self):
|
|
||||||
self._stop_req = True
|
|
||||||
self._set_status("Stopping…")
|
|
||||||
|
|
||||||
def start_detect(self):
|
|
||||||
"""Start the detection pipeline in a background thread."""
|
|
||||||
input_path = self.input_path_var.get().strip()
|
|
||||||
if not input_path:
|
|
||||||
messagebox.showerror("No input", "Select a video file first.")
|
|
||||||
return
|
|
||||||
if not os.path.isfile(input_path):
|
|
||||||
messagebox.showerror("File not found", f"Cannot find:\n{input_path}")
|
|
||||||
return
|
|
||||||
|
|
||||||
available, msg = check_segmenter_available()
|
|
||||||
if not available:
|
|
||||||
messagebox.showerror("Missing dependency", msg)
|
|
||||||
return
|
|
||||||
|
|
||||||
self._running = True
|
|
||||||
self._stop_req = False
|
|
||||||
self.detect_btn.configure(state="disabled")
|
|
||||||
self.stop_btn.configure(state="normal")
|
|
||||||
self.progress.configure(mode="indeterminate")
|
|
||||||
self.progress.start()
|
|
||||||
self._set_status("Starting detection…")
|
|
||||||
|
|
||||||
threading.Thread(target=self._detect_worker, args=(input_path,), daemon=True).start()
|
|
||||||
|
|
||||||
def _detect_worker(self, input_path):
|
|
||||||
output_dir = self.output_dir_var.get().strip() or str(Path(input_path).parent)
|
|
||||||
os.makedirs(output_dir, exist_ok=True)
|
|
||||||
stem = Path(input_path).stem
|
|
||||||
|
|
||||||
wav_path = os.path.join(output_dir, f"{stem}_audio.wav")
|
|
||||||
|
|
||||||
try:
|
|
||||||
# Step 1: Extract audio
|
|
||||||
def progress_cb(msg):
|
|
||||||
self.app.after(0, lambda: self._set_status(msg))
|
|
||||||
|
|
||||||
extract_audio(input_path, wav_path, self.ffmpeg, progress_cb=progress_cb)
|
|
||||||
|
|
||||||
if self._stop_req:
|
|
||||||
self._cleanup_and_finish(wav_path, "Stopped by user.")
|
|
||||||
return
|
|
||||||
|
|
||||||
# Step 2: Run segmentation
|
|
||||||
self.app.after(0, lambda: self._set_status("Running audio segmentation (this may take a while)…"))
|
|
||||||
raw_segments = run_segmentation(wav_path, progress_cb=progress_cb)
|
|
||||||
|
|
||||||
if self._stop_req:
|
|
||||||
self._cleanup_and_finish(wav_path, "Stopped by user.")
|
|
||||||
return
|
|
||||||
|
|
||||||
self._raw_segments = raw_segments
|
|
||||||
|
|
||||||
# Step 3: Merge
|
|
||||||
self.app.after(0, lambda: self._set_status("Merging singing segments…"))
|
|
||||||
singing = merge_singing_segments(
|
|
||||||
raw_segments,
|
|
||||||
max_gap=self.gap_var.get(),
|
|
||||||
min_duration=self.min_dur_var.get(),
|
|
||||||
padding=self.padding_var.get(),
|
|
||||||
)
|
|
||||||
self._singing_segments = singing
|
|
||||||
|
|
||||||
# Step 4: Export
|
|
||||||
if singing:
|
|
||||||
fps = self.fps_var.get()
|
|
||||||
source_name = Path(input_path).name
|
|
||||||
|
|
||||||
edl_path = os.path.join(output_dir, f"{stem}_singing.edl")
|
|
||||||
csv_path = os.path.join(output_dir, f"{stem}_singing.csv")
|
|
||||||
markers_path = os.path.join(output_dir, f"{stem}_markers.csv")
|
|
||||||
json_path = os.path.join(output_dir, f"{stem}_singing.json")
|
|
||||||
|
|
||||||
export_edl(singing, edl_path, fps=fps,
|
|
||||||
title=f"{stem} - Singing", source_filename=source_name)
|
|
||||||
export_csv(singing, csv_path)
|
|
||||||
export_markers_csv(singing, markers_path, fps=fps)
|
|
||||||
export_json(singing, json_path)
|
|
||||||
|
|
||||||
if self.autocut_var.get():
|
|
||||||
export_ffmpeg_script(singing, input_path, output_dir)
|
|
||||||
|
|
||||||
total_singing = sum(s["duration"] for s in singing)
|
|
||||||
result_msg = (f"✓ Found {len(singing)} songs · "
|
|
||||||
f"Total singing: {format_timecode(total_singing)} · "
|
|
||||||
f"Exported to: {output_dir}")
|
|
||||||
|
|
||||||
self.app.after(0, lambda: self._show_results(singing, edl_path, output_dir))
|
|
||||||
else:
|
|
||||||
result_msg = "No singing segments detected. Try lowering min duration or increasing gap."
|
|
||||||
|
|
||||||
# Cleanup WAV
|
|
||||||
if os.path.exists(wav_path):
|
|
||||||
os.remove(wav_path)
|
|
||||||
|
|
||||||
self.app.after(0, lambda: self._set_status(result_msg))
|
|
||||||
|
|
||||||
except Exception as e:
|
|
||||||
self.app.after(0, lambda: self._set_status(f"Error: {e}", error=True))
|
|
||||||
if os.path.exists(wav_path):
|
|
||||||
try:
|
|
||||||
os.remove(wav_path)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
self.app.after(0, self._finish_detect)
|
|
||||||
|
|
||||||
def _cleanup_and_finish(self, wav_path, msg):
|
|
||||||
if os.path.exists(wav_path):
|
|
||||||
try:
|
|
||||||
os.remove(wav_path)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
self.app.after(0, lambda: self._set_status(msg))
|
|
||||||
self.app.after(0, self._finish_detect)
|
|
||||||
|
|
||||||
def _finish_detect(self):
|
|
||||||
self._running = False
|
|
||||||
self._stop_req = False
|
|
||||||
self.detect_btn.configure(state="normal")
|
|
||||||
self.stop_btn.configure(state="disabled")
|
|
||||||
self.progress.stop()
|
|
||||||
self.progress.configure(mode="determinate")
|
|
||||||
self.progress.set(1.0 if self._singing_segments else 0)
|
|
||||||
|
|
||||||
def _show_results(self, segments, edl_path, output_dir):
|
|
||||||
"""Display detection results in the Results section."""
|
|
||||||
# Clear previous
|
|
||||||
for w in self.song_list_frame.winfo_children():
|
|
||||||
w.destroy()
|
|
||||||
self.export_frame.pack_forget()
|
|
||||||
|
|
||||||
self.results_lbl.configure(
|
|
||||||
text=f"Found {len(segments)} singing segments:",
|
|
||||||
text_color=("gray10", "gray90"))
|
|
||||||
|
|
||||||
self.song_list_frame.pack(fill="x", pady=(8, 0))
|
|
||||||
|
|
||||||
# Header
|
|
||||||
hdr = ctk.CTkFrame(self.song_list_frame, fg_color=("gray88", "gray20"), corner_radius=8)
|
|
||||||
hdr.pack(fill="x", pady=(0, 4))
|
|
||||||
for text, w in [("#", 40), ("Start", 90), ("End", 90), ("Duration", 80)]:
|
|
||||||
ctk.CTkLabel(hdr, text=text, width=w, anchor="center",
|
|
||||||
font=ctk.CTkFont(size=11, weight="bold")).pack(side="left", padx=4, pady=4)
|
|
||||||
|
|
||||||
# Rows
|
|
||||||
for seg in segments:
|
|
||||||
row = ctk.CTkFrame(self.song_list_frame, corner_radius=6, height=30)
|
|
||||||
row.pack(fill="x", pady=2)
|
|
||||||
row.pack_propagate(False)
|
|
||||||
|
|
||||||
vals = [
|
|
||||||
(f"Song {seg['index']}", 40),
|
|
||||||
(format_timecode(seg["start"]), 90),
|
|
||||||
(format_timecode(seg["end"]), 90),
|
|
||||||
(f"{seg['duration']:.0f}s", 80),
|
|
||||||
]
|
|
||||||
for text, w in vals:
|
|
||||||
ctk.CTkLabel(row, text=text, width=w, anchor="center",
|
|
||||||
font=ctk.CTkFont(size=11)).pack(side="left", padx=4)
|
|
||||||
|
|
||||||
# Total
|
|
||||||
total = sum(s["duration"] for s in segments)
|
|
||||||
ctk.CTkLabel(self.song_list_frame,
|
|
||||||
text=f"Total singing time: {format_timecode(total)}",
|
|
||||||
font=ctk.CTkFont(size=12, weight="bold")).pack(anchor="w", pady=(8, 0))
|
|
||||||
|
|
||||||
# Export info
|
|
||||||
self.export_frame.pack(fill="x", padx=14, pady=(4, 10))
|
|
||||||
for w in self.export_frame.winfo_children():
|
|
||||||
w.destroy()
|
|
||||||
|
|
||||||
ctk.CTkLabel(self.export_frame,
|
|
||||||
text=f"Files exported to: {output_dir}",
|
|
||||||
font=ctk.CTkFont(size=11), text_color=("gray45", "gray55"),
|
|
||||||
wraplength=600, anchor="w", justify="left").pack(anchor="w")
|
|
||||||
|
|
||||||
info_lines = [
|
|
||||||
"• _singing.edl → DaVinci Resolve: File → Import → Timeline",
|
|
||||||
"• _markers.csv → DaVinci Resolve: Timeline → Import Markers from CSV",
|
|
||||||
"• _singing.csv → Human-readable segment list",
|
|
||||||
"• _singing.json → Programmatic use",
|
|
||||||
]
|
|
||||||
if self.autocut_var.get():
|
|
||||||
info_lines.append("• _autocut.bat → Run to auto-cut singing segments (no DaVinci needed)")
|
|
||||||
|
|
||||||
for line in info_lines:
|
|
||||||
ctk.CTkLabel(self.export_frame, text=line,
|
|
||||||
font=ctk.CTkFont(size=11),
|
|
||||||
text_color=("gray50", "gray55"), anchor="w").pack(anchor="w")
|
|
||||||
|
|
||||||
# ── Public API for Quick Flow ─────────────────────────────────────────────
|
|
||||||
|
|
||||||
def run_detect_headless(self, input_path: str, output_dir: str,
|
|
||||||
gap: float, min_dur: float, padding: float,
|
|
||||||
fps: float, autocut: bool,
|
|
||||||
progress_cb=None) -> list:
|
|
||||||
"""
|
|
||||||
Run detection synchronously (called from Quick Flow worker thread).
|
|
||||||
Returns list of singing segments.
|
|
||||||
"""
|
|
||||||
os.makedirs(output_dir, exist_ok=True)
|
|
||||||
stem = Path(input_path).stem
|
|
||||||
wav_path = os.path.join(output_dir, f"{stem}_audio.wav")
|
|
||||||
|
|
||||||
try:
|
|
||||||
extract_audio(input_path, wav_path, self.ffmpeg, progress_cb=progress_cb)
|
|
||||||
|
|
||||||
raw_segments = run_segmentation(wav_path, progress_cb=progress_cb)
|
|
||||||
|
|
||||||
if progress_cb:
|
|
||||||
progress_cb("Merging singing segments…")
|
|
||||||
|
|
||||||
singing = merge_singing_segments(raw_segments,
|
|
||||||
max_gap=gap, min_duration=min_dur, padding=padding)
|
|
||||||
|
|
||||||
if singing:
|
|
||||||
source_name = Path(input_path).name
|
|
||||||
export_edl(singing, os.path.join(output_dir, f"{stem}_singing.edl"),
|
|
||||||
fps=fps, title=f"{stem} - Singing", source_filename=source_name)
|
|
||||||
export_csv(singing, os.path.join(output_dir, f"{stem}_singing.csv"))
|
|
||||||
export_markers_csv(singing, os.path.join(output_dir, f"{stem}_markers.csv"), fps=fps)
|
|
||||||
export_json(singing, os.path.join(output_dir, f"{stem}_singing.json"))
|
|
||||||
if autocut:
|
|
||||||
export_ffmpeg_script(singing, input_path, output_dir)
|
|
||||||
|
|
||||||
if os.path.exists(wav_path):
|
|
||||||
os.remove(wav_path)
|
|
||||||
|
|
||||||
return singing
|
|
||||||
|
|
||||||
except Exception:
|
|
||||||
if os.path.exists(wav_path):
|
|
||||||
try:
|
|
||||||
os.remove(wav_path)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
raise
|
|
||||||
64
ui/theme.py
@@ -1,64 +0,0 @@
|
|||||||
"""
|
|
||||||
Theme, DPI, and shared styling constants.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import sys
|
|
||||||
|
|
||||||
# ── DPI awareness (call before any Tk/CTk window) ────────────────────────────
|
|
||||||
def setup_dpi():
|
|
||||||
if sys.platform == "win32":
|
|
||||||
try:
|
|
||||||
import ctypes
|
|
||||||
ctypes.windll.shcore.SetProcessDpiAwareness(2)
|
|
||||||
except Exception:
|
|
||||||
try:
|
|
||||||
import ctypes
|
|
||||||
ctypes.windll.user32.SetProcessDPIAware()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def dpi_scale(widget) -> float:
|
|
||||||
try:
|
|
||||||
return max(1.0, widget.winfo_fpixels("1i") / 96.0)
|
|
||||||
except Exception:
|
|
||||||
return 1.0
|
|
||||||
|
|
||||||
|
|
||||||
# ── Color accents ─────────────────────────────────────────────────────────────
|
|
||||||
# Each feature area has its own accent color scheme: (light_mode, dark_mode)
|
|
||||||
|
|
||||||
ACCENT_BLUE = {
|
|
||||||
"fg": ("#3B82F6", "#2563EB"),
|
|
||||||
"hover": ("#2563EB", "#1D4ED8"),
|
|
||||||
}
|
|
||||||
|
|
||||||
ACCENT_PURPLE = {
|
|
||||||
"fg": ("#EDE9FE", "#3B1F6E"),
|
|
||||||
"text": ("#5B21B6", "#C4B5FD"),
|
|
||||||
"hover": ("#DDD6FE", "#4C2889"),
|
|
||||||
"preview": ("#7C3AED", "#A78BFA"),
|
|
||||||
}
|
|
||||||
|
|
||||||
ACCENT_GREEN = {
|
|
||||||
"fg": ("#10B981", "#059669"),
|
|
||||||
"hover": ("#059669", "#047857"),
|
|
||||||
}
|
|
||||||
|
|
||||||
ACCENT_AMBER = {
|
|
||||||
"fg": ("#F59E0B", "#D97706"),
|
|
||||||
"hover": ("#D97706", "#B45309"),
|
|
||||||
}
|
|
||||||
|
|
||||||
# Neutral button style
|
|
||||||
BTN_NEUTRAL = {
|
|
||||||
"fg_color": ("gray80", "gray25"),
|
|
||||||
"text_color": ("gray10", "gray90"),
|
|
||||||
"hover_color": ("gray70", "gray35"),
|
|
||||||
}
|
|
||||||
|
|
||||||
BTN_SIDE = {
|
|
||||||
"fg_color": ("gray82", "gray22"),
|
|
||||||
"text_color": ("gray10", "gray90"),
|
|
||||||
"hover_color": ("gray72", "gray32"),
|
|
||||||
}
|
|
||||||
@@ -1,42 +0,0 @@
|
|||||||
"""
|
|
||||||
Timecode & duration formatting utilities.
|
|
||||||
"""
|
|
||||||
|
|
||||||
|
|
||||||
def fmt_dur(seconds: float) -> str:
|
|
||||||
"""Format a duration as H:MM:SS or M:SS for display."""
|
|
||||||
if seconds < 0:
|
|
||||||
return "…"
|
|
||||||
if seconds == 0:
|
|
||||||
return "—"
|
|
||||||
s = int(round(seconds))
|
|
||||||
h, rem = divmod(s, 3600)
|
|
||||||
m, sec = divmod(rem, 60)
|
|
||||||
return f"{h}:{m:02d}:{sec:02d}" if h else f"{m}:{sec:02d}"
|
|
||||||
|
|
||||||
|
|
||||||
def fmt_elapsed(s: float) -> str:
|
|
||||||
if s < 60:
|
|
||||||
return f"{int(s)}s"
|
|
||||||
m, sec = divmod(int(s), 60)
|
|
||||||
return f"{m}m {sec:02d}s"
|
|
||||||
|
|
||||||
|
|
||||||
def format_timecode(seconds: float) -> str:
|
|
||||||
"""Format seconds as HH:MM:SS."""
|
|
||||||
h = int(seconds // 3600)
|
|
||||||
m = int((seconds % 3600) // 60)
|
|
||||||
s = int(seconds % 60)
|
|
||||||
return f"{h:02d}:{m:02d}:{s:02d}"
|
|
||||||
|
|
||||||
|
|
||||||
def seconds_to_smpte(seconds: float, fps: float = 29.97) -> str:
|
|
||||||
"""Convert seconds to SMPTE timecode HH:MM:SS:FF."""
|
|
||||||
total_frames = int(seconds * fps)
|
|
||||||
ff = total_frames % int(round(fps))
|
|
||||||
total_seconds = total_frames // int(round(fps))
|
|
||||||
ss = total_seconds % 60
|
|
||||||
total_minutes = total_seconds // 60
|
|
||||||
mm = total_minutes % 60
|
|
||||||
hh = total_minutes // 60
|
|
||||||
return f"{hh:02d}:{mm:02d}:{ss:02d}:{ff:02d}"
|
|
||||||
30
vite.config.ts
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
import { defineConfig } from "vite";
|
||||||
|
|
||||||
|
// @ts-expect-error process is a nodejs global
|
||||||
|
const host = process.env.TAURI_DEV_HOST;
|
||||||
|
|
||||||
|
// https://vite.dev/config/
|
||||||
|
export default defineConfig(async () => ({
|
||||||
|
|
||||||
|
// Vite options tailored for Tauri development and only applied in `tauri dev` or `tauri build`
|
||||||
|
//
|
||||||
|
// 1. prevent Vite from obscuring rust errors
|
||||||
|
clearScreen: false,
|
||||||
|
// 2. tauri expects a fixed port, fail if that port is not available
|
||||||
|
server: {
|
||||||
|
port: 1420,
|
||||||
|
strictPort: true,
|
||||||
|
host: host || false,
|
||||||
|
hmr: host
|
||||||
|
? {
|
||||||
|
protocol: "ws",
|
||||||
|
host,
|
||||||
|
port: 1421,
|
||||||
|
}
|
||||||
|
: undefined,
|
||||||
|
watch: {
|
||||||
|
// 3. tell Vite to ignore watching `src-tauri`
|
||||||
|
ignored: ["**/src-tauri/**"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}));
|
||||||