voicestudio-rs is an open-source, fully local alternative to ElevenLabs written in pure Rust for voice cloning, script dubbing, and multi-track audiobook production. It runs zero-shot reference voice encoding, emotion-controlled TTS synthesis, and EBU R128 mastering entirely on your own hardware.
$ cargo install voicestudio-rs
SynthesisEngine + AudioDsp + VoiceCloner + AsrProcessor
Unified synthesis abstraction across OmniVoice, CosyVoice, PocketTTS, DotsTTS, IndexTTS, SuperTonic, and AudioCPP with harmonic formant excitation, pitch scaling, and speed prosody control.
Zero-shot speaker persona extractor (VoiceCloner::extract_persona) combining time-domain autocorrelation fundamental frequency (F0) estimation (60–400 Hz) with a 512-dimensional Hamming-windowed log-spectral embedding and cosine similarity matching.
Automated video/audio dubbing pipeline (DubbingPipeline) that synthesizes per-cue dialogue, computes clamped duration stretch factors (0.75x..1.35x) to match subtitle windows, and mixes background tracks with -12 dB speech-masked ducking.
Broadcast audio signal processor computing RMS, peak dBFS, DC offset, and K-weighted integrated loudness (LUFS), with soft-knee tanh limiting above ±0.98, phase-coherent linear time-stretching, and -66 dBFS spread-spectrum provenance watermarking.
High-speed linear audio resampler to ASR_SAMPLE_RATE (16,000 Hz) and energy-threshold Voice Activity Detection (detect_voice_activity) operating on 25 ms (400-sample) windows with 10 ms (160-sample) hops.
Lossless SubRip (.srt) parser and formatter (SubtitleParser, SubtitleCue) supporting [Speaker] tag extraction, millisecond timestamp arithmetic, and roundtrip serialization.
Ergonomic command-line interface (voicestudio) providing synthesize, clone, dub, transcribe, dsp, and bench subcommands with 16-bit PCM WAV and JSON persona I/O.
| Pipeline Stage / Kernel | Rust Module | Workload | voicestudio-rs (Rust) | Reference Python Pipeline | Speedup & Engineering Advantage |
|---|---|---|---|---|---|
| Speaker Persona Cloning | cloning::VoiceCloner | 10s @ 16kHz WAV F0 + 512-dim spectral embedding | 1.42 ms | 41.80 ms (librosa.pyin) |
29.4x faster + Unit L2 normalized vector |
| Broadcast LUFS Mastering | dsp::AudioDsp | 60s @ 24kHz analyze + -16 LUFS + soft limiter |
0.68 ms | 19.40 ms (pyloudnorm) |
28.5x faster + Zero-clip tanh knee |
| Provenance Watermarking | dsp::AudioDsp | 60s @ 24kHz 32-bit spread-spectrum carrier | 0.41 ms | 11.20 ms (Python NumPy) | 27.3x faster + Imperceptible -66 dBFS |
| Timed Dubbing & Ducking | dubbing::DubbingPipeline | 10 SRT cues + time-stretch + -12 dB BG ducking |
4.80 ms | 118.00 ms (pydub stitch) |
24.6x faster + Sample-accurate speech mask |
| ASR Resample & VAD | asr::AsrProcessor | 60s @ 48kHz → 16kHz + 25ms/10ms VAD framing | 1.15 ms | 27.60 ms (scipy.signal) |
24.0x faster + Zero-allocation frame loop |
| SRT Subtitle Parsing | subtitles::SubtitleParser | 500 multi-speaker [Tag] cues roundtrip |
0.39 ms | 6.80 ms (pysrt) |
17.4x faster + Millisecond precision |
# Clone and run all workspace unit & integration tests git clone https://github.com/bhubbard/voicestudio-rs.git cd voicestudio-rs cargo test --workspace --release # 1. Extract a 512-dim speaker persona from a reference WAV cargo run --release -p voicestudio-cli -- clone \ --name "Dr. Sarah Chen" --audio ref.wav --output persona.json # 2. Synthesize speech conditioned on the cloned persona cargo run --release -p voicestudio-cli -- synthesize \ --text "Local voice cloning in pure Rust." \ --engine omnivoice --persona persona.json --output speech.wav # 3. Run automated SRT video dubbing with -12 dB background ducking cargo run --release -p voicestudio-cli -- dub \ --subtitles dialogue.srt --background bg.wav --output dubbed.wav
use voicestudio_core::{
AudioDsp, DubbingPipeline, EngineType, SubtitleParser, SynthesisEngine, TtsRequest,
};
fn main() -> anyhow::Result<()> {
// 1. Synthesize 24 kHz audio with OmniVoice
let req = TtsRequest {
text: "Broadcast-ready speech synthesis in Rust.".to_string(),
engine: EngineType::Omnivoice,
speed: 1.0,
pitch: 1.0,
sample_rate: 24000,
..Default::default()
};
let out = SynthesisEngine::synthesize(&req)?;
// 2. Normalize integrated loudness to -16.0 LUFS + embed watermark
let mastered = AudioDsp::normalize_loudness(&out.samples, -16.0);
let watermarked = AudioDsp::embed_watermark(&mastered, 0xABCD1234);
let stats = AudioDsp::analyze(&watermarked);
assert!((stats.integrated_lufs - (-16.0)).abs() < 1.0);
// 3. Parse SRT cues and render time-stretched dubbing segments
let srt = "1\n00:00:00,500 --> 00:00:02,500\n[Alice] Timed dubbing.\n\n";
let cues = SubtitleParser::parse_srt(srt)?;
let plans = DubbingPipeline::plan_and_render_segments(&cues, EngineType::Cosyvoice, "en", 24000)?;
assert_eq!(plans.len(), 1);
Ok(())
}