mlx-video-rs — Apple Silicon Diffusion Transformer (DiT) Video Engine in Rust
mlx-video-rs is a pure Rust port of mlx-video for generating high-definition video clips from text and image prompts on Apple Silicon Macs. It accelerates Wan2.1, LTX-Video, and CogVideoX diffusion transformers using Metal unified memory, 4-bit quantization, and tiled 3D-VAE decoding.
$ cargo install mlx-video-rs
Architecture & Performance
Built from the ground up for Metal
Combining the safety and speed of Rust with Apple MLX unified memory architecture.
Apple Silicon Unified Memory
Zero-copy checkpoint streaming from SafeTensors directly to Metal GPU unified RAM without duplicate CPU memory allocations.
3-Way Factorized 3D RoPE
Rotary Position Embeddings decomposed across temporal (F), height (H), and width (W) dimensions for fast spatial-temporal attention.
Flow Matching Schedulers
High-precision Euler and 2nd-order DPM++ 2M continuous vector-field integration with dynamic sample shifting.
Causal 3D VAE & Tiling
CausalConv3d decoding preserving temporal continuity, combined with spatial/temporal tiling to eliminate VRAM OOM errors on long generations.
LoRA Dynamic Weight Fusion
Automatic rank/alpha extraction and instant on-the-fly matrix injection (ΔW = s • α/r • B × A) with multi-LoRA stacking support.
Native Image-to-Video
Lanczos downsampling and 36-channel latent concatenation (16 noisy + 16 image + 4 mask) for Wan2.2 I2V animation.
Supported Checkpoints
Model Matrix & Presets
Comprehensive support across both Wan and LTX diffusion video transformer families.
| Model Name | Category | Parameters | DiT Dim | Layers | Heads | Default Steps |
|---|---|---|---|---|---|---|
| Wan2.1 T2V (1.3B) | Text-to-Video | 1.3 Billion | 1536 | 30 | 12 | 40 steps |
| Wan2.1 T2V (14B) | Text-to-Video | 14.0 Billion | 5120 | 40 | 40 | 40 steps |
| Wan2.2 T2V (14B) | Text-to-Video (Dual) | 14.0 Billion | 5120 | 40 | 40 | 40 steps |
| Wan2.2 I2V (14B) | Image-to-Video | 14.0 Billion | 5120 (in=36) | 40 | 40 | 40 steps |
| Wan2.2 TI2V (5B) | Text+Image-to-Video | 5.0 Billion | 3072 | 32 | 24 | 40 steps |
| LTX-2 Distilled | Multi-Stage Flow | Variable | 2048 | 28 | 32 | 8 steps |
| LTX-2 Dev | Multi-Stage Flow | Variable | 2048 | 28 | 32 | 40 steps |
Developer Tools
Interactive CLI Command Generator
Configure generation parameters and generate ready-to-run mlx-video terminal commands.
System Sizing
Hardware Recommendations
Memory allocations scaled for Apple Silicon unified RAM.
16GB – 24GB RAM
Recommended: Wan2.1-1.3B & LTX-2 Distilled at 768x512 with 49–65 frames.
36GB – 48GB RAM
Recommended: Wan2.2-TI2V-5B and Wan2.1-14B (4-bit quantized / tiled) at 832x480 with 81 frames.
64GB – 128GB+ RAM
Recommended: Full uncompressed 14B Dual models, Wan2.2 I2V, and 720p HD long-sequence video generation.