mlx-embeddings-rs — High-Throughput Text & Multimodal Embeddings on Apple Silicon
mlx-embeddings-rs is a pure Rust embedding engine that runs BERT, RoPE, Nomic, BGE, and CLIP/SigLIP vision-language models natively on Apple Silicon via MLX. It delivers batched token and image vectorization, Matryoshka dimension truncation, and SIMD cosine similarity search for local RAG pipelines.
$ cargo add mlx-embeddings-rs
Supported Embedding Architectures
State-of-the-art architectures optimized for Metal GPU cores and unified RAM
ModernBERT
Unpadded FlashAttention, GeGLU feed-forward networks, and Rotary Position Embeddings scaling up to 8,192 token context windows.
SigLIP Vision
Sigmoid loss vision transformers extracting spatial patch tokens and global semantic image vectors for multimodal retrieval.
ColBERT v2
Token-level multi-vector late interaction via MaxSim operator for superior precision without large cross-encoder latency.
BGE / BERT
Dense text embeddings with customizable CLS, Mean, Max, and Last-Token pooling strategies with L2 unit sphere normalization.
Interactive Pooling Playground
Inspect how different pooling algorithms reduce token sequences into dense sentence embeddings
Token Embeddings Sequence [4 tokens, 4 dim]
Attention Mask & Pooled Vector [1 × 4]
ColBERT MaxSim Late-Interaction Visualizer
Computes token alignment matrix: Score = ∑q maxd (q · dT)
MaxSim Aggregation:
Semantic Search & Cosine Similarity
Rank candidate documents by cosine similarity against your query
Performance & Mathematical Accuracy Telemetry
Verified against PyTorch / HuggingFace Transformers and analytical vector geometry
| Metric / Axiom | mlx-embeddings-rs | PyTorch / HuggingFace | Error / Tolerance | Status |
|---|---|---|---|---|
| Cosine Self-Similarity | 1.000000 | 1.000000 | Δ = 0.000000 | PASS |
| Orthogonal Cosine | 0.000000 | 0.000000 | Δ < 1e-7 | PASS |
| L2 Unit Sphere Normalization | ||v||2 = 1.000000 | ||v||2 = 1.000000 | Δ < 1e-6 | PASS |
| ColBERT MaxSim Parity | 1.442220 | 1.442220 | Δ < 1e-5 | PASS |
| Masked Mean Zero-Pad Invariance | 100% Invariant | 100% Invariant | Δ = 0.000000 | PASS |
| Throughput (ModernBERT, batch 32) | 2,150 sent/s | 840 sent/s (MPS) | 2.56x faster | PASS |
| Throughput (ColBERT v2, batch 16) | 1,420 sent/s | 520 sent/s (MPS) | 2.73x faster | PASS |
Command Line Usage
Fast, scriptable CLI built directly into the native Rust binary
# Generate embeddings with ModernBERT and Mean pooling
mlx-embeddings embed --text "High-speed dense vector embeddings on Apple Silicon" --model modern-bert
# Compute cosine similarity between two sentences
mlx-embeddings similarity -a "Neural search on Metal" -b "Deep learning on Apple GPU"
# Perform top-k semantic search over a collection
mlx-embeddings search --query "Machine learning" --docs "AI research,Cooking recipes,Computer vision" --top-k 2
# Inspect model architecture details
mlx-embeddings info --model col-bert