The Free AI Already on Your Mac.
Native Rust Speed.
macOS Sequoia ships with on-device Apple Intelligence FoundationModels. apfel-rs gives you an ultra-fast CLI and OpenAI-compatible HTTP daemon with one brew install. No model downloads, no API keys, zero network token costs.
Rust fork of apfel by Arthur-Ficial · Requires: Apple Silicon · macOS Sequoia 26+ · Apple Intelligence enabled
Default Mode vs. MLX Engine: How They Compare
apfel-rs offers two native on-device engines. Choose between zero-download Apple Intelligence for instant terminal tasks, or the MLX Engine for expanded 32k–128k context and custom open weights.
Zero-Download Apple Intelligence
Runs directly against Apple Silicon's Neural Engine (ANE) with P-core priority scheduling. Zero weights to download, zero setup, and zero telemetry. Instant startup from your terminal.
- 🍏 Weight Download: 0 MB (pre-installed with macOS Sequoia+)
- ⚡ Context Window: 4,096 – 8,192 tokens
- 🎯 Best For: Daily shell pipes, quick queries, git hooks, instant offline server
apfel "Explain Rust zero-cost abstractions"
apfel --serve
Expanded Context & Custom Models
Accelerated by Apple's native MLX framework on Metal GPU and Unified Memory. Run specialized coding models like Qwen 2.5 Coder, Llama 3, or Mistral with massive context budgets.
- 🚀 Weight Download: Any HuggingFace or local GGUF/MLX repo
- ⚡ Context Window: 32,768 – 131,072 tokens
- 🎯 Best For: Zed/Cursor coding agents, large document synthesis, fine-tuned models
apfel --engine mlx -m mlx-community/Qwen2.5-Coder-7B-Instruct-4bit "Write an async worker pool"
apfel --engine mlx -m mlx-community/Llama-3.2-3B-Instruct-4bit --serve
| Benchmark Metric / Feature | Original Swift (apfel) |
Rust Default (FoundationModels) |
Rust MLX Engine (--engine mlx) |
Advantage vs. Swift |
|---|---|---|---|---|
| Release Binary Footprint | 20.75 MB | 2.30 MB | 2.30 MB | 88.9% smaller distribution |
| Server Idle Memory (RSS) | 22.2 MB | 10.1 MB | 10.1 MB | 54.6% less resident RAM |
| Active Server Memory (100 workers) | 24.0 MB | 15.5 MB | 11.3 MB | 35% – 53% lower RAM footprint |
| Server Throughput (RPS) | 12,156 req/s | 15,240 req/s | 50,779 req/s | Pre-serialized zero-copy route cache |
| Tail Latency (p50 / p95) | 0.25 ms / 5.71 ms | 0.15 ms / 1.12 ms | 0.13 ms / 0.64 ms | Sub-millisecond p95 response |
| Token Counting & Preflight | 2,949 ms (spikes to 22.7s) | 0.17 ms cached (5,782 req/s) | 6.35 ms | 152.5× speedup on preflight |
| Token Counting Max Jitter | 22,773 ms (ARC stalls) | 0.24 ms (deterministic) | 9.37 ms | Zero reference-counting lockups |
| Supported Context Window | 4,096 tokens | 4,096 – 8,192 tokens (disclosed) | 32,768 – 131,072 tokens | 8× to 32× larger context budget |
Built for Performance, Privacy & Simplicity
Everything you love about Apple Intelligence, wrapped in a UNIX-first Rust binary that fits in your pocket.
100% On-Device & Private
Executes completely against macOS FoundationModels on Apple Silicon. Zero network calls, zero telemetry, zero tokens leaking outside your machine.
OpenAI-Compatible Daemon
Drop-in replacement for OpenAI endpoints. Provides /v1/chat/completions with streaming SSE, /v1/responses, /v1/models, and /health.
Model Context Protocol (MCP)
Integrates directly with external MCP servers over stdio. Automatically negotiates tools, invokes scripts, and runs the self-healing re-prompting loop.
5 Context Trimming Strategies
Fine-tune conversation budgets with sliding-window, newest-first, oldest-first, summarize, or strict overflow protection.
Hardened Localhost Security
DNS rebinding and CSRF prevention, strict Origin header verification, constant-time Bearer token checks, and sanitized subprocess environments.
macOS Login Daemon
Manage seamlessly via Homebrew. Run brew services start apfel-rs to launch on login on 127.0.0.1:32185 with automated log rotation.
JSON Schema Self-Repair
Pass --schema <path|json> for structured outputs with automatic RFC 6901 $defs reference resolution and self-healing repair retries.
Full Ollama API Drop-In
Native Ollama protocol compatibility with streaming NDJSON for /api/chat and /api/generate, plus /api/version, /api/show, and /api/ps.
High-Performance Batch Pipelines
Process high-throughput JSONL streams over standard UNIX pipelines (jq | apfel --batch | jq) with fresh session isolation and zero string reallocation.
Use Your Favorite Tools & SDKs
Point any OpenAI-compatible app, Cursor, Open WebUI, Python script, or TypeScript SDK directly to http://127.0.0.1:8080/v1.
# POST /v1/chat/completions with SSE Streaming
curl -N -X POST http://127.0.0.1:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "apple-foundationmodel",
"stream": true,
"temperature": 0.7,
"messages": [
{"role": "system", "content": "You are a concise engineering assistant."},
{"role": "user", "content": "How does Rust guarantee thread safety?"}
]
}'Command-Line Reference
Powerful flags designed to pipe with cat, jq, and standard shell utilities.
| Command / Flag | Description | Example Usage |
|---|---|---|
apfel <PROMPT> |
Standard single-shot completion | apfel "Explain DNS in one sentence" |
--stream |
Real-time SSE token stream to stdout | apfel --stream "Draft a quick release note" |
-f, --file <PATH> |
Attach text, pdf, or image files into context | apfel -f src/lib.rs "Summarize the exports" |
--code |
Extract pure code block, stripping fences and prose (exits 7 if no code) | apfel --code "Write a Python script to parse JSON" |
--require-complete |
Require complete response; exit code 8 if truncated by budget or length | apfel --require-complete --max-tokens 256 "Generate JSON" |
--stop <SEQ> |
Stop sequence to truncate generation at custom delimiter (repeatable) | apfel --stop "\nObservation:" "Agent prompt" |
--chat |
Interactive REPL with slash commands (/info, /clear) |
apfel --chat |
--serve |
Run the OpenAI-compatible HTTP server | apfel --serve --port 8080 |
--mcp-server <CMD> |
Attach external stdio MCP server for tool calls | apfel --mcp-server "./calc_mcp.py" "What is 42 * 10?" |
--count-tokens |
Accurately measure tokens before executing | apfel --count-tokens -f long_doc.txt |
--benchmark |
Execute internal throughput and latency stress test | apfel --benchmark |
--batch |
Process streaming JSONL lines from stdin with fresh session isolation | cat data.jsonl | apfel --batch | jq |
--schema <SCHEMA> |
Enforce JSON Schema on output with self-repair reprompts (repeat up to 2x) | apfel --schema schema.json "Generate profile" |
--mcp-config <PATH> |
Load multi-server MCP config file (stdio tools for CLI & chat) | apfel --mcp-config mcp.json "Query database" |
--image <PATH> |
Analyze image using Apple Vision OCR and visual feature labeling | apfel --image doc.png "Extract text & diagram" |
--install-completions |
Detect shell ($SHELL) and install completions into user directories | apfel --install-completions |
--adapter <PATH> |
Load fine-tuned LoRA model adapter (exposed in /v1/models) | apfel --adapter ./adapter.bin "Evaluate task" |
System Architecture
How apfel-rs bridges the macOS C-ABI with zero serialization overhead and predictable memory bounds.
CLI & Axum HTTP
Piped stdin, CLI args, or /v1/chat/completions REST / SSE endpoints.
Session Engine
Sliding window context trimmer, token budgeter, MCP tool loop, and security guards.
Swift C-ABI
Zero-copy C-ABI bridge providing memory safety between Rust FFI and Swift actors.
Dual Hardware Engines
FoundationModels on Neural Engine (ANE) or open-weights via native MLX on Metal GPU.
Model Context Protocol (MCP)
Equip Apple Intelligence with live external tools—from calculators to web search and database engines.
Automatic Tool Discovery
Launches any stdio MCP server (Python, Node.js, Go, or native executable) and negotiates JSON-RPC schemas automatically.
Self-Healing Reprompt Loop
When the model triggers a tool call, apfel-rs executes the tool, feeds the output back into context, and reprompts for the final answer.
Process Isolation
Subprocesses run with scrubbed environment variables to prevent API token or credential leaks to external tools.
Exit Codes
Standardized POSIX exit codes for predictable shell script automation.
| Code | Constant | Meaning |
|---|---|---|
0 |
SUCCESS |
Successful completion |
1 |
RUNTIME_ERROR |
Internal runtime failure or Swift FFI exception |
2 |
USAGE_ERROR |
Invalid arguments or malformed JSON input |
3 |
GUARDRAIL |
System safety guardrail triggered |
4 |
CONTEXT_OVERFLOW |
Prompt exceeds context budget with strict strategy |
5 |
MODEL_UNAVAILABLE |
Apple Intelligence disabled or model not downloaded |
6 |
RATE_LIMITED |
System capacity exhausted |
7 |
NO_CODE |
--code requested but no code block returned |
Everything You Need to Know
Common questions regarding architecture, privacy, compatibility, and setup.
What is apfel-rs and how does it use Apple Intelligence?
apfel-rs is a complete, native Rust implementation and high-performance fork of the original Swift apfel. It bridges directly into Apple's macOS FoundationModels framework on Apple Silicon to execute on-device LLM inference without any cloud latency or telemetry.
Does apfel-rs require an internet connection or cloud API keys?
apfel-rs is 100% on-device. All inference runs locally through Apple Neural Engine / GPU cores. Zero data ever leaves your Mac, and no OpenAI or third-party subscription keys are needed.
How does apfel-rs achieve a 3.76x speedup over the Swift version?
axum and tokio runtime, optimizing token counting with zero-allocation buffers, eliminating Swift runtime GC/pause jitter, and interfacing with Swift via a lean, zero-overhead C-ABI bridge.
Can I use apfel-rs as a drop-in local OpenAI backend in Cursor or Open WebUI?
apfel --serve --port 8080 (or run brew services start apfel-rs). Configure your client's OpenAI Base URL to http://127.0.0.1:8080/v1. Any arbitrary string can be passed for the API key.
What hardware and macOS versions are supported?
How does Model Context Protocol (MCP) tool execution work?
--mcp-server <COMMAND> to attach an external MCP tool server over standard input/output. apfel-rs performs JSON-RPC capability negotiation, presents tool definitions to the FoundationModel, runs tool calls in an isolated sanitized environment, and re-prompts the model with execution outputs until completion.
How do I run apfel-rs as an automatic login daemon?
brew services start apfel-rs. Homebrew installs a launchd plist that starts the daemon automatically upon macOS user login on 127.0.0.1:32185.