Skip to main content

stream_quantize_checkpoint

Function stream_quantize_checkpoint 

Source
pub fn stream_quantize_checkpoint(
    src_dir: &Path,
    dst_dir: &Path,
    lb_cfg: &LittleBitConfig,
    opts: &StreamingQuantizeOptions,
) -> Result<StreamingQuantizeReport>
Expand description

Quantize a dense local checkpoint in streaming layer-by-layer fashion without loading the full model weights into RAM.

Handles single-file and multi-shard input checkpoints across all supported architectures, including Phi-3/4 qkv_proj splitting, OPT project_in/project_out, and Gemma 4 fused 3D MoE expert tensors (gate_up_proj / down_proj, loaded one expert slice at a time).

ยงErrors

Returns an error if configs are invalid, required weights are malformed, or disk I/O fails.