Expand description
LLM architectures, dynamic LittleBit patching, KV cache, and Hugging Face Hub integration.
Re-exports§
pub use config::ModelArchitecture;pub use config::ModelConfig;pub use config::ResolvedRopeParams;pub use config::TokenIdSpec;pub use error::ModelError;pub use error::Result;pub use gemma::GemmaModel;pub use gemma4::Gemma4Expert;pub use gemma4::Gemma4Experts;pub use gemma4::Gemma4MoE;pub use gemma4::Gemma4Model;pub use gemma4::Gemma4Router;pub use gemma4::RouterDispatch;pub use hub::load_dense_model;pub use hub::resolve_model_dir;pub use hub::LittleBitModel;pub use hub::LoadOptions;pub use hub::PushOptions;pub use layer::apply_activation;pub use layer::causal_mask;pub use layer::repeat_kv;pub use layer::Embedding;pub use layer::KvCache;pub use layer::LayerNorm;pub use layer::LinearLayer;pub use layer::LinearLoadSpec;pub use layer::NormKind;pub use layer::RmsNorm;pub use layer::RmsNormNoScale;pub use layer::RopeMode;pub use layer::RotaryEmbedding;pub use llama::LlamaModel;pub use model::CausalLm;pub use model::CausalLmOutput;pub use model::GenerateConfig;pub use model::HiddenStatesMode;pub use model::ParameterStats;pub use opt::OptModel;pub use patch::derive_layer_seed;pub use patch::patch_linear_layer;pub use patch::PatchPolicy;pub use patch::PatchSummary;pub use phi::PhiModel;pub use phi_attn::PhiAttentionProjection;pub use streaming::stream_quantize_checkpoint;pub use streaming::StreamingQuantizeOptions;pub use streaming::StreamingQuantizeReport;
Modules§
- config
- Hugging Face
config.jsonparsing and architecture normalization across all supported LittleBit families. - error
- Typed errors for
littlebit-models. - gemma
- Gemma 2 and Gemma 3 decoder architectures.
- gemma4
- Gemma 4 Dense and Mixture-of-Experts (MoE) decoder architectures.
- hub
- Hugging Face Hub integration and checkpoint packaging (
quantization/hub.pyparity). - layer
- Differentiable neural network building blocks for quantized and dense LLMs.
- llama
- Unified decoder implementation for Llama, Llama 2/3, Qwen2/2.5/QwQ, and Qwen3.
- model
- Unified
CausalLmabstraction dispatching across all supported LLM families. - opt
- Meta OPT causal language model architecture.
- patch
- Dynamic LittleBit patching matching upstream
quantization/utils/quant_util.py. - phi
- Phi-3 and Phi-4 decoder model using
PhiAttentionProjection(PhiQKVSplitAttention). - phi_
attn - Phi-3 / Phi-4 QKV splitting projection (
PhiQKVSplitAttentionparity). - streaming
- Streaming layer-by-layer quantization for large checkpoints under constrained RAM.