Skip to main content

Crate littlebit_models

Crate littlebit_models 

Source
Expand description

LLM architectures, dynamic LittleBit patching, KV cache, and Hugging Face Hub integration.

Re-exports§

pub use config::ModelArchitecture;
pub use config::ModelConfig;
pub use config::ResolvedRopeParams;
pub use config::TokenIdSpec;
pub use error::ModelError;
pub use error::Result;
pub use gemma::GemmaModel;
pub use gemma4::Gemma4Expert;
pub use gemma4::Gemma4Experts;
pub use gemma4::Gemma4MoE;
pub use gemma4::Gemma4Model;
pub use gemma4::Gemma4Router;
pub use gemma4::RouterDispatch;
pub use hub::load_dense_model;
pub use hub::resolve_model_dir;
pub use hub::LittleBitModel;
pub use hub::LoadOptions;
pub use hub::PushOptions;
pub use layer::apply_activation;
pub use layer::causal_mask;
pub use layer::repeat_kv;
pub use layer::Embedding;
pub use layer::KvCache;
pub use layer::LayerNorm;
pub use layer::LinearLayer;
pub use layer::LinearLoadSpec;
pub use layer::NormKind;
pub use layer::RmsNorm;
pub use layer::RmsNormNoScale;
pub use layer::RopeMode;
pub use layer::RotaryEmbedding;
pub use llama::LlamaModel;
pub use model::CausalLm;
pub use model::CausalLmOutput;
pub use model::GenerateConfig;
pub use model::HiddenStatesMode;
pub use model::ParameterStats;
pub use opt::OptModel;
pub use patch::derive_layer_seed;
pub use patch::patch_linear_layer;
pub use patch::PatchPolicy;
pub use patch::PatchSummary;
pub use phi::PhiModel;
pub use phi_attn::PhiAttentionProjection;
pub use streaming::stream_quantize_checkpoint;
pub use streaming::StreamingQuantizeOptions;
pub use streaming::StreamingQuantizeReport;

Modules§

config
Hugging Face config.json parsing and architecture normalization across all supported LittleBit families.
error
Typed errors for littlebit-models.
gemma
Gemma 2 and Gemma 3 decoder architectures.
gemma4
Gemma 4 Dense and Mixture-of-Experts (MoE) decoder architectures.
hub
Hugging Face Hub integration and checkpoint packaging (quantization/hub.py parity).
layer
Differentiable neural network building blocks for quantized and dense LLMs.
llama
Unified decoder implementation for Llama, Llama 2/3, Qwen2/2.5/QwQ, and Qwen3.
model
Unified CausalLm abstraction dispatching across all supported LLM families.
opt
Meta OPT causal language model architecture.
patch
Dynamic LittleBit patching matching upstream quantization/utils/quant_util.py.
phi
Phi-3 and Phi-4 decoder model using PhiAttentionProjection (PhiQKVSplitAttention).
phi_attn
Phi-3 / Phi-4 QKV splitting projection (PhiQKVSplitAttention parity).
streaming
Streaming layer-by-layer quantization for large checkpoints under constrained RAM.