pub enum CausalLm {
LlamaFamily(LlamaModel),
Phi(PhiModel),
Gemma(GemmaModel),
Gemma4(Box<Gemma4Model>),
Opt(Box<OptModel>),
}Expand description
Unified causal language model supporting all architectures in SamsungLabs/LittleBit plus Gemma 4.
Variants§
LlamaFamily(LlamaModel)
Llama, Llama 2/3, Qwen2/2.5/QwQ, and Qwen3.
Phi(PhiModel)
Phi-3 and Phi-4 (including PhiQKVSplitAttention).
Gemma(GemmaModel)
Gemma 2 and Gemma 3.
Gemma4(Box<Gemma4Model>)
Gemma 4 Dense and Mixture-of-Experts.
Opt(Box<OptModel>)
Meta OPT.
Implementations§
Source§impl CausalLm
impl CausalLm
Sourcepub fn from_tensors(
config: ModelConfig,
tensors: &HashMap<String, Tensor>,
littlebit: Option<&LittleBitConfig>,
weight_state: WeightState,
inference_mode: InferenceMode,
dtype: DType,
device: &Device,
) -> Result<Self>
pub fn from_tensors( config: ModelConfig, tensors: &HashMap<String, Tensor>, littlebit: Option<&LittleBitConfig>, weight_state: WeightState, inference_mode: InferenceMode, dtype: DType, device: &Device, ) -> Result<Self>
Construct a model from a parsed ModelConfig and loaded checkpoint tensors.
§Errors
Returns an error if the architecture is unsupported or required weights are missing.
Sourcepub fn new_random(
config: ModelConfig,
dtype: DType,
device: &Device,
seed: u64,
) -> Result<Self>
pub fn new_random( config: ModelConfig, dtype: DType, device: &Device, seed: u64, ) -> Result<Self>
Initialize a compact or full-sized random model for testing and benchmarking.
§Errors
Returns an error if architecture detection or tensor allocation fails.
Sourcepub fn forward(
&self,
input_ids: &Tensor,
cache: Option<&mut KvCache>,
hidden_mode: HiddenStatesMode,
) -> Result<CausalLmOutput>
pub fn forward( &self, input_ids: &Tensor, cache: Option<&mut KvCache>, hidden_mode: HiddenStatesMode, ) -> Result<CausalLmOutput>
Execute causal forward pass over input_ids (batch, seq_len).
§Errors
Returns an error if layer computation fails.
Sourcepub fn apply_littlebit_patch(
&mut self,
lb_cfg: &LittleBitConfig,
train_mode: TrainMode,
base_seed: u64,
) -> Result<PatchSummary>
pub fn apply_littlebit_patch( &mut self, lb_cfg: &LittleBitConfig, train_mode: TrainMode, base_seed: u64, ) -> Result<PatchSummary>
Apply LittleBit / LittleBit-2 quantization patch across all linear projections
(excluding lm_head, router, splitting qkv_proj on Phi, and scaling k_proj/v_proj by kv_factor).
§Errors
Returns an error if config validation or SVD/ITQ decomposition fails.
Sourcepub fn convert_to_packed(&mut self) -> Result<()>
pub fn convert_to_packed(&mut self) -> Result<()>
Convert all LittleBitLinear layers into memory-compact 1-bit PackedLittleBitLinear modules.
§Errors
Returns an error if packing fails.
Sourcepub fn prepare_for_training(&mut self, exclude: &[&str]) -> Result<()>
pub fn prepare_for_training(&mut self, exclude: &[&str]) -> Result<()>
Prepare model parameters for QAT training, making all non-excluded parameters trainable
and freezing parameters whose module path contains any pattern in exclude.
Matches upstream utils/utils.py::prepare_model_for_training.
§Errors
Returns an error if wrapping parameters in [Var] fails.
Sourcepub fn freeze_matching(&mut self, exclude: &[&str])
pub fn freeze_matching(&mut self, exclude: &[&str])
Freeze all parameters matching any substring in exclude.
Sourcepub fn trainable_vars(&self) -> Vec<Var>
pub fn trainable_vars(&self) -> Vec<Var>
Collect all active trainable [Var] parameters across the model.
Sourcepub fn parameter_stats(&self) -> ParameterStats
pub fn parameter_stats(&self) -> ParameterStats
Compute trainable and total scalar parameter counts.
Sourcepub fn pack_state_dict(&self) -> Result<HashMap<String, Tensor>>
pub fn pack_state_dict(&self) -> Result<HashMap<String, Tensor>>
Export full model state dictionary with LittleBit layers binarized & 1-bit packed.
§Errors
Returns an error if packing any layer fails.
Sourcepub fn config(&self) -> &ModelConfig
pub fn config(&self) -> &ModelConfig
Borrow underlying Hugging Face ModelConfig.
Trait Implementations§
Auto Trait Implementations§
impl !RefUnwindSafe for CausalLm
impl !UnwindSafe for CausalLm
impl Freeze for CausalLm
impl Send for CausalLm
impl Sync for CausalLm
impl Unpin for CausalLm
impl UnsafeUnpin for CausalLm
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> ErasedDestructor for Twhere
T: 'static,
§impl<T> Identity for Twhere
T: ?Sized,
impl<T> Identity for Twhere
T: ?Sized,
impl<T, U> Imply<T> for U
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more