pub struct Gemma4Model { /* private fields */ }Expand description
Causal language model supporting Gemma 4 Dense and Mixture-of-Experts checkpoints.
Implementations§
Source§impl Gemma4Model
impl Gemma4Model
Sourcepub fn from_tensors(
config: ModelConfig,
tensors: &HashMap<String, Tensor>,
spec: LinearLoadSpec<'_>,
) -> Result<Self>
pub fn from_tensors( config: ModelConfig, tensors: &HashMap<String, Tensor>, spec: LinearLoadSpec<'_>, ) -> Result<Self>
Load Gemma 4 Dense or MoE weights from a state dictionary.
Automatically supports both standalone (model.*) and multimodal (model.language_model.* /
language_model.model.*) weight prefixes, as well as fused 3D MoE experts and LittleBit packed experts.
§Errors
Returns ModelError::MissingWeight or ModelError::UnsupportedArchitecture on incompatible configs.
Sourcepub fn new_random(
config: ModelConfig,
dtype: DType,
device: &Device,
rng: &mut ChaCha8Rng,
) -> Result<Self>
pub fn new_random( config: ModelConfig, dtype: DType, device: &Device, rng: &mut ChaCha8Rng, ) -> Result<Self>
Initialize a random Gemma 4 Dense or MoE model for testing and benchmarking.
§Errors
Returns an error if tensor allocation or configuration validation fails.
Sourcepub fn layer_moe(&self, layer_idx: usize) -> Option<&Gemma4MoE>
pub fn layer_moe(&self, layer_idx: usize) -> Option<&Gemma4MoE>
Borrow a decoder layer’s MoE block (useful for testing router / expert quantization states).
Sourcepub fn layer_moe_mut(&mut self, layer_idx: usize) -> Option<&mut Gemma4MoE>
pub fn layer_moe_mut(&mut self, layer_idx: usize) -> Option<&mut Gemma4MoE>
Mutably borrow a decoder layer’s MoE block (useful in unit tests).
Sourcepub fn forward(
&self,
input_ids: &Tensor,
cache: Option<&mut KvCache>,
hidden_mode: HiddenStatesMode,
) -> Result<CausalLmOutput>
pub fn forward( &self, input_ids: &Tensor, cache: Option<&mut KvCache>, hidden_mode: HiddenStatesMode, ) -> Result<CausalLmOutput>
Execute causal forward pass over input_ids (batch, seq_len).
§Errors
Returns an error if layer computation fails.
Trait Implementations§
Source§impl Clone for Gemma4Model
impl Clone for Gemma4Model
Auto Trait Implementations§
impl !RefUnwindSafe for Gemma4Model
impl !UnwindSafe for Gemma4Model
impl Freeze for Gemma4Model
impl Send for Gemma4Model
impl Sync for Gemma4Model
impl Unpin for Gemma4Model
impl UnsafeUnpin for Gemma4Model
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> ErasedDestructor for Twhere
T: 'static,
§impl<T> Identity for Twhere
T: ?Sized,
impl<T> Identity for Twhere
T: ?Sized,
impl<T, U> Imply<T> for U
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more