pub struct Gemma4MoE {
pub post_feedforward_layernorm_1: RmsNorm,
pub pre_feedforward_layernorm_2: RmsNorm,
pub router: Gemma4Router,
pub experts: Gemma4Experts,
pub post_feedforward_layernorm_2: RmsNorm,
}Expand description
Complete Mixture-of-Experts sub-block inside a Gemma4Model decoder layer.
Fields§
§post_feedforward_layernorm_1: RmsNormPost-normalization on the parallel dense shared MLP output.
pre_feedforward_layernorm_2: RmsNormPre-normalization before routed expert execution.
router: Gemma4RouterUnquantized top-k router.
experts: Gemma4ExpertsSparse expert ensemble (Dense, LittleBit, or PackedLittleBitLinear).
post_feedforward_layernorm_2: RmsNormPost-normalization on the summed sparse expert output.
Trait Implementations§
Auto Trait Implementations§
impl !RefUnwindSafe for Gemma4MoE
impl !UnwindSafe for Gemma4MoE
impl Freeze for Gemma4MoE
impl Send for Gemma4MoE
impl Sync for Gemma4MoE
impl Unpin for Gemma4MoE
impl UnsafeUnpin for Gemma4MoE
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> ErasedDestructor for Twhere
T: 'static,
§impl<T> Identity for Twhere
T: ?Sized,
impl<T> Identity for Twhere
T: ?Sized,
impl<T, U> Imply<T> for U
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more