pub enum LinearLayer {
Dense {
weight: Param,
bias: Option<Param>,
},
LittleBit(Box<LittleBitLinear>),
Packed(Box<PackedLittleBitLinear>),
}Expand description
Unified linear projection supporting dense weights, differentiable LittleBit QAT layers, and compact 1-bit packed inference layers.
Variants§
Dense
Standard dense linear transformation x @ W^T + b.
Fields
weight: ParamWeight tensor (out_features, in_features).
LittleBit(Box<LittleBitLinear>)
Differentiable or unpacked LittleBit factorized linear layer.
Packed(Box<PackedLittleBitLinear>)
Memory-compact 1-bit packed LittleBit inference layer.
Implementations§
Source§impl LinearLayer
impl LinearLayer
Sourcepub fn new_random(
shape: LayerShape,
dtype: DType,
device: &Device,
rng: &mut ChaCha8Rng,
) -> Result<Self>
pub fn new_random( shape: LayerShape, dtype: DType, device: &Device, rng: &mut ChaCha8Rng, ) -> Result<Self>
Construct a random dense linear layer with normal initialization (std = 0.02).
§Errors
Returns an error if tensor creation fails.
Sourcepub fn from_tensors(
prefix: &str,
tensors: &HashMap<String, Tensor>,
shape: LayerShape,
spec: LinearLoadSpec<'_>,
) -> Result<Self>
pub fn from_tensors( prefix: &str, tensors: &HashMap<String, Tensor>, shape: LayerShape, spec: LinearLoadSpec<'_>, ) -> Result<Self>
Load a linear layer from a state dictionary, detecting packed LittleBit, unpacked LittleBit, or dense weights automatically.
§Errors
Returns ModelError::MissingWeight if expected checkpoint keys are missing.
Sourcepub fn in_features(&self) -> usize
pub fn in_features(&self) -> usize
Input feature dimension.
Sourcepub fn out_features(&self) -> usize
pub fn out_features(&self) -> usize
Output feature dimension.
Sourcepub fn bias_mode(&self) -> BiasMode
pub fn bias_mode(&self) -> BiasMode
Return whether this layer currently carries a bias parameter.
Sourcepub fn forward(&self, x: &Tensor) -> Result<Tensor>
pub fn forward(&self, x: &Tensor) -> Result<Tensor>
Execute linear forward pass on [..., in_features].
§Errors
Returns an error if shapes mismatch or tensor math fails.
Sourcepub fn convert_to_littlebit(
&mut self,
config: &LittleBitConfig,
ratio_factor: f64,
train_mode: TrainMode,
seed: u64,
) -> Result<()>
pub fn convert_to_littlebit( &mut self, config: &LittleBitConfig, ratio_factor: f64, train_mode: TrainMode, seed: u64, ) -> Result<()>
Convert a dense layer into LinearLayer::LittleBit via SVD (and optional Joint-ITQ).
§Errors
Returns an error if called on an already-packed layer or if factorization fails.
Sourcepub fn convert_to_packed(&mut self) -> Result<()>
pub fn convert_to_packed(&mut self) -> Result<()>
Convert a LinearLayer::LittleBit layer to compact 1-bit LinearLayer::Packed.
§Errors
Returns an error if packing fails.
Sourcepub fn pack_state_dict(&self, prefix: &str) -> Result<HashMap<String, Tensor>>
pub fn pack_state_dict(&self, prefix: &str) -> Result<HashMap<String, Tensor>>
Export layer weights in packed LittleBit or dense format under prefix.
§Errors
Returns an error if serialization conversion fails.
Sourcepub fn trainable_vars(&self) -> Vec<Var>
pub fn trainable_vars(&self) -> Vec<Var>
Collect trainable [Var] parameters inside this layer.
Sourcepub fn make_trainable(&mut self) -> Result<()>
pub fn make_trainable(&mut self) -> Result<()>
Convert all dense weights into trainable [Var]s (no-op for Packed).
§Errors
Returns an error if converting tensors to Var fails.
Sourcepub fn param_count(&self) -> usize
pub fn param_count(&self) -> usize
Total scalar parameter elements stored in this layer.
Trait Implementations§
Source§impl Clone for LinearLayer
impl Clone for LinearLayer
Auto Trait Implementations§
impl !RefUnwindSafe for LinearLayer
impl !UnwindSafe for LinearLayer
impl Freeze for LinearLayer
impl Send for LinearLayer
impl Sync for LinearLayer
impl Unpin for LinearLayer
impl UnsafeUnpin for LinearLayer
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> ErasedDestructor for Twhere
T: 'static,
§impl<T> Identity for Twhere
T: ?Sized,
impl<T> Identity for Twhere
T: ?Sized,
impl<T, U> Imply<T> for U
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more