pub struct Embedding {
pub weight: Param,
pub scale: Option<f64>,
}Expand description
Token or positional embedding table (vocab_size, embed_dim).
Fields§
§weight: ParamWeight table (num_embeddings, embed_dim).
scale: Option<f64>Optional multiplicative embedding scale (e.g. sqrt(hidden_size) for Gemma).
Implementations§
Source§impl Embedding
impl Embedding
Sourcepub fn from_tensors(
prefix: &str,
tensors: &HashMap<String, Tensor>,
scale: Option<f64>,
dtype: DType,
device: &Device,
) -> Result<Self>
pub fn from_tensors( prefix: &str, tensors: &HashMap<String, Tensor>, scale: Option<f64>, dtype: DType, device: &Device, ) -> Result<Self>
Sourcepub fn new_random(
num_embeddings: usize,
embed_dim: usize,
scale: Option<f64>,
dtype: DType,
device: &Device,
rng: &mut ChaCha8Rng,
) -> Result<Self>
pub fn new_random( num_embeddings: usize, embed_dim: usize, scale: Option<f64>, dtype: DType, device: &Device, rng: &mut ChaCha8Rng, ) -> Result<Self>
Random initialization (num_embeddings, embed_dim) with std = 0.02.
§Errors
Returns an error if tensor creation fails.
Sourcepub fn forward(&self, ids: &Tensor) -> Result<Tensor>
pub fn forward(&self, ids: &Tensor) -> Result<Tensor>
Embed token indices [..., seq_len] into [..., seq_len, embed_dim].
§Errors
Returns an error if indexing or reshaping fails.
Sourcepub fn forward_lm_head(&self, hidden: &Tensor) -> Result<Tensor>
pub fn forward_lm_head(&self, hidden: &Tensor) -> Result<Tensor>
Project hidden states [..., embed_dim] through tied embedding weights -> [..., num_embeddings].
§Errors
Returns an error if matrix multiplication fails.
Sourcepub fn trainable_vars(&self) -> Vec<Var>
pub fn trainable_vars(&self) -> Vec<Var>
Collect trainable Var if unfrozen.
Sourcepub fn make_trainable(&mut self) -> Result<()>
pub fn make_trainable(&mut self) -> Result<()>
Sourcepub fn pack_state_dict(&self, prefix: &str, out: &mut HashMap<String, Tensor>)
pub fn pack_state_dict(&self, prefix: &str, out: &mut HashMap<String, Tensor>)
Export {prefix}.weight.
Trait Implementations§
Auto Trait Implementations§
impl !RefUnwindSafe for Embedding
impl !UnwindSafe for Embedding
impl Freeze for Embedding
impl Send for Embedding
impl Sync for Embedding
impl Unpin for Embedding
impl UnsafeUnpin for Embedding
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> ErasedDestructor for Twhere
T: 'static,
§impl<T> Identity for Twhere
T: ?Sized,
impl<T> Identity for Twhere
T: ?Sized,
impl<T, U> Imply<T> for U
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more