pub struct LlamaModel { /* private fields */ }Expand description
Causal language model supporting Llama / Llama 2/3, Qwen2 / Qwen2.5 / QwQ, and Qwen3.
Implementations§
Source§impl LlamaModel
impl LlamaModel
Sourcepub fn from_tensors(
config: ModelConfig,
tensors: &HashMap<String, Tensor>,
spec: LinearLoadSpec<'_>,
) -> Result<Self>
pub fn from_tensors( config: ModelConfig, tensors: &HashMap<String, Tensor>, spec: LinearLoadSpec<'_>, ) -> Result<Self>
Construct a Llama/Qwen2/Qwen3 model from a loaded state dictionary.
§Errors
Returns crate::ModelError::MissingWeight or tensor shape errors if checkpoint tensors are missing.
Sourcepub fn new_random(
config: ModelConfig,
dtype: DType,
device: &Device,
rng: &mut ChaCha8Rng,
) -> Result<Self>
pub fn new_random( config: ModelConfig, dtype: DType, device: &Device, rng: &mut ChaCha8Rng, ) -> Result<Self>
Initialize a random Llama/Qwen2/Qwen3 model (for testing and synthetic benchmarks).
§Errors
Returns an error if tensor creation fails.
Sourcepub fn forward(
&self,
input_ids: &Tensor,
cache: Option<&mut KvCache>,
hidden_mode: HiddenStatesMode,
) -> Result<CausalLmOutput>
pub fn forward( &self, input_ids: &Tensor, cache: Option<&mut KvCache>, hidden_mode: HiddenStatesMode, ) -> Result<CausalLmOutput>
Forward pass returning logits (batch, seq_len, vocab_size) and optional hidden states.
§Errors
Returns an error if tensor shapes are invalid or layer forward passes fail.
Trait Implementations§
Source§impl Clone for LlamaModel
impl Clone for LlamaModel
Auto Trait Implementations§
impl !RefUnwindSafe for LlamaModel
impl !UnwindSafe for LlamaModel
impl Freeze for LlamaModel
impl Send for LlamaModel
impl Sync for LlamaModel
impl Unpin for LlamaModel
impl UnsafeUnpin for LlamaModel
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> ErasedDestructor for Twhere
T: 'static,
§impl<T> Identity for Twhere
T: ?Sized,
impl<T> Identity for Twhere
T: ?Sized,
impl<T, U> Imply<T> for U
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more