pub struct OptModel { /* private fields */ }Expand description
Meta OPT causal language model (model_type = "opt").
Implementations§
Source§impl OptModel
impl OptModel
Sourcepub fn from_tensors(
config: ModelConfig,
tensors: &HashMap<String, Tensor>,
spec: LinearLoadSpec<'_>,
) -> Result<Self>
pub fn from_tensors( config: ModelConfig, tensors: &HashMap<String, Tensor>, spec: LinearLoadSpec<'_>, ) -> Result<Self>
Load OPT weights from a Hugging Face or LittleBit state dictionary.
§Errors
Returns crate::ModelError::MissingWeight if expected checkpoint keys are missing.
Sourcepub fn new_random(
config: ModelConfig,
dtype: DType,
device: &Device,
rng: &mut ChaCha8Rng,
) -> Result<Self>
pub fn new_random( config: ModelConfig, dtype: DType, device: &Device, rng: &mut ChaCha8Rng, ) -> Result<Self>
Sourcepub fn forward(
&self,
input_ids: &Tensor,
cache: Option<&mut KvCache>,
hidden_mode: HiddenStatesMode,
) -> Result<CausalLmOutput>
pub fn forward( &self, input_ids: &Tensor, cache: Option<&mut KvCache>, hidden_mode: HiddenStatesMode, ) -> Result<CausalLmOutput>
Forward pass returning logits (batch, seq_len, vocab_size) and optional hidden states.
§Errors
Returns an error if layer forward passes fail.
Trait Implementations§
Auto Trait Implementations§
impl !RefUnwindSafe for OptModel
impl !UnwindSafe for OptModel
impl Freeze for OptModel
impl Send for OptModel
impl Sync for OptModel
impl Unpin for OptModel
impl UnsafeUnpin for OptModel
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> ErasedDestructor for Twhere
T: 'static,
§impl<T> Identity for Twhere
T: ?Sized,
impl<T> Identity for Twhere
T: ?Sized,
impl<T, U> Imply<T> for U
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more