pub struct LittleBitModel {
pub model: CausalLm,
pub littlebit_config: LittleBitConfig,
pub base_model_id: Option<String>,
pub source_dir: Option<PathBuf>,
}Expand description
LittleBit quantized model wrapper supporting local persistence and Hugging Face Hub upload/download.
Fields§
§model: CausalLmUnderlying causal language model.
littlebit_config: LittleBitConfigResolved LittleBit quantization hyperparameters.
base_model_id: Option<String>Optional source base model identifier (e.g., meta-llama/Llama-2-7b-hf).
source_dir: Option<PathBuf>Local directory from which the model or tokenizer was resolved.
Implementations§
Source§impl LittleBitModel
impl LittleBitModel
Sourcepub fn new(
model: CausalLm,
littlebit_config: LittleBitConfig,
base_model_id: Option<String>,
source_dir: Option<PathBuf>,
) -> Self
pub fn new( model: CausalLm, littlebit_config: LittleBitConfig, base_model_id: Option<String>, source_dir: Option<PathBuf>, ) -> Self
Wrap a quantized CausalLm alongside its [LittleBitConfig] and optional base model ID.
Sourcepub fn from_pretrained(
model_id_or_path: &str,
options: &LoadOptions,
) -> Result<Self>
pub fn from_pretrained( model_id_or_path: &str, options: &LoadOptions, ) -> Result<Self>
Load a pre-quantized LittleBit model from a local path or Hugging Face Hub repository ID.
Implements upstream priority:
options.overrideslittlebit_config.jsonconfig.jsonfallback- Defaults
§Errors
Returns an error if downloading, config resolution, or weight loading fails.
Sourcepub fn save_pretrained(
&self,
save_dir: &Path,
tokenizer_src: Option<&Path>,
cast_policy: CastPolicy,
) -> Result<()>
pub fn save_pretrained( &self, save_dir: &Path, tokenizer_src: Option<&Path>, cast_policy: CastPolicy, ) -> Result<()>
Save littlebit_config.json, base_model.json, config.json, packed model.safetensors,
model.safetensors.index.json, and tokenizer files (when found) to save_dir.
§Errors
Returns an error if packing weights or writing files fails.
Sourcepub fn generate_readme(&self, repo_id: &str) -> String
pub fn generate_readme(&self, repo_id: &str) -> String
Generate a Hugging Face Hub model card matching upstream LittleBitModel._generate_readme.
Sourcepub fn push_to_hub(
&self,
repo_id: &str,
options: &PushOptions,
) -> Result<String>
pub fn push_to_hub( &self, repo_id: &str, options: &PushOptions, ) -> Result<String>
Save the quantized checkpoint and upload it to Hugging Face Hub.
§Errors
Returns ModelError::Hub or I/O errors if staging or uploading fails.
Trait Implementations§
Source§impl Clone for LittleBitModel
impl Clone for LittleBitModel
Auto Trait Implementations§
impl !RefUnwindSafe for LittleBitModel
impl !UnwindSafe for LittleBitModel
impl Freeze for LittleBitModel
impl Send for LittleBitModel
impl Sync for LittleBitModel
impl Unpin for LittleBitModel
impl UnsafeUnpin for LittleBitModel
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> ErasedDestructor for Twhere
T: 'static,
§impl<T> Identity for Twhere
T: ?Sized,
impl<T> Identity for Twhere
T: ?Sized,
impl<T, U> Imply<T> for U
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more