pub struct StreamingQuantizeOptions {
pub seed: u64,
pub cast_policy: CastPolicy,
pub max_shard_bytes: usize,
pub device: Device,
pub dtype: DType,
pub text_only: bool,
pub base_model_id: Option<String>,
}Expand description
Configuration options for stream_quantize_checkpoint.
Fields§
§seed: u64Deterministic RNG seed for randomized SVD and Joint-ITQ (42 default).
cast_policy: CastPolicySerialization cast policy for scale/norm tensors (ToBf16 or Preserve).
max_shard_bytes: usizeMaximum byte size per output shard before flushing to disk (2 GiB default).
device: DeviceCompute device used during SVD/ITQ decomposition (Device::Cpu default).
dtype: DTypeFloating-point compute dtype during factorization (DType::F32 default).
text_only: boolSkip multimodal vision/audio tower weights when quantizing multimodal checkpoints.
base_model_id: Option<String>Optional base model ID recorded in base_model.json.
Trait Implementations§
Source§impl Clone for StreamingQuantizeOptions
impl Clone for StreamingQuantizeOptions
Source§impl Debug for StreamingQuantizeOptions
impl Debug for StreamingQuantizeOptions
Auto Trait Implementations§
impl Freeze for StreamingQuantizeOptions
impl RefUnwindSafe for StreamingQuantizeOptions
impl Send for StreamingQuantizeOptions
impl Sync for StreamingQuantizeOptions
impl Unpin for StreamingQuantizeOptions
impl UnsafeUnpin for StreamingQuantizeOptions
impl UnwindSafe for StreamingQuantizeOptions
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> ErasedDestructor for Twhere
T: 'static,
§impl<T> Identity for Twhere
T: ?Sized,
impl<T> Identity for Twhere
T: ?Sized,
impl<T, U> Imply<T> for U
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more