Skip to main content

ShardedSafetensorsWriter

Struct ShardedSafetensorsWriter 

Source
pub struct ShardedSafetensorsWriter { /* private fields */ }
Expand description

Incremental sharded safetensors writer that flushes completed shards to disk and frees their in-memory buffers as quantization progresses.

Implementations§

Source§

impl ShardedSafetensorsWriter

Source

pub fn new( output_dir: impl AsRef<Path>, max_shard_bytes: usize, cast_policy: CastPolicy, ) -> Result<Self>

Initialize a streaming sharded safetensors writer under output_dir.

§Errors

Returns LittleBitError::InvalidConfig if max_shard_bytes == 0 or creating output_dir fails.

Source

pub fn add_tensor(&mut self, key: String, tensor: Tensor) -> Result<()>

Stage a tensor for writing, applying CastPolicy and recording its serialized byte size.

§Errors

Returns an error if moving or casting the tensor fails.

Source

pub fn flush_if_needed(&mut self) -> Result<()>

Flush the current pending shard to disk if pending_bytes >= max_shard_bytes.

§Errors

Returns an error if writing the shard fails.

Source

pub fn flush_shard(&mut self) -> Result<()>

Force-flush any buffered tensors into a numbered temporary shard and clear memory.

§Errors

Returns an error if writing the shard file fails.

Source

pub fn finish(self) -> Result<usize>

Finalize shard filenames (model.safetensors or model-00001-of-NNNNN.safetensors), write model.safetensors.index.json, and return the number of output shards written.

§Errors

Returns an error if flushing, renaming, or writing model.safetensors.index.json fails.

Trait Implementations§

Source§

impl Debug for ShardedSafetensorsWriter

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
§

impl<T> ByRef<T> for T

§

fn by_ref(&self) -> &T

§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

§

impl<T> ErasedDestructor for T
where T: 'static,

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

§

impl<T, U> Imply<T> for U
where T: ?Sized, U: ?Sized,

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
§

impl<T> Pointable for T

§

const ALIGN: usize

The alignment of pointer.
§

type Init = T

The type for initializers.
§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

§

fn vzip(self) -> V