pub struct ShardedSafetensorsWriter { /* private fields */ }Expand description
Incremental sharded safetensors writer that flushes completed shards to disk and frees their in-memory buffers as quantization progresses.
Implementations§
Source§impl ShardedSafetensorsWriter
impl ShardedSafetensorsWriter
Sourcepub fn new(
output_dir: impl AsRef<Path>,
max_shard_bytes: usize,
cast_policy: CastPolicy,
) -> Result<Self>
pub fn new( output_dir: impl AsRef<Path>, max_shard_bytes: usize, cast_policy: CastPolicy, ) -> Result<Self>
Initialize a streaming sharded safetensors writer under output_dir.
§Errors
Returns LittleBitError::InvalidConfig if max_shard_bytes == 0 or creating output_dir fails.
Sourcepub fn add_tensor(&mut self, key: String, tensor: Tensor) -> Result<()>
pub fn add_tensor(&mut self, key: String, tensor: Tensor) -> Result<()>
Stage a tensor for writing, applying CastPolicy and recording its serialized byte size.
§Errors
Returns an error if moving or casting the tensor fails.
Sourcepub fn flush_if_needed(&mut self) -> Result<()>
pub fn flush_if_needed(&mut self) -> Result<()>
Flush the current pending shard to disk if pending_bytes >= max_shard_bytes.
§Errors
Returns an error if writing the shard fails.
Sourcepub fn flush_shard(&mut self) -> Result<()>
pub fn flush_shard(&mut self) -> Result<()>
Force-flush any buffered tensors into a numbered temporary shard and clear memory.
§Errors
Returns an error if writing the shard file fails.
Sourcepub fn finish(self) -> Result<usize>
pub fn finish(self) -> Result<usize>
Finalize shard filenames (model.safetensors or model-00001-of-NNNNN.safetensors),
write model.safetensors.index.json, and return the number of output shards written.
§Errors
Returns an error if flushing, renaming, or writing model.safetensors.index.json fails.
Trait Implementations§
Auto Trait Implementations§
impl !RefUnwindSafe for ShardedSafetensorsWriter
impl !UnwindSafe for ShardedSafetensorsWriter
impl Freeze for ShardedSafetensorsWriter
impl Send for ShardedSafetensorsWriter
impl Sync for ShardedSafetensorsWriter
impl Unpin for ShardedSafetensorsWriter
impl UnsafeUnpin for ShardedSafetensorsWriter
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
impl<T> ErasedDestructor for Twhere
T: 'static,
impl<T, U> Imply<T> for U
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more