Expand description
Neural Transformer compression and decompression codec for tufazip backed by Candle.
Ported from nn.py and checkpoint_utils.py: includes the Peri-RMSNorm / Post-RMSNorm
SwiGLU Transformer with RoPE or learned-relative attention, online Adam updates,
periodic prefix retraining, QPI1 prior loading, and atomic checkpoint resume.
Structs§
- Adam
- Pure-Rust PyTorch-compatible Adam optimizer (without decoupled weight decay) with state export/import.
- Checkpoint
Metadata - Serializable metadata written alongside
latest.safetensorsfor resumable execution. - Codec
Config - Complete configuration for running the neural compressor or decompressor.
- Deterministic
Rng - Deterministic, checkpointable PRNG for weight initialization and replay dropout.
- Loaded
Checkpoint - Loaded checkpoint metadata and tensor map.
- Model
Config - Transformer architectural configuration.
- Optim
Config - Online Adam optimizer hyperparameters.
- Piecewise
LrSchedule - Linearly interpolated piecewise learning-rate schedule (
lr0[,step1,lr1,...]). - Prior
Load Info - Summary of loading a
QPI1prior archive into aTransformer. - Retrain
Config - Periodic prefix retraining (replay) hyperparameters.
- RngState
- Exact serializable state of
DeterministicRng. - RunSummary
- Metrics reported upon completion of a neural coding or estimation run.
- Transformer
- Neural Transformer language model matching
nn.py.
Enums§
- Autocast
Policy - Controls whether
ComputeDType::Bf16activates on CPU or only GPU accelerators. - Codec
Error - Errors returned by the neural codec, model initialization, optimizer, or checkpoints.
- Codec
Mode - Operating mode for
crate::run_codec. - ComputeD
Type - Model activation precision during forward/backward passes.
- Device
Choice - Hardware execution backend preference.
- Forward
Mode - Forward pass execution mode.
- Norm
Style - Residual block normalization placement.
- Online
Loss - Online supervision target mode.
- Output
Slice - Selects which sequence positions are projected through the vocabulary head.
- Position
Type - Positional attention mechanism.
- RunOutcome
- Outcome returned by
run_codec.
Constants§
- CHECKPOINT_
VERSION - Checkpoint schema version.
- DONE_
NAME - Sentinel filename marking successful codec completion.
- REQUEUE_
EXIT_ CODE - Exit code returned when stopping cleanly at a block boundary for requeue/resume.
- STOP_
NAME - Sentinel filename requesting clean checkpoint-and-stop at the next block boundary.
Functions§
- clear_
stop - Removes
STOP_REQUESTEDindirif present. - clip_
grad_ norm - Computes the global L2 norm across
vars’ gradients and rescalesgradsin place when abovemax_norm. - is_done
- Returns
trueifdir/DONEexists. - load_
checkpoint - Loads an existing checkpoint from
dir, verifyinglatest.safetensorsagainsttensors_sha256. - lr_at
- Linear warmup followed by cosine decay to
min_lr. - mark_
done - Atomically writes the
DONEcompletion marker indir. - request_
stop - Writes
STOP_REQUESTEDindir. - resolve_
device - Resolves a Candle [
Device] matching the requestedDeviceChoice. - run_
codec - Runs online neural compression, decompression, or entropy estimation.
- save_
checkpoint - Saves a resumable checkpoint (
latest.safetensors+latest.json) atomically underdir. - stop_
requested - Returns
trueifdir/STOP_REQUESTEDexists.