pub struct TrainingConfig {Show 17 fields
pub seed: u64,
pub num_train_epochs: f64,
pub batch_size: usize,
pub gradient_accumulation_steps: usize,
pub warmup_ratio: f64,
pub lr: f64,
pub adam_beta1: f64,
pub adam_beta2: f64,
pub adam_eps: f64,
pub weight_decay: f64,
pub max_grad_norm: f64,
pub cos_min_ratio: f64,
pub l2l_loss_scale: f64,
pub save_dir: Option<PathBuf>,
pub exp_name: String,
pub run_name: String,
pub exclude_from_grad: Vec<String>,
}Expand description
Hyperparameters for KdTrainer.
Fields§
§seed: u64Random seed for batch shuffling.
num_train_epochs: f64Number of training epochs (supports fractional epochs like 0.5 or 5.0).
batch_size: usizePer-step micro-batch size.
gradient_accumulation_steps: usizeNumber of micro-batches accumulated per optimizer step.
warmup_ratio: f64Fraction of total optimizer steps spent warming up LR linearly.
lr: f64Peak AdamW learning rate.
adam_beta1: f64AdamW beta1.
adam_beta2: f64AdamW beta2.
adam_eps: f64AdamW numerical epsilon.
weight_decay: f64AdamW weight decay.
max_grad_norm: f64Global gradient L2 clip threshold (0.0 disables clipping).
cos_min_ratio: f64Minimum cosine LR ratio at end of decay (cos_min_ratio).
l2l_loss_scale: f64Multiplier for summed layer-to-layer hidden-state MSE loss (10.0 default).
save_dir: Option<PathBuf>Output directory where checkpoints and metrics logs are saved (optional).
exp_name: StringExperiment name metadata.
run_name: StringRun name metadata.
exclude_from_grad: Vec<String>Module name substrings excluded from training (["lm_head", "embed"] default).
Trait Implementations§
Source§impl Clone for TrainingConfig
impl Clone for TrainingConfig
Source§impl Debug for TrainingConfig
impl Debug for TrainingConfig
Source§impl Default for TrainingConfig
impl Default for TrainingConfig
Source§impl<'de> Deserialize<'de> for TrainingConfig
impl<'de> Deserialize<'de> for TrainingConfig
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
Auto Trait Implementations§
impl Freeze for TrainingConfig
impl RefUnwindSafe for TrainingConfig
impl Send for TrainingConfig
impl Sync for TrainingConfig
impl Unpin for TrainingConfig
impl UnsafeUnpin for TrainingConfig
impl UnwindSafe for TrainingConfig
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> DeserializeOwned for Twhere
T: for<'de> Deserialize<'de>,
impl<T> ErasedDestructor for Twhere
T: 'static,
§impl<T> Identity for Twhere
T: ?Sized,
impl<T> Identity for Twhere
T: ?Sized,
impl<T, U> Imply<T> for U
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more