Skip to main content

l2l_mse_loss

Function l2l_mse_loss 

Source
pub fn l2l_mse_loss(
    student_reps: &[Tensor],
    teacher_reps: &[Tensor],
    scale: f64,
) -> Result<Tensor>
Expand description

Compute layer-to-layer MSE loss summed over aligned transformer hidden states and scaled by scale.

Matches KDTrainer.mse_loss accumulation in utils/kd_utils.py.

ยงErrors

Returns an error if hidden state counts or tensor shapes mismatch.