Skip to main content

compute_kd_loss

Function compute_kd_loss 

Source
pub fn compute_kd_loss(
    student_out: &CausalLmOutput,
    teacher_out: &CausalLmOutput,
    l2l_loss_scale: f64,
) -> Result<KdLossBreakdown>
Expand description

Compute combined Knowledge Distillation loss (KD + L2L MSE) from student and teacher outputs.

Slices &hidden_states[1..] to skip embedding outputs, matching upstream outputs.hidden_states[1:].

ยงErrors

Returns an error if hidden_states are missing or loss calculation fails.