Skip to main content

repeat_kv

Function repeat_kv 

Source
pub fn repeat_kv(x: &Tensor, n_rep: usize) -> Result<Tensor>
Expand description

Repeat key/value heads n_rep times along dimension 1 for Grouped-Query Attention (GQA).

ยงErrors

Returns an error if tensor is not rank-4 or reshape fails.