Skip to main content

Module phi_attn

Module phi_attn 

Source
Expand description

Phi-3 / Phi-4 QKV splitting projection (PhiQKVSplitAttention parity).

Enumsยง

PhiAttentionProjection
Attention QKV projection for Phi-3 / Phi-4 supporting both fused dense checkpoints (qkv_proj) and split LittleBit checkpoints (q_proj, k_proj, v_proj).