pub enum PhiAttentionProjection {
Fused {
qkv_proj: Box<LinearLayer>,
},
Split {
q_proj: Box<LinearLayer>,
k_proj: Box<LinearLayer>,
v_proj: Box<LinearLayer>,
},
}Expand description
Attention QKV projection for Phi-3 / Phi-4 supporting both fused dense checkpoints
(qkv_proj) and split LittleBit checkpoints (q_proj, k_proj, v_proj).
Variants§
Fused
Original fused Hugging Face Phi-3/Phi-4 qkv_proj dense layer.
Fields
qkv_proj: Box<LinearLayer>Combined (q_dim + 2 * kv_dim, hidden_size) linear projection.
Split
Split Q/K/V projections produced by PhiQKVSplitAttention, allowing separate
kv_factor scaling on k_proj and v_proj.
Fields
q_proj: Box<LinearLayer>Query projection (num_heads * head_dim, hidden_size).
k_proj: Box<LinearLayer>Key projection (num_kv_heads * head_dim, hidden_size).
v_proj: Box<LinearLayer>Value projection (num_kv_heads * head_dim, hidden_size).
Implementations§
Source§impl PhiAttentionProjection
impl PhiAttentionProjection
Sourcepub fn from_tensors(
attn_prefix: &str,
tensors: &HashMap<String, Tensor>,
cfg: &ModelConfig,
littlebit_cfg: Option<&LittleBitConfig>,
spec_base: LinearLoadSpec<'_>,
) -> Result<Self>
pub fn from_tensors( attn_prefix: &str, tensors: &HashMap<String, Tensor>, cfg: &ModelConfig, littlebit_cfg: Option<&LittleBitConfig>, spec_base: LinearLoadSpec<'_>, ) -> Result<Self>
Load either split Q/K/V weights (when present in a quantized checkpoint) or fused qkv_proj.
§Errors
Returns ModelError::MissingWeight if neither split nor fused projections exist.
Sourcepub fn project(
&self,
hidden_states: &Tensor,
cfg: &ModelConfig,
) -> Result<(Tensor, Tensor, Tensor)>
pub fn project( &self, hidden_states: &Tensor, cfg: &ModelConfig, ) -> Result<(Tensor, Tensor, Tensor)>
Project hidden_states into (query, key, value) tensors of shape [..., dim].
§Errors
Returns an error if underlying linear projections or slicing fail.
Sourcepub fn split_qkv(&mut self, cfg: &ModelConfig) -> Result<()>
pub fn split_qkv(&mut self, cfg: &ModelConfig) -> Result<()>
Convert fused qkv_proj into separate q_proj, k_proj, and v_proj layers
matching upstream PhiQKVSplitAttention._initialize_parameters.
§Errors
Returns an error if slicing weights/biases fails or if qkv_proj is already quantized.
Sourcepub fn apply_littlebit_patch(
&mut self,
attn_prefix: &str,
cfg: &ModelConfig,
lb_cfg: &LittleBitConfig,
policy: &PatchPolicy,
train_mode: TrainMode,
base_seed: u64,
summary: &mut PatchSummary,
) -> Result<()>
pub fn apply_littlebit_patch( &mut self, attn_prefix: &str, cfg: &ModelConfig, lb_cfg: &LittleBitConfig, policy: &PatchPolicy, train_mode: TrainMode, base_seed: u64, summary: &mut PatchSummary, ) -> Result<()>
Split fused qkv_proj if needed and patch q_proj, k_proj, v_proj with LittleBit.
§Errors
Returns an error if splitting or LittleBit factorization fails.
Sourcepub fn convert_to_packed(&mut self) -> Result<()>
pub fn convert_to_packed(&mut self) -> Result<()>
Convert any LittleBit projections inside to compact 1-bit Packed representation.
§Errors
Returns an error if packing fails.
Sourcepub fn pack_state_dict(
&self,
attn_prefix: &str,
out: &mut HashMap<String, Tensor>,
) -> Result<()>
pub fn pack_state_dict( &self, attn_prefix: &str, out: &mut HashMap<String, Tensor>, ) -> Result<()>
Export weights under {attn_prefix}.{qkv_proj|q_proj,k_proj,v_proj}.*.
§Errors
Returns an error if tensor export fails.
Sourcepub fn trainable_vars(&self) -> Vec<Var>
pub fn trainable_vars(&self) -> Vec<Var>
Collect trainable variables.
Sourcepub fn param_count(&self) -> usize
pub fn param_count(&self) -> usize
Total scalar parameters across Q/K/V projections.
Trait Implementations§
Source§impl Clone for PhiAttentionProjection
impl Clone for PhiAttentionProjection
Auto Trait Implementations§
impl !RefUnwindSafe for PhiAttentionProjection
impl !UnwindSafe for PhiAttentionProjection
impl Freeze for PhiAttentionProjection
impl Send for PhiAttentionProjection
impl Sync for PhiAttentionProjection
impl Unpin for PhiAttentionProjection
impl UnsafeUnpin for PhiAttentionProjection
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> ErasedDestructor for Twhere
T: 'static,
§impl<T> Identity for Twhere
T: ?Sized,
impl<T> Identity for Twhere
T: ?Sized,
impl<T, U> Imply<T> for U
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more