Expand description
LittleBit-2 Joint Iterative Quantization (Joint-ITQ) latent rotation.
Functionsยง
- apply_
joint_ itq - Jointly align SVD factors
U(out_features, split_dim)andV(split_dim, in_features)viaX_combined = [U; V^T], updatingU <- U @ RandV <- R^T @ V. - compute_
itq_ rotation - Find orthogonal rotation
Rof shape(dim, dim)aligningX((rows, dim)) to the binary hypercube:min_R || sign(X @ R) - X @ R ||_F^2. - random_
orthogonal_ matrix - Generate a Haar-distributed orthogonal
(dim, dim)matrix via QR with positiveRdiagonal.