Sigma: Differential Rescaling of Query, Key and Value for Efficient Language Models
Paper • 2501.13629 • Published • 48
🔄 In a Training Loop
Fredi
Fredithefish
AI & ML interests
LLMs, Audio, Diffusion Models
Recent Activity
liked a model 1 day ago
Qwen/Qwen3.8-Flash-Next liked a model 3 days ago
netflix/void-model liked a model 3 days ago
facebook/sam-3d-objectsOrganizations
upvoted a paper over 1 year ago
upvoted 2
articles almost 2 years ago
Article
Fixing Gradient Accumulation
- +4
lysandre, ArthurZ, muellerzr, ydshieh, BenjaminB, pcuenq
• • 66 Article
How to build a custom text classifier without days of human labeling
sdiazlor
• • 57 upvoted 5
papers almost 2 years ago
Self-Rewarding Language Models
Paper • 2401.10020 • Published • 156
Law of the Weakest Link: Cross Capabilities of Large Language Models
Paper • 2409.19951 • Published • 54
SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration
Paper • 2410.02367 • Published • 50
RATIONALYST: Pre-training Process-Supervision for Improving Reasoning
Paper • 2410.01044 • Published • 35
VinePPO: Unlocking RL Potential For LLM Reasoning Through Refined Credit Assignment
Paper • 2410.01679 • Published • 27
upvoted an article almost 2 years ago
Article
wHy DoNt YoU jUsT uSe ThE lLaMa ToKeNiZeR??
catherinearnett
• • 55 upvoted a paper about 2 years ago