Scaling Embeddings Outperforms Scaling Experts in Language Models
Paper • 2601.21204 • Published • 105
Junyuan Shang
sjy1203
AI & ML interests
NLP
Recent Activity
upvoted a paper 21 days ago
Scaling Embeddings Outperforms Scaling Experts in Language Models upvoted a paper about 1 month ago
Native Audio-Visual Alignment for Generation authored a paper 5 months ago
DHA: Learning Decoupled-Head Attention from Transformer Checkpoints via Adaptive Heads FusionOrganizations
None yet
upvoted a paper 21 days ago
upvoted a paper about 1 month ago
authored 3
papers 5 months ago
DHA: Learning Decoupled-Head Attention from Transformer Checkpoints via Adaptive Heads Fusion
Paper • 2406.06567 • Published • 1
NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Time
Paper • 2408.03675 • Published • 1
ERNIE 5.0 Technical Report
Paper • 2602.04705 • Published • 269
upvoted a paper 5 months ago
authored 6
papers 6 months ago
ERNIE-Doc: A Retrospective Long-Document Modeling Transformer
Paper • 2012.15688 • Published • 1
ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
Paper • 2107.02137 • Published
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
Paper • 2112.12731 • Published • 1
Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking
Paper • 2502.13842 • Published
Mixture of Hidden-Dimensions Transformer
Paper • 2412.05644 • Published • 2
VideoAR: Autoregressive Video Generation via Next-Frame & Scale Prediction
Paper • 2601.05966 • Published • 23
upvoted a paper 6 months ago
liked a
model 10 months ago
