You can just stack more layers.
MV-Split fixes the silent mean-mode collapse in ultra-deep transformers — stable to 1000 layers.
📄 Paper: arxiv.org/abs/2605.06169
🌐 Project: erwold.github.io/mv-split
💻 Code: github.com/erwold/mv-split
🤗 Models: huggingface.co/StableKirito/m…
StableKirito
77 posts



