Cut Your Losses in Large-Vocabulary Language Models
Paper • 2411.09009 • Published • 49
Papers - Text - Training - Loss - Cut Cross Entropy
Note Putting all the pieces together yields a memory-efficient low-latency cross-entropy loss
Note Putting all the pieces together yields a memory-efficient low-latency cross-entropy loss