🤏 quecto mode
appvoid
appvoid
AI & ML interests
singularity through byte-level tokens and small language models
Recent Activity
repliedto their post about 10 hours ago
Byte-level state-space models. That sounded pretty scary for a scientist decades ago. Now we have:
1. Knowledge that deeper layers train smoothly.
2. Knowledge that Transformers work but is quadratic on sequence length.
3. Knowledge that SSMs work even better. Numerically unstable sometimes.
4. Speculative-decoding.
5. Open high-quality data.
6. Knowledge that KD works.
It slowly feels like is no longer a bad idea. new activity about 10 hours ago
BananaMind/BananaMind-2.1-Unified:This network integrates information in an unusual way liked a model about 13 hours ago
BananaMind/BananaMind-2.1-Unified