SODA models trained on Yodas+Emilia+Nemotron for 500B tokens from scratch
AI & ML interests
multimodal, audio, speech, llms
Papers
View all PapersIsoFLOP Models trained on Yodas+Emilia+Nemotron from budgets of 3e18 to 3e20
-
soda-research/discrete-audio-isoflop-3e20-4.24B-d2816-L27-B64-b829f0
4B • Updated • 2 -
soda-research/discrete-audio-isoflop-3e20-1.93B-d2048-L20-B128-ca249f
2B • Updated • 3 -
soda-research/discrete-audio-isoflop-3e20-2.62B-d2304-L23-B128-29fe40
3B • Updated -
soda-research/discrete-audio-isoflop-3e20-1.68B-d1920-L19-B128-a41e32
2B • Updated • 4
Pretraining Discrete Audio Data -- Interleaved Sequence using Mimi (8 codebooks)
SODA models trained on Yodas+Emilia+Nemotron for 500B tokens from scratch
IsoFLOP Models trained on Yodas+Emilia+Nemotron from budgets of 3e18 to 3e20
-
soda-research/discrete-audio-isoflop-3e20-4.24B-d2816-L27-B64-b829f0
4B • Updated • 2 -
soda-research/discrete-audio-isoflop-3e20-1.93B-d2048-L20-B128-ca249f
2B • Updated • 3 -
soda-research/discrete-audio-isoflop-3e20-2.62B-d2304-L23-B128-29fe40
3B • Updated -
soda-research/discrete-audio-isoflop-3e20-1.68B-d1920-L19-B128-a41e32
2B • Updated • 4
SODA models trained on Yodas+Emilia+Nemotron for 500B tokens with a Qwen3 initialization
Pretraining Discrete Audio Data -- Interleaved Sequence using Mimi (8 codebooks)