electron271/graig-experiment-4
Text Generation
ā¢
4B
ā¢
Updated
ā¢
331
ā¢
1
Creitin Gameplays
CreitinGameplays
AI & ML interests
text generation models
Recent Activity
liked
a model
about 5 hours ago
electron271/graig-experiment-4
liked
a model
5 days ago
moonshotai/Kimi-K2.5
liked
a model
about 1 month ago
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
Organizations
liked
a
model
about 5 hours ago
liked
a
model
5 days ago
liked
a
model
about 1 month ago
liked
a
Space
2 months ago
updated
a
Space
2 months ago
published
a
Space
2 months ago
liked
a
Space
2 months ago
liked
a
model
3 months ago
reacted to
nouamanetazi's
post with š
3 months ago
Post
4366
After training šš¦šØš„ššš on ššš ššššš¬ for nearly a month, I've come to realize something most people overlook: š¢š§šš«šš¬šš«š®ššš®š«š š¢š¬ šš”š š¦šš¤š-šØš«-šš«ššš¤ šššššØš« š¢š§ ššš šš«šš¢š§š¢š§š . š„
Everyone talks about model architecture and data quality. And yes, those matter immensely. But here's what nobody tells you: when your training run fails at 2 AM because of mysterious šššš šš«š«šØš«š¬, or when your expensive GPU cluster is running at šš% šššš¢šš¢šš§šš², the problem isn't your model. It's most probably a š¦š¢š¬š®š¬š šØš šš”š š”šš«šš°šš«š. š ļø
Questions that seemed simple but had no clear answers: Why is ššØš šš«šš¢š§š¢š§š š¬š„šØš°šš« šš”šš§ ššš§š¬š š¦šØššš„š¬? Which šššš šš„šš š¬ should we actually set? How often should we checkpoint without killing throughput?
That's why we built šš”š šš¦šØš„ šš«šš¢š§š¢š§š šš„šš²ššØšØš¤ š: a complete guide covering everything from model architecture and data curation to the SmolLM3 training marathon, post-training techniques, and crucially, the š¢š§šš«šš¬šš«š®ššš®š«š š„šš²šš« that most teams get wrong.
We validated real vs theoretical bandwidth across the entire stack: šššš š”š¢ššš¢š§š š šš/š¬, šššš¢š§š¤ š.š š«šššš”š¢š§š ššš šš/š¬, šššš ššš§š šš šš.š šš/š¬. Then we ran collective operations across ššš šššš¬ (16 nodes, 8xH100s each) and measured how performance degrades at scale: all-reduce drops from ššš šš/š¬ on a single node to ššš-ššš šš/š¬ across 16 nodes.
If you've ever wondered why your training runs are slower than they should be, or you're planning to scale up and want to avoid expensive mistakes, this guide might save you weeks of debugging.
šš”š šš¦šØš„ šš«šš¢š§š¢š§š šš„šš²ššØšØš¤: https://lnkd.in/e5MKXUHS
Shared with ā¤ļø by the HuggingFace team
Everyone talks about model architecture and data quality. And yes, those matter immensely. But here's what nobody tells you: when your training run fails at 2 AM because of mysterious šššš šš«š«šØš«š¬, or when your expensive GPU cluster is running at šš% šššš¢šš¢šš§šš², the problem isn't your model. It's most probably a š¦š¢š¬š®š¬š šØš šš”š š”šš«šš°šš«š. š ļø
Questions that seemed simple but had no clear answers: Why is ššØš šš«šš¢š§š¢š§š š¬š„šØš°šš« šš”šš§ ššš§š¬š š¦šØššš„š¬? Which šššš šš„šš š¬ should we actually set? How often should we checkpoint without killing throughput?
That's why we built šš”š šš¦šØš„ šš«šš¢š§š¢š§š šš„šš²ššØšØš¤ š: a complete guide covering everything from model architecture and data curation to the SmolLM3 training marathon, post-training techniques, and crucially, the š¢š§šš«šš¬šš«š®ššš®š«š š„šš²šš« that most teams get wrong.
We validated real vs theoretical bandwidth across the entire stack: šššš š”š¢ššš¢š§š š šš/š¬, šššš¢š§š¤ š.š š«šššš”š¢š§š ššš šš/š¬, šššš ššš§š šš šš.š šš/š¬. Then we ran collective operations across ššš šššš¬ (16 nodes, 8xH100s each) and measured how performance degrades at scale: all-reduce drops from ššš šš/š¬ on a single node to ššš-ššš šš/š¬ across 16 nodes.
If you've ever wondered why your training runs are slower than they should be, or you're planning to scale up and want to avoid expensive mistakes, this guide might save you weeks of debugging.
šš”š šš¦šØš„ šš«šš¢š§š¢š§š šš„šš²ššØšØš¤: https://lnkd.in/e5MKXUHS
Shared with ā¤ļø by the HuggingFace team
liked
a
Space
4 months ago
updated
2
datasets
4 months ago
published
2
datasets
4 months ago
liked
a
dataset
4 months ago
liked
a
model
4 months ago
updated
a
Space
5 months ago
Is this demo powered by hugging face?
š
š¤
1
2
#124 opened 6 months ago
by
terobox
liked
a
model
5 months ago
updated
a
model
6 months ago