Whisper Web
🎤
1.23k
Transcribe audio files to text instantly
Zach Zimmermann
ZachZimm
AI & ML interests
None yet
Recent Activity
liked
a Space
8 days ago
Xenova/whisper-web
liked
a Space
2 months ago
anycoderapps/LongCat-Image-Edit
liked
a Space
2 months ago
abdul9999/NoWatermark
Organizations
None yet
liked
a
Space
8 days ago
liked
2
Spaces
2 months ago
liked
a
model
4 months ago
reacted to
ZennyKenny's
post with 🤗
10 months ago
Post
1449
Submitted my first dataset for the Reasoning Datasets Competition! https://huggingface.co/datasets/ZennyKenny/TRON-dataset-v.1.0
This dataset is designed to post-train Metareasoning agents, or those agents whose job it is to quickly (and importantly, cheaply) reason through whether it makes sense to launch a full reasoning job or simply use a simple completions job.
There's still plenty of time to join the competition! https://www.bespokelabs.ai/blog/reasoning-datasets-competition
Generation notebook (linked in dataset) is open source and pretty well generalized if I don't say so myself, so you can use it to make your own Metareasoning datasets.
Shoutout to @onekq for his inspiring comment on this topic.
This dataset is designed to post-train Metareasoning agents, or those agents whose job it is to quickly (and importantly, cheaply) reason through whether it makes sense to launch a full reasoning job or simply use a simple completions job.
There's still plenty of time to join the competition! https://www.bespokelabs.ai/blog/reasoning-datasets-competition
Generation notebook (linked in dataset) is open source and pretty well generalized if I don't say so myself, so you can use it to make your own Metareasoning datasets.
Shoutout to @onekq for his inspiring comment on this topic.
liked
2
models
11 months ago
liked
2
models
over 1 year ago
liked
a
Space
over 1 year ago
liked
a
model
over 1 year ago
reacted to
singhsidhukuldeep's
post with 🤯
over 1 year ago
Post
2851
Meta Researchers: How many compute hours should we use to train Llama 3.1?
Mr. Zuck: Yes! 🤖💪
Good folks at @AIatMeta did not just release the models but also published a 92-page detailed paper 📄 on their findings and technical aspects of the models and their training process!
Generally, we just gobble up these weights and forget the compute infrastructure used to train these models. 🖥️🚀
Here are some interesting findings about the computing infrastructure of Llamas:
- Llama 1 and 2 models were trained on @Meta 's AI Research SuperCluster. Llama 3 was migrated to Meta’s production clusters! 📊
- That's 16,000 H100 GPUs, with each GPU featuring 700W TDP and 80GB HBM3, arranged in Meta’s Grand Teton AI server platform. 🖥️🔋
- What about storing checkpoints? Used Tectonic, a distributed file system, for storage, with capacities reaching 240 PB and peak throughput of 7 TB/s. 💾📈
- Meta's mad lads saved each GPU’s model state, ranging from 1 MB to 4 GB per GPU, for recovery and debugging. 🛠️🔍
If this sounds big, well, they document the humungous challenges that come with it:
- In the 54-day training period, there were 466 job interruptions. 🕒🔄
- About 78% of unexpected interruptions were attributed to confirmed or suspected hardware issues. Mostly GPUs! 💥🖥️
- Saving all checkpoints is cool until you do it for the 300B+ parameters model. The bursty nature of checkpoint writes, essential for state-saving during training, periodically saturated the storage fabric, impacting performance. 📉💾
- With all this, effective training time—measured as the time spent on useful training over the elapsed time—was higher than 90%. ⏱️📊
I think this is the stuff that movies can be made on! 🎬🌟
Paper: https://ai.meta.com/research/publications/the-llama-3-herd-of-models/
Mr. Zuck: Yes! 🤖💪
Good folks at @AIatMeta did not just release the models but also published a 92-page detailed paper 📄 on their findings and technical aspects of the models and their training process!
Generally, we just gobble up these weights and forget the compute infrastructure used to train these models. 🖥️🚀
Here are some interesting findings about the computing infrastructure of Llamas:
- Llama 1 and 2 models were trained on @Meta 's AI Research SuperCluster. Llama 3 was migrated to Meta’s production clusters! 📊
- That's 16,000 H100 GPUs, with each GPU featuring 700W TDP and 80GB HBM3, arranged in Meta’s Grand Teton AI server platform. 🖥️🔋
- What about storing checkpoints? Used Tectonic, a distributed file system, for storage, with capacities reaching 240 PB and peak throughput of 7 TB/s. 💾📈
- Meta's mad lads saved each GPU’s model state, ranging from 1 MB to 4 GB per GPU, for recovery and debugging. 🛠️🔍
If this sounds big, well, they document the humungous challenges that come with it:
- In the 54-day training period, there were 466 job interruptions. 🕒🔄
- About 78% of unexpected interruptions were attributed to confirmed or suspected hardware issues. Mostly GPUs! 💥🖥️
- Saving all checkpoints is cool until you do it for the 300B+ parameters model. The bursty nature of checkpoint writes, essential for state-saving during training, periodically saturated the storage fabric, impacting performance. 📉💾
- With all this, effective training time—measured as the time spent on useful training over the elapsed time—was higher than 90%. ⏱️📊
I think this is the stuff that movies can be made on! 🎬🌟
Paper: https://ai.meta.com/research/publications/the-llama-3-herd-of-models/
- 3 replies
reacted to
clem's
post with 👍
over 1 year ago
Post
2503
This is the week of small AI language models!
-
- 4 replies
liked
a
model
over 1 year ago
liked
2
models
almost 2 years ago
liked
a
dataset
about 2 years ago
liked
a
model
about 2 years ago
liked
a
dataset
about 2 years ago