Whoops! I almost convinced myself with Sea AI lab's results: the experiments were right in front of me, and it made perfect sense why switching to FP16 (provides precision with 10 mantissa bits) eliminates the training-inference mismatch, BUT on A100 (Ampere arch).
I was puzzled by why their paper claims "bfloat16" training crashes -- since we trained for 100,000 GPU hours and 7K+ training steps for both dense and MoEs in the ScaleRL paper stably without any crashes. I think it matters what kind of GPUs they used -- they mention in the…
There were times my mind would just go blank looking at huge, scary codebases, but with better prompts, ai tools helps me untangle everything so much faster.
A quick quiz! Evaluate the one liner (complex list comprehension) of a simple problem in seconds.
I've touched graphs, DP problems in C++ and now back to solve with python - starting from basic data structures for a warm-up. (can't promise to be consistent let's see how far it goes...) You can find DSA questions i solved in C++ here: github.com/Khan-Ramsha/DS…
Saavdhaan rahe, Satark rahe... Taking risk is another obsession of mine; it throws me into unpredictable spaces!
how it feels to yolo your frontier model training run on pytorch nightlies
United States 趨勢
- 1. #DWTS 16K posts
- 2. Virginia 300K posts
- 3. Alix 5,634 posts
- 4. Louisville 97.6K posts
- 5. Elaine 41.4K posts
- 6. Flav 4,987 posts
- 7. Jay Jones 44.8K posts
- 8. Abigail Spanberger 41K posts
- 9. #WWENXT 7,795 posts
- 10. Carrie Ann 1,113 posts
- 11. Dylan 23.4K posts
- 12. Collin Chandler N/A
- 13. #Election2025 7,172 posts
- 14. Tyrese Maxey 2,569 posts
- 15. Jets 137K posts
- 16. New Jersey 340K posts
- 17. College Football Playoff 5,690 posts
- 18. Danielle 7,180 posts
- 19. #DancingWithTheStars N/A
- 20. MD-11 21.2K posts
Something went wrong.
Something went wrong.