harshith
@theharshithh
parallel tensor splitter @simplismartHQ, prev @onfinance_ai
Vous pourriez aimer
had just shipped a repo for training speculative decoding heads to speed up inference of llms by ~3x. get any base model, train a few speculative heads, see the difference in throughput. 🧵on more details. 1/n
please take this adivice from me and do not waste weeks on @LiteLLM. it JUST DOESNT WORK. 1. core repo is 200mb, where 180mb is images and gifs. 2. we had to contribute to our forks and install to get it working. how is so hard to write python transformation fns. just write…
instagram's codebase is ~20mn lines of python. do what u will with information js devs, im sorry. python is really all you need.
AI has been built on one vendor’s stack for too long. AMD’s GPUs now offer state-of-the-art peak compute and memory bandwidth — but the lack of mature software / the “CUDA moat” keeps that power locked away. Time to break it and ride into our multi-silicon future. 🌊 It's been a…
scaling rl is an infra problem than a ml problem. apart from the fact that training is just waiting for rollouts to be completed and updating.
and ladies and gentlemen, its all compute and data. cheetah was a previous checkpoint of composer. build your platform, collect data, train and win!
and ladies and gentlemen, its all compute and data. cheetah was a previous checkpoint of composer. build your platform, collect data, train and win!
testing something here. ignore
here is something I worked on a few months before, you can train a few speculative decoding heads and infer out of that. end to end setup. can be modified to train for eagle spec heads. leave a star if found helpful! github.com/theharshith/sp…
Here's your weekend challenge: Implement speculative decoding. Step 1: Read the following paper and/or blog: arxiv.org/abs/2211.17192 galacodes.hashnode.dev/speculative-de… (cc @jaygala223) Step 2: Choose a family of models which come in various sizes. My choice would be the Gemma3 or Qwen…
the only company i want to work for: @anyscalecompute
Companies i'd defo work for if my startup goes balls up: @atomicworkhq @PrimeIntellect @NousResearch @clonerobotics @PalantirTech @GroqInc @Starcloud_Inc1 @cursor_ai @brexHQ @ExaAILabs
John Wick of CUDA kernels.
“win the internet for a day” as a service the more i think about this, the more it makes sense. so much value unlock for everyone in general.
i wonder how they plan moving data in and data out, latest nasa's latest number is 900mbps. assuming it would be for large scale pretraining, they would launch "space jobs". not sure if this is helpful for realtime inference. exciting time to be alive
Our TPUs are headed to space! Inspired by our history of moonshots, from quantum computing to autonomous driving, Project Suncatcher is exploring how we could one day build scalable ML compute systems in space, harnessing more of the sun’s power (which emits more power than 100…
"While you’re crafting the perfect launch tweet, support tickets pile up. While you’re updating your bio with “Backed by [Famous Fund],” customers churn. While you’re performing success, someone boring is building" while I sort of agree of what @nikunj is telling here but also,…
no we will take care of inference, you just release those weights
"Open-sourcing does not equal free; running inference servers comes with costs." U need to buy a computer to run Linux too "Open-sourcing the weights of large models is different from open-source software; there is no reverse contribution from the developer community."…
Tell me you don't know about AI infra without telling me you don't know about AI infra.
the most important figure of the week
United States Tendances
- 1. #TT_Telegram_sam11adel N/A
- 2. LeBron 82.2K posts
- 3. #DWTS 53.5K posts
- 4. #LakeShow 3,915 posts
- 5. Peggy 18.8K posts
- 6. Whitney 15.9K posts
- 7. Reaves 8,463 posts
- 8. Keyonte George 1,913 posts
- 9. Orioles 7,079 posts
- 10. Macklin Celebrini 2,559 posts
- 11. Jazz 27.2K posts
- 12. Grayson 7,038 posts
- 13. Taylor Ward 3,595 posts
- 14. #TheFutureIsTeal 1,566 posts
- 15. DUSD N/A
- 16. Tatum 16.6K posts
- 17. #Lakers 1,615 posts
- 18. Rantanen 2,096 posts
- 19. ELAINE 17.4K posts
- 20. Dylan 25.1K posts
Something went wrong.
Something went wrong.