Mike Carroll
@mikecarroll_eng
Engineer. Previously @Facebook
Bạn có thể thích
Training Andrej Karpathy’s Nanochat on 4x RTX 3090s at 225W each: Step 2,694/21,400 (12.59% done) Loss: 3.14 Runtime: 6.78 hours Throughput: 3,600 tok/sec Temps: 52-57°C VRAM: 19GB/24GB per card Total cost: 15$ at 55h Zero errors, perfectly stable
I don't like courses. Most were a waste of time. Yes, even at Stanford. If you're new to ML, take CS231N.
your honor i object, i dont know about harvard but stanford literally releases SOTA courses
My meeting budget: 5 min - meet someone new 10 min - solve a problem 15 min - identify + solve a problem Parkinson’s law: work expands so as to fill the time available for its completion.
MIT's 6.851: Advanced Data Structures (Spring'21) courses.csail.mit.edu/6.851/spring21/ This has been on my recommendation list for a while, and the Memory hierarchy discussions are great in the context of cache-oblivious algorithms.
"Cache‑Oblivious Algorithms and Data Structures" by Erik D. Demaine erikdemaine.org/papers/BRICS20… This is a foundational survey on designing cache‑oblivious algorithms and data structures that perform as well as cache‑aware approaches that require hardcoding cache size (M) and block…
50 LLM Projects with Source Code to Become a Pro 1. Beginner-Level LLM Projects → Text Summarizer using OpenAI API → Chatbot for Customer Support → Sentiment Analysis with GPT Models → Resume Optimizer using LLMs → Product Description Generator → AI-Powered Grammar…
I left my plans for weekend to read this recent blog from HuggingFace 🤗 on how they maintain the most critical AI library: transformers. → 1M lines of Python, → 1.3M installations, → thousands of contributors, → a true engineering masterpiece, Here's what I learned:…
Excited to release new repo: nanochat! (it's among the most unhinged I've written). Unlike my earlier similar repo nanoGPT which only covered pretraining, nanochat is a minimal, from scratch, full-stack training/inference pipeline of a simple ChatGPT clone in a single,…
You're not depressed, you just lost your quest.
🔥Free Google Collab notebooks to implement every Machine Learning Algorithm from scratch Link in comment
how i got here: > i used to be and still tend towards having an obsessive/addictive personality > put many years of my life into video games > it was only 2 years ago i started to turn that around because i got other interests and starting really looking forward to the future >…
found a repo that has a massive collection of Machine Learning system design case studies used in the real world, from Stripe, Spotify, Netflix, Meta, GitHub, Twitter/X, and much more link in replies
Copy-pasting PyTorch code is fast — using an AI coding model is even faster — but both skip the learning. That's why I asked my students to write by hand ✍️. 🔽 Download: byhand.ai/pytorch After the exercise, my students can understand what every line really does and…
70 Python Projects with Source code for Developers Step 1: Beginner Foundations → Hello World Web App → Calculator (CLI) → To-Do List CLI → Number Guessing Game → Countdown Timer → Dice Roll Simulator → Coin Flip Simulator → Password Generator → Palindrome Checker →…
everything you need to get started in one repo
System prompts are getting outdated! Here's a counterintuitive lesson from building real-world Agents: Writing giant system prompts doesn't improve an Agent's performance; it often makes it worse. For example, you add a rule about refund policies. Then one about tone. Then…
Nethack is the best benchmark you've never heard of. I'd say that it makes ALE look like a toy, but well... it is
Introducing Scalable Option Learning (SOL☀️), a blazingly fast hierarchical RL algorithm that makes progress on long-horizon tasks and demonstrates positive scaling trends on the largely unsolved NetHack benchmark, when trained for 30 billion samples. Details, paper and code in >
it's insane to me how little attention the llm.q repo has it's a fully C/C++/CUDA implementation of multi-gpu (zero + fsdp), quantized LLM training with support for selective AC it's genuinely the coolest OSS thing I've seen this year (what's crazier is 1 person wrote it!)
Finally had a chance to listen through this pod with Sutton, which was interesting and amusing. As background, Sutton's "The Bitter Lesson" has become a bit of biblical text in frontier LLM circles. Researchers routinely talk about and ask whether this or that approach or idea…
.@RichardSSutton, father of reinforcement learning, doesn’t think LLMs are bitter-lesson-pilled. My steel man of Richard’s position: we need some new architecture to enable continual (on-the-job) learning. And if we have continual learning, we don't need a special training…
As the divide between the super-rich and the rest widens, this strategy becomes increasingly relevant.
United States Xu hướng
- 1. #2025MAMAVOTE 1.4M posts
- 2. #KonamiWorldSeriesSweepstakes N/A
- 3. Tyla 16.5K posts
- 4. Fetterman 68.2K posts
- 5. No Kings 150K posts
- 6. Deport Harry Sisson 25.6K posts
- 7. Somalia 30.6K posts
- 8. Miguel Vick N/A
- 9. Dave Dombrowski N/A
- 10. GTreasury 4,320 posts
- 11. #SpiritDay 1,386 posts
- 12. Yung Miami N/A
- 13. Andrade 6,713 posts
- 14. #thursdayvibes 3,825 posts
- 15. Ninja Gaiden 25.6K posts
- 16. #WorldFoodDay 36.6K posts
- 17. Mila 17.7K posts
- 18. Starting 5 7,781 posts
- 19. Tomonobu Itagaki 19.2K posts
- 20. Turkey Leg Hut N/A
Something went wrong.
Something went wrong.