Jonathan Ross, Founder & CEO @ Groq: NVIDIA vs Groq - The Future of Training vs Inference | E1260

How AI Gets Smarter and Faster: Computer Brains Explained

• How can AI make its own learning materials and get super smart?
  
  Imagine teaching yourself! Right now, AI learns from things humans made on the internet. But what if AI could:

  1. Make its own homework - AI can create super smart practice problems
  2. Keep only the best stuff - Just like keeping only A+ work
  3. Learn from its own best work - Getting smarter each time

  It's like if you could teach yourself math, then create harder problems, solve them, and keep getting better forever! This helps AI grow much faster than we thought possible.
• How will combining quick thinking and careful reasoning make AI better?
  
  AI needs two superpowers to be really smart:

  1. Fast thinking - Like when you just "know" 2+2=4 without counting
  2. Slow, careful thinking - Like when you work out a big math problem step-by-step

  Imagine a chess master! They can see good moves instantly (fast thinking) but also plan many moves ahead (careful reasoning). When we combine these powers, AI gets much smarter than using either one alone. It's like having both super speed AND super strength instead of just one superpower!
• How can new company owners prepare for huge AI breakthroughs?
  
  Be ready for big jumps, not just small steps!

  1. Solve problems that aren't fixed yet - Like building a medical helper that never makes mistakes
  2. Get ready BEFORE the wave hits - Position yourself where technology is heading
  3. Focus on the big changes coming - Like when AI stops making stuff up or can follow multi-step plans perfectly

  It's like learning to surf before the big wave comes! Don't just build slightly better versions of what exists today. Instead, imagine what will be possible when AI makes its next big jump forward.

AI Thinking vs AI Working: What's the Difference?

• How will AI working being harder than AI learning change computer stuff?
  
  Training is like teaching an AI by showing it examples. Inference is when the AI uses what it learned to do work. Imagine:

  1. Companies will make special chips just for AI working (inference), not just for AI learning
  2. Using AI will get cheaper because these special chips will use less electricity
  3. More AI helpers everywhere because it won't cost so much to run them
  4. Big computer rooms will be built just for running AI workers

  It's like how we have special machines in factories to do specific jobs!
• How will different computer companies work together with new AI tools?
  
  Think of making AI work like making a sandwich:

  1. Model providers are like recipe makers who create AI brains
  2. Infrastructure providers are like kitchen makers who build places for AI to work
  3. Application developers are like sandwich makers who make things we can use

  With new special AI chips:

  1. Everyone might team up more - sharing their special skills
  2. More AI supermarkets where you can rent AI brains without buying expensive computers
  3. Small companies can make cool AI things without needing lots of money
• What happens when we think of AI chips as robot workers instead of just computers?
  
  Thinking of AI chips as "robot employees" instead of just machines changes everything!

  1. Companies will compare costs - "Should we hire a person for $50,000 a year or buy an AI chip for $10,000?"
  2. AI workers never sleep - They can work 24 hours without getting tired or needing breaks
  3. AI workers don't quit - Once they learn something, they don't forget or leave for another job
  4. We need different skills - People will need to learn how to work WITH AI instead of doing jobs AI can do
  5. Offices might change - We might need fewer desks but more electricity and computer rooms

  It's like having robot helpers who can do certain jobs while humans do other jobs robots can't do!

Computer Brains: How LPUs and GPUs Power AI Differently

• How are LPUs different from GPUs, and why does this save money?
  
  LPUs are like assembly lines while GPUs are like tiny factories that need to be rebuilt for each task.

  1. LPUs keep everything inside: They keep all the AI's "thoughts" (parameters) inside the chips.
  2. GPUs use outside memory: They have to constantly grab information from separate memory chips.
  3. LPUs use less energy: This makes them about 3 times more efficient.
  4. LPUs cost less: They can run AI models for about 5 times less money than GPUs.

  This matters because running big AI models is super expensive, and cheaper chips mean more people can use AI!
• Why is keeping AI information inside chips better than using outside memory?
  
  Keeping information inside chips instead of using outside memory is like having everything you need right at your desk versus running to the library every few minutes!

  1. It's much faster: The AI doesn't waste time grabbing information from far away.
  2. It uses less energy: Moving data between chips uses lots of power - like shouting across a playground instead of whispering to someone next to you.
  3. It's more predictable: Everything happens on a schedule, like a train, not like unpredictable car traffic.
  4. You can connect many chips: Like building with LEGO blocks, you can connect hundreds or thousands of chips together for bigger AI.

  Future AI chips will likely try to keep more information close by instead of far away!
• What happens if all AI chips use 3 times less energy?
  
  If AI chips used 3 times less energy, it would change everything about AI!

  1. More AI everywhere: AI could run in more places with limited power, like phones and small devices.
  2. Cheaper to use: Companies would pay much less to run AI, so more people could afford it.
  3. Less environmental impact: AI uses LOTS of electricity - using 3 times less would help our planet.
  4. Faster innovation: When things cost less, more people can experiment and try new ideas.
  5. Different companies might win: The companies with the most energy-efficient chips could become the new leaders.

  It's like when cars became more fuel-efficient - suddenly more people could drive farther for less money!

Who Will Win the AI Race: USA, China, or Europe?

• How do different countries build AI differently, and who might win?
  
  Each country builds AI in their own special way:

  1. USA: Lots of companies competing and trying new ideas. This makes them very creative but sometimes they waste money.
  2. China: The government helps companies work together. This makes things happen fast, but they might not allow AI to think freely.
  3. Europe: Very careful with rules to keep AI safe. This protects people but might slow down new ideas.

  Imagine if you're running in a race - USA runs creatively but sometimes trips, China runs in a straight line following instructions, and Europe checks every step before running!
• Does controlling what AI can say help or hurt a country's chances to win?
  
  When governments tell AI what it can and cannot say:

  1. It limits creativity: If AI can't think freely, it can't come up with brand new ideas.
  2. It scares away smart people: Imagine if you couldn't say certain things - you might want to go somewhere else!
  3. It creates less helpful AI: AI needs to learn from all kinds of information to be smart.

  Think of it like a teacher who only lets you read certain books. You might miss important stories that could help you learn! Countries that let AI explore more ideas might end up with smarter AI in the long run.
• How can Europe make better rules that allow new ideas but keep people safe?
  
  Europe could make better AI rules by:

  1. Creating special AI cities: Places where new ideas can be tested more easily with fewer rules.
  2. Letting people start work right away: In Europe, sometimes you have to wait 6 months to start a new job - that slows everything down!
  3. Focusing on what to build, not just what to stop: Instead of just making rules about what AI can't do, help people build good AI.
  4. Bringing smart people together: Create places where people who like taking risks can meet and share ideas.

  It's like making a playground with safety rules, but still letting kids try fun new games!

How Groq Makes Money: Computer Chips for AI Helpers

• Is Groq's way of getting other people to pay for computer stuff special?
  
  Groq found a cool trick! Instead of spending their own money to build computer parts, they let other companies pay for it. It's like if you wanted to build a lemonade stand:

  1. Normal way: You save your allowance to buy wood, paint, and lemons
  2. Groq's way: Your friend pays for everything while you focus on making the best lemonade recipe

  This is special because Groq can grow super fast without needing lots of money. Their partners are happy too because they share the money made from selling computer time!
• Why is Groq okay making less money on each computer chip than Nvidia?
  
  Think about cookies!

  1. Nvidia: Sells each cookie for $1 but only makes a few cookies (earning 80¢ profit per cookie)
  2. Groq: Sells each cookie for 25¢ but makes LOTS of cookies (earning 5¢ profit per cookie)

  Groq wants to sell many more chips at lower prices. This helps them:

  1. Grow faster because more people can afford their chips
  2. Take over the market for running AI helpers (called "inference")
  3. Let Nvidia keep making expensive chips for training AI

  They'd rather make a little money from lots of customers than lots of money from a few customers!