What is a GPU, and why does AI need them?
A GPU (graphics processing unit) is a chip that does huge numbers of simple calculations at the same time. Built to draw 3D graphics, that parallel power now makes GPUs the main engine for training and running AI.
Updated September 24, 2026
A GPU, or graphics processing unit, is a chip designed to do a huge number of simple math operations at the same time. It was invented to render 3D graphics for games. It turns out the same trick, massive parallel math, is exactly what modern AI needs, which is why GPUs now fill the world's AI data centers.
GPU vs. CPU
- A CPU (the main processor in every computer) has a handful of powerful cores and works through tasks mostly one after another. It runs the system and juggles many kinds of jobs.
- A GPU has hundreds or thousands of smaller cores that run the same operation on lots of data at once.
Why AI runs on GPUs
A neural network is layer upon layer of matrix math: multiplying big grids of numbers. That work splits neatly across thousands of cores. GPUs are used both to train models and to run them for users (inference).
Three things made GPUs dominant:
- Special math units. NVIDIA added Tensor Cores built for AI's matrix math.
- Fast memory. AI GPUs stack high-bandwidth memory (HBM) right next to the chip. NVIDIA's H200 has 141 GB of it.
- Software. NVIDIA's CUDA platform, launched in 2006 and 2007, let developers use GPUs for general computing, and a large library of AI tools grew on top of it.
Who makes them
IBM names NVIDIA, AMD and Intel as the major GPU makers. NVIDIA's data-center systems, such as the GB200 NVL72 rack with 72 Blackwell GPUs, and AMD's Instinct accelerators compete for AI workloads. Some tech giants also design their own AI chips, like Google's TPUs.
A short history
NVIDIA says it invented the GPU in 1999 with the GeForce 256, a graphics card for PCs. Over the 2010s, AI researchers discovered GPUs could train neural networks far faster than CPUs, and the chips became the backbone of the AI boom.
Quick questions
What is the difference between a GPU and a CPU?
A CPU has a few powerful cores that handle general tasks mostly one after another. A GPU has thousands of simpler cores that do many calculations at the same time.
What is the difference between a GPU and a TPU?
TPUs are Google's custom chips built only for machine learning and shine on huge, matrix-heavy models. GPUs are more flexible and handle a wider range of work, including custom operations.
Why does AI need so much GPU memory?
Large AI models have to keep billions of numbers close to the chip. AI GPUs use stacked high-bandwidth memory; NVIDIA's H200, for example, has 141 GB running at 4.8 TB per second.
Who makes AI GPUs?
NVIDIA is the best known, with AMD and Intel also making GPUs. Some companies, like Google, design their own AI chips instead.
Sources: IBM: What is a GPU?, AWS: What is a GPU?, NVIDIA: Why GPUs are great for AI, NVIDIA: H200 Tensor Core GPU