A→Z
A2ZAI
Back to Glossary
concepts

GPU (Graphics Processing Unit)

Specialized processor essential for training and running AI models efficiently.

Share:

Definition

GPUs are processors originally designed for graphics but now essential for AI due to their parallel processing capabilities.

Why GPUs for AI: - Thousands of cores for parallel processing - Matrix operations are parallelizable - Much faster than CPUs for AI workloads - High memory bandwidth

AI-Specific GPUs: - Tensor Cores: Specialized matrix units - High-bandwidth memory (HBM) - Large VRAM capacity - NVLink for multi-GPU

VRAM (Memory) Matters: - Limits model size you can run - 8GB: Small models - 24GB: Medium models - 80GB+: Large models, training

Cloud GPU Options: - AWS (p4d, p5 instances) - Google Cloud (A100, H100) - Azure (NC, ND series) - Lambda Labs, CoreWeave, etc.

Examples

Running Llama 3 8B requires ~16GB VRAM, fitting on RTX 4090.

Related Terms

Training

The process of teaching an AI model to perform tasks by exposing it to data.

Inference

Using a trained AI model to make predictions on new, unseen data.

NVIDIA

Leading GPU manufacturer whose hardware powers most AI training and inference.

Want more AI knowledge?

Get bite-sized AI concepts delivered to your inbox.

Free intelligence briefs. No spam, unsubscribe anytime.

Discussion

Browse all termsTake AI 101 Course