Definition
Test-time compute refers to allocating more computational resources when the model is generating outputs, rather than only investing compute during training.
Key Insight: Instead of just making models bigger, you can make them "think longer" on hard problems.
Techniques: - Chain of thought reasoning - Multiple sampling and voting - Tree search over responses - Self-verification - Extended thinking time
OpenAI o1 Approach: - Model generates internal reasoning - More tokens for complex problems - Self-checks before responding - Trade-off: slower but more accurate
When It Helps: - Math problems - Coding challenges - Complex reasoning - Multi-step logic
Trade-offs: - Longer response times - Higher inference costs - May not help simple tasks
Examples
OpenAI o1 spending more "thinking tokens" on a hard math problem to ensure the answer is correct.
Related Terms
Want more AI knowledge?
Get bite-sized AI concepts delivered to your inbox.
Free intelligence briefs. No spam, unsubscribe anytime.