NVIDIA GeForce RTX 4090 24GB
24 GB VRAM is the practical ceiling for many 70B-class quantized models on a single consumer card.
Gear
Hardware for running local models
Honest picks for people who build AI agents and want to run models on their own machines. Original icons only — no product photos or prices. Open Amazon to compare current options.
As an Amazon Associate I earn from qualifying purchases. Some links are affiliate links; we may earn a commission at no extra cost to you. Disclosure
GPUs, Macs and storage
Local agent loops are VRAM- and RAM-bound more often than CPU-bound. Pick for memory first, then speed.
NVIDIA GeForce RTX 4080 Super 16GB
A strong mid/high option when 24 GB is more than you need; fine for 32B–34B Q4/Q5 local agents.
Apple Mac Mini (M4 / M4 Pro)
Unified memory makes Macs a quiet, efficient host for Apple-silicon builds of llama.cpp, MLX and Ollama.
64 GB DDR5 desktop RAM kit
CPU offload and large KV caches eat system RAM fast; 64 GB is a sensible floor for serious local agent loops.
2 TB NVMe SSD (PCIe 4.0)
Model weights are multi-gigabyte files; a fast 2 TB drive keeps several GGUF/MLX variants on hand without thrashing.
Compact mini PC with discrete GPU
A small desktop with an RTX-class card is often quieter and cheaper than a full tower for always-on local agents.
Books worth keeping nearby
Short shelf for builders who want systems intuition, not hype.
AI Engineering — Chip Huyen
Practical guide to building production LLM applications, evaluation and systems design.
Build a Large Language Model (From Scratch) — Sebastian Raschka
Implements an LLM step by step; excellent for understanding what your local stack is doing.
Hands-On Large Language Models — Jay Alammar & Maarten Grootendorst
Clear walkthrough of embeddings, retrieval and fine-tuning with modern tooling.
Designing Machine Learning Systems — Chip Huyen
Broader ML systems context that still applies when agents call tools and models in production.
Notes
- Links use our Amazon Associates tag (
jrghouseofip-20) andrel="nofollow sponsored". - Hardware links go to Amazon search results on purpose — specific GPU SKUs and sellers change often.
- Book links use known ISBN/ASIN identifiers. Prefer the edition that matches your format (print or Kindle).
- Icons are original line drawings for the item type, not brand logos or store product photos.
- For API model prices instead of local hardware, see model prices and best models for agents.