DK-013 — The State of AI (2026)

🧠 The State of AI, Mid-2026: Models, Architecture, World Models, and the Vocabulary You Need to Sound Like You Actually Know What’s Going On

“A model is not an agent. An agent is a model wearing a harness.”

Everyone’s cousin has an opinion about AI right now. Most of it is vibes. This post is not vibes. It’s the actual mental model — from how a transformer is trained, to why “RAG is dead” is wrong, to why your brain is still the most absurd piece of computing hardware in the known universe, to whether letting an AI manage a team of other AIs is a good idea.

We’ll go from foundations to frontier. If you already know what a transformer is, skip ahead — but even the “basic” sections have a twist you probably haven’t seen framed this way.

Foundations: What Actually Happens When You “Train” a Model
The Vocabulary Upgrade: Fine-Tuning vs. Retraining vs. Training From Scratch
RAG in 2026: Is It Dead, Reborn, or Just Renamed?
The Model Landscape, Mid-2026: US Frontier and Chinese Frontier
Hallucination: Why It Happens and Why It Never Fully Goes Away
AI Harness, Human-in-the-Loop, AI Literacy, AI Governance — The New Core Vocabulary
World Models: The Other AGI Bet
Multi-Agent Dev Teams: Should an AI Be Your Project Manager?
Data Leakage and Why “Just Paste It Into ChatGPT” Is a Corporate Risk
AI Sovereignty: Why Countries Suddenly Care Who Owns the Weights
Why So Many Smart People Think LLMs Will Become AGI (and Why Others Don’t)
Your Brain vs. a Supercomputer: An Honest Comparison
How to Actually Use AI Well: Augmentation, Not Autopilot

🔬 Chapter 1: Foundations — What Actually Happens When You “Train” a Model

Before the vocabulary, you need the shape of the thing. A large language model (LLM) is, underneath everything, a very large function that takes in a sequence of tokens (fragments of words) and predicts the probability distribution of the next token. That’s it. Everything — reasoning, code generation, poetry — is an emergent side effect of getting extremely good at that one prediction task, at a scale of hundreds of billions to trillions of parameters.

The Three-Stage Pipeline

Modern frontier model training happens in (roughly) three stages, and understanding these stages is the single highest-leverage thing you can learn this year, because almost every piece of 2026 vocabulary is really just “which stage are we talking about.”

Stage 1 — Pretraining. The model is shown an enormous corpus (trillions of tokens: web text, code, books, licensed data, synthetic data) and trained purely to predict the next token. No instructions, no “be helpful.” This is where the model absorbs grammar, facts, world knowledge, and — controversially — a huge share of its emergent reasoning ability. This stage costs the most compute by far, often tens of thousands of GPUs/TPUs running for months.

Stage 2 — Post-training / alignment. This is where a raw pretrained model (which just completes text — including completing it badly) becomes an assistant. This has multiple sub-steps:

Supervised Fine-Tuning (SFT): the model is shown curated examples of ideal question-answer pairs.
RLHF (Reinforcement Learning from Human Feedback): human raters rank multiple model outputs; a reward model is trained on those rankings; the base model is then optimized against that reward model.
RLAIF (RL from AI Feedback): the same idea, but an AI model does the ranking instead of humans — cheaper and more scalable, and increasingly dominant in 2026 pipelines because human-ranking doesn’t scale to trillions of comparisons.
RLVR (RL from Verifiable Rewards): for domains with a checkable ground truth — math, code that either compiles and passes tests or doesn’t — the model is rewarded directly against an automated verifier rather than a learned preference model. This is the technique behind the big 2025–2026 jump in “reasoning models” that show their chain-of-thought before answering.

Stage 3 — Inference-time scaling. Instead of (or in addition to) making the model bigger, you let it “think longer” at answer time — generating longer chains of reasoning, sampling multiple candidate answers and picking the best one, or running tool calls in a loop. This is why you now see model variants labeled “high / xhigh / max thinking” — the same underlying weights, given more inference-time compute budget.

The Brain Analogy, Done Properly (Part 1)

People love to say “the AI’s brain is like a human brain.” It’s a useful lie for stage 1 and a bad one for stage 2. Pretraining resembles a child absorbing language and the statistical structure of the world through sheer exposure — genuinely brain-like in spirit, if not in mechanism. But RLHF/RLVR has no clean biological analogue: it’s closer to an extremely intensive, automated version of a teacher grading millions of practice essays and a coach correcting your form in real time, simultaneously, at a scale no human institution could ever run. Keep this distinction in your head — we’ll come back to it in Chapter 12.

📚 Chapter 2: Fine-Tuning vs. Retraining vs. Training From Scratch

This trio confuses even people who use AI daily, so let’s nail it down with a concrete anchor: cost and how much of the model’s “brain” you’re touching.

Term	What it means	Rough cost	Analogy
Training from scratch	Random-initialized weights, full pretraining pipeline on trillions of tokens	$10M–$100M+, months, thousands of accelerators	Growing a new brain from a fertilized egg
Retraining	Re-running the (or an updated) pretraining process, often to incorporate a new architecture, new data mix, or to produce the “next generation” (e.g., GPT-4 → GPT-5)	Similar cost profile to training from scratch — it usually is training from scratch, just using lessons learned from the last generation	Raising a second child with better parenting knowledge, not surgery on the first child
Fine-tuning	Taking an already-pretrained model and continuing training on a smaller, task- or domain-specific dataset, adjusting some or all weights	$10s–$10,000s, hours to days, a handful of GPUs	An adult doctor doing a specialized residency — the general medical brain is already there

Two important 2026 nuances:

Full fine-tuning vs. parameter-efficient fine-tuning (PEFT). Full fine-tuning touches every weight. Techniques like LoRA (Low-Rank Adaptation) and QLoRA instead freeze the original weights and train small “adapter” matrices bolted onto the network — cheaper, faster, reversible, and now the default way most companies customize an open-weight model (Llama, Qwen, DeepSeek) for their own data.
“Retraining” is often confused with fine-tuning in casual conversation, but the distinction that actually matters in a business context is: are you changing what the model fundamentally knows and how it reasons (retraining/pretraining), or are you changing how it behaves on top of what it already knows (fine-tuning)? If your company’s internal jargon or tone is the issue, fine-tuning (or honestly, just good prompting/RAG) solves it. If the model is missing entire domains of world knowledge, no amount of fine-tuning substitutes for a model that was pretrained on that domain.

🔎 Chapter 3: RAG in 2026 — Dead, Reborn, or Just Renamed?

RAG (Retrieval-Augmented Generation) means: instead of relying purely on what the model memorized during pretraining, you retrieve relevant documents from an external knowledge base at query time and stuff them into the model’s context window before it answers.

“We have 1M-token context now, do we still need RAG?”

This is the single most-debated architecture question of H1 2026, and the honest answer is: yes, but RAG itself has changed shape.

The naive 2023-era argument against RAG was: context windows are growing (many frontier and open-weight models now advertise context windows well past 200K tokens, with some Chinese models like Qwen’s long-context variants marketing 1M-token windows), so “just put everything in context” should make retrieval unnecessary.

Reality in 2026 pushed back on that for three concrete reasons:

Cost and latency. Feeding a million tokens into every query — even when only 200 words of it are relevant — is enormously wasteful. RAG is a cost-and-speed optimization as much as a knowledge one.
“Lost in the middle.” Even models with huge nominal context windows show measurably worse recall for information buried in the middle of a very long context versus the beginning or end. Bigger windows didn’t fully solve this; retrieval that surfaces exactly the relevant chunk still outperforms “dump everything in.”
Freshness and permissions. A model’s weights are frozen after training. RAG is the only practical way to give a model access to information that changed after training, or information that’s private/permissioned and should never be baked into weights in the first place (this matters enormously for the data-leakage discussion in Chapter 9).

What “modern RAG” actually looks like now

The 2023 version of RAG was: chunk documents → embed → cosine-similarity search → stuff top-k chunks into the prompt. That naive pipeline is now considered legacy. The modern (2026) RAG stack typically layers in:

Hybrid retrieval — combining dense vector search with classic keyword/BM25 search, because pure embedding search misses exact-match terms (product SKUs, legal citations, error codes).
Re-ranking — a second, smaller model re-scores the top 50–100 retrieved chunks before the final 5–10 go into the prompt, catching cases where the embedding model’s similarity score was misleading.
Agentic RAG — instead of one retrieval pass, the model itself decides iteratively what to search for, evaluates whether the results actually answer the question, and re-queries if not. This blurs RAG into the “agent with tools” paradigm entirely: retrieval becomes just one tool among many (web search, database query, code execution) that the model calls when it decides it needs more information, rather than a rigid pre-processing step.
GraphRAG — building a knowledge graph of entities and relationships from the documents first, so the model can answer multi-hop questions (“who was the manager of the person who signed this contract’s original 2019 version?”) that pure similarity search handles poorly.

So the honest one-line answer to “do we still need RAG”: the term “RAG” is dissolving into “an agent that has retrieval as one of its tools,” but the underlying need — grounding answers in real, current, permissioned data rather than frozen weights — is more important than ever, not less.

🌍 Chapter 4: The Model Landscape, Mid-2026

This section will be stale in a month — that’s the nature of the field right now — but here’s the honest lay of the land as of July 2026.

US Frontier

OpenAI: shipped the GPT-5 family through several point releases across 2026 (GPT-5.2, GPT-5.4, GPT-5.5 as the current shipping default in ChatGPT, with a GPT-5.6 “Sol-tier” preview for the heaviest reasoning workloads). Strength: broad ecosystem, tool integrations, and strong agentic/terminal-workflow benchmarks.
Anthropic: the Claude lineup moved through Opus 4.5 → 4.6 → 4.7 → 4.8 across the first half of 2026, with Claude Sonnet 5 as the new balanced default and Claude Opus 4.8 as the flagship. Anthropic also introduced a tier above Opus — the Mythos class — with Claude Fable 5 as its safety-hardened public variant. Independent benchmark trackers generally put the Claude family ahead on real-world coding-agent tasks (like resolving actual GitHub issues) and long-document coherence.
Google DeepMind: the Gemini 3 family (3 Pro → 3.1 Pro → 3.5 Flash/Pro) leads several reasoning and multimodal benchmarks and has the deepest native integration with Google Workspace, Search, and — notably — the world-model work covered in Chapter 7.
xAI: Grok 4 and its successors compete aggressively on price and lead some knowledge-recall benchmarks (Humanity’s Last Exam), with the cheapest flagship-class output pricing on the market as of mid-2026.
Meta: Llama 4 remains the most-used fully open-weight Western model family, competing with Chinese open-weight labs more than with the closed US labs above.

Chinese Frontier

The Chinese ecosystem stopped being “the cheap alternative” sometime in 2025 and became a genuine second pole of frontier capability by mid-2026:

DeepSeek (V3.2, with V4 delayed by reported compute/chip constraints) remains the price-performance benchmark everyone else gets compared against — open-weight, extremely cheap per token, competitive on reasoning.
Alibaba’s Qwen (Qwen3, Qwen3.5/3.6 Max/Plus) offers some of the largest advertised context windows in the industry and leads on multilingual capability and agent-framework tooling (Qwen3 Agent + Qwen-Image).
Zhipu AI’s GLM (GLM-5, GLM-5.1) has repeatedly led open-weight coding benchmarks, in some evaluations surpassing closed Western models like Gemini 3 Pro on agentic coding tasks.
Moonshot AI’s Kimi (K2, K2.5, K2.6) is specifically tuned for long-context retrieval and document-heavy workflows and became, per several benchmark trackers, the first open-weight model to beat a GPT-5.4 variant on the SWE-Bench Pro coding benchmark.
MiniMax and StepFun round out a now-crowded field, with StepFun’s Step 3.5 Flash pushing pricing down to roughly $0.10 per million input tokens — undercutting Western pricing by an order of magnitude.

The strategic read: Chinese labs have converged on “good enough to be a top-3 model, radically cheaper, mostly open-weight.” That combination is exactly what’s driving the sovereignty and self-hosting conversation in Chapter 10 — a country or company that doesn’t want to depend on a single foreign API now has credible, near-frontier options it can run on its own infrastructure.

🎭 Chapter 5: Hallucination — Why It Happens and Why It Doesn’t Fully Go Away

Hallucination = the model generating confident, fluent, and wrong information. It is not a bug that will simply be “patched out” — it’s a structural consequence of how these models work, for three compounding reasons:

The model is a probability machine, not a fact database. It predicts plausible next tokens. Plausible and true are correlated, not identical. When the true answer is rare or ambiguous in training data, “plausible-sounding” and “correct” diverge.
Compression loss. Trillions of tokens of training data get compressed into a fixed number of parameters. Specific facts (a phone number, a rarely-cited statistic, an exact court case citation) are exactly the kind of low-frequency information that gets “smoothed over” during this compression, the same way a JPEG at high compression loses fine detail while keeping the overall picture recognizable.
Reward hacking during RLHF/RLVR. Human raters (and AI raters) tend to prefer confident-sounding, complete answers over answers that hedge or say “I don’t know.” This subtly trains models to guess rather than abstain, because guessing wins more preference comparisons even when it’s sometimes wrong.

What actually reduces hallucination in practice (2026 state of the art):

RAG/grounding (Chapter 3) — give the model the actual source instead of asking it to recall from memory.
RLVR training on domains with checkable answers, which measurably improves calibration in those domains.
Explicit “I don’t know” reward shaping — some labs now specifically reward abstention on genuinely unanswerable questions rather than only rewarding correctness.
Tool use — letting the model call a calculator, a search engine, or a code interpreter rather than “mentally” computing or recalling.

None of this eliminates hallucination. It reduces its rate and shifts where it shows up (usually: the more obscure, low-frequency, or recent the fact, the higher the risk).

🧩 Chapter 6: The New Core Vocabulary

AI Harness

An agent harness is the software scaffolding around a model that turns “a model that predicts text” into “an agent that gets things done.” The formulation that’s become standard in 2026 engineering circles: Agent = Model + Harness.

The harness includes: the system prompt and task instructions, the tool definitions the model can call (file editing, web search, code execution, MCP servers), the sandboxed execution environment, memory/state persistence across turns, and the verification/feedback loop that checks the agent’s work and feeds errors back in. The insight that made “harness” a term of art rather than just “infrastructure”: a fixed model’s real-world task performance can vary enormously depending purely on harness quality — a well-engineered harness can make a mid-tier model outperform a frontier model wrapped in a sloppy harness. Harness engineering — designing constraints, verification steps, and feedback loops for agents — is now treated as its own discipline, separate from prompt engineering.

Human-in-the-Loop (HITL)

A design pattern where a human reviews, approves, or can interrupt an AI system’s action at a defined checkpoint, rather than letting the system act fully autonomously. In 2026, this has split into two flavors that are worth distinguishing:

HITL-for-quality: a human reviews outputs before they ship (e.g., approving AI-drafted emails).
HITL-for-safety: a human must explicitly authorize any consequential action (deleting data, spending money, sending external communications), regardless of how confident the agent is — analogous to a two-person rule in security-sensitive fields.

AI Literacy

The practical skill of understanding what a model can and can’t reliably do, how to prompt and verify it, and where its failure modes (hallucination, sycophancy, stale knowledge) are likely to bite you. It is increasingly treated as a baseline professional skill, the way “spreadsheet literacy” was in the 1990s — not “can you use the tool” but “do you understand its failure modes well enough not to get burned by it.”

AI Governance

The broader set of policies, regulations, and organizational controls around how AI is developed and deployed: model evaluations before release, incident reporting, the EU AI Act’s risk-tiered obligations, export controls on advanced chips and (increasingly) on advanced models themselves, and internal corporate policies about which tools employees may feed sensitive data into. Governance is where the technical (harness, HITL) and the political (sovereignty, export controls) layers meet.

🌌 Chapter 7: World Models — The Other AGI Bet

If 2023–2025 was defined by “scale the transformer, add reasoning,” 2026’s most interesting parallel bet is world models — and the two camps genuinely disagree about which path leads to general intelligence.

A world model is a system trained not to predict the next word, but to predict how an environment evolves — given the current state of a scene and an action, what does the next moment look like? Google DeepMind’s Genie 3, released as a research preview in 2025 and opened to the public in January 2026, generates fully interactive, navigable 3D environments in real time (24 frames per second) from a text prompt, with the model having taught itself physics and spatial consistency purely from watching video — no hand-coded physics engine underneath. NVIDIA’s Cosmos platform pursues a similar goal for robotics and autonomous-vehicle simulation, trained on an enormous corpus of real-world driving, industrial, and human-interaction video. Waymo has already integrated Genie-derived world models into its self-driving simulation stack.

The philosophical split matters: Yann LeCun (formerly Meta’s chief AI scientist, now running his own world-model lab) has argued for years that language models alone will never reach general intelligence, because text is a low-bandwidth, already-compressed representation of the world — a child learns intuitive physics (that unsupported objects fall) from years of raw sensory exposure, not from reading sentences about gravity. His architecture family (JEPA — Joint Embedding Predictive Architecture) is built specifically around learning to predict future states of the world in an abstract representation space, rather than predicting pixels or tokens.

Meanwhile, DeepMind explicitly frames Genie 3 as infrastructure for training better agents — the point isn’t the video generation itself, but giving an AI agent an effectively unlimited curriculum of realistic, interactive practice environments to learn in, sidestepping the fact that the real world is slow, expensive, and dangerous to practice in directly.

Honest caveat, which the researchers themselves are candid about: current world models still lose object permanence and violate physics over longer time horizons, are extremely compute- and data-hungry to train, and have no agreed-upon benchmark for “does it actually understand the world” the way we have SWE-bench for coding. This is a genuine research frontier, not a solved problem being productized — treat any breathless “world models solve AGI” headline with real skepticism, including the ones in this article.

👥 Chapter 8: Multi-Agent Dev Teams — Should an AI Be Your Project Manager?

The pattern getting a lot of attention in 2026: instead of one AI coding agent, you run several — one writes code, one writes tests, one reviews pull requests — coordinated by an orchestrator agent acting like a project manager: breaking down the task, assigning subtasks, and merging results. Frameworks like LangGraph, CrewAI, and cloud-native orchestration services (Microsoft’s Foundry Agent Service, Google’s Agent Development Kit) have made this pattern accessible enough that “give it a try” no longer requires a research team.

Does it actually work? The honest, non-hyped answer, based on both vendor-reported figures and independent write-ups, is: it depends heavily on task shape, and the gains are real but narrower than the marketing suggests.

It clearly helps on tasks that decompose cleanly into independent, parallelizable subtasks with objective success criteria — refactors, migrations, “add tests to every module,” incident-response triage where multiple hypotheses can be checked in parallel. Some published incident-response trials report dramatically higher rates of actionable output from multi-agent setups versus a single agent working sequentially.
It struggles on tasks requiring a single coherent architectural vision, where splitting work across agents introduces the same coordination overhead and miscommunication risk you’d get splitting the work across humans who don’t talk to each other enough — except the “AI PM” doesn’t yet have the judgment to notice when two agents’ outputs subtly contradict each other’s assumptions.
The orchestrator itself becomes a new single point of failure and a new attack/error surface — if the “PM agent” misunderstands the task or drops context, everything downstream inherits that error, often silently, because no individual worker agent is positioned to catch a planning-level mistake.

Practical read for a team considering this in 2026: treat “AI-managed multi-agent dev team” as a legitimate productivity tool for well-scoped, verifiable work — not as a replacement for a human tech lead’s judgment on ambiguous, high-stakes architecture decisions. The harness (Chapter 6) matters enormously here: teams that invest in strong verification gates between agent handoffs report far better outcomes than teams that just chain agents together and hope.

🔒 Chapter 9: Data Leakage — Why “Just Paste It Into the AI” Is a Real Risk

Every time you paste a document into a consumer AI chat interface, three separate risks are worth distinguishing, because they get conflated constantly:

Training-data risk: does the provider use your input to train future models? (Enterprise/API tiers from major providers generally contractually exclude this by default; free consumer tiers sometimes don’t — always check the specific product’s current data-use policy rather than assuming.)
Retention/breach risk: even if it’s not used for training, is it logged, and could it leak via a security incident, a subpoena, or an employee with access?
Jurisdictional risk: which country’s servers and laws govern that data once it leaves your organization’s boundary? This is precisely what’s driving Chapter 10.

The practical 2026 corporate response has been a wave of private/sovereign AI deployments — running open-weight models (Llama, Qwen, DeepSeek, GLM are all popular choices specifically because they’re open-weight and self-hostable) inside a company’s or country’s own infrastructure, trading some frontier capability for full control over where data physically goes and who can access it. Surveys from major consultancies in 2026 report a large share of enterprise AI leaders now naming “enabling private or sovereign AI” as their single biggest adoption barrier — not because they don’t want AI, but because their risk and compliance functions won’t sign off on the default cloud-API path for sensitive data categories.

🏛️ Chapter 10: AI Sovereignty — Why Nations Care Who Owns the Weights

AI/data sovereignty is the idea that a nation (or a large organization) should maintain control over the entire lifecycle of its AI systems — not just where the data physically sits, but who trained the model, on what data, and whether the organization can keep running it if the original vendor relationship ends.

2026 made this concrete rather than theoretical: the World Economic Forum’s Davos meeting in January 2026 had sovereign AI as its dominant technology theme; the EU launched OpenEuroLLM to build models across 24 European languages rather than depend entirely on US labs; the UAE built the Falcon/Jais model family through G42 and MBZUAI; India’s IndiaAI Mission hosted a Global South AI summit that drew commitments from over a hundred countries; and export-control fights over advanced chips (and, per this notice you may already be aware of, over certain frontier models themselves) became a recurring feature of US-China AI relations through the first half of 2026.

The strategic logic, stripped of rhetoric: a country or company entirely dependent on one foreign vendor’s API is exposed to that vendor’s pricing changes, policy changes, service interruptions, or export-control decisions it has zero say over. Building (or credibly threatening to build) a domestic alternative is as much geopolitical leverage as it is a technical achievement — which is exactly why the rise of capable, cheap, open-weight Chinese models in Chapter 4 has become a geopolitical talking point and not just a technology story.

🤔 Chapter 11: Why So Many Smart People Think LLMs Will Become AGI (and Why Others Don’t)

Two camps, both containing genuinely serious researchers:

The “scaling is enough” camp points to an empirical pattern that has held up remarkably well since around 2020: model capability improves smoothly and predictably as you scale up parameters, data, and compute together (the “scaling laws”). If that trend holds — the argument goes — and you add better reasoning training (RLVR) and longer inference-time thinking, there’s no obvious wall preventing continued improvement all the way to human-level generality across most cognitive tasks. Dario Amodei of Anthropic has repeatedly argued powerful, broadly capable AI is close enough to be planning for economically (his Davos 2026 remarks about AI-driven GDP growth and unemployment risk reflect exactly this view).

The skeptics’ camp — including, notably, Demis Hassabis of Google DeepMind, who stated at the same Davos gathering that current systems remain “nowhere near” human-level general intelligence — points to specific, stubborn gaps: models still lack persistent, updating memory the way humans do; they don’t have grounded, embodied understanding of the physical world (hence the world-model research bet in Chapter 7); they hallucinate in ways that suggest a difference in kind, not just degree, from human error; and there’s real debate about whether “reasoning” in a chain-of-thought is genuine multi-step logic or an increasingly sophisticated form of pattern completion that merely looks like reasoning on the benchmarks we’ve chosen to measure.

Neither camp is being unserious. The honest 2026 state of the debate is: capability curves keep climbing on the benchmarks we have, but there’s genuine, unresolved disagreement about whether those benchmarks measure the things that actually constitute “general intelligence,” or whether we’re getting extremely good at exactly the tasks we chose to test.

🧠 Chapter 12: Your Brain vs. a Supercomputer — An Honest Comparison

This is the part that should make you feel better, not worse, about being a biological intelligence.

A modern frontier LLM runs on data-center hardware: separate memory (RAM/storage) and processing units (GPUs/TPUs) connected by comparatively slow interconnects, consuming on the order of tens of megawatts across a training run, with data constantly shuttled back and forth between “where it’s stored” and “where it’s computed.”

Your brain does not have this separation. Roughly 86 billion neurons, each with thousands of synaptic connections, perform storage and computation in the same physical substrate — a synapse’s strength (a “weight,” in ML terms) is both the stored memory and the active computational element simultaneously. There’s no separate “RAM” being copied over a bus to a separate “CPU.” This is called in-memory computing, and it’s a property AI hardware researchers are still trying to replicate with neuromorphic chips, because the classic CPU/GPU-plus-separate-RAM architecture (the “von Neumann architecture,” dating to the 1940s) is fundamentally memory-bandwidth-limited in a way biological brains simply aren’t.

And the power budget is almost embarrassing to compare: your brain runs on roughly 20 watts — less than a household lightbulb — for continuous, real-time, multimodal sensory processing, motor control, memory formation and retrieval, language, planning, and social reasoning, simultaneously, for decades, without a training/inference split (you learn while you operate, constantly, online, from a tiny number of examples). A frontier model’s training run alone consumes megawatts for months; inference at scale consumes additional megawatts continuously across data centers serving millions of users.

None of this means LLMs are “fake intelligence” — the comparison isn’t really fair in either direction, because brains and transformers optimize for different things under different constraints, evolved (in one case literally) for different purposes. But it’s worth sitting with: your brain is, on pure energy-efficiency-per-unit-of-general-capability, still the most efficient known general intelligence in the universe. That’s not a consolation prize. That’s a genuinely unmatched engineering achievement that evolution produced with zero design meetings.

🛠️ Chapter 13: How to Actually Use AI Well

Given everything above, here’s the practical synthesis:

Treat the model as a very well-read, very fast, occasionally-confidently-wrong collaborator — not an oracle. Verify anything load-bearing (numbers, citations, legal/medical specifics) against a primary source, especially anything that falls into the “low-frequency in training data” hallucination risk zone from Chapter 5.
Use RAG/grounding for anything that changed after the model’s knowledge cutoff, or that’s private to you. Don’t rely on frozen weights for current facts.
Keep a human in the loop at consequential decision points, proportional to the stakes — reviewing a drafted email is low-stakes; letting an agent autonomously spend money or delete production data is not, regardless of how good the agent’s track record has been so far.
Use AI to widen your thinking, not replace it. The most reliable pattern across serious users in 2026 isn’t “ask AI, accept answer” — it’s “ask AI, get a first draft or a set of angles you hadn’t considered, then apply your own judgment on top.” The tasks AI still reliably fails at are exactly the ones requiring genuine judgment under ambiguity, taste, or accountability — which, not coincidentally, is where human value is concentrating.
Match the tool to the task shape, the same lesson as Chapter 8’s multi-agent teams: well-scoped, verifiable, parallelizable work is where AI (single-agent or multi-agent) shines; ambiguous, high-stakes, judgment-heavy work is where you should still be doing the driving.

🖥️ Chapter 14: Reading Model Size, and What Hardware You Actually Need

You’ll see model names like “8B,” “32B,” “70B,” “671B,” or “1.8T.” Here’s how to read that number, and what it actually costs you in silicon.

What “B” and “T” mean

The number is the parameter count — how many individual weights the neural network has, in billions (B) or trillions (T). Roughly speaking, more parameters = more capacity to store knowledge and represent complex patterns, but it’s not a clean line to “smarter.” A well-trained 32B model can beat a poorly-trained 70B model. Parameter count is a proxy for capacity, not a guarantee of quality — training data quality and post-training (Chapter 1) matter enormously too.

Dense vs. Mixture-of-Experts (MoE): A “dense” model activates every parameter for every token. An MoE model (DeepSeek-V3/V4, GLM-5, many Qwen variants) has many “expert” sub-networks and only routes each token through a small subset of them. So a “671B” MoE model might only activate ~37B parameters per token — you still need enough memory to hold all 671B in VRAM, but the actual compute per token is much closer to a 37B dense model. This is why MoE models can be both huge and comparatively fast.

The one formula that matters: VRAM ≈ parameters × bytes-per-parameter

Precision	Bytes/param	Rough VRAM for weights alone
FP16 / BF16 (full)	2 bytes	7B → ~14GB, 70B → ~140GB
INT8	1 byte	7B → ~7GB, 70B → ~70GB
Q4 (4-bit quantization)	0.5 bytes	7B → ~3.5–4GB, 70B → ~35–40GB

Quantization = compressing the weights to lower precision after training, trading a small amount of accuracy for a large drop in memory footprint. Q4/Q5 (GGUF format, used by llama.cpp/Ollama) is the default choice for running models locally — the accuracy loss is usually small enough not to matter for most day-to-day use.

Add on top of the weights: KV cache (memory that grows with context length × number of concurrent users — often the actual bottleneck in production, not the weights) and a working-memory overhead of roughly 10–20%.

Practical spec table: running models locally

Model size class	Example models	Minimum viable GPU	Comfortable setup
1B–3B (“tiny,” on-device)	Llama 3.2 1B/3B, Qwen2.5 0.5B–3B, Gemma 3 1B/4B	Any modern laptop GPU, 8GB VRAM, even CPU-only	RTX 4060/5060 (8–12GB)
7B–14B (“small,” daily-driver local assistant)	Llama 3.1 8B, Qwen3 8B/14B, Mistral 7B	12–16GB VRAM (RTX 4070/5070 Ti) at Q4	RTX 5090 (32GB) at FP16
30B–32B (“mid,” strong local coding assistant)	Qwen3 32B, DeepSeek-R1-Distill-32B, Gemma 3 27B	RTX 5090 (32GB) at Q4	RTX PRO 6000 Blackwell (96GB) at FP16
70B (“large,” near frontier quality)	Llama 3.3 70B, Qwen2.5 72B	RTX PRO 6000 Blackwell (96GB) at FP8, or 2× RTX 5090 (32GB each) split across GPUs	2–4× H100/H200, or Mac Studio Ultra (192GB+ unified memory) at Q4/Q8
100B–700B+ MoE (“frontier-class,” e.g., DeepSeek-V3/V4, GLM-5, Kimi K2)	—	4–8× H100/H200/B200 (80–192GB each) with NVLink	8× B200/B300 DGX node, or a multi-node cluster for the biggest MoE releases

Rule of thumb for “how many GPUs do I need”: take the model’s total parameter count, multiply by your target precision’s bytes-per-parameter, add ~20% for KV cache and overhead, then divide by your GPU’s VRAM. If the answer is more than 1, you need multi-GPU serving with tensor parallelism (splitting each layer’s computation across GPUs, needs fast interconnect like NVLink) or pipeline parallelism (splitting different layers onto different GPUs, more tolerant of slower interconnect but adds latency).

Designing for inference: serving many users, not just yourself

Running a model for yourself and serving hundreds of concurrent users are different engineering problems:

Continuous batching: instead of processing one request fully before starting the next, the serving engine dynamically batches multiple users’ in-flight requests together on the GPU — this is the single biggest throughput win for multi-user serving, and it’s the default behavior in modern serving engines (see tools below).
KV cache management: each user’s conversation keeps a growing cache of attention keys/values. This is usually the real memory bottleneck at scale, not the model weights — techniques like PagedAttention (treating KV cache like OS virtual memory pages, popularized by vLLM) let you serve far more concurrent users out of the same VRAM by avoiding memory fragmentation.
Speculative decoding: a small “draft” model quickly guesses several tokens ahead, and the big model verifies them in one pass instead of generating token-by-token — meaningful latency reduction with no quality loss when done correctly.
Load balancing across replicas: for real production scale, you run multiple copies of the model behind a router, not one giant instance — horizontal scaling, same as any other web service.

Recommended serving tools (2026):

vLLM — the most widely adopted open-source serving engine, built around PagedAttention, continuous batching, and broad model support. Good default choice.
SGLang — strong alternative, particularly fast for structured generation and agent workloads with heavy tool-calling.
TensorRT-LLM (NVIDIA) — lower-level, maximum throughput on NVIDIA hardware if you’re willing to do more tuning.
Ollama / llama.cpp — best for single-user local use and quick prototyping, not built for high-concurrency production serving.

Vector databases for RAG (Chapter 3)

If you’re building a retrieval pipeline, the vector database is where your document embeddings live and get searched:

pgvector — a Postgres extension; the pragmatic default if you already run Postgres and don’t want a new piece of infrastructure to operate.
Qdrant — open-source, fast, popular for self-hosted production RAG with good filtering support alongside vector search.
Milvus — built for very large scale (billions of vectors), the choice once you outgrow single-node setups.
Weaviate — open-source with built-in hybrid (keyword + vector) search and a plugin ecosystem for re-ranking.
Pinecone — fully managed/hosted, the easiest path if you don’t want to operate the database yourself, at the cost of it being closed and cloud-only.

Rule of thumb: start with pgvector if you already have Postgres and your corpus is under a few million chunks; move to Qdrant or Milvus once you need dedicated scale, and only reach for a managed service like Pinecone if you specifically want to offload operations rather than infrastructure cost.

📋 Chapter 15: The Cheat Sheet — Every Chapter, One Table

If you only screenshot one part of this entire post, screenshot this. Each row = one thing you should be able to drop into conversation and sound like you know exactly what’s going on.

#	Topic	Keyword(s)	The one-line hint to remember
1	Training pipeline	Pretraining → SFT → RLHF/RLAIF/RLVR → inference-time scaling	Pretraining = raw knowledge from predicting next-token; post-training = shaping it into an assistant; “high/max thinking” modes = more compute spent at answer time, not a different model
1	Reward signal types	RLHF vs RLAIF vs RLVR	RLHF = humans rank answers; RLAIF = AI ranks answers (scales better); RLVR = graded against an automatic checker (math/code) — this is what powers “reasoning models”
2	Model customization	Fine-tuning vs. retraining vs. training from scratch	Scratch/retrain = touching the whole brain, $10M+; fine-tune = specializing an existing brain, $10s–$1000s; LoRA/QLoRA = cheap adapter-based fine-tuning, freeze the base, train small add-on matrices
3	Grounding models in real data	RAG → Agentic RAG → GraphRAG	Long context ≠ retrieval solved (cost, “lost in the middle,” freshness/permissions); modern RAG = hybrid search + re-ranking + the model deciding when to search, i.e. retrieval becomes just one tool in an agent’s toolbox
4	US frontier models (mid-2026)	GPT-5.5/5.6, Claude Sonnet 5/Opus 4.8/Fable 5, Gemini 3.1 Pro/3.5, Grok 4, Llama 4	OpenAI = ecosystem + agentic workflows; Anthropic = coding-agent + long-doc coherence leader; Google = multimodal + Workspace + world models; xAI = cheapest flagship pricing
4	China frontier models (mid-2026)	DeepSeek V3.2/V4, Qwen3.x, GLM-5/5.1, Kimi K2.6, StepFun, MiniMax	DeepSeek = price/performance benchmark; Qwen = huge context + multilingual; GLM = open-weight coding leader; Kimi = long-context/agentic RAG specialist; whole field now genuinely near-frontier, mostly open-weight
5	Hallucination	Compression loss + reward hacking + no ground truth	Model predicts plausible, not true; rare facts get “smoothed” like JPEG compression; RLHF subtly rewards confident guessing over honest “I don’t know”
5	Fixing hallucination	Grounding, RLVR, abstention rewards, tool use	None of these eliminate it — they shrink it and push it toward the rarest/most recent facts
6	Turning a model into an agent	AI harness	Agent = Model + Harness; harness = system prompt + tools + sandbox + memory + verification loop; a great harness can beat a frontier model in a bad harness
6	Safety pattern	Human-in-the-loop (HITL)	HITL-for-quality (review before it ships) vs. HITL-for-safety (must approve consequential actions — money, deletion, external comms)
6	New baseline skill	AI literacy	Knowing what the model reliably can’t do (hallucination, staleness, sycophancy) — the 2026 equivalent of “spreadsheet literacy”
6	Policy layer	AI governance	Evals before release, incident reporting, EU AI Act risk tiers, export controls — where technical (harness) meets political (sovereignty)
7	The other AGI bet	World models — Genie 3, Cosmos, JEPA	Predicts how a scene evolves, not the next word; DeepMind frames it as infinite training environments for agents; LeCun (JEPA) argues language alone can never reach general intelligence
7	Honest caveat	Object permanence, physics drift, no benchmark	World models still break over long time horizons — treat “world models solve AGI” headlines skeptically
8	AI-managed teams	Multi-agent orchestration, orchestrator = “AI PM”	Works well: parallelizable, verifiable tasks (refactors, incident triage). Fails: tasks needing one coherent vision — orchestrator becomes a silent single point of failure
9	Corporate risk	Data leakage — training risk, retention risk, jurisdiction risk	Three separate risks, don’t conflate them; check the specific product’s data-use policy, don’t assume
10	Geopolitics	AI/data sovereignty	Control over the whole lifecycle (data + model + infrastructure), not just server location; driven directly by cheap, capable, open-weight Chinese models (row 4) needing no foreign API dependency
11	The AGI debate	Scaling-is-enough (Amodei) vs. skeptics (Hassabis: “nowhere near”)	Both camps are serious; disagreement is really about whether current benchmarks measure the thing that is general intelligence
12	Brain vs. computer	In-memory computing, ~20 watts, von Neumann bottleneck	Your brain fuses memory + compute in the same synapse, no separate RAM/CPU shuttle; runs on a lightbulb’s worth of power vs. megawatts for a training run — most energy-efficient general intelligence known, full stop
13	Using AI well	Augment, don’t autopilot	Verify load-bearing facts, ground with RAG for anything post-cutoff/private, HITL proportional to stakes, match tool to task shape (row 8)
14	Reading model size	1B–3B tiny, 7–14B small, 30–32B mid, 70B large, 100B+ MoE frontier	Bigger ≠ automatically smarter; dense vs. MoE — MoE activates only a slice of total params per token (e.g. 671B total / ~37B active)
14	The VRAM formula	VRAM ≈ params × bytes/param (+ KV cache)	FP16 = 2 bytes/param, INT8 = 1, Q4 quantization = 0.5 — this is why Q4/GGUF is the default for local models
14	Local hardware picks	RTX 5090 (32GB) sweet spot; RTX PRO 6000 Blackwell (96GB) for 70B; H100/H200/B200 for frontier MoE	If (params × bytes/param × 1.2) ÷ your GPU’s VRAM > 1, you need multi-GPU — tensor parallelism (fast interconnect) or pipeline parallelism (slower-tolerant)
14	Serving many users	Continuous batching, PagedAttention/KV cache, speculative decoding	KV cache (grows with users × context), not weights, is usually the real bottleneck at scale
14	Serving tools	vLLM, SGLang, TensorRT-LLM (production) vs. Ollama, llama.cpp (single-user local)	vLLM = safest default; SGLang = fast agentic/tool-calling workloads; TensorRT-LLM = max NVIDIA-tuned throughput
14	Vector databases	pgvector → Qdrant/Milvus → Pinecone	Start pgvector if you already run Postgres; scale to Qdrant/Milvus; use Pinecone only to offload ops, not cost

This has been a long one. If you made it here, you now know more precise, currently-accurate AI vocabulary than most people writing “AI trend” listicles this year — and, more importantly, you know why each term exists, which is the part that doesn’t go stale in six months even when the model names do.

Last updated on 2026