Why This Distinction Keeps Getting Blurred

People use "AI" to mean a lot of different things in the same breath — GPT-4 is "AI," ChatGPT is "AI," a company's customer support bot built on top of both is also "AI." But these are three genuinely different things, sitting at different layers …

Why This Matters

Almost every FastAPI app eventually needs something it shouldn't have hardcoded in the source code: a database URL, an LLM API key, a JWT secret, a third-party service token. When you're building GenAI apps, this shows up on day one, because you can't call an LLM API …

Why This Matters

If you've built even a basic FastAPI app, you've probably stumbled onto /docs and been pleasantly surprised — a fully interactive API documentation page, complete with every endpoint, request schema, and a "Try it out" button, without writing a single line of documentation yourself. This isn't a small …

The Question That Started This

I once asked the same question to the same model twice and got two noticeably different answers. My first thought was that something was broken. It wasn't. That variation is a feature, controlled by a couple of settings you've probably seen in every LLM API …

The Misconception I Had

When I first heard that an LLM can "call a function" or "use a tool," I pictured the model reaching out and running code: checking the weather, querying a database, sending an email. It sounded almost magical, and a little scary.

The reality is much simpler …

Why LLM Apps Need Different Observability

Traditional apps fail in predictable ways. A function throws an exception. A query times out. The error is binary: it worked or it didn't, and the stack trace points you to the line.

LLM apps fail softly. The API call succeeds. The response arrives …

What We're Building

Retrieval-Augmented Generation (RAG) is the pattern where an LLM answers questions using documents you provide, rather than relying only on what it learned during training. It's the foundation of most real-world AI apps: document Q&A, support bots, knowledge assistants.

This post builds a working RAG API …

The Question That Trips Up Every Model

Ask an LLM how many times the letter "r" appears in the word "strawberry" and there's a good chance it gets it wrong. Ask it to reverse a simple word and it fumbles. Count vowels in a sentence — wrong again. For a system …

A New Kind of Security Problem

When you build a web app, you learn early about SQL injection — the attack where user input slips out of its intended slot and starts being interpreted as a command. Prompt injection is the same idea, but for LLMs. User-supplied text escapes its role …

The Problem With Long System Prompts

If you've built anything with an LLM API, you've probably added a system prompt. At first it's a sentence or two. Then you add instructions. Then examples. Then context about your product. Before long, your system prompt is a few hundred lines and every …