Why This Distinction Keeps Getting Blurred

People use "AI" to mean a lot of different things in the same breath — GPT-4 is "AI," ChatGPT is "AI," a company's customer support bot built on top of both is also "AI." But these are three genuinely different things, sitting at different layers …

Why This Matters

Almost every FastAPI app eventually needs something it shouldn't have hardcoded in the source code: a database URL, an LLM API key, a JWT secret, a third-party service token. When you're building GenAI apps, this shows up on day one, because you can't call an LLM API …

Why This Matters

If you've built even a basic FastAPI app, you've probably stumbled onto /docs and been pleasantly surprised — a fully interactive API documentation page, complete with every endpoint, request schema, and a "Try it out" button, without writing a single line of documentation yourself. This isn't a small …

The Question That Started This

I once asked the same question to the same model twice and got two noticeably different answers. My first thought was that something was broken. It wasn't. That variation is a feature, controlled by a couple of settings you've probably seen in every LLM API …

The Misconception I Had

When I first heard that an LLM can "call a function" or "use a tool," I pictured the model reaching out and running code: checking the weather, querying a database, sending an email. It sounded almost magical, and a little scary.

The reality is much simpler …

The Problem With Long System Prompts

If you've built anything with an LLM API, you've probably added a system prompt. At first it's a sentence or two. Then you add instructions. Then examples. Then context about your product. Before long, your system prompt is a few hundred lines and every …

The Day Nobody Knew Why It Got Worse

Here's a scenario that happens on almost every LLM project eventually. The app was giving good answers on Monday. By Thursday, users are complaining that responses feel off. Someone says, "I think I tweaked the prompt a bit last week." Someone else …

The Bug That Taught Me This

The first time I built something that fed LLM output into other code, it worked beautifully for about a day. Then one response came back as:

Sure! Here's the JSON you asked for:

{"category": "billing", "priority": 2}

Let me know if you need …

The Question I Couldn't Answer

For a long time, my process for improving a prompt was embarrassingly simple: change the wording, try two or three inputs, decide it "feels better," and move on. Then one day someone asked me a basic question: "Is the new version actually better than the …

The Moment It Clicked

For a long time, "using an LLM" meant one thing for me: sign up, paste an API key, send a request to someone else's server, and watch the token counter tick up. Then I tried running a model on my own laptop, with the Wi-Fi turned …