Prompt reliability
Practical habits for getting less brittle model behavior before adding heavier tooling.
6 notes
Start with Keep the system prompt short enough to auditPlatPhormNews field notebook
AI insights for humans
A practical notebook for understanding model behavior: prompting habits, interpretability methods, verification rituals, and tools that make AI systems less mysterious.
Aha lab
Watch the idea move before you read the rule.
Each animation is a deterministic concept trace, not a live model run. The goal is to make the hidden review move visible enough to use.
Uncertainty trace
The sentence is fluent. The token path may not be.
This toy trace turns a polished answer into token confidence. The cold tokens are where a reviewer should slow down first.
Aha
A low-margin token is where the model nearly chose a different world.
Thedecoded
paperdecoded
wasdecoded
publisheddecoded
indecoded
2024decoded
bydecoded
OpenAIdecoded
The generated text looks continuous, but each token still carries a margin against alternatives.
Operator move
Read names, dates, citations, and refusal-boundary words with the token margin visible.
Notebook state
The corpus is small enough to read, structured enough to operate.
17
published notes
6
topic tags
17
aha moments
3
reading paths
Tip6
Method9
Tool2
Reading paths
Three ways through the notebook
Interpretability fieldwork
Ways to inspect internal signals and move from pretty explanations to causal evidence.
6 notes
Start with Read the logprobs, not the answerVerification discipline
Small rituals that keep AI-assisted work tied to evidence, tests, and explicit uncertainty.
4 notes
Start with Turn every important claim into a receiptEntries
The working notebook
17 of 17 notes visible
- Method2026 · 06 · 18
01Ask for counter-evidence before confidence
A model that only explains why its answer is right will happily polish a mistake. Ask it to list the strongest evidence against its answer before it scores confidence. The missing objection is often the thing that saves you.
Confidence after counter-evidence is more useful than confidence after a sales pitch.
#reliability
Aha momentThe model is usually best at self-correction before it has been rewarded for sounding certain.
Try this: Ask for the answer, the strongest objection, what evidence would change the answer, and then confidence.
- Method2026 · 05 · 27
02Turn every important claim into a receipt
For high-stakes answers, make the model attach each claim to a source, calculation, line number, trace id, or explicit assumption. Unsupported claims do not disappear, but they become visible enough to reject.
#verification
Aha momentA citation is not decoration. It is a handle you can pull to see whether the sentence is load-bearing.
Try this: Make every claim end with one receipt: source, row, command, calculation, trace id, or explicit assumption.
- Tip2026 · 05 · 03
03Keep the system prompt short enough to audit
A giant instruction stack feels powerful until nobody can tell which rule caused the behavior. Keep the non-negotiables small, move task detail into the user prompt, and test the system prompt like code.
If you cannot diff it cleanly, you cannot debug it cleanly.
#prompting
Aha momentA system prompt is product code. If it is too long to review, it is too long to trust.
Try this: Rewrite the system prompt into role, hard boundary, output contract, and escalation rule.
- Method2026 · 04 · 12
04Use refusal boundaries as test fixtures
Do not only test the happy path. Save examples that should be refused, partially answered, or redirected, then run them every time you change instructions. Safety behavior drifts just like product behavior.
#evals
Aha momentSafety quality is not whether the model refuses. It is whether it refuses at the right boundary.
Try this: Keep allowed, disallowed, and allowed-with-constraints prompts as fixtures and run them after prompt edits.
- Method2025 · 11 · 02
05Read the logprobs, not the answer
The text a model gives you is the tip of the iceberg. Ask the API for token logprobs and you can see where it was confident versus where it basically flipped a coin. A fluent sentence built on 51% tokens is a guess wearing a suit.
Confidence ≠ correctness. But low confidence is a great place to look for hallucinations.
#interpretability
Aha momentThe final text is only the winning token path. The uncertainty is still visible in the runners-up.
Try this: Render low-margin tokens first, especially names, dates, citations, and first tokens after a policy boundary.
- Tip2025 · 10 · 28
06Make it think before it answers
Put the reasoning before the conclusion, never after. If you ask for the answer first and the explanation second, the explanation is just a story it invents to justify a choice it already made. Order matters more than wording.
#prompting
Aha momentExplanation after an answer often explains a commitment; reasoning before an answer can still change it.
Try this: Put scratch work, constraints, and uncertainty checks before the final answer field.
- Tool2025 · 10 · 19
07BertViz for watching attention move
An open-source visualizer that shows which tokens each attention head is looking at, layer by layer. You start to notice heads that only track quotation marks, or ones that follow subject-verb agreement across a whole paragraph.
Most heads are boring. The interesting 5% are worth the dig.
#tooling
Aha momentAttention is not the whole mind, but it is a useful map of what information is being routed where.
Try this: Inspect heads for repeated roles: bracket matching, quote tracking, subject agreement, or copied labels.
- Tip2025 · 10 · 11
08Tell it what NOT to do, sparingly
Negative instructions are weak anchors. 'Don't mention X' often makes X more likely because you just put X in the context. Prefer describing the thing you DO want. Reserve negatives for hard safety rails, not style.
#prompting
Aha momentNegation still puts the forbidden idea into context. Attention does not forget because grammar says not to.
Try this: Rewrite prohibitions as the positive shape of the output you want before adding hard safety rails.
- Method2025 · 09 · 30
09Probe for a concept with a linear classifier
Freeze the model, grab the hidden activations for a pile of examples, and train a tiny linear probe to predict some property (is this text about money? is it past tense?). If a simple line separates them, the concept is already represented in there, plain as day.
Cheap, fast, and weirdly revealing.
#interpretability
Aha momentIf a tiny linear classifier can read a concept from hidden states, the model has already organized that signal.
Try this: Probe one concept across layers and look for where accuracy first rises above a simple baseline.
- Tip2025 · 09 · 22
10Temperature is a creativity dial, not a quality dial
Turning it up doesn't make answers smarter, it makes them less predictable. For extraction, classification, and anything with a right answer, keep it near zero. For brainstorming, let it wander. People mix these up constantly.
#prompting
Aha momentTemperature changes variance, not intelligence. It widens the lottery; it does not improve the tickets.
Try this: Set temperature near zero for extraction and scoring, then raise it only when variety is the product.
- Method2025 · 09 · 14
11Sparse autoencoders to untangle neurons
Individual neurons are polysemantic, one neuron fires for 'the Golden Gate Bridge' and also 'feeling trapped' and also semicolons. Train a sparse autoencoder on the activations and you pull those mixed signals apart into cleaner, single-meaning features.
This is roughly how the 'Golden Gate Claude' demo worked.
#interpretability
Aha momentSparse features give you named-ish knobs because they separate mixed neuron behavior into cleaner directions.
Try this: Treat an SAE feature as a hypothesis, then test whether steering it changes the behavior you care about.
- Tool2025 · 09 · 03
12Keep a tokenizer open in a tab
Half of weird model behavior is a tokenization story. ' apple' and 'apple' are different tokens. Numbers split in unintuitive ways. Pasting your prompt into a tokenizer viewer takes ten seconds and explains a surprising number of bugs.
#tooling
Aha momentThe model never sees your characters. It sees chunks, spaces, merges, and weird number splits.
Try this: Paste the exact prompt, including spaces and separators, into a tokenizer before debugging prompt behavior.
- Method2025 · 08 · 25
13Sample many, keep the majority
For reasoning tasks, generate the same answer five or ten times at moderate temperature and take the most common result. A single greedy answer is fragile; the consensus across samples is far more reliable. Slow, but it works.
#reliability
Aha momentWrong reasoning paths tend to scatter; correct reasoning paths often converge.
Try this: Sample several answers, normalize the final claim, and inspect disagreement before majority voting.
- Tip2025 · 08 · 12
14Mind the middle of long contexts
Models attend strongly to the beginning and end of a long prompt and get foggy in the middle. Put the instructions and the most important facts at the edges. If something keeps getting ignored, it's probably buried in the soft center.
Sometimes called 'lost in the middle'.
#prompting
Aha momentA fact can be present and still practically invisible if it sits in the soft middle of a long prompt.
Try this: Move critical constraints to the opening contract and repeat the single highest-risk fact at the end.
- Method2025 · 07 · 30
15Activation patching to find the cause
Run the model twice, once on a clean prompt and once on a corrupted one, then copy a single activation from one run into the other. If the output flips, you just found a component that carries the behavior. It's causal, not just correlational.
The closest thing in this field to a controlled experiment.
#interpretability
Aha momentCorrelation says a signal is present. Patching asks whether changing that signal changes the outcome.
Try this: Patch one layer-position activation from a clean run into a corrupted run and track whether the answer flips.
- Tip2025 · 07 · 18
16Your examples teach format louder than content
In few-shot prompts the model copies the shape of your examples first: the punctuation, the casing, the length, the label words. Get the format dead consistent and the task often solves itself. Sloppy examples teach sloppiness.
#prompting
Aha momentFew-shot examples teach surface form first. The model copies the pattern before it understands your intention.
Try this: Make labels, punctuation, casing, separators, and example length boringly consistent.
- Method2025 · 07 · 05
17The logit lens: decode the middle layers
Take the hidden state from an intermediate layer and run it through the model's output head early. You get the model's 'best guess so far' at that depth. Watching the prediction sharpen layer by layer is like a time-lapse of a thought forming.
#interpretability
Aha momentThe answer often appears before the wording is polished. Middle layers can show the thought taking shape.
Try this: Decode intermediate layers and watch when the target answer first becomes linearly readable.