The Honest Machine

The demo is simple: a photo of a broken thing goes in, and a grounded diagnosis pinned to the real object comes back, then a short ladder of visual steps, then, if it goes well, a thing in your house that works again.

But the demo isn't the interesting part. The interesting part is everything we did to make sure the app would rather admit it's unsure than hand you a confident, beautiful, wrong answer. Because the only metric that counts here isn't engagement, or time in app, or how slick the render looks. It's whether the faucet actually stops leaking. Bytes turned into atoms.

Three ideas are worth telling in full. Each is more interesting for what it refuses to do than for what it does.

We stopped asking the AI to guess where things are

Here's a thing that surprises people: a frontier vision model can describe your faucet in fluent paragraphs, but ask it for the exact pixel where the cartridge sits and it falls apart. Pointing, the skill of emitting raw coordinates, is one of its single worst abilities. It's genuinely good at seeing and genuinely bad at pointing.

So we flipped the problem. Instead of asking the model to invent coordinates, a small, fast detector looks at your photo first and burns a handful of numbered badges onto the real object, like a row of stickers. Then the analysis model does the one thing it's good at: it picks a number. Boring, deterministic code, not the model, looks up where sticker seven actually is and pins the circle there.

Don't ask the model to do the thing it's bad at. Ask it to do the thing it's good at, and let plain, boring, deterministic code do the rest.The spine of the whole story

The case of the broken glass

Every good engineering story has a bug that turns out to be a mirror. Ours is about a shattered wine glass. The app produced a perfectly reasonable cleanup plan, and then one step, "dispose of the glass safely," came back with an ugly empty placeholder.

The obvious read was that our image picker was broken. It wasn't. It was doing its job correctly, on garbage we were quietly feeding it. The component that scores how well a photo matches had been told to grade a real-world action, sweeping up shards, as if it were a "find me this exact intact part" task. Shown a perfect photo of glass on a floor, it rejected it. Correctly.

The fix didn't come from nudging a prompt and hoping. Someone sat down and read, the candidates, the reasons each was rejected, the actual inputs, and the diagnosis fell out of looking, not guessing. That's the difference between an AI product that improves and one that just shuffles its prompts forever.

The reranker was never broken. It was correctly rejecting garbage we were feeding it.Found by inspection, not by tuning the prompt

When the repair can kill you, the code decides

This is where the product stops looking like a clever consumer app and starts looking like something built by people who lie awake thinking about bodily harm. The dangerous failure here isn't a chatbot saying something embarrassing. It's an app cheerfully walking a first-timer into a gas line, a live panel, or a brake system.

You could try to handle that by asking the model nicely, in its prompt, to refuse dangerous repairs. We don't trust that. Prompts are suggestions. So a deterministic check runs over the model's output, completely independent of whatever the model was told. If the diagnosis touches a category that can hurt you, the check overrides the model's own "safe to DIY" flag and ships a wall that connects you to a professional. The model doesn't get a vote.

And the guard is itself guarded against over-refusing, tested in both directions: that it refuses the genuinely dangerous thing, and that it never starts turning away a perfectly safe washer swap. A safety system that refuses everything isn't safe. It's just useless, and a useless one gets switched off.

On a physical-harm product, the model is allowed to be wrong about danger. The code is not.The governing principle, stated plainly

An honest plain drawing beats a confident wrong photo

It's not enough for the plan to be safe. The picture has to be safe and true. So every visual in the system converges on a single gate: if an image can't be trusted, either because it doesn't faithfully depict the step or because it shows a hazard, it's simply deleted, nulled out before it ever loads. You never see the confident, beautiful, wrong render. You see something humbler and truer instead.

Treat the model as the component you constrain, not the one you trust. Build the deterministic floor first. Let the model raise the ceiling.Defense in depth, in two lines

Every guard is also a teacher

Here's the quiet business genius underneath all the safety machinery. A raw photo plus a query isn't proprietary. Google sees those too. The asset nobody else has is the labeled outcome: which image a real user accepted, which one drew a "not it, Jin," where exactly they tapped, and whether the repair actually finished.

Every guard, every abstention, every graceful downgrade was designed to write that signal down. So every honest "I'm not sure" isn't a dead end. It's a labeled example of exactly where the system reached its edge, which is exactly the data you need to push that edge out. The corpus stops being a log and starts paying compound interest.

The frontier isn't a smarter talker

Most of the AI conversation right now is about machines that talk. This is a machine pointed at the physical world, at the leaking faucet and the live panel and the floor full of glass, where confidence without grounding has consequences you can cut yourself on.

So the horizon we're building toward isn't a more impressive render. It's calibrated honesty: a real, math-backed sense of how sure the system is, independent of the model's own say-so. Teaching a machine not just to see, but to know what it doesn't know. On a product where a wrong answer meets a real hand and a real pipe, that's the whole game.

The interesting frontier isn't making the model more impressive. It's making it know what it doesn't know.The real horizon