The Machine Did Exactly What We Asked Of It

A while ago, someone on my team had a great line for when some tool went nuts:

The machine did exactly what we asked of it.

It's funny for about half a second, because the natural response is to immediately blame the machine. Look at this (wrong) thing. How did it go in this direction? Why emphasize that strange part? How did it interpret that sentence... Oh.

The line was born in the more deterministic setting, when some if/then logic smashed against some edge case we hadn't ever thought possible. Today, the line's all grown up and has become increasingly useful whenever an AI agent returns something confident and beautiful, but wrong.

You go back and read what you actually asked it for. Well that's often a deeply humbling exercise because the typical case is that the machine hasn't really done something insane. You gave it some initial ambiguity, only now barely noticeable. The machine took that ambiguity and ran, making choice after choice about it. Proceeding with unbridled confidence that would often be shocking in the real world, like a runaway intern, extremely talented but who doesn't bother with small-minded questions.

We all think we're clear communicators. But human beings cover for each other, have been covering all along. I'm pretty sure of that being true based on all my years of successfully being alive.

We take whatever is on the docket today and stitch it together with the last five similar conversations, small contradictions, and our EQ about all the parties involved. We might ask a question, but mostly we'll just get on with it.

How are AI agents supposed to work with that?

Is the machine sitting there deciding whether I deserve the benefit of the doubt? Not in the way I mean the words.

Claude, tell me what's going on, make no mistakes: "The fault, dear Brutus, is not in our stars, but in ourselves."

Human collaboration depends on EQ, of course, but also something more akin to "human repair."^[1]

Working together is a mess in the best circumstances, despite whatever we might think. People are continuously smoothing over all of this kind of stuff, reconstructing intent from incomplete instructions and missing context. The bumps in the road are endless.

Somebody talks or sends you something. What kind of puzzle is this? Well, it's a thousand-piece puzzle. Except the box is half empty, but, oh hey, look here are the rest of the pieces. They're in this drawer (our brains): some of what happened yesterday, a little bit of what this person typically cares about, the tone they used. Everything else we know about everything.

We put all the pieces on the table, move them around for a while, and marvel: it's perfect.

Or close enough. It's 997 pieces but that's still pretty good.

Puzzle box reassembly is such a normal course of human activity that I'd mostly stopped seeing it. Until AI agents made it extremely, sometimes painfully, visible again. That drawer with the missing pieces? The right one is hidden from our LLM helpers, and now they're checking all the world's drawers real quick.

Which means a lot of what I've casually thought of as "clear communication" was actually more of a joint effort. Do any of us actively realize this? I'm going to say no and just skate past whether I'm out on a limb. Because I know now that I've often said something that turned out to be incomplete, because the machines keep showing me that I do this. So many times before, somebody else opened their brain drawer to supply the missing pieces, and ignored the pieces I sent that were from another puzzle. They did all that, disappeared for a while, and handed the whole thing back to me in the right shape. Yep, that was a Golden Gate Bridge puzzle, thank you, I see the bridge.

Job well done everyone.

The natural conclusion was "hey now we have a bridge" and so certainly I'd explained myself well. Not so sure about that anymore.

On certain things they set out to do, agents are both capable (sometimes exceptionally so) and weirdly literal (in highly unpredictable ways). Like people. But the ways they infer what you might mean are their own, not ours but kind of ours, sort of, even if how they even got where they got is often mysterious to us. Still, if the puzzle is the Golden Gate, why do they give it back with...wings? Probably because you handed over half a puzzle, confidently assuming the other side will know which drawer to open, but...oh god, what's that?

So you stare at what you gave it, channeling your best beginner's mind. Sound it out, read it slow.

Sigh, I can see how you'd add the wings. I sort of like the wings.

The hypothetical wings are a nice poke to the ego. They make all the invisible human repair work so so obvious. Front and center.

How many times has somebody handed me back a perfectly normal Golden Gate Bridge? A lot. And how many times had I given them mostly bridge pieces, but a few things from an airplane puzzle?

A lot. Probably?

The other side of all those transactions remembered some earlier conversation about the Golden Gate and ignored the jet pieces. Halfway through they just made a judgment about what a sane person would have meant. At the end, the bridge looks like a bridge. We move on.

Whereas the machine occasionally presents you with a bridge-plane, which yeah is annoying.

"Do you like this?" No. But I can see how it did exactly what I asked.