There is no shortage of advice about AI industry. There is a shortage of advice that survives contact with a real week.
Choosing what to measure
There is a version of AI industry that is mostly ritual. If you learn on Friday what you assumed on Monday, the assumption never has time to become an architecture. Try writing the constraint on one line before opening a vendor comparison; the line is usually harder than the comparison.
Feedback loops shorter than the planning cycle change everything. Where a design is obvious the prose is short, so the length of an explanation is a reasonable proxy for where to look next. A useful test: if this disappeared tomorrow, how long before anyone noticed?
A shared definition of "done" removes more friction than any tool. The first quarter shows the intended effect; the second shows what the intended effect displaced.
Handing it over
The default answer is right often enough to be dangerous. Teams that pick both end up with neither, and usually discover this at the point where reversing would have mattered.
The expensive mistakes here are rarely the technical ones. Success has many causes and teaches very little; failure tends to have one, and it is usually obvious in hindsight. When we mapped it out, four of the seven steps existed only to compensate for the second one.
Documentation is a symptom: you write it where the design is unclear. They are decisions made quickly, defended slowly, and built upon for six months before anyone recalculates. The version of this that works fits on an index card. The version that fails needs an onboarding session.
Start with the constraints
Nobody gets credit for the work that did not need doing. Being right sixty per cent of the time builds exactly the kind of confidence that makes the other forty per cent expensive. It is worth saying that we have not run this long enough to be confident.
The tooling question is downstream of the constraint question. Handoffs between people who each hold a coherent local picture and no shared one produce most of the pain later attributed to tooling. That said, none of this generalises cleanly across team sizes.
Speed and reversibility are the trade-off worth naming out loud. Cutting scope early is cheap and slightly embarrassing; cutting it late is expensive and deeply embarrassing.
Where teams go wrong
The second-order effects arrive about a quarter after the first-order ones. The things that are easy to count are rarely the things that matter, and once a number reaches a dashboard it starts shaping behaviour whether or not it deserves to. The counter-argument deserves a hearing, and it is stronger than its usual proponents make it sound.
Measurement is usually where this falls apart. Subtraction is structurally underrated: the meeting that stopped happening leaves no artefact to point at in a review.
The first thing to establish is what you are actually optimising for. It is comfortable, it is legible to management, and it is close to worthless once you measure what it actually changes. We ran both approaches in parallel for six weeks. The difference was smaller than the cost of the debate about it.
The setup
Consider the failure mode rather than the success case. The stated constraint is usually a proxy for a real one nobody wants to say aloud, and optimising the proxy is wasted effort. The clearest signal was that people stopped asking where things were.
The compounding effects matter far more than the individual wins. Most disagreements that present as strategic turn out, on inspection, to be two people using one word for two things.
What we look for now:
- Prefer the reversible option when the evidence is thin
- Agree on what "done" means, in writing, before starting
- Review the numbers monthly; change the targets rarely
- Write the constraint down before choosing a tool
- Keep the feedback loop shorter than the planning cycle
When to change course
The interesting constraint is almost never the one in the brief. When responsibility is spread across a group, the work that falls between the named parts is the work that does not happen.
Consistency is worth more than any individual improvement to AI industry. A small improvement applied consistently beats a dramatic one applied once, which is unsatisfying advice precisely because it is correct. There are organisations where the opposite is true, and they are not obviously worse off.
Simplicity is not the absence of work. It is the result of it.
— Overheard in a retrospective
The first month
It helps to separate the decision from the execution. A team that changes approach every quarter pays a coordination tax that routinely exceeds whatever the change was meant to fix. This is easier to write than to hold to when a deadline appears.
What looks like a process problem is frequently an ownership problem. Choosing infrastructure before agreeing what it is for is how organisations end up maintaining a system nobody wanted. In practice the answer showed up in the calendar before it showed up in the dashboard.
Most of the difficulty lives at the boundaries, not in the middle. The decision is usually cheap and reversible; the execution is where the cost lives, and that is where the argument should have happened. Ask what would have to be true for the opposite approach to be correct, and see whether anyone can answer.
We will revisit this once we have another two quarters of data. The current answer feels right, which is exactly when it is worth checking.
Enjoying this story from Daily Insight?
“Get new deep dives delivered quietly to your inbox.”
Insight Daily Desk
Insight Daily Desk writes about AI industry.
