At OSS Ventures, we’ve spent the past few years creating AI-first companies and deploying them in real factories. Today, five of those products are live and running in production environments. They’re not demos, not prototypes, but systems that ship real work, every day, in dozens of industrial contexts.
If I had to distill one of the most important lessons we’ve learned from this journey, it’s this: when you build AI for operations, you are not just designing the workflow and its performance targets. You are designing for the human who supervises it. You are designing for what we’ve come to call the AI controller.
This role didn’t exist in our vocabulary when we began. At first, we thought we were simply automating tasks. But as deployments matured, it became clear that every AI system created a new layer of responsibility — someone who had to watch over, correct, and ultimately feel accountable for what the AI was doing. In factories, this role resembles the machine controller or the quality manager. In AI, it is the AI controller. And designing for that person is as important as designing the model itself.
The Emergence of the AI Controller
Think about what a controller does in a traditional factory setting. Their job is not to turn bolts or weld parts. It is to supervise systems, ensure that processes run as intended, and catch deviations before they propagate downstream. They translate high-level objectives into operating parameters and act as the last line of defense against defects.
The AI controller plays a similar part. Their work falls into two main responsibilities. First, they must “code the machine” — not literally by writing Python, but by deeply understanding what the AI system does, configuring it, and making sure it behaves in predictable ways. They need to feel like they are in control, not at the mercy of a probabilistic engine.
Second, they act as the “last-mile quality check.” They scan outputs for patterns, catch mistakes, and make sure nothing unacceptable reaches the next step of the process. Just as a human inspector won’t let a defective part move along the production line, the AI controller won’t let a hallucinated answer or a misclassified item slip through unnoticed.
In other words, the AI controller is not a passive user. They are a supervisor, a quality gatekeeper, and a translator between probabilistic systems and deterministic expectations.
Explainability Before Accuracy
One of the surprises in our deployments was how often user adoption correlated more with explainability than raw accuracy. In one of our companies, the AI model’s performance dipped — accuracy fell from 96% to 92%. By most metrics, that would be a regression. And yet, user adoption and satisfaction skyrocketed at exactly that moment. Why? Because we released explainability features.
For the first time, operators could see not just the output, but the reasoning behind it. They could peek under the hood, understand why a decision was made, and intervene when necessary. The AI no longer felt like a black box. It felt like a partner.
This reflects a deeper truth. People in factories don’t want magic. They want control. They want systems they can trust, debug, and improve. In manufacturing, tolerances define what is “good enough.” A part can deviate slightly from the target and still be acceptable, as long as the deviation is explainable and controlled. The same applies in AI: a slightly less accurate but explainable system can be far more valuable than a “perfect” but opaque one.
The Need for Depth, Breadth, and Control
Designing for the AI controller means building systems that operate on multiple levels. At the surface, the system must present a clear, human-readable explanation of what it’s doing. Strip away the math, keep the logic. “Here is what I saw. Here is what I decided. Here is why.”
Beneath that, the system must allow for deeper inspection. Logs, traces, and detailed views are essential. The controller must be able to dive down, explore the decision chain, and find where a problem originated. And equally important, they must be able to resurface quickly to the high-level view and make corrective changes.
The metaphor I keep returning to is the cockpit. Pilots don’t fly with one gauge. They have a whole panel of instruments, designed for situational awareness. Some are simple — fuel remaining, altitude. Some are complex — engine readouts, weather radar. The art of cockpit design is not throwing more dials at the pilot but organizing them so that the essential information is always available, and deeper detail is accessible when needed. The AI controller’s “cockpit” must do the same.
Sometimes Determinism Wins
In one of our AI-native companies, we started with a step-by-step guided workflow powered by AI. It seemed intuitive: let the AI walk the user through each decision. But over time, adoption began to plateau. We made a radical shift. Instead of having the AI guide every step, we let it propose a workflow and then let AI transform that workflow into deterministic code. In other words, AI proposed, humans approved, and the result became fixed.
The impact was immediate. Users trusted the system more, even though less “AI magic” was happening in real time. Why? Because they knew the final code was deterministic. They could read it, understand it, and rely on it.
This points to a counterintuitive lesson. Sometimes, the best way to build trust in a probabilistic system is to freeze its output into something deterministic. The AI becomes the ideation engine, but the final process is human-owned code. Variability is managed not by chasing the perfect model but by constraining it into something stable.
The next obvious step, that this particular company is in the process of, is having AI propose to better the code based on observations and new learnings.
Factories have a century of experience managing variance. They don’t eliminate it; they tame it. The same is true here. Sometimes the way to tame variance is to convert it into determinism at the right stage of the process.
The Business Stakes of Design
Why does all this matter? Because adoption lives or dies by the AI controller experience. You can build the most accurate, sophisticated AI model in the world, but if the operator doesn’t trust it, it won’t stick. If the controller can’t see what’s happening, can’t correct errors, or feels out of control, the system will be sidelined.
We’ve seen this repeatedly. AI demos are fun. They generate excitement. But when the pilot ends and the system must live day-to-day in a factory, the questions change. Managers ask about failure modes. Operators ask about edge cases. Controllers ask how to override the system when it goes wrong.
If those answers aren’t built into the design, adoption craters. If they are, adoption soars. ROI doesn’t come from model performance alone; it comes from user trust.
This is why designing for the AI controller is not a UX afterthought. It is the business-critical layer that determines whether your AI investment creates value or not.
Building Like Manufacturers
Factories have long known that the last step in any process is quality control. Every station checks its own work before passing it on. Defects are caught as early as possible, because letting them propagate downstream multiplies the cost.
The AI controller is the embodiment of that principle. They are the quality check in a probabilistic system. They don’t eliminate variability; they manage it. They don’t replace the machine; they ensure it runs within tolerances.
When we design for them, we are not just designing better user interfaces. We are designing systems that can be trusted in production. We are building AI the way manufacturers build factories: process-driven, variance-aware, quality-obsessed.
Future of Programmable Factories and Programmable AI need modern users
Factories are converging with computers. Machines act like CPUs, ERPs like operating systems, workflows like APIs. In that metaphor, the AI controller is the operator at the console — the one who supervises, tunes, and optimizes.
In the future, as factories become more programmable, AI will be the compiler and the optimizer. But compilers only work if programmers understand the code they generate. Optimizers only matter if you know what you’re optimizing for. Without clarity and supervision, power becomes chaos.
That is why the AI controller is not a transitional role. It is a permanent one. Just as factories will always need quality managers, AI systems will always need controllers. The form may evolve — more automation, more autonomy — but the principle will remain: someone must be responsible for making the system understandable, trustworthy, and accountable.
The skillset for this AI controller position is still fluctuating and everything is not clear. But some hints at the future exist : for example, that they’re system thinkers, love detail and big picture, and act decisively.
Closing Reflections
We are still learning. Every deployment teaches us something new. Sometimes accuracy drops but adoption rises, because explainability matters more than precision. Sometimes probabilistic models give way to deterministic code, because stability matters more than flexibility.
These are not failures. They are the exhilarating “aha” moments of building. They remind us that AI in operations is not about replacing humans. It is about designing new forms of collaboration between humans and machines. It is about giving operators not just tools, but instruments they can trust.
Designing for the AI controller means acknowledging that the human-in-the-loop is not a weakness. It is the strength that makes AI usable in the real world. And if history is any guide, the winners in this new era will be the ones who understand that — and build like manufacturers.