Everyone assumes the AI orchestration was the hard part of building a security scanning platform.

It was not.

The AI pipeline (that is, vulnerability assessment and remediation research) worked relatively quickly. Straightforward.

The part that required three full rebuilds was deployment and usability: making the platform work reliably in constrained customer environments without demanding specialized configuration from users.

The system

The platform orchestrated established open-source security tools such as OpenVAS and Nmap, consumed their output, and produced reports.

The AI reporting layer was the simplest part. Deployment and usability were where the complexity lived.

Why this was hard

Network scanning sounds simple until you try to make it work for someone who is not a network engineer.

We were asking users to initiate vulnerability scans in constrained environments without understanding the underlying infrastructure. Early approaches required too much setup and specialized knowledge.

We could not design our way through this from a whiteboard. We needed to iterate against real user constraints.

The four iterations

V1: I assumed AI could carry me through unfamiliar territory

V1 was built in early 2025 when AI development tooling was just getting popular. I was tasked with evaluating these tools and standardizing them for the rest of the company.

For the first couple of months, I leaned too heavily on AI to generate code in a domain I did not understand. I did not have the networking and security expertise to make informed architecture decisions. The AI was confidently producing output I could not evaluate.

I got a working prototype together, but it was not going to scale. I had a demo I could not debug. When something broke, I did not know enough about the scanning stack to figure out why, so every fix was a guess. The architecture was shaky because my understanding of the underlying systems was shaky.

V2: I assumed a better tech stack and deeper knowledge would solve the deployment problem

I stepped back and used AI to accelerate learning instead of skipping it. I built many small prototypes — harness projects for individual pieces of the system — until I understood the full picture before integrating anything.

I also simplified the tech stack. That made development faster and more reliable, but it did not solve the deployment problem.

With the stack solid, the real question became: how do you make deployment work reliably in a constrained customer environment?

V3: I assumed I could shift more of the work into the customer environment

I next tried shifting more of the workload into the customer environment. It worked in test conditions, but it was too intrusive and resource-intensive for practical deployment.

V4: I assumed we needed to reduce customer-side configuration

We connected with a field expert on security scanning who had solved a similar problem a decade earlier working as an MSP.

His suggestion helped reframe the problem: keep more orchestration in managed infrastructure and reduce the amount of customer-side network configuration.

That was the approach we needed: keep more complexity in managed infrastructure and minimize customer-side setup.

This version brought the platform to alpha.

What I got wrong

The iterations themselves were not the mistake. Iterating on a hard UX problem is how product development works.

First, I did not spend enough time learning the subject matter before building. I should have used AI to accelerate my learning from the start and built small prototypes for each piece of the system instead of trying to assemble the whole thing at once.

Second, I did not bring in real users early enough. Several approaches took meaningful time and failed as soon as we tested basic assumptions. Earlier conversations would have exposed practical barriers before I built around them.

The pattern is simple: validate the riskiest assumption with the cheapest possible test before building the infrastructure around it.

When to do something different

The complexity came from making a specialized workflow usable without requiring users to understand the underlying infrastructure. A more technical audience may tolerate more setup and direct access to raw output, which changes the amount of product and UX work required.

If you have a team and a designer, you can also parallelize the UX work with the infrastructure work. I was doing both sequentially as a solo CTO, which meant every UX dead end also stalled the infrastructure progress.

And if you are building any product that touches client networks or client machines, talk to your users before you build the deployment model. The networking constraints are almost never what you assume they are from the outside.

What’s next

Next up: why free-text handoffs between AI agents kill your pipeline, and what to use instead.

If you want that when it goes live, sign up below for alerts.