Your AI vendor might be a concept car
What matters isn't what they built; it's whether they'll still be building it with you in a year.
A concept car is engineered to be seen, not driven.
On the rotating platform, it looks more advanced than anything on the road: doors lifting on cue, dashboard glowing under perfect light, beautiful from every angle. Underneath, it may have no drivetrain, no crash structure, no serviceability, and none of the economics required to put 10,000 of them on a highway. Nobody at the show is lying because the car did exactly what it was built to do. The trouble starts when someone mistakes it for the finished product.
AI prototypes have started to work the same way. A few years ago, they looked like prototypes, rough enough to remind everyone in the room how much engineering was left to do. That visual warning label is gone. Now a demo can look complete enough to be mistaken for a product, when all it has proven is that an idea can work.
The same test should apply to the teams offering to build your product. An engineering partner can be a concept car too.
TL;DR | |
A slick demo proves an idea works. It doesn't prove a vendor can turn that idea into software you'll still be running in three years. | |
Judge them on how they disagree with you, how they think about systems (not just features), where they refuse to use AI, and whether they hand you the keys when it's done. |
The demo is one dimension of the work
Our engineering teams know that software rarely lives where people think it does. It does not live on the screen you demo. It lives underneath: how data moves, what happens when an integration fails, how permissions work, how the system behaves with malformed inputs, whether anyone can reconstruct what happened after something goes wrong, whether the architecture survives ten users becoming ten thousand.
The interface is the part you can point at. The rest is the product.
AI has compressed that visible part dramatically. A small team can now produce in days an experience that once took months, and that is real progress. It is also an optical illusion: the first 20% of the work has started to look like 80% of the product. The demo is one dimension of the job, the one built to be seen.
The rest of product development is the discipline of getting you to your next milestone predictably, and that is where evaluating a partner on whether they "do AI" tells you almost nothing.
Commercial software still has to function as a system. That means answers to questions like:
Where does the data come from, and where does it go?
What happens when the model is uncertain, the user disagrees with it, or the work moves off the happy path?
How does the product get deployed, monitored, supported, upgraded, debugged?
And, most importantly, which parts should use AI at all?
These questions are less exciting than the demo. They are also much closer to whether you have a product. We learned that sharply in healthcare, where thin answers about data, security, integrations, auditability, and workflow surface fast.
But healthcare is only an extreme version of a broader truth. Every serious product eventually meets the road. The question is whether the people building it designed for that road or only for the showroom.
How to recognize a concept-car vendor
I spent years on the buyer side of this. As a senior engineering leader at Philips and Arcadia, I made outsourcing decisions, stood up offshore teams, hired vendors, and lived with the consequences.
They disagree with you.
Strong opinions, loosely held. A good partner argues a position hard and drops it the moment the evidence changes, because they optimize for the product, not the comfort of the current meeting. Watch what happens when you float a shaky idea: a concept-car vendor nods; a real one says, "that will cost more than you think, and here is the assumption I would test first." The nod is cheaper today and far more expensive later.
They think in systems
You cannot build in a space you do not understand, so a strong partner maps the space before the feature. They ask about the systems your product will touch, the data moving through it, the roles, the failure modes, the handoffs. Ask a concept-car vendor how the feature works, and they will tell you for an hour. Ask a real one the same question, and they will start asking you questions back.
They know AI is not for everything
"We do AI" is not a differentiator. The tell is whether a team can name where AI does not belong. A partner who reaches for a model to validate a date field, where a rule is cheaper, faster, and testable, is selling you the platform under the lights. A partner who says "that part should be deterministic, save the model for where it earns its cost" is building you a product.
They hand you the keys and the map
A product you own should not go dark the day the vendor leaves. The keys are the repository, the cloud account, the credentials, the domain. The map is the documentation that lets another competent team pick the work up. A concept-car vendor leaves you something that only runs on their track and only they can drive. A real partner makes ownership explicit from the start and writes down how it works before you ask.
The breadth is the capability
Most competent engineering teams can build a feature. The better question is whether they understand how the puzzle comes together to deliver a business benefit
Can they move from discovery to architecture, from architecture to implementation, from the happy path to the edge cases, from a standalone feature to the systems around it, from a handful of users to production load, from launch day to the unglamorous reality of operating software for years after? And can they make good product decisions the entire way?
That is a much higher bar than producing an impressive prototype. It is also what customers are actually buying.
Look past the platform
The concept car is useful. It can show a direction before anyone spends the money to manufacture it. It can create excitement. It can help people see something that did not exist before. AI prototypes do the same thing, and the mistake is not building them. The mistake is forgetting what they prove.
A prototype can tell you an idea deserves another step. It cannot tell you that every step after it has already been solved. A polished vendor presentation can tell you a team knows how to present its capabilities. It cannot tell you whether they have the breadth to turn an idea into a product you can actually own, operate, and grow.
The bottom line
Before you sign anything, run the team through the same test:
Watch how they handle disagreement, not just how they answer questions.
Ask where they think AI does not belong in the build.
Check who owns the repository, the credentials, and the domain once the engagement ends.
Judge the breadth of the work they can carry, not the polish of the demo they showed you.
The car under the lights will always look impressive. Find out who can actually get it onto the road.
FAQ
What's the difference between an AI prototype and an AI product?
A prototype shows an idea can work. A product has to keep working, with real data, real load, real failures, and someone accountable for fixing what breaks.
Why do AI demos look so much more finished than they used to?
AI has compressed the visible layer of software, the interface, dramatically. A small team can build something that looks finished in days.
That compression hasn't touched the invisible layer: data, permissions, failure handling, deployment, and monitoring. That's still slow, still hard, and still where most of the real work sits.
What should I ask an AI vendor before signing a contract?
Ask how they'd handle a bad idea you proposed, where they think AI does not belong in the build, who owns the repository and credentials once the engagement ends, and how the system behaves when something goes wrong rather than only when it goes right.
Why does this matter more in healthcare software?
Thin answers about data handling, security, integrations, and auditability surface fast in healthcare, where the cost of getting it wrong is high, and the scrutiny is built into the industry. It's an extreme version of a truth that applies everywhere: every product eventually meets the road.
Isn't a working prototype proof that a vendor can deliver?
It's proof the idea works. Carrying it through architecture, edge cases, production load, and years of operation is a different skill, and it's the one that actually matters. Those are different skills, and the gap between them is where most AI initiatives stall.
Author
Geo Marquez
SVP, Growth
Share post
