The AI product lifecycle: building products with foundation models

If ML products taught us how to build systems that learn from data, AI products built on foundation models teach us something slightly different: how to build products by orchestrating existing intelligence, not training it from scratch.

This is where teams may get confused.

They take the ML lifecycle — discovery, data, modeling, deployment — and apply it directly to products built on GPTs, Claude, Gemini, or open-source LLMs.

That approach is not wrong.
But it’s incomplete.

Most foundation-model products still follow the ML lifecycle. What changes is where effort, risk, and leverage concentrate. Modeling is often compressed or outsourced behind an API, while system design, orchestration, evaluation, and cost control become the dominant product concerns.

The core ML fundamentals still apply. Teams that ignore data quality, evaluation, error analysis, and monitoring fail just as hard as before. But the locus of decision-making shifts away from training and toward how the model is used inside a broader product and process.

Foundation models change where product decisions live, which trade-offs matter most, and how PMs create leverage — not by eliminating ML thinking, but by rebalancing it.

In earlier posts on the ML Product Lifecycle and AI Product Strategy, the focus was on custom models: data feasibility, labeling, training, and drift. This builds on that foundation, while shifting the spotlight to what’s different when the “model” already exists.

This is the AI Product Lifecycle for foundation-model-based products.

‍

What changes when you build with foundation models

When you use foundation models, you are no longer primarily designing a model.
You are designing a product on top of the model.

Instead of asking:

Can we train this?

You ask:

How do we shape, constrain, adapt, and trust something that already knows a lot — but not exactly what we need?

That changes the lifecycle.

Discovery still matters.
Data still matters.
Deployment still matters.

But the center of gravity moves to:

• model selection
• adaptation strategy
• prompting and context
• cost and latency
• AI evals and iteration

Data doesn’t disappear. Instead, its role changes. Rather than being primarily used to train a model from scratch, data is increasingly used to shape behavior, ground outputs, improve reliability, and differentiate the product.

Let’s walk through it.

1. Discovery: Is a foundation model the right tool?

Discovery for AI products doesn’t start with models — it starts with user value and constraints.

But when foundation models are in play, there’s an extra layer of realism required.

You’re not asking:

“Can AI solve this?”

You’re asking:

“Can an existing model solve this well enough, safely enough, and cheaply enough?”

That’s a very different question.

At this stage, PMs should pressure-test:

• What job is the user trying to get done?
• What level of correctness, consistency, and explainability does that job require?
• Where would a model’s uncertainty actually be unacceptable?

For example, summarizing internal documents for faster decision-making is very different from generating customer-facing advice in healthcare or finance.

The output might look similar.
The tolerance for error is not.

Good AI discovery makes those constraints explicit before the system hardens — and validates them early through quick experiments with candidate models and APIs, not just prompt drafts.

2. Model selection: Capability, control, and cost

Model selection is not new. ML teams have always selected algorithms, architectures, and vendors.

What’s different in foundation-model-based products is what you are selecting — a pre-trained, general-purpose model with opaque internals — and what you implicitly accept along with it.

Choosing a model is no longer a downstream technical decision. It becomes a primary product and business decision.

PMs must help teams navigate trade-offs like:

• closed vs. open models, and what that implies for transparency and control
• text-only vs. multimodal capability, now and on the provider’s roadmap
• API convenience vs. infrastructure ownership, including portability risk
• fast iteration vs. long-term cost, driven by token economics
• data privacy vs. performance, especially for regulated domains
  
  

By selecting a foundation model, teams also inherit:

• opaque internal behavior they cannot fully inspect or debug,
• an external roadmap they do not control,
• pricing, policy, and availability risk set by the provider.
  
  

For example:

• If your product handles sensitive or regulated data, a self-hosted or open-source model may be necessary — not for accuracy, but for control.
  
• If your product lives or dies by latency, the “best” model on paper may be unusable in practice.
  
• If margins are thin, token economics will matter more than raw capability.
  
  

This is where many teams make an early mistake: optimizing for impressive demos instead of durable product economics and risk.

3. Adaptation: Prompting, fine-tuning, or retrieval?

Once a model is selected, the next decision is how much to adapt it.

This is where PM judgment matters most — because over-adapting is just as dangerous as under-adapting.

Prompting: shaping behavior through instructions

Prompting is the fastest and cheapest way to adapt a foundation model.

It’s also the most misunderstood.

Prompts don’t just ask questions — they define:

• tone and style
• structure and length
• constraints and safety rules
• what knowledge to activate

In product terms, prompts are part of your UX.

A weak prompt leads to unpredictable behavior.
A well-designed prompt creates consistency users can trust.

PMs should treat prompts like product features:

• version them
• test them
• evaluate them with explicit AI evals
• document what works and why

Prompting can scale to production — including long-lived systems — when prompts are modularized, paired with strong AI evals, and combined with retrieval or guardrails. Its real limits show up not at prototype scale, but under distribution shift, hidden coupling with context, and when hard guarantees are required.

Fine-tuning: teaching the model your domain

Fine-tuning goes deeper.

Instead of reminding the model what to do on every request, you bake desired behavior into the model using examples. This can improve reliability, reduce prompt complexity, and simplify downstream systems — but it also introduces additional operational and governance considerations.

Fine-tuning tends to make sense when:

• behavior must be consistent, not re-negotiated on every request (e.g. support tone, moderation policy)
• alignment matters more than raw domain knowledge
• prompt + RAG complexity becomes hard to manage
• you have sufficient labeled examples to encode behavior reliably

PMs don’t fine-tune models themselves — but they do decide whether the added complexity is justified by product needs, including evaluation, versioning, and long-term maintainability.

A common mistake is jumping to fine-tuning too early, when better prompting, retrieval, or guardrails would have solved the problem.

An equally costly mistake is avoiding fine-tuning when behavior truly needs to be baked in.

RAG: grounding models in your data

Retrieval-Augmented Generation (RAG) is a complementary technique, not a middle ground between prompting and fine-tuning. It can be combined with either.

Instead of retraining the model, RAG supplies relevant context at query time — documents, policies, knowledge bases, internal data.

RAG improves grounding and freshness, but it also shifts the main failure mode from generation to retrieval.

Key risks include:

• silent retrieval failures that look like confident model errors
• poor chunking or metadata degrading performance more than no RAG
• permissions bugs turning into security incidents
• freshness vs. consistency trade-offs that can erode trust
  
  

Because of this, RAG requires its own evals and monitoring, focused on retrieval quality and end-to-end correctness.

Many “LLM failures” in production are really retrieval and data-prep failures, not model issues.

4. Deployment: Where AI becomes a product

A foundation model is useless until it’s embedded in a user's experience (workflow).

Deployment is where PMs define:

• who sees AI output
• when it appears
• what action it triggers
• what happens when it’s wrong or uncertain

This is also where costs become real.

Long prompts and rich context feel cheap during experimentation — but inference cost is what kills products in production.

PMs must constantly balance:
quality ↔ cost
speed ↔ reliability
automation ↔ human oversight

Most AI products fail here — not because the model is bad, but because the integration doesn’t create real value.

5. Evaluation, monitoring, and learning

Foundation models don't drift from data distribution changes in the same way custom models do — but their behavior still changes in production.

This happens when:

• providers update models
• prompts, context, or sampling settings change
• input distributions shift
• user behavior and goals evolve
  
  

That’s why continuous evaluation is required in production.

PMs should ensure evaluation covers:

• task success and regressions
• safety and alignment, especially for customer-facing products
• latency and cost
• behavior under failure (API errors, missing retrieval, fallbacks to humans)

Teams should also plan for fallback strategies and escalation paths when systems fail.

AI products don’t improve by accident.
They improve through ongoing observation, evaluation, and adjustment.

The real shift: From models to systems

The biggest lesson of the AI Product Lifecycle is this:
AI is rarely just a stand alone product.
It’s a component in a larger system.

The best AI PMs don’t obsess over models.
They design learning systems that balance user value, business constraints, cost and reliability, safety and trust. 

Foundation models give teams incredible leverage — but only if that leverage is used intentionally.

Building an AI product? Let’s get the hard decisions right

If the ML Product Lifecycle taught PMs how to work with data and models, the AI Product Lifecycle teaches PMs how to work with capability, context, and constraint.

At Chovik, this is exactly where we help teams: turning powerful AI capabilities into products that are usable, scalable, and defensible.

If you’re building an AI product with foundation models and want help making the right decisions early, let’s talk.

‍