Putting AI into a legacy system without it falling over
10 June 2026 · 5 min read · Daniyal Malik
The hardest GenAI work I do usually isn't the AI. It's the legacy system it has to live inside. Greenfield demos are easy. Putting a model into a ten-year-old enterprise stack — where the data lives in a SQL Server nobody fully understands and the frontend talks to a CMS from 2014 — is where most "AI initiatives" quietly die.
Where the model actually struggles
Legacy systems break the assumptions that RAG demos quietly rely on:
- The knowledge is in PDFs, scanned documents, and people's heads — not a clean vector store.
- The data is inconsistent — the same field means three different things in three different tables.
- There's no API for half of what you need; it's a database view, a nightly export, or a screen.
When this goes wrong, the model didn't fail. The integration did.
How I ground it
At BUPA, I connected a modern React/TypeScript frontend to a legacy .NET Core + Sitecore backend through a BFF (backend-for-frontend) — a thin layer that speaks the new system's language on one side and the old system's on the other. Putting AI into legacy is the same move: an anti-corruption layer that turns messy legacy data into clean, structured, retrievable knowledge, so the model never touches the mess directly.
- Ingest and structure with tools like AWS Glue or LlamaParse — parse the PDFs, normalise the fields, and verify against ground truth before anything reaches retrieval.
- Retrieve, don't fine-tune — legacy knowledge changes constantly; retrieval keeps it current without a training loop.
- Keep the rules in code — the legacy system has compliance and entitlement logic the model must never break, so that stays as deterministic guardrails, not as hopeful prompting.
- Human-in-the-loop and Step Functions for the workflows where a wrong answer carries real cost.
The mindset
You don't bolt AI onto a legacy system. You build a clean boundary the AI can trust, and you do the boring data work behind it. The teams that win treat the integration as the project and the model as a component — not the other way around.
This is most of what production GenAI actually is. If that's the wall you're hitting, let's talk through it.