

By Trucision staff
Key takeaways: AI agents in a lab are only as reliable as what they remember, what they're allowed to see, and how they're tested. If memory lives inside a vendor's tool, context is whatever the prompt happened to include, and testing means a generic benchmark, you can't defend the output to an auditor. Your lab needs three things it controls: one shared memory where every write is approved and traced, an eval set built from your own closed records, and context limited to the right systems and the right SOP revision.
Most labs are starting to use AI in small, separate ways. Staff ask ChatGPT or Microsoft Copilot about an SOP. A vendor's assistant sits inside the instrument software. Someone wrote a script to summarize deviations. Each one works well enough on its own.
The trouble starts when you ask a simple question: what does the AI actually know about our lab, and who decided that? In most labs, nobody can answer it.
When each tool keeps its own memory, the same question gets different answers. One assistant still cites last year's SOP revision. Another picked up a workaround an analyst typed in once and now repeats it as if it were procedure. A third knows nothing about the calibration decision QA made last month.
None of this shows up as an error. It shows up as small inconsistencies that someone eventually has to untangle, usually during an investigation or an audit.
In a regulated lab, that's not a convenience problem. A fact an AI relies on is part of how a record was produced. If you can't say where it came from, who approved it and which version was in use, you can't defend the record.
Lab memory is the set of facts your AI agents are allowed to rely on across tasks. Typical examples:
These are facts a good senior analyst carries in their head. The difference is that an agent's memory can be written down, versioned and checked.
The safest pattern for shared lab memory is simple, and it mirrors how you already control documents:
This gives you what ALCOA+ asks of any record: attributable, original, accurate, and available when an inspector asks how a conclusion was reached. It also means a fact can be retired cleanly when an SOP changes, rather than lingering in some tool's history.
AI vendors quote benchmark scores. Those scores tell you how a model does on someone else's questions. They don't tell you how it handles your methods, your instruments, your naming conventions or the way your QA team writes a deviation.
Small changes can move results a lot, and you can see this for yourself. Take ten closed deviations and ask an assistant to classify each one. Then ask again with the categories listed in a different order, or with a few extra options added. If the answers shift, a generic benchmark wasn't going to warn you. Only testing on your own records will.
A lab eval set is a collection of your own closed, reviewed records, with known correct outcomes:
Before anything changes, whether that's a new model version, a new prompt or a new batch of memory, you replay the eval set and compare. If the agent gets worse on your records, the change doesn't ship. Reviewer edits and rejections from live work feed back into the eval set, not silently into the model, so it gets sharper over time.
This is the same logic as qualifying an instrument. You don't accept the manufacturer's spec sheet as proof it works in your lab. You test it on your own samples.
Context is everything an agent sees when it does a task. Too little, and it guesses. Too much, and it picks up the wrong revision, an unrelated record or data it shouldn't have.
Controlled context means:
Keep them. Staff like them, and they're good at general work. The point is that the lab's memory, eval set and context rules shouldn't live inside a general assistant's chat history. They should live in a system the lab controls, and the assistant should ask that system when it needs a lab answer. Standards such as MCP (Model Context Protocol) make that connection possible: the assistant calls a lab tool, and the tool answers from approved memory, with sources.
If a vendor can't answer these clearly, the AI may still be useful for drafts, but it isn't ready to sit inside your quality system.
This is the model Trucision is building toward: agents propose, people approve, and every agent reads from one versioned lab memory that traces back to the LIMS result, instrument log or SOP revision behind each fact. Changes are tested against the lab's own closed records before they go live. Trucision manages that lab operations layer end to end, and assistants such as ChatGPT and Copilot can connect to it as a tool.
A collection of a lab's own closed and reviewed records, such as deviations, CAPAs and OOS investigations, with known correct outcomes. It is used to test an AI agent before go-live and before any change to the model, prompt or memory.
Because the lab can't see, approve or version what the AI relies on. In a regulated lab, a fact used to produce a record needs a source, an approver and a version, just like a controlled document.
No. In a controlled design, agents only propose new facts. A qualified person approves each write, and reviewer feedback goes into the eval set rather than silently changing the model.
Yes. Connect them to a lab-controlled system through a standard such as MCP, so lab answers come from approved memory and your records rather than from the assistant's own chat history.
A step-by-step guide to building your first eval set from 25 closed records: which records to pick, how to label the correct outcome, and a simple scoring sheet to compare any AI tool, including ChatGPT and Copilot. Send me the starter kit.
Bring one closed deviation and we'll show you what an approved, traced draft would look like. Book a 30-minute call.
Scenarios in this article are illustrative examples.