On this page
An AI answering system can retrieve the wrong answer perfectly. If its source says the office closes at six after the business changed its hours to five, better search relevance will not make the answer current. Knowledge freshness is an operating process with owners and decisions, not merely a model setting.
Begin with the facts the system is allowed to use. Opening hours, service areas, eligibility rules, prices, appointment instructions, and escalation contacts change at different rates. Treat each kind of information according to its consequence and the process that maintains it.
Separate the source from the search copy
The authoritative source may be a controlled business record, approved document, or maintained page. The retrieval index is a processed copy used to find information. A successful index update does not prove that the source itself was correct, and an approved source change does not prove that the index has incorporated it.
Record both stages. Who approves the business fact? Who operates ingestion and retrieval? What evidence shows the public answering system now uses the approved version? These questions prevent a change from disappearing between content and engineering responsibilities.
The AI knowledge-base guide covers the broader collection. An expiry process focuses on what happens as individual facts age or are replaced.
Give each source a practical owner
An owner should be able to confirm or route a change, not merely appear in a spreadsheet. Assign the team responsible for the fact and a backup route when that person is unavailable.
For example, operations may own opening hours, while a service manager owns appointment requirements. The AI team can maintain the technical pipeline without being authorized to invent either policy.
Keep the ownership record close to the content's maintenance process. A separate list that nobody sees during an update is likely to drift. The useful question is who will notice and act when the information becomes uncertain.
Distinguish effective dates from review dates
An effective date says when the information applies. A review date says when someone checked it. An expiry or retirement rule says when the system should stop relying on it without another decision. These dates answer different questions.
A holiday schedule may be approved weeks ahead but apply only on specified days. A standing policy may remain valid until replaced, with periodic review. A temporary promotion may need a firm end condition. Do not give all three the same generic “last updated” field and assume the model will infer the difference.
Where time zones or local locations matter, make them explicit in the source. A date boundary at headquarters may not describe the closing time of every branch.
Decide what happens when review is overdue
An overdue review is a state to handle, not proof that the fact is false. Define whether the system may continue using it, qualify the answer, route to a person, or withhold a consequential claim until it is confirmed.
Choose the behavior according to the information's role and risk. An old description of the building's color and an uncertain price commitment do not need identical treatment. NIST's framework supports documenting limitations and responsibilities; the specific policy must fit the business and use case.
Avoid allowing the model to create a fresh date or an apparent confirmation merely because the caller asks whether the information is current. Confirmation needs evidence from the maintained source or the authorized person.
Track replacement and retirement explicitly
When a new version replaces an old one, identify the relationship. Remove or retire obsolete material from the active retrieval path according to the system's design. Keeping both versions searchable without a clear precedence rule can produce conflicting answers.
Preserve historical material where the business has a legitimate reason, but separate it from current answering content. An archive label in a file name may not survive document processing or be visible in every retrieved fragment.
Check summaries, copied FAQs, and location-specific pages that repeat the same fact. Updating the main policy while leaving an old summary active can make the answer depend on which fragment happens to rank first.
Test the question a caller would actually ask
After an update, query the deployed answering path with representative wording. Include direct questions, common paraphrases, and a case that used to produce the old answer. Inspect the retrieved source and the final response.
A source appearing in a retrieval result is not enough if the generated answer still states the old value or adds an unsupported exception. Conversely, a correct answer once does not prove every branch uses the current source.
Use synthetic scenarios and avoid real customer data. The AI receptionist launch test provides a broader set of operational checks; freshness tests should become part of that release and maintenance process.
Use a small change record
| Field | Purpose |
|---|---|
| Source and topic | Identifies the fact being maintained |
| Business owner | Establishes who can approve it |
| Effective period | States when the information applies |
| Review or expiry condition | Triggers the next decision |
| Replaces | Identifies material that should stop being active |
| Ingestion evidence | Shows the technical copy was updated |
| Answer checks | Shows representative deployed responses |
| Fallback | Explains what happens if the fact cannot be confirmed |
Keep the record proportionate. A small business may use a controlled document and a short checklist; a larger service may automate parts of the workflow. The required outcome is the same: an update can be traced from approval to the answer a caller hears.
Monitor the failures that reveal stale knowledge
Review corrections, escalations, conflicting-source reports, and repeated caller challenges for signs of outdated content. Do not count every escalation as a defect; a system correctly refusing to guess may be following its intended boundary.
When an error occurs, identify whether the source was wrong, the update was not ingested, retrieval selected an obsolete version, or generation departed from the source. Fixing the correct stage is more durable than adding another prompt sentence to hide the symptom.
Document the resolution and retest the affected questions. The result should be an answering system whose current claims have a current authority, with a clear route when that authority is missing.
Sources and further reading
Primary and contextual sources used to verify definitions or give readers a relevant next resource.
- NIST: AI RMF core AI risk management includes documented roles, limitations, monitoring, and accountability across the system lifecycle.
- Microsoft: Retrieval augmented generation Retrieval can ground generated responses in indexed source information, making the source and retrieval process part of answer quality.