Good agents need
good diagnostics.

Classic RAG

Retrieve relevant passages, then generate an answer. A sensible starting point for approved purchasing policies.

Hybrid retrieval and reranking

Combine keyword and vector search, then optionally reorder results. Useful when Business Central item numbers and descriptive language appear together.

Context-enriched RAG

Add document context to chunks before indexing. Helps a short policy extract retain its meaning.

Graph RAG

Use extracted relationships and summaries to explore connected information. Consider for questions across suppliers and policy documents; it does not replace ledger calculations.

Agentic RAG

An agent plans and iterates searches. Useful for questions needing several evidence sources, with explicit limits on time and tool calls.

Multimodal RAG

Retrieve evidence from material such as text and images using suitable extraction and models. Scanned delivery notes need evaluation beyond plain-text search.

The right document never appears

Symptom

An agent cannot find an approved purchasing policy.

Check

Inspect the retrieved passages before looking at the answer. Check ingestion status, search terms and the requesting user’s access.

Corrective action

Correct the ingestion or identity filter that excluded the document. Do not grant broader permissions simply to produce an answer.

Verify

Try a known answer as an authorised user, then confirm an unauthorised user still cannot retrieve it.

Similar words, wrong item

Symptom

A question about one Business Central item returns a similarly described product.

Check

Compare exact identifiers, company filters and search results. Check that query and index embeddings are compatible.

Corrective action

Evaluate keyword plus vector retrieval. Preserve item identifiers and apply validated company constraints before selecting evidence.

Verify

Use exact-ID, misspelling and ambiguous-description questions. Confirm citations identify the intended item.

A policy extract loses its conditions

Symptom

The answer quotes an approval threshold but omits the exception in the next paragraph.

Check

Read the indexed chunk alongside its parent section. Look for missing headings, table labels and adjacent conditions.

Corrective action

Rework chunk boundaries or add contextual metadata; rebuild affected entries. Choose the strategy using representative questions.

Verify

Ask about both the general rule and its exception. Both must be supported by the retrieved evidence.

Yesterday’s information answers today’s question

Symptom

An agent presents an archived policy or an outdated stock figure as current.

Check

Inspect source version, effective date, ingestion time and how updates and deletions reach the index.

Corrective action

Use a tested refresh and deletion process for documents. For current Business Central stock or balances, call an authorised API or report with explicit company and date context.

Verify

Change a test document, then retire it. Confirm both changes propagate. Compare live numeric answers with the authoritative business record.

A convincing answer goes beyond the evidence

Symptom

The answer invents an approval rule that its citation does not contain.

Check

Separate retrieval quality from answer quality: did the evidence contain the answer, and did the generated response stay within it?

Corrective action

Require evidence for material claims and a clear “not enough information” response. Evaluate retrieval and grounding separately.

Verify

Include answerable, ambiguous and unanswerable questions. Review whether each citation actually supports its claim.

Information crosses a company boundary

Symptom

A user sees another Business Central company’s internal documents.

Check

Trace user identity and document-level authorisation through retrieval. An application’s access is not proof of a visitor’s entitlement.

Corrective action

Enforce access using trusted identity and permission data on the server. Keep permission changes synchronised. Some native Azure AI Search permission features are preview; check support before choosing them.

Verify

Test two companies, revoked access and missing identity. Confirm restricted evidence never reaches the model.

A retrieved document tries to direct the agent

Symptom

A supplier attachment includes instructions to ignore rules or change payment details.

Check

Treat instructions embedded in external material as untrusted data. Inspect the trace for tool calls influenced by that content.

Corrective action

Separate evidence from instructions, restrict tool permissions and require approval for sensitive business changes. Use supported injection defences as one layer.

Verify

Use controlled malicious test documents. Verify the agent cannot perform an unauthorised vendor-bank change.

A connector returns a 400 error

Microsoft documents a Copilot Studio connector failure when its response exceeds 500 KB. First inspect the actual error; not every 400 has this cause.

For an illustrative ledger query, reduce records and fields using supported connector inputs, then test representative larger results. Add pagination only where the connector supports it.

Authentication, throttling or timeouts

Capture the error code and correlation ID. Check connection identity and permissions for access failures; inspect limits and service status for throttling or timeouts.

For Business Central integrations, use bounded retries where appropriate, honour retry guidance and narrow expensive queries. Retest the failing path under expected load.

A retry creates a second document

Engineering recommendation: a timeout does not prove a write failed. Before retrying a Business Central order creation, reconcile the first attempt using a stable request reference.

Design duplicate prevention around the operation’s supported behaviour. Test a lost response after a successful write and confirm only one document exists.

The agent keeps searching

Engineering recommendation: cap search iterations, elapsed time and tool calls. Record whether each extra search added useful evidence.

For a supplier investigation, define a completion condition and a hand-off when evidence is insufficient. Test difficult questions as well as successful ones.