DataBP POC
DataBP POC / Documents / Test plan

DIG-656 Test Plan: Morgan

The route's tests and the live probes.

Morgan, the agent (DIG-656). Built from Pilot knowledge 111 item 819, version 2, as kept while building; private details such as addresses, identifiers and paths are masked here.

Test Plan

Branch Delivery Document — Level 4 — Test Plan

Effort DIG-656 — Morgan: build — branch rayserrano2735/dig-656-morgan-build
Governed by Standard: Branch Delivery Documents (K#12 I#27)
References Design: Morgan, the POC's agent: design document, knowledge 111 item 817 v8 — implementation guide: DIG-656 Implementation Guide: Morgan, a pluggable agent service, knowledge 111 item 818 — QA standard: K#2 I#167
Status v1 — DRAFT — written 2026-11-20 13:50Z (written inside Ray's workflow approval of 2026-11-20 08:46 ET, answer 1y)
Audience Claude, who builds and tests; Ray, who decides
Author Ray Serrano; edited by Claude — 2026-11-20
Classification Internal — Data Management
Official copy method Copied from this governed template by AGENTIC.DATASKILLS.KNOWLEDGE_ITEM_FROM_TEMPLATE; template SHA-256 recorded in the created item

1. Executive summary

Model Logic areas Brute-force paths Unique paths after collapse
agent.routes.dbt_semantic_layer LA-1 the three tools 6 5
agent.core.plugins LA-2 choosing persona and route 4 3
agent.core.conversation LA-3 Sage's loop and guard, kept Sage's own Sage's tests

2. Logic areas

LA-1 The three tools (route)

Dimensions: the tool (list-metrics, get-dimensions, query-metrics) by the outcome (answered, refused). Brute force 6. Collapse: list-metrics has no refusal of its own, so 5. Design D-2, D-3.

LA-2 Choosing persona and route (plugins)

Dimensions: the persona setting known or unknown, the route setting known or unknown. Brute force 4. Collapse: either unknown fails the same way at start, so 3. Design D-9.

LA-3 Sage's loop and guard (core)

Sage's own logic areas and tests (DV Digest design D-10 to D-13), kept and run against a fake route. Design D-1.

Each logic area states its dimensions, brute-force count, collapse and reason, then provides this table:

Path Scenario Expected result Unit-test name
1 list-metrics The 17 metrics with name, label, description test_list_metrics
2 get-dimensions, known metric Its dimension names test_get_dimensions
3 get-dimensions, unknown metric Refusal naming the metrics test_unknown_metric_refused
4 query-metrics answered Rows and labels; createQuery carries the metric and group_by test_query_metrics
5 query-metrics failed Refusal carrying the Semantic Layer's error test_failed_query_refused
6 Persona and route known Morgan and the Semantic Layer route loaded test_plugins_load
7 Persona unknown Start fails, naming the setting test_unknown_persona
8 Route unknown Start fails, naming the setting test_unknown_route
9 Sage's conversation tests As in DV Digest agent/tests/test_core.py

2.1 Coverage by changed file, all seven layers

Every code file the branch changes or adds, from git diff --name-only against its base at a stated clock, one row per file: every layer filled with its named artifacts, or "n/a: reason" (COVERAGE: EVERY FILE, EVERY LAYER, K#2 I#226). The row count equals the diff's; existing tests on a changed file are kept and named beside the new.

File Change 1 Built-ins 2 Expectations 3 Unit tests 4 Logic assertions 5 Detectors 6 Reconciliations 7 Semantic surface
agent/ (new folder) Added: core, route, persona, deployment, tests n/a: no dbt models n/a LA-1 to LA-3 the guard, LA-3 CloudWatch events (design 8) the probes against dbt's MCP the catalogue of 17 metrics

3. Seven-layer proof plan

Layer Named artifacts Expected result Applicability and reason
1 — Built-ins None n/a No dbt models change
2 — Expectations None n/a No dbt models change
3 — Unit tests Paths 1 to 9 Each seen failing before its code, then passing LA-1 to LA-3
4 — Logic assertions The guard: a planted reply with a figure and no query is never served Error, no text released Design D-1, Sage's D-13
5 — Detectors CloudWatch events start, catalogue, model call, tool request, upstream error Present after the probes Design 8
6 — Reconciliations Each probe figure against dbt's MCP and the answer key Equal Morgan, Hex and the MCP agree
7 — Semantic surface The catalogue given to Claude; each probe's query evidence 17 metrics; real metric names The Semantic Layer is Morgan's only input

3.1 Full-build guardrails

Artifacts Expected result
Deployment read back: reserved concurrency 2, response streaming, CORS origins, daily cap 200, no secret in the package; the page read live in Chrome As the design's D-6 and D-7 set
The structure comparison: each committed DDL file, one per data model the branch maintains, compared with the structure the build left in that environment, after every build in dev, QA and production (Ray, 2026-11-01) Identical, keys and descriptions included; a difference fails the cycle

4. Consumer-question probes

Question Runnable probe Expected result
Q-1 Through the function URL, then on the page: "What did October's disagreements cost?" and "Why is subscriber 43 unbilled?" 1,410 and 2,070 dollars; 17 units beyond licensed, 850 dollars; query evidence under each
Q-2 Ray talks to Morgan on the page His approval

4.1 Finance workbook and initial UAT proofs

For finance reporting branches, name row/key/value parity checks against the shared report query, totals at the declared grain, Excel types/date/rounding checks and refresh preservation of consumer observations. State consumer-review cases, dispositions and acceptance evidence for the exact workbook revision under K#12 I#27. Technical verification does not imply consumer acceptance. Otherwise state NA with a reason.

NA: not a finance reporting branch.

5. QA on production-shaped data

5.1 Added and changed tests on QA data

Test Expected result
Every test above DataBP has no QA environment; the QA certification is waived for DataBP (Ray, 2026-11-19, answer 1y); Morgan has one deployment (design D-7)

5.2 Regression against production

n/a: Morgan is new; nothing in production depends on it.

5.3 Consumer probes and reconciliations on QA data

The probes run against the one deployment.

5.4 Layer 7 in the QA collection

The catalogue check runs against the one deployment.

6. Test artifact inventory

Artifact classification Count Sections that make up the count
Unit tests 8 new, plus Sage's kept 2
Logic assertions 1 3
Reconciliations 2 probes 4

7. Origin

Test or proof Design decision proved
Paths 1 to 5 D-2, D-3
Paths 6 to 8 D-9
Path 9 and the guard D-1
The probes D-4, D-5
The guardrails D-6, D-7

This plan cites the design's decisions and adds none. A test requiring a missing decision returns to the design.

Compliance record

Built by the author last, before the document is presented for agreement (Standard: Branch Delivery Documents, K#12 I#27, "Compliance record before agreement"): every section and field of this template shown present and filled, by infra/workstation/pilot/compliance_sections.py; every rule of that standard and of the standards it cites, each with the passage that meets it or marked NOT MET; and the decisions the document carries. While this section reads NOT BUILT, the document is not presented for agreement.

NOT BUILT.