What did my suite never check?
The problem
An endpoint can be called by a hundred setup helpers and have its response asserted by none of them. The status comes back, the fields go unread, and a renamed field breaks a client you never tested.
Code coverage cannot see this. It counts lines that ran, not whether a check read what they returned. Contract coverage counts the endpoints, statuses and fields your assertions actually matched.
Do it
1. Open the report from a real run
Download l4-coverage.prototrace. The file is a zip archive. Open the report inside it at resources/run/JsonReportSink/run-artifact-1/report.json. A local run writes the same data to TestResults/Northstar.ProtoTest/report.json.
The journey writes a project over REST and reads it back over GraphQL. Look for these rows:
| Row | Reading |
|---|---|
coverage REST POST /api/v1/projects, covered | the endpoint and the status the test asserted |
traffic REST POST /api/v1/projects · 201, with $.id, $.name, $.slug, $.status, $.environmentCount, $.createdAtUtc | the fields the response carried and no assertion mentioned |
gate no error findings, passed | the run gate from the last lesson |
The summary reads CoverageTotal: 1, Covered: 1, Uncovered: 0. It counts the write endpoint only. The six fields are not part of it.
2. Read the two rows in the JSON
{ "TargetName": "Northstar:Northstar", "Category": "REST",
"Identifier": "POST /api/v1/projects", "Kind": "coverage",
"Status": "Success", "IsCovered": true }
{ "TargetName": "Northstar:Northstar", "Category": "REST traffic",
"Identifier": "POST /api/v1/projects · 201", "Kind": "traffic",
"Status": "Neutral",
"Message": "Fields that arrived in a response but no shape assertion mentioned.",
"Children": ["$.id", "$.name", "$.slug", "$.status", "$.environmentCount", "$.createdAtUtc"] }
The first row is the covered claim. The second is the gap.
3. Find which assertion claims a field
Should.HaveHttpStatus(...)covers the endpoint and the status.Should.MatchShape(...)covers every property path it matched, such asnameandstatus.JsonValue.Any()mentions a whole value but not the fields inside it, so those fields stay in the traffic section.
Compare the traffic row with the shape assertions in your own journeys. Every field it lists is a field no assertion named.
What happened
Two collectors produced those rows. The composition registers both on the REST client:
1app.AddRest(rest => rest2.CaptureAttachments()3.AddClient(NorthstarTargets.Api)4.AddCollector<RestCoverageCollector>()5.AddCollector<RestTrafficCoverageCollector>())
- Keep the evidence
Request and response artifacts land in the trace, which is what the fields are read from.
- Count what was called
RestCoverageCollector reports every REST endpoint the suite called, with its hit count.
- List what was only seen
RestTrafficCoverageCollector reports the response fields no shape assertion mentioned, in their own section. It never counts them as covered.
samples/Northstar.ProtoTest/Setup.cs. Collectors gather across the run; the sinks registered below them write the report.A covered row says what the suite checked. A traffic row says what it only saw. A field no assertion names can break silently, and the traffic section shows that gap without pretending it is covered.
Check yourself
The REST write asserts only the status, and the report lists six response fields as unasserted. The GraphQL read of the same project asserts its name and status. Why are the REST fields still unasserted?
Coverage is kept per target and per protocol. A GraphQL shape assertion claims GraphQL fields. It says nothing about a REST response.
The REST write response was only judged on its status, so every field it carried is listed in the traffic section as observed and unasserted. Adding one Should.MatchShape to the write removes the fields it names from the traffic row.
Remember
- Code coverage says lines ran. Contract coverage says what the suite checked about the API.
- A status check covers the endpoint and the status. A shape assertion claims a field.
- The traffic section lists what arrived in a response and no assertion mentioned.
Go deeper
- Coverage and observations: collectors for other contracts, such as OpenAPI documents and GraphQL schemas, and how to write your own.