Skip to main content

Run four failures on purpose

The problem​

A failure you cannot reproduce is hard to trust. You want to see a real integration failure on demand, with the cause known in advance.

The Northstar sample suite ships four deliberate failures, each paired with a passing example. Compare them to see how the test controls time, creates data, chooses an address or checks an error response.

Do it​

1. Turn the failing tests on​

The sample skips the deliberate failures unless you enable them. Use the default local configuration, with nothing listening on port 5099.

Set one environment variable and run the sample:

$env:ProtoTest__Sample__Drills = "true"
dotnet test samples/Northstar.ProtoTest

Remove ProtoTest__Sample__Drills again before running the suite normally.

With those prerequisites, expect four deliberate failures: time, state, environment and visibility. Their paired examples should pass. A fifth journey adds a finding and calls Assert.Warn, so its trace records a partial outcome.

You can also use the saved traces below, which come from the same sample.

2. Open a card​

Each card shows what a failing test recorded and what the test beside it does instead.

What a failure looks like

Four questions a failing integration test usually asks. Open one to compare the drill that fails with the test that holds, as their traces recorded them.

Each card is a real recorded run. The drill failed on purpose, and the paired test runs the same journey the right way. Each card links its own archive.

Look for one habit per pair. Time moves the clock instead of waiting. State creates the data it reads. Environment takes the address from the run. Visibility checks the expected error status and body.

The visibility failure already includes the response body in its status error. Its paired test checks that the empty name produces HTTP 400 and the expected validation error.

3. Read one fix line by line​

The time failure waits a real second, which does not advance the test clock. Its paired test advances that clock eight days, then calls the application to observe the overdue invoice.

Moving the test clock alone does not run billing. The next request does.

This is the body of TheTestClockClosesTheDueWindow:

FailureDrills.cs5 notes
1var invoice = await Proto.Context.Data().IssueInvoiceAsync();
2

3Proto.Context.Clock.Advance(TimeSpan.FromDays(8));
4using var organization = await Proto.Context.Rest().GetAsync("/api/v1/organization");
5organization
6.Should.HaveHttpStatus(HttpStatusCode.OK)
7.Should.MatchShape(new { status = SubscriptionStatuses.PastDue });
8

9using var paid = await Proto.Context.Rest()
10.Body(new PayInvoiceRequest(PaymentMethods.Visa))
11.PostAsync("/api/v1/invoices/{invoiceId}/pay", new { invoiceId = invoice.Id });
12paid
13.Should.HaveHttpStatus(HttpStatusCode.OK)
14.Should.MatchShape(new { status = InvoiceStatuses.Paid });
  1. Build and provision the invoice

    The local provisioner records usage and advances the tenant clock to issue an invoice. Tenant teardown removes the data.

  2. Move the test clock

    Advance moves the test's clock and records a clock.advance event. The application runs inside the test process and reads that same clock.

  3. Call through the composed client

    Rest() uses the client supplied by the run. Here it calls the in-process application, which reads the test clock.

  4. Assert the property the behavior depends on

    For a status value mismatch, MatchShape reports the JSON path, expected value and actual value.

  5. Pay, then check the result

    The second call reuses the same client, the same context and the same trace.

From samples/Northstar.ProtoTest/FailureDrills.cs. The failing test next to it waits on real time.

What happened​

Each deliberate failure exposes an assumption. Real time moves the test clock, a hardcoded project exists, a local port serves the application, or an invalid request succeeds.

The paired examples replace those assumptions with explicit setup, the client supplied by the run, and checks of the expected response. Compare both the requests and the assertions when reading each pair.

Each failure left a trace you can download from its card and open in the viewer. The next lesson reads one.

Check yourself​

The saved environment failure ran for about two seconds and its trace holds no HTTP request. What can you learn from the trace and source together?

Verify
Download l0-environment-drill.prototrace, open it in the viewer, and compare its execution phase with l0-environment-fix.prototrace.

Remember​

  • Each deliberate failure exposes an assumption about time, state, the environment or the expected response.
  • The paired fix changes one habit: the clock, the data, the address or the assertion.
  • Both halves leave a trace, so you can compare them.

Go deeper​