Why integration tests get hard

An integration test checks how running parts of a system work together, such as an API and a database. It catches problems that a test of one isolated component would miss.

Most teams know what goes wrong next. A test passes on your machine and fails on CI. It fails once and passes on retry. It fails with a message that does not say why, and someone spends an hour finding out. After a while, people stop reading its failures.

Those failures look random, but most of them have one of four causes: time, state, environment and visibility. This page shows each one with an example from the sample suite. The suite pairs every example with a deliberate failure and a passing test, and the later lessons run those drills and read their traces.

Time

What you see: the test passes locally and fails on a busy CI machine.

A test issues an invoice, waits one real second and expects the subscription to be past due. The application still reports it as active. It reads a test clock, which advances only when the test moves it. The check fails on $.status: it expected past_due and read active.

The fix advances that clock beyond the invoice's due date: Proto.Context.Clock.Advance(TimeSpan.FromDays(8)). The application then reports past_due, as the test expects.

The question to ask: who moves the clock? Learn it in Reliable tests.

State

What you see: the test passes alone and fails in the suite, or the other way round.

A test reads the project with id prj_1 without creating it. The request returns 404. A hardcoded id does not establish that the record exists or belongs to this test.

The fix creates a project inside the test's own tenant, an organization prepared during setup. It checks that the tenant's project list contains that project. Teardown deletes the tenant and its data.

The question to ask: what does the test share with others? Learn it in Good tests.

Environment

What you see: the test works on one machine and fails with a connection error on another.

A test opens a raw HttpClient on 127.0.0.1:5099, assuming an application is listening there. In the recorded drill, the connection fails. The raw client bypasses ProtoTest's request recording, so the trace has no request operation for that call.

The fix uses the test's client, Proto.Context.Rest(). It takes its address from the suite's composition: the setup that chooses applications, dependencies and their addresses. The test can then use the environment selected by that setup without hardcoding an address.

The question to ask: where does the address come from? Learn it in Good tests and Reliable tests.

Visibility

What you see: the test fails, and the message does not tell you why.

A test sends an empty project name and asserts only the status. It expected 201 Created and the check reported 400. The response body said validation_failed and named the empty parameter. ProtoTest includes that body in the failure message, but the test did not check it.

The passing test expects 400 Bad Request and checks the problem body's code and message. It verifies that the application rejected the empty name for the expected reason.

The question to ask: what can the test show when it fails? Learn it in Understand failures.

What a passing test has handled

A test you can trust has handled each cause that applies to it. For time-dependent behavior, it controls or observes time. For mutable data, it decides who creates it, who can see it and who removes it. It gets its addresses from the suite's setup, and its assertions explain a mismatch.

Not every test needs to move a clock or create data. When a failure does look random, the four questions above are where to start looking.

Next: Run the sample suite.