Skip to main content

Host and lifecycle

What it is​

ProtoHost is built once per test process by your runner's assembly setup. It owns the dependency injection container, runs suite-wide hooks and run gates, starts infrastructure, and starts and completes each test.

ProtoExecutionContext exists for exactly one test. It holds that test's clients, typed state, resources, attachments and observations. See Execution context.

1 hooks → 2 attributes → 3 body → 4 attributes back → 5 hooks back → 6 publish → 7 dispose

Hooks run before all attributes. A setup failure rolls back only what completed.

How it works​

Building the host​

The runner's setup class receives an IProtoHostBuilder:

protected override void Configure(IProtoHostBuilder builder) =>
    builder
        .ConfigureAppConfiguration(configuration => configuration.AddJsonFile("appsettings.Test.json", optional: true))
        .ConfigureServices(services => services.AddSingleton<IClock, FixedClock>())
        .ConfigureTracing(trace => trace.OutputPath = "TestResults/run.prototrace")
        .ConfigureTestIds(ids => ids.RunPrefix = 42)
        .AddRunHook<StartDependenciesHook>()
        .AddTestHook<ResetMailboxHook>()
        .AddApplication("Api", app => app.AddRest(rest => rest.AddClient("Api")));
IProtoHostBuilder ConfigureServices(Action<IServiceCollection> configure);
IProtoHostBuilder ConfigureAppConfiguration(Action<IConfigurationBuilder> configure);
IProtoHostBuilder ConfigureTracing(Action<ProtoTraceOptions> configure);
IProtoHostBuilder ConfigureTestIds(Action<ProtoTestIdOptions> configure);
IProtoHostBuilder AddTestHook<THook>() where THook : class, IProtoTestHook;
IProtoHostBuilder AddRunHook<TRunHook>() where TRunHook : class, IProtoRunHook;
IProtoHostBuilder AddRunGate<TGate>() where TGate : class, IProtoRunGate;
IProtoHostBuilder AddRunGate(string name, Func<ProtoRunGateContext, ProtoRunGateResult> evaluate);
IProtoHostBuilder AddResource(IProtoResource resource);
ProtoHost Build();

Every integration's Add... method is an extension on this same builder. A builder can only build once. The option tables for ProtoTestIdOptions and ProtoTraceOptions are in Configuration.

The run​

  1. Runner → ProtoHost1 StartAsync
  2. ProtoHost → Run hooks2 BeforeRunAsync, ascending Order
  3. Infrastructure starts in registration order.
  4. ProtoHost → Run hooks3 AfterInfrastructureAsync, ascending Order
  5. The tests run.
  6. Runner → ProtoHost4 StopAsync
  7. ProtoHost → Run hooks5 AfterRunAsync, descending Order
  8. Gates evaluate, report sinks export, run resources release, then the .prototrace is written.
  • If a BeforeRunAsync throws, the hooks that already started get their AfterRunAsync in reverse, and startup fails.
  • If an AfterInfrastructureAsync throws, startup fails the same way: every hook gets its AfterRunAsync in reverse, the started infrastructure releases, and a retry runs the whole start again.
  • StopAsync runs every AfterRunAsync even if some throw, then reports all failures together.
  • Once stopping has begun, starting a new test throws.

ProtoTest's own run hooks are ordered to run last on the way out. Run gates evaluate first, report sinks export next, and run-scoped resources release. The trace archive is written last, so it can include the reports.

A test​

When the runner starts a test:

  1. A ProtoExecutionContext and a new DI scope are created, and the context becomes Proto.Context for this async flow.
  2. Every IProtoTestHook runs BeforeTestAsync, in ascending Order. ProtoTest's client-initializing hook has the lowest possible order, so clients exist before any of your hooks run.
  3. Every ProtoAttribute on the test runs BeforeTestAsync, in ascending Order.
  4. Your test body runs.

All hooks run before any attribute, whatever their Order values. Among attributes, only Order matters. Whether an attribute sits on the class or the method does not change when it runs, and ties keep class attributes first.

When the runner completes the test:

  1. Attributes run AfterTestAsync in reverse order.
  2. Hooks run AfterTestAsync in reverse order.
  3. Attachments are published to the runner.
  4. context.DisposeAsync releases owned resources in reverse registration order, then disposes the DI scope.
  5. The test's trace artifacts are captured and the recorder is completed with its outcome. Proto.Context is cleared.

Teardown attempts each step, even when an earlier one throws. A teardown failure is recorded as an Error finding. By default it also fails the test when the body passed: the runner reports that test as failed, and the message says the test body passed and names the cleanup failure (the first exception, then how many more). A body that already failed keeps that failure as the message the runner shows; the cleanup failures are attached after it, so a cleanup error never hides a failed assertion. The trace agrees: a passing body whose cleanup failed is Failed, with that same message.

CleanupFailures selects the behaviour. Fail is the default. Report is the previous behaviour: the finding is recorded, the runner result stays what the body produced, and a passing body reads as Partial in the trace.

builder.ConfigureCleanup(options => options.CleanupFailures = ProtoCleanupFailureMode.Report);

Configuration binds the same choice from ProtoTest:CleanupFailures (Fail or Report) and wins over the code value. CompleteTestAsync still rethrows one cleanup failure as-is and aggregates several. The runner adapters complete through ProtoTestScope, which applies this rule, so the failure belongs to the test that leaked rather than to the run as a whole.

A test that skips itself after its context starts still owns its resources. With CleanupFailures = Fail, a failing release makes that test fail, even though its body was skipped. An unknown body outcome follows the same rule. The runner error says that the body was skipped or unknown; it does not claim the body passed. The trace records the cleanup failure as the test's failure too. With Report, a skipped or unknown outcome stays unchanged and the cleanup error remains in the findings and teardown entries. A skip evaluated before the context starts owns no resources and needs no cleanup.

When setup fails​

If a hook or attribute throws during setup, ProtoTest rolls back. Only the components that completed their BeforeTestAsync get their AfterTestAsync, in reverse. The one that threw does not.

FirstHook:Before
SecondHook:Before
FirstAttribute:Before
FailingAttribute:Before   <- throws
FirstAttribute:After
SecondHook:After
FirstHook:After

The test is recorded as failed, and the trace shows a Rollback phase instead of Teardown. The exception reads "Test setup failed and completed lifecycle components were rolled back."

The same reversal in the recording, the sample suite's project journey: setup runs attributes Application, NorthstarTenant, SignedInAs, NorthstarMember, and teardown answers NorthstarMember, SignedInAs, NorthstarTenant, Application.

Setup order, teardown reversalNorthstar.ProtoTest.ProjectsJourney.CreatingAProjectReturnsIt
  1. Setup500.5 ms

    Hooks first in ascending Order, then attributes in ascending Order. The client hook runs before everything else.

    Recorded operations (4)
    • ProtoClientInitializerHookhook.before · Order First: clients exist before any other hook
    • SqlConnectionHook, NorthstarScenarioHookhook.before · Order -1000 each
    • ProtoHttpAuthLifecycleHook twicehook.before · Order 100: auth applies last
    • Application, NorthstarTenant, SignedInAs, NorthstarMemberattribute.before · Orders First, -200, -100, 0
  2. Teardown36.7 ms

    The same components walk back: attributes in reverse, then hooks in reverse, then publish and release.

    Recorded operations (4)
    • NorthstarMember, SignedInAs, NorthstarTenant, Applicationattribute.after · reverse of setup
    • Auth hooks, scenario, SQL, completion, initializerhook.after · reverse of setup
    • Request, response, expected shape, scenario summaryattachment.publish · after every hook and attribute finished
    • Tenant cleanup, services, connection, consumerresources.release · reverse registration order
Read from the project journey the first lesson writes in docs/static/lessons/l1-first-journey.prototrace. The viewer draws the same trace from that archive (download it and drop it on the viewer).

This is why teardown code should tolerate partial setup. The sample environment attribute uses TryResolve rather than Resolve in its AfterTestAsync for exactly that reason.

One test per async flow​

A context is tied to the async flow that started it. Starting a second test on the same flow before completing the first throws, as does completing a test from a different host. Skipped tests never reach this point. A skip condition is evaluated before StartTestAsync, so there is no context to complete.

How to use it​

Runner packages start and complete tests for you. The hand-driven surface lives on Host API.

Test ids​

Every test gets a numeric id, and everything the test produces is keyed by it. That covers trace entries, archived artifacts, and often the data your own attributes create ($"test-{context.TestId}").

An id is a run prefix followed by a sequence: with the defaults, 482913000001, 482913000002, and so on.

ProtoTestIdOptionsDefault
RunPrefixa random six-digit number per host
SequenceDigits6 (1 to 9)

The random prefix keeps ids from colliding when several test processes create data in the same shared environment. Set a fixed RunPrefix, for example from a CI build number, when you want ids you can trace back to a pipeline run. Ids are at most 18 digits, and running out of sequence numbers throws.

To replace the scheme entirely, register your own IProtoTestIdGenerator:

public interface IProtoTestIdGenerator
{
    ProtoTestId Next(MethodInfo testMethod);
}

Run gates and resources​

AddRunGate registers a check that runs once, after the last test and before the reports are written. It can see everything the run's collectors produced. ProtoRunGateContext exposes Items, ItemsOfKind, InCategory, ForTarget and WithStatus, plus coverage helpers such as CoverageFor(target). A failed gate throws ProtoRunGateException out of AfterRunAsync. The delegate overload is the quick form:

builder.AddRunGate("no error findings", context => context
    .ItemsOfKind(ProtoReportItemKinds.Finding)
    .Any(item => item.Status == ProtoReportStatus.Error)
    ? ProtoRunGateResult.Failed("The run recorded error findings.")
    : ProtoRunGateResult.Passed("No error findings were recorded."));
ResultMeaning
Passedthe run continues to the reports
Warningrecorded, but the run continues
Failedthrows ProtoRunGateException out of AfterRunAsync
Skippedrecorded. A gate that returns no result is treated as failed.

AddResource(IProtoResource) registers an already-created, run-scoped resource, such as a started container or a connection. The host owns it and releases it with the run, without starting anything. AddInfrastructure is the variant that starts with the run and fills settings, as Infrastructure explains.

What the trace shows​

  • Each test's record opens with a test.setup operation. It carries one hook.before and hook.after per hook, with its type and order, and one attribute.before and attribute.after per attribute. The test's own operations follow.
  • A failed setup shows a Rollback phase instead of Teardown, and only the components that completed appear.
  • Attachments land on the record of the operation that produced them. Resources appear as entities, and each release writes a resource.release operation. A framework-managed client records only a failed release, as an event (Clients).
  • Run-level pieces are entities: capabilities, infrastructure with its state, and readiness probes. Run gates evaluate report items after the last test and before the reports export.
  • The trace archive is written last, after the reports, so a report's coverage is inside the archive.
  • A teardown failure is recorded as an Error finding. By default it also fails a test whose body passed. CleanupFailures set to Report leaves the test's own outcome unchanged and records a passing body as Partial.

Limits​

  • One context per async flow. Starting a second test on the same flow before completing the first throws, and completing a test from a different host throws. Proto.Context is flow-local, so work that escapes the test's flow cannot read it. See Execution context and Concurrency.
  • One build per builder. Build() can only run once, and once stopping has begun, starting a new test throws.
  • A skipped test never reaches the lifecycle. It never creates a context. See Skip conditions.
  • A teardown failure does not replace a body failure. The body's error stays the one the runner shows, with the cleanup failures attached. A body that passed is reported as failed unless CleanupFailures is Report. The finding is recorded either way.
  • Ids cannot grow past 18 digits. Running out of sequence numbers throws.