Skip to main content

Run the sample suite

The problem​

Before you trust a test framework, you want to see it work. This lesson runs the Northstar sample suite and finds the files it produces. The suite tests a small application through an API, a database, a browser and a message broker.

The default configuration runs locally. Tests that need an unavailable broker or browser skip with a reason.

Do it​

1. Run the suite​

git clone https://github.com/MSeys/ProtoTest.git
cd ProtoTest
dotnet test samples/Northstar.ProtoTest

The repository's global.json selects the build SDK. The sample targets net8.0, so it also needs the .NET 8 runtimes listed above.

The first run restores packages before building and testing. With the checked-in settings, the suite starts the application and uses a local SQLite database. You do not need to start an application, database server or message broker yourself.

This recorded run had Playwright's Chromium browser installed and no broker configured:

Failed:     0, Passed:    14, Skipped:     6, Total:    20

With the default settings, the browser test skips if Playwright's Chromium browser is missing. Your counts can differ with the checkout and configuration. Look for passing tests, expected skips and no failures. A result with zero tests does not confirm a successful run.

2. Read the skips​

The example includes expected skips. Run the suite with detailed output to read each reason:

dotnet test samples/Northstar.ProtoTest --logger "console;verbosity=detailed"

Look for the broker journey. With no broker configured, its reason reads "No broker is configured; set ProtoTest:Messaging:Broker=container."

The four failure drills and the warning journey also skip by default. They deliberately fail or report a warning when enabled. Leave ProtoTest:Sample:Drills set to false for this lesson.

3. Find the evidence​

In samples/Northstar.ProtoTest/, find the files under the build output:

bin/Debug/net8.0/TestResults/
    prototest-{runId}.prototrace    the steps ProtoTest recorded, as one file
    Northstar.ProtoTest/
        report.json             the verdict, as data
        report.html             the same verdict, as a page

Open report.html in a browser to see test results and route coverage. The .prototrace archive is called a trace. You will open it in the viewer in the third lesson.

What happened​

One command built the application, started it inside the test process, ran the suite, and wrote a trace and two reports.

The broker test requires a capability: something the configured framework can provide to a test. With no broker configured, that capability is absent. The test declares that it needs a broker, so ProtoTest skips it before its test body runs.

Other skips have different causes. The browser test checks whether its browser is available. The drills opt out until you enable them. Read the reason to tell these cases apart.

Check yourself​

The run is green and the broker journey did not run. What reason did it give, and where did that reason come from?

Verify
Run the test command with --logger "console;verbosity=detailed" and find the skipped test in the summary. The reason is printed beside it.

Remember​

  • One command runs the whole sample. The run leaves a trace, a JSON report and an HTML report.
  • A skip can mean a missing requirement or a deliberately disabled test. Read its reason in the runner output.
  • The same packages that build the sample go into your own project.

Go deeper​