Writing integration tests

dwe test runs your project’s deploy pipeline — and whatever assertions or commands you add — inside a fresh, throwaway copy of the project. Nothing you do here touches the environment you’re working in. This guide is the authoring workflow; the field-by-field schema lives in ../reference/config/tests.md.

Ports are isolated automatically

dwe test gives each isolated copy its own host ports, so a scenario runs alongside your working environment — and any other project holding those ports. Every host port your enabled services declare under services.<name>.ports is remapped to a freshly allocated free port in the copy, automatically — you route nothing through vars and write no port config in the scenario. Both the ports_free preflight and — for projects that source their compose bindings from services.<name>.ports (directly, or via an exports.env entry from: services.<name>.ports.<x>) — the actual container bind read the port from that same place, so the remap moves them together.

A scenario step that needs a remapped port references it the normal way: ${services.<name>.ports.<x>}.

A compose host port written as a variable ("${VALKEY_PORT:-6379}:6379") is covered too, as long as dwe can trace the variable: an exports.env rule from: vars.<path> makes the runner allocate a free port for <path> and write it into the copy’s vars: — again with no scenario config. The step reads it as ${vars.<path>}. A scenario that wants a specific port pins it with env.vars: { <path>: 6380 }.

Two shapes stay outside the remap, and they fail differently. A host port hardcoded straight in a raw compose file (8080:8080) is a blocking raw_host_port finding: dwe test run refuses the scenario before the copy exists, so nothing ever binds your working environment’s port. A port interpolated from a variable that no active exports.env rule routes to a port is not blocking — it bypasses both the remap and the ports_free preflight, the copy binds the same port as your working environment, and dwe test run and dwe validate only warn, with an interpolated_host_port finding naming the variable and both routing fixes. The fix is the same for both: declare the port under services.<name>.ports, or export the variable from a vars.<path> through an exports.env rule.

Your first scenario

Create workspace/tests/smoke.yml:

description: "Clean deploy comes up healthy"

steps:
  - name: "app answers"
    type: builtin
    cmd: http_check
    with:
      url: "http://localhost:${services.app.ports.http}/health"
      status: 200

Run it:

dwe test run smoke

This copies your project into an isolated tree, generates a fresh local.yml with freshly allocated free host ports, runs dwe validate then a real dwe deploy run inside the copy, checks the endpoint, and tears the whole thing down. A scenario with no steps: at all is already a useful test — “deploy with these parameters succeeds.”

Testing a variant of your stack

env: describes how this scenario’s environment differs from your defaults — service toggles and var overrides:

description: "Deploy with redis disabled — cache falls back to in-memory"

env:
  services:
    disable: [redis]

steps:
  - name: "app still answers without redis"
    type: builtin
    cmd: http_check
    with:
      url: "http://localhost:${services.app.ports.http}/health"
      status: 200

Each scenario file is one isolated environment — write one per meaningful variant (a service disabled, a feature flag var flipped, a different port layout) rather than trying to parametrize a single file.

Asserting on more than “the port answers”

Steps use the exact same schema as workspace/deploy.yml: type: shell, type: dwe, type: command, type: builtin, with when: conditions. Any predicate builtin — file_exists, tcp_reachable, containers_running, env_keys_present, http_check — works directly as a step body and behaves as a pass/fail assertion, not just inside check::

steps:
  - name: "containers are up"
    type: builtin
    cmd: containers_running
    with: { services: [app, db] }

  - name: "app answers"
    type: builtin
    cmd: http_check
    with: { url: "http://localhost:${services.app.ports.http}/health", status: 200, contains: "ok" }

${...} in a step’s with:/cmd: resolves against the copy’s config before the step runs, and file_exists-style paths resolve relative to the copy root — assertions always look at the disposable environment, never your working tree.

Testing a project command with type: command

The second use case from the spec — “create a DB dump, verify the file appears” — is just a type: command step calling a regular user command, followed by an assertion:

steps:
  - name: "create dump"
    type: command
    cmd: db:dump

  - name: "dump file exists"
    type: builtin
    cmd: file_exists
    with: { path: "dumps/db-latest.sql.gz" }

type: command steps can call private commands, so you can keep test-only commands (e.g. a command that seeds fixture data, or dumps to a fixed filename instead of a timestamped one) out of the everyday dwe commands listing:

workspace/commands/testing.yml

group: testing
commands:
  - id: db:dump
    private: true
    type: shell
    cmd: "docker compose exec -T db pg_dump -U app app > dumps/db-latest.sql.gz"

(hide commands are skipped by pipelines entirely, so they don’t work here — use private for test-only commands you still want runnable from a scenario.)

Debugging a failing scenario

When a scenario fails (deploy failure, step failure, or timeout), the runner attempts to collect a failure report into .dwe/tests/reports/<scenario>/ — automatically, before teardown runs, so it survives the environment being torn down. Collection is best-effort: if it can’t create the report directory the path is left empty, and individual artifacts may be partial or blank when a capture fails — but the run always reports the failure regardless.

dwe test run smoke   # fails

ls .dwe/tests/reports/smoke/
# pipeline.log        — the scenario's deploy/steps pipeline log
# compose-ps.txt       — docker compose ps --all inside the copy
# container-logs.txt   — combined container logs (last 200 lines each)

This is the report you’d attach to a CI failure, or read locally without touching Docker at all. It’s overwritten on each non-passing run, so it always reflects the latest failure.

When the report isn’t enough and you need to inspect the live environment itself, rerun with --keep:

dwe test run --keep smoke

Teardown is skipped (and no failure report is collected, since the environment itself is kept); dwe test run prints the compose project name and the copy’s path so you can cd in, inspect containers with docker compose -p <project> ps, or open a shell inside a service. The manifest stays on disk too, so a second dwe test run smoke refuses to start until you clean up.

Clean up a kept run — or anything orphaned by a crashed run — with dwe test clean:

dwe test clean --dry-run    # see what would be removed, without touching anything
dwe test clean smoke        # tear down just this scenario's kept/orphaned environment
dwe test clean               # sweep every manifested environment

clean is manifest-driven and never guesses at a compose project name: it only ever tears down environments it has a manifest for, skips any scenario whose flock is currently held by a live run, and separately (report-only) lists Docker compose projects that look like dwe test output but have no manifest — those it flags for you to remove by hand rather than destroying automatically.

Bounding a slow or hanging step

Any step can carry its own timeout: — the same opt-in engine field deploy.yml steps have. Useful when a scenario step is a tcp_reachable/http_check assertion (or a shell command) that should fail fast instead of running to the scenario’s own overall timeout::

steps:
  - name: "app answers quickly"
    type: builtin
    cmd: http_check
    timeout: 5s
    with:
      url: "http://localhost:${services.app.ports.http}/health"
      status: 200

Absent or timeout: 0 leaves the step unbounded. The timeout only bounds a step body that honors Go’s context cancellation — subprocess steps (type: shell/type: dwe) and ctx-aware builtins are covered; a step blocked on interactive input is not force-interrupted (irrelevant here, since scenario runs are always non-interactive). See Step fields for the full contract.

Catching mistakes before you run anything

dwe validate tests statically checks every workspace/tests/*.yml file — scenario name, timeout: parse, env.services references, step schema (including builtin with: params and when: conditions), and type: command references — without touching Docker or spinning up a copy. It’s cheap enough to run on every CI push, ahead of the much slower dwe test run:

dwe validate tests

It also surfaces compose isolation hazards as warnings, so you can fix them before a dwe test run ever blocks on one.

Resolving an isolation failure

dwe test run scans the compose files the scenario’s copy will run — the project’s own, with the scenario’s env: applied — for constructs that bypass compose’s project-name scoping. container_name: and literal (non-templated) host ports are blocking; external:/explicitly-name:d volumes and networks are warnings only. The scan happens before the copy is made, so a blocking finding fails the scenario with nothing created — no copy, no containers, no teardown, and no failure report collected for that run (one left by an earlier failure of the same scenario stays in place) — with a message naming the offending construct:

blocking compose isolation hazard(s), refusing to run: service db sets container_name: myapp-db — bypasses compose project-name scoping and collides with any other project/run using the same fixed name — pass --skip-isolation-check to downgrade to a warning

Fix it at the source when you can:

• Literal host port (8080:8080 in a raw compose file) — move it onto services.<name>.ports so dwe test can see and remap it automatically (see Ports are isolated automatically above), or interpolate it from a variable an exports.env rule exports from: vars.<path>, which dwe test then remaps on its own.
• container_name: — drop it; compose already names containers deterministically from the project + service name, and a fixed container_name: is what causes the collision in the first place.

When the finding is a false positive, or fixing it isn’t practical right now, downgrade every finding to a warning and proceed:

dwe test run --skip-isolation-check smoke

See Compose isolation scanner for the full list of flagged constructs and the fail/warn tiering.

Host scripts and the project name

Inside dwe test the copy runs under its own compose project name, and dwe hands that name to the shell steps and host commands it starts as COMPOSE_PROJECT_NAME (see who gets it). A host script that computes its own -p from the project prefix and name ignores it and talks to your working stack — the scenario passes while the script seeds, dumps or truncates the live database.

Take the name from $COMPOSE_PROJECT_NAME. For a script you also run by hand, keep your old value as the fallback:

PROJECT="${COMPOSE_PROJECT_NAME:-dwe-myproj}"
docker compose -p "$PROJECT" exec db psql

For a script only dwe runs, make a missing name fatal instead of falling back to the live stack:

PROJECT="${COMPOSE_PROJECT_NAME:?run this through dwe}"

The :? form also covers the places dwe does not inject the name: a shell when: predicate or the builtin shell probe in a scenario sees no COMPOSE_PROJECT_NAME at all, so a :- fallback there resolves to the live stack.

dwe validate tests catches the common mistake — a -p / --project-name or COMPOSE_PROJECT_NAME= value built without $COMPOSE_PROJECT_NAME, in host commands, scripts, pipeline and scenario shell steps, and scripts they call one level deep. It never guesses, so it cannot see a name passed in as $1, read with read, returned by a command substitution, set in a script two files deep, or a hand-built container name (docker exec myproj-db-1 …); those stay yours to check. See Host scripts that build their own compose project name for the full rule.

Running the whole suite

dwe test run                                # every scenario under workspace/tests/*.yml, sorted
dwe test run smoke db-dump                  # just these two, by name
dwe test list                               # scenario names + descriptions
dwe test run --timeout 5m                   # override every scenario's own timeout: field
dwe test run --skip-isolation-check smoke   # downgrade blocking isolation findings to warnings
dwe test run --parallel 3                   # run up to 3 scenarios concurrently

Exit codes make this CI-friendly out of the box: 0 all passed, 1 at least one failed, 2 a scenario couldn’t even be prepared (bad name, held lock, scenario file error). --output json gives a machine-readable report of the same result.

Running scenarios in parallel

Each scenario already runs fully isolated — its own copy, its own compose project, its own auto-remapped host ports — so nothing stops several from running at once. --parallel N runs up to N scenarios concurrently:

dwe test run --parallel 3                   # up to 3 at a time, across the whole suite
dwe test run --parallel 2 smoke redis-off   # these two, side by side

Effective parallelism is min(N, scenario count), so --parallel 8 over three scenarios runs three workers. At more than one worker the per-scenario streaming output is replaced by a compact live view — one row per scenario with a spinner, coarse phase, and elapsed time, finalizing to ✓ <name> passed or ✗ <name> failed — step "…":

  ✓ [12s] smoke      passed
  ⠹ [4s]  redis-off  deploying…
  ⠹ [6s]  cache-on   running steps…
running 2/3 scenarios…

Each scenario’s full deploy/pipeline log still goes to its own copy (.dwe/tests/runs/<scenario>/.dwe/logs/test.log), and a failing scenario’s report is collected exactly as in a sequential run. Piped/CI runs (no TTY) print flat scenario <name>: started / scenario <name>: passed lines instead of the live block; --output json is unchanged by --parallel.

--parallel 1 (the default) keeps today’s behavior byte-for-byte: full streaming pipeline output, one scenario at a time.

Watch shared caches. Scenarios that all hammer the same shared: true package cache (composer, npm) can contend when run together — cold-cache installs serialize on the package manager’s own lock file. Parallelize scenarios with warm or disjoint caches freely; keep heavy cold-cache installs sequential, and size N to what your Docker daemon can comfortably run at once.

Cross-links

• ../reference/config/tests.md — full scenario schema, isolation model, .dwe/tests/ layout, teardown order, dwe validate tests, the compose isolation scanner, exit codes, documented limitations.
• ../reference/config/deploy/index.md — the full step-field table, including the general timeout: field.
• ../reference/config/deploy/builtins.md — every builtin available to steps:, including http_check and predicate-as-assertion semantics.
• author-project-commands.md — authoring the type: command steps a scenario can call, including private commands.
• preflight-checks.md — the ports_free check, which reads services.<name>.ports and so moves together with the test copy’s host-port remap.