workspace/tests/

Declarative integration-test scenarios (dwe test).

Contents

• Purpose
• File layout
• Scenario schema
  • description
  • env.services
  • env.vars and auto ports
  • timeout
  • steps
• What a scenario run does
• Isolation model
• .dwe/tests/ layout
• Teardown
• Failure reports
• dwe test run
  • --parallel N
• dwe test list
  • Cost profile (--output json)
• dwe test clean
• dwe validate tests
  • Host scripts that build their own compose project name
• Compose isolation scanner
  • Volumes acknowledged by shared: true
  • Interpolated host ports
• Exit codes
• JSON output
• Documented limitations
• Related commands

Purpose

A dwe project’s deploy pipeline is normally only ever exercised against the developer’s one working environment — there’s no way to answer “does a clean deploy still work with these config changes?” without touching (or risking) the environment you work in.

dwe test runs the project’s deploy pipeline — and any follow-up assertions or project commands you declare — inside a fresh, fully isolated, disposable copy of the project. Each scenario is one YAML file under workspace/tests/: one isolated environment, one clean-slate deploy with that scenario’s parameters, followed by ordered steps using the same step schema as deploy.yml.

Isolation targets dwe-managed state only (compose project, ports, volumes, files under .dwe/) — see Documented limitations for what it doesn’t cover.

File layout

workspace/tests/<scenario>.yml — one file per scenario. The scenario name is the file basename, not a field (symmetric with workspace/services/<name>/service.yml): a file at workspace/tests/redis-off.yml defines a scenario named redis-off.

Scenario names must already be valid compose-project-name fragments: lowercase alphanumerics, -, _, no leading separator (^[a-z0-9][a-z0-9_-]*$). Names are never sanitised — a non-matching filename is a hard error at discovery time, for every file under workspace/tests/*.yml/*.yaml.

The loader is strict (KnownFields(true), matching the pipeline-loader family), with one deliberate divergence from that family: an empty or all-comment scenario file is an error, not an absent-and-defaulted document — a scenario has no meaningful default.

An absent workspace/tests/ directory is not an error: dwe test list reports no scenarios and dwe test run (no args) runs nothing.

Scenario schema

description: "Deploy with redis disabled"

env:
  services:
    disable: [redis]        # and/or enable: [...]
  vars:
    app.http_port: auto     # auto = allocate a free host port before deploy
    db.port: auto
    db.password: "test-pw"  # any vars override; lands in the copy's local.yml

timeout: 15m                # wall-clock budget for the whole scenario

steps:                      # ordered, run AFTER the implicit deploy; deploy.yml step format
  - name: "app answers"
    type: builtin
    cmd: http_check
    with: { url: "http://localhost:${vars.app.http_port}/up", status: 200, contains: "OK" }

  - name: "create dump"
    type: command            # regular user command; private commands allowed
    cmd: db:dump

  - name: "dump exists"
    type: builtin
    cmd: file_exists
    with: { path: "dumps/db-latest.sql.gz" }

description

Human-readable summary shown verbatim by dwe test list — not translated (display strings elsewhere in dwe are localised, but scenario descriptions are user content and stay as written, spec §8).

env.services

env:
  services:
    enable: [worker]
    disable: [redis]

Force-enables or force-disables named services in the copy, on top of whatever workspace/local.yml currently has enabled. Maps to services.<name>.enabled: true/false in the generated local.yml (below) — same effect as dwe services enable/disable --apply, just scoped to the copy.

env.vars and auto ports

env:
  vars:
    app.http_port: auto
    db.password: "test-pw"

Each key is a dot-path relative to vars. (app.http_port → vars: { app: { http_port: … } } in the copy’s local.yml), overriding whatever the project or the developer’s own local.yml set.

auto is the one magic value: before deploy, the runner allocates a free host port and writes the concrete number in its place. Scenario steps see the allocated value through ${vars.app.http_port} (loader-side rendering, below).

auto is not required to isolate a host port. Every host port an enabled service declares under services.<name>.ports is remapped automatically, and so is every vars: path a compose host port reads through an exports.env rule (see Automatic host-port isolation below) — a scenario that mentions neither still gets a free port for both. env.vars: { …: auto } remains for the residual case: a var that no compose port reads, which the runner would otherwise have no reason to allocate. Writing auto for a path the runner already remaps is harmless and changes nothing.

Writing a concrete number instead pins the path: the copy binds exactly that port and the automatic remap steps aside. null, "" and a missing key do not pin — the export would fall back to its own default and bind the live stack’s port, so those still get an allocated port. Neither does a map or a list at the path: it is structure, not a port, and the allocated port replaces it.

timeout

Wall-clock budget for the whole scenario (deploy + all steps), e.g. 15m. Parsed with time.ParseDuration and must be strictly positive. Precedence: dwe test run --timeout flag (overrides every scenario) > this field > a 30-minute default. On expiry, the in-flight subprocess or step is killed, the scenario is marked failed, and teardown still runs (under its own fresh deadline, not the expired one).

steps

Ordinary pipeline steps, in the same schema deploy.yml uses — type: shell / command / dwe / builtin, with when: — resolved and executed by the same engine. There is no service:/dir: field on steps (deploy doesn’t have one either); a step that needs to run against a service goes through type: command and lets the user command own that detail. A scenario with no steps: is valid and already useful: “deploy with these parameters succeeds and containers come up healthy.”

A step may also set its own timeout: (e.g. timeout: 5s on an http_check step) — the same general, opt-in engine field documented in Step fields; absent or 0 leaves the step unbounded.

type: command steps dispatch through the project’s regular command registry, including private commands — so a project can define test-only commands (e.g. db:dump above) and keep them out of the everyday command listing while still exercising them from a scenario. hide commands are different: pipelines skip them per the existing contract, so a test-only command should be private, not hide.

Any KindPredicate builtin (file_exists, executable_in_path, tcp_reachable, containers_running, env_keys_present, config_keys_present, the shell predicate, and the http_check builtin below) may be used directly as a step body, where it behaves as an assertion: false fails the step with the predicate’s own message, true succeeds silently. This is a general engine capability (applies to deploy.yml/reset.yml/lifecycle.yml steps too), not test-only — see the builtins reference for the full mechanics, including the always-run exemption that keeps an assertion from being skipped by deploy’s up-to-date gates.

http_check (new builtin, predicate kind) complements tcp_reachable for web stacks that need a few seconds after up before answering:

- name: "app answers"
  type: builtin
  cmd: http_check
  with:
    url: "http://localhost:${vars.app.http_port}/up"
    status: 200
    contains: "OK"          # optional body substring
    retries: 10             # optional; defaults to 0
    interval: 1s
    timeout: 2s

See the builtins reference for the full parameter list and retry semantics.

${...} rendering happens in the pipeline engine, exactly once. Scenario steps resolve through the same path as deploy.yml steps, so cmd, the string leaves of with, check, timeout and a shell when: are rendered at resolve time against the copy’s resolved config — ${vars.app.http_port} resolves to the concrete allocated port. There is no scenario-local pre-pass: rendering is not idempotent, and a second one would double-expand a var whose value is itself a ${...} reference, so a scenario would test something the deploy pipeline never runs.

The same rules as deploy steps therefore apply here: an unrecognized head (${HOME}, a typo) is left as a literal ${...} rather than collapsing to an empty string, and ${param.*} / ${context.*} / ${files.*} / ${generated.*} / ${args} have no source on this path — using one fails the scenario at resolve time with a message naming it. Paths in builtin params (e.g. file_exists path:) resolve relative to the copy’s root — assertions always inspect the test environment, never the original tree.

A failing step fails the scenario; remaining steps are skipped.

What a scenario run does

Per scenario, in order:

1. Acquire a per-scenario flock, load and validate the scenario file.
2. Scan the scenario’s view of the project for compose isolation hazards and plan the host ports the copy will allocate. A blocking finding fails the scenario here, before anything is created.
3. Copy the project into an isolated tree.
4. Generate the copy’s local.yml (seed + this scenario’s env: + the planned ports + a fresh identity) and a docker identity file, then write a durable run manifest.
5. Run dwe validate in the copy (cheap fail-fast).
6. Run dwe deploy run --silent in the copy — the real deploy pipeline, not a re-composed approximation.
7. Run steps: in-process against the copy’s config.
8. Tear everything down (unless --keep).

Isolation model

Tree copy. The runner copies the project into .dwe/tests/runs/<scenario>/ (inside the project root, so it stays within Docker Desktop’s/OrbStack’s file-sharing scope for bind mounts). Copy selection is git-aware: tracked + untracked-but-not-ignored files (git ls-files -co --exclude-standard), always excluding .dwe/, .env, and .git/. A path git lists but that’s absent from the worktree (an uncommitted deletion) is skipped — the worktree wins, so a locally deleted file stays deleted in the copy too. Symlinks are recreated as symlinks; permissions are preserved. Without git (or on a git failure), the runner falls back to a full directory copy with the same exclusions, plus a warning that gitignored artifacts are now included.

The developer’s own workspace/local.yml is gitignored and therefore not copied — instead it’s seeded into the generated local.yml (below), so locally-required vars and locally-enabled services still reach the test deploy.

Compose identity. Each run gets <base>-t-<scenario>-<run-id> as its compose project name (<base> = project.prefix if set, else project.name; <run-id> is 6 random hex chars — the scenario name alone isn’t a uniqueness boundary across two simultaneous clones of the same project). This partitions containers, networks, and non-shared volumes automatically; container lookup stays label-based (com.docker.compose.project), never name-guessed. The identity is written into the copy two ways:

• if the copy has workspace/docker.yml → a generated workspace/docker.local.yml with just project_name: (the local layer always wins);
• if the copy has no workspace/docker.yml → a generated workspace/docker.yml with project_name: plus an explicit empty [] for every args: key, so the generated file stays semantics-neutral (no docker.yml means zero-value args config today — an args:-less generated file would silently opt every command into its defaults, which would be a behavior change). Any stray copied docker.local.yml is removed in this branch, since it would otherwise activate for the first time.

Generated local.yml precedence (low → high):

1. seed = the original project’s local.yml (absent file → empty), with compose.extra and services.<name>.compose.extra stripped (each strip warns — those reference gitignored overlay files the copy doesn’t contain);
2. this scenario’s env.vars / env.services, plus every port the runner allocates for this copy — the remapped services.<name>.ports and every vars: path it remaps, including traced paths the scenario never mentions (below);
3. identity: project: { prefix: <compose project name> } and update: { mode: "off" } (no self-update prompts inside a disposable test run).

Automatic host-port isolation. Every host port declared under services.<name>.ports by a service that will be enabled in the copy is remapped to a freshly allocated free port, written into the generated local.yml as a services.<name>.ports.<x> override (the original port’s scheme is preserved). Because ports_free preflight reads that same field — and a project that sources its compose host bindings from services.<name>.ports (directly, or via an exports.env entry from: services.<name>.ports.<x>) binds from it too — the preflight and the actual bind move together, so a scenario runs alongside the working environment with no port config. Every vars: path the copy remaps — the traced ones below and any env.vars: { …: auto } — is allocated in the same batch. All ports come from one allocation pass (all listeners opened before any is closed, guaranteeing intra-batch uniqueness); the copy’s ports_free preflight still catches host-level races, and on a deploy failure that looks like a host-port bind conflict (with any allocated port present) the runner re-allocates every port and retries the deploy exactly once before failing the scenario.

Interpolated host ports are remapped through the variable’s exports.env rule. A compose host port written as a variable ("${VALKEY_PORT:-6379}:6379") moves when the scanner can trace that variable to a value the copy controls — the last exports.env rule with that name: whose when: is empty or truthy, exactly as .env is rendered:

• from: vars.<path> → always remapped, with no scenario config: the runner allocates a free port for <path> in the same batch as everything else and writes it into the generated local.yml. A scenario that pins the path to a concrete number (env.vars: { ports.valkey: 6380 }) keeps that number instead — the author’s decision about which port the copy binds, not a collision.
• from: services.<name>.ports.<x> → remapped in the scenarios that remap <name>: enabled in the copy and not listed under env.services.disable (a required: true service stays enabled, so it stays covered).
• anything else — another path, a .port sub-path, an undeclared port, a falsy when:, a variable from the host environment or a hand-written .env — is not remapped. The variable keeps whatever value it would have had outside a test run: an expression carrying a default falls back to it ("${VALKEY_PORT:-6379}:6379" → 6379) and a value coming from the host environment or a hand-written .env survives, both of which make the copy bind the working environment’s port; a variable with neither resolves to the empty string, and compose then publishes no host port for that entry at all. The scanner warns in every one of these cases.

The scan behind this runs on the scenario’s view of the project: the working tree with the scenario’s env: applied and the per-developer compose.extra overlays removed — as the copy’s seeded local.yml removes them — evaluated before the copy is made. dwe test list --output json scans the same view, so a compose file reached only through a compose.extra overlay is invisible to everything it reports: the copy never runs it. dwe validate tests builds the view per scenario too, but only to decide coverage — its own scan is project-wide, so a finding of any other kind in such an overlay still warns there.

shared: true volumes resolve to their verbatim names and are reused as-is — the deliberate package-cache exception (composer, npm, …). A shared volume holding real (non-cache) data is therefore visible to every test run too.

State. Journal, locks, generated-value store, prompt cache, logs — everything lives under the copy’s own .dwe/, disjoint from the original. .env is regenerated inside the copy by the deploy itself. The Docker daemon (image/build caches) and ~/.config/dwe (user-level preferences, binary overrides) are shared by design — “all state is project-local” holds for runtime state, not for the daemon or user-level config.

What each run gets on its own:

Guaranteed	How
A unique compose project name	<base>-t-<scenario>-<run-id>, written into the copy’s docker config (above)
Host ports declared under services.<name>.ports	remapped to free ports in the generated local.yml
Host ports a compose file reads from a vars: path through exports.env	a free port from the same batch, written as that vars: path in the generated local.yml
Runtime state	the copy’s own .dwe/ — journal, locks, logs, prompt cache
Generated values	the copy’s own generated-value store, under that .dwe/

Everything else is listed under Documented limitations.

Who sees the copy’s COMPOSE_PROJECT_NAME. The runner strips every COMPOSE_* variable from its own process and runs scenarios concurrently, so no process-wide value can carry the name. dwe derives it from the copy’s config each time it starts a child:

Child process	Gets the copy’s name
type: shell steps and shell check: — scenario steps: and the copy’s deploy	yes, as COMPOSE_PROJECT_NAME
type: shell / type: script commands reached through a type: command step	yes, as COMPOSE_PROJECT_NAME plus COMPOSE_FILE
Container commands (service_exec / service_run) reached through a type: command step	yes — dwe passes the copy’s name as -p
type: dwe steps	yes — the child dwe runs in the copy and resolves the name from the copy’s config itself
Shell when: predicates and the builtin shell probe	no — nothing is injected; in scenario steps: they see no COMPOSE_PROJECT_NAME at all
Anything started outside dwe — a script run by hand, another terminal	no

files_gate only checks files and starts no shell, so it has nothing to receive. A host script that builds its own docker compose -p instead of reading $COMPOSE_PROJECT_NAME bypasses all of this: dwe validate tests warns about it, and the guide has the recipe.

.dwe/tests/ layout

All paths below are relative to the original project root (never the copy):

Path	Purpose
.dwe/tests/runs/<scenario>/	The disposable copy for a scenario’s current/last run
.dwe/tests/locks/<scenario>.lock	Per-scenario flock (never the project-wide deploy.lock/snapshot.lock)
.dwe/tests/manifests/<scenario>-<run-id>.yml	Durable run manifest — written before any Docker interaction
.dwe/tests/reports/<scenario>/	Failure artifacts from the scenario’s most recent non-passing, non---keep run (see Failure reports)

The manifest (scenario, run_id, compose_project, copy_path, bridge_dir, report_dir, created_at) is the sole input to teardown: a run that dies mid-way (crash, --keep, a killed process) is still fully describable from its manifest and copy contents alone, without touching the working environment or guessing at names.

Teardown

Runs by default after every scenario that created an environment (pass/fail/timeout/Ctrl+C) — a scenario stopped by the isolation gate before the copy was made has nothing to tear down — driven only by the manifest, in order: docker compose down --remove-orphans (never -v, and the copy’s args.down policy is bypassed so a project that sets args.down: ["-v"] cannot delete a shared cache volume referenced as a plain named volume in a raw compose file) → reap any remaining containers labelled with the manifest’s exact com.docker.compose.project value → remove the test project’s own volumes (prefix-filtered by compose project name; shared: volumes survive, same semantics as dwe reset) → stop any bridge daemon the deploy started in the copy → remove the copy directory → delete the manifest → release the flock. Each step is best-effort — a failure is logged and later steps still run.

--keep skips every step above, leaves the manifest and copy in place, and prints the compose project name, the copy path, and a cleanup hint. A subsequent dwe test run of the same scenario name fails fast (a kept run’s manifest still exists) rather than silently deleting the kept environment out from under you — clean it up manually, or run dwe test clean.

A non-passing --keep run prints a second line naming where the evidence actually is — <copy>/.dwe/logs/ and docker compose -p <project> logs. No failure report is collected under --keep (the environment is still alive, so a snapshot of it would be redundant), and without that line the absence reads as an omission rather than a deliberate trade.

Failure reports

When a scenario does not pass (deploy failure, step failure, or timeout) and --keep was not used, the runner attempts to collect a failure report into .dwe/tests/reports/<scenario>/ before teardown destroys the environment — so the debugging material survives. The directory is cleared and rewritten on every non-passing run that got as far as creating the environment (the latest failure is what you debug); a passing scenario or a --keep run never touches it. A scenario stopped by the isolation gate is the one other exception: it fails before anything is created, so no collection is attempted and a report left by an earlier failure of that scenario stays in place — check the run’s own warnings, not the directory’s timestamp.

File	Source
pipeline.log	copy of the scenario’s pipeline log (.dwe/logs/test.log inside the disposable copy)
compose-ps.txt	docker compose ps --all against the copy (--all, so a service that crashed or exited during deploy still shows up — the running-only default would drop exactly the service a failure report exists to surface)
container-logs.txt	docker compose logs --no-color --tail 200 for the copy’s containers, combined into one file

Collection is best-effort throughout and runs under its own fresh timeout (never the scenario’s own, possibly-expired, deadline): a missing pipeline log, a docker command that partially fails, or the collector itself timing out is warned and never changes the scenario’s pass/fail outcome. If the copy’s docker config can’t be loaded (the same condition that can break deploy itself), collection falls back to docker ps -a / docker logs, filtered by the run’s exact com.docker.compose.project label, so a report is still produced even when compose is unusable.

The report path is surfaced in dwe test run’s text output (a report: <dir> line under a non-passing scenario) and in its JSON output as report_dir (empty for a passing scenario, a --keep run, or when the report directory could not be created).

dwe test run

dwe test run [scenario...]
    --keep                    # skip teardown; print project name, copy path, cleanup hint
    --timeout <duration>      # override every scenario's own timeout (e.g. 15m)
    --skip-isolation-check    # downgrade blocking isolation findings to warnings
    --parallel N              # run up to N scenarios concurrently (default 1)

No arguments runs every scenario under workspace/tests/*.yml, in sorted name order. Named arguments run exactly those scenarios (an unknown name fails before anything runs, exit code 2). By default scenarios run sequentially. Ctrl+C (SIGINT/SIGTERM) cancels the scenario(s) currently running, tears them down, and skips the rest — already-completed scenarios are still reported.

Output is the standard live pipeline reporter per scenario (the same look as dwe deploy run), followed by a summary line, e.g.:

2 passed, 1 failed (redis-off: step "tests/app answers")

dwe test requires a project — unlike read-only docs commands, it is not usable outside one.

The global --verbose / --debug flags propagate into the copy’s own dwe validate and dwe deploy run subprocesses, so the diagnostic level applies to the deploy the scenario actually exercises — not just the runner’s own bookkeeping. In the default sequential text mode the subprocess trace streams to stderr live; under --parallel or --output json (where the subprocess output is not streamed) it is captured in that copy’s run log (.dwe/tests/runs/<scenario>/.dwe/logs/test.log). --debug wins over --verbose (it is a superset), matching every other command.

--parallel N

--parallel N (default 1) runs up to N scenarios concurrently. Effective parallelism is min(N, scenario count) — --parallel 8 with two scenarios runs two workers; --parallel 8 with one scenario runs one. Ordering of the output (text summary and JSON scenarios array) is always the original name order, independent of completion order.

• --parallel 1 (the default) is byte-identical to today. When effective parallelism is 1 — the flag is absent, set to 1, or there are fewer scenarios than requested workers — the sequential streaming path runs unchanged: the standard live pipeline reporter per scenario, exactly as dwe deploy run looks.
• At effective parallelism > 1 the streaming output is replaced by a compact aggregated view. One sticky row per scenario shows a spinner, the scenario name, a coarse phase (preparing…, validating…, deploying…, deploy retry…, running steps…, collecting report…, tearing down…), and an elapsed stopwatch; on completion the row finalizes to ✓ <name> passed, ✗ <name> failed — step "…", or ✗ <name> error (a scenario that could not be prepared). A footer tracks running k/n scenarios…. When there are more scenarios than the terminal has room for (the sticky block is capped to the terminal height), the scenarios beyond that budget report via the same flat scenario <name>: started / scenario <name>: <status> lines rather than a sticky row. The per-scenario deploy/pipeline output is not streamed to the terminal — it goes to that copy’s own run log only (.dwe/tests/runs/<scenario>/.dwe/logs/test.log), and a failing scenario’s failure report is collected as usual. Warnings are prefixed [<scenario>] warning: … and printed to stderr without disturbing the block.
• Piped / non-TTY runs (CI) degrade to flat scenario <name>: started / scenario <name>: <status> lines per scenario instead of the live block — the summary and exit code are unchanged.
• JSON mode (--output json) is unaffected by --parallel: the payload shape is identical and, as with every read-only surface, live output and warnings are silenced regardless of parallelism.

Exit codes are unchanged (see Exit codes): any scenario that could not be prepared → 2, else any failed scenario → 1, else 0.

Isolation holds unchanged under parallelism. Each scenario already runs in its own copy dir, under its own per-scenario flock, with its own per-run-id compose project and manifest, and with every host port auto-remapped to a freshly allocated free port (see Isolation model). Port allocation is additionally process-wide race-safe: a lease set guarantees two concurrent scenarios in the same dwe test run never receive the same host port (cross-process races between separate invocations stay covered by each copy’s ports_free preflight plus the one deploy retry).

Shared package-cache contention. Scenarios that reuse the same shared: true cache volume (a composer or npm cache, say) can contend when run in parallel — package managers take lock files, and simultaneous cold-cache installs against one volume can slow each other down or trip a manager’s own locking. Prefer not to parallelize scenarios that each perform a heavy cold-cache install against a shared cache; scenarios with warm caches or disjoint caches parallelize cleanly. The Docker daemon load of N simultaneous deploys (image pulls, builds, container starts) is your call — pick N to match the host.

dwe test list

dwe test list

Lists every scenario under workspace/tests/*.yml with its description:, verbatim. An absent workspace/tests/ directory lists nothing and is not an error.

Cost profile (--output json)

dwe test list --output json carries a cost_profile object per scenario — what running that scenario would cost, and how far its isolation reaches. Every chain-derived number is computed on the scenario’s view of the project, so the per-developer compose.extra overlays are excluded: a compose service that exists only in workspace/local.yml counts toward nothing here, because the copy never receives it.

{
  "scenarios": [
    {
      "name": "redis-off",
      "description": "Deploy with redis disabled",
      "cost_profile": {
        "enabled_services": 3,
        "build_services": ["app"],
        "external_images": ["postgres:16", "redis:7"],
        "max_start_period_seconds": 30,
        "shared_volumes": 1,
        "isolation_findings": [{"kind": "external_volume", "resource": "composer-cache", "shared": true}],
        "host_steps": 2
      }
    }
  ]
}

Field	Meaning
enabled_services	dwe services enabled after this scenario’s env.services overlay — the number that actually differs between two scenarios of the same project. A required: true service stays enabled even when a scenario disables it, exactly as the config loader resolves the generated local.yml.
build_services	Compose services declaring build: in the enabled chain, sorted.
external_images	Distinct image: references of compose services that do not build locally, sorted — what a cold run would have to pull. A service that builds carries a local tag (image: myproject-app:dev), which is not something to pull and is therefore excluded.
max_start_period_seconds	The largest healthcheck start_period in the enabled chain. Max, not sum: docker compose up --wait waits in parallel, so a sum over-estimates the more services a project has. A disabled or unparseable healthcheck contributes nothing.
shared_volumes	Count of docker.yml volumes declared shared: true. These resolve to their verbatim names and are written into by a test run.
isolation_findings	The non-blocking findings of the compose isolation scanner — named_volume / external_volume / named_network / external_network, i.e. resources shared with the working environment, plus interpolated_host_port, a host port interpolated from a variable this scenario’s copy does not remap. The blocking kinds (container_name, raw_host_port) are omitted: they abort the scenario before the copy is even made. A volume acknowledged by a docker.yml shared: true declaration stays listed and carries "shared": true (the key is omitted otherwise) — the profile reports facts, not verdicts, so a caller that only wants unacknowledged hazards filters on it. Full messages come from dwe validate tests.
host_steps	Steps this scenario would run on the host, outside the container sandbox: its own steps, the deploy pipeline it triggers (project-wide plus the enabled services’), and the workspace/validate.yml checks the run’s dwe validate and dwe deploy run preflight execute in the copy — a check counts when it is type: builtin with cmd: shell or a type: command (the loader restricts those to shell / script user commands), subject to the same services: gate the checks loader applies. Pipeline steps are counted descending into parallel: groups. The counted step forms are type: shell, type: builtin with cmd: shell, a type: dwe whose subcommand re-enters project-authored code (cmd, run, restart, deploy, reset, test), a type: command whose command resolves to a host-executing one, and any step carrying a shell when: or a host-executing check: (including check: auto). A referenced command counts as host-executing when it declares argv_append_from: (the expression runs on the host even for a container command), or — absent that — when it is neither service_exec nor service_run, applying the same type: dwe / type: builtin rules recursively and, for a workflow, counting a sub-step that is itself host-executing or carries a cmd: when:. dwe’s own subcommands (docker up --wait, info, render … — the whole built-in default pipeline) are not counted: they are dwe machinery acting on the disposable copy, which is what the scenario exists to run. A step is counted once however many of those it uses, and a phase-level shell when: counts as one. A type: command reference that does not resolve counts as a host step — the field gates an unattended run, so the unknown case closes the gate. Host side effects are not sandboxed.

Two properties are deliberate:

• Facts only — there is no cheap/expensive verdict field. What counts as cheap enough to run unattended is a policy that belongs to the caller (for AI agents, the dwe skill states the rule), not to the CLI.
• It reports whether there is a build, not what the build costs. The dominant factor — whether the Docker layer cache is warm, seconds versus many minutes — has no static source and is not modelled.

The profile is omitted (not empty, not an error) when the project state it needs does not load: dwe test list takes no locks, touches no Docker, and requires no loadable config — it is the command you reach for while the config is mid-edit, and it keeps working there. The plain-text listing never computes a profile at all.

dwe test clean

dwe test clean [scenario...]
    --dry-run                 # report what would be swept without tearing anything down

Removes test environments left behind by dwe test run --keep or an interrupted/crashed run. clean is strictly manifest-driven: it enumerates .dwe/tests/manifests/*.yml and reuses the exact same Teardown a normal scenario run performs at the end of dwe test run — nothing is ever destroyed by guessing a compose project name from a pattern.

With no arguments, every manifested scenario is swept. Passing scenario names restricts the sweep to those. A scenario whose flock (.dwe/tests/locks/<scenario>.lock) is currently held by a live dwe test run is skipped, never torn down — clean never contests a run in progress. --dry-run reports what would be swept without tearing anything down (each scenario’s flock is still acquired-and-released to correctly classify a live run as skipped rather than sweepable).

If a manifest’s teardown does not complete cleanly (e.g. a container or volume removal step failed partway through), that entry is reported as failed, never as swept — Teardown is best-effort and may have removed some resources (even the manifest itself) before hitting the failure, so counting it as swept would hide a real leftover from the next run.

A best-effort, report-only scan additionally lists Docker compose projects matching this project’s test-name prefix (<base>-t-, the same base dwe test run uses) that have no manifest at all — these are reported as orphans and are never destroyed automatically; remove them by hand once you’ve confirmed they’re safe to drop. If the project’s own root config can’t be loaded, the orphan scan is skipped (warned) but every manifested environment is still swept — a broken or mid-edit config must not block a recovery sweep.

JSON output

dwe test clean --output json

{
  "dry_run": false,
  "swept": [{"scenario": "smoke", "compose_project": "myapp-t-smoke-a1b2c3", "copy_path": ".dwe/tests/runs/smoke"}],
  "skipped": [{"scenario": "redis-off", "compose_project": "myapp-t-redis-off-9f1e2d", "copy_path": ".dwe/tests/runs/redis-off", "reason": "live"}],
  "failed": [],
  "orphans": [{"compose_project": "myapp-t-old-9f8e7d", "note": "no manifest — remove manually"}]
}

As with run/list, live output (per-entry warnings) is silenced on stderr in JSON mode.

dwe validate tests

dwe validate tests

A dwe validate domain (Domain() == "tests") that statically checks every workspace/tests/*.yml scenario file without touching Docker — validate-only, never wired into preflight. Run it in CI before dwe test run to catch scenario authoring mistakes without spinning up a disposable copy.

Per file, in order:

• load — LoadScenario (strict KnownFields(true), empty file rejected); this also covers name validation (ValidateScenarioName rejects — does not rewrite — a bad file basename), so an invalid filename surfaces here.
• timeout: — the scenario’s own timeout: field must parse via time.ParseDuration and be strictly positive (mirrors resolveScenarioTimeout’s runtime contract); a parse failure or non-positive duration is an error.
• env.services — every enable/disable entry must name a service that exists in the project’s merged config; an unknown name is an error.
• steps — all steps are rendered and resolved as one whole phase, exactly like a real run (pipeline.ResolvePhaseSteps over a single synthetic phase) — this catches step schema errors, invalid builtin with: params, broken when: conditions, and duplicate top-level step names (a per-step check would miss the last one, since uniqueness is a whole-phase invariant). Rendering substitutes any env.vars entry whose value is the literal auto with a valid placeholder host port, so ${vars.db.port} renders to a valid int and a tcp_reachable/http_check step validates normally — a genuinely bad param (e.g. status: nope) still errors even next to a templated url:. A var populated only post-deploy (a ${generated.*} secret, or a var the deploy itself creates) is absent at validate time and may produce a spurious diagnostic; give it a project-level default to avoid this — the validator sees pre-deploy config, the real run sees post-deploy config.
• type: command steps — each command ID is looked up in the project’s command registry; an unknown ID is an error.
• compose isolation — see Compose isolation scanner below; findings that are not acknowledged as shared are emitted once per project as warnings (an interpolated_host_port finding only when some scenario leaves its port unremapped; never errors — the tiered fail/warn policy applies only to dwe test run, not to static validation).

The domain also scans the host shell text scenarios can run — see Host scripts that build their own compose project name.

Host scripts that build their own compose project name

A second validator, tests.host_project_name, warns when host shell text hands docker compose a project name not derived from $COMPOSE_PROJECT_NAME. Inside dwe test, shell steps and scripts get the copy’s name in COMPOSE_PROJECT_NAME; a script that assembles its own -p from the project prefix and name addresses the live stack instead. Like the rest of the domain it is validate-only, stays silent without workspace/tests/, and its warnings fail dwe validate --strict.

Sources:

Source	Text scanned
type: shell user commands in every command file, hidden ones included	cmd, or the -c payload of a POSIX-shell argv ([sh, -c, <payload>], [bash, -lc, <payload>])
type: script user commands whose script.shell is empty or a POSIX shell (sh, bash, dash, ksh, zsh)	the path, plan, run and cleanup files
type: shell steps and shell check: (including parallel: sub-steps) in workspace/deploy.yml, workspace/reset.yml, workspace/lifecycle.yml (run and stop), each service’s deploy.yml / reset.yml, and every scenario	cmd
A script an inline cmd above runs as bash <path>, sh <path> or ./<path> (path relative to the project root)	the file, one level deep; a ./<path> file only when it has no shebang or a POSIX-shell one

A file reached from several sources is scanned once. The scanner looks at compose invocations in command position — docker compose, docker-compose or ${DOCKER:-docker} compose, also behind VAR=value prefixes and sudo, env, exec, command, time, nice with their options (sudo -u root, nice -n 10) — reads the -p / --project-name global flag, and also checks every COMPOSE_PROJECT_NAME= assignment. A value that references $COMPOSE_PROJECT_NAME anywhere passes (${COMPOSE_PROJECT_NAME:-dwe-myproj}, ${OVERRIDE:-$COMPOSE_PROJECT_NAME}). For a bare $PROJECT it reads every assignment to PROJECT in the same text and warns only when all of them build the name without $COMPOSE_PROJECT_NAME; when they disagree it stays silent. A -p after the subcommand (docker compose exec db psql -p 5432) is not compose’s flag and is never read.

PROJECT="${PROJECT_PREFIX:-dwe}-${PROJECT_NAME:-myproj}"
docker compose -p "$PROJECT" exec db psql     # warns

The fix keeps the old expression as the fallback, so the script still works when run by hand:

PROJECT="${COMPOSE_PROJECT_NAME:-${PROJECT_PREFIX:-dwe}-${PROJECT_NAME:-myproj}}"

The warning’s hint prints this shape with the offending value filled in. In a script file the location is line N; in YAML it is the command ID or step address plus the line inside cmd.

The scanner never guesses. It skips, and never flags:

• a literal name (-p myproj), a positional ($1, ${1:-…}) and a command substitution ($(…), backticks);
• a variable with no assignment in the same text — set by read, a for loop, or another file;
• a script more than one referenced file deep, and a referenced path containing an expansion;
• a referenced path under a command’s workdir: — references resolve against the project root only;
• docker --context … compose, which is not anchored;
• an argv without a shell and -c — [docker, compose, …] runs without a shell, and a script file in [bash, <path>] is not followed;
• scripts for other interpreters (script.shell: python3, a ./tool.py with a python shebang);
• files outside the project (an absolute path, a ../ path, a symlink leading out) and files over 1 MiB;
• container-side commands.

Compose isolation scanner

dwe test’s isolation model (above) partitions containers, networks, and non-shared volumes by compose project name — but a handful of raw-compose constructs bypass that scoping entirely and can collide with, or attach to, the working environment. The scanner (config.ScanComposeIsolation) parses the project’s active compose files (cfg.ComposeFiles()) for these constructs and flags them:

Construct	Kind	Severity
container_name: (any occurrence)	container_name	Blocking — a literal container name collides directly with the working environment
A literal host port (single, e.g. "8080:80", or range, e.g. "8080-8090:80-90") not modelled via services.<name>.ports	raw_host_port	Blocking — bypasses both the automatic port remap and ports_free
A host port that is exactly one variable ("${VALKEY_PORT:-6379}:6379", published: "${PORT}")	interpolated_host_port	Warning when the variable is untraced, or reads a service port the scenario does not remap; a variable traced to a vars: path is remapped automatically and never warns. See Interpolated host ports
external: true volume/network	external_volume / external_network	Warning — a shared-resource hazard, not a hard collision
An explicit name: on a volume/network	named_volume / named_network	Warning — same hazard class as external:

The two volume kinds are silenced when the volume is declared shared: true in docker.yml.

Volumes acknowledged by shared: true

A volume the project already declares under docker.yml resources.volumes.<key> with shared: true is a deliberate cross-project resource — a package cache dwe creates itself through ensure_before — not an isolation mistake. The scanner reads workspace/docker.yml (plus docker.local.yml) and marks such findings acknowledged, so the recipe below is silent instead of warning twice per volume on every run:

workspace/docker.yml

resources:
  volumes:
    npm_cache:
      name: dwe_npm_cache
      shared: true

compose.yaml

volumes:
  npm_cache:
    external: true
    name: dwe_npm_cache

The match is by effective name: the volume’s explicit name: after the -f chain collapses (a later overlay’s name: wins), else the name carried by a legacy long-form external: { name: … }, else the compose map key when neither is set, compared against the shared docker.yml volume’s own name (a shared: true volume resolves to its name: verbatim, without the project prefix). A mismatch is not acknowledged, and neither is shared: false. Both the external_volume and the named_volume finding for that volume are marked together.

Networks are never acknowledged — docker.yml declares no network resources, so there is nothing to match against. Blocking kinds are never acknowledged either.

Container-port-only entries (random host port) are not flagged, and neither is a host-port token that mixes a variable with other text (80${X}, ${A}${B}) — only a literal port number or range, or a token that is exactly one variable. IPv6-bracketed hosts ([::1]:8080:80) are out of scope and are not flagged (rare in dev compose). A container_name: finding is always emitted regardless of which compose service it’s on — mapping a compose service back to a dwe service key isn’t reliable, so false positives are cleared with --skip-isolation-check rather than suppressed at the source.

The scanner itself has no opinion on severity beyond the intrinsic Blocking flag — each caller decides what to do with a finding:

• dwe test run — runs the scan on the scenario’s view of the project before the copy is made (the same scan decides which vars: paths get an allocated port). Every finding that is not acknowledged as shared, and not an interpolated host port the scenario covers, is printed as a warning. A blocking finding fails the scenario immediately, with nothing created: no copy, no compose project, no teardown and no failure report collected for that run (one left by an earlier failure of the same scenario stays in place) — the scenario is reported failed (exit code 1), and under --keep there is no kept copy to print. --skip-isolation-check downgrades every finding — blocking or not — to a warning and the run proceeds. See the ports prerequisite for how to avoid a raw_host_port finding in the first place: model the port under services.<name>.ports.
• dwe validate tests — emits every finding that is not acknowledged as shared as a warning (an interpolated host port only while some scenario leaves it uncovered), regardless of Blocking; static validation never fails a build over an isolation hazard, it only surfaces it early.
• dwe test list --output json — lists the non-blocking findings the scenario does not cover under cost_profile.isolation_findings, acknowledged ones included and marked "shared": true.

Interpolated host ports

A host port written as a variable ("${VALKEY_PORT:-6379}:6379", $PORT, ${PORT?} …) moves only when the copy controls the variable’s value. The scanner follows the variable to its exports.env rule — the last one with that name: whose when: is empty or truthy, exactly as .env is rendered — and what the rule reads decides the rest:

The variable’s rule	Covered in a scenario when	Message
from: vars.<path>	always — the runner allocates a free port for <path> in every copy, and a scenario that sets env.vars: { <path>: <number> } pins its own port instead	states the fact: the variable is remapped through vars.<path>
from: services.<name>.ports.<x>, with <x> declared under services.<name>.ports	the runner remaps <name>’s ports — the service is enabled in the scenario and not listed under env.services.disable (a required: true service cannot be disabled, so it stays covered)	says <name>’s ports are remapped only in scenarios where it is enabled and not listed under env.services.disable
anything else — another path, a .port sub-path, an undeclared port, a falsy when:, no rule at all (host environment, a hand-written .env, the :- default)	only when its compose file is not in the scenario’s stack (see below)	names the two routing fixes: export the port from a declared service port (from: services.<name>.ports.<port>), or from a vars.<path> through an exports.env rule, which dwe test then remaps automatically

The first row is why a traced variable needs no scenario config at all, and why removing an env.vars: { <path>: auto } line changes nothing. A required: true service listed under env.services.disable stays enabled in the copy, exactly as the config loader resolves the generated local.yml, so its ports are still remapped and a second-row finding is covered.

Whatever the rule, a scenario also covers the finding when the compose file that publishes the port is not in its stack: a service’s own compose files (compose: and compose_after: in its service.yml, plus its local.yml overlays) leave the chain when the scenario disables the service, so nothing in them binds a port in that copy. A required: true service stays enabled, so its files stay in; a service declared in the root compose file never leaves it. A file reached only through a per-developer compose.extra overlay is never in the stack — no view carries those.

The finding is never blocking. Each caller filters it by its own view:

• dwe test run drops it when the scenario being run covers it — always, for a vars:-traced variable — and prints it as a warning otherwise.
• dwe test list --output json does the same per scenario, so cost_profile.isolation_findings never lists a vars:-traced variable; when: conditions are evaluated with the scenario’s service toggles applied.
• dwe validate tests has no scenario of its own, so it drops the finding only when every scenario covers it, and otherwise appends (not covered in scenarios: a, b). A vars:-traced variable is therefore silent in a project whose scenarios all load, and a scenario file that fails to load covers nothing. A finding of the last table row warns without a scenario list, unless no scenario’s stack includes its compose file. Here when: is evaluated on the project’s own config, so a rule gated on a service that is off by default but enabled by every scenario still warns.

Because dwe test run runs dwe validate inside the copy, and the copy carries every scenario file, dwe test run a still streams this project-wide warning while scenario b leaves an untraced variable uncovered. It does not block the run.

Exit codes

Code	Meaning
0	Every scenario passed (run); sweep completed, including nothing to sweep (clean — skipped-live entries and orphans do not affect this)
1	At least one scenario failed — deploy failure, step failure, or timeout (run); at least one manifest’s teardown did not complete cleanly (clean)
2	A scenario — or the run itself — could not even be prepared: unknown scenario name, scenario load/parse error, flock held by a concurrent run, a kept prior run’s manifest still present (run)

clean maps a hard error (e.g. an unreadable manifests directory) to a non-zero exit through the standard CLI error envelope, separately from the 0/1 sweep-outcome codes above.

JSON output (run / list)

dwe test run --output json
dwe test list --output json

{
  "scenarios": [
    {"name": "redis-off", "status": "passed", "duration_seconds": 4.213},
    {"name": "cache-on", "status": "failed", "failed_step": "tests/http_check", "duration_seconds": 2.101, "report_dir": ".dwe/tests/reports/cache-on"}
  ],
  "summary": "1 passed, 1 failed"
}

status is one of passed, failed, error (error = the scenario could not be prepared — copy/config/manifest/validate failure; distinct from a deploy or step failure, which is failed). failed_step and report_dir are omitted when empty (a passing scenario has neither). report_dir is the failure report directory for a non-passing scenario; omitted for a passing scenario, a --keep run, a scenario stopped by the isolation gate before the copy was made (no collection is attempted), or when collection could not create the report directory. As with every other read-only/report surface, live pipeline output and the summary line are silenced in JSON mode — the file log under .dwe/logs/ still records everything. The isolation gate is the exception: it fires before the run log exists, so under --output json a scenario it stops shows up only as failed with neither failed_step nor report_dir, and its finding messages are not recorded anywhere — re-run without --output json, or run dwe validate tests, to read them.

dwe test list --output json additionally carries a per-scenario cost profile.

Documented limitations

• .git/ is excluded from the copy. A deploy or scenario step that shells out to git against the project root will fail or behave differently inside the copy.
• Named compose resources bypass isolation. container_name:, explicitly named networks/volumes, and external: true in raw compose files ignore the compose project-name scoping and can collide with — or attach to — the working environment. This is why teardown never uses compose down -v. The compose isolation scanner detects these constructs and, for container_name: and literal host ports, fails the scenario before deploy (downgradeable with --skip-isolation-check) — it does not make them safe, it surfaces them before they cause a collision. A volume declared shared: true in docker.yml is acknowledged and reported no longer: it still bypasses isolation, deliberately.
• Names a script builds itself are not isolated. A host script that assembles a compose project name (docker compose -p "${PREFIX}-${NAME}") or a container name (docker exec myproj-db-1 …) from the project prefix and name reaches the working environment, not the copy. Take the name from $COMPOSE_PROJECT_NAME (see the guide). dwe validate tests warns about a self-built -p or COMPOSE_PROJECT_NAME= value, but not about a hand-built container name.
• Host ports dwe cannot trace aren’t isolated. The automatic remap covers ports declared under services.<name>.ports and vars: paths an exports.env rule routes into a compose interpolation; a host port hardcoded straight in a raw compose file (8080:8080), or interpolated from a variable no active rule traces, bypasses it — and ports_free preflight sees neither. Declare the port under services.<name>.ports, or export the variable from a vars.<path> through an exports.env rule. The isolation scanner flags a literal as a blocking raw_host_port finding and an untraced variable as a non-blocking interpolated_host_port warning — only a warning, so a scenario that ignores it still binds the original port.
• Host side effects of a project’s own deploy/scenario steps aren’t sandboxed. A shell step touching absolute paths, ~, or bind mounts outside the project affects the real host, same as it would from a real deploy. dwe test isolates dwe-managed state (files, containers, volumes, networks, ports) — not arbitrary side effects a step chooses to have.
• The copy is not atomic. Nothing locks the original project while git ls-files and the copy run; editing files during a test run can produce a mixed snapshot.
• ~/.config/dwe and the Docker daemon’s image/build caches are shared, by design (see Isolation model).

Related commands

• dwe deploy run — the real command run inside each scenario’s copy
• dwe validate — the fail-fast check run before deploy inside each copy
• dwe validate tests — static scenario validation without Docker (see above)
• dwe reset run — shares the volume-removal semantics teardown reuses
• deploy.yml / reset.yml — the step schema steps: reuses (types, with:, when:)
• Builtins — http_check and predicate-as-assertion step bodies
• Conditions and Actions — typed conditions/actions available to when: and step bodies