End-to-end testing with Probe

Probe can act as the observer of an end-to-end test: your test (a browser, an app on an emulator, an API client) sends its traffic through Probe, and Probe tells you exactly what that test sent and what came back. The same eight steps work from a script over the REST API and from an AI agent over the MCP tools.

Everything below lives in Probe's memory: a test run and its test rules are never saved, never sent to the app and never change your saved Map Local, Map Remote or Mock rules.

The loop

Set up once: read the REST port and token from api-handshake.json in Probe's data folder (~/.probe on macOS, %USERPROFILE%\.probe on Windows, or $PROBE_DATA_DIR when that is set).

DATA=${PROBE_DATA_DIR:-$HOME/.probe}
REST="http://127.0.0.1:$(jq -r .port "$DATA/api-handshake.json")"
TOKEN=$(jq -r .token "$DATA/api-handshake.json")
api() { curl -s -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' "$@"; }

Begin a run. Requests that start after this belong to it. When several test shards share one Probe, give each its own scope header value and have the shard send that header. expect_hosts checks up front that Probe can see those hosts (an ignored or non-intercepted host answers host_decrypted: false with a hint).
Terminal window
```
RUN=$(api -X POST "$REST/api/v1/test-runs" \
  -d '{"name":"checkout","scope":{"header":{"name":"X-Probe-Test","value":"shard-1"}},"expect_hosts":["api.example.com"]}' | jq -r .id)
```
MCP: probe_begin_test_run (header_value, expect_hosts).
Drive the app. Run your test with its traffic going through Probe (see client setup); the shard sends X-Probe-Test: shard-1.
Wait for the traffic instead of sleeping. It answers as soon as enough matching responses exist, or matched: false after the timeout (not an error: call again with the same cursor).
Terminal window
```
api "$REST/api/v1/traffic/wait?test_run_id=$RUN&host=api.example.com&path=/orders&min_count=1&timeout_ms=20000"
```
MCP: probe_wait_for_request.
Assert on what was sent. A failing assertion is still HTTP 200; read passed and each result's reason.

A browser also loads scripts, styles, images and fonts, so a count over the whole run counts those too: pass a filter (host / path) to count only the requests you mean.
Terminal window
```
api -X POST "$REST/api/v1/traffic/assert" -d "{\"test_run_id\":\"$RUN\",\"filter\":{\"host\":\"api.example.com\",\"path\":\"/orders\"},\"assertions\":[
  {\"target\":\"count\",\"op\":\"equals\",\"value\":1},
  {\"target\":\"status\",\"op\":\"lessThan\",\"value\":400},
  {\"target\":\"req_header:Authorization\",\"op\":\"exists\"}]}"
```
MCP: probe_assert_traffic (a failed assertion is a tool error listing the failures).
Stage a mock or a fault for this run only with a test rule: a Map Local response, a Map Remote rewrite, or a Mock rule served by the run's own listener (mock_base_url in the answer). The rule applies only inside the run's scope and is tried before your own rules; its responses carry X-Probe-Test-Run and X-Probe-Rule. The run must have a scope (header value, recommended, or client or host): a run begun without one refuses test rules with 409 runScopeRequired, since they would rewrite every client's traffic.
Terminal window
```
api -X POST "$REST/api/v1/test-runs/$RUN/rules" \
  -d '{"kind":"map_local","rule":{"url_pattern":"https://api.example.com/orders","inline_body":"{\"error\":\"down\"}","inline_format":"json","status_code":503}}'
```
MCP: probe_add_test_rule.
Reset the run's rules between scenarios (the run stays), or DELETE /api/v1/test-runs?all=true to end every run and remove every test rule.
Terminal window
```
api -X DELETE "$REST/api/v1/test-runs/$RUN/rules"
```
MCP: probe_reset_test_rules.
Export a HAR of the run for your CI artifacts or another tool. Bodies Probe did not keep are never written as content; _probe says why.
Terminal window
```
curl -s -H "Authorization: Bearer $TOKEN" "$REST/api/v1/traffic/export/har?test_run_id=$RUN" -o probe-run.har
```
MCP: probe_export_har (with output_path the probe-mcp process writes the file).
End the run. settle_ms waits for requests already in flight to land. An ended run stays readable for an hour; GET /api/v1/test-runs/{id} (MCP: probe_get_test_run) shows its stats (evicted, cleared, capture_interrupted tell you when some requests may be missing).
Terminal window
```
api -X POST "$REST/api/v1/test-runs/$RUN/end" -d '{"settle_ms":500}'
```
MCP: probe_end_test_run.

A run id that belongs to an earlier Probe process answers 404 with code runFromPreviousInstance: begin a new run after Probe restarts.

Client setup

The proxy port is Probe's proxy port (9099 by default). Probe's CA must be trusted by whatever sends the traffic. The certificate server is on the proxy port minus one (9098 by default) and serves:

Path	Format
`/download/cert.pem`	PEM, for Node, curl, Python, browsers' test profiles
`/download/cert.der`	DER
`/download/android`	DER, the format the Android installer expects
`/download/ios`	iOS configuration profile (`.mobileconfig`)
`/download/macos`	PEM, served as a `.crt` file for macOS (the same certificate as `cert.pem`)

An agent doesn't need this table: probe_get_ca (REST GET /api/v1/ca) returns the CA's PEM, its SHA-256 fingerprint, the spki_sha256 value for the Chromium flag below, these download URLs, the proxy address, whether this computer trusts the CA, and one line per client from the recipes below. To trust the CA for every app on this Mac or PC, probe_install_ca (REST POST /api/v1/ca/install) asks the running app to run its own Install button; a person must approve the operating system's prompt on screen, an already trusted CA answers already_trusted without a prompt, and on macOS Probe's other windows must be closed first (409 childWindowsOpen otherwise). Without the app, trust the CA per process instead.

The Playwright browser recipe below was run against a live Probe. The other recipes, including Node-side clients with NODE_EXTRA_CA_CERTS, were checked against the tools' own documentation only.

curl

curl -x http://127.0.0.1:9099 --cacert probe-ca.pem https://api.example.com/orders

Fetch probe-ca.pem once with curl -o probe-ca.pem http://127.0.0.1:9098/download/cert.pem.

Python

REQUESTS_CA_BUNDLE=probe-ca.pem HTTPS_PROXY=http://127.0.0.1:9099 python my_test.py   # requests
SSL_CERT_FILE=probe-ca.pem HTTPS_PROXY=http://127.0.0.1:9099 python my_test.py        # urllib, httpx and other OpenSSL users

Playwright

Browsers: point the context at the proxy and make Chromium trust Probe's CA. The preferred way pins the CA's public key, so only certificates Probe issues are accepted:

# base64 SHA-256 of the Probe CA's public key (SPKI)
openssl x509 -in probe-ca.pem -pubkey -noout | openssl pkey -pubin -outform der \
  | openssl dgst -sha256 -binary | openssl base64

export default defineConfig({
  use: {
    proxy: { server: 'http://127.0.0.1:9099' },
    launchOptions: { args: [`--ignore-certificate-errors-spki-list=${process.env.PROBE_CA_SPKI}`] },
    extraHTTPHeaders: { 'X-Probe-Test': 'shard-1' },
  },
});

The quick option is ignoreHTTPSErrors: true in use: it works in every browser, but it accepts any certificate, not only Probe's. API clients that run in Node (request fixture, fetch) need the CA instead: NODE_EXTRA_CA_CERTS.

NODE_EXTRA_CA_CERTS=probe-ca.pem npx playwright test   # Node-side HTTP clients

extraHTTPHeaders puts the shard header on every request the test sends, which is how a run scoped to shard-1 recognises them.

Android emulator

Start the emulator with Probe as its proxy, then install the CA from the emulator's browser:

emulator -avd Pixel_API_34 -http-proxy http://127.0.0.1:9099

10.0.2.2 is the host machine as the emulator sees it. Then follow Install CA on Android. Since Android 7, apps do not trust user-installed CAs unless they opt in: a debug build needs a network_security_config that trusts user certificates, otherwise only browsers and apps that already trust the user store show their HTTPS traffic.

iOS simulator

Add the CA to the simulator's keychain:

curl -o probe-ca.pem http://127.0.0.1:9098/download/cert.pem
xcrun simctl keychain booted add-root-cert probe-ca.pem

The simulator has no proxy setting of its own: it uses the Mac's network configuration, so its traffic only reaches Probe when Probe has turned on the system proxy. That is the trade-off: the isolated CI mode below (PROBE_NO_SYSTEM_PROXY=1) leaves the system proxy alone, so it cannot capture the iOS simulator. For simulator tests, run Probe normally with Settings → General → Auto-enable system proxy on (Probe switches the system proxy off again when you stop it). See Install CA on iOS for a physical device.

CI scripts

scripts/e2e/probe-e2e.sh (macOS) and scripts/e2e/probe-e2e.ps1 (Windows) wrap the loop around your test command:

scripts/e2e/probe-e2e.sh --app ./Probe.app/Contents/MacOS/Probe --shard shard-1 \
  --assertions asserts.json --artifacts ./probe-artifacts -- npx playwright test

scripts\e2e\probe-e2e.ps1 -App 'C:\Probe\probe.exe' -Shard shard-1 -Assertions asserts.json -- npx playwright test

They:

refuse to start while a Probe process is running or ports 9099 / 9098 are busy (they never stop a process they did not start), then start the app with PROBE_DATA_DIR=<temp dir>, PROBE_NO_SYSTEM_PROXY=1 and a temporary home (HOME / CFFIXED_USER_HOME on macOS, USERPROFILE / LOCALAPPDATA / APPDATA on Windows), so the data folder, the certificate folder and the system proxy are not the installed Probe's;
wait for api-handshake.json and /api/v1/health, download the CA and begin a run (scoped to --shard when given);
run your command with HTTP_PROXY / HTTPS_PROXY / ALL_PROXY, NODE_EXTRA_CA_CERTS, SSL_CERT_FILE, CURL_CA_BUNDLE, PROBE_E2E_RUN_ID, PROBE_E2E_SHARD, PROBE_E2E_REST and PROBE_E2E_TOKEN set (so the test can add test rules itself);
end the run, post asserts.json ({"assertions":[...],"filter":{...}}; test_run_id is added for you) and export probe-run.har, run.json and assertions.json into the artifacts folder;
stop only the app process they started.

Meant for CI machines. On a developer machine the app's preferences (NSUserDefaults com.vinhtnk.mitmproxy on macOS, HKCU\Software\Guide\MITMProxy on Windows) are still shared with an installed Probe; for full isolation use a separate user account, or a copy of the app with its own bundle ID.

Exit code: your command's code if it failed (passed through as is), otherwise 1 when an assertion failed, 2 when Probe could not be started or driven (or rejected the assertions file), else 0. The scripts need curl and either jq or python3 on macOS, and only PowerShell on Windows. The scripts take the proxy port from the app's startup line and fail instead of guessing; on Windows, where that line may not be printed, pass -ProxyPort. They have not been run in CI here; the repository's own smoke test (scripts/smoke-test.sh) runs the same loop against the real app.