API Genie

A run of the server, every exchange of every service and protocol, kept as files that a replay reads back as a mock, in a test suite too.

Session recording

--record DIR keeps a run whole: every exchange the server answered, of every service and every protocol, in the order it happened, each marked with the test that sent it, as files a replay can read and a reviewer can diff:

apigenie-server openapi.yaml --proxy-to http://localhost:8080 --record sessions/checkout
apigenie-server --config apigenie-server.yaml --record sessions/checkout   # several services
apigenie-server session export sessions/checkout --har checkout.har

sessions/checkout/
  session.yaml                     # the session and the exchanges in order
  pets/0001-getPet.yaml            # an HTTP exchange of the service pets (a stub)
  pets/0002-createPet.yaml
  events/0003-ws-in-sendMessage.yaml   # a WebSocket message from a client
  events/0004-ws-out-sendMessage.yaml  # a message to it
  chat/0005-Chat.Talk.yaml         # a gRPC call with its messages (a stub)

A server of one document writes the files in DIR itself (--stubs DIR and the GraphQL examples skip session.yaml). session.yaml names the session (the directory’s name unless record.name or /record/start says), the server, when it started and stopped, the services (name, document, specification, the SHA-256 of the document’s bytes, mount) and the exchanges:

name: checkout
server: apigenie-server 1.0.0
started: 2026-10-13T10:00:00.000Z
stopped: 2026-10-13T10:00:42.512Z
services:
  - {name: pets, spec: pets.yaml, specification: openapi, sha256: cfada1c0..., mount: /pets}
  - {name: events, spec: events.yaml, specification: asyncapi, sha256: 9b2f..., mount: /events}
exchanges:
  - {sequence: 1, id: 2, service: pets, test: checkout-happy, kind: http, time: 2026-10-13T10:00:01.555Z,
     durationMs: 3, source: proxy, summary: GET /pets/1 200, file: pets/0001-getPet.yaml}
  - {sequence: 3, id: 7, service: events, test: checkout-happy, kind: ws, time: ..., source: mock,
     summary: ws-in rooms/7 notice, file: events/0003-ws-in-sendMessage.yaml}

The exchange files. Each has a session note (the index’s line) next to the protocol’s own form, so what already reads that form reads the file:

http: {session, stubs: [stub]}, the stub form of --record-dir, so --stubs sessions/checkout/pets makes a mock answer with the service’s exchanges as before (the first matching stub answers, whatever the test). OpenAPI exchanges, AsyncAPI HTTP channels and GraphQL requests that aren’t one operation answered with JSON (batches, errors, GraphiQL). The answer is kept as the client got it, less the CORS headers the server adds to every answer. The request’s X-Apigenie-Test header is left out.
graphql: one operation and its answer, in the example form (name, operation, query, variables, response, status) the mock’s examples load; the note’s path is the endpoint.
grpc: {session, stubs: [stub]}, the stub form of a .proto file’s --record-dir, with the whole streams (at most 100 messages per side); the note’s order ([in, out, in, out]) says which side each message came from, code the status. The x-apigenie-test metadata is left out.

ws and mqtt: one file per message of either side:

session: {sequence: 3, id: 7, service: events, test: checkout-happy, kind: ws, time: ..., source: mock}
message:
  direction: in                  # from the client; out: to it
  channel: rooms/7               # the channel's address, and the route it matched
  route: rooms/{roomId}
  operation: sendMessage
  name: notice
  contentType: application/json
  connection: 4
  json: {text: hi}               # text: for other text, base64: for bytes

Tests. An exchange’s test is the X-Apigenie-Test header of the request or of the WebSocket handshake, or the x-apigenie-test metadata of a gRPC call; MQTT messages have none of their own. Without one, it is the current mark: POST /__apigenie/record/mark {"test": "checkout-happy"} names the test of the exchanges to come ({} clears it), for clients that can’t send the header. A Playwright suite sends the header with page.setExtraHTTPHeaders and marks the test in beforeEach.

Order. sequence numbers the exchanges in the order they completed, which is the log’s order; a request still in flight when a short one is answered comes after it. Each exchange keeps its time and durationMs (a gRPC call its started), so a tool can sort by arrival. What isn’t an exchange isn’t recorded: requests the client abandoned or the simulation dropped, CORS preflights, the admin API’s own traffic, the dashboard.

Redaction. The values of Authorization, Cookie, Set-Cookie, Proxy-Authorization and X-Api-Key are written as [redacted], as in --record-dir. --record-redact (or record.redact) adds header and metadata names, and JSON pointers, which start with /, to values of JSON bodies, message payloads, gRPC messages and GraphQL variables and answers: --record-redact X-Session --record-redact /owner/password. A pointer is matched exactly (/items/0/key is the first item’s key). A redacted request body no longer matches on replay.

Start and stop. --record records from the start and writes session.yaml when the server stops; the startup line says Recording the session to DIR (record in JSON). The admin API records a part of a run: POST /__apigenie/record/start {"dir": "sessions/run-7", "name": "run 7"} (dir defaults to record.dir; 409 while a session is on, 400 for a directory that isn’t empty), POST /__apigenie/record/stop (the session’s dir, name, exchanges, file; 409 without one), GET /__apigenie/record (recording, and the same while one is on, test for the mark). The dashboard’s Traffic screen has Record session and Save session. A session has at most 10 000 exchanges: past that it counts on but writes no more files, with one warning in the log and truncated: true in the index. When the server dies before writing session.yaml, the exchange files tell: session export rebuilds the index from their notes.

Export. apigenie-server session export DIR --har FILE writes the HTTP exchanges, GraphQL operations (as POST requests to the endpoint) and gRPC calls (as POST requests with application/grpc+json bodies, the messages as JSON arrays, and a grpc-status header) as a HAR 1.2 file, in the session’s order, each entry with _apigenie.service, test, sequence and kind; WebSocket and MQTT messages have no form in HAR and are left out (the command says how many). URLs keep the services’ mount prefixes.

The design is ADR-168.

Session replay

--replay DIR serves the services of a recorded session the way they answered when it was recorded, without the real servers: each request gets the next unused recorded exchange of its test that matches it, so a test suite replays the same run every time, and fails when the application asks for something the recording doesn’t have.

apigenie-server openapi.yaml --record sessions/checkout --proxy-to http://localhost:8080   # once
apigenie-server openapi.yaml --replay sessions/checkout                                    # every run
apigenie-server --config apigenie-server.yaml   # replay: {dir: sessions/checkout}, several services

Tests and order. A request’s test is its X-Apigenie-Test header (of a WebSocket handshake too) or a gRPC call’s x-apigenie-test metadata. It draws from the exchanges recorded under that test, in their order: the first GET /pets/1 of test checkout gets the first one recorded for it, the second the second, and a third matches none. Tests replay side by side, each with its own counters, so the parallel workers of a suite don’t take each other’s exchanges. Requests without a tag share one test that draws from the whole session in order, for suites that don’t tag their requests (and MQTT, whose clients send none). POST /__apigenie/replay/reset {"test": "checkout"} (or with the header) restarts a test’s counters, at the start of each test; without a test it restarts every test.

What is compared. The method and path; the query, in any order; the headers the recording kept (not Host, User-Agent, Accept-Encoding and the other headers a client changes from run to run: Date, Origin, Referer, Accept-Language, Cache-Control, the Sec-* headers, traceparent, X-Request-Id, …), a recorded [redacted] value standing for any; and the body, JSON regardless of key order and number formatting. Left out:

strings that are both RFC 3339 dates or date-times (2026-10-13, 2026-10-13T10:00:00Z), in bodies and query parameters;
the fields of an OpenAPI request body that its schema marks readOnly, which a client sends back as it got them (id, createdAt);
what --replay-ignore (or replay.ignore) names: headers, and JSON pointers into bodies, messages, gRPC messages and GraphQL variables (--replay-ignore /cartId --replay-ignore /items/*/addedAt; * stands for any index or key).

When nothing matches. --replay-missing fail, the default, answers with a 502 application/problem+json naming the closest recording (of the same test and kind; the same operation first, then the fewest differences) and how the request differs from it:

{
  "type": "https://apigenie.pl/mock/errors#replay-missing",
  "title": "No recorded exchange matches",
  "status": 502,
  "detail": "no recorded exchange of test 'checkout' matches POST /carts; the closest, pets/0004-createCart.yaml, differs at /items/0/quantity",
  "test": "checkout",
  "closest": {"file": "pets/0004-createCart.yaml", "sequence": 4, "summary": "POST /carts 201", "used": false},
  "differences": [{"where": "/items/0/quantity", "recorded": 1, "received": 2}]
}

used: true says the matching exchange was replayed already in that test (a request sent once more than when it was recorded). generate lets the mock answer (stubs, the document store, examples, generated values); passthrough forwards the request to --proxy-to, and with --replay-update records the run (replayed and forwarded exchanges alike) into DIR.update and puts it in place of DIR when the server stops: the way to refresh a recording after the application changed on purpose. A request that matched none is reported either way.

Every protocol.

HTTP (OpenAPI, AsyncAPI HTTP channels, GraphQL batches): the recorded status, headers and body, with X-Apigenie-Mock-Source: replay:0002-getPet.yaml.
GraphQL queries and mutations: by operation name, query (formatting aside) and variables; the recorded answer. One none matches gets a REPLAY_MISSING error (status 502) with the same members in extensions.replay.
GraphQL subscriptions (graphql-transport-ws, graphql-ws, SSE) and AsyncAPI WebSocket channels: a connection replays a recorded connection of its route and test as the server’s side: what the server sent first (connection_ack, the first scheduled messages) at once, then, after each message the client sends that is the recorded one, the messages that followed it. The subscriptions’ ids are the client’s own. A message that isn’t the recorded one gets the problem as a message (GraphQL: an error message). The scheduled messages and generated replies are off.
MQTT: a subscriber of a topic gets the messages a subscriber got before any client published; a published message that is the next recorded one gets the messages that followed it.
gRPC: unary and server-streaming calls by their metadata and first request, client streams by all their requests, bidirectional streams in their recorded interleaving (the session’s order): a message the client sends that isn’t the recorded one fails the call. A call none matches fails with FAILED_PRECONDITION, the problem in the apigenie-replay-problem trailer.

WebSocket and MQTT messages can’t be forwarded one at a time: under passthrough they are answered as under fail.

The report. When the server stops it prints how many recorded exchanges were replayed, the ones never replayed and the requests that matched none, with their tests (with --log-format json, one {"event": "replay", ...} line), and exits with status 5 if there are any; --replay-update prints the report but doesn’t fail. GET /__apigenie/replay gives the same report at any time, with passed and the exchanges each test used since its reset. The startup line says Replaying the session in DIR (N exchanges, missing: fail) (replay in JSON). A document that changed since the session was recorded (another SHA-256) and a service the session lacks are warned about at startup. --stubs, the GraphQL examples and the gRPC stubs stay: the per-exchange forms, where the first match answers whatever the test.

The design is ADR-169.

Replay in a test suite

A Playwright suite recorded against the real backend once replays it in every run, each test with its own exchanges. The server starts with the suite and every request of a test carries its name:

// playwright.config.ts
import { defineConfig } from "@playwright/test";

export default defineConfig({
  webServer: {
    command: "apigenie-server openapi.yaml --port 4010 --replay recordings/checkout",
    url: "http://127.0.0.1:4010/__apigenie",
    reuseExistingServer: false,
  },
  globalTeardown: "./replay-teardown.ts",
});

// checkout.spec.ts
import { expect, test } from "@playwright/test";

const admin = "http://127.0.0.1:4010/__apigenie";

test.beforeEach(async ({ page }, testInfo) => {
  const name = testInfo.titlePath.join(" > ");
  // every request of the page names the test, the API's included
  await page.setExtraHTTPHeaders({ "X-Apigenie-Test": name });
  // the test starts at its first recorded exchange, also when it is retried
  await fetch(`${admin}/replay/reset`, { method: "POST", body: JSON.stringify({ test: name }) });
});

test("checks out a cart", async ({ page }) => {
  await page.goto("/cart");
  await page.getByRole("button", { name: "Check out" }).click();
  await expect(page.getByText("Thank you")).toBeVisible();
});

// replay-teardown.ts: the run fails when the application drifted from the recording
export default async function teardown() {
  const report = await (await fetch("http://127.0.0.1:4010/__apigenie/replay")).json();
  if (!report.passed) {
    throw new Error(
      `the replay failed: ${report.neverReplayedCount} exchanges never replayed, ` +
        `${report.unmatchedCount} requests matched none: ${JSON.stringify(report.unmatched, null, 2)}`,
    );
  }
}

The server answers the CORS preflight of the header for a page served from another origin. To record the suite, run it once with --record recordings/checkout --proxy-to https://staging.example.com in place of --replay: the same header tags the recorded exchanges with the tests. Playwright stops its web servers without letting them finish, so such a recording has no session.yaml; the replay reads the index from the exchange files, as session export does. On CI, a request the recording lacks gets a 502, so the test that sent it fails at that step, with the closest recording and the differences in the response; the teardown fails the run when a test no longer sends what was recorded (exchanges never replayed). A job that runs the server itself sees the same as its exit status:

apigenie-server openapi.yaml --port 4010 --replay recordings/checkout --log-format json > replay.log &
server=$!
npx playwright test
curl -s -X POST http://127.0.0.1:4010/__apigenie/shutdown
wait $server   # 5 when the run drifted from the recording; replay.log has the report

Session recording and replay

Session recording

Session replay

Replay in a test suite