API Genie

With --proxy-to, an OpenAPI document is not mocked: every request goes to the real server, and the mock checks the request and the server’s response against the document. (A GraphQL schema and an AsyncAPI document’s WebSocket, MQTT and HTTP channels work the same way, with checks of their own: see GraphQL contract testing, WebSocket contract testing, MQTT contract testing and HTTP channels.) Point your tests, your frontend or a load test at the mock instead of the server, and the traffic becomes a contract test. To send the requests yourself, from .http files or generated from the document, with the same checks and reports, use apigenie test. For the checks and what report and enforce mean, explained with a server to try them on, see What the validating proxy checks.

apigenie-server openapi.yaml --proxy-to http://localhost:8080 --report contract.json --fail-on errors

The upstream gets the request as it came, without the mock’s __ query parameters and X-Apigenie-Mock-* headers, with Via, X-Forwarded-For, X-Forwarded-Host and X-Forwarded-Proto, and Accept-Encoding: identity so that bodies can be checked. A path in the URL (http://localhost:8080/api) is put before the request’s path. The admin API, CORS preflights and answers of scripts’ onRequest stay local; the global network simulation, a request’s Prefer: delay= or drop=, and scripts’ onResponse apply to the upstream’s responses. Scenario rules and examples don’t: nothing is mocked, unless the server fails in fallback mode (below).

What is checked. Each problem is a violation with a check, a severity and a location:

Check	Severity	When
`request.path-or-method.unknown`	warn	The document has no operation for the method and path.
`request.operation.deprecated`, `request.parameter.deprecated`	info	A deprecated operation or parameter is used (the response gets `Deprecation: true`).
`request.authorization.missing`, `request.authorization.rejected`	error	No credentials for any security requirement; credentials other than the configured ones (`security.mode: enforce`).
`request.authorization.oauth2-scheme`, `request.authorization.oauth2-scope`	warn	`apigenie test` only: a request with an OAuth 2 token for an operation that has no OAuth 2 or OpenID Connect scheme; a token requested with scopes the operation doesn’t declare, or without those it requires.
`request.path.incompatible`, `request.query.incompatible`, `request.header.incompatible`, `request.cookie.incompatible`	error	A parameter is missing or doesn’t match its schema (every style).
`request.query.unknown`	warn	An undeclared query parameter (`validation.unknown_query_parameters: error`).
`request.content-type.incompatible`, `request.body.malformed`, `request.body.incompatible`	error	The body’s media type isn’t declared, the body can’t be parsed (JSON, XML, multipart), or doesn’t match its schema (for XML, see XML bodies).
`response.status.unknown`	error for 2xx, else warn	No response covers the status (exact code, then `2XX`, then `default`).
`response.status.default`	info	Only `default` covers the status.
`response.status.server-error`	info	The server answered with 5xx.
`response.content-type.incompatible`	error	The response’s media type isn’t one of the declared ones.
`response.body.incompatible`, `response.body.malformed`	error	The body is missing, doesn’t match its schema (`writeOnly` properties aren’t required), or isn’t JSON or XML where the media type says so.
`response.body.unexpected`	warn	A body where the response declares no content.
`response.body.unvalidatable`	info	A body that can’t be checked: binary, compressed.
`response.header.incompatible`	error	A declared header is missing when required, or doesn’t match its schema.
`response.header.undefined`	off	A header the response doesn’t declare (common ones such as `Date` or `ETag` aside).

contract.severity in the configuration changes a check’s severity (error, warn, info) or turns it off (ignore).

Report or enforce. By default a proxy reports only (validation.request and validation.response are log unless the configuration says otherwise), since test suites send invalid requests on purpose to see the server refuse them. Every response then carries the violations of its exchange:

X-Apigenie-Mock-Source: proxy
X-Apigenie-Mock-Operation: getPet
X-Apigenie-Mock-Violations: 1
X-Apigenie-Mock-Violations-Detail: [{"location":["response","body","/id"],"check":"response.body.incompatible","code":"type","severity":"error","message":"expected type integer, found string"}]

The detail is cut at 8 KB, with a last item counting what was left out. The log has the same violations, upstreamStatus for each request, and fallback: true for the mock’s own answers in fallback mode.

--strict (or enforce in the configuration) turns errors into answers. A request with an error doesn’t reach the server: it gets 401 or 403 (security), 415 (media type), 400 (malformed body) or 422. A response with an error is replaced by a 502:

{
  "type": "https://apigenie.pl/mock/errors#invalid-upstream-response",
  "title": "The upstream response does not match the specification",
  "status": 502,
  "detail": "response.body/id: expected type integer, found string",
  "validation": [{"location": "response.body/id", "code": "type", "message": "expected type integer, found string",
                  "severity": "error", "check": "response.body.incompatible", "path": ["response", "body", "/id"]}],
  "upstream": {"status": 200, "contentType": "application/json"}
}

Warnings and notes never block anything. A server that can’t be reached is answered with 502 (upstream-unreachable, upstream-bad-response, upstream-response-too-large), one that doesn’t answer in time with 504 (upstream-timeout).

Fallback mode. With --proxy-mode fallback (or proxy.mode: fallback), every request still goes to the server first, but the mock answers itself, as it would without --proxy-to (examples, generated data, scenario rules, Prefer), when the server can’t be reached, doesn’t answer in time, or answers 404 or 501 for an operation of the document. A frontend can then work against a server that is down or only partly implemented. The answer says so:

X-Apigenie-Mock-Source: generated
X-Apigenie-Mock-Upstream-Status: 404

The log record has fallback: true and upstreamStatus (absent when the server didn’t answer). The server’s 404 or 501 isn’t checked against the document, so it isn’t a response.status.unknown violation. Requests for no operation of the document are not the mock’s to answer: the server’s 404 reaches the client. An answer that isn’t HTTP or is too large is still a 502: the server did answer.

Record and replay. --record-dir DIR writes each exchange the server answered as a stub file, getPet-1.yaml (GET_health-1.yaml for a route of no operation; numbers already taken are skipped), and --stubs DIR makes a mock without --proxy-to answer with them:

apigenie-server openapi.yaml --proxy-to http://localhost:8080 --record-dir recorded
apigenie-server openapi.yaml --stubs recorded

stubs:
  - name: getPet-1
    operationId: getPet
    request:
      method: GET
      path: /pets/1                # as the client sent it, without the proxy's prefix
      query: full=true             # without the mock's __ parameters
      headers: {Accept: application/json, Authorization: "[redacted]"}
    # match: {headers: [X-Tenant]} # these request headers must be equal too
    response:
      status: 200
      headers: {Content-Type: application/json, ETag: '"v1"'}
      body: {id: 1, name: Rex}     # JSON; text: for other text, base64: for bytes

A request matches a stub when its method, path, query (in any order) and body (JSON compared as JSON) are the same; headers count only when match.headers names them. The first stub that matches answers, files in name order; its source is stub:getPet-1. Requests still go through security and request validation; the answer isn’t checked again. Answers the mock gave or replaced (fallback, --strict) aren’t recorded. The values of Authorization, Cookie, Set-Cookie, Proxy-Authorization and X-Api-Key are written as [redacted] (proxy.redact sets the list), and what the connection or the proxy adds (Host, User-Agent, Via, X-Forwarded-*, Date, …) is left out. A recorded header that matters for matching has to be restored by hand.

HAR. --record-har session.har writes every exchange through the proxy as an HTTP Archive (HAR 1.2) when the server stops: the server’s answers and the mock’s own (fallback answers, rejected requests, replaced responses, the problems of a failed server). Each entry’s comment is proxy or mock, and _apigenie has the source, the operation, the server’s status, fallback and the violations. Headers of proxy.redact are written as [redacted]; binary bodies are in Base64, and bodies above 1 MiB are left out (their sizes stay). Browsers’ developer tools, Postman and HAR viewers read it.

Coverage. The mock counts what the traffic exercised: each operation, its request body (seen with a 2xx or 3xx answer) and each declared response (200, 4XX, default). An operation is covered when all of these were seen, partial, or not-tested; requests for no operation are listed as undocumented. Mocked traffic is counted too, without --proxy-to or in fallback mode; the report tells the two apart: summary.sources and each operation’s sources count the answers of the server (proxy) and of the mock (mock, also the problems of a failed server and rejected requests), mockedResponses the statuses the mock answered with, and, when the mock answered any, the summary printed at the end has a Sources: line.

Reports. GET /__apigenie/contract returns the report at any time, and DELETE starts it over (a reload does too). When the server stops (Ctrl+C or POST /__apigenie/shutdown) it prints a summary, writes --report (JSON) and --report-junit (one test case per operation, skipped when not tested, failed with error violations; one per undocumented route; one for the coverage threshold), and exits with 5 when --fail-on or --min-coverage isn’t met. In CI:

apigenie-server openapi.yaml --proxy-to http://localhost:8080 --port 4010 \
  --report contract.json --report-junit contract.xml --fail-on errors --min-coverage 80 &
API_URL=http://localhost:4010 npm test
curl -X POST http://localhost:4010/__apigenie/shutdown
wait $!   # 5 when the contract failed

The JSON report has summary (requests, their sources, violations by severity, coverage of branches and operations, the gate with its reasons), operations (the JSON pointer of the operation in the document, e.g. /paths/~1pets/get, requests, responses seen and missing, undeclared statuses, sources and the mock’s statuses, violations by check), undocumented and violations (each distinct one with its count and the first time it was seen). Its formatVersion is the version of the format, which changes only when a member is removed or changes meaning (see Compatibility); version, its former name, is deprecated and will be removed in the next release. The JSON Schema of the members the OpenAPI, AsyncAPI, GraphQL and gRPC reports share is apigenie-contract-report.json.

Contract testing (validating proxy)