# Validation

Source: https://apigenie.pl/docs/reference/cli/test/validation
Part of the reference of [test](https://apigenie.pl/docs/reference/cli/test.md).
Its other pages: [Request files](https://apigenie.pl/docs/reference/cli/test/requests.md), [Variables and OAuth 2](https://apigenie.pl/docs/reference/cli/test/variables.md), [Assertions and scripts](https://apigenie.pl/docs/reference/cli/test/assertions.md), [Generated requests](https://apigenie.pl/docs/reference/cli/test/generate.md), [GraphQL requests](https://apigenie.pl/docs/reference/cli/test/graphql.md), [WebSocket and MQTT requests](https://apigenie.pl/docs/reference/cli/test/websocket-mqtt.md), [Kafka requests](https://apigenie.pl/docs/reference/cli/test/kafka.md), [AMQP requests](https://apigenie.pl/docs/reference/cli/test/amqp.md), [gRPC requests](https://apigenie.pl/docs/reference/cli/test/grpc.md), [Importing collections](https://apigenie.pl/docs/reference/cli/test/import.md), [httpYac compatibility](https://apigenie.pl/docs/reference/cli/test/httpyac.md).

With `--spec`, every request, response and message is checked against the document with the
checks of the [validating proxy](/docs/reference/server): an OpenAPI or Swagger 2.0 document for
HTTP requests, a GraphQL schema for GraphQL requests, a `.proto` file for `GRPC` requests, and an
AsyncAPI document for WebSocket, MQTT, Kafka and AMQP requests and the requests of its HTTP
channels.

With `--spec`, each request is bound to an operation by its method and path, as the validating
proxy does, then checked: parameters, body, security (credentials must be present), and the
response's status, media type, headers and body. The checks, their severities and `--severity` are
those of [contract testing](/docs/reference/server/contract-testing).
XML request and response bodies are mapped by the schema's `xml` objects and checked like JSON ones
(see [XML bodies](/docs/reference/server/validation#xml-bodies)).
The coverage counts each operation, its request body and each declared response.

A request sent with an [OAuth 2](/docs/reference/cli/test/variables#oauth-2) token is also checked against the operation's security
requirements (or the document's), as warnings by default:

| Check | Default | Meaning |
| --- | --- | --- |
| `request.authorization.oauth2-scheme` | `warn` | The operation has no `oauth2` or `openIdConnect` scheme (or no security at all) |
| `request.authorization.oauth2-scope` | `warn` | A scope of `oauth2_scope` isn't declared for the operation, or no requirement has all its scopes in `oauth2_scope` |

When several OAuth 2 schemes apply, the one whose flow's `tokenUrl` is the token endpoint is
used. Without `oauth2_scope` the scopes aren't checked.

A Swagger 2.0 specification is converted to OpenAPI 3.2 in memory, as `apigenie-server` does (see
[Swagger 2.0 documents](/docs/reference/server#swagger-20-documents)); no file is
written. What the conversion can only approximate is told once before the requests are sent, as
`SPEC:LINE: info: POINTER: MESSAGE` lines on the standard error (`{"event":"info",...}` with
`--formatter json`), and the report's operation pointers point into the Swagger file.

With a GraphQL schema as `--spec` (`.graphql`, `.graphqls` or `.gql`), the GraphQL requests are
checked by the [GraphQL validating proxy](/docs/reference/server/graphql#contract-testing-of-a-graphql-api)'s checks. The
operation is validated against the schema before it is sent: an invalid one (a syntax error, an
unknown field or argument, an unknown `operationName`, variables of the wrong type) fails the
request, which isn't sent, with the errors the server would give (`invalid request: Cannot query
field "nope" on type "Book".`), and the run goes on. The response is checked against the request's
selection set: the shape of `data` and `errors`, the type of each value, `null` in non-null fields
that no error's `path` explains (`null-in-non-null`), `__typename` of abstract types, and errors
of operations the schema allows (`response.schema-drift`). Each event of a subscription is checked
the same way, at `next[0]`, `next[1]`, ... (`next[1]/data/storyAdded/title: Story.title is an
Int`). Deprecated fields selected are notes (`request.field.deprecated`). Requests that aren't
GraphQL requests aren't checked.

An answer with `errors` next to `data` is valid GraphQL: the operation ran, but fields failed. Each
of those errors is a warning (`response.field-error`), so a server that answers every field with
an error doesn't pass unnoticed. `--fail-on warnings` fails such requests, and
`--severity response.field-error=error` makes them errors. The coverage counts the fields of the
schema's types that requests selected, whether or not the answers had them, and each root field;
`--min-coverage` is the share of fields.

```text
PASS GRAPHQL http://localhost:4000/graphql -> 200 (7 ms) GetBook
FAIL GRAPHQL http://localhost:4000/graphql Broken
    invalid request: Cannot query field "nope" on type "Book".
Requests: 1 passed, 1 failed, 0 skipped
Contract: 2 requests (0 to other paths), 1 errors, 0 warnings
Coverage: 40.0% of 15 fields, 1 of 5 root fields
  not tested: Query.books, Query.node, Mutation.addBook, Mutation.deleteBook
```

With an AsyncAPI 3.x `--spec`, WS and MQTT requests are checked against the document's channels.
The channel of a `WS` request is the one served at its URL's path; that of an `MQTT` request is the
one of the topic it publishes to (or of its first subscription without wildcards), and each message
received is matched by its own topic. `# @channel` names the channel when its address can't be
matched (`address: null`, wildcard filters), `# @operation` checks the messages sent as that
operation, and `# @operation off` checks nothing. A path or topic of no channel is a note, and its
messages aren't checked.

The messages sent are the client's: they are checked against the channel's `receive` operations.
The messages received are the application's: they are checked against its `send` operations or,
when they answer a request, against the reply's messages. When a message sent matched an operation
with a `reply`, the first later message on the reply's channel that is one of the reply's messages
is its reply:

| Check | Default | Meaning |
| --- | --- | --- |
| `message.reply.missing` | `warn` | No reply came while the request waited |
| `message.reply.uncorrelated` | `error` | The reply's `correlationId` (in the payload, or in a header over MQTT 5) isn't the request's, or is missing; over MQTT 5, the reply doesn't repeat the request's Correlation Data |
| `message.properties.invalid` | `error` | MQTT 5: a message's properties don't match its MQTT bindings (see [MQTT 5 properties](/docs/reference/cli/test/websocket-mqtt#mqtt-5-properties)) |

Violations are located in the message: `sent[0]/message/carrier`, `received[1]/message/requestId`,
`sent[0]/message/property.contentType`. Over MQTT 5 a message's user properties are its headers:
a `correlationId` or a reply address in `$message.header#/...` is read from them, and they are
checked against the message's `headers` (`message.headers.invalid`). MQTT 3.1.1 has no message
headers: such a `correlationId` or reply address can't be checked, and the request gets a note
saying so. WS and MQTT
requests aren't checked against an OpenAPI document. The coverage counts each message of each
operation, those of HTTP channels included, as the
[AsyncAPI validating proxy](/docs/reference/server/asyncapi#contract-testing-of-a-websocket-api) does.

## Requests of HTTP channels

HTTP requests are checked against the AsyncAPI document's HTTP channels (the channels of `http` and
`https` servers) with the checks of the
[validating proxy](/docs/reference/server/asyncapi#contract-testing-of-http-channels), so both
report the same violations:

- The channel and operation are found by the request's path (under a server's `pathname` or not)
  and method: a `receive` operation's method is its HTTP binding's (`POST` by default); a `send`
  operation is asked for with `GET`, as the mock serves it. `# @operation` names the operation
  instead, whatever the method; a path that isn't its channel's is an error.
- A `receive` operation's request: the body against the operation's messages (`body/...`), the
  query against the HTTP binding's `query` schema, and the header fields against the message's
  `headers` and its HTTP binding's `headers` (its traits' too). Its 2XX response: the body and
  header fields against the reply's messages (`response/...`).
- A `send` operation's 2XX response with a body: against the operation's messages. Its header
  fields aren't checked: they are those of the webhook the application sends, which doesn't pass
  `apigenie test`.
- Responses of other statuses, and the reply's `statusCode` binding, aren't checked.

| Check | Default | Meaning |
| --- | --- | --- |
| `operation.query.invalid` | `error` | The query doesn't match the HTTP binding's `query` (`query.dryRun`) |
| `message.headers.invalid` | `error` | The header fields the message's header schemas name don't match them (`header.X-Batch-Size`) |

Query parameters and header fields are read as the type their schema says (`integer`, `number`,
`boolean`, an `array` from repeated parameters); header fields the schemas don't name are left
out. A path of no HTTP channel, or a method its channel doesn't take, is a note, and the request
isn't checked.

```http
### A carrier's status batch
POST {{http}}/http/carriers/dhl/events?dryRun=true
Content-Type: application/json

{"events":[{"trackingNumber":"CE1234567890","statusCode":"DLV","occurredAt":"2026-10-08T10:00:00Z"}]}

### The webhooks, asked for as the mock serves send operations
GET {{http}}/http/
```