Migrate from Buf

This guide moves what buf lint, buf breaking and buf format do for a repository of Protobuf files to apigenie lint, apigenie diff and apigenie format: the rules by name, the buf.yaml settings, the CI step, and what stays with Buf, since buf generate and the Buf Schema Registry have no counterpart. It starts from Buf 1.75 with a v2 buf.yaml and the files under proto/.

You need the CLI installed; the Quickstart shows how. Keep Buf installed until the last step, so you can compare the two reports on the same files, and keep it after that if your code is generated with buf generate.

What changes and what stays

Buf API Genie
Lint buf lint apigenie lint proto/**/*.proto
Breaking changes buf breaking --against '.git#branch=main' apigenie diff main.proto shop.proto, file by file
Format buf format -w, buf format --exit-code apigenie format proto/**/*.proto, apigenie format --check
Configuration buf.yaml apigenie.yaml, sections commands.lint.protobuf, commands.diff.protobuf and commands.format.protobuf (Configuration)
Lint rules 45 in the MINIMAL, BASIC, STANDARD, COMMENTS and UNARY_RPC categories 39 in the recommended, strict and all rulesets (Protobuf rules)
Severities every violation fails error fails, warning does not; set per rule
Ignoring one place // buf:lint:ignore RULE, ignore and ignore_only by path // apigenie-lint-disable rule on the declaration
Imports the module’s directories, deps from the registry --proto-path, import_paths in the configuration, the directories above the file; the well-known types built in
Report formats --error-format text, json, msvs, junit, github-actions, gitlab-code-quality --formatter stylish, json, visualstudio, junit, checkstyle, sarif, …
Files proto2, proto3, editions proto3
Code generation buf generate, buf.gen.yaml stays with Buf; apigenie-generator writes Elixir gRPC clients and servers only

Run both on the same files

buf lint
apigenie lint proto/**/*.proto
  12:1   warning  Service name 'Shop' should end with 'Service'                 service-suffix
  31:3   warning  Enum zero value 'STATUS_NONE' should end with '_UNSPECIFIED'  enum-zero-value-suffix

  2 issues (0 errors, 2 warnings)
  0 errors, 0 warnings are potentially auto-fixable

apigenie lint takes files, so the shell’s ** glob, or a list, stands in for Buf’s module directory. Without configuration it applies the recommended ruleset: the structural rules at error, the naming and style rules at warning, which the command reports without failing. Buf’s STANDARD category fails on each of them. To fail the same way, extend strict or raise the rules you care about to error (below).

Imports are looked up on the paths given with --proto-path, then in import_paths of the configuration, then in the directory of the importing file and the directories above it, and google/protobuf/*.proto are built in. A Buf module whose files import each other by their path from the module root needs that root on the path:

apigenie lint --proto-path proto proto/**/*.proto

Replace buf.yaml

version: v2
lint:
  use:
    - STANDARD
    - COMMENTS
  except:
    - PACKAGE_VERSION_SUFFIX
  service_suffix: API
  ignore:
    - proto/vendor
breaking:
  use:
    - FILE

becomes

version: '1.0'
commands:
  lint:
    protobuf:
      extends: [strict]
      import_paths: [proto]
      rules:
        package-version-suffix: off
        service-suffix: [warning, API]
        field-comment-defined: error
        oneof-comment-defined: error
        enum-value-comment-defined: error
  diff:
    protobuf:
      extends: [recommended]
  • use names categories; extends names a ruleset: recommended (29 rules), strict (34: it adds the comments of messages, enums, services and RPCs, sorted imports, the version suffix and spelling) or all (39: every comment, and no streaming). The table below maps each rule.
  • except is off on the rule; a rule’s options follow its severity in a list.
  • ignore by path has no counterpart: leave the directory out of the files you pass, or disable the rules in the files with // apigenie-lint-disable.
  • breaking.use has no equivalent list: the recommended rules of the differ rate every change (below).

The lint rules by name

Buf API Genie
DIRECTORY_SAME_PACKAGE, PACKAGE_DEFINED, PACKAGE_DIRECTORY_MATCH, PACKAGE_NO_IMPORT_CYCLE, PACKAGE_SAME_DIRECTORY directory-same-package, package-defined, package-directory-match, package-no-import-cycle, package-same-directory
ENUM_PASCAL_CASE, ENUM_VALUE_UPPER_SNAKE_CASE, FIELD_LOWER_SNAKE_CASE, MESSAGE_PASCAL_CASE, ONEOF_LOWER_SNAKE_CASE, PACKAGE_LOWER_SNAKE_CASE, RPC_PASCAL_CASE, SERVICE_PASCAL_CASE, FILE_LOWER_SNAKE_CASE enum-naming, enum-value-naming, field-naming, message-naming, oneof-naming, package-naming, rpc-naming, service-naming, file-naming
ENUM_NO_ALLOW_ALIAS enum-no-allow-alias
ENUM_VALUE_PREFIX, ENUM_ZERO_VALUE_SUFFIX enum-value-prefix, enum-zero-value-suffix (the suffix is an option)
IMPORT_NO_PUBLIC, IMPORT_USED import-no-public, import-used; also import-no-weak, import-resolved and, in strict, imports-sorted
PACKAGE_SAME_CSHARP_NAMESPACE, PACKAGE_SAME_GO_PACKAGE, PACKAGE_SAME_JAVA_MULTIPLE_FILES, PACKAGE_SAME_JAVA_PACKAGE, PACKAGE_SAME_PHP_NAMESPACE, PACKAGE_SAME_RUBY_PACKAGE, PACKAGE_SAME_SWIFT_PREFIX package-options-consistent, one rule for the language options
PACKAGE_VERSION_SUFFIX package-version-suffix, in strict
RPC_REQUEST_RESPONSE_UNIQUE, RPC_REQUEST_STANDARD_NAME, RPC_RESPONSE_STANDARD_NAME, SERVICE_SUFFIX rpc-request-response-unique, rpc-request-standard-name, rpc-response-standard-name, service-suffix (the suffix is an option)
COMMENT_ENUM, COMMENT_MESSAGE, COMMENT_RPC, COMMENT_SERVICE enum-comment-defined, message-comment-defined, rpc-comment-defined, service-comment-defined, in strict
COMMENT_ENUM_VALUE, COMMENT_FIELD, COMMENT_ONEOF enum-value-comment-defined, field-comment-defined, oneof-comment-defined, in all
RPC_NO_CLIENT_STREAMING, RPC_NO_SERVER_STREAMING rpc-no-client-streaming, rpc-no-server-streaming, in all
ENUM_FIRST_VALUE_ZERO, FIELD_NOT_REQUIRED, SYNTAX_SPECIFIED not checked: the linter reads proto3 files, where protoc enforces the three
PROTOVALIDATE, STABLE_PACKAGE_NO_IMPORT_UNSTABLE not checked

Buf has nothing for spell-check, which checks the comments with a dictionary (in strict, at warning).

Breaking changes

buf breaking compares a module with another input, a Git ref included. apigenie diff compares two files, so CI fetches the old version first:

git show main:proto/shop/v1/shop.proto > main.proto
apigenie diff main.proto proto/shop/v1/shop.proto
shop.proto
  19  1      breaking  Modified message 'Customer'
  19  1  non-breaking    Removed field 'fax'
  25  3      breaking    Modified field 'full_name'
  25  3      breaking      Modified field name

The command exits with 1 on a breaking change, as buf breaking does. Fields and enum values are matched by number, so a renamed field is a modified one, and removing a field whose number the new file reserves is non-breaking, the way Protobuf retires fields. The files are compared on their own: to compare the declarations of imported files too, bundle both versions first.

Buf’s categories pick what counts: FILE and PACKAGE protect the generated code, WIRE and WIRE_JSON the encoded messages. The recommended rules of the differ rate as breaking what breaks either: a removed or renamed declaration, a changed type, label or oneof, the types or the streaming of a method, the package, an option that names the generated code such as go_package. For a WIRE policy, where names and compatible types may change, the differ’s rules say so, with a wire_compatible condition:

version: '1.0'
commands:
  diff:
    protobuf:
      extends: [recommended]
      rules:
        field:
          name: non_breaking
          type:
            modify:
              if: wire_compatible
              then: non_breaking
              else: breaking

diff.protobuf lists the properties of every node type, the conditions, and overrides for a part of the file, which stand in for Buf’s ignore and ignore_only.

Format

buf format -w is apigenie format on the files, and buf format --exit-code is apigenie format --check, which writes nothing and exits with 1 when a file is not formatted. The formatter writes one statement per line, normalizes the spacing and keeps every comment. With the recommended settings it also sorts imports and option statements; extends: [original] under commands.format keeps every order, and the Protobuf settings sort declarations, fields or methods when you want that. proto2 and editions files are reported and left untouched.

The report in CI

--error-format junit is --formatter junit for apigenie lint; msvs is visualstudio; json keeps its name; github-actions and gitlab-code-quality have no counterpart, and apigenie diff writes json, markdown, html, stylish, unix and visualstudio. On GitHub, the GitHub Action runs the three commands with their results as annotations on the lines of the pull request, and diff compares each file with its version at the base of the pull request on its own:

on: pull_request
jobs:
  proto:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: mykulyak/api-genie-action@v0
        with:
          command: lint
          files: proto/**/*.proto
      - uses: mykulyak/api-genie-action@v0
        with:
          command: format
          files: proto/**/*.proto
      - uses: mykulyak/api-genie-action@v0
        with:
          command: diff
          files: proto/**/*.proto

What stays with Buf

  • Code generation. buf generate with buf.gen.yaml and its plugins has no counterpart. apigenie-generator reads .proto files for an Elixir gRPC client and an Elixir gRPC server only.
  • The Buf Schema Registry. deps and --against-registry have no counterpart; the imported files have to be in the repository, on an import path.
  • proto2 and editions. The linter, the differ and the formatter read proto3 files.

What Buf lacks: a gRPC mock of the .proto file, served with stubs, reflection and recording, and apigenie test for GRPC requests in .http files, checked against the file.

Remove Buf from the checks

When the two reports agree on the files you have: replace buf lint, buf breaking and buf format in the scripts and the pipeline, keep buf generate and its buf.gen.yaml, and keep buf.yaml for it, without the lint and breaking sections.

Was this helpful?