Migrate from Buf
This guide moves what buf lint, buf breaking and buf format do for a repository of Protobuf
files to apigenie lint, apigenie diff and apigenie format: the rules by name, the
buf.yaml settings, the CI step, and what stays with Buf, since buf generate and the Buf
Schema Registry have no counterpart. It starts from Buf 1.75 with a v2 buf.yaml and the files
under proto/.
You need the CLI installed; the Quickstart shows how. Keep Buf installed until
the last step, so you can compare the two reports on the same files, and keep it after that if
your code is generated with buf generate.
What changes and what stays
| Buf | API Genie | |
|---|---|---|
| Lint | buf lint |
apigenie lint proto/**/*.proto |
| Breaking changes | buf breaking --against '.git#branch=main' |
apigenie diff main.proto shop.proto, file by file |
| Format | buf format -w, buf format --exit-code |
apigenie format proto/**/*.proto, apigenie format --check |
| Configuration | buf.yaml |
apigenie.yaml, sections commands.lint.protobuf, commands.diff.protobuf and commands.format.protobuf (Configuration) |
| Lint rules | 45 in the MINIMAL, BASIC, STANDARD, COMMENTS and UNARY_RPC categories |
39 in the recommended, strict and all rulesets (Protobuf rules) |
| Severities | every violation fails | error fails, warning does not; set per rule |
| Ignoring one place | // buf:lint:ignore RULE, ignore and ignore_only by path |
// apigenie-lint-disable rule on the declaration |
| Imports | the module’s directories, deps from the registry |
--proto-path, import_paths in the configuration, the directories above the file; the well-known types built in |
| Report formats | --error-format text, json, msvs, junit, github-actions, gitlab-code-quality |
--formatter stylish, json, visualstudio, junit, checkstyle, sarif, … |
| Files | proto2, proto3, editions | proto3 |
| Code generation | buf generate, buf.gen.yaml |
stays with Buf; apigenie-generator writes Elixir gRPC clients and servers only |
Run both on the same files
buf lint
apigenie lint proto/**/*.proto
12:1 warning Service name 'Shop' should end with 'Service' service-suffix
31:3 warning Enum zero value 'STATUS_NONE' should end with '_UNSPECIFIED' enum-zero-value-suffix
2 issues (0 errors, 2 warnings)
0 errors, 0 warnings are potentially auto-fixable
apigenie lint takes files, so the shell’s ** glob, or a list, stands in for Buf’s module
directory. Without configuration it applies the recommended
ruleset: the structural rules at error, the naming and style rules at warning, which the
command reports without failing. Buf’s STANDARD category fails on each of them. To fail the
same way, extend strict or raise the rules you care about to error (below).
Imports are looked up on the paths given with --proto-path, then in import_paths of the
configuration, then in the directory of the importing file and the directories above it, and
google/protobuf/*.proto are built in. A Buf module whose files import each other by their
path from the module root needs that root on the path:
apigenie lint --proto-path proto proto/**/*.proto
Replace buf.yaml
version: v2
lint:
use:
- STANDARD
- COMMENTS
except:
- PACKAGE_VERSION_SUFFIX
service_suffix: API
ignore:
- proto/vendor
breaking:
use:
- FILE
becomes
version: '1.0'
commands:
lint:
protobuf:
extends: [strict]
import_paths: [proto]
rules:
package-version-suffix: off
service-suffix: [warning, API]
field-comment-defined: error
oneof-comment-defined: error
enum-value-comment-defined: error
diff:
protobuf:
extends: [recommended]
usenames categories;extendsnames a ruleset:recommended(29 rules),strict(34: it adds the comments of messages, enums, services and RPCs, sorted imports, the version suffix and spelling) orall(39: every comment, and no streaming). The table below maps each rule.exceptisoffon the rule; a rule’s options follow its severity in a list.ignoreby path has no counterpart: leave the directory out of the files you pass, or disable the rules in the files with// apigenie-lint-disable.breaking.usehas no equivalent list: therecommendedrules of the differ rate every change (below).
The lint rules by name
| Buf | API Genie |
|---|---|
DIRECTORY_SAME_PACKAGE, PACKAGE_DEFINED, PACKAGE_DIRECTORY_MATCH, PACKAGE_NO_IMPORT_CYCLE, PACKAGE_SAME_DIRECTORY |
directory-same-package, package-defined, package-directory-match, package-no-import-cycle, package-same-directory |
ENUM_PASCAL_CASE, ENUM_VALUE_UPPER_SNAKE_CASE, FIELD_LOWER_SNAKE_CASE, MESSAGE_PASCAL_CASE, ONEOF_LOWER_SNAKE_CASE, PACKAGE_LOWER_SNAKE_CASE, RPC_PASCAL_CASE, SERVICE_PASCAL_CASE, FILE_LOWER_SNAKE_CASE |
enum-naming, enum-value-naming, field-naming, message-naming, oneof-naming, package-naming, rpc-naming, service-naming, file-naming |
ENUM_NO_ALLOW_ALIAS |
enum-no-allow-alias |
ENUM_VALUE_PREFIX, ENUM_ZERO_VALUE_SUFFIX |
enum-value-prefix, enum-zero-value-suffix (the suffix is an option) |
IMPORT_NO_PUBLIC, IMPORT_USED |
import-no-public, import-used; also import-no-weak, import-resolved and, in strict, imports-sorted |
PACKAGE_SAME_CSHARP_NAMESPACE, PACKAGE_SAME_GO_PACKAGE, PACKAGE_SAME_JAVA_MULTIPLE_FILES, PACKAGE_SAME_JAVA_PACKAGE, PACKAGE_SAME_PHP_NAMESPACE, PACKAGE_SAME_RUBY_PACKAGE, PACKAGE_SAME_SWIFT_PREFIX |
package-options-consistent, one rule for the language options |
PACKAGE_VERSION_SUFFIX |
package-version-suffix, in strict |
RPC_REQUEST_RESPONSE_UNIQUE, RPC_REQUEST_STANDARD_NAME, RPC_RESPONSE_STANDARD_NAME, SERVICE_SUFFIX |
rpc-request-response-unique, rpc-request-standard-name, rpc-response-standard-name, service-suffix (the suffix is an option) |
COMMENT_ENUM, COMMENT_MESSAGE, COMMENT_RPC, COMMENT_SERVICE |
enum-comment-defined, message-comment-defined, rpc-comment-defined, service-comment-defined, in strict |
COMMENT_ENUM_VALUE, COMMENT_FIELD, COMMENT_ONEOF |
enum-value-comment-defined, field-comment-defined, oneof-comment-defined, in all |
RPC_NO_CLIENT_STREAMING, RPC_NO_SERVER_STREAMING |
rpc-no-client-streaming, rpc-no-server-streaming, in all |
ENUM_FIRST_VALUE_ZERO, FIELD_NOT_REQUIRED, SYNTAX_SPECIFIED |
not checked: the linter reads proto3 files, where protoc enforces the three |
PROTOVALIDATE, STABLE_PACKAGE_NO_IMPORT_UNSTABLE |
not checked |
Buf has nothing for spell-check, which checks the
comments with a dictionary (in strict, at warning).
Breaking changes
buf breaking compares a module with another input, a Git ref included. apigenie diff compares
two files, so CI fetches the old version first:
git show main:proto/shop/v1/shop.proto > main.proto
apigenie diff main.proto proto/shop/v1/shop.proto
shop.proto
19 1 breaking Modified message 'Customer'
19 1 non-breaking Removed field 'fax'
25 3 breaking Modified field 'full_name'
25 3 breaking Modified field name
The command exits with 1 on a breaking change, as buf breaking does. Fields and enum values are
matched by number, so a renamed field is a modified one, and removing a field whose number the new
file reserves is non-breaking, the way Protobuf retires fields. The files are compared on their
own: to compare the declarations of imported files too, bundle
both versions first.
Buf’s categories pick what counts: FILE and PACKAGE protect the generated code, WIRE and
WIRE_JSON the encoded messages. The recommended rules of the differ rate as breaking what
breaks either: a removed or renamed declaration, a changed type, label or oneof, the types or the
streaming of a method, the package, an option that names the generated code such as
go_package. For a WIRE policy, where names and compatible types may change, the differ’s
rules say so, with a wire_compatible condition:
version: '1.0'
commands:
diff:
protobuf:
extends: [recommended]
rules:
field:
name: non_breaking
type:
modify:
if: wire_compatible
then: non_breaking
else: breaking
diff.protobuf lists the properties of every node type,
the conditions, and overrides for a part of the file, which stand in for Buf’s ignore and
ignore_only.
Format
buf format -w is apigenie format on the files, and buf format --exit-code is
apigenie format --check, which writes nothing and exits with 1 when a file is not formatted.
The formatter writes one statement per line, normalizes the spacing and keeps every comment.
With the recommended settings it also sorts imports and option statements; extends: [original]
under commands.format keeps every order, and the Protobuf settings
sort declarations, fields or methods when you want that. proto2 and editions files are reported
and left untouched.
The report in CI
--error-format junit is --formatter junit for apigenie lint; msvs is visualstudio;
json keeps its name; github-actions and gitlab-code-quality have no counterpart, and
apigenie diff writes json, markdown, html, stylish, unix and visualstudio. On
GitHub, the GitHub Action runs the three commands with their results
as annotations on the lines of the pull request, and diff compares each file with its version
at the base of the pull request on its own:
on: pull_request
jobs:
proto:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: mykulyak/api-genie-action@v0
with:
command: lint
files: proto/**/*.proto
- uses: mykulyak/api-genie-action@v0
with:
command: format
files: proto/**/*.proto
- uses: mykulyak/api-genie-action@v0
with:
command: diff
files: proto/**/*.proto
What stays with Buf
- Code generation.
buf generatewithbuf.gen.yamland its plugins has no counterpart.apigenie-generatorreads.protofiles for an Elixir gRPC client and an Elixir gRPC server only. - The Buf Schema Registry.
depsand--against-registryhave no counterpart; the imported files have to be in the repository, on an import path. - proto2 and editions. The linter, the differ and the formatter read proto3 files.
What Buf lacks: a gRPC mock of the .proto file, served with
stubs, reflection and recording, and apigenie test for GRPC requests
in .http files, checked against the file.
Remove Buf from the checks
When the two reports agree on the files you have: replace buf lint, buf breaking and
buf format in the scripts and the pipeline, keep buf generate and its buf.gen.yaml, and
keep buf.yaml for it, without the lint and breaking sections.