protolock

package module
v0.17.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 19, 2023 License: BSD-3-Clause Imports: 12 Imported by: 2

README

protolock

Track your .proto files and prevent changes to messages and services which impact API compatibility.

CircleCI GoDoc

Why

Ever accidentally break your API compatibility while you're busy fixing problems? You may have forgotten to reserve the field number of a message or you re-ordered fields after removing a property. Maybe a new team member was not familiar with the backward-compatibility of Protocol Buffers and made an easy mistake.

protolock attempts to help prevent this from happening.

Overview

  1. Initialize your repository:

     $ protolock init
 # creates a `proto.lock` file

  2. Add changes to .proto messages or services, verify no breaking changes made:

     $ protolock status
 CONFLICT: "Channel" is missing ID: 108, which had been reserved [path/to/file.proto]
 CONFLICT: "Channel" is missing ID: 109, which had been reserved [path/to/file.proto]

  3. Commit a new state of your .protos (rewrites proto.lock if no warnings):

     $ protolock commit
 # optionally provide --force flag to disregard warnings

  4. Integrate into your protobuf compilation step:

     $ protolock status && protoc -I ...

In all, prevent yourself from compiling your protobufs and generating code if breaking changes have been made.

Recommended: commit the output proto.lock file into your version control system

Install

If you have Go installed, you can install protolock by running:

  • Go >= 1.17:

    go install github.com/nilslice/protolock/cmd/protolock@latest

  • Go < 1.17:

    go get github.com/nilslice/protolock/cmd/protolock

Otherwise, download a pre-built binary for Windows, macOS, or Linux from the latest release page.

Usage

protolock <command> [options]

Commands:
	-h, --help, help	display the usage information for protolock
	init			initialize a proto.lock file from current tree
	status			check for breaking changes and report conflicts
	commit			rewrite proto.lock file with current tree if no conflicts (--force to override)

Options:
	--strict [true]		enable strict mode and enforce all built-in rules
	--debug	[false]		enable debug mode and output debug messages
	--ignore 		comma-separated list of filepaths to ignore
	--force [false]		forces commit to rewrite proto.lock file and disregards warnings
	--plugins 		comma-separated list of executable protolock plugin names
	--lockdir [.]		directory of proto.lock file
	--protoroot [.]		root of directory tree containing proto files
	--uptodate [false]	enforce that proto.lock file is up-to-date with proto files

Rules Enforced

No Using Reserved Fields

Compares the current vs. updated Protolock definitions and will return a list of warnings if any message's previously reserved fields or IDs are now being used as part of the same message.

No Removing Reserved Fields

Compares the current vs. updated Protolock definitions and will return a list of warnings if any reserved field has been removed.

Note: This rule is not enforced when strict mode is disabled.

No Changing Field IDs

Compares the current vs. updated Protolock definitions and will return a list of warnings if any field ID number has been changed.

No Changing Field Types

Compares the current vs. updated Protolock definitions and will return a list of warnings if any field type has been changed.

No Changing Field Names

Compares the current vs. updated Protolock definitions and will return a list of warnings if any message's previous fields have been renamed.

Note: This rule is not enforced when strict mode is disabled.

No Removing Fields Without Reserve

Compares the current vs. updated Protolock definitions and will return a list of warnings if any field has been removed without a corresponding reservation of that field name or ID.

No Removing RPCs

Compares the current vs. updated Protolock definitions and will return a list of warnings if any RPCs provided by a Service have been removed.

Note: This rule is not enforced when strict mode is disabled.

No Changing RPC Signature

Compares the current vs. updated Protolock definitions and will return a list of warnings if any RPC signature has been changed while using the same name.

Docker

docker pull nilslice/protolock:latest
docker run -v $(pwd):/protolock -w /protolock nilslice/protolock init

Plugins

The default rules enforced by protolock may not cover everything you want to do. If you have custom checks you'd like run on your .proto files, create a plugin, and have protolock run it and report your warnings. Read the wiki to learn more about creating and using plugins.

Contributing

Please feel free to make pull requests with better support for various rules, optimized code and overall tests. Filing an issue when you encounter a bug or any unexpected behavior is very much appreciated.

For current issues, see: open issues

Acknowledgement

Thank you to Ernest Micklei for his work on the excellent parser heavily relied upon by this tool and many more: https://github.com/emicklei/proto

Documentation

Index

Constants

View Source 
const (
	// FileSep is the string representation of the OS-specific path separator.
	FileSep = string(filepath.Separator)

	// ProtoSep is an OS-ambiguous path separator to encode into the proto.lock
	// file. Use OsPath and ProtoPath funcs to convert.
	ProtoSep = ":/:"
)
View Source 
const (
	// CommentSkip tells the parse step to skip the comparable entity.
	CommentSkip = "@protolock:skip"
)
View Source 
const LockFileName = "proto.lock"

Variables

View Source 
var ErrOutOfDate = errors.New("proto.lock file is not up-to-date with source")

ErrOutOfDate indicates that the locked definitions are ahead or behind of the source proto definitions within the tree.

View Source 
var (
	// ErrSkipEntry indicates that the CommentSkip hint was found.
	ErrSkipEntry = errors.New("protolock: skip entry hint encountered")
)
View Source 
var (
	ErrWarningsFound = errors.New("comparison found one or more warnings")
)
View Source 
var (
	// Rules provides a complete list of all funcs to be run to compare
	// a set of Protolocks. This list should be updated as new RuleFunc's
	// are added to this package.
	Rules = []Rule{
		{
			Name: "NoUsingReservedFields",
			Func: NoUsingReservedFields,
		},
		{
			Name: "NoRemovingReservedFields",
			Func: NoRemovingReservedFields,
		},
		{
			Name: "NoRemovingFieldsWithoutReserve",
			Func: NoRemovingFieldsWithoutReserve,
		},
		{
			Name: "NoChangingFieldIDs",
			Func: NoChangingFieldIDs,
		},
		{
			Name: "NoChangingFieldTypes",
			Func: NoChangingFieldTypes,
		},
		{
			Name: "NoChangingFieldNames",
			Func: NoChangingFieldNames,
		},
		{
			Name: "NoRemovingRPCs",
			Func: NoRemovingRPCs,
		},
		{
			Name: "NoChangingRPCSignature",
			Func: NoChangingRPCSignature,
		},
		{
			Name: "NoMovingExistingFieldsIntoOrOutOfOneof",
			Func: NoMovingExistingFieldsIntoOrOutOfOneof,
		},
	}
)

Functions

func Commit 

func Commit(cfg Config) (io.Reader, error)

Commit will return an io.Reader with the lock representation data for caller to use as needed.

func HandleReport added in v0.11.0 

func HandleReport(report *Report, w io.Writer, err error) (int, error)

HandleReport checks a report for warnigs and writes warnings to an io.Writer. The returned int (an exit code) is 1 if warnings are encountered.

func Init 

func Init(cfg Config) (io.Reader, error)

Init will return an io.Reader with the lock representation data for caller to use as needed.

func SetDebug 

func SetDebug(status bool)

SetDebug enables the user to toggle debug mode on and off.

func SetStrict 

func SetStrict(mode bool)

SetStrict enables the user to toggle strict mode on and off.

Types

type Config added in v0.10.0 

type Config struct {
	LockDir   string
	ProtoRoot string
	Ignore    string
	UpToDate  bool
	Debug     bool
}

func NewConfig added in v0.10.0 

func NewConfig(
	lockDir, protoRoot, ignores string,
	upToDate, debug bool,
) (*Config, error)

func (*Config) LockFileExists added in v0.10.0 

func (cfg *Config) LockFileExists() bool

func (*Config) LockFilePath added in v0.10.0 

func (cfg *Config) LockFilePath() string

type Definition 

type Definition struct {
	Filepath Protopath `json:"protopath,omitempty"`
	Def      Entry     `json:"def,omitempty"`
}

type Entry 

type Entry struct {
	Enums    []Enum    `json:"enums,omitempty"`
	Messages []Message `json:"messages,omitempty"`
	Services []Service `json:"services,omitempty"`
	Imports  []Import  `json:"imports,omitempty"`
	Package  Package   `json:"package,omitempty"`
	Options  []Option  `json:"options,omitempty"`
}

func Parse added in v0.9.1 

func Parse(filename string, r io.Reader) (Entry, error)

type Enum 

type Enum struct {
	Name          string      `json:"name,omitempty"`
	EnumFields    []EnumField `json:"enum_fields,omitempty"`
	ReservedIDs   []int       `json:"reserved_ids,omitempty"`
	ReservedNames []string    `json:"reserved_names,omitempty"`
	AllowAlias    bool        `json:"allow_alias,omitempty"`
	Options       []Option    `json:"options,omitempty"`
}

type EnumField 

type EnumField struct {
	Name    string   `json:"name,omitempty"`
	Integer int      `json:"integer,omitempty"`
	Options []Option `json:"options,omitempty"`
}

type Field 

type Field struct {
	ID          int      `json:"id,omitempty"`
	Name        string   `json:"name,omitempty"`
	Type        string   `json:"type,omitempty"`
	IsRepeated  bool     `json:"is_repeated,omitempty"`
	IsOptional  bool     `json:"optional,omitempty"`
	IsRequired  bool     `json:"required,omitempty"`
	Options     []Option `json:"options,omitempty"`
	OneofParent string   `json:"oneof_parent,omitempty"`
}

type Import added in v0.9.0 

type Import struct {
	Path string `json:"path,omitempty"`
}

type Map 

type Map struct {
	KeyType string `json:"key_type,omitempty"`
	Field   Field  `json:"field,omitempty"`
}

type Message 

type Message struct {
	Name          string    `json:"name,omitempty"`
	Fields        []Field   `json:"fields,omitempty"`
	Maps          []Map     `json:"maps,omitempty"`
	ReservedIDs   []int     `json:"reserved_ids,omitempty"`
	ReservedNames []string  `json:"reserved_names,omitempty"`
	Filepath      Protopath `json:"filepath,omitempty"`
	Messages      []Message `json:"messages,omitempty"`
	Options       []Option  `json:"options,omitempty"`
}

type Option added in v0.9.0 

type Option struct {
	Name       string   `json:"name,omitempty"`
	Value      string   `json:"value,omitempty"`
	Aggregated []Option `json:"aggregated,omitempty"`
}

type Package added in v0.10.0 

type Package struct {
	Name string `json:"name,omitempty"`
}

type ProtoFile 

type ProtoFile struct {
	ProtoPath Protopath
	Entry     Entry
}

type Protolock 

type Protolock struct {
	Definitions []Definition `json:"definitions,omitempty"`
}

func FromReader added in v0.11.2 

func FromReader(r io.Reader) (Protolock, error)

FromReader unmarshals a proto.lock file into a Protolock struct.

func (*Protolock) Equal added in v0.13.0 

func (p *Protolock) Equal(q *Protolock) bool

Check whether one lockfile is equal to another.

type Protopath added in v0.9.0 

type Protopath string

Protopath is a type to assist in OS filepath transformations

func OSPath added in v0.9.0 

func OSPath(ProtoPath Protopath) Protopath

OSPath converts a path in the Protopath format to the OS path format

func ProtoPath added in v0.9.0 

func ProtoPath(OSPath Protopath) Protopath

ProtoPath converts a path in the OS path format to Protopath format

func (Protopath) String added in v0.9.0 

func (p Protopath) String() string

type RPC 

type RPC struct {
	Name        string   `json:"name,omitempty"`
	InType      string   `json:"in_type,omitempty"`
	OutType     string   `json:"out_type,omitempty"`
	InStreamed  bool     `json:"in_streamed,omitempty"`
	OutStreamed bool     `json:"out_streamed,omitempty"`
	Options     []Option `json:"options,omitempty"`
}

type Report 

type Report struct {
	Current  Protolock `json:"current,omitempty"`
	Updated  Protolock `json:"updated,omitempty"`
	Warnings []Warning `json:"warnings,omitempty"`
}

func Compare added in v0.11.2 

func Compare(current, update Protolock) (*Report, error)

Compare returns a Report struct and an error which indicates that there is one or more warnings to report to the caller. If no error is returned, the Report can be ignored.

func Status 

func Status(cfg Config) (*Report, error)

Status will report on any issues encountered when comparing the updated tree of parsed proto files and the current proto.lock file.

type Rule added in v0.11.0 

type Rule struct {
	Name string
	Func RuleFunc
}

type RuleFunc 

type RuleFunc func(current, updated Protolock) ([]Warning, bool)

RuleFunc defines the common signature for a function which can compare Protolock states and determine if issues exist.

type Service 

type Service struct {
	Name     string    `json:"name,omitempty"`
	RPCs     []RPC     `json:"rpcs,omitempty"`
	Filepath Protopath `json:"filepath,omitempty"`
}

type Warning 

type Warning struct {
	Filepath Protopath `json:"filepath,omitempty"`
	Message  string    `json:"message,omitempty"`
	RuleName string    `json:"rulename,omitempty"`
}

func NoChangingFieldIDs 

func NoChangingFieldIDs(cur, upd Protolock) ([]Warning, bool)

NoChangingFieldIDs compares the current vs. updated Protolock definitions and will return a list of warnings if any field ID number has been changed.

func NoChangingFieldNames 

func NoChangingFieldNames(cur, upd Protolock) ([]Warning, bool)

NoChangingFieldNames compares the current vs. updated Protolock definitions and will return a list of warnings if any message's previous fields have been renamed. This rule is only enforced when strict mode is enabled.

func NoChangingFieldTypes 

func NoChangingFieldTypes(cur, upd Protolock) ([]Warning, bool)

NoChangingFieldTypes compares the current vs. updated Protolock definitions and will return a list of warnings if any field type has been changed.

func NoChangingRPCSignature 

func NoChangingRPCSignature(cur, upd Protolock) ([]Warning, bool)

NoChangingRPCSignature compares the current vs. updated Protolock definitions and will return a list of warnings if any RPC signature has been changed while using the same name.

func NoMovingExistingFieldsIntoOrOutOfOneof added in v0.17.0 

func NoMovingExistingFieldsIntoOrOutOfOneof(cur, upd Protolock) ([]Warning, bool)

Existing fields must not be moved into or out of a oneof. This is a backwards-incompatible change in the Go protobuf stubs. per https://google.aip.dev/180#moving-into-oneofs

func NoRemovingFieldsWithoutReserve 

func NoRemovingFieldsWithoutReserve(cur, upd Protolock) ([]Warning, bool)

NoRemovingFieldsWithoutReserve compares the current vs. updated Protolock definitions and will return a list of warnings if any field has been removed without a corresponding reservation of that field name or ID.

func NoRemovingRPCs 

func NoRemovingRPCs(cur, upd Protolock) ([]Warning, bool)

NoRemovingRPCs compares the current vs. updated Protolock definitions and will return a list of warnings if any RPCs provided by a Service have been removed. This rule is only enforced when strict mode is enabled.

func NoRemovingReservedFields 

func NoRemovingReservedFields(cur, upd Protolock) ([]Warning, bool)

NoRemovingReservedFields compares the current vs. updated Protolock definitions and will return a list of warnings if any reserved field has been removed. This rule is only enforced when strict mode is enabled.

func NoUsingReservedFields 

func NoUsingReservedFields(cur, upd Protolock) ([]Warning, bool)

NoUsingReservedFields compares the current vs. updated Protolock definitions and will return a list of warnings if any message's previously reserved fields or IDs are now being used as part of the same message.

Source Files

View all Source files

Directories

Path Synopsis
cmd
protolock
plugin-samples
plugin-sample
plugin-sample-error
plugin-sample-wasm Module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.