protovalidate

protovalidate-go

protovalidate-go is the Go language implementation of protovalidate designed to validate Protobuf messages at runtime based on user-defined validation constraints. Powered by Google's Common Expression Language (CEL), it provides a flexible and efficient foundation for defining and evaluating custom validation rules. The primary goal of protovalidate is to help developers ensure data consistency and integrity across the network without requiring generated code.

The protovalidate project

Head over to the core protovalidate repository for:

• The API definition: used to describe validation constraints
• Documentation: how to apply protovalidate effectively
• Migration tooling: incrementally migrate from protoc-gen-validate
• Conformance testing utilities: for acceptance testing of protovalidate implementations

Other protovalidate runtime implementations:

• C++: protovalidate-cc
• Java: protovalidate-java
• Python: protovalidate-python

And others coming soon:

• TypeScript: protovalidate-ts

For Connect see connectrpc/validate-go.

Installation

To install the package, use the go get command from within your Go module:

go get github.com/bufbuild/protovalidate-go

Import the package into your Go project:

import "github.com/bufbuild/protovalidate-go"

Remember to always check for the latest version of protovalidate-go on the project's GitHub releases page to ensure you're using the most up-to-date version.

Usage

Implementing validation constraints

Validation constraints are defined directly within .proto files. Documentation for adding constraints can be found in the protovalidate project README and its comprehensive docs.

syntax = "proto3";

package my.package;

import "google/protobuf/timestamp.proto";
import "buf/validate/validate.proto";

message Transaction {
  uint64 id = 1 [(buf.validate.field).uint64.gt = 999];
  google.protobuf.Timestamp purchase_date = 2;
  google.protobuf.Timestamp delivery_date = 3;
  
  string price = 4 [(buf.validate.field).cel = {
    id: "transaction.price",
    message: "price must be positive and include a valid currency symbol ($ or £)",
    expression: "(this.startsWith('$') || this.startsWith('£')) && double(this.substring(1)) > 0"
  }];

  option (buf.validate.message).cel = {
    id: "transaction.delivery_date",
    message: "delivery date must be after purchase date",
    expression: "this.delivery_date > this.purchase_date"
  };
}

Buf managed mode

protovalidate-go assumes the constraint extensions are imported into the generated code via buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go.

If you are using Buf managed mode to augment Go code generation, ensure that the protovalidate module is excluded in your buf.gen.yaml:

version: v1
# <snip>
managed:
  enabled: true
  go_package_prefix:
    except:
      - buf.build/bufbuild/protovalidate
# <snip>

Example

package main

import (
	"fmt"
	"time"
	
	pb "github.com/path/to/generated/protos"
	"github.com/bufbuild/protovalidate-go"
	"google.golang.org/protobuf/types/known/timestamppb"
)

func main() {
	msg := &pb.Transaction{
		Id:           1234,
		Price:        "$5.67",
		PurchaseDate: timestamppb.New(time.Now()),
		DeliveryDate: timestamppb.New(time.Now().Add(time.Hour)),
	}

	v, err := protovalidate.New()
	if err != nil {
		fmt.Println("failed to initialize validator:", err)
	}

	if err = v.Validate(msg); err != nil {
		fmt.Println("validation failed:", err)
	} else {
		fmt.Println("validation succeeded")
	}
}

Lazy mode

protovalidate-go defaults to lazily construct validation logic for Protobuf message types the first time they are encountered. A validator's internal cache can be pre-warmed with the WithMessages or WithDescriptors options during initialization:

validator, err := protovalidate.New(
  protovalidate.WithMessages(
    &pb.MyFoo{}, 
    &pb.MyBar{}, 
  ),
)

Lazy mode uses a copy on write cache stategy to reduce the required locking. While performance is sub-microsecond, the overhead can be further reduced by disabling lazy mode with the WithDisableLazy option. Note that all expected messages must be provided during initialization of the validator:

validator, err := protovalidate.New(
  protovalidate.WithDisableLazy(true),
  protovalidate.WithMessages(
    &pb.MyFoo{},
    &pb.MyBar{},
  ),
)

Support legacy protoc-gen-validate constraints

The protovalidate-go module comes with a legacy package which adds opt-in support for existing protoc-gen-validate constraints. Provide thelegacy.WithLegacySupport option when initializing the validator:

validator, err := protovalidate.New(
  legacy.WithLegacySupport(legacy.ModeMerge),
)

protoc-gen-validate code generation is not used by protovalidate-go. The legacy package assumes the protoc-gen-validate extensions are imported into the generated code via github.com/envoyproxy/protoc-gen-validate/validate.

A migration tool is also available to incrementally upgrade legacy constraints in .proto files.

Performance

Benchmarks are provided to test a variety of use-cases. Generally, after the initial cold start, validation on a message is sub-microsecond and only allocates in the event of a validation error.

[circa 14 September 2023]
goos: darwin
goarch: arm64
pkg: github.com/bufbuild/protovalidate-go
BenchmarkValidator
BenchmarkValidator/ColdStart-10              4192  246278 ns/op  437698 B/op  5955 allocs/op
BenchmarkValidator/Lazy/Valid-10         11816635   95.08 ns/op       0 B/op     0 allocs/op
BenchmarkValidator/Lazy/Invalid-10        2983478   380.5 ns/op     649 B/op    15 allocs/op
BenchmarkValidator/Lazy/FailFast-10      12268683   98.22 ns/op     168 B/op     3 allocs/op
BenchmarkValidator/PreWarmed/Valid-10    12209587   90.36 ns/op       0 B/op     0 allocs/op
BenchmarkValidator/PreWarmed/Invalid-10   3098940   394.1 ns/op     649 B/op    15 allocs/op
BenchmarkValidator/PreWarmed/FailFast-10 12291523   99.27 ns/op     168 B/op     3 allocs/op
PASS

Ecosystem

• protovalidate core repository
• Buf
• CEL Go
• CEL Spec

Legal

Offered under the Apache 2 license.