protoc-gen-go-hash

protoc-gen-go-hash

protoc-gen-go-hash is a simple compiler plugin to protoc. It adds one method called Hash to the message struct. This is used to produce a deterministic hash out of the protocol buffer message.

Background

At Refurbed we have a caching system that supports A/B testing. Therefore, we have the need to cache protocol buffer messages. We cannot really get a hash of the actual protobuf binary data since the default serialization is not deterministic, neither can we hash the json marshalled object because objects are unordered collections of name/value pairs. So we are left with using hashstructure.

Prerequisites

• Go 1.19
• Protocol Buffer Compiler
• Go protocol Buffers plugin

Usage

Install

In order to install the protoc-gen-go-hash plugin to use it in your projects first run:

go install gitlab.com/refurbed-community/oss/protoc-gen-go-hash

Then import hashstructure dependency in your project:

go get github.com/mitchellh/hashstructure/v2

And finally generate the protobuf messages you require:

protoc --go_out=. --go-hash_out=. <your proto files>

Local Development

First clone the repo.

Then build the protoc-gen-go-hash plugin:

go build ./cmd/protoc-gen-go-hash

And finally generate the example protobuf message:

protoc --plugin=protoc-gen-go-hash=./protoc-gen-go-hash --go_out=. --go_opt=paths=source_relative --go-hash_out=. --go-hash_opt=paths=source_relative example/example.proto

Outcome

For the example protobuf file, the following two files will be created, example.pb.go and example_hash.pb.go:

proto-gen-hash/
├─ example/
│  ├─ example.proto
│  ├─ example.pb.go
│  ├─ example_hash.pb.go

example.pb.go is the familiar code generated by the protoc go plugin. And then the example_hash.pb.go will look like this:

// Code generated by protoc-gen-go-hash. DO NOT EDIT.

package example

import (
	"fmt"
	"github.com/mitchellh/hashstructure/v2"
)

func (x *Hello) Hash() (uint64, error) {
	if x == nil {
		return 0, fmt.Errorf("message is defined as nil")
	}
	return hashstructure.Hash(*x, hashstructure.FormatV2, nil)
}

Therefore, the following code:

h := &example.Hello{}
h.CustomerId = "1234"
fmt.Println(h.Hash())

prints out:

10958536349413352795 <nil>