protoc-gen-docjson

protoc-gen-docjson

A protobuf compiler plugin that generates data to use in rendering documentation templates from protobuf specs.

This plugin generates a JSON file containing meta data from protobuf specifications that is useful for building (in an automated fashion) documentation in HTML, Markdown, etc., for easier reading and distribution. This allows using the comments in the protobuf specifications as documentation in other formats without duplication of effort. In turn, this helps to keep documentation up to date, since the text for the documentation is in the protobuf specifications themselves.

Installation

Using protoc-gen-docjson requires the protobuf compiler (protoc) to be installed. It is what calls protoc-gen-docjson with the required information about the protobuf files. In turn, the protoc-gen-docjson executable must be in your execution path, or, alternatively, provide protoc the full path to the executable (using the --plugin parameter).

env GOBIN=/my/bin/dir go install github.com/cuberat/protoc-gen-docjson/cmd/protoc-gen-docjson@latest

Running

The following command will generate a file named docs.json in the ${OUT_DIR} directory.

cd ${proto_dir} && protoc -I${proto_dir} \
    --docjson_out="${OUT_DIR}" \
    --docjson_opt=outfile=docs.json \
    file1 file2 file3

It is important to cd to the directory containing the protobuf specifications if you need to document custom options (extensions to the various *Options messages defined in the descriptor protobuf), as the plugin needs to be able to open the files itself in that case. Alternatively, you may provide the protobuf directory path to the plugin by adding the proto= parameter to the --docjson_opt parameter (separate parameters to --docjson_opt with commas). E.g.,

cd ${proto_dir} && protoc -I${PROTO_DIR} \
    --docjson_out="${OUT_DIR}" \
    --docjson_opt=outfile=docs.json,proto="${PROTO_DIR}" \
    file1 file2 file3

Output Structure

Top-Level Fields

file_name_list

A list of the file names provided to the protobuf compiler. The file names are ordered such that files are listed before other files that include them. The file names are actually relative paths. E.g., if you have a subdirectory in your protobuf specification directory, files in the that directory will be prefixed with the directory name.

file_map

A map of file names to file descriptors.

service_name_list

A list of fully-qualified service names appearing in the input files.

service_map

A map of fully-qualified service names to service descriptors.

message_name_list

A list of fully-qualified message names.

message_map

A map of fully-qualified message names to message descriptors.

extension_name_list

A list of fully-qualified extension names.

extension_map

A map of fully-qualified extension names to extension descriptors

enum_name_list

A list of fully-qualified enumerations.

enum_map

A map of fully-qualfied enumerations to enumeration descriptors

message_deps

A map of fully-qualified message names to a list of the messages and enumerations they depend on.

service_deps

A map of fully-qualified service names to a list of the messages and enumerations they depend on.

service_file_deps

A map of fully-qualified service names to a list of the files containing messages and enumerations they depend on.

Common Fields

The descriptors describe below have some fields in common, so those are described here.

Comments

This is a description generated from the leading_comments and trailing_comments fields. If both the leading_comments and trailing_comments fields are non-empty, the description is generated by appending a space to the content of leading_comments and then appending the content of trailing_comments. Otherwise, the content of whichever fields is populated will be used as the description.

Leading comments appear before, but are attached to, an item in the protobuf specification. The following is an example of a leading comment:

// Leading comment for the name field.
string name = 1;

This is not a leading comment, because it is not attached (there is a blank line between the comment and the field specification):

// NOT a leading comment for the name field.

string name = 1;

Trailing comments appear to the right of an item in the protobuf specification. The following is an example of a trailing comment:

int32 bar = 3; // Trailing comment for the bar field.

This field contains a list of leading detached comments. That is, these are comments appearing between the item in question and the item above it, but they are not attached to the item. In the following example, the block comment is a leading detached comment:

/*
 * Leading detached comment for the syntax statement.
 * A second line.
 */

// This is the syntax statement leading comment.
syntax = "proto3"; // This is a trailing comment for syntax.

Both protobuf specification comment styles are supported here and throughout the specifications:

// Syntax leading comment for a proto file in a subdirectory. Both block-style
// comments and line-style comments work throughout the specifications.

// This is a syntax statement leading comment.
syntax = "proto3";

custom_options

custom_options is a map of fully-qualified names of extensions that extend protobuf descriptor options. For example, if a method_not_implemented extension is defined like so:

extend google.protobuf.MethodOptions {
  optional bool method_not_implemented = 51235;
}

and the package is specified as MyServices.Tester, and a service method is defined like so:

rpc RunTestV2(MyServices.Tester.TesterRequest)
        returns (MyServices.Tester.TesterResponse) {
            option (MyServices.Tester.method_not_implemented) = true;
        }

then the custom_options field will look like the following:

"custom_options": {
  "method_not_implemented": true
}

This gives you more information to use when rendering templates, e.g., highlight the fact that this service method is not ready to use yet. You can find more details on custom options on the protobuf.dev website.

Descriptors

File Descriptor

Fields:

• name: relative path to the file.
• package: package name specified by the package declaration.
• messages: a list of messages. See the Message Descriptor section for details.
• enums: a list of enumerations. See the Enum Descriptor section for details.
• services: a list of services. See the Service Descriptor section for details.
• dependencies: a list of files this file depends on (imports) that are in the list of files provided to the protobuf compiler.
• external_dependencies: a list of files this file depends on (imports) that are not in the list of files provided to the protobuf compiler.
• options: a map of options specific to files. See the File Options section for details.
• extensions: a list of extensions defined in this file.
• syntax: a syntax descriptor.
• custom_options: a map of custom options. See the custom_options section for details.
• declared_custom_options: if an extension was defined to extend one of the structures used to represent protobuf specifications (e.g., google.protobuf.MessageOptions), information on that extension (same information as in the extensions field) is provided here as a map of type to list of extensions. The valid types are file, service, message, field, enum_decl, and enum_val.
• Comment fields

Syntax Declaration

• version: protobuf syntax version. E.g., "proto2", "proto3".
• Comment fields

See the FileOptions message in descriptor.proto for official documentation on these options.

• java_package
• java_outer_classname
• java_multiple_files
• java_string_check_utf8
• go_package
• deprecated
• cc_enable_arenas
• objc_class_prefix
• csharp_namespace
• swift_prefix
• php_class_prefix
• php_namespace
• php_metadata_namespace
• ruby_package

Service Descriptor

• name: name of the service.
• full_name: fully-qualified name of the service.
• methods: list of method descriptors.
• options: a service options descriptor.
• custom_options: map of custom options.
• Comment fields

See the ServiceOptions message in descriptor.proto for official documentation on this option.

• deprecated

Method Descriptor

• name: name of the method.
• full_name: full-qualified name of the method.
• request_type: message name indicating the type in the request.
• request_full_type: fully-qualified message name indicating the type in the request.
• request_streaming: boolean indicating whether this method supports client streaming.
• response_type: message name indicating the type in the response.
• response_full_type full-qualified message name indicating the type in the response.
• response_streaming: boolean indicating whether this method supports server streaming.
• options: a method options descriptor.
• custom_options: map of custom options.
• Comment fields

Method Options

See the MethodOptions message in descriptor.proto for official documentation on this option.

• deprecated

Message Descriptor

• name: message name.
• full_name: fully-qualified message name.
• fields: a list of field descriptors.
• nested_messages: a list of nested messages. Each message is another Message Descriptor, which can, in turn, have nested messages.
• enums: list of enum descriptors.
• oneof_decl: a list of oneof descriptors.
• options: a message options descriptor.
• custom_options: map of custom options.
• Comment fields

See the MessageOptions message in descriptor.proto for official documentation on these options.

• deprecated
• map_entry

Field Descriptor

• name: name of the field.
• full_name: fully-qualified name of the field.
• type: type of the field, as a string. E.g., "bool".
• full_type: fully-qualified type of the field. This differs from type when it is a message or enum.
• kind: similar to type, except if the field type is a message or enum, the value will be "message" or "enum".
• label: the label in the field declaration. E.g., "repeated", "optional".
• field_number: the field number/slot number of this field.
• default_value: default value for the field.
• oneof_index: index representing which oneof in the message this field is in. Check in_oneof to distinguish between an unset value for oneof_index (in which case, the field is not in a oneof, but this field still exists with the value zero) and an actual index.
• in_oneof: boolean indicating whether this field is declared inside a oneof.
• oneof_name: name of the oneof this field is declared in.
• oneof_full_name: fully-qualified oneof this field is declared in.
• options: a field options descriptor.
• custom_options: map of custom options.
• Comment fields

Field Options

See the FieldOptions message in descriptor.proto for official documentation on these options.

• ctype
• packed
• jstype
• lazy
• deprecated

Oneof Descriptor

• name: name of the oneof.
• full_name: fully-qualified name of the oneof.
• Comment fields

Enum Descriptor

• name: name of the enum.
• full_name: fully-qualified name of the enum.
• description: comment before (but attached to) the enum declaration.
• Comment fields: see the Comments section.
• defined_in: the name of the file the enum is declared in.

See the EnumOptions message in descriptor.proto for official documentation on these options.

• allow_alias
• deprecated

Enum Value Descriptor

• name: the name of the enum value.
• number: the number of the enum value.
• options: an enum options descriptor.
• custom_options: a map of custom options.
• Comment fields: see the Comments section.

Enum Value Options

See the EnumValueOptions message in descriptor.proto for official documentation on this option.

• deprecated

Extension Descriptor

• name: name of the extension. E.g., "field_required".
• full_name: fully-qualified name of the extension. E.g., "MyServices.Tester.field_required".
• field_number: the field number/slot number for this field. E.g., 51234.
• type: type of the field in the extension. E.g., "bool".
• extendee: the extended protobuf message name. E.g., "google.protobuf.MessageOptions".
• Comment fields: see the Comments section.
• defined_in: the name of the file the extension is declared in.

Glossary

• Descriptor: a data structure describing a thing. Descriptors described in this document are not the same structures as the protobuf descriptors used for the output of this plugin, but they tend to be modeled after them.
• Fully-qualified name: a name prefixed with the package name, and, if applicable, the parent message name, etc.