centraldogma

package module
v0.0.0-...-69da0db Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 2, 2023 License: Apache-2.0 Imports: 25 Imported by: 1

README

Central Dogma client for Go

Usage

Getting started
import "go.linecorp.com/centraldogma"

// create a client with OAuth2 token
// See also: https://line.github.io/centraldogma/auth.html#application-token
centraldogma.NewClientWithToken(baseURL, token, nil)
Customize transport

If transport is nil (like above), http2.Transport is used by default.

You could inject your own transport easily:

import "golang.org/x/net/http2"

tr := &http2.Transport{
    DisableCompression: false,
}

// create a client with custom transport
centraldogma.NewClientWithToken(baseURL, token, tr)
Example
package sample

import (
	"context"
	"sync/atomic"
	"time"

	"go.linecorp.com/centraldogma"
)

// CentralDogmaFile represents a file in application repository, stored on Central Dogma server.
type CentralDogmaFile struct {
	client            atomic.Value
	BaseURL           string       `yaml:"base_url" json:"base_url"`
	Token             string       `yaml:"token" json:"token"`
	Project           string       `yaml:"project" json:"project"`
	Repo              string       `yaml:"repo" json:"repo"`
	Path              string       `yaml:"path" json:"path"`
	LastKnownRevision atomic.Value `yaml:"-" json:"-"`
	TimeoutSec        int          `yaml:"timeout_sec" json:"timeout_sec"`
}

func (c *CentralDogmaFile) getClientOrSet() (*centraldogma.Client, error) {
	if v, stored := c.client.Load().(*centraldogma.Client); stored {
		return v, nil
	}

	// create a new client
	dogmaClient, err := centraldogma.NewClientWithToken(c.BaseURL, c.Token, nil)
	if err != nil {
		return nil, err
	}

	// store
	c.client.Store(dogmaClient)

	return dogmaClient, nil
}

// Fetch file content from Central Dogma.
func (c *CentralDogmaFile) Fetch(ctx context.Context) (b []byte, err error) {
	dogmaClient, err := c.getClientOrSet()
	if err != nil {
		return
	}

	entry, _, err := dogmaClient.GetFile(ctx, c.Project, c.Repo, "", &centraldogma.Query{
		Path: c.Path,
		Type: centraldogma.Identity,
	})
	if err != nil {
		return
	}

	// set last known revision
	c.LastKnownRevision.Store(entry.Revision)

	b = entry.Content
	return
}

// Watch changes on remote file.
func (c *CentralDogmaFile) Watch(ctx context.Context, callback func([]byte)) error {
	dogmaClient, err := c.getClientOrSet()
	if err != nil {
		return err
	}

	ch, closer, err := dogmaClient.WatchFile(ctx, c.Project, c.Repo, &centraldogma.Query{
		Path: c.Path,
		Type: centraldogma.Identity,
	}, time.Duration(c.TimeoutSec)*time.Second)
	if err != nil {
		return err
	}

	go func() {
		for {
			select {
			case <-ctx.Done():
				closer()
				return

			case changes := <-ch:
				if changes.Err == nil {
					callback(changes.Entry.Content)
				}
			}
		}
	}()

	return nil
}
Building CLI

We use Go Modules (formerly known as vgo) to manage the dependencies.

# Opt-in for Go Modules.
$ export GO111MODULE=on

# Set up the GOPATH.
$ mkdir go
$ export GOPATH="$(pwd)/go"

# Clone the repository.
$ cd "$GOPATH"
$ git clone https://github.com/line/centraldogma-go src/go.linecorp.com/centraldogma

# Build the CLI.
$ cd "$GOPATH/src/go.linecorp.com/centraldogma/internal/app/dogma"
$ go build
$ ./dogma -help

Documentation

Overview

Package centraldogma provides a Go client for accessing Central Dogma. Visit https://line.github.io/centraldogma/ to learn more about Central Dogma.

Usage: 

import "go.linecorp.com/centraldogma"

Create a client with the username and password, then use the client to access the Central Dogma HTTP APIs. For example: 

username := "foo"
password := "bar"
client, err := centraldogma.NewClientWithToken("https://localhost:443", "myToken", nil)

projects, res, err := client.ListProjects(context.Background())

Note that all of the APIs are using the https://godoc.org/context which can pass cancellation and deadlines for handling a request.

Index

Constants

View Source 
const (
	DefaultChannelBuffer = 128

	UnknownHttpStatusCode = 0

	DefaultClientName = "centralDogmaClient"
)

Variables

View Source 
var (
	ErrLatestNotSet = fmt.Errorf("latest is not set yet")

	ErrQueryMustBeSet = fmt.Errorf("query should not be nil")

	ErrWatcherClosed = fmt.Errorf("watcher is closed")

	ErrTokenEmpty = fmt.Errorf("token should not be empty")

	ErrTransportMustBeSet = fmt.Errorf("transport should not be nil")

	ErrTransportMustNotBeOAuth2 = fmt.Errorf("transport cannot be oauth2.Transport")

	ErrMetricCollectorConfigMustBeSet = fmt.Errorf("metric collector config should not be nil")
)

Functions

func DefaultHTTP2Transport 

func DefaultHTTP2Transport(baseURL string) (*http2.Transport, error)

DefaultHTTP2Transport returns a http2.Transport which could be used on cleartext or encrypted connection depending on the scheme of the baseURL.

func DefaultMetricCollectorConfig 

func DefaultMetricCollectorConfig(name string) (c *metrics.Config)

DefaultMetricCollectorConfig returns default metric collector config.

func DefaultOAuth2Transport 

func DefaultOAuth2Transport(baseURL, token string, transport http.RoundTripper) (*oauth2.Transport, error)

DefaultOAuth2Transport returns an oauth2.Transport which internally uses the specified transport and attaches the specified token to every request using the authorization header. If the transport is a type of oauth2.Transport, it will throw an error.

func GlobalPrometheusMetricCollector 

func GlobalPrometheusMetricCollector(config *metrics.Config) (m *metrics.Metrics, err error)

GlobalPrometheusMetricCollector returns global metric collector which sinks to Prometheus metrics endpoint. Be aware that function may cause panic on error.

func StatsdMetricCollector 

func StatsdMetricCollector(config *metrics.Config, addr string) (m *metrics.Metrics, err error)

StatsdMetricCollector returns metric collector which sinks to statsd endpoint.

func StatsiteMetricCollector 

func StatsiteMetricCollector(config *metrics.Config, addr string) (m *metrics.Metrics, err error)

StatsiteMetricCollector returns metric collector which sinks to statsite endpoint.

Types

type Author 

type Author struct {
	Name  string `json:"name,omitempty"`
	Email string `json:"email,omitempty"`
}

type Change 

type Change struct {
	Path    string      `json:"path"`
	Type    ChangeType  `json:"type"`
	Content interface{} `json:"content,omitempty"`
}

Change represents a change to commit in the repository.

func (*Change) MarshalJSON 

func (c *Change) MarshalJSON() ([]byte, error)

func (*Change) UnmarshalJSON 

func (c *Change) UnmarshalJSON(b []byte) error

type ChangeType 

type ChangeType int
const (
	UpsertJSON ChangeType = iota + 1
	UpsertText
	Remove
	Rename
	ApplyJSONPatch
	ApplyTextPatch
)

func (ChangeType) String 

func (c ChangeType) String() string

String returns the string value of ChangeType

type Client 

type Client struct {
	// contains filtered or unexported fields
}

A Client communicates with the Central Dogma server API.

func NewClientWithToken 

func NewClientWithToken(baseURL, token string, transport http.RoundTripper) (*Client, error)

NewClientWithToken returns a Central Dogma client which communicates the server at baseURL, using the specified token and transport. If transport is nil, http2.Transport is used by default.

func (*Client) CreateProject 

func (c *Client) CreateProject(ctx context.Context, name string) (pro *Project, httpStatusCode int, err error)

CreateProject creates a project.

func (*Client) CreateRepository 

func (c *Client) CreateRepository(
	ctx context.Context, projectName, repoName string) (repo *Repository, httpStatusCode int, err error)

CreateRepository creates a repository.

func (*Client) FileWatcher 

func (c *Client) FileWatcher(projectName, repoName string, query *Query) (*Watcher, error)

FileWatcher returns a Watcher which notifies its listeners when the result of the given Query becomes available or changes. For example: 

query := &Query{Path: "/a.json", Type: Identity}
watcher := client.FileWatcher("foo", "bar", query)

myCh := make(chan interface{})
watcher.Watch(func(revision int, value interface{}) {
    myCh <- value
})
myValue := <-myCh

func (*Client) GetDiff 

func (c *Client) GetDiff(ctx context.Context,
	projectName, repoName, from, to string, query *Query) (change *Change, httpStatusCode int, err error)

GetDiff returns the diff of a file between two revisions. If the from and to are not specified, this will return the diff from the init to the latest revision.

func (*Client) GetDiffs 

func (c *Client) GetDiffs(ctx context.Context,
	projectName, repoName, from, to, pathPattern string) (changes []*Change, httpStatusCode int, err error)

GetDiffs returns the diffs of the files that match the given path pattern. A path pattern is a variant of glob:

  • "/**": find all files recursively
  • "*.json": find all JSON files recursively
  • "/foo/*.json": find all JSON files under the directory /foo
  • "/&#42;/foo.txt": find all files named foo.txt at the second depth level
  • "*.json,/bar/*.txt": use comma to match any patterns

If the from and to are not specified, this will return the diffs from the init to the latest revision.

func (*Client) GetFile 

func (c *Client) GetFile(
	ctx context.Context, projectName, repoName, revision string, query *Query) (entry *Entry,
	httpStatusCode int, err error)

GetFile returns the file at the specified revision and path with the specified Query.

func (*Client) GetFiles 

func (c *Client) GetFiles(ctx context.Context,
	projectName, repoName, revision, pathPattern string) (entries []*Entry, httpStatusCode int, err error)

GetFiles returns the files that match the given path pattern. A path pattern is a variant of glob:

  • "/**": find all files recursively
  • "*.json": find all JSON files recursively
  • "/foo/*.json": find all JSON files under the directory /foo
  • "/&#42;/foo.txt": find all files named foo.txt at the second depth level
  • "*.json,/bar/*.txt": use comma to match any patterns

func (*Client) GetHistory 

func (c *Client) GetHistory(ctx context.Context,
	projectName, repoName, from, to, pathPattern string, maxCommits int) (commits []*Commit,
	httpStatusCode int, err error)

GetHistory returns the history of the files that match the given path pattern. A path pattern is a variant of glob:

  • "/**": find all files recursively
  • "*.json": find all JSON files recursively
  • "/foo/*.json": find all JSON files under the directory /foo
  • "/&#42;/foo.txt": find all files named foo.txt at the second depth level
  • "*.json,/bar/*.txt": use comma to match any patterns

If the from and to are not specified, this will return the history from the init to the latest revision.

func (*Client) ListFiles 

func (c *Client) ListFiles(ctx context.Context,
	projectName, repoName, revision, pathPattern string) (entries []*Entry, httpStatusCode int, err error)

ListFiles returns the list of files that match the given path pattern. A path pattern is a variant of glob:

  • "/**": find all files recursively
  • "*.json": find all JSON files recursively
  • "/foo/*.json": find all JSON files under the directory /foo
  • "/&#42;/foo.txt": find all files named foo.txt at the second depth level
  • "*.json,/bar/*.txt": use comma to match any patterns

func (*Client) ListProjects 

func (c *Client) ListProjects(ctx context.Context) (pros []*Project, httpStatusCode int, err error)

ListProjects returns the list of projects.

func (*Client) ListRemovedProjects 

func (c *Client) ListRemovedProjects(ctx context.Context) (removedPros []*Project, httpStatusCode int, err error)

ListRemovedProjects returns the list of removed projects.

func (*Client) ListRemovedRepositories 

func (c *Client) ListRemovedRepositories(
	ctx context.Context, projectName string) (removedRepos []*Repository, httpStatusCode int, err error)

ListRemovedRepositories returns the list of the removed repositories which can be unremoved using UnremoveRepository.

func (*Client) ListRepositories 

func (c *Client) ListRepositories(
	ctx context.Context, projectName string) (repos []*Repository, httpStatusCode int, err error)

ListRepositories returns the list of repositories.

func (*Client) NormalizeRevision 

func (c *Client) NormalizeRevision(
	ctx context.Context, projectName, repoName, revision string) (normalizedRev int, httpStatusCode int, err error)

NormalizeRevision converts the relative revision number to the absolute revision number(e.g. -1 -> 3).

func (*Client) PurgeProject 

func (c *Client) PurgeProject(ctx context.Context, name string) (httpStatusCode int, err error)

PurgeProject purges a project which was removed before.

func (*Client) PurgeRepository 

func (c *Client) PurgeRepository(ctx context.Context, projectName, repoName string) (httpStatusCode int, err error)

PurgeRepository purges a repository which was removed before.

func (*Client) Push 

func (c *Client) Push(ctx context.Context, projectName, repoName, baseRevision string,
	commitMessage *CommitMessage, changes []*Change) (result *PushResult, httpStatusCode int, err error)

Push pushes the specified changes to the repository.

func (*Client) RemoveProject 

func (c *Client) RemoveProject(ctx context.Context, name string) (httpStatusCode int, err error)

RemoveProject removes a project. A removed project can be unremoved using UnremoveProject.

func (*Client) RemoveRepository 

func (c *Client) RemoveRepository(ctx context.Context, projectName, repoName string) (httpStatusCode int, err error)

RemoveRepository removes a repository. A removed repository can be unremoved using UnremoveRepository.

func (*Client) RepoWatcher 

func (c *Client) RepoWatcher(projectName, repoName, pathPattern string) (*Watcher, error)

RepoWatcher returns a Watcher which notifies its listeners when the repository that matched the given pathPattern becomes available or changes. For example: 

watcher := client.RepoWatcher("foo", "bar", "/*.json")

myCh := make(chan interface{})
watcher.Watch(func(revision int, value interface{}) {
    myCh <- value
})
myValue := <-myCh

func (*Client) SecurityEnabled 

func (c *Client) SecurityEnabled() (bool, error)

SecurityEnabled returns whether the security of the server is enabled or not.

func (*Client) SetMetricCollector 

func (c *Client) SetMetricCollector(m *metrics.Metrics)

SetMetricCollector sets metric collector for the client. For example, with Prometheus: 

config := centraldogma.DefaultMetricCollectorConfig("client_name")
metricCollector := centraldogma.GlobalPrometheusMetricCollector(config)
client.SetMetricCollector(metricCollector)

Or Statsd: 

config := centraldogma.DefaultMetricCollectorConfig("client_name")
metricCollector, err := centraldogma.StatsdMetricCollector(config, "127.0.0.1:8125")
if err != nil {
    panic(err)
}
client.SetMetricCollector(metricCollector)

func (*Client) UnremoveProject 

func (c *Client) UnremoveProject(ctx context.Context, name string) (pro *Project, httpStatusCode int, err error)

UnremoveProject unremoves a removed project.

func (*Client) UnremoveRepository 

func (c *Client) UnremoveRepository(
	ctx context.Context, projectName, repoName string) (repo *Repository, httpStatusCode int, err error)

UnremoveRepository unremoves a repository.

func (*Client) WatchFile 

func (c *Client) WatchFile(
	ctx context.Context,
	projectName, repoName string, query *Query,
	timeout time.Duration,
) (result <-chan WatchResult, closer func(), err error)

WatchFile watches on file changes. The watched result will be returned through the returned channel. The API also provides a manual closer to stop watching and release underlying resources. In short, watching will be stopped in case either context is cancelled or closer is called. Manually closing returned channel is unsafe and may cause sending on closed channel error. Usage: 

   query := &Query{Path: "/a.json", Type: Identity}
   ctx := context.Background()
   changes, closer, err := client.WatchFile(ctx, "foo", "bar", query, 2 * time.Minute)
   if err != nil {
		 panic(err)
   }
   defer closer() // stop watching and release underlying resources.

   /* close(changes) */ // manually closing is unsafe, don't do this.

   for {
       select {
         case <-ctx.Done():
            ...

         case change := <-changes:
            // got change
            json.Unmarshal(change.Entry.Content, &expect)
            ...
       }
   }

func (*Client) WatchRepository 

func (c *Client) WatchRepository(
	ctx context.Context,
	projectName, repoName, pathPattern string,
	timeout time.Duration,
) (result <-chan WatchResult, closer func(), err error)

WatchRepository watches on repository changes. The watched result will be returned through the returned channel. The API also provides a manual closer to stop watching and release underlying resources. In short, watching will be stopped in case either context is cancelled or closer is called. Manually closing returned channel is unsafe and may cause sending on closed channel error. Usage: 

   query := &Query{Path: "/a.json", Type: Identity}
   ctx := context.Background()
   changes, closer, err := client.WatchRepository(ctx, "foo", "bar", "/*.json", 2 * time.Minute)
   if err != nil {
		 panic(err)
   }
   defer closer() // stop watching and release underlying resources.

   /* close(changes) */ // manually closing is unsafe, don't do this.

   for {
       select {
         case <-ctx.Done():
            ...

         case change := <-changes:
            // got change
            json.Unmarshal(change.Entry.Content, &expect)
            ...
       }
   }

type Commit 

type Commit struct {
	Revision      int           `json:"revision"`
	Author        Author        `json:"author,omitempty"`
	CommitMessage CommitMessage `json:"commitMessage,omitempty"`
	PushedAt      string        `json:"pushedAt,omitempty"`
}

Commit represents a commit in the repository.

type CommitMessage 

type CommitMessage struct {
	Summary string `json:"summary"`
	Detail  string `json:"detail,omitempty"`
	Markup  string `json:"markup,omitempty"`
}

CommitMessages represents a commit message in the repository.

type Entry 

type Entry struct {
	Path       string       `json:"path"`
	Type       EntryType    `json:"type"` // can be JSON, TEXT or DIRECTORY
	Content    EntryContent `json:"content,omitempty"`
	Revision   int          `json:"revision,omitempty"`
	URL        string       `json:"url,omitempty"`
	ModifiedAt string       `json:"modifiedAt,omitempty"`
}

Entry represents an entry in the repository.

func (*Entry) MarshalJSON 

func (c *Entry) MarshalJSON() ([]byte, error)

func (*Entry) UnmarshalJSON 

func (c *Entry) UnmarshalJSON(b []byte) error

type EntryContent 

type EntryContent []byte

EntryContent represents the content of an entry

func (*EntryContent) UnmarshalJSON 

func (e *EntryContent) UnmarshalJSON(b []byte) error

type EntryType 

type EntryType int
const (
	JSON EntryType = iota + 1
	Text
	Directory
)

func (EntryType) String 

func (c EntryType) String() string

String returns the string value of EntryType

type Project 

type Project struct {
	Name      string `json:"name"`
	Creator   Author `json:"creator,omitempty"`
	URL       string `json:"url,omitempty"`
	CreatedAt string `json:"createdAt,omitempty"`
}

Project represents a project in the Central Dogma server.

type PushResult 

type PushResult struct {
	Revision int    `json:"revision"`
	PushedAt string `json:"pushedAt"`
}

PushResult represents a result of push in the repository.

type Query 

type Query struct {
	Path string
	// QueryType can be "identity" or "json_path". "identity" is used to retrieve the content as it is.
	// "json_path" applies a series of JSON path to the content.
	// See https://github.com/json-path/JsonPath/blob/master/README.md
	Type        QueryType
	Expressions []string
}

Query specifies a query on a file.

type QueryType 

type QueryType int
const (
	Identity QueryType = iota + 1
	JSONPath
)

type Repository 

type Repository struct {
	Name         string `json:"name"`
	Creator      Author `json:"creator,omitempty"`
	HeadRevision int    `json:"headRevision,omitempty"`
	URL          string `json:"url,omitempty"`
	CreatedAt    string `json:"createdAt,omitempty"`
}

Repository represents a repository in the Central Dogma server.

type WatchListener 

type WatchListener func(result WatchResult)

WatchListener listens to Watcher.

type WatchResult 

type WatchResult struct {
	Revision       int   `json:"revision"`
	Entry          Entry `json:"entry,omitempty"`
	HttpStatusCode int
	Err            error
}

WatchResult represents a result from watch operation.

type Watcher 

type Watcher struct {
	// contains filtered or unexported fields
}

Watcher watches the changes of a repository or a file.

func (*Watcher) AwaitInitialValue 

func (w *Watcher) AwaitInitialValue() *WatchResult

AwaitInitialValue awaits for the initial value to be available.

func (*Watcher) AwaitInitialValueWith 

func (w *Watcher) AwaitInitialValueWith(timeout time.Duration) *WatchResult

AwaitInitialValueWith awaits for the initial value to be available during the specified timeout.

func (*Watcher) Close 

func (w *Watcher) Close()

Close stops watching the file specified in the Query or the pathPattern in the repository.

func (*Watcher) Latest 

func (w *Watcher) Latest() *WatchResult

Latest returns the latest Revision and value of WatchFile() or WatchRepository() result.

func (*Watcher) Watch 

func (w *Watcher) Watch(listener WatchListener) error

Watch registers a func that will be invoked when the value of the watched entry becomes available or changes.

Source Files

View all Source files

Directories

Path Synopsis
internal
app/dogma Module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.