charmstore

package module
v0.0.0-...-1aca525 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 17, 2014 License: LGPL-3.0 Imports: 24 Imported by: 0

README

juju/charmstore

Store and publish Juju charms.

Installation

To start using the charm store, run the following:

go get github.com/juju/charmstore

Go dependencies

The project uses godeps (https://launchpad.net/godeps) to manage Go dependencies. After installing the application, you can update the dependencies to the revision specified in the dependencies.tsv file with the following:

make deps

Use make create-deps to update the dependencies file.

Devlopment environment

A couple of system packages are required in order to set up a charm store development environment. To install them, run the following:

make sysdeps

At this point, from the root of this branch, run the command::

make install

The command above builds and installs the charm store binaries, and places them in $GOPATH/bin. This is the list of the installed commands:

  • charmload: populate the database with charms from Launchpad;
  • charmd: start the charm store server;
  • charm-admin: manage published charms.

A description of each command can be found below.

Testing

Run make check to test the application. Run make help to display help about all the available make targets.

Populate the charms database

The charm store creates a MongoDB database named "juju" and stores info about charms in the MongoDB "juju.charms" collection. Also charm files are stored in a GridFS named "juju.charmfs".

To populate the database with the charms published in Launchpad, run the following command:

charmload cmd/charmd/config.yaml

Note: the operation takes a large amount of time and disk space to complete: at the time of this writing it takes ~2:30h and ~4GB to store ~1050 charms, but this can vary significantly based on your machine/connection speed. The process can be stopped by typing ^C. To check the imported charm count, you can run the following:

mongo --eval "db.getSiblingDB('juju').charms.count()"

Charmstore server

Once the charms database is fully populated, it is possible to interact with charm data using the charm store server. It can be started with the following command:

charmd cmd/charmd/config.yaml

The same result can be achieved more easily by running make server.

At this point the server starts listening on port 8080 (as specified in the config YAML file). The server exposes the following API:

/charm-info

A GET call to /charm-info returns info about one or more charms, including its canonical URL, revision, SHA256 checksum and VCS revision digest. The returned info is in JSON format. For instance a request to /charm-info?charms=cs:trusty/juju-gui returns the following response:

{"cs:trusty/juju-gui": {
    "canonical-url": "cs:trusty/juju-gui",
    "revision": 3,
    "sha256": "a15c77f3f92a0fb7b61e9...",
    "digest": jeff.pihach@canonical.com-20140612210347-6cc9su1jqjkhbi84"
}}
/charm-event:

A GET call to /charm-event returns info about an event occurred in the life of the specified charm(s). Currently two types of events are logged: "published" (a charm has been published and it's available in the store) and "publish-error" (an error occurred while importing the charm). E.g. a call to /charm-event?charms=cs:trusty/juju-gui generates the following JSON response:

{"cs:trusty/juju-gui": {
    "kind": "published",
    "revision": 3,
    "digest": "jeff.pihach@canonicalcom-20140612210347-6cc9su1jqjkhbi84",
    "time": "2014-06-16T14:41:19Z"
}}
/charm/

The charm API provides the ability to download a charm as a Zip archive, given the charm identifier. For instance, it is possible to download the Juju GUI charm by performing a GET call to /charm/trusty/juju-gui-42. Both the revision and OS series can be omitted, e.g. /charm/juju-gui will download the last revision of the Juju GUI charm with support to the more recent Ubuntu LTS series.

/stats/counter/

Stats can be retrieved by calling /stats/counter/{key} where key is a query that specifies the counter stats to calculate and return.

For instance, a call to /stats/counter/charm-bundle:* returns the number of times a charm has been downloaded from the store. To get the same value for a specific charm, it is possible to filter the results by passing the charm series and name, e.g. /stats/counter/charm-bundle:trusty:juju-gui.

The results can be grouped by specifying the by query (possible values are day and week), and time delimited using the start and stop queries.

It is also possible to list the results by passing list=1. For example, a GET call to /stats/counter/charm-bundle:trusty:*?by=day&list=1 returns an aggregated count of trusty charms downloads, grouped by charm and day, similar to the following:

charm-bundle:trusty:juju-gui  2014-06-17  5
charm-bundle:trusty:mysql     2014-06-17  1

Manage published charms

The charm-admin command is used to manage the store contents. Currently the only implemented sub-command is delete-charm, which removes a charm from the store, e.g.:

charm-admin delete-charm --config cmd/charmd/config.yaml --url trusty/mysql

Run charm-admin help for the complete command's help.

Documentation

Overview

The charmstore package is capable of storing and updating charms in a MongoDB database, as well as maintaining further information about them such as the VCS revision the charm was loaded from and the URLs for the charms.

Index

Constants

View Source 
const DefaultSeries = "precise"
View Source 
const (
	UpdateTimeout = 600e9
)

Variables

View Source 
var (
	ErrUpdateConflict  = errors.New("charm update in progress")
	ErrRedundantUpdate = errors.New("charm is up-to-date")

	// Note that this error message is part of the API, since it's sent
	// both in charm-info and charm-event responses as errors indicating
	// that the given charm or charm event wasn't found.
	ErrNotFound = errors.New("entry not found")
)

Functions

func PublishBazaarBranch 

func PublishBazaarBranch(store *Store, urls []*charm.URL, burl string, digest string) error

PublishBazaarBranch checks out the Bazaar branch from burl and publishes its latest revision at urls in the given store. The digest parameter must be the most recent known Bazaar revision id for the branch tip. If publishing this specific digest for these URLs has been attempted already, the publishing procedure may abort early. The published digest is the Bazaar revision id of the checked out branch's tip, though, which may differ from the digest parameter.

func PublishCharmsDistro 

func PublishCharmsDistro(store *Store, apiBase lpad.APIBase) error

PublishCharmsDistro publishes all branch tips found in the /charms distribution in Launchpad onto store under the "cs:" scheme. apiBase specifies the Launchpad base API URL, such as lpad.Production or lpad.Staging. Errors found while processing one or more branches are all returned as a PublishBranchErrors value.

Types

type CharmDir 

type CharmDir interface {
	Meta() *charm.Meta
	Config() *charm.Config
	Actions() *charm.Actions
	SetRevision(revision int)
	BundleTo(w io.Writer) error
}

CharmDir matches the part of the interface of *charm.Dir that is necessary to publish a charm. Using this interface rather than *charm.Dir directly makes testing some aspects of the store possible.

type CharmEvent 

type CharmEvent struct {
	Kind     CharmEventKind
	Digest   string
	Revision int
	URLs     []*charm.URL
	Errors   []string `bson:",omitempty"`
	Warnings []string `bson:",omitempty"`
	Time     time.Time
}

CharmEvent is a record for an event relating to one or more charm URLs.

type CharmEventKind 

type CharmEventKind int
const (
	EventPublished CharmEventKind = iota + 1
	EventPublishError

	EventKindCount
)

func (CharmEventKind) String 

func (k CharmEventKind) String() string

type CharmInfo 

type CharmInfo struct {
	// contains filtered or unexported fields
}

func (*CharmInfo) Actions 

func (ci *CharmInfo) Actions() *charm.Actions

Actions returns the charm.ACtions details for the stored charm.

func (*CharmInfo) BundleSha256 

func (ci *CharmInfo) BundleSha256() string

BundleSha256 returns the sha256 checksum for the stored charm bundle.

func (*CharmInfo) BundleSize 

func (ci *CharmInfo) BundleSize() int64

BundleSize returns the size for the stored charm bundle.

func (*CharmInfo) Config 

func (ci *CharmInfo) Config() *charm.Config

Config returns the charm.Config details for the stored charm.

func (*CharmInfo) Digest 

func (ci *CharmInfo) Digest() string

Digest returns the unique identifier that represents the charm data imported. This is typically set to the VCS revision digest.

func (*CharmInfo) Meta 

func (ci *CharmInfo) Meta() *charm.Meta

Meta returns the charm.Meta details for the stored charm.

func (*CharmInfo) Revision 

func (ci *CharmInfo) Revision() int

Revision returns the store charm's revision.

type CharmPublisher 

type CharmPublisher struct {
	// contains filtered or unexported fields
}

A CharmPublisher is responsible for importing a charm dir onto the store.

func (*CharmPublisher) Publish 

func (p *CharmPublisher) Publish(charm CharmDir) error

Publish bundles charm and writes it to the store. The written charm bundle will have its revision set to the result of Revision. Publish must be called only once for a CharmPublisher.

func (*CharmPublisher) Revision 

func (p *CharmPublisher) Revision() int

Revision returns the revision that will be assigned to the published charm.

type Config 

type Config struct {
	MongoURL string `yaml:"mongo-url"`
	APIAddr  string `yaml:"api-addr"`
}

func ReadConfig 

func ReadConfig(path string) (*Config, error)

type Counter 

type Counter struct {
	Key    []string
	Prefix bool
	Count  int64
	Time   time.Time
}

type CounterRequest 

type CounterRequest struct {
	// Key and Prefix determine the counter keys to match.
	// If Prefix is false, Key must match exactly. Otherwise, counters
	// must begin with Key and have at least one more key token.
	Key    []string
	Prefix bool

	// If List is true, matching counters are aggregated under their
	// prefixes instead of being returned as a single overall sum.
	//
	// For example, given the following counts:
	//
	//   {"a", "b"}: 1,
	//   {"a", "c"}: 3
	//   {"a", "c", "d"}: 5
	//   {"a", "c", "e"}: 7
	//
	// and assuming that Prefix is true, the following keys will
	// present the respective results if List is true:
	//
	//        {"a"} => {{"a", "b"}, 1, false},
	//                 {{"a", "c"}, 3, false},
	//                 {{"a", "c"}, 12, true}
	//   {"a", "c"} => {{"a", "c", "d"}, 3, false},
	//                 {{"a", "c", "e"}, 5, false}
	//
	// If List is false, the same key prefixes will present:
	//
	//        {"a"} => {{"a"}, 16, true}
	//   {"a", "c"} => {{"a", "c"}, 12, false}
	//
	List bool

	// By defines the period covered by each aggregated data point.
	// If unspecified, it defaults to ByAll, which aggregates all
	// matching data points in a single entry.
	By CounterRequestBy

	// Start, if provided, changes the query so that only data points
	// ocurring at the given time or afterwards are considered.
	Start time.Time

	// Stop, if provided, changes the query so that only data points
	// ocurring at the given time or before are considered.
	Stop time.Time
}

CounterRequest represents a request to aggregate counter values.

type CounterRequestBy 

type CounterRequestBy int
const (
	ByAll CounterRequestBy = iota
	ByDay
	ByWeek
)

type PublishBranchError 

type PublishBranchError struct {
	URL string
	Err error
}

type PublishBranchErrors 

type PublishBranchErrors []PublishBranchError

func (PublishBranchErrors) Error 

func (errs PublishBranchErrors) Error() string

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server is an http.Handler that serves the HTTP API of juju so that juju clients can retrieve published charms.

func NewServer 

func NewServer(store *Store) (*Server, error)

NewServer returns a new *Server using store.

func (*Server) ServeHTTP 

func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP serves an http request. This method turns *Server into an http.Handler.

type Store 

type Store struct {
	// contains filtered or unexported fields
}

Store holds a connection to a charm store.

func Open 

func Open(mongoAddr string) (store *Store, err error)

Open creates a new session with the store. It connects to the MongoDB server at the given address (as expected by the Mongo function in the labix.org/v2/mgo package).

func (*Store) CharmEvent 

func (s *Store) CharmEvent(url *charm.URL, digest string) (*CharmEvent, error)

CharmEvent returns the most recent event associated with url and digest. If the specified event isn't found the error ErrUnknownChange will be returned. If digest is empty, any digest will match.

func (*Store) CharmInfo 

func (s *Store) CharmInfo(url *charm.URL) (*CharmInfo, error)

CharmInfo retrieves the CharmInfo value for the charm at url.

func (*Store) CharmPublisher 

func (s *Store) CharmPublisher(urls []*charm.URL, digest string) (p *CharmPublisher, err error)

CharmPublisher returns a new CharmPublisher for importing a charm that will be made available in the store at all of the provided URLs. The digest parameter must contain the unique identifier that represents the charm data being imported (e.g. the VCS revision sha1). ErrRedundantUpdate is returned if all of the provided urls are already associated to that digest.

func (*Store) Close 

func (s *Store) Close()

Close terminates the connection with the store.

func (*Store) Counters 

func (s *Store) Counters(req *CounterRequest) ([]Counter, error)

Counters aggregates and returns counter values according to the provided request.

func (*Store) DeleteCharm 

func (s *Store) DeleteCharm(url *charm.URL) ([]*CharmInfo, error)

DeleteCharm deletes the charms matching url. If no revision is specified, all revisions of the charm are deleted.

func (*Store) IncCounter 

func (s *Store) IncCounter(key []string) error

IncCounter increases by one the counter associated with the composed key.

func (*Store) LockUpdates 

func (s *Store) LockUpdates(urls []*charm.URL) (l *UpdateLock, err error)

LockUpdates acquires a server-side lock for updating a single charm that is supposed to be made available in all of the provided urls. If the lock can't be acquired in any of the urls, an error will be immediately returned. In the usual case, any locking done is undone when an error happens, or when l.Unlock is called. If something else goes wrong, the locks will also expire after the period defined in UpdateTimeout.

func (*Store) LogCharmEvent 

func (s *Store) LogCharmEvent(event *CharmEvent) (err error)

LogCharmEvent records an event related to one or more charm URLs.

func (*Store) OpenCharm 

func (s *Store) OpenCharm(url *charm.URL) (info *CharmInfo, rc io.ReadCloser, err error)

OpenCharm opens for reading via rc the charm currently available at url. rc must be closed after dealing with it or resources will leak.

func (*Store) Series 

func (s *Store) Series(ref charm.Reference) ([]string, error)

Series returns all the series available for a charm reference, in descending order of preference. LTS releases preferred over non-LTS

type UpdateLock 

type UpdateLock struct {
	// contains filtered or unexported fields
}

UpdateLock represents an acquired update lock over a set of charm URLs.

func (*UpdateLock) Unlock 

func (l *UpdateLock) Unlock()

Unlock removes the previously acquired server-side lock that prevents other processes from attempting to update a set of charm URLs.

Source Files

View all Source files

Directories

Path Synopsis
cmd
charm-admin
charmd
charmload

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.