libsquash

package module
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 27, 2015 License: MIT Imports: 15 Imported by: 0

README

libsquash

Library

GoDoc

This is based on jwilder/docker-squash, but the squashing functionality is extracted into a package named libsquash

libsquash is different from docker-squash in that libsquash...

  1. does not write any layer data to disk
  2. does not require sudo
  3. does not shell out to or require the installation of tar

Other information:

License

MIT

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrorMultipleBranchesSameParent is an edge case that libsquash does not currently handle
	ErrorMultipleBranchesSameParent = errors.New("this image is a full " +
		"repository export w/ multiple images in it. Please generate the export " +
		"from a specific image ID or tag.",
	)

	// ErrorNoFROM is returned when o root layer can be found
	ErrorNoFROM = errors.New("no root layer found")
)
View Source 
var (
	// ErrorNoLast is returned if it cannot be determined, by traversing the
	// layers, what the last layer is. This should probably never happen, so if
	// this error does occur, it's probably the result of the image tarball
	// being malformed in some way
	ErrorNoLast = errors.New("unable to determine last layer in image")
)
View Source 
var LayerFileIgnoreRegex = regexp.MustCompile(`^\.$|^\.\.$|^\.\/$`)

LayerFileIgnoreRegex is the regex for identifying files that should be of type Ignore

View Source 
var Verbose bool

Verbose will print out debugging info when set to true. Should be set manually in code only for debugging purposes

Functions

func Squash 

func Squash(instream io.Reader, outstream io.Writer, imageIDOut io.Writer) error

Squash squashes a docker image where instream is an io.Reader for the image tarball, outstream is an io.Writer to which the squashed image tarball will be written, and imageIDOut is an io.Writer to which the id of the squashed image will be written

The steps are as follows:

1. Go through stream, tee'ing it to a tempfile, get layer configs and layer->file lists

2. Using the metadata, go through the tar stream again (from the tempfile), build the squash layer, build the final image tar, and write it to our output stream

3. (as a cleanup step, write the id of the final layer, which the daemon will use as the image id)

Types

type Config 

type Config struct {
	AttachStderr    bool
	AttachStdin     bool
	AttachStdout    bool
	Cmd             []string
	CPUShares       int64    `json:"CpuShares"`
	DNS             []string `json:"Dns"` // For Docker API v1.9 and below only
	Domainname      string
	Entrypoint      []string
	Env             []string
	ExposedPorts    map[Port]struct{}
	Hostname        string
	Image           string
	Memory          int64
	MemorySwap      int64
	NetworkDisabled bool
	OnBuild         []string
	OpenStdin       bool
	PortSpecs       []string
	StdinOnce       bool
	Tty             bool
	User            string
	Volumes         map[string]struct{}
	VolumesFrom     string
	WorkingDir      string
}

Config is a sub config of LayerConfig that represents the cumulative config

type ContainerConfig 

type ContainerConfig struct {
	AttachStderr    bool
	AttachStdin     bool
	AttachStdout    bool
	Cmd             []string
	CPUShares       int64    `json:"CpuShares"`
	DNS             []string `json:"Dns"`
	Domainname      string
	Entrypoint      []string
	Env             []string
	Hostname        string
	Image           string
	Memory          int64
	MemorySwap      int64
	NetworkDisabled bool
	OnBuild         []string
	OpenStdin       bool
	PortSpecs       []string
	StdinOnce       bool
	Tty             bool
	User            string
	Volumes         map[string]struct{}
	VolumesFrom     string
}

ContainerConfig is a sub config of LayerConfig that represents the config for the given layer only

type Export added in v0.2.0 

type Export struct {
	Layers       map[string]*Layer
	Repositories map[string]*tagInfo
	// contains filtered or unexported fields
}

An Export contains the layers of the image as well as various forms of metadata. An export "ingests" this metadata and then uses it to determine how to compose the squashed image

func NewExport added in v0.2.0 

func NewExport() *Export

NewExport returns a fully initialized *Export

func (*Export) ChildOf added in v0.2.0 

func (e *Export) ChildOf(parent string) *Layer

ChildOf returns the child layer or nil of the parent

func (*Export) FirstSquash added in v0.2.0 

func (e *Export) FirstSquash() *Layer

FirstSquash finds the first layer marked with the token #(squash)

func (*Export) GetByID added in v0.2.0 

func (e *Export) GetByID(idPrefix string) (*Layer, error)

GetByID returns an exportedImaged with a prefix matching ID. An error is returned multiple exportedImages matched.

func (*Export) IngestImageMetadata added in v0.2.0 

func (e *Export) IngestImageMetadata(tarstream io.Reader) error

IngestImageMetadata walks the files in the "tarstream" tarball and checks for several things:

1. check for the "repositories" file - if present, determine if it cancels the squash

2. check for each "json" file, read it into a data structure

3. check for each "layer.tar" file, noting which filesystem files are present or deleted via aufs-style whiteout files (.wh..wh.<file>)

To determine what files should come from each layer.tar (which was the last to modify that file), we build a list like so: 

fileToLayers:
------------
file1: []layer
file2: []layer

After completing the processing of the tarball, this function calls another that translates the list as follows (note: a uuid is the unique identifier for a layer): 

layerToFiles:
------------
uuid1: file1 -> true, file3 -> true
uuid2: file2 -> true

The next time we we iterate over the image tarball, we only have one layer.tar at a time. We need to know, based on the uuid of that layer.tar, which files to pull from it. That requires the layerToFiles structure.

func (*Export) InsertLayer added in v0.2.0 

func (e *Export) InsertLayer(parent string) (*Layer, error)

InsertLayer inserts a new layer after "parent" with the token #(squash) command. Return the new layer

func (*Export) Last added in v0.2.0 

func (e *Export) Last() *Layer

Last returns the layer found last in the list

func (*Export) RebuildImage added in v0.2.0 

func (e *Export) RebuildImage(squashLayer *Layer, outstream io.Writer, squashLayerFile *os.File) (imageID string, err error)

RebuildImage builds the final image tarball using the following process:

1. Open up a new tar stream that writes to the output stream

2. For each layer that should be in the final tarball (based on the current LayerConfig data), write the four rquired files

  1. <uuid>/ -> directory, no file contents
  2. <uuid>/VERSION -> contents always the same
  3. <uuid>/json -> the LayerConfig
  4. <uuid>/layer.tar -> the tarball for the given layer a. if this is the #(squash) layer, it should contain all of the image's data b. if it is any other layer, it will contain only 2x 512 byte blocks of \x00 (this is the way to represent an empty tarball)

func (*Export) ReplaceLayer added in v0.2.0 

func (e *Export) ReplaceLayer(orig *Layer) error

ReplaceLayer refreshes layer "orig" by updating the Created timestamp and ID and wiring it in correctly

func (*Export) RewriteChildren added in v0.2.0 

func (e *Export) RewriteChildren(from *Layer, squashID string) error

RewriteChildren should only be called internally by SquashLayers.

RewriteChildren contains the core logic about how to modify all layers that are inherited by the squash layer. The logic is as follows:

  • if the layer modifies the filesystem (is a RUN or a #(nop) ADD)
  • remove it
  • that layer will effectively be merged into the squash layer
  • history from these layers will be lost
  • if the layer does NOT modify the filesystem (is any other command type)
  • keep it, but give it a new ID and timestamp
  • the history of that layer and its changes (e.g. new env vars, new workdir, etc.) will be preserved

func (*Export) Root added in v0.2.0 

func (e *Export) Root() *Layer

Root returns the top layer in the export

func (*Export) SquashLayers added in v0.2.0 

func (e *Export) SquashLayers(into, from *Layer, tarstream io.Reader, outstream io.Writer) (imageID string, err error)

SquashLayers produces the #(squash) layer from the contents in the tarball, rewrites the subsequent layers, using e.RewriteChildren, and then rewrites the final image tar by calling e.RebuildImage

type Layer added in v0.2.0 

type Layer struct {
	// LayerConfig is the layer config json
	LayerConfig *LayerConfig

	// DirHeader is the header for <uuid>/
	DirHeader *tar.Header

	// VersionHeader is the header for <uuid>/VERSION
	VersionHeader *tar.Header

	// JSONHeader is the header for <uuid>/json
	JSONHeader *tar.Header

	// LayerTarHeader is the header for <uuid>/layer.tar
	LayerTarHeader *tar.Header
}

Layer represents a layer inside the docker image. A Layer consists of a struct representation of the json config and placeholders for the tar headers for the various required files

func (*Layer) Clone added in v0.2.0 

func (l *Layer) Clone() *Layer

Clone returns a copy of layer l

func (*Layer) Cmd added in v0.2.0 

func (l *Layer) Cmd() string

Cmd is a convenience function that prints out the command for layer "l". The command is shortened to be no more than 60 characters

type LayerConfig 

type LayerConfig struct {
	ID                string           `json:"id"`
	Parent            string           `json:"parent,omitempty"`
	Comment           string           `json:"comment"`
	Created           time.Time        `json:"created"`
	V1ContainerConfig *ContainerConfig `json:"ContainerConfig,omitempty"`  // Docker 1.0.0, 1.0.1
	V2ContainerConfig *ContainerConfig `json:"container_config,omitempty"` // All other versions
	Container         string           `json:"container"`
	Config            *Config          `json:"config,omitempty"`
	DockerVersion     string           `json:"docker_version"`
	Architecture      string           `json:"architecture"`
}

LayerConfig is the top-level json config for ar docker layer

func NewLayerConfig added in v0.2.0 

func NewLayerConfig(id, parent, comment string) *LayerConfig

NewLayerConfig produces an empty LayerConfig, initialized with a few sensible defaults

func (*LayerConfig) ContainerConfig 

func (l *LayerConfig) ContainerConfig() *ContainerConfig

ContainerConfig normalizes the V1 and V2ContainerConfig and returns the correct version

type LayerFileType added in v0.2.0 

type LayerFileType uint8

LayerFileType is a type for identifying a file in a tarball as specific to how this library parses a docker image 

const (
	// Ignore is for ".", "..", and "./" (identified by LayerFileIgnoreRegex)
	Ignore LayerFileType = iota

	// Repositories is for "repositories"
	Repositories

	// Directory is for "<uuid>/"
	Directory

	// JSON is for "<uuid>/json"
	JSON

	// LayerTar is for "<uuid>/layer.tar"
	LayerTar

	// Version is for "<uuid>/VERSION"
	Version

	// Unknown is for files that cannot be otherwise identified
	Unknown
)

func ParseType added in v0.2.0 

func ParseType(t *tarball.TarFile) LayerFileType

ParseType returns the LayerFileType of the given tar file

type Port 

type Port string

Port is a type for representing docker port mappings

func (Port) Port 

func (p Port) Port() string

Port returns the number of the port.

func (Port) Proto 

func (p Port) Proto() string

Proto returns the name of the protocol.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.