parcello

package module
v0.8.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 13, 2019 License: MIT Imports: 17 Imported by: 27

README

Parcello

Documentation License Build Status Coverage Go Report Card

Golang Resource Bundler

Parcel

Overview

Parcello is a simple resource manager for Golang that allows embedding assets like SQL, bash scripts and images. That allows easy release management by deploying just a single binary rather than many files.

Roadmap

Note that we may introduce breaking changes until we reach v1.0.

  • Rename the tool in order not to clash with parcel-bundler
  • Support http.FileSystem
  • Bundle resource as ZIP archive in the end of built Golang binary
  • Support embedded COFF resources (postponed until we accomplish a spike that works on all platforms)

Installation

GitHub
$ go get -u github.com/phogolabs/parcello
$ go install github.com/phogolabs/parcello/cmd/parcello
Homebrew (for Mac OS X)
$ brew tap phogolabs/tap
$ brew install parcello

Usage

You can use the parcello command line interface to bundle the desired resources recursively:

$ parcello -r -d <resource_dir_source> -b <bundle_dir_destination>

However, the best way to use the tool is via go generate. In order to embed all resource in particular directory, you should make it a package that has the following comment:

// Package database contains the database artefacts of GOM as embedded resource
package database

//go:generate parcello -r

When you run:

$ go generate ./...

The tools will create a resource.go file that contains all embedded resource in that directory and its subdirectories as zip archive which is registered in parcello.ResourceManager.

You can read the content in the following way:

// Import the package that includes 'resource.go'
import _ "database"

file, err := parcello.Open("your_sub_directory_name/your_file_name")

The parcello package provides an abstraction of FileSystem interface:

// FileSystem provides primitives to work with the underlying file system
type FileSystem interface {
	// A FileSystem implements access to a collection of named files.
	http.FileSystem
	// Walk walks the file tree rooted at root, calling walkFn for each file or
	// directory in the tree, including root.
	Walk(dir string, fn filepath.WalkFunc) error
	// OpenFile is the generalized open call; most users will use Open
	OpenFile(name string, flag int, perm os.FileMode) (File, error)
}

That is implemented by the following:

That allows easy replacement of the file system with the bundled resources and vice versa.

If you want to work in dev mode, you should set the following environment variables before you start your application:

$ export PARCELLO_DEV_ENABLED=1
$ # if the application resource directory is different than the current working directory
$ export PARCELLO_RESOURCE_DIR=./public

Note that downsides of this resource embedding approach are that your compile time may increase significantly.

If you have such a issue, you can bundle the resource at the end of your binary as zip archive. You can do this via parcello CLI:

$ go build your_binary
$ parcello -r -d <resource_dir_source> -b <path_to_your_binary> -t bundle

Command Line Interface

$ parcello -h

NAME:
   parcello - Golang Resource Bundler and Embedder

USAGE:
   parcello [global options]

VERSION:
   0.8

COMMANDS:
     help, h  Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --bundle-path value, -b value    path to the bundle directory or binary (default: ".")
   --ignore value, -i value         ignore file name
   --include-docs                   include API documentation in generated source code
   --quiet, -q                      disable logging
   --recursive, -r                  embed or bundle the resources recursively
   --resource-dir value, -d value   path to directory (default: ".")
   --resource-type value, -t value  resource type. (supported: bundle, source-code) (default: "source-code")
   --help, -h                       show help
   --version, -v                    print the version

Example

You can check working example.

Contributing

We are open for any contributions. Just fork the project.

logo made by Good Wave CC 3.0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrReadOnly is returned if the file is read-only and write operations are disabled.
	ErrReadOnly = errors.New("File is read-only")
	// ErrWriteOnly is returned if the file is write-only and read operations are disabled.
	ErrWriteOnly = errors.New("File is write-only")
	// ErrIsDirectory is returned if the file under operation is not a regular file but a directory.
	ErrIsDirectory = errors.New("Is directory")
)
View Source
var ErrSkipResource = fmt.Errorf("Skip Resource Error")

ErrSkipResource skips a particular file from processing

View Source
var (
	// Manager keeps track of all resources
	Manager = DefaultManager(osext.Executable)
)

Functions

func AddResource

func AddResource(resource []byte)

AddResource adds resource to the default resource manager Note that the method may panic if the resource not exists

Types

type Bundle

type Bundle struct {
	// Name of the resource
	Name string
	// Count returns the count of files in the bundle
	Count int
	// Body of the resource
	Body []byte
}

Bundle represents a bundled resource

type Bundler

type Bundler struct {
	// Logger prints each step of compression
	Logger io.Writer
	// Compressor compresses the resources
	Compressor Compressor
	// FileSystem represents the underlying file system
	FileSystem FileSystem
}

Bundler bundles the resources to the provided binary

func (*Bundler) Bundle

func (e *Bundler) Bundle(ctx *BundlerContext) error

Bundle bundles the resources to the provided binary

type BundlerContext

type BundlerContext struct {
	// Name of the binary
	Name string
	// FileSystem represents the underlying file system
	FileSystem FileSystem
}

BundlerContext the context of this bundler

type Composer

type Composer interface {
	// Compose composes from an archive
	Compose(bundle *Bundle) error
}

Composer composes the resources

type Compressor

type Compressor interface {
	// Compress compresses given source
	Compress(ctx *CompressorContext) (*Bundle, error)
}

Compressor compresses given resource

type CompressorConfig

type CompressorConfig struct {
	// Logger prints each step of compression
	Logger io.Writer
	// Filename is the name of the compressed bundle
	Filename string
	// IgnorePatterns provides a list of all files that has to be ignored
	IgnorePatterns []string
	// Recurive enables embedding the resources recursively
	Recurive bool
}

CompressorConfig controls how the code generation happens

type CompressorContext

type CompressorContext struct {
	// FileSystem file system that contain the files which will be compressed
	FileSystem FileSystem
	// Offset that should be applied
	Offset int64
}

CompressorContext used for the compression

type Dir

type Dir string

Dir implements FileSystem using the native file system restricted to a specific directory tree.

func (Dir) Add

func (d Dir) Add(resource *Resource) error

Add adds resource bundle to the dir. (noop)

func (Dir) Dir

func (d Dir) Dir(name string) (FileSystemManager, error)

Dir returns a sub-manager for given path

func (Dir) Open

func (d Dir) Open(name string) (ReadOnlyFile, error)

Open opens the named file for reading. If successful, methods on the returned file can be used for reading; the associated file descriptor has mode O_RDONLY. If there is an error, it will be of type *PathError.

func (Dir) OpenFile

func (d Dir) OpenFile(name string, flag int, perm os.FileMode) (File, error)

OpenFile is the generalized open call; most users will use Open

func (Dir) Walk

func (d Dir) Walk(dir string, fn filepath.WalkFunc) error

Walk walks the file tree rooted at root, calling walkFn for each file or directory in the tree, including root.

type Embedder

type Embedder struct {
	// Logger prints each step of compression
	Logger io.Writer
	// Composer composes the resources
	Composer Composer
	// Compressor compresses the resources
	Compressor Compressor
	// FileSystem represents the underlying file system
	FileSystem FileSystem
}

Embedder embeds the resources to the provided package

func (*Embedder) Embed

func (e *Embedder) Embed() error

Embed embeds the resources to the provided package

type ExecutableFunc

type ExecutableFunc func() (string, error)

ExecutableFunc returns the executable path

type File

type File interface {

	// A File is returned by a FileSystem's Open method and can be
	ReadOnlyFile
	// Writer is the interface that wraps the basic Write method.
	io.Writer
	// ReaderAt reads at specific position
	io.ReaderAt
}

File is the bundle file

func Open

func Open(name string) (File, error)

Open opens an embedded resource for read

type FileSystem

type FileSystem interface {
	// A FileSystem implements access to a collection of named files.
	http.FileSystem
	// Walk walks the file tree rooted at root, calling walkFn for each file or
	// directory in the tree, including root.
	Walk(dir string, fn filepath.WalkFunc) error
	// OpenFile is the generalized open call; most users will use Open
	OpenFile(name string, flag int, perm os.FileMode) (File, error)
}

FileSystem provides primitives to work with the underlying file system

type FileSystemManager

type FileSystemManager interface {
	// FileSystem is the underlying file system
	FileSystem
	// Dir returns a sub-file-system
	Dir(name string) (FileSystemManager, error)
	// Add resource bundle to the manager
	Add(resource *Resource) error
}

FileSystemManager is a file system that can create sub-file-systems

func DefaultManager

func DefaultManager(executable ExecutableFunc) FileSystemManager

DefaultManager creates a FileSystemManager based on whether dev mode is enabled

func ManagerAt

func ManagerAt(path string) FileSystemManager

ManagerAt returns manager at given path

type Generator

type Generator struct {
	// FileSystem represents the underlying file system
	FileSystem FileSystem
	// Config controls how the code generation happens
	Config *GeneratorConfig
}

Generator generates an embedable resource

func (*Generator) Compose

func (g *Generator) Compose(bundle *Bundle) error

Compose generates an embedable resource for given directory

type GeneratorConfig

type GeneratorConfig struct {
	// Package determines the name of the package
	Package string
	// InlcudeDocs determines whether to include documentation
	InlcudeDocs bool
}

GeneratorConfig controls how the code generation happens

type Node

type Node struct {
	// Name of the node
	Name string
	// IsDir returns true if the node is directory
	IsDir bool
	// Mutext keeps the node thread safe
	Mutex *sync.RWMutex
	// ModTime returns the last modified time
	ModTime time.Time
	// Content of the node
	Content *[]byte
	// Children of the node
	Children []*Node
}

Node represents a node in resource tree

type ReadOnlyFile

type ReadOnlyFile = http.File

ReadOnlyFile is the bundle file

type Resource

type Resource struct {
	// Body of the resource
	Body io.ReaderAt
	// Size of the body
	Size int64
}

Resource represents a resource

func BinaryResource

func BinaryResource(data []byte) *Resource

BinaryResource creates a binary resource

type ResourceFile

type ResourceFile struct {
	*memfs.MemFile
	// contains filtered or unexported fields
}

ResourceFile represents a *bytes.Buffer that can be closed

func NewResourceFile

func NewResourceFile(node *Node) *ResourceFile

NewResourceFile creates a new Buffer

func (*ResourceFile) Readdir

func (b *ResourceFile) Readdir(n int) ([]os.FileInfo, error)

Readdir reads the contents of the directory associated with file and returns a slice of up to n FileInfo values, as would be returned

func (*ResourceFile) Stat

func (b *ResourceFile) Stat() (os.FileInfo, error)

Stat returns the FileInfo structure describing file. If there is an error, it will be of type *PathError.

type ResourceFileInfo

type ResourceFileInfo struct {
	Node *Node
}

ResourceFileInfo represents a hierarchy node in the resource manager

func (*ResourceFileInfo) IsDir

func (n *ResourceFileInfo) IsDir() bool

IsDir returns true if the node is directory

func (*ResourceFileInfo) ModTime

func (n *ResourceFileInfo) ModTime() time.Time

ModTime returns the modification time

func (*ResourceFileInfo) Mode

func (n *ResourceFileInfo) Mode() os.FileMode

Mode returns the file mode bits

func (*ResourceFileInfo) Name

func (n *ResourceFileInfo) Name() string

Name returns the base name of the file

func (*ResourceFileInfo) Size

func (n *ResourceFileInfo) Size() int64

Size returns the length in bytes for regular files

func (*ResourceFileInfo) Sys

func (n *ResourceFileInfo) Sys() interface{}

Sys returns the underlying data source

type ResourceManager

type ResourceManager struct {

	// NewReader creates a new ZIP Reader
	NewReader func(io.ReaderAt, int64) (*zip.Reader, error)
	// contains filtered or unexported fields
}

ResourceManager represents a virtual in memory file system

func NewResourceManager

func NewResourceManager(cfg *ResourceManagerConfig) (*ResourceManager, error)

NewResourceManager creates a new manager

func (*ResourceManager) Add

func (m *ResourceManager) Add(resource *Resource) error

Add adds resource to the manager

func (*ResourceManager) Dir

Dir returns a sub-manager for given path

func (*ResourceManager) Open

func (m *ResourceManager) Open(name string) (ReadOnlyFile, error)

Open opens an embedded resource for read

func (*ResourceManager) OpenFile

func (m *ResourceManager) OpenFile(name string, flag int, perm os.FileMode) (File, error)

OpenFile is the generalized open call; most users will use Open

func (*ResourceManager) Walk

func (m *ResourceManager) Walk(dir string, fn filepath.WalkFunc) error

Walk walks the file tree rooted at root, calling walkFn for each file or directory in the tree, including root.

type ResourceManagerConfig

type ResourceManagerConfig struct {
	// Path to the archive
	Path string
	// FileSystem that stores the archive
	FileSystem FileSystem
}

ResourceManagerConfig represents the configuration for Resource Manager

type ZipCompressor

type ZipCompressor struct {
	// Config controls how the compression is made
	Config *CompressorConfig
}

ZipCompressor compresses content as GZip tarball

func (*ZipCompressor) Compress

func (e *ZipCompressor) Compress(ctx *CompressorContext) (*Bundle, error)

Compress compresses given source in tar.gz

Directories

Path Synopsis
cmd
parcello
Command Line Interface of Embedo.
Command Line Interface of Embedo.
public
Package public contains embedded resources
Package public contains embedded resources
This file was generated by counterfeiter This file was generated by counterfeiter This file was generated by counterfeiter This file was generated by counterfeiter This file was generated by counterfeiter
This file was generated by counterfeiter This file was generated by counterfeiter This file was generated by counterfeiter This file was generated by counterfeiter This file was generated by counterfeiter

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL