tpkg

package
v0.0.0-...-85991d7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 13, 2023 License: LGPL-2.1 Imports: 19 Imported by: 3

Documentation

Overview

Package tpkg provides functions to manage Toit packages.

Key concepts:

  • Package: a reusable component.
  • Package description (Desc): description of a package that is distributed independently of a package. The description serves to find a package and all of its dependencies. Package resolution (deciding which versions...) is done with descriptions alone, without downloading any sources.
  • Package specification (Spec): a specification shipped with a package that contains relevant information. Fundamentally, a specification is not necessary, as only a description is necessary to make packages usable. In practice, we use specifications to automatically build descriptions (together with additional information that can often be extracted automatically). For example, dependencies should be written into a specification file. We also use specification files to recognize folders as packages. If such a file exists, we know that the folder should be treated as package.
  • Registry: a place where descriptions of available packages can be found. There can be multiple registries, with different properties. For example, some registries can be public, while others can be private.
  • Lock file: the result of resolving all versions of an application. Contains an easy way for the compiler to find concrete sources for each package the application transitively uses. One can think of the entries in the lock file as a mapping from package-name (used as dependency) to an absolute path. However, since lock files are often shared (and checked in), the mapping is a bit more complicated and usually doesn't have absolute paths.

Index

Constants

View Source
const (
	// ProjectPackagesPath provides the path, relative to the project's root,
	// into which packages should be downloaded.
	ProjectPackagesPath = ".packages"

	DefaultSpecName     = "package.yaml"
	DefaultLockFileName = "package.lock"

	// The directory inside registries, where descriptions should be stored.
	PackageDescriptionDir = "packages"

	// The default filename for description files.
	DescriptionFileName = "desc.yaml"
)
View Source
const TestGitPathHost = "path.toit.local"

Variables

This section is empty.

Functions

func DownloadGit

func DownloadGit(ctx context.Context, o DownloadGitOptions) (string, error)

func InitDirectory

func InitDirectory(projectRoot string, ui UI) error

InitDirectory initializes the project root as the root for a package or application. If no root is given, initializes the current directory instead.

func IsErrAlreadyReported

func IsErrAlreadyReported(e error) bool

IsErrAlreadyReported returns whether 'e' is the ErrAlreadyReported error.

func URLVersionToRelPath

func URLVersionToRelPath(url string, version string) string

Types

type AllowLocalDepsFlag

type AllowLocalDepsFlag int
const (
	AllowLocalDeps AllowLocalDepsFlag = iota
	ReportLocalDeps
	DisallowLocalDeps
)

type Cache

type Cache struct {
	// contains filtered or unexported fields
}

Cache handles all package-Cache related functionality. It keeps track of where the caches are, and how to compute paths for packages.

func NewCache

func NewCache(registryPath string, ui UI, options ...CacheOption) Cache

NewCache creates a new package cache and uses the registryPath as the locations where git registries will be installed can be found.

func (Cache) CreatePackagesCacheDir

func (c Cache) CreatePackagesCacheDir(projectRootPath string, ui UI) error

CreatePackagesCacheDir creates the package cache dir. If the directory doesn't exist yet, creates it, and writes a README explaining what the directory is for, and what is allowed to be done.

func (Cache) FindPkg

func (c Cache) FindPkg(rootPath string, url string, version string) (string, error)

FindPkg searches for the path of 'url'-'version' in the cache. If it's not found returns "".

func (Cache) FindRegistry

func (c Cache) FindRegistry(url string) (string, error)

FindRegistry searches for the path of the registry with the given url in the cache. If it's not found returns "".

func (Cache) PkgInstallPath

func (c Cache) PkgInstallPath(projectRootPath string) string

func (Cache) PreferredPkgPath

func (c Cache) PreferredPkgPath(projectRootPath string, url string, version string) string

PreferredPkgPath returns the preferred path for the package url-version.

func (Cache) PreferredRegistryPath

func (c Cache) PreferredRegistryPath(url string) string

PreferredRegistryPath returns the preferred path for the given registry url.

func (Cache) SpecPathFor

func (c Cache) SpecPathFor(projectRootPath string, url string, version string) (string, error)

Returns the path to the specification of the package url-version. If the package is not in the cache returns "".

type CacheOption

type CacheOption interface {
	// contains filtered or unexported methods
}

CacheOption defines the optional parameters for NewCache.

func WithPkgCachePath

func WithPkgCachePath(paths ...string) CacheOption

WithPkgCachePath sets the locations where packages can be found.

func WithPkgInstallPath

func WithPkgInstallPath(path string) CacheOption

WithPkgInstallPath sets the locations where git registries can be found.

func WithRegistryCachePath

func WithRegistryCachePath(paths ...string) CacheOption

WithRegistryCachePath sets the locations where git registries can be found.

type DependencyMap

type DependencyMap map[string]SpecPackage

DependencyMap is a map from prefix to package.

type Desc

type Desc struct {
	Name        string `yaml:"name" json:"name"`
	Description string `yaml:"description,omitempty" json:"description"`

	License string `yaml:"license,omitempty" json:"license"`
	// We might want to add a url-kind in the future, but currently we
	// don't need it, and just assume that every url is a git location.
	URL         string          `yaml:"url" json:"url"`
	Version     string          `yaml:"version" json:"version"`
	Environment DescEnvironment `yaml:"environment,omitempty" json:"environment,omitempty"`

	// The git-hash of the package.
	Hash string `yaml:"hash,omitempty" json:"hash"`

	Deps []descPackage `yaml:"dependencies,omitempty" json:"dependencies"`
	// contains filtered or unexported fields
}

Desc describes a Package. The description is used in registries. It contains all the information necessary to download and install a package. Furthermore, it contains all dependency information. Fundamentally, a description is sufficient to determine which versions a program wants to use, before downloading any source. The package resolution mechanism only needs descriptions.

func NewDesc

func NewDesc(name string, description string, url string, version string, sdk string, license string, hash string, deps []descPackage) *Desc

func ScrapeDescriptionAt

func ScrapeDescriptionAt(path string, allowsLocalDeps AllowLocalDepsFlag, isVerbose bool, ui UI) (*Desc, error)

func ScrapeDescriptionGit

func ScrapeDescriptionGit(ctx context.Context, url string, v string, allowsLocalDeps AllowLocalDepsFlag, isVerbose bool, ui UI) (*Desc, error)

func (*Desc) IDCompare

func (d *Desc) IDCompare(other *Desc) int

IDCompare compares this and the other description with respect to their ids. The ID of a description is its URL combined with the version. Invalid versions are considered equal to themselves and less than valid ones.

func (*Desc) PackageDir

func (d *Desc) PackageDir() string

func (*Desc) Parse

func (d *Desc) Parse(b []byte, ui UI) error

TODO (jesper): Parse and WriteTo should preserve comments.

func (*Desc) ParseFile

func (d *Desc) ParseFile(filename string, ui UI) error

func (*Desc) ParseString

func (d *Desc) ParseString(str string, ui UI) error

func (*Desc) Validate

func (d *Desc) Validate(ui UI) error

func (*Desc) WriteInDir

func (d *Desc) WriteInDir(outDir string) (string, error)

func (*Desc) WriteToFile

func (d *Desc) WriteToFile() error

func (*Desc) WriteYAML

func (d *Desc) WriteYAML(writer io.Writer) error

type DescEnvironment

type DescEnvironment struct {
	SDK string `yaml:"sdk,omitempty" json:"sdk,omitempty"`
}

type DescRegistries

type DescRegistries []DescRegistry

func (DescRegistries) WithoutLowerVersions

func (descs DescRegistries) WithoutLowerVersions() (DescRegistries, error)

WithoutLowerVersions discards descriptions of packages where a higher version exists. If a constraint is given, then descriptions are first filtered out according to the constraint.

type DescRegistry

type DescRegistry struct {
	Desc     *Desc
	Registry Registry
}

DescRegistry combines a description with the registry it comes from.

type DownloadGitOptions

type DownloadGitOptions struct {
	Directory  string
	URL        string
	Version    string
	Hash       string
	UI         UI
	NoReadOnly bool
}

type LockFile

type LockFile struct {

	// SDK constraint, if any.
	// Must be of form '^version'.
	SDK string `yaml:"sdk,omitempty"`
	// Prefixes for the entry module.
	Prefixes PrefixMap `yaml:"prefixes,omitempty"`
	// All dependent packages: from package-id to their PackageEntry
	Packages map[string]PackageEntry `yaml:"packages,omitempty"`
	// contains filtered or unexported fields
}

LockFile represents a lock file.

func ReadLockFile

func ReadLockFile(path string) (*LockFile, error)

ReadLockFile reads the lock-file at the given path.

func (*LockFile) WriteToFile

func (lf *LockFile) WriteToFile() error

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager serves as entry point for all package-management related operations. Use NewManager to create a new manager.

func NewManager

func NewManager(registries Registries, cache Cache, sdkVersion *version.Version, ui UI, track tracking.Track) *Manager

NewManager returns a new Manager.

type PackageEntry

type PackageEntry struct {
	URL      compiler.URIPath `yaml:"url,omitempty"`
	Name     string           `yaml:"name,omitempty"`
	Version  string           `yaml:"version,omitempty"`
	Path     compiler.Path    `yaml:"path,omitempty"`
	Hash     string           `yaml:"hash,omitempty"`
	Prefixes PrefixMap        `yaml:"prefixes,omitempty"`
}

PackageEntry corresponds to a resolved package. If 'path' is given, the package is at the location given by the path. The path can be absolute or relative to the lock file. If 'url' is given, then 'version' must be given as well. The entry then refers to a non-local package and is found in the package cache.

func (PackageEntry) Validate

func (pe PackageEntry) Validate(ui UI) error

Validate ensures that the receiver is a valid LockFileEntry.

type PrefixMap

type PrefixMap map[string]string

PrefixMap has a mapping from prefix to package-id.

type ProjectPaths

type ProjectPaths struct {
	// Project root.
	ProjectRootPath string

	// The path of the lock file for the current project.
	LockFile string

	// The path of the spec file for the current project.
	SpecFile string
}

func NewProjectPaths

func NewProjectPaths(projectRoot string, lockPath string, specPath string) (*ProjectPaths, error)

setupPaths sets the spec and lock file, searching in the given directory.

Does not overwrite a set value (m.SpecFile or m.LockFile). If the given directory is empty, starts the search in the current working directory. If a file doesn't exists, returns the path for it in the given directory.

type ProjectPkgManager

type ProjectPkgManager struct {
	*Manager

	// The project relevant Paths.
	Paths *ProjectPaths
}

ProjectPackageManager: a package manager for a specific project.

func NewProjectPkgManager

func NewProjectPkgManager(manager *Manager, paths *ProjectPaths) *ProjectPkgManager

func (*ProjectPkgManager) CleanPackages

func (m *ProjectPkgManager) CleanPackages() error

CleanPackages removes unused downloaded packages from the local cache.

func (*ProjectPkgManager) Install

func (m *ProjectPkgManager) Install(ctx context.Context, forceRecompute bool) error

Install downloads all dependencies. Simply downloads all dependencies, if forceRecompute is false, and a lock file without local dependencies exists. Otherwise (re)computes the lockfile, giving preference to versions that are listed in the lockfile (if it exists).

func (*ProjectPkgManager) InstallLocalPkg

func (m *ProjectPkgManager) InstallLocalPkg(ctx context.Context, name string, path string) (string, error)

InstallLocalPkg installs the local package at the given path. When provided, the package is installed with the given name. Otherwise, the packages name is extracted from the path. Returns the name that was used for the package. TODO(florian): the package name should be extracted from the package.yaml, or the README.

func (*ProjectPkgManager) InstallURLPkg

func (m *ProjectPkgManager) InstallURLPkg(ctx context.Context, name string, id string) (string, string, error)

InstallURLPkg install the package identified by its identifier id. The id can be a (suffix of a) package URL, or a package name. The identifier can also be suffixed by a `@` followed by a version. When provided, the package is installed with the given name. Otherwise, the packages name (extracted from the description) is used. Returns (name, package-string, err).

func (*ProjectPkgManager) PrintLockFile

func (m *ProjectPkgManager) PrintLockFile() error

PrintLockFile prints the contents of the lock file for the current project.

func (*ProjectPkgManager) PrintSpecFile

func (m *ProjectPkgManager) PrintSpecFile() error

PrintSpecFile prints the contents of the spec file for the current project.

func (*ProjectPkgManager) Uninstall

func (m *ProjectPkgManager) Uninstall(ctx context.Context, name string) error

func (*ProjectPkgManager) Update

func (m *ProjectPkgManager) Update(ctx context.Context) error

type Registries

type Registries []Registry

func (Registries) MatchName

func (registries Registries) MatchName(name string) (DescRegistries, error)

MatchName searches for the given name in all registries, but requires a full match.

func (Registries) SearchAll

func (registries Registries) SearchAll(needle string) (DescRegistries, error)

SearchAll searches for the given needle in the names and descriptions of all registries.

func (Registries) SearchName

func (registries Registries) SearchName(name string) (DescRegistries, error)

SearchName searches for the given name in all registries.

func (Registries) SearchURL

func (registries Registries) SearchURL(url string) (DescRegistries, error)

SearchURLVersion searches for the package with the given url and version in all registries.

func (Registries) SearchURLVersion

func (registries Registries) SearchURLVersion(url string, version string) (DescRegistries, error)

SearchURLVersion searches for the package with the given url and version in all registries.

type Registry

type Registry interface {
	// Name of the registry.
	Name() string
	// Loads the registry into memory.
	// Synchronizes the registry first, if 'sync' is true.
	// Synchronization installs the registry first if necessary. It then
	// downloads the latest packages.
	Load(ctx context.Context, sync bool, cache Cache, ui UI) error
	// Clears the cache, if there is any.
	ClearCache(ctx context.Context, cache Cache, ui UI) error
	// Describes this registry. Used when showing where a specification comes from.
	Describe() string
	// All the loaded entries. If the registry hasn't been loaded yet returns nil.
	Entries() []*Desc
	// Searches for the given package name in the registry.
	// Returns all matching packages.
	SearchName(name string) ([]*Desc, error)
	// Searches for the given package name in the registry.
	// THe name must match completely.
	// Returns all matching packages.
	MatchName(name string) ([]*Desc, error)
	// Searches for needle.
	// The search uses all description information (including description, authors, ...)
	// to find the package.
	SearchAll(needle string) ([]*Desc, error)
	// Searches for a package with the given URL and version.
	SearchURLVersion(url string, version string) ([]*Desc, error)
	// Searches for all package with the given URL.
	SearchURL(url string) ([]*Desc, error)
	// contains filtered or unexported methods
}

Registry is a source of package descriptions.

func NewGitRegistry

func NewGitRegistry(name string, url string, cache Cache) (Registry, error)

NewGitRegistry creates a new registry that is backed by a git-repository. The data is fetched (cloned) during 'Load' when 'sync' is true.

func NewLocalRegistry

func NewLocalRegistry(name string, path string) Registry

NewLocalRegistry creates a new path registry. Path registries simply find all package descriptions in a certain path.

func NewSSHGitRegistry

func NewSSHGitRegistry(name string, url string, cache Cache, sshPath string, branch string) (Registry, error)

type RegistryConfig

type RegistryConfig struct {
	Name string       `yaml:"name"`
	Kind RegistryKind `yaml:"kind"`
	Path string       `yaml:"path"`
}

RegistryConfig can be used to load a registry with LoadRegistry or LoadRegistries.

func (RegistryConfig) Load

func (cfg RegistryConfig) Load(ctx context.Context, sync bool, clearCache bool, cache Cache, ui UI) (Registry, error)

Load loads the registry given by its configuration.

type RegistryConfigs

type RegistryConfigs []RegistryConfig

func (RegistryConfigs) Load

func (configs RegistryConfigs) Load(ctx context.Context, sync bool, cache Cache, ui UI) (Registries, error)

Load takes the registry configuration and loads the corresponding registries into memory.

type RegistryKind

type RegistryKind string

RegistryKind specifies how to load a registry. See PathKind.

const (
	// RegistryKindLocal specifies that the corresponding registry should treated like
	// a simple folder with descriptions in it.
	RegistryKindLocal RegistryKind = "local"
	// RegistryKindGit specifies that the registry is backed by a git-repository.
	RegistryKindGit RegistryKind = "git"
)

func (RegistryKind) IsValid

func (k RegistryKind) IsValid() bool

IsValid returns whether the registry kind is valid. The kind value should be one of the exported kinds. See PathKind.

type Solution

type Solution struct {
	// contains filtered or unexported fields
}

Solution is a map from pkg-url to a set of version-strings.

type Solver

type Solver struct {
	// contains filtered or unexported fields
}

Solver is a simple constraint solver for the Toit package manager.

func NewSolver

func NewSolver(registries Registries, sdkVersion *version.Version, ui UI) (*Solver, error)

func (*Solver) SetPreferred

func (s *Solver) SetPreferred(preferred []versionedURL)

SetPreferred marks the list of versionedURLs as preferred.

func (*Solver) Solve

func (s *Solver) Solve(minSDK *version.Version, deps []SolverDep) *Solution

type SolverDep

type SolverDep struct {
	// contains filtered or unexported fields
}

SolverDep represents a dependency for the solver. It needs the target's package name and the version constraints for it.

func NewSolverDep

func NewSolverDep(url string, constraintString string) (SolverDep, error)

type Spec

type Spec struct {
	Name        string          `yaml:"name,omitempty"`
	Description string          `yaml:"description,omitempty"`
	License     string          `yaml:"license,omitempty"`
	Environment SpecEnvironment `yaml:"environment,omitempty"`
	Deps        DependencyMap   `yaml:"dependencies,omitempty"`
	// contains filtered or unexported fields
}

Spec specifies a Package. This specification is used for two (partially overlapping) purposes: 1. create a package description for package registries. 2. specify the prefix-dependency mapping.

func NewSpecFromLockFile

func NewSpecFromLockFile(lf *LockFile) (*Spec, error)

func ReadSpec

func ReadSpec(path string, ui UI) (*Spec, error)

ReadSpec reads the spec-file at the given path.

func (*Spec) BuildLockFile

func (s *Spec) BuildLockFile(solution *Solution, cache Cache, registries Registries, ui UI) (*LockFile, error)

BuildLockFile generates a lock file using the given solution. Assumes that all packages in the solution are used.

func (*Spec) BuildSolverDeps

func (s *Spec) BuildSolverDeps(ui UI) ([]SolverDep, error)

BuildSolverDeps builds an array of SolverDeps from the content of the lock file. This involves looking into spec files of local dependencies.

func (*Spec) Parse

func (s *Spec) Parse(b []byte, ui UI) error

TODO (jesper): Parse and WriteYAML should preserve comments.

func (*Spec) ParseFile

func (s *Spec) ParseFile(filename string, ui UI) error

func (*Spec) ParseString

func (s *Spec) ParseString(str string, ui UI) error

func (*Spec) Validate

func (s *Spec) Validate(ui UI) error

func (*Spec) WriteToFile

func (s *Spec) WriteToFile() error

func (*Spec) WriteYAML

func (s *Spec) WriteYAML(writer io.Writer) error

type SpecEnvironment

type SpecEnvironment struct {
	SDK string `yaml:"sdk,omitempty"`
}

type SpecPackage

type SpecPackage struct {
	// URL is the github URL (for cloning) of the package.
	// It uniquely identifies the package (all its versions) in the registry.
	URL string `yaml:"url,omitempty"`
	// Version is a version constraint on the package.
	// A missing version constraint allows any version of the package.
	Version string `yaml:"version,omitempty"`
	// Path is set if the package should be found locally.
	// This field overrides all other fields. This makes it possible to
	// temporarily (during development) switch to a local version.
	Path compiler.Path `yaml:"path,omitempty"`
}

SpecPackage identifies a package in the spec file. A valid instance has at least URL or Path set.

func (*SpecPackage) Validate

func (sp *SpecPackage) Validate(prefix string, ui UI) error

type StringVersion

type StringVersion struct {
	// contains filtered or unexported fields
}

type UI

type UI interface {
	// ReportError signals an error to the user.
	// The format string is currently compatible with fmt.Printf.
	// Returns ErrAlreadyReported.
	ReportError(format string, a ...interface{}) error

	// ReportError signals a warning to the user.
	// The format string is currently compatible with fmt.Printf.
	ReportWarning(format string, a ...interface{})

	// ReportInfo reports interesting information.
	ReportInfo(format string, a ...interface{})
}

UI allows this package to interact with the user.

This package will report user-facing errors (like missing packages) through this interface. The package might report multiple errors. If an action wasn't successful (like installing a package), then the package reports the error and then returns an AlreadyReportedError. This indicates to the caller that the operation failed, but that no further information needs to be printed.

var (
	// ErrAlreadyReported can be used to signal that an error has
	// been reported, and that no further action needs to be taken.
	// The returned error should be interchangeable. That is, one should
	// be able to call this function multiple times and just return any
	// of the received errors.
	// In case the error gets printed anyway, we have a sensible error message
	// instead of "already reported" or similar.
	ErrAlreadyReported = fmt.Errorf("package management error")

	// FmtUI is simple version of UI that uses 'fmt' to report warnings and errors.
	FmtUI UI = fmtUI{}
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL