component

package
v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 7, 2021 License: GPL-3.0 Imports: 20 Imported by: 0

Documentation

Index

Constants

View Source 
const NumberPossibleComponents = 8

NumberPossibleComponents is the number of subcomponents plus "dataset".

View Source 
const WritePerm = os.FileMode(int(0644))

WritePerm is the file permission for the written files

Variables

View Source 
var DatasetFields = []string{"commit", "cm", "structure", "st", "body", "bd", "meta", "md", "readme", "rm", "viz", "vz", "transform", "tf", "rendered", "rd", "stats"}

DatasetFields is a list of valid dataset field identifiers

View Source 
var (
	// ErrNoDatasetFiles indicates no data
	ErrNoDatasetFiles = fmt.Errorf("no dataset files provided")
)
View Source 
var IsDatasetField = regexp.MustCompile("(?i)^(" + strings.Join(DatasetFields, "|") + ")($|\\.)")

IsDatasetField can be used to check if a string is a dataset field identifier

Functions

func AllSubcomponentNames 

func AllSubcomponentNames() []string

AllSubcomponentNames is the names of subcomponents that can live on a collection.

func ExpandListedComponents 

func ExpandListedComponents(container Component, resolver qfs.Filesystem) error

ExpandListedComponents will read whatever is necessary in order to discover all of the components that exist within this observation. For example, if a "dataset" exists, it will be read to find out if it contains a "meta", a "structure", etc. No other components are expanded, but this may change in the future if we decide another component can contain some other component. If the "dataset" file does not exist, an empty dataset component will be created.

func GetKnownFilenames 

func GetKnownFilenames() map[string][]string

GetKnownFilenames returns a map containing all possible filenames (filebase and extension) for any file that can represent a component of a dataset.

func IsKnownFilename added in v0.9.4 

func IsKnownFilename(fullpath string, known map[string][]string) bool

IsKnownFilename returns whether the file is a known component filename.

func OpenEntryReader 

func OpenEntryReader(file *os.File, format string) (dsio.EntryReader, error)

OpenEntryReader opens a entry reader for the file, determining the schema automatically TODO(dlong): Move this to dataset.dsio

func SerializeBody 

func SerializeBody(source interface{}, st *dataset.Structure) ([]byte, error)

SerializeBody writes the source, which must be an array or object, TODO(dlong): Move this to dataset.dsio

func ToDataset 

func ToDataset(comp Component) (*dataset.Dataset, error)

ToDataset converts a component to a dataset. Should only be used on a component representing an entire dataset.

Types

type BaseComponent 

type BaseComponent struct {
	Subcomponents  map[string]Component
	ProblemKind    string
	ProblemMessage string
	// File information:
	ModTime    time.Time
	SourceFile string
	IsLoaded   bool
	Format     string
}

BaseComponent is the data elements common to any component

func (*BaseComponent) Base 

func (bc *BaseComponent) Base() *BaseComponent

Base returns the common base data for the component

func (*BaseComponent) GetSubcomponent 

func (bc *BaseComponent) GetSubcomponent(name string) Component

GetSubcomponent returns the component with the given name

func (*BaseComponent) LoadFile 

func (bc *BaseComponent) LoadFile() (map[string]interface{}, error)

LoadFile opens the source file for the component and unmarshals it, adds errors for duplicate components and parse errors

func (*BaseComponent) RemoveSubcomponent 

func (bc *BaseComponent) RemoveSubcomponent(name string)

RemoveSubcomponent removes the component with the given name

func (*BaseComponent) SetErrorAsProblem 

func (bc *BaseComponent) SetErrorAsProblem(kind string, err error)

SetErrorAsProblem converts the error into a problem and assigns it

func (*BaseComponent) SetSubcomponent 

func (bc *BaseComponent) SetSubcomponent(name string, base BaseComponent) Component

SetSubcomponent constructs a component of the appropriate type and adds it as a subcomponent

type BodyComponent 

type BodyComponent struct {
	BaseComponent
	Resolver       qfs.PathResolver
	BodyFile       qfs.File
	Structure      *dataset.Structure
	InferredSchema map[string]interface{}
	Value          interface{}
}

BodyComponent represents a body component

func NewBodyComponent 

func NewBodyComponent(file string) *BodyComponent

NewBodyComponent returns a body component for the given source file

func (*BodyComponent) Compare 

func (bc *BodyComponent) Compare(compare Component) (bool, error)

Compare compares to another component

func (*BodyComponent) DropDerivedValues 

func (bc *BodyComponent) DropDerivedValues()

DropDerivedValues drops derived values from the component

func (*BodyComponent) LoadAndFill 

func (bc *BodyComponent) LoadAndFill(ds *dataset.Dataset) error

LoadAndFill loads data from the component source file and assigngs it

func (*BodyComponent) RemoveFrom 

func (bc *BodyComponent) RemoveFrom(dirPath string) error

RemoveFrom removes the component file from the directory

func (*BodyComponent) StructuredData 

func (bc *BodyComponent) StructuredData() (interface{}, error)

StructuredData returns the body as a map[string] or []interface{}, depending on top-level type

func (*BodyComponent) WriteTo 

func (bc *BodyComponent) WriteTo(dirPath string) (targetFile string, err error)

WriteTo writes the component as a file to the directory

type CommitComponent 

type CommitComponent struct {
	BaseComponent
	Value *dataset.Commit
}

CommitComponent represents a commit component

func (*CommitComponent) Compare 

func (cc *CommitComponent) Compare(compare Component) (bool, error)

Compare compares to another component

func (*CommitComponent) DropDerivedValues 

func (cc *CommitComponent) DropDerivedValues()

DropDerivedValues drops derived values from the component

func (*CommitComponent) LoadAndFill 

func (cc *CommitComponent) LoadAndFill(ds *dataset.Dataset) error

LoadAndFill loads data from the component source file and assigngs it

func (*CommitComponent) RemoveFrom 

func (cc *CommitComponent) RemoveFrom(dirPath string) error

RemoveFrom removes the component file from the directory

func (*CommitComponent) StructuredData 

func (cc *CommitComponent) StructuredData() (interface{}, error)

StructuredData returns the commit as a map[string]

func (*CommitComponent) WriteTo 

func (cc *CommitComponent) WriteTo(dirPath string) (targetFile string, err error)

WriteTo writes the component as a file to the directory

type Component 

type Component interface {
	Base() *BaseComponent
	Compare(Component) (bool, error)
	WriteTo(dirPath string) (targetFile string, err error)
	RemoveFrom(dirPath string) error
	DropDerivedValues()
	LoadAndFill(*dataset.Dataset) error
	StructuredData() (interface{}, error)
}

Component represents one of two things, either a single component (meta, body), or a collection of components, such as an entire dataset or a directory of files that encode components.

func ConvertDatasetToComponents 

func ConvertDatasetToComponents(ds *dataset.Dataset, qfilesys qfs.Filesystem) Component

ConvertDatasetToComponents will convert a dataset to a component collection

func ListDirectoryComponents 

func ListDirectoryComponents(dir string) (Component, error)

ListDirectoryComponents lists the relevant files and reads them into a component collection object. The resulting object has stat'ed each file, and has their mtimes, but no files have been read from disk. Conflicting files (such as both a "body.csv" and "body.json") will cause the "ProblemKind" and "ProblemMessage" fields to be set. Other conflicts may also exist, such as "meta" being in both "dataset.json" and "meta.json", but this function does not detect these kinds of problems because it does not read any files.

type DatasetComponent 

type DatasetComponent struct {
	BaseComponent
	Value *dataset.Dataset
}

DatasetComponent represents a dataset with components

func (*DatasetComponent) Compare 

func (dc *DatasetComponent) Compare(compare Component) (bool, error)

Compare compares to another component

func (*DatasetComponent) DropDerivedValues 

func (dc *DatasetComponent) DropDerivedValues()

DropDerivedValues drops derived values from the component

func (*DatasetComponent) LoadAndFill 

func (dc *DatasetComponent) LoadAndFill(ds *dataset.Dataset) error

LoadAndFill loads data from the component source file and assigngs it

func (*DatasetComponent) RemoveFrom 

func (dc *DatasetComponent) RemoveFrom(dirPath string) error

RemoveFrom removes the component file from the directory

func (*DatasetComponent) StructuredData 

func (dc *DatasetComponent) StructuredData() (interface{}, error)

StructuredData returns the dataset as a map[string]

func (*DatasetComponent) WriteTo 

func (dc *DatasetComponent) WriteTo(dirPath string) (targetFile string, err error)

WriteTo writes the component as a file to the directory

type FilesysComponent 

type FilesysComponent struct {
	BaseComponent
}

FilesysComponent represents a collection of components existing as files on a filesystem

func (*FilesysComponent) Compare 

func (fc *FilesysComponent) Compare(compare Component) (bool, error)

Compare compares to another component

func (*FilesysComponent) DropDerivedValues 

func (fc *FilesysComponent) DropDerivedValues()

DropDerivedValues drops derived values from the component

func (*FilesysComponent) IsEmpty 

func (fc *FilesysComponent) IsEmpty() bool

IsEmpty returns whether the component collection is empty

func (*FilesysComponent) LoadAndFill 

func (fc *FilesysComponent) LoadAndFill(ds *dataset.Dataset) error

LoadAndFill loads data from the component source file and assigngs it

func (*FilesysComponent) RemoveFrom 

func (fc *FilesysComponent) RemoveFrom(dirPath string) error

RemoveFrom removes the component file from the directory

func (*FilesysComponent) StructuredData 

func (fc *FilesysComponent) StructuredData() (interface{}, error)

StructuredData cannot be returned for a filesystem

func (*FilesysComponent) WriteTo 

func (fc *FilesysComponent) WriteTo(dirPath string) (targetFile string, err error)

WriteTo writes the component as a file to the directory

type MetaComponent 

type MetaComponent struct {
	BaseComponent
	Value *dataset.Meta
	// Prevent the component from being written. Used for testing.
	DisableSerialization bool
}

MetaComponent represents a meta component

func (*MetaComponent) Compare 

func (mc *MetaComponent) Compare(compare Component) (bool, error)

Compare compares to another component

func (*MetaComponent) DropDerivedValues 

func (mc *MetaComponent) DropDerivedValues()

DropDerivedValues drops derived values from the component

func (*MetaComponent) LoadAndFill 

func (mc *MetaComponent) LoadAndFill(ds *dataset.Dataset) error

LoadAndFill loads data from the component source file and assigngs it

func (*MetaComponent) RemoveFrom 

func (mc *MetaComponent) RemoveFrom(dirPath string) error

RemoveFrom removes the component file from the directory

func (*MetaComponent) StructuredData 

func (mc *MetaComponent) StructuredData() (interface{}, error)

StructuredData returns the meta as a map[string]

func (*MetaComponent) WriteTo 

func (mc *MetaComponent) WriteTo(dirPath string) (targetFile string, err error)

WriteTo writes the component as a file to the directory

type ReadmeComponent 

type ReadmeComponent struct {
	BaseComponent
	Resolver qfs.PathResolver
	Value    *dataset.Readme
}

ReadmeComponent represents a readme component

func (*ReadmeComponent) Compare 

func (rc *ReadmeComponent) Compare(compare Component) (bool, error)

Compare compares to another component

func (*ReadmeComponent) DropDerivedValues 

func (rc *ReadmeComponent) DropDerivedValues()

DropDerivedValues drops derived values from the component

func (*ReadmeComponent) LoadAndFill 

func (rc *ReadmeComponent) LoadAndFill(ds *dataset.Dataset) error

LoadAndFill loads data from the component source file and assigns it

func (*ReadmeComponent) RemoveFrom 

func (rc *ReadmeComponent) RemoveFrom(dirPath string) error

RemoveFrom removes the component file from the directory

func (*ReadmeComponent) StructuredData 

func (rc *ReadmeComponent) StructuredData() (interface{}, error)

StructuredData returns the readme as a map[string]

func (*ReadmeComponent) WriteTo 

func (rc *ReadmeComponent) WriteTo(dirPath string) (targetFile string, err error)

WriteTo writes the component as a file to the directory

type StructureComponent 

type StructureComponent struct {
	BaseComponent
	Value           *dataset.Structure
	SchemaInference func(*dataset.Dataset) (map[string]interface{}, error)
}

StructureComponent represents a structure component

func (*StructureComponent) Compare 

func (sc *StructureComponent) Compare(compare Component) (bool, error)

Compare compares to another component

func (*StructureComponent) DropDerivedValues 

func (sc *StructureComponent) DropDerivedValues()

DropDerivedValues drops derived values from the component

func (*StructureComponent) LoadAndFill 

func (sc *StructureComponent) LoadAndFill(ds *dataset.Dataset) error

LoadAndFill loads data from the component source file and assigngs it

func (*StructureComponent) RemoveFrom 

func (sc *StructureComponent) RemoveFrom(dirPath string) error

RemoveFrom removes the component file from the directory

func (*StructureComponent) StructuredData 

func (sc *StructureComponent) StructuredData() (interface{}, error)

StructuredData returns the structure as a map[string]

func (*StructureComponent) WriteTo 

func (sc *StructureComponent) WriteTo(dirPath string) (targetFile string, err error)

WriteTo writes the component as a file to the directory

type TransformComponent added in v0.9.2 

type TransformComponent struct {
	BaseComponent
	Resolver qfs.PathResolver
	Value    *dataset.Transform
}

TransformComponent represents a transform component

func (*TransformComponent) Compare added in v0.9.2 

func (tc *TransformComponent) Compare(compare Component) (bool, error)

Compare compares to another component

func (*TransformComponent) DropDerivedValues added in v0.9.2 

func (tc *TransformComponent) DropDerivedValues()

DropDerivedValues drops derived values from the component

func (*TransformComponent) LoadAndFill added in v0.9.2 

func (tc *TransformComponent) LoadAndFill(ds *dataset.Dataset) error

LoadAndFill loads data from the component soutce file and assigns it

func (*TransformComponent) RemoveFrom added in v0.9.2 

func (tc *TransformComponent) RemoveFrom(dirPath string) error

RemoveFrom removes the component file from the directory

func (*TransformComponent) StructuredData added in v0.9.2 

func (tc *TransformComponent) StructuredData() (interface{}, error)

StructuredData returns the transform as a map[string]

func (*TransformComponent) WriteTo added in v0.9.2 

func (tc *TransformComponent) WriteTo(dirPath string) (targetFile string, err error)

WriteTo writes the component as a file to the directory

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.