skywalker: github.com/dixonwille/skywalker Index | Examples | Files

package skywalker

import "github.com/dixonwille/skywalker"

Package skywalker walks through a filesystem concurrently.

Index

Examples

Package Files

skywalker.go

type ListType Uses

type ListType int

ListType is used to specify how to handle the contents of a list

const (
    //LTBlacklist is used to specify that a list is to exclude the contents.
    LTBlacklist ListType = iota
    //LTWhitelist is used to specify that a list is to include the contents.
    LTWhitelist
)

type Skywalker Uses

type Skywalker struct {
    //Root is where the file walker starts. It is converted to an absolute path before start.
    Root string

    //List and ListType should only be used for fine filtering of paths.
    //It uses https://github.com/gobwas/glob for glob checking on each patch check.
    ListType ListType
    List     []string

    //ExtList and ExtListType are used to narrow down the files by their extensions.
    //Make sure to include the preceding ".".
    ExtListType ListType
    ExtList     []string

    //DirList and DirListType are used to narrow down by directories.
    //Will skip the appropriate directories and their files/subfolders.
    DirListType ListType
    DirList     []string

    //NumWorkers are how many workers are listening to the queue to do the work.
    NumWorkers int

    //QueueSize is how many paths to queue up at a time.
    //Useful for fine control over memory usage if needed.
    QueueSize int

    //Worker is the function that is called on each file/directory.
    Worker Worker

    //FilesOnly should be set to true if you only want to queue up files.
    FilesOnly bool
    // contains filtered or unexported fields
}

Skywalker can concurrently go through files in Root and call Worker on everything. It is recommended to use DirList and ExtList as much as possible as it is more perfomant than List.

Code:

package main

import (
    "fmt"
    "sort"
    "strings"
    "sync"

    "github.com/dixonwille/skywalker"
)

type ExampleWorker struct {
    *sync.Mutex
    found []string
}

func (ew *ExampleWorker) Work(path string) {
    //This is where the necessary work should be done.
    //This will get concurrently so make sure it is thread safe if you need info across threads.
    ew.Lock()
    defer ew.Unlock()
    ew.found = append(ew.found, path)
}

func main() {
    //Following two functions are only to create and destroy data for the example
    defer teardownData()
    standupData()

    ew := new(ExampleWorker)
    ew.Mutex = new(sync.Mutex)

    //root is the root directory of the data that was stood up above
    sw := skywalker.New(root, ew)
    sw.DirListType = skywalker.LTBlacklist
    sw.DirList = []string{"sub"}
    sw.ExtListType = skywalker.LTWhitelist
    sw.ExtList = []string{".pdf"}
    err := sw.Walk()
    if err != nil {
        fmt.Println(err)
        return
    }
    sort.Sort(sort.StringSlice(ew.found))
    for _, f := range ew.found {
        show := strings.Replace(f, sw.Root, "", 1)
        show = strings.Replace(show, "\\", "/", -1)
        fmt.Println(show)
    }
}

func New Uses

func New(root string, worker Worker) *Skywalker

New creates a new Skywalker that can walk through the specified root and calls the Worker on each file and/or directory. Defaults Skywalker to have 20 workers, a QueueSize of 100 and only queue files.

func (*Skywalker) Walk Uses

func (sw *Skywalker) Walk() error

Walk goes through the files and folders in Root and calls the worker on each. Checks the lists specified to check whether it should ignore files or directories. It also handles the creation of workers and queues needed for walking.

type Worker Uses

type Worker interface {
    Work(path string)
}

Worker is anything that knows what to do with a path.

Package skywalker imports 6 packages (graph) and is imported by 1 packages. Updated 2017-08-05. Refresh now. Tools for package owners.