README
¶
go-github-walk
Go package for walking all of the files in a GitHub repository.
Documentation
Example
package main
import (
"context"
"flag"
"fmt"
"github.com/google/go-github/github"
"github.com/sfomuseum/go-github-walk"
)
func main() {
walker_uri := flag.String("walker-uri", "", "A valid go-github-walk.GitHubWalker URI string.")
flag.Parse()
ctx := context.Background()
cb := func(ctx context.Context, contents *github.RepositoryContent) error {
fmt.Println(*contents.Path)
return nil
}
w, _ := walk.NewGitHubWalker(ctx, *walker_uri)
w.WalkURI(ctx, "", cb)
}
For example:
$> ./bin/walk -walker-uri 'walk://sfomuseum-data/sfomuseum-data-maps?access_token={ACCESS_TOKEN}&concurrent=1' data
data/147/788/175/3/1477881753.geojson
data/147/788/175/7/1477881757.geojson
data/171/295/239/3/1712952393.geojson
data/136/039/135/1/1360391351.geojson
... and so on
See also
Documentation
¶
Overview ¶
package walk provides methods for walking all (or some) of the files in a GitHub repository.
Index ¶
Constants ¶
const DEFAULT_BRANCH string = "main"
DEFAULT_BRANCH is the assumed default branch for any given GitHub repository.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type GitHubWalker ¶
type GitHubWalker struct {
// contains filtered or unexported fields
}
GitHubWalker is a struct that wraps operations for walking all the files in a GitHub repository.
func NewGitHubWalker ¶
func NewGitHubWalker(ctx context.Context, uri string) (*GitHubWalker, error)
NewGitHubWalker will create a new `GitHubWalker` instance from details defined in uri. uri takes the form of:
walk://sfomuseum-data/sfomuseum-data-collection?access_token={ACCESS_TOKEN}&concurrent=1
Where it's component part are:
scheme: `walk`, but this can be anything. host: A valid GitHub user or organization name. path: A valid GitHub repository name.
And it's allowable query parameters are:
access_token: A valid GitHub API access token (required). branch: A valid GitHub repository branch to walk. concurrent: A boolean flag indicating whether directory contents should be processed concurrently. wait-on-reset: A boolean flag indicating whether on an GitHub API rate limit error the GitHubWalker should pause until the specified reset time.
func (*GitHubWalker) WalkURI ¶
func (gw *GitHubWalker) WalkURI(ctx context.Context, uri string, cb WalkCallbackFunc) error
WalkURI will retrieve uri and if it is a file pass it to cb for final processing. If the contents of uri is a directory then each of its children will be processed by calling gw.WalkURI.
type WalkCallbackFunc ¶
type WalkCallbackFunc func(context.Context, *github.RepositoryContent) error
WalkCallbackFunc defines a custom callback function to be invoked for every file in a Github repository.
Source Files
Directories