Clone matching repos on GitHub
$ github-clone-all [flags] {query}
github-clone-all
is a small command to clone all repositories matching to given query and
language via GitHub Search API. To know the detail of query, please read
official document for GitHub Repository Search. The query should be in
GitHub search syntax and cannot be empty. It clones many repositories in parallel. Please see
-help
option to know all flags.
Repository is cloned to 'dest' directory. It is $cwd/repos
by default and can be specified with
-dest
flag. And in order to reduce size of cloned repositories, -extract
option is available.
-extract
only leaves files matching to given regular expression.
Because of restriction of GitHub search API, max number of results is 1000 repositories. And you may
need to gain GitHub API token in advance to avoid reaching API rate limit. github-clone-all
will
refer the token via -token
flag or $GITHUB_TOKEN
environment variable.
All arguments in {query}
are regarded as query. For example, github-clone-all foo bar
will search
foo bar
. But quoting the query is recommended to avoid conflicting with shell special characters
as github-clone-all 'foo bar'
.
Installation
Use go get
or released binaries.
$ go get github.com/rhysd/github-clone-all
Example
$ github-clone-all -extract '(\.vim|vimrc)$' 'language:vim fork:false stars:>1'
Above command will clone first 1000 repositories into 'repos' directory in the current working
directory. And it only leaves files whose file name ends with .vim
or vimrc
.
So it collects many Vim script codes from famous repositories on GitHub.
Query condition:
- language is 'vim'
- not a fork repo
- stars of repo is more than 1
$ github-clone-all -count 1 'language:javascript'
Above command will clone the most popular repository of JavaScript on GitHub.
$ github-clone-all -dry 'language:go'
Above command will only list up most popular 1000 repositories of Go instead of cloning them.
$ github-clone-all -deep 'user:YOUR_USER_NAME fork:false'
Above command will clone all your repositories (except for forks) with full history.
How to get GitHub API token
- Visit https://github.com/settings/tokens in a browser
- Click 'Generate new token'
- Add token description
- Without checking any checkbox, click 'Generate token'
- Generated token is shown at the top of your tokens list
Use github-clone-all programmatically
github-clone-all
consists of tiny main.go
and ghca
package. You can import ghca
to utilize
functions of the tool.
import "github.com/rhysd/github-clone-all/ghca"
Please read documentation for more detail.
License
MIT license