Starchive
About The Project
There are lots of awesome projects out there, and its common to star your
favorite projects/repos. Sometimes those projects become archived, or disappear
from the internet for various reasons. In order to archive most of these
projects and ensure you still have a copy of them, this project aims to enable
a quick binary to use with cron, and enable scheduled backups of all those
repos.
Getting Started
Prerequisites
Prerequisities of installing this application include the potential compilation,
and retrieval of a github user access token.
Compiling
Some pre-requisites for compiling this binary are the following packages:
golang
, and make
.
macOS
brew install golang make
debian
sudo apt install golang make
Clone the repository
git clone git@github.com:ChrisSwanson/Starchive.git && cd Starchive
Compile the binary
make build
Once the binary is done being compiled, the build can be found in build
directory.
Create Github Personal Access Token
Instructions can be found here on how to create a github personal access
token.
This application will just require basic access to public_repo
(browse public
repositories).
Configuration
It is recommended that this application be setup with cron to enable regularly
scheduled backups of github starred projects.
This was built with simplicity in mind, all flags have a 1:1 relation that can
be used in a configuration file.
Flags
--dir $TARGET_DIRECTORY
- the target directory to populate with git clones and pulls.
--token $GITHUB_PERSONAL_ACCESS_TOKEN
- The token flag is used to provide the github personal access token. This is
used to provide context to which it will follow the starred repo
--debug
- This flag is utilized to enable debug logging.
config.yml
dir: /tmp/starchive
debug: true
token: mypersonalgithubaccesstoken1234567890abc
Installation
Copy the binary
To make this binary more accessible, copy to your local bin directory
cp build/starchive /usr/local/bin/
Create a cronjob, with the path for your config in line:
I.E. if the config.yml file is provided in /etc/starchive/config.yml,
amd we want to run hourly at the top of the hour, we could enter something like
this:
crontab -l | { cat; echo "0 * * * * cd /etc/starchive && /usr/local/bin/starchive"; } | crontab -
License
Distributed under the MIT License. See LICENSE
for more information.
FAQ
Why does this project use a github personal access token, when this can just screen scrape?
Github rate limits
can be more restrictive for unauthenticated API based requests, and APIs are
going to be more consistent than screen scraping and breaking in future builds.
Additionally the github personal access token is used to determine your
username.
In the link provided for Github Rate Limits (at the time of writing this),
github allows for the following rate limits:
Auth |
Rate Limit |
Unauthenticated |
60/hour |
Personal Access Token |
1000/hour |
Basic / OAuth |
5000/hour |