Crates Mirror
Download all crates on Rust official crates site
and keep sync with it. This can be used to setup a static mirror site
of https://crates.io. This can be used with
cargo-mirror to make
dependency download faster when building Rust project.
This fork adds more implementations of the crates-mirror
Requirements
- Good bandwidth (at least can access aws-s3 service of us region)
- Large hard disk (at least 30G, the current size is 18G, 2018-11-15.)
Go
How
-
Clone this repo: git clone https://github.com/EmilSodergren/crates-mirror.git
-
go get github.com/mattn/go-sqlite3
-
go install
-
Add a config.json file in your current directory, or place it wherever and pass the path as an argument.
-
Run the program $GOBIN/crates-mirror [/path/to/config.json]
The config.json has the following structure:
{
"indexurl":"https://github.com/rust-lang/crates.io-index",
"registrypath":"/path/to/the/crates.io-index",
"cratespath":"/path/to/the/downloaded/crate/files",
"dbpath":"/path/to/the/metadata/database",
"updateindex":true <--- if not present, will use true as default
}
Python (The original)
- Python3 (yes python3, python2 is dead)
How
- Clone this repo:
git clone https://github.com/tennix/crates-mirror
- Fire a python virtualenv:
cd crates-mirror
pyvenv env
source env/bin/activate
pip install -r requirements.txt
- Run this program:
python app.py
- (Optional)Serve a mirror site:
cd crates
python -m http.server 8000
Note: for production, you should make this program auto-restarted
when dies (supervisord like tools is
needed). And also use a production web server (nginx, apache etc.) to
serve the mirror site