README
¶
DU.
Simple and fast concurrent downloader & uploader.
Installation ❘ CLI Usage ❘ Module Usage ❘ License
Comparison
Comparison in cloud server:
Installation
Download and install the latest release:
# go to tmp dir.
cd /tmp
# Download latest version.
curl -sfL https://git.io/getdu | sh
# Make the binary executable.
chmod +x /tmp/bin/du
# Move the binary to your PATH
sudo mv /tmp/bin/du /usr/bin/du
Or Go ahead compile it yourself:
go get github.com/g-tool/du/cmd/du
Note: these packages are not maintained by g-tool
Command Line Tool Usage
Simple usage:
du https://example.com/file.mp4
You can specify destination path:
du -o /path/to/save https://example.com/file.mp4
You can download multiple URLs and save them to directory:
du --dir /path/to/dir https://example.com/file.mp4 https://example.com/file2.mp4
You can download multiple URLs from a file:
du --dir /path/to/dir -f urls.txt
You can pipe multiple URLs:
cat urls.txt | du --dir /path/to/dir
Docs for available flags:
du help
Module Usage
You can use DU to download large files in your go code, the usage is simple as the CLI tool:
package main
import "github.com/go-tool/du"
func main() {
d := du.New()
err := d.Download("http://localhost/file.ext", "/path/to/save")
if err != nil {
// ..
}
}
For more see PkgDocs.
How It Works?
DU takes advantage of the HTTP range requests support in servers RFC 7233, if the server supports partial content DU split the file into chunks, then starts downloading and merging the chunks into the destinaton file concurrently.
License
DU is provided under the MIT License © Tacey Wong.
Documentation
¶
Index ¶
- Constants
- Variables
- func GetFilename(URL string) string
- func NewRequest(ctx context.Context, method, URL string) (req *http.Request, err error)
- type Chunk
- type Download
- func (d Download) AvgSpeed() uint64
- func (d Download) Context() context.Context
- func (d *Download) DownloadChunk(c Chunk, dest *os.File) error
- func (d Download) GetInfo() (*Info, error)
- func (d *Download) Init() (err error)
- func (d Download) IsRangeable() bool
- func (d *Download) Name() string
- func (d *Download) RunProgress(fn ProgressFunc)
- func (d Download) Size() uint64
- func (d Download) Speed() uint64
- func (d *Download) Start() (err error)
- func (d Download) TotalCost() time.Duration
- func (d Download) TotalSize() uint64
- func (d *Download) Write(b []byte) (int, error)
- type Got
- type Info
- type ProgressFunc
Examples ¶
Constants ¶
const DefaulUserAgent = "Got/1.0"
DefaulUserAgent is the default Got user agent to send http requests.
Variables ¶
var ChunkPool = &sync.Pool{ New: func() interface{} { return new(Chunk) }, }
ChunkPool helps in multi *Download files.
var DefaultClient = &http.Client{ Transport: &http.Transport{ MaxIdleConns: 10, IdleConnTimeout: 30 * time.Second, TLSHandshakeTimeout: 5 * time.Second, Proxy: http.ProxyFromEnvironment, }, }
DefaultClient is the default http client for got requests.
var DefaultFileName = "got.output"
DefaultFileName is the fallback name for GetFilename.
var ErrDownloadAborted = errors.New("Operation aborted")
ErrDownloadAborted - When download is aborted by the OS before it is completed, ErrDownloadAborted will be triggered
Functions ¶
func GetFilename ¶
GetFilename it returns default file name from a URL.
Types ¶
type Chunk ¶
type Chunk struct {
// Chunk start pos.
Start uint64
// Chunk end pos.
End uint64
// Path name where this chunk downloaded.
Path string
// Done to check is this chunk downloaded.
Done chan struct{}
}
Chunk is a partial content range.
type Download ¶
type Download struct {
Client *http.Client
Concurrency uint
URL, Dir, Dest string
Interval, ChunkSize, MinChunkSize, MaxChunkSize uint64
StopProgress bool
// contains filtered or unexported fields
}
Download holds downloadable file config and infos.
Example ¶
// Just for testing
destPath := createTemp()
defer clean(destPath)
ctx := context.Background()
dl := got.NewDownload(ctx, testUrl, destPath)
// Init
if err := dl.Init(); err != nil {
fmt.Println(err)
}
// Start download
if err := dl.Start(); err != nil {
fmt.Println(err)
}
fmt.Println("Done")
Output: Done
func NewDownload ¶
NewDownload returns new *Download with context.
func (*Download) DownloadChunk ¶
DownloadChunk downloads a file chunk.
func (*Download) Init ¶
Init set defaults and split file into chunks and gets Info, you should call Init before Start
func (Download) IsRangeable ¶
IsRangeable returns file server partial content support state.
func (*Download) RunProgress ¶
func (d *Download) RunProgress(fn ProgressFunc)
RunProgress runs ProgressFunc based on Interval and updates lastSize.
type Got ¶
type Got struct {
ProgressFunc
Client *http.Client
// contains filtered or unexported fields
}
Got holds got download config.
Example ¶
// Just for testing
destPath := createTemp()
defer clean(destPath)
g := got.New()
err := g.Download(testUrl, destPath)
if err != nil {
log.Fatal(err)
return
}
fmt.Println("done")
Output: done
Example (WithContext) ¶
// Just for testing
destPath := createTemp()
defer clean(destPath)
ctx := context.Background()
g := got.NewWithContext(ctx)
err := g.Download(testUrl, destPath)
if err != nil {
log.Fatal(err)
return
}
fmt.Println("done")
Output: done
func NewWithContext ¶
NewWithContext wants Context and returns *Got with default http client.
type ProgressFunc ¶
type ProgressFunc func(d *Download)
ProgressFunc to show progress state, called by RunProgress based on interval.
Source Files
Directories