README
¶
ranger - io.ReaderAt with range requests! 
INSTALL
$ go get howett.net/ranger
OVERVIEW
Package ranger provides an implementation of io.ReaderAt and io.ReadSeeker which makes partial document requests. Ranger ships with a range fetcher that operates on an HTTP resource using the Range: header.
USE
package main
import (
"archive/zip"
"io"
"howett.net/ranger"
"net/url"
"os"
)
func main() {
url, _ := url.Parse("http://example.com/example.zip")
reader, _ := ranger.NewReader(&ranger.HTTPRanger{URL: url})
length, _ := reader.Length()
zipreader, _ := zip.NewReader(reader, length)
data := make([]byte, zipreader.File[0].UncompressedSize64)
rc, _ := zipreader.File[0].Open()
io.ReadFull(rc, data)
rc.Close()
}
Documentation
¶
Overview ¶
Package ranger implements an io.ReaderAt and io.ReadSeeker-compliant implementation of a caching HTTP range request client.
Index ¶
Examples ¶
Constants ¶
const DefaultBlockSize int = 128 * 1024
DefaultBlockSize is the default size for the blocks that are downloaded from the server and cached.
Variables ¶
var ( // ErrResourceChanged is the error returned by Read when the underlying resource's integrity can no longer be verified. // In the case of HTTP, this usually happens when the remote document's validator has changed. ErrResourceChanged = errors.New("unsatisfiable range request; resource may have changed") // ErrResourceNotFound is returned by the first Read operation that determines that a resource is inaccessible. ErrResourceNotFound = errors.New("resource not found") )
Functions ¶
This section is empty.
Types ¶
type ByteRange ¶
type ByteRange struct {
Start, End int64
}
ByteRange represents a not-yet-fetched range of bytes
type HTTPClient ¶
type HTTPClient interface {
Do(*http.Request) (*http.Response, error)
Get(string) (*http.Response, error)
Head(string) (*http.Response, error)
}
HTTPClient is an interface describing the methods required from net/http.Client
type HTTPRanger ¶
type HTTPRanger struct {
URL *url.URL
Client HTTPClient
// contains filtered or unexported fields
}
HTTPRanger is a RangeFetcher that uses the HTTP Range: header to fetch blocks.
HTTPRanger first makes a HEAD request and then between 0 and Length()/BlockSize GET requests, attempting whenever possible to optimize for a lower number of requests.
No network requests are made until the first I/O-related function call.
func (*HTTPRanger) ExpectedLength ¶
func (r *HTTPRanger) ExpectedLength() (int64, error)
ExpectedLength returns the length, in bytes, of the ranged-over file.
func (*HTTPRanger) FetchRanges ¶
func (r *HTTPRanger) FetchRanges(ranges []ByteRange) ([]Block, error)
FetchRanges requests ranges from the HTTP server.
type RangeFetcher ¶
type RangeFetcher interface {
FetchRanges([]ByteRange) ([]Block, error)
ExpectedLength() (int64, error)
}
RangeFetcher is the interface that wraps the FetchBlocks method.
FetchBlocks fetches the specified block ranges and returns any errors encountered in doing so.
Length returns the length, in bytes, of the ranged-over source.
Initialize, called once and passed the Reader's block size, performs any necessary setup tasks for the RangeFetcher
type Reader ¶
type Reader struct {
// the range fetcher with which to download blocks
Fetcher RangeFetcher
// size of the blocks fetched from the source and cached; lower values translate to lower memory usage, but typically require more requests
BlockSize int
// contains filtered or unexported fields
}
Reader is an io.ReaderAt and io.ReadSeeker backed by a partial block store.
Example ¶
url, _ := url.Parse(testServer.URL + "/b.zip")
reader, _ := NewReader(&HTTPRanger{URL: url})
length, _ := reader.Length()
zipreader, _ := zip.NewReader(reader, length)
for i, v := range zipreader.File {
fmt.Printf("[%d]: %s (%d bytes)\n", i, v.Name, v.UncompressedSize64)
}
data := make([]byte, 16)
rc, _ := zipreader.File[0].Open()
defer rc.Close()
io.ReadFull(rc, data)
fmt.Printf("Data from f00: `%s`\n", string(data))
Output: [0]: f00 (640 bytes) [1]: f01 (640 bytes) [2]: f02 (640 bytes) [3]: f03 (640 bytes) [4]: f04 (640 bytes) [5]: f05 (640 bytes) [6]: f06 (640 bytes) [7]: f07 (640 bytes) [8]: f08 (640 bytes) [9]: f09 (640 bytes) Data from f00: `f0000000000BEGIN`
func NewReader ¶
func NewReader(fetcher RangeFetcher) (*Reader, error)
NewReader returns a newly-initialized Reader, which also initializes its provided RangeFetcher. It returns the new reader and an error, if any.
func (*Reader) Read ¶
Read reads len(p) bytes from ranged-over source. It returns the number of bytes read and the error, if any. EOF is signaled by a zero count with err set to io.EOF.
func (*Reader) ReadAt ¶
ReadAt reads len(p) bytes from the ranged-over source. It returns the number of bytes read and the error, if any. ReadAt always returns a non-nil error when n < len(b). At end of file, that error is io.EOF.
Source Files