README
¶
s3link
An experiment in using the Storj Network via the AWS SDK for Go
background
In order to increase user adoption, Storj runs an S3-compliant web service called Gateway-MT. Gateway-MT has increased user adoption, but compromises on a core feature of the Storj Network in that it requires centralized HTTP endpoints. Analysis of various Amazon SDKs indicates it would be difficult to code generate "one size fits all" S3-API-compatible bindings across languages. Indeed [and perhaps because they span service types], most Amazon SDKs can be described as having "HTTP as the architecture".
Thus an initial attempt to integrate with the AWS SDK for Go by implementing an interface was abandoned. This more language agnostic solution accepts the overhead of serializing code to HTTP streams, which are then processed without actually sending those streams. Application logic from Storj's Gateway-ST project handles adapting the request back to native libuplink Storj Network requests.
usage
The Gateway-ST / Minio libraries used by this code implement request signing. Thus, the credentials and host names must match between the AWS SDK for Go session variable and the S3Link logic. EG:
awsSession, _ := session.NewSession(&aws.Config{
Credentials: credentials.NewStaticCredentials("placeholder", "placeholder", ""),
Endpoint: aws.String("http://s3.amazonaws.com"),
Region: aws.String("us-east-1"),
S3ForcePathStyle: aws.Bool(true),
})
client := s3.New(awsSession, aws.NewConfig().WithHTTPClient(s3link.NewHTTPClient()))
downloader := s3manager.NewDownloader(awsSession, func(d *s3manager.Downloader) { d.S3 = client }),
uploader := s3manager.NewUploader(awsSession, func(u *s3manager.Uploader) { u.S3 = client }),
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var GlobalActiveCred auth.Credentials
GlobalActiveCred exposes minio's cmd.globalActiveCred.
nolint: golint
var GlobalServerRegion string
GlobalServerRegion exposes minio's cmd.globalServerRegion.
nolint: golint
Functions ¶
func NewHTTPClient ¶
NewHTTPClient returns a new http.Client with StorjTransport as the Transport. client := s3.New(awsSession, aws.NewConfig().WithHTTPClient(&s3link.NewHTTPClient()})) downloader := s3manager.NewDownloader(awsSession, func(d *s3manager.Downloader) { d.S3 = client }) uploader := s3manager.NewUploader(awsSession, func(u *s3manager.Uploader) { u.S3 = client })
Types ¶
type StorjTransport ¶
type StorjTransport struct {
// contains filtered or unexported fields
}
StorjTransport can be used as an http.Transport, but uses the Storj Network directly via DRCP, not HTTP.
func NewStorjTransport ¶
func NewStorjTransport() StorjTransport
NewStorjTransport can be used as an http.Client{Transport: NewStorjTransport()}. A serialized Storj access grant should be passed in via the environment variables "ACCESS".
Source Files