mongo-driver: go.mongodb.org/mongo-driver/mongo/gridfs Index | Examples | Files

package gridfs

import "go.mongodb.org/mongo-driver/mongo/gridfs"

Package gridfs provides a MongoDB GridFS API. See https://docs.mongodb.com/manual/core/gridfs/ for more information about GridFS and its use cases.

Buckets

The main type defined in this package is Bucket. A Bucket wraps a mongo.Database instance and operates on two collections in the database. The first is the files collection, which contains one metadata document per file stored in the bucket. This collection is named "<bucket name>.files". The second is the chunks collection, which contains chunks of files. This collection is named "<bucket name>.chunks".

Uploading a File

Files can be uploaded in two ways:

1. OpenUploadStream/OpenUploadStreamWithID - These methods return an UploadStream instance. UploadStream
implements the io.Writer interface and the Write() method can be used to upload a file to the database.

2. UploadFromStream/UploadFromStreamWithID - These methods take an io.Reader, which represents the file to
upload. They internally create a new UploadStream and close it once the operation is complete.

Downloading a File

Similar to uploads, files can be downloaded in two ways:

1. OpenDownloadStream/OpenDownloadStreamByName - These methods return a DownloadStream instance. DownloadStream
implements the io.Reader interface. A file can be read either using the Read() method or any standard library
methods that reads from an io.Reader such as io.Copy.

2. DownloadToStream/DownloadToStreamByName - These methods take an io.Writer, which represents the download
destination. They internally create a new DownloadStream and close it once the operation is complete.

Index

Examples

Package Files

bucket.go doc.go download_stream.go upload_stream.go

Constants

const DefaultChunkSize int32 = 255 * 1024 // 255 KiB

DefaultChunkSize is the default size of each file chunk.

const UploadBufferSize = 16 * 1024 * 1024 // 16 MiB

UploadBufferSize is the size in bytes of one stream batch. Chunks will be written to the db after the sum of chunk lengths is equal to the batch size.

Variables

var ErrFileNotFound = errors.New("file with given parameters not found")

ErrFileNotFound occurs if a user asks to download a file with a file ID that isn't found in the files collection.

var ErrStreamClosed = errors.New("stream is closed or aborted")

ErrStreamClosed is an error returned if an operation is attempted on a closed/aborted stream.

var ErrWrongIndex = errors.New("chunk index does not match expected index")

ErrWrongIndex is used when the chunk retrieved from the server does not have the expected index.

var ErrWrongSize = errors.New("chunk size does not match expected size")

ErrWrongSize is used when the chunk retrieved from the server does not have the expected size.

type Bucket Uses

type Bucket struct {
    // contains filtered or unexported fields
}

Bucket represents a GridFS bucket.

func NewBucket Uses

func NewBucket(db *mongo.Database, opts ...*options.BucketOptions) (*Bucket, error)

NewBucket creates a GridFS bucket.

func (*Bucket) Delete Uses

func (b *Bucket) Delete(fileID interface{}) error

Delete deletes all chunks and metadata associated with the file with the given file ID.

If this operation requires a custom write deadline to be set on the bucket, it cannot be done concurrently with other write operations operations on this bucket that also require a custom deadline.

Code:

var bucket *gridfs.Bucket
var fileID primitive.ObjectID

if err := bucket.Delete(fileID); err != nil {
    log.Fatal(err)
}

func (*Bucket) DownloadToStream Uses

func (b *Bucket) DownloadToStream(fileID interface{}, stream io.Writer) (int64, error)

DownloadToStream downloads the file with the specified fileID and writes it to the provided io.Writer. Returns the number of bytes written to the steam and an error, or nil if there was no error.

If this download requires a custom read deadline to be set on the bucket, it cannot be done concurrently with other read operations operations on this bucket that also require a custom deadline.

Code:

var bucket *gridfs.Bucket
var fileID primitive.ObjectID

fileBuffer := bytes.NewBuffer(nil)
if _, err := bucket.DownloadToStream(fileID, fileBuffer); err != nil {
    log.Fatal(err)
}

func (*Bucket) DownloadToStreamByName Uses

func (b *Bucket) DownloadToStreamByName(filename string, stream io.Writer, opts ...*options.NameOptions) (int64, error)

DownloadToStreamByName downloads the file with the given name to the given io.Writer.

If this download requires a custom read deadline to be set on the bucket, it cannot be done concurrently with other read operations operations on this bucket that also require a custom deadline.

func (*Bucket) Drop Uses

func (b *Bucket) Drop() error

Drop drops the files and chunks collections associated with this bucket.

If this operation requires a custom write deadline to be set on the bucket, it cannot be done concurrently with other write operations operations on this bucket that also require a custom deadline

Code:

var bucket *gridfs.Bucket

if err := bucket.Drop(); err != nil {
    log.Fatal(err)
}

func (*Bucket) Find Uses

func (b *Bucket) Find(filter interface{}, opts ...*options.GridFSFindOptions) (*mongo.Cursor, error)

Find returns the files collection documents that match the given filter.

If this download requires a custom read deadline to be set on the bucket, it cannot be done concurrently with other read operations operations on this bucket that also require a custom deadline.

Code:

var bucket *gridfs.Bucket

// Specify a filter to find all files with a length greater than 1000 bytes.
filter := bson.D{
    {"length", bson.D{{"$gt", 1000}}},
}
cursor, err := bucket.Find(filter)
if err != nil {
    log.Fatal(err)
}
defer func() {
    if err := cursor.Close(context.TODO()); err != nil {
        log.Fatal(err)
    }
}()

type gridfsFile struct {
    Name   string `bson:"filename"`
    Length int64  `bson:"length"`
}
var foundFiles []gridfsFile
if err = cursor.All(context.TODO(), &foundFiles); err != nil {
    log.Fatal(err)
}

for _, file := range foundFiles {
    fmt.Printf("filename: %s, length: %d\n", file.Name, file.Length)
}

func (*Bucket) OpenDownloadStream Uses

func (b *Bucket) OpenDownloadStream(fileID interface{}) (*DownloadStream, error)

OpenDownloadStream creates a stream from which the contents of the file can be read.

Code:

var bucket *gridfs.Bucket
var fileID primitive.ObjectID

downloadStream, err := bucket.OpenDownloadStream(fileID)
if err != nil {
    log.Fatal(err)
}
defer func() {
    if err := downloadStream.Close(); err != nil {
        log.Fatal(err)
    }
}()

// Use SetReadDeadline to force a timeout if the download does not succeed in 2 seconds.
if err = downloadStream.SetReadDeadline(time.Now().Add(2 * time.Second)); err != nil {
    log.Fatal(err)
}

fileBuffer := bytes.NewBuffer(nil)
if _, err := io.Copy(fileBuffer, downloadStream); err != nil {
    log.Fatal(err)
}

func (*Bucket) OpenDownloadStreamByName Uses

func (b *Bucket) OpenDownloadStreamByName(filename string, opts ...*options.NameOptions) (*DownloadStream, error)

OpenDownloadStreamByName opens a download stream for the file with the given filename.

func (*Bucket) OpenUploadStream Uses

func (b *Bucket) OpenUploadStream(filename string, opts ...*options.UploadOptions) (*UploadStream, error)

OpenUploadStream creates a file ID new upload stream for a file given the filename.

Code:

var fileContent []byte
var bucket *gridfs.Bucket

// Specify the Metadata option to include a "metadata" field in the files collection document.
uploadOpts := options.GridFSUpload().SetMetadata(bson.D{{"metadata tag", "tag"}})
uploadStream, err := bucket.OpenUploadStream("filename", uploadOpts)
if err != nil {
    log.Fatal(err)
}
defer func() {
    if err = uploadStream.Close(); err != nil {
        log.Fatal(err)
    }
}()

// Use SetWriteDeadline to force a timeout if the upload does not succeed in 2 seconds.
if err = uploadStream.SetWriteDeadline(time.Now().Add(2 * time.Second)); err != nil {
    log.Fatal(err)
}

if _, err = uploadStream.Write(fileContent); err != nil {
    log.Fatal(err)
}

func (*Bucket) OpenUploadStreamWithID Uses

func (b *Bucket) OpenUploadStreamWithID(fileID interface{}, filename string, opts ...*options.UploadOptions) (*UploadStream, error)

OpenUploadStreamWithID creates a new upload stream for a file given the file ID and filename.

func (*Bucket) Rename Uses

func (b *Bucket) Rename(fileID interface{}, newFilename string) error

Rename renames the stored file with the specified file ID.

If this operation requires a custom write deadline to be set on the bucket, it cannot be done concurrently with other write operations operations on this bucket that also require a custom deadline

Code:

var bucket *gridfs.Bucket
var fileID primitive.ObjectID

if err := bucket.Rename(fileID, "new file name"); err != nil {
    log.Fatal(err)
}

func (*Bucket) SetReadDeadline Uses

func (b *Bucket) SetReadDeadline(t time.Time) error

SetReadDeadline sets the read deadline for this bucket

func (*Bucket) SetWriteDeadline Uses

func (b *Bucket) SetWriteDeadline(t time.Time) error

SetWriteDeadline sets the write deadline for this bucket.

func (*Bucket) UploadFromStream Uses

func (b *Bucket) UploadFromStream(filename string, source io.Reader, opts ...*options.UploadOptions) (primitive.ObjectID, error)

UploadFromStream creates a fileID and uploads a file given a source stream.

If this upload requires a custom write deadline to be set on the bucket, it cannot be done concurrently with other write operations operations on this bucket that also require a custom deadline.

Code:

var fileContent []byte
var bucket *gridfs.Bucket

// Specify the Metadata option to include a "metadata" field in the files collection document.
uploadOpts := options.GridFSUpload().SetMetadata(bson.D{{"metadata tag", "tag"}})
fileID, err := bucket.UploadFromStream("filename", bytes.NewBuffer(fileContent), uploadOpts)
if err != nil {
    log.Fatal(err)
}

fmt.Printf("new file created with ID %s", fileID)

func (*Bucket) UploadFromStreamWithID Uses

func (b *Bucket) UploadFromStreamWithID(fileID interface{}, filename string, source io.Reader, opts ...*options.UploadOptions) error

UploadFromStreamWithID uploads a file given a source stream.

If this upload requires a custom write deadline to be set on the bucket, it cannot be done concurrently with other write operations operations on this bucket that also require a custom deadline.

type DownloadStream Uses

type DownloadStream struct {
    // contains filtered or unexported fields
}

DownloadStream is a io.Reader that can be used to download a file from a GridFS bucket.

func (*DownloadStream) Close Uses

func (ds *DownloadStream) Close() error

Close closes this download stream.

func (*DownloadStream) Read Uses

func (ds *DownloadStream) Read(p []byte) (int, error)

Read reads the file from the server and writes it to a destination byte slice.

func (*DownloadStream) SetReadDeadline Uses

func (ds *DownloadStream) SetReadDeadline(t time.Time) error

SetReadDeadline sets the read deadline for this download stream.

func (*DownloadStream) Skip Uses

func (ds *DownloadStream) Skip(skip int64) (int64, error)

Skip skips a given number of bytes in the file.

type Upload Uses

type Upload struct {
    // contains filtered or unexported fields
}

Upload contains options to upload a file to a bucket.

type UploadStream Uses

type UploadStream struct {
    *Upload // chunk size and metadata
    FileID  interface{}
    // contains filtered or unexported fields
}

UploadStream is used to upload a file in chunks. This type implements the io.Writer interface and a file can be uploaded using the Write method. After an upload is complete, the Close method must be called to write file metadata.

func (*UploadStream) Abort Uses

func (us *UploadStream) Abort() error

Abort closes the stream and deletes all file chunks that have already been written.

func (*UploadStream) Close Uses

func (us *UploadStream) Close() error

Close writes file metadata to the files collection and cleans up any resources associated with the UploadStream.

func (*UploadStream) SetWriteDeadline Uses

func (us *UploadStream) SetWriteDeadline(t time.Time) error

SetWriteDeadline sets the write deadline for this stream.

func (*UploadStream) Write Uses

func (us *UploadStream) Write(p []byte) (int, error)

Write transfers the contents of a byte slice into this upload stream. If the stream's underlying buffer fills up, the buffer will be uploaded as chunks to the server. Implements the io.Writer interface.

Package gridfs imports 16 packages (graph) and is imported by 7 packages. Updated 2020-01-15. Refresh now. Tools for package owners.