gdrive

package module
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 4, 2021 License: MIT Imports: 20 Imported by: 0

README

Afero Google Drive Driver

Build codecov Go Report Card GoDoc

About

It provides an afero filesystem implementation of a Google Drive backend.

This was created to provide a backend to the ftpserver but can definitely be used in any other code.

I'm very opened to any improvement through issues or pull-request that might lead to a better implementation or even better testing.

Key points

  • Download & upload file streaming
  • 60% coverage: This isn't great, I intend to improve it to reach 80%. As it's a third-party API more would take way too much time.
  • Very carefully linted

Known limitations

  • File appending / seeking for write is not supported because Google Drive doesn't support it, it could be simulated by rewriting entire files.
  • Chmod is saved as a property and not used at this time.
  • No files listing cache. This means that every directory of a path results in a request that happens every single time a path is analyzed. In other word opening /a/b/c/file.txt for writing will create at least 4 request, and the same will happen for reading.

How to use

Note: Errors handling is skipped for brevity but you definitely have to handle it.


import (
	"github.com/fclairamb/afero-gdrive/oauthhelper"
)

func main() {
  // We declare the OAuh2 app
  helper := oauthhelper.Auth{
    ClientID:     os.Getenv("GOOGLE_CLIENT_ID"),
    ClientSecret: os.Getenv("GOOGLE_CLIENT_SECRET"),
    Authenticate: func(url string) (string, error) {
     return "", ErrNotSupported
    },
  }

  // Pass a token 
  token, _ := base64.StdEncoding.DecodeString(os.Getenv("GOOGLE_TOKEN"))
  helper.Token = new(oauth2.Token)
  json.Unmarshal(token, helper.Token)
	
  // Initialize the authenticated client
  client, _ := helper.NewHTTPClient(context.Background())
  
  // Initialize the FS from the athenticated http client
  fs, _ := New(client)

  // And use it
  file, _ := fs.OpenFile("my_file.txt", os.O_WRONLY, 0777)
  file.WriteString("Hello world !")
  file.Close()
}

Credits

This is a fork from T4cC0re/gdriver which is itself a fork of eun/gdriver.

The code was massively modified. Most of the changes are improvements. The file listing API from the original implemented based on callbacks is much better though, but this was needed to respect the afero API.

Documentation

Overview

Package gdrive provides an afero Fs interface to Google Drive API

Index

Constants

This section is empty.

Variables

View Source
var ErrEmptyPath = errors.New("path cannot be empty")

ErrEmptyPath is returned when an empty path is sent

View Source
var ErrForbiddenOnRoot = errors.New("forbidden for root directory")

ErrForbiddenOnRoot is returned when an operation is performed on the root node

View Source
var ErrInvalidSeek = errors.New("invalid seek offset")

ErrInvalidSeek is returned when the seek operation is not doable

View Source
var ErrNotImplemented = errors.New("not implemented")

ErrNotImplemented is returned when this operation is not (yet) implemented

View Source
var ErrNotSupported = errors.New("google drive doesn't support this operation")

ErrNotSupported is returned when this operations is not supported by Google Drive

View Source
var ErrOpenMissingFlag = errors.New("you need to specify a read or write flag")

ErrOpenMissingFlag is returned when neither read nor write flags are passed

View Source
var ErrReadAndWriteNotSupported = errors.New("option O_RDWR is not supported")

ErrReadAndWriteNotSupported is returned when the O_RDWR flag is passed

View Source
var ErrReadOnly = errors.New("we're in a read-only mode")

ErrReadOnly means a write operation was performed on a file opened in read-only

View Source
var ErrUnknownBufferType = errors.New("unknown buffer type")

ErrUnknownBufferType is returned when a un unknown buffer is specified

View Source
var ErrWriteOnly = errors.New("we're in write-only mode")

ErrWriteOnly means a write operation was performed on a file opened in write-only

Functions

func IsNotExist

func IsNotExist(e error) bool

IsNotExist returns true if the error is an FileNotExistError

Types

type APIWrapper

type APIWrapper struct {
	UseCache bool
	// contains filtered or unexported fields
}

APIWrapper allows to wrap some GDrive API calls to perform some caching

func NewAPIWrapper

func NewAPIWrapper(srv *drive.Service, logger log.Logger) *APIWrapper

NewAPIWrapper instantiates a new APIWrapper

func (*APIWrapper) TotalNbCalls

func (a *APIWrapper) TotalNbCalls() int

TotalNbCalls returns the total number of calls performed to the API

type DriveAPICallError

type DriveAPICallError struct {
	Err error
}

DriveAPICallError wraps an error that was returned by the Google Drive API

func (*DriveAPICallError) Error

func (e *DriveAPICallError) Error() string

func (*DriveAPICallError) Unwrap

func (e *DriveAPICallError) Unwrap() error

Unwrap exposes the Google Drive API returned error

type DriveStreamError

type DriveStreamError struct {
	Err error
}

DriveStreamError wraps an error that happened while using a stream opened from the Google Drive API

func (*DriveStreamError) Error

func (e *DriveStreamError) Error() string

func (*DriveStreamError) Unwrap

func (e *DriveStreamError) Unwrap() error

Unwrap exposes the underlying stream error

type File

type File struct {
	*FileInfo        // FileInfo contains the core fileInfo
	Path      string // Path is the complete path of hte file
	// contains filtered or unexported fields
}

File represents the managed file structure

func (*File) AsAfero

func (f *File) AsAfero() afero.File

AsAfero provides a cast to afero interface for easier testing

func (*File) Close

func (f *File) Close() error

Close closes the file This marks the end of the file write.

func (*File) Read

func (f *File) Read(p []byte) (int, error)

func (*File) ReadAt

func (f *File) ReadAt(p []byte, off int64) (n int, err error)

ReadAt reads a file at a specific offset

func (*File) Readdir

func (f *File) Readdir(count int) ([]os.FileInfo, error)

Readdir provides a list of file information

func (*File) Readdirnames

func (f *File) Readdirnames(n int) ([]string, error)

Readdirnames provides a list of directory names

func (*File) Seek

func (f *File) Seek(offset int64, whence int) (int64, error)

Seek sets the offset for the next Read or Write to offset

func (*File) Stat

func (f *File) Stat() (os.FileInfo, error)

Stat provides stat file information

func (*File) Sync

func (f *File) Sync() error

Sync forces a file synchronization. This has no effect here.

func (*File) Truncate

func (f *File) Truncate(int64) error

Truncate should truncate a file to a specific size. But this method is not supported by the google drive API.

func (*File) Write

func (f *File) Write(p []byte) (int, error)

func (*File) WriteAt

func (f *File) WriteAt(p []byte, off int64) (n int, err error)

WriteAt writes some bytes at a specified offset

func (*File) WriteString

func (f *File) WriteString(s string) (ret int, err error)

WriteString writes a string

type FileExistError

type FileExistError struct {
	Path string
}

FileExistError will be thrown if an File exists

func (FileExistError) Error

func (e FileExistError) Error() string

type FileHasMultipleEntriesError

type FileHasMultipleEntriesError struct {
	Path string
}

FileHasMultipleEntriesError will be returned when the same file name is present multiple times in the same directory.

func (FileHasMultipleEntriesError) Error

type FileInfo

type FileInfo struct {
	// contains filtered or unexported fields
}

FileInfo represents File information for a File or directory

func (*FileInfo) CreateTime

func (i *FileInfo) CreateTime() time.Time

CreateTime returns the time when this File was created

func (*FileInfo) DriveFile

func (i *FileInfo) DriveFile() *drive.File

DriveFile returns the underlaying drive.File

func (*FileInfo) IsDir

func (i *FileInfo) IsDir() bool

IsDir returns true if this File is a directory

func (*FileInfo) ModTime

func (i *FileInfo) ModTime() time.Time

ModTime returns the modification time

func (*FileInfo) Mode

func (i *FileInfo) Mode() os.FileMode

Mode returns the file mode bits

func (*FileInfo) Name

func (i *FileInfo) Name() string

Name returns the name of the File or directory

func (*FileInfo) ParentPath

func (i *FileInfo) ParentPath() string

ParentPath returns the parent path of the File or directory

func (*FileInfo) Path

func (i *FileInfo) Path() string

Path returns the full path to this File or directory

func (*FileInfo) Size

func (i *FileInfo) Size() int64

Size returns the bytes for this File

func (*FileInfo) Sys

func (i *FileInfo) Sys() interface{}

Sys provides underlying data source

type FileIsDirectoryError

type FileIsDirectoryError struct {
	Path string
}

FileIsDirectoryError will be thrown if a File is a directory

func (FileIsDirectoryError) Error

func (e FileIsDirectoryError) Error() string

type FileIsNotDirectoryError

type FileIsNotDirectoryError struct {
	Fi   *FileInfo
	Path string
}

FileIsNotDirectoryError will be thrown if a File is not a directory

func (FileIsNotDirectoryError) Error

func (e FileIsNotDirectoryError) Error() string

type FileNotExistError

type FileNotExistError struct {
	Path string
}

FileNotExistError will be thrown if a File was not found

func (FileNotExistError) Error

func (e FileNotExistError) Error() string

type GDriver

type GDriver struct {
	Logger              log.Logger
	LogReaderAndWriters bool
	TrashForDelete      bool
	WriteBufferType     WriteBufferType
	WriteBufferSize     int
	// contains filtered or unexported fields
}

GDriver can be used to access google drive in a traditional File-folder-path pattern

func New

func New(client *http.Client, opts ...Option) (*GDriver, error)

New creates a new Google Drive driver, client must me an authenticated instance for google drive

func (*GDriver) AsAfero

func (d *GDriver) AsAfero() afero.Fs

AsAfero provides a cast to afero interface for easier testing

func (*GDriver) Chmod

func (d *GDriver) Chmod(path string, mode os.FileMode) error

Chmod changes the mode of the named file to mode.

func (*GDriver) Chown

func (d *GDriver) Chown(string, int, int) error

Chown changes the ownership of a file

func (*GDriver) Chtimes

func (d *GDriver) Chtimes(path string, atime time.Time, mTime time.Time) error

Chtimes changes the access and modification times of the named file

func (*GDriver) Create

func (d *GDriver) Create(name string) (afero.File, error)

Create creates a file in the filesystem, returning the file and an error, if any happens.

func (*GDriver) DeleteDirectory

func (d *GDriver) DeleteDirectory(path string) error

DeleteDirectory will delete a directory and its descendants

func (*GDriver) ListTrash

func (d *GDriver) ListTrash(filePath string, _ int) ([]*FileInfo, error)

ListTrash lists the contents of the trash if you specify directories it will only list the trash contents of the specified directories

func (*GDriver) Mkdir

func (d *GDriver) Mkdir(path string, perm os.FileMode) error

Mkdir creates a directory in the filesystem, return an error if any happens.

func (*GDriver) MkdirAll

func (d *GDriver) MkdirAll(path string, _ os.FileMode) error

MkdirAll creates a directory path and all parents that does not exist yet.

func (*GDriver) Name

func (d *GDriver) Name() string

Name provides the name of this filesystem

func (*GDriver) Open

func (d *GDriver) Open(name string) (afero.File, error)

Open a File for reading.

func (*GDriver) OpenFile

func (d *GDriver) OpenFile(path string, flag int, _ os.FileMode) (afero.File, error)

OpenFile opens a File in the traditional os.Open way

func (*GDriver) Remove

func (d *GDriver) Remove(path string) error

Remove removes a file identified by name, returning an error, if any happens.

func (*GDriver) RemoveAll

func (d *GDriver) RemoveAll(path string) error

RemoveAll will delete a File or directory, if directory it will also delete its descendants

func (*GDriver) Rename

func (d *GDriver) Rename(oldPath, newPath string) error

Rename moves a File or directory to a new path

func (*GDriver) SetDrive added in v0.2.8

func (d *GDriver) SetDrive(drive string) error

Set the drive to act on

func (*GDriver) SetRootDirectory

func (d *GDriver) SetRootDirectory(path string) (*FileInfo, error)

SetRootDirectory changes the working root directory use this if you want to do certain operations in a special directory path should always be the absolute real path

func (*GDriver) Stat

func (d *GDriver) Stat(path string) (os.FileInfo, error)

Stat gives a FileInfo for a File or directory

type HashMethod

type HashMethod int

HashMethod is the hashing method to use for GetFileHash

type NoFileInformationError

type NoFileInformationError struct {
	Fi   *FileInfo
	Path string
}

NoFileInformationError is returned when a given directory didn't provide any file info. This error is bit confusing and needs reviewing.

func (NoFileInformationError) Error

func (e NoFileInformationError) Error() string

type Option

type Option func(driver *GDriver) error

Option can be used to pass optional Options to GDriver

func RootDirectory

func RootDirectory(path string) Option

RootDirectory sets the root directory for all operations

func RootDrive added in v0.2.9

func RootDrive(drive string, path string) Option

Sets the drive to act on

type WriteBufferType

type WriteBufferType string

WriteBufferType defines the type of buffer we want to use to read & write files

const (
	// WriteBufferNone means no buffer
	WriteBufferNone WriteBufferType = ""
	// WriteBufferSimple means a simple io.Buffer
	WriteBufferSimple WriteBufferType = "simple"
	// WriteBufferAsync means an asynchronous io.Buffer
	WriteBufferAsync WriteBufferType = "async"
	// WriteBufferChan means an asynchronous channel-based set of buffers
	WriteBufferChan WriteBufferType = "chan"
)

Directories

Path Synopsis
Package cache allows to
Package cache allows to
Package iohelper should brings tools to help manage IOs
Package iohelper should brings tools to help manage IOs
log
Package log provides a simple interface to handle logging
Package log provides a simple interface to handle logging
gokit
Package gokit defines a simple implementation of the logging interface
Package gokit defines a simple implementation of the logging interface
Package oauthhelper provide an OAuth2 authentication and token management helper
Package oauthhelper provide an OAuth2 authentication and token management helper
This program helps the setup of credentials for tests
This program helps the setup of credentials for tests

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL