galaxy_fds_sdk_golang

package module
v0.0.0-...-37dfdd7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 19, 2020 License: Apache-2.0 Imports: 21 Imported by: 0

README

Warning

这个FDS client进入维护阶段，不会再添加新的功能，仅修复重大bug。

对于新用户，请使用: github.com/XiaoMi/go-fds.

对于老用户，如果有深度使用fds go client的需求，强烈建议迁移到新版。

旧版本存在一下问题：

  1. 代码结构不清晰，不利于维护
  2. 不支持流式接口，也很难支持
  3. 不支持Context
  4. 未遵循Go的一些默认代码规范

galaxy-fds-sdk-golang

Golang SDK for Xiaomi File Data Storage.

install

内部用户请不要使用go get 方式获取，直接克隆代码即可

go get github.com/XiaoMi/galaxy-fds-sdk-golang

#example

package main

import (
	fds "github.com/XiaoMi/galaxy-fds-sdk-golang"
	"log"
	"os"
)

const (
	BUCKET_NAME = "test"
	APP_KEY     = "APP_KEY"
	SECRET_KEY  = "SECRET_KEY"
	REGION_NAME = "cnbj0" // region
	END_POINT   = "" // fds domain
)

func main() {
	objectName := "test_object"

	client := fds.NEWFDSClient(APP_KEY, SECRET_KEY, REGION_NAME, END_POINT, false, false)
	client.Create_Bucket(BUCKET_NAME)
	content := []byte("object content data")
	result, err := client.Put_Object(BUCKET_NAME, objectName, content, "", nil)
	if err != nil {
		log.Println(err)
		os.Exit(1)
	}
	log.Println(result.ObjectName)

	fdsObject, err := client.Get_Object(BUCKET_NAME, objectName, 0, -1)
	if err != nil {
		log.Println(err)
		os.Exit(2)
	}
	log.Println(string(fdsObject.ObjectContent))
}

Changes

20170822:

  1. 修正README.md example中的错误
  2. fix因为struct内变量名字引起的multi upload失败问题
  3. 使用go fmt 工具格式化所有代码，使代码风格一致

20170823

  1. 修复get_object代码中存在的逻辑漏洞

20170829

  1. 添加方法注释：List_Bucket
  2. 方法 List_Authorized_Buckets 实现并添加该方法注释
  3. 方法 List_Trash_Object 实现并添加该方法注释
  4. fix List_Bucket方法因返回值为空字符串导致json解析失败的问题

20170831

  1. 添加下载Object到本地的SDK调用接口，Download_Object
  2. 添加Download_Object_With_Uri接口
  3. 添加Get_Object_With_Uri 接口

20170906

  1. 添加Put_Object_With_Uri调用接口

20170907

  1. 添加Get_Object_Acl调用接口
  2. 修复NewFDSObjectListing中因为"lastModified":null字段产生的bug
  3. 修复Grantee struct json注释bug
  4. 添加新接口Set_Object_Acl_New，原Set_Object_Acl接口保留，使用Set_Object_Acl_New更规范ACL的定义
  5. 添加Get_Object_ACL接口
  6. 添加Delete_Object_ACL 接口
  7. 添加Set_Bucket_ACL 接口
  8. 添加Delete_Bucket_ACL 接口
  9. 添加Get_Bucket_ACL 接口
  10. 添加Generate_Download_Object_Uri 接口

20171013

  1. 修复readme中的example code中的bug
  2. 修复 Test/FDSClient_test.go 中的bug
  3. 添加Get_Bucket 接口，但是该接口与java中语义不同，请谨慎使用，java中该方法不返回任何值

20180612

  1. 修复download中的分片错误，该错误会导致小于50M的文件无法下载

Documentation

Index

Constants

View Source 
const (
	URI_CDN                            = "cdn"
	URI_FDS_SUFFIX                     = ".fds.api.xiaomi.com/"
	URI_FDS_CDN_SUFFIX                 = ".fds.api.mi-img.com/"
	URI_HTTP_PREFIX                    = "http://"
	URI_HTTPS_PREFIX                   = "https://"
	USER_DEFINED_METADATA_PREFIX       = "x-xiaomi-meta-"
	DELIMITER                          = "/"
	DEFAULT_LIST_MAX_KEYS              = 1000
	GALAXY_ACCESS_KEY_ID               = "GalaxyAccessKeyId"
	EXPIRES                            = "Expires"
	SIGNATURE                          = "Signature"
	SLICE_SIZE                   int64 = 52428800 // 50*1024*1024 下载时分片上限为50MB，低于50MB不可分片
)
View Source 
const (
	PERMISSION_READ         = "READ"
	PERMISSION_WRITE        = "WRITE"
	PERMISSION_FULL_CONTROL = "FULL_CONTROL"
	PERMISSION_USER         = "USER"
	PERMISSION_GROUP        = "GROUP"
)

permission

View Source 
const (
	REGION_CNBJ0    = "cnbj0"
	REGION_CNBJ1    = "cnbj1"
	REGION_CNBJ2    = "cnbj2"
	REGION_AWSBJ0   = "awsbj0"
	REGION_AWSUSOR0 = "awsusor0"
	REGION_AWSSGP0  = "awssgp0"
	REGION_AWSDE0   = "awsde0"
)

Variables

View Source 
var ALL_USERS = map[string]string{"id": "ALL_USERS"}
View Source 
var AUTHENTICATED_USERS = map[string]string{"id": "AUTHENTICATED_USERS"}
View Source 
var PRE_DEFINED_METADATA = []string{"cache-control",
	"content-encoding",
	"content-length",
	"content-md5",
	"content-type",
}
View Source 
var SUB_RESOURCE_MAP = map[string]string{
	"acl":                "",
	"quota":              "",
	"uploads":            "",
	"partNumber":         "",
	"uploadId":           "",
	"storageAccessToken": "",
	"metadata":           "",
}

Functions

func Signature 

func Signature(app_secret, method, u string, headers map[string][]string) (string, error)

func Uri_To_Bucket_And_Object 

func Uri_To_Bucket_And_Object(uri string) (string, string)

name: 

Uri_To_Bucket_And_Object
将指定Uri转换成bucketname和objectname

param: 

url: 要下载的object的uri地址，格式为：fds://bucketname/objectname

return: 

string: bucket name
string: object name

example: 

Not available now

Types

type FDSAuth 

type FDSAuth struct {
	UrlBase      string
	Method       string
	Data         []byte
	Content_Md5  string
	Content_Type string
	Headers      *map[string]string
	Params       *map[string]string
}

type FDSClient 

type FDSClient struct {
	AppKey      string
	AppSecret   string
	RegionName  string
	EndPoint    string
	EnableHttps bool
	EnableCDN   bool
}

func NEWFDSClient 

func NEWFDSClient(appkey, appSecret, regionName string, endPoint string, enableHttps, enableCDN bool) *FDSClient

func (*FDSClient) Abort_MultipartUpload 

func (c *FDSClient) Abort_MultipartUpload(initPartuploadResult *Model.InitMultipartUploadResult) error

func (*FDSClient) Auth 

func (c *FDSClient) Auth(auth FDSAuth) (*http.Response, error)

func (*FDSClient) CompleteMultipartUpload 

func (c *FDSClient) CompleteMultipartUpload(initResult *Model.InitMultipartUploadResult,
	metadata http.Header, uploadPartResultList *Model.UploadPartList) (*Model.PutObjectResult, error)

CompleteMultipartUpload completes multipart upload

func (*FDSClient) Complete_Multipart_Upload 

func (c *FDSClient) Complete_Multipart_Upload(initPartuploadResult *Model.InitMultipartUploadResult,
	uploadPartResultList *Model.UploadPartList) (*Model.PutObjectResult, error)

Complete_Multipart_Upload completes multipart upload Deprecated: Use CompleteMultipartUpload

func (*FDSClient) Create_Bucket 

func (c *FDSClient) Create_Bucket(bucketname string) (bool, error)

func (*FDSClient) Delete_Bucket 

func (c *FDSClient) Delete_Bucket(bucketname string) (bool, error)

func (*FDSClient) Delete_Bucket_ACL 

func (c *FDSClient) Delete_Bucket_ACL(bucketname string, acl Model.ACL) (bool, error)

name: 

Delete_Bucket_ACL

description: 

删除指定bucket的ACL规则

param: 

bucketname:  要删除ACL的bucket
acl:         要删除的ACL

return: 

bool:  如果执行正常则返回true，发生错误是返回false
error: 正常返回nil，异常返回error Code

example: 

Not available now

func (*FDSClient) Delete_Object 

func (c *FDSClient) Delete_Object(bucketname, objectname string) (bool, error)

func (*FDSClient) Delete_Object_ACL 

func (c *FDSClient) Delete_Object_ACL(bucketname, objectname string, acl Model.ACL) (bool, error)

name: 

Delete_Object_ACL

description: 

删除指定Object的ACL规则

param: 

bucketname:  要获取ACL的Object所在的bucket
objectname:  要回去ACL的Object
acl:         要删除的ACL

return: 

bool:  如果执行正常则返回true，发生错误是返回false
error: 正常返回nil，异常返回error Code

example: 

Not available now

func (*FDSClient) Delete_Objects 

func (c *FDSClient) Delete_Objects(bucketname string, prefix []string) error

func (*FDSClient) Delete_Objects_With_Prefix 

func (c *FDSClient) Delete_Objects_With_Prefix(bucketname, prefix string) error

func (*FDSClient) Download_Object 

func (c *FDSClient) Download_Object(bucketname, objectname, filename string) (*string, error)

name: 

Download_Object
将指定的object下载到本地文件，并不会判断本地是否存在同名文件，如果存在同名文件则文件内容会被重写，文件inode可能并不会改变,因为文件

param: 

bucketname: 使用的bucket的名字
objectname: 要下载的object的名字
filename:   本地要写入的文件名字

return: 

*string: 返回服务器端该文件的md5值，用户校验本地文件是否完整
error: 正常返回nil，异常返回error Code

example: 

Not available now

func (*FDSClient) Download_Object_With_Uri 

func (c *FDSClient) Download_Object_With_Uri(url, filename string) (*string, error)

name: 

Download_Object_With_Uri
将指定Uri的object下载到本地文件，并不会判断本地是否存在同名文件，如果存在同名文件则文件内容会被重写，文件inode可能并不会改变,因为文件

param: 

url: 要下载的object的uri地址，格式为：fds://bucketname/objectname
filename:   本地要写入的文件名字

return: 

string: bucketname
string: objectname

example: 

Not available now

func (*FDSClient) GeneratePresignedURI 

func (c *FDSClient) GeneratePresignedURI(bucketname, objectname, method string, subResources []string, expiration int64, headers map[string][]string) (string, error)

GeneratePresignedURI generates presigned uri for all actions

func (*FDSClient) Generate_Download_Object_Uri 

func (c *FDSClient) Generate_Download_Object_Uri(bucketname, objectname string) string

name: 

Generate_Download_Object_Uri

description: 

生成object的下载链接

param: 

bucketname： 要生成链接的object原来所在的bucket
objectname： 要生成链接的object名字

return: 

string: 返回指定object的下载地址

example: 

Not available now

func (*FDSClient) Generate_Presigned_URI 

func (c *FDSClient) Generate_Presigned_URI(bucketname, objectname, method string, expiration int64, headers map[string][]string) (string, error)

Generate_Presigned_URI generates presigned uri Deprecated: Use GeneratePresignedURI

func (*FDSClient) GetBaseUri 

func (c *FDSClient) GetBaseUri() string

func (*FDSClient) GetUploadURL 

func (c *FDSClient) GetUploadURL() string

func (*FDSClient) Get_Bucket 

func (c *FDSClient) Get_Bucket(bucketname string) (*Model.BucketInfo, error)

name: 

Get_Bucket

param: 

bucketname: 要获取信息的bucket名字

return: 

*Model.BucketInfo: 由bucketinfo结构体，包含bucket相关信息
error: 正常返回nil，异常返回error Code

example: 

Not available now

Exception: 这个接口跟java中定义不同，请谨慎使用，java中不返回任何值

func (*FDSClient) Get_Bucket_ACL 

func (c *FDSClient) Get_Bucket_ACL(bucketname string) (*Model.ACL, error)

name: 

Get_Bucket_ACL

description: 

获取指定bucket的ACL规则

param: 

bucketname:  要获取ACL的bucket

return: 

bool:  如果执行正常则返回true，发生错误是返回false
error: 正常返回nil，异常返回error Code

example: 

Not available now

func (*FDSClient) Get_Object 

func (c *FDSClient) Get_Object(bucketname, objectname string, position int64, size int64) (*Model.FDSObject, error)

name: 

Get_Object
获取指定的object

param: 

bucketname: 要获取object所属的bucketname
objectname: 要获取object的name
position:   指定要获取object的其实位置
size:       指定要获取object内容的大小，-1位最大，值必须为正数

return: 

*FDSObject: 返回与object相关信息
error:      正常返回nil，异常返回error Code

example: 

Not available now

func (*FDSClient) Get_Object_ACL 

func (c *FDSClient) Get_Object_ACL(bucketname, objectname string) (*Model.ACL, error)

name: 

Get_Object_ACL

description: 

获取指定Object的ACL规则

param: 

bucketname:  要获取ACL的Object所在的bucket
objectname:  要回去ACL的Object

return: 

*Model.ACL:  获取到ACL结构体
error: 正常返回nil，异常返回error Code

example: 

Not available now

func (*FDSClient) Get_Object_Meta 

func (c *FDSClient) Get_Object_Meta(bucketname, objectname string) (*Model.FDSMetaData, error)

func (*FDSClient) Get_Object_Reader 

func (c *FDSClient) Get_Object_Reader(bucketname, objectname string, position int64, size int64) (*io.ReadCloser, error)

func (*FDSClient) Get_Object_With_Uri 

func (c *FDSClient) Get_Object_With_Uri(uri string, position, size int64) (*Model.FDSObject, error)

name: 

Get_Object_With_Uri
获取使用uri标识的object

param: 

uri:      包含要获取object名字的uri，格式为：fds://
position: 指定获取object的起始位置
size:     指定要获取object内容的大小

return: 

*FDSObject: 返回与object相关信息
error:      正常返回nil，异常返回error Code

example: 

Not available now

func (*FDSClient) InitMultipartUpload 

func (c *FDSClient) InitMultipartUpload(bucketName, objectName string, metadata http.Header) (*Model.InitMultipartUploadResult, error)

InitMultipartUpload is a new interface to init multipart upload, I can not save this this library

func (*FDSClient) Init_MultiPart_Upload 

func (c *FDSClient) Init_MultiPart_Upload(bucketname, objectname string, contentType string) (*Model.InitMultipartUploadResult, error)

func (*FDSClient) Is_Bucket_Exists 

func (c *FDSClient) Is_Bucket_Exists(bucketname string) (bool, error)

func (*FDSClient) Is_Object_Exists 

func (c *FDSClient) Is_Object_Exists(bucketname, objectname string) (bool, error)

func (*FDSClient) List_Authorized_Buckets 

func (c *FDSClient) List_Authorized_Buckets() ([]string, error)

name: 

List_Authorized_Buckets

param: 

默认不使用任何参数，使用用户配置的AK SK，列出由该AK SK所属用户组所有被授权的bucket

return: 

[]string: 由bucket name组成的slice
error: 正常返回nil，异常返回error Code

example: 

Not available now

func (*FDSClient) List_Bucket 

func (c *FDSClient) List_Bucket() ([]string, error)

name: 

List_Bucket

param: 

默认不使用任何参数，使用用户配置的AK SK，列出由该AK SK所属用户组创建的bucket

return: 

[]string: 由bucket name组成的slice
error: 正常返回nil，异常返回error Code

example: 

Not available now

func (*FDSClient) List_Multipart_Uploads 

func (c *FDSClient) List_Multipart_Uploads(bucketName, prefix,
	delimiter string, maxKeys int) (*Model.FDSListMultipartUploadsResult, error)

func (*FDSClient) List_Next_Batch_Of_Objects 

func (c *FDSClient) List_Next_Batch_Of_Objects(previous *Model.FDSObjectListing) (*Model.FDSObjectListing, error)

func (*FDSClient) List_Object 

func (c *FDSClient) List_Object(bucketname, prefix, delimiter string, maxKeys int) (*Model.FDSObjectListing, error)

prefix需要改进

func (*FDSClient) List_Parts 

func (c *FDSClient) List_Parts(bucketName, objectName, uploadId string) (*Model.UploadPartList, error)

func (*FDSClient) List_Trash_Object 

func (c *FDSClient) List_Trash_Object(prefix, delimiter string, maxKeys int) (*Model.FDSObjectListing, error)

name: 

List_Trash_Object

description: 

列出用户配置的AK SK，列出由该AK SK所在org下响应bucket中被删除的object

param: 

prefix: The prefix of bucket_name/object_name
delimiter: The delimiter used in listing
maxKeys: 每次获取最大的bucket数量

return: 

*Model.FDSObjectListing: 包含object信息的结构体
error: 正常返回nil，异常返回error Code

example: 

Not available now

func (*FDSClient) Post_Object 

func (c *FDSClient) Post_Object(bucketname string, data []byte, filetype string) (string, error)

v1类型：objectname由服务端随机生成唯一名字

func (*FDSClient) Prefetch_Object 

func (c *FDSClient) Prefetch_Object(bucketname, objectname string) (bool, error)

func (*FDSClient) Put_Object 

func (c *FDSClient) Put_Object(bucketname string, objectname string,
	data []byte, contentType string,
	headers *map[string]string) (*Model.PutObjectResult, error)

v2类型 自定义文件名 如果object已存在，将会覆盖

func (*FDSClient) Put_Object_With_Uri 

func (c *FDSClient) Put_Object_With_Uri(url string, data []byte, contentType string,
	headers *map[string]string) (*Model.PutObjectResult, error)

name: 

Put_Object_With_Uri

description: 

通过用户指定的URI上传object

param: 

url:         上传object所使用的uri
data:        object内容
contentType: object内容类型
headers:     头信息

return: 

*Model.PutObjectResult: 上传结果信息
error: 正常返回nil，异常返回error Code

example: 

Not available now

func (*FDSClient) Refresh_Object 

func (c *FDSClient) Refresh_Object(bucketname, objectname string) (bool, error)

func (*FDSClient) Rename_Object 

func (c *FDSClient) Rename_Object(bucketname, src_objectname, dst_objectname string) (bool, error)

func (*FDSClient) Restore_Object 

func (c *FDSClient) Restore_Object(bucketname, objectname string) error

name: 

Restore_Object

description: 

恢复指定删除的object

param: 

bucketname： 被删除object原来所在的bucket
objectname： 被删除的object名字

return: 

error: 正常返回nil，异常返回error Code

example: 

Not available now

TODO 防止restore时替换掉原来的Object的冲突检测

func (*FDSClient) SetObjectMetadata 

func (c *FDSClient) SetObjectMetadata(bucketname string, objectname string, metadata Model.FDSMetaData) (bool, error)

SetObjectMetadata method will change object's metadata without puting object

func (*FDSClient) Set_Bucket_ACL 

func (c *FDSClient) Set_Bucket_ACL(bucketname string, acl Model.ACL) (bool, error)

name: 

Set_Bucket_ACL

description: 

设置指定bucket的ACL规则

param: 

bucketname:  要修改ACL规则的bucket
acl:         ACL规则

return: 

bool:  如果执行正常则返回true，发生错误是返回false
error: 正常返回nil，异常返回error Code

example: 

Not available now

func (*FDSClient) Set_Object_Acl 

func (c *FDSClient) Set_Object_Acl(bucketname, objectname string, acl map[string]interface{}) (bool, error)

func (*FDSClient) Set_Object_Acl_New 

func (c *FDSClient) Set_Object_Acl_New(bucketname, objectname string, acl Model.ACL) (bool, error)

name: 

Set_Object_Acl_New

description: 

修改指定object的ACL规则

param: 

bucketname:  要修改ACL的object所在的bucket
objectname:  要修改ACL的object
acl:         要设置的ACL规则

return: 

bool:  如果执行正常则返回true，发生错误是返回false
error: 正常返回nil，异常返回error Code

example: 

Not available now

func (*FDSClient) Set_Public 

func (c *FDSClient) Set_Public(bucketname, objectname string, disable_prefetch bool) (bool, error)

func (*FDSClient) Upload_Part 

func (c *FDSClient) Upload_Part(initUploadPartResult *Model.InitMultipartUploadResult, partnumber int, data []byte) (*Model.UploadPartResult, error)

Source Files

View all Source files

Directories

Path Synopsis
examples
01-put-object
02-multipart-upload
03-presigned-url

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.