store

package
v0.2.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 13, 2023 License: AGPL-3.0 Imports: 10 Imported by: 0

Documentation

Overview

Package store contains functions for writing parsed feeds to cloud storage.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func GetSession 

func GetSession(creds *AWSCredentials) (sesh *session.Session, err error)

GetSession is a helper function that returns an AWS session that can be reused for multiple writes. It takes in a pointer to an AWSCredentials struct and returns a pointer to an AWS connection and a nil, else a nil and the connection error.

func MustOpen 

func MustOpen(path string) *leveldb.DB

MustOpen either initializes a new leveldb database at the path of the string provided or opens an existing leveldb database found at that path.

func Upload 

func Upload(s *session.Session, doc Document, bucket string) error

Upload a single Document to the specified S3 bucket using the established AWS session, setting file information including name (the feedID, language code, year, month, day, and hash of the content), the content size and type, and the encryption on the uploaded file.

func VerifyCredentials 

func VerifyCredentials(creds *AWSCredentials) bool

VerifyCredentials is a helper function that verifies the credentials are correct and the bucket exists, returning true if so else false.

Types

type AWSCredentials 

type AWSCredentials struct {
	Region string
	Bucket string
}

AWSCredentials stores the region and bucket needed to access an S3 bucket.

type Document 

type Document struct {
	FeedID       string
	LanguageCode string
	Year         int
	Month        string
	Day          int
	Title        string
	Description  string
	Content      []byte
	Encoding     string `json:",omitempty"`
	Link         string
}

A Document is the generic representation of an individual entry in a feed

type Feed 

type Feed struct {
	URL    string `json:"url"`
	Active bool   `json:"active"`
	Error  string `json:"error"`
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.