scrape

package
v0.0.0-...-ad4c16b Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 18, 2017 License: MIT Imports: 14 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Many

func Many(screenNames []string, maxPerScreenName int) (TweetCollection, *MultiError)

Many returns the most recent tweets for a given collection of screen names

func Run

func Run()

Run is a function to trigger command line functionality

Types

type MultiError

type MultiError struct {
	Errors []error
}

MultiError is a collection of errors that may have occurred during a parallel operation

func (*MultiError) Error

func (e *MultiError) Error() string

type Tweet

type Tweet struct {
	IDStr      string
	ScreenName string
	CreatedAt  time.Time
	Body       string
}

Tweet is a struct representing a scraped tweet

type TweetCollection

type TweetCollection []Tweet

TweetCollection is a slice of tweets with extra functionality

func Single

func Single(screenName string, maxTweets int) (collection TweetCollection, err error)

Single scrapes most recent tweets from a single user, up to a maximum

func (TweetCollection) Oldest

func (tc TweetCollection) Oldest() *Tweet

Oldest returns the oldest tweet in the collection

func (TweetCollection) Subtract

func (tc TweetCollection) Subtract(other TweetCollection) TweetCollection

Subtract subtracts the second collection from the first in a set-like manner

func (TweetCollection) Unique

func (tc TweetCollection) Unique() TweetCollection

Unique returns a copy collection that contains only tweets with unique IDs

type TweetCollectionExportJSON

type TweetCollectionExportJSON struct {
	ExportTimestamp int64          `json:"export_timestamp"`
	UserCounts      map[string]int `json:"user_counts"`
	Tweets          []TweetJSON    `json:"tweets"`
}

TweetCollectionExportJSON is the marshaling structure for a tweet collection

type TweetJSON

type TweetJSON struct {
	IDStr      string `json:"id_str"`
	ScreenName string `json:"screen_name"`
	CreatedAt  int64  `json:"created_at"`
	Body       string `json:"body"`
}

TweetJSON is the marshaling structure for a tweet

Directories

Path Synopsis
cmd

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL