faapi

Overview ¶

Package faapi uses GET requests and parsing to get metadata from the main sections of FurAffinity. These include submissions, user-pages, galleries and searches.

The package loads cookies from a JSON formatted file and will return structs with the requested metadata.

Types and Values ¶

The main type in the package is faapi.FAAPI.

type FAAPI struct {
  Cookies []*http.Cookie
  Request *http.Request
  Interval time.Duration
}

The Cookies and Request fields are assigned using faapi.LoadCookies() and faapi.MakeRequest() methods. The thus assigned cookies and request will be used by the rest of the package to perform the GET requests. The Interval field specifies how long to wait between requests (i.e. throttle).

The other two types are used to return parsed metadata from the requests.

type Submission struct {
  ID          int
  Title       string
  Author      string
  Date        string
  Tags        []string
  Description string
  FileURL     string
  File        []byte
}

type User struct {
  Username string
  Userid   string
  Title    string
  Status   string
  Date     string
  Profile  string
}

The field names describe their content. For Submission.ID an int type has been used instead of string because the a submission ID on FA can only be a number, the other fields use simple strings (or slices of them) for easier handling.

The Submission type contains the metadata that the package can parse from the submission page. The date is the only modified field: the package converts from the "Mmm D, YYYY" format to "YYYY-MM-DD" for easier sorting and reformatting. Tags are stored in a non-sorted slice.

The Description field contains the HTML source of the submission description, directly extracted from the response. The only edit the package performs is to add "http:" to src attributes that lack it (i.e. resources internal to FA). The FileURL field stores the link to the submission file (with "http:" added). The File field stores the binary data of the submission file.

The User type contains the metadata extracted from a user's main page.

User.Userid and User.Username store the same value but in two different formats. Userid is the url-compatible version of the username, so it lacks capital letters and the underscore character is removed. Username instead stores the user-formatted version of the name (i.e. with capital letters, underscore). The Title is the small "description" chosen by the user (e.g. Writer, Artist, etc...) and extracted as is.

The Status can be active, suspended or deceased at the moment and it depends from the first character that FA automatically prefixes to the user's name (e.g. "~UserName" for an active username).

Date is the date the user joined, formatted to "YYYY-MM-DD".

The Profile is the HTML source of the main description written by the user.

Methods ¶

The FAAPI type supports two types of methods, one to initialize the api, the other to get data off the forum.

func (api *FAAPI) LoadCookies(cookiesFile string) (err error)

func (api *FAAPI) MakeRequest() (err error)

func (api *FAAPI) GetSubmission(id string) (sub Submission, err error)

func (api *FAAPI) GetUser(id string) (usr User, err error)

func (api *FAAPI) GetGallery(user string, page int) (subs []Submission, next int, err error)

func (api *FAAPI) GetScraps(user string, page int) (subs []Submission, next int, err error)

func (api *FAAPI) GetFavorites(user string, page int) (subs []Submission, next int, err error)

func (api *FAAPI) GetSearch(args map[string]string, page int) (subs []Submission, next int, err error)

LoadCookies() and MakeRequest() are used to initialize the type. The first takes a string as filename and opens the corresponding file. If the file is properly formatted (in JSON) then the cookies contained in it will be loaded into the api as a slice of http.Cookie. The cookies file needs to contain elements with at least name and value keys, others are not necessary. MakeRequest() simply takes the cookies slice returned and saved by LoadCookies() and uses it to create a http.Request object which is then saved in the FAAPI struct.

The Get* methods return single structs or slices of what their name suggests.

GetSubmission() and GetUser() return a single struct of type Submission and User respectively. These are the calls that provide the most information for both user and submission.

GetGallery(), GetScraps(), GetFavorites() and GetSearch() all return a slice of Submission. Gallery, scraps and favorites take a userid as argument together with a page value. The slices returned contain normal Submission structs but with less information: the Date, Tags, Description and FileURL fields are empty. This is because these methods parse the links contained withing the user folders (or search pages) which do not contain those values. The second item the methods return is the next page in the specific section. If no next page page can be found (i.e. there is no next page button) then `next` will have value -1. For gallery, scraps and searches the next page item is just the passed page argument increased by 1, for favorites it is the id of the next page.

GetFavorites() page works differently than the others. FA uses an id for favorites pages. 0 can be used to get the first page and 1 for the last, but those in between will need the specific page id.

GetSearch() takes a map argument with both keys and values of type string. These are then encoded using the URL. Values method AddValues and are then encoded and added to the URL.RawQuery field of the http.Request.

The Submission type supports a single method.

func (sub *Submission) GetFile() (err error)

GetFile() downloads the file pointed at by FileURL and saves its binary data into the File field.