Documentation ¶
Overview ¶
Credits go to github.com/SlyMarbo/rss for inspiring this solution.
Author: jim teeuwen <jimteeuwen@gmail.com> Dependencies: go-pkg-xmlx (http://github.com/jteeuwen/go-pkg-xmlx) This package allows us to fetch Rss and Atom feeds from the internet. They are parsed into an object tree which is a hybrid of both the RSS and Atom standards. Supported feeds are: - Rss v0.91, 0.91 and 2.0 - Atom 1.0 The package allows us to maintain cache timeout management. This prevents us from querying the servers for feed updates too often and risk ip bams. Appart from setting a cache timeout manually, the package also optionally adheres to the TTL, SkipDays and SkipHours values specied in the feeds themselves. Note that the TTL, SkipDays and SkipHour fields are only part of the RSS spec. For Atom feeds, we use the CacheTimeout in the Feed struct. Because the object structure is a hybrid between both RSS and Atom specs, not all fields will be filled when requesting either an RSS or Atom feed. I have tried to create as many shared fields as possiblem but some of them simply do not occur in either the RSS or Atom spec.
Index ¶
- func NewDatabase() *database
- type Author
- type Category
- type Channel
- type ChannelHandler
- type ChannelHandlerFunc
- type Cloud
- type Content
- type Enclosure
- type Extension
- type Feed
- func (this *Feed) CanUpdate() bool
- func (this *Feed) Fetch(uri string, charset xmlx.CharsetFunc) (err error)
- func (this *Feed) FetchBytes(uri string, content []byte, charset xmlx.CharsetFunc) (err error)
- func (this *Feed) FetchClient(uri string, client *http.Client, charset xmlx.CharsetFunc) (err error)
- func (this *Feed) GetVersionInfo(doc *xmlx.Document) (ftype string, fversion [2]int)
- func (this *Feed) IgnoreCacheOnce()
- func (this *Feed) LastUpdate() time.Time
- func (this *Feed) SecondsTillUpdate() int64
- func (this *Feed) SetUserAgent(s string)
- func (this *Feed) TillUpdate() (time.Duration, error)
- type Generator
- type Image
- type Input
- type Item
- type ItemHandler
- type ItemHandlerFunc
- type Link
- type MissingRssNodeError
- type Source
- type SubTitle
- type UnsupportedFeedError
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NewDatabase ¶
func NewDatabase() *database
Types ¶
type Channel ¶
type Channel struct { Title string Links []Link Description string Language string Copyright string ManagingEditor string WebMaster string PubDate string LastBuildDate string Docs string Categories []*Category Generator Generator TTL int Rating string SkipHours []int SkipDays []int Image Image Items []*Item Cloud Cloud TextInput Input Extensions map[string]map[string][]Extension // Atom fields Id string Rights string Author Author SubTitle SubTitle }
type ChannelHandler ¶
func NewDatabaseChannelHandler ¶
func NewDatabaseChannelHandler(db *database, chanhandler ChannelHandler) ChannelHandler
type ChannelHandlerFunc ¶
func (ChannelHandlerFunc) ProcessChannels ¶
func (h ChannelHandlerFunc) ProcessChannels(f *Feed, newchannels []*Channel)
type Feed ¶
type Feed struct { // Custom cache timeout in minutes. CacheTimeout int // Make sure we adhere to the cache timeout specified in the feed. If // our CacheTimeout is higher than that, we will use that instead. EnforceCacheLimit bool // Type of feed. Rss, Atom, etc Type string // Version of the feed. Major and Minor. Version [2]int // Channels with content. Channels []*Channel // Url from which this feed was created. Url string // contains filtered or unexported fields }
func New ¶
func New(cachetimeout int, enforcecachelimit bool, ch ChannelHandlerFunc, ih ItemHandlerFunc) *Feed
New is a helper function to stay semi-compatible with the old code. Includes the database handler to ensure that this approach is functionally identical to the old database/handlers version.
func NewWithHandlers ¶
func NewWithHandlers(cachetimeout int, enforcecachelimit bool, ch ChannelHandler, ih ItemHandler) *Feed
NewWithHandler creates a new feed with handlers. People should use this approach from now on.
func (*Feed) CanUpdate ¶
This function returns true or false, depending on whether the CacheTimeout value has expired or not. Additionally, it will ensure that we adhere to the RSS spec's SkipDays and SkipHours values (if Feed.EnforceCacheLimit is set to true). If this function returns true, you can be sure that a fresh feed update will be performed.
func (*Feed) Fetch ¶
func (this *Feed) Fetch(uri string, charset xmlx.CharsetFunc) (err error)
Fetch retrieves the feed's latest content if necessary.
The charset parameter overrides the xml decoder's CharsetReader. This allows us to specify a custom character encoding conversion routine when dealing with non-utf8 input. Supply 'nil' to use the default from Go's xml package.
This is equivalent to calling FetchClient with http.DefaultClient
func (*Feed) FetchBytes ¶
Fetch retrieves the feed's content from the []byte
The charset parameter overrides the xml decoder's CharsetReader. This allows us to specify a custom character encoding conversion routine when dealing with non-utf8 input. Supply 'nil' to use the default from Go's xml package.
func (*Feed) FetchClient ¶
func (this *Feed) FetchClient(uri string, client *http.Client, charset xmlx.CharsetFunc) (err error)
Fetch retrieves the feed's latest content if necessary.
The charset parameter overrides the xml decoder's CharsetReader. This allows us to specify a custom character encoding conversion routine when dealing with non-utf8 input. Supply 'nil' to use the default from Go's xml package.
The client parameter allows the use of arbitrary network connections, for example the Google App Engine "URL Fetch" service.
func (*Feed) GetVersionInfo ¶
Returns the type of the feed, ie. "atom" or "rss", and the version number as an array. The first item in the array is the major and the second the minor version number.
func (*Feed) IgnoreCacheOnce ¶
func (this *Feed) IgnoreCacheOnce()
Until the next *successful* fetching of the feed's content, the fetcher will ignore all cache values and update interval hints, and always attempt to retrieve a fresh copy of the feed.
func (*Feed) LastUpdate ¶
This returns a timestamp of the last time the feed was updated.
func (*Feed) SecondsTillUpdate ¶
Returns the number of seconds needed to elapse before the feed should update.
func (*Feed) SetUserAgent ¶
type Item ¶
type Item struct { // RSS and Shared fields Title string Links []*Link Description string Author Author Categories []*Category Comments string Enclosures []*Enclosure Guid *string PubDate string Source *Source // Atom specific fields Id string Generator *Generator Contributors []string Content *Content Updated string Extensions map[string]map[string][]Extension }
type ItemHandler ¶
func NewDatabaseItemHandler ¶
func NewDatabaseItemHandler(db *database, itemhandler ItemHandler) ItemHandler
type ItemHandlerFunc ¶
func (ItemHandlerFunc) ProcessItems ¶
func (h ItemHandlerFunc) ProcessItems(f *Feed, ch *Channel, newitems []*Item)
type MissingRssNodeError ¶
type MissingRssNodeError struct{}
func (*MissingRssNodeError) Error ¶
func (err *MissingRssNodeError) Error() string
type UnsupportedFeedError ¶
func (*UnsupportedFeedError) Error ¶
func (err *UnsupportedFeedError) Error() string