Documentation ¶
Index ¶
- Variables
- func AddProductToCache(c *redis.Client, scrapedProduct *AmazonProduct, duration time.Duration) (err error)
- func ConvertHTMLEntities(htmlString string) (clean string)
- func NewScraperServer(client *redis.Client) v1.WebScraperServer
- func StoreProduct(c *redis.Client, scrapedProduct *AmazonProduct) (err error)
- type AmazonProduct
Constants ¶
This section is empty.
Variables ¶
View Source
var ( //ErrMissingASIN is shared error message, returns if asin is missing ErrMissingASIN = errors.New("missing ASIN in request") )
Functions ¶
func AddProductToCache ¶
func AddProductToCache(c *redis.Client, scrapedProduct *AmazonProduct, duration time.Duration) (err error)
AddProductToCache cache AmazonProduct with key cacheProduct:{ASIN}
func ConvertHTMLEntities ¶
ConvertHTMLEntities converts scraped html entities and returns clean string
func NewScraperServer ¶
func NewScraperServer(client *redis.Client) v1.WebScraperServer
NewScraperServer takes a new redis client for scraper server
func StoreProduct ¶
func StoreProduct(c *redis.Client, scrapedProduct *AmazonProduct) (err error)
StoreProduct save AmazonProduct into Redis with key product:{ASIN}
Types ¶
type AmazonProduct ¶
type AmazonProduct struct { Asin string `json:"asin"` Name string `json:"name"` Categories []string `json:"categories"` Ranks []string `json:"ranks"` Dimensions []string `json:"dimensions"` CreatedAt string `json:"created_at"` }
AmazonProduct is the default product struct for ASIN service
func FetchProduct ¶
func FetchProduct(c *redis.Client, asin string) (product AmazonProduct, err error)
FetchProduct get AmazonProduct from Redis Todo: Fetch product will be used in Phase 2 for limited caching
func GetProductFromCache ¶
func GetProductFromCache(c *redis.Client, asin string) (product AmazonProduct, err error)
GetProductFromCache tries to grab cached product from cacheProduct with key cacheProduct:{ASIN}
func (*AmazonProduct) GetProductInfoByASIN ¶
func (product *AmazonProduct) GetProductInfoByASIN() (res *colly.Response, err error)
GetProductInfoByASIN takes asin, build target url, and returns product info
Click to show internal directories.
Click to hide internal directories.