cerebro

package
v0.1.4-alpha Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 26, 2020 License: MIT Imports: 28 Imported by: 0

README

🧠 Cerebro

Cerebro is the application that finds characters, character sources, and character issues from external sources and imports those resources as local resources into the appearances database so that all a characters.

CLI Commands

  • cerebro import [resource]: Imports external resources as local resources. Available resources: characters, charactersources, characterissues

What counts as an appearance

characterissue.go contains the logic for aggregating a character's issues and counting it as an appearance and persisting it.

There are two categories for the type of appearances: main and alternate appearances.

Main appearances refer to the original character (for DC the rules are a little different). Alternate appearances refer to a character's alternate reality appearances, such as the Marvel Ultimate line.

Here are the rules for what counts as an appearance.

What does NOT count:

  • Variants
  • Reprints, such as 2nd printings or issues reprinted in another language.
  • Flip-books, ashcans, magazines, trade paperbacks, etc.

Marvel:

Main

  • The character's 616 counterpart makes an appearance.
  • The character's 616 counterpart was transported into another reality, such as House of M or Age of X.

Alternate

  • The character appears, such as Spider-Man 2099, but in a difference universe, such as the 2099 or Ultimate line.

DC:

Main

  • Due to DC's nature, almost all appearance are main appearances, except comics about other mediums, such as TV shows, video games, movies, etc.

Alternate

  • Comics involving tv shows, video games, and movies are alternate appearances.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ParseCharacterName

func ParseCharacterName(s string) string

ParseCharacterName parses the character name from the given string.

func ParsePublisherName

func ParsePublisherName(s string) string

ParsePublisherName parses the publisher name from the given string.

func SearchableName

func SearchableName(s string, parensIndex int) string

SearchableName returns a search-friendly name for the external source and removes any parentheses and periods. If parensIndex is > -1, it will get the name within the parentheses.

Types

type AlterEgoIdentifier

type AlterEgoIdentifier struct {
	// contains filtered or unexported fields
}

AlterEgoIdentifier is the struct for identifying an alter ego.

func NewAlterEgoIdentifier

func NewAlterEgoIdentifier(h HTTPClient, svc comic.CharacterServicer) *AlterEgoIdentifier

NewAlterEgoIdentifier creates a new identifier.

func (*AlterEgoIdentifier) Name

Name gets the alter-ego name for a character.

type AlterEgoImporter

type AlterEgoImporter struct {
	// contains filtered or unexported fields
}

AlterEgoImporter imports the alter ego as an other name for a character

func NewAlterEgoImporter

func NewAlterEgoImporter(h HTTPClient, svc comic.CharacterServicer) *AlterEgoImporter

NewAlterEgoImporter creates a new alter ego importer.

func (*AlterEgoImporter) Import

func (i *AlterEgoImporter) Import(slugs []comic.CharacterSlug) error

Import imports a character's other_name by identifying a real name from an external source.

type CharacterCBExtractor

type CharacterCBExtractor struct {
	// contains filtered or unexported fields
}

CharacterCBExtractor parses a character's sources and into CharacterVendorInfo.

func NewCharacterCBExtractor

func NewCharacterCBExtractor(externalSource externalissuesource.ExternalSource) *CharacterCBExtractor

NewCharacterCBExtractor creates a new character CB vendor extractor from the params.

func (*CharacterCBExtractor) Extract

Extract parses the vendor information from a character's many character sources.

type CharacterImportResult

type CharacterImportResult struct {
	ExternalCharacter *ExternalCharacter // Returns nil if the external character wasn't fetched.
	LocalCharacter    *comic.Character   // Returns nil if the character wasn't imported.
	Error             error              // Returns an error if something bad happened.
}

CharacterImportResult represents the result of an import, with the `ExternalCharacter` being the external source and the `LocalCharacter` being the object that was imported from the external source. An error could happen, so the `LocalCharacter` returns `nil` with an `Error` set if the character wasn't imported.

type CharacterImporter

type CharacterImporter interface {
	ImportAll() error
}

CharacterImporter is the interface for importing characters.

type CharacterIssueImporter

type CharacterIssueImporter struct {
	// contains filtered or unexported fields
}

CharacterIssueImporter is the importer for getting a character's issues from a character source.

func NewCharacterIssueImporter

func NewCharacterIssueImporter(
	db *pg.DB,
	redis *redis.Client) *CharacterIssueImporter

NewCharacterIssueImporter creates a new character issue importer.

func (*CharacterIssueImporter) ImportAll

func (i *CharacterIssueImporter) ImportAll(slugs []comic.CharacterSlug, doReset bool) error

ImportAll imports characters from the specified slugs and creates the sync log for each character and sets it to PENDING, then sequentially imports the issues for the character. Fatals if failed to create a sync log or character cannot be fetched. If doReset is set to true, it will delete all associated character issues first and re-import new ones.

func (*CharacterIssueImporter) ImportWithSyncLog

func (i *CharacterIssueImporter) ImportWithSyncLog(character comic.Character, syncLog *comic.CharacterSyncLog, doReset bool) error

ImportWithSyncLog does A LOT. It's for importing a character's issues with an existing sync log attached. Imports a character's issues from their character sources and polls an external source for issue information and then persists the character's appearances to the db and Redis. A channel is opened listening for a SIGINT if the caller quits the process. In that case, the character sync log is set to failed and the process quits cleanly.

type CharacterSourceImportItem

type CharacterSourceImportItem struct {
	Source *comic.CharacterSource
	Error  error
}

CharacterSourceImportItem represents issue sources whose import was attempted and an error if an unexpected event occurred.

type CharacterSourceImporter

type CharacterSourceImporter struct {
	// contains filtered or unexported fields
}

CharacterSourceImporter is responsible for importing a characters' sources into a persistence layer.

func NewCharacterSourceImporter

func NewCharacterSourceImporter(db comic.ORM) *CharacterSourceImporter

NewCharacterSourceImporter returns the implementation for the character source importer.

func (*CharacterSourceImporter) Import

func (i *CharacterSourceImporter) Import(slugs []comic.CharacterSlug, isStrict bool) error

Import with the specified character criteria, concurrently imports character sources from an external source. If strict is set to true, it will import sources whose name _exactly_ matches the character's name (case insensitive). Otherwise, it will import all sources that match the search result.

type CharacterVendorExtractor

type CharacterVendorExtractor interface {
	// Extract extracts character source information into CharacterVendorInfo.
	Extract(sources []*comic.CharacterSource) (CharacterVendorInfo, error)
}

CharacterVendorExtractor parses information about a vendor for a character.

type CharacterVendorInfo

type CharacterVendorInfo struct {
	// VendorIDs contains a map of VendorIDs and their corresponding URLs.
	VendorIDs map[ExternalVendorID]ExternalVendorURL
	// MainSources contains all the VendorIDs that are main sources.
	MainSources map[ExternalVendorID]bool
	// AltSources contains all the VendorIDs that are alternate sources.
	AltSources map[ExternalVendorID]bool
}

CharacterVendorInfo contains information about a character's vendor IDs and the main and alternate sources associated to the character's sources.

func (CharacterVendorInfo) AppearanceType

func (vi CharacterVendorInfo) AppearanceType(issue *comic.Issue) comic.AppearanceType

AppearanceType determines the type of appearance from the issue's vendor ID. So if the issue's vendor ID is in the externalInfo's mainSources map, it's a main source, etc.

type DcCharactersImporter

type DcCharactersImporter struct {
	// contains filtered or unexported fields
}

DcCharactersImporter imports characters from the DC API to the local repository.

func NewDCCharactersImporter

func NewDCCharactersImporter(db *pg.DB) *DcCharactersImporter

NewDCCharactersImporter returns the implementation for the DC Characters importer.

func (*DcCharactersImporter) ImportAll

func (dci *DcCharactersImporter) ImportAll() error

ImportAll launches goroutines to import characters from the DC API. Returns an error if there is a system error or an error fetching from the API.

type ExternalCharacter

type ExternalCharacter struct {
	Publisher    string
	VendorID     string // The vendor's unique identifier of the external character.
	Name         string
	Description  string
	ThumbnailURL string
	URL          string
}

ExternalCharacter represents a character from a remote source, such as the Marvel API.

type ExternalVendorID

type ExternalVendorID string

ExternalVendorID is a vendor ID for a third-party vendor.

type ExternalVendorURL

type ExternalVendorURL string

ExternalVendorURL is a vendor URL from a third-party.

func (ExternalVendorURL) String

func (u ExternalVendorURL) String() string

String gets the string value of the external URL.

type HTTPClient

type HTTPClient interface {
	Get(url string) (resp *http.Response, err error)
}

HTTPClient is an interface for handling HTTP calls.

type MarvelCharactersImporter

type MarvelCharactersImporter struct {
	// contains filtered or unexported fields
}

MarvelCharactersImporter imports characters from the Marvel API into a local repository.

func NewMarvelCharactersImporter

func NewMarvelCharactersImporter(db *pg.DB) *MarvelCharactersImporter

NewMarvelCharactersImporter returns the implementation for the Marvel Characters importer.

func (*MarvelCharactersImporter) ImportAll

func (mci *MarvelCharactersImporter) ImportAll() error

ImportAll launches goroutines to import characters from the Marvel API. Returns an error if there is a system error or an error fetching from the API.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL