Documentation ¶
Overview ¶
Package gryffin is an application scanning infrastructure.
Index ¶
- func GenRandomID() string
- func SetLogWriter(w io.Writer)
- func SetMemoryStore(m *GryffinStore)
- type Fingerprint
- type Fuzzer
- type GryffinStore
- type HTTPDoer
- type Job
- type LogMessage
- type PublishMessage
- type Renderer
- type Scan
- func (s *Scan) CrawlAsync(r Renderer)
- func (s *Scan) Error(service string, err error)
- func (s *Scan) Fuzz(fuzzer Fuzzer) (int, error)
- func (s *Scan) IsDuplicatedPage() bool
- func (s *Scan) IsScanAllowed() bool
- func (s *Scan) Json() []byte
- func (s *Scan) Log(v interface{})
- func (s *Scan) Logf(format string, a ...interface{})
- func (s *Scan) Logm(service, msg string)
- func (s *Scan) Logmf(service, format string, a ...interface{})
- func (s *Scan) MergeRequest(req *http.Request)
- func (s *Scan) Poke(client HTTPDoer) (err error)
- func (s *Scan) RateLimit() int
- func (s *Scan) ReadResponseBody()
- func (s *Scan) ShouldCrawl() bool
- func (s *Scan) Spawn() *Scan
- func (s *Scan) UpdateFingerprint()
- type SerializableRequest
- type SerializableResponse
- type SerializableScan
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func SetMemoryStore ¶
func SetMemoryStore(m *GryffinStore)
SetMemoryStore sets the package internal global variable for the memory store.
Types ¶
type Fingerprint ¶
type Fingerprint struct { Origin uint64 // origin URL uint64 // origin + path Request uint64 // method, url, body RequestFull uint64 // request + header ResponseSimilarity uint64 }
Fingerprint contains all the different types of hash for the Scan (Request & Response)
type GryffinStore ¶
type GryffinStore struct { Oracles map[string]*distance.Oracle Hashes map[string]bool Hits map[string]int Mu sync.RWMutex // contains filtered or unexported fields }
GryffinStore includes data and handles for Gryffin message processing,
func NewGryffinStore ¶
func NewGryffinStore() *GryffinStore
func NewSharedGryffinStore ¶
func NewSharedGryffinStore() *GryffinStore
func (*GryffinStore) GetRcvChan ¶
func (s *GryffinStore) GetRcvChan() chan []byte
func (*GryffinStore) GetSndChan ¶
func (s *GryffinStore) GetSndChan() chan []byte
func (*GryffinStore) Hit ¶
func (s *GryffinStore) Hit(prefix string) bool
type LogMessage ¶
LogMessage contains the data fields to be marshalled as JSON for forwarding to the log processor.
type PublishMessage ¶
type PublishMessage struct { F string // function, i.e. See or Seen T string // type (kind), i.e. oracle or hash K string // key V string // value }
PublishMessage is the data in the messages handled by Gryffin.
type Renderer ¶
Renderer is an interface for implementation HTML DOM renderer and obtain the response body and links. Since DOM construction is very likely to be asynchronous, we return the channels to receive response and links.
type Scan ¶
type Scan struct { // ID is a random ID to identify this particular scan. // if ID is empty, this scan should not be performed (but record for rate limiting). ID string Job *Job Request *http.Request RequestBody string Response *http.Response ResponseBody string Cookies []*http.Cookie Fingerprint Fingerprint HitCount int }
A Scan consists of the job, target, request and response.
func NewScanFromJson ¶
NewScanFromJson creates a Scan from the passed JSON blob.
func (*Scan) CrawlAsync ¶
CrawlAsync run the crawling asynchronously.
func (*Scan) Error ¶
TODO - LogFmt (fmt string) TODO - LogI (interface) Error logs the error for the given service.
func (*Scan) IsDuplicatedPage ¶
IsDuplicatedPage checks if we should proceed based on the Response
func (*Scan) IsScanAllowed ¶
IsScanAllowed check if the request URL is allowed per Job.DomainsAllowed.
func (*Scan) Log ¶
func (s *Scan) Log(v interface{})
Log encodes the given argument as JSON and writes it to the log writer.
func (*Scan) MergeRequest ¶
MergeRequest merge the request field in scan with the existing one.
func (*Scan) RateLimit ¶
RateLimit checks whether we are under the allowed rate for crawling the site. It returns a delay time to wait to check for ReadyToCrawl again.
func (*Scan) ReadResponseBody ¶
func (s *Scan) ReadResponseBody()
ReadResponseBody read Response.Body and fill it to ReadResponseBody. It will also reconstruct the io.ReaderCloser stream.
func (*Scan) ShouldCrawl ¶
ShouldCrawl checks if the links should be queued for next crawl.
func (*Scan) UpdateFingerprint ¶
func (s *Scan) UpdateFingerprint()
UpdateFingerprint updates the fingerprint field.
type SerializableRequest ¶
SerializableRequest is a Scan extended with serializable request field.
type SerializableResponse ¶
type SerializableResponse struct { *http.Response Request *SerializableRequest }
SerializableResponse is a Scan extended with serializable response field.
type SerializableScan ¶
type SerializableScan struct { *Scan Request *SerializableRequest Response *SerializableResponse }
SerializableScan is a Scan extended with serializable request and response fields.
Directories ¶
Path | Synopsis |
---|---|
cmd
|
|
Package data provides an interface for common data store operations.
|
Package data provides an interface for common data store operations. |
fuzzer
|
|
Package distance is a go library for computing the proximity of the HTML pages.
|
Package distance is a go library for computing the proximity of the HTML pages. |