gryffin: github.com/yahoo/gryffin Index | Files | Directories

package gryffin

import "github.com/yahoo/gryffin"

Package gryffin is an application scanning infrastructure.

Index

Package Files

global.go gryffin.go serialize.go session.go util.go

func GenRandomID Uses

func GenRandomID() string

GenRandomID generates a random ID.

func SetLogWriter Uses

func SetLogWriter(w io.Writer)

func SetMemoryStore Uses

func SetMemoryStore(m *GryffinStore)

type Fingerprint Uses

type Fingerprint struct {
    Origin             uint64 // origin
    URL                uint64 // origin + path
    Request            uint64 // method, url, body
    RequestFull        uint64 // request + header
    ResponseSimilarity uint64
}

Fingerprint contains all the different types of hash for the Scan (Request & Response)

type Fuzzer Uses

type Fuzzer interface {
    Fuzz(*Scan) (int, error)
}

Fuzzer runs the fuzzing.

type GryffinStore Uses

type GryffinStore struct {
    Oracles map[string]*distance.Oracle
    Hashes  map[string]bool
    Hits    map[string]int
    // contains filtered or unexported fields
}

func NewGryffinStore Uses

func NewGryffinStore() *GryffinStore

func NewSharedGryffinStore Uses

func NewSharedGryffinStore() *GryffinStore

func (*GryffinStore) GetRcvChan Uses

func (s *GryffinStore) GetRcvChan() chan []byte

func (*GryffinStore) GetSndChan Uses

func (s *GryffinStore) GetSndChan() chan []byte

func (*GryffinStore) Hit Uses

func (s *GryffinStore) Hit(prefix string) bool

func (*GryffinStore) See Uses

func (s *GryffinStore) See(prefix string, kind string, v uint64)

func (*GryffinStore) Seen Uses

func (s *GryffinStore) Seen(prefix string, kind string, v uint64, r uint8) bool

type HTTPDoer Uses

type HTTPDoer interface {
    Do(*http.Request) (*http.Response, error)
}

HTTPDoer interface is to be implemented by http.Client

type Job Uses

type Job struct {
    ID             string
    DomainsAllowed []string // Domains that we would crawl
}

Job stores the job id and config (if any).

type LogMessage Uses

type LogMessage struct {
    Service string
    Msg     string
    Method  string
    Url     string
    JobID   string
}

LogMessage contains the data fields to be marshall as a json for forwarding to the log processor.

type PublishMessage Uses

type PublishMessage struct {
    F   string // function, i.e. See or Seen
    T   string // type (kind), i.e. oracle or hash
    K   string // key
    V   string // value
}

type Renderer Uses

type Renderer interface {
    Do(*Scan)
    GetRequestBody() <-chan *Scan
    GetLinks() <-chan *Scan
}

Renderer is an interface for implementation HTML DOM renderer and obtain the response body and links. Since DOM construction is very likely to be asynchronous, we return the channels to receive response and links.

type Scan Uses

type Scan struct {
    // ID is a random ID to identify this particular scan.
    // if ID is empty, this scan should not be performed (but record for rate limiting).
    ID           string
    Job          *Job
    Request      *http.Request
    RequestBody  string
    Response     *http.Response
    ResponseBody string
    Cookies      []*http.Cookie
    Fingerprint  Fingerprint
    HitCount     int
}

A Scan consists of the job, target, request and response.

func NewScan Uses

func NewScan(method, url, post string) *Scan

NewScan creates a scan.

func NewScanFromJson Uses

func NewScanFromJson(b []byte) *Scan

func (*Scan) CrawlAsync Uses

func (s *Scan) CrawlAsync(r Renderer)

CrawlAsync run the crawling asynchronously.

func (*Scan) Error Uses

func (s *Scan) Error(service string, err error)

TODO - LogFmt (fmt string) TODO - LogI (interface)

func (*Scan) Fuzz Uses

func (s *Scan) Fuzz(fuzzer Fuzzer) (int, error)

Scan runs the vulnerability fuzzer, return the issue count

func (*Scan) IsDuplicatedPage Uses

func (s *Scan) IsDuplicatedPage() bool

IsDuplicatedPage checks if we should proceed based on the Response

func (*Scan) IsScanAllowed Uses

func (s *Scan) IsScanAllowed() bool

IsScanAllowed check if the request URL is allowed per Job.DomainsAllowed.

func (*Scan) Json Uses

func (s *Scan) Json() []byte

func (*Scan) Log Uses

func (s *Scan) Log(v interface{})

func (*Scan) Logf Uses

func (s *Scan) Logf(format string, a ...interface{})

func (*Scan) Logm Uses

func (s *Scan) Logm(service, msg string)

Logm sends a LogMessage to Log processor.

func (*Scan) Logmf Uses

func (s *Scan) Logmf(service, format string, a ...interface{})

func (*Scan) MergeRequest Uses

func (s *Scan) MergeRequest(req *http.Request)

MergeRequest merge the request field in scan with the existing one.

func (*Scan) Poke Uses

func (s *Scan) Poke(client HTTPDoer) (err error)

Poke checks if the target is up.

func (*Scan) RateLimit Uses

func (s *Scan) RateLimit() int

RateLimit checks whether we are under the allowed rate for crawling the site. It returns a delay time to wait to check for ReadyToCrawl again.

func (*Scan) ReadResponseBody Uses

func (s *Scan) ReadResponseBody()

ReadResponseBody read Response.Body and fill it to ReadResponseBody. It will also reconstruct the io.ReaderCloser stream.

func (*Scan) ShouldCrawl Uses

func (s *Scan) ShouldCrawl() bool

ShouldCrawl checks if the links should be queued for next crawl.

func (*Scan) Spawn Uses

func (s *Scan) Spawn() *Scan

Spawn spawns a new scan object with a different ID.

func (*Scan) UpdateFingerprint Uses

func (s *Scan) UpdateFingerprint()

UpdateFingerprint updates the fingerprint field.

type SerializableRequest Uses

type SerializableRequest struct {
    *http.Request
    Cancel string
}

type SerializableResponse Uses

type SerializableResponse struct {
    *http.Response
    Request *SerializableRequest
}

type SerializableScan Uses

type SerializableScan struct {
    *Scan
    Request  *SerializableRequest
    Response *SerializableResponse
}

Directories

PathSynopsis
dataPackage data provides an interface for common data store operations.
html-distancePackage html-distance is a go library for computing the proximity of the HTML pages.
renderer
renderer/resource

Package gryffin imports 16 packages (graph) and is imported by 1 packages. Updated 2016-07-26. Refresh now. Tools for package owners.