graphproc

package
v0.0.0-...-cbbc74b Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 22, 2021 License: MIT Imports: 20 Imported by: 0

Documentation

Index

Constants

View Source
const (
	MESSAGE peventid = iota + 1
	FMETRIC
	IMETRIC
)

Graphline event IDs

Variables

This section is empty.

Functions

func AWSDownloadS3Bucket

func AWSDownloadS3Bucket(s *State, payload *Payload) error

AWSDownloadS3Bucket : graph stage that downloads a file from an S3 bucket and returns it as raw bytes

func BaseFunctionName

func BaseFunctionName(i interface{}) string

BaseFunctionName :

func BytesMatrix

func BytesMatrix(buffer []byte) [][]string

BytesMatrix : create matrix from text stored in a byte slice

func CallerName

func CallerName() string

CallerName :

func CutFields

func CutFields(s *State, payload *Payload) error

CutFields :

func DeleteWhiteListIOC

func DeleteWhiteListIOC(s *State, payload *Payload) error

DeleteWhiteListIOC :

func DomainIOC

func DomainIOC(s *State, payload *Payload) error

DomainIOC : search and return DOMAIN

func DurationTime

func DurationTime(name string, mesg []byte, scope *Scope) error

DurationTime : aspect to calculate the duration and send an event. Uses the times from StartTime and EndTime

func EndTime

func EndTime(name string, mesg []byte, scope *Scope) error

EndTime : aspect that sends the end time as an event

func EventInit

func EventInit()

EventInit : initiate the event channel and start a go routine to process events

func FastLogger

func FastLogger(s string, arg ...interface{})

FastLogger : fast structured logger for hot paths

func FileRead

func FileRead(s *State, payload *Payload) error

FileRead : graph stage to read a file into the raw bytes

func FileTextReader

func FileTextReader(s *State, payload *Payload) error

FileTextReader : graph stage to read a text file one line at a time and send as a stream

state of the stream, open or closed, is stored in the payload.Data field. Closed is sent after
file has been read

func FileTextWriter

func FileTextWriter(s *State, payload *Payload) error

FileTextWriter : graph stage to write a stream to a text file one line at a time

looks for closed status in payload.Data to close the file

func FileWrite

func FileWrite(s *State, payload *Payload) error

FileWrite : graph stage to write the raw bytes to a file

func FilterWhiteListIOC

func FilterWhiteListIOC(s *State, payload *Payload) error

FilterWhiteListIOC :

func FunctionName

func FunctionName(i interface{}) string

FunctionName :

func IOC

func IOC(re *regexp.Regexp, rawbytes []byte, hashtype string) ([]byte, error)

IOC : search and return the IOC

func IOCDataToJson

func IOCDataToJson(s *State, payload *Payload) error

IOCDataToJson : marshall the Data field to JSON in the Raw field

func IOCtoData

func IOCtoData(s *State, payload *Payload) error

IOCtoData : conver the Raw field IOC strings to a slice in the Data field

func Ipv4IOC

func Ipv4IOC(s *State, payload *Payload) error

Ipv4IOC : search and return IPV4

func Ipv6IOC

func Ipv6IOC(s *State, payload *Payload) error

Ipv6IOC : search and return IPV6

func Logger

func Logger(s string, arg ...interface{})

Logger : simple structured logger

func MatrixBytes

func MatrixBytes(matrix [][]string, buffer []byte) ([]byte, error)

MatrixBytes : convert matrix to text in a byte slice

func MatrixCut

func MatrixCut(matrix [][]string, cutrow []bool) [][]string

MartrixCut : delete fields from a matrix

func MatrixSelect

func MatrixSelect(matrix [][]string, selectrow []bool) [][]string

MatrixSelect : select fields from a matrix

func Md5IOC

func Md5IOC(s *State, payload *Payload) error

Md5IOC : search and return MD5

func NormalIOC

func NormalIOC(s *State, payload *Payload) error

NormalIOC : convert some typical values to normal values

func ProcessEvent

func ProcessEvent()

ProcessEvent : every second, process events from the event channel

func SelectFields

func SelectFields(s *State, payload *Payload) error

SelectFields :

func SendEvent

func SendEvent(e Gevent)

SendEvent : send event the the event channel

func Sha1IOC

func Sha1IOC(s *State, payload *Payload) error

Sha1IOC : search and return SHA1

func Sha256IOC

func Sha256IOC(s *State, payload *Payload) error

Sha256IOC : search and return SHA256

func StartTime

func StartTime(name string, mesg []byte, scope *Scope) error

StartTime : aspect that sends start time as an event

func SubGraph

func SubGraph(s *State, payload *Payload) error

SubGraph : graph stage to execute a sub-graph

The subgraph must have been made public by PublishGraphs()

func Trace

func Trace(name string, mesg []byte, scope *Scope) error

Trace : trace aspect that sends trace events

func UrlIOC

func UrlIOC(s *State, payload *Payload) error

UrlIOC : search and return URL

func WhiteListIOC

func WhiteListIOC(s *State, payload *Payload) error

WhiteListIOC :

Types

type Aspect

type Aspect struct {
	// contains filtered or unexported fields
}

Aspect :

func NewAspect

func NewAspect(f func(string, []byte, *Scope) error, t time.Duration, v int64, action AspectAction) Aspect

NewAspect :

func (*Aspect) Execute

func (a *Aspect) Execute(name string, mesg []byte, scope *Scope) AspectAction

Execute :

type AspectAction

type AspectAction int

AspectAction :

const (
	RETRY AspectAction = iota + 1
	CANCEL
	CONTINUE
	DEALDLINE
	TIMEOUT
	ERROR
	START
	END
)

Aspect action :

type Edge

type Edge struct {
	//In     *Vertex
	Out      *Vertex
	Epayload *Payload
	Estate   *State
}

Edge : graph edge type

type FileState

type FileState struct {
	Status   string
	Ffile    *os.File
	Fscanner *bufio.Scanner
}

FileState : store that state of file scanner

type Gevent

type Gevent struct {
	// contains filtered or unexported fields
}

Gevent : graphline events for logging, observability, metrics, etc.

type Graph

type Graph struct {
	V    []*Vertex
	E    []*Edge
	Path []*Vertex
	// contains filtered or unexported fields
}

Graph : graph type of vertexes and edges and execution path

func NewGraph

func NewGraph() *Graph

NewGraph : create a new graph

func (*Graph) BuildPath

func (g *Graph) BuildPath()

Path : build the execution path by walking through the graph depth first until a vertex is found with incomplete inputs

go back to the last vertex with a fork in the path and start the next walk
func (g *Graph) Link(v1 *Vertex, v2 *Vertex)

Link : link two vertexes by creating an edge from vertex 1 to vertex 2

func (*Graph) NewEdge

func (g *Graph) NewEdge() *Edge

NewEdge : create a new edge

func (*Graph) NewVertex

func (g *Graph) NewVertex(n string) *Vertex

NewVertex : create a new vertex

func (*Graph) NextVertex

func (g *Graph) NextVertex() *Vertex

NextVertex : get the next vertex in the path, returns nil if at the path end

type Graphline

type Graphline struct {
	// contains filtered or unexported fields
}

Graphline : map with the set of graphs and map of application stages used in the graphs

func NewGraphline

func NewGraphline() Graphline

NewGraphline : create a new graphline

func PublicGraphs

func PublicGraphs() *Graphline

PublicGraphs : return the graphline that was published

func (*Graphline) CallStage

func (g *Graphline) CallStage(graphid string, name string, state *State) (int, error)

CallStage : deprecated

func (*Graphline) Execute

func (g *Graphline) Execute(gname string, payload *Payload) error

Execute : execute a graph based on its name

func (*Graphline) PrintPath

func (g *Graphline) PrintPath(graphid string)

PrintPath : print the path for diagnostic purposes

func (*Graphline) PublishGraphs

func (g *Graphline) PublishGraphs()

PublishGraphs : publish the graphline if needed elsewhere (subgraph execution stage)

func (*Graphline) RegisterAspect

func (g *Graphline) RegisterAspect(graphid string, action AspectAction, stage func(*State, *Payload) error, newasp *Aspect) error

RegisterAspect : register an application aspect function for use in a graph

func (*Graphline) RegisterStage

func (g *Graphline) RegisterStage(sfunc func(*State, *Payload) error) error

RegisterStage : register an application stage function for use in a graph

func (*Graphline) RunAspects

func (g *Graphline) RunAspects(name string, mesg []byte, scope *Scope, asp map[string]*Aspect) error

RunAspects : run the aspects for a graph stage

func (*Graphline) Sequence

func (g *Graphline) Sequence(graphspec string) ([]string, error)

Sequence : parse and generate the execution sequence for a graph specification

func (*Graphline) SetState

func (g *Graphline) SetState(gname string, name string, state *State) (int, error)

SetState : deprecated

type IOCs

type IOCs struct {
	Md5    []string `json:"md5"`
	Sha1   []string `json:"sha1"`
	Sha256 []string `json:"sha256"`
	Ipv4   []string `json:"ipv4"`
	Ipv6   []string `json:"ipv6"`
	Url    []string `json:"url"`
	Domain []string `json:"domain"`
}

type Parser

type Parser struct {
	// contains filtered or unexported fields
}

Parser : represents a parser.

func NewParser

func NewParser(r io.Reader) *Parser

NewParser : returns a new instance of Parser

func (*Parser) Parse

func (p *Parser) Parse() ([]string, []*Graph, error)

Parse : parse graph definition and output the slice of graphs

type Payload

type Payload struct {
	Raw  []byte
	Data interface{}
}

Payload : graph stage payload type

func NewPayload

func NewPayload() *Payload

NewPayload : create a new payload type

func (*Payload) Append

func (p *Payload) Append(dest, src *Payload) (*Payload, error)

Append : deep append the src payload to the destination payload. Data field must be map or slice

func (*Payload) Copy

func (p *Payload) Copy(dest, src *Payload) (*Payload, error)

Copy : deep copy the src payload to the destination payload

func (*Payload) GetData

func (p *Payload) GetData(key string) (interface{}, error)

GetData : get the data field when using convention of map of interface values

func (*Payload) SetData

func (p *Payload) SetData(key string, value interface{}) error

SetData : set the data field when using convention of map of interface values

type Scanner

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner : lexical scanner

func NewScanner

func NewScanner(r io.Reader) *Scanner

NewScanner : create a new instance of Scanner

func (*Scanner) Scan

func (s *Scanner) Scan() (tok Token, lit string)

Scan : returns the next token and literal value

type Scope

type Scope struct {
	// contains filtered or unexported fields
}

Scope :

type Stage

type Stage struct {
	// contains filtered or unexported fields
}

Stage : graph execution stage type

func NewStage

func NewStage() *Stage

NewStage : create and new graph execution stage type

func (*Stage) BuildState

func (s *Stage) BuildState(cfg interface{})

BuildState

type State

type State struct {
	// contains filtered or unexported fields
}

State : graph stage state type

func NewState

func NewState() *State

NewState : create a state type

func (*State) Config

func (s *State) Config(field string) (string, bool)

Config : get a field value when using the convention of map of interface values

func (*State) SetConfig

func (s *State) SetConfig(field string, value interface{})

SetConfig : set a field value when using the convention of map of interface values

type Token

type Token int

Token represents a lexical token.

const (
	// Special tokens
	ILLEGAL Token = iota
	EOF
	WS
	// Literals
	IDENT // vertex names
	// Misc characters
	SEMICOLON    // ;
	COLON        //	:
	PERIOD       // .
	UNDERSCORE   // _
	COMMA        //	,
	PIPE         //	|
	DOUBLEQUOTE  //	"
	LEFTBRACE    // {
	RIGHTBRACE   //	}
	LEFTBRACKET  // [
	RIGHTBRACKET //	]
	ESCAPE       // \
)

type Vertex

type Vertex struct {
	Name   string
	Prev   []*Edge
	Next   []*Edge
	Vstage *Stage
	// contains filtered or unexported fields
}

Vertex : graph vertex type

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL