halfpike: github.com/johnsiilver/halfpike Index | Examples | Files

package halfpike

import "github.com/johnsiilver/halfpike"

Package halfpike provides a lexer/parser framework library that can simplify lexing and parsing by using a very limited subset of the regexp syntax. This prevents many of the common errors encountered when trying to parse output from devices where the complete language syntax is unknown and can change between releases. Routers and other devices with human readable output or badly mangled formats within a standard (such as XML or JSON).

Called halfpike, because this solution is a mixture of Rob Pike's lexer talk and the use of regex's within a single line of output to do captures in order to store a value within a struct type.

A similar method replaced complex regex captures at a large search company's network group to prevent accidental empty matches and other bad behavior from regexes that led to issues in automation stacks. It allowed precise diagnosis of problems and readable code (complex regexes are not easily readable).

Code:

package main

import (
    "context"
    "fmt"
    "net"
    "strconv"
    "strings"
    "time"

    "github.com/kylelemons/godebug/pretty"
)

func main() {
    // A slice of structs that has a .Validate() method on it.
    // This is where our data will be stored.
    neighbors := BGPNeighbors{}

    // Creates our parers object that our various ParseFn functions will use to move
    // through the input.
    p, err := NewParser(showBGPNeighbor, neighbors)
    if err != nil {
        panic(err)
    }

    // An object that contains various ParseFn methods.
    states := &PeerParser{}

    // Parses our content in showBGPNeighbor and begins parsing with states.FindPeer
    // which is a ParseFn.
    if err := Parse(context.Background(), p, states.FindPeer); err != nil {
        panic(err)
    }

    // Because we pass in a slice, we have to do a reassign to get the changed value.
    neighbors = p.Validator.(BGPNeighbors)
    fmt.Println(pretty.Sprint(neighbors))

}

// showBGPNeighbor is the output we are going to lex/parse.
var showBGPNeighbor = `
Peer: 10.10.10.2+179 AS 22     Local: 10.10.10.1+65406 AS 17   
  Type: External    State: Established    Flags: <Sync>
  Last State: OpenConfirm   Last Event: RecvKeepAlive
  Last Error: None
  Options: <Preference PeerAS Refresh>
  Holdtime: 90 Preference: 170
  Number of flaps: 0
  Peer ID: 10.10.10.2       Local ID: 10.10.10.1       Active Holdtime: 90
  Keepalive Interval: 30         Peer index: 0   
  BFD: disabled, down
  Local Interface: ge-1/2/0.0                       
  NLRI for restart configured on peer: inet-unicast
  NLRI advertised by peer: inet-unicast
  NLRI for this session: inet-unicast
  Peer supports Refresh capability (2)
  Restart time configured on the peer: 120
  Stale routes from peer are kept for: 300
  Restart time requested by this peer: 120
  NLRI that peer supports restart for: inet-unicast
  NLRI that restart is negotiated for: inet-unicast
  NLRI of received end-of-rib markers: inet-unicast
  NLRI of all end-of-rib markers sent: inet-unicast
  Peer supports 4 byte AS extension (peer-as 22)
  Peer does not support Addpath
  Table inet.0 Bit: 10000
    RIB State: BGP restart is complete
    Send state: in sync
    Active prefixes:              0
    Received prefixes:            0
    Accepted prefixes:            0
    Suppressed due to damping:    2
    Advertised prefixes:          0
  Last traffic (seconds): Received 10   Sent 6    Checked 1   
  Input messages:  Total 8522   Updates 1       Refreshes 0     Octets 161922
  Output messages: Total 8433   Updates 0       Refreshes 0     Octets 160290
  Output Queue[0]: 0

Peer: 10.10.10.6+54781 AS 22   Local: 10.10.10.5+179 AS 17   
  Type: External    State: Established    Flags: <Sync>
  Last State: OpenConfirm   Last Event: RecvKeepAlive
  Last Error: None
  Options: <Preference PeerAS Refresh>
  Holdtime: 90 Preference: 170
  Number of flaps: 0
  Peer ID: 10.10.10.6       Local ID: 10.10.10.1       Active Holdtime: 90
  Keepalive Interval: 30         Peer index: 1   
  BFD: disabled, down                   
  Local Interface: ge-0/0/1.5                       
  NLRI for restart configured on peer: inet-unicast
  NLRI advertised by peer: inet-unicast
  NLRI for this session: inet-unicast
  Peer supports Refresh capability (2)
  Restart time configured on the peer: 120
  Stale routes from peer are kept for: 300
  Restart time requested by this peer: 120
  NLRI that peer supports restart for: inet-unicast
  NLRI that restart is negotiated for: inet-unicast
  NLRI of received end-of-rib markers: inet-unicast
  NLRI of all end-of-rib markers sent: inet-unicast
  Peer supports 4 byte AS extension (peer-as 22)
  Peer does not support Addpath
  Table inet.0 Bit: 10000
    RIB State: BGP restart is complete
    Send state: in sync
    Active prefixes:              0
    Received prefixes:            0
    Accepted prefixes:            0
    Suppressed due to damping:    0
    Advertised prefixes:          0
  Last traffic (seconds): Received 12   Sent 6    Checked 33  
  Input messages:  Total 8527   Updates 1       Refreshes 0     Octets 162057
  Output messages: Total 8430   Updates 0       Refreshes 0     Octets 160233
  Output Queue[0]: 0
 `

// PeerType is the type of peer the neighbor is.
type PeerType uint8

// BGP neighbor types.
const (
    // PTUnknown indicates the neighbor type is unknown.
    PTUnknown PeerType = 0
    // PTExternal indicates the neighbor is external to the router's AS.
    PTExternal PeerType = 1
    // PTInternal indicates the neighbort is intneral to the router's AS.
    PTInternal PeerType = 2
)

type BGPState uint8

// BGP connection states.
const (
    NSUnknown     BGPState = 0
    NSActive      BGPState = 1
    NSConnect     BGPState = 2
    NSEstablished BGPState = 3
    NSIdle        BGPState = 4
    NSOpenConfirm BGPState = 5
    NSOpenSent    BGPState = 6
    NSRRClient    BGPState = 7
)

type RIBState uint8

const (
    RSUnknown    RIBState = 0
    RSComplete   RIBState = 2
    RSInProgress RIBState = 3
)

type SendState uint8

const (
    RSSendUnknown     SendState = 0
    RSSendSync        SendState = 1
    RSSendNotSync     SendState = 2
    RSSendNoAdvertise SendState = 3
)

// BGPNeighbors is a collection of BGPNeighbors for a router.
type BGPNeighbors []*BGPNeighbor

// Vaildate implements Validator.Validate().
func (b BGPNeighbors) Validate() error {
    for _, v := range b {
        if err := v.Validate(); err != nil {
            return err
        }
    }
    return nil
}

// BGPNeighbor provides information about a router's BGP Neighbor.
type BGPNeighbor struct {
    // PeerIP is the IP address of the neighbor.
    PeerIP net.IP
    // PeerPort is the IP port of the peer.
    PeerPort uint32
    // PeerAS is the peers autonomous system number.
    PeerAS int
    // LocalIP is the IP address on this router the neighbor connects to.
    LocalIP net.IP
    // LocaPort is the IP port on this router the neighbor connects to.
    LocalPort uint32
    // LocalAS is the local autonomous system number.
    LocalAS int
    // Type is the type of peer.
    Type PeerType
    // State is the current state of the BGP peer.
    State BGPState
    // LastState is the previous state of the BGP peer.
    LastState BGPState
    // HoldTime is how long to consider the neighbor valid after not hearing a keep alive.
    HoldTime time.Duration
    // Preference is the BGP preference value.
    Preference int
    // PeerID is the ID the peer uses to identify itself.
    PeerID net.IP
    // LocalID is the ID the local router uses to identify itself.
    LocalID   net.IP
    InetStats map[int]*InetStats

    initCalled bool
}

func (b *BGPNeighbor) init() {
    b.Preference = -1
    b.initCalled = true
    b.LocalAS, b.PeerAS = -1, -1
}

// Vaildate implements Validator.Validate().
func (b *BGPNeighbor) Validate() error {
    if !b.initCalled {
        return fmt.Errorf("internal error: BGPNeighbor.init() was not called")
    }

    switch {
    case b.PeerIP == nil:
        return fmt.Errorf("PeerIP was nil")
    case b.LocalIP == nil:
        return fmt.Errorf("LocalIP was nil")
    case b.PeerID == nil:
        return fmt.Errorf("PeerID was nil")
    case b.LocalID == nil:
        return fmt.Errorf("LocalID was nil")
    }

    switch uint32(0) {
    case b.PeerPort:
        return fmt.Errorf("PeerPort was 0")
    case b.LocalPort:
        return fmt.Errorf("LocalPort was 0")
    }

    switch 0 {
    case int(b.Type):
        return fmt.Errorf("Type was not set")
    case int(b.LastState):
        return fmt.Errorf("LastState was not set")
    case int(b.State):
        return fmt.Errorf("State was not set")
    }

    switch -1 {
    case b.Preference:
        return fmt.Errorf("Preference was not set")
    case b.LocalAS:
        return fmt.Errorf("LocalAS was not set")
    case b.PeerAS:
        return fmt.Errorf("PeerAS was not set")
    }

    for _, v := range b.InetStats {
        if err := v.Validate(); err != nil {
            err = fmt.Errorf(err.Error())
        }
    }
    return nil
}

// InetStats contains information about the route table.
type InetStats struct {
    ID                 int
    Bit                int
    RIBState           RIBState
    SendState          SendState
    ActivePrefixes     int
    RecvPrefixes       int
    AcceptPrefixes     int
    SurpressedPrefixes int
    AdvertisedPrefixes int
}

func (b *InetStats) init() {
    b.Bit = -1
    b.ActivePrefixes = -1
    b.RecvPrefixes = -1
    b.AcceptPrefixes = -1
    b.SurpressedPrefixes = -1
}

// Validate implements Validator.
func (b *InetStats) Validate() error {
    switch -1 {
    case b.Bit:
        return fmt.Errorf("InetStats: Bit was not parsed from the input")
    case b.ActivePrefixes:
        return fmt.Errorf("InetStats(Bit==%d): ActivePrefixes was not parsed from the input", b.Bit)
    case b.AcceptPrefixes:
        return fmt.Errorf("InetStats(Bit==%d): AcceptPrefixes was not parsed from the input", b.Bit)
    case b.SurpressedPrefixes:
        return fmt.Errorf("InetStats(Bit==%d): SurpressedPrefixes was not parsed from the input", b.Bit)
    }

    switch {
    case b.RIBState == RSUnknown:
        return fmt.Errorf("InetStats(Bit==%d): RIBState was unknown, which indicates the parser is broken on input", b.Bit)
    case b.SendState == RSSendUnknown:
        return fmt.Errorf("InetStats(Bit==%d): SendState was unknown, which indicates the parser is broken on input", b.Bit)
    }
    return nil
}

// PeerParse is a collection of ParseFn for parsing top level entries in "show bgp neigbhors".
type PeerParser struct {
    parser *Parser
    peers  BGPNeighbors
}

// lastPeer returns the last *BgPNeighbor added to our *BGPNeighbors slice.
func (pe *PeerParser) lastPeer() *BGPNeighbor {
    if len(pe.peers) == 0 {
        return nil
    }
    return pe.peers[len(pe.peers)-1]
}

func (pe *PeerParser) errorf(s string, a ...interface{}) ParseFn {
    rec := pe.lastPeer()
    if rec == nil {
        return pe.parser.Errorf(s, a...)
    }

    return pe.parser.Errorf("Peer(%s+%d):Local(%s+%d) entry: %s", rec.PeerIP, rec.PeerPort, rec.LocalIP, rec.LocalPort, fmt.Sprintf(s, a...))
}

var peerRecStart = []string{"Peer:", Skip, "AS", Skip, "Local:", Skip, "AS", Skip}

// Peer: 10.10.10.2+179 AS 22     Local: 10.10.10.1+65406 AS 17
func (pe *PeerParser) FindPeer(ctx context.Context, p *Parser) ParseFn {
    const (
        peerIPPort  = 1
        peerASNum   = 3
        localIPPort = 5
        localASNum  = 7
    )
    pe.peers = p.Validator.(BGPNeighbors)
    pe.parser = p

    rec := &BGPNeighbor{}
    rec.init()

    line, err := p.FindStart(peerRecStart)
    if err != nil {
        if len(pe.peers) == 0 {
            p.Errorf("did not locate the start of our list of peers within the output")
        }
        return nil
    }
    if p.EOF(line) {
        return p.Errorf("received the start of a Peer statement, but then an EOF: %#+v", line)
    }

    // Get peer's ip and port.
    ip, port, err := ipPort(line.Items[peerIPPort].Val)
    if err != nil {
        return p.Errorf("coud not retrieve a valid peer IP and port from: %#+v", line)
    }
    rec.PeerIP = ip
    rec.PeerPort = uint32(port)

    // Get peer's AS.
    as, err := line.Items[peerASNum].ToInt()
    if err != nil {
        return p.Errorf("could not retrieve the peer AS num from: %#+v", line)
    }
    rec.PeerAS = as

    // Get local ip and port.
    ip, port, err = ipPort(line.Items[localIPPort].Val)
    if err != nil {
        return p.Errorf("coud not retrieve a valid local IP and port from: %#+v", line)
    }
    rec.LocalIP = ip
    rec.LocalPort = uint32(port)

    // Get local AS.
    as, err = line.Items[peerASNum].ToInt()
    if err != nil {
        return p.Errorf("could not retrieve the peer AS num from: %#+v", line)
    }
    rec.LocalAS = as

    pe.peers = append(pe.peers, rec)
    return pe.typeState
}

var toPeerType = map[string]PeerType{
    "Internal": PTInternal,
    "External": PTExternal,
}

var toState = map[string]BGPState{
    "Active":                 NSActive,
    "Connect":                NSConnect,
    "Established":            NSEstablished,
    "Idle":                   NSIdle,
    "OpenConfirm":            NSOpenConfirm,
    "OpenSent":               NSOpenSent,
    "route reflector client": NSRRClient,
}

// Type: External    State: Established    Flags: <Sync>
func (pe *PeerParser) typeState(ctx context.Context, p *Parser) ParseFn {
    const (
        peerType = 1
        state    = 3
    )

    line := p.Next()

    rec := pe.lastPeer()

    if !p.IsAtStart(line, []string{"Type:", Skip, "State:", Skip}) {
        return pe.errorf("did not have the expected 'Type' and 'State' declarations following peer line")
    }

    t, ok := toPeerType[line.Items[peerType].Val]
    if !ok {
        return pe.errorf("Type was not 'Internal' or 'External', was %s", line.Items[peerType].Val)
    }
    rec.Type = t

    s, ok := toState[line.Items[state].Val]
    if !ok {
        return pe.errorf("BGP State was not one of the accepted types (Active, Connect, ...), was %s", line.Items[state].Val)
    }
    rec.State = s

    return pe.lastState
}

// Last State: OpenConfirm   Last Event: RecvKeepAlive
func (pe *PeerParser) lastState(ctx context.Context, p *Parser) ParseFn {
    line := p.Next()

    rec := pe.lastPeer()

    if !p.IsAtStart(line, []string{"Last", "State:", Skip}) {
        return pe.errorf("did not have the expected 'Last State:', got %#+v", line)
    }

    s, ok := toState[line.Items[2].Val]
    if !ok {
        return pe.errorf("BGP last state was not one of the accepted types (Active, Connect, ...), was %s", line.Items[2].Val)
    }
    rec.LastState = s
    return pe.holdTimePref
}

// Holdtime: 90 Preference: 170
func (pe *PeerParser) holdTimePref(ctx context.Context, p *Parser) ParseFn {
    const (
        hold = 1
        pref = 3
    )

    rec := pe.lastPeer()

    line, until, err := p.FindUntil([]string{"Holdtime:", Skip, "Preference:", Skip}, peerRecStart)
    if err != nil {
        return pe.errorf("reached end of file before finding Holdtime and Preference line")
    }
    if until {
        return pe.errorf("reached next entry before finding Holdtime and Preference line")
    }

    ht, err := line.Items[hold].ToInt()
    if err != nil {
        return pe.errorf("Holdtime was not an integer, was %s", line.Items[hold].Val)
    }
    prefVal, err := line.Items[pref].ToInt()
    if err != nil {

        return pe.errorf("Preference was not an integer, was %s", line.Items[pref].Val)
    }

    rec.HoldTime = time.Duration(ht) * time.Second
    rec.Preference = prefVal
    return pe.peerIDLocalID
}

// Peer ID: 10.10.10.6       Local ID: 10.10.10.1       Active Holdtime: 90
func (pe *PeerParser) peerIDLocalID(ctx context.Context, p *Parser) ParseFn {
    const (
        peer  = 2
        local = 5
    )

    rec := pe.lastPeer()

    line, until, err := p.FindUntil([]string{"Peer", "ID:", Skip, "Local", "ID:", Skip}, peerRecStart)
    if err != nil {
        return pe.errorf("reached end of file before finding PeerID and LocalID")
    }
    if until {
        return pe.errorf("reached next entry before finding PeerID and LocalID")
    }
    pid := net.ParseIP(line.Items[peer].Val)
    if pid == nil {
        return pe.errorf("PeerID does not appear to be an IP: was %s", line.Items[peer].Val)
    }
    loc := net.ParseIP(line.Items[local].Val)
    if loc == nil {
        return pe.errorf("LocalID does not appear to be an IP: was %s", line.Items[local].Val)
    }
    rec.PeerID = pid
    rec.LocalID = loc
    return pe.findTableStats
}

// Table inet.0 Bit: 10000
func (pe *PeerParser) findTableStats(ctx context.Context, p *Parser) ParseFn {
    p.Validator = pe.peers

    _, until, err := p.FindUntil([]string{"Table", Skip, "Bit:", Skip}, peerRecStart)
    if err != nil {
        return nil
    }
    if until {
        return pe.FindPeer
    }
    p.Backup()
    ts := &tableStats{peer: pe}
    return ts.start
}

/*
Table inet.0 Bit: 10000
    RIB State: BGP restart is complete
    Send state: in sync
    Active prefixes:              0
    Received prefixes:            0
    Accepted prefixes:            0
    Suppressed due to damping:    0
    Advertised prefixes:          0
*/
type tableStats struct {
    peer  *PeerParser
    stats *InetStats
    rec   *BGPNeighbor
}

func (t *tableStats) errorf(s string, a ...interface{}) ParseFn {
    if t.stats == nil {
        return t.peer.errorf("Table(unknown): %s", fmt.Sprintf(s, a...))
    }
    return t.peer.errorf("Table(ID: %d, Bit: %d): %s", t.stats.ID, t.stats.Bit, fmt.Sprintf(s, a...))
}

// Table inet.0 Bit: 10000
func (t *tableStats) start(ctx context.Context, p *Parser) ParseFn {
    const (
        table = 1
        bit   = 3
    )
    t.rec = t.peer.lastPeer()

    line := p.Next()

    tvals := strings.Split(line.Items[table].Val, `.`)
    if len(tvals) != 2 {
        return t.errorf("had Table entry with table id that wasn't in a format I understand: %s", line.Items[table].Val)
    }
    i, err := strconv.Atoi(tvals[1])
    if err != nil {
        return t.errorf("had Table entry with table id that wasn't an integer: %s", tvals[1])
    }

    b, err := line.Items[bit].ToInt()
    if err != nil {
        return t.errorf("had Table entry with bits id that wasn't an integer: %s", line.Items[bit].Val)
        return nil
    }
    t.stats = &InetStats{
        ID:  i,
        Bit: b,
    }

    return t.ribState
}

var toRIBState = map[string]RIBState{
    "restart is complete": RSComplete,
    "estart in progress":  RSInProgress,
}

// RIB State: BGP restart is complete
func (t *tableStats) ribState(ctx context.Context, p *Parser) ParseFn {
    const begin = 3

    line := p.Next()

    if !p.IsAtStart(line, []string{"RIB", "State:", "BGP", Skip}) {
        return t.errorf("did not have the RIB State as expected")
        return nil
    }

    s := ItemJoin(line, begin, -1)
    v, ok := toRIBState[s]
    if !ok {
        return t.errorf("did not have a valid RIB State, had: %q", s)
    }

    t.stats.RIBState = v
    return t.sendState
}

var toSendState = map[string]SendState{
    "in sync":         RSSendSync,
    "not in sync":     RSSendNotSync,
    "not advertising": RSSendNoAdvertise,
}

// Send state: in sync
func (t *tableStats) sendState(ctx context.Context, p *Parser) ParseFn {
    const begin = 2

    line := p.Next()

    if !p.IsAtStart(line, []string{"Send", "state:", Skip}) {
        return t.errorf("did not have the Send state as expected")
        return nil
    }

    s := ItemJoin(line, begin, -1)
    v, ok := toSendState[s]
    if !ok {
        return t.errorf("did not have recognized Send state, had %s", s)
    }

    t.stats.SendState = v
    return t.active
}

// Active prefixes:              0
func (t *tableStats) active(ctx context.Context, p *Parser) ParseFn {
    i, err := t.intKeyVal([]string{"Active", "prefixes:", Skip}, p)
    if err != nil {
        return t.errorf(err.Error())
    }
    t.stats.ActivePrefixes = i
    return t.received
}

// Received prefixes:            0
func (t *tableStats) received(ctx context.Context, p *Parser) ParseFn {
    i, err := t.intKeyVal([]string{"Received", "prefixes:", Skip}, p)
    if err != nil {
        return t.errorf(err.Error())
    }
    t.stats.RecvPrefixes = i
    return t.accepted
}

// Accepted prefixes:            0
func (t *tableStats) accepted(ctx context.Context, p *Parser) ParseFn {
    i, err := t.intKeyVal([]string{"Accepted", "prefixes:", Skip}, p)
    if err != nil {
        return t.errorf(err.Error())
    }
    t.stats.AcceptPrefixes = i
    return t.supressed
}

// Suppressed due to damping:    0
func (t *tableStats) supressed(ctx context.Context, p *Parser) ParseFn {
    i, err := t.intKeyVal([]string{"Suppressed", "due", "to", "damping:", Skip}, p)
    if err != nil {
        return t.errorf(err.Error())
    }
    t.stats.SurpressedPrefixes = i
    return t.advertised
}

// Advertised prefixes:          0
func (t *tableStats) advertised(ctx context.Context, p *Parser) ParseFn {
    i, err := t.intKeyVal([]string{"Advertised", "prefixes:", Skip}, p)
    if err != nil {
        return t.errorf(err.Error())
    }
    t.stats.AdvertisedPrefixes = i
    return t.recordStats
}

func (t *tableStats) recordStats(ctx context.Context, p *Parser) ParseFn {
    if t.rec.InetStats == nil {
        t.rec.InetStats = map[int]*InetStats{}
    }
    t.rec.InetStats[t.stats.ID] = t.stats

    return t.peer.findTableStats
}

func (t *tableStats) intKeyVal(name []string, p *Parser) (int, error) {
    line := p.Next()
    if !p.IsAtStart(line, name) {
        return 0, fmt.Errorf("did not have %s as expected", strings.Join(name, " "))
    }

    item := line.Items[len(name)-1]
    v, err := item.ToInt()
    if err != nil {
        return 0, fmt.Errorf("did not have %s value as a int, had %v", strings.Join(name, " "), item.Val)
    }
    return v, nil
}

func ipPort(s string) (net.IP, int, error) {
    sp := strings.Split(s, `+`)
    if len(sp) != 2 {
        return nil, 0, fmt.Errorf("IP address and port could not be found with syntax <ip>+<port>: %s", s)
    }
    ip := net.ParseIP(sp[0])
    if ip == nil {
        return nil, 0, fmt.Errorf("IP address could not be parsed: %s", sp[0])
    }
    port, err := strconv.Atoi(sp[1])
    if err != nil {
        return nil, 0, fmt.Errorf("IP port could not be parsed from: %s", sp[1])
    }
    return ip, port, nil
}

Code:

package main

import (
    "context"
    "fmt"
    "regexp"
    "strconv"
    "strings"

    "github.com/kylelemons/godebug/pretty"
)

var showIntBrief = `
Doesn't matter what comes before
what we are looking for
Physical interface: ge-3/0/2, Enabled, Physical link is Up
  Link-level type: 52, MTU: 1522, Speed: 1000mbps, Loopback: Disabled,
  This is just some trash
Physical interface: ge-3/0/3, Enabled, Physical link is Up
  Link-level type: ppp, MTU: 1522, Speed: 1000mbps, Loopback: Disabled,
  This doesn't matter either
`

func main() {
    // Creates our parers object that our various ParseFn functions will use to move
    // through the input.
    p, err := NewParser(showIntBrief, Interfaces{})
    if err != nil {
        panic(err)
    }

    // An object that contains various ParseFn methods.
    states := &interBriefParsers{}

    // Parses our content in showBGPNeighbor and begins parsing with states.FindPeer
    // which is a ParseFn.
    if err := Parse(context.Background(), p, states.findInterface); err != nil {
        panic(err)
    }

    // Because we pass in a slice, we have to do a reassign to get the changed value.
    fmt.Println(pretty.Sprint(p.Validator.(Interfaces)))

}

type LinkLevel int8

const (
    LLUnknown  LinkLevel = 0
    LL52       LinkLevel = 1
    LLPPP      LinkLevel = 2
    LLEthernet LinkLevel = 3
)

type InterState int8

const (
    IStateUnknown  InterState = 0
    IStateEnabled  InterState = 1
    IStateDisabled InterState = 2
)

type InterStatus int8

const (
    IStatUnknown InterStatus = 0
    IStatUp      InterStatus = 1
    IStatDown    InterStatus = 2
)

// Interfaces is a collection of Interface information for a device.
type Interfaces []*Interface

func (i Interfaces) Validate() error {
    for _, v := range i {
        if err := v.Validate(); err != nil {
            return err
        }
    }
    return nil
}

// Interface is a brief decription of a network interface.
type Interface struct {
    // VendorDesc is the name a vendor gives the interface, like ge-10/2/1.
    VendorDesc string
    // Blade is the blade in the routing chassis.
    Blade int
    // Pic is the pic position on the blade.
    Pic int
    // Port is the port in the pic.
    Port int
    // State is the interface's current state.
    State InterState
    // Status is the interface's current status.
    Status InterStatus
    // LinkLevel is the type of encapsulation used on the link.
    LinkLevel LinkLevel
    // MTU is the maximum amount of bytes that can be sent on the frame.
    MTU int
    // Speed is the interface's speed in bits per second.
    Speed int

    initCalled bool
}

// init initializes Interface.
func (i *Interface) init() {
    i.Blade = -1
    i.Pic = -1
    i.Port = -1
    i.MTU = -1
    i.Speed = -1
    i.initCalled = true
}

// Validate implements halfpike.Validator.
func (i *Interface) Validate() error {
    if !i.initCalled {
        return fmt.Errorf("an Interface did not have init() called before storing data")
    }

    if i.VendorDesc == "" {
        return fmt.Errorf("an Interface did not have VendorDesc assigned")
    }

    switch -1 {
    case i.Blade:
        return fmt.Errorf("Interface(%s): Blade was not set", i.VendorDesc)
    case i.Pic:
        return fmt.Errorf("Interface(%s): Pic was not set", i.VendorDesc)
    case i.Port:
        return fmt.Errorf("Interface(%s): Port was not set", i.VendorDesc)
    case i.MTU:
        return fmt.Errorf("Interface(%s): MTU was not set", i.VendorDesc)
    case i.Speed:
        return fmt.Errorf("Interface(%s): Speed was not set", i.VendorDesc)
    }

    switch {
    case i.State == IStateUnknown:
        return fmt.Errorf("Interface(%s): State was not set", i.VendorDesc)
    case i.Status == IStatUnknown:
        return fmt.Errorf("Interface(%s): Status was not set", i.VendorDesc)
    case i.LinkLevel == LLUnknown:
        return fmt.Errorf("Interface(%s): LinkLevel was not set", i.VendorDesc)
    }

    return nil
}

type interBriefParsers struct {
    parser *Parser
    inters Interfaces
}

func (i *interBriefParsers) errorf(s string, a ...interface{}) ParseFn {
    if len(i.inters) > 0 {
        v := i.current().VendorDesc
        if v != "" {
            return i.parser.Errorf("interface(%s): %s", v, fmt.Sprintf(s, a...))
        }
    }
    return i.parser.Errorf(s, a...)
}

var phyStart = []string{"Physical", "interface:", Skip, Skip, "Physical", "link", "is", Skip}

// Physical interface: ge-3/0/2, Enabled, Physical link is Up
func (i *interBriefParsers) findInterface(ctx context.Context, p *Parser) ParseFn {
    if i.parser == nil {
        i.parser = p
    }

    // The Skip here says that we need to have an item here, but we don't care what it is.
    // This way we can deal with dynamic values and ensure we
    // have the minimum values we need.
    // p.FindItemsRegexStart() can be used if you require more
    // complex matching of static values.
    _, err := p.FindStart(phyStart)
    if err != nil {
        if len(i.inters) == 0 {
            return i.errorf("could not find a physical interface in the output")
        }
        return nil
    }
    // Create our new entry.
    inter := &Interface{}
    inter.init()
    i.inters = append(i.inters, inter)

    p.Backup() // I like to start all ParseFn with either Find...() or p.Next() for consistency.
    return i.phyInter
}

var toInterState = map[string]InterState{
    "Enabled,":  IStateEnabled,
    "Disabled,": IStateDisabled,
}

var toStatus = map[string]InterStatus{
    "Up":   IStatUp,
    "Down": IStatDown,
}

// Physical interface: ge-3/0/2, Enabled, Physical link is Up
func (i *interBriefParsers) phyInter(ctx context.Context, p *Parser) ParseFn {
    // These are indexes within the line where our values are.
    const (
        name        = 2
        stateIndex  = 3
        statusIndex = 7
    )
    line := p.Next() // fetches the next line of ouput.

    i.current().VendorDesc = line.Items[name].Val[:len(line.Items[name].Val)-1] // this will be ge-3/0/2 in the example above
    if err := i.interNameSplit(line.Items[name].Val); err != nil {
        return i.errorf("error parsing the name into blade/pic/port: %s", err)
    }

    state, ok := toInterState[line.Items[stateIndex].Val]
    if !ok {
        return i.errorf("error parsing the interface state, got %s is not a known state", line.Items[stateIndex].Val)
    }
    i.current().State = state

    status, ok := toStatus[line.Items[statusIndex].Val]
    if !ok {
        return i.errorf("error parsing the interface status, got %s which is not a known status", line.Items[statusIndex].Val)
    }
    i.current().Status = status
    return i.findLinkLevel
}

var toLinkLevel = map[string]LinkLevel{
    "52,":       LL52,
    "ppp,":      LLPPP,
    "ethernet,": LLEthernet,
}

// Link-level type: 52, MTU: 1522, Speed: 1000mbps, Loopback: Disabled,
func (i *interBriefParsers) findLinkLevel(ctx context.Context, p *Parser) ParseFn {
    const (
        llTypeIndex = 2
        mtuIndex    = 4
        speedIndex  = 6
    )

    line, until, err := p.FindUntil([]string{"Link-level", "type:", Skip, "MTU:", Skip, "Speed:", Skip}, phyStart)
    if err != nil {
        return i.errorf("did not find Link-level before end of file reached")
    }
    if until {
        return i.errorf("did not find Link-level before finding the next interface")
    }

    ll, ok := toLinkLevel[line.Items[llTypeIndex].Val]
    if !ok {
        return i.errorf("unknown link level type: %s", line.Items[llTypeIndex].Val)
    }
    i.current().LinkLevel = ll

    mtu, err := strconv.Atoi(strings.Split(line.Items[mtuIndex].Val, ",")[0])
    if err != nil {
        return i.errorf("mtu did not seem to be a valid integer: %s", line.Items[mtuIndex].Val)
    }
    i.current().MTU = mtu

    if err := i.speedSplit(line.Items[speedIndex].Val); err != nil {
        return i.errorf("problem interpreting the interface speed: %s", err)
    }

    return i.record
}

// record our data back to the parser.
func (i *interBriefParsers) record(ctx context.Context, p *Parser) ParseFn {
    i.parser.Validator = i.inters
    return i.findInterface
}

// ge-3/0/2
var interNameRE = regexp.MustCompile(`(?P<inttype>ge)-(?P<blade>\d+)/(?P<pic>\d+)/(?P<port>\d+),`)

func (i *interBriefParsers) interNameSplit(s string) error {
    matches, err := Match(interNameRE, s)
    if err != nil {
        return fmt.Errorf("error disecting the interface name(%s): %s", s, err)
    }

    for k, v := range matches {
        if k == "inttype" {
            continue
        }
        in, err := strconv.Atoi(v)
        if err != nil {
            return fmt.Errorf("could not convert value for %s(%s) to an integer", k, v)
        }
        switch k {
        case "blade":
            i.current().Blade = in
        case "pic":
            i.current().Pic = in
        case "port":
            i.current().Port = in
        }
    }
    return nil
}

var speedRE = regexp.MustCompile(`(?P<bits>\d+)(?P<desc>(kbps|mbps|gbps))`)
var bitsMultiplier = map[string]int{
    "kbps": 1000,
    "mbps": 1000 * 1000,
    "gbps": 1000 * 1000 * 1000,
}

func (i *interBriefParsers) speedSplit(s string) error {
    matches, err := Match(speedRE, s)
    if err != nil {
        return fmt.Errorf("error disecting the interfacd speed(%s): %s", s, err)
    }

    multi, ok := bitsMultiplier[matches["desc"]]
    if !ok {
        return fmt.Errorf("could not decipher the interface speed measurement: %s", matches["desc"])
    }

    bits, err := strconv.Atoi(matches["bits"])
    if err != nil {
        return fmt.Errorf("interface speed does not seem to be a integer: %s", matches["bits"])
    }
    i.current().Speed = bits * multi
    return nil
}

func (i *interBriefParsers) current() *Interface {
    if len(i.inters) == 0 {
        return nil
    }
    return i.inters[len(i.inters)-1]
}

Index

Examples

Package Files

halfpike.go interstate_string.go

Constants

const Skip = "$.<skip>.$"

Any provides a special string for FindStart that will skip an item.

func ItemJoin Uses

func ItemJoin(line Line, start, end int) string

ItemJoin takes a line, the inclusive beginning index and the non-inclusive ending index and joins all the values with a single space between them. -1 for start or end means from the absolute begin or end of the line slice. This will automatically remove the carriage return or EOF items.

func Match Uses

func Match(re *regexp.Regexp, s string) (map[string]string, error)

Match returns matches of the regex with keys set to the submatch names. If these are not named submatches (aka `(?P<name>regex)`) this will probably panic. A match that is empty string will cause an error to return.

func Parse Uses

func Parse(ctx context.Context, p *Parser, start ParseFn) error

Parse begins parsing content from the underlying input to Parser. Parsing starts with the "start" ParseFn until a returned ParseFn == nil. Calling p.HasError() will return an error if there was one. If err == nil, the Validator object passed to Parser should have .Validate() called to ensure all data is correct.

type Item Uses

type Item struct {
    // Type is the type of item that is stored in .Val.
    Type ItemType
    // Val is the value of the item that was in the text output.
    Val string
    // contains filtered or unexported fields
}

Item represents a token created by the Lexer.

func (Item) IsZero Uses

func (i Item) IsZero() bool

IsZero indicates the Item is the zero value.

func (Item) ToFloat Uses

func (i Item) ToFloat() (float64, error)

ToFloat returns the value as a float64 type. if the Item.Type is not itemFloat, this will panic.

func (Item) ToInt Uses

func (i Item) ToInt() (int, error)

ToInt returns the value as an int type. If the Item.Type is not ItemInt, this will panic.

type ItemType Uses

type ItemType int

ItemType describes the type of item being emitted by the Lexer. There are predefined ItemType(s) and the rest are defined by the user.

const (
    // ItemUnknown indicates that the Item is an unknown. This should only happen on
    // a Item that is the zero type.
    ItemUnknown ItemType = iota
    // ItemEOF indicates that the end of input is reached. No further tokens will be sent.
    ItemEOF
    // ItemText indicates that it is a block of text separated by some type of space (including tabs).
    // This may contain numbers, but if it is not a pure number it is contained in here.
    ItemText
    // ItemInt indicates that an integer was found.
    ItemInt
    // ItemFloat indicates that a float was found.
    ItemFloat
    // ItemEOL indicates the end of a line was reached.
    ItemEOL
)

type Line Uses

type Line struct {
    // Items are the Item(s) that make up a line.
    Items []Item
    // LineNum is the line number in the content this represents, starting at 1.
    LineNum int
    // Raw is the actual raw string that made up the line.
    Raw string
}

Line represents a line in the input.

type ParseFn Uses

type ParseFn func(ctx context.Context, p *Parser) ParseFn

ParseFn handles parsing items provided by a lexer into an object that implements the Validator interface.

type Parser Uses

type Parser struct {
    Validator Validator
    // contains filtered or unexported fields
}

Parser parses items coming from the Lexer and puts the values into *struct that must satisfy the Validator interface. It provides helper methods for recording an Item directory to a field handling text conversions. More complex types such as conversion to time.Time or custom objects are not covered.

func NewParser Uses

func NewParser(input string, val Validator) (*Parser, error)

NewParser is the constructor for Parser.

func (*Parser) Backup Uses

func (p *Parser) Backup() Line

Backup undoes a Next() call and returns the items in the previous line.

func (*Parser) Close Uses

func (p *Parser) Close()

TODO(johnsiilver): Implement this or we are going to leak goroutines. TODO(johnsiilver): Update the reset to also shut down the lexer so that we stop leaking.

func (*Parser) EOF Uses

func (p *Parser) EOF(line Line) bool

EOF returns true if the last Item in []Item is a ItemEOF.

func (*Parser) Errorf Uses

func (p *Parser) Errorf(str string, args ...interface{}) ParseFn

Errorf records an error in parsing. The ParseFn should immediately return nil. Errorf will always return a nil ParseFn.

func (*Parser) FindREStart Uses

func (p *Parser) FindREStart(find []*regexp.Regexp) (Line, error)

FindREStart looks for a match of [n]*regexp.Regexp against [n]Item.Val continuing to call .Next() until a match is found or EOF is reached. Once this is found, Line is returned. This is done from the current position.

func (*Parser) FindStart Uses

func (p *Parser) FindStart(find []string) (Line, error)

FindStart looks for an exact match of starting items in a line represented by Line continuing to call .Next() until a match is found or EOF is reached. Once this is found, Line is returned. This is done from the current position.

func (*Parser) FindUntil Uses

func (p *Parser) FindUntil(find []string, until []string) (matchFound Line, untilFound bool, err error)

FindUntil searches a Line until it matches "find", matches "until" or reaches the EOF. If "find" is matched, we return the Line. If "until" is matched, we call .Backup() and return true. This is useful when you wish to discover a line that represent a sub-entry of a record (find) but wish to stop searching if you find the beginning of the next record (until).

func (*Parser) HasError Uses

func (p *Parser) HasError() error

HasError returns if the Parser encountered an error.

func (*Parser) IsAtStart Uses

func (p *Parser) IsAtStart(line Line, find []string) bool

IsAtStart checks to see that "find" is at the beginning of "line".

func (*Parser) IsREStart Uses

func (p *Parser) IsREStart(line Line, find []*regexp.Regexp) bool

IsREStart checks to see that matches to "find" is at the beginning of "line".

func (*Parser) Next Uses

func (p *Parser) Next() Line

Next moves to the next Line sent from the Lexer. That Line is returned. If we haven't received the next Line, the Parser will block until that Line has been received.

func (*Parser) Peek Uses

func (p *Parser) Peek() Line

Peek returns the item in the next position, but does not change the current position.

func (*Parser) Reset Uses

func (p *Parser) Reset(s string, val Validator) error

Reset will reset the Parsers internal attributes for parsing new input "s" into "val".

type Validator Uses

type Validator interface {
    // Validate indicates if the type validates or not.
    Validate() error
}

Validator provides methods to validate that a data type is okay.

Package halfpike imports 7 packages (graph). Updated 2019-03-07. Refresh now. Tools for package owners.