safer

package module
v1.5.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 3, 2021 License: Apache-2.0 Imports: 8 Imported by: 1

README

SAFER Scraper API [GoLang]

Go Report Card Go Reference codecov Tests

An API to scrape data from the Department of Transportation's Safety and Fitness Electronic Records (SAFER) System.

Scraping is performed using htmlquery, this project's only non std lib dependency.

Installation

go get -u github.com/brandenc40/safer

Client Methods

// GetCompanyByDOTNumber - Get a company snapshot by the companies DOT number. Returns ErrCompanyNotFound if
// no company is found
func (c *Client) GetCompanyByDOTNumber(dotNumber string) (*CompanySnapshot, error)

// GetCompanyByMCMX - Get a company snapshot by the companies MC/MX number. Returns ErrCompanyNotFound if no
// company is found.
//
// Note: do not include the prefix. (e.g. use "133655" not "MC-133655")
func (c *Client) GetCompanyByMCMX(mcmx string) (*CompanySnapshot, error)

// SearchCompaniesByName - Search for all carriers with a given name. Name queries will return the best matched results
// in a slice of CompanyResult structs.
func (c *Client) SearchCompaniesByName(name string) ([]CompanyResult, error)
Build a new Client
package main

import (
	"github.com/brandenc40/safer"
)

func main() {
	client := safer.NewClient()
	//... use the client
}
Scraping Benchmark

Benchmarks only test the time taken to parse the html and map it back to the output. Server time is ignored here.

goos: darwin
goarch: arm64
pkg: github.com/brandenc40/safer
BenchmarkClient_GetCompanyByDOTNumber-8             9145            132111 ns/op           91557 B/op       2672 allocs/op
BenchmarkClient_Search_4Results-8                  94274             12821 ns/op            9559 B/op        285 allocs/op
BenchmarkClient_Search_484Results-8                 1048           1137919 ns/op          826245 B/op      24774 allocs/op
PASS
ok      github.com/brandenc40/safer     4.942s

Documentation

Index

Examples

Constants

This section is empty.

Variables

View Source 
var (
	// ErrCompanyNotFound is thrown when a company is not found for the searched MC/MX/DOT number
	ErrCompanyNotFound = errors.New("company not found")
)

Functions

This section is empty.

Types

type Client added in v1.0.0 

type Client struct {
	// contains filtered or unexported fields
}

Client for scraping company details from SAFER

func NewClient added in v1.0.0 

func NewClient() *Client

NewClient build's a new Client interface

func (*Client) GetCompanyByDOTNumber added in v1.0.0 

func (c *Client) GetCompanyByDOTNumber(dotNumber string) (*CompanySnapshot, error)

GetCompanyByDOTNumber - Get a company snapshot by the companies DOT number. Returns ErrCompanyNotFound if no company is found

Example 
package main

import (
	"fmt"
	"log"

	"github.com/brandenc40/safer"
)

func main() {
	client := safer.NewClient()

	snapshot, err := client.GetCompanyByDOTNumber("1003306")
	if err != nil {
		log.Fatalln(err)
	}
	fmt.Printf("%#v", snapshot)
}

Output:

func (*Client) GetCompanyByMCMX added in v1.0.0 

func (c *Client) GetCompanyByMCMX(mcmx string) (*CompanySnapshot, error)

GetCompanyByMCMX - Get a company snapshot by the companies MC/MX number. Returns ErrCompanyNotFound if no company is found.

Note: do not include the prefix. (e.g. use "133655" not "MC-133655")

Example 
package main

import (
	"fmt"
	"log"

	"github.com/brandenc40/safer"
)

func main() {
	client := safer.NewClient()

	snapshot, err := client.GetCompanyByMCMX("133655")
	if err != nil {
		log.Fatalln(err)
	}
	fmt.Printf("%#v", snapshot)
}

Output:

func (*Client) SearchCompaniesByName added in v1.0.0 

func (c *Client) SearchCompaniesByName(name string) ([]CompanyResult, error)

SearchCompaniesByName - Search for all carriers with a given name. Name queries will return the best matched results in a slice of CompanyResult structs.

Example 
package main

import (
	"fmt"
	"log"

	"github.com/brandenc40/safer"
)

func main() {
	client := safer.NewClient()

	res, err := client.SearchCompaniesByName("Schneider")
	if err != nil {
		log.Fatalln(err)
	}
	fmt.Printf("%#v", res[0])
}

Output:

type CompanyResult 

type CompanyResult struct {
	Name      string `json:"name"`
	DOTNumber string `json:"dot_number"`
	Location  string `json:"location"`
}

CompanyResult is the search result returned from a company query by name

type CompanySnapshot 

type CompanySnapshot struct {
	USVehicleInspections     InspectionSummary `json:"us_vehicle_inspections"`
	USDriverInspections      InspectionSummary `json:"us_driver_inspections"`
	USHazmatInspections      InspectionSummary `json:"us_hazmat_inspections"`
	USIEPInspections         InspectionSummary `json:"us_iep_inspections"`
	CanadaVehicleInspections InspectionSummary `json:"canada_vehicle_inspections"`
	CanadaDriverInspections  InspectionSummary `json:"canada_driver_inspections"`
	USCrashes                CrashSummary      `json:"us_crashes"`
	CanadaCrashes            CrashSummary      `json:"canada_crashes"`
	Safety                   SafetyRating      `json:"safety"`
	LatestUpdateDate         *time.Time        `json:"latest_update_date"`
	OutOfServiceDate         *time.Time        `json:"out_of_service_date"`
	MCS150FormDate           *time.Time        `json:"mcs_150_form_date"`
	OperationClassification  []string          `json:"operation_classification"`
	CarrierOperation         []string          `json:"carrier_operation"`
	CargoCarried             []string          `json:"cargo_carried"`
	LegalName                string            `json:"legal_name"`
	DBAName                  string            `json:"dba_name"`
	EntityType               string            `json:"entity_type"`
	PhysicalAddress          string            `json:"physical_address"`
	Phone                    string            `json:"phone"`
	MailingAddress           string            `json:"mailing_address"`
	DOTNumber                string            `json:"dot_number"`
	StateCarrierID           string            `json:"state_carrier_id"`
	MCMXFFNumbers            []string          `json:"mc_mx_ff_numbers"`
	DUNSNumber               string            `json:"duns_number"`
	MCS150Mileage            int               `json:"mcs_150_mileage"`
	MCS150Year               string            `json:"mcs_150_year"`
	OperatingStatus          string            `json:"operating_status"`
	PowerUnits               int               `json:"power_units"`
	Drivers                  int               `json:"drivers"`
}

CompanySnapshot data parsed from the https://safer.fmcsa.dot.gov/CompanySnapshot.aspx website

type CrashSummary 

type CrashSummary struct {
	Fatal  int `json:"fatal"`
	Injury int `json:"injury"`
	Tow    int `json:"tow"`
	Total  int `json:"total"`
}

CrashSummary for 24 months prior to LatestUpdateDate

type InspectionSummary 

type InspectionSummary struct {
	Inspections     int     `json:"inspections"`
	OutOfService    int     `json:"out_of_service"`
	OutOfServicePct float32 `json:"out_of_service_pct"`
	NationalAverage float32 `json:"national_average"`
}

InspectionSummary for 24 months prior to LatestUpdateDate.

Note: NationalAverage not available for Canadian summaries

type SafetyRating 

type SafetyRating struct {
	RatingDate *time.Time `json:"rating_date"`
	ReviewDate *time.Time `json:"review_date"`
	Rating     string     `json:"rating"`
	Type       string     `json:"type"`
}

SafetyRating current as of LatestUpdateDate

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.