quirk

package module
v2.1.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 18, 2019 License: Apache-2.0 Imports: 14 Imported by: 1

README

GoDoc Go Report Card License

Quirk is a library used to seamlessly use upsert procedures in Dgraph without going through the hassle yourself.

Some Quick Notes about Quirk v2

With the recent update of Dgraph and dgo, Quirk has been versioned to v2.0.0 and is now using the new module tag github.com/damienfamed75/quirk/v2. If you are using any Dgraph version below 1.1.0 and wish to still use quirk then please go ahead and download quirk v1.

Also Quirk will be updating to utilize further functionality added to dgo to optimize mutations as soon as possible. At the moment please enjoy Quirk v2 though in all its glory!

Install

To download Quirk v2, run this command to download the package.

go get github.com/damienfamed75/quirk/v2

If you wish to download Quirk v1, then run this command instead.

go get github.com/damienfamed75/quirk

Note if you are using Quirk v1 all imports in Go must not contain /v2

Using quirk

Here is a quick example of using a quirk client to insert a single node.

package main

import (
    "context"
    "fmt"

    "github.com/damienfamed75/quirk/v2"
    "github.com/dgraph-io/dgo/v2/protos/api"
    "github.com/dgraph-io/dgo/v2"
    "google.golang.org/grpc"
)

func main() {
    // Create some data to insert in Dgraph.
    person := struct {
        // These quirk tags are required.
        // It lets the quirk client know that this is the name
        // of the predicate in Dgraph.
        Name   string `quirk:"name"`
        SSN    string `quirk:"ssn,unique"` // Add unique if it should be upserted.
        Policy string `quirk:"policy,unique"`
        // As part of Dgraph 1.1.0 you can add types by just using the
        // "dgraph.type" tag using a string and Quirk will handle it correctly.
        Type string `quirk:"dgraph.type"`
    }{
        Name:   "Damien",
        SSN:    "123-12-1234",
        Policy: "ABCDAMIEN",
        Type:   "Person",
    }

    // Dial with GRPC to Dgraph as usual.
    conn, err := grpc.Dial("127.0.0.1:9080", grpc.WithInsecure())
    if err != nil {
        fmt.Println(err)
    }
    defer conn.Close()

    // Create the normal dgo client as usual.
    dg := dgo.NewDgraphClient(api.NewDgraphClient(conn))

    // Add the schema to Dgraph.
    // Make sure to mark the unique predicates with the "@upsert" directive.
    err = dg.Alter(context.Background(), &api.Operation{
        Schema: `
            name:   string @index(hash) .
            ssn:    string @index(hash) @upsert .
            policy: string @index(hash) @upsert .
        `,
    })
    if err != nil {
        fmt.Println(err)
    }

    // Create a new quirk client.
    q := quirk.NewClient()

    // Insert a single node. If we run this file multiple times you will
    // see that this node is never added twice.
    uids, err := q.InsertNode(context.Background(), dg, &quirk.Operation{
        SetSingleStruct: &person,
    })
    if err != nil {
        fmt.Println(err)
    }

    // Print out the returned node and its uid.
    for n, u := range uids {
        fmt.Printf("UIDMap: name[%s] uid[%s]\n", n, u)
    }
}

Contributing

Go ahead and create a PR or an issue. I'm always open for new contributions.

Documentation

Overview

Package quirk provides the main quirk.Client which is used to insert nodes into Dgraph. This client works with multiple types of data and will insert them concurrently if using the `multi` Operations found in quirk.Operation.

To get started using the quirk client you must create a Dgraph client using Dgraph's official `dgo` package at: 

https://github.com/dgraph-io/dgo

Once creating a Dgraph client you may begin using the functionality of the quirk client. Which currently is just using the `InsertNode` function.

To see examples on how to use the quirk client further then check out the 

`examples/`

directory found in the root of the repository. Or you can check out the Godoc examples listed below.

Example (InsertSingleNode) 
// Ignoring error handling for brevity.

// Dial for Dgraph using grpc.
conn, _ := grpc.Dial("127.0.0.1:9080", grpc.WithInsecure())
defer conn.Close()

// Create a new Dgraph client for our mutations.
dg := dgo.NewDgraphClient(api.NewDgraphClient(conn))

// Alter the schema to be equal to our schema variable.
dg.Alter(context.Background(), &api.Operation{Schema: `
		name: string @index(hash) .
		ssn: string @index(hash) @upsert .
		policy: string @index(hash) @upsert .
	`})

// Create the Quirk Client with a debug logger.
// The debug logger is just for demonstration purposes or for debugging.
c := quirk.NewClient(quirk.WithLogger(quirk.NewDebugLogger()))

// Use the quirk client to insert a single node.
uidMap, _ := c.InsertNode(context.Background(), dg, &quirk.Operation{
	SetSingleStruct: &struct {
		Name   string `quirk:"name"`
		SSN    string `quirk:"ssn,unique"`
		Policy int    `quirk:"policy,unique"`
	}{
		Name:   "Damien",
		SSN:    "126",
		Policy: 61238,
	},
})

// Finally print out the UIDs of the nodes that were inserted or updated.
// The key is going to be either your assigned "name" predicate.
// Note: If you wish to use another predicate beside "name"
// you may set that when creating the client and using
// quirk.WithPredicateKey(predicateName string)
for k, v := range uidMap {
	log.Printf("UIDMap: [%s] [%s:%v]\n", k, v.Value(), v.IsNew())
}

Output:

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewCustomLogger 

func NewCustomLogger(level []byte, config zapcore.EncoderConfig) yalp.Logger

NewCustomLogger returns a *yalp.CustomLogger with the desired level and encoder configuration that may be passed in.

func NewDebugLogger 

func NewDebugLogger() yalp.Logger

NewDebugLogger is a debug level zap logger that can be used when testing.

func NewNilLogger 

func NewNilLogger() yalp.Logger

NewNilLogger returns the *yalp.NilLogger logger which doesn't log anything.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is used to store enough data and help manage the logger when inserting nodes into Dgraph using a proper upsert procedure.

func NewClient 

func NewClient(confs ...ClientConfiguration) *Client

NewClient will setup a new client with the passed in configurations if so chosen to use any.

func (*Client) GetPredicateKey 

func (c *Client) GetPredicateKey() string

GetPredicateKey returns the name of the field(predicate) that will be used to label inserted nodes. By default this is "name"

Example 
client := quirk.NewClient()

// Should be "name"
fmt.Println(client.GetPredicateKey())

_ = client

Output:

func (*Client) InsertMultiDynamicNode added in v2.1.0 

func (c *Client) InsertMultiDynamicNode(ctx context.Context, dg *dgo.Dgraph, dat ...interface{}) (map[string]UID, error)

InsertMultiDynamicNode takes in a variadic number of interfaces as data. This function was added, because converting everything to []interface{} in someone's program proved to be inconvenient.

func (*Client) InsertNode 

func (c *Client) InsertNode(ctx context.Context, dg *dgo.Dgraph, o *Operation) (map[string]UID, error)

InsertNode takes in an Operation to determine if multiple nodes will be added or a single node. Then the function will return a map of the returned successful UIDs with the key being the predicate key value. By default this will be the "name" predicate value.

Example (DynamicMap) 
conn, _ := grpc.Dial("127.0.0.1:9080", grpc.WithInsecure())
defer conn.Close()

dg := dgo.NewDgraphClient(api.NewDgraphClient(conn))
c := quirk.NewClient()

// Maps do not support unique predicates.
// Interface maps (Dynamic maps) support multiple datatypes in Dgraph.
data := make(map[string]interface{})
data["name"] = "Damien"
data["age"] = 19

c.InsertNode(context.Background(), dg, &quirk.Operation{
	SetDynamicMap: data,
})

Output:
Example (MultiDupleNode) 
conn, _ := grpc.Dial("127.0.0.1:9080", grpc.WithInsecure())
defer conn.Close()

dg := dgo.NewDgraphClient(api.NewDgraphClient(conn))
c := quirk.NewClient()

data := []*quirk.DupleNode{
	&quirk.DupleNode{
		Identifier: "Damien",
		Duples: []quirk.Duple{
			{Predicate: "name", Object: "Damien"},
			{Predicate: "ssn", Object: "126", IsUnique: true},
			{Predicate: "policy", Object: 61238, IsUnique: true},
		},
	},
	&quirk.DupleNode{
		Identifier: "George",
		Duples: []quirk.Duple{
			{Predicate: "name", Object: "George"},
			{Predicate: "ssn", Object: "125", IsUnique: true},
			{Predicate: "policy", Object: 67234, IsUnique: true},
		},
	},
}

c.InsertNode(context.Background(), dg, &quirk.Operation{
	SetMultiDupleNode: data,
})

Output:
Example (MultiStruct) 
conn, _ := grpc.Dial("127.0.0.1:9080", grpc.WithInsecure())
defer conn.Close()

dg := dgo.NewDgraphClient(api.NewDgraphClient(conn))
c := quirk.NewClient()

type Person struct {
	Name   string `quirk:"name"`
	SSN    string `quirk:"ssn,unique"`
	Policy int    `quirk:"policy,unique"`
}

// Multi structs must be inserted using a slice of interfaces.
data := []interface{}{
	&Person{
		Name:   "Damien",
		SSN:    "126",
		Policy: 61238,
	}, &Person{
		Name:   "George",
		SSN:    "125",
		Policy: 67234,
	},
}

c.InsertNode(context.Background(), dg, &quirk.Operation{
	SetMultiStruct: data,
})

Output:
Example (SingleDupleNode) 
conn, _ := grpc.Dial("127.0.0.1:9080", grpc.WithInsecure())
defer conn.Close()

dg := dgo.NewDgraphClient(api.NewDgraphClient(conn))
c := quirk.NewClient()

data := &quirk.DupleNode{
	Identifier: "Damien",
	Duples: []quirk.Duple{
		{Predicate: "name", Object: "Damien"},
		{Predicate: "ssn", Object: "126", IsUnique: true},
		{Predicate: "policy", Object: 61238, IsUnique: true},
	},
}

c.InsertNode(context.Background(), dg, &quirk.Operation{
	SetSingleDupleNode: data,
})

Output:
Example (SingleStruct) 
conn, _ := grpc.Dial("127.0.0.1:9080", grpc.WithInsecure())
defer conn.Close()

dg := dgo.NewDgraphClient(api.NewDgraphClient(conn))
c := quirk.NewClient()

// Single struct must be a reference to a struct in order for
// reflect to not throw a panic.
data := &struct {
	Name   string `quirk:"name"`
	SSN    string `quirk:"ssn,unique"`
	Policy int    `quirk:"policy,unique"`
}{
	Name:   "Damien",
	SSN:    "126",
	Policy: 61238,
}

c.InsertNode(context.Background(), dg, &quirk.Operation{
	SetSingleStruct: data,
})

Output:
Example (StringMap) 
conn, _ := grpc.Dial("127.0.0.1:9080", grpc.WithInsecure())
defer conn.Close()

dg := dgo.NewDgraphClient(api.NewDgraphClient(conn))
c := quirk.NewClient()

// Maps do not support unique predicates.
data := make(map[string]string)
data["name"] = "Damien"
data["age"] = "19"

c.InsertNode(context.Background(), dg, &quirk.Operation{
	SetStringMap: data,
})

Output:

type ClientConfiguration 

type ClientConfiguration func(*Client)

ClientConfiguration is used to pass in options to change the client and customize it to the user's liking.

func WithLogger 

func WithLogger(l yalp.Logger) ClientConfiguration

WithLogger sets the logger used by the quirk client. By default this is quirk.NewNilLogger.

Example 
client := quirk.NewClient(
	quirk.WithLogger(quirk.NewDebugLogger()),
)

_ = client

Output:

func WithMaxWorkerCount 

func WithMaxWorkerCount(count int) ClientConfiguration

WithMaxWorkerCount will set the maximum workers that will be spun when using a Multi operation.

func WithPredicateKey 

func WithPredicateKey(predicateName string) ClientConfiguration

WithPredicateKey sets the field(predicate) that will be used to label inserted nodes. By default this is "name"

Example 
client := quirk.NewClient(
	quirk.WithPredicateKey("label"),
)

// Now that the predicate key is set to "label" the returning UID map
// will use the predicate value of "label" for the key rather than "name"

// Should be "label"
fmt.Println(client.GetPredicateKey())

_ = client

Output:

func WithTemplate 

func WithTemplate(tmpl string) ClientConfiguration

WithTemplate sets the field in the Quirk client that uses a progress bar to show the nodes being inserted with multi node sets.

Example 
// Using github.com/cheggaaa/pb/v3 for progress bar.
client := quirk.NewClient(
	quirk.WithTemplate(`{{ "Custom:" }} {{ bar . "[" "-" (cycle . ">" ) " " "]"}} [{{etime . | cyan }}:{{rtime . | cyan }}]`),
)

_ = client

Output:

type Duple 

type Duple struct {
	// Predicate acts as a key.
	Predicate string
	// Object is the data representing the predicate.
	Object interface{}
	// IsUnique stores whether or not to treat this as an upsert or not.
	IsUnique bool
	// contains filtered or unexported fields
}

Duple is a structural way of giving the quirk client enough information about a node to create triples and insert them into Dgraph.

Example 
duple := &quirk.Duple{
	Predicate: "name",
	Object:    "Damien",
	IsUnique:  false,
}

_ = duple

Output:

type DupleNode 

type DupleNode struct {
	Identifier string
	Duples     []Duple
}

DupleNode is the container for a duple node.

Example 
node := &quirk.DupleNode{
	Identifier: "person", // used for the key in the returned UID Map.
	Duples: []quirk.Duple{ // predicate value pairs.
		{Predicate: "name", Object: "Damien"},
		{Predicate: "ssn", Object: "126", IsUnique: true},
		{Predicate: "policy", Object: 61238, IsUnique: true},
	},
}

_ = node

Output:

func (*DupleNode) AddDuples 

func (d *DupleNode) AddDuples(duple ...Duple) *DupleNode

AddDuples appends new duples given in the function. Then returns the reference to the DupleNode. This function doesn't support updating previously added Duples though. This should only be used when trying to optimize appending new duples.

Example 
node := &quirk.DupleNode{}

// Adds new Duples. This doesn't support updating predicate values.
node.AddDuples(
	quirk.Duple{Predicate: "age", Object: 20},
	quirk.Duple{Predicate: "username", Object: "damienfamed75"},
)

_ = node

Output:

func (*DupleNode) Find 

func (d *DupleNode) Find(predicate string) *Duple

Find will return a reference to a duple given that it is found in the slice of duples in the DupleNode.

Example 
node := &quirk.DupleNode{
	Duples: []quirk.Duple{
		{Predicate: "name", Object: "Damien"},
	},
}

// Find a Duple stored in the DupleNode.
name := node.Find("name")

fmt.Printf("%s: %v unique[%v]\n", name.Predicate, name.Object, name.IsUnique)

_ = node

Output:

func (*DupleNode) SetOrAdd 

func (d *DupleNode) SetOrAdd(duple Duple) *DupleNode

SetOrAdd will set a pre existing duple in the DupleNode or if the Duple doesn't exist, then it will be added to the Node.

Example 
node := &quirk.DupleNode{
	Duples: []quirk.Duple{
		{Predicate: "age", Object: 19},
	},
}

// Updates the previous valued stored in DupleNode.
node.SetOrAdd(
	quirk.Duple{Predicate: "age", Object: 20},
)

_ = node

Output:

func (*DupleNode) Unique 

func (d *DupleNode) Unique() (duples []Duple)

Unique will loop through the Duples and return a new slice containing all duples that are marked as unique.

Example 
node := &quirk.DupleNode{
	Identifier: "person", // used for the key in the returned UID Map.
	Duples: []quirk.Duple{ // predicate value pairs.
		{Predicate: "name", Object: "Damien"},
		{Predicate: "ssn", Object: "126", IsUnique: true},
		{Predicate: "policy", Object: 61238, IsUnique: true},
	},
}

// returns a slice of all the predicates labeled as unique.
uniqueDuples := node.Unique()

// Should be 2.
fmt.Printf("num of unique nodes [%v]\n", len(uniqueDuples))

_ = node

Output:

type Error 

type Error struct {
	ExtErr   error
	Msg      string
	File     string
	Function string
}

Error is a general error that isn't super specific. Just used for when there needs to be more context relating to an error.

func (*Error) Error 

func (e *Error) Error() string

func (*Error) Unwrap added in v2.1.6 

func (e *Error) Unwrap() error

Unwrap is used to return any external errors. This function was implemented for Go 1.13.

type Operation 

type Operation struct {
	SetMultiStruct     []interface{}
	SetSingleStruct    interface{}
	SetStringMap       map[string]string
	SetDynamicMap      map[string]interface{}
	SetSingleDupleNode *DupleNode
	SetMultiDupleNode  []*DupleNode
}

Operation is the main parameter used when calling quirk client methods. Note: only one of these should be filled at a time, because only one will be executed and taken care of as seen in client.go

type QueryError 

type QueryError struct {
	ExtErr   error
	Msg      string
	Function string
	Query    string
}

QueryError is used for functions in the query.go file.

func (*QueryError) Error 

func (e *QueryError) Error() (res string)

func (*QueryError) Unwrap added in v2.1.6 

func (e *QueryError) Unwrap() error

Unwrap is used to return any external errors. This function was implemented for Go 1.13.

type TransactionError 

type TransactionError struct {
	ExtErr   error
	Msg      string
	Function string
	RDF      string
}

TransactionError is for when a transaction fails during a mutation.

func (*TransactionError) Error 

func (e *TransactionError) Error() string

func (*TransactionError) Unwrap added in v2.1.6 

func (e *TransactionError) Unwrap() error

Unwrap is used to return any external errors. This function was implemented for Go 1.13.

type UID 

type UID struct {
	// contains filtered or unexported fields
}

UID is used to identify the ID's given to the user and retrieved back to be put as the object of a predicate. This way quirk can handle the UID how they're supposed to be handled. Note: Use this struct as the Object for Duples to create relationships.

func (UID) IsNew 

func (u UID) IsNew() bool

IsNew returns a simple boolean value indicating whether or not this node was a newly added node or if it was pre existing in Dgraph.

func (UID) Value 

func (u UID) Value() string

Value returns the raw string value of the UID. Note: Do not use this as the Object for Duples to create relationships.

Source Files

View all Source files

Directories

Path Synopsis
examples
edges
largeload
schemacheck
setdynamicmap
setmulti
setmultiduple
setsingle
setsingleduple
setstringmap
withpredicatekey

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.