quirk

package module
v0.0.0-...-5f0fd85 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 29, 2019 License: Apache-2.0 Imports: 15 Imported by: 0

README

GoDoc Go Report Card License

Quirk is a library used to seamlessly use upsert procedures in Dgraph without going through the hassle yourself.

Install

Run this command to download the package.

go get github.com/damienfamed75/quirk

Using quirk

Here is a quick example of using a quirk client to insert a single node.

package main

import (
    "context"
    "fmt"

    "github.com/dgraph-io/dgo/protos/api"
    "github.com/damienfamed75/quirk"
    "github.com/dgraph-io/dgo"
    "google.golang.org/grpc"
)

func main() {
    // Create some data to insert in Dgraph.
    person := struct {
        // These quirk tags are required.
        // It lets the quirk client know that this is the name
        // of the predicate in Dgraph.
        Name   string `quirk:"name"`
        SSN    string `quirk:"ssn,unique"` // Add unique if it should be upserted.
        Policy string `quirk:"policy,unique"`
    }{
        Name:   "Damien",
        SSN:    "123-12-1234",
        Policy: "ABCDAMIEN",
    }

    // Dial with GRPC to Dgraph as usual.
    conn, err := grpc.Dial("127.0.0.1:9080", grpc.WithInsecure())
    if err != nil {
        fmt.Println(err)
    }
    defer conn.Close()

    // Create the normal dgo client as usual.
    dg := dgo.NewDgraphClient(api.NewDgraphClient(conn))

    // Add the schema to Dgraph.
    // Make sure to mark the unique predicates with the "@upsert" directive.
    err = dg.Alter(context.Background(), &api.Operation{
        Schema: `
            name:   string @index(hash) .
            ssn:    string @index(hash) @upsert .
            policy: string @index(hash) @upsert .
        `,
    })
    if err != nil {
        fmt.Println(err)
    }

    // Create a new quirk client.
    q := quirk.NewClient()

    // Insert a single node. If we run this file multiple times you will
    // see that this node is never added twice.
    uids, err := q.InsertNode(context.Background(), dg, &quirk.Operation{
        SetSingleStruct: &person,
    })
    if err != nil {
        fmt.Println(err)
    }

    // Print out the returned node and its uid.
    for n, u := range uids {
        fmt.Printf("UIDMap: name[%s] uid[%s]\n", n, u)
    }
}

Contributing

Go ahead and create a PR or an issue. I'm always open for new contributions.

Documentation

Overview

Package quirk provides the main quirk.Client which is used to insert nodes into Dgraph. This client works with multiple types of data and will insert them concurrently if using the `multi` Operations found in quirk.Operation.

To get started using the quirk client you must create a Dgraph client using Dgraph's official `dgo` package at: 

https://github.com/dgraph-io/dgo

Once creating a Dgraph client you may begin using the functionality of the quirk client. Which currently is just using the `InsertNode` function.

To see examples on how to use the quirk client further then check out the 

`examples/`

directory found in the root of the repository. Or you can check out the Godoc examples listed below.

Example (InsertSingleNode) 
package main

import (
	"context"
	"log"

	"github.com/damienfamed75/quirk"
	"github.com/dgraph-io/dgo"
	"github.com/dgraph-io/dgo/protos/api"
	"google.golang.org/grpc"
)

func main() {
	// Ignoring error handling for brevity.

	// Dial for Dgraph using grpc.
	conn, _ := grpc.Dial("127.0.0.1:9080", grpc.WithInsecure())
	defer conn.Close()

	// Create a new Dgraph client for our mutations.
	dg := dgo.NewDgraphClient(api.NewDgraphClient(conn))

	// Alter the schema to be equal to our schema variable.
	dg.Alter(context.Background(), &api.Operation{Schema: `
		name: string @index(hash) .
		ssn: string @index(hash) @upsert .
		policy: string @index(hash) @upsert .
	`})

	// Create the Quirk Client with a debug logger.
	// The debug logger is just for demonstration purposes or for debugging.
	c := quirk.NewClient(quirk.WithLogger(quirk.NewDebugLogger()))

	// Use the quirk client to insert a single node.
	uidMap, _ := c.InsertNode(context.Background(), dg, &quirk.Operation{
		SetSingleStruct: &struct {
			Name   string `quirk:"name"`
			SSN    string `quirk:"ssn,unique"`
			Policy int    `quirk:"policy,unique"`
		}{
			Name:   "Damien",
			SSN:    "126",
			Policy: 61238,
		},
	})

	// Finally print out the UIDs of the nodes that were inserted or updated.
	// The key is going to be either your assigned "name" predicate.
	// Note: If you wish to use another predicate beside "name"
	// you may set that when creating the client and using
	// quirk.WithPredicateKey(predicateName string)
	for k, v := range uidMap {
		log.Printf("UIDMap: [%s] [%s:%v]\n", k, v.Value(), v.IsNew())
	}
}

Output:

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewCustomLogger 

func NewCustomLogger(level []byte, config zapcore.EncoderConfig) yalp.Logger

NewCustomLogger returns a *yalp.CustomLogger with the desired level and encoder configuration that may be passed in.

func NewDebugLogger 

func NewDebugLogger() yalp.Logger

NewDebugLogger is a debug level zap logger that can be used when testing.

func NewNilLogger 

func NewNilLogger() yalp.Logger

NewNilLogger returns the *yalp.NilLogger logger which doesn't log anything.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is used to store enough data and help manage the logger when inserting nodes into Dgraph using a proper upsert procedure.

func NewClient 

func NewClient(confs ...ClientConfiguration) *Client

NewClient will setup a new client with the passed in configurations if so chosen to use any.

func (*Client) GetPredicateKey 

func (c *Client) GetPredicateKey() string

GetPredicateKey returns the name of the field(predicate) that will be used to label inserted nodes. By default this is "name"

Example 
package main

import (
	"fmt"

	"github.com/damienfamed75/quirk"
)

func main() {
	client := quirk.NewClient()

	// Should be "name"
	fmt.Println(client.GetPredicateKey())

	_ = client
}

Output:

func (*Client) InsertNode 

func (c *Client) InsertNode(ctx context.Context, dg *dgo.Dgraph, o *Operation) (map[string]UID, error)

InsertNode takes in an Operation to determine if multiple nodes will be added or a single node. Then the function will return a map of the returned successful UIDs with the key being the predicate key value. By default this will be the "name" predicate value.

Example (DynamicMap) 
package main

import (
	"context"

	"github.com/damienfamed75/quirk"
	"github.com/dgraph-io/dgo"
	"github.com/dgraph-io/dgo/protos/api"
	"google.golang.org/grpc"
)

func main() {
	conn, _ := grpc.Dial("127.0.0.1:9080", grpc.WithInsecure())
	defer conn.Close()

	dg := dgo.NewDgraphClient(api.NewDgraphClient(conn))
	c := quirk.NewClient()

	// Maps do not support unique predicates.
	// Interface maps (Dynamic maps) support multiple datatypes in Dgraph.
	data := make(map[string]interface{})
	data["name"] = "Damien"
	data["age"] = 19

	c.InsertNode(context.Background(), dg, &quirk.Operation{
		SetDynamicMap: data,
	})
}

Output:
Example (MultiDupleNode) 
package main

import (
	"context"

	"github.com/damienfamed75/quirk"
	"github.com/dgraph-io/dgo"
	"github.com/dgraph-io/dgo/protos/api"
	"google.golang.org/grpc"
)

func main() {
	conn, _ := grpc.Dial("127.0.0.1:9080", grpc.WithInsecure())
	defer conn.Close()

	dg := dgo.NewDgraphClient(api.NewDgraphClient(conn))
	c := quirk.NewClient()

	data := []*quirk.DupleNode{
		&quirk.DupleNode{
			Identifier: "Damien",
			Duples: []quirk.Duple{
				{Predicate: "name", Object: "Damien"},
				{Predicate: "ssn", Object: "126", IsUnique: true},
				{Predicate: "policy", Object: 61238, IsUnique: true},
			},
		},
		&quirk.DupleNode{
			Identifier: "George",
			Duples: []quirk.Duple{
				{Predicate: "name", Object: "George"},
				{Predicate: "ssn", Object: "125", IsUnique: true},
				{Predicate: "policy", Object: 67234, IsUnique: true},
			},
		},
	}

	c.InsertNode(context.Background(), dg, &quirk.Operation{
		SetMultiDupleNode: data,
	})
}

Output:
Example (MultiStruct) 
package main

import (
	"context"

	"github.com/damienfamed75/quirk"
	"github.com/dgraph-io/dgo"
	"github.com/dgraph-io/dgo/protos/api"
	"google.golang.org/grpc"
)

func main() {
	conn, _ := grpc.Dial("127.0.0.1:9080", grpc.WithInsecure())
	defer conn.Close()

	dg := dgo.NewDgraphClient(api.NewDgraphClient(conn))
	c := quirk.NewClient()

	type Person struct {
		Name   string `quirk:"name"`
		SSN    string `quirk:"ssn,unique"`
		Policy int    `quirk:"policy,unique"`
	}

	// Multi structs must be inserted using a slice of interfaces.
	data := []interface{}{
		&Person{
			Name:   "Damien",
			SSN:    "126",
			Policy: 61238,
		}, &Person{
			Name:   "George",
			SSN:    "125",
			Policy: 67234,
		},
	}

	c.InsertNode(context.Background(), dg, &quirk.Operation{
		SetMultiStruct: data,
	})
}

Output:
Example (SingleDupleNode) 
package main

import (
	"context"

	"github.com/damienfamed75/quirk"
	"github.com/dgraph-io/dgo"
	"github.com/dgraph-io/dgo/protos/api"
	"google.golang.org/grpc"
)

func main() {
	conn, _ := grpc.Dial("127.0.0.1:9080", grpc.WithInsecure())
	defer conn.Close()

	dg := dgo.NewDgraphClient(api.NewDgraphClient(conn))
	c := quirk.NewClient()

	data := &quirk.DupleNode{
		Identifier: "Damien",
		Duples: []quirk.Duple{
			{Predicate: "name", Object: "Damien"},
			{Predicate: "ssn", Object: "126", IsUnique: true},
			{Predicate: "policy", Object: 61238, IsUnique: true},
		},
	}

	c.InsertNode(context.Background(), dg, &quirk.Operation{
		SetSingleDupleNode: data,
	})
}

Output:
Example (SingleStruct) 
package main

import (
	"context"

	"github.com/damienfamed75/quirk"
	"github.com/dgraph-io/dgo"
	"github.com/dgraph-io/dgo/protos/api"
	"google.golang.org/grpc"
)

func main() {
	conn, _ := grpc.Dial("127.0.0.1:9080", grpc.WithInsecure())
	defer conn.Close()

	dg := dgo.NewDgraphClient(api.NewDgraphClient(conn))
	c := quirk.NewClient()

	// Single struct must be a reference to a struct in order for
	// reflect to not throw a panic.
	data := &struct {
		Name   string `quirk:"name"`
		SSN    string `quirk:"ssn,unique"`
		Policy int    `quirk:"policy,unique"`
	}{
		Name:   "Damien",
		SSN:    "126",
		Policy: 61238,
	}

	c.InsertNode(context.Background(), dg, &quirk.Operation{
		SetSingleStruct: data,
	})
}

Output:
Example (StringMap) 
package main

import (
	"context"

	"github.com/damienfamed75/quirk"
	"github.com/dgraph-io/dgo"
	"github.com/dgraph-io/dgo/protos/api"
	"google.golang.org/grpc"
)

func main() {
	conn, _ := grpc.Dial("127.0.0.1:9080", grpc.WithInsecure())
	defer conn.Close()

	dg := dgo.NewDgraphClient(api.NewDgraphClient(conn))
	c := quirk.NewClient()

	// Maps do not support unique predicates.
	data := make(map[string]string)
	data["name"] = "Damien"
	data["age"] = "19"

	c.InsertNode(context.Background(), dg, &quirk.Operation{
		SetStringMap: data,
	})
}

Output:

type ClientConfiguration 

type ClientConfiguration func(*Client)

ClientConfiguration is used to pass in options to change the client and customize it to the user's liking.

func WithLogger 

func WithLogger(l yalp.Logger) ClientConfiguration

WithLogger sets the logger used by the quirk client. By default this is quirk.NewNilLogger.

Example 
package main

import (
	"github.com/damienfamed75/quirk"
)

func main() {
	client := quirk.NewClient(
		quirk.WithLogger(quirk.NewDebugLogger()),
	)

	_ = client
}

Output:

func WithMaxWorkerCount 

func WithMaxWorkerCount(count int) ClientConfiguration

WithMaxWorkerCount will set the maximum workers that will be spun when using a Multi operation.

func WithPredicateKey 

func WithPredicateKey(predicateName string) ClientConfiguration

WithPredicateKey sets the field(predicate) that will be used to label inserted nodes. By default this is "name"

Example 
package main

import (
	"fmt"

	"github.com/damienfamed75/quirk"
)

func main() {
	client := quirk.NewClient(
		quirk.WithPredicateKey("label"),
	)

	// Now that the predicate key is set to "label" the returning UID map
	// will use the predicate value of "label" for the key rather than "name"

	// Should be "label"
	fmt.Println(client.GetPredicateKey())

	_ = client
}

Output:

func WithTemplate 

func WithTemplate(tmpl string) ClientConfiguration

WithTemplate sets the field in the Quirk client that uses a progress bar to show the nodes being inserted with multi node sets.

Example 
package main

import (
	"github.com/damienfamed75/quirk"
)

func main() {
	// Using github.com/cheggaaa/pb/v3 for progress bar.
	client := quirk.NewClient(
		quirk.WithTemplate(`{{ "Custom:" }} {{ bar . "[" "-" (cycle . ">" ) " " "]"}} [{{etime . | cyan }}:{{rtime . | cyan }}]`),
	)

	_ = client
}

Output:

type Duple 

type Duple struct {
	// Predicate acts as a key.
	Predicate string
	// Object is the data representing the predicate.
	Object interface{}
	// IsUnique stores whether or not to treat this as an upsert or not.
	IsUnique bool
	// contains filtered or unexported fields
}

Duple is a structural way of giving the quirk client enough information about a node to create triples and insert them into Dgraph.

Example 
package main

import (
	"github.com/damienfamed75/quirk"
)

func main() {
	duple := &quirk.Duple{
		Predicate: "name",
		Object:    "Damien",
		IsUnique:  false,
	}

	_ = duple
}

Output:

type DupleNode 

type DupleNode struct {
	Identifier string
	Duples     []Duple
}

DupleNode is the container for a duple node.

Example 
package main

import (
	"github.com/damienfamed75/quirk"
)

func main() {
	node := &quirk.DupleNode{
		Identifier: "person", // used for the key in the returned UID Map.
		Duples: []quirk.Duple{ // predicate value pairs.
			{Predicate: "name", Object: "Damien"},
			{Predicate: "ssn", Object: "126", IsUnique: true},
			{Predicate: "policy", Object: 61238, IsUnique: true},
		},
	}

	_ = node
}

Output:

func (*DupleNode) AddDuples 

func (d *DupleNode) AddDuples(duple ...Duple) *DupleNode

AddDuples appends new duples given in the function. Then returns the reference to the DupleNode. This function doesn't support updating previously added Duples though. This should only be used when trying to optimize appending new duples.

Example 
package main

import (
	"github.com/damienfamed75/quirk"
)

func main() {
	node := &quirk.DupleNode{}

	// Adds new Duples. This doesn't support updating predicate values.
	node.AddDuples(
		quirk.Duple{Predicate: "age", Object: 20},
		quirk.Duple{Predicate: "username", Object: "damienfamed75"},
	)

	_ = node
}

Output:

func (*DupleNode) Find 

func (d *DupleNode) Find(predicate string) *Duple

Find will return a reference to a duple given that it is found in the slice of duples in the DupleNode.

Example 
package main

import (
	"fmt"

	"github.com/damienfamed75/quirk"
)

func main() {
	node := &quirk.DupleNode{
		Duples: []quirk.Duple{
			{Predicate: "name", Object: "Damien"},
		},
	}

	// Find a Duple stored in the DupleNode.
	name := node.Find("name")

	fmt.Printf("%s: %v unique[%v]\n", name.Predicate, name.Object, name.IsUnique)

	_ = node
}

Output:

func (*DupleNode) SetOrAdd 

func (d *DupleNode) SetOrAdd(duple Duple) *DupleNode

SetOrAdd will set a pre existing duple in the DupleNode or if the Duple doesn't exist, then it will be added to the Node.

Example 
package main

import (
	"github.com/damienfamed75/quirk"
)

func main() {
	node := &quirk.DupleNode{
		Duples: []quirk.Duple{
			{Predicate: "age", Object: 19},
		},
	}

	// Updates the previous valued stored in DupleNode.
	node.SetOrAdd(
		quirk.Duple{Predicate: "age", Object: 20},
	)

	_ = node
}

Output:

func (*DupleNode) Unique 

func (d *DupleNode) Unique() (duples []Duple)

Unique will loop through the Duples and return a new slice containing all duples that are marked as unique.

Example 
package main

import (
	"fmt"

	"github.com/damienfamed75/quirk"
)

func main() {
	node := &quirk.DupleNode{
		Identifier: "person", // used for the key in the returned UID Map.
		Duples: []quirk.Duple{ // predicate value pairs.
			{Predicate: "name", Object: "Damien"},
			{Predicate: "ssn", Object: "126", IsUnique: true},
			{Predicate: "policy", Object: 61238, IsUnique: true},
		},
	}

	// returns a slice of all the predicates labeled as unique.
	uniqueDuples := node.Unique()

	// Should be 2.
	fmt.Printf("num of unique nodes [%v]\n", len(uniqueDuples))

	_ = node
}

Output:

type Error 

type Error struct {
	ExtErr   error
	Msg      string
	File     string
	Function string
}

Error is a general error that isn't super specific. Just used for when there needs to be more context relating to an error.

func (*Error) Error 

func (e *Error) Error() string

type Operation 

type Operation struct {
	SetMultiStruct     []interface{}
	SetSingleStruct    interface{}
	SetStringMap       map[string]string
	SetDynamicMap      map[string]interface{}
	SetSingleDupleNode *DupleNode
	SetMultiDupleNode  []*DupleNode
}

Operation is the main parameter used when calling quirk client methods. Note: only one of these should be filled at a time, because only one will be executed and taken care of as seen in client.go

type QueryError 

type QueryError struct {
	ExtErr   error
	Msg      string
	Function string
	Query    string
}

QueryError is used for functions in the query.go file.

func (*QueryError) Error 

func (e *QueryError) Error() (res string)

type TransactionError 

type TransactionError struct {
	ExtErr   error
	Msg      string
	Function string
	RDF      string
}

TransactionError is for when a transaction fails during a mutation.

func (*TransactionError) Error 

func (e *TransactionError) Error() string

type UID 

type UID struct {
	// contains filtered or unexported fields
}

UID is used to identify the ID's given to the user and retrieved back to be put as the object of a predicate. This way quirk can handle the UID how they're supposed to be handled. Note: Use this struct as the Object for Duples to create relationships.

func (UID) IsNew 

func (u UID) IsNew() bool

IsNew returns a simple boolean value indicating whether or not this node was a newly added node or if it was pre existing in Dgraph.

func (UID) Value 

func (u UID) Value() string

Value returns the raw string value of the UID. Note: Do not use this as the Object for Duples to create relationships.

Source Files

View all Source files

Directories

Path Synopsis
examples
edges
largeload
setdynamicmap
setmulti
setmultiduple
setsingle
setsingleduple
setstringmap
withpredicatekey

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.