paginate

package module
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 2, 2021 License: MIT Imports: 13 Imported by: 0

README

paginate

Build Status MIT license GoDoc

Overview

Package paginate provides basic pagination capabilities to paginate sql database tables with Go.

How to use?

package main

import (
	"database/sql"
	"encoding/json"
	"fmt"
	_ "github.com/lib/pq"
	"github.com/ottotech/paginate"
	"log"
	"net/url"
	"time"
)

type Employee struct {
	ID           int                  `paginate:"id;col=id;param=employee_id"`
	Name         string               `paginate:"filter;col=name"`
	LastName     string               `paginate:"filter;col=last_name"`
	WorkerNumber paginate.NullInt     `paginate:"col=worker_number"`
	DateJoined   time.Time            `paginate:"col=date_joined"`
	Salary       float64              `paginate:"filter;col=salary"`
	NullText     paginate.NullString  `paginate:"col=null_text"`
	NullVarchar  paginate.NullString  `paginate:"col=null_varchar"`
	NullBool     paginate.NullBool    `paginate:"col=null_bool"`
	NullDate     paginate.NullTime    `paginate:"col=null_date"`
	NullInt      paginate.NullInt     `paginate:"col=null_int"`
	NullFloat    paginate.NullFloat64 `paginate:"col=null_float"`
}

func main() {
    dbUri := "user=root password=secret host=localhost " +
        "port=5432 dbname=events_db sslmode=disable"
    
    db, err := sql.Open("postgres", dbUri)
    if err != nil {
        panic(err)
    }
    
    if err = db.Ping(); err != nil {
        panic(err)
    }
    fmt.Println("You connected to your database.")
    
    u, err := url.Parse("http://localhost?name=Ringo")
    if err != nil {
        // Handle error gracefully...
    }
    
    paginator, err := paginate.NewPaginator(Employee{}, "postgres", *u, paginate.TableName("employees"))
    if err != nil {
        // Handle error gracefully...
    }
    
    cmd, args, err := paginator.Paginate()
    if err != nil {
        // Handle error gracefully...
    }
    
    rows, err := db.Query(cmd, args...)
    if err != nil {
        // Handle error gracefully...
    }
    defer rows.Close()
    
    for rows.Next() {
        err = rows.Scan(paginator.GetRowPtrArgs()...)
        if err != nil {
            // Handle error gracefully...
        }
    }
    
    if err = rows.Err(); err != nil {
        // Handle error gracefully...
    }
    
    results := make([]Employee, 0)
    
    for paginator.NextData() {
        employee := Employee{}
        err = paginator.Scan(&employee)
        if err != nil {
            // Handle error gracefully...
        }
        results = append(results, employee)
    }
    
    // You should be able to see the paginated data inside results. 
    fmt.Println(results)
    
    // You should be able to serialize into json the paginated data.
    sb, err := json.Marshal(results)
    if err != nil {
        log.Fatal(err)
    }
    
    fmt.Println(sb)
}

Contributing

Notes

Paginator does not take into consideration performance since it uses the OFFSET sql argument which reads and counts all rows from the beginning until it reaches the requested page. For not too big datasets Paginator will just work fine. If you care about performance because you are dealing with heavy data you might want to write a custom solution for that.

License

Released under MIT license, see LICENSE for details.

Documentation

Overview

Package paginate provides a basic Paginator object to paginate records of a single database table. Its primary job is to generate a raw sql command with the corresponding arguments that can be executed against a sequel database with the sql driver of your preference.

Paginator also provides some utility functions like GetRowPtrArgs, NextData, and Scan, to make easy to retrieve and read the paginated data.

Paginator also handles basic filtering of records with the parameters coming from a request url.

And Paginator also returns a PaginationResponse which contains useful information for clients to do proper pagination.

For filtering database records the following operators are available. Use these with the parameters in the request url:

eq  = "="
gt  = ">"
lt  = "<"
gte = ">="
lte = "<="
ne  = "<>"

For ordering records based on column names use the following syntax in the url with the “sort“ parameter. For sorting in ascending order use the plus (+) sign, and for sorting in descending order use the minus (-) sign:

http://localhost/employees?name=rob&sort=+name,-age

When parameters with the equal sign (=) in the request url are repeated, Paginator will interpret this as an IN sql clause. So for example given a database table “Employees“ and a request url like:

http://localhost/employees?name=julia&name=mark

Paginator will produce an sql query similar to this:

SELECT id, name FROM employees WHERE name IN($1,$2) ORDER BY id LIMIT 30 OFFSET 0

Similarly, when parameters with the notequal sign (<>) in the request url are repeated, Paginator will interpret this as a NOT IN sql clause. So for example, given a database table “Employees“ and a request url like:

http://localhost/employees?name<>julia&name<>mark

Paginator will produce an sql query similar to this:

SELECT id, name FROM employees WHERE name NOT IN($1,$2) ORDER BY id LIMIT 30 OFFSET 0

Example of the table struct field tags and their meanings (use a ; to specify multiple tags at the same time):

// Paginator will infer the database column name from the tag "col",
// so in this case the column name would be "person_name".
// If no "col" tag is given Paginator will infer the database
// column name from the name of the struct field in snake lowercase
// form. So given a struct field "MyName", Paginator will infer the
// the database column name as "my_name".
Name string `paginate:"col=person_name"`

// By default all the columns of the given table cannot be filtered
// with the parameters coming from a request url. Nevertheless, a user can
// explicitly tell Paginator to filter a column by specifying the "filter"
// tag in the table struct fields.
LastName string `paginate:"filter"`

// By default, once a database column has been defined as filterable
// with the tag "filter", Paginator will map the request parameters with
// the column names of the given table and it will filter the table
// accordingly. However, in some cases you might want to have different
// names for your request parameters in order to be more expressive.
// In those cases use the tag "param" to define a custom request parameter
// name that can be mapped to a column of the database table.
// So, for example, in this case a request parameter "person_id" will be
// used to filter the records of the given table based on the values of
// the column "id".
ID int `paginate:"col=id;param=person_id"`

// The tag "id" is required. If it is not given, Paginator cannot be instantiated
// and it will return an error. The tag "id" allows Paginator to keep the same order
// between pages and results. In simple words, it will make the pagination deterministic.
// Usually, the column used with this tag should be the primary key or unique identifier
// of the given table.
ID int `paginate:"id"`

NOTES:

Paginator does not take into consideration performance since it uses the OFFSET sql argument which reads and counts all rows from the beginning until it reaches the requested page. For not too big datasets Paginator will just work fine. If you care about performance because you are dealing with heavy data you might want to write a custom solution for that.

Index

Constants

This section is empty.

Variables

View Source
var ErrPaginatorIsClosed = errors.New("paginate: Paginator is closed")

ErrPaginatorIsClosed is an error returned by Scan when trying to Scan on a Paginator that is closed. That is, a paginator whose values were already scanned.

Functions

This section is empty.

Types

type InnerJoin added in v1.1.0

type InnerJoin struct {
	// contains filtered or unexported fields
}

func NewInnerJoinClause added in v1.1.0

func NewInnerJoinClause(dialect string) (InnerJoin, error)

func (*InnerJoin) On added in v1.1.0

func (clause *InnerJoin) On(column, targetTable, targetColumn string)

type NullBool

type NullBool struct {
	Bool  bool
	Valid bool // Valid is true if Bool is not NULL
}

func (NullBool) MarshalJSON

func (nb NullBool) MarshalJSON() ([]byte, error)

func (*NullBool) Scan

func (nb *NullBool) Scan(value interface{}) error

func (*NullBool) UnmarshalJSON

func (nb *NullBool) UnmarshalJSON(data []byte) error

func (NullBool) Value

func (nb NullBool) Value() (driver.Value, error)

type NullFloat64

type NullFloat64 struct {
	Float64 float64
	Valid   bool // Valid is true if Float64 is not NULL
}

func (NullFloat64) MarshalJSON

func (n NullFloat64) MarshalJSON() ([]byte, error)

func (*NullFloat64) Scan

func (n *NullFloat64) Scan(value interface{}) error

func (*NullFloat64) UnmarshalJSON

func (n *NullFloat64) UnmarshalJSON(data []byte) error

func (NullFloat64) Value

func (n NullFloat64) Value() (driver.Value, error)

type NullInt

type NullInt struct {
	Int   int
	Valid bool // Valid is true if Int is not NULL
}

func (NullInt) MarshalJSON

func (ni NullInt) MarshalJSON() ([]byte, error)

func (*NullInt) Scan

func (ni *NullInt) Scan(value interface{}) error

func (*NullInt) UnmarshalJSON

func (ni *NullInt) UnmarshalJSON(data []byte) error

func (NullInt) Value

func (ni NullInt) Value() (driver.Value, error)

type NullString

type NullString struct {
	String string
	Valid  bool // Valid is true if String is not NULL
}

func (NullString) MarshalJSON

func (ns NullString) MarshalJSON() ([]byte, error)

func (*NullString) Scan

func (ns *NullString) Scan(value interface{}) error

func (*NullString) UnmarshalJSON

func (ns *NullString) UnmarshalJSON(data []byte) error

func (NullString) Value

func (ns NullString) Value() (driver.Value, error)

type NullTime

type NullTime struct {
	Time  time.Time
	Valid bool // Valid is true if Time is not NULL
}

func (NullTime) MarshalJSON

func (nt NullTime) MarshalJSON() ([]byte, error)

func (*NullTime) Scan

func (nt *NullTime) Scan(value interface{}) error

func (*NullTime) UnmarshalJSON

func (nt *NullTime) UnmarshalJSON(data []byte) error

func (NullTime) Value

func (nt NullTime) Value() (driver.Value, error)

type Option

type Option func(p *paginator) error

func OrderByAsc added in v1.0.0

func OrderByAsc(column string) Option

OrderByAsc is an option for NewPaginator that allows you to add a custom specific sql ascending ORDER BY clause. This is useful when, for example, you want to have control on the sorting from the backend. Trying to sort by the "id" defined in the given table (through the tag "id") will not work, since Paginator will always sort the results in a deterministic way, so it will not consider the given "id" for sorting.

func OrderByDesc added in v1.0.0

func OrderByDesc(column string) Option

OrderByDesc is an option for NewPaginator that allows you to add a custom specific sql descending ORDER BY clause. This is useful when, for example, you want to have control on the sorting from the backend. Trying to sort by the "id" defined in the given table (through the tag "id") will not work, since Paginator will always sort the results in a deterministic way, so it will not consider te given "id" for sorting.

func PageSize

func PageSize(size uint) Option

PageSize is an option for NewPaginator which indicates the size of the record set that we want our paginator object to produce per page. “size“ should be an uint value greater than zero. Use this option if you want finer control on the pagination size. Using this option will override the “page_size“ parameter coming from the request in the url.URL.

func TableName

func TableName(name string) Option

TableName is an option for NewPaginator which indicates the name of the table that paginator will paginate in the database. Use this option if the name of your database table cannot be inferred by the given struct table name.

type PaginationResponse

type PaginationResponse struct {
	PageNumber      int  `json:"page_number"`
	NextPageNumber  int  `json:"next_page_number"`
	HasNextPage     bool `json:"has_next_page"`
	HasPreviousPage bool `json:"has_previous_page"`
	PageCount       int  `json:"page_count"`
	TotalSize       int  `json:"total_size"`
}

PaginationResponse contains information about the pagination.

Clients of this package can use PaginationResponse to paginate further their data.

type Paginator

type Paginator interface {
	// Paginate will return an sql command with the corresponding arguments,
	// so it can be run against any sql driver.
	Paginate() (sql string, args []interface{}, err error)

	// GetRowPtrArgs will prepare the next pointer arguments that can be scanned
	// by sql.Rows.Scan.
	//
	// Always run GetRowPtrArgs when scanning the queried rows with the sql package,
	// for example:
	//
	//   rows, _ := db.Query(myQuery)
	//	 for rows.Next() {
	//		err = rows.Scan(paginator.GetRowPtrArgs()...)
	//		if err != nil {
	//			log.Fatal(err)
	//		}
	//	 }
	//
	// Every time GetRowPtrArgs gets called it will save the previous scanned values
	// internally in the Paginator object so you can read/scan them later.
	GetRowPtrArgs() []interface{}

	// NextData will loop over the saved values created by GetRowPtrArgs until
	// all the paginated data has been scanned by Scan. Always use NextData
	// followed by a call to Scan.
	NextData() bool

	// Scan will copy the next paginated data in the given destination. The given destination
	// should be a pointer instance of the same type of the given “table“.
	//
	// Scan converts columns read from the database into the following
	// common Go types:
	//
	//    *string
	//	  *[]byte
	//    *int, *int8, *int16, *int32, *int64
	//    *uint, *uint8, *uint16, *uint32, *uint64
	//    *bool
	//    *float32, *float64
	//
	// Scan will also convert nullable fields of type string, int32, int64, float64,
	// bool, time.Time to their default zero values with the following helpers provided
	// by the sql package (As the user of this package you do not have to care about this,
	// paginate will automatically handle this for you):
	//
	// 	  - sql.NullString
	// 	  - sql.NullInt32
	// 	  - sql.NullInt64
	// 	  - sql.NullFloat64
	// 	  - sql.NullBool
	// 	  - sql.NullTime
	//
	// For other nullable fields that you might want Scan to handle, use
	// the nullable types provided by this package.
	Scan(dest interface{}) error

	// Response returns a PaginationResponse containing useful information about
	// the pagination, so that clients can do proper and subsequent pagination
	// operations.
	Response() PaginationResponse

	// AddWhereClause adds a custom raw where clause that paginator can use to
	// filter out the rows of the target table in the database. Usually, you will
	// use this when the backend needs to filter the records based on internal logic,
	// like, for example: permissions, ownership, etc. Add where clauses before
	// calling Paginator.Paginate.
	AddWhereClause(clause RawWhereClause) error

	// AddJoinClause adds a custom join clause that paginator can use to join
	// multiple tables and columns for pagination.
	AddJoinClause(clause interface{}) error
}

Paginator wraps pagination behaviors.

Paginator should be used following the next steps in the same order:

  1. Initialize a Paginator instance with NewPaginator.
  2. Call Paginate to create the query and arguments to be executed with an sql driver.
  3. Call GetRowPtrArgs when scanning the rows inside the sql.Rows.Next loop.
  4. Call NextData to loop over the paginated data and Scan the data afterwards.
  5. Call Scan inside the NextData loop to copy the paginated data to the given destination.
  6. Call Response to get useful information about the pagination operation.

For more information, see the examples folder to check how to use Paginator.

func NewPaginator

func NewPaginator(table interface{}, dialect string, u url.URL, opts ...Option) (Paginator, error)

NewPaginator creates a Paginator object ready to paginate data from a database table.

The table parameter should be a struct object with fields representing the target database table you want to paginate. The dialect parameter should be a string representing the sql dialect you are using "postgres" or "mysql", for example. For available options you can pass to Paginator check: TableName and PageSize.

When the PageSize option is not given paginator will try to get the page size from the request parameter “page_size“. If there is no “page_size“ parameter NewPaginator will set the Paginator with the default page size which is 30. When the TableName option is not given, NewPaginator will infer the database table name from the table argument given, so it will extract the name from the struct variable.

type RawWhereClause

type RawWhereClause struct {
	// contains filtered or unexported fields
}

func NewRawWhereClause added in v1.0.0

func NewRawWhereClause(dialect string) (RawWhereClause, error)

NewRawWhereClause will give you a validated instance of a RawWhereClause object.

Use this constructor whenever you want to create a custom sql "where" clause that Paginator can use to paginate your table accordingly.

Example of how to use the returned RawWhereClause object:

...
paginator, _ := NewPaginator(MyTable{}, "mysql", *url)
rawWhereSql, err := NewRawWhereClause("mysql")
if err != nil {
   // Handle error gracefully.
}

rawWhereSql.AddPredicate("name LIKE ? OR last_name LIKE ?")
rawWhereSql.AddArg("%ringo%")
rawWhereSql.AddArg("%smith%")

err = paginator.AddWhereClause(rawWhereSql)
if err != nil {
   // Handle error gracefully.
}

func (*RawWhereClause) AddArg

func (raw *RawWhereClause) AddArg(v interface{})

AddArg adds the given argument to the RawWhereClause instance. If your custom raw sql where clause requires more than one argument you should call AddArg multiple times.

func (*RawWhereClause) AddPredicate

func (raw *RawWhereClause) AddPredicate(predicate string)

AddPredicate adds the given predicate to a RawWhereClause instance. If your sql "where" clause requires multiple arguments use the question mark symbol "?" as placeholders, later use AddArg to add the arguments you need.

func (RawWhereClause) String

func (raw RawWhereClause) String() string

String returns the RawWhereClause predicate without arguments as string.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL