wikibrief

package module
v0.0.0-...-887c23e Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 19, 2019 License: MIT Imports: 16 Imported by: 4

README

wikibrief

GoDoc Reference Build Status Go Report Card Bugs Coverage Lines of Code Maintainability Rating Reliability Rating Security Rating Vulnerabilities

Description

Package wikibrief is a golang package that provides utility functions for extracting information from Wikipedia dumps.

Installation

This package can be installed with the go get command:

go get github.com/negapedia/wikibrief

Documentation

API documentation can be found in the associated godoc reference

Documentation

Index

Constants

View Source
const AnonimousUserID uint32 = 0

AnonimousUserID is the UserID value assumed by revisions done by an anonimous user

Variables

This section is empty.

Functions

func FanOut

func FanOut(ctx context.Context, in <-chan EvolvingPage, n int) (out []<-chan EvolvingPage)

FanOut creates n copies of the given EvolvingPage channel; if n < 1 panics.

func New

func New(ctx context.Context, fail func(err error) error, tmpDir, lang string, restrict bool) <-chan EvolvingPage

New digest the latest wikipedia dump of the specified language into the output channel. The revision channel of each page must be exhausted (or the context cancelled), doing otherwise may result in a deadlock. The ctx and fail together should behave in the same manner as if created with WithFail - https://godoc.org/github.com/ebonetti/ctxutils#WithFail The condition restrict restricts the digest to just one dump file, used for testing purposes.

Types

type EvolvingPage

type EvolvingPage struct {
	PageID          uint32
	Title, Abstract string
	TopicID         uint32
	Revisions       <-chan Revision
}

EvolvingPage represents a wikipedia page that is being edited. Revisions is closed when there are no more revisions. Revision channel must be exhausted (or the context cancelled), doing otherwise may result in a deadlock.

type Revision

type Revision struct {
	ID, UserID uint32
	IsBot      bool
	Text, SHA1 string
	IsRevert   uint32
	Timestamp  time.Time
}

Revision represents a revision of a page.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL