containers

package

v0.0.0-...-f8fee01 Latest Latest Go to latest Published: Mar 16, 2024 License: GPL-3.0 Imports: 11 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/algo7/TripAdvisor-Review-Scraper

Documentation ¶

Index ¶

func ContainerConfigGenerator(locationURL string, locationName string, uploadIdentifier string, ...) *container.Config
func CreateContainer(containerConfig *container.Config) string
func PullImage(image string)
func ReleaseProxyContainer(containerID string)
func RemoveContainer(containerID string)
func Scrape(uploadIdentifier string, targetName string, containerID string)
func TailLog(containerID string) io.Reader
type Container
- func ListContainersByType(containerType string) []Container
type ProxyContainer
- func AcquireProxyContainer() ProxyContainer

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

func ContainerConfigGenerator ¶

func ContainerConfigGenerator(
	locationURL string, locationName string, uploadIdentifier string,
	proxyAddress string, vpnRegion string) *container.Config

ContainerConfigGenerator generates the container config depending on the scrape target

func CreateContainer ¶

func CreateContainer(containerConfig *container.Config) string

CreateContainer creates a container then returns the container ID

func PullImage ¶

func PullImage(image string)

PullImage pulls the given image from a registry

func ReleaseProxyContainer ¶

func ReleaseProxyContainer(containerID string)

ReleaseProxyContainer releases the lock on a proxy container

func RemoveContainer ¶

func RemoveContainer(containerID string)

RemoveContainer removes the container with the given ID

func Scrape ¶

func Scrape(uploadIdentifier string, targetName string, containerID string)

Scrape creates a container, runs it, tails the log and wait for it to exit, and export the file name

func TailLog ¶

func TailLog(containerID string) io.Reader

TailLog tails the log of the container with the given ID

Types ¶

type Container ¶

type Container struct {
	ContainerID    *string
	TaskOwner      *string
	TargetName     *string
	URL            *string
	IPAddress      *string
	VPNRegion      *string
	ProxySOCKSPort *string
	ProxyHTTPPort  *string
}

Container information

func ListContainersByType ¶

func ListContainersByType(containerType string) []Container

ListContainersByType lists all containers of the given type. Available container types:

"scraper": Lists all scraper containers.
"proxy": Lists all proxy containers.

Example:

scraperContainers := ListContainersByType("scraper")
proxyContainers := ListContainersByType("proxy")

type ProxyContainer ¶

type ProxyContainer struct {
	ContainerID  string
	ProxyAddress string
	VPNRegion    string
}

ProxyContainer information

func AcquireProxyContainer ¶

func AcquireProxyContainer() ProxyContainer

AcquireProxyContainer acquires a lock on a proxy container and returns its ID

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL