staticgen

package module
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 23, 2020 License: MIT Imports: 18 Imported by: 0

README

Staticgen

A static website generator that lets you use HTTP servers and frameworks you already know. Just tell Staticgen how to start your server, then watch it crawl your site and generate a static version with all of the pages and assets required.

About

If you're unfamiliar, you can actually use the decades-old wget command to output a static website from a dynamic one, this project is purpose-built for the same idea, letting your team to use whatever HTTP servers and frameworks you're already familiar with, in any language.

I haven't done any scientific benchmarks or comparisons yet, but here are some results on my 2014 8-core MBP:

Installation

Via gobinaries.com:

$ curl -sf https://gobinaries.com/tj/staticgen/cmd/staticgen | sh

Configuration

Configuration is stored within a ./static.json file in your project's root directory. The following options are available:

  • command — The server command executed before crawling.
  • url — The target website to crawl. Defaults to "http://127.0.0.1:3000".
  • dir — The static website output directory. Defaults to "build".
  • pages — A list of paths added to crawl, typically including unlinked pages such as landing pages. Defaults to [].
  • concurrency — The number of concurrent pages to crawl. Defaults to 30.

Guide

First create the ./static.json configuration file, for example here's the config for Go server, the only required property is command:

{
  "command": "go run main.go",
  "concurrency": 50,
  "dir": "dist"
}

Below is an example of a Node.js server, note that NODE_ENV is assigned to production so that optimizations such as Express template caches are used to improve serving performance.

{
  "command": "NODE_ENV=production node server.js"
}

Run the staticgen command to start the pre-rendering process:

$ staticgen

Staticgen executes the command you provided, waits for the server to become available on the url configured. The pages and assets are copied to the dir configured and then your server is shut down.

By default the timeout for the generation process is 15 minutes, depending on your situation you may want to increase or decrease this with the -t, --timeout flag, here are some examples:

$ staticgen -t 30s
$ staticgen -t 15m
$ staticgen -t 1h

When launching the command, Staticgen sets the STATICGEN environment variable to 1, allowing you to alter behaviour if necessary.

To view the pre-rendered site run the following command to start a static file server and open the browser:

$ staticgen serve

See the examples directory for full examples.

Notes

Staticgen does not pre-render using a headless browser, this makes it faster, however it means that you cannot rely on client-side JavaScript manipulating the page.

GoDoc

Sponsors

This project is sponsored by CTO.ai, making it easy for development teams to create and share workflow automations without leaving the command line.

And my GitHub sponsors:

Documentation

Overview

Package staticgen provides static website generation from a live server.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Config 

type Config struct {
	// URL is the target website to crawl. Defaults to "http://127.0.0.1:3000".
	URL string `json:"url"`

	// Dir is the static website output directory. Defaults to "build".
	Dir string `json:"dir"`

	// Command is the optional server command executed before crawling.
	Command string `json:"command"`

	// Pages is a list of paths added to crawl, typically
	// including unlinked pages such as error pages,
	// landing pages and so on.
	Pages []string `json:"pages"`

	// Concurrency is the number of concurrent pages to crawl. Defaults to 30.
	Concurrency int `json:"concurrency"`

	// Allow404 can be enabled to opt-in to pages resulting in a 404,
	// which otherwise lead to an error.
	Allow404 bool `json:"allow_404"`
}

Config is the static website generator configuration.

func (*Config) Load 

func (c *Config) Load(path string) error

Load configuration from the given path.

type Event 

type Event interface {
	// contains filtered or unexported methods
}

Event is an event.

type EventStartCrawl 

type EventStartCrawl struct{}

EventStartCrawl .

type EventStartedServer 

type EventStartedServer struct {
	Command string
	URL     string
}

EventStartedServer .

type EventStartingServer 

type EventStartingServer struct {
	Command string
	URL     string
}

EventStartingServer .

type EventStopCrawl 

type EventStopCrawl struct{}

EventStopCrawl .

type EventStoppingServer 

type EventStoppingServer struct{}

EventStoppingServer .

type EventVisitedResource 

type EventVisitedResource struct {
	Target
	Duration   time.Duration
	StatusCode int
	Filename   string
	Error      error
}

EventVisitedResource .

type Generator 

type Generator struct {
	// Config used for crawling and producing the static website.
	Config

	// HTTPClient ...
	HTTPClient *http.Client
	// contains filtered or unexported fields
}

Generator is a static website generator.

func (*Generator) Report 

func (g *Generator) Report(ch chan<- Event)

Report registers a channel for reporting on events.

func (*Generator) Run 

func (g *Generator) Run(ctx context.Context) error

Run starts the configured server command, starts to perform crawling, and waits for completion before shutting down the configured server.

func (*Generator) Start 

func (g *Generator) Start(ctx context.Context) error

Start loads configuration from ./static.json, starts the configured server, and begins the crawling process.

func (*Generator) Wait 

func (g *Generator) Wait() error

Wait for crawling to complete.

type Reporter 

type Reporter struct {
	// contains filtered or unexported fields
}

A Reporter outputs a human-friendly report of events.

func (*Reporter) Report 

func (r *Reporter) Report(ch <-chan Event) <-chan struct{}

Report on the given event channel. Returns a channel which is closed when the event channel is closed, and all reporting has been completed.

type Target 

type Target struct {
	Parent *url.URL
	URL    *url.URL
}

Target is a target URL.

Source Files

View all Source files

Directories

Path Synopsis
_examples
go
cmd
staticgen
internal
crawler
Package crawler provides a website crawler.
 Package crawler provides a website crawler.
deduplicator
Package deduplicator provides URL deduplication.
 Package deduplicator provides URL deduplication.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.