web

Overview ¶

Package web implements a basic web site serving framework. The two fundamental types in this package are Site and Page.

Sites ¶

A Site is an http.Handler that serves requests from a file system. Use NewSite(fsys) to create a new Site.

The Site is defined primarily by the content of its file system fsys, which holds files to be served as well as templates for converting Markdown or HTML fragments into full HTML pages.

Pages ¶

A Page, which is a map[string]interface{}, is the raw data that a Site renders into a web page. Typically a Page is loaded from a *.html or *.md file in the file system fsys, although dynamic pages can be computed and passed to ServePage as well, as described in “Serving Dynamic Pages” below.

For a Page loaded from the file system, the key-value pairs in the map are initialized from the YAML or JSON metadata block at the top of a Markdown or HTML file, which looks like (YAML):

---
key: value
...
---

or (JSON):

<!--{
	"Key": "value",
	...
}-->

By convention, key-value pairs loaded from a metadata block use lower-case keys. For historical reasons, keys in JSON metadata are converted to lower-case when read, so that the two headers above both refer to a key with a lower-case k.

A few keys have special meanings:

The key-value pair “status: n” sets the HTTP response status to the integer code n.

The key-value pair “redirect: url” causes requests for this page redirect to the given relative or absolute URL.

The key-value pair “layout: name” selects the page layout template with the given name. See the next section, “Page Rendering”, for details about layout and rendering.

In addition to these explicit key-value pairs, pages loaded from the file system have a few implicit key-value pairs added by the page loading process:

• File: the path in fsys to the file containing the page
• FileData: the file body, with the key-value metadata stripped
• URL: this page's URL path (/x/y/z for x/y/z.md, /x/y/ for x/y/index.md)

The key “Content” is added during during the rendering process. See “Page Rendering” for details.

Page Rendering ¶

A Page's content is rendered in two steps: conversion to content, and framing of content.

To convert a page to content, the page's file body (its FileData key, a []byte) is parsed and executed as an HTML template, with the page itself passed as the template input data. The template output is then interpreted as Markdown (perhaps with embedded HTML), and converted to HTML. The result is stored in the page under the key “Content”, with type template.HTML.

A page's conversion to content can be skipped entirely in dynamically-generated pages by setting the “Content” key before passing the page to ServePage.

The second step is framing the content in the overall site HTML, which is done by executing the site template, again using the Page itself as the template input data.

The site template is constructed from two files in the file system. The first file is the fsys's “site.tmpl”, which provides the overall HTML frame for the site. The second file is a layout-specific template file, selected by the Page's “layout: name” key-value pair. The renderer searches for “name.tmpl” in the directory containing the page's file, then in the parent of that directory, and so on up to the root. If no such template is found, the rendering fails and reports that error. As a special case, “layout: none” skips the second file entirely.

If there is no “layout: name” key-value pair, then the renderer tries using an implicit “layout: default”, but if no such “default.tmpl” template file can be found, the renderer uses an implicit “layout: none” instead.

By convention, the site template and the layout-specific template are connected as follows. The site template, at the point where the content should be rendered, executes:

{{block "layout" .}}{{.Content}}{{end}}

The layout-specific template overrides this block by defining its own template named “layout”. For example:

{{define "layout"}}
Here's some <blink>great</blink> content: {{.Content}}
{{end}}

The use of the “block” template construct ensures that if there is no layout-specific template, the content will still be rendered.

Page Template Functions ¶

In this web server, templates can themselves be invoked as functions. See https://pkg.go.dev/rsc.io/tmplfunc for more details about that feature.

During page rendering, both when rendering a page to content and when framing the content, the following template functions are available (in addition to those provided by the template package itself and the per-template functions just mentioned).

In all functions taking a file path f, if the path begins with a slash, it is interpreted relative to the fsys root. Otherwise, it is interpreted relative to the directory of the current page's URL.

The “{{add x y}}”, “{{sub x y}}”, “{{mul x y}}”, and “{{div x y}}” functions provide basic math on arguments of type int.

The “{{code f [start [end]]}}” function returns a template.HTML of a formatted display of code lines from the file f. If both start and end are omitted, then the display shows the entire file. If only the start line is specified, then the display shows that single line. If both start and end are specified, then the display shows a range of lines starting at start up to and including end. The arguments start and end can take two forms: a number indicates a specific line number, and a string is taken to be a regular expresion indicating the earliest matching line in the file (or, for end, the earliest matching line after the start line). Any lines ending in “OMIT” are elided from the display.

For example:

{{code "hello.go" `^func main` `^}`}}

The “{{data f}}” function reads the file f, decodes it as YAML, and then returns the resulting data, typically a map[string]interface{}. It is effectively shorthand for “{{yaml (file f)}}”.

The “{{file f}}” function reads the file f and returns its content as a string.

The “{{first n slice}}” function returns a slice of the first n elements of slice, or else slice itself when slice has fewer than n elements.

The “{{markdown text}}” function interprets text (a string) as Markdown and returns the equivalent HTML as a template.HTML.

The “{{page f}}” function returns the page data (a Page) for the static page contained in the file f. The lookup ignores trailing slashes in f as well as the presence or absence of extensions like .md, .html, /index.md, and /index.html, making it possible for f to be a relative or absolute URL path instead of a file path.

The “{{pages glob}}” function returns a slice of page data (a []Page) for all pages loaded from files or directories in fsys matching the given glob (a string), according to the usual file path rules (if the glob starts with slash, it is interpreted relative to the fsys root, and otherwise relative to the directory of the page's URL). If the glob pattern matches a directory, the page for the directory's index.md or index.html is used.

For example:

Here are all the articles:
{{range (pages "/articles/*")}}
- [{{.title}}]({{.URL}})
{{end}}

The “{{raw s}}” function converts s (a string) to type template.HTML without any escaping, to allow using s as raw Markdown or HTML in the final output.

The “{{yaml s}}” function decodes s (a string) as YAML and returns the resulting data. It is most useful for defining templates that accept YAML-structured data as a literal argument. For example:

{{define "quote info"}}
{{with (yaml .info)}}
.text
— .name{{if .title}}, .title{{end}}
{{end}}

{{quote `
  text: If a program is too slow, it must have a loop.
  name: Ken Thompson
`}}

The “path” and “strings” functions return package objects with methods for every top-level function in these packages (except path.Split, which has more than one non-error result and would not be invokable). For example, “{{strings.ToUpper "abc"}}”.

Serving Requests ¶

A Site is an http.Handler that serves requests by consulting the underlying file system and constructing and rendering pages, as well as serving binary and text files.

To serve a request for URL path /p, if fsys has a file p/index.md, p/index.html, p.md, or p.html (in that order of preference), then the Site opens that file, parses it into a Page, renders the page as described in the “Page Rendering” section above, and responds to the request with the generated HTML. If the request URL does not match the parsed page's URL, then the Site responds with a redirect to the canonical URL.

Otherwise, if fsys has a directory p and the Site can find a template “dir.tmpl” in that directory or a parent, then the Site responds with the rendering of

Page{
	"URL": "/p/",
	"File": "p",
	"layout": "dir",
	"dir": []fs.FileInfo(dir),
}

where dir is the directory contents.

Otherwise, if fsys has a file p containing valid UTF-8 text (at least up to the first kilobyte of the file) and the Site can find a template “text.tmpl” in that file's directory or a parent, and the file is not named robots.txt, and the file does not have a .css, .js, .svg, or .ts extension, then the Site responds with the rendering of

Page{
	"URL": "/p",
	"File": "p",
	"layout": "texthtml",
	"texthtml": template.HTML(texthtml),
}

where texthtml is the text file as rendered by the golang.org/x/website/internal/texthtml package. In the texthtml.Config, GoComments is set to true for file names ending in .go; the h URL query parameter, if present, is passed as Highlight, and the s URL query parameter, if set to lo:hi, is passed as a single-range Selection.

If the request has the URL query parameter m=text, then the text file content is not rendered or framed and is instead served directly as a plain text response.

If the request is for a file with a .ts extension the file contents are transformed from TypeScript to JavaScript and then served with a Content-Type=text/javascript header.

Otherwise, if none of those cases apply but the request path p does exist in the file system, then the Site passes the request to an http.FileServer serving from fsys. This last case handles binary static content as well as textual static content excluded from the text file case above.

Otherwise, the Site responds with the rendering of

Page{
	"URL": r.URL.Path,
	"status": 404,
	"layout": "error",
	"error": err,
}

where err is the “not exist” error returned by fs.Stat(fsys, p). (See also the “Serving Errors” section below.)

Serving Dynamic Requests ¶

Of course, a web site may wish to serve more than static content. To allow dynamically generated web pages to make use of page rendering and site templates, the Site.ServePage method can be called with a dynamically generated Page value, which will then be rendered and served as the result of the request.

Serving Errors ¶

If an error occurs while serving a request r, the Site responds with the rendering of

Page{
	"URL": r.URL.Path,
	"status": 500,
	"layout": "error",
	"error": err,
}

If that rendering itself fails, the Site responds with status 500 and the cryptic page text “error rendering error”.

The Site.ServeError and Site.ServeErrorStatus methods provide a way for dynamic servers to generate similar responses.