dyd

command module
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 2, 2023 License: BSD-3-Clause Imports: 24 Imported by: 0

README

dyd logo

Compile HTML with embedded Go to a single binary

Command dyd compiles HTML with embedded Go scripts to a self-contained, go install-able, single binary server producing dynamic web pages.

Hello world

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>dyd Hello world</title>
  </head>
  <body>
    Hello world!
    <p><? write("req #%d for %s at %s", ctx.ReqSequence, ctx.Request.URL, time.Now().Format(time.RFC1123Z)) ?>
  </body>
</html>

The above file is all you need to produce a web server for a hello world example.

Prerequisities

  • A working Go toolchain must be installed to use dyd. You can install Go from here.

  • The goimports tool is needed. To install goimports issue

$ go install golang.org/x/tools/cmd/goimports@latest

Installation

$ go install modernc.org/dyd@latest

Command line arguments

$ dyd [-v] [-addr=<arg>] [-serve] [-modinit=<arg>] [-clean] [-htmlfmt] path

'path' points to the web server root, it defaults to ".".

-v adds more verbose output.

-addr is the web server address. It's the same argument as the first http.ListeAndServe wants. It defaults to ":6226".

-modinit creates a go.mod file with module path at 'path'. Ignored if a mod file at 'path' already exists.

-clean removes all generated files in the tree starting at 'path' and exits.

-serve compiles and runs the server.

-htmlfmt attempts to format html files. Use with caution, formatting may in some cases lose information. A backup is made before overwriting the HTML file. For example

<link rel="stylesheet" href=<? write("%q", ctx.Request.URL.Query().Get("style")) ?>>

The preprocessor can handle this just fine, the formatter cannot. Write instead

<? write(`<link rel="stylesheet" href=%q>`, ctx.Request.URL.Query().Get("style")) ?>

Usage example

Using the hello world example from above in index.html.

0:~/tmp/dyd$ ls -la
total 12
drwxr-xr-x 2 jnml jnml 4096 May 22 13:02 .
drwxr-xr-x 7 jnml jnml 4096 May 22 12:42 ..
-rw-r--r-- 1 jnml jnml  269 May 22 12:57 index.html
0:~/tmp/dyd$

Generate and run the server (-v is optional).

0:~/tmp/dyd$ dyd -v -modinit example.com/hello -serve
executing [go mod init example.com/hello]
preprocessing
executing [/home/jnml/bin/goimports -l -w /home/jnml/tmp/dyd/index.html.go]
building site map
url /index.html produced by example.com/hello.Serve_index_1html()
generating dyd.go
executing [/home/jnml/bin/goimports -l -w dyd.go]
generating /home/jnml/tmp/dyd/main.go
executing [/home/jnml/bin/goimports -l -w /home/jnml/tmp/dyd/main.go]
executing [go mod tidy]
starting server at :6226
executing [go run .]

Stop the server.

^C
130:~/tmp/dyd$

How it works

Every directory is - or becomes - a Go package. dyd looks for .html files and generates Go functions serving the content. Any Go code can appear within HTML processing instructions <? and ?>. This code is inserted into the serving function. Go code can refer to two arguments of the serving function. The 'ctx' argument provides context. The type is documented at http://modernc.org/dyd/dyd#Context. The other argument is 'write' and its a function declared as

write func(s string, args ...any) error

The write function is only a small helper writing to the ctx.ResponseWriter.

write("%d %s", 123, s)

is the same as

fmt.Fprintf(ctx.ResponseWriter, "%d %s", 123, s)

The web root directory must be package main, directories under the root cannot be package main.

If there's no main.go/func main() in the web root, dyd will create one. If a manually written main.go/func main() exists, it will be kept. To invoke the web server in your manual func main(), add this as the first line of the function body

defer dydStart()

or

defer func() { log.Fatal(dydStart()) }()

etc. The 'dydStart' will be defined automatically in dyd.go. Consider this file name reserved by dyd.

Let's see what was generated for the Hello world example.

130:~/tmp/dyd$ ls -la
total 32
drwxr-xr-x 2 jnml jnml 4096 May 22 13:03 .
drwxr-xr-x 7 jnml jnml 4096 May 22 12:42 ..
-rw-r----- 1 jnml jnml  328 May 22 13:03 dyd.go
-rw-r--r-- 1 jnml jnml   66 May 22 13:03 go.mod
-rw-r--r-- 1 jnml jnml  149 May 22 13:03 go.sum
-rw-r--r-- 1 jnml jnml  269 May 22 12:57 index.html
-rw-r----- 1 jnml jnml  516 May 22 13:03 index.html.go
-rw-r----- 1 jnml jnml  107 May 22 13:03 main.go
0:~/tmp/dyd$

The dyd.go file contains the dydStart function. It registers all the serving functions to handle the respective URLs.

// Code generated by dyd, DO NOT EDIT.

package main

import (
	"modernc.org/dyd/dyd"
)

// dydStart starts the webapp. dydStart always returns a non-nil error.
func dydStart() error {
	app, err := dyd.NewApp()
	if err != nil {
		return err
	}

	app.Bind("/index.html", Serve_index_1html)
	return app.ListenAndServe(":6226")
}

Preprocessed index.html is in index.html.go.

// Code generated by dyd, DO NOT EDIT.

package main

import (
	"time"

	"modernc.org/dyd/dyd"
)

// Serve_index_1html produces /index.html.
func Serve_index_1html(ctx *dyd.Context, write func(s string, args ...any) error) {
	write(`<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>dyd Hello world</title>
  </head>
  <body>
    Hello world!
    <p>`)
	write("req #%d for %s at %s", ctx.ReqSequence, ctx.Request.URL, time.Now().Format(time.RFC1123Z))
	write(`
  </body>
</html>
`)
}

And finally, main.go.

// Code generated by dyd, DO NOT EDIT.

package main

import "log"

func main() {
	log.Fatal(dydStart())
}

Serving dynamic HTML without a .html file

You can write a function with this signature in any of the directories

func Serve_<X>(ctx *dyd.Context, write func(s string, args ...any) error)

Replace <X> with what this function returns (copied from modernc.org/dyd/etc.go)

func encodeFileName(s string) string {
	s = strings.ReplaceAll(s, "_", "_0")
	s = strings.ReplaceAll(s, ".", "_1")
	return strings.ReplaceAll(s, "-", "_2")
}

For example, "index.html" becomes "index_1html" and the complete function name will be "Serve_index_1html".

dyd can find such functions and adds them, using the decoded file name to the site map so the respective URLs will get handled by the user function. The function needs, of course, to produce a complete HTML document.

HTML file specification

From dyd's POV a HTML file is an UTF-8 encoded text file with zero or more processing instructions embedded. Processing instructions between the <? and ?> tags are used as Go code verbatim. The rest of the HTML file is used as HTML and again verbatim. That means a HTML file intended for dyd consumption does not have to be a complete, well-formed HTML document. Sometimes it is not because, for example, opening/closing tags can come from both HTML and Go, possibly intermixed. The preprocessor does not parse anything found in a HTML file, it looks only for the PI tags.

Also, the embedded Go code can contain characters forbidden in a HTML file. The well-formedness of HTML can be checked only after the web server fully produces a complete document and this is not computable at dyd/Go compile time in the general case.

The processing instruction opening and closing tags cannot nest and there's no escape sequence for the PI tags.

File name conventions

  • HTML file names should preferably match [a-zA-Z0-9-.]*.

  • HTML files matching ^.*_\.html$, like top_.html, are not externally bound, the web server will not handle their URLs. But all HTML URLs can be used by ctx.Include().

Server side includes

Use ctx.Include(path) to insert static/dynamic HTML, for example.

<div>
  <? ctx.Include("/layout/top_.html") ?>
</div>

See examples/include for a full, working demo.

Some batteries included

W3.CSS

Use ctx.W3CSS() to get the content of https://www.w3schools.com/w3css/4/w3.css without using an internet connection. It can be used like this

<!DOCTYPE html>
<html>
  <head>
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <style><? write("%s", ctx.W3CSS()) ?></style>
  </head>
  <body>
  </body>
</html>

More information about W3.CSS can be found at https://www.w3schools.com/w3css/default.asp.

The database

Use ctx.DBConn() to get a connection to the persistent database of the web server instance, created on demand in the working directory. It's the CGo-free SQLite database (http://modernc.org/sqlite).

See examples/db for a full, working demo.

The key/value store

Use ctx.KV() to access a persistent key/value store, created on demand within the web server instance database. Full documentation available at http://modernc.org/dyd/dyd.

See examples/kv for a full, working demo.

Documentation

Overview

Compile HTML with embedded Go to a single binary

Command dyd compiles HTML with embedded Go scripts to a self-contained, go install-able, single binary server producing dynamic web pages.

Hello world 

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>dyd Hello world</title>
  </head>
  <body>
    Hello world!
    <p><? write("req #%d for %s at %s", ctx.ReqSequence, ctx.Request.URL, time.Now().Format(time.RFC1123Z)) ?>
  </body>
</html>

The above file is all you need to produce a web server for a hello world example.

Prerequisities

- A working Go toolchain must be installed to use dyd. You can install Go from here: https://go.dev/dl/.

- The goimports tool is needed. To install goimports issue 

$ go install golang.org/x/tools/cmd/goimports@latest

Installation 

$ go install modernc.org/dyd@latest

Command line arguments 

$ dyd [-v] [-addr=<arg>] [-serve] [-modinit=<arg>] [-clean] [-htmlfmt] path

'path' points to the web server root, it defaults to ".".

-v adds more verbose output.

-addr is the web server address. It's the same argument as the first http.ListeAndServe wants. It defaults to ":6226".

-modinit creates a go.mod file with module path <arg> at 'path'. Ignored if a mod file at 'path' already exists.

-clean removes all generated files in the tree starting at 'path' and exits.

-serve compiles and runs the server.

-htmlfmt attempts to format html files. Use with caution, formatting may in some cases lose information. A backup is made before overwriting the HTML file. For example 

<link rel="stylesheet" href=<? write("%q", ctx.Request.URL.Query().Get("style")) ?>>

The preprocessor can handle this just fine, the formatter cannot. Write instead 

<? write(`<link rel="stylesheet" href=%q>`, ctx.Request.URL.Query().Get("style")) ?>

Usage example

Using the hello world example from above in index.html. 

0:~/tmp/dyd$ ls -la
total 12
drwxr-xr-x 2 jnml jnml 4096 May 22 13:02 .
drwxr-xr-x 7 jnml jnml 4096 May 22 12:42 ..
-rw-r--r-- 1 jnml jnml  269 May 22 12:57 index.html
0:~/tmp/dyd$

Generate and run the server (`-v` is optional). 

0:~/tmp/dyd$ dyd -v -modinit example.com/hello -serve
executing [go mod init example.com/hello]
preprocessing
executing [/home/jnml/bin/goimports -l -w /home/jnml/tmp/dyd/index.html.go]
building site map
url /index.html produced by example.com/hello.Serve_index_1html()
generating dyd.go
executing [/home/jnml/bin/goimports -l -w dyd.go]
generating /home/jnml/tmp/dyd/main.go
executing [/home/jnml/bin/goimports -l -w /home/jnml/tmp/dyd/main.go]
executing [go mod tidy]
starting server at :6226
executing [go run .]

Stop the server. 

^C
130:~/tmp/dyd$

How it works

Every directory is - or becomes - a Go package. dyd looks for .html files and generates Go functions serving the content. Any Go code can appear within HTML processing instructions `<?` and `?>`. This code is inserted into the serving function. Go code can refer to two arguments of the serving function. The 'ctx' argument provides context. The type is documented at http://modernc.org/dyd/dyd#Context. The other argument is 'write' and its a function declared as 

write func(s string, args ...any) error

The write function is only a small helper writing to the ctx.ResponseWriter. 

write("%d %s", 123, s)

is the same as 

fmt.Fprintf(ctx.ResponseWriter, "%d %s", 123, s)

The web root directory must be package main, directories under the root cannot be package main.

If there's no main.go/func main() in the web root, dyd will create one. If a manually written main.go/func main() exists, it will be kept. To invoke the web server in your manual func main(), add this as the first line of the function body 

defer dydStart()

or 

defer func() { log.Fatal(dydStart()) }()

etc. The 'dydStart' will be defined automatically in dyd.go. Consider this file name reserved by dyd.

Let's see what was generated for the Hello world example. 

130:~/tmp/dyd$ ls -la
total 32
drwxr-xr-x 2 jnml jnml 4096 May 22 13:03 .
drwxr-xr-x 7 jnml jnml 4096 May 22 12:42 ..
-rw-r----- 1 jnml jnml  328 May 22 13:03 dyd.go
-rw-r--r-- 1 jnml jnml   66 May 22 13:03 go.mod
-rw-r--r-- 1 jnml jnml  149 May 22 13:03 go.sum
-rw-r--r-- 1 jnml jnml  269 May 22 12:57 index.html
-rw-r----- 1 jnml jnml  516 May 22 13:03 index.html.go
-rw-r----- 1 jnml jnml  107 May 22 13:03 main.go
0:~/tmp/dyd$

The dyd.go file contains the `dydStart` function. It registers all the serving functions to handle the respective URLs. 

// Code generated by dyd, DO NOT EDIT.

package main

import (
	"modernc.org/dyd/dyd"
)

// dydStart starts the webapp. dydStart always returns a non-nil error.
func dydStart() error {
	app, err := dyd.NewApp()
	if err != nil {
		return err
	}

	app.Bind("/index.html", Serve_index_1html)
	return app.ListenAndServe(":6226")
}

Preprocessed index.html is in index.html.go. 

// Code generated by dyd, DO NOT EDIT.

package main

import (
	"time"

	"modernc.org/dyd/dyd"
)

// Serve_index_1html produces /index.html.
func Serve_index_1html(ctx *dyd.Context, write func(s string, args ...any) error) {
	write(`<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>dyd Hello world</title>
  </head>
  <body>
    Hello world!
    <p>`)
	write("req #%d for %s at %s", ctx.ReqSequence, ctx.Request.URL, time.Now().Format(time.RFC1123Z))
	write(`
  </body>
</html>
`)
}

And finally, main.go. 

// Code generated by dyd, DO NOT EDIT.

package main

import "log"

func main() {
	log.Fatal(dydStart())
}

Serving dynamic HTML without a .html file

You can write a function with this signature in any of the directories 

func Serve_<X>(ctx *dyd.Context, write func(s string, args ...any) error)

Replace <X> with what this function returns (copied from modernc.org/dyd/etc.go) 

func encodeFileName(s string) string {
	s = strings.ReplaceAll(s, "_", "_0")
	s = strings.ReplaceAll(s, ".", "_1")
	return strings.ReplaceAll(s, "-", "_2")
}

For example, "index.html" becomes "index_1html" and the complete function name will be "Serve_index_1html".

dyd can find such functions and adds them, using the decoded file name to the site map so the respective URLs will get handled by the user function. The function needs, of course, to produce a complete HTML document.

HTML file specification

From dyd's POV a HTML file is an UTF-8 encoded text file with zero or more processing instructions embedded. Processing instructions between the `<?` and `?>` tags are used as Go code verbatim. The rest of the HTML file is used as HTML and again verbatim. That means a HTML file intended for dyd consumption does not have to be a complete, well-formed HTML document. Sometimes it is not because, for example, opening/closing tags can come from both HTML and Go, possibly intermixed. the preprocessor does not parse anything found in a HTML file, it looks only for the PI tags.

Also, the embedded Go code can contain characters forbidden in a HTML file. The well-formedness of HTML can be checked only after the web server fully produces a complete document and this is not computable at dyd/Go compile time in the general case.

The processing instruction opening and closing tags cannot nest and there's no escape sequence for the PI tags.

File name conventions

HTML file names should preferably match `[a-zA-Z0-9-.]*`.

HTML files matching `^.*_\.html$`, like top_.html, are not externally bound, the web server will not handle their URLs. But all HTML URLs can be used by `ctx.Include()`.

Server side includes

Use `ctx.Include(path)` to insert static/dynamic HTML, for example. 

<div>
  <? ctx.Include("/layout/top_.html") ?>
</div>

See examples/include for a full, working demo.

Some batteries included

W3.CSS

Use `ctx.W3CSS()` to get the content of https://www.w3schools.com/w3css/4/w3.css without using an internet connection. It can be used like this 

<!DOCTYPE html>
<html>
  <head>
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <style><? write("%s", ctx.W3CSS()) ?></style>
  </head>
  <body>
  </body>
</html>

More information about W3.CSS can be found at https://www.w3schools.com/w3css/default.asp.

The database

Use `ctx.DBConn()` to get a connection to the persistent database of the web server instance, created on demand in the working directory. It's the CGo-free SQLite database (http://modernc.org/sqlite).

See examples/db for a full, working demo.

The key/value store

Use `ctx.KV()` to access a persistent key/value store, created on demand within the web server instance database. Full documentation available at http://modernc.org/dyd/dyd.

See examples/kv for a full, working demo.

Source Files

View all Source files

Directories

Path Synopsis
Package dyd provides runtime support for web servers created by the dyd command.
 Package dyd provides runtime support for web servers created by the dyd command.
examples
db2
dynamicstyles
static
Command getdyd.org is the web server for www.getdyd.org.
 Command getdyd.org is the web server for www.getdyd.org.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.