README
¶
WebLoop
Scriptable, headless WebKit with a Go API. Like PhantomJS, but for Go. Render static HTML versions of dynamic JavaScript applications, automate browsing, run arbitrary JavaScript in a browser window context, etc., all from Go or the command line.
Requirements
- Go >= 1.2rc1 (due to #3250)
- WebKitGTK+ >= 2.0.0
- go-webkit2
For instructions on installing these dependencies, see the go-webkit2 README.
To install WebLoop, run: go get github.com/sourcegraph/webloop/...
Usage
Static HTML rendering reverse proxy
The included command static-reverse-proxy proxies a dynamic JavaScript application and serves an equivalent statically rendered HTML website to clients. Run it with:
$ go install github.com/sourcegraph/webloop/...
$ static-reverse-proxy
For example, to proxy a dynamic application at http://example.com and serve an equivalent statically rendered HTML website on http://localhost:13000, run:
$ static-reverse-proxy -target=http://example.com -http=:13000
Run with -h to see more information.
Rendering static HTML from a dynamic, single-page AngularJS app
StaticRenderer is an HTTP handler that serves a static HTML version of a
dynamic web application. Use it like:
staticHandler := &webloop.StaticRenderer{
TargetBaseURL: "http://dynamic-app.example.com",
WaitTimeout: time.Second * 3,
ReturnUnfinishedPages: true
}
http.Handle("/", staticHandler)
See the examples/angular-static-seo/ directory for example code. Run the included binary with:
$ go run examples/angular-static-seo/server.go
Instructions will be printed for accessing the 2 local demo HTTP servers. Run
with -h to see more information.
Operating a headless WebKit and running arbitrary JavaScript in the page
package webloop_test
import (
"fmt"
"os"
"runtime"
"github.com/gotk3/gotk3/gtk"
"github.com/sourcegraph/webloop"
)
func Example() {
gtk.Init(nil)
go func() {
runtime.LockOSThread()
gtk.Main()
}()
ctx := webloop.New()
view := ctx.NewView()
defer view.Close()
view.Open("http://google.com")
err := view.Wait()
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to load URL: %s", err)
os.Exit(1)
}
res, err := view.EvaluateJavaScript("document.title")
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to run JavaScript: %s", err)
os.Exit(1)
}
fmt.Printf("JavaScript returned: %q\n", res)
// output:
// JavaScript returned: "Google"
}
See webloop_test.go for more examples.
TODO
- Set up CI testing. This is difficult because all of the popular CI services run older versions of Ubuntu that make it difficult to install WebKitGTK+ >= 2.0.0.
- Add the ability for JavaScript code to send messages to WebLoop, similar to [PhantomJS's callPhantom] (https://github.com/ariya/phantomjs/wiki/API-Reference-WebPage#oncallback) mechanism.
Users
- WebLoop is used to render static HTML pages on Sourcegraph for search engine crawlers
Contributors
See the AUTHORS file for a list of contributors.
Submit contributions via GitHub pull request. Patches should include tests and should pass golint.
Documentation
¶
Overview ¶
Example ¶
package main
import (
"fmt"
"os"
"runtime"
"github.com/gotk3/gotk3/gtk"
"github.com/sourcegraph/webloop"
)
func main() {
gtk.Init(nil)
go func() {
runtime.LockOSThread()
gtk.Main()
}()
ctx := webloop.New()
view := ctx.NewView()
defer view.Close()
view.Open("http://google.com")
err := view.Wait()
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to load URL: %s", err)
os.Exit(1)
}
res, err := view.EvaluateJavaScript("document.title")
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to run JavaScript: %s", err)
os.Exit(1)
}
fmt.Printf("JavaScript returned: %q\n", res)
}
Output: JavaScript returned: "Google"
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ErrLoadFailed = errors.New("load failed")
ErrLoadFailed indicates that the View failed to load the requested resource.
Functions ¶
This section is empty.
Types ¶
type StaticRenderer ¶
type StaticRenderer struct {
// TargetBaseURL is the baseURL of the dynamic content URLs.
TargetBaseURL string
// Context is the WebLoop context to create views in.
Context Context
// WaitTimeout is the maximum duration to wait for a loaded page to set
// window.$renderStaticReady.
WaitTimeout time.Duration
// ReturnUnfinishedPages is whether a page that has not set
// window.$renderStaticReady after WaitTimeout is sent to the browser in a
// (potentially) unfinished state. If false, an HTTP 502 Bad Gateway error
// will be returned.
//
// If you are unsure of whether all accessible pages set
// window.$renderStaticReady (perhaps you could forget to do so on a few
// pages), then setting ReturnUnfinishedPages would suppress errors for
// those pages, at the possible expense of sending out unfinished pages that
// take a long time to load.
ReturnUnfinishedPages bool
// RemoveJavaScript indicates whether <script> tags will be removed. When
// generating static HTML pages from a dynamic JavaScript app, this is often
// necessary because the JavaScript expects to run on a non-bootstrapped
// page. This option is not guaranteed to disable all <script> tags and
// should relied upon for security purposes.
RemoveScripts bool
// Log is the logger to use for log messages. If nil, there is no log
// output.
Log *log.Logger
// contains filtered or unexported fields
}
StaticRenderer generates and returns static HTML based on a snapshot of a Web page's computed HTML.
func (*StaticRenderer) Release ¶
func (h *StaticRenderer) Release()
Release releases resources used by this handler, such as the view. If this handler is reused after calling Release, the view is automatically recreated.
func (*StaticRenderer) ServeHTTP ¶
func (h *StaticRenderer) ServeHTTP(w http.ResponseWriter, r *http.Request)
ServeHTTP implements net/http.Handler.
func (*StaticRenderer) StartGTK ¶
func (h *StaticRenderer) StartGTK()
StartGTK ensures that the GTK+ main loop has started. If it has already been started by StartGTK, it will not start it again. If another goroutine is already running the GTK+ main loop, StartGTK's behavior is undefined.
type View ¶
View represents a WebKit view that can load resources at a given URL and query information about them.
func (*View) Close ¶
func (v *View) Close()
Close closes the view and releases associated resources. Ensure that Close is called after all other pending operations on View have returned, or they may hang indefinitely.
func (*View) EvaluateJavaScript ¶
EvaluateJavaScript runs the JavaScript in script in the view's context and returns the script's result as a Go value.
Source Files
Directories