screener

package module
v0.0.0-...-efcac61 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 25, 2024 License: MIT Imports: 24 Imported by: 0

README ¶

screener logo

screener is a command-line interface (CLI) and Golang library for capturing screenshots of web pages. Uses Rod for headless browsing.

Features

  • Stream URLs: Input URLs via standard input (STDIN) for real-time processing.
  • Fixed Page Load Wait: Define a maximum wait time for web page loading before capturing screenshots.
  • Redirect Handling: Customize redirect behavior to follow or ignore URL changes.
  • Unique screenshots: Uses fuzzy-hashing to prevent saving duplicate screenshots, useful for large-scale scanning.
  • Concurrency: Support for concurrent requests for faster processing.
  • Certificate Error Handling: Option to ignore SSL certificate errors for testing environments.
  • HTTP/2 Control: Disable HTTP/2 for compatibility with various server configurations.
  • Custom User-Agent: Set a custom user-agent for requests to simulate different browsers or devices.
  • Imprint URL in Image: Choose to include the URL directly in the captured image for context and reference.

Installation

Go

go install github.com/root4loot/screener/cmd/screener@latest

Docker

git clone https://github.com/root4loot/screener.git && cd screener
docker build -t screener .
docker run -it -v "$(pwd)/screenshots:/app/screenshots" screener -t example.com

Usage

Usage: screener [options] (-t <target> | -l <targets.txt>)

INPUT:
   -t,  --target                   single target
   -l,  --list                     input file containing list of targets (one per line)

CONFIGURATIONS:
   -c,   --concurrency             number of concurrent requests                 (Default: 10)
   -to,  --timeout                 timeout for screenshot capture                (Default: 15 seconds)
   -ua,  --user-agent              set user agent                                (Default: Chrome Headless)
   -su,  --save-unique             save unique screenshots only                  (Default: false)
   -dh,  --disable-http2           disable HTTP2                                 (Default: true)
   -fr,  --follow-redirects        follow redirects                              (Default: true)
   -cw,  --capture-width           screenshot pixel width                        (Default: 1366)
   -ch,  --capture-height          screenshot pixel height                       (Default: 768)
   -cf,  --capture-full            capture full page                             (Default: 768)
   -fw,  --fixed-wait              fixed wait time before capturing (seconds)    (Default: 2)
   -dc,  --delay-between-capture   delay between capture (seconds)               (Default: 0)
   -ice, --ignore-cert-err         ignore certificate errors                     (Default: true)
   -isc, --ignore-status-codes     ignore HTTP status codes  (comma separated)   (Default: [])
   -s,   --silence                 silence output                                (Default: false)

OUTPUT:
   -o,   --outfolder               save images to given folder                   (Default: ./screenshots)
   -nu,  --no-url                  do not imprint URL in image                   (Default: false)
   -s,   --silence                 silence output
   -v,   --verbose                 verbose output
         --version                 display version

Example

Screenshot Single Target

Capture a single target. If the scheme (http/https) is not specified, then it will default to https and fallback to http if the former fails.

$ screener -t "example.com"
[screener] (INF) Preparing screenshot: https://example.com
[screener] (RES) Successful screenshot: https://example.com/

Screenshot Multiple Targets

Capture multiple targets.

$ cat targets.txt
142.250.74.110
google.com
bugcrowd.com
hackerone.com/sitemap.xml
http://example.com
https://scanme.sh

Note that targets can be IP, domain, or full URL.

$ screener -l targets.txt
[screener] (RES) Saved screenshot to screenshots/http_example.com.png
[screener] (RES) Saved screenshot to screenshots/https_hackerone.com_sitemap.xml.png
[screener] (RES) Saved screenshot to screenshots/https_scanme.sh.png
[screener] (RES) Saved screenshot to screenshots/https_142.250.74.110.png
[screener] (RES) Saved screenshot to screenshots/https_google.com.png
[screener] (RES) Saved screenshot to screenshots/https_bugcrowd.com.png

You may also "stream" targets to screener, capturing screenshots as they are received:

$ cat targets.txt | screener
[screener] (RES) Saved screenshot to screenshots/https_142.250.74.110.png
[screener] (RES) Saved screenshot to screenshots/https_google.com.png
[screener] (RES) Saved screenshot to screenshots/https_bugcrowd.com.png
[screener] (RES) Saved screenshot to screenshots/https_hackerone.com_sitemap.xml.png
[screener] (RES) Saved screenshot to screenshots/http_example.com.png
[screener] (RES) Saved screenshot to screenshots/https_scanme.sh.png

When dealing with many same-site URLs, use the -su or --save-unique flags to avoid saving multiple copies of the same screenshot. This makes it easier to sort through your screenshots! In the following example, we're using recrawl to crawl a target site and pipe its results to screener with the --save-unique flag set. For more information, see recrawl.

$ recrawl --target "hackerone.com" --hide-status --hide-media | screener --save-unique
[recrawl] (INF) Hiding status codes: true
[recrawl] (INF) Hiding media: [.png .jpg .jpeg .woff .woff2 .ttf .eot .svg .gif .ico .webp .mp4 .webm .mp3 .wav .flac .aac .ogg .m4a .flv .avi .mov .wmv .swf .mkv .m4v .3gp .3g2]
[recrawl] (INF) Notice: Output is being piped. 'Result' logs will be formatted accordingly.
[recrawl] (INF) Crawling target: https://hackerone.com

[screener] (INF) Skipping duplicate screenshot for https://hackerone.com/robots.txt
[screener] (RES) Saved screenshot to screenshots/https_hackerone.com_robots.txt.png
[screener] (INF) Skipping duplicate screenshot for https://www.hackerone.com/node/12420/
[screener] (RES) Saved screenshot to screenshots/https_www.hackerone.com_node_12420.png
[screener] (INF) Skipping duplicate screenshot for https://www.hackerone.com/product/challenge/
[screener] (RES) Saved screenshot to screenshots/https_www.hackerone.com_product_challenge.png
[screener] (INF) Skipping duplicate screenshot for https://www.hackerone.com/node/9916/
[screener] (RES) Saved screenshot to screenshots/https_www.hackerone.com_node_9916.png
...

Example Screenshot

screenshot example

Tips

  • Use -nu or --no-url flag to remove the URL from the image.
  • Use -su or --save-unique flag to save only screenshots that are unique.
  • macOS users can quickly access websites from screenshots: Press Space to preview an image, then mouse over the URL imprinted at the bottom. You can often click the link directly with Command + Click. If this method doesn't work, open the image in the Preview app to click the URL.

Library Example 📦

go get github.com/root4loot/screener

package main

import (
	"fmt"

	"github.com/root4loot/screener"
)

func main() {
	// Create runner with default options
	runner := screener.NewRunner()
	runner.Options.SaveScreenshots = true

	// Capture a single URL
	result := runner.Run("https://example.com", "https://hackerone.com")

	// Process the result
	for _, result := range result {
		fmt.Println(result.TargetURL, result.LandingURL, result.Error, len(result.Image))
	}
}

For more, see examples

License

See LICENSE

Contributing

See CONTRIBUTING.md

Documentation ¶

Index ¶

Constants ¶

View Source 
const Version = "0.0.0"

Variables ¶

This section is empty.

Functions ¶

func Init ¶ 

func Init()

func SetLogLevel ¶ 

func SetLogLevel(options *Options)

SetLogLevel initiates the logger and sets the log level based on the options

Types ¶

type Options ¶ 

type Options struct {
	Concurrency             int            // number of concurrent requests
	CaptureHeight           int            // height of the capture
	CaptureWidth            int            // width of the capture
	Timeout                 int            // Timeout for each capture (seconds)
	IgnoreCertificateErrors bool           // Ignore certificate errors
	DisableHTTP2            bool           // Disable HTTP2
	SaveScreenshots         bool           // Save screenshot to file
	SaveScreenshotsPath     string         // Path to save screenshots
	SaveUnique              bool           // Save unique screenshots only
	Scope                   *goscope.Scope // Scope to use
	UserAgent               string         // User agent to use
	MaxWait                 int            // Max wait time in seconds before taking screenshot, regardless of page load completion
	FixedWait               int            // Fixed wait time in seconds before taking screenshot, regardless of page load completion
	IgnoreStatusCodes       []int64        // List of status codes to ignore
	DelayBetweenCapture     int            // Delay in seconds between captures for multiple targets
	FollowRedirects         bool           // Follow redirects
	CaptureFull             bool           // Whether to take a full page screenshot
	ImprintURL              bool           // Whether to include the URL in the image
	Silence                 bool           // Silence output
	Verbose                 bool           // Verbose logging
}

Options contains options for the runner

func DefaultOptions ¶ 

func DefaultOptions() *Options

DefaultOptions returns default options

type Result ¶ 

type Result struct {
	Target     string
	TargetURL  string
	LandingURL string
	Image      []byte
	Error      error
}

func (Result) WriteToFolder ¶ 

func (result Result) WriteToFolder(writeFolderPath string) (filename string, err error)

type Runner ¶ 

type Runner struct {
	Options *Options
	// contains filtered or unexported fields
}

func NewRunner ¶ 

func NewRunner() *Runner

NewRunner returns a new runner

func NewRunnerWithOptions ¶ 

func NewRunnerWithOptions(options Options) *Runner

NewRunnerWithOptions returns a new runner with the specified options

func (*Runner) GetCustomFlags ¶ 

func (r *Runner) GetCustomFlags() []chromedp.ExecAllocatorOption

getCustomFlags returns custom chromedp.ExecAllocatorOptions based on the Runner's Options.

func (*Runner) Run ¶ 

func (r *Runner) Run(targets ...string) (results []Result)

Run captures one or more targets and returns the results. It handles both single and multiple targets.

func (*Runner) RunAsync ¶ 

func (r *Runner) RunAsync(resultsChan chan<- Result, targets ...string)

RunAsync captures multiple targets asynchronously and streams the results using channels.

Source Files ¶

View all Source files

Directories ¶

Path Synopsis
cmd
screener
examples
run
runAsync

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.