README
¶
OONI Probe Measurement Engine
This directory contains private Go packages. Collectively these package are the measurement engine that empowers OONI Probe. Most of the code in this package derives from the ooni/probe-engine discontinued repository.
Reading documentation from the command line
You can read the Go documentation of a package by using go doc -all.
For example:
go doc -all ./internal/netxlite
Reading documentation in the browser
You can install the pkgsite tool using this command:
go install golang.org/x/pkgsite@latest
To run pkgsite, use:
pkgsite
Then visit http://127.0.0.1:8080/github.com/ooni/probe-cli/v3 with your browser to browse the documentation.
Getting information about a package
Use the go list -json subcommand. For example:
go list -json ./internal/netxlite
Generating a dependency graph
You can get a graph of the dependencies using kisielk/godepgraph.
For example:
godepgraph -s -novendor -p golang.org,gitlab.com ./internal/engine | dot -Tpng -o deps.png
You can further tweak which packages to exclude by appending
prefixes to the list passed to the -p flag.
Tutorials
The tutorial package contains tutorials on writing new experiments, using measurements libraries, and networking code.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package bytecounter contains code to track the number of bytes sent and received by a probe.
|
Package bytecounter contains code to track the number of bytes sent and received by a probe. |
|
Package checkincache contains an on-disk cache for check-in responses.
|
Package checkincache contains an on-disk cache for check-in responses. |
|
cmd
|
|
|
apitool
Command apitool is a simple tool to fetch individual OONI measurements.
|
Command apitool is a simple tool to fetch individual OONI measurements. |
|
buildtool/internal/buildtoolmodel
Package buildtoolmodel contains the model for buildtool.
|
Package buildtoolmodel contains the model for buildtool. |
|
buildtool/internal/buildtooltest
Package buildtooltest contains code for testing buildtool.
|
Package buildtooltest contains code for testing buildtool. |
|
gardener
Command gardener helps with test-lists management.
|
Command gardener helps with test-lists management. |
|
gardener/internal/aggregationapi
Package aggregationapi allows one to query the OONI aggregation API.
|
Package aggregationapi allows one to query the OONI aggregation API. |
|
gardener/internal/dnsfix
Package dnsfix implements the dnsfix command.
|
Package dnsfix implements the dnsfix command. |
|
gardener/internal/dnsreport
Package dnsreport implements the dnsreport subcommand.
|
Package dnsreport implements the dnsreport subcommand. |
|
gardener/internal/sync
Package sync implements the sync subcommand.
|
Package sync implements the sync subcommand. |
|
gardener/internal/testlists
Package testlists contains code to read/rewrite the test lists.
|
Package testlists contains code to read/rewrite the test lists. |
|
getresources
Command getresources downloads the resources
|
Command getresources downloads the resources |
|
ghgen
Command ghgen regenerates selected GitHub actions.
|
Command ghgen regenerates selected GitHub actions. |
|
miniooni
Command miniooni is a simple binary for research and QA purposes with a CLI interface similar to MK and OONI Probe v2.x.
|
Command miniooni is a simple binary for research and QA purposes with a CLI interface similar to MK and OONI Probe v2.x. |
|
oohelper
Command oohelper contains a simple command line client for the Web Connectivity test helper.
|
Command oohelper contains a simple command line client for the Web Connectivity test helper. |
|
oohelperd
Command oohelperd implements the Web Connectivity test helper.
|
Command oohelperd implements the Web Connectivity test helper. |
|
oonireport
Command oonireport uploads reports stored on disk to the OONI collector.
|
Command oonireport uploads reports stored on disk to the OONI collector. |
|
ooporthelper
command ooporthelper implements the Port Filtering test helper
|
command ooporthelper implements the Port Filtering test helper |
|
printversion
Command printversion prints the current version of this repository.
|
Command printversion prints the current version of this repository. |
|
tinyjafar
Command tinyjafar implements a subset of the CLI flags of the original jafar tool.
|
Command tinyjafar implements a subset of the CLI flags of the original jafar tool. |
|
Package engine contains the engine API.
|
Package engine contains the engine API. |
|
Package enginelocate implements IP lookup, resolver lookup, and geolocation.
|
Package enginelocate implements IP lookup, resolver lookup, and geolocation. |
|
Package enginenetx contains engine-specific network-extensions.
|
Package enginenetx contains engine-specific network-extensions. |
|
Package engineresolver contains the resolver used by the OONI engine.
|
Package engineresolver contains the resolver used by the OONI engine. |
|
experiment
|
|
|
dash
Package dash implements the DASH network experiment.
|
Package dash implements the DASH network experiment. |
|
dnscheck
Package dnscheck contains the DNS check experiment.
|
Package dnscheck contains the DNS check experiment. |
|
dnsping
Package dnsping is the experimental dnsping experiment.
|
Package dnsping is the experimental dnsping experiment. |
|
echcheck
Package echcheck contains the ECH blocking network experiment.
|
Package echcheck contains the ECH blocking network experiment. |
|
example
Package example contains a simple example experiment.
|
Package example contains a simple example experiment. |
|
fbmessenger
Package fbmessenger contains the Facebook Messenger network experiment.
|
Package fbmessenger contains the Facebook Messenger network experiment. |
|
hhfm
Package hhfm contains the HTTP Header Field Manipulation network experiment.
|
Package hhfm contains the HTTP Header Field Manipulation network experiment. |
|
hirl
Package hirl contains the HTTP Invalid Request Line network experiment.
|
Package hirl contains the HTTP Invalid Request Line network experiment. |
|
httphostheader
Package httphostheader contains the HTTP host header network experiment.
|
Package httphostheader contains the HTTP host header network experiment. |
|
ndt7
Package ndt7 contains the ndt7 network experiment.
|
Package ndt7 contains the ndt7 network experiment. |
|
portfiltering
Package portfiltering implements the portfiltering experiment
|
Package portfiltering implements the portfiltering experiment |
|
psiphon
Package psiphon implements the psiphon network experiment.
|
Package psiphon implements the psiphon network experiment. |
|
quicping
Package quicping implements the quicping network experiment.
|
Package quicping implements the quicping network experiment. |
|
riseupvpn
Package riseupvpn contains the RiseupVPN network experiment.
|
Package riseupvpn contains the RiseupVPN network experiment. |
|
signal
Package signal contains the Signal network experiment.
|
Package signal contains the Signal network experiment. |
|
simplequicping
Package simplequicping is the experimental simplequicping experiment.
|
Package simplequicping is the experimental simplequicping experiment. |
|
sniblocking
Package sniblocking contains the SNI blocking network experiment.
|
Package sniblocking contains the SNI blocking network experiment. |
|
stunreachability
Package stunreachability contains the STUN reachability experiment.
|
Package stunreachability contains the STUN reachability experiment. |
|
tcpping
Package tcpping is the experimental tcpping experiment.
|
Package tcpping is the experimental tcpping experiment. |
|
telegram
Package telegram contains the Telegram network experiment.
|
Package telegram contains the Telegram network experiment. |
|
tlsmiddlebox
Package tlsmiddlebox implements the tlsmiddlebox experiment
|
Package tlsmiddlebox implements the tlsmiddlebox experiment |
|
tlsping
Package tlsping is the experimental tlsping experiment.
|
Package tlsping is the experimental tlsping experiment. |
|
tlstool
Package tlstool contains a TLS tool that we are currently using for running quick and dirty experiments.
|
Package tlstool contains a TLS tool that we are currently using for running quick and dirty experiments. |
|
tlstool/internal
Package internal contains the implementation of tlstool.
|
Package internal contains the implementation of tlstool. |
|
tor
Package tor contains the tor experiment.
|
Package tor contains the tor experiment. |
|
torsf
Package torsf contains the torsf experiment.
|
Package torsf contains the torsf experiment. |
|
urlgetter
Package urlgetter implements a nettest that fetches a URL.
|
Package urlgetter implements a nettest that fetches a URL. |
|
vanillator
Package vanillator contains the vanilla_tor experiment.
|
Package vanillator contains the vanilla_tor experiment. |
|
webconnectivity
Package webconnectivity implements OONI's Web Connectivity experiment.
|
Package webconnectivity implements OONI's Web Connectivity experiment. |
|
webconnectivity/internal
Package internal contains internal code.
|
Package internal contains internal code. |
|
webconnectivitylte
Package webconnectivitylte implements the web_connectivity experiment.
|
Package webconnectivitylte implements the web_connectivity experiment. |
|
whatsapp
Package whatsapp contains the WhatsApp network experiment.
|
Package whatsapp contains the WhatsApp network experiment. |
|
feature
|
|
|
psiphonfeat
Package psiphonfeat implements the psiphon feature.
|
Package psiphonfeat implements the psiphon feature. |
|
Package flagx contains extensions for the standard library flag package.
|
Package flagx contains extensions for the standard library flag package. |
|
Package fsx contains io/fs extensions.
|
Package fsx contains io/fs extensions. |
|
Package geoipx contains code to use the embedded MaxMind-like databases.
|
Package geoipx contains code to use the embedded MaxMind-like databases. |
|
Package httpapi contains code for calling HTTP APIs.
|
Package httpapi contains code for calling HTTP APIs. |
|
Package httpx contains http extensions.
|
Package httpx contains http extensions. |
|
Package hujsonx contains github.com/tailscale/hujson extensions.
|
Package hujsonx contains github.com/tailscale/hujson extensions. |
|
Package humanize is like dustin/go-humanize.
|
Package humanize is like dustin/go-humanize. |
|
Package idnax contains IDNA extensions.
|
Package idnax contains IDNA extensions. |
|
Package inputparser contains code to parse experiments input.
|
Package inputparser contains code to parse experiments input. |
|
Package kvstore implements model.KeyValueStore.
|
Package kvstore implements model.KeyValueStore. |
|
legacy
|
|
|
assetsdir
Package assetsdir contains code to cleanup the assets dir.
|
Package assetsdir contains code to cleanup the assets dir. |
|
kvstore2dir
Package kvstore2dir migrates $OONI_HOME/kvstore2 to $OONI_HOME/engine.
|
Package kvstore2dir migrates $OONI_HOME/kvstore2 to $OONI_HOME/engine. |
|
legacymodel
Package legacymodel contains legacy content that used to be in internal/model
|
Package legacymodel contains legacy content that used to be in internal/model |
|
measurex
Package measurex contains measurement extensions.
|
Package measurex contains measurement extensions. |
|
mockable
Package mockable contains mockable objects
|
Package mockable contains mockable objects |
|
netx
Package netx contains code to perform network measurements.
|
Package netx contains code to perform network measurements. |
|
tracex
Package tracex performs measurements using tracing.
|
Package tracex performs measurements using tracing. |
|
Package logmodel contains the core log model.
|
Package logmodel contains the core log model. |
|
Package logx contains logging extensions.
|
Package logx contains logging extensions. |
|
Package measurexlite contains measurement extensions.
|
Package measurexlite contains measurement extensions. |
|
Package memoryless helps repeated calls to a function be distributed across time in a memoryless fashion.
|
Package memoryless helps repeated calls to a function be distributed across time in a memoryless fashion. |
|
Package minipipeline implements a minimal data processing pipeline used to analyze local measurements collected by OONI Probe.
|
Package minipipeline implements a minimal data processing pipeline used to analyze local measurements collected by OONI Probe. |
|
Package mlablocatev2 implements m-lab locate services API v2.
|
Package mlablocatev2 implements m-lab locate services API v2. |
|
Package mocks contains mocks for implementing unit tests.
|
Package mocks contains mocks for implementing unit tests. |
|
Package model contains the shared interfaces and data structures.
|
Package model contains the shared interfaces and data structures. |
|
Package multierror contains code to manage multiple errors.
|
Package multierror contains code to manage multiple errors. |
|
Package must contains functions that panic on error.
|
Package must contains functions that panic on error. |
|
Package netemx contains code to run integration tests using github.com/ooni/netem.
|
Package netemx contains code to run integration tests using github.com/ooni/netem. |
|
Package netxlite contains network extensions.
|
Package netxlite contains network extensions. |
|
Package ooapi describes how the OONI API works.
|
Package ooapi describes how the OONI API works. |
|
Package oonirun contains code to run OONI experiments.
|
Package oonirun contains code to run OONI experiments. |
|
Package optional contains safer code to handle optional values.
|
Package optional contains safer code to handle optional values. |
|
Package platform allows you to obtain the platform name.
|
Package platform allows you to obtain the platform name. |
|
Package probeservices contains code to contact OONI probe services.
|
Package probeservices contains code to contact OONI probe services. |
|
Package progress contains utilities to emit progress.
|
Package progress contains utilities to emit progress. |
|
Package ptx contains code to use pluggable transports.
|
Package ptx contains code to use pluggable transports. |
|
Package randx contains math/rand extensions.
|
Package randx contains math/rand extensions. |
|
Package registry contains a registry of all the available experiments.
|
Package registry contains a registry of all the available experiments. |
|
Package runtimex contains runtime extensions.
|
Package runtimex contains runtime extensions. |
|
Package shellx helps to write shell-like Go code.
|
Package shellx helps to write shell-like Go code. |
|
shellxtesting
Package shellxtesting supports shellx testing.
|
Package shellxtesting supports shellx testing. |
|
Package strcasex converts strings to various cases.
|
Package strcasex converts strings to various cases. |
|
Package stuninput contains stun targets as well as code to format such targets according to various conventions.
|
Package stuninput contains stun targets as well as code to format such targets according to various conventions. |
|
Package testingproxy contains shared test cases for the proxies.
|
Package testingproxy contains shared test cases for the proxies. |
|
Package testingquic allows to retrieve the domain and endpoint to use for all the integration tests that use QUIC.
|
Package testingquic allows to retrieve the domain and endpoint to use for all the integration tests that use QUIC. |
|
Package testingsock5 is a netem-aware fork of https://github.com/armon/go-socks5.
|
Package testingsock5 is a netem-aware fork of https://github.com/armon/go-socks5. |
|
Package testingx contains code useful for testing.
|
Package testingx contains code useful for testing. |
|
Package throttling wraps connections to measure throttling.
|
Package throttling wraps connections to measure throttling. |
|
Package torlogs contains code to read tor logs.
|
Package torlogs contains code to read tor logs. |
|
Package tunnel allows to create tunnels to speak with OONI backends and other services.
|
Package tunnel allows to create tunnels to speak with OONI backends and other services. |
|
mocks
Package mocks contains mocks for tunnel.
|
Package mocks contains mocks for tunnel. |
|
tutorial
|
|
|
dslx/chapter02
## Getting started
|
## Getting started |
|
experiment/torsf/chapter01
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
experiment/torsf/chapter02
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
generator
Command generator generates or re-generates the tutorial chapters.
|
Command generator generates or re-generates the tutorial chapters. |
|
measurex/chapter01
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
measurex/chapter02
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
measurex/chapter03
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
measurex/chapter04
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
measurex/chapter05
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
measurex/chapter06
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
measurex/chapter07
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
measurex/chapter08
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
measurex/chapter09
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
measurex/chapter10
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
measurex/chapter11
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
measurex/chapter12
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
measurex/chapter13
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
measurex/chapter14
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
netxlite/chapter01
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
netxlite/chapter02
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
netxlite/chapter03
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
netxlite/chapter04
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
netxlite/chapter05
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
netxlite/chapter06
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
netxlite/chapter07
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
netxlite/chapter08
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
Package version contains version information.
|
Package version contains version information. |
|
Package webconnectivityalgo contains Web Connectivity algorithms.
|
Package webconnectivityalgo contains Web Connectivity algorithms. |
|
Package webconnectivityqa contains code to perform Web Connectivity QA.
|
Package webconnectivityqa contains code to perform Web Connectivity QA. |
|
Package x contains highly experimental packages.
|
Package x contains highly experimental packages. |
|
dsljavascript
Package dsljavascript allows running experiments written in JavaScript.
|
Package dsljavascript allows running experiments written in JavaScript. |
|
dsljson
Package dsljson allows expressing the measurement DSL using JSON.
|
Package dsljson allows expressing the measurement DSL using JSON. |
|
dslvm
Package dslvm contains low-level code for implementing the measurements DSL.
|
Package dslvm contains low-level code for implementing the measurements DSL. |
|
dslx
Package dslx implements a domain specific language for writing network experiments.
|
Package dslx implements a domain specific language for writing network experiments. |
Click to show internal directories.
Click to hide internal directories.