README
¶
AdGuard content blocking library
Pure GO library that implements AdGuard filtering rules syntax.
You can learn more about AdGuard filtering rules syntax from this article.
TODO:
- Basic filtering rules
- Core blocking syntax
- Basic engine
- Basic rules validation (don't match everything, unexpected modifiers, etc)
- Domain modifier semantics: https://github.com/AdguardTeam/AdguardBrowserExtension/issues/1474
- TLD support: https://kb.adguard.com/en/general/how-to-create-your-own-ad-filters#wildcard-for-tld
- Benchmark basic rules matching
- Hosts matching rules
- /etc/hosts matching
- $badfilter support for host-blocking network rules
- Memory optimization
- Tech document
- Cosmetic rules
- Basic element hiding and CSS rules
- Proper CSS rules validation
- ExtCSS rules
- ExtCSS rules validation
- Scriptlet rules
- JS rules
- Basic element hiding and CSS rules
- Proxy implementation
- Simple MITM proxy example
- Add cosmetic filters to the proxy example
- Handling cosmetic modifiers $elemhide, $generichide, $jsinject
- (!) Server certificate verification - it should pass badssl.com/dashboard/
- Use fetch metadata to detect the content type: https://www.w3.org/TR/fetch-metadata/
- Unit tests coverage
- Fix TODOs
- Proxy - handle CSP (including tags with CSP)
- Proxy - proper blocking page code
- Proxy - unblocking via a temporary cookie
- Proxy - content script caching
- Proxy - content script compression
- Proxy - brotli support (see here)
- Content script - babel plugin
- Content script - apply ExtCSS rules
- Content script - styles protection
- Content script - JS unit tests
- Content script - GO unit tests
- HTML filtering rules
- Advanced modifiers
- $important
- $replace
- $csp
- $cookie
- $redirect
- $badfilter
- $badfilter (https://github.com/AdguardTeam/CoreLibs/issues/1241)
- $ping modifier (https://github.com/AdguardTeam/CoreLibs/issues/1258)
How to use
TODO
Documentation
¶
Overview ¶
Package urlfilter contains implementation of AdGuard content blocking engine
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type CosmeticEngine ¶
type CosmeticEngine struct {
// contains filtered or unexported fields
}
CosmeticEngine combines all the cosmetic rules and allows to quickly find all rules matching this or that hostname
func NewCosmeticEngine ¶
func NewCosmeticEngine(s *filterlist.RuleStorage) *CosmeticEngine
NewCosmeticEngine builds a new cosmetic engine from the specified rule storage
func (*CosmeticEngine) Match ¶
func (e *CosmeticEngine) Match(hostname string, includeCSS, includeJS, includeGenericCSS bool) CosmeticResult
Match builds scripts and styles that needs to be injected into the specified page hostname is the page hostname includeCSS defines if we should inject any CSS and element hiding rules (see $elemhide) includeJS defines if we should inject JS into the page (see $jsinject) includeGenericCSS defines if we should inject generic CSS and element hiding rules (see $generichide) TODO: Additionally, we should provide a method that writes result to an io.Writer
type CosmeticResult ¶
type CosmeticResult struct {
ElementHiding StylesResult
CSS StylesResult
JS ScriptsResult
}
CosmeticResult represents all scripts and styles that needs to be injected into the page
type DNSEngine ¶
type DNSEngine struct {
// RulesCount is the count of rules loaded to the engine.
RulesCount int
// contains filtered or unexported fields
}
DNSEngine combines host rules and network rules and is supposed to quickly find matching rules for hostnames. First, it looks over network rules and returns first rule found. Then, if nothing found, it looks up the host rules.
func NewDNSEngine ¶
func NewDNSEngine(s *filterlist.RuleStorage) *DNSEngine
NewDNSEngine parses the specified filter lists and returns a DNSEngine built from them. key of the map is the filter list ID, value is the raw content of the filter list.
func (*DNSEngine) Match ¶
Match finds a matching rule for the specified hostname. It returns true and the list of rules found or false and nil. A list of rules is returned when there are multiple host rules matching the same domain, for example:
192.168.0.1 example.local 2000::1 example.local
func (*DNSEngine) MatchRequest ¶ added in v0.11.0
func (d *DNSEngine) MatchRequest(dReq *DNSRequest) (res *DNSResult, matched bool)
MatchRequest matches the specified DNS request. The return parameter matched is true if the result has a basic network rule or some host rules.
For compatibility reasons, it is also false when there are DNS rewrite and other kinds of special network rules, so users who need those will need to ignore the matched return parameter and instead inspect the results of the corresponding DNSResult getters.
TODO(ameshkov): return nil when there's no match. Currently, the logic is flawed because it analyzes the DNSResult even when matched is false and looks for $dnsrewrite rules.
type DNSRequest ¶ added in v0.11.0
type DNSRequest struct {
// ClientIP is the IP address to match against $client modifiers. The
// default zero value won't be considered.
ClientIP netip.Addr
// ClientName is the name to match against $client modifiers. The default
// empty value won't be considered.
ClientName string
// Hostname is the hostname to filter.
Hostname string
// SortedClientTags is the list of tags to match against $ctag modifiers.
SortedClientTags []string
// DNSType is the type of the resource record (RR) of a DNS request, for
// example "A" or "AAAA". See [rules.RRValue] for all acceptable constants
// and their corresponding values.
DNSType rules.RRType
// Answer if the filtering request is for filtering a DNS response.
Answer bool
}
DNSRequest represents a DNS query with associated metadata.
type DNSResult ¶ added in v0.9.0
type DNSResult struct {
// NetworkRule is the matched network rule, if any. If it is nil,
// HostRulesV4 and HostRulesV6 may still contain matched hosts-file style
// rules.
NetworkRule *rules.NetworkRule
// HostRulesV4 are the host rules with IPv4 addresses.
HostRulesV4 []*rules.HostRule
// HostRulesV6 are the host rules with IPv6 addresses.
HostRulesV6 []*rules.HostRule
// NetworkRules are all matched network rules. These include unprocessed
// DNS rewrites, exception rules, and so on.
NetworkRules []*rules.NetworkRule
}
DNSResult is the result of matching a DNS filtering request.
func (*DNSResult) DNSRewrites ¶ added in v0.14.0
func (res *DNSResult) DNSRewrites() (nrules []*rules.NetworkRule)
DNSRewrites returns $dnsrewrite network rules applying exception logic. For example, rules like:
||example.com^$dnsrewrite=127.0.0.1 ||example.com^$dnsrewrite=127.0.0.2 @@||example.com^$dnsrewrite=127.0.0.1
Will result in example.com being rewritten to only return 127.0.0.2.
To get all DNS rewrite rules without applying any exception logic, use (*DNSResult).DNSRewritesAll.
func (*DNSResult) DNSRewritesAll ¶ added in v0.14.0
func (res *DNSResult) DNSRewritesAll() (nrules []*rules.NetworkRule)
DNSRewritesAll returns all $dnsrewrite network rules. To get the rules with exception logic applied, use (*DNSResult).DNSRewrites.
type Engine ¶
type Engine struct {
// contains filtered or unexported fields
}
Engine represents the filtering engine with all the loaded rules
func NewEngine ¶
func NewEngine(s *filterlist.RuleStorage) *Engine
NewEngine parses the filtering rules and creates a filtering engine of them
func (*Engine) GetCosmeticResult ¶ added in v0.6.0
func (e *Engine) GetCosmeticResult(hostname string, option rules.CosmeticOption) CosmeticResult
GetCosmeticResult gets cosmetic result for the specified hostname and cosmetic options
func (*Engine) MatchRequest ¶ added in v0.6.0
func (e *Engine) MatchRequest(r *rules.Request) (res *rules.MatchingResult)
MatchRequest - matches the specified request against the filtering engine and returns the matching result.
type NetworkEngine ¶
type NetworkEngine struct {
// RulesCount is the count of rules added to the engine.
RulesCount int
// contains filtered or unexported fields
}
NetworkEngine is the engine that supports quick search over network rules.
func NewNetworkEngine ¶
func NewNetworkEngine(s *filterlist.RuleStorage) (engine *NetworkEngine)
NewNetworkEngine builds an instance of the network engine. This method scans the specified rule storage and adds all rules.NetworkRule found there to the internal lookup tables.
func NewNetworkEngineSkipStorageScan ¶ added in v0.15.0
func NewNetworkEngineSkipStorageScan(s *filterlist.RuleStorage) (engine *NetworkEngine)
NewNetworkEngineSkipStorageScan creates a new instance of *NetworkEngine, but unlike NewNetworkEngine it does not scans the storage.
func (*NetworkEngine) AddRule ¶ added in v0.15.0
func (n *NetworkEngine) AddRule(f *rules.NetworkRule, storageIdx int64)
AddRule adds rule to the network engine.
func (*NetworkEngine) Match ¶
func (n *NetworkEngine) Match(r *rules.Request) (*rules.NetworkRule, bool)
Match searches over all filtering rules loaded to the engine. It returns true if a match was found alongside the matching rule.
func (*NetworkEngine) MatchAll ¶
func (n *NetworkEngine) MatchAll(r *rules.Request) (result []*rules.NetworkRule)
MatchAll finds all rules matching the specified request regardless of the rule types. It will find both allowlist and blocklist rules.
type ScriptsResult ¶ added in v0.6.0
ScriptsResult contains scripts to be executed on a page
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package main is responsible for the command-line interface of the urlfilter content filtering proxy.
|
Package main is responsible for the command-line interface of the urlfilter content filtering proxy. |
|
examples
|
|
|
Package filterlist provides methods to work with filter lists.
|
Package filterlist provides methods to work with filter lists. |
|
Package filterutil contains helper functions used by urlfilter
|
Package filterutil contains helper functions used by urlfilter |
|
Package lookup implements index structures that we use to improve matching speed in the engines.
|
Package lookup implements index structures that we use to improve matching speed in the engines. |
|
Package proxy contains implementation of the filtering MITM proxy
|
Package proxy contains implementation of the filtering MITM proxy |
|
Package rules contains implementation of all kinds of blocking rules
|
Package rules contains implementation of all kinds of blocking rules |