mirror of
https://github.com/novatiq/packages.git
synced 2026-04-30 15:38:40 +01:00
adblock: release 1.1.0
* support more router modes, as long as the firewall and the DNS server
are enabled
* new source sites can be added & changed via awk ruleset in uci config
* source domain count, last update time & overall count will be stored
in uci config
* added 3 new source sites:
ransomware tracker
rolist/easylist
winspy
* switch to minimal inline base64 encoded 1×1 GIF for pixel server
(separate png image no longer needed)
* simplified uci parser
* source download & domain sort optimization
* add whitelist parser with wildcard support
* reduced code size & various cleanups
* updated documentation
Signed-off-by: Dirk Brenken <openwrt@brenken.org>
(cherry picked from commit b9aeb82c95)
This commit is contained in:
Dirk Brenken
committed by
Hannu Nyman
Hannu Nyman
parent
9aa63712b3
commit
bb5f4f6d29
+97
-29
@@ -7,6 +7,8 @@ A lot of people already use adblocker plugins within their desktop browsers, but
|
||||
* support of the following domain blocklist sources (free for private usage, for commercial use please check their individual licenses):
|
||||
* [adaway](https://adaway.org)
|
||||
* => infrequent updates, approx. 400 entries (enabled by default)
|
||||
* [blacklist]()
|
||||
* => static local blacklist, located by default in '/etc/adblock/adblock.blacklist'
|
||||
* [disconnect](https://disconnect.me)
|
||||
* => numerous updates on the same day, approx. 6.500 entries (enabled by default)
|
||||
* [dshield](http://dshield.org)
|
||||
@@ -19,8 +21,12 @@ A lot of people already use adblocker plugins within their desktop browsers, but
|
||||
* => daily updates, approx. 1.500 entries
|
||||
* [openphish](https://openphish.com)
|
||||
* => numerous updates on the same day, approx. 1.800 entries
|
||||
* [palevotracker](https://palevotracker.abuse.ch)
|
||||
* [palevo tracker](https://palevotracker.abuse.ch)
|
||||
* => daily updates, approx. 15 entries
|
||||
* [ransomware tracker](https://ransomwaretracker.abuse.ch)
|
||||
* => daily updates, approx. 150 entries
|
||||
* [rolist/easylist](https://easylist-downloads.adblockplus.org/rolist+easylist.txt)
|
||||
* => weekly updates, approx. 600 entries
|
||||
* [ruadlist/easylist](https://code.google.com/p/ruadlist)
|
||||
* => weekly updates, approx. 2.000 entries
|
||||
* [shallalist](http://www.shallalist.de) (categories "adv" "costtraps" "spyware" "tracker" "warez" enabled by default)
|
||||
@@ -33,82 +39,144 @@ A lot of people already use adblocker plugins within their desktop browsers, but
|
||||
* => weekly updates, approx. 12.000 entries
|
||||
* [winhelp](http://winhelp2002.mvps.org)
|
||||
* => infrequent updates, approx. 15.000 entries
|
||||
* [winspy](https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/hostsBlockWindowsSpy.txt)
|
||||
* => infrequent updates, approx. 120 entries
|
||||
* [yoyo](http://pgl.yoyo.org/adservers)
|
||||
* => weekly updates, approx. 2.500 entries (enabled by default)
|
||||
* [zeustracker](https://zeustracker.abuse.ch)
|
||||
* [zeus tracker](https://zeustracker.abuse.ch)
|
||||
* => daily updates, approx. 440 entries
|
||||
* zero-conf like automatic installation & setup, usually no manual changes needed (i.e. ip address, network devices etc.)
|
||||
* supports a wide range of router modes, as long as the firewall and the DNS server are enabled
|
||||
* full IPv4 and IPv6 support
|
||||
* each blocklist source will be updated and processed separately
|
||||
* timestamp check to download and process only updated adblock list sources
|
||||
* overall duplicate removal in separate adblock lists (will be automatically disabled on low memory systems)
|
||||
* adblock source list parsing by fast & flexible regex rulesets
|
||||
* additional white- and blacklist support for manual overrides
|
||||
* additional whitelist for manual overrides, located by default in /etc/adblock/adblock.whitelist
|
||||
* quality checks during & after update of adblock lists to ensure a reliable dnsmasq service
|
||||
* basic adblock statistics via iptables packet counters for each chain
|
||||
* basic adblock statistics via iptables packet counters
|
||||
* list states, (overall) list counts & last update time will be stored in uci config
|
||||
* status & error logging to stdout and syslog
|
||||
* use a dynamic uhttpd instance as an adblock pixel server
|
||||
* use dynamic iptables rulesets for adblock related redirects/rejects
|
||||
* openwrt init system support (start/stop/restart/reload)
|
||||
* hotplug support, the adblock start will be triggered by wan 'ifup' event
|
||||
* optional: adblock list backup/restore (disabled by default)
|
||||
* optional: add new adblock sources via uci config (see example below)
|
||||
|
||||
## Prerequisites
|
||||
* [openwrt](https://openwrt.org), tested with latest stable release (Chaos Calmer 15.05) and with current trunk (Designated Driver > r47025)
|
||||
* usual openwrt setup with 'iptables' & 'uhttpd', additional required software packages:
|
||||
* [openwrt](https://openwrt.org), tested with latest stable release (Chaos Calmer) and with current trunk (Designated Driver)
|
||||
* usual openwrt setup with enabled 'iptables', 'dnsmasq' and 'uhttpd' - dump AP modes are _not_ supported!
|
||||
* additional required software packages:
|
||||
* wget
|
||||
* optional: 'kmod-ipt-nat6' for IPv6 support
|
||||
* the above dependencies and requirements will be checked during package installation & script runtime
|
||||
|
||||
## Designated Driver Installation & Usage
|
||||
* install the adblock package (*opkg install adblock*)
|
||||
* start the adblock service with */etc/init.d/adblock start* and check *logread -e "adblock"* for adblock related information
|
||||
* optional: enable/disable your required adblock list sources in */etc/config/adblock* - 'adaway', 'disconnect' and 'yoyo' are enabled by default
|
||||
* install 'adblock' (_opkg install adblock_)
|
||||
* adblock starts automatically during boot/wan-ifup event, check _logread -e "adblock"_ for adblock related information
|
||||
* optional: start/restart/stop the adblock service manually with _/etc/init.d/adblock_
|
||||
* optional: enable/disable your required adblock list sources in _/etc/config/adblock_ - 'adaway', 'disconnect' and 'yoyo' are enabled by default
|
||||
* optional: maintain the adblock service in luci under 'System => Startup'
|
||||
|
||||
## LuCI adblock companion package
|
||||
For easy management of the various blocklist sources and and the adblock options there is also a nice & efficient LuCI frontend available.
|
||||
Please install the package 'luci-app-adblock' (*opkg install luci-app-adblock*). Then you will find the application in LuCI located under 'Services' menu.
|
||||
Thanks to Hannu Nyman for this great adblock LuCI frontend!
|
||||
* for easy management of the various blocklist sources and adblock options there is also a nice & efficient LuCI frontend available
|
||||
* install 'luci-app-adblock' (_opkg install luci-app-adblock_)
|
||||
* the application is located in LuCI under 'Services' menu
|
||||
* _Thanks to Hannu Nyman for this great adblock LuCI frontend!_
|
||||
|
||||
## Chaos Calmer installation notes
|
||||
* currently the adblock package is *not* part of the CC package repository
|
||||
* download the latest adblock package *adblock_x.xx.x-1_all.ipk* from a development snapshot [package directory](https://downloads.openwrt.org/snapshots/trunk/ar71xx/nand/packages/packages)
|
||||
* due to server hardware troubles the package directory link above may not work, if so please check the [main openwrt download area](https://downloads.openwrt.org) manually
|
||||
* manual transfer the package to your router and install the opkg package as usual
|
||||
* 'adblock' and 'luci-app-adblock' are _not_ available as .ipk packages in the Chaos Calmer download repository
|
||||
* download both packages from a development snapshot package directory:
|
||||
* for 'adblock' look [here](https://downloads.openwrt.org/snapshots/trunk/ar71xx/generic/packages/packages/)
|
||||
* for 'luci-app-adblock' look [here](https://downloads.openwrt.org/snapshots/trunk/ar71xx/generic/packages/luci/)
|
||||
* manually transfer the packages to your routers temp directory (with tools like _sshfs_ or _winscp_)
|
||||
* install the packages with _opkg install <...>_ as described above
|
||||
|
||||
## Tweaks
|
||||
* there is no need to enable all blacklist sites at once, for normal use one to three adblock list sources should be sufficient
|
||||
* if you really need to handle all blacklists at once add an usb stick or any other storage device to enlarge your temp directory with a swap partition => see [openwrt wiki](https://wiki.openwrt.org/doc/uci/fstab) for further details
|
||||
* add personal domain white- or blacklist entries as an additional blocklist source, one domain per line (wildcards & regex are not allowed!), by default both empty lists are located in */etc/adblock*
|
||||
* enable the backup/restore feature, to restore automatically the latest stable backup of your adblock lists in case of any (partial) processing error (i.e. a single blocklist source server is down). Please use an (external) solid partition and *not* your volatile router temp directory for this
|
||||
* for a scheduled call of the adblock service via */etc/init.d/adblock start* add an appropriate crontab entry
|
||||
* in case of any script runtime errors, you should enable script debugging: for this please change the value of the main 'DEBUG' switch, you'll find it in the header of */usr/bin/adblock-update.sh*
|
||||
* to process & store all blocklist sources at once it might helpful to enlarge your temp directory with a swap partition => see [openwrt wiki](https://wiki.openwrt.org/doc/uci/fstab) for further details
|
||||
* add domain white- or blacklist entries to always-allow or -deny certain (sub) domains, by default both lists are located in _/etc/adblock_. Please add one domain per line - ip addresses, wildcards & regex are _not_ allowed (see example below)
|
||||
* enable the backup/restore feature, to restore automatically the latest stable backup of your adblock lists in case of any (partial) processing error (i.e. a single blocklist source is down). Please use an (external) solid partition and _not_ your volatile router temp directory for this
|
||||
* for a scheduled call of the adblock service via _/etc/init.d/adblock start_ add an appropriate crontab entry (see example below)
|
||||
* in case of any script runtime errors, you should enable script debugging: for this please change the 'DEBUG' variable in the header of _/usr/bin/adblock-update.sh_ from '0' to '1' and start this script afterwards (without any parameter)
|
||||
* you could add new blocklist sources on your own via uci config, all you need is a source url and an awk one-liner (see example below)
|
||||
|
||||
## Further adblock config options
|
||||
* usually the adblock autodetection works quite well and no manual config overrides are needed, all options apply to 'global' adblock config section:
|
||||
* usually the adblock autodetection works quite well and no manual config overrides are needed, all options apply to the 'global' config section:
|
||||
* adb\_enabled => main switch to enable/disable adblock service (default: '1', enabled)
|
||||
* adb\_cfgver => config version string (do not change!) - adblock checks this entry and automatically applies the current config, if none or an older revision was found.
|
||||
* adb\_wanif => name of the logical wan interface (default: 'wan')
|
||||
* adb\_cfgversion => config version string (do not change!) - adblock will check this entry during startup
|
||||
* adb\_lanif => name of the logical lan interface (default: 'lan')
|
||||
* adb\_port => port of the adblock uhttpd instance (default: '65535')
|
||||
* adb\_nullport => port of the adblock uhttpd instance (default: '65535')
|
||||
* adb\_nullipv4 => IPv4 blackhole ip address (default: '192.0.2.1')
|
||||
* adb\_nullipv6 => IPv6 blackhole ip address (default: '::ffff:c000:0201')
|
||||
* adb\_forcedns => redirect all DNS queries to local dnsmasq resolver (default: '1', enabled)
|
||||
|
||||
## Examples
|
||||
|
||||
**example cronjob for a regular update of the block lists:**
|
||||
<pre><code>
|
||||
# configuration found in /etc/crontabs/root
|
||||
# start adblock script once a day at 6 a.m.
|
||||
#
|
||||
0 06 * * * /etc/init.d/adblock start
|
||||
</code></pre>
|
||||
|
||||
**example blacklist entry (/etc/adblock/adblock.blacklist):**
|
||||
<pre><code>
|
||||
ads.example.com
|
||||
</code></pre>
|
||||
|
||||
This rule blocks:
|
||||
http://ads.example.com/foo.gif
|
||||
http://server1.ads.example.com/foo.gif
|
||||
https://ads.example.com:8000/
|
||||
|
||||
This rule doesn't block:
|
||||
http://ads.example.com.ua/foo.gif
|
||||
http://example.com/
|
||||
|
||||
**example whitelist entry (/etc/adblock/adblock.whitelist):**
|
||||
<pre><code>
|
||||
analytics.com
|
||||
</code></pre>
|
||||
|
||||
This rule removes _all_ domains from the blocklists with this string in it, i.e.:
|
||||
google-analytics.com
|
||||
ssl.google-analytics.com
|
||||
api.gameanalytics.com
|
||||
photos.daily-deals.analoganalytics.com
|
||||
adblockanalytics.com
|
||||
|
||||
**example to add [rolist+easylist](https://easylist-downloads.adblockplus.org/rolist+easylist.txt) as a new source:**
|
||||
<pre><code>
|
||||
add a new source section in adblock config, take an existing easylist source as a template and change only the source name, the url and the description
|
||||
|
||||
config source 'ruadlist'
|
||||
option enabled '0'
|
||||
option adb_src 'https://easylist-downloads.adblockplus.org/ruadlist+easylist.txt'
|
||||
option adb_src_rset '{FS=\"[|^]\"} \$0 ~/^\|\|([A-Za-z0-9_-]+\.){1,}[A-Za-z]+\^$/{print tolower(\$3)}'
|
||||
option adb_src_desc 'focus on russian ad related domains plus generic easylist additions, weekly updates, approx. 2.000 entries'
|
||||
|
||||
config source 'rolist'
|
||||
option enabled '0'
|
||||
option adb_src 'https://easylist-downloads.adblockplus.org/rolist+easylist.txt'
|
||||
option adb_src_rset '{FS=\"[|^]\"} \$0 ~/^\|\|([A-Za-z0-9_-]+\.){1,}[A-Za-z]+\^$/{print tolower(\$3)}'
|
||||
option adb_src_desc 'focus on romanian ad related domains plus generic easylist additions, weekly updates, approx. 600 entries'
|
||||
</code></pre>
|
||||
|
||||
## Background
|
||||
This adblock package is a dns/dnsmasq based adblock solution for openwrt.
|
||||
Queries to ad/abuse domains are never forwarded and always replied with a local IP address which may be IPv4 or IPv6.
|
||||
For that purpose adblock uses an ip address from the private 'TEST-NET-1' subnet (192.0.2.1 / ::ffff:c000:0201) by default.
|
||||
Furthermore all ad/abuse queries will be filtered by ip(6)tables and redirected to internal adblock pixel server (in PREROUTING chain) or rejected (in FORWARD or OUTPUT chain).
|
||||
All iptables and uhttpd related adblock additions are non-destructive, no hard-coded changes in 'firewall.user', 'uhttpd' config or any other openwrt related config files. There is *no* adblock background daemon running, the (scheduled) start of the adblock service keeps only the adblock lists up-to-date.
|
||||
All iptables and uhttpd related adblock additions are non-destructive, no hard-coded changes in 'firewall.user', 'uhttpd' config or any other openwrt related config files. There is _no_ adblock background daemon running, the (scheduled) start of the adblock service keeps only the adblock lists up-to-date.
|
||||
|
||||
## Support
|
||||
Please join the adblock discussion in this [openwrt forum thread](https://forum.openwrt.org/viewtopic.php?id=59803) or contact me by mail <openwrt@brenken.org>
|
||||
|
||||
## Removal
|
||||
* stop all adblock related services with */etc/init.d/adblock stop*
|
||||
* optional: remove the adblock package (*opkg remove adblock*)
|
||||
* stop all adblock related services with _/etc/init.d/adblock stop_
|
||||
* optional: remove the adblock package (_opkg remove adblock_)
|
||||
|
||||
Have fun!
|
||||
Dirk
|
||||
|
||||
Reference in New Issue
Block a user