Using hosts to block ads and malware in Catalina

[bogdanw]

As Apple restricted extensions' capabilities, an alternative approach might more effective. The hosts file can be used to block acces to defined domains, thus preventig every application from accesing it. By using custom lists of domains, ads and malware can be blocked.

GitHub user Dani3lSun wrote a simple and efective script: adblock_hosts.sh Mac OS X adblock shell script using system hosts file (Block ads system wide) https://gist.github.com/Dani3lSun/83ebfa77fbb21a6873ef but it hasn't been updated in a while.

I made minor changes, updating the links, ading NoCoin adblock list and a line to remove the temporary directory created. My version can be found here https://gist.github.com/b0gdanw/2716bc05be8eef49424b105d39370105

Usage:
1. Download https://gist.githubusercontent.com/...46847c0e2a4bc302b099727c230b/adblock_hosts.sh in your home folder
Or open Terminal and run

curl -O https://gist.githubusercontent.com/b0gdanw/2716bc05be8eef49424b105d39370105/raw/fe06ab01309746847c0e2a4bc302b099727c230b/adblock_hosts.sh

2. Change file permissions to executable

chmod +x adblock_hosts.sh

3. Run

sudo ./adblock_hosts.sh

(Alternatively, you can skip step 2 and run sudo bash adblock_hosts.sh)

The script downloads six text files containing the domains to be blocked, merges them into one, renames the original hosts file to hosts_org and copies the new host file to /private/etc/hosts
The original and the hosts file created are also saved in the home folder as hosts_org.txt and hosts_new.txt

To revert to the original hosts file

sudo rm -f /private/etc/hosts; sudo mv /private/etc/hosts_org /private/etc/hosts

and to flush DNS cache

dscacheutil -flushcache

Advantages:
- no resources used
- applies to every application (Safari, Chrome, Mail and any other that connects to the Internet)

Disadvantages:
- it has to be manually updated
- no whitelist or possibility to disable for certain sites

Further reading
About the hosts file https://en.wikipedia.org/wiki/Hosts_(file)
Idea for the script and source for the lists AdAway for Android https://github.com/AdAway/AdAway
Other domain lists:
NoCoin adblock list https://github.com/hoshsadiq/adblock-nocoin-list
StevenBlack/hosts https://github.com/StevenBlack/hosts
Application to manage hosts file:
Gas Mask - simple hosts file manager for macOS https://github.com/2ndalpha/gasmask

 

[Stephen.R]

As Apple restricted extensions' capabilities, an alternative approach might more effective. The hosts file can be used to block acces to defined domains, thus preventig every application from accesing it. By using custom lists of domains, ads and malware can be blocked.

Assuming you mean Safari extensions - there is literally built in support for dedicated content blocking plugins, that work across macOS and iOS (and derivatives like iPadOS).

In contrast to the 'traditional' type of extension that runs JavaScript and would 'cancel' requests to URLs in a black list (or not in a white list), while potentially also reporting every URL you access to a third party, Content Blockers run no javascript, and the plugin itself receives no information about what URLs you visit. The plugin provides a manifest of rules to Safari, and they're pre-compiled so that blocking is efficient and fast.


I've done the hosts file thing, I've done the system proxy thing, and when support came, I did the "proper" Safari plugin using JS to block trackers/ads, they were indeed all better options when the alternative/history was a plugin relying on the likes of SIMBL, and would break with practically every OS update.

But none of them can hold a candle a content blocker. Fast. Resource Efficient. Privacy is enforced. Officially supported, wth a stable API.

 

[bogdanw]

Permissions for AdBlock and Ghostery Lite installed from the AppStore in Catalina Beta 8.

 

[Stephen.R]

None of those is required if the blocking itself uses a content blocker.

And honestly, using Ghostery to protect your privacy is like hiring Michael Jackson to babysit your kids.

 

[tyc0746]

I just use LittleSnitch, so I can block on an app/process/system level.

 

[Stephen.R]

I just use LittleSnitch, so I can block on an app/process/system level.

I may be missing some new feature in LS, but how can that possibly work for blocking unwanted requests in a webpage?

 

[MacGizmo]

I may be missing some new feature in LS, but how can that possibly work for blocking unwanted requests in a webpage?

It doesn't.

 

[bogdanw]

None of those is required if the blocking itself uses a content blocker.
And honestly, using Ghostery to protect your privacy is like hiring Michael Jackson to babysit your kids.

Yes, the content blocker part works without user interaction. But the point of an extension is to allow user interaction (allowing ads on certain sites, adding elements to be blocked) and that comes with those permissions.

If you don't trust them, build your own Ghostery https://github.com/ghostery/GhosterySafari

I just use LittleSnitch, so I can block on an app/process/system level.

I'm testing Little Snitch in Mojave, here is my latest lsrules file for Safari https://github.com/b0gdanw/Little-Snitch/blob/master/Safari_sb_unified_hosts_gambling.lsrules
created with stevenblack_unified_hosts_V1.3.py https://github.com/naveednajam/Little-Snitch---Rule-Groups

 

[allan.nyholm]

The hosts file I use with current release version of macOS and Gas Mask is this MVPS Hosts and it has been quite good.

I appreciate all the other helpful tips. I will look further into this with Catalina once released in full.

 

[bogdanw]

The hosts file I use with current release version of macOS and Gas Mask is this MVPS Hosts and its's been quite good.

MVPS hosts file is included in the script, here are all the sources from my version:

# Host 1
https://raw.githubusercontent.com/AdAway/adaway.github.io/master/hosts.txt
# Host 2
https://hosts-file.net/ad_servers.txt
# Host 3
http://winhelp2002.mvps.org/hosts.txt
# Host 4
"https://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext"
# Host 5
https://www.malwaredomainlist.com/hostslist/hosts.txt
# Host 6
https://someonewhocares.org/hosts/zero/hosts
# Host 7
https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.txt

 

[bogdanw]

For tyc0746 and anyone else interested: AdAway for Safari using Little Snitch lsrules on macOS Mojave (10.14) and macOS Catalina (10.15) https://github.com/b0gdanw/AdAwayForSafari
The script SSLforApache.sh enables SSL in Apache for localhost, required to host the lsrules for Little Snitch
The script AdAwayForSafari.sh generates the lsrules file based on AdAway hosts
RestoreApacheConfig.sh restores Apache's default configuration, removing the SSL certificate
Tested on macOS Mojave (10.14.6) and macOS Catalina (10.15.1). Usage: download SSLforApache.sh and AdAwayForSafari.sh, drag&drop into Terminal SSLforApache.sh then AdAwayForSafari.sh.