You might have set up some great rules in your iptables to act as a kill switch, only to find out that on installing Docker and running some containers they completely override your rules.
Fortunately the fix is easy. Let me show you how.
1. Required Reading
I’m going to apply the fix to my setup, if you are fairly advanced you can skip to the next section for the fix, if you have trouble come back here and read the articles at the following links.
- The very basics of securing your linux system with iptables, more here.
- How to create a VPN kill switch for your linux system, more here.
2. System iptables Overview
A brief overview of what your system iptables should look like before we install Docker, and apply the Docker kill switch.
Systemd Service Unit Script
If you have followed the above articles you will have the following bash script saved at ‘/usr/local/bin/vpnkillswitch_granular.sh‘ and you may have added more rules to allow ports and services for your needs.
#!/bin/bash
#Flush all chains, delete custom chains.
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
#Block all traffic as default.
iptables --policy INPUT DROP
iptables --policy FORWARD DROP
iptables --policy OUTPUT DROP
#Create a custom chain to contain rules to allow related and established traffic.
iptables -N RELATED_AND_ESTABLISHED
iptables -A RELATED_AND_ESTABLISHED -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
#Divert input traffic to the RELATED_AND_ESTABLISHED chain.
iptables -A INPUT -j RELATED_AND_ESTABLISHED
#iptables -A OUTPUT -d 10.29.0.1 -j ACCEPT
#Let's allow the loopback device.
iptables -A OUTPUT -o lo -j ACCEPT
iptables -A RELATED_AND_ESTABLISHED -i lo -j ACCEPT
#iptables -A OUTPUT -o tun+ -p icmp -j ACCEPT
#If you want to, you can also allow access to your local network.
#The most common local network is class C, 192.168.0.0/16
iptables -A OUTPUT -d 10.0.0.0/8 -j ACCEPT
#You must check what local network class you car connected to.
#Adjust for Class C, 192.168.0.0/16 Class B, 172.16.0.0/12 and class A 10.0.0.0/8
#let's create a chain named OUTPUT_VPN_KILL_SWITCH to hold our VPN rules
iptables -N OUTPUT_VPN_KILL_SWITCH
#iptables --policy OUTPUT_VPN_KILL_SWITCH DROP
#let's pop a reroute to our vpn rules at the top of the OUTPUT chain using -I insert
iptables -A OUTPUT_VPN_KILL_SWITCH -p udp -m udp --dport 1194 -j ACCEPT
#iptables -A OUTPUT_VPN_KILL_SWITCH -o tun+ -j ACCEPT
#iptables -A OUTPUT_VPN_KILL_SWITCH -d 10.17.0.1 -j ACCEPT
#port 443 https
iptables -A OUTPUT_VPN_KILL_SWITCH -o tun+ -p tcp --dport 443 -j ACCEPT
#port 80, used by apt-get for example.
iptables -A OUTPUT_VPN_KILL_SWITCH -o tun+ -p tcp --dport 80 -j ACCEPT
#port 123 , NTP , network time sync
iptables -A OUTPUT_VPN_KILL_SWITCH -o tun+ -p udp --dport 123 -j ACCEPT
#port 22, used by ssh
iptables -A OUTPUT_VPN_KILL_SWITCH -o tun+ -p tcp --dport 22 -j ACCEPT
#port 53, used by github clone
iptables -A OUTPUT_VPN_KILL_SWITCH -o tun+ -p tcp --dport 53 -j ACCEPT
#allow ping
iptables -A OUTPUT -o tun+ -p icmp -j ACCEPT
#port 993, enable for IMAPS
#iptables -A OUTPUT_VPN_KILL_SWITCH -o tun+ -p tcp --dport 993 -j ACCEPT
#port 465 TLS messages, SMTP
#iptables -A OUTPUT_VPN_KILL_SWITCH -o tun+ -p tcp --dport 465 -j ACCEPT
iptables -I OUTPUT -j OUTPUT_VPN_KILL_SWITCH
iptables State
On system start, your systemd service script will be run and create the following iptables.
[email protected]:/usr/local/bin# iptables -L -v
Chain INPUT (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
1 69 RELATED_AND_ESTABLISHED all -- any any anywhere anywhere
Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
1 69 OUTPUT_VPN_KILL_SWITCH all -- any any anywhere anywhere
0 0 ACCEPT all -- any lo anywhere anywhere
0 0 ACCEPT all -- any any anywhere 10.0.0.0/8
0 0 ACCEPT icmp -- any tun+ anywhere anywhere
Chain OUTPUT_VPN_KILL_SWITCH (1 references)
pkts bytes target prot opt in out source destination
1 69 ACCEPT udp -- any any anywhere anywhere udp dpt:openvpn
0 0 ACCEPT tcp -- any tun+ anywhere anywhere tcp dpt:https
0 0 ACCEPT tcp -- any tun+ anywhere anywhere tcp dpt:http
0 0 ACCEPT tcp -- any tun+ anywhere anywhere tcp dpt:ssh
0 0 ACCEPT tcp -- any tun+ anywhere anywhere tcp dpt:domain
Chain RELATED_AND_ESTABLISHED (1 references)
pkts bytes target prot opt in out source destination
1 69 ACCEPT all -- any any anywhere anywhere ctstate RELATED,ESTABLISHED
1 180 ACCEPT all -- lo any anywhere anywhere 3. iptable State After Docker Install
After installing Docker, Docker will add its own iptables chains and rules.
Your modified iptables will look like this.
[email protected]:/home/ubuntu# iptables -L -v
Chain INPUT (policy DROP 21 packets, 2224 bytes)
pkts bytes target prot opt in out source destination
21 2224 RELATED_AND_ESTABLISHED all -- any any anywhere anywhere
Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 DOCKER-USER all -- any any anywhere anywhere
0 0 DOCKER-ISOLATION-STAGE-1 all -- any any anywhere anywhere
0 0 ACCEPT all -- any docker0 anywhere anywhere ctstate RELATED,ESTABLISHED
0 0 DOCKER all -- any docker0 anywhere anywhere
0 0 ACCEPT all -- docker0 !docker0 anywhere anywhere
0 0 ACCEPT all -- docker0 docker0 anywhere anywhere
Chain OUTPUT (policy DROP 299 packets, 20946 bytes)
pkts bytes target prot opt in out source destination
318 22704 OUTPUT_VPN_KILL_SWITCH all -- any any anywhere anywhere
22 2264 ACCEPT all -- any lo anywhere anywhere
0 0 ACCEPT all -- any any anywhere 10.0.0.0/8
0 0 ACCEPT icmp -- any tun+ anywhere anywhere
Chain DOCKER (1 references)
pkts bytes target prot opt in out source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
pkts bytes target prot opt in out source destination
0 0 DOCKER-ISOLATION-STAGE-2 all -- docker0 !docker0 anywhere anywhere
0 0 RETURN all -- any any anywhere anywhere
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
pkts bytes target prot opt in out source destination
0 0 DROP all -- any docker0 anywhere anywhere
0 0 RETURN all -- any any anywhere anywhere
Chain DOCKER-USER (1 references)
pkts bytes target prot opt in out source destination
0 0 RETURN all -- any any anywhere anywhere
Chain OUTPUT_VPN_KILL_SWITCH (1 references)
pkts bytes target prot opt in out source destination
0 0 ACCEPT udp -- any any anywhere anywhere udp dpt:openvpn
0 0 ACCEPT tcp -- any tun+ anywhere anywhere tcp dpt:https
0 0 ACCEPT tcp -- any tun+ anywhere anywhere tcp dpt:http
0 0 ACCEPT tcp -- any tun+ anywhere anywhere tcp dpt:ssh
0 0 ACCEPT tcp -- any tun+ anywhere anywhere tcp dpt:domain
Chain RELATED_AND_ESTABLISHED (1 references)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all -- any any anywhere anywhere ctstate RELATED,ESTABLISHED
1 180 ACCEPT all -- lo any anywhere anywhere
I know this looks complicated, but we only need to pay attention to the ‘DOCKER-USER‘ chain.
The ‘DOCKER-USER‘ chain rules are run on network traffic before the traffic is forwarded to whatever other rules docker creates. Adding a few simple rules here will apply a VPN kill switch to all your docker containers.
4. Docker VPN Kill Switch Solution
For the most basic Docker VPN kill switch you only need three iptables rules.
Order is extremely important, so I am going to use the ‘-I‘ option with a rule positional argument.
Rule 1
Here we insert ‘-I‘ a rule in to DOCKER-USER at position 1 at the top of the chain.
The rule is applied to an output interface ‘-o‘, and we set the output interface to our vpn interface ‘tun+‘. The rule jumps ‘-j‘ all traffic to an ACCEPT state.
iptables -I DOCKER-USER 1 -o tun+ -j ACCEPTRule 2
In position 2 of chain DOCKER-USER we state that for the input interface ‘-i‘ named ‘tun+‘, our VPN interface, we will only jump traffic to an ACCEPT state if it is in response to user initiated outward traffic.
iptables -I DOCKER-USER 2 -i tun+ -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPTRule 3
In 3rd place in chain DOCKER-USER we drop absolutely everything else and jump ‘-j‘ all traffic to the DROP state.
iptables -I DOCKER-USER 3 -j DROPThat is it! Your docker containers will be denied network access when your VPN is stopped.
You can if you like be more granular and replace rule 1 with a series of port specific output rules as we have down in the other articles for the general system.
5. Docker Kill Switch iptables State
Here is what your iptables should look like after applying kill switch rules to ‘DOCKER-USER‘.
[email protected]:/home/ubuntu# iptables -L -v
Chain INPUT (policy DROP 24 packets, 2443 bytes)
pkts bytes target prot opt in out source destination
24 2443 RELATED_AND_ESTABLISHED all -- any any anywhere anywhere
Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 DOCKER-USER all -- any any anywhere anywhere
0 0 DOCKER-ISOLATION-STAGE-1 all -- any any anywhere anywhere
0 0 ACCEPT all -- any docker0 anywhere anywhere ctstate RELATED,ESTABLISHED
0 0 DOCKER all -- any docker0 anywhere anywhere
0 0 ACCEPT all -- docker0 !docker0 anywhere anywhere
0 0 ACCEPT all -- docker0 docker0 anywhere anywhere
Chain OUTPUT (policy DROP 1931 packets, 121K bytes)
pkts bytes target prot opt in out source destination
1953 123K OUTPUT_VPN_KILL_SWITCH all -- any any anywhere anywhere
25 2483 ACCEPT all -- any lo anywhere anywhere
0 0 ACCEPT all -- any any anywhere 10.0.0.0/8
0 0 ACCEPT icmp -- any tun+ anywhere anywhere
Chain DOCKER (1 references)
pkts bytes target prot opt in out source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
pkts bytes target prot opt in out source destination
0 0 DOCKER-ISOLATION-STAGE-2 all -- docker0 !docker0 anywhere anywhere
0 0 RETURN all -- any any anywhere anywhere
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
pkts bytes target prot opt in out source destination
0 0 DROP all -- any docker0 anywhere anywhere
0 0 RETURN all -- any any anywhere anywhere
Chain DOCKER-USER (1 references)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all -- any tun+ anywhere anywhere
0 0 ACCEPT all -- tun+ any anywhere anywhere ctstate RELATED,ESTABLISHED
0 0 DROP all -- any any anywhere anywhere
0 0 RETURN all -- any any anywhere anywhere
Chain OUTPUT_VPN_KILL_SWITCH (1 references)
pkts bytes target prot opt in out source destination
0 0 ACCEPT udp -- any any anywhere anywhere udp dpt:openvpn
0 0 ACCEPT tcp -- any tun+ anywhere anywhere tcp dpt:https
0 0 ACCEPT tcp -- any tun+ anywhere anywhere tcp dpt:http
0 0 ACCEPT tcp -- any tun+ anywhere anywhere tcp dpt:ssh
0 0 ACCEPT tcp -- any tun+ anywhere anywhere tcp dpt:domain
Chain RELATED_AND_ESTABLISHED (1 references)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all -- any any anywhere anywhere ctstate RELATED,ESTABLISHED
1 180 ACCEPT all -- lo any anywhere anywhere 6. Permanent Persistent iptables Solution
Unfortunately on restart your new iptable rules will not persist.
We need to apply these rules to your iptables once docker has added its iptables rules, and we will do this with a new Systemd unit script.
Step 1 – Create Unit Script
Head over to the ‘/usr/local/bin‘ directory and create a script named ‘dockerkillswitch.sh‘ and add the following content.
#!/bin/bash
iptables -I DOCKER-USER 1 -o tun+ -j ACCEPT
iptables -I DOCKER-USER 2 -i tun+ -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
iptables -I DOCKER-USER 3 -j DROPMake sure you set the appropriate executable permissions on this script file,
[email protected]:/usr/local/bin# chmod 755 dockerkillswitch.sh Step 2 – Create The systemd Unit Service File
Head over to the ‘/etc/systemd/system‘ directory and create a file named ‘dockerkillswitch.service‘
[Unit]
Description=Apply Docker Kill Switch Rules To DOCKER-USER chain.
After=docker.service
BindsTo=docker.service
[Service]
Type=oneshot
ExecStart=/usr/local/bin/dockerkillswitch.sh
[Install]
WantedBy=multi-user.targetOur service file has a Description, we state that we want our script to run after the docker service (After=docker.service) and we set BindsTo=docker.service so that our script only runs if the docker.service is in an active state.
We set our service Type=oneshot, which lets systemd know we want a short-lived process and that systemd should wait for our process to exit before continuing with other units. Ideally we want our kill switch in place.
We state that once the rest of the conditions specified in this service file are complete, we run our script, by setting ExecStart=/usr/local/bin/dockerkillswitch.sh
We want our script to run when the system state has most other services also running, and will accept logins from multiple users before the GUI being in a ready state. This is done by setting WantedBy=multi-user.target
If you are curious you can inspect the docker.system unit file at the following location ‘/lib/systemd/system/docker.service‘
Step 3 -Enable Your dockerkillswitch service
One line to do this,
[email protected]:/etc/systemd/system# systemctl enable dockerkillswitch
Created symlink /etc/systemd/system/multi-user.target.wants/dockerkillswitch.service → /etc/systemd/system/dockerkillswitch.service.
Step 4 – Restart
Restart your system and enjoy your VPN kill switch across you native system, and all of your docker containers.
7. Further Notes
VPN Connection Tip
I have noticed that on linux mint occasionally the VPN fails to start.
In that case I just turn off all the networks,
then back on again. So expect this part to be a little fiddly, this isn’t the fault of the iptables work we have carried out, so don’t worry.
Here is a docker container, and my local machine pinging through the VPN.
When I turn off the VPN, the pings stop, and I can’t access the internet using any other applications.
The VPN kill switch works.
VPN OPENVPN Persistence Tip
If you have a VPN connection dropping there may be multiple reasons why, for example my issue is described here:
https://rexbytes.com/2023/01/29/portable-laptop-pi-hole-and-unbound-dns-resolver/#unboundresolving
It’s difficult to troubleshoot, but if all else fails you might like to try the following:
There are a couple of settings that are not visible in connection manager and if you want a permanent always on VPN connection you are going to have to set them manually. First you need to identify your vpn connection by using the following command.
[email protected]:~# nmcli connection show
NAME UUID TYPE DEVICE
your.securevpn.net.udp 4d42e4c7-b805-4df1-a05d-6beba1763e73 vpn wlp4s0
docker0 aa132a0e-cdf0-4c21-aef5-41a46658e118 bridge docker0
tun0 4cda97bc-14bc-4064-887b-7868919eca8b tun tun0
VirginMedia_5G 1c44e9ab-af6e-4b21-a3f5-114502909f83 wifi wlp4s0
My VPN connection is called ‘your.securevpn.net.udp‘
To view all the available settings for a network connection you need to use the following command.
nmcli connection show your.securevpn.net.udp Or if you want to check the value of an existing setting,
nmcli connection show your.securevpn.net.udp | grep vpn.persistentThere are quite a few settings so I won’t list them all here.
We are interested in setting the following two values, and their commands are…
nmcli connection modify your.securevpn.net.udp connection.autoconnect-retries 0
nmcli connection modify your.securevpn.net.udp vpn.persistent yesThis will keep your VPN connection on as best as the machine can.
To set them back to the default values,
nmcli connection modify your.securevpn.net.udp connection.autoconnect-retries -1
nmcli connection modify your.securevpn.net.udp vpn.persistent noAlternative Docker Kill Switch Script
I like to keep all my rules separate and organised, a lot like my email inbox rules.
So, the following script creates a chain named ‘DOCKER_VPN_KILL_SWITCH‘ to hold the three rules we already covered. Note that the commands to add the rules now target the ‘DOCKER_VPN_KILL_SWITCH‘ chain.
#!/bin/bash
iptables -N DOCKER_VPN_KILL_SWITCH
iptables -I DOCKER_VPN_KILL_SWITCH 1 -o tun+ -j ACCEPT
iptables -I DOCKER_VPN_KILL_SWITCH 2 -i tun+ -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
iptables -I DOCKER_VPN_KILL_SWITCH 3 -j DROP
iptables -I DOCKER-USER 1 -j DOCKER_VPN_KILL_SWITCH
The ‘DOCKER-USER‘ chain still needs to process theses rules, which is why we insert ‘-I‘ a redirect to our ‘DOCKER_VPN_KILL_SWITCH‘ at the very top of the ‘DOCKER-USER‘ chain.
On substituting with this alternative script your new iptables will look like this,
[email protected]:/usr/local/bin# iptables -L -v
Chain INPUT (policy DROP 26 packets, 2573 bytes)
pkts bytes target prot opt in out source destination
2334 2533K RELATED_AND_ESTABLISHED all -- any any anywhere anywhere
Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
68 5712 DOCKER-USER all -- any any anywhere anywhere
0 0 DOCKER-ISOLATION-STAGE-1 all -- any any anywhere anywhere
0 0 ACCEPT all -- any docker0 anywhere anywhere ctstate RELATED,ESTABLISHED
0 0 DOCKER all -- any docker0 anywhere anywhere
0 0 ACCEPT all -- docker0 !docker0 anywhere anywhere
0 0 ACCEPT all -- docker0 docker0 anywhere anywhere
Chain OUTPUT (policy DROP 1066 packets, 71223 bytes)
pkts bytes target prot opt in out source destination
3094 253K OUTPUT_VPN_KILL_SWITCH all -- any any anywhere anywhere
25 2340 ACCEPT all -- any lo anywhere anywhere
55 4068 ACCEPT all -- any any anywhere 10.0.0.0/8
8 672 ACCEPT icmp -- any tun+ anywhere anywhere
Chain DOCKER (1 references)
pkts bytes target prot opt in out source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
pkts bytes target prot opt in out source destination
0 0 DOCKER-ISOLATION-STAGE-2 all -- docker0 !docker0 anywhere anywhere
0 0 RETURN all -- any any anywhere anywhere
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
pkts bytes target prot opt in out source destination
0 0 DROP all -- any docker0 anywhere anywhere
0 0 RETURN all -- any any anywhere anywhere
Chain DOCKER-USER (1 references)
pkts bytes target prot opt in out source destination
68 5712 DOCKER_VPN_KILL_SWITCH all -- any any anywhere anywhere
0 0 RETURN all -- any any anywhere anywhere
Chain DOCKER_VPN_KILL_SWITCH (1 references)
pkts bytes target prot opt in out source destination
21 1764 ACCEPT all -- any tun+ anywhere anywhere
21 1764 ACCEPT all -- tun+ any anywhere anywhere ctstate RELATED,ESTABLISHED
26 2184 DROP all -- any any anywhere anywhere
Chain OUTPUT_VPN_KILL_SWITCH (1 references)
pkts bytes target prot opt in out source destination
1122 125K ACCEPT udp -- any any anywhere anywhere udp dpt:openvpn
107 11170 ACCEPT tcp -- any tun+ anywhere anywhere tcp dpt:https
714 39741 ACCEPT tcp -- any tun+ anywhere anywhere tcp dpt:http
0 0 ACCEPT tcp -- any tun+ anywhere anywhere tcp dpt:ssh
0 0 ACCEPT tcp -- any tun+ anywhere anywhere tcp dpt:domain
Chain RELATED_AND_ESTABLISHED (1 references)
pkts bytes target prot opt in out source destination
2309 2531K ACCEPT all -- any any anywhere anywhere ctstate RELATED,ESTABLISHED
1 180 ACCEPT all -- lo any anywhere anywhere
You can see the new ‘DOCKER_VPN_KILL_SWITCH‘ containing the rules, and the target redirect at the top of the ‘DOCKER_USER‘ chain.
You can now add more granular firewall rules in chain ‘DOCKER_VPN_KILL_SWITCH‘ and have an overall better organised iptables.
Now you have read all the above, I thought I would let you know i’ve written a python package to do this all for you.
[…] VPN Kill Switch With Docker […]