So it's great that shorewall has a DOCKER option. It saves docker's
rules on restart, and docker can do it's own iptables thing when a new
docker starts up and ports need to be forwarded, etc. Awesome.
However, the current implementation of DOCKER in shorewall introduces a
huge security flaw. And it's one essentially mentioned on the DOCKER
website about iptables use:
Docker installs two custom iptables chains named DOCKER-USER and
DOCKER, and it ensures that incoming packets are always checked by
these two chains first.
All of Docker's iptables rules are added to the DOCKER chain. Do not
manipulate this chain manually. If you need to add rules which load
before Docker's rules, add them to the DOCKER-USER chain. These rules
are applied before any rules Docker creates automatically.
Rules added to the FORWARD chain -- either manually, or by another
iptables-based firewall -- are evaluated after these chains. This
means that if you expose a port through Docker, this port gets exposed
no matter what rules your firewall has configured. If you want those
rules to apply even when a port gets exposed through Docker, you must
add these rules to the DOCKER-USER chain.
The crux of this problem is, because shorewall adds it's rules to the
FORWARD chain after the jumps to DOCKER-USER and DOCKER chains, any
rules one might have had to drop/reject connections to ports exposed by
docker are NEVER evaluated - as the connection has already been ACCEPTED.
Example:
Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source
destination
1261K 1560M DOCKER-USER all -- * * 0.0.0.0/0
0.0.0.0/0
1261K 1560M DOCKER-ISOLATION-STAGE-1 all -- * *
0.0.0.0/0 0.0.0.0/0
404 464K ACCEPT all -- * br-2cba0ae1535f
0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
3 180 DOCKER all -- * br-2cba0ae1535f
0.0.0.0/0 0.0.0.0/0
404 30528 ACCEPT all -- br-2cba0ae1535f !br-2cba0ae1535f
0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- br-2cba0ae1535f br-2cba0ae1535f
0.0.0.0/0 0.0.0.0/0
596K 775M ACCEPT all -- * br-02873a83aa96
0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
1754 107K DOCKER all -- * br-02873a83aa96
0.0.0.0/0 0.0.0.0/0
663K 785M ACCEPT all -- br-02873a83aa96 !br-02873a83aa96
0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- br-02873a83aa96 br-02873a83aa96
0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- * docker0 0.0.0.0/0
0.0.0.0/0 ctstate RELATED,ESTABLISHED
0 0 DOCKER all -- * docker0 0.0.0.0/0
0.0.0.0/0
0 0 ACCEPT all -- docker0 !docker0 0.0.0.0/0
0.0.0.0/0
0 0 ACCEPT all -- docker0 docker0 0.0.0.0/0
0.0.0.0/0
0 0 ACCEPT all -- * docker_gwbridge
0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
0 0 DOCKER all -- * docker_gwbridge
0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- docker_gwbridge !docker_gwbridge
0.0.0.0/0 0.0.0.0/0
91640 151M eno4_fwd all -- eno4 * 0.0.0.0/0
0.0.0.0/0
89503 74M bond0_fwd all -- bond0 * 0.0.0.0/0
0.0.0.0/0
172K 83M br_fwd all -- br-+ * 0.0.0.0/0
0.0.0.0/0
0 0 dock_frwd all -- docker0 * 0.0.0.0/0
0.0.0.0/0
0 0 DROP all -- * * 0.0.0.0/0
0.0.0.0/0 ADDRTYPE match dst-type BROADCAST
0 0 DROP all -- * * 0.0.0.0/0
0.0.0.0/0 ADDRTYPE match dst-type ANYCAST
0 0 DROP all -- * * 0.0.0.0/0
0.0.0.0/0 ADDRTYPE match dst-type MULTICAST
0 0 LOG all -- * * 0.0.0.0/0
0.0.0.0/0 limit: up to 1/sec burst 10 mode srcip LOG flags
0 level 6 prefix "FORWARD REJECT "
0 0 reject all -- * * 0.0.0.0/0
0.0.0.0/0 [goto]
0 0 DROP all -- docker_gwbridge docker_gwbridge
0.0.0.0/0 0.0.0.0/0
My firewall rules denying connections from eno4 -> docker, or eno4 -> a
bridge (which is also created by docker, I have more than one docker
bridge for service network isolation), are not evaluated until you get
to the last 4 lines above. But that's too late, because:
Chain DOCKER (4 references)
pkts bytes target prot opt in out source
destination
1 60 ACCEPT tcp -- !br-02873a83aa96 br-02873a83aa96
0.0.0.0/0 172.20.0.2 tcp dpt:9050
0 0 ACCEPT tcp -- !br-02873a83aa96 br-02873a83aa96
0.0.0.0/0 172.20.0.2 tcp dpt:9040
20 1200 ACCEPT tcp -- !br-02873a83aa96 br-02873a83aa96
0.0.0.0/0 172.20.0.2 tcp dpt:9030
1639 99999 ACCEPT tcp -- !br-02873a83aa96 br-02873a83aa96
0.0.0.0/0 172.20.0.2 tcp dpt:9001
3 180 ACCEPT tcp -- !br-2cba0ae1535f br-2cba0ae1535f
0.0.0.0/0 172.21.0.2 tcp dpt:3334
So as you can see, if I had wanted to put some kind of firewall rule in
place limiting the source of WHO can access those ports opened by
docker, I could not. Worse, if I wanted to do something as simple as
limiting connections to specific ports to coming from specific
interfaces (eg. eno4 vs bond0), I could not.
The only viable solution I can think of would be to allow for specifying
chain names (eg. rather than inserting shorewall rules into the FORWARD
chain, it would insert into the ${FORWARD_CHAIN} chain from
shorewall.conf).
This could be a docker-specific thing, namely that, if DOCKER=Yes, then
the interface-base forwarding rules (and possibly the
broadcast/multicast drop rules) would be added to the DOCKER-USER chain.
Though it could be more generic such that you could specify all chains
in the config:
FILTER_INPUT_CHAIN=INPUT
FILTER_OUTPUT_CHAIN=OUTPUT
FILTER_FORWARD_CHAIN=FORWARD
And then I could modify the FILTER_FORWARD_CHAIN to be DOCKER-USER -
though the all/all policy rules would have to not go in the
FILTER_FORWARD_CHAIN, or the docker rules would never be reached.
This is a rather large security hole if you use DOCKER=Yes (and docker)
and shorewall.
I tried NOT using DOCKER=yes and just using the docker-proxy stuff (ie.
host binding ports), but docker-proxy has issues and will often enough
not be able to establish a connection correctly. So using docker's
iptables support (which creates the appropriate NAT rules to redirect to
the correct ports) is a much better way to go.
