On 8/4/2020 2:24 PM, Preston A. Elder wrote:
> So it's great that shorewall has a DOCKER option. It saves docker's
> rules on restart, and docker can do it's own iptables thing when a new
> docker starts up and ports need to be forwarded, etc. Awesome.
>
> However, the current implementation of DOCKER in shorewall introduces a
> huge security flaw. And it's one essentially mentioned on the DOCKER
> website about iptables use:
>
>> Docker installs two custom iptables chains named DOCKER-USER and
>> DOCKER, and it ensures that incoming packets are always checked by
>> these two chains first.
>
>> All of Docker's iptables rules are added to the DOCKER chain. Do not
>> manipulate this chain manually. If you need to add rules which load
>> before Docker's rules, add them to the DOCKER-USER chain. These rules
>> are applied before any rules Docker creates automatically.
>
>> Rules added to the FORWARD chain -- either manually, or by another
>> iptables-based firewall -- are evaluated after these chains. This
>> means that if you expose a port through Docker, this port gets exposed
>> no matter what rules your firewall has configured. If you want those
>> rules to apply even when a port gets exposed through Docker, you must
>> add these rules to the DOCKER-USER chain.
>
> The crux of this problem is, because shorewall adds it's rules to the
> FORWARD chain after the jumps to DOCKER-USER and DOCKER chains, any
> rules one might have had to drop/reject connections to ports exposed by
> docker are NEVER evaluated - as the connection has already been ACCEPTED.
>
> Example:
>
>> Chain FORWARD (policy DROP 0 packets, 0 bytes)
>> pkts bytes target prot opt in out source
>> destination
>> 1261K 1560M DOCKER-USER all -- * * 0.0.0.0/0
>> 0.0.0.0/0
>> 1261K 1560M DOCKER-ISOLATION-STAGE-1 all -- * *
>> 0.0.0.0/0 0.0.0.0/0
>> 404 464K ACCEPT all -- * br-2cba0ae1535f
>> 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
>> 3 180 DOCKER all -- * br-2cba0ae1535f
>> 0.0.0.0/0 0.0.0.0/0
>> 404 30528 ACCEPT all -- br-2cba0ae1535f !br-2cba0ae1535f
>> 0.0.0.0/0 0.0.0.0/0
>> 0 0 ACCEPT all -- br-2cba0ae1535f br-2cba0ae1535f
>> 0.0.0.0/0 0.0.0.0/0
>> 596K 775M ACCEPT all -- * br-02873a83aa96
>> 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
>> 1754 107K DOCKER all -- * br-02873a83aa96
>> 0.0.0.0/0 0.0.0.0/0
>> 663K 785M ACCEPT all -- br-02873a83aa96 !br-02873a83aa96
>> 0.0.0.0/0 0.0.0.0/0
>> 0 0 ACCEPT all -- br-02873a83aa96 br-02873a83aa96
>> 0.0.0.0/0 0.0.0.0/0
>> 0 0 ACCEPT all -- * docker0 0.0.0.0/0
>> 0.0.0.0/0 ctstate RELATED,ESTABLISHED
>> 0 0 DOCKER all -- * docker0 0.0.0.0/0
>> 0.0.0.0/0
>> 0 0 ACCEPT all -- docker0 !docker0 0.0.0.0/0
>> 0.0.0.0/0
>> 0 0 ACCEPT all -- docker0 docker0 0.0.0.0/0
>> 0.0.0.0/0
>> 0 0 ACCEPT all -- * docker_gwbridge
>> 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
>> 0 0 DOCKER all -- * docker_gwbridge
>> 0.0.0.0/0 0.0.0.0/0
>> 0 0 ACCEPT all -- docker_gwbridge !docker_gwbridge
>> 0.0.0.0/0 0.0.0.0/0
>> 91640 151M eno4_fwd all -- eno4 * 0.0.0.0/0
>> 0.0.0.0/0
>> 89503 74M bond0_fwd all -- bond0 * 0.0.0.0/0
>> 0.0.0.0/0
>> 172K 83M br_fwd all -- br-+ * 0.0.0.0/0
>> 0.0.0.0/0
>> 0 0 dock_frwd all -- docker0 * 0.0.0.0/0
>> 0.0.0.0/0
>> 0 0 DROP all -- * * 0.0.0.0/0
>> 0.0.0.0/0 ADDRTYPE match dst-type BROADCAST
>> 0 0 DROP all -- * * 0.0.0.0/0
>> 0.0.0.0/0 ADDRTYPE match dst-type ANYCAST
>> 0 0 DROP all -- * * 0.0.0.0/0
>> 0.0.0.0/0 ADDRTYPE match dst-type MULTICAST
>> 0 0 LOG all -- * * 0.0.0.0/0
>> 0.0.0.0/0 limit: up to 1/sec burst 10 mode srcip LOG flags
>> 0 level 6 prefix "FORWARD REJECT "
>> 0 0 reject all -- * * 0.0.0.0/0
>> 0.0.0.0/0 [goto]
>> 0 0 DROP all -- docker_gwbridge docker_gwbridge
>> 0.0.0.0/0 0.0.0.0/0
>
> My firewall rules denying connections from eno4 -> docker, or eno4 -> a
> bridge (which is also created by docker, I have more than one docker
> bridge for service network isolation), are not evaluated until you get
> to the last 4 lines above. But that's too late, because:
>
>
>> Chain DOCKER (4 references)
>> pkts bytes target prot opt in out source
>> destination
>> 1 60 ACCEPT tcp -- !br-02873a83aa96 br-02873a83aa96
>> 0.0.0.0/0 172.20.0.2 tcp dpt:9050
>> 0 0 ACCEPT tcp -- !br-02873a83aa96 br-02873a83aa96
>> 0.0.0.0/0 172.20.0.2 tcp dpt:9040
>> 20 1200 ACCEPT tcp -- !br-02873a83aa96 br-02873a83aa96
>> 0.0.0.0/0 172.20.0.2 tcp dpt:9030
>> 1639 99999 ACCEPT tcp -- !br-02873a83aa96 br-02873a83aa96
>> 0.0.0.0/0 172.20.0.2 tcp dpt:9001
>> 3 180 ACCEPT tcp -- !br-2cba0ae1535f br-2cba0ae1535f
>> 0.0.0.0/0 172.21.0.2 tcp dpt:3334
>
>
> So as you can see, if I had wanted to put some kind of firewall rule in
> place limiting the source of WHO can access those ports opened by
> docker, I could not. Worse, if I wanted to do something as simple as
> limiting connections to specific ports to coming from specific
> interfaces (eg. eno4 vs bond0), I could not.
>
> The only viable solution I can think of would be to allow for specifying
> chain names (eg. rather than inserting shorewall rules into the FORWARD
> chain, it would insert into the ${FORWARD_CHAIN} chain from
> shorewall.conf).
>
> This could be a docker-specific thing, namely that, if DOCKER=Yes, then
> the interface-base forwarding rules (and possibly the
> broadcast/multicast drop rules) would be added to the DOCKER-USER chain.
> Though it could be more generic such that you could specify all chains
> in the config:
> FILTER_INPUT_CHAIN=INPUT
> FILTER_OUTPUT_CHAIN=OUTPUT
> FILTER_FORWARD_CHAIN=FORWARD
>
> And then I could modify the FILTER_FORWARD_CHAIN to be DOCKER-USER -
> though the all/all policy rules would have to not go in the
> FILTER_FORWARD_CHAIN, or the docker rules would never be reached.
>
> This is a rather large security hole if you use DOCKER=Yes (and docker)
> and shorewall.
>
> I tried NOT using DOCKER=yes and just using the docker-proxy stuff (ie.
> host binding ports), but docker-proxy has issues and will often enough
> not be able to establish a connection correctly. So using docker's
> iptables support (which creates the appropriate NAT rules to redirect to
> the correct ports) is a much better way to go.
>
An other way to this would be to use Shorewall provided that you use
the '--iptables=false' (1) option.
1)
https://docs.docker.com/engine/reference/commandline/dockerd/#run-multiple-daemons
--
Matt Darfeuille <m...@shorewall.org>
Shorewall Project Committee, one of four core members
https://sourceforge.net/p/shorewall/mailman/message/36596609/
https://shorewall.org
_______________________________________________
Shorewall-users mailing list
Shorewall-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/shorewall-users
