On 8/4/2020 5:24 AM, Preston A. Elder wrote:
> So it's great that shorewall has a DOCKER option. It saves docker's
> rules on restart, and docker can do it's own iptables thing when a new
> docker starts up and ports need to be forwarded, etc. Awesome.
>
> However, the current implementation of DOCKER in shorewall introduces a
> huge security flaw. And it's one essentially mentioned on the DOCKER
> website about iptables use:
>
>> Docker installs two custom iptables chains named DOCKER-USER and
>> DOCKER, and it ensures that incoming packets are always checked by
>> these two chains first.
>
>> All of Docker's iptables rules are added to the DOCKER chain. Do not
>> manipulate this chain manually. If you need to add rules which load
>> before Docker's rules, add them to the DOCKER-USER chain. These rules
>> are applied before any rules Docker creates automatically.
>
>> Rules added to the FORWARD chain -- either manually, or by another
>> iptables-based firewall -- are evaluated after these chains. This
>> means that if you expose a port through Docker, this port gets exposed
>> no matter what rules your firewall has configured. If you want those
>> rules to apply even when a port gets exposed through Docker, you must
>> add these rules to the DOCKER-USER chain.
>
> The crux of this problem is, because shorewall adds it's rules to the
> FORWARD chain after the jumps to DOCKER-USER and DOCKER chains, any
> rules one might have had to drop/reject connections to ports exposed by
> docker are NEVER evaluated - as the connection has already been ACCEPTED.
>
> Example:
>
>> Chain FORWARD (policy DROP 0 packets, 0 bytes)
>> pkts bytes target prot opt in out source
>> destination
>> 1261K 1560M DOCKER-USER all -- * * 0.0.0.0/0
>> 0.0.0.0/0
>> 1261K 1560M DOCKER-ISOLATION-STAGE-1 all -- * *
>> 0.0.0.0/0 0.0.0.0/0
>> 404 464K ACCEPT all -- * br-2cba0ae1535f
>> 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
>> 3 180 DOCKER all -- * br-2cba0ae1535f
>> 0.0.0.0/0 0.0.0.0/0
>> 404 30528 ACCEPT all -- br-2cba0ae1535f !br-2cba0ae1535f
>> 0.0.0.0/0 0.0.0.0/0
>> 0 0 ACCEPT all -- br-2cba0ae1535f br-2cba0ae1535f
>> 0.0.0.0/0 0.0.0.0/0
>> 596K 775M ACCEPT all -- * br-02873a83aa96
>> 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
>> 1754 107K DOCKER all -- * br-02873a83aa96
>> 0.0.0.0/0 0.0.0.0/0
>> 663K 785M ACCEPT all -- br-02873a83aa96 !br-02873a83aa96
>> 0.0.0.0/0 0.0.0.0/0
>> 0 0 ACCEPT all -- br-02873a83aa96 br-02873a83aa96
>> 0.0.0.0/0 0.0.0.0/0
>> 0 0 ACCEPT all -- * docker0 0.0.0.0/0
>> 0.0.0.0/0 ctstate RELATED,ESTABLISHED
>> 0 0 DOCKER all -- * docker0 0.0.0.0/0
>> 0.0.0.0/0
>> 0 0 ACCEPT all -- docker0 !docker0 0.0.0.0/0
>> 0.0.0.0/0
>> 0 0 ACCEPT all -- docker0 docker0 0.0.0.0/0
>> 0.0.0.0/0
>> 0 0 ACCEPT all -- * docker_gwbridge
>> 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
>> 0 0 DOCKER all -- * docker_gwbridge
>> 0.0.0.0/0 0.0.0.0/0
>> 0 0 ACCEPT all -- docker_gwbridge !docker_gwbridge
>> 0.0.0.0/0 0.0.0.0/0
>> 91640 151M eno4_fwd all -- eno4 * 0.0.0.0/0
>> 0.0.0.0/0
>> 89503 74M bond0_fwd all -- bond0 * 0.0.0.0/0
>> 0.0.0.0/0
>> 172K 83M br_fwd all -- br-+ * 0.0.0.0/0
>> 0.0.0.0/0
>> 0 0 dock_frwd all -- docker0 * 0.0.0.0/0
>> 0.0.0.0/0
>> 0 0 DROP all -- * * 0.0.0.0/0
>> 0.0.0.0/0 ADDRTYPE match dst-type BROADCAST
>> 0 0 DROP all -- * * 0.0.0.0/0
>> 0.0.0.0/0 ADDRTYPE match dst-type ANYCAST
>> 0 0 DROP all -- * * 0.0.0.0/0
>> 0.0.0.0/0 ADDRTYPE match dst-type MULTICAST
>> 0 0 LOG all -- * * 0.0.0.0/0
>> 0.0.0.0/0 limit: up to 1/sec burst 10 mode srcip LOG flags
>> 0 level 6 prefix "FORWARD REJECT "
>> 0 0 reject all -- * * 0.0.0.0/0
>> 0.0.0.0/0 [goto]
>> 0 0 DROP all -- docker_gwbridge docker_gwbridge
>> 0.0.0.0/0 0.0.0.0/0
>
> My firewall rules denying connections from eno4 -> docker, or eno4 -> a
> bridge (which is also created by docker, I have more than one docker
> bridge for service network isolation), are not evaluated until you get
> to the last 4 lines above. But that's too late, because:
>
>
>> Chain DOCKER (4 references)
>> pkts bytes target prot opt in out source
>> destination
>> 1 60 ACCEPT tcp -- !br-02873a83aa96 br-02873a83aa96
>> 0.0.0.0/0 172.20.0.2 tcp dpt:9050
>> 0 0 ACCEPT tcp -- !br-02873a83aa96 br-02873a83aa96
>> 0.0.0.0/0 172.20.0.2 tcp dpt:9040
>> 20 1200 ACCEPT tcp -- !br-02873a83aa96 br-02873a83aa96
>> 0.0.0.0/0 172.20.0.2 tcp dpt:9030
>> 1639 99999 ACCEPT tcp -- !br-02873a83aa96 br-02873a83aa96
>> 0.0.0.0/0 172.20.0.2 tcp dpt:9001
>> 3 180 ACCEPT tcp -- !br-2cba0ae1535f br-2cba0ae1535f
>> 0.0.0.0/0 172.21.0.2 tcp dpt:3334
>
>
> So as you can see, if I had wanted to put some kind of firewall rule in
> place limiting the source of WHO can access those ports opened by
> docker, I could not. Worse, if I wanted to do something as simple as
> limiting connections to specific ports to coming from specific
> interfaces (eg. eno4 vs bond0), I could not.
>
> The only viable solution I can think of would be to allow for specifying
> chain names (eg. rather than inserting shorewall rules into the FORWARD
> chain, it would insert into the ${FORWARD_CHAIN} chain from
> shorewall.conf).
>
> This could be a docker-specific thing, namely that, if DOCKER=Yes, then
> the interface-base forwarding rules (and possibly the
> broadcast/multicast drop rules) would be added to the DOCKER-USER chain.
> Though it could be more generic such that you could specify all chains
> in the config:
> FILTER_INPUT_CHAIN=INPUT
> FILTER_OUTPUT_CHAIN=OUTPUT
> FILTER_FORWARD_CHAIN=FORWAR>
> And then I could modify the FILTER_FORWARD_CHAIN to be DOCKER-USER -
> though the all/all policy rules would have to not go in the
> FILTER_FORWARD_CHAIN, or the docker rules would never be reached.This doesn't work. it would result in NO traffic reaching the Docker chains, since Shorewall lets nothing fall off the end of the FORWARD chain. And packets only reach the last rule in the FORWARD chain if there is a configuration error. > > This is a rather large security hole if you use DOCKER=Yes (and docker) > and shorewall. > > I tried NOT using DOCKER=yes and just using the docker-proxy stuff (ie. > host binding ports), but docker-proxy has issues and will often enough > not be able to establish a connection correctly. So using docker's > iptables support (which creates the appropriate NAT rules to redirect to > the correct ports) is a much better way to go. > There was never an intention that Shorewall-generated rules be able to override those configured by Docker. There was a desire that either Shorewall or Docker should be able to be restarted without having to restart the other. Since restarting Docker always results in Docker's filter rules being first, starting/restarting Shorewall while Docker is running produces that same result. I have said a number of times that if I were to run Shorewall and Docker on one physical system, I would run at least one of them in a virtual machine. Note that doing so requires that the Shorewall-generated ruleset make the decision about what traffic to send to Docker, but it keeps the two products out of each other's hair and insures an understandable security framework. Also, if I had it to do over again, I would not implement any Docker support in Shorewall. I think doing so was a mistake. -Tom -- Tom Eastep \ Q: What do you get when you cross a mobster Shoreline, \ with an international standard? Washington, USA \ A: Someone who makes you an offer you http://shorewall.org \ can't understand \________________________________________
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users
