Re: Port forwarding for Guix containers

[Zhu Zihao]

Thank you Jason. Your code looks good, but after some search and
reading, I found it's a very very complicate issue for networking
between containers, it may not available to manage it in a declarative
way(or say Guix way).

So I decide to continue to use Docker, and leave iptables for Docker to
play. Now I use nftables to setup firewall rules personally.

Thank you again.

Jason Conroy writes:

> Hi Zihao,
>
> It sounds like you're running Guix for your host OS and want to have Guix
> containers inside of that? If that's so, then my existing config won't be
> much use to you: right now I'm running my Guix containers (the `guix system
> container` shell scripts) inside of Debian via systemd.
>
> But in case it helps, I think this is how you could approximate what
> "docker run --network --publish ..." does:
>
> 1) Create a persistent network namespace with `ip netns add`.
> 2) Use `ip link add` to create a pair of virtual ethernet interfaces (veth)
> - one for the host and one for the container.
> 3) Use `ip link set  netns ` so that one of the veth
> interfaces appears inside of the namespace, while its peer remains on the
> host side.
> 4) Assign each of the veth interfaces an address in the same subnet, but
> choose a subset that's unused on your system. For example, 192.168.0.1 and
> 192.168.0.2 within the subnet 192.168.0.0/24.
> 5) Bring up the interfaces with `ip link set  up`. Do the same for
> the loopback interface (lo) inside the namespace.
> 6) Inside the namespace, set up a default route using the address of the
> veth interface on the host side.
> 7) Use iptables to configure source network address translation (SNAT) for
> the traffic originating from the namespace so that it can connect to
> external hosts (e.g. via eth0).
> 8) Enable IP forwarding: set /proc/sys/net/ipv4/ip_forward to 1, and add
> related rules to iptables' FORWARD chain (if your default iptables policy
> is to DROP packets).
> 9) Finally, use iptables again to enable port forwarding (DNAT) from
> external hosts to your container.
>
> Here, "do X inside of a namespace" usually means `ip netns exec 
> `. When the command is /bin/bash you can explore the namespace's
> environment interactively. The namespace persists until you call `ip netns
> del `.
>
> With the exception of #9, there are examples of each task in the script I
> mentioned up-thread:
> https://gist.github.com/dpino/6c0dca1742093346461e11aa8f608a99#file-ns-inet-sh
>
> For my purposes, dynamic configuration of namespaces, interfaces, routes,
> etc. (like Docker does) seems unnecessarily complicated and fragile, so
> I've taken the approach of setting up my namespaces once at boot, and then
> the container startup script is as simple as `ip netns exec 
> `. Even when the Guix container itself shuts down
> and restarts, the namespace settings above are unchanged.
>
> How would these network settings be implemented using Guix services? I
> don't have experience in this area, so the following is just a guess:
> iptables-service seems suitable for tasks #7 - #9, and there's
> static-networking-service for assigning addresses in task #4 (but I think
> it will only know about the veth interface outside the namespace, not the
> one inside). For the rest, I think you'd need to define some new service to
> set up the namespace and virtual interfaces, and ensure that this service
> runs before static-networking-service.
>
> Hope that helps,
>
> Jason
>
> On Mon, Nov 23, 2020 at 11:22 AM Zhu Zihao  wrote:
>
>>
>> That's what I want to say, thank you!
>>
>> I want to combine different software in containers in docker-compose
>> like way. It's more similar with a system container then a `guix
>> environment` container.
>>
>> I'm not a Docker hater, but docker will corrupt your iptables entry and
>> make the system impure. If you wanna use iptables-service-type and
>> docker-service-type together, when you run `herd restart iptables`. All
>> docker specific rules will be erased.
>>
>> > Supposing that we've developed some system container that starts a
>> service
>> > on port N. If we want to run another instance of the same container, we
>> > first need to override the port number for the service in our
>> > operating-system, otherwise the service in the second container will fail
>> > to bind to port N in the shared network namespace. With a couple of
>> > one-service containers this may not be so hard, but system containers in
>> > general could have lots of services, and the authors of individual
>> > containers may not want to worry about choosing port numbers that are
>> > mutually disjoint from those in all other containers (and those used by
>> the
>> > container host itself).
>>
>> --
>> Retrieve my PGP public key: https://meta.sr.ht/~citreu.pgp
>>
>> Zihao
>>


-- 
Retrieve my PGP public key: https://meta.sr.ht/~citreu.pgp

Zihao


signature.asc
Description: PGP signature

Re: Port forwarding for Guix containers

[Jason Conroy]

Hi Zihao,

It sounds like you're running Guix for your host OS and want to have Guix
containers inside of that? If that's so, then my existing config won't be
much use to you: right now I'm running my Guix containers (the `guix system
container` shell scripts) inside of Debian via systemd.

But in case it helps, I think this is how you could approximate what
"docker run --network --publish ..." does:

1) Create a persistent network namespace with `ip netns add`.
2) Use `ip link add` to create a pair of virtual ethernet interfaces (veth)
- one for the host and one for the container.
3) Use `ip link set  netns ` so that one of the veth
interfaces appears inside of the namespace, while its peer remains on the
host side.
4) Assign each of the veth interfaces an address in the same subnet, but
choose a subset that's unused on your system. For example, 192.168.0.1 and
192.168.0.2 within the subnet 192.168.0.0/24.
5) Bring up the interfaces with `ip link set  up`. Do the same for
the loopback interface (lo) inside the namespace.
6) Inside the namespace, set up a default route using the address of the
veth interface on the host side.
7) Use iptables to configure source network address translation (SNAT) for
the traffic originating from the namespace so that it can connect to
external hosts (e.g. via eth0).
8) Enable IP forwarding: set /proc/sys/net/ipv4/ip_forward to 1, and add
related rules to iptables' FORWARD chain (if your default iptables policy
is to DROP packets).
9) Finally, use iptables again to enable port forwarding (DNAT) from
external hosts to your container.

Here, "do X inside of a namespace" usually means `ip netns exec 
`. When the command is /bin/bash you can explore the namespace's
environment interactively. The namespace persists until you call `ip netns
del `.

With the exception of #9, there are examples of each task in the script I
mentioned up-thread:
https://gist.github.com/dpino/6c0dca1742093346461e11aa8f608a99#file-ns-inet-sh

For my purposes, dynamic configuration of namespaces, interfaces, routes,
etc. (like Docker does) seems unnecessarily complicated and fragile, so
I've taken the approach of setting up my namespaces once at boot, and then
the container startup script is as simple as `ip netns exec 
`. Even when the Guix container itself shuts down
and restarts, the namespace settings above are unchanged.

How would these network settings be implemented using Guix services? I
don't have experience in this area, so the following is just a guess:
iptables-service seems suitable for tasks #7 - #9, and there's
static-networking-service for assigning addresses in task #4 (but I think
it will only know about the veth interface outside the namespace, not the
one inside). For the rest, I think you'd need to define some new service to
set up the namespace and virtual interfaces, and ensure that this service
runs before static-networking-service.

Hope that helps,

Jason

On Mon, Nov 23, 2020 at 11:22 AM Zhu Zihao  wrote:

>
> That's what I want to say, thank you!
>
> I want to combine different software in containers in docker-compose
> like way. It's more similar with a system container then a `guix
> environment` container.
>
> I'm not a Docker hater, but docker will corrupt your iptables entry and
> make the system impure. If you wanna use iptables-service-type and
> docker-service-type together, when you run `herd restart iptables`. All
> docker specific rules will be erased.
>
> > Supposing that we've developed some system container that starts a
> service
> > on port N. If we want to run another instance of the same container, we
> > first need to override the port number for the service in our
> > operating-system, otherwise the service in the second container will fail
> > to bind to port N in the shared network namespace. With a couple of
> > one-service containers this may not be so hard, but system containers in
> > general could have lots of services, and the authors of individual
> > containers may not want to worry about choosing port numbers that are
> > mutually disjoint from those in all other containers (and those used by
> the
> > container host itself).
>
> --
> Retrieve my PGP public key: https://meta.sr.ht/~citreu.pgp
>
> Zihao
>

Re: Port forwarding for Guix containers

[Zhu Zihao]

I try to search a lightweight network sharing solution for containers
without Docker. But when my search query contains keyword "container",
almost all results referring to Docker... :(

-- 
Retrieve my PGP public key: https://meta.sr.ht/~citreu.pgp

Zihao


signature.asc
Description: PGP signature

Re: Port forwarding for Guix containers

[Zhu Zihao]

That's what I want to say, thank you!

I want to combine different software in containers in docker-compose
like way. It's more similar with a system container then a `guix
environment` container.

I'm not a Docker hater, but docker will corrupt your iptables entry and
make the system impure. If you wanna use iptables-service-type and
docker-service-type together, when you run `herd restart iptables`. All
docker specific rules will be erased. 

> Supposing that we've developed some system container that starts a service
> on port N. If we want to run another instance of the same container, we
> first need to override the port number for the service in our
> operating-system, otherwise the service in the second container will fail
> to bind to port N in the shared network namespace. With a couple of
> one-service containers this may not be so hard, but system containers in
> general could have lots of services, and the authors of individual
> containers may not want to worry about choosing port numbers that are
> mutually disjoint from those in all other containers (and those used by the
> container host itself).

-- 
Retrieve my PGP public key: https://meta.sr.ht/~citreu.pgp

Zihao


signature.asc
Description: PGP signature

Re: Port forwarding for Guix containers

[Jason Conroy]

Hi Edouard, I completely agree that this sort of composition is convenient
for a group of services that share a single dependency graph. For example,
when deploying a web application in a container, one could also provide an
nginx server there for its exclusive use, and maybe even a database server
(if the db doesn't have other clients).

But when it comes to deploying separate, unrelated applications (or
isolated instances of an app, like Production and Testing), I believe that
development, testing, and maintenance are all simplified by keeping the
containers separate. For example, after a `guix pull` with a major package
update, I might want to rebuild a single application container and test the
new package there. The alternative is to bring down all applications,
upgrade them all, test them all, and roll back everything if there's at
least one issue. When there are many applications, this doesn't scale well.

I guess I see `guix system container` as analogous to `guix environment` in
this regard: a single environment may contain many binaries that work
together, but users are encouraged to set up different environments for
Project A and Project B.

I'm not sure whether Zihao's situation is similar, but that's my
perspective. :)

Jason


On Sun, Nov 22, 2020 at 10:10 AM  wrote:

>
> The trick is that guix being written in scheme, operating system
> declarations can be written in a composable way, where the port N can be
> a parameter.
>
> The way I see it, it is when you compose all your services together on
> one host that you decide which service gets which port, and declare all
> that in a single operating system declaration (which is made up of
> modular, task specific, smaller OS declarations).
>
> I don't know if I'm being clear. I can't provide code because I did not
> have the time to do it yet, but this was my understanding of how guix
> works.
>
> Jason Conroy writes:
>
> > I agree with Zihao that containers have certain use cases where it's
> > important to use separate network namespaces for each instance, with
> > traffic forwarded selectively between host and guest. Security (and hence
> > firewalling) is part of the issue, but it's also about the container's
> > maintainability and reproducibility.
> >
> > Supposing that we've developed some system container that starts a
> service
> > on port N. If we want to run another instance of the same container, we
> > first need to override the port number for the service in our
> > operating-system, otherwise the service in the second container will fail
> > to bind to port N in the shared network namespace. With a couple of
> > one-service containers this may not be so hard, but system containers in
> > general could have lots of services, and the authors of individual
> > containers may not want to worry about choosing port numbers that are
> > mutually disjoint from those in all other containers (and those used by
> the
> > container host itself).
> >
> > Aside from the risk that one container's port bindings will prevent
> another
> > container from working, there's also the risk of unintended dependencies:
> > we might start a container thinking that it's self-contained, when really
> > it depends on a service belonging to the container's host or to another
> > container. This is why I consider the shared namespace a reproducibility
> > problem.
> >
> > Lately I've been experimenting with a modified version of this script
> > <
> https://gist.github.com/dpino/6c0dca1742093346461e11aa8f608a99#file-ns-inet-sh
> >
> > to set up a network namespace with its own interface and routes, and then
> > run a guix system container inside. Because the container is built with
> the
> > -N flag, its services will bind to the virtual interface inside the
> network
> > namespace. Processes inside the container can access the internet, while
> > processes on the host (but outside the container) can access the
> container
> > services via the IP address bound to the container's interface.
> >
> > Next, to make the container's services accessible to other hosts, there
> are
> > a couple of options. One is to enable port forwarding from the host's
> > external interface to the container's IP address using iptables
> > . If the
> > container is hosting a web service, another choice (as Edouard mentions)
> is
> > for the host to run some sort of reverse proxy that forwards incoming
> > requests to the container's port. For example, nginx and Apache can both
> do
> > this.
> >
> > It would be really nice if guix system containers had this namespacing
> > ability built in, but it sounds complex.
> >
> > On Sat, Nov 21, 2020 at 10:03 AM zimoun 
> wrote:
> >
> >> Hi,
> >>
> >> On Fri, 20 Nov 2020 at 19:26, Christopher Baines 
> wrote:
> >> > Zhu Zihao  writes:
> >> >
> >> >> I found guix container "created by `guix environment --container` or
> >> >> `guix system container`" is very useful to isolate some 

Re: Port forwarding for Guix containers

[edk]

The trick is that guix being written in scheme, operating system
declarations can be written in a composable way, where the port N can be
a parameter.

The way I see it, it is when you compose all your services together on
one host that you decide which service gets which port, and declare all
that in a single operating system declaration (which is made up of
modular, task specific, smaller OS declarations).

I don't know if I'm being clear. I can't provide code because I did not
have the time to do it yet, but this was my understanding of how guix works.

Jason Conroy writes:

> I agree with Zihao that containers have certain use cases where it's
> important to use separate network namespaces for each instance, with
> traffic forwarded selectively between host and guest. Security (and hence
> firewalling) is part of the issue, but it's also about the container's
> maintainability and reproducibility.
>
> Supposing that we've developed some system container that starts a service
> on port N. If we want to run another instance of the same container, we
> first need to override the port number for the service in our
> operating-system, otherwise the service in the second container will fail
> to bind to port N in the shared network namespace. With a couple of
> one-service containers this may not be so hard, but system containers in
> general could have lots of services, and the authors of individual
> containers may not want to worry about choosing port numbers that are
> mutually disjoint from those in all other containers (and those used by the
> container host itself).
>
> Aside from the risk that one container's port bindings will prevent another
> container from working, there's also the risk of unintended dependencies:
> we might start a container thinking that it's self-contained, when really
> it depends on a service belonging to the container's host or to another
> container. This is why I consider the shared namespace a reproducibility
> problem.
>
> Lately I've been experimenting with a modified version of this script
> 
> to set up a network namespace with its own interface and routes, and then
> run a guix system container inside. Because the container is built with the
> -N flag, its services will bind to the virtual interface inside the network
> namespace. Processes inside the container can access the internet, while
> processes on the host (but outside the container) can access the container
> services via the IP address bound to the container's interface.
>
> Next, to make the container's services accessible to other hosts, there are
> a couple of options. One is to enable port forwarding from the host's
> external interface to the container's IP address using iptables
> . If the
> container is hosting a web service, another choice (as Edouard mentions) is
> for the host to run some sort of reverse proxy that forwards incoming
> requests to the container's port. For example, nginx and Apache can both do
> this.
>
> It would be really nice if guix system containers had this namespacing
> ability built in, but it sounds complex.
>
> On Sat, Nov 21, 2020 at 10:03 AM zimoun  wrote:
>
>> Hi,
>>
>> On Fri, 20 Nov 2020 at 19:26, Christopher Baines  wrote:
>> > Zhu Zihao  writes:
>> >
>> >> I found guix container "created by `guix environment --container` or
>> >> `guix system container`" is very useful to isolate some service. But
>> >> it only supports fully isolated network namespace or just share with
>> >> host, it's not so safe IMO.
>> >
>> > I'll assume that a fully isolated network namespace is safer in whatever
>> > way you're referring to than a shared network namespace. However, for a
>> > shared network namespace, what threats is that not safe in respect to?
>> >
>> > In the shared network namespace scenario, you are free to use a
>> > firewall, which could help protect against threats coming from other
>> > machines, for example by creating a list of IP addresses which are
>> > allowed to connect, and dropping any other traffic.
>>
>> I do not know about the initial motivation and I do not know either if
>> it makes sense in the context of “guix environment”.  One point is that
>> Docker [1] provides a way to specify the firewall rules.  Well, somehow,
>> something similar as ’--share’ but for network.
>>
>>
>> 1: 
>>
>> All the best,
>> simon
>>
>>

Re: Port forwarding for Guix containers

[Jason Conroy]

I agree with Zihao that containers have certain use cases where it's
important to use separate network namespaces for each instance, with
traffic forwarded selectively between host and guest. Security (and hence
firewalling) is part of the issue, but it's also about the container's
maintainability and reproducibility.

Supposing that we've developed some system container that starts a service
on port N. If we want to run another instance of the same container, we
first need to override the port number for the service in our
operating-system, otherwise the service in the second container will fail
to bind to port N in the shared network namespace. With a couple of
one-service containers this may not be so hard, but system containers in
general could have lots of services, and the authors of individual
containers may not want to worry about choosing port numbers that are
mutually disjoint from those in all other containers (and those used by the
container host itself).

Aside from the risk that one container's port bindings will prevent another
container from working, there's also the risk of unintended dependencies:
we might start a container thinking that it's self-contained, when really
it depends on a service belonging to the container's host or to another
container. This is why I consider the shared namespace a reproducibility
problem.

Lately I've been experimenting with a modified version of this script

to set up a network namespace with its own interface and routes, and then
run a guix system container inside. Because the container is built with the
-N flag, its services will bind to the virtual interface inside the network
namespace. Processes inside the container can access the internet, while
processes on the host (but outside the container) can access the container
services via the IP address bound to the container's interface.

Next, to make the container's services accessible to other hosts, there are
a couple of options. One is to enable port forwarding from the host's
external interface to the container's IP address using iptables
. If the
container is hosting a web service, another choice (as Edouard mentions) is
for the host to run some sort of reverse proxy that forwards incoming
requests to the container's port. For example, nginx and Apache can both do
this.

It would be really nice if guix system containers had this namespacing
ability built in, but it sounds complex.

On Sat, Nov 21, 2020 at 10:03 AM zimoun  wrote:

> Hi,
>
> On Fri, 20 Nov 2020 at 19:26, Christopher Baines  wrote:
> > Zhu Zihao  writes:
> >
> >> I found guix container "created by `guix environment --container` or
> >> `guix system container`" is very useful to isolate some service. But
> >> it only supports fully isolated network namespace or just share with
> >> host, it's not so safe IMO.
> >
> > I'll assume that a fully isolated network namespace is safer in whatever
> > way you're referring to than a shared network namespace. However, for a
> > shared network namespace, what threats is that not safe in respect to?
> >
> > In the shared network namespace scenario, you are free to use a
> > firewall, which could help protect against threats coming from other
> > machines, for example by creating a list of IP addresses which are
> > allowed to connect, and dropping any other traffic.
>
> I do not know about the initial motivation and I do not know either if
> it makes sense in the context of “guix environment”.  One point is that
> Docker [1] provides a way to specify the firewall rules.  Well, somehow,
> something similar as ’--share’ but for network.
>
>
> 1: 
>
> All the best,
> simon
>
>

Re: Port forwarding for Guix containers

[Edouard Klein]

zimoun writes:

> Hi,
>
> On Fri, 20 Nov 2020 at 19:26, Christopher Baines  wrote:
>> Zhu Zihao  writes:
>>
>>> I found guix container "created by `guix environment --container` or
>>> `guix system container`" is very useful to isolate some service. But
>>> it only supports fully isolated network namespace or just share with
>>> host, it's not so safe IMO.
>>
>> I'll assume that a fully isolated network namespace is safer in whatever
>> way you're referring to than a shared network namespace. However, for a
>> shared network namespace, what threats is that not safe in respect to?
>>
>> In the shared network namespace scenario, you are free to use a
>> firewall, which could help protect against threats coming from other
>> machines, for example by creating a list of IP addresses which are
>> allowed to connect, and dropping any other traffic.
>
> I do not know about the initial motivation and I do not know either if
> it makes sense in the context of “guix environment”.  One point is that
> Docker [1] provides a way to specify the firewall rules.  Well, somehow,
> something similar as ’--share’ but for network.
>
>
> 1: 
>

My .02€:

I am in the camp of letting the container do the job with an operating
system declaration, and keeping guix simple. That way, one can choose
e.g. nginx to do the proxying, or an actual firewall, etc. The right
tool for the right job.

Sure it's not as easy as docker's -p option, but it's more secure and
cleaner.



> All the best,
> simon

Re: Port forwarding for Guix containers

[zimoun]

Hi,

On Fri, 20 Nov 2020 at 19:26, Christopher Baines  wrote:
> Zhu Zihao  writes:
>
>> I found guix container "created by `guix environment --container` or
>> `guix system container`" is very useful to isolate some service. But
>> it only supports fully isolated network namespace or just share with
>> host, it's not so safe IMO.
>
> I'll assume that a fully isolated network namespace is safer in whatever
> way you're referring to than a shared network namespace. However, for a
> shared network namespace, what threats is that not safe in respect to?
>
> In the shared network namespace scenario, you are free to use a
> firewall, which could help protect against threats coming from other
> machines, for example by creating a list of IP addresses which are
> allowed to connect, and dropping any other traffic.

I do not know about the initial motivation and I do not know either if
it makes sense in the context of “guix environment”.  One point is that
Docker [1] provides a way to specify the firewall rules.  Well, somehow,
something similar as ’--share’ but for network.


1: 

All the best,
simon

Re: Port forwarding for Guix containers

[Christopher Baines]

Zhu Zihao  writes:

> I found guix container "created by `guix environment --container` or
> `guix system container`" is very useful to isolate some service. But
> it only supports fully isolated network namespace or just share with
> host, it's not so safe IMO.

I'll assume that a fully isolated network namespace is safer in whatever
way you're referring to than a shared network namespace. However, for a
shared network namespace, what threats is that not safe in respect to?

In the shared network namespace scenario, you are free to use a
firewall, which could help protect against threats coming from other
machines, for example by creating a list of IP addresses which are
allowed to connect, and dropping any other traffic.

If it's not on another machine, but on the same machine, there's
probably more to worry about than the network if you're assuming another
process is malicious, it could potentially escape from the isolation put
in place by Linux, or use excessive resources to attempt to disrupt
other processes.

Chris


signature.asc
Description: PGP signature

Re: Port forwarding for Guix containers

[Bonface M. K.]

"Zhu Zihao"  writes:

> Hi, Guix users!
>
>
> I found guix container "created by `guix
> environment --container` or `guix system
> container`" is very useful to isolate some
> service. But it only supports fully isolated
> network namespace or just share with host, it's
> not so safe IMO.
>
>
>
> In Docker, there's port forwarding, allows you to
> share some ports of Guix container with host. I
> just learn something about docker's network
> mechanism, it looks quite complicated. It use veth
> pair, network bridge and even iptables. Is there
> some idiomatic way to implement such port
> forwarding feature for Guix containers?
>
>
> Any answer or suggestions are appreicated.
>

I don't know if this is helpful, but you could
start a container with network access by passing
the "-N" flag. Here's an example:

--8<---cut here---start->8---
./pre-inst-env guix environment -N --ad-hoc wget -- wget "some-url"
--8<---cut here---end--->8---

I don't know if this is feasible, but you could
try using some third party tool to limit ports
from _inside_ the container...

HTH!

-- 
Bonface M. K. <https://www.bonfacemunyoki.com>
Chief Emacs Bazu / Rieng ya software sare
Mchochezi of: <https://upbookclub.com> / Twitter: @BonfaceKilz
GPG Key: D4F09EB110177E03C28E2FE1F5BBAE1E0392253F


signature.asc
Description: PGP signature

Port forwarding for Guix containers

[Zhu Zihao]

Hi, Guix users!


I found guix container "created by `guix environment --container` or `guix 
system container`" is very useful to isolate some service. But it only supports 
fully isolated network namespace or just share with host, it's not so safe IMO.



In Docker, there's port forwarding, allows you to share some ports of Guix 
container with host. I just learn something about docker's network mechanism, 
it looks quite complicated. It use veth pair, network bridge and even iptables. 
Is there some idiomatic way to implement such port forwarding feature for Guix 
containers?


Any answer or suggestions are appreicated.


--


Zihao