Hi Jaime,

Sorry for taking so long to get back to you on this one. I spent sometime
tonight trying to troubleshoot this but haven't been able to get the host
-> docker connection working yet but perhaps some experts can help
troubleshoot with the information below.

The test scenario is as follows. I manually spun up 2 hosts in a test
environment one "builder" (Host-A) and one "docker". I did not modify
anything in the image other than just starting it up and launching a
centos:7 container on the docker host.

container: centos:7 image running on "docker"
docker: The basic docker VM provided by ODL infra
builder: The generic build VM provided by ODL infra

Scenario 1: container ping docker host: SUCCESS
Scenario 2: container ping builder vm: SUCCESS
Scenario 3: docker host ping container: SUCCESS
Scenario 4: builder vm ping docker host: SUCCESS
Scenario 5: builder vm ping container: FAIL

So it seems the only direction that doesn't work is when another vm
(builder) pinging the docker host. According to the docker documentation
here:

https://docs.docker.com/engine/userguide/networking/
default_network/container-communication/

We should be setting iptables -P FORWARD ACCEPT to allow another host to
reach inside the docker container on the docker host however even after
setting this things are still unrouteable.

Below is a list of the iptables rules that's active on the system.


Jaime sent me some configuration for the docker host as follows.

- enable IP forwarding in B
        echo 1 > /proc/sys/net/ipv4/ip_forward

This is not necessary in our cloud as ip_forward is already enabled by
default.


- clean ip tables in B
        sudo iptables -F
        sudo iptables -t nat -F

In my testing cleaning the iptables in this way breaks the docker
container's ability to route to the outside world so I would not recommend
cleaning the iptables as this will make the container not be able to reach
anything at all.

Any ideas on what else I should check or try to configure?

Regards,
Thanh


# iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
DOCKER-USER  all  --  0.0.0.0/0            0.0.0.0/0
DOCKER-ISOLATION  all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            ctstate
RELATED,ESTABLISHED
DOCKER     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain DOCKER (1 references)
target     prot opt source               destination

Chain DOCKER-ISOLATION (1 references)
target     prot opt source               destination
RETURN     all  --  0.0.0.0/0            0.0.0.0/0

Chain DOCKER-USER (1 references)
target     prot opt source               destination
RETURN     all  --  0.0.0.0/0            0.0.0.0/0



On Fri, Dec 29, 2017 at 7:48 AM, Jaime Caamaño Ruiz <jcaam...@suse.de>
wrote:

> I took another look into this in oxygen.
>
> The test that fails uses docker to setup multiple OVS instances. All
> these instances run on a docker bridge network and must have
> connectivity to ODL each with distinct IP from that network. To achieve
> this, masquerading is disabled for the docker network, ip forwarding is
> enabled on the docker host and a route for this network via the docker
> host is added to the odl host.
>
> This setup worked fine before the cloud provider transition. But now
> packets are dropped between the docker host and the odl host when the
> source IP is that of the docker network. I have verified forwarding is
> done correctly on the docker host. I have also verified that the docker
> network is reachable from the odl host, so the problem is on the other
> direction.
>
> I am out of ideas now but this might be out of our control and the test
> invalid as is.
>
> BR
> Jaime.
>
>
>
> On Fri, 2017-12-22 at 12:17 -0800, Jamo Luhrsen wrote:
> > On 12/22/2017 07:45 AM, Sam Hague wrote:
> > >
> > >
> > > On Fri, Dec 22, 2017 at 7:37 AM, Jaime Caamaño Ruiz <jcaamano@suse.
> > > de <mailto:jcaam...@suse.de>> wrote:
> > >
> > >     Hello Jamo
> > >
> > >     Took a quick look.
> > >
> > >     The problem seems to be that the odl-sfc-openflow-renderer
> > > feature is
> > >     installed but the dependant bundles are not activated. This
> > > feature is
> > >     in charge of writing the flows, and the test fails because
> > > there are no
> > >     flows on the  switches.
> >
> > yeah, I saw a bunch of ugly karaf.log messages as well. There is an
> > NPE showing
> > up from sfc code that may or may not be important. Check this log:
> >
> > https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/sfc-csit-3
> > node-docker-full-deploy-all-carbon/390/odl1_karaf.log.gz
> >
> > >     This is not the only feature with this problem, seems to me
> > > than more
> > >     than one bundle is just not properly started. So something is
> > > wrong in
> > >     the feature or blueprint handling. Unfortunately, older logs
> > > are not
> > >     available to compare but there were no changes in Carbon since
> > > the last
> > >     success job.
> >
> > I tried a job with Carbon SR2 since that should probably have been
> > from
> > before when it was passing. It failed as well. The logs didn't save,
> > but
> > you can see in the console log that it's the same failure:
> >
> > https://jenkins.opendaylight.org/releng/view/sfc/job/sfc-csit-3node-d
> > ocker-full-deploy-all-carbon/392/console
> >
> > Thanks,
> > JamO
> >
> >
> > >     Oxygen job is also failing from Dec 9. On this case, seems that
> > > there
> > >     is no connectivity between the docker network running in the
> > > tools
> > >     system and ODL. Something that we fixed few days ago caused by
> > > a new
> > >     tools docker image that did not have ip forward enabled. But
> > > now I dont
> > >     know what is the problem. When did we effectively switch to new
> > > cloud
> > >     provider?
> > >
> > > the switch was last week so around the 15th.
> > >
> > >
> > >     BR
> > >     Jaime.
> > >
> > >
> > >     On Thu, 2017-12-21 at 16:57 -0800, Jamo Luhrsen wrote:
> > >     > Hi SFC,
> > >     >
> > >     > I can't remember if there was ever any discussion on this one
> > > yet,
> > >     > but there is a
> > >     > consistent failure it would be nice to see fixed. Or if there
> > > is a
> > >     > low sev bug
> > >     > we can point to, I'll edit the test to list that bug in the
> > > report.
> > >     >
> > >     > looks like there is something not showing up in an ovs docker
> > >     > containers flow
> > >     > table. Specifically, it's not finding "*actions=pop_nsh*"
> > >     >
> > >     > I'm vaguely remembering something regarding this, but can't
> > > recall it
> > >     > exactly.
> > >     >
> > >     > here's an example:
> > >     > https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/sf
> > > c-csit-3
> > >     <https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/sfc
> > > -csit-3>
> > >     > node-docker-full-deploy-all-carbon/390/log.html.gz#s1-s1-s1
> > >     >
> > >     >
> > >     > Thanks,
> > >     > JamO
> > >     > _______________________________________________
> > >     > sfc-dev mailing list
> > >     > sfc-dev@lists.opendaylight.org <mailto:sfc-dev@lists.opendayl
> > > ight.org>
> > >     > https://lists.opendaylight.org/mailman/listinfo/sfc-dev <http
> > > s://lists.opendaylight.org/mailman/listinfo/sfc-dev>
> > >     >
> > >     _______________________________________________
> > >     integration-dev mailing list
> > >     integration-...@lists.opendaylight.org <mailto:integration-dev@
> > > lists.opendaylight.org>
> > >     https://lists.opendaylight.org/mailman/listinfo/integration-dev
> > >     <https://lists.opendaylight.org/mailman/listinfo/integration-de
> > > v>
> > >
> > >
> >
> >
> _______________________________________________
> integration-dev mailing list
> integration-...@lists.opendaylight.org
> https://lists.opendaylight.org/mailman/listinfo/integration-dev
>
_______________________________________________
sfc-dev mailing list
sfc-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/sfc-dev

Reply via email to