[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[dsclose]

Github user dsclose commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-222195257
  
@luhaijiao I'd recommend trying e7a63be

Port forwarding works for me. I've not tried the VPN functionality.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[luhaijiao]

Github user luhaijiao commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-222182040
  
@dsclose we installed your updated commits  d582358 and bf285e1 ONLY in our 
environment to solve the RvR issue (VR network services intermittent hang due 
to public interface is up on backup VR), however, it's not working as expected, 
 the status of eth2 is still up on backup VR.  Do we need install all the 
commits ?  or it's probably our environment issue ?

Besides,  if we install all the commits,  the port forwarding and VPN seems 
getting broken.  We are investigating more.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[dsclose]

Github user dsclose commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-222085574
  
@swill agreed; in particular, my note concerning the merge of lines 299 and 
300 - i've not even tested that locally - not had a chance to even figure out 
what it's doing!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-221904409
  
I actually had a test running overnight for this one and it had similar 
results.

```
Tests Run: 85
  Skipped: 0
   Failed: 14
   Errors: 6
 Duration: 9h 36m 41s
```

I think this one will need some work still.  I will run it again to see 
what we get as a result and I will post the next run so you know what the 
latest issues are.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-221901259
  
I will retest this one.  Thanks for kicking it off again, we are green now. 
 :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[dsclose]

Github user dsclose commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-221867807
  
@swill no worries. Happy to do that - the CI output was vast and I wasn't 
looking forward to combing through it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[dsclose]

GitHub user dsclose reopened a pull request:

https://github.com/apache/cloudstack/pull/1519

Cloudstack 9339: Virtual Routers do not handle Multiple Public Interfaces

This PR addresses CLOUDSTACK-9339 and may need a code review from someone 
familiar with the System VM scripts. In particular, this PR has not been tested 
in a VPC RvR context. Only standalone routers and RvR routers have been 
demonstrated.

- **d582358: Leave public interfaces down in backup redundant routers.** 
Previously backup routers were bringing all interfaces up and thus arping 
public IPs away from the master router.
- **9ee1eb6: Add the default gateway to the main routing table when 
interfaces are configured.** The gateway for the first public IP was always 
being added to the main routing table. Sometimes a router would consequently 
add the gateway for an IP other than the default source-NAT IP. This would 
prevent outbound connectivity for guest VMs.
- **ad9d72f: Add default gateway to device-specific routing tables.** 
Link-level routes were being put into the device-specific routing tables 
(accessed via firewall marks) but these are unnecessary. Instead, the default 
gateway is needed to allow the kernel to make an appropriate routing decision.
- **8db879e: Only mark guest connections when they are part of a 
static-NAT.** Guest connections were being marked with a zero. This added no 
functionality and prevented static-NAT rules from routing outbound traffic 
properly as device-specific routing tables would not be used. Instead, all 
traffic would be routed out via the default public interface.
- **788b1be: Allow forwarding and collect network stats on any public 
interface.** Forwarding rules and network stats were limited to eth2 on RvR 
networks. This needed to be decoupled from eth2 and reapplied to whichever 
interface was under consideration.
- **b19e8aa: Ensure that CONNMARK --restore-mark only appears once.** This 
is a bit of a hack and can do with being improved. The CONNMARK rule was not 
being picked up by the de-duplication logic in CsNetfilter and was being added 
twice. This caused checksum errors on packets traversing NAT.
- **bf285e1: Transition to master state should add all necessary routes.** 
Now that backup routers keep their interfaces down, the route logic executed at 
configuration-time cannot be applied. Instead, once the interface is brought up 
during a transition to master, routers must re-evaluate what routes are needed 
and add them. Unfortunately I couldn't see a way to re-use the existing route 
logic with the variables that I had in scope so there is some duplication. In 
some cases, routers did not successfully arp IPs away from the old master so 
some arp logic was added. During a failover most connections with guest VMs 
will be maintained with only minor packet loss. SSH sessions implemented via 
port-forwarding rules on an interface other than the source-NAT interface 
consistently get dropped, however, so the failover isn't quite seamless. It's 
possible that there's an easy fix for that.

I expect that a number of tests may need to be modified/written as part of 
this PR. Any feedback or pointers would be useful as initially I'll be relying 
on the CI failures to tell me where to look.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dsclose/cloudstack CLOUDSTACK-9339

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1519.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1519


commit e7a63be161bdd14c985a8b483bffe4bfdaa3f5d4
Author: dean.close 
Date:   2016-05-09T10:31:26Z

CLOUDSTACK-9339: Handle multiple public subnets on virtual routers.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[dsclose]

Github user dsclose closed the pull request at:

https://github.com/apache/cloudstack/pull/1519


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-221862724
  
@dsclose sorry to do this to you.  Can you close and reopen again to kick 
off the jobs again?  Thanks...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[dsclose]

GitHub user dsclose reopened a pull request:

https://github.com/apache/cloudstack/pull/1519

Cloudstack 9339: Virtual Routers do not handle Multiple Public Interfaces

This PR addresses CLOUDSTACK-9339 and may need a code review from someone 
familiar with the System VM scripts. In particular, this PR has not been tested 
in a VPC RvR context. Only standalone routers and RvR routers have been 
demonstrated.

- **d582358: Leave public interfaces down in backup redundant routers.** 
Previously backup routers were bringing all interfaces up and thus arping 
public IPs away from the master router.
- **9ee1eb6: Add the default gateway to the main routing table when 
interfaces are configured.** The gateway for the first public IP was always 
being added to the main routing table. Sometimes a router would consequently 
add the gateway for an IP other than the default source-NAT IP. This would 
prevent outbound connectivity for guest VMs.
- **ad9d72f: Add default gateway to device-specific routing tables.** 
Link-level routes were being put into the device-specific routing tables 
(accessed via firewall marks) but these are unnecessary. Instead, the default 
gateway is needed to allow the kernel to make an appropriate routing decision.
- **8db879e: Only mark guest connections when they are part of a 
static-NAT.** Guest connections were being marked with a zero. This added no 
functionality and prevented static-NAT rules from routing outbound traffic 
properly as device-specific routing tables would not be used. Instead, all 
traffic would be routed out via the default public interface.
- **788b1be: Allow forwarding and collect network stats on any public 
interface.** Forwarding rules and network stats were limited to eth2 on RvR 
networks. This needed to be decoupled from eth2 and reapplied to whichever 
interface was under consideration.
- **b19e8aa: Ensure that CONNMARK --restore-mark only appears once.** This 
is a bit of a hack and can do with being improved. The CONNMARK rule was not 
being picked up by the de-duplication logic in CsNetfilter and was being added 
twice. This caused checksum errors on packets traversing NAT.
- **bf285e1: Transition to master state should add all necessary routes.** 
Now that backup routers keep their interfaces down, the route logic executed at 
configuration-time cannot be applied. Instead, once the interface is brought up 
during a transition to master, routers must re-evaluate what routes are needed 
and add them. Unfortunately I couldn't see a way to re-use the existing route 
logic with the variables that I had in scope so there is some duplication. In 
some cases, routers did not successfully arp IPs away from the old master so 
some arp logic was added. During a failover most connections with guest VMs 
will be maintained with only minor packet loss. SSH sessions implemented via 
port-forwarding rules on an interface other than the source-NAT interface 
consistently get dropped, however, so the failover isn't quite seamless. It's 
possible that there's an easy fix for that.

I expect that a number of tests may need to be modified/written as part of 
this PR. Any feedback or pointers would be useful as initially I'll be relying 
on the CI failures to tell me where to look.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dsclose/cloudstack CLOUDSTACK-9339

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1519.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1519


commit e7a63be161bdd14c985a8b483bffe4bfdaa3f5d4
Author: dean.close 
Date:   2016-05-09T10:31:26Z

CLOUDSTACK-9339: Handle multiple public subnets on virtual routers.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[dsclose]

Github user dsclose closed the pull request at:

https://github.com/apache/cloudstack/pull/1519


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-221637516
  
@dsclose can you kick it off again.  It is just a timeout which caused it 
to fail, so kicking it off again could fix it.  `No output has been received in 
the last 10m0s`.

I will CI this right now.  Thx...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[dsclose]

Github user dsclose commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-221611699
  
Same issue. I don't think this is related to the PR. I'll leave this be for 
now.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[dsclose]

Github user dsclose commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-221600590
  
> No output has been received in the last 10m0s, this potentially indicates 
a stalled build or something wrong with the build itself.
>
>The build has been terminated

Kicking it off again.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[dsclose]

GitHub user dsclose reopened a pull request:

https://github.com/apache/cloudstack/pull/1519

Cloudstack 9339: Virtual Routers do not handle Multiple Public Interfaces

This PR addresses CLOUDSTACK-9339 and may need a code review from someone 
familiar with the System VM scripts. In particular, this PR has not been tested 
in a VPC RvR context. Only standalone routers and RvR routers have been 
demonstrated.

- **d582358: Leave public interfaces down in backup redundant routers.** 
Previously backup routers were bringing all interfaces up and thus arping 
public IPs away from the master router.
- **9ee1eb6: Add the default gateway to the main routing table when 
interfaces are configured.** The gateway for the first public IP was always 
being added to the main routing table. Sometimes a router would consequently 
add the gateway for an IP other than the default source-NAT IP. This would 
prevent outbound connectivity for guest VMs.
- **ad9d72f: Add default gateway to device-specific routing tables.** 
Link-level routes were being put into the device-specific routing tables 
(accessed via firewall marks) but these are unnecessary. Instead, the default 
gateway is needed to allow the kernel to make an appropriate routing decision.
- **8db879e: Only mark guest connections when they are part of a 
static-NAT.** Guest connections were being marked with a zero. This added no 
functionality and prevented static-NAT rules from routing outbound traffic 
properly as device-specific routing tables would not be used. Instead, all 
traffic would be routed out via the default public interface.
- **788b1be: Allow forwarding and collect network stats on any public 
interface.** Forwarding rules and network stats were limited to eth2 on RvR 
networks. This needed to be decoupled from eth2 and reapplied to whichever 
interface was under consideration.
- **b19e8aa: Ensure that CONNMARK --restore-mark only appears once.** This 
is a bit of a hack and can do with being improved. The CONNMARK rule was not 
being picked up by the de-duplication logic in CsNetfilter and was being added 
twice. This caused checksum errors on packets traversing NAT.
- **bf285e1: Transition to master state should add all necessary routes.** 
Now that backup routers keep their interfaces down, the route logic executed at 
configuration-time cannot be applied. Instead, once the interface is brought up 
during a transition to master, routers must re-evaluate what routes are needed 
and add them. Unfortunately I couldn't see a way to re-use the existing route 
logic with the variables that I had in scope so there is some duplication. In 
some cases, routers did not successfully arp IPs away from the old master so 
some arp logic was added. During a failover most connections with guest VMs 
will be maintained with only minor packet loss. SSH sessions implemented via 
port-forwarding rules on an interface other than the source-NAT interface 
consistently get dropped, however, so the failover isn't quite seamless. It's 
possible that there's an easy fix for that.

I expect that a number of tests may need to be modified/written as part of 
this PR. Any feedback or pointers would be useful as initially I'll be relying 
on the CI failures to tell me where to look.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dsclose/cloudstack CLOUDSTACK-9339

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1519.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1519


commit e7a63be161bdd14c985a8b483bffe4bfdaa3f5d4
Author: dean.close 
Date:   2016-05-09T10:31:26Z

CLOUDSTACK-9339: Handle multiple public subnets on virtual routers.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[dsclose]

Github user dsclose closed the pull request at:

https://github.com/apache/cloudstack/pull/1519


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[dsclose]

Github user dsclose closed the pull request at:

https://github.com/apache/cloudstack/pull/1519


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[dsclose]

GitHub user dsclose reopened a pull request:

https://github.com/apache/cloudstack/pull/1519

Cloudstack 9339: Virtual Routers do not handle Multiple Public Interfaces

This PR addresses CLOUDSTACK-9339 and may need a code review from someone 
familiar with the System VM scripts. In particular, this PR has not been tested 
in a VPC RvR context. Only standalone routers and RvR routers have been 
demonstrated.

- **d582358: Leave public interfaces down in backup redundant routers.** 
Previously backup routers were bringing all interfaces up and thus arping 
public IPs away from the master router.
- **9ee1eb6: Add the default gateway to the main routing table when 
interfaces are configured.** The gateway for the first public IP was always 
being added to the main routing table. Sometimes a router would consequently 
add the gateway for an IP other than the default source-NAT IP. This would 
prevent outbound connectivity for guest VMs.
- **ad9d72f: Add default gateway to device-specific routing tables.** 
Link-level routes were being put into the device-specific routing tables 
(accessed via firewall marks) but these are unnecessary. Instead, the default 
gateway is needed to allow the kernel to make an appropriate routing decision.
- **8db879e: Only mark guest connections when they are part of a 
static-NAT.** Guest connections were being marked with a zero. This added no 
functionality and prevented static-NAT rules from routing outbound traffic 
properly as device-specific routing tables would not be used. Instead, all 
traffic would be routed out via the default public interface.
- **788b1be: Allow forwarding and collect network stats on any public 
interface.** Forwarding rules and network stats were limited to eth2 on RvR 
networks. This needed to be decoupled from eth2 and reapplied to whichever 
interface was under consideration.
- **b19e8aa: Ensure that CONNMARK --restore-mark only appears once.** This 
is a bit of a hack and can do with being improved. The CONNMARK rule was not 
being picked up by the de-duplication logic in CsNetfilter and was being added 
twice. This caused checksum errors on packets traversing NAT.
- **bf285e1: Transition to master state should add all necessary routes.** 
Now that backup routers keep their interfaces down, the route logic executed at 
configuration-time cannot be applied. Instead, once the interface is brought up 
during a transition to master, routers must re-evaluate what routes are needed 
and add them. Unfortunately I couldn't see a way to re-use the existing route 
logic with the variables that I had in scope so there is some duplication. In 
some cases, routers did not successfully arp IPs away from the old master so 
some arp logic was added. During a failover most connections with guest VMs 
will be maintained with only minor packet loss. SSH sessions implemented via 
port-forwarding rules on an interface other than the source-NAT interface 
consistently get dropped, however, so the failover isn't quite seamless. It's 
possible that there's an easy fix for that.

I expect that a number of tests may need to be modified/written as part of 
this PR. Any feedback or pointers would be useful as initially I'll be relying 
on the CI failures to tell me where to look.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dsclose/cloudstack CLOUDSTACK-9339

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1519.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1519


commit e7a63be161bdd14c985a8b483bffe4bfdaa3f5d4
Author: dean.close 
Date:   2016-05-09T10:31:26Z

CLOUDSTACK-9339: Handle multiple public subnets on virtual routers.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[dsclose]

Github user dsclose commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1519#discussion_r64537215
  
--- Diff: systemvm/patches/debian/config/opt/cloud/bin/cs/CsAddress.py ---
@@ -288,14 +290,16 @@ def post_configure(self, address):
 """ The steps that must be done after a device is configured """
 route = CsRoute()
 if not self.get_type() in ["control"]:
-route.add_table(self.dev)
 
-CsRule(self.dev).addMark()
+if self.dev != 'eth0':
+route.add_table(self.dev)
+CsRule(self.dev).addMark()
+self.set_mark()
 
 interfaces = [CsInterface(address, self.config)]
 CsHelper.reconfigure_interfaces(self.cl, interfaces)
 
--- End diff --

Lines 299 and 300 were added as part of c41edc1. I've not investigated what 
those lines do, I've just naively overcome the merge conflicts. I'll look at 
the results of CI and follow from there.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[dsclose]

Github user dsclose commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-221501553
  
@swill taking a look now.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-221278559
  
@dsclose if you can fix the merge conflicts I can run this again and see 
what is outstanding.  Thanks...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[luhaijiao]

Github user luhaijiao commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-221202635
  
@dclose I think this PR is important for RvR environment too,  particularly 
to the commits d582358 and bf285e1.  Would be very nice to have it in 4.9 if 
you can review the CI results and fix the issues.

thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-218842740
  
@dsclose we have merge conflicts on this one now.  Also, prior to merging 
the PRs that caused the conflicts, I ran the above CI.  You will probably want 
to review the results of that CI run to fix some things.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-218842214
  


### CI RESULTS

```
Tests Run: 85
  Skipped: 0
   Failed: 9
   Errors: 10
 Duration: 10h 09m 15s
```

**Summary of the problem(s):**
```
ERROR: Create a redundant VPC with 1 Tier, 1 VM, 1 ACL, 1 PF and test 
Network GC Nics
--
Traceback (most recent call last):
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_vpc_redundant.py", line 
281, in tearDown
raise Exception("Warning: Exception during cleanup : %s" % e)
Exception: Warning: Exception during cleanup : Execute cmd: 
deletenetworkoffering failed, due to: errorCode: 431, errorText:Can't delete 
network offering 22 as its used by 1 networks. To make the network offering 
unavaiable, disable it
--
Additional details in: /tmp/MarvinLogs/test_network_8UJKVV/results.txt
```

```
ERROR: Test iptables default INPUT/FORWARD policies on VPC router
--
Traceback (most recent call last):
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_routers_iptables_default_policy.py",
 line 302, in test_01_single_VPC_iptables_policies
self.entity_manager.do_vpc_test()
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_routers_iptables_default_policy.py",
 line 490, in do_vpc_test
self.check_ssh_into_vm(vm.get_vm(), vm.get_ip())
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_routers_iptables_default_policy.py",
 line 525, in check_ssh_into_vm
raise Exception("Failed to SSH into VM - %s" % 
(public_ip.ipaddress.ipaddress))
Exception: Failed to SSH into VM - 192.168.23.9
--
Additional details in: /tmp/MarvinLogs/test_network_8UJKVV/results.txt
```

```
ERROR: Test redundant router internals
--
Traceback (most recent call last):
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_routers_network_ops.py", 
line 510, in test_03_RVR_Network_check_router_state
zoneid=self.zone.id
  File "/usr/lib/python2.7/site-packages/marvin/lib/base.py", line 2780, in 
create
return Network(apiclient.createNetwork(cmd).__dict__)
  File 
"/usr/lib/python2.7/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py", 
line 1887, in createNetwork
response = self.connection.marvinRequest(command, 
response_type=response, method=method)
  File "/usr/lib/python2.7/site-packages/marvin/cloudstackConnection.py", 
line 379, in marvinRequest
raise e
CloudstackAPIException: Execute cmd: createnetwork failed, due to: 
errorCode: 530, errorText:Failed to implement persistent guest network
--
Additional details in: /tmp/MarvinLogs/test_network_8UJKVV/results.txt
```

```
ERROR: Test to verify access to loadbalancer haproxy admin stats page
--
Traceback (most recent call last):
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_internal_lb.py", line 
763, in test_03_vpc_internallb_haproxy_stats_on_all_interfaces
self.execute_internallb_haproxy_tests(vpc_offering)
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_internal_lb.py", line 
838, in execute_internallb_haproxy_tests
applb.sourceipaddress, self.get_ssh_client(vm, 5), settings)
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_internal_lb.py", line 
497, in get_ssh_client
self.fail("Unable to create ssh connection: " % e)
TypeError: not all arguments converted during string formatting
--
Additional details in: /tmp/MarvinLogs/test_network_8UJKVV/results.txt
```

```
ERROR: Test to verify access to loadbalancer haproxy admin stats page
--
Traceback (most recent call last):
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_internal_lb.py", line 
784, in test_04_rvpc_internallb_haproxy_stats_on_all_interfaces
self.execute_internallb_haproxy_tests(redundant_vpc_offering)
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_internal_lb.py", line 
838, in execute_internallb_haproxy_tests
applb.sourceipaddress, self.get_ssh_client(vm, 5), settings)
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_internal_lb.py", line 
497, in get_ssh_client
  

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[dsclose]

Github user dsclose commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-217495575
  
@swill ok, thanks, good luck with the master.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-217491038
  
There is an issue currently on master which I am trying to get sorted out, 
but there should not be a problem on `4.7`.  It looks like Jenkins may be out 
of diskspace again looking at that error.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[dsclose]

Github user dsclose commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-217490305
  
Same error over at #1519 - something introduced in an earlier commit?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[dsclose]

Github user dsclose commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-217489945
  
I rebased against the latest 4.7 before force pushing. Has an error been 
introduced along the way?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Cloudstack 9339: Virtual Routers do not h...

[dsclose]

Github user dsclose commented on the pull request:

https://github.com/apache/cloudstack/pull/1519#issuecomment-217397801
  
Looks like the build environment isn't sufficient on jenkins-test-a20:

```
[ERROR] Java HotSpot(TM) 64-Bit Server VM warning: Insufficient space for 
shared memory file:
   30583
Try using the -Djava.io.tmpdir= option to select an alternate temp location.
```

Doing another force push.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---