[GitHub] cloudstack pull request: Make the generated json files unique to p...

[remibergsma]

GitHub user remibergsma opened a pull request:

https://github.com/apache/cloudstack/pull/1470

Make the generated json files unique to prevent concurrency issues

The json files now have UUIDs to prevent them from getting overwritten 
before they've been executed. Prevents config to be pushed to the wrong router 
:-s

```
2016-02-25 18:32:23,797 DEBUG [c.c.a.t.Request] 
(AgentManager-Handler-1:null) (logid:) Seq 2-4684025087442026584: Processing:  
{ Ans: , MgmtId: 90520732674657, via: 2, Ver: v1, Flags: 10, 
[{"com.cloud.agent.api.routing.GroupA
nswer":{"results":["null - success: null","null - success: [INFO] 
update_config.py :: Processing incoming file => 
vm_dhcp_entry.json.4ea45061-2efb-4467-8eaa-db3d77fb0a7b\n[INFO] Processing JSON 
file vm_dhcp_entry.json.4ea4506
1-2efb-4467-8eaa-db3d77fb0a7b\n"],"result":true,"wait":0}}] }
```

On the router:
```
2016-02-25 18:32:23,416  merge.py __moveFile:298 Processed file written to 
/var/cache/cloud/processed/vm_dhcp_entry.json.4ea45061-2efb-4467-8eaa-db3d77fb0a7b.gz
```

![screen shot 2016-02-25 at 19 33 
17](https://cloud.githubusercontent.com/assets/1630096/13329886/acdae4fc-dbf6-11e5-8c20-fdc6661f5a0b.png)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/remibergsma/cloudstack 
47_fix_concurrency_issue

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1470.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1470


commit f632215b0493b210201fa7590db00ce64cf25e4a
Author: Miguel Ferreira 
Date:   2016-02-25T13:32:09Z

Make the generated json files unique to prevent concurrency issues

Backported from Cosmic.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[jburwell]

Github user jburwell commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-207082450
  
@remibergsma this looks like a good improvement to processing on the VR.  
Please look through my feedback on #1465 regarding string and logging 
performance.  They are relatively small changes that have a big impact.

Also, is there is way to craft a Marvin test to trigger this issue?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[swill]

Github user swill commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1470#discussion_r59290164
  
--- Diff: 
core/src/com/cloud/agent/resource/virtualnetwork/facade/AbstractConfigItemFacade.java
 ---
@@ -104,13 +109,25 @@ public static AbstractConfigItemFacade 
getInstance(final Class

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[jburwell]

Github user jburwell commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1470#discussion_r59294640
  
--- Diff: 
core/src/com/cloud/agent/resource/virtualnetwork/facade/AbstractConfigItemFacade.java
 ---
@@ -104,13 +109,25 @@ public static AbstractConfigItemFacade 
getInstance(final Class

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[jburwell]

Github user jburwell commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1470#discussion_r59294765
  
--- Diff: 
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
 ---
@@ -292,6 +292,9 @@ public ExecutionResult executeInVR(final String 
routerIp, final String script, f
 if (details == null) {
 details = parser.getLines();
 }
+
+s_logger.debug("Executing script in VR " + script);
--- End diff --

Please wrap in a ``if (s_logger.isDebugEnabled())`` block to avoid 
unnecessary string creation due to pre-method string concatenation when 
``DEBUG`` is not enabled.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[jburwell]

Github user jburwell commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1470#discussion_r59294795
  
--- Diff: 
core/src/com/cloud/agent/resource/virtualnetwork/facade/AbstractConfigItemFacade.java
 ---
@@ -104,13 +109,25 @@ public static AbstractConfigItemFacade 
getInstance(final Class generateConfigItems(final ConfigBase 
configuration) {
 final List cfg = new LinkedList<>();
 
-final ConfigItem configFile = new 
FileConfigItem(VRScripts.CONFIG_PERSIST_LOCATION, destinationFile, 
gson.toJson(configuration));
+final String remoteFilename = 
appendUuidToJsonFiles(destinationFile);
+s_logger.debug("Transformed filename " + destinationFile + " to " 
+ remoteFilename);
--- End diff --

Please wrap in a ``if (s_logger.isDebugEnabled())`` block to avoid 
unnecessary string creation due to pre-method string concatenation when 
``DEBUG`` is not enabled.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[jburwell]

Github user jburwell commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1470#discussion_r59294770
  
--- Diff: 
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
 ---
@@ -300,6 +303,8 @@ public ExecutionResult createFileInVR(final String 
routerIp, final String path,
 final File permKey = new File("/root/.ssh/id_rsa.cloud");
 String error = null;
 
+s_logger.debug("Creating file in VR " + filename);
--- End diff --

Please wrap in a ``if (s_logger.isDebugEnabled())`` block to avoid 
unnecessary string creation due to pre-method string concatenation when 
``DEBUG`` is not enabled.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[rhtyd]

Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-216226822
  
Good improvement, LGTM (just code review)

tag:needlove


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[kiwiflyer]

Github user kiwiflyer commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-218446038
  
Remi,

I'm seeing some issues with this PR, related to the way the VR scripts 
utilize the argv file name for other uses. We spent a fair bit of time trying 
to run down a problem with static routes not being hot plugged. The odd thing 
was, a failover to the backup, or a restart of the network brought up the 
static routes fine (after applying your static route failover PR - thanks for 
that!).

I think the problem is in the way /opt/cloud/bin/update_config.py calls out 
to configure.main, that in turn calls the individual process classes:

def finish_config():
# Converge
returncode = configure.main(sys.argv)
sys.exit(returncode)

sys.argv here, happens in this PRs case to be static_routes.json.[GUID]. 

In configure.py, def main(argv) relies on the argv to be the actual json 
file payload from /etc/cloudstack/, or in the case where you want to configure 
every service, you just specify cmd_line.json. In our case, what we see is that 
/etc/cloudstack/static_routes.json gets correctly replaced with the valid data, 
but the configure method never calls out to CsStaticRoutes here:

if process_file in ["cmd_line.json", "static_routes.json"]:
logging.debug("Configuring static routes")
static_routes = CsStaticRoutes("staticroutes", config)
static_routes.process()

I took a look at this in Cosmic last night and I didn't see any additional 
code within the systemvm scripts to handle this differently, so this may also 
be an issue in Cosmic.

I would suspect, this pretty much affects the hot-plugging of any new 
configuration.

I've learnt a lot about how horrible the VR scripts are in this adventure 
and honestly I think the community needs to work on not just cleaning these up, 
but making them more bulletproof in general. 

I hope this points you in the right direction.

- Si



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-218578631
  


### CI RESULTS

```
Tests Run: 85
  Skipped: 0
   Failed: 24
   Errors: 10
 Duration: 10h 46m 24s
```

**Summary of the problem(s):**
```
ERROR: Create a redundant VPC with 1 Tier, 1 VM, 1 ACL, 1 PF and test 
Network GC Nics
--
Traceback (most recent call last):
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_vpc_redundant.py", line 
281, in tearDown
raise Exception("Warning: Exception during cleanup : %s" % e)
Exception: Warning: Exception during cleanup : Execute cmd: 
deletenetworkoffering failed, due to: errorCode: 431, errorText:Can't delete 
network offering 22 as its used by 1 networks. To make the network offering 
unavaiable, disable it
--
Additional details in: /tmp/MarvinLogs/test_network_JPKNVA/results.txt
```

```
ERROR: Test iptables default INPUT/FORWARD policies on VPC router
--
Traceback (most recent call last):
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_routers_iptables_default_policy.py",
 line 302, in test_01_single_VPC_iptables_policies
self.entity_manager.do_vpc_test()
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_routers_iptables_default_policy.py",
 line 490, in do_vpc_test
self.check_ssh_into_vm(vm.get_vm(), vm.get_ip())
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_routers_iptables_default_policy.py",
 line 525, in check_ssh_into_vm
raise Exception("Failed to SSH into VM - %s" % 
(public_ip.ipaddress.ipaddress))
Exception: Failed to SSH into VM - 192.168.23.9
--
Additional details in: /tmp/MarvinLogs/test_network_JPKNVA/results.txt
```

```
ERROR: test suite for 
--
Traceback (most recent call last):
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/suite.py", line 
228, in run
self.tearDown()
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/suite.py", line 
351, in tearDown
self.teardownContext(ancestor)
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/suite.py", line 
367, in teardownContext
try_run(context, names)
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/util.py", line 471, 
in try_run
return func()
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_router_dhcphosts.py", 
line 169, in tearDownClass
raise Exception("Warning: Exception during cleanup : %s" % e)
Exception: Warning: Exception during cleanup : Job failed: {jobprocstatus : 
0, created : u'2016-05-11T10:18:25+0200', jobresult : {errorcode : 530, 
errortext : u'Failed to delete network'}, cmd : 
u'org.apache.cloudstack.api.command.user.network.DeleteNetworkCmd', userid : 
u'c8508137-1732-11e6-923a-5254001daa61', jobstatus : 2, jobid : 
u'5b50924d-00dc-44dc-b23a-70d11c529b6a', jobresultcode : 530, jobresulttype : 
u'object', jobinstancetype : u'Network', accountid : 
u'c8506953-1732-11e6-923a-5254001daa61'}
--
Additional details in: /tmp/MarvinLogs/test_network_JPKNVA/results.txt
```

```
ERROR: Test to verify access to loadbalancer haproxy admin stats page
--
Traceback (most recent call last):
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_internal_lb.py", line 
763, in test_03_vpc_internallb_haproxy_stats_on_all_interfaces
self.execute_internallb_haproxy_tests(vpc_offering)
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_internal_lb.py", line 
838, in execute_internallb_haproxy_tests
applb.sourceipaddress, self.get_ssh_client(vm, 5), settings)
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_internal_lb.py", line 
497, in get_ssh_client
self.fail("Unable to create ssh connection: " % e)
TypeError: not all arguments converted during string formatting
--
Additional details in: /tmp/MarvinLogs/test_network_JPKNVA/results.txt
```

```
ERROR: Test to verify access to loadbalancer haproxy admin stats page
--
Traceback (most recent call last):
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_internal_lb.py", line 
784, in test_04_rvpc_

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[remibergsma]

Github user remibergsma commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-218595232
  
Worked with @swill and @kiwiflyer and found the issue: one bit was missing 
in the backport from Cosmic. Will look into that tomorrow.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-218648913
  
Thanks for the hard work @remibergsma.  👍 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[remibergsma]

Github user remibergsma commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-219005330
  
@swill @kiwiflyer I run the integration tests over night and they look much 
better now. I only see some test fail in `smoke/test_network.py` that seem to 
have the same root cause. Will look into that tonight.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[remibergsma]

Github user remibergsma commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-219037267
  
Pushed the fix and the tests now pass:

```
[root@cs2 cloudstack]# cat /tmp//MarvinLogs/test_network_7VU0DJ/results.txt 
Test for port forwarding on source NAT ... === TestName: 
test_01_port_fwd_on_src_nat | Status : SUCCESS ===
ok
Test for port forwarding on non source NAT ... === TestName: 
test_02_port_fwd_on_non_src_nat | Status : SUCCESS ===
ok
Test for reboot router ... === TestName: test_reboot_router | Status : 
SUCCESS ===
ok
Test for Router rules for network rules on acquired public IP ... === 
TestName: test_network_rules_acquired_public_ip_1_static_nat_rule | Status : 
SUCCESS ===
ok
Test for Router rules for network rules on acquired public IP ... === 
TestName: test_network_rules_acquired_public_ip_2_nat_rule | Status : SUCCESS 
===
ok
Test for Router rules for network rules on acquired public IP ... === 
TestName: test_network_rules_acquired_public_ip_3_Load_Balancer_Rule | Status : 
SUCCESS ===
ok

--
Ran 6 tests in 1519.164s

OK
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-219074669
  
I will rerun CI on this now.  Thx...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[kiwiflyer]

Github user kiwiflyer commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-219076184
  
We have the fixes pulled in for testing.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-219077134
  
@kiwiflyer thanks, I will run CI against it as well.  It is good to have CI 
posted as well as manual verification.  that gives us the most confidence that 
everything is in a good place.  👍 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[kiwiflyer]

Github user kiwiflyer commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-219137126
  
Hot add static route:


![hot_add_static_route](https://cloud.githubusercontent.com/assets/17278194/15259361/77dad958-1916-11e6-904c-d0ae35b8554a.png)



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[kiwiflyer]

Github user kiwiflyer commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-219137269
  
Add a new ACL, add a rule to it and apply it to a Tier:


![acs_before](https://cloud.githubusercontent.com/assets/17278194/15259376/94178760-1916-11e6-845d-62fc715d0630.png)


![acl_after](https://cloud.githubusercontent.com/assets/17278194/15259378/9766df42-1916-11e6-9cc8-af58d2ffe377.png)




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[kiwiflyer]

Github user kiwiflyer commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-219137332
  
Add a private gw:


![add_pg](https://cloud.githubusercontent.com/assets/17278194/15259388/a3d1da2a-1916-11e6-8077-d9980fc8641a.png)



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[kiwiflyer]

Github user kiwiflyer commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-219137482
  
Kill the Master VR:


![int_before_fo](https://cloud.githubusercontent.com/assets/17278194/15259402/b4659c14-1916-11e6-8169-b397f102.png)


![kill_the_vr](https://cloud.githubusercontent.com/assets/17278194/15259406/b8539466-1916-11e6-9351-b510c8abdf7b.png)


![int_after_fo](https://cloud.githubusercontent.com/assets/17278194/15259409/bc168d10-1916-11e6-9145-e98c4efdcfd8.png)




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[kiwiflyer]

Github user kiwiflyer commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-219137807
  
LGTM based on the manual testing above.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-219413266
  


### CI RESULTS

```
Tests Run: 57
  Skipped: 0
   Failed: 4
   Errors: 14
 Duration: 5h 15m 07s
```

**Summary of the problem(s):**
```
FAIL: Test create, assign, remove of an Internal LB with roundrobin http 
traffic to 3 vm's in a Single VPC
--
Traceback (most recent call last):
  File 
"/data/git/cs2/cloudstack/test/integration/smoke/test_internal_lb.py", line 
599, in test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80
self.execute_internallb_roundrobin_tests(vpc_offering)
  File 
"/data/git/cs2/cloudstack/test/integration/smoke/test_internal_lb.py", line 
680, in execute_internallb_roundrobin_tests
"Failed to assign virtual machine(s) to loadbalancer: %s" % e)
AssertionError: Failed to assign virtual machine(s) to loadbalancer: Job 
failed: {jobprocstatus : 0, created : u'2016-05-13T20:12:50+0200', cmd : 
u'org.apache.cloudstack.api.command.user.loadbalancer.AssignToLoadBalancerRuleCmd',
 userid : u'6c3cdb74-1924-11e6-8d73-5254001daa62', jobstatus : 2, jobid : 
u'ad3c6ab0-4bd4-4420-8041-cc0bf3daad0e', jobresultcode : 530, jobresulttype : 
u'object', jobresult : {errorcode : 530, errortext : u'Failed to assign load 
balancer rule'}, accountid : u'6c3cc409-1924-11e6-8d73-5254001daa62'}
--
Additional details in: /tmp/MarvinLogs/test_network_CAWK92/results.txt
```

```
FAIL: Test create, assign, remove of an Internal LB with roundrobin http 
traffic to 3 vm's in a Redundant VPC
--
Traceback (most recent call last):
  File 
"/data/git/cs2/cloudstack/test/integration/smoke/test_internal_lb.py", line 
617, in test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80
self.execute_internallb_roundrobin_tests(redundant_vpc_offering)
  File 
"/data/git/cs2/cloudstack/test/integration/smoke/test_internal_lb.py", line 
680, in execute_internallb_roundrobin_tests
"Failed to assign virtual machine(s) to loadbalancer: %s" % e)
AssertionError: Failed to assign virtual machine(s) to loadbalancer: Job 
failed: {jobprocstatus : 0, created : u'2016-05-13T20:21:07+0200', cmd : 
u'org.apache.cloudstack.api.command.user.loadbalancer.AssignToLoadBalancerRuleCmd',
 userid : u'6c3cdb74-1924-11e6-8d73-5254001daa62', jobstatus : 2, jobid : 
u'3ad0b5e0-9e4c-4e66-b21d-b1293647dfe7', jobresultcode : 530, jobresulttype : 
u'object', jobresult : {errorcode : 530, errortext : u'Failed to assign load 
balancer rule'}, accountid : u'6c3cc409-1924-11e6-8d73-5254001daa62'}
--
Additional details in: /tmp/MarvinLogs/test_network_CAWK92/results.txt
```

```
FAIL: Test to verify access to loadbalancer haproxy admin stats page
--
Traceback (most recent call last):
  File 
"/data/git/cs2/cloudstack/test/integration/smoke/test_internal_lb.py", line 
763, in test_03_vpc_internallb_haproxy_stats_on_all_interfaces
self.execute_internallb_haproxy_tests(vpc_offering)
  File 
"/data/git/cs2/cloudstack/test/integration/smoke/test_internal_lb.py", line 
829, in execute_internallb_haproxy_tests
"Failed to assign virtual machine(s) to loadbalancer: %s" % e)
AssertionError: Failed to assign virtual machine(s) to loadbalancer: Job 
failed: {jobprocstatus : 0, created : u'2016-05-13T20:30:12+0200', cmd : 
u'org.apache.cloudstack.api.command.user.loadbalancer.AssignToLoadBalancerRuleCmd',
 userid : u'6c3cdb74-1924-11e6-8d73-5254001daa62', jobstatus : 2, jobid : 
u'723f596c-2727-44ef-8c34-166c7d3d5a19', jobresultcode : 530, jobresulttype : 
u'object', jobresult : {errorcode : 530, errortext : u'Failed to assign load 
balancer rule'}, accountid : u'6c3cc409-1924-11e6-8d73-5254001daa62'}
--
Additional details in: /tmp/MarvinLogs/test_network_CAWK92/results.txt
```

```
FAIL: Test to verify access to loadbalancer haproxy admin stats page
--
Traceback (most recent call last):
  File 
"/data/git/cs2/cloudstack/test/integration/smoke/test_internal_lb.py", line 
784, in test_04_rvpc_internallb_haproxy_stats_on_all_interfaces
self.execute_internallb_haproxy_tests(redundant_vpc_offering)
  File 
"/data/git/cs2/cloudstack/test/integration/smoke/test_internal_lb.py", line 
829, in execute_internallb_haproxy_tests
"Failed to assign virtual machine(s) to loadbalancer: %s" % e)
AssertionError: Failed to assign virtual

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-21941
  
As you can see, I am still having a hell of a time with CI on this one.  :(


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[remibergsma]

Github user remibergsma commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-219432167
  
@swill I can only reproduce the loadbalance errors, not the others. When I 
look at the errors I doubt they are related. They work fine for me. But anyway, 
no more time available for me to work on this unfortunately.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-219462622
  
I will run CI against it again today to see if it was an issue with that 
environment.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-219702648
  


### CI RESULTS

```
Tests Run: 170
  Skipped: 0
   Failed: 9
   Errors: 0
 Duration: 8h 52m 46s
```

**Summary of the problem(s):**
```
FAIL: Test create, assign, remove of an Internal LB with roundrobin http 
traffic to 3 vm's in a Single VPC
--
Traceback (most recent call last):
  File 
"/data/git/cs2/cloudstack/test/integration/smoke/test_internal_lb.py", line 
599, in test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80
self.execute_internallb_roundrobin_tests(vpc_offering)
  File 
"/data/git/cs2/cloudstack/test/integration/smoke/test_internal_lb.py", line 
680, in execute_internallb_roundrobin_tests
"Failed to assign virtual machine(s) to loadbalancer: %s" % e)
AssertionError: Failed to assign virtual machine(s) to loadbalancer: Job 
failed: {jobprocstatus : 0, created : u'2016-05-17T07:07:36+0200', cmd : 
u'org.apache.cloudstack.api.command.user.loadbalancer.AssignToLoadBalancerRuleCmd',
 userid : u'ddb2d545-1bb3-11e6-8da1-5254001daa62', jobstatus : 2, jobid : 
u'5bb71bfe-c77b-4196-88d5-a46db7d4e7d0', jobresultcode : 530, jobresulttype : 
u'object', jobresult : {errorcode : 530, errortext : u'Failed to assign load 
balancer rule'}, accountid : u'ddb2a73f-1bb3-11e6-8da1-5254001daa62'}
--
Additional details in: /tmp/MarvinLogs/test_network_ADU7VD/results.txt
```

```
FAIL: Test create, assign, remove of an Internal LB with roundrobin http 
traffic to 3 vm's in a Redundant VPC
--
Traceback (most recent call last):
  File 
"/data/git/cs2/cloudstack/test/integration/smoke/test_internal_lb.py", line 
617, in test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80
self.execute_internallb_roundrobin_tests(redundant_vpc_offering)
  File 
"/data/git/cs2/cloudstack/test/integration/smoke/test_internal_lb.py", line 
680, in execute_internallb_roundrobin_tests
"Failed to assign virtual machine(s) to loadbalancer: %s" % e)
AssertionError: Failed to assign virtual machine(s) to loadbalancer: Job 
failed: {jobprocstatus : 0, created : u'2016-05-17T07:16:03+0200', cmd : 
u'org.apache.cloudstack.api.command.user.loadbalancer.AssignToLoadBalancerRuleCmd',
 userid : u'ddb2d545-1bb3-11e6-8da1-5254001daa62', jobstatus : 2, jobid : 
u'9aca0ddb-6219-4f32-a043-da1368a4d38b', jobresultcode : 530, jobresulttype : 
u'object', jobresult : {errorcode : 530, errortext : u'Failed to assign load 
balancer rule'}, accountid : u'ddb2a73f-1bb3-11e6-8da1-5254001daa62'}
--
Additional details in: /tmp/MarvinLogs/test_network_ADU7VD/results.txt
```

```
FAIL: Test to verify access to loadbalancer haproxy admin stats page
--
Traceback (most recent call last):
  File 
"/data/git/cs2/cloudstack/test/integration/smoke/test_internal_lb.py", line 
763, in test_03_vpc_internallb_haproxy_stats_on_all_interfaces
self.execute_internallb_haproxy_tests(vpc_offering)
  File 
"/data/git/cs2/cloudstack/test/integration/smoke/test_internal_lb.py", line 
829, in execute_internallb_haproxy_tests
"Failed to assign virtual machine(s) to loadbalancer: %s" % e)
AssertionError: Failed to assign virtual machine(s) to loadbalancer: Job 
failed: {jobprocstatus : 0, created : u'2016-05-17T07:24:48+0200', cmd : 
u'org.apache.cloudstack.api.command.user.loadbalancer.AssignToLoadBalancerRuleCmd',
 userid : u'ddb2d545-1bb3-11e6-8da1-5254001daa62', jobstatus : 2, jobid : 
u'fd397573-11af-41e4-9c02-2a78cb2e6dc1', jobresultcode : 530, jobresulttype : 
u'object', jobresult : {errorcode : 530, errortext : u'Failed to assign load 
balancer rule'}, accountid : u'ddb2a73f-1bb3-11e6-8da1-5254001daa62'}
--
Additional details in: /tmp/MarvinLogs/test_network_ADU7VD/results.txt
```

```
FAIL: Test to verify access to loadbalancer haproxy admin stats page
--
Traceback (most recent call last):
  File 
"/data/git/cs2/cloudstack/test/integration/smoke/test_internal_lb.py", line 
784, in test_04_rvpc_internallb_haproxy_stats_on_all_interfaces
self.execute_internallb_haproxy_tests(redundant_vpc_offering)
  File 
"/data/git/cs2/cloudstack/test/integration/smoke/test_internal_lb.py", line 
829, in execute_internallb_haproxy_tests
"Failed to assign virtual machine(s) to loadbalancer: %s" % e)
AssertionError: Failed to assign virtual

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-219702802
  
Everything is duplicated in these results because I ran it a second time to 
see if any of the errors cleared up on a second run.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---