[GitHub] cloudstack pull request: CLOUDSTACK-9299: Out-of-band Management f...

[rhtyd]

Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1502#issuecomment-218666241
  
@jburwell fixed the ProcessRunner issues, please do a final review and LGTM 
or suggest changes. Thanks.

@nvazquez @swill I've fixed two CI issues (Travis and Jenkins issues) in 
this PR as well


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: OSPF: adding dynamically routing capabili...

[abhinandanprateek]

Github user abhinandanprateek commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1371#discussion_r62967326
  
--- Diff: utils/src/main/java/com/cloud/utils/net/cidr/CIDR4.java ---
@@ -0,0 +1,172 @@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+
+package com.cloud.utils.net.cidr;
+
+import java.net.Inet4Address;
+import java.net.Inet6Address;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+
+import org.apache.commons.lang.NotImplementedException;
+import org.apache.log4j.Logger;
+
+import com.cloud.utils.net.NetUtils;
+
+/**
+ */
+public class CIDR4 implements CIDR {
+protected final static Logger s_logger = Logger.getLogger(CIDR4.class);
+
+protected InetAddress baseAddress;
+protected int cidrMask;
+private int addressInt;
+private final int addressEndInt;
+
+protected CIDR4(Inet4Address newaddr, int mask) {
+cidrMask = mask;
+addressInt = ipv4AddressToInt(newaddr);
+int newmask = ipv4PrefixLengthToMask(mask);
+addressInt &= newmask;
+try {
+baseAddress = intToIPv4Address(addressInt);
+} catch (UnknownHostException e) {
+// this should never happen
--- End diff --

Changed it to throw Error.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9299: Out-of-band Management f...

[rhtyd]

Github user rhtyd closed the pull request at:

https://github.com/apache/cloudstack/pull/1502


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9299: Out-of-band Management f...

[rhtyd]

GitHub user rhtyd reopened a pull request:

https://github.com/apache/cloudstack/pull/1502

CLOUDSTACK-9299: Out-of-band Management for CloudStack

Support access to a host’s out-of-band management interface (e.g. IPMI, 
iLO,
DRAC, etc.) to manage host power operations (on/off etc.) and querying 
current
power state in CloudStack.

Given the wide range of out-of-band management interfaces such as iLO and 
iDRA,
the service implementation allows for development of separate drivers as 
plugins.
This feature comes with a ipmitool based driver that uses the
ipmitool (http://linux.die.net/man/1/ipmitool) to communicate with any
out-of-band management interface that support IPMI 2.0.

This feature allows following common use-cases:
- Restarting stalled/failed hosts
- Powering off under-utilised hosts
- Powering on hosts for provisioning or to increase capacity
- Allowing system administrators to see the current power state of the host

For testing this feature, please install `ipmitool` (using yum/apt/brew) 
and `ipmisim`:
https://pypi.python.org/pypi/ipmisim

The default ipmitool location is assumed in /usr/bin, if this is different 
in your env please fix the global setting, see FS for details on various global 
settings.

FS:

https://cwiki.apache.org/confluence/display/CLOUDSTACK/Out-of-band+Management+for+CloudStack

/cc @jburwell @swill @abhinandanprateek @murali-reddy @borisstoyanov

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/shapeblue/cloudstack outofband-master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1502.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1502


commit 07564469e9a2ddd0334e8bdd30deed3ed53c3f09
Author: Rohit Yadav 
Date:   2015-12-28T11:07:03Z

CLOUDSTACK-9299: Out-of-band Management for CloudStack

Support access to a host’s out-of-band management interface (e.g. IPMI, 
iLO,
DRAC, etc.) to manage host power operations (on/off etc.) and querying 
current
power state in CloudStack.

Given the wide range of out-of-band management interfaces such as iLO and 
iDRA,
the service implementation allows for development of separate drivers as 
plugins.
This feature comes with a ipmitool based driver that uses the
ipmitool (http://linux.die.net/man/1/ipmitool) to communicate with any
out-of-band management interface that support IPMI 2.0.

This feature allows following common use-cases:
- Restarting stalled/failed hosts
- Powering off under-utilised hosts
- Powering on hosts for provisioning or to increase capacity
- Allowing system administrators to see the current power state of the host

For testing this feature `ipmisim` can be used:
https://pypi.python.org/pypi/ipmisim

FS:

https://cwiki.apache.org/confluence/display/CLOUDSTACK/Out-of-band+Management+for+CloudStack

Signed-off-by: Rohit Yadav 

commit 4d5e8df2f952c4e0594c8fc4d11181f6d3da3811
Author: Rohit Yadav 
Date:   2016-05-06T00:22:07Z

travis: Use patched version of ipmitool for tests

- For out-of-band management feature (CLOUDSTACK-9299) use patched version 
of
  ipmitool that would work on trusty travis machines
- The ipmitool used is from xenial/16.04 release with patch from RedHat
  https://bugzilla.redhat.com/show_bug.cgi?id=1286035
- Installs ipmitool from xenial repositories to get all the dependencies
  and then install patched deb version
- Skip test if the known failure occurs

Signed-off-by: Rohit Yadav 

commit e122bbfbfcf13dcec6e4848e8a8b0047d060
Author: Rohit Yadav 
Date:   2016-05-11T08:59:22Z

HypervisorUtilsTest: increate timeout to 8seconds

Increases timeout to a larger value to avoid failures in VM environments 
such as
TravisCI.

Signed-off-by: Rohit Yadav 

commit 6135f6d98fa6b363b0468c17a8e713aeb5054437
Author: Rohit Yadav 
Date:   2016-05-12T05:34:37Z

CLOUDSTACK-9378: Fix for #1497

Reorder cleanup items so cleanup won't fail

Signed-off-by: Rohit Yadav 




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9378: Fix for #1497

[rhtyd]

Github user rhtyd commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1539#discussion_r62967100
  
--- Diff: test/integration/smoke/test_list_ids_parameter.py ---
@@ -194,10 +194,7 @@ def setUpClass(cls):
 cls._cleanup = [
 cls.disk_offering,
 cls.account,
-cls.service_offering,
-cls.snapshot_1,
-cls.snapshot_2,
-cls.snapshot_3
+cls.service_offering
--- End diff --

I've tried to fix this, by reordering the items by ownership here: 

https://github.com/apache/cloudstack/pull/1502/commits/6135f6d98fa6b363b0468c17a8e713aeb5054437


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: OSPF: adding dynamically routing capabili...

[abhinandanprateek]

Github user abhinandanprateek commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1371#discussion_r62965518
  
--- Diff: utils/src/main/java/com/cloud/utils/net/NetUtils.java ---
@@ -1159,6 +1181,25 @@ public static boolean validateGuestCidr(final String 
cidr) {
 }
 }
 
+public static boolean validateGuestCidr(final CIDR cidr) throws 
BadCIDRException {
--- End diff --

it is being used by code other than ospf to check for RFC1918 compliance.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Metrics UI - Incorrect metrics data for zone/cluster

[Rashmi Dixit]

Hi,

I came across an issue in the metrics for zone and cluster. I have logged a bug 
for the same. https://issues.apache.org/jira/browse/CLOUDSTACK-9377

To quickly summarize the issue: APIs have a pagesize restriction - the 
default.page.size global setting is used as the max result count - value is 
typically 500. In this case, listClusters and listHosts APIS are used to gather 
metrics data for zone and clusters. If there are 600 hosts, then one call to 
listHosts will only return 500 hosts. The usage data is summed up for 500 
hosts, however average is calculated based the total hosts which is 600.

To correct this, UI should fetch ALL hosts for ALL clusters to get the right 
result. Ideally this kind of calculation should be done at the API level - 
large number of hosts will result in performance degradation. The right way to 
fix this would be to add a new API for metrics calculations at zones/cluster 
level.

@Rohit, since you are the original contributor, can you help me with the 
background on this? Any comments/suggestions on doing this differently are most 
welcome.

Thanks!
Rashmi

Rashmi Dixit
Principal Product Engineer | CloudPlatform | www.accelerite.com




DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Accelerite, a Persistent Systems business. It is intended only for 
the use of the individual or entity to which it is addressed. If you are not 
the intended recipient, you are not authorized to read, retain, copy, print, 
distribute or use this message. If you have received this communication in 
error, please notify the sender and delete all copies of this message. 
Accelerite, a Persistent Systems business does not accept any liability for 
virus infected mails.

[GitHub] cloudstack pull request: OSPF: adding dynamically routing capabili...

[abhinandanprateek]

Github user abhinandanprateek commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1371#discussion_r62963628
  
--- Diff: systemvm/patches/debian/config/opt/cloud/bin/cs/CsQuagga.py ---
@@ -0,0 +1,85 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+import logging
+import os.path
+import re
+from cs.CsDatabag import CsDataBag
+from CsProcess import CsProcess
+from CsFile import CsFile
+import CsHelper
+
+OSPFD_CONF = "/etc/quagga/ospfd.conf"
+OSPFD_CONF_NEW = "/etc/quagga/ospfd.conf.new"
+ZEBRA_CONF = "/etc/quagga/zebra.conf"
+ZEBRA_CONF_NEW = "/etc/quagga/zebra.conf.new"
+
+
+class CsQuagga(CsDataBag):
+""" Manage Load Balancer entries """
+
+def process(self):
+#create quagga config
+logging.debug("CsQuagga: zebra" + str(self.dbag["zebra_config"]))
+logging.debug("CsQuagga: ospfd" + str(self.dbag["ospf_config"]))
+
+zebra_config = self.dbag["zebra_config"].split(',')
+ospf_config = self.dbag["ospf_config"].split(',')
+changed = False
+#process zebra
+zebra_conf_new = CsFile(ZEBRA_CONF_NEW)
+zebra_conf_new.empty()
+for x in zebra_config:
+[zebra_conf_new.append(w, -1) for w in x.split('\n')]
+
+zebra_conf_new.commit()
+zebra_conf = CsFile(ZEBRA_CONF)
+if not zebra_conf.compare(zebra_conf_new):
+CsHelper.copy(ZEBRA_CONF_NEW, ZEBRA_CONF)
+changed = True
+
+#process ospfd
+ospfd_conf_new = CsFile(OSPFD_CONF_NEW)
+ospfd_conf_new.empty()
+for x in ospf_config:
+[ospfd_conf_new.append(w, -1) for w in x.split('\n')]
+
+ospfd_conf_new.commit()
+ospfd_conf = CsFile(OSPFD_CONF)
+if not ospfd_conf.compare(ospfd_conf_new):
+CsHelper.copy(OSPFD_CONF_NEW, OSPFD_CONF)
+changed = True
+
+if changed:
+#reset zebra and quagga
+proc = CsProcess(['/var/run/quagga.pid'])
+logging.debug("CsQuagga: Resetted quagga ")
+if not proc.find():
+logging.debug("CsQuagga:: will restart Quagga!")
+CsHelper.service("quagga", "restart")
+else:
+logging.debug("CsQuagga:: will reload Quagga!")
+CsHelper.service("quagga", "reload")
--- End diff --

In debian quagga service will restart both ospfd and zebra.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9378: Fix for #1497

[rhtyd]

Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1539#issuecomment-218657103
  
I think one way to cleanup would be to do it case by case basis for each 
test case manually instead of relying on the class level cleanup. Also, it's 
best to avoid using `cls` instead use cleanup for instance (use self.cleanup). 
I personally prefer to avoid class level setups, if we use instance 
`self.stuff` -- we know things are going to be created and cleaned-up for each 
test case.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9348: Use non-blocking SSL han...

[rhtyd]

Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1493#issuecomment-218656836
  
@kiwiflyer along with this patch, have you also applied the optimization 
patch #1534 (make sure to apply the patch on both mgmt server and KVM agent) ? 
We can debug your issue to see if it's caused by NioConnection by changing 
log4j settings to `TRACE`. Edit `/etc/cloudstack/agent/log4j-cloud.xml`, and 
add a category with name="com.cloud.utils.nio" and priority value="TRACE" and 
watch out for issues in the logs. Alternatively, you have enable DEBUG/TRACE on 
com.cloud and org.apache.cloudstack categories to see what's happening. 
Finally, if you've an IDE or jdb you can attach a debugger to the agent with 
JAVA_OPTS settings `-Xdebug 
-Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n` and connect on 
remote port 8787 (or any other custom port). Let me know if that helps, or you 
need more help in finding out the issue.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: 4.9/master bountycastle changes

[rhtyd]

Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1511#issuecomment-218656345
  
@DaanHoogland done


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: 4.9/master bountycastle changes

[rhtyd]

Github user rhtyd closed the pull request at:

https://github.com/apache/cloudstack/pull/1511


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: 4.9/master bountycastle changes

[rhtyd]

GitHub user rhtyd reopened a pull request:

https://github.com/apache/cloudstack/pull/1511

4.9/master bountycastle changes

This PR adds bountycastle related version and code fixes from @DaanHoogland 
on top of https://github.com/apache/cloudstack/pull/1510

I could not fix java compilation issues when using version 1.54 with 
openjdk 1.7.0_99, @DaanHoogland can you take over? The branch in on asf remote 
so you can push/pull as necessary. Thanks.

cc @swill 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/cloudstack 4.9-bountycastle-daan

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1511.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1511


commit 2d82198acad1cab0755cfc18bdbc5ba7ee8f66ca
Author: Daan Hoogland 
Date:   2016-04-22T10:58:33Z

maven: upgrade bounty castle version to 1.54

Upgrades bountycastle dependency version to bcprov-15on v1.54

commit 4817cb6ab1ed94a2a6c056b6d09f8219a98c04a1
Author: Daan Hoogland 
Date:   2016-04-25T12:05:40Z

use more safe getCertificate(s) call




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: OSPF: adding dynamically routing capabili...

[abhinandanprateek]

Github user abhinandanprateek commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1371#discussion_r62962859
  
--- Diff: 
server/src/org/apache/cloudstack/network/topology/BasicNetworkVisitor.java ---
@@ -316,4 +317,9 @@ public boolean visit(final StaticRoutesRules 
staticRoutesRules) throws ResourceU
 public boolean visit(final AdvancedVpnRules vpnRules) throws 
ResourceUnavailableException {
 throw new CloudRuntimeException("AdvancedVpnRules not implemented 
in Basic Network Topology.");
 }
+
+@Override
+public boolean visit(final QuaggaRules vpnRules) throws 
ResourceUnavailableException {
+throw new CloudRuntimeException("AdvancedVpnRules not implemented 
in Basic Network Topology.");
--- End diff --

CloudRuntimeException is UI friendly and is thrown to pop up the error 
message on CS UI. It is also consistent with other similar methods. I think if 
we find strong reasons to change it we should do it for all the other similar 
methods too.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9378: Fix for #1497

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1539#issuecomment-218654451
  
Sorry for the delay getting to this.  I think this probably works in all 
cases, but I suspect it will not clean up correctly always.  Because this was 
working sometimes and failing sometimes, I believe that means that the cleanup 
of `account` does not always clean up the snapshots (otherwise it would have 
never worked).

I think the code should be like this to make sure it cleans up everything 
up and does not fail.

```
cls._cleanup = [
cls.snapshot_1,
cls.snapshot_2,
cls.snapshot_3,
cls.disk_offering,
cls.account,
cls.service_offering
]
```

Does this make sense to you guys?  This way it will remove the snapshots 
first and then it will clean up everything else.  I could be wrong, but since 
the current code ever worked, I think this guarantees that the cleanup of 
everything will happen and the `account` won't error when it tries to clean up. 
 I guess if the account keeps a reference to the snapshot after it is deleted, 
that could cause the account deletion to fail.  Feedback please...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9378: Fix for #1497

[rhtyd]

Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1539#issuecomment-218652190
  
LGTM, cc @swill let's merge this first to avoid other PRs failing


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8715: Add VirtIO channel to al...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/985#issuecomment-218651517
  
Does this require a change to the system VM template?  I am pulling this in 
for testing now...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9287 - Fix unique mac address ...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1483#issuecomment-218649701
  
I need some code review on this one.  Thanks...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Apply static routes on change to master s...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1472#issuecomment-218649198
  
Thanks gents.  I will get this merged...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Handle private gateways more reliably

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1474#issuecomment-218649126
  
Thank you sir.  I will get this merged...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-218648913
  
Thanks for the hard work @remibergsma.  👍 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Remove extraneous log directory and add c...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1537#issuecomment-218648825
  
This one is ready other than Jenkins.  @PaulAngus, can you please either 
force push the PR or close and reopen it so it kicks off Jenkins so we can get 
this PR all green.  It is ready to merge otherwise...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-6975: Prevent dnsmasq from sta...

[kiwiflyer]

Github user kiwiflyer commented on the pull request:

https://github.com/apache/cloudstack/pull/1514#issuecomment-218611825
  
LGTM. Tested in hardware lab and works as designed.

tag:mergeready


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Handle private gateways more reliably

[kiwiflyer]

Github user kiwiflyer commented on the pull request:

https://github.com/apache/cloudstack/pull/1474#issuecomment-218607576
  
LGTM. Tested in a hardware lab. 

tag:mergeready


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Apply static routes on change to master s...

[kiwiflyer]

Github user kiwiflyer commented on the pull request:

https://github.com/apache/cloudstack/pull/1472#issuecomment-218607629
  
tag:mergeready


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Apply static routes on change to master s...

[dmabry]

Github user dmabry commented on the pull request:

https://github.com/apache/cloudstack/pull/1472#issuecomment-218600092
  
LGTM.  We tested this in our Lab on HW and all static routes on each PG 
came up as expected.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9373: Class methods over-shawd...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1528#issuecomment-218597484
  
I think this one is good to go...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Apply static routes on change to master s...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1472#issuecomment-218597071
  
@rafaelweingartner yes, it will be forward merged.  Sorry I only just got 
around to answering your question.  I need one more code review on this one.  
Thanks...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[remibergsma]

Github user remibergsma commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-218595232
  
Worked with @swill and @kiwiflyer and found the issue: one bit was missing 
in the backport from Cosmic. Will look into that tomorrow.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: [CLOUDSTACK-9207] Test to verify restarti...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1304#issuecomment-218596132
  
The additional run of these tests in a new environment gives the same 
results.  I think this needs some work still...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Apply static routes on change to master s...

[kiwiflyer]

Github user kiwiflyer commented on the pull request:

https://github.com/apache/cloudstack/pull/1472#issuecomment-218592243
  
LGTM. Tested in a hardware lab.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Fix Sync of template.properties in Swift

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1331#issuecomment-218591057
  
@syed can you have a look at this one.

I am not sure why, but I am having a lot of trouble with this PR.  I have 
not been able to get a test run to actually works.  I have tried to run CI on 
this 3 times and I always get the following:

```
INFO  [c.c.r.ResourceManagerImpl] (ApiServer-1:ctx-7bf8070e ctx-cdcb3f39) 
(logid:2044d020) Trying to add a new host at http://kvm1 in data center 1
INFO  [o.a.c.f.j.i.AsyncJobManagerImpl] 
(AsyncJobMgr-Heartbeat-1:ctx-412302f9) (logid:5b139200) Begin cleanup expired 
async-jobs
INFO  [o.a.c.f.j.i.AsyncJobManagerImpl] 
(AsyncJobMgr-Heartbeat-1:ctx-412302f9) (logid:5b139200) End cleanup expired 
async-jobs
INFO  [o.a.c.e.o.NetworkOrchestrator] (Network-Scavenger-1:ctx-a52a3147) 
(logid:86cda124) NetworkGarbageCollector uses '10' seconds for GC interval.
INFO  [c.c.h.k.d.LibvirtServerDiscoverer] (ApiServer-1:ctx-7bf8070e 
ctx-cdcb3f39) (logid:2044d020) cloudstack agent setup command failed: 
cloudstack-setup-agent  -m 192.168.22.61 -z 1 -p 1 -c 1 -g 
57acd541-cd37-34ea-af6a-dc6ecd007325 -a --pubNic=cloudbr0 --prvNic=cloudbr1 
--guestNic=cloudbr0 --hypervisor=kvm
WARN  [c.c.r.ResourceManagerImpl] (ApiServer-1:ctx-7bf8070e ctx-cdcb3f39) 
(logid:2044d020) Unable to find the server resources at http://kvm1
INFO  [c.c.u.e.CSExceptionErrorCode] (ApiServer-1:ctx-7bf8070e 
ctx-cdcb3f39) (logid:2044d020) Could not find exception: 
com.cloud.exception.DiscoveryException in error code list for exceptions
WARN  [o.a.c.a.c.a.h.AddHostCmd] (ApiServer-1:ctx-7bf8070e ctx-cdcb3f39) 
(logid:2044d020) Exception: 
com.cloud.exception.DiscoveryException: Unable to add the host
at 
com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:802)
at 
com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:597)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at 
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy159.discoverHosts(Unknown Source)
at 
org.apache.cloudstack.api.command.admin.host.AddHostCmd.execute(AddHostCmd.java:142)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:150)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:698)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:529)
at com.cloud.api.ApiServer.handle(ApiServer.java:434)
at 
org.apache.http.protocol.HttpService.doService(HttpService.java:423)
at 
org.apache.http.protocol.HttpService.handleRequest(HttpService.java:341)
at 
com.cloud.api.ApiServer$WorkerTask.runInContext(ApiServer.java:1236)
at 
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at 
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
INFO  [c.c.a.ApiServer] (ApiServer-1:ctx-7bf8070e ctx-cdcb3f39) 
(logid:2044d020) Unable to add the host
```

The fact that both Jenkins and Travis are failing may also be relevant.  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or 

[GitHub] cloudstack pull request: CLOUDSTACK-8800 : Improved the listVirtua...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1444#issuecomment-218579831
  
I think this one is pretty much ready.  I have one code review on this one. 
 Can I get one more person to look over this for me?  Thanks...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8800 : Improved the listVirtua...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1444#issuecomment-218579019
  


### CI RESULTS

```
Tests Run: 85
  Skipped: 0
   Failed: 1
   Errors: 0
 Duration: 9h 12m 09s
```

**Summary of the problem(s):**
```
FAIL: Test redundant router internals
--
Traceback (most recent call last):
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_routers_network_ops.py", 
line 483, in test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
"Attempt to retrieve google.com index page should be successful once 
rule is added!"
AssertionError: Attempt to retrieve google.com index page should be 
successful once rule is added!
--
Additional details in: /tmp/MarvinLogs/test_network_F298R0/results.txt
```



**Associated Uploads**

**`/tmp/MarvinLogs/DeployDataCenter__May_11_2016_00_51_52_QTWNUF:`**
* 
[dc_entries.obj](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1444/tmp/MarvinLogs/DeployDataCenter__May_11_2016_00_51_52_QTWNUF/dc_entries.obj)
* 
[failed_plus_exceptions.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1444/tmp/MarvinLogs/DeployDataCenter__May_11_2016_00_51_52_QTWNUF/failed_plus_exceptions.txt)
* 
[runinfo.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1444/tmp/MarvinLogs/DeployDataCenter__May_11_2016_00_51_52_QTWNUF/runinfo.txt)

**`/tmp/MarvinLogs/test_network_F298R0:`**
* 
[failed_plus_exceptions.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1444/tmp/MarvinLogs/test_network_F298R0/failed_plus_exceptions.txt)
* 
[results.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1444/tmp/MarvinLogs/test_network_F298R0/results.txt)
* 
[runinfo.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1444/tmp/MarvinLogs/test_network_F298R0/runinfo.txt)

**`/tmp/MarvinLogs/test_vpc_routers_6U8HTX:`**
* 
[failed_plus_exceptions.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1444/tmp/MarvinLogs/test_vpc_routers_6U8HTX/failed_plus_exceptions.txt)
* 
[results.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1444/tmp/MarvinLogs/test_vpc_routers_6U8HTX/results.txt)
* 
[runinfo.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1444/tmp/MarvinLogs/test_vpc_routers_6U8HTX/runinfo.txt)


Uploads will be available until `2016-07-11 02:00:00 +0200 CEST`

*Comment created by [`upr comment`](https://github.com/cloudops/upr).*



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Make the generated json files unique to p...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1470#issuecomment-218578631
  


### CI RESULTS

```
Tests Run: 85
  Skipped: 0
   Failed: 24
   Errors: 10
 Duration: 10h 46m 24s
```

**Summary of the problem(s):**
```
ERROR: Create a redundant VPC with 1 Tier, 1 VM, 1 ACL, 1 PF and test 
Network GC Nics
--
Traceback (most recent call last):
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_vpc_redundant.py", line 
281, in tearDown
raise Exception("Warning: Exception during cleanup : %s" % e)
Exception: Warning: Exception during cleanup : Execute cmd: 
deletenetworkoffering failed, due to: errorCode: 431, errorText:Can't delete 
network offering 22 as its used by 1 networks. To make the network offering 
unavaiable, disable it
--
Additional details in: /tmp/MarvinLogs/test_network_JPKNVA/results.txt
```

```
ERROR: Test iptables default INPUT/FORWARD policies on VPC router
--
Traceback (most recent call last):
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_routers_iptables_default_policy.py",
 line 302, in test_01_single_VPC_iptables_policies
self.entity_manager.do_vpc_test()
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_routers_iptables_default_policy.py",
 line 490, in do_vpc_test
self.check_ssh_into_vm(vm.get_vm(), vm.get_ip())
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_routers_iptables_default_policy.py",
 line 525, in check_ssh_into_vm
raise Exception("Failed to SSH into VM - %s" % 
(public_ip.ipaddress.ipaddress))
Exception: Failed to SSH into VM - 192.168.23.9
--
Additional details in: /tmp/MarvinLogs/test_network_JPKNVA/results.txt
```

```
ERROR: test suite for 
--
Traceback (most recent call last):
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/suite.py", line 
228, in run
self.tearDown()
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/suite.py", line 
351, in tearDown
self.teardownContext(ancestor)
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/suite.py", line 
367, in teardownContext
try_run(context, names)
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/util.py", line 471, 
in try_run
return func()
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_router_dhcphosts.py", 
line 169, in tearDownClass
raise Exception("Warning: Exception during cleanup : %s" % e)
Exception: Warning: Exception during cleanup : Job failed: {jobprocstatus : 
0, created : u'2016-05-11T10:18:25+0200', jobresult : {errorcode : 530, 
errortext : u'Failed to delete network'}, cmd : 
u'org.apache.cloudstack.api.command.user.network.DeleteNetworkCmd', userid : 
u'c8508137-1732-11e6-923a-5254001daa61', jobstatus : 2, jobid : 
u'5b50924d-00dc-44dc-b23a-70d11c529b6a', jobresultcode : 530, jobresulttype : 
u'object', jobinstancetype : u'Network', accountid : 
u'c8506953-1732-11e6-923a-5254001daa61'}
--
Additional details in: /tmp/MarvinLogs/test_network_JPKNVA/results.txt
```

```
ERROR: Test to verify access to loadbalancer haproxy admin stats page
--
Traceback (most recent call last):
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_internal_lb.py", line 
763, in test_03_vpc_internallb_haproxy_stats_on_all_interfaces
self.execute_internallb_haproxy_tests(vpc_offering)
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_internal_lb.py", line 
838, in execute_internallb_haproxy_tests
applb.sourceipaddress, self.get_ssh_client(vm, 5), settings)
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_internal_lb.py", line 
497, in get_ssh_client
self.fail("Unable to create ssh connection: " % e)
TypeError: not all arguments converted during string formatting
--
Additional details in: /tmp/MarvinLogs/test_network_JPKNVA/results.txt
```

```
ERROR: Test to verify access to loadbalancer haproxy admin stats page
--
Traceback (most recent call last):
  File 
"/data/git/cs1/cloudstack/test/integration/smoke/test_internal_lb.py", line 
784, in 

[GitHub] cloudstack pull request: Notify listeners when a host has been add...

[mike-tutkowski]

Github user mike-tutkowski commented on the pull request:

https://github.com/apache/cloudstack/pull/816#issuecomment-218553761
  
@swill OK, looks like are OK here.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9265 cleanup around httpclient...

[DaanHoogland]

Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/1385#issuecomment-218544251
  
It was stuck due to another jenkins job running. I am still not sure I saw 
what I saw. when I stopped 1252, shortly after 1253 through 1256 finished.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9299: Out-of-band Management f...

[serg38]

Github user serg38 commented on the pull request:

https://github.com/apache/cloudstack/pull/1502#issuecomment-218543261
  
@swill PR1539 passed Jenkins and Travis. After you merge it should resolve 
the issue in other PRs 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: 4.9/master bountycastle changes

[DaanHoogland]

Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/1511#issuecomment-218542671
  
Good point, I am numb to travis timeouts. RM work remnance :(


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/1403#issuecomment-218540891
  
Did a review, I have some remarks, some questions as well.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62892495
  
--- Diff: server/src/com/cloud/storage/snapshot/SnapshotManagerImpl.java ---
@@ -144,6 +145,8 @@
 @Inject
 SnapshotDataStoreDao _snapshotStoreDao;
 @Inject
+private DataStoreManager _dataStoreMgr;
--- End diff --

there is already a dataStoreMgr. why are two needed? (also underscore names 
are against naming convention)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[jburwell]

Github user jburwell commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62892410
  
--- Diff: core/src/com/cloud/agent/api/StartupRoutingCommand.java ---
@@ -35,7 +35,7 @@
 long memory;
 long dom0MinMemory;
 boolean poolSync;
-
+private boolean _supportsClonedVolumes;
--- End diff --

+1 as well.  Ridding of odd Hungarian notation and "_" would be a very 
things for the codebase.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[jburwell]

Github user jburwell commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62892193
  
--- Diff: 
engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/StorageSystemDataMotionStrategy.java
 ---
@@ -172,78 +232,211 @@ private void validate(SnapshotInfo snapshotInfo) {
 }
 }
 
-private Void handleCreateTemplateFromSnapshot(SnapshotInfo 
snapshotInfo, TemplateInfo templateInfo, 
AsyncCompletionCallback callback) {
+private boolean usingBackendSnapshotFor(SnapshotInfo snapshotInfo) {
+String property = getProperty(snapshotInfo.getId(), 
"takeSnapshot");
+
+return Boolean.parseBoolean(property);
+}
+
+private void handleCreateTemplateFromSnapshot(SnapshotInfo 
snapshotInfo, TemplateInfo templateInfo, 
AsyncCompletionCallback callback) {
 try {
 snapshotInfo.processEvent(Event.CopyingRequested);
 }
 catch (Exception ex) {
 throw new CloudRuntimeException("This snapshot is not 
currently in a state where it can be used to create a template.");
 }
 
-HostVO hostVO = getHost(snapshotInfo.getDataStore().getId());
-DataStore srcDataStore = snapshotInfo.getDataStore();
-
-String value = 
_configDao.getValue(Config.PrimaryStorageDownloadWait.toString());
-int primaryStorageDownloadWait = NumbersUtil.parseInt(value, 
Integer.parseInt(Config.PrimaryStorageDownloadWait.getDefaultValue()));
-CopyCommand copyCommand = new CopyCommand(snapshotInfo.getTO(), 
templateInfo.getTO(), primaryStorageDownloadWait, 
VirtualMachineManager.ExecuteInSequence.value());
+HostVO hostVO = getHost(snapshotInfo);
 
-String errMsg = null;
+boolean usingBackendSnapshot = 
usingBackendSnapshotFor(snapshotInfo);
+boolean computeClusterSupportsResign = 
computeClusterSupportsResign(hostVO.getClusterId());
 
-CopyCmdAnswer copyCmdAnswer = null;
+if (usingBackendSnapshot && !computeClusterSupportsResign) {
+throw new CloudRuntimeException("Unable to locate an 
applicable host with which to perform a resignature operation");
+}
 
 try {
-_volumeService.grantAccess(snapshotInfo, hostVO, srcDataStore);
+if (usingBackendSnapshot) {
+createVolumeFromSnapshot(hostVO, snapshotInfo, true);
+}
 
-Map srcDetails = 
getSnapshotDetails(_storagePoolDao.findById(srcDataStore.getId()), 
snapshotInfo);
+DataStore srcDataStore = snapshotInfo.getDataStore();
 
-copyCommand.setOptions(srcDetails);
+String value = 
_configDao.getValue(Config.PrimaryStorageDownloadWait.toString());
+int primaryStorageDownloadWait = NumbersUtil.parseInt(value, 
Integer.parseInt(Config.PrimaryStorageDownloadWait.getDefaultValue()));
+CopyCommand copyCommand = new 
CopyCommand(snapshotInfo.getTO(), templateInfo.getTO(), 
primaryStorageDownloadWait, VirtualMachineManager.ExecuteInSequence.value());
+
+String errMsg = null;
+
+CopyCmdAnswer copyCmdAnswer = null;
 
-copyCmdAnswer = (CopyCmdAnswer)_agentMgr.send(hostVO.getId(), 
copyCommand);
-}
-catch (Exception ex) {
-throw new CloudRuntimeException(ex.getMessage());
-}
-finally {
 try {
-_volumeService.revokeAccess(snapshotInfo, hostVO, 
srcDataStore);
+// If we are using a back-end snapshot, then we should 
still have access to it from the hosts in the cluster that hostVO is in
+// (because we passed in true as the third parameter to 
createVolumeFromSnapshot above).
+if (usingBackendSnapshot == false) {
+_volumeService.grantAccess(snapshotInfo, hostVO, 
srcDataStore);
+}
+
+Map srcDetails = 
getSnapshotDetails(snapshotInfo);
+
+copyCommand.setOptions(srcDetails);
+
+copyCmdAnswer = 
(CopyCmdAnswer)_agentMgr.send(hostVO.getId(), copyCommand);
 }
 catch (Exception ex) {
-s_logger.debug(ex.getMessage(), ex);
+throw new CloudRuntimeException(ex.getMessage());
 }
+finally {
+try {
+_volumeService.revokeAccess(snapshotInfo, hostVO, 
srcDataStore);
+}
+catch (Exception ex) {
+s_logger.debug(ex.getMessage(), ex);
+}
 
-if (copyCmdAnswer == null 

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62891804
  
--- Diff: server/src/com/cloud/storage/StorageManagerImpl.java ---
@@ -209,6 +211,8 @@
 @Inject
 protected HostDao _hostDao;
 @Inject
+private HostDetailsDao _hostDetailsDao;
--- End diff --

unused


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62890825
  
--- Diff: 
plugins/storage/volume/solidfire/src/org/apache/cloudstack/storage/datastore/driver/SolidFirePrimaryDataStoreDriver.java
 ---
@@ -692,12 +1145,38 @@ private String deleteSnapshot(SnapshotInfo 
snapshotInfo, long storagePoolId) {
 _storagePoolDao.update(storagePoolId, storagePool);
 }
 catch (Exception ex) {
-s_logger.debug(SolidFireUtil.LOG_PREFIX + "Failed to delete 
SolidFire volume. CloudStack snapshot ID: " + snapshotId, ex);
+s_logger.debug(SolidFireUtil.LOG_PREFIX + "Issue in 
'deleteSnapshot(SnapshotInfo, long)'. CloudStack snapshot ID: " + csSnapshotId, 
ex);
 
-errMsg = ex.getMessage();
+throw ex;
 }
+}
+
+private void deleteTemplate(TemplateInfo template, long storagePoolId) 
{
+try {
+SolidFireUtil.SolidFireConnection sfConnection = 
SolidFireUtil.getSolidFireConnection(storagePoolId, _storagePoolDetailsDao);
+
+long sfTemplateVolumeId = 
getVolumeIdFrom_iScsiPath(template.getInstallPath());
 
-return errMsg;
+SolidFireUtil.deleteSolidFireVolume(sfConnection, 
sfTemplateVolumeId);
+
+VMTemplateStoragePoolVO templatePoolRef = 
_tmpltPoolDao.findByPoolTemplate(storagePoolId, template.getId());
+
+_tmpltPoolDao.remove(templatePoolRef.getId());
+
+StoragePoolVO storagePool = 
_storagePoolDao.findById(storagePoolId);
+
+// getUsedBytes(StoragePool) will not include the template to 
delete because the "template_spool_ref" table has already been updated by this 
point
+long usedBytes = getUsedBytes(storagePool);
+
+storagePool.setUsedBytes(usedBytes < 0 ? 0 : usedBytes);
+
+_storagePoolDao.update(storagePoolId, storagePool);
+}
+catch (Exception ex) {
--- End diff --

???


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62890766
  
--- Diff: 
plugins/storage/volume/solidfire/src/org/apache/cloudstack/storage/datastore/driver/SolidFirePrimaryDataStoreDriver.java
 ---
@@ -665,22 +950,190 @@ private void updateSnapshotDetails(long 
csSnapshotId, long sfNewVolumeId, long s
 _snapshotDetailsDao.persist(snapshotDetail);
 }
 
-// return null for no error message
-private String deleteSnapshot(SnapshotInfo snapshotInfo, long 
storagePoolId) {
-String errMsg = null;
+private String createVolume(VolumeInfo volumeInfo, long storagePoolId) 
{
+verifySufficientBytesForStoragePool(volumeInfo, storagePoolId);
+verifySufficientIopsForStoragePool(volumeInfo.getMinIops() != null 
? volumeInfo.getMinIops() : getDefaultMinIops(storagePoolId), storagePoolId);
+
+SolidFireUtil.SolidFireConnection sfConnection = 
SolidFireUtil.getSolidFireConnection(storagePoolId, _storagePoolDetailsDao);
+
+long sfAccountId = getCreateSolidFireAccountId(sfConnection, 
volumeInfo.getAccountId(), storagePoolId);
+
+long csSnapshotId = getCsIdForCloning(volumeInfo.getId(), 
"cloneOfSnapshot");
+long csTemplateId = getCsIdForCloning(volumeInfo.getId(), 
"cloneOfTemplate");
+
+SolidFireUtil.SolidFireVolume sfVolume;
+
+if (csSnapshotId > 0) {
+// We are supposed to create a clone of the underlying volume 
or snapshot that supports the CloudStack snapshot.
+sfVolume = createClone(sfConnection, csSnapshotId, volumeInfo, 
sfAccountId, storagePoolId, DataObjectType.SNAPSHOT);
+} else if (csTemplateId > 0) {
+// Clone from template.
+sfVolume = createClone(sfConnection, csTemplateId, volumeInfo, 
sfAccountId, storagePoolId, DataObjectType.TEMPLATE);
+
+long volumeSize = 
getDataObjectSizeIncludingHypervisorSnapshotReserve(volumeInfo, 
_storagePoolDao.findById(storagePoolId));
+
+if (volumeSize > sfVolume.getTotalSize()) {
+// Expand the volume to include HSR.
+SolidFireUtil.modifySolidFireVolume(sfConnection, 
sfVolume.getId(), volumeSize, getVolumeAttributes(volumeInfo),
+sfVolume.getMinIops(), sfVolume.getMaxIops(), 
sfVolume.getBurstIops());
+
+// Get the SolidFire volume from the SAN again because we 
just updated its size.
+sfVolume = SolidFireUtil.getSolidFireVolume(sfConnection, 
sfVolume.getId());
+}
+}
+else {
+sfVolume = createSolidFireVolume(sfConnection, volumeInfo, 
sfAccountId);
+}
+
+String iqn = sfVolume.getIqn();
+
+VolumeVO volume = _volumeDao.findById(volumeInfo.getId());
+
+volume.set_iScsiName(iqn);
+volume.setFolder(String.valueOf(sfVolume.getId()));
+volume.setPoolType(StoragePoolType.IscsiLUN);
+volume.setPoolId(storagePoolId);
+
+_volumeDao.update(volume.getId(), volume);
+
+updateVolumeDetails(volume.getId(), sfVolume.getTotalSize());
+
+StoragePoolVO storagePool = 
_storagePoolDao.findById(storagePoolId);
+
+long capacityBytes = storagePool.getCapacityBytes();
+// getUsedBytes(StoragePool) will include the bytes of the newly 
created volume because
+// updateVolumeDetails(long, long) has already been called for 
this volume
+long usedBytes = getUsedBytes(storagePool);
+
+storagePool.setUsedBytes(usedBytes > capacityBytes ? capacityBytes 
: usedBytes);
+
+_storagePoolDao.update(storagePoolId, storagePool);
+
+return iqn;
+}
+
+private void createTempVolume(SnapshotInfo snapshotInfo, long 
storagePoolId) {
+long csSnapshotId = snapshotInfo.getId();
+
+SnapshotDetailsVO snapshotDetails = 
_snapshotDetailsDao.findDetail(csSnapshotId, SolidFireUtil.SNAPSHOT_ID);
+
+if (snapshotDetails == null || snapshotDetails.getValue() == null) 
{
+throw new 
CloudRuntimeException("'createTempVolume(SnapshotInfo, long)' should not be 
invoked unless " + SolidFireUtil.SNAPSHOT_ID + " exists.");
+}
+
+SolidFireUtil.SolidFireConnection sfConnection = 
SolidFireUtil.getSolidFireConnection(storagePoolId, _storagePoolDetailsDao);
+
+snapshotDetails = _snapshotDetailsDao.findDetail(csSnapshotId, 
"tempVolume");
+
+if (snapshotDetails != null && snapshotDetails.getValue() != null 
&& snapshotDetails.getValue().equalsIgnoreCase("create")) {
+long sfAccountId = 

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62886608
  
--- Diff: 
plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resource/CitrixResourceBase.java
 ---
@@ -2351,17 +2376,52 @@ public SR getIscsiSR(final Connection conn, final 
String srNameLabel, final Stri
 throw new CloudRuntimeException(msg, e);
 }
 }
+
 deviceConfig.put("SCSIid", scsiid);
 
-final String result = SR.probe(conn, host, deviceConfig, 
type, smConfig);
+String result = SR.probe(conn, host, deviceConfig, type, 
smConfig);
+
 String pooluuid = null;
+
 if (result.indexOf("") != -1) {
 pooluuid = result.substring(result.indexOf("") + 
6, result.indexOf("")).trim();
 }
 
 if (pooluuid == null || pooluuid.length() != 36) {
 sr = SR.create(conn, host, deviceConfig, new Long(0), 
srNameLabel, srNameLabel, type, "user", true, smConfig);
-} else {
+}
+else {
+if (resignature) {
--- End diff --

maybe a method createIfResignable()? thsi file is a blob of blobs, let's 
get to reducing the problem called CitrixResourceBase


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62886063
  
--- Diff: 
plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resource/CitrixResourceBase.java
 ---
@@ -1792,10 +1796,26 @@ protected void fillHostInfo(final Connection conn, 
final StartupRoutingCommand c
 cmd.setPod(_pod);
 
cmd.setVersion(CitrixResourceBase.class.getPackage().getImplementationVersion());
 
+try {
+final String cmdLine = "xe sm-list | grep \"resigning of 
duplicates\"";
+
+final XenServerUtilitiesHelper xenServerUtilitiesHelper = 
getXenServerUtilitiesHelper();
+
+Pair result = 
xenServerUtilitiesHelper.executeSshWrapper(_host.getIp(), 22, _username, null, 
getPwdFromQueue(), cmdLine);
+
+boolean supportsClonedVolumes = result != null && 
result.first() != null && result.first() &&
+result.second() != null && 
result.second().length() > 0;
+
+cmd.setSupportsClonedVolumes(supportsClonedVolumes);
+} catch (Exception ex) {
--- End diff --

no checked exceptions are being thrown, why a catch all? is there a 
specific exception that might threaten us?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62885313
  
--- Diff: 
plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resource/CitrixResourceBase.java
 ---
@@ -164,9 +165,12 @@
  *
  */
 public abstract class CitrixResourceBase implements ServerResource, 
HypervisorResource, VirtualRouterDeployer {
-
+/**
+ * RELVMOISCSI = used for resigning metadata (like SR UUID and VDI 
UUID when a
+ * particular storage manager is installed on a XenServer host (for 
back-end snapshots to work))
+ */
--- End diff --

I would put the value javadoc on the value and describe only the whole enum 
here. (PR coming to that end on another enum, shortly)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9348: Use non-blocking SSL han...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1493#issuecomment-218522071
  
No worries.  👍   I am really short on sleep, but I was pretty confident. 
 :P


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9348: Use non-blocking SSL han...

[nvazquez]

Github user nvazquez commented on the pull request:

https://github.com/apache/cloudstack/pull/1493#issuecomment-218520703
  
Sorry, I confused PRs. Please ignore my last comment.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9299: Out-of-band Management f...

[nvazquez]

Github user nvazquez commented on the pull request:

https://github.com/apache/cloudstack/pull/1502#issuecomment-218520519
  
@rhtyd @kiwiflyer @swill a PR for fixing the problem #1539 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9348: Use non-blocking SSL han...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1493#issuecomment-218519246
  
@nvazquez I don't think #1539 is related to this PR.  I think this one has 
completely separate issues...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62881271
  
--- Diff: 
engine/storage/volume/src/org/apache/cloudstack/storage/volume/VolumeServiceImpl.java
 ---
@@ -554,6 +574,51 @@ protected Void 
managedCopyBaseImageCallback(AsyncCallbackDispatcher callback, CreateVolumeContext context) {
+CreateCmdResult result = callback.getResult();
+VolumeApiResult res = new VolumeApiResult(null);
+
+res.setResult(result.getResult());
+
+AsyncCallFuture future = context.getFuture();
+DataObject templateOnPrimaryStoreObj = context.getVolume();
+
+if (result.isSuccess()) {
+
((TemplateObject)templateOnPrimaryStoreObj).setInstallPath(result.getPath());
+
templateOnPrimaryStoreObj.processEvent(Event.OperationSuccessed, 
result.getAnswer());
+}
+else {
+templateOnPrimaryStoreObj.processEvent(Event.OperationFailed);
+}
+
+future.complete(res);
+
+return null;
+}
+
+protected Void 
copyManagedTemplateCallback(AsyncCallbackDispatcher callback, CreateBaseImageContext context) {
--- End diff --

same here, why Void and returning a null? returning null is a bad smell to 
me.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62881152
  
--- Diff: 
engine/storage/volume/src/org/apache/cloudstack/storage/volume/VolumeServiceImpl.java
 ---
@@ -554,6 +574,51 @@ protected Void 
managedCopyBaseImageCallback(AsyncCallbackDispatcher callback, CreateVolumeContext context) {
+CreateCmdResult result = callback.getResult();
+VolumeApiResult res = new VolumeApiResult(null);
+
+res.setResult(result.getResult());
+
+AsyncCallFuture future = context.getFuture();
+DataObject templateOnPrimaryStoreObj = context.getVolume();
+
+if (result.isSuccess()) {
+
((TemplateObject)templateOnPrimaryStoreObj).setInstallPath(result.getPath());
+
templateOnPrimaryStoreObj.processEvent(Event.OperationSuccessed, 
result.getAnswer());
+}
+else {
+templateOnPrimaryStoreObj.processEvent(Event.OperationFailed);
+}
+
+future.complete(res);
+
+return null;
--- End diff --

why Void and return null instead of void and return?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9348: Use non-blocking SSL han...

[nvazquez]

Github user nvazquez commented on the pull request:

https://github.com/apache/cloudstack/pull/1493#issuecomment-218514854
  
@rhtyd @kiwiflyer @swill a PR for fixing the problem #1539 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62880956
  
--- Diff: 
engine/storage/volume/src/org/apache/cloudstack/storage/volume/VolumeServiceImpl.java
 ---
@@ -134,6 +146,14 @@
 EndPointSelector _epSelector;
 @Inject
 HostDao _hostDao;
+@Inject
+private PrimaryDataStoreDao _storagePoolDao;
--- End diff --

not used


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62880949
  
--- Diff: 
engine/storage/volume/src/org/apache/cloudstack/storage/volume/VolumeServiceImpl.java
 ---
@@ -134,6 +146,14 @@
 EndPointSelector _epSelector;
 @Inject
 HostDao _hostDao;
+@Inject
+private PrimaryDataStoreDao _storagePoolDao;
+@Inject
+private HostDetailsDao _hostDetailsDao;
--- End diff --

not used


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62880464
  
--- Diff: 
engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/XenserverSnapshotStrategy.java
 ---
@@ -289,7 +291,7 @@ public boolean deleteSnapshot(Long snapshotId) {
 @Override
 public boolean revertSnapshot(SnapshotInfo snapshot) {
 if (canHandle(snapshot,SnapshotOperation.REVERT) == 
StrategyPriority.CANT_HANDLE) {
-throw new UnsupportedOperationException("Reverting not 
supported. Create a template or volume based on the snapshot instead.");
+throw new CloudRuntimeException("Reverting not supported. 
Create a template or volume based on the snapshot instead.");
--- End diff --

why change the type of the exception?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62880347
  
--- Diff: 
engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/StorageSystemSnapshotStrategy.java
 ---
@@ -347,38 +395,87 @@ private String getProperty(long snapshotId, String 
property) {
 return null;
 }
 
-private HostVO getHost(Long hostId, VolumeVO volumeVO) {
-HostVO hostVO = _hostDao.findById(hostId);
+private HostVO getHost(long volumeId) {
+VolumeVO volumeVO = _volumeDao.findById(volumeId);
+
+Long vmInstanceId = volumeVO.getInstanceId();
+VMInstanceVO vmInstanceVO = _vmInstanceDao.findById(vmInstanceId);
+
+Long hostId = null;
+
+// if the volume to snapshot is associated with a VM
+if (vmInstanceVO != null) {
+hostId = vmInstanceVO.getHostId();
+
+// if the VM is not associated with a host
+if (hostId == null) {
+hostId = vmInstanceVO.getLastHostId();
+}
+}
+
+return getHost(volumeVO.getDataCenterId(), hostId);
+}
+
+private HostVO getHost(long zoneId, Long hostId) {
+HostVO hostVO = getHost(zoneId, true);
 
 if (hostVO != null) {
 return hostVO;
 }
 
-// pick a host in any XenServer cluster that's in the applicable 
zone
+hostVO = _hostDao.findById(hostId);
 
-long zoneId = volumeVO.getDataCenterId();
+if (hostVO != null) {
+return hostVO;
+}
+
+hostVO = getHost(zoneId, false);
 
-List clusters = _mgr.searchForClusters(zoneId, 
new Long(0), Long.MAX_VALUE, HypervisorType.XenServer.toString());
+if (hostVO != null) {
+return hostVO;
+}
+
+throw new CloudRuntimeException("Unable to locate an applicable 
host");
+}
+
+private HostVO getHost(long zoneId, boolean 
computeClusterMustSupportResign) {
+List clusters = _mgr.searchForClusters(zoneId, 
0L, Long.MAX_VALUE, HypervisorType.XenServer.toString());
 
 if (clusters == null) {
-throw new CloudRuntimeException("Unable to locate an 
applicable cluster");
+clusters = new ArrayList<>();
 }
 
+Collections.shuffle(clusters, new Random(System.nanoTime()));
+
+clusters:
 for (Cluster cluster : clusters) {
 if (cluster.getAllocationState() == AllocationState.Enabled) {
 List hosts = 
_hostDao.findByClusterId(cluster.getId());
 
 if (hosts != null) {
+Collections.shuffle(hosts, new 
Random(System.nanoTime()));
+
 for (HostVO host : hosts) {
 if (host.getResourceState() == 
ResourceState.Enabled) {
-return host;
+if (computeClusterMustSupportResign) {
+if 
(_clusterDao.computeClusterSupportsResign(cluster.getId())) {
+return host;
+}
+else {
+// no other host in the cluster in 
question should be able to satisfy our requirements here, so move on to the 
next cluster
+continue clusters;
+}
+}
+else {
+return host;
+}
 }
 }
 }
 }
 }
 
-throw new CloudRuntimeException("Unable to locate an applicable 
cluster");
+return null;
--- End diff --

me no like, maybe a local private checked exception to have it handled in 
the top getHost method?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62879575
  
--- Diff: 
engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/StorageSystemSnapshotStrategy.java
 ---
@@ -347,38 +395,87 @@ private String getProperty(long snapshotId, String 
property) {
 return null;
 }
 
-private HostVO getHost(Long hostId, VolumeVO volumeVO) {
-HostVO hostVO = _hostDao.findById(hostId);
+private HostVO getHost(long volumeId) {
+VolumeVO volumeVO = _volumeDao.findById(volumeId);
+
+Long vmInstanceId = volumeVO.getInstanceId();
+VMInstanceVO vmInstanceVO = _vmInstanceDao.findById(vmInstanceId);
+
+Long hostId = null;
+
+// if the volume to snapshot is associated with a VM
+if (vmInstanceVO != null) {
+hostId = vmInstanceVO.getHostId();
+
+// if the VM is not associated with a host
+if (hostId == null) {
+hostId = vmInstanceVO.getLastHostId();
+}
+}
+
+return getHost(volumeVO.getDataCenterId(), hostId);
+}
+
+private HostVO getHost(long zoneId, Long hostId) {
+HostVO hostVO = getHost(zoneId, true);
 
 if (hostVO != null) {
 return hostVO;
 }
 
-// pick a host in any XenServer cluster that's in the applicable 
zone
+hostVO = _hostDao.findById(hostId);
 
-long zoneId = volumeVO.getDataCenterId();
+if (hostVO != null) {
+return hostVO;
+}
+
+hostVO = getHost(zoneId, false);
 
-List clusters = _mgr.searchForClusters(zoneId, 
new Long(0), Long.MAX_VALUE, HypervisorType.XenServer.toString());
+if (hostVO != null) {
+return hostVO;
+}
+
+throw new CloudRuntimeException("Unable to locate an applicable 
host");
+}
+
+private HostVO getHost(long zoneId, boolean 
computeClusterMustSupportResign) {
+List clusters = _mgr.searchForClusters(zoneId, 
0L, Long.MAX_VALUE, HypervisorType.XenServer.toString());
 
 if (clusters == null) {
-throw new CloudRuntimeException("Unable to locate an 
applicable cluster");
+clusters = new ArrayList<>();
 }
 
+Collections.shuffle(clusters, new Random(System.nanoTime()));
+
+clusters:
 for (Cluster cluster : clusters) {
 if (cluster.getAllocationState() == AllocationState.Enabled) {
 List hosts = 
_hostDao.findByClusterId(cluster.getId());
 
 if (hosts != null) {
+Collections.shuffle(hosts, new 
Random(System.nanoTime()));
+
 for (HostVO host : hosts) {
 if (host.getResourceState() == 
ResourceState.Enabled) {
-return host;
+if (computeClusterMustSupportResign) {
+if 
(_clusterDao.computeClusterSupportsResign(cluster.getId())) {
+return host;
+}
+else {
+// no other host in the cluster in 
question should be able to satisfy our requirements here, so move on to the 
next cluster
--- End diff --

a usefull comment! kudos, that rarely happens ;)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62878912
  
--- Diff: 
engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/StorageSystemSnapshotStrategy.java
 ---
@@ -72,8 +77,10 @@
 private static final Logger s_logger = 
Logger.getLogger(StorageSystemSnapshotStrategy.class);
 
 @Inject private AgentManager _agentMgr;
+@Inject private ClusterDao _clusterDao;
 @Inject private DataStoreManager _dataStoreMgr;
 @Inject private HostDao _hostDao;
+@Inject private HostDetailsDao _hostDetailsDao;
--- End diff --

warning for not used


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62876283
  
--- Diff: 
engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/StorageSystemDataMotionStrategy.java
 ---
@@ -361,59 +603,221 @@ private String getProperty(long snapshotId, String 
property) {
 }
 
 private Map getVolumeDetails(VolumeInfo volumeInfo) {
-Map sourceDetails = new HashMap();
+Map volumeDetails = new HashMap();
 
 VolumeVO volumeVO = _volumeDao.findById(volumeInfo.getId());
 
 long storagePoolId = volumeVO.getPoolId();
 StoragePoolVO storagePoolVO = 
_storagePoolDao.findById(storagePoolId);
 
-sourceDetails.put(DiskTO.STORAGE_HOST, 
storagePoolVO.getHostAddress());
-sourceDetails.put(DiskTO.STORAGE_PORT, 
String.valueOf(storagePoolVO.getPort()));
-sourceDetails.put(DiskTO.IQN, volumeVO.get_iScsiName());
+volumeDetails.put(DiskTO.STORAGE_HOST, 
storagePoolVO.getHostAddress());
+volumeDetails.put(DiskTO.STORAGE_PORT, 
String.valueOf(storagePoolVO.getPort()));
+volumeDetails.put(DiskTO.IQN, volumeVO.get_iScsiName());
 
 ChapInfo chapInfo = _volumeService.getChapInfo(volumeInfo, 
volumeInfo.getDataStore());
 
 if (chapInfo != null) {
-sourceDetails.put(DiskTO.CHAP_INITIATOR_USERNAME, 
chapInfo.getInitiatorUsername());
-sourceDetails.put(DiskTO.CHAP_INITIATOR_SECRET, 
chapInfo.getInitiatorSecret());
-sourceDetails.put(DiskTO.CHAP_TARGET_USERNAME, 
chapInfo.getTargetUsername());
-sourceDetails.put(DiskTO.CHAP_TARGET_SECRET, 
chapInfo.getTargetSecret());
+volumeDetails.put(DiskTO.CHAP_INITIATOR_USERNAME, 
chapInfo.getInitiatorUsername());
+volumeDetails.put(DiskTO.CHAP_INITIATOR_SECRET, 
chapInfo.getInitiatorSecret());
+volumeDetails.put(DiskTO.CHAP_TARGET_USERNAME, 
chapInfo.getTargetUsername());
+volumeDetails.put(DiskTO.CHAP_TARGET_SECRET, 
chapInfo.getTargetSecret());
 }
 
-return sourceDetails;
+return volumeDetails;
+}
+
+private Map getSnapshotDetails(SnapshotInfo 
snapshotInfo) {
+Map snapshotDetails = new HashMap();
+
+long storagePoolId = snapshotInfo.getDataStore().getId();
+StoragePoolVO storagePoolVO = 
_storagePoolDao.findById(storagePoolId);
+
+snapshotDetails.put(DiskTO.STORAGE_HOST, 
storagePoolVO.getHostAddress());
+snapshotDetails.put(DiskTO.STORAGE_PORT, 
String.valueOf(storagePoolVO.getPort()));
+
+long snapshotId = snapshotInfo.getId();
+
+snapshotDetails.put(DiskTO.IQN, getProperty(snapshotId, 
DiskTO.IQN));
+
+snapshotDetails.put(DiskTO.CHAP_INITIATOR_USERNAME, 
getProperty(snapshotId, DiskTO.CHAP_INITIATOR_USERNAME));
+snapshotDetails.put(DiskTO.CHAP_INITIATOR_SECRET, 
getProperty(snapshotId, DiskTO.CHAP_INITIATOR_SECRET));
+snapshotDetails.put(DiskTO.CHAP_TARGET_USERNAME, 
getProperty(snapshotId, DiskTO.CHAP_TARGET_USERNAME));
+snapshotDetails.put(DiskTO.CHAP_TARGET_SECRET, 
getProperty(snapshotId, DiskTO.CHAP_TARGET_SECRET));
+
+return snapshotDetails;
 }
 
-public HostVO getHost(long dataStoreId) {
-StoragePoolVO storagePoolVO = 
_storagePoolDao.findById(dataStoreId);
+private HostVO getHost(SnapshotInfo snapshotInfo) {
+HostVO hostVO = getHost(snapshotInfo.getDataCenterId(), true);
 
-List clusters = 
_mgr.searchForClusters(storagePoolVO.getDataCenterId(), new Long(0), 
Long.MAX_VALUE, HypervisorType.XenServer.toString());
+if (hostVO == null) {
+hostVO = getHost(snapshotInfo.getDataCenterId(), false);
 
-if (clusters == null) {
-throw new CloudRuntimeException("Unable to locate an 
applicable cluster");
+if (hostVO == null) {
+throw new CloudRuntimeException("Unable to locate an 
applicable host in data center with ID = " + snapshotInfo.getDataCenterId());
+}
 }
 
-for (Cluster cluster : clusters) {
-if (cluster.getAllocationState() == AllocationState.Enabled) {
-List hosts = 
_hostDao.findByClusterId(cluster.getId());
+return hostVO;
+}
 
-if (hosts != null) {
-for (HostVO host : hosts) {
-if (host.getResourceState() == 
ResourceState.Enabled) {
-return host;
- 

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62876182
  
--- Diff: 
engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/StorageSystemDataMotionStrategy.java
 ---
@@ -361,59 +603,221 @@ private String getProperty(long snapshotId, String 
property) {
 }
 
 private Map getVolumeDetails(VolumeInfo volumeInfo) {
-Map sourceDetails = new HashMap();
+Map volumeDetails = new HashMap();
 
 VolumeVO volumeVO = _volumeDao.findById(volumeInfo.getId());
 
 long storagePoolId = volumeVO.getPoolId();
 StoragePoolVO storagePoolVO = 
_storagePoolDao.findById(storagePoolId);
 
-sourceDetails.put(DiskTO.STORAGE_HOST, 
storagePoolVO.getHostAddress());
-sourceDetails.put(DiskTO.STORAGE_PORT, 
String.valueOf(storagePoolVO.getPort()));
-sourceDetails.put(DiskTO.IQN, volumeVO.get_iScsiName());
+volumeDetails.put(DiskTO.STORAGE_HOST, 
storagePoolVO.getHostAddress());
+volumeDetails.put(DiskTO.STORAGE_PORT, 
String.valueOf(storagePoolVO.getPort()));
+volumeDetails.put(DiskTO.IQN, volumeVO.get_iScsiName());
 
 ChapInfo chapInfo = _volumeService.getChapInfo(volumeInfo, 
volumeInfo.getDataStore());
 
 if (chapInfo != null) {
-sourceDetails.put(DiskTO.CHAP_INITIATOR_USERNAME, 
chapInfo.getInitiatorUsername());
-sourceDetails.put(DiskTO.CHAP_INITIATOR_SECRET, 
chapInfo.getInitiatorSecret());
-sourceDetails.put(DiskTO.CHAP_TARGET_USERNAME, 
chapInfo.getTargetUsername());
-sourceDetails.put(DiskTO.CHAP_TARGET_SECRET, 
chapInfo.getTargetSecret());
+volumeDetails.put(DiskTO.CHAP_INITIATOR_USERNAME, 
chapInfo.getInitiatorUsername());
+volumeDetails.put(DiskTO.CHAP_INITIATOR_SECRET, 
chapInfo.getInitiatorSecret());
+volumeDetails.put(DiskTO.CHAP_TARGET_USERNAME, 
chapInfo.getTargetUsername());
+volumeDetails.put(DiskTO.CHAP_TARGET_SECRET, 
chapInfo.getTargetSecret());
 }
 
-return sourceDetails;
+return volumeDetails;
+}
+
+private Map getSnapshotDetails(SnapshotInfo 
snapshotInfo) {
+Map snapshotDetails = new HashMap();
+
+long storagePoolId = snapshotInfo.getDataStore().getId();
+StoragePoolVO storagePoolVO = 
_storagePoolDao.findById(storagePoolId);
+
+snapshotDetails.put(DiskTO.STORAGE_HOST, 
storagePoolVO.getHostAddress());
+snapshotDetails.put(DiskTO.STORAGE_PORT, 
String.valueOf(storagePoolVO.getPort()));
+
+long snapshotId = snapshotInfo.getId();
+
+snapshotDetails.put(DiskTO.IQN, getProperty(snapshotId, 
DiskTO.IQN));
+
+snapshotDetails.put(DiskTO.CHAP_INITIATOR_USERNAME, 
getProperty(snapshotId, DiskTO.CHAP_INITIATOR_USERNAME));
+snapshotDetails.put(DiskTO.CHAP_INITIATOR_SECRET, 
getProperty(snapshotId, DiskTO.CHAP_INITIATOR_SECRET));
+snapshotDetails.put(DiskTO.CHAP_TARGET_USERNAME, 
getProperty(snapshotId, DiskTO.CHAP_TARGET_USERNAME));
+snapshotDetails.put(DiskTO.CHAP_TARGET_SECRET, 
getProperty(snapshotId, DiskTO.CHAP_TARGET_SECRET));
+
+return snapshotDetails;
 }
 
-public HostVO getHost(long dataStoreId) {
-StoragePoolVO storagePoolVO = 
_storagePoolDao.findById(dataStoreId);
+private HostVO getHost(SnapshotInfo snapshotInfo) {
+HostVO hostVO = getHost(snapshotInfo.getDataCenterId(), true);
 
-List clusters = 
_mgr.searchForClusters(storagePoolVO.getDataCenterId(), new Long(0), 
Long.MAX_VALUE, HypervisorType.XenServer.toString());
+if (hostVO == null) {
+hostVO = getHost(snapshotInfo.getDataCenterId(), false);
 
-if (clusters == null) {
-throw new CloudRuntimeException("Unable to locate an 
applicable cluster");
+if (hostVO == null) {
+throw new CloudRuntimeException("Unable to locate an 
applicable host in data center with ID = " + snapshotInfo.getDataCenterId());
+}
 }
 
-for (Cluster cluster : clusters) {
-if (cluster.getAllocationState() == AllocationState.Enabled) {
-List hosts = 
_hostDao.findByClusterId(cluster.getId());
+return hostVO;
+}
 
-if (hosts != null) {
-for (HostVO host : hosts) {
-if (host.getResourceState() == 
ResourceState.Enabled) {
-return host;
- 

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62876040
  
--- Diff: 
engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/StorageSystemDataMotionStrategy.java
 ---
@@ -361,59 +608,225 @@ private String getProperty(long snapshotId, String 
property) {
 }
 
 private Map getVolumeDetails(VolumeInfo volumeInfo) {
-Map sourceDetails = new HashMap();
+Map volumeDetails = new HashMap();
 
 VolumeVO volumeVO = _volumeDao.findById(volumeInfo.getId());
 
 long storagePoolId = volumeVO.getPoolId();
 StoragePoolVO storagePoolVO = 
_storagePoolDao.findById(storagePoolId);
 
-sourceDetails.put(DiskTO.STORAGE_HOST, 
storagePoolVO.getHostAddress());
-sourceDetails.put(DiskTO.STORAGE_PORT, 
String.valueOf(storagePoolVO.getPort()));
-sourceDetails.put(DiskTO.IQN, volumeVO.get_iScsiName());
+volumeDetails.put(DiskTO.STORAGE_HOST, 
storagePoolVO.getHostAddress());
+volumeDetails.put(DiskTO.STORAGE_PORT, 
String.valueOf(storagePoolVO.getPort()));
+volumeDetails.put(DiskTO.IQN, volumeVO.get_iScsiName());
 
 ChapInfo chapInfo = _volumeService.getChapInfo(volumeInfo, 
volumeInfo.getDataStore());
 
 if (chapInfo != null) {
-sourceDetails.put(DiskTO.CHAP_INITIATOR_USERNAME, 
chapInfo.getInitiatorUsername());
-sourceDetails.put(DiskTO.CHAP_INITIATOR_SECRET, 
chapInfo.getInitiatorSecret());
-sourceDetails.put(DiskTO.CHAP_TARGET_USERNAME, 
chapInfo.getTargetUsername());
-sourceDetails.put(DiskTO.CHAP_TARGET_SECRET, 
chapInfo.getTargetSecret());
+volumeDetails.put(DiskTO.CHAP_INITIATOR_USERNAME, 
chapInfo.getInitiatorUsername());
+volumeDetails.put(DiskTO.CHAP_INITIATOR_SECRET, 
chapInfo.getInitiatorSecret());
+volumeDetails.put(DiskTO.CHAP_TARGET_USERNAME, 
chapInfo.getTargetUsername());
+volumeDetails.put(DiskTO.CHAP_TARGET_SECRET, 
chapInfo.getTargetSecret());
+}
+
+return volumeDetails;
+}
+
+private Map getSnapshotDetails(SnapshotInfo 
snapshotInfo) {
+Map snapshotDetails = new HashMap();
+
+long storagePoolId = snapshotInfo.getDataStore().getId();
+StoragePoolVO storagePoolVO = 
_storagePoolDao.findById(storagePoolId);
+
+snapshotDetails.put(DiskTO.STORAGE_HOST, 
storagePoolVO.getHostAddress());
+snapshotDetails.put(DiskTO.STORAGE_PORT, 
String.valueOf(storagePoolVO.getPort()));
+
+long snapshotId = snapshotInfo.getId();
+
+snapshotDetails.put(DiskTO.IQN, getProperty(snapshotId, 
DiskTO.IQN));
+
+snapshotDetails.put(DiskTO.CHAP_INITIATOR_USERNAME, 
getProperty(snapshotId, DiskTO.CHAP_INITIATOR_USERNAME));
+snapshotDetails.put(DiskTO.CHAP_INITIATOR_SECRET, 
getProperty(snapshotId, DiskTO.CHAP_INITIATOR_SECRET));
+snapshotDetails.put(DiskTO.CHAP_TARGET_USERNAME, 
getProperty(snapshotId, DiskTO.CHAP_TARGET_USERNAME));
+snapshotDetails.put(DiskTO.CHAP_TARGET_SECRET, 
getProperty(snapshotId, DiskTO.CHAP_TARGET_SECRET));
+
+return snapshotDetails;
+}
+
+private HostVO getHost(SnapshotInfo snapshotInfo) {
+HostVO hostVO = getHost(snapshotInfo.getDataCenterId(), true);
+
+if (hostVO == null) {
+hostVO = getHost(snapshotInfo.getDataCenterId(), false);
+
+if (hostVO == null) {
+throw new CloudRuntimeException("Unable to locate an 
applicable host");
+}
 }
 
-return sourceDetails;
+return hostVO;
 }
 
-public HostVO getHost(long dataStoreId) {
-StoragePoolVO storagePoolVO = 
_storagePoolDao.findById(dataStoreId);
+private HostVO getHost(Long zoneId, boolean 
computeClusterMustSupportResign) {
+if (zoneId == null) {
+throw new CloudRuntimeException("Zone ID cannot be null.");
+}
 
-List clusters = 
_mgr.searchForClusters(storagePoolVO.getDataCenterId(), new Long(0), 
Long.MAX_VALUE, HypervisorType.XenServer.toString());
+List clusters = _mgr.searchForClusters(zoneId, 
new Long(0), Long.MAX_VALUE, HypervisorType.XenServer.toString());
 
 if (clusters == null) {
-throw new CloudRuntimeException("Unable to locate an 
applicable cluster");
+clusters = new ArrayList<>();
 }
 
+Collections.shuffle(clusters, new Random(System.nanoTime()));
+

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62875666
  
--- Diff: 
engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/StorageSystemDataMotionStrategy.java
 ---
@@ -255,99 +473,123 @@ private Void 
handleCreateVolumeFromSnapshotBothOnStorageSystem(SnapshotInfo snap
 
 VolumeApiResult result = future.get();
 
+if (volumeDetail != null) {
+_volumeDetailsDao.remove(volumeDetail.getId());
+}
+
 if (result.isFailed()) {
-s_logger.debug("Failed to create a volume: " + 
result.getResult());
+s_logger.warn("Failed to create a volume: " + 
result.getResult());
 
 throw new CloudRuntimeException(result.getResult());
 }
-}
-catch (Exception ex) {
-throw new CloudRuntimeException(ex.getMessage());
-}
 
-volumeInfo = _volumeDataFactory.getVolume(volumeInfo.getId(), 
volumeInfo.getDataStore());
+volumeInfo = _volumeDataFactory.getVolume(volumeInfo.getId(), 
volumeInfo.getDataStore());
 
-volumeInfo.processEvent(Event.MigrationRequested);
+volumeInfo.processEvent(Event.MigrationRequested);
 
-volumeInfo = _volumeDataFactory.getVolume(volumeInfo.getId(), 
volumeInfo.getDataStore());
+volumeInfo = _volumeDataFactory.getVolume(volumeInfo.getId(), 
volumeInfo.getDataStore());
 
-HostVO hostVO = getHost(snapshotInfo.getDataStore().getId());
-
-String value = 
_configDao.getValue(Config.PrimaryStorageDownloadWait.toString());
-int primaryStorageDownloadWait = NumbersUtil.parseInt(value, 
Integer.parseInt(Config.PrimaryStorageDownloadWait.getDefaultValue()));
-CopyCommand copyCommand = new CopyCommand(snapshotInfo.getTO(), 
volumeInfo.getTO(), primaryStorageDownloadWait, 
VirtualMachineManager.ExecuteInSequence.value());
+if (useCloning) {
+copyCmdAnswer = performResignature(volumeInfo, hostVO);
+}
+else {
+// asking for a XenServer host here so we don't always 
prefer to use XenServer hosts that support resigning
+// even when we don't need those hosts to do this kind of 
copy work
+hostVO = getHost(snapshotInfo.getDataCenterId(), false);
 
-CopyCmdAnswer copyCmdAnswer = null;
+copyCmdAnswer = performCopyOfVdi(volumeInfo, snapshotInfo, 
hostVO);
+}
 
-try {
-_volumeService.grantAccess(snapshotInfo, hostVO, 
snapshotInfo.getDataStore());
-_volumeService.grantAccess(volumeInfo, hostVO, 
volumeInfo.getDataStore());
+if (copyCmdAnswer == null || !copyCmdAnswer.getResult()) {
+if (copyCmdAnswer != null && 
!StringUtils.isEmpty(copyCmdAnswer.getDetails())) {
+errMsg = copyCmdAnswer.getDetails();
+}
+else {
+errMsg = "Unable to create volume from snapshot";
+}
+}
+}
+catch (Exception ex) {
+errMsg = ex.getMessage() != null ? ex.getMessage() : "Copy 
operation failed";
+}
 
-Map srcDetails = 
getSnapshotDetails(_storagePoolDao.findById(snapshotInfo.getDataStore().getId()),
 snapshotInfo);
+CopyCommandResult result = new CopyCommandResult(null, 
copyCmdAnswer);
 
-copyCommand.setOptions(srcDetails);
+result.setResult(errMsg);
 
-Map destDetails = getVolumeDetails(volumeInfo);
+callback.complete(result);
+}
 
-copyCommand.setOptions2(destDetails);
+/**
+ * If the underlying storage system is making use of read-only 
snapshots, this gives the storage system the opportunity to
+ * create a volume from the snapshot so that we can copy the VHD file 
that should be inside of the snapshot to secondary storage.
+ *
+ * The resultant volume must be writable because we need to resign the 
SR and the VDI that should be inside of it before we copy
+ * the VHD file to secondary storage.
+ *
+ * If the storage system is using writable snapshots, then nothing 
need be done by that storage system here because we can just
+ * resign the SR and the VDI that should be inside of the snapshot 
before copying the VHD file to secondary storage.
+ */
+private void createVolumeFromSnapshot(HostVO hostVO, SnapshotInfo 
snapshotInfo, boolean keepGrantedAccess) {
+SnapshotDetailsVO snapshotDetails = 

[GitHub] cloudstack pull request: CLOUDSTACK-9265 cleanup around httpclient...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1385#issuecomment-218505040
  
@DaanHoogland it finished!!!  :P


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Remove extraneous log directory and add c...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1537#issuecomment-218503981
  
I am testing now...  👍 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62875028
  
--- Diff: 
engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/StorageSystemDataMotionStrategy.java
 ---
@@ -172,78 +254,214 @@ private void validate(SnapshotInfo snapshotInfo) {
 }
 }
 
-private Void handleCreateTemplateFromSnapshot(SnapshotInfo 
snapshotInfo, TemplateInfo templateInfo, 
AsyncCompletionCallback callback) {
+private boolean usingBackendSnapshotFor(SnapshotInfo snapshotInfo) {
+String property = getProperty(snapshotInfo.getId(), 
"takeSnapshot");
+
+return Boolean.parseBoolean(property);
+}
+
+private void handleCreateTemplateFromSnapshot(SnapshotInfo 
snapshotInfo, TemplateInfo templateInfo, 
AsyncCompletionCallback callback) {
 try {
 snapshotInfo.processEvent(Event.CopyingRequested);
 }
 catch (Exception ex) {
 throw new CloudRuntimeException("This snapshot is not 
currently in a state where it can be used to create a template.");
 }
 
-HostVO hostVO = getHost(snapshotInfo.getDataStore().getId());
-DataStore srcDataStore = snapshotInfo.getDataStore();
+HostVO hostVO = getHost(snapshotInfo);
 
-String value = 
_configDao.getValue(Config.PrimaryStorageDownloadWait.toString());
-int primaryStorageDownloadWait = NumbersUtil.parseInt(value, 
Integer.parseInt(Config.PrimaryStorageDownloadWait.getDefaultValue()));
-CopyCommand copyCommand = new CopyCommand(snapshotInfo.getTO(), 
templateInfo.getTO(), primaryStorageDownloadWait, 
VirtualMachineManager.ExecuteInSequence.value());
-
-String errMsg = null;
+boolean usingBackendSnapshot = 
usingBackendSnapshotFor(snapshotInfo);
+boolean computeClusterSupportsResign = 
_clusterDao.computeClusterSupportsResign(hostVO.getClusterId());
 
-CopyCmdAnswer copyCmdAnswer = null;
+if (usingBackendSnapshot && !computeClusterSupportsResign) {
+throw new CloudRuntimeException("Unable to locate an 
applicable host with which to perform a resignature operation");
+}
 
 try {
-_volumeService.grantAccess(snapshotInfo, hostVO, srcDataStore);
+if (usingBackendSnapshot) {
+createVolumeFromSnapshot(hostVO, snapshotInfo, true);
+}
 
-Map srcDetails = 
getSnapshotDetails(_storagePoolDao.findById(srcDataStore.getId()), 
snapshotInfo);
+DataStore srcDataStore = snapshotInfo.getDataStore();
 
-copyCommand.setOptions(srcDetails);
+String value = 
_configDao.getValue(Config.PrimaryStorageDownloadWait.toString());
+int primaryStorageDownloadWait = NumbersUtil.parseInt(value, 
Integer.parseInt(Config.PrimaryStorageDownloadWait.getDefaultValue()));
+CopyCommand copyCommand = new 
CopyCommand(snapshotInfo.getTO(), templateInfo.getTO(), 
primaryStorageDownloadWait, VirtualMachineManager.ExecuteInSequence.value());
+
+String errMsg = null;
+
+CopyCmdAnswer copyCmdAnswer = null;
 
-copyCmdAnswer = (CopyCmdAnswer)_agentMgr.send(hostVO.getId(), 
copyCommand);
-}
-catch (Exception ex) {
-throw new CloudRuntimeException(ex.getMessage());
-}
-finally {
 try {
-_volumeService.revokeAccess(snapshotInfo, hostVO, 
srcDataStore);
+// If we are using a back-end snapshot, then we should 
still have access to it from the hosts in the cluster that hostVO is in
+// (because we passed in true as the third parameter to 
createVolumeFromSnapshot above).
+if (usingBackendSnapshot == false) {
+_volumeService.grantAccess(snapshotInfo, hostVO, 
srcDataStore);
+}
+
+Map srcDetails = 
getSnapshotDetails(snapshotInfo);
+
+copyCommand.setOptions(srcDetails);
+
+copyCmdAnswer = 
(CopyCmdAnswer)_agentMgr.send(hostVO.getId(), copyCommand);
 }
 catch (Exception ex) {
-s_logger.debug(ex.getMessage(), ex);
+throw new CloudRuntimeException("Failed to create template 
from snapshot : " + ex.getMessage());
 }
+finally {
+try {
+_volumeService.revokeAccess(snapshotInfo, hostVO, 
srcDataStore);
+}
+catch (Exception ex) {
+s_logger.warn(ex.getMessage(), ex);
+ 

[GitHub] cloudstack pull request: 4.9/master bountycastle changes

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1511#issuecomment-218503116
  
@DaanHoogland well a timeout in combination with a connection is not being 
able to be established seems relevant since it is related to the code that has 
changed. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62874090
  
--- Diff: 
engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/StorageSystemDataMotionStrategy.java
 ---
@@ -172,78 +232,211 @@ private void validate(SnapshotInfo snapshotInfo) {
 }
 }
 
-private Void handleCreateTemplateFromSnapshot(SnapshotInfo 
snapshotInfo, TemplateInfo templateInfo, 
AsyncCompletionCallback callback) {
+private boolean usingBackendSnapshotFor(SnapshotInfo snapshotInfo) {
+String property = getProperty(snapshotInfo.getId(), 
"takeSnapshot");
+
+return Boolean.parseBoolean(property);
+}
+
+private void handleCreateTemplateFromSnapshot(SnapshotInfo 
snapshotInfo, TemplateInfo templateInfo, 
AsyncCompletionCallback callback) {
 try {
 snapshotInfo.processEvent(Event.CopyingRequested);
 }
 catch (Exception ex) {
 throw new CloudRuntimeException("This snapshot is not 
currently in a state where it can be used to create a template.");
 }
 
-HostVO hostVO = getHost(snapshotInfo.getDataStore().getId());
-DataStore srcDataStore = snapshotInfo.getDataStore();
-
-String value = 
_configDao.getValue(Config.PrimaryStorageDownloadWait.toString());
-int primaryStorageDownloadWait = NumbersUtil.parseInt(value, 
Integer.parseInt(Config.PrimaryStorageDownloadWait.getDefaultValue()));
-CopyCommand copyCommand = new CopyCommand(snapshotInfo.getTO(), 
templateInfo.getTO(), primaryStorageDownloadWait, 
VirtualMachineManager.ExecuteInSequence.value());
+HostVO hostVO = getHost(snapshotInfo);
 
-String errMsg = null;
+boolean usingBackendSnapshot = 
usingBackendSnapshotFor(snapshotInfo);
+boolean computeClusterSupportsResign = 
computeClusterSupportsResign(hostVO.getClusterId());
 
-CopyCmdAnswer copyCmdAnswer = null;
+if (usingBackendSnapshot && !computeClusterSupportsResign) {
+throw new CloudRuntimeException("Unable to locate an 
applicable host with which to perform a resignature operation");
+}
 
 try {
-_volumeService.grantAccess(snapshotInfo, hostVO, srcDataStore);
+if (usingBackendSnapshot) {
+createVolumeFromSnapshot(hostVO, snapshotInfo, true);
+}
 
-Map srcDetails = 
getSnapshotDetails(_storagePoolDao.findById(srcDataStore.getId()), 
snapshotInfo);
+DataStore srcDataStore = snapshotInfo.getDataStore();
 
-copyCommand.setOptions(srcDetails);
+String value = 
_configDao.getValue(Config.PrimaryStorageDownloadWait.toString());
+int primaryStorageDownloadWait = NumbersUtil.parseInt(value, 
Integer.parseInt(Config.PrimaryStorageDownloadWait.getDefaultValue()));
+CopyCommand copyCommand = new 
CopyCommand(snapshotInfo.getTO(), templateInfo.getTO(), 
primaryStorageDownloadWait, VirtualMachineManager.ExecuteInSequence.value());
+
+String errMsg = null;
+
+CopyCmdAnswer copyCmdAnswer = null;
 
-copyCmdAnswer = (CopyCmdAnswer)_agentMgr.send(hostVO.getId(), 
copyCommand);
-}
-catch (Exception ex) {
-throw new CloudRuntimeException(ex.getMessage());
-}
-finally {
 try {
-_volumeService.revokeAccess(snapshotInfo, hostVO, 
srcDataStore);
+// If we are using a back-end snapshot, then we should 
still have access to it from the hosts in the cluster that hostVO is in
+// (because we passed in true as the third parameter to 
createVolumeFromSnapshot above).
+if (usingBackendSnapshot == false) {
+_volumeService.grantAccess(snapshotInfo, hostVO, 
srcDataStore);
+}
+
+Map srcDetails = 
getSnapshotDetails(snapshotInfo);
+
+copyCommand.setOptions(srcDetails);
+
+copyCmdAnswer = 
(CopyCmdAnswer)_agentMgr.send(hostVO.getId(), copyCommand);
 }
 catch (Exception ex) {
-s_logger.debug(ex.getMessage(), ex);
+throw new CloudRuntimeException(ex.getMessage());
 }
+finally {
+try {
+_volumeService.revokeAccess(snapshotInfo, hostVO, 
srcDataStore);
+}
+catch (Exception ex) {
+s_logger.debug(ex.getMessage(), ex);
+}
 
-if (copyCmdAnswer == 

Re: Question: commands.properties.in deprecated

[Will Stevens]

I think this is the extent of the documentation at this point.

https://github.com/apache/cloudstack-docs-admin/pull/37/files

Rohit, maybe you can shed some light on this from a developer's perspective
and how we should be handling this going forward?  Are there official docs
which covered how to use 'commands.properties.in'?  If so, we should update
them to reflect this new feature...



*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Wed, May 11, 2016 at 10:40 AM, Tutkowski, Mike  wrote:

> Hi,
>
>
> I see in afcbd27 that we've deprecated commands.properties.in.
>
>
> Is there documentation on the new way that we specify permissions for a
> given API these days?
>
>
> Thanks!
>
> Mike
>

[GitHub] cloudstack pull request: CLOUDSTACK-9351: Add ids parameter to res...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1497#issuecomment-218500189
  
Thank you, I will merge it assuming Jenkins and Travis come back clean.  
Thx... 👍 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9351: Add ids parameter to res...

[nvazquez]

Github user nvazquez commented on the pull request:

https://github.com/apache/cloudstack/pull/1497#issuecomment-218499768
  
Done @swill @serg38 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9378: Fix for #1497

[nvazquez]

GitHub user nvazquez opened a pull request:

https://github.com/apache/cloudstack/pull/1539

CLOUDSTACK-9378: Fix for #1497

JIRA TICKET: https://issues.apache.org/jira/browse/CLOUDSTACK-9378

After #1497 was merged, test_list_ids_parameter.py failed on Travis. It was 
noticed that snapshots could be removed from cleaup

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nvazquez/cloudstack fix1497

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1539.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1539


commit 19a26afdef01d091944f3d279030717075e6570f
Author: nvazquez 
Date:   2016-05-11T15:40:10Z

CLOUDSTACK-9378: Fix for #1497




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62872442
  
--- Diff: 
engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/StorageSystemDataMotionStrategy.java
 ---
@@ -172,78 +232,211 @@ private void validate(SnapshotInfo snapshotInfo) {
 }
 }
 
-private Void handleCreateTemplateFromSnapshot(SnapshotInfo 
snapshotInfo, TemplateInfo templateInfo, 
AsyncCompletionCallback callback) {
+private boolean usingBackendSnapshotFor(SnapshotInfo snapshotInfo) {
+String property = getProperty(snapshotInfo.getId(), 
"takeSnapshot");
+
+return Boolean.parseBoolean(property);
+}
+
+private void handleCreateTemplateFromSnapshot(SnapshotInfo 
snapshotInfo, TemplateInfo templateInfo, 
AsyncCompletionCallback callback) {
 try {
 snapshotInfo.processEvent(Event.CopyingRequested);
 }
 catch (Exception ex) {
 throw new CloudRuntimeException("This snapshot is not 
currently in a state where it can be used to create a template.");
 }
 
-HostVO hostVO = getHost(snapshotInfo.getDataStore().getId());
-DataStore srcDataStore = snapshotInfo.getDataStore();
-
-String value = 
_configDao.getValue(Config.PrimaryStorageDownloadWait.toString());
-int primaryStorageDownloadWait = NumbersUtil.parseInt(value, 
Integer.parseInt(Config.PrimaryStorageDownloadWait.getDefaultValue()));
-CopyCommand copyCommand = new CopyCommand(snapshotInfo.getTO(), 
templateInfo.getTO(), primaryStorageDownloadWait, 
VirtualMachineManager.ExecuteInSequence.value());
+HostVO hostVO = getHost(snapshotInfo);
 
-String errMsg = null;
+boolean usingBackendSnapshot = 
usingBackendSnapshotFor(snapshotInfo);
+boolean computeClusterSupportsResign = 
computeClusterSupportsResign(hostVO.getClusterId());
 
-CopyCmdAnswer copyCmdAnswer = null;
+if (usingBackendSnapshot && !computeClusterSupportsResign) {
+throw new CloudRuntimeException("Unable to locate an 
applicable host with which to perform a resignature operation");
+}
 
 try {
-_volumeService.grantAccess(snapshotInfo, hostVO, srcDataStore);
+if (usingBackendSnapshot) {
+createVolumeFromSnapshot(hostVO, snapshotInfo, true);
+}
 
-Map srcDetails = 
getSnapshotDetails(_storagePoolDao.findById(srcDataStore.getId()), 
snapshotInfo);
+DataStore srcDataStore = snapshotInfo.getDataStore();
 
-copyCommand.setOptions(srcDetails);
+String value = 
_configDao.getValue(Config.PrimaryStorageDownloadWait.toString());
+int primaryStorageDownloadWait = NumbersUtil.parseInt(value, 
Integer.parseInt(Config.PrimaryStorageDownloadWait.getDefaultValue()));
+CopyCommand copyCommand = new 
CopyCommand(snapshotInfo.getTO(), templateInfo.getTO(), 
primaryStorageDownloadWait, VirtualMachineManager.ExecuteInSequence.value());
+
+String errMsg = null;
+
+CopyCmdAnswer copyCmdAnswer = null;
 
-copyCmdAnswer = (CopyCmdAnswer)_agentMgr.send(hostVO.getId(), 
copyCommand);
-}
-catch (Exception ex) {
-throw new CloudRuntimeException(ex.getMessage());
-}
-finally {
 try {
--- End diff --

I can not see what Rafael means. I think the code has changed since he made 
the comment. I do see this method as growing out of proportion so some 
refactorring for readability and maintainability seems in order.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62868269
  
--- Diff: engine/schema/src/com/cloud/dc/dao/ClusterDaoImpl.java ---
@@ -260,4 +268,41 @@ public boolean remove(Long id) {
 sc.setParameters("dataCenterId", zoneId);
 return customSearch(sc, null);
 }
+
+@Override
+public boolean computeClusterSupportsResign(long clusterId) {
--- End diff --

sugested name change: computeWhetherClusterSupportsResign or even 
computeWhetherClusterSupportsResigning


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Taking fast and efficient volume snapshot...

[DaanHoogland]

Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1403#discussion_r62867309
  
--- Diff: 
engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/PrimaryDataStoreInfo.java
 ---
@@ -36,6 +36,7 @@
 static final String CHAP_INITIATOR_SECRET = "chapInitiatorSecret";
 static final String CHAP_TARGET_USERNAME = "chapTargetUsername";
 static final String CHAP_TARGET_SECRET = "chapTargetSecret";
+static final String REMOVE_AFTER_COPY = "removeAfterCopy";
--- End diff --

not lethal but these strings seem to warrant an enum as well, do they?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Notify listeners when a host has been add...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/816#issuecomment-218489553
  
I think we are definitely having issues with it not cleaning correctly.  I 
have seen a couple cases where we get the "unable to update index" when it 
tries to do a git checkout because of problems with cleanup, so that is 
possible.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9351: Add ids parameter to res...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1497#issuecomment-218489998
  
Can you create a new PR to fix this and we will get it tested right away.  
Thanks for being on top of this.  👍 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: [CLOUDSTACK-9207] Test to verify restarti...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1304#issuecomment-218483982
  
The added test fails, but lots of tests failed.  I will rebuild my 
environment and test again to see if anything changes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: [CLOUDSTACK-9207] Test to verify restarti...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1304#issuecomment-218483641
  


### CI RESULTS

```
Tests Run: 7
  Skipped: 0
   Failed: 4
   Errors: 0
 Duration: 0h 37m 50s
```

**Summary of the problem(s):**
```
FAIL: Test restart network with cleanup
--
Traceback (most recent call last):
  File 
"/data/git/cs2/cloudstack/test/integration/component/test_vpc_network.py", line 
2851, in test_restart_network_with_cleanup
self.validateResult(result)
  File 
"/data/git/cs2/cloudstack/test/integration/component/test_vpc_network.py", line 
2749, in validateResult
self.fail("IP is missing from vpcVR")
AssertionError: IP is missing from vpcVR
--
Additional details in: /tmp/MarvinLogs/test_vpc_network_14GQ49/results.txt
```

```
FAIL: Test network rules after starting a VpcVr that
--
Traceback (most recent call last):
  File 
"/data/git/cs2/cloudstack/test/integration/component/test_vpc_network.py", line 
2465, in test_02_start_vm_network_gc
"Ping to outside world from VM should be successful"
AssertionError: Ping to outside world from VM should be successful
--
Additional details in: /tmp/MarvinLogs/test_vpc_network_14GQ49/results.txt
```

```
FAIL: Test Stop all the Vms that are part of the a Network
--
Traceback (most recent call last):
  File 
"/data/git/cs2/cloudstack/test/integration/component/test_vpc_network.py", line 
2557, in test_03_restart_vpcvr
(self.public_ip_1.ipaddress.ipaddress, e))
AssertionError: Failed to SSH into VM - 192.168.23.54, SSH connection has 
Failed. Waited 600s. Error is SSH Connection Failed
--
Additional details in: /tmp/MarvinLogs/test_vpc_network_14GQ49/results.txt
```

```
FAIL: Test update Network that is part of a VPC to a network
--
Traceback (most recent call last):
  File 
"/data/git/cs2/cloudstack/test/integration/component/test_vpc_network.py", line 
1910, in test_01_network_services_upgrade
"Ping to outside world from VM should be successful"
AssertionError: Ping to outside world from VM should be successful
--
Additional details in: /tmp/MarvinLogs/test_vpc_network_14GQ49/results.txt
```



**Associated Uploads**

**`/tmp/MarvinLogs/DeployDataCenter__May_11_2016_08_32_17_9O5EIE:`**
* 
[dc_entries.obj](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1304/tmp/MarvinLogs/DeployDataCenter__May_11_2016_08_32_17_9O5EIE/dc_entries.obj)
* 
[failed_plus_exceptions.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1304/tmp/MarvinLogs/DeployDataCenter__May_11_2016_08_32_17_9O5EIE/failed_plus_exceptions.txt)
* 
[runinfo.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1304/tmp/MarvinLogs/DeployDataCenter__May_11_2016_08_32_17_9O5EIE/runinfo.txt)

**`/tmp/MarvinLogs/test_vpc_network_14GQ49:`**
* 
[failed_plus_exceptions.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1304/tmp/MarvinLogs/test_vpc_network_14GQ49/failed_plus_exceptions.txt)
* 
[results.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1304/tmp/MarvinLogs/test_vpc_network_14GQ49/results.txt)
* 
[runinfo.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1304/tmp/MarvinLogs/test_vpc_network_14GQ49/runinfo.txt)


Uploads will be available until `2016-07-11 02:00:00 +0200 CEST`

*Comment created by [`upr comment`](https://github.com/cloudops/upr).*



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9351: Add ids parameter to res...

[serg38]

Github user serg38 commented on the pull request:

https://github.com/apache/cloudstack/pull/1497#issuecomment-218480013
  
The issue that it couldn't find a snaphsot because it is already cleaned by 
account cleanup.
 +cls._cleanup = [
 +cls.disk_offering,
 +cls.account,
 +cls.service_offering,
 +cls.snapshot_1,
 +cls.snapshot_2,
 +cls.snapshot_3
 +]
 +

Snapshot cleanup can be safely removed since account cleanup will remove 
them automatically.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Question: commands.properties.in deprecated

[Tutkowski, Mike]

Hi,


I see in afcbd27 that we've deprecated commands.properties.in.


Is there documentation on the new way that we specify permissions for a given 
API these days?


Thanks!

Mike

[GitHub] cloudstack pull request: Notify listeners when a host has been add...

[DaanHoogland]

Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/816#issuecomment-218480720
  
That is the rat test. it is usually right. I didn't see a new file without 
license though. May one of the to with a license starting with an empty comment 
line were the problem. Otherwise the job may be the problem in the sense of not 
cleaning the workspace properly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: DAO: Hit the cache for entity flagged as ...

[marcaurele]

GitHub user marcaurele reopened a pull request:

https://github.com/apache/cloudstack/pull/1532

DAO: Hit the cache for entity flagged as removed too

I came along this part of the code and I don't see any reason why the cache 
should not be used when fetching with the "removed" ones. It will help decrease 
the number of DB queries.

*It can be merged in many CS versions*

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/exoscale/cloudstack fix/db-cache-miss

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1532.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1532


commit 696440a6754a60888fce7a82e8022ca7278b9be5
Author: Marc-Aurèle Brothier 
Date:   2016-05-04T12:28:44Z

dao: Hit the cache for entity flagged as removed too since they are put
in cache afterwards.

commit 56fe0ec53dbc1331f1f4875f6f9e9bea62760c38
Author: Marc-Aurèle Brothier 
Date:   2016-05-09T09:39:04Z

Rewrite the change for method findByIdIncludingRemoved(ID id) and also 
change the sibling method findById(final ID id)




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Test failure on master?

[Will Stevens]

I can't point to this PR for this, but I have noticed an increase in random
failures in my CI runs since this code has gone in.  I have not tracked it
down to this, but resources are tight in those environments, so if we are
getting CPU maxing out, that could potentially account for the higher
failure rates in my CI environments...

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Wed, May 11, 2016 at 10:17 AM, Will Stevens 
wrote:

> Rohit, I have seen quite a few issues with this feature so far.  The
> change you made in #1538 does not change the actual code at all, it just
> reduces the number of tests, so you are less likely to run into the
> problem, but the problem still exists.
>
> I am CCing in Simon Weller as well.  I was talking to him this morning and
> he had this to say (unprompted).
>
> Will, We're still seeing odd issues with that NIO SSL concurrency patch
>> (1493), even after pulling in the additional PR 1534. The latest problem
>> we've seen is 100% cpu on the agents for no apparent reason. I reverted
>> both patches from our QA lab this morning and the problem has gone away.
>
>
> I pulled it into a second lab where we have haproxy setup to load balance
>> and the same behaviour occurs
>
>
> top - 08:18:15 up 1 day, 17:08,  5 users,  load average: 1.92, 2.22, 2.09
>> Tasks: 223 total,   1 running, 222 sleeping,   0 stopped,   0 zombie
>> %Cpu(s): 22.2 us, 11.9 sy,  0.0 ni, 65.8 id,  0.0 wa,  0.0 hi,  0.1 si,
>>  0.0 st
>> KiB Mem : 32673608 total, 28312176 free,  3512104 used,   849328
>> buff/cache
>> KiB Swap:  4194300 total,  4194300 free,0 used. 28757568 avail Mem
>>
>>   PID USER  PR  NIVIRTRESSHR S  %CPU %MEM TIME+
>> COMMAND
>>
>> 17985 root  20   0 6937720 162816  22196 S 100.3  0.5   3:24.84
>> /usr/lib/jvm/jre/bin/java -Xms256m -Xmx2048m -cp
>> /usr/share/cloudstack-agent/lib/activatio+
>> 15587 root  20   0 1733288 375976  12164 S 100.0  1.2  10:42.36
>> /usr/libexec/qemu-kvm -name v-46-VM -S -machine
>> pc-i440fx-rhel7.0.0,accel=kvm,usb=off -m 1+
>>  4480 root  20   0  909604 305292  12264 S   0.7  0.9   1:10.21
>> /usr/libexec/qemu-kvm -name r-44-VM -S -machine
>> pc-i440fx-rhel7.0.0,accel=kvm,usb=off -m 2+
>>  5188 root  20   0  957548 323420  12216 S   0.7  1.0   1:07.35
>> /usr/libexec/qemu-kvm -name r-45-VM -S -machine
>> pc-i440fx-rhel7.0.0,accel=kvm,usb=off -m 2+
>> 18336 root  20   0  157840   2392   1556 R   0.7  0.0   0:00.14 top
>>
>>
>> 19023 root  20   0 1002156 449720  12372 S   0.7  1.4  10:57.69
>> /usr/libexec/qemu-kvm -name r-32-VM -S -machine
>> pc-i440fx-rhel7.0.0,accel=kvm,usb=off -m 2+
>
>
>  I am considering reverting this feature (both PRs) until we can
> understand what is causing this and we can stabilize this code so it does
> not cause us problems.  With this type of behavior, I am not confident with
> this code in production right now...
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Wed, May 11, 2016 at 5:36 AM, Rohit Yadav 
> wrote:
>
>> Please follow up on PR #1538 and comment if that fixes the issue on OSX.
>>
>> Regards.
>>
>> Regards,
>>
>> Rohit Yadav
>>
>> rohit.ya...@shapeblue.com
>> www.shapeblue.com
>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> @shapeblue
>>
>> -Original Message-
>> From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com]
>> Sent: Wednesday, May 11, 2016 2:49 PM
>> To: dev@cloudstack.apache.org
>> Subject: RE: Test failure on master?
>>
>> I don't have OSX, but it seems to be working on Travis and Linux env in
>> general.
>> I'll send a PR that relaxes malicious client attacks, and ask you to
>> review in your env -- Koushik and Mike.
>>
>> Regards,
>>
>> Rohit Yadav
>>
>> rohit.ya...@shapeblue.com
>> www.shapeblue.com
>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>>
>> -Original Message-
>> From: Koushik Das [mailto:koushik@accelerite.com]
>> Sent: Wednesday, May 11, 2016 12:22 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: Test failure on master?
>>
>> I am also seeing the same failure happening randomly. OS X El Capitan
>> 10.11.4.
>>
>> Results :
>>
>> Tests in error:
>>   NioTest.testConnection:152 > TestTimedOut test timed out after 6
>> milliseco...
>>
>> Tests run: 200, Failures: 0, Errors: 1, Skipped: 13
>>
>>
>> 
>> From: Tutkowski, Mike 
>> Sent: Tuesday, May 10, 2016 6:31:23 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: Test failure on master?
>>
>> Oh, and it's the OS of my MacBook Pro.
>>
>> > On May 10, 2016, at 6:59 AM, Tutkowski, Mike 
>> wrote:
>> >
>> > Hi,
>> >
>> > The environment is Mac OS X El Capitan 10.11.4.
>> >

[GitHub] cloudstack pull request: CLOUDSTACK-9348: Use non-blocking SSL han...

[kiwiflyer]

Github user kiwiflyer commented on the pull request:

https://github.com/apache/cloudstack/pull/1493#issuecomment-218473922
  

![image](https://cloud.githubusercontent.com/assets/17278194/15184123/94553146-1759-11e6-8dec-c4e0bb0c1795.png)



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9348: Use non-blocking SSL han...

[kiwiflyer]

Github user kiwiflyer commented on the pull request:

https://github.com/apache/cloudstack/pull/1493#issuecomment-218473890
  
@rhtyd We're still seeing some odd behaviour related to the agent with this 
PR (and PR1534) applied to 2 of our hardware labs. What we're seeing is 100% 
cpu on the agent (KVM). There's nothing obvious in the debug logs indicating a 
problem. I've tried it both behind haproxy and directly to the management 
server and we see the same symptoms. Let me know what we can provide to help in 
terms of debugging.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: 4.9/master bountycastle changes

[DaanHoogland]

Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/1511#issuecomment-218473575
  
a timeout @swill. in travis that is not a good indication of not working. I 
wouldn't know where to look. @rhtyd can you close and reopen?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: DAO: Hit the cache for entity flagged as ...

[marcaurele]

Github user marcaurele closed the pull request at:

https://github.com/apache/cloudstack/pull/1532


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Test failure on master?

[Will Stevens]

Rohit, I have seen quite a few issues with this feature so far.  The change
you made in #1538 does not change the actual code at all, it just reduces
the number of tests, so you are less likely to run into the problem, but
the problem still exists.

I am CCing in Simon Weller as well.  I was talking to him this morning and
he had this to say (unprompted).

Will, We're still seeing odd issues with that NIO SSL concurrency patch
> (1493), even after pulling in the additional PR 1534. The latest problem
> we've seen is 100% cpu on the agents for no apparent reason. I reverted
> both patches from our QA lab this morning and the problem has gone away.


I pulled it into a second lab where we have haproxy setup to load balance
> and the same behaviour occurs


top - 08:18:15 up 1 day, 17:08,  5 users,  load average: 1.92, 2.22, 2.09
> Tasks: 223 total,   1 running, 222 sleeping,   0 stopped,   0 zombie
> %Cpu(s): 22.2 us, 11.9 sy,  0.0 ni, 65.8 id,  0.0 wa,  0.0 hi,  0.1 si,
>  0.0 st
> KiB Mem : 32673608 total, 28312176 free,  3512104 used,   849328 buff/cache
> KiB Swap:  4194300 total,  4194300 free,0 used. 28757568 avail Mem
>
>   PID USER  PR  NIVIRTRESSHR S  %CPU %MEM TIME+
> COMMAND
>
> 17985 root  20   0 6937720 162816  22196 S 100.3  0.5   3:24.84
> /usr/lib/jvm/jre/bin/java -Xms256m -Xmx2048m -cp
> /usr/share/cloudstack-agent/lib/activatio+
> 15587 root  20   0 1733288 375976  12164 S 100.0  1.2  10:42.36
> /usr/libexec/qemu-kvm -name v-46-VM -S -machine
> pc-i440fx-rhel7.0.0,accel=kvm,usb=off -m 1+
>  4480 root  20   0  909604 305292  12264 S   0.7  0.9   1:10.21
> /usr/libexec/qemu-kvm -name r-44-VM -S -machine
> pc-i440fx-rhel7.0.0,accel=kvm,usb=off -m 2+
>  5188 root  20   0  957548 323420  12216 S   0.7  1.0   1:07.35
> /usr/libexec/qemu-kvm -name r-45-VM -S -machine
> pc-i440fx-rhel7.0.0,accel=kvm,usb=off -m 2+
> 18336 root  20   0  157840   2392   1556 R   0.7  0.0   0:00.14 top
>
>
> 19023 root  20   0 1002156 449720  12372 S   0.7  1.4  10:57.69
> /usr/libexec/qemu-kvm -name r-32-VM -S -machine
> pc-i440fx-rhel7.0.0,accel=kvm,usb=off -m 2+


 I am considering reverting this feature (both PRs) until we can understand
what is causing this and we can stabilize this code so it does not cause us
problems.  With this type of behavior, I am not confident with this code in
production right now...

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Wed, May 11, 2016 at 5:36 AM, Rohit Yadav 
wrote:

> Please follow up on PR #1538 and comment if that fixes the issue on OSX.
>
> Regards.
>
> Regards,
>
> Rohit Yadav
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
> -Original Message-
> From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com]
> Sent: Wednesday, May 11, 2016 2:49 PM
> To: dev@cloudstack.apache.org
> Subject: RE: Test failure on master?
>
> I don't have OSX, but it seems to be working on Travis and Linux env in
> general.
> I'll send a PR that relaxes malicious client attacks, and ask you to
> review in your env -- Koushik and Mike.
>
> Regards,
>
> Rohit Yadav
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>
> -Original Message-
> From: Koushik Das [mailto:koushik@accelerite.com]
> Sent: Wednesday, May 11, 2016 12:22 PM
> To: dev@cloudstack.apache.org
> Subject: Re: Test failure on master?
>
> I am also seeing the same failure happening randomly. OS X El Capitan
> 10.11.4.
>
> Results :
>
> Tests in error:
>   NioTest.testConnection:152 > TestTimedOut test timed out after 6
> milliseco...
>
> Tests run: 200, Failures: 0, Errors: 1, Skipped: 13
>
>
> 
> From: Tutkowski, Mike 
> Sent: Tuesday, May 10, 2016 6:31:23 PM
> To: dev@cloudstack.apache.org
> Subject: Re: Test failure on master?
>
> Oh, and it's the OS of my MacBook Pro.
>
> > On May 10, 2016, at 6:59 AM, Tutkowski, Mike 
> wrote:
> >
> > Hi,
> >
> > The environment is Mac OS X El Capitan 10.11.4.
> >
> > Thanks!
> > Mike
> >
> >> On May 10, 2016, at 5:51 AM, Will Stevens 
> wrote:
> >>
> >> I think I can verify that this is still happening on master for him
> >> because you changed the timeout (and the number of tests run, etc)
> >> when you pushed the fix in #1534.  So by looking at the timeout of
> >> 6, we can verify that it is the latest code from master being run.
> >>
> >> I do think we need to revisit this to make sure we don't have
> >> intermittent issues with this test.
> >>
> >> Thx guys...
> >>
> >> *Will STEVENS*
> >> Lead Developer
> >>
> >> *CloudOps* *| *Cloud Solutions Experts
> >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> >> 

Re: Virtual router failed to start on KVM + Linux Bridges

[Prashanth Manthena]

Thank you Wei & Remi !!

I will start using the latest system VM templates.

With regards,
Prashanth

On Wed, May 11, 2016 at 2:20 PM, Remi Bergsma 
wrote:

> Hi Prashanth,
>
> As Wei suggested, when you use CloudStack 4.6/4.7/4.8/4.9 you also need
> the systemvm template with version 4.6.0. This does include the needed
> Python modules. The 4.4 template you used won't fly. You will run into
> other missing packages and changes as well.
>
> Regards, Remi
>
> Sent from my iPhone
>
> > On 11 May 2016, at 13:50, Prashanth Manthena <
> prashanth.manth...@nuagenetworks.net> wrote:
> >
> > Hi Will,
> >
> > I was able to put a break-point and debug the actual issue in the VR
> > (system VM template), it's a Python import error of the package
> "netaddr",
> > which can be fixed by updating the system template with the "netaddr"
> > package installed in it. Hopefully, it fixes this issue.
> >
> > Here is the error in VR's /var/log/cloud.log:
> >
> > Wed May 11 10:29:37 UTC 2016 : VR config: configuation format version 1.0
> > Wed May 11 10:29:37 UTC 2016 : VR config: creating file:
> > /var/cache/cloud/ip_asss
> > ociations.json
> > Wed May 11 10:29:37 UTC 2016 : VR config: create file success
> > Wed May 11 10:29:37 UTC 2016 : VR config: executing:
> > /opt/cloud/bin/update_confii
> > g.py ip_associations.json
> > Traceback (most recent call last):
> >  File "/opt/cloud/bin/update_config.py", line 20, in 
> >from merge import QueueFile
> >  File "/opt/cloud/bin/merge.py", line 23, in 
> >import cs_ip
> >  File "/opt/cloud/bin/cs_ip.py", line 19, in 
> >from netaddr import *
> > ImportError: No module named netaddr
> > Wed May 11 10:29:37 UTC 2016 : VR config: executing failed:
> > /opt/cloud/bin/updatt
> > e_config.py ip_associations.json
> >
> > With regards,
> > Prashanth
> >
> > On Wed, May 11, 2016 at 12:06 PM, Prashanth Manthena <
> > prashanth.manth...@nuagenetworks.net> wrote:
> >
> >> Hi Will,
> >>
> >> I am hitting an issue similar to CLOUDSTACK-9013 on master (4.9.0). Here
> >> are the management server error logs:
> >>
> >> 2016-05-10 08:24:03,109 ERROR
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (Work-Job-Executor-3:ctx-5069ea04 job-247/job-250 ctx-48a763a9)
> >> (logid:2735b868) VR config: execution failed:
> >> "/opt/cloud/bin/update_config.py ip_associations.json", check
> >> /var/log/cloud.log in VR for details
> >>
> >> 2016-05-10 08:24:03,109 WARN
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (Work-Job-Executor-3:ctx-5069ea04 job-247/job-250 ctx-48a763a9)
> >> (logid:2735b868) Command: com.cloud.agent.api.Command failed while
> starting
> >> virtual router
> >>
> >> 2016-05-10 08:24:03,109 INFO  [c.c.v.VirtualMachineManagerImpl]
> >> (Work-Job-Executor-3:ctx-5069ea04 job-247/job-250 ctx-48a763a9)
> >> (logid:2735b868) The guru did not like the answers so stopping
> >> VM[DomainRouter|r-30-VM]
> >>
> >> Response:
> >>
> >> 2016-05-10 08:24:03,067 DEBUG [c.c.a.t.Request]
> >> (AgentManager-Handler-10:null) (logid:) Seq 1-4347381015296081926:
> >> Processing:  { Ans: , MgmtId: 275619426932736, via: 1, Ver: v1, Flags:
> 10,
> >>
> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":30,"name":"r-30-VM","type":"DomainRouter","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":268435456,"maxRam":268435456,"arch":"x86_64","os":"Debian
> >> GNU/Linux 5.0 (64-bit)","platformEmulator":"Debian GNU/Linux
> >> 5","bootArgs":" template=domP name=r-30-VM eth2ip=10.100.100.117
> >> eth2mask=255.255.255.0 gateway=10.100.100.1 eth0ip=10.10.1.1
> >> eth0mask=255.255.255.0 domain=cs2cloud.internal cidrsize=24
> >> dhcprange=10.10.1.1 eth1ip=169.254.2.130 eth1mask=255.255.0.0
> type=router
> >> disable_rp_filter=true dns1=128.251.10.29 dns2=4.4.4.4
> >>
> baremetalnotificationsecuritykey=ODFTmygvs7oVAHVFZMmv7RpvsNNsawY9tf6sfeMvkUJlmEhWXGli5sO_oWAr52vrF18Z5xMgzjclu9P7G_w9Fg
> >>
> baremetalnotificationapikey=pguqxBGrlU2ItCuTHa7xb6pmjrpl9KbVMEE7o6TKrJ-mjDoaRayL_Nj21VkqrKvbtc10L8BxPqaYIhnGJkeHfw
> >> host=10.30.36.11
> >>
> port=8080","enableHA":true,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"Pj8GDPW2vqLfkdOMoCOogg","vncAddr":"10.100.100.11","params":{},"uuid":"f406776f-3b4c-4d23-917e-715ceaea0a7a","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"f44bbb4e-a788-4034-9389-dc1fa47dce70","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"63a122cd-ac74-3cc1-85b0-e3ad105234dc","id":1,"poolType":"NetworkFilesystem","host":"
> >> andcvtb05.be.alcatel-lucent.com
> >> ","path":"/andcvtb05/storage","port":2049,"url":"NetworkFilesystem://
> >>
> andcvtb05.be.alcatel-lucent.com/andcvtb05/storage/?ROLE=Primary=63a122cd-ac74-3cc1-85b0-e3ad105234dc
> >>
> 

[GitHub] cloudstack pull request: DAO: Hit the cache for entity flagged as ...

[DaanHoogland]

Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/1532#issuecomment-218472093
  
@marcaurele instead of force push you can close and reopen after a few 
seconds. This usually works as well and seves you switching windows 
browser->terminal->browser ;)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Remove extraneous log directory and add c...

[DaanHoogland]

Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/1537#issuecomment-218470411
  
LGTM, @swill, I am being very formal but can you queue this for integration 
tests?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9265 cleanup around httpclient...

[DaanHoogland]

Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/1385#issuecomment-218467871
  
@swill I closed and reopened. It started building anew but is now stuck in 
the archiving phase for quite some time.
```
[INFO] 

[INFO] BUILD SUCCESS
[INFO] 

[INFO] Total time: 40:46 min
[INFO] Finished at: 2016-05-11T10:07:41+00:00
[INFO] Final Memory: 181M/1862M
[INFO] 

[CHECKSTYLE] Collecting checkstyle analysis files...
[CHECKSTYLE] Finding all files that match the pattern 
**/checkstyle-result.xml
[CHECKSTYLE] Parsing 96 files in 
/home/jenkins/jenkins-slave/workspace/cloudstack-pr-analysis
```
I am trying to find where it got stuck. At least aftyer duplicate code 
analysis...:|


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9351: Add ids parameter to res...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1497#issuecomment-218469220
  
@serg38 I had a bunch of people re-push PRs last night and this morning to 
get everything green and 3 or 4 are failing with this.  

Here are a couple off the top of my head:  #1502, #1376


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9351: Add ids parameter to res...

[swill]

Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1497#issuecomment-218464050
  
@nvazquez this seems to have broken travis for everyone.  We have this 
issue showing up in pretty much all travis runs right now `ContextSuite 
context=TestListIdsParams>:teardown`.

Can you investigate?  Thanks...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9299: Out-of-band Management f...

[nvazquez]

Github user nvazquez commented on the pull request:

https://github.com/apache/cloudstack/pull/1502#issuecomment-218468571
  
@rhtyd sure, I'll examine this. This tests are introduced in #1497, I'll 
work on this


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-9351: Add ids parameter to res...

[nvazquez]

Github user nvazquez commented on the pull request:

https://github.com/apache/cloudstack/pull/1497#issuecomment-218467920
  
@swill it's strange, as @serg38 says it succeeds for us and for 
@koushik-das, I'll investigate. Is there any log of the failure?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---