[GitHub] blueorangutan commented on issue #2048: CLOUDSTACK-9880: Expansion of Management IP Range.

[GitBox]

blueorangutan commented on issue #2048: CLOUDSTACK-9880: Expansion of 
Management IP Range.
URL: https://github.com/apache/cloudstack/pull/2048#issuecomment-352985449
 
 
   Trillian test result (tid-1835)
   Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 28540 seconds
   Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2048-t1835-kvm-centos7.zip
   Smoke tests completed. 64 look OK, 3 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   test_01_vpc_privategw_acl | `Failure` | 51.70 | test_privategw_acl.py
   test_02_vpc_privategw_static_routes | `Failure` | 148.02 | 
test_privategw_acl.py
   test_03_vpc_privategw_restart_vpc_cleanup | `Failure` | 127.97 | 
test_privategw_acl.py
   test_04_rvpc_privategw_static_routes | `Failure` | 238.68 | 
test_privategw_acl.py
   test_02_create_template_with_checksum_sha1 | `Error` | 5.23 | 
test_templates.py
   test_03_create_template_with_checksum_sha256 | `Error` | 5.24 | 
test_templates.py
   test_04_create_template_with_checksum_md5 | `Error` | 5.26 | 
test_templates.py
   test_01_vpc_remote_access_vpn | `Error` | 50.84 | test_vpc_vpn.py
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nvazquez commented on issue #2146: Multidisk ova import

[GitBox]

nvazquez commented on issue #2146: Multidisk ova import
URL: https://github.com/apache/cloudstack/pull/2146#issuecomment-352969262
 
 
   Sure @rhtyd, conflicts fixed


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nvazquez commented on issue #2035: CLOUDSTACK-9867:VM snapshot on primary storage usage metrics

[GitBox]

nvazquez commented on issue #2035: CLOUDSTACK-9867:VM snapshot on primary 
storage usage metrics
URL: https://github.com/apache/cloudstack/pull/2035#issuecomment-352959690
 
 
   @rhtyd sure, conflicts fixed


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2104: CLOUDSTACK-9908 : Primary Storage allocated capacity goes very high after VM snapshot

[GitBox]

blueorangutan commented on issue #2104: CLOUDSTACK-9908 : Primary Storage 
allocated capacity goes very high after VM snapshot
URL: https://github.com/apache/cloudstack/pull/2104#issuecomment-352949554
 
 
   Trillian test result (tid-1831)
   Environment: vmware-55u3 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 52359 seconds
   Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2104-t1831-vmware-55u3.zip
   Smoke tests completed. 56 look OK, 11 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   test_00_deploy_vm_root_resize | `Error` | 5.93 | 
test_deploy_vm_root_resize.py
   ContextSuite context=TestDeployVmRootSize>:teardown | `Error` | 0.16 | 
test_deploy_vm_root_resize.py
   test_01_deploy_vms_storage_tags | `Error` | 40.77 | test_primary_storage.py
   test_01_deploy_vms_storage_tags | `Error` | 40.81 | test_primary_storage.py
   ContextSuite context=TestStorageTags>:teardown | `Error` | 85.72 | 
test_primary_storage.py
   test_01_vpc_privategw_acl | `Failure` | 142.30 | test_privategw_acl.py
   test_02_vpc_privategw_static_routes | `Failure` | 505.00 | 
test_privategw_acl.py
   test_03_vpc_privategw_restart_vpc_cleanup | `Failure` | 640.91 | 
test_privategw_acl.py
   test_04_rvpc_privategw_static_routes | `Failure` | 917.66 | 
test_privategw_acl.py
   test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true | `Failure` | 
560.57 | test_routers_network_ops.py
   test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false | `Failure` | 
584.99 | test_routers_network_ops.py
   test_02_list_snapshots_with_removed_data_store | `Error` | 35.69 | 
test_snapshots.py
   test_02_list_snapshots_with_removed_data_store | `Error` | 35.71 | 
test_snapshots.py
   test_05_stop_ssvm | `Error` | 325.75 | test_ssvm.py
   test_06_stop_cpvm | `Error` | 145.65 | test_ssvm.py
   test_08_reboot_cpvm | `Failure` | 0.04 | test_ssvm.py
   test_02_create_template_with_checksum_sha1 | `Error` | 5.24 | 
test_templates.py
   test_03_create_template_with_checksum_sha256 | `Error` | 5.22 | 
test_templates.py
   test_04_create_template_with_checksum_md5 | `Error` | 5.21 | 
test_templates.py
   test_01_volume_usage | `Error` | 25.88 | test_usage.py
   test_01_create_volume | `Error` | 25.75 | test_volumes.py
   test_02_attach_volume | `Error` | 25.23 | test_volumes.py
   test_03_download_attached_volume | `Error` | 5.09 | test_volumes.py
   test_04_delete_attached_volume | `Error` | 5.09 | test_volumes.py
   test_05_detach_volume | `Error` | 5.09 | test_volumes.py
   test_06_download_detached_volume | `Error` | 5.08 | test_volumes.py
   test_07_resize_fail | `Error` | 10.24 | test_volumes.py
   test_08_resize_volume | `Error` | 5.09 | test_volumes.py
   test_09_delete_detached_volume | `Error` | 20.29 | test_volumes.py
   ContextSuite context=TestVolumes>:teardown | `Error` | 219.27 | 
test_volumes.py
   ContextSuite context=TestVPCRedundancy>:teardown | `Error` | 602.90 | 
test_vpc_redundant.py
   ContextSuite context=TestRVPCSite2SiteVpn>:setup | `Error` | 0.00 | 
test_vpc_vpn.py
   ContextSuite context=TestVpcRemoteAccessVpn>:setup | `Error` | 0.00 | 
test_vpc_vpn.py
   ContextSuite context=TestVpcSite2SiteVpn>:setup | `Error` | 0.00 | 
test_vpc_vpn.py
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nvazquez commented on issue #2305: [PLACEHOLDER] CLOUDSTACK-10126: Separate Subnet for CPVM and SSVM

[GitBox]

nvazquez commented on issue #2305: [PLACEHOLDER] CLOUDSTACK-10126: Separate 
Subnet for CPVM and SSVM
URL: https://github.com/apache/cloudstack/pull/2305#issuecomment-352949270
 
 
   Closing this PR as work is continued on #2368 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nvazquez closed pull request #2305: [PLACEHOLDER] CLOUDSTACK-10126: Separate Subnet for CPVM and SSVM

[GitBox]

nvazquez closed pull request #2305: [PLACEHOLDER] CLOUDSTACK-10126: Separate 
Subnet for CPVM and SSVM
URL: https://github.com/apache/cloudstack/pull/2305
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/pom.xml b/pom.xml
index 96fe92503e2..aa8b87c1560 100644
--- a/pom.xml
+++ b/pom.xml
@@ -18,6 +18,7 @@
 under the License.
 
 -->
+
 http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
   xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/maven-v4_0_0.xsd;>
   4.0.0


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nvazquez commented on issue #2368: CLOUDSTACK-10126: Separate Subnet for SSVM and CPVM

[GitBox]

nvazquez commented on issue #2368: CLOUDSTACK-10126: Separate Subnet for SSVM 
and CPVM
URL: https://github.com/apache/cloudstack/pull/2368#issuecomment-352944809
 
 
   @rhtyd @DaanHoogland @borisstoyanov @nitin-maharana please review


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nvazquez opened a new pull request #2368: CLOUDSTACK-10126: Separate Subnet for SSVM and CPVM

[GitBox]

nvazquez opened a new pull request #2368: CLOUDSTACK-10126: Separate Subnet for 
SSVM and CPVM
URL: https://github.com/apache/cloudstack/pull/2368
 
 
   JIRA Ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-10126
   
   ### Background
   This extends work presented on #2048 on which the ability to extend the 
management range is provided. 
   
   ### Description
   By this feature it is possible to dedicate a created range for SSVM and CPVM 
(system vms) and provide a VLAN ID for its range.
   
   A new boolean global configuration is added: 
system.vm.management.ip.reservation.mode.strictness. If enabled, the use of 
System VMs management IP reservation is strict, preferred if not. Default value 
is false (preferred).
   - Strict reservation: System VMs should try to get a private IP from a range 
marked for system vms. If not available, deployment fails
   - Preferred reservation: System VMS will try to get a private IP from a 
range marked for system vms. If not available, IP for range not marked for 
system vms is taken.
   
   ### UI Changes
   - Management range creation:
   
![image](https://user-images.githubusercontent.com/5295080/34187659-4b99b404-e511-11e7-86a3-9db473fe87b0.png)
   
   - Ranges dedicated for system vms are marked with checkbox under 'System 
VMs' column:
   
![image](https://user-images.githubusercontent.com/5295080/34187735-9fda235a-e511-11e7-85eb-9191d42658e7.png)
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2211: CLOUDSTACK-10013: Migrate systemvmtemplate to Debian9

[GitBox]

blueorangutan commented on issue #2211: CLOUDSTACK-10013: Migrate 
systemvmtemplate to Debian9
URL: https://github.com/apache/cloudstack/pull/2211#issuecomment-352941800
 
 
   Trillian test result (tid-1833)
   Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 24417 seconds
   Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2211-t1833-kvm-centos7.zip
   Smoke tests completed. 65 look OK, 1 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   test_01_vpc_remote_access_vpn | `Error` | 30.67 | test_vpc_vpn.py
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2281: CLOUDSTACK-10102: New network type (L2)

[GitBox]

blueorangutan commented on issue #2281: CLOUDSTACK-10102: New network type (L2)
URL: https://github.com/apache/cloudstack/pull/2281#issuecomment-352937530
 
 
   Trillian test result (tid-1832)
   Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 28508 seconds
   Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2281-t1832-kvm-centos7.zip
   Smoke tests completed. 63 look OK, 4 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   test_01_vpc_privategw_acl | `Failure` | 57.00 | test_privategw_acl.py
   test_02_vpc_privategw_static_routes | `Failure` | 208.74 | 
test_privategw_acl.py
   test_03_vpc_privategw_restart_vpc_cleanup | `Failure` | 153.34 | 
test_privategw_acl.py
   test_04_rvpc_privategw_static_routes | `Failure` | 294.56 | 
test_privategw_acl.py
   test_02_create_template_with_checksum_sha1 | `Error` | 5.29 | 
test_templates.py
   test_03_create_template_with_checksum_sha256 | `Error` | 5.22 | 
test_templates.py
   test_04_create_template_with_checksum_md5 | `Error` | 5.23 | 
test_templates.py
   test_06_download_detached_volume | `Failure` | 147.53 | test_volumes.py
   test_01_vpc_remote_access_vpn | `Error` | 50.81 | test_vpc_vpn.py
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rafaelweingartner commented on a change in pull request #1740: CLOUDSTACK-9572 Snapshot on primary storage not cleaned up after Stor?

[GitBox]

rafaelweingartner commented on a change in pull request #1740: CLOUDSTACK-9572 
Snapshot on primary storage not cleaned up after Stor?
URL: https://github.com/apache/cloudstack/pull/1740#discussion_r157915732
 
 

 ##
 File path: 
engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/SnapshotDataFactoryImpl.java
 ##
 @@ -64,6 +64,24 @@ public SnapshotInfo getSnapshot(DataObject obj, DataStore 
store) {
 return so;
 }
 
+@Override
+public List getSnapshots(long volumeId, DataStoreRole role) {
+
+SnapshotDataStoreVO snapshotStore = 
snapshotStoreDao.findByVolume(volumeId, role);
+if (snapshotStore == null) {
+return null;
 
 Review comment:
   This will cause a NPE. Consider returning an empty list.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] voloshanenko opened a new pull request #2367: Fix ACL_INBOUND/OUTBOUND rules for PrivateGateway

[GitBox]

voloshanenko opened a new pull request #2367: Fix ACL_INBOUND/OUTBOUND rules 
for PrivateGateway
URL: https://github.com/apache/cloudstack/pull/2367
 
 
   We found bug in ACL rules for PrivateGateway for VPC
   
   At a glance - rules not applied - switching Allow All or Deny All (default 
ACL) - showed as completed - but rules missed.
   Result - traffic via PrivateGateway blocked by next DROP rule in next chains
   
   How to reproduce:
   1. Enable PrivateGateway for Cloudstack
   2. Create VPC
   3. Provision new PrivateGateway inside VPC with some VLAN
   4. Change ACL (optional step to show that problem not in initial 
configuration but in config itself)
   
   Expected:
   ACL rules applied (inserted) into correspondig ACL_INBOUND/OUTBOUND chanins 
for PrivateGateway interface (ethX) based on ACL which user choose
   
   Current:
   No rules inserted. ACL_INBOUND/OUTBOUND_ethX - empty. Traffic blocked by 
next DROP rule in FORWARD chain
   
   Affect - all our corporate customers blocked with access to their own nets 
via PG and vice-versa.
   
   Root cause:
   Issue happened because of CsNetFilter.py logic for inserting rules for 
ACL_INBOUND/OUTBOUND chains.
   
   We choose rule numebr to isnert right before last DROP rule - but forget 
about fact - that if chain empty - we also return 0 as insert position. Which 
not true for iptables - numeration started from 0.
   
   So we need very small patch to handle this special case - if number of rules 
inside chain equal to zero - return 1, else - return count of rules inside 
chain.
   
   It's found only one - just because be default for PrivateGateway - we didn't 
insert any "service rules" (if SourceNat for PrivateGteway not ticked) - and we 
have by default empty ACL_INBOUND/OUTBOUND chains. Because same insert happened 
for all VPC networks (but when we call this insert - we already have at least 1 
rule inside chains - and we successfully can process)
   
   https://issues.apache.org/jira/browse/CLOUDSTACK-10200


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rafaelweingartner commented on issue #1740: CLOUDSTACK-9572 Snapshot on primary storage not cleaned up after Stor?

[GitBox]

rafaelweingartner commented on issue #1740: CLOUDSTACK-9572 Snapshot on primary 
storage not cleaned up after Stor?
URL: https://github.com/apache/cloudstack/pull/1740#issuecomment-352927548
 
 
   I have the same doubt as @syed here.
   
   > Why is it expected that latest snapshot be present on the primary storage 
after it has been copied? Shouldn't it be deleted?
   
   I am having a problem that might be cause by the problem discussed here. The 
problem I am having is:
   `create snapshot operation Failed for snapshotId: 3711, reason: 
SR_BACKEND_FAILURE_109. The snapshot chain is too long 
SR_BACKEND_FAILURE_109The snapshot chain is too long`


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rafaelweingartner commented on issue #1740: CLOUDSTACK-9572 Snapshot on primary storage not cleaned up after Stor?

[GitBox]

rafaelweingartner commented on issue #1740: CLOUDSTACK-9572 Snapshot on primary 
storage not cleaned up after Stor?
URL: https://github.com/apache/cloudstack/pull/1740#issuecomment-352927548
 
 
   I have the same doubt as @syed here.
   
   > Why is it expected that latest snapshot be present on the primary storage 
after it has been copied? Shouldn't it be deleted?
   
   I am having a problem that might be cause by this one:
   `create snapshot operation Failed for snapshotId: 3711, reason: 
SR_BACKEND_FAILURE_109. The snapshot chain is too long 
SR_BACKEND_FAILURE_109The snapshot chain is too long`


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nvazquez commented on issue #2048: CLOUDSTACK-9880: Expansion of Management IP Range.

[GitBox]

nvazquez commented on issue #2048: CLOUDSTACK-9880: Expansion of Management IP 
Range.
URL: https://github.com/apache/cloudstack/pull/2048#issuecomment-352917718
 
 
   @blueorangutan test


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2048: CLOUDSTACK-9880: Expansion of Management IP Range.

[GitBox]

blueorangutan commented on issue #2048: CLOUDSTACK-9880: Expansion of 
Management IP Range.
URL: https://github.com/apache/cloudstack/pull/2048#issuecomment-352917790
 
 
   @nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been 
kicked to run smoke tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2048: CLOUDSTACK-9880: Expansion of Management IP Range.

[GitBox]

blueorangutan commented on issue #2048: CLOUDSTACK-9880: Expansion of 
Management IP Range.
URL: https://github.com/apache/cloudstack/pull/2048#issuecomment-352905755
 
 
   Packaging result: ?centos6 ?centos7 ?debian. JID-1431


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2048: CLOUDSTACK-9880: Expansion of Management IP Range.

[GitBox]

blueorangutan commented on issue #2048: CLOUDSTACK-9880: Expansion of 
Management IP Range.
URL: https://github.com/apache/cloudstack/pull/2048#issuecomment-352900152
 
 
   @nvazquez a Jenkins job has been kicked to build packages. I'll keep you 
posted as I make progress.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nvazquez commented on issue #2048: CLOUDSTACK-9880: Expansion of Management IP Range.

[GitBox]

nvazquez commented on issue #2048: CLOUDSTACK-9880: Expansion of Management IP 
Range.
URL: https://github.com/apache/cloudstack/pull/2048#issuecomment-352899927
 
 
   @blueorangutan package


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #1740: CLOUDSTACK-9572 Snapshot on primary storage not cleaned up after Stor?

[GitBox]

blueorangutan commented on issue #1740: CLOUDSTACK-9572 Snapshot on primary 
storage not cleaned up after Stor?
URL: https://github.com/apache/cloudstack/pull/1740#issuecomment-352888771
 
 
   Trillian test result (tid-1829)
   Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 33073 seconds
   Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr1740-t1829-kvm-centos7.zip
   Smoke tests completed. 63 look OK, 4 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   ContextSuite context=TestDeployVirtioSCSIVM>:setup | `Error` | 0.00 | 
test_deploy_virtio_scsi_vm.py
   test_01_vpc_privategw_acl | `Failure` | 71.91 | test_privategw_acl.py
   test_02_vpc_privategw_static_routes | `Failure` | 238.93 | 
test_privategw_acl.py
   test_03_vpc_privategw_restart_vpc_cleanup | `Failure` | 258.79 | 
test_privategw_acl.py
   test_04_rvpc_privategw_static_routes | `Failure` | 309.27 | 
test_privategw_acl.py
   test_02_create_template_with_checksum_sha1 | `Error` | 5.25 | 
test_templates.py
   test_03_create_template_with_checksum_sha256 | `Error` | 5.23 | 
test_templates.py
   test_04_create_template_with_checksum_md5 | `Error` | 5.26 | 
test_templates.py
   test_01_vpc_remote_access_vpn | `Error` | 65.91 | test_vpc_vpn.py
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2350: Cloudstack 10170 - fixes resource tags security bugs and adds account tags support

[GitBox]

blueorangutan commented on issue #2350: Cloudstack 10170 - fixes resource tags 
security bugs and adds account tags support
URL: https://github.com/apache/cloudstack/pull/2350#issuecomment-352886023
 
 
   Trillian test result (tid-1825)
   Environment: vmware-55u3 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 51327 seconds
   Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2350-t1825-vmware-55u3.zip
   Smoke tests completed. 56 look OK, 11 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   test_3d_gpu_support | `Error` | 1516.34 | test_deploy_vgpu_enabled_vm.py
   test_00_deploy_vm_root_resize | `Error` | 1.85 | 
test_deploy_vm_root_resize.py
   ContextSuite context=TestDeployVmRootSize>:teardown | `Error` | 0.22 | 
test_deploy_vm_root_resize.py
   test_01_deploy_vms_storage_tags | `Error` | 24.27 | test_primary_storage.py
   test_01_deploy_vms_storage_tags | `Error` | 24.36 | test_primary_storage.py
   ContextSuite context=TestStorageTags>:teardown | `Error` | 76.05 | 
test_primary_storage.py
   test_01_vpc_privategw_acl | `Failure` | 166.15 | test_privategw_acl.py
   test_02_vpc_privategw_static_routes | `Failure` | 477.06 | 
test_privategw_acl.py
   test_03_vpc_privategw_restart_vpc_cleanup | `Failure` | 474.22 | 
test_privategw_acl.py
   test_04_rvpc_privategw_static_routes | `Failure` | 841.48 | 
test_privategw_acl.py
   test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true | `Failure` | 
519.62 | test_routers_network_ops.py
   test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false | `Failure` | 
523.14 | test_routers_network_ops.py
   test_02_list_snapshots_with_removed_data_store | `Error` | 23.67 | 
test_snapshots.py
   test_02_list_snapshots_with_removed_data_store | `Error` | 23.72 | 
test_snapshots.py
   test_05_stop_ssvm | `Error` | 329.58 | test_ssvm.py
   test_06_stop_cpvm | `Error` | 146.73 | test_ssvm.py
   test_07_reboot_ssvm | `Error` | 66.74 | test_ssvm.py
   test_08_reboot_cpvm | `Failure` | 0.05 | test_ssvm.py
   test_02_create_template_with_checksum_sha1 | `Error` | 5.22 | 
test_templates.py
   test_03_create_template_with_checksum_sha256 | `Error` | 5.25 | 
test_templates.py
   test_04_create_template_with_checksum_md5 | `Error` | 5.23 | 
test_templates.py
   test_01_volume_usage | `Error` | 14.98 | test_usage.py
   test_01_create_volume | `Error` | 16.92 | test_volumes.py
   test_02_attach_volume | `Error` | 18.84 | test_volumes.py
   test_03_download_attached_volume | `Error` | 1.09 | test_volumes.py
   test_04_delete_attached_volume | `Error` | 1.10 | test_volumes.py
   test_05_detach_volume | `Error` | 0.10 | test_volumes.py
   test_06_download_detached_volume | `Error` | 1.10 | test_volumes.py
   test_07_resize_fail | `Error` | 2.26 | test_volumes.py
   test_08_resize_volume | `Error` | 1.10 | test_volumes.py
   test_09_delete_detached_volume | `Error` | 21.87 | test_volumes.py
   test_10_list_volumes | `Failure` | 370.18 | test_volumes.py
   ContextSuite context=TestVolumes>:teardown | `Error` | 370.38 | 
test_volumes.py
   ContextSuite context=TestVpcRemoteAccessVpn>:setup | `Error` | 2441.66 | 
test_vpc_vpn.py
   ContextSuite context=TestVpcSite2SiteVpn>:setup | `Error` | 3956.89 | 
test_vpc_vpn.py
   
   
   Additional tests completed. 0 look ok, 1 have error(s)
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   test_06_template_tag | `Error` | 10.60 | test_tags.py
   test_08_volume_tag | `Error` | 2.27 | test_tags.py
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] krissterckx commented on issue #2259: CLOUDSTACK-10024: Network migration support

[GitBox]

krissterckx commented on issue #2259: CLOUDSTACK-10024: Network migration 
support
URL: https://github.com/apache/cloudstack/pull/2259#issuecomment-352885731
 
 
   @borisstoyanov can you reassess/approve and re-kick Trillian please.  We aim 
to merge in this week. Thanks.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2042: CLOUDSTACK-9875: Unable to re-apply Explicit dedication to VM

[GitBox]

blueorangutan commented on issue #2042: CLOUDSTACK-9875: Unable to re-apply 
Explicit dedication to VM
URL: https://github.com/apache/cloudstack/pull/2042#issuecomment-352877051
 
 
   Trillian test result (tid-1830)
   Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 29914 seconds
   Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2042-t1830-kvm-centos7.zip
   Smoke tests completed. 63 look OK, 4 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   test_01_vpc_privategw_acl | `Failure` | 71.86 | test_privategw_acl.py
   test_02_vpc_privategw_static_routes | `Failure` | 168.16 | 
test_privategw_acl.py
   test_03_vpc_privategw_restart_vpc_cleanup | `Failure` | 122.99 | 
test_privategw_acl.py
   test_04_rvpc_privategw_static_routes | `Failure` | 304.56 | 
test_privategw_acl.py
   test_02_create_template_with_checksum_sha1 | `Error` | 5.27 | 
test_templates.py
   test_03_create_template_with_checksum_sha256 | `Error` | 5.24 | 
test_templates.py
   test_04_create_template_with_checksum_md5 | `Error` | 5.25 | 
test_templates.py
   test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | `Failure` | 399.10 | 
test_vpc_redundant.py
   test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | `Error` | 510.21 | 
test_vpc_redundant.py
   test_01_vpc_remote_access_vpn | `Error` | 60.90 | test_vpc_vpn.py
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2366: CLOUDSTACK-10168 - VR duplicate entries in /etc/hosts when reusing VM name

[GitBox]

blueorangutan commented on issue #2366: CLOUDSTACK-10168 - VR duplicate entries 
in /etc/hosts when reusing VM name
URL: https://github.com/apache/cloudstack/pull/2366#issuecomment-352874854
 
 
   Packaging result: ?centos6 ?centos7 ?debian. JID-1430


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #1760: CLOUDSTACK-9593: userdata: enforce data is a multiple of 4 characters

[GitBox]

blueorangutan commented on issue #1760: CLOUDSTACK-9593: userdata: enforce data 
is a multiple of 4 characters
URL: https://github.com/apache/cloudstack/pull/1760#issuecomment-352870322
 
 
   Trillian test result (tid-1828)
   Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 28907 seconds
   Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr1760-t1828-kvm-centos7.zip
   Smoke tests completed. 62 look OK, 5 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   test_01_vpc_privategw_acl | `Failure` | 46.70 | test_privategw_acl.py
   test_02_vpc_privategw_static_routes | `Failure` | 218.40 | 
test_privategw_acl.py
   test_03_vpc_privategw_restart_vpc_cleanup | `Failure` | 198.33 | 
test_privategw_acl.py
   test_04_rvpc_privategw_static_routes | `Failure` | 308.92 | 
test_privategw_acl.py
   test_02_create_template_with_checksum_sha1 | `Error` | 5.26 | 
test_templates.py
   test_03_create_template_with_checksum_sha256 | `Error` | 5.22 | 
test_templates.py
   test_04_create_template_with_checksum_md5 | `Error` | 5.24 | 
test_templates.py
   test_08_resize_volume | `Error` | 312.38 | test_volumes.py
   test_09_delete_detached_volume | `Error` | 186.87 | test_volumes.py
   test_10_list_volumes | `Error` | 172.32 | test_volumes.py
   test_01_vpc_remote_access_vpn | `Error` | 45.81 | test_vpc_vpn.py
   test_hostha_kvm_host_degraded | `Error` | 26.01 | test_hostha_kvm.py
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on issue #2232: CLOUDSTACK-10040 Upload volume fails when management server can not r?

[GitBox]

rhtyd commented on issue #2232: CLOUDSTACK-10040 Upload volume fails when 
management server can not r?
URL: https://github.com/apache/cloudstack/pull/2232#issuecomment-352868878
 
 
   Thanks for the pr @yvsubhash, closing this as 2074 has been merged. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd closed pull request #2232: CLOUDSTACK-10040 Upload volume fails when management server can not r?

[GitBox]

rhtyd closed pull request #2232: CLOUDSTACK-10040 Upload volume fails when 
management server can not r?
URL: https://github.com/apache/cloudstack/pull/2232
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/server/src/com/cloud/storage/VolumeApiServiceImpl.java 
b/server/src/com/cloud/storage/VolumeApiServiceImpl.java
index 3330cc74a26..a46beb40ea7 100644
--- a/server/src/com/cloud/storage/VolumeApiServiceImpl.java
+++ b/server/src/com/cloud/storage/VolumeApiServiceImpl.java
@@ -135,6 +135,7 @@
 import 
org.apache.cloudstack.engine.subsystem.api.storage.VolumeService.VolumeApiResult;
 import org.apache.cloudstack.framework.async.AsyncCallFuture;
 import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
 import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
 import org.apache.cloudstack.framework.jobs.AsyncJob;
 import org.apache.cloudstack.framework.jobs.AsyncJobExecutionContext;
@@ -173,7 +174,7 @@
 import java.util.UUID;
 import java.util.concurrent.ExecutionException;
 
-public class VolumeApiServiceImpl extends ManagerBase implements 
VolumeApiService, VmWorkJobHandler {
+public class VolumeApiServiceImpl extends ManagerBase implements 
VolumeApiService, VmWorkJobHandler, Configurable {
 private final static Logger s_logger = 
Logger.getLogger(VolumeApiServiceImpl.class);
 public static final String VM_WORK_JOB_HANDLER = 
VolumeApiServiceImpl.class.getSimpleName();
 
@@ -260,6 +261,17 @@
 static final ConfigKey VmJobCheckInterval = new 
ConfigKey("Advanced", Long.class, "vm.job.check.interval", "3000",
 "Interval in milliseconds to check if the job is complete", false);
 
+static final ConfigKey ValidateURLExistence =
+new ConfigKey(
+"Advanced",
+Boolean.class,
+"validate.url.existence",
+"true",
+"Whether url of the given volume during upload needs to be 
validated for existence",
+false,
+ConfigKey.Scope.Zone);
+
+
 private long _maxVolumeSizeInGb;
 private final StateMachine2 
_volStateMachine;
 
@@ -403,8 +415,11 @@ private boolean validateVolume(Account caller, long 
ownerId, Long zoneId, String
 throw new InvalidParameterValueException("File:// type urls 
are currently unsupported");
 }
 UriUtils.validateUrl(format, url);
-// check URL existence
-UriUtils.checkUrlExistence(url);
+
+// check URL existence
+if(ValidateURLExistence.value()) {
+UriUtils.checkUrlExistence(url);
+}
 // Check that the resource limit for secondary storage won't be 
exceeded
 
_resourceLimitMgr.checkResourceLimit(_accountMgr.getAccount(ownerId), 
ResourceType.secondary_storage, UriUtils.getRemoteSize(url));
 } else {
@@ -3066,4 +3081,15 @@ private VmWorkJobVO createPlaceHolderWork(long 
instanceId) {
 
 return workJob;
 }
+
+@Override
+public String getConfigComponentName() {
+return VolumeApiService.class.getSimpleName();
+}
+
+@Override
+public ConfigKey[] getConfigKeys() {
+return new ConfigKey[] { ValidateURLExistence};
+}
+
 }


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2366: CLOUDSTACK-10168 - VR duplicate entries in /etc/hosts when reusing VM name

[GitBox]

blueorangutan commented on issue #2366: CLOUDSTACK-10168 - VR duplicate entries 
in /etc/hosts when reusing VM name
URL: https://github.com/apache/cloudstack/pull/2366#issuecomment-352868279
 
 
   @rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted 
as I make progress.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on issue #2366: CLOUDSTACK-10168 - VR duplicate entries in /etc/hosts when reusing VM name

[GitBox]

rhtyd commented on issue #2366: CLOUDSTACK-10168 - VR duplicate entries in 
/etc/hosts when reusing VM name
URL: https://github.com/apache/cloudstack/pull/2366#issuecomment-352868008
 
 
   @blueorangutan package 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2203: [CLOUDSTACK-10029] All private, public, and guest interfaceses are marked as untagged ca?

[GitBox]

blueorangutan commented on issue #2203: [CLOUDSTACK-10029] All private, public, 
and guest interfaceses are marked as untagged ca?
URL: https://github.com/apache/cloudstack/pull/2203#issuecomment-352867209
 
 
   @rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been 
kicked to run smoke tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on issue #2203: [CLOUDSTACK-10029] All private, public, and guest interfaceses are marked as untagged ca?

[GitBox]

rhtyd commented on issue #2203: [CLOUDSTACK-10029] All private, public, and 
guest interfaceses are marked as untagged ca?
URL: https://github.com/apache/cloudstack/pull/2203#issuecomment-352867031
 
 
   @blueorangutan test 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2211: CLOUDSTACK-10013: Migrate systemvmtemplate to Debian9

[GitBox]

blueorangutan commented on issue #2211: CLOUDSTACK-10013: Migrate 
systemvmtemplate to Debian9
URL: https://github.com/apache/cloudstack/pull/2211#issuecomment-352856628
 
 
   Packaging result: ?centos6 ?centos7 ?debian. JID-1429


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on issue #1554: CLOUDSTACK-9602: API: improve resource limits comprehension

[GitBox]

rhtyd commented on issue #1554: CLOUDSTACK-9602: API: improve resource limits 
comprehension
URL: https://github.com/apache/cloudstack/pull/1554#issuecomment-352855948
 
 
   Test LGTM, additional review requested for merging this.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] khos2ow commented on issue #2352: CLOUDSTACK-10175: prevent VPC list leakage

[GitBox]

khos2ow commented on issue #2352: CLOUDSTACK-10175: prevent VPC list leakage
URL: https://github.com/apache/cloudstack/pull/2352#issuecomment-352855972
 
 
   @rhtyd yes I will.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on issue #2320: CLOUDSTACK-10178: Hotfixes to make 4.10 work

[GitBox]

rhtyd commented on issue #2320: CLOUDSTACK-10178: Hotfixes to make 4.10 work
URL: https://github.com/apache/cloudstack/pull/2320#issuecomment-352850794
 
 
   @bwsw I see most of these commits are already in master, do you want to 
squash your commits and if there are no new changes/commits for master we can 
merge using `-X ours` to effective ignore changes in master while forward 
merging. Thoughts?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on issue #2352: CLOUDSTACK-10175: prevent VPC list leakage

[GitBox]

rhtyd commented on issue #2352: CLOUDSTACK-10175: prevent VPC list leakage
URL: https://github.com/apache/cloudstack/pull/2352#issuecomment-352851020
 
 
   @khos2ow the test_deploy_virtio_scsi_vm related failures are new, rest are 
known issues and ignorable. can you check?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[cloudstack] 19/21: CLOUDSTACK-10194: Use packer for building systemvmtemplates

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git

commit 28e0270abae3abedecfea9050b48e65fa0e6e41b
Author: Rohit Yadav 
AuthorDate: Sun Dec 17 12:55:26 2017 +0530

CLOUDSTACK-10194: Use packer for building systemvmtemplates

- This migrates the current systemvmtemplate build system from
  veewee/virtualbox to packer and qemu based.
- This also introduces and updates a CentOS7 built-in template.
- Remove old appliance build scripts and files.
- Adds iftop package (CLOUDSTACK-9785)

Signed-off-by: Rohit Yadav 
---
 pom.xml|   8 +-
 tools/appliance/.ruby-version  |   1 -
 tools/appliance/.rvmrc |  24 --
 tools/appliance/README.md  | 105 +-
 tools/appliance/build.sh   | 360 +++--
 tools/appliance/builtin/http/kickstart.cfg |  91 ++
 .../scripts/base.sh}   |  28 +-
 .../{Gemfile => builtin/scripts/cleanup.sh}|  20 +-
 tools/appliance/builtin/template.json  |  53 +++
 tools/appliance/convert/Convert.class  | Bin 984 -> 0 bytes
 tools/appliance/convert/Convert.java   |  36 ---
 tools/appliance/convert_ovf_vbox_to_esx.xslt   |  42 ---
 tools/appliance/definitions/builtin/base.sh|  14 -
 tools/appliance/definitions/builtin/cleanup.sh |  21 --
 tools/appliance/definitions/builtin/definition.rb  |  33 --
 .../definitions/builtin/install-xs-tools.sh|  10 -
 tools/appliance/definitions/builtin/ks.cfg |  35 --
 tools/appliance/definitions/builtin/postinstall.sh |  54 
 tools/appliance/definitions/builtin/zerodisk.sh|   3 -
 tools/appliance/definitions/debianbase/cleanup.sh  |  49 ---
 .../definitions/debianbase/configure_login.sh  |  78 -
 .../appliance/definitions/debianbase/definition.rb |  80 -
 tools/appliance/definitions/debianbase/preseed.cfg | 129 
 tools/appliance/definitions/debianbase/zerodisk.sh |  43 ---
 tools/appliance/definitions/devcloud/base.sh   |  12 -
 tools/appliance/definitions/devcloud/cleanup.sh|  21 --
 tools/appliance/definitions/devcloud/definition.rb |  45 ---
 .../appliance/definitions/devcloud/postinstall.sh  |  60 
 tools/appliance/definitions/devcloud/preseed.cfg   | 357 
 tools/appliance/definitions/devcloud/zerodisk.sh   |  11 -
 tools/appliance/shar_cloud_scripts.sh  |   2 +-
 .../http}/preseed.cfg  |  13 +-
 .../scripts}/apt_upgrade.sh|   5 +-
 .../scripts}/authorized_keys.sh|   0
 .../scripts}/cleanup.sh|   0
 .../scripts}/configure_acpid.sh|   0
 .../scripts}/configure_conntrack.sh|   0
 .../scripts}/configure_grub.sh |   4 +
 .../scripts}/configure_locale.sh   |   0
 .../scripts}/configure_login.sh|   0
 .../scripts}/configure_networking.sh   |   0
 .../scripts}/configure_persistent_config.sh|   1 -
 .../scripts}/configure_systemvm_services.sh|   0
 .../scripts}/definition.rb |   4 +-
 .../scripts}/finalize.sh   |   8 +-
 .../scripts}/install_systemvm_packages.sh  |  16 +-
 tools/appliance/systemvmtemplate/template.json |  82 +
 tools/appliance/test.sh| 180 ---
 tools/appliance/vbox_disk_clean.rb |  49 ---
 tools/appliance/vbox_vm_clean.rb   |  77 -
 tools/vagrant/devcloud/Vagrantfile | 206 
 tools/vagrant/devcloud/templates/tmpl/1/1/README   |   1 -
 tools/vagrant/devcloud/templates/tmpl/1/5/README   |   2 -
 tools/vagrant/systemvm/.gitignore  |  52 ---
 tools/vagrant/systemvm/.ruby-version   |   1 -
 tools/vagrant/systemvm/.rvmrc  |  24 --
 tools/vagrant/systemvm/Gemfile |  20 --
 tools/vagrant/systemvm/README.md   |  30 --
 tools/vagrant/systemvm/VBoxManage  |  41 ---
 tools/vagrant/systemvm/Vagrantfile | 123 ---
 tools/vagrant/systemvm/test.sh | 213 
 tools/vagrant/systemvm/vagrant.pub |   1 -
 62 files changed, 340 insertions(+), 2638 deletions(-)

diff --git a/pom.xml b/pom.xml
index 39a611d..00a3542 100644
--- a/pom.xml
+++ b/pom.xml
@@ -880,14 +880,8 @@
   
services/secondary-storage/conf/agent.properties
   
services/secondary-storage/conf/environment.properties
   test/systemvm/README.md
-  

[cloudstack] 15/21: CLOUDSTACK-9501: route is not available by default on CentOS7 (#1637)

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git

commit cd58d1bb54faa37c0b3e912830261027128869a5
Author: lmrv 
AuthorDate: Thu Dec 14 00:54:05 2017 +0530

CLOUDSTACK-9501: route is not available by default on CentOS7 (#1637)

This adds the `net-tools` dependency on CentOS cloudstack-agent rpms.
This will provide ifconfig, route and other tools that may be used
by CloudStack scripts and utilities.

Signed-off-by: Rohit Yadav 
---
 packaging/centos7/cloud.spec | 1 +
 1 file changed, 1 insertion(+)

diff --git a/packaging/centos7/cloud.spec b/packaging/centos7/cloud.spec
index 219b8b6..f16858a 100644
--- a/packaging/centos7/cloud.spec
+++ b/packaging/centos7/cloud.spec
@@ -104,6 +104,7 @@ Requires: bridge-utils
 Requires: ebtables
 Requires: iptables
 Requires: ethtool
+Requires: net-tools
 Requires: iproute
 Requires: ipset
 Requires: perl

-- 
To stop receiving notification emails like this one, please contact
"commits@cloudstack.apache.org" .

[cloudstack] 12/21: CLOUDSTACK-10181: Skip rVR test cases for VMware until supported

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git

commit bfc871ef2fd625b53a05440b09a8e6b7c3c803b5
Author: Rohit Yadav 
AuthorDate: Sat Dec 9 00:43:25 2017 +0530

CLOUDSTACK-10181: Skip rVR test cases for VMware until supported

With this change, rVR related test cases will be skipped for VMware
where this feature is not properly supported.

Signed-off-by: Rohit Yadav 
---
 test/integration/smoke/test_internal_lb.py | 11 +++
 test/integration/smoke/test_privategw_acl.py   | 11 +++
 test/integration/smoke/test_routers_network_ops.py | 13 -
 test/integration/smoke/test_vpc_redundant.py   | 11 +++
 test/integration/smoke/test_vpc_vpn.py | 10 ++
 5 files changed, 55 insertions(+), 1 deletion(-)

diff --git a/test/integration/smoke/test_internal_lb.py 
b/test/integration/smoke/test_internal_lb.py
index 5b4c663..379bbb0 100644
--- a/test/integration/smoke/test_internal_lb.py
+++ b/test/integration/smoke/test_internal_lb.py
@@ -50,6 +50,8 @@ from marvin.lib.common import (get_zone,
get_template,
list_network_offerings)
 
+from marvin.lib.decoratorGenerators import skipTestIf
+
 from nose.plugins.attrib import attr
 
 import logging
@@ -280,6 +282,7 @@ class TestInternalLb(cloudstackTestCase):
 
 cls.zone = get_zone(cls.apiclient, testClient.getZoneForTests())
 cls.domain = get_domain(cls.apiclient)
+
 cls.logger.debug("Creating compute offering: %s" 
%cls.services["compute_offering"]["name"])
 cls.compute_offering = ServiceOffering.create(
 cls.apiclient,
@@ -302,6 +305,12 @@ class TestInternalLb(cloudstackTestCase):
%s" % (cls.account.name,
   cls.account.id))
 
+# Skip rVR related test cases for VMware, for details see
+# https://issues.apache.org/jira/browse/CLOUDSTACK-10181
+cls.rvrNotSupported = False
+if cls.hypervisor.lower() in ['vmware']:
+cls.rvrNotSupported = True
+
 cls._cleanup = [cls.account, cls.compute_offering]
 return
 
@@ -598,6 +607,7 @@ class TestInternalLb(cloudstackTestCase):
 self.cleanup.insert(0, vpc_offering)
 self.execute_internallb_roundrobin_tests(vpc_offering)
 
+@skipTestIf("rvrNotSupported")
 @attr(tags=["smoke", "advanced"], required_hardware="true")
 def test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80(self):
 """
@@ -766,6 +776,7 @@ class TestInternalLb(cloudstackTestCase):
 
 self.execute_internallb_haproxy_tests(vpc_offering)
 
+@skipTestIf("rvrNotSupported")
 @attr(tags=["smoke", "advanced"], required_hardware="true")
 def test_04_rvpc_internallb_haproxy_stats_on_all_interfaces(self):
 """ Test to verify access to loadbalancer haproxy admin stats page
diff --git a/test/integration/smoke/test_privategw_acl.py 
b/test/integration/smoke/test_privategw_acl.py
index b86b96e..d48eb42 100644
--- a/test/integration/smoke/test_privategw_acl.py
+++ b/test/integration/smoke/test_privategw_acl.py
@@ -25,6 +25,7 @@ from marvin.lib.base import *
 from marvin.lib.common import *
 from nose.plugins.attrib import attr
 from marvin.codes import PASS
+from marvin.lib.decoratorGenerators import skipTestIf
 
 import time
 import logging
@@ -182,6 +183,14 @@ class TestPrivateGwACL(cloudstackTestCase):
 cls.services["service_offering"])
 cls._cleanup = [cls.service_offering]
 
+cls.hypervisor = cls.testClient.getHypervisorInfo()
+
+# Skip rVR related test cases for VMware, for details see
+# https://issues.apache.org/jira/browse/CLOUDSTACK-10181
+cls.rvrNotSupported = False
+if cls.hypervisor.lower() in ['vmware']:
+cls.rvrNotSupported = True
+
 cls.logger = logging.getLogger('TestPrivateGwACL')
 cls.stream_handler = logging.StreamHandler()
 cls.logger.setLevel(logging.DEBUG)
@@ -289,6 +298,7 @@ class TestPrivateGwACL(cloudstackTestCase):
 
 self.performVPCTests(vpc_off, restart_with_cleanup = True)
 
+@skipTestIf("rvrNotSupported")
 @attr(tags=["advanced"], required_hardware="true")
 def test_04_rvpc_privategw_static_routes(self):
 self.logger.debug("Creating a Redundant VPC offering..")
@@ -301,6 +311,7 @@ class TestPrivateGwACL(cloudstackTestCase):
 
 self.performVPCTests(vpc_off)
 
+@skipTestIf("rvrNotSupported")
 @attr(tags=["advanced"], required_hardware="true")
 def _test_05_rvpc_privategw_check_interface(self):
 self.logger.debug("Creating a Redundant VPC offering..")
diff --git a/test/integration/smoke/test_routers_network_ops.py 

[cloudstack] 01/21: CLOUDSTACK-10013: Migrate systemvmtemplate to Debian9

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git

commit a9df3bd09239003a8f854a17446a9a1069faf821
Author: Rohit Yadav 
AuthorDate: Sun Jul 23 18:01:35 2017 +0200

CLOUDSTACK-10013: Migrate systemvmtemplate to Debian9

  SystemVM changes to work on Debian 9
- Migrate away from chkconfig to systemctl
- Remove xenstore-utils override deb pkg
- Fix runlevel in sysv scripts for systemd

Signed-off-by: Rohit Yadav 
---
 systemvm/patches/debian/buildsystemvm.sh   | 577 -
 systemvm/patches/debian/config/etc/init.d/cloud|  10 +-
 .../debian/config/etc/init.d/cloud-early-config|  23 +-
 .../debian/config/etc/init.d/cloud-passwd-srvr |  10 +-
 systemvm/patches/debian/config/etc/init.d/postinit |   6 +-
 systemvm/patches/debian/config/etc/rc.local|   2 +-
 systemvm/patches/debian/config/etc/ssh/sshd_config |   1 -
 .../debian/config/opt/cloud/bin/patchsystemvm.sh   | 128 ++---
 tools/appliance/build.sh   |   2 +-
 .../definitions/systemvmtemplate/apt_upgrade.sh|   9 +-
 .../systemvmtemplate/authorized_keys.sh|   1 +
 .../definitions/systemvmtemplate/build_time.sh |  26 -
 .../definitions/systemvmtemplate/cleanup.sh|   1 -
 .../definitions/systemvmtemplate/configure_grub.sh |   6 +-
 .../systemvmtemplate/configure_login.sh|  15 +-
 .../systemvmtemplate/configure_networking.sh   |  25 +-
 .../configure_systemvm_services.sh |  94 +++-
 .../definitions/systemvmtemplate/definition.rb |  36 +-
 .../systemvmtemplate/install_systemvm_packages.sh  |  32 +-
 .../definitions/systemvmtemplate/preseed.cfg   |  69 ++-
 .../definitions/systemvmtemplate/zerodisk.sh   |   9 +-
 21 files changed, 267 insertions(+), 815 deletions(-)

diff --git a/systemvm/patches/debian/buildsystemvm.sh 
b/systemvm/patches/debian/buildsystemvm.sh
deleted file mode 100755
index a34b1dd..000
--- a/systemvm/patches/debian/buildsystemvm.sh
+++ /dev/null
@@ -1,577 +0,0 @@
-#!/bin/bash
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements.  See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership.  The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License.  You may obtain a copy of the License at
-# 
-#   http://www.apache.org/licenses/LICENSE-2.0
-# 
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied.  See the License for the
-# specific language governing permissions and limitations
-# under the License.
-
-echo ""
-echo " Note there is a new systemvm build script based on "
-echo " Veewee(Vagrant) under tools/appliance."
-echo ""
-
-set -e
-set -x
-
-IMAGENAME=systemvm
-LOCATION=/var/lib/images/systemvm
-PASSWORD=password
-#APT_PROXY=192.168.1.115:3142/
-APT_PROXY=
-HOSTNAME=systemvm
-SIZE=2000
-DEBIAN_MIRROR=ftp.us.debian.org/debian
-MINIMIZE=true
-CLOUDSTACK_RELEASE=4.0
-offset=4096
-baseimage() {
-  mkdir -p $LOCATION
-  #dd if=/dev/zero of=$IMAGELOC bs=1M  count=$SIZE
-  dd if=/dev/zero of=$IMAGELOC bs=1M seek=$((SIZE - 1)) count=1
-  loopdev=$(losetup -f)
-  losetup $loopdev $IMAGELOC
-  parted $loopdev -s 'mklabel msdos'
-  parted $loopdev -s 'mkpart primary ext3 4096B -1'
-  sleep 2 
-  losetup -d $loopdev
-  loopdev=$(losetup --show -o $offset -f $IMAGELOC )
-  mkfs.ext3  -L ROOT $loopdev
-  mkdir -p $MOUNTPOINT
-  tune2fs -c 100 -i 0 $loopdev
-  sleep 2 
-  losetup -d $loopdev
-  
-  mount -o loop,offset=$offset $IMAGELOC  $MOUNTPOINT
-  
-  #debootstrap --variant=minbase 
--keyring=/usr/share/keyrings/debian-archive-keyring.gpg wheezy $MOUNTPOINT 
http://${APT_PROXY}${DEBIAN_MIRROR}
-  debootstrap --variant=minbase --arch=i386 wheezy $MOUNTPOINT 
http://${APT_PROXY}${DEBIAN_MIRROR}
-}
-
-
-fixapt() {
-  if [ "$APT_PROXY" != "" ]; then
-  cat >> etc/apt/apt.conf.d/01proxy << EOF
-Acquire::http::Proxy "http://${APT_PROXY};;
-EOF
-  fi
-
-  cat > etc/apt/sources.list << EOF
-deb http://http.debian.net/debian/ wheezy main contrib non-free
-deb-src http://http.debian.net/debian/ wheezy main contrib non-free
-
-deb http://security.debian.org/ wheezy/updates main
-deb-src http://security.debian.org/ wheezy/updates main
-
-deb http://http.debian.net/debian/ wheezy-backports main
-deb-src http://http.debian.net/debian/ wheezy-backports main
-EOF
-
-  cat >> etc/apt/apt.conf << EOF
-APT::Default-Release "stable"; 

[cloudstack] 04/21: CLOUDSTACK-7853: Fix ping timeout edge case and refactor code

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git

commit 0560584e859c6c7776108eb9a16e8737fe1fc3a3
Author: Rohit Yadav 
AuthorDate: Sat Dec 2 15:50:34 2017 +0530

CLOUDSTACK-7853: Fix ping timeout edge case and refactor code

Refresh InaccurateClock every 10seconds, refactor code to get ping timeout
and ping interval.

Signed-off-by: Rohit Yadav 
---
 .../src/com/cloud/agent/manager/AgentManagerImpl.java| 16 
 .../main/java/com/cloud/utils/time/InaccurateClock.java  |  2 +-
 2 files changed, 9 insertions(+), 9 deletions(-)

diff --git 
a/engine/orchestration/src/com/cloud/agent/manager/AgentManagerImpl.java 
b/engine/orchestration/src/com/cloud/agent/manager/AgentManagerImpl.java
index 325f3ec..7815c76 100644
--- a/engine/orchestration/src/com/cloud/agent/manager/AgentManagerImpl.java
+++ b/engine/orchestration/src/com/cloud/agent/manager/AgentManagerImpl.java
@@ -213,7 +213,7 @@ public class AgentManagerImpl extends ManagerBase 
implements AgentManager, Handl
 _nodeId = ManagementServerNode.getManagementServerId();
 s_logger.info("Configuring AgentManagerImpl. management server node 
id(msid): " + _nodeId);
 
-final long lastPing = (System.currentTimeMillis() >> 10) - (long) 
(PingTimeout.value() * PingInterval.value());
+final long lastPing = (System.currentTimeMillis() >> 10) - 
getTimeout();
 _hostDao.markHostsAsDisconnected(_nodeId, lastPing);
 
 registerForHostEvents(new BehindOnPingListener(), true, true, false);
@@ -241,8 +241,12 @@ public class AgentManagerImpl extends ManagerBase 
implements AgentManager, Handl
 return true;
 }
 
+protected int getPingInterval() {
+return PingInterval.value();
+}
+
 protected long getTimeout() {
-return (long) (PingTimeout.value() * PingInterval.value());
+return (long) (Math.ceil(PingTimeout.value() * PingInterval.value()));
 }
 
 @Override
@@ -358,10 +362,6 @@ public class AgentManagerImpl extends ManagerBase 
implements AgentManager, Handl
 return null;
 }
 
-protected int getPingInterval() {
-return PingInterval.value();
-}
-
 @Override
 public Answer send(final Long hostId, final Command cmd) throws 
AgentUnavailableException, OperationTimedoutException {
 final Commands cmds = new Commands(Command.OnError.Stop);
@@ -623,7 +623,7 @@ public class AgentManagerImpl extends ManagerBase 
implements AgentManager, Handl
 }
 }
 
-_monitorExecutor.scheduleWithFixedDelay(new MonitorTask(), 
PingInterval.value(), PingInterval.value(), TimeUnit.SECONDS);
+_monitorExecutor.scheduleWithFixedDelay(new MonitorTask(), 
getPingInterval(), getPingInterval(), TimeUnit.SECONDS);
 
 return true;
 }
@@ -1515,7 +1515,7 @@ public class AgentManagerImpl extends ManagerBase 
implements AgentManager, Handl
 attache = createAttacheForDirectConnect(host, resource);
 final StartupAnswer[] answers = new StartupAnswer[cmds.length];
 for (int i = 0; i < answers.length; i++) {
-answers[i] = new StartupAnswer(cmds[i], attache.getId(), 
PingInterval.value());
+answers[i] = new StartupAnswer(cmds[i], attache.getId(), 
getPingInterval());
 }
 attache.process(answers);
 
diff --git a/utils/src/main/java/com/cloud/utils/time/InaccurateClock.java 
b/utils/src/main/java/com/cloud/utils/time/InaccurateClock.java
index 2a22853..e03231d 100644
--- a/utils/src/main/java/com/cloud/utils/time/InaccurateClock.java
+++ b/utils/src/main/java/com/cloud/utils/time/InaccurateClock.java
@@ -63,7 +63,7 @@ public class InaccurateClock extends StandardMBean implements 
InaccurateClockMBe
 public synchronized String restart() {
 turnOff();
 s_executor = Executors.newScheduledThreadPool(1, new 
NamedThreadFactory("InaccurateClock"));
-s_executor.scheduleAtFixedRate(new SetTimeTask(), 0, 60, 
TimeUnit.SECONDS);
+s_executor.scheduleAtFixedRate(new SetTimeTask(), 0, 10, 
TimeUnit.SECONDS);
 return "Restarted";
 }
 

-- 
To stop receiving notification emails like this one, please contact
"commits@cloudstack.apache.org" .

[cloudstack] 14/21: CLOUDSTACK-10187: Don't delete vifs for VPCs with source nat

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git

commit 02d479b928c830687d877a2771fbc4bc14fa2112
Author: Rohit Yadav 
AuthorDate: Wed Dec 13 01:28:18 2017 +0530

CLOUDSTACK-10187: Don't delete vifs for VPCs with source nat

On XenServer, both redundant router's vifs were getting deleted when any
PF rule is removed from any of the acquired public IPs. This fix
ensures that lastIp is set to `false` when processed by hypervisor
resources to avoid removing of VIFs when VPCs have any source nat IP.

Signed-off-by: Rohit Yadav 
---
 .../com/cloud/network/router/CommandSetupHelper.java  | 19 ++-
 test/integration/smoke/test_vpc_redundant.py  |  4 ++--
 2 files changed, 16 insertions(+), 7 deletions(-)

diff --git a/server/src/com/cloud/network/router/CommandSetupHelper.java 
b/server/src/com/cloud/network/router/CommandSetupHelper.java
index cadaf4f..c8d85fe 100644
--- a/server/src/com/cloud/network/router/CommandSetupHelper.java
+++ b/server/src/com/cloud/network/router/CommandSetupHelper.java
@@ -475,7 +475,7 @@ public class CommandSetupHelper {
 
 public void createAssociateIPCommands(final VirtualRouter router, final 
List ips, final Commands cmds, final long vmId) {
 final String ipAssocCommand = "IPAssocCommand";
-createRedundantAssociateIPCommands(router, ips, cmds, ipAssocCommand, 
vmId);
+createRedundantAssociateIPCommands(router, ips, cmds, ipAssocCommand, 
false);
 }
 
 public void createNetworkACLsCommands(final List 
rules, final VirtualRouter router, final Commands cmds, final long 
guestNetworkId,
@@ -645,7 +645,7 @@ public class CommandSetupHelper {
 
 final String ipAssocCommand = "IPAssocVpcCommand";
 if (router.getIsRedundantRouter()) {
-createRedundantAssociateIPCommands(router, ips, cmds, 
ipAssocCommand, 0);
+createRedundantAssociateIPCommands(router, ips, cmds, 
ipAssocCommand, true);
 return;
 }
 
@@ -742,7 +742,7 @@ public class CommandSetupHelper {
 }
 }
 
-public void createRedundantAssociateIPCommands(final VirtualRouter router, 
final List ips, final Commands cmds, final String 
ipAssocCommand, final long vmId) {
+public void createRedundantAssociateIPCommands(final VirtualRouter router, 
final List ips, final Commands cmds, final String 
ipAssocCommand, final boolean isVPC) {
 
 // Ensure that in multiple vlans case we first send all ip addresses of
 // vlan1, then all ip addresses of vlan2, etc..
@@ -840,7 +840,16 @@ public class CommandSetupHelper {
 }
 
 // for network if the ips does not have any rules, then only last 
ip
-List userIps = 
_ipAddressDao.listByAssociatedNetwork(associatedWithNetworkId, null);
+final List userIps = 
_ipAddressDao.listByAssociatedNetwork(associatedWithNetworkId, null);
+boolean hasSourceNat = false;
+if (isVPC && userIps.size() > 0 && userIps.get(0) != null) {
+// All ips should belong to a VPC
+final Long vpcId = userIps.get(0).getVpcId();
+final List sourceNatIps = 
_ipAddressDao.listByAssociatedVpc(vpcId, true);
+if (sourceNatIps != null && sourceNatIps.size() > 0) {
+hasSourceNat = true;
+}
+}
 
 int ipsWithrules = 0;
 int ipsStaticNat = 0;
@@ -864,7 +873,7 @@ public class CommandSetupHelper {
 cmd.setAccessDetail(NetworkElementCommand.ZONE_NETWORK_TYPE, 
dcVo.getNetworkType().toString());
 
 // if there is 1 static nat then it will be checked for remove at 
the resource
-if (ipsWithrules == 0 && ipsStaticNat == 0) {
+if (ipsWithrules == 0 && ipsStaticNat == 0 && !hasSourceNat) {
 // there is only one ip address for the network.
 cmd.setAccessDetail(NetworkElementCommand.NETWORK_PUB_LAST_IP, 
"true");
 } else {
diff --git a/test/integration/smoke/test_vpc_redundant.py 
b/test/integration/smoke/test_vpc_redundant.py
index b87ce43..e7b29b1 100644
--- a/test/integration/smoke/test_vpc_redundant.py
+++ b/test/integration/smoke/test_vpc_redundant.py
@@ -610,7 +610,7 @@ class TestVPCRedundancy(cloudstackTestCase):
 
 gc_wait = Configurations.list(self.apiclient, name="network.gc.wait")
 gc_interval = Configurations.list(self.apiclient, 
name="network.gc.interval")
-
+
 self.logger.debug("network.gc.wait is ==> %s" % gc_wait)
 self.logger.debug("network.gc.interval is ==> %s" % gc_interval)
 
@@ -628,7 +628,7 @@ class TestVPCRedundancy(cloudstackTestCase):
 
 @attr(tags=["advanced", "intervlan"], required_hardware="true")
 def 

[cloudstack] 17/21: CLOUDSTACK-10013: Fix VMware related issues and fix misc tests

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git

commit 463bb0a16de7c3f6231f7af3c37607474224329a
Author: Rohit Yadav 
AuthorDate: Wed Dec 13 01:42:05 2017 +0530

CLOUDSTACK-10013: Fix VMware related issues and fix misc tests

This fixes test failures around VMware with the new systemvmtemplate.
In addition:

- Does not skip rVR related test cases for VMware
- Removes rc.local
- Processes unprocessed cmd_line.json
- Fixed NPEs around VMware tests/code
- On VMware, use udevadm to reconfigure nic/mac address than rebooting
- Fix proper acpi shutdown script for faster systemvm shutdowns
- Give at least 256MB of swap for VRs to avoid OOM on VMware
- Fixes smoke tests for environment related failures

Signed-off-by: Rohit Yadav 
---
 .../com/cloud/vm/VirtualMachineManagerImpl.java|  2 +-
 python/lib/cloudutils/utilities.py |  4 +-
 server/src/com/cloud/server/StatsCollector.java| 20 +++-
 systemvm/agent/conf/log4j-cloud.xml| 12 ++---
 systemvm/debian/etc/issue  |  3 --
 systemvm/debian/etc/rc.local   | 19 
 systemvm/debian/opt/cloud/bin/cs/CsDhcp.py |  2 +-
 systemvm/debian/opt/cloud/bin/setup/common.sh  |  8 ++--
 systemvm/debian/opt/cloud/bin/setup/router.sh  |  6 +--
 systemvm/debian/opt/cloud/bin/setup/vpcrouter.sh   |  7 +--
 systemvm/debian/opt/cloud/bin/update_config.py |  6 +++
 .../opt/cloud/templates/keepalived.conf.templ  |  2 +-
 .../smoke/test_deploy_virtio_scsi_vm.py|  3 +-
 .../smoke/test_deploy_vm_root_resize.py| 56 +-
 test/integration/smoke/test_internal_lb.py | 10 
 test/integration/smoke/test_iso.py |  0
 test/integration/smoke/test_list_ids_parameter.py  |  0
 .../smoke/test_nested_virtualization.py|  0
 test/integration/smoke/test_privategw_acl.py   |  8 
 test/integration/smoke/test_routers_network_ops.py | 10 
 test/integration/smoke/test_ssvm.py|  4 +-
 test/integration/smoke/test_vm_life_cycle.py   |  0
 test/integration/smoke/test_volumes.py |  2 +
 test/integration/smoke/test_vpc_redundant.py   | 10 
 test/integration/smoke/test_vpc_vpn.py | 10 
 .../systemvmtemplate/configure_acpid.sh| 13 ++---
 .../definitions/systemvmtemplate/configure_grub.sh |  4 +-
 .../configure_systemvm_services.sh | 10 
 .../systemvmtemplate/install_systemvm_packages.sh  |  4 +-
 .../definitions/systemvmtemplate/preseed.cfg   |  6 +--
 ui/scripts/network.js  |  2 +-
 31 files changed, 92 insertions(+), 151 deletions(-)

diff --git 
a/engine/orchestration/src/com/cloud/vm/VirtualMachineManagerImpl.java 
b/engine/orchestration/src/com/cloud/vm/VirtualMachineManagerImpl.java
index 74927b9..50e53e3 100755
--- a/engine/orchestration/src/com/cloud/vm/VirtualMachineManagerImpl.java
+++ b/engine/orchestration/src/com/cloud/vm/VirtualMachineManagerImpl.java
@@ -4743,8 +4743,8 @@ public class VirtualMachineManagerImpl extends 
ManagerBase implements VirtualMac
 final VMInstanceVO vm = _entityMgr.findById(VMInstanceVO.class, 
work.getVmId());
 if (vm == null) {
 s_logger.info("Unable to find vm " + work.getVmId());
+throw new CloudRuntimeException("Unable to find VM id=" + 
work.getVmId());
 }
-assert vm != null;
 
 orchestrateStop(vm.getUuid(), work.isCleanup());
 return new Pair(JobInfo.Status.SUCCEEDED, 
null);
diff --git a/python/lib/cloudutils/utilities.py 
b/python/lib/cloudutils/utilities.py
index 05a2a58..b9d4763 100755
--- a/python/lib/cloudutils/utilities.py
+++ b/python/lib/cloudutils/utilities.py
@@ -216,8 +216,8 @@ class serviceOpsUbuntu(serviceOps):
 class serviceOpsRedhat7(serviceOps):
 def isServiceRunning(self, servicename):
 try:
-o = bash("systemctl status " + servicename)
-if "running" in o.getStdout() or "start" in o.getStdout() or 
"Running" in o.getStdout():
+o = bash("systemctl is-active " + servicename)
+if "inactive" not in o.getStdout():
 return True
 else:
 return False
diff --git a/server/src/com/cloud/server/StatsCollector.java 
b/server/src/com/cloud/server/StatsCollector.java
index d32ed98..b66fa5f 100644
--- a/server/src/com/cloud/server/StatsCollector.java
+++ b/server/src/com/cloud/server/StatsCollector.java
@@ -64,6 +64,7 @@ import com.cloud.cluster.ManagementServerHostVO;
 import com.cloud.cluster.dao.ManagementServerHostDao;
 import com.cloud.dc.Vlan.VlanType;
 import com.cloud.dc.VlanVO;
+import 

[cloudstack] 09/21: CLOUDSTACK-10010: Port marvin test from #2190 by @swill

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git

commit 0e4de4bf10c0fce3e88515dd2a290e5570e1ae0f
Author: Will Stevens 
AuthorDate: Sat Dec 2 21:11:40 2017 +0530

CLOUDSTACK-10010: Port marvin test from #2190 by @swill

This ports the S2S config test by @swill from #2190 with additional
changes to make robust and environment agnostic.

Signed-off-by: Rohit Yadav 
---
 test/integration/smoke/test_vpc_vpn.py | 468 -
 1 file changed, 463 insertions(+), 5 deletions(-)

diff --git a/test/integration/smoke/test_vpc_vpn.py 
b/test/integration/smoke/test_vpc_vpn.py
index 83c244b..b3dfab2 100644
--- a/test/integration/smoke/test_vpc_vpn.py
+++ b/test/integration/smoke/test_vpc_vpn.py
@@ -1138,8 +1138,19 @@ class TestRVPCSite2SiteVpn(cloudstackTestCase):
 self.apiclient, customer2_response.id, vpn1_response['id'])
 self.debug("VPN connection created for VPC %s" % vpc1.id)
 
-self.assertEqual(
-vpnconn2_response['state'], "Connected", "Failed to connect 
between VPCs!")
+def checkVpnConnected():
+connections = Vpn.listVpnConnection(
+self.apiclient,
+listall='true',
+vpcid=vpc2.id)
+if isinstance(connections, list):
+return connections[0].state == 'Connected', None
+return False, None
+
+# Wait up to 60 seconds for passive connection to show up as Connected
+res, _ = wait_until(2, 30, checkVpnConnected)
+if not res:
+self.fail("Failed to connect between VPCs, see VPN state as 
Connected")
 
 # acquire an extra ip address to use to ssh into vm2
 try:
@@ -1174,9 +1185,8 @@ class TestRVPCSite2SiteVpn(cloudstackTestCase):
 
 if ssh_client:
 # run ping test
-packet_loss = ssh_client.execute(
-"/bin/ping -c 3 -t 10 " + vm1.nic[0].ipaddress + " |grep 
packet|cut -d ' ' -f 7| cut -f1 -d'%'")[0]
-self.assert_(int(packet_loss) == 0, "Ping did not succeed")
+packet_loss = ssh_client.execute("/bin/ping -c 3 -t 10 " + 
vm1.nic[0].ipaddress + " | grep packet | sed 's/.*received, //g' | sed 's/[% 
]*packet.*//g'")[0]
+self.assert_(int(packet_loss) < 50, "Ping did not succeed")
 else:
 self.fail("Failed to setup ssh connection to %s" % vm2.public_ip)
 
@@ -1189,3 +1199,451 @@ class TestRVPCSite2SiteVpn(cloudstackTestCase):
 cleanup_resources(cls.apiclient, cls.cleanup)
 except Exception, e:
 raise Exception("Cleanup failed with %s" % e)
+
+
+class TestVPCSite2SiteVPNMultipleOptions(cloudstackTestCase):
+
+@classmethod
+def setUpClass(cls):
+cls.logger = logging.getLogger('TestVPCSite2SiteVPNMultipleOptions')
+cls.stream_handler = logging.StreamHandler()
+cls.logger.setLevel(logging.DEBUG)
+cls.logger.addHandler(cls.stream_handler)
+
+testClient = super(TestVPCSite2SiteVPNMultipleOptions, 
cls).getClsTestClient()
+cls.apiclient = testClient.getApiClient()
+cls.services = Services().services
+
+cls.zone = get_zone(cls.apiclient, testClient.getZoneForTests())
+cls.domain = get_domain(cls.apiclient)
+
+cls.compute_offering = ServiceOffering.create(
+cls.apiclient,
+cls.services["compute_offering"]
+)
+
+cls.account = Account.create(
+cls.apiclient, services=cls.services["account"])
+
+cls.hypervisor = testClient.getHypervisorInfo()
+
+cls.logger.debug("Downloading Template: %s from: %s" % 
(cls.services["template"][
+ cls.hypervisor.lower()], 
cls.services["template"][cls.hypervisor.lower()]["url"]))
+cls.template = Template.register(cls.apiclient, 
cls.services["template"][cls.hypervisor.lower(
+)], cls.zone.id, hypervisor=cls.hypervisor.lower(), 
account=cls.account.name, domainid=cls.domain.id)
+cls.template.download(cls.apiclient)
+
+if cls.template == FAILED:
+assert False, "get_template() failed to return template"
+
+cls.logger.debug("Successfully created account: %s, id: \
+   %s" % (cls.account.name,
+  cls.account.id))
+
+cls.cleanup = [cls.account, cls.compute_offering]
+return
+
+def _get_ssh_client(self, virtual_machine, services, retries):
+""" Setup ssh client connection and return connection
+vm requires attributes public_ip, public_port, username, password """
+
+try:
+ssh_client = SshClient(
+virtual_machine.public_ip,
+services["virtual_machine"]["ssh_port"],
+

[cloudstack] 06/21: CLOUDSTACK-10129: UX improvements and event timeline

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git

commit 406e71424d394c6d4dd231718cfcdd0b6f20f6af
Author: Rohit Yadav 
AuthorDate: Tue Nov 28 19:25:09 2017 +0530

CLOUDSTACK-10129: UX improvements and event timeline

- Fixes timezone issue where dates show up as nvalid in UI
- Introduces new event timeline listing/filtering of events
- Several UI improvements to add columns in list views
- Bulk operations support in instance list view to shutdown and destroy
  multiple-selected VMs (limitation: after operation, redundant entries
  may show up in the list view, refreshing VM list view fixes that)
- Align table thead/tbody to avoid splitting of tables

Signed-off-by: Rohit Yadav 
---
 .../org/apache/cloudstack/api/ApiConstants.java|   2 +
 .../api/command/user/event/ListEventsCmd.java  |   7 +
 .../cloudstack/api/response/EventResponse.java |   2 +-
 .../schema/src/com/cloud/user/UserAccountVO.java   |   4 +
 engine/schema/src/com/cloud/user/UserVO.java   |   4 +
 .../src/com/cloud/api/query/QueryManagerImpl.java  |  10 +-
 ui/css/cloudstack3.css |   3 +-
 ui/l10n/en.js  |   3 +
 ui/scripts/events.js   |  20 +-
 ui/scripts/instances.js| 275 -
 ui/scripts/metrics.js  |  13 +-
 ui/scripts/network.js  |  51 +++-
 ui/scripts/sharedFunctions.js  |   2 +-
 ui/scripts/storage.js  |  30 ++-
 ui/scripts/system.js   | 112 +++--
 ui/scripts/templates.js|  27 ++
 ui/scripts/ui/widgets/dataTable.js |  13 +-
 17 files changed, 412 insertions(+), 166 deletions(-)

diff --git a/api/src/org/apache/cloudstack/api/ApiConstants.java 
b/api/src/org/apache/cloudstack/api/ApiConstants.java
index a5bd95f..89deeef 100644
--- a/api/src/org/apache/cloudstack/api/ApiConstants.java
+++ b/api/src/org/apache/cloudstack/api/ApiConstants.java
@@ -205,6 +205,7 @@ public class ApiConstants {
 public static final String OUTOFBANDMANAGEMENT_POWERSTATE = 
"outofbandmanagementpowerstate";
 public static final String OUTOFBANDMANAGEMENT_ENABLED = 
"outofbandmanagementenabled";
 public static final String PARAMS = "params";
+public static final String PARENT_ID = "parentid";
 public static final String PARENT_DOMAIN_ID = "parentdomainid";
 public static final String PASSWORD = "password";
 public static final String SHOULD_UPDATE_PASSWORD = 
"update_passwd_on_host";
@@ -274,6 +275,7 @@ public class ApiConstants {
 public static final String SNAPSHOT_QUIESCEVM = "quiescevm";
 public static final String SOURCE_ZONE_ID = "sourcezoneid";
 public static final String START_DATE = "startdate";
+public static final String START_ID = "startid";
 public static final String START_IP = "startip";
 public static final String START_IPV6 = "startipv6";
 public static final String START_PORT = "startport";
diff --git 
a/api/src/org/apache/cloudstack/api/command/user/event/ListEventsCmd.java 
b/api/src/org/apache/cloudstack/api/command/user/event/ListEventsCmd.java
index a4934fa..b98c308 100644
--- a/api/src/org/apache/cloudstack/api/command/user/event/ListEventsCmd.java
+++ b/api/src/org/apache/cloudstack/api/command/user/event/ListEventsCmd.java
@@ -65,6 +65,9 @@ public class ListEventsCmd extends 
BaseListProjectAndAccountResourcesCmd {
 @Parameter(name = ApiConstants.TYPE, type = CommandType.STRING, 
description = "the event type (see event types)")
 private String type;
 
+@Parameter(name = ApiConstants.START_ID, type = CommandType.UUID, 
entityType = EventResponse.class, description = "the parent/start ID of the 
event, when provided this will list all the events with the start/parent ID 
including the parent event")
+private Long startId;
+
 /
 /// Accessors ///
 /
@@ -97,6 +100,10 @@ public class ListEventsCmd extends 
BaseListProjectAndAccountResourcesCmd {
 return type;
 }
 
+public Long getStartId() {
+return startId;
+}
+
 /
 /// API Implementation///
 /
diff --git a/api/src/org/apache/cloudstack/api/response/EventResponse.java 
b/api/src/org/apache/cloudstack/api/response/EventResponse.java
index 5ce66ed..da15434 100644
--- a/api/src/org/apache/cloudstack/api/response/EventResponse.java
+++ 

[cloudstack] 18/21: CLOUDSTACK-10193: Use other64Linux for systemvms on VMware

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git

commit a9175efff249992b39c9e214556f72a09c98420e
Author: Rohit Yadav 
AuthorDate: Sun Dec 17 12:26:35 2017 +0530

CLOUDSTACK-10193: Use other64Linux for systemvms on VMware

In default/fresh installations, the guest os type for systemvms with id=15
or Debian 5 (32-bit) can cause memory allocation issues to guest. Using
Other Linux 64-bit as guest OS systemvms get all the allocated RAM. This
avoids OOM related kernel panics for certain VRs such as rVRs, lbvm etc.

Signed-off-by: Rohit Yadav 
---
 engine/schema/resources/META-INF/db/schema-41000to41100.sql | 4 
 1 file changed, 4 insertions(+)

diff --git a/engine/schema/resources/META-INF/db/schema-41000to41100.sql 
b/engine/schema/resources/META-INF/db/schema-41000to41100.sql
index 2a2fe2d..76bcb3f 100644
--- a/engine/schema/resources/META-INF/db/schema-41000to41100.sql
+++ b/engine/schema/resources/META-INF/db/schema-41000to41100.sql
@@ -495,3 +495,7 @@ UPDATE `cloud`.`monitoring_services` SET 
pidfile="/var/run/apache2/apache2.pid"
 
 -- Boost secondary storage systemvm
 UPDATE `cloud`.`service_offering` SET ram_size=1024, cpu=2 WHERE 
vm_type="secondarystoragevm" and cpu=1 and ram_size=512;
+
+-- Use 'Other Linux 64-bit' as guest os for the default systemvmtemplate for 
VMware
+-- This fixes a memory allocation issue to systemvms on VMware/ESXi
+UPDATE `cloud`.`vm_template` SET guest_os_id=99 WHERE id=8;

-- 
To stop receiving notification emails like this one, please contact
"commits@cloudstack.apache.org" .

[cloudstack] 21/21: CLOUDSTACK-10001: Fix incorrect total host memory in responses

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git

commit 69a57a3a195691a318747ae3bbcf7049ca8d9397
Author: Rohit Yadav 
AuthorDate: Tue Dec 19 11:45:24 2017 +0530

CLOUDSTACK-10001: Fix incorrect total host memory in responses

This fixes incorrect total host memory in listHosts and related host
responses, regression introduced in #2120.

Signed-off-by: Rohit Yadav 
---
 server/src/com/cloud/api/query/dao/HostJoinDaoImpl.java | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/server/src/com/cloud/api/query/dao/HostJoinDaoImpl.java 
b/server/src/com/cloud/api/query/dao/HostJoinDaoImpl.java
index 4d411f2..4cb8a76 100644
--- a/server/src/com/cloud/api/query/dao/HostJoinDaoImpl.java
+++ b/server/src/com/cloud/api/query/dao/HostJoinDaoImpl.java
@@ -159,8 +159,8 @@ public class HostJoinDaoImpl extends 
GenericDaoBase implements
 Long mem = host.getMemReservedCapacity() + 
host.getMemUsedCapacity();
 Long cpu = host.getCpuReservedCapacity() + 
host.getCpuUsedCapacity();
 
-hostResponse.setMemoryTotal(mem);
-Float totalMemorywithOverprovisioning=new 
Float((host.getTotalMemory()*ApiDBUtils.getMemOverprovisioningFactor(host.getClusterId(;
+hostResponse.setMemoryTotal(host.getTotalMemory());
+Float totalMemorywithOverprovisioning = host.getTotalMemory() 
* ApiDBUtils.getMemOverprovisioningFactor(host.getClusterId());
 
hostResponse.setMemWithOverprovisioning(totalMemorywithOverprovisioning.toString());
 hostResponse.setMemoryAllocated(mem);
 
@@ -180,7 +180,7 @@ public class HostJoinDaoImpl extends 
GenericDaoBase implements
 
 hostResponse.setHypervisorVersion(host.getHypervisorVersion());
 
-Float cpuWithOverprovisioning = new Float(host.getCpus() * 
host.getSpeed() * ApiDBUtils.getCpuOverprovisioningFactor(host.getClusterId()));
+Float cpuWithOverprovisioning = host.getCpus() * 
host.getSpeed() * ApiDBUtils.getCpuOverprovisioningFactor(host.getClusterId());
 String cpuAlloc = decimalFormat.format(((float)cpu / 
cpuWithOverprovisioning * 100f)) + "%";
 hostResponse.setCpuAllocated(cpuAlloc);
 
hostResponse.setCpuWithOverprovisioning(cpuWithOverprovisioning.toString());
@@ -308,10 +308,10 @@ public class HostJoinDaoImpl extends 
GenericDaoBase implements
 Long mem = host.getMemReservedCapacity() + 
host.getMemUsedCapacity();
 Long cpu = host.getCpuReservedCapacity() + 
host.getCpuReservedCapacity();
 
-hostResponse.setMemoryTotal(mem);
-Float memWithOverprovisioning =new 
Float((host.getTotalMemory()*ApiDBUtils.getMemOverprovisioningFactor(host.getClusterId(;
+hostResponse.setMemoryTotal(host.getTotalMemory());
+Float memWithOverprovisioning = host.getTotalMemory() * 
ApiDBUtils.getMemOverprovisioningFactor(host.getClusterId());
 
hostResponse.setMemWithOverprovisioning(memWithOverprovisioning.toString());
-
hostResponse.setMemoryAllocated(decimalFormat.format((float)mem/ 
memWithOverprovisioning*100f).toString()+"%");
+hostResponse.setMemoryAllocated(decimalFormat.format((float) 
mem / memWithOverprovisioning * 100.0f) +"%");
 
 String hostTags = host.getTag();
 hostResponse.setHostTags(host.getTag());

-- 
To stop receiving notification emails like this one, please contact
"commits@cloudstack.apache.org" .

[cloudstack] 10/21: CLOUDSTACK-10013: Make the generated VR/json files unique (ports #1470)

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git

commit 427df5056eadd98479011f0667d13090b12fe821
Author: Remi Bergsma 
AuthorDate: Sat Dec 2 23:19:14 2017 +0530

CLOUDSTACK-10013: Make the generated VR/json files unique (ports #1470)

This ports PR #1470 by @remibergsma.

Make the generated json files unique to prevent concurrency issues:
The json files now have UUIDs to prevent them from getting overwritten
before they've been executed. Prevents config to be pushed to the wrong
router.

2016-02-25 18:32:23,797 DEBUG [c.c.a.t.Request] 
(AgentManager-Handler-1:null) (logid:) Seq 2-4684025087442026584: Processing:  
{ Ans: , MgmtId: 90520732674657, via: 2, Ver: v1, Flags: 10, 
[{"com.cloud.agent.api.routing.GroupA
nswer":{"results":["null - success: null","null - success: [INFO] 
update_config.py :: Processing incoming file => 
vm_dhcp_entry.json.4ea45061-2efb-4467-8eaa-db3d77fb0a7b\n[INFO] Processing JSON 
file vm_dhcp_entry.json.4ea4506
1-2efb-4467-8eaa-db3d77fb0a7b\n"],"result":true,"wait":0}}] }

On the router:
2016-02-25 18:32:23,416  merge.py __moveFile:298 Processed file written to 
/var/cache/cloud/processed/vm_dhcp_entry.json.4ea45061-2efb-4467-8eaa-db3d77fb0a7b.gz

Signed-off-by: Rohit Yadav 
---
 .../facade/AbstractConfigItemFacade.java   |  21 +-
 .../kvm/resource/LibvirtComputingResource.java |   5 +
 systemvm/patches/debian/etc/rc.local   |   9 -
 systemvm/patches/debian/opt/cloud/bin/configure.py | 220 +
 systemvm/patches/debian/opt/cloud/bin/merge.py |  37 ++--
 .../patches/debian/opt/cloud/bin/update_config.py  |  22 ++-
 6 files changed, 159 insertions(+), 155 deletions(-)

diff --git 
a/core/src/com/cloud/agent/resource/virtualnetwork/facade/AbstractConfigItemFacade.java
 
b/core/src/com/cloud/agent/resource/virtualnetwork/facade/AbstractConfigItemFacade.java
index f017384..a083012 100644
--- 
a/core/src/com/cloud/agent/resource/virtualnetwork/facade/AbstractConfigItemFacade.java
+++ 
b/core/src/com/cloud/agent/resource/virtualnetwork/facade/AbstractConfigItemFacade.java
@@ -22,6 +22,8 @@ package com.cloud.agent.resource.virtualnetwork.facade;
 import java.util.Hashtable;
 import java.util.LinkedList;
 import java.util.List;
+import java.util.UUID;
+import org.apache.log4j.Logger;
 
 import com.cloud.agent.api.BumpUpPriorityCommand;
 import com.cloud.agent.api.SetupGuestNetworkCommand;
@@ -58,6 +60,8 @@ import com.google.gson.GsonBuilder;
 
 public abstract class AbstractConfigItemFacade {
 
+private static final Logger s_logger = 
Logger.getLogger(AbstractConfigItemFacade.class);
+
 private final static Gson gson;
 
 private static Hashtable flyweight = new Hashtable();
@@ -104,13 +108,26 @@ public abstract class AbstractConfigItemFacade {
 return instance;
 }
 
+private static String appendUuidToJsonFiles(final String filename) {
+String remoteFileName = new String(filename);
+if (remoteFileName.endsWith("json")) {
+remoteFileName += "." + UUID.randomUUID().toString();
+}
+return remoteFileName;
+}
+
 protected List generateConfigItems(final ConfigBase 
configuration) {
 final List cfg = new LinkedList<>();
 
-final ConfigItem configFile = new 
FileConfigItem(VRScripts.CONFIG_PERSIST_LOCATION, destinationFile, 
gson.toJson(configuration));
+final String remoteFilename = appendUuidToJsonFiles(destinationFile);
+if (s_logger.isDebugEnabled()) {
+s_logger.debug("Transformed filename: " + destinationFile + " to: 
" + remoteFilename);
+}
+
+final ConfigItem configFile = new 
FileConfigItem(VRScripts.CONFIG_PERSIST_LOCATION, remoteFilename, 
gson.toJson(configuration));
 cfg.add(configFile);
 
-final ConfigItem updateCommand = new 
ScriptConfigItem(VRScripts.UPDATE_CONFIG, destinationFile);
+final ConfigItem updateCommand = new 
ScriptConfigItem(VRScripts.UPDATE_CONFIG, remoteFilename);
 cfg.add(updateCommand);
 
 return cfg;
diff --git 
a/plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
 
b/plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
index 51b9737..ebd36bd 100644
--- 
a/plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
+++ 
b/plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
@@ -330,6 +330,9 @@ public class LibvirtComputingResource extends 
ServerResourceBase implements Serv
 if (details == null) {
 details = parser.getLines();
 }
+
+

[cloudstack] 02/21: CLOUDSTACK-10013: Make Debian9 systemvmtemplates work with systemd

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git

commit b9ffbeea027e72d6b197fcf819ba8df6817ba018
Author: Wido den Hollander 
AuthorDate: Thu Aug 10 13:51:11 2017 +0200

CLOUDSTACK-10013: Make Debian9 systemvmtemplates work with systemd

- Load the nf_conntrack_ipv6 module for IPv6 connection tracking on SSVM
- Move systemd services to /etc and enable services after they have been
  installed
- Disable most services by default and enable in cloud-early-config
- Start services after enabling them using systemd
- In addition remove /etc/init.d/cloud as this is no longer needed and
  done by systemd
- Accept DOS/MBR as file format for ISO images as well

Under Debian 7 the 'file' command would return:

  debian-9.1.0-amd64-netinst.iso: ISO 9660 CD-ROM filesystem data UDF 
filesystem data

Under Debian 9 however it will return

  debian-9.1.0-amd64-netinst.iso: DOS/MBR boot sector

This would make the HTTPTemplateDownloader in the Secondary Storage VM 
refuse the ISO as
a valid template because it's not a correct format.

Changes this behavior so that it accepts both.
This allows us to use Debian 9 as a System VM template.

Not sure though if enabling them is enough for systemd to still start them
on first boot

Signed-off-by: Wido den Hollander 
---
 .gitignore |   2 +
 systemvm/patches/debian/config/etc/init.d/cloud| 155 -
 .../debian/config/etc/init.d/cloud-early-config|  55 ++--
 .../systemvmtemplate/configure_conntrack.sh|   1 +
 .../configure_systemvm_services.sh |  22 +--
 .../utils/imagestore/ImageStoreUtil.java   |   2 +-
 6 files changed, 30 insertions(+), 207 deletions(-)

diff --git a/.gitignore b/.gitignore
index 29b4ffc..1a73724 100644
--- a/.gitignore
+++ b/.gitignore
@@ -97,3 +97,5 @@ systemvm/.pydevproject
 test/.pydevprojec
 plugins/hypervisors/kvm/.pydevproject
 scripts/.pydevproject
+*.qcow2
+*.raw
diff --git a/systemvm/patches/debian/config/etc/init.d/cloud 
b/systemvm/patches/debian/config/etc/init.d/cloud
deleted file mode 100755
index 9b3a63b..000
--- a/systemvm/patches/debian/config/etc/init.d/cloud
+++ /dev/null
@@ -1,155 +0,0 @@
-#!/bin/bash
-### BEGIN INIT INFO
-# Provides:  cloud
-# Required-Start:$local_fs cloud-early-config
-# Required-Stop: $local_fs
-# Default-Start: 3 4 5
-# Default-Stop:  0 1 6
-# Short-Description: Start up the CloudStack cloud service
-### END INIT INFO
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements.  See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership.  The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License.  You may obtain a copy of the License at
-#
-#   http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied.  See the License for the
-# specific language governing permissions and limitations
-# under the License.
-
-#set -x
-
-ENABLED=0
-[ -e /etc/default/cloud ] && . /etc/default/cloud
-
-CMDLINE=$(cat /var/cache/cloud/cmdline)
-
-if [ ! -z $CLOUD_DEBUG ];then
-  LOG_FILE=/var/log/cloud/cloud.out
-else
-  LOG_FILE=/dev/null
-fi
-
-TYPE="router"
-for i in $CMDLINE
-  do
-# search for foo=bar pattern and cut out foo
-FIRSTPATTERN=$(echo $i | cut -d= -f1)
-case $FIRSTPATTERN in 
-  type)
-  TYPE=$(echo $i | cut -d= -f2)
-  ;;
-esac
-done
-
-# Source function library.
-if [ -f /etc/init.d/functions ]
-then
-  . /etc/init.d/functions
-fi
-
-if [ -f ./lib/lsb/init-functions ]
-then
-  . /lib/lsb/init-functions
-fi
-
-_success() {
-  if [ -f /etc/init.d/functions ]
-  then
-success
-  else
-echo "Success"
-  fi
-}
-
-_failure() {
-  if [ -f /etc/init.d/functions ]
-  then
-failure
-  else
-echo "Failed"
-  fi
-}
-RETVAL=$?
-CLOUDSTACK_HOME="/usr/local/cloud"
-if [ -f  $CLOUDSTACK_HOME/systemvm/utils.sh ];
-then
-  . $CLOUDSTACK_HOME/systemvm/utils.sh
-else
-  _failure
-fi
-
-# mkdir -p /var/log/vmops
-
-start() {
-   local pid=$(get_pids)
-   if [ "$pid" != "" ]; then
-   echo "CloudStack cloud sevice is already running, PID = $pid"
-   return 0
-   fi
-
-   echo -n "Starting CloudStack cloud service (type=$TYPE) "
-   if [ -f $CLOUDSTACK_HOME/systemvm/run.sh ];
-   then
- if [ "$pid" == "" ]
- then
-   (cd $CLOUDSTACK_HOME/systemvm; nohup ./run.sh > 

[cloudstack] 08/21: CLOUDSTACK-10013: Fix ipsec VPN configuration

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git

commit 0b0252572ab9dbd1a24b2ee6386a5dd1d460db14
Author: Rohit Yadav 
AuthorDate: Fri Dec 1 20:58:18 2017 +0530

CLOUDSTACK-10013: Fix ipsec VPN configuration

- Fixes strongswan/ipsec, l2tpd and pppd configs
- Uses auto=route in ipsec configs
- Fixes road-warrior setup
- Fixes site-to-site VPN with automatic connection configuration
- Fixes vpc_vpn tests

Signed-off-by: Rohit Yadav 
---
 systemvm/patches/debian/opt/cloud/bin/configure.py | 38 +-
 .../patches/debian/opt/cloud/bin/ipsectunnel.sh| 10 +-
 .../debian/opt/cloud/bin/monitor_service.sh|  2 --
 systemvm/patches/vpn/etc/ipsec.d/l2tp.conf | 12 +++
 systemvm/patches/vpn/etc/ppp/options.xl2tpd|  2 --
 systemvm/patches/vpn/opt/cloud/bin/vpn_l2tp.sh | 18 +-
 test/integration/smoke/test_vpc_vpn.py | 26 +++
 7 files changed, 55 insertions(+), 53 deletions(-)

diff --git a/systemvm/patches/debian/opt/cloud/bin/configure.py 
b/systemvm/patches/debian/opt/cloud/bin/configure.py
index 68d1c26..b8a3e02 100755
--- a/systemvm/patches/debian/opt/cloud/bin/configure.py
+++ b/systemvm/patches/debian/opt/cloud/bin/configure.py
@@ -546,10 +546,6 @@ class CsSite2SiteVpn(CsDataBag):
 ikepolicy=obj['ike_policy'].replace(';','-')
 esppolicy=obj['esp_policy'].replace(';','-')
 
-pfs='no'
-if 'modp' in esppolicy:
-pfs='yes'
-
 if rightpeer in self.confips:
 self.confips.remove(rightpeer)
 file = CsFile(vpnconffile)
@@ -557,7 +553,6 @@ class CsSite2SiteVpn(CsDataBag):
 file.search("conn ", "conn vpn-%s" % rightpeer)
 file.addeq(" left=%s" % leftpeer)
 file.addeq(" leftsubnet=%s" % obj['local_guest_cidr'])
-file.addeq(" leftnexthop=%s" % obj['local_public_gateway'])
 file.addeq(" right=%s" % rightpeer)
 file.addeq(" rightsubnet=%s" % peerlist)
 file.addeq(" type=tunnel")
@@ -567,9 +562,8 @@ class CsSite2SiteVpn(CsDataBag):
 file.addeq(" ikelifetime=%s" % 
self.convert_sec_to_h(obj['ike_lifetime']))
 file.addeq(" esp=%s" % esppolicy)
 file.addeq(" lifetime=%s" % self.convert_sec_to_h(obj['esp_lifetime']))
-file.addeq(" pfs=%s" % pfs)
 file.addeq(" keyingtries=2")
-file.addeq(" auto=start")
+file.addeq(" auto=route")
 if 'encap' not in obj:
 obj['encap']=False
 file.addeq(" forceencaps=%s" % CsHelper.bool_to_yn(obj['encap']))
@@ -585,10 +579,20 @@ class CsSite2SiteVpn(CsDataBag):
 logging.info("Configured vpn %s %s", leftpeer, rightpeer)
 CsHelper.execute("ipsec rereadsecrets")
 
-# This will load the new config and start the connection when needed 
since auto=start in the config
+# This will load the new config
 CsHelper.execute("ipsec reload")
 os.chmod(vpnsecretsfile, 0400)
 
+for i in xrange(3):
+result = CsHelper.execute('ipsec status vpn-%s | grep "%s"' % 
(rightpeer, peerlist.split(",", 1)[0]))
+if len(result) > 0:
+break
+time.sleep(1)
+
+# With 'auto=route', connections are established on an attempt to
+# communicate over the S2S VPN. This uses ping to initialize the 
connection.
+CsHelper.execute("timeout 5 ping -c 3 %s" % (peerlist.split("/", 
1)[0].replace(".0", ".1")))
+
 def convert_sec_to_h(self, val):
 hrs = int(val) / 3600
 return "%sh" % hrs
@@ -658,6 +662,7 @@ class CsRemoteAccessVpn(CsDataBag):
 self.confips = []
 
 logging.debug(self.dbag)
+
 for public_ip in self.dbag:
 if public_ip == "id":
 continue
@@ -665,12 +670,13 @@ class CsRemoteAccessVpn(CsDataBag):
 
 #Enable remote access vpn
 if vpnconfig['create']:
+shutdownIpsec = False
 logging.debug("Enabling  remote access vpn  on "+ public_ip)
 
 dev = CsHelper.get_device(public_ip)
 if dev == "":
-logging.error("Request for ipsec to %s not possible 
because ip is not configured", public_ip)
-continue
+logging.error("Request for ipsec to %s not possible 
because ip is not configured", public_ip)
+continue
 
 CsHelper.start_if_stopped("ipsec")
 self.configure_l2tpIpsec(public_ip, self.dbag[public_ip])
@@ -682,7 +688,6 @@ class CsRemoteAccessVpn(CsDataBag):
 CsHelper.execute("ipsec rereadsecrets")
 else:
 logging.debug("Disabling remote access vpn .")
-

[cloudstack] 16/21: CLOUDSTACK-9953: Resize root disk for VMware when full clone is enabled

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git

commit d644774d72d87e73d3e6f7cf32c2dbd1be22b9ed
Author: Rohit Yadav 
AuthorDate: Sat Dec 16 14:19:28 2017 +0530

CLOUDSTACK-9953: Resize root disk for VMware when full clone is enabled

Resize for VMware root disk should only be performed during VM start
when vmware.create.full.clone is true i.e. the disk chain length is one.

Signed-off-by: Rohit Yadav 
---
 .../hypervisor/vmware/resource/VmwareResource.java | 34 ++
 1 file changed, 15 insertions(+), 19 deletions(-)

diff --git 
a/plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
 
b/plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
index 40ffdf4..7f8c2a5 100644
--- 
a/plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
+++ 
b/plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
@@ -2093,9 +2093,9 @@ public class VmwareResource implements 
StoragePoolResource, ServerResource, Vmwa
 hyperHost.setRestartPriorityForVM(vmMo, 
DasVmPriority.HIGH.value());
 }
 
-//For resizing root disk.
+// For resizing root disk.
 if (rootDiskTO != null && !hasSnapshot) {
-resizeRootDisk(vmMo, rootDiskTO, hyperHost, context);
+resizeRootDiskOnVMStart(vmMo, rootDiskTO, hyperHost, context);
 }
 
 //
@@ -2165,28 +2165,24 @@ public class VmwareResource implements 
StoragePoolResource, ServerResource, Vmwa
 return path + fileType;
 }
 
-private void resizeRootDisk(VirtualMachineMO vmMo, DiskTO rootDiskTO, 
VmwareHypervisorHost hyperHost, VmwareContext context) throws Exception
-{
-Pair vdisk = getVirtualDiskInfo(vmMo, 
appendFileType(rootDiskTO.getPath(), ".vmdk"));
+private void resizeRootDiskOnVMStart(VirtualMachineMO vmMo, DiskTO 
rootDiskTO, VmwareHypervisorHost hyperHost, VmwareContext context) throws 
Exception {
+final Pair vdisk = getVirtualDiskInfo(vmMo, 
appendFileType(rootDiskTO.getPath(), ".vmdk"));
 assert(vdisk != null);
 
-Long reqSize=((VolumeObjectTO)rootDiskTO.getData()).getSize()/1024;
-VirtualDisk disk = vdisk.first();
-if (reqSize > disk.getCapacityInKB())
-{
-VirtualMachineDiskInfo diskInfo = 
getMatchingExistingDisk(vmMo.getDiskInfoBuilder(), rootDiskTO, hyperHost, 
context);
+final Long reqSize = ((VolumeObjectTO)rootDiskTO.getData()).getSize() 
/ 1024;
+final VirtualDisk disk = vdisk.first();
+if (reqSize > disk.getCapacityInKB()) {
+final VirtualMachineDiskInfo diskInfo = 
getMatchingExistingDisk(vmMo.getDiskInfoBuilder(), rootDiskTO, hyperHost, 
context);
 assert (diskInfo != null);
-String[] diskChain = diskInfo.getDiskChain();
+final String[] diskChain = diskInfo.getDiskChain();
 
-if (diskChain != null && diskChain.length>1)
-{
-s_logger.error("Unsupported Disk chain length "+ 
diskChain.length);
-throw new Exception("Unsupported Disk chain length "+ 
diskChain.length);
+if (diskChain != null && diskChain.length > 1) {
+s_logger.warn("Disk chain length for the VM is greater than 
one, skipping resizing of root disk.");
+return;
 }
-if (diskInfo.getDiskDeviceBusName() == null || 
!diskInfo.getDiskDeviceBusName().toLowerCase().startsWith("scsi"))
-{
-s_logger.error("Unsupported root disk device bus "+ 
diskInfo.getDiskDeviceBusName() );
-throw new Exception("Unsupported root disk device bus "+ 
diskInfo.getDiskDeviceBusName());
+if (diskInfo.getDiskDeviceBusName() == null || 
!diskInfo.getDiskDeviceBusName().toLowerCase().startsWith("scsi")) {
+s_logger.warn("Resizing of root disk is only support for scsi 
device/bus, the provide disk's device bus name is " + 
diskInfo.getDiskDeviceBusName());
+return;
 }
 
 disk.setCapacityInKB(reqSize);

-- 
To stop receiving notification emails like this one, please contact
"commits@cloudstack.apache.org" .

[cloudstack] 13/21: CLOUDSTACK-10193: Fix smoke tests failures with new systemvmtemplate

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git

commit 475da63ebefbf21d57aa6386484b46fc6e92fa65
Author: Rohit Yadav 
AuthorDate: Sat Dec 9 16:37:31 2017 +0530

CLOUDSTACK-10193: Fix smoke tests failures with new systemvmtemplate

- Several systemvmtemplate optimizations
- Uses new macchinina template for running smoke tests
- Switch to latest Debian 9.3.0 release for systemvmtemplate
- Introduce a new `get_test_template` that uses tiny test template
  such as macchinina as defined test_data.py
- rVR related fixes and improvements

Signed-off-by: Rohit Yadav 
---
 .travis.yml| 11 +--
 .../framework/jobs/impl/AsyncJobManagerImpl.java   |  4 +
 .../com/cloud/resource/ResourceManagerImpl.java|  2 +-
 server/src/com/cloud/server/StatsCollector.java| 12 ++-
 .../secondary/SecondaryStorageVmManager.java   |  6 +-
 .../SecondaryStorageManagerImpl.java   |  2 +-
 systemvm/debian/etc/issue  |  2 +-
 systemvm/debian/etc/rc.local   | 28 ---
 .../etc/systemd/system/cloud-postinit.service  |  2 +-
 systemvm/debian/opt/cloud/bin/cs/CsApp.py  |  2 +-
 systemvm/debian/opt/cloud/bin/cs/CsRedundant.py| 11 ++-
 systemvm/debian/opt/cloud/bin/passwd_server| 26 --
 systemvm/debian/opt/cloud/bin/passwd_server_ip | 29 ---
 systemvm/debian/opt/cloud/bin/setup/common.sh  | 61 +-
 .../debian/opt/cloud/bin/setup/consoleproxy.sh |  2 +-
 systemvm/debian/opt/cloud/bin/setup/dhcpsrvr.sh|  2 +-
 systemvm/debian/opt/cloud/bin/setup/elbvm.sh   |  2 +-
 systemvm/debian/opt/cloud/bin/setup/ilbvm.sh   |  2 +-
 .../debian/opt/cloud/bin/setup/patchsystemvm.sh|  4 +
 systemvm/debian/opt/cloud/bin/setup/postinit.sh| 46 +--
 systemvm/debian/opt/cloud/bin/setup/router.sh  |  5 --
 systemvm/debian/opt/cloud/bin/setup/secstorage.sh  |  2 +-
 systemvm/debian/opt/cloud/bin/setup/vpcrouter.sh   |  5 --
 systemvm/debian/opt/cloud/bin/update_config.py |  7 --
 systemvm/debian/opt/cloud/bin/vpc_passwd_server| 32 
 .../opt/cloud/templates/keepalived.conf.templ  | 11 ++-
 test/integration/smoke/test_accounts.py| 22 +++---
 test/integration/smoke/test_affinity_groups.py | 10 +--
 .../smoke/test_affinity_groups_projects.py | 11 +--
 test/integration/smoke/test_deploy_vm_iso.py   |  7 +-
 .../smoke/test_deploy_vm_with_userdata.py  | 11 +--
 ...st_deploy_vms_with_varied_deploymentplanners.py |  9 ++-
 test/integration/smoke/test_host_maintenance.py|  6 +-
 test/integration/smoke/test_hostha_kvm.py  |  4 +-
 test/integration/smoke/test_internal_lb.py | 53 ++---
 test/integration/smoke/test_list_ids_parameter.py  | 10 +--
 test/integration/smoke/test_loadbalance.py | 10 +--
 test/integration/smoke/test_metrics_api.py |  4 +-
 test/integration/smoke/test_multipleips_per_nic.py |  7 +-
 .../smoke/test_nested_virtualization.py| 15 ++--
 test/integration/smoke/test_network.py | 36 -
 test/integration/smoke/test_network_acl.py |  9 ++-
 test/integration/smoke/test_nic_adapter_type.py|  6 +-
 test/integration/smoke/test_password_server.py |  7 +-
 test/integration/smoke/test_portforwardingrules.py |  6 +-
 test/integration/smoke/test_privategw_acl.py   |  7 +-
 test/integration/smoke/test_projects.py|  7 +-
 test/integration/smoke/test_reset_vm_on_reboot.py  |  7 +-
 test/integration/smoke/test_router_dhcphosts.py| 12 +--
 test/integration/smoke/test_router_dns.py  | 11 ++-
 test/integration/smoke/test_router_dnsservice.py   | 11 +--
 test/integration/smoke/test_routers.py | 12 ++-
 .../smoke/test_routers_iptables_default_policy.py  | 15 ++--
 test/integration/smoke/test_routers_network_ops.py | 71 ++---
 test/integration/smoke/test_service_offerings.py   | 11 ++-
 test/integration/smoke/test_snapshots.py   | 65 +--
 test/integration/smoke/test_templates.py   | 22 +++---
 test/integration/smoke/test_usage.py   | 22 +-
 test/integration/smoke/test_vpc_redundant.py   | 55 ++---
 test/integration/smoke/test_vpc_router_nics.py | 30 ---
 test/integration/smoke/test_vpc_vpn.py | 92 +++---
 tools/appliance/build.sh   |  4 +-
 .../definitions/systemvmtemplate/apt_upgrade.sh|  9 ++-
 .../definitions/systemvmtemplate/cleanup.sh|  5 +-
 .../definitions/systemvmtemplate/configure_grub.sh |  4 +-
 .../definitions/systemvmtemplate/definition.rb |  8 +-
 .../definitions/systemvmtemplate/finalize.sh   |  3 +-
 

[cloudstack] branch debian9-systemvmtemplate updated (b91e885 -> 69a57a3)

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a change to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git.


omit b91e885  CLOUDSTACK-10127: KVM + Ovs: Incorrect devId on static nat
omit 1c65ac4  CLOUDSTACK-10001: Fix incorrect total host memory in responses
omit 0c320fe  CLOUDSTACK-10013: Add systemvm 4.11 migration in 4.10->4.11 
upgrade path
omit 5650ff2  CLOUDSTACK-10194: Use packer for building systemvmtemplates
omit 23387f4  CLOUDSTACK-10193: Use other64Linux for systemvms on VMware
omit a797f84  CLOUDSTACK-10013: Fix VMware related issues and fix misc tests
omit 64701a7  CLOUDSTACK-9953: Resize root disk for VMware when full clone 
is enabled
omit 4e24e06  CLOUDSTACK-9501: route is not available by default on CentOS7 
(#1637)
omit b2f95de  CLOUDSTACK-10187: Don't delete vifs for VPCs with source nat
omit 9984371  CLOUDSTACK-10193: Fix smoke tests failures with new 
systemvmtemplate
omit c7264a2  CLOUDSTACK-10181: Skip rVR test cases for VMware until 
supported
omit 990d51b  CLOUDSTACK-10013: SystemVM codebase refactorings and 
improvements
omit e0857bb  CLOUDSTACK-10013: Make the generated VR/json files unique 
(ports #1470)
omit cbfba0a  CLOUDSTACK-10010: Port marvin test from #2190 by @swill
omit 6409f62  CLOUDSTACK-10013: Fix ipsec VPN configuration
omit aba3bbe  CLOUDSTACK-10013: SystemVM.iso refactoring and cleanups
omit c8d5165  CLOUDSTACK-10129: UX improvements and event timeline
omit 0a64c30  CLOUDSTACK-9348: Improve Nio SSH handshake buffers
omit bc59dad  CLOUDSTACK-7853: Fix ping timeout edge case and refactor code
omit 13cd40d  CLOUDSTACK-10013: Debian9 SystemVM appliance improvements
omit 6767724  CLOUDSTACK-10013: Make Debian9 systemvmtemplates work with 
systemd
omit f718238  CLOUDSTACK-10013: Migrate systemvmtemplate to Debian9
 add 6e3495f  CLOUDSTACK-10190: Duplicate public VLAN for two different 
admin accounts (#2361)
 add 9179bd5  CLOUDSTACK-10195: CloudStack MySQL HA problem - No database 
selected (#2364)
 new a9df3bd  CLOUDSTACK-10013: Migrate systemvmtemplate to Debian9
 new b9ffbee  CLOUDSTACK-10013: Make Debian9 systemvmtemplates work with 
systemd
 new 5ea7d19  CLOUDSTACK-10013: Debian9 SystemVM appliance improvements
 new 0560584  CLOUDSTACK-7853: Fix ping timeout edge case and refactor code
 new 6bd772f  CLOUDSTACK-9348: Improve Nio SSH handshake buffers
 new 406e714  CLOUDSTACK-10129: UX improvements and event timeline
 new 22f7d09  CLOUDSTACK-10013: SystemVM.iso refactoring and cleanups
 new 0b02525  CLOUDSTACK-10013: Fix ipsec VPN configuration
 new 0e4de4b  CLOUDSTACK-10010: Port marvin test from #2190 by @swill
 new 427df50  CLOUDSTACK-10013: Make the generated VR/json files unique 
(ports #1470)
 new cea9553  CLOUDSTACK-10013: SystemVM codebase refactorings and 
improvements
 new bfc871e  CLOUDSTACK-10181: Skip rVR test cases for VMware until 
supported
 new 475da63  CLOUDSTACK-10193: Fix smoke tests failures with new 
systemvmtemplate
 new 02d479b  CLOUDSTACK-10187: Don't delete vifs for VPCs with source nat
 new cd58d1b  CLOUDSTACK-9501: route is not available by default on CentOS7 
(#1637)
 new d644774  CLOUDSTACK-9953: Resize root disk for VMware when full clone 
is enabled
 new 463bb0a  CLOUDSTACK-10013: Fix VMware related issues and fix misc tests
 new a9175ef  CLOUDSTACK-10193: Use other64Linux for systemvms on VMware
 new 28e0270  CLOUDSTACK-10194: Use packer for building systemvmtemplates
 new 3112500  CLOUDSTACK-10013: Add systemvm 4.11 migration in 4.10->4.11 
upgrade path
 new 69a57a3  CLOUDSTACK-10001: Fix incorrect total host memory in responses

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (b91e885)
\
 N -- N -- N   refs/heads/debian9-systemvmtemplate (69a57a3)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

The 21 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../src/com/cloud/cluster/ClusterManagerImpl.java  |  13 +-
 .../test/com/cloud/utils/db/TransactionTest.java   | 166 --
 .../kvm/resource/LibvirtComputingResource.java |  20 +-
 

[cloudstack] 05/21: CLOUDSTACK-9348: Improve Nio SSH handshake buffers

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git

commit 6bd772f81ae9d55c54537561fe538b280f791846
Author: Rohit Yadav 
AuthorDate: Thu Nov 30 16:51:48 2017 +0530

CLOUDSTACK-9348: Improve Nio SSH handshake buffers

Use a holder class to pass buffers, fixes potential leak.

Signed-off-by: Rohit Yadav 
---
 .../agent/manager/ClusteredAgentManagerImpl.java   |  2 +-
 utils/src/main/java/com/cloud/utils/nio/Link.java  | 72 +-
 .../main/java/com/cloud/utils/nio/NioClient.java   |  2 +-
 .../java/com/cloud/utils/nio/NioConnection.java|  2 +-
 4 files changed, 58 insertions(+), 20 deletions(-)

diff --git 
a/engine/orchestration/src/com/cloud/agent/manager/ClusteredAgentManagerImpl.java
 
b/engine/orchestration/src/com/cloud/agent/manager/ClusteredAgentManagerImpl.java
index 2ebfeb5..0b9899e 100644
--- 
a/engine/orchestration/src/com/cloud/agent/manager/ClusteredAgentManagerImpl.java
+++ 
b/engine/orchestration/src/com/cloud/agent/manager/ClusteredAgentManagerImpl.java
@@ -519,7 +519,7 @@ public class ClusteredAgentManagerImpl extends 
AgentManagerImpl implements Clust
 sslEngine.setUseClientMode(true);
 
sslEngine.setEnabledProtocols(SSLUtils.getSupportedProtocols(sslEngine.getEnabledProtocols()));
 sslEngine.beginHandshake();
-if (!Link.doHandshake(ch1, sslEngine, true)) {
+if (!Link.doHandshake(ch1, sslEngine)) {
 ch1.close();
 throw new IOException(String.format("SSL: 
Handshake failed with peer management server '%s' on %s:%d ", peerName, ip, 
port));
 }
diff --git a/utils/src/main/java/com/cloud/utils/nio/Link.java 
b/utils/src/main/java/com/cloud/utils/nio/Link.java
index 8f1b811..35211c8 100644
--- a/utils/src/main/java/com/cloud/utils/nio/Link.java
+++ b/utils/src/main/java/com/cloud/utils/nio/Link.java
@@ -32,6 +32,8 @@ import java.security.GeneralSecurityException;
 import java.security.KeyStore;
 import java.security.SecureRandom;
 import java.util.concurrent.ConcurrentLinkedQueue;
+import java.util.concurrent.Executor;
+import java.util.concurrent.Executors;
 
 import javax.net.ssl.KeyManagerFactory;
 import javax.net.ssl.SSLContext;
@@ -462,7 +464,7 @@ public class Link {
 return buffer;
 }
 
-public static ByteBuffer handleBufferUnderflow(final SSLEngine engine, 
ByteBuffer buffer) {
+public static ByteBuffer handleBufferUnderflow(final SSLEngine engine, 
final ByteBuffer buffer) {
 if (engine == null || buffer == null) {
 return buffer;
 }
@@ -475,14 +477,14 @@ public class Link {
 return replaceBuffer;
 }
 
-private static boolean doHandshakeUnwrap(final SocketChannel 
socketChannel, final SSLEngine sslEngine,
+private static HandshakeHolder doHandshakeUnwrap(final SocketChannel 
socketChannel, final SSLEngine sslEngine,
  ByteBuffer peerAppData, 
ByteBuffer peerNetData, final int appBufferSize) throws IOException {
 if (socketChannel == null || sslEngine == null || peerAppData == null 
|| peerNetData == null || appBufferSize < 0) {
-return false;
+return new HandshakeHolder(peerAppData, peerNetData, false);
 }
 if (socketChannel.read(peerNetData) < 0) {
 if (sslEngine.isInboundDone() && sslEngine.isOutboundDone()) {
-return false;
+return new HandshakeHolder(peerAppData, peerNetData, false);
 }
 try {
 sslEngine.closeInbound();
@@ -492,7 +494,7 @@ public class Link {
 sslEngine.closeOutbound();
 // After closeOutbound the engine will be set to WRAP state,
 // in order to try to send a close message to the client.
-return true;
+return new HandshakeHolder(peerAppData, peerNetData, true);
 }
 peerNetData.flip();
 SSLEngineResult result = null;
@@ -503,7 +505,10 @@ public class Link {
 s_logger.error(String.format("SSL error caught during unwrap data: 
%s, for local address=%s, remote address=%s. The client may have invalid 
ca-certificates.",
 sslException.getMessage(), 
socketChannel.getLocalAddress(), socketChannel.getRemoteAddress()));
 sslEngine.closeOutbound();
-return false;
+return new HandshakeHolder(peerAppData, peerNetData, true);
+}
+if (result == null) {
+return new HandshakeHolder(peerAppData, peerNetData, false);
 }
 switch (result.getStatus()) {
 case OK:
@@ -519,23 +524,23 @@ public class Link {
   

[cloudstack] 20/21: CLOUDSTACK-10013: Add systemvm 4.11 migration in 4.10->4.11 upgrade path

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git

commit 31125007b347d44f231f290c1e046d280af02a60
Author: Rohit Yadav 
AuthorDate: Tue Dec 19 11:06:59 2017 +0530

CLOUDSTACK-10013: Add systemvm 4.11 migration in 4.10->4.11 upgrade path

This moves the systevmtemplate migration logic from previous upgrade path
to 4.10.0.0->4.11.0.0 upgrade path.

Signed-off-by: Rohit Yadav 
---
 .../com/cloud/upgrade/dao/Upgrade41000to41100.java | 189 -
 .../com/cloud/upgrade/dao/Upgrade4930to41000.java  | 184 +---
 2 files changed, 191 insertions(+), 182 deletions(-)

diff --git a/engine/schema/src/com/cloud/upgrade/dao/Upgrade41000to41100.java 
b/engine/schema/src/com/cloud/upgrade/dao/Upgrade41000to41100.java
index c59ead9..fbe9d78 100644
--- a/engine/schema/src/com/cloud/upgrade/dao/Upgrade41000to41100.java
+++ b/engine/schema/src/com/cloud/upgrade/dao/Upgrade41000to41100.java
@@ -17,13 +17,25 @@
 
 package com.cloud.upgrade.dao;
 
-import com.cloud.utils.exception.CloudRuntimeException;
-
 import java.io.InputStream;
 import java.sql.Connection;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.utils.exception.CloudRuntimeException;
 
 public class Upgrade41000to41100 implements DbUpgrade {
 
+final static Logger LOG = Logger.getLogger(Upgrade41000to41100.class);
+
 @Override
 public String[] getUpgradableVersionRange() {
 return new String[] {"4.10.0.0", "4.11.0.0"};
@@ -52,8 +64,181 @@ public class Upgrade41000to41100 implements DbUpgrade {
 
 @Override
 public void performDataMigration(Connection conn) {
+updateSystemVmTemplates(conn);
 }
 
+@SuppressWarnings("serial")
+private void updateSystemVmTemplates(final Connection conn) {
+LOG.debug("Updating System Vm template IDs");
+// Get all hypervisors in use
+final Set hypervisorsListInUse = new 
HashSet();
+try (PreparedStatement pstmt = conn.prepareStatement("select 
distinct(hypervisor_type) from `cloud`.`cluster` where removed is null"); 
ResultSet rs = pstmt.executeQuery()) {
+while (rs.next()) {
+switch (Hypervisor.HypervisorType.getType(rs.getString(1))) {
+case XenServer:
+
hypervisorsListInUse.add(Hypervisor.HypervisorType.XenServer);
+break;
+case KVM:
+
hypervisorsListInUse.add(Hypervisor.HypervisorType.KVM);
+break;
+case VMware:
+
hypervisorsListInUse.add(Hypervisor.HypervisorType.VMware);
+break;
+case Hyperv:
+
hypervisorsListInUse.add(Hypervisor.HypervisorType.Hyperv);
+break;
+case LXC:
+
hypervisorsListInUse.add(Hypervisor.HypervisorType.LXC);
+break;
+case Ovm3:
+
hypervisorsListInUse.add(Hypervisor.HypervisorType.Ovm3);
+break;
+default:
+break;
+}
+}
+} catch (final SQLException e) {
+LOG.error("updateSystemVmTemplates: Exception caught while getting 
hypervisor types from clusters: " + e.getMessage());
+throw new CloudRuntimeException("updateSystemVmTemplates:Exception 
while getting hypervisor types from clusters", e);
+}
+
+final Map NewTemplateNameList = new 
HashMap() {
+{
+put(Hypervisor.HypervisorType.KVM, "systemvm-kvm-4.11");
+put(Hypervisor.HypervisorType.VMware, "systemvm-vmware-4.11");
+put(Hypervisor.HypervisorType.XenServer, 
"systemvm-xenserver-4.11");
+put(Hypervisor.HypervisorType.Hyperv, "systemvm-hyperv-4.11");
+put(Hypervisor.HypervisorType.LXC, "systemvm-lxc-4.11");
+put(Hypervisor.HypervisorType.Ovm3, "systemvm-ovm3-4.11");
+}
+};
+
+final Map 
routerTemplateConfigurationNames = new HashMap() {
+{
+put(Hypervisor.HypervisorType.KVM, "router.template.kvm");
+put(Hypervisor.HypervisorType.VMware, 
"router.template.vmware");
+put(Hypervisor.HypervisorType.XenServer, 

[GitHub] rhtyd commented on issue #1762: CLOUDSTACK-9595 Transactions are not getting retried in case of datab?

[GitBox]

rhtyd commented on issue #1762: CLOUDSTACK-9595 Transactions are not getting 
retried in case of datab?
URL: https://github.com/apache/cloudstack/pull/1762#issuecomment-352848141
 
 
   @yvsubhash I've left some additional remarks, please see them. After commit 
triaging, looks like I made an error, and it seems the issue was not caused by 
this commit/PR. I found the https://github.com/apache/cloudstack/pull/2304 to 
be an issue on KVM (in my branch this commit/pr was also included causing 
incorrect nic dev ids to be assigned). I'll continue testing, but do see my 
remarks and send another PR if you find further scope for improvements, thanks. 
/cc @rafaelweingartner 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on issue #2304: CLOUDSTACK-10127: KVM + Ovs: Incorrect devId on static nat

[GitBox]

rhtyd commented on issue #2304: CLOUDSTACK-10127: KVM + Ovs: Incorrect devId on 
static nat
URL: https://github.com/apache/cloudstack/pull/2304#issuecomment-352846227
 
 
   @fmaximus I did testing in my branch/pr #2211 and found that the dev id is 
not assigned in case of private gate and this fails with an issue similar to 
https://issues.apache.org/jira/browse/CLOUDSTACK-9759 where when nic_dev_id is 
not provided, it leads to ethNone is ips.json and fails VPC deployment and 
related test as seen in test failures on this PR. Please fix this and test 
against a normal non-ovs (bridge) env, thanks.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on a change in pull request #1762: CLOUDSTACK-9595 Transactions are not getting retried in case of datab?

[GitBox]

rhtyd commented on a change in pull request #1762: CLOUDSTACK-9595 Transactions 
are not getting retried in case of datab?
URL: https://github.com/apache/cloudstack/pull/1762#discussion_r157832926
 
 

 ##
 File path: server/src/com/cloud/network/IpAddressManagerImpl.java
 ##
 @@ -901,40 +916,28 @@ public PublicIp assignDedicateIpAddress(Account owner, 
final Long guestNtwkId, f
 
 PublicIp ip = null;
 try {
-ip = Transaction.execute(new 
TransactionCallbackWithException() {
 
 Review comment:
   @yvsubhash this method is no longer bound in transaction and may cause 
failures. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on a change in pull request #1762: CLOUDSTACK-9595 Transactions are not getting retried in case of datab?

[GitBox]

rhtyd commented on a change in pull request #1762: CLOUDSTACK-9595 Transactions 
are not getting retried in case of datab?
URL: https://github.com/apache/cloudstack/pull/1762#discussion_r157831617
 
 

 ##
 File path: server/src/com/cloud/network/IpAddressManagerImpl.java
 ##
 @@ -901,40 +916,28 @@ public PublicIp assignDedicateIpAddress(Account owner, 
final Long guestNtwkId, f
 
 PublicIp ip = null;
 try {
-ip = Transaction.execute(new 
TransactionCallbackWithException() {
-@Override
-public PublicIp doInTransaction(TransactionStatus status) 
throws InsufficientAddressCapacityException {
-Account owner = _accountDao.acquireInLockTable(ownerId);
-
-if (owner == null) {
-// this ownerId comes from owner or type Account. See 
the class "AccountVO" and the annotations in that class
-// to get the table name and field name that is 
queried to fill this ownerid.
-ConcurrentOperationException ex = new 
ConcurrentOperationException("Unable to lock account");
-throw ex;
-}
-if (s_logger.isDebugEnabled()) {
-s_logger.debug("lock account " + ownerId + " is 
acquired");
-}
-boolean displayIp = true;
-if (guestNtwkId != null) {
-Network ntwk = _networksDao.findById(guestNtwkId);
-displayIp = ntwk.getDisplayNetwork();
-} else if (vpcId != null) {
-VpcVO vpc = _vpcDao.findById(vpcId);
-displayIp = vpc.isDisplay();
-}
+Account ownerAccount = _accountDao.acquireInLockTable(ownerId);
 
-PublicIp ip = fetchNewPublicIp(dcId, null, null, owner, 
VlanType.VirtualNetwork, guestNtwkId, isSourceNat, false, null, false, vpcId, 
displayIp);
-IPAddressVO publicIp = ip.ip();
-
-markPublicIpAsAllocated(publicIp);
-_ipAddressDao.update(publicIp.getId(), publicIp);
+if (ownerAccount == null) {
+// this ownerId comes from owner or type Account. See the 
class "AccountVO" and the annotations in that class
+// to get the table name and field name that is queried to 
fill this ownerid.
+ConcurrentOperationException ex = new 
ConcurrentOperationException("Unable to lock account");
+throw ex;
+}
+if (s_logger.isDebugEnabled()) {
+s_logger.debug("lock account " + ownerId + " is acquired");
+}
+boolean displayIp = true;
+if (guestNtwkId != null) {
+Network ntwk = _networksDao.findById(guestNtwkId);
+displayIp = ntwk.getDisplayNetwork();
+} else if (vpcId != null) {
+VpcVO vpc = _vpcDao.findById(vpcId);
+displayIp = vpc.isDisplay();
+}
 
-return ip;
-}
-});
+return fetchNewPublicIp(dcId, null, null, owner, 
VlanType.VirtualNetwork, guestNtwkId, isSourceNat, false, null, false, vpcId, 
displayIp);
 
 Review comment:
   After this, we're not doing the following:
   ```
   -IPAddressVO publicIp = ip.ip();
-
-markPublicIpAsAllocated(publicIp);
-_ipAddressDao.update(publicIp.getId(), publicIp);
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on a change in pull request #1762: CLOUDSTACK-9595 Transactions are not getting retried in case of datab?

[GitBox]

rhtyd commented on a change in pull request #1762: CLOUDSTACK-9595 Transactions 
are not getting retried in case of datab?
URL: https://github.com/apache/cloudstack/pull/1762#discussion_r157830563
 
 

 ##
 File path: server/src/com/cloud/network/IpAddressManagerImpl.java
 ##
 @@ -836,35 +835,51 @@ public IPAddressVO doInTransaction(TransactionStatus 
status) throws Insufficient
 @DB
 @Override
 public void markPublicIpAsAllocated(final IPAddressVO addr) {
-
-Transaction.execute(new TransactionCallbackNoReturn() {
-@Override
-public void doInTransactionWithoutResult(TransactionStatus status) 
{
-Account owner = 
_accountMgr.getAccount(addr.getAllocatedToAccountId());
-synchronized (this) {
+synchronized (_allocatedLock) {
+Transaction.execute(new TransactionCallbackNoReturn() {
+@Override
+public void doInTransactionWithoutResult(TransactionStatus 
status) {
+Account owner = 
_accountMgr.getAccount(addr.getAllocatedToAccountId());
 if (_ipAddressDao.lockRow(addr.getId(), true) != null) {
 IPAddressVO userIp = 
_ipAddressDao.findById(addr.getId());
 if (userIp.getState() == IpAddress.State.Allocating || 
addr.getState() == IpAddress.State.Free) {
 addr.setState(IpAddress.State.Allocated);
-_ipAddressDao.update(addr.getId(), addr);
-// Save usage event
-if (owner.getAccountId() != 
Account.ACCOUNT_ID_SYSTEM) {
-VlanVO vlan = 
_vlanDao.findById(addr.getVlanId());
-String guestType = 
vlan.getVlanType().toString();
-if (!isIpDedicated(addr)) {
-
UsageEventUtils.publishUsageEvent(EventTypes.EVENT_NET_IP_ASSIGN, 
owner.getId(), addr.getDataCenterId(), addr.getId(),
-addr.getAddress().toString(),
-addr.isSourceNat(), guestType, 
addr.getSystem(), addr.getClass().getName(), addr.getUuid());
-}
-if (updateIpResourceCount(addr)) {
-
_resourceLimitMgr.incrementResourceCount(owner.getId(), ResourceType.public_ip);
+if (_ipAddressDao.update(addr.getId(), addr)) {
+// Save usage event
+if (owner.getAccountId() != 
Account.ACCOUNT_ID_SYSTEM) {
+VlanVO vlan = 
_vlanDao.findById(addr.getVlanId());
+String guestType = 
vlan.getVlanType().toString();
+if (!isIpDedicated(addr)) {
+
UsageEventUtils.publishUsageEvent(EventTypes.EVENT_NET_IP_ASSIGN, 
owner.getId(), addr.getDataCenterId(), addr.getId(),
+addr.getAddress().toString(),
+addr.isSourceNat(), guestType, 
addr.getSystem(), addr.getClass().getName(), addr.getUuid());
+}
+if (updateIpResourceCount(addr)) {
+
_resourceLimitMgr.incrementResourceCount(owner.getId(), ResourceType.public_ip);
+}
 }
 }
 }
 }
 }
-}
-});
+});
+}
+}
+
+@DB
+private void markPublicIpAsAllocating(final IPAddressVO addr) {
+synchronized (_allocatingLock) {
+Transaction.execute(new TransactionCallbackNoReturn() {
+@Override
+public void doInTransactionWithoutResult(TransactionStatus 
status) {
+
+if (_ipAddressDao.lockRow(addr.getId(), true) != null) {
 
 Review comment:
   Here why is an explicit locking of Row called? And there is no 
retry/exception/logging when row may not be locked.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on a change in pull request #1762: CLOUDSTACK-9595 Transactions are not getting retried in case of datab?

[GitBox]

rhtyd commented on a change in pull request #1762: CLOUDSTACK-9595 Transactions 
are not getting retried in case of datab?
URL: https://github.com/apache/cloudstack/pull/1762#discussion_r157830563
 
 

 ##
 File path: server/src/com/cloud/network/IpAddressManagerImpl.java
 ##
 @@ -836,35 +835,51 @@ public IPAddressVO doInTransaction(TransactionStatus 
status) throws Insufficient
 @DB
 @Override
 public void markPublicIpAsAllocated(final IPAddressVO addr) {
-
-Transaction.execute(new TransactionCallbackNoReturn() {
-@Override
-public void doInTransactionWithoutResult(TransactionStatus status) 
{
-Account owner = 
_accountMgr.getAccount(addr.getAllocatedToAccountId());
-synchronized (this) {
+synchronized (_allocatedLock) {
+Transaction.execute(new TransactionCallbackNoReturn() {
+@Override
+public void doInTransactionWithoutResult(TransactionStatus 
status) {
+Account owner = 
_accountMgr.getAccount(addr.getAllocatedToAccountId());
 if (_ipAddressDao.lockRow(addr.getId(), true) != null) {
 IPAddressVO userIp = 
_ipAddressDao.findById(addr.getId());
 if (userIp.getState() == IpAddress.State.Allocating || 
addr.getState() == IpAddress.State.Free) {
 addr.setState(IpAddress.State.Allocated);
-_ipAddressDao.update(addr.getId(), addr);
-// Save usage event
-if (owner.getAccountId() != 
Account.ACCOUNT_ID_SYSTEM) {
-VlanVO vlan = 
_vlanDao.findById(addr.getVlanId());
-String guestType = 
vlan.getVlanType().toString();
-if (!isIpDedicated(addr)) {
-
UsageEventUtils.publishUsageEvent(EventTypes.EVENT_NET_IP_ASSIGN, 
owner.getId(), addr.getDataCenterId(), addr.getId(),
-addr.getAddress().toString(),
-addr.isSourceNat(), guestType, 
addr.getSystem(), addr.getClass().getName(), addr.getUuid());
-}
-if (updateIpResourceCount(addr)) {
-
_resourceLimitMgr.incrementResourceCount(owner.getId(), ResourceType.public_ip);
+if (_ipAddressDao.update(addr.getId(), addr)) {
+// Save usage event
+if (owner.getAccountId() != 
Account.ACCOUNT_ID_SYSTEM) {
+VlanVO vlan = 
_vlanDao.findById(addr.getVlanId());
+String guestType = 
vlan.getVlanType().toString();
+if (!isIpDedicated(addr)) {
+
UsageEventUtils.publishUsageEvent(EventTypes.EVENT_NET_IP_ASSIGN, 
owner.getId(), addr.getDataCenterId(), addr.getId(),
+addr.getAddress().toString(),
+addr.isSourceNat(), guestType, 
addr.getSystem(), addr.getClass().getName(), addr.getUuid());
+}
+if (updateIpResourceCount(addr)) {
+
_resourceLimitMgr.incrementResourceCount(owner.getId(), ResourceType.public_ip);
+}
 }
 }
 }
 }
 }
-}
-});
+});
+}
+}
+
+@DB
+private void markPublicIpAsAllocating(final IPAddressVO addr) {
+synchronized (_allocatingLock) {
+Transaction.execute(new TransactionCallbackNoReturn() {
+@Override
+public void doInTransactionWithoutResult(TransactionStatus 
status) {
+
+if (_ipAddressDao.lockRow(addr.getId(), true) != null) {
 
 Review comment:
   Here why is an explicit locking of Row called?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on a change in pull request #1762: CLOUDSTACK-9595 Transactions are not getting retried in case of datab?

[GitBox]

rhtyd commented on a change in pull request #1762: CLOUDSTACK-9595 Transactions 
are not getting retried in case of datab?
URL: https://github.com/apache/cloudstack/pull/1762#discussion_r157830379
 
 

 ##
 File path: server/src/com/cloud/network/IpAddressManagerImpl.java
 ##
 @@ -836,35 +835,51 @@ public IPAddressVO doInTransaction(TransactionStatus 
status) throws Insufficient
 @DB
 @Override
 public void markPublicIpAsAllocated(final IPAddressVO addr) {
-
-Transaction.execute(new TransactionCallbackNoReturn() {
-@Override
-public void doInTransactionWithoutResult(TransactionStatus status) 
{
-Account owner = 
_accountMgr.getAccount(addr.getAllocatedToAccountId());
-synchronized (this) {
+synchronized (_allocatedLock) {
+Transaction.execute(new TransactionCallbackNoReturn() {
+@Override
+public void doInTransactionWithoutResult(TransactionStatus 
status) {
+Account owner = 
_accountMgr.getAccount(addr.getAllocatedToAccountId());
 if (_ipAddressDao.lockRow(addr.getId(), true) != null) {
 IPAddressVO userIp = 
_ipAddressDao.findById(addr.getId());
 if (userIp.getState() == IpAddress.State.Allocating || 
addr.getState() == IpAddress.State.Free) {
 addr.setState(IpAddress.State.Allocated);
-_ipAddressDao.update(addr.getId(), addr);
-// Save usage event
-if (owner.getAccountId() != 
Account.ACCOUNT_ID_SYSTEM) {
-VlanVO vlan = 
_vlanDao.findById(addr.getVlanId());
-String guestType = 
vlan.getVlanType().toString();
-if (!isIpDedicated(addr)) {
-
UsageEventUtils.publishUsageEvent(EventTypes.EVENT_NET_IP_ASSIGN, 
owner.getId(), addr.getDataCenterId(), addr.getId(),
-addr.getAddress().toString(),
-addr.isSourceNat(), guestType, 
addr.getSystem(), addr.getClass().getName(), addr.getUuid());
-}
-if (updateIpResourceCount(addr)) {
-
_resourceLimitMgr.incrementResourceCount(owner.getId(), ResourceType.public_ip);
+if (_ipAddressDao.update(addr.getId(), addr)) {
 
 Review comment:
   @yvsubhash I've found regressions in some tests, specifically around private 
gw tests where failures look like this:
   ```
   2017-12-19 22:53:00,817 DEBUG [c.c.a.t.Request] 
(AgentManager-Handler-20:null) (logid:) Seq 1-4815473901565903002: Processing:  
{ Ans: , MgmtId: 2485222984626, via: 1, Ver: v1, Flags: 10, 
[{"com.cloud.agent.api.routing.GroupAnswer":{"results":["null - success: 
Creating file in VR, with ip: 169.254.1.135, file: 
ip_associations.json.82a129d8-a849-4a63-af8b-3b885abd3b32","null - failed: 
[INFO] update_config.py :: Processing incoming file => 
ip_associations.json.82a129d8-a849-4a63-af8b-3b885abd3b32[INFO] Processing JSON 
file ip_associations.json.82a129d8-a849-4a63-af8b-3b885abd3b32Traceback (most 
recent call last):  File \"/opt/cloud/bin/update_config.py\", line 143, in 
process_file()  File \"/opt/cloud/bin/update_config.py\", line 57, 
in process_filefinish_config()  File \"/opt/cloud/bin/update_config.py\", 
line 45, in finish_configreturncode = configure.main(sys.argv)  File 
\"/opt/cloud/bin/configure.py\", line 1006, in main
config.address().process()  File \"/opt/cloud/bin/cs/CsAddress.py\", line 104, 
in processip = CsIP(dev, self.config)  File 
\"/opt/cloud/bin/cs/CsAddress.py\", line 261, in __init__self.dnum = 
hex(int(dev[3:]))ValueError: invalid literal for int() with base 10: 
'None'"],"result":false,"wait":0}}] }  2017-12-19 22:53:00,818 DEBUG 
[c.c.a.t.Request] (API-Job-Executor-47:ctx-52e4a845 job-76 ctx-b4b917c1) 
(logid:a5289d87) Seq 1-4815473901565903002: Received:  { Ans: , MgmtId: 
2485222984626, via: 1(centos7-kvm1), Ver: v1, Flags: 10, { GroupAnswer } }
   ```
   On deeper analysis, I found that `nic_dev_id` was not passed to a VPC router 
with no VMs, one network and a private way via a ip_associations.json and it 
caused interface to have ethNone defined in ips.json instead of eth2 etc. (this 
is similar to https://issues.apache.org/jira/browse/CLOUDSTACK-9759). Since, 
the only change in master is your PR I suspect there is some regression and 
that patch you've submitted here may be different that you private fork/branch. 
/cc @rafaelweingartner 
   
   For example, in this code if update fails for some reason it should throw an 
exception or log?


This is an automated message from the Apache Git Service.
To 

[GitHub] blueorangutan commented on issue #2281: CLOUDSTACK-10102: New network type (L2)

[GitBox]

blueorangutan commented on issue #2281: CLOUDSTACK-10102: New network type (L2)
URL: https://github.com/apache/cloudstack/pull/2281#issuecomment-352829593
 
 
   @nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been 
kicked to run smoke tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nvazquez commented on issue #2281: CLOUDSTACK-10102: New network type (L2)

[GitBox]

nvazquez commented on issue #2281: CLOUDSTACK-10102: New network type (L2)
URL: https://github.com/apache/cloudstack/pull/2281#issuecomment-352829323
 
 
   @blueorangutan test


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2281: CLOUDSTACK-10102: New network type (L2)

[GitBox]

blueorangutan commented on issue #2281: CLOUDSTACK-10102: New network type (L2)
URL: https://github.com/apache/cloudstack/pull/2281#issuecomment-352828874
 
 
   Packaging result: ?centos6 ?centos7 ?debian. JID-1428


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #1554: CLOUDSTACK-9602: API: improve resource limits comprehension

[GitBox]

blueorangutan commented on issue #1554: CLOUDSTACK-9602: API: improve resource 
limits comprehension
URL: https://github.com/apache/cloudstack/pull/1554#issuecomment-352828596
 
 
   Trillian test result (tid-1827)
   Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 29335 seconds
   Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr1554-t1827-kvm-centos7.zip
   Smoke tests completed. 63 look OK, 4 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   ContextSuite context=TestTemplateHierarchy>:setup | `Error` | 1591.58 | 
test_accounts.py
   test_01_vpc_privategw_acl | `Failure` | 46.68 | test_privategw_acl.py
   test_02_vpc_privategw_static_routes | `Failure` | 263.63 | 
test_privategw_acl.py
   test_03_vpc_privategw_restart_vpc_cleanup | `Failure` | 198.28 | 
test_privategw_acl.py
   test_04_rvpc_privategw_static_routes | `Failure` | 258.61 | 
test_privategw_acl.py
   test_02_create_template_with_checksum_sha1 | `Error` | 5.25 | 
test_templates.py
   test_03_create_template_with_checksum_sha256 | `Error` | 5.26 | 
test_templates.py
   test_04_create_template_with_checksum_md5 | `Error` | 5.25 | 
test_templates.py
   test_01_vpc_remote_access_vpn | `Error` | 50.93 | test_vpc_vpn.py
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2281: CLOUDSTACK-10102: New network type (L2)

[GitBox]

blueorangutan commented on issue #2281: CLOUDSTACK-10102: New network type (L2)
URL: https://github.com/apache/cloudstack/pull/2281#issuecomment-352822032
 
 
   @nvazquez a Jenkins job has been kicked to build packages. I'll keep you 
posted as I make progress.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nvazquez commented on issue #2281: CLOUDSTACK-10102: New network type (L2)

[GitBox]

nvazquez commented on issue #2281: CLOUDSTACK-10102: New network type (L2)
URL: https://github.com/apache/cloudstack/pull/2281#issuecomment-352821798
 
 
   @blueorangutan package


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] khos2ow commented on issue #2365: CLOUDSTACK-10197: Rename xentools iso for XenServer 7.0+

[GitBox]

khos2ow commented on issue #2365: CLOUDSTACK-10197: Rename xentools iso for 
XenServer 7.0+
URL: https://github.com/apache/cloudstack/pull/2365#issuecomment-352816862
 
 
   @rhtyd damn! I haven't seen that link!!! thanks.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2046: CLOUDSTACK-7958: Add configuration for limit to CIDRs for Admin API calls

[GitBox]

blueorangutan commented on issue #2046: CLOUDSTACK-7958: Add configuration for 
limit to CIDRs for Admin API calls
URL: https://github.com/apache/cloudstack/pull/2046#issuecomment-352816311
 
 
   Packaging result: ?centos6 ?centos7 ?debian. JID-1427


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on issue #2365: CLOUDSTACK-10197: Rename xentools iso for XenServer 7.0+

[GitBox]

rhtyd commented on issue #2365: CLOUDSTACK-10197: Rename xentools iso for 
XenServer 7.0+
URL: https://github.com/apache/cloudstack/pull/2365#issuecomment-352815764
 
 
   @khos2ow yes, download the marvin logs zip file from the test above, extract 
and see the logs for that test to get hints etc.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on issue #2365: CLOUDSTACK-10197: Rename xentools iso for XenServer 7.0+

[GitBox]

rhtyd commented on issue #2365: CLOUDSTACK-10197: Rename xentools iso for 
XenServer 7.0+
URL: https://github.com/apache/cloudstack/pull/2365#issuecomment-352815764
 
 
   @khos2ow yes, download the marvin logs from the test above, extract and see 
the logs for that test to get hints etc.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] khos2ow commented on issue #2365: CLOUDSTACK-10197: Rename xentools iso for XenServer 7.0+

[GitBox]

khos2ow commented on issue #2365: CLOUDSTACK-10197: Rename xentools iso for 
XenServer 7.0+
URL: https://github.com/apache/cloudstack/pull/2365#issuecomment-352810031
 
 
   @rhtyd is there a way to see the error log?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2046: CLOUDSTACK-7958: Add configuration for limit to CIDRs for Admin API calls

[GitBox]

blueorangutan commented on issue #2046: CLOUDSTACK-7958: Add configuration for 
limit to CIDRs for Admin API calls
URL: https://github.com/apache/cloudstack/pull/2046#issuecomment-352808357
 
 
   @rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted 
as I make progress.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on issue #2046: CLOUDSTACK-7958: Add configuration for limit to CIDRs for Admin API calls

[GitBox]

rhtyd commented on issue #2046: CLOUDSTACK-7958: Add configuration for limit to 
CIDRs for Admin API calls
URL: https://github.com/apache/cloudstack/pull/2046#issuecomment-352808093
 
 
   @blueorangutan package 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on issue #2365: CLOUDSTACK-10197: Rename xentools iso for XenServer 7.0+

[GitBox]

rhtyd commented on issue #2365: CLOUDSTACK-10197: Rename xentools iso for 
XenServer 7.0+
URL: https://github.com/apache/cloudstack/pull/2365#issuecomment-352807826
 
 
   I see new iso related failures, rest are known issues. Can you check @khos2ow


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2365: CLOUDSTACK-10197: Rename xentools iso for XenServer 7.0+

[GitBox]

blueorangutan commented on issue #2365: CLOUDSTACK-10197: Rename xentools iso 
for XenServer 7.0+
URL: https://github.com/apache/cloudstack/pull/2365#issuecomment-352780567
 
 
   Trillian test result (tid-1822)
   Environment: xenserver-65sp1 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 40212 seconds
   Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2365-t1822-xenserver-65sp1.zip
   Smoke tests completed. 61 look OK, 6 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   test_04_extract_Iso | `Failure` | 5.21 | test_iso.py
   test_01_vpc_privategw_acl | `Failure` | 91.89 | test_privategw_acl.py
   test_02_vpc_privategw_static_routes | `Failure` | 298.83 | 
test_privategw_acl.py
   test_03_vpc_privategw_restart_vpc_cleanup | `Failure` | 480.51 | 
test_privategw_acl.py
   test_04_rvpc_privategw_static_routes | `Failure` | 646.04 | 
test_privategw_acl.py
   test_02_create_template_with_checksum_sha1 | `Error` | 5.22 | 
test_templates.py
   test_03_create_template_with_checksum_sha256 | `Error` | 5.21 | 
test_templates.py
   test_04_create_template_with_checksum_md5 | `Error` | 5.21 | 
test_templates.py
   test_04_extract_template | `Failure` | 5.16 | test_templates.py
   ContextSuite context=TestISOUsage>:setup | `Error` | 0.00 | test_usage.py
   test_01_volume_usage | `Error` | 15.75 | test_usage.py
   test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | `Failure` | 465.68 | 
test_vpc_redundant.py
   test_04_rvpc_network_garbage_collector_nics | `Failure` | 407.38 | 
test_vpc_redundant.py
   test_05_rvpc_multi_tiers | `Failure` | 412.43 | test_vpc_redundant.py
   test_01_vpc_remote_access_vpn | `Error` | 166.47 | test_vpc_vpn.py
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rafaelweingartner commented on issue #2333: Add Java Default Certificat Authorities into the keystore if using a custom cert SSL

[GitBox]

rafaelweingartner commented on issue #2333: Add Java Default Certificat 
Authorities into the keystore if using a custom cert SSL
URL: https://github.com/apache/cloudstack/pull/2333#issuecomment-352767689
 
 
   @svenvogel now the problem is that you are bringing a lot of changes from 
master to 4.10.
   Can you rebase only your commits on top of 4.10?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2320: CLOUDSTACK-10178: Hotfixes to make 4.10 work

[GitBox]

blueorangutan commented on issue #2320: CLOUDSTACK-10178: Hotfixes to make 4.10 
work
URL: https://github.com/apache/cloudstack/pull/2320#issuecomment-352758966
 
 
   Trillian test result (tid-1824)
   Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 25952 seconds
   Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2320-t1824-kvm-centos7.zip
   Smoke tests completed. 54 look OK, 2 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   test_02_edit_iso | `Failure` | 0.10 | test_iso.py
   test_05_iso_permissions | `Failure` | 0.07 | test_iso.py
   test_04_rvpc_privategw_static_routes | `Failure` | 548.78 | 
test_privategw_acl.py
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nvazquez commented on issue #2281: CLOUDSTACK-10102: New network type (L2)

[GitBox]

nvazquez commented on issue #2281: CLOUDSTACK-10102: New network type (L2)
URL: https://github.com/apache/cloudstack/pull/2281#issuecomment-352753047
 
 
   Thanks @rafaelweingartner, I pushed a commit with suggested improvements


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2352: CLOUDSTACK-10175: prevent VPC list leakage

[GitBox]

blueorangutan commented on issue #2352: CLOUDSTACK-10175: prevent VPC list 
leakage
URL: https://github.com/apache/cloudstack/pull/2352#issuecomment-352747221
 
 
   Trillian test result (tid-1823)
   Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 32782 seconds
   Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2352-t1823-kvm-centos7.zip
   Smoke tests completed. 63 look OK, 4 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   test_04_verify_guest_lspci | `Error` | 651.57 | test_deploy_virtio_scsi_vm.py
   test_06_verify_guest_lspci_again | `Error` | 654.22 | 
test_deploy_virtio_scsi_vm.py
   test_01_vpc_privategw_acl | `Failure` | 71.90 | test_privategw_acl.py
   test_02_vpc_privategw_static_routes | `Failure` | 299.42 | 
test_privategw_acl.py
   test_03_vpc_privategw_restart_vpc_cleanup | `Failure` | 122.93 | 
test_privategw_acl.py
   test_04_rvpc_privategw_static_routes | `Failure` | 288.88 | 
test_privategw_acl.py
   test_02_create_template_with_checksum_sha1 | `Error` | 5.23 | 
test_templates.py
   test_03_create_template_with_checksum_sha256 | `Error` | 5.23 | 
test_templates.py
   test_04_create_template_with_checksum_md5 | `Error` | 5.22 | 
test_templates.py
   test_01_vpc_remote_access_vpn | `Error` | 91.53 | test_vpc_vpn.py
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] wido commented on issue #2046: CLOUDSTACK-7958: Add configuration for limit to CIDRs for Admin API calls

[GitBox]

wido commented on issue #2046: CLOUDSTACK-7958: Add configuration for limit to 
CIDRs for Admin API calls
URL: https://github.com/apache/cloudstack/pull/2046#issuecomment-352743584
 
 
   @rhtyd Fixed!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd closed pull request #2364: [CLOUDSTACK-10195] CloudStack MySQL HA problem -- No database selected

[GitBox]

rhtyd closed pull request #2364: [CLOUDSTACK-10195] CloudStack MySQL HA problem 
-- No database selected
URL: https://github.com/apache/cloudstack/pull/2364
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/framework/cluster/src/com/cloud/cluster/ClusterManagerImpl.java 
b/framework/cluster/src/com/cloud/cluster/ClusterManagerImpl.java
index 2e20f241e01..d4717cada55 100644
--- a/framework/cluster/src/com/cloud/cluster/ClusterManagerImpl.java
+++ b/framework/cluster/src/com/cloud/cluster/ClusterManagerImpl.java
@@ -22,7 +22,6 @@
 import java.nio.channels.SocketChannel;
 import java.rmi.RemoteException;
 import java.sql.Connection;
-import java.sql.SQLException;
 import java.sql.SQLNonTransientException;
 import java.sql.SQLRecoverableException;
 import java.util.ArrayList;
@@ -546,7 +545,7 @@ protected void runInContext() {
 profiler.start();
 
 profilerHeartbeatUpdate.start();
-
txn.transitToUserManagedConnection(getHeartbeatConnection());
+
txn.transitToAutoManagedConnection(TransactionLegacy.CLOUD_DB);
 if (s_logger.isTraceEnabled()) {
 s_logger.trace("Cluster manager heartbeat update, 
id:" + _mshostId);
 }
@@ -597,7 +596,6 @@ protected void runInContext() {
 invalidHeartbeatConnection();
 }
 } finally {
-
txn.transitToAutoManagedConnection(TransactionLegacy.CLOUD_DB);
 txn.close("ClusterHeartbeat");
 }
 }
@@ -616,15 +614,6 @@ private boolean isRootCauseConnectionRelated(Throwable e) {
 return false;
 }
 
-private Connection getHeartbeatConnection() throws SQLException {
-if (_heartbeatConnection == null) {
-final Connection conn = 
TransactionLegacy.getStandaloneConnectionWithException();
-_heartbeatConnection = new 
ConnectionConcierge("ClusterManagerHeartbeat", conn, false);
-}
-
-return _heartbeatConnection.conn();
-}
-
 private void invalidHeartbeatConnection() {
 if (_heartbeatConnection != null) {
 final Connection conn = 
TransactionLegacy.getStandaloneConnection();
diff --git a/framework/db/test/com/cloud/utils/db/TransactionTest.java 
b/framework/db/test/com/cloud/utils/db/TransactionTest.java
deleted file mode 100644
index fed663285d1..000
--- a/framework/db/test/com/cloud/utils/db/TransactionTest.java
+++ /dev/null
@@ -1,166 +0,0 @@
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements.  See the NOTICE file
-// distributed with this work for additional information
-// regarding copyright ownership.  The ASF licenses this file
-// to you under the Apache License, Version 2.0 (the
-// "License"); you may not use this file except in compliance
-// the License.  You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing,
-// software distributed under the License is distributed on an
-// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-// KIND, either express or implied.  See the License for the
-// specific language governing permissions and limitations
-// under the License.
-package com.cloud.utils.db;
-
-import java.sql.Connection;
-import java.sql.PreparedStatement;
-import java.sql.SQLException;
-
-import org.junit.After;
-import org.junit.AfterClass;
-import org.junit.Assert;
-import org.junit.BeforeClass;
-import org.junit.Test;
-
-import com.cloud.utils.component.ComponentContext;
-import com.cloud.utils.exception.CloudRuntimeException;
-
-/**
- * A test fixture to test APIs or bugs found for Transaction class. This test 
fixture will do one time setup before
- * all its testcases to set up a test db table, and then tear down these test 
db artifacts after all testcases are run.
- *
- */
-public class TransactionTest {
-
-@BeforeClass
-public static void oneTimeSetup() {
-try (
-Connection conn = TransactionLegacy.getStandaloneConnection();
-PreparedStatement pstmt =
-conn.prepareStatement("CREATE TABLE `cloud`.`test` (" + 
"`id` bigint unsigned NOT NULL UNIQUE AUTO_INCREMENT," + "`fld_int` int 
unsigned,"
-+ "`fld_long` bigint unsigned," + "`fld_string` 
varchar(255)," + "PRIMARY KEY (`id`)" + ") ENGINE=InnoDB DEFAULT 
CHARSET=utf8;");
-) {
-
-pstmt.execute();
-
-} catch (SQLException e) {
-throw new CloudRuntimeException("Problem with sql", e);
-}
-}
-
-

[cloudstack] branch master updated: CLOUDSTACK-10195: CloudStack MySQL HA problem - No database selected (#2364)

[rohit]

This is an automated email from the ASF dual-hosted git repository.

rohit pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cloudstack.git


The following commit(s) were added to refs/heads/master by this push:
 new 9179bd5  CLOUDSTACK-10195: CloudStack MySQL HA problem - No database 
selected (#2364)
9179bd5 is described below

commit 9179bd54d4d721c52c0f7c041cd0f5d714172073
Author: Rafael Weingärtner 
AuthorDate: Tue Dec 19 10:25:13 2017 -0200

CLOUDSTACK-10195: CloudStack MySQL HA problem - No database selected (#2364)

When using CloudStack with database HA configuration user receives warnings 
constantly saying “No database selected”.

This problem happens at 
com.cloud.cluster.ClusterManagerImpl.getHeartbeatTask().new 
ManagedContextRunnable(){...}.runInContext(), line 550. The scheme of the 
database is not properly set in the transaction when database HA options are 
enabled.
---
 .../src/com/cloud/cluster/ClusterManagerImpl.java  |  13 +-
 .../test/com/cloud/utils/db/TransactionTest.java   | 166 -
 2 files changed, 1 insertion(+), 178 deletions(-)

diff --git a/framework/cluster/src/com/cloud/cluster/ClusterManagerImpl.java 
b/framework/cluster/src/com/cloud/cluster/ClusterManagerImpl.java
index 2e20f24..d4717ca 100644
--- a/framework/cluster/src/com/cloud/cluster/ClusterManagerImpl.java
+++ b/framework/cluster/src/com/cloud/cluster/ClusterManagerImpl.java
@@ -22,7 +22,6 @@ import java.net.InetSocketAddress;
 import java.nio.channels.SocketChannel;
 import java.rmi.RemoteException;
 import java.sql.Connection;
-import java.sql.SQLException;
 import java.sql.SQLNonTransientException;
 import java.sql.SQLRecoverableException;
 import java.util.ArrayList;
@@ -546,7 +545,7 @@ public class ClusterManagerImpl extends ManagerBase 
implements ClusterManager, C
 profiler.start();
 
 profilerHeartbeatUpdate.start();
-
txn.transitToUserManagedConnection(getHeartbeatConnection());
+
txn.transitToAutoManagedConnection(TransactionLegacy.CLOUD_DB);
 if (s_logger.isTraceEnabled()) {
 s_logger.trace("Cluster manager heartbeat update, 
id:" + _mshostId);
 }
@@ -597,7 +596,6 @@ public class ClusterManagerImpl extends ManagerBase 
implements ClusterManager, C
 invalidHeartbeatConnection();
 }
 } finally {
-
txn.transitToAutoManagedConnection(TransactionLegacy.CLOUD_DB);
 txn.close("ClusterHeartbeat");
 }
 }
@@ -616,15 +614,6 @@ public class ClusterManagerImpl extends ManagerBase 
implements ClusterManager, C
 return false;
 }
 
-private Connection getHeartbeatConnection() throws SQLException {
-if (_heartbeatConnection == null) {
-final Connection conn = 
TransactionLegacy.getStandaloneConnectionWithException();
-_heartbeatConnection = new 
ConnectionConcierge("ClusterManagerHeartbeat", conn, false);
-}
-
-return _heartbeatConnection.conn();
-}
-
 private void invalidHeartbeatConnection() {
 if (_heartbeatConnection != null) {
 final Connection conn = 
TransactionLegacy.getStandaloneConnection();
diff --git a/framework/db/test/com/cloud/utils/db/TransactionTest.java 
b/framework/db/test/com/cloud/utils/db/TransactionTest.java
deleted file mode 100644
index fed6632..000
--- a/framework/db/test/com/cloud/utils/db/TransactionTest.java
+++ /dev/null
@@ -1,166 +0,0 @@
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements.  See the NOTICE file
-// distributed with this work for additional information
-// regarding copyright ownership.  The ASF licenses this file
-// to you under the Apache License, Version 2.0 (the
-// "License"); you may not use this file except in compliance
-// the License.  You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing,
-// software distributed under the License is distributed on an
-// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-// KIND, either express or implied.  See the License for the
-// specific language governing permissions and limitations
-// under the License.
-package com.cloud.utils.db;
-
-import java.sql.Connection;
-import java.sql.PreparedStatement;
-import java.sql.SQLException;
-
-import org.junit.After;
-import org.junit.AfterClass;
-import org.junit.Assert;
-import org.junit.BeforeClass;
-import org.junit.Test;
-
-import com.cloud.utils.component.ComponentContext;
-import com.cloud.utils.exception.CloudRuntimeException;
-
-/**
- * A test fixture to test APIs or bugs found for Transaction class. This test 
fixture 

[GitHub] rhtyd commented on issue #2364: [CLOUDSTACK-10195] CloudStack MySQL HA problem -- No database selected

[GitBox]

rhtyd commented on issue #2364: [CLOUDSTACK-10195] CloudStack MySQL HA problem 
-- No database selected
URL: https://github.com/apache/cloudstack/pull/2364#issuecomment-352734703
 
 
   Merging based on reviews and test results.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2149: CLOUDSTACK-9932 snapshot is getting deleted while volume is in creating state

[GitBox]

blueorangutan commented on issue #2149: CLOUDSTACK-9932 snapshot is getting 
deleted while volume is in creating state
URL: https://github.com/apache/cloudstack/pull/2149#issuecomment-352731862
 
 
   Packaging result: ?centos6 ?centos7 ?debian. JID-1426


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2104: CLOUDSTACK-9908 : Primary Storage allocated capacity goes very high after VM snapshot

[GitBox]

blueorangutan commented on issue #2104: CLOUDSTACK-9908 : Primary Storage 
allocated capacity goes very high after VM snapshot
URL: https://github.com/apache/cloudstack/pull/2104#issuecomment-352729942
 
 
   @rhtyd a Trillian-Jenkins test job (centos7 mgmt + vmware-55u3) has been 
kicked to run smoke tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on issue #2104: CLOUDSTACK-9908 : Primary Storage allocated capacity goes very high after VM snapshot

[GitBox]

rhtyd commented on issue #2104: CLOUDSTACK-9908 : Primary Storage allocated 
capacity goes very high after VM snapshot
URL: https://github.com/apache/cloudstack/pull/2104#issuecomment-352729831
 
 
   @blueorangutan test centos7 vmware-55u3
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2042: CLOUDSTACK-9875: Unable to re-apply Explicit dedication to VM

[GitBox]

blueorangutan commented on issue #2042: CLOUDSTACK-9875: Unable to re-apply 
Explicit dedication to VM
URL: https://github.com/apache/cloudstack/pull/2042#issuecomment-352729732
 
 
   @rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been 
kicked to run smoke tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #1740: CLOUDSTACK-9572 Snapshot on primary storage not cleaned up after Stor?

[GitBox]

blueorangutan commented on issue #1740: CLOUDSTACK-9572 Snapshot on primary 
storage not cleaned up after Stor?
URL: https://github.com/apache/cloudstack/pull/1740#issuecomment-352729720
 
 
   @rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been 
kicked to run smoke tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on issue #1740: CLOUDSTACK-9572 Snapshot on primary storage not cleaned up after Stor?

[GitBox]

rhtyd commented on issue #1740: CLOUDSTACK-9572 Snapshot on primary storage not 
cleaned up after Stor?
URL: https://github.com/apache/cloudstack/pull/1740#issuecomment-352729579
 
 
   @blueorangutan test


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on issue #2042: CLOUDSTACK-9875: Unable to re-apply Explicit dedication to VM

[GitBox]

rhtyd commented on issue #2042: CLOUDSTACK-9875: Unable to re-apply Explicit 
dedication to VM
URL: https://github.com/apache/cloudstack/pull/2042#issuecomment-352729492
 
 
   @blueorangutan test


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #1760: CLOUDSTACK-9593: userdata: enforce data is a multiple of 4 characters

[GitBox]

blueorangutan commented on issue #1760: CLOUDSTACK-9593: userdata: enforce data 
is a multiple of 4 characters
URL: https://github.com/apache/cloudstack/pull/1760#issuecomment-352729487
 
 
   @rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been 
kicked to run smoke tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on issue #1760: CLOUDSTACK-9593: userdata: enforce data is a multiple of 4 characters

[GitBox]

rhtyd commented on issue #1760: CLOUDSTACK-9593: userdata: enforce data is a 
multiple of 4 characters
URL: https://github.com/apache/cloudstack/pull/1760#issuecomment-352729409
 
 
   @blueorangutan test


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] blueorangutan commented on issue #2149: CLOUDSTACK-9932 snapshot is getting deleted while volume is in creating state

[GitBox]

blueorangutan commented on issue #2149: CLOUDSTACK-9932 snapshot is getting 
deleted while volume is in creating state
URL: https://github.com/apache/cloudstack/pull/2149#issuecomment-352726471
 
 
   @rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted 
as I make progress.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on issue #2149: CLOUDSTACK-9932 snapshot is getting deleted while volume is in creating state

[GitBox]

rhtyd commented on issue #2149: CLOUDSTACK-9932 snapshot is getting deleted 
while volume is in creating state
URL: https://github.com/apache/cloudstack/pull/2149#issuecomment-352726242
 
 
   @blueorangutan package


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rafaelweingartner commented on issue #2364: [CLOUDSTACK-10195] CloudStack MySQL HA problem -- No database selected

[GitBox]

rafaelweingartner commented on issue #2364: [CLOUDSTACK-10195] CloudStack MySQL 
HA problem -- No database selected
URL: https://github.com/apache/cloudstack/pull/2364#issuecomment-352723561
 
 
   @khos2ow, @nvazquez, @DaanHoogland, @GabrielBrascher can some of you review 
this PR as well?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rhtyd commented on issue #2363: 4.8.0.1 rc20160525 t1247

[GitBox]

rhtyd commented on issue #2363: 4.8.0.1 rc20160525 t1247
URL: https://github.com/apache/cloudstack/pull/2363#issuecomment-352723418
 
 
   Thanks @rafaelweingartner I'll give another day to @toshen to reply, if 
there is no answer we can close this PR as potential spam/mistake.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rafaelweingartner commented on issue #2363: 4.8.0.1 rc20160525 t1247

[GitBox]

rafaelweingartner commented on issue #2363: 4.8.0.1 rc20160525 t1247
URL: https://github.com/apache/cloudstack/pull/2363#issuecomment-352722954
 
 
   @rhtyd if there is no answer here, I vote for closing this PR.
   That is one of the biggest benefits of using the Gitbox experiment.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rafaelweingartner commented on a change in pull request #2281: CLOUDSTACK-10102: New network type (L2)

[GitBox]

rafaelweingartner commented on a change in pull request #2281: 
CLOUDSTACK-10102: New network type (L2)
URL: https://github.com/apache/cloudstack/pull/2281#discussion_r157728263
 
 

 ##
 File path: server/src/com/cloud/network/NetworkModelImpl.java
 ##
 @@ -1823,6 +1826,9 @@ public boolean isNetworkAvailableInDomain(long 
networkId, long domainId) {
 
 @Override
 public Set getAvailableIps(Network network, String requestedIp) {
+if (network.getCidr() == null) {
+return null;
 
 Review comment:
   Instead of `null`, what about an empty set here?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services