date:20131017

Hey Darren,

I'm having a look at the branch. Takes some time so i will get back to you when 
i have something.

Did you run the bvt test suite on this branch already?

Cheers,

Hugo

On Oct 16, 2013, at 6:59 PM, Darren Shepherd  
wrote:

> I need as many people as possible to review this branch.  I'm still
> testing it out, but I wanted to get as many eyes on it as possible.
> This is a huge cross cutting change.  This branch is the changes to
> use a new Transaction API that will be consistent Spring TX's style so
> that we can eventually move to it.  You can get a bit more context
> from 
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Database+Transactions
> 
> Having spent so much time looking at the transaction management in
> ACS, I'm well convinced we need to adopt Spring TX as soon as we can.
> I've found just too many bugs.  It will be a painful transitiion, as
> you can see from this branch.  It will also be very tricky, but I'll
> figure it out.
> 
> If you are reviewing this branch, use a diff tool that ignores
> whitespace.  Also if you don't know about "git difftool -d", you
> should use that.  Just know, its was 10x more tedious and painful for
> me to make this change then it is for you to review it :)
> 
> Darren

Re: [MERGE] txn-refactor

2013-10-17 Thread Sebastien Goasguen


On Oct 16, 2013, at 9:08 PM, Darren Shepherd  
wrote:

> How do these merge emails work?  Do I just wait and certain amount of time 
> and if nobody says no I just do it?  I've tested the branch as much as  I'd 
> like, so just waiting for input. 
> 

I think there is a 72 hours window, but we might want to wait one week to get 
this one merged ?

> Darren
> 
>> On Oct 16, 2013, at 9:59 AM, Darren Shepherd  
>> wrote:
>> 
>> I need as many people as possible to review this branch.  I'm still
>> testing it out, but I wanted to get as many eyes on it as possible.
>> This is a huge cross cutting change.  This branch is the changes to
>> use a new Transaction API that will be consistent Spring TX's style so
>> that we can eventually move to it.  You can get a bit more context
>> from 
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Database+Transactions
>> 
>> Having spent so much time looking at the transaction management in
>> ACS, I'm well convinced we need to adopt Spring TX as soon as we can.
>> I've found just too many bugs.  It will be a painful transitiion, as
>> you can see from this branch.  It will also be very tricky, but I'll
>> figure it out.
>> 
>> If you are reviewing this branch, use a diff tool that ignores
>> whitespace.  Also if you don't know about "git difftool -d", you
>> should use that.  Just know, its was 10x more tedious and painful for
>> me to make this change then it is for you to review it :)
>> 
>> Darren

Re: [MERGE] txn-refactor

Hey Darren,

Looking through the code it looks like this more an API change than an actual 
redesign of the transaction code? I like the resulting code a lot better than 
the existing way of doing it. As far as i can see you wrapped the existing 
TransactionLegacy way of doing it (txn.start / txn.commit) inside of the new 
Transaction functions execute and executeWithException. So if i understand it 
correctly, nothing changed in how transactions are actually handled, except 
that the code can now be easily changed to use spring TX.

Also you made that changes to a couple of classes to use the new api, but the 
majority of the classes still need to be done. It might be nice to annotate the 
TransactionLegacy class with @Deprecated so we can easily identify what needs 
to be done?

The new code is not covered by any new unit test yet? I couldn't check the 
cobertura result yet as there are some build and test failures. Can you have a 
look at this build : 
http://jenkins.buildacloud.org/job/cloudstack-maven-build-with-branch-parameter/3/.
  You can kick off that job yourself from jenkins if you like to do the full 
test. Didn't do the noredist build test yet as the normal build failed.

In short there are some things to fix before we can merge this.

Cheers,

Hugo

On Oct 17, 2013, at 9:04 AM, Hugo Trippaers  wrote:

> Hey Darren,
> 
> I'm having a look at the branch. Takes some time so i will get back to you 
> when i have something.
> 
> Did you run the bvt test suite on this branch already?
> 
> Cheers,
> 
> Hugo
> 
> On Oct 16, 2013, at 6:59 PM, Darren Shepherd  
> wrote:
> 
>> I need as many people as possible to review this branch.  I'm still
>> testing it out, but I wanted to get as many eyes on it as possible.
>> This is a huge cross cutting change.  This branch is the changes to
>> use a new Transaction API that will be consistent Spring TX's style so
>> that we can eventually move to it.  You can get a bit more context
>> from 
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Database+Transactions
>> 
>> Having spent so much time looking at the transaction management in
>> ACS, I'm well convinced we need to adopt Spring TX as soon as we can.
>> I've found just too many bugs.  It will be a painful transitiion, as
>> you can see from this branch.  It will also be very tricky, but I'll
>> figure it out.
>> 
>> If you are reviewing this branch, use a diff tool that ignores
>> whitespace.  Also if you don't know about "git difftool -d", you
>> should use that.  Just know, its was 10x more tedious and painful for
>> me to make this change then it is for you to review it :)
>> 
>> Darren
>

Snapshot Create on 4.2

2013-10-17 Thread Antonio Petrocelli


Hello,

after upgrade to 4.2 snapshot creation is stuck on *Ready *status.

before status was *BackedUp.

*Is there a parameter to set?

Thanks! *
*

Re: Review Request 14556: CLOUDSTACK-4766: Add timeout if vm does not reach running state

2013-10-17 Thread sanjeev n


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14556/#review27113
---

Ship it!


Patch is good to go.

- sanjeev n


On Oct. 9, 2013, 9:50 a.m., Girish Shilamkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14556/
> ---
> 
> (Updated Oct. 9, 2013, 9:50 a.m.)
> 
> 
> Review request for cloudstack, sanjeev n and venkata swamy babu  budumuru.
> 
> 
> Bugs: CLOUDSTACK-4766
> https://issues.apache.org/jira/browse/CLOUDSTACK-4766
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> The test use to wait for ever for the vm to attain Running state.
> Added a timeout so it does not get into infinite loop.
> 
> 
> Diffs
> -
> 
>   test/integration/component/test_reset_ssh_keypair.py ace4499 
> 
> Diff: https://reviews.apache.org/r/14556/diff/
> 
> 
> Testing
> ---
> 
> Verified that the timeout works. 
> 
> test_01_reset_keypair_normal_user 
> (test_reset_ssh_keypair.TestResetSSHKeyUserRights)Verify API 
> resetSSHKeyForVirtualMachine for non admin non root ... FAIL
> test_02_reset_keypair_domain_admin 
> (test_reset_ssh_keypair.TestResetSSHKeyUserRights)
> Verify API resetSSHKeyForVirtualMachine for domain admin non root ... ok
> test_03_reset_keypair_root_admin 
> (test_reset_ssh_keypair.TestResetSSHKeyUserRights)
> Verify API resetSSHKeyForVirtualMachine for domain admin root ... ok
> test_01_reset_ssh_keys (test_reset_ssh_keypair.TestResetSSHKeypair)
> Test Reset SSH keys for VM  already having SSH key ... ok
> test_02_reset_ssh_key_password_enabled_template 
> (test_reset_ssh_keypair.TestResetSSHKeypair)
> Reset SSH keys for VM  created from password enabled template and ... ok
> test_03_reset_ssh_with_no_key (test_reset_ssh_keypair.TestResetSSHKeypair)
> Reset SSH key for VM  having no SSH key ... ok
> test_04_reset_key_passwd_enabled_no_key 
> (test_reset_ssh_keypair.TestResetSSHKeypair)
> Reset SSH keys for VM  created from password enabled template and ... ok
> test_05_reset_key_in_running_state 
> (test_reset_ssh_keypair.TestResetSSHKeypair)
> Reset SSH keys for VM  already having SSH key when VM is in running ... ok
> test_06_reset_key_passwd_enabled_vm_running 
> (test_reset_ssh_keypair.TestResetSSHKeypair)
> Reset SSH keys for VM  created from password enabled template and ... ok
> test_07_reset_keypair_invalid_params 
> (test_reset_ssh_keypair.TestResetSSHKeypair)
> Verify API resetSSHKeyForVirtualMachine with incorrect parameters … ok
> 
> ==
> FAIL: test_01_reset_keypair_normal_user 
> (test_reset_ssh_keypair.TestResetSSHKeyUserRights)
> Verify API resetSSHKeyForVirtualMachine for non admin non root
> --
> Traceback (most recent call last):
>   File "/root/girish/test_reset_ssh_keypair.py", line 1218, in 
> test_01_reset_keypair_normal_user
> % (vms[0].name, self.services["timeout"]))
> AssertionError: The virtual machine 974d1275-b747-441c-83b2-1795de4d87df 
> failed to start even after 10 minutes
> 
> --
> 
> 
> Thanks,
> 
> Girish Shilamkar
> 
>

Re: Review Request 14557: CLOUDSTACK-4747: Rename testcase name to use lesser characters


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14557/#review27114
---


Commit c5e1c4725c3658cae967c63cbae0ffe598f227ef in branch refs/heads/master 
from Girish Shilamkar
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=c5e1c47 ]

CLOUDSTACK-4747: Rename testcase name to use lesser characters

Renamed testcase name and also initialised _cleanup so that
it does not break on non-NS Cloudstack setup.

Signed-off-by: venkataswamybabu budumuru 


- ASF Subversion and Git Services


On Oct. 11, 2013, 8:31 a.m., Girish Shilamkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14557/
> ---
> 
> (Updated Oct. 11, 2013, 8:31 a.m.)
> 
> 
> Review request for cloudstack, Sowmya Krishnan and SrikanteswaraRao Talluri.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Renamed testcase name and also initialised _cleanup so that
> it does not break on non-NS Cloudstack setup.
> 
> 
> Diffs
> -
> 
>   test/integration/component/test_netscaler_nw_off.py 3139257 
> 
> Diff: https://reviews.apache.org/r/14557/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Girish Shilamkar
> 
>

Re: Review Request 14557: CLOUDSTACK-4747: Rename testcase name to use lesser characters


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14557/#review27115
---


Commit 08e69f130f142b0b034b6f42715ae0d1ca390c12 in branch refs/heads/4.2 from 
Girish Shilamkar
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=08e69f1 ]

CLOUDSTACK-4747: Rename testcase name to use lesser characters

Renamed testcase name and also initialised _cleanup so that
it does not break on non-NS Cloudstack setup.

Signed-off-by: venkataswamybabu budumuru 
(cherry picked from commit c5e1c4725c3658cae967c63cbae0ffe598f227ef)


- ASF Subversion and Git Services


On Oct. 11, 2013, 8:31 a.m., Girish Shilamkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14557/
> ---
> 
> (Updated Oct. 11, 2013, 8:31 a.m.)
> 
> 
> Review request for cloudstack, Sowmya Krishnan and SrikanteswaraRao Talluri.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Renamed testcase name and also initialised _cleanup so that
> it does not break on non-NS Cloudstack setup.
> 
> 
> Diffs
> -
> 
>   test/integration/component/test_netscaler_nw_off.py 3139257 
> 
> Diff: https://reviews.apache.org/r/14557/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Girish Shilamkar
> 
>

[JENKINS] systemvm build jobs

Heya,

The systemvm jobs on master are failing because the netinst iso for 7.0.0 is no 
longer available from the debian mirrors. I've added the images to the 
userContent on the master so we can use the copy-to-slave plugin to get them. 
I'm updating and testing the jobs now.

Cheers,

Hugo

Re: Review Request 14556: CLOUDSTACK-4766: Add timeout if vm does not reach running state


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14556/#review27116
---


Commit e3bcdc16a11d7452b5bf6ce5e5993dcd008526a6 in branch refs/heads/4.2 from 
Girish Shilamkar
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=e3bcdc1 ]

CLOUDSTACK-4766: Add timeout if vm does not reach running state

The tests use to wait for ever for the vm to attain Running state.
Added a timeout so it does not get into infinite loop.

Signed-off-by: venkataswamybabu budumuru 


- ASF Subversion and Git Services


On Oct. 9, 2013, 9:50 a.m., Girish Shilamkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14556/
> ---
> 
> (Updated Oct. 9, 2013, 9:50 a.m.)
> 
> 
> Review request for cloudstack, sanjeev n and venkata swamy babu  budumuru.
> 
> 
> Bugs: CLOUDSTACK-4766
> https://issues.apache.org/jira/browse/CLOUDSTACK-4766
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> The test use to wait for ever for the vm to attain Running state.
> Added a timeout so it does not get into infinite loop.
> 
> 
> Diffs
> -
> 
>   test/integration/component/test_reset_ssh_keypair.py ace4499 
> 
> Diff: https://reviews.apache.org/r/14556/diff/
> 
> 
> Testing
> ---
> 
> Verified that the timeout works. 
> 
> test_01_reset_keypair_normal_user 
> (test_reset_ssh_keypair.TestResetSSHKeyUserRights)Verify API 
> resetSSHKeyForVirtualMachine for non admin non root ... FAIL
> test_02_reset_keypair_domain_admin 
> (test_reset_ssh_keypair.TestResetSSHKeyUserRights)
> Verify API resetSSHKeyForVirtualMachine for domain admin non root ... ok
> test_03_reset_keypair_root_admin 
> (test_reset_ssh_keypair.TestResetSSHKeyUserRights)
> Verify API resetSSHKeyForVirtualMachine for domain admin root ... ok
> test_01_reset_ssh_keys (test_reset_ssh_keypair.TestResetSSHKeypair)
> Test Reset SSH keys for VM  already having SSH key ... ok
> test_02_reset_ssh_key_password_enabled_template 
> (test_reset_ssh_keypair.TestResetSSHKeypair)
> Reset SSH keys for VM  created from password enabled template and ... ok
> test_03_reset_ssh_with_no_key (test_reset_ssh_keypair.TestResetSSHKeypair)
> Reset SSH key for VM  having no SSH key ... ok
> test_04_reset_key_passwd_enabled_no_key 
> (test_reset_ssh_keypair.TestResetSSHKeypair)
> Reset SSH keys for VM  created from password enabled template and ... ok
> test_05_reset_key_in_running_state 
> (test_reset_ssh_keypair.TestResetSSHKeypair)
> Reset SSH keys for VM  already having SSH key when VM is in running ... ok
> test_06_reset_key_passwd_enabled_vm_running 
> (test_reset_ssh_keypair.TestResetSSHKeypair)
> Reset SSH keys for VM  created from password enabled template and ... ok
> test_07_reset_keypair_invalid_params 
> (test_reset_ssh_keypair.TestResetSSHKeypair)
> Verify API resetSSHKeyForVirtualMachine with incorrect parameters … ok
> 
> ==
> FAIL: test_01_reset_keypair_normal_user 
> (test_reset_ssh_keypair.TestResetSSHKeyUserRights)
> Verify API resetSSHKeyForVirtualMachine for non admin non root
> --
> Traceback (most recent call last):
>   File "/root/girish/test_reset_ssh_keypair.py", line 1218, in 
> test_01_reset_keypair_normal_user
> % (vms[0].name, self.services["timeout"]))
> AssertionError: The virtual machine 974d1275-b747-441c-83b2-1795de4d87df 
> failed to start even after 10 minutes
> 
> --
> 
> 
> Thanks,
> 
> Girish Shilamkar
> 
>

Re: Review Request 14556: CLOUDSTACK-4766: Add timeout if vm does not reach running state


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14556/#review27117
---


Commit 9cc557509265d8faacca134e8bfeb3a31ddbeb31 in branch refs/heads/master 
from Girish Shilamkar
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=9cc5575 ]

CLOUDSTACK-4766: Add timeout if vm does not reach running state

The tests use to wait for ever for the vm to attain Running state.
Added a timeout so it does not get into infinite loop.

Signed-off-by: venkataswamybabu budumuru 
(cherry picked from commit e3bcdc16a11d7452b5bf6ce5e5993dcd008526a6)


- ASF Subversion and Git Services


On Oct. 9, 2013, 9:50 a.m., Girish Shilamkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14556/
> ---
> 
> (Updated Oct. 9, 2013, 9:50 a.m.)
> 
> 
> Review request for cloudstack, sanjeev n and venkata swamy babu  budumuru.
> 
> 
> Bugs: CLOUDSTACK-4766
> https://issues.apache.org/jira/browse/CLOUDSTACK-4766
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> The test use to wait for ever for the vm to attain Running state.
> Added a timeout so it does not get into infinite loop.
> 
> 
> Diffs
> -
> 
>   test/integration/component/test_reset_ssh_keypair.py ace4499 
> 
> Diff: https://reviews.apache.org/r/14556/diff/
> 
> 
> Testing
> ---
> 
> Verified that the timeout works. 
> 
> test_01_reset_keypair_normal_user 
> (test_reset_ssh_keypair.TestResetSSHKeyUserRights)Verify API 
> resetSSHKeyForVirtualMachine for non admin non root ... FAIL
> test_02_reset_keypair_domain_admin 
> (test_reset_ssh_keypair.TestResetSSHKeyUserRights)
> Verify API resetSSHKeyForVirtualMachine for domain admin non root ... ok
> test_03_reset_keypair_root_admin 
> (test_reset_ssh_keypair.TestResetSSHKeyUserRights)
> Verify API resetSSHKeyForVirtualMachine for domain admin root ... ok
> test_01_reset_ssh_keys (test_reset_ssh_keypair.TestResetSSHKeypair)
> Test Reset SSH keys for VM  already having SSH key ... ok
> test_02_reset_ssh_key_password_enabled_template 
> (test_reset_ssh_keypair.TestResetSSHKeypair)
> Reset SSH keys for VM  created from password enabled template and ... ok
> test_03_reset_ssh_with_no_key (test_reset_ssh_keypair.TestResetSSHKeypair)
> Reset SSH key for VM  having no SSH key ... ok
> test_04_reset_key_passwd_enabled_no_key 
> (test_reset_ssh_keypair.TestResetSSHKeypair)
> Reset SSH keys for VM  created from password enabled template and ... ok
> test_05_reset_key_in_running_state 
> (test_reset_ssh_keypair.TestResetSSHKeypair)
> Reset SSH keys for VM  already having SSH key when VM is in running ... ok
> test_06_reset_key_passwd_enabled_vm_running 
> (test_reset_ssh_keypair.TestResetSSHKeypair)
> Reset SSH keys for VM  created from password enabled template and ... ok
> test_07_reset_keypair_invalid_params 
> (test_reset_ssh_keypair.TestResetSSHKeypair)
> Verify API resetSSHKeyForVirtualMachine with incorrect parameters … ok
> 
> ==
> FAIL: test_01_reset_keypair_normal_user 
> (test_reset_ssh_keypair.TestResetSSHKeyUserRights)
> Verify API resetSSHKeyForVirtualMachine for non admin non root
> --
> Traceback (most recent call last):
>   File "/root/girish/test_reset_ssh_keypair.py", line 1218, in 
> test_01_reset_keypair_normal_user
> % (vms[0].name, self.services["timeout"]))
> AssertionError: The virtual machine 974d1275-b747-441c-83b2-1795de4d87df 
> failed to start even after 10 minutes
> 
> --
> 
> 
> Thanks,
> 
> Girish Shilamkar
> 
>

Re: Review Request 14531: CLOUDSTACK-702: Verify Userdata and Password service on alias ip on VR


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14531/#review27118
---


Commit 7a7fb61a17911987838a2b6744d594c840df7c2e in branch refs/heads/master 
from sanjeevneelarapu
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=7a7fb61 ]

CLOUDSTACK-702: Verify Userdata,password services 1.Added two tests to vefiy 
userdata and password services on alias ip on VR

Conflicts:

test/integration/component/maint/test_multiple_ip_ranges.py

Signed-off-by: sanjeevneelarapu 
Signed-off-by: venkataswamybabu budumuru 


- ASF Subversion and Git Services


On Oct. 8, 2013, 11:17 a.m., sanjeev n wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14531/
> ---
> 
> (Updated Oct. 8, 2013, 11:17 a.m.)
> 
> 
> Review request for cloudstack, venkata swamy babu  budumuru, SrikanteswaraRao 
> Talluri, and Prasanna Santhanam.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Verify Userdata and Password service on alias ip on VR
> 1.Added two tests to verify userdata and password service after ip alias 
> creation on VR.
> 2.Tests will deploy vm in new CIDR,checks alias creation and verify services 
> on alias ip addresses on VR.
> 
> 
> Diffs
> -
> 
>   test/integration/component/maint/test_multiple_ip_ranges.py 68b5979 
> 
> Diff: https://reviews.apache.org/r/14531/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> sanjeev n
> 
>

Re: Review Request 14531: CLOUDSTACK-702: Verify Userdata and Password service on alias ip on VR


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14531/#review27119
---


Commit 8b0894b27c45c3e89824ece3a6c47c8143d3ae76 in branch refs/heads/4.2 from 
sanjeevneelarapu
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=8b0894b ]

CLOUDSTACK-702: Verify Userdata,password services 1.Added two tests to vefiy 
userdata and password services on alias ip on VR

Conflicts:

test/integration/component/maint/test_multiple_ip_ranges.py

Signed-off-by: sanjeevneelarapu 
Signed-off-by: venkataswamybabu budumuru 
(cherry picked from commit 7a7fb61a17911987838a2b6744d594c840df7c2e)


- ASF Subversion and Git Services


On Oct. 8, 2013, 11:17 a.m., sanjeev n wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14531/
> ---
> 
> (Updated Oct. 8, 2013, 11:17 a.m.)
> 
> 
> Review request for cloudstack, venkata swamy babu  budumuru, SrikanteswaraRao 
> Talluri, and Prasanna Santhanam.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Verify Userdata and Password service on alias ip on VR
> 1.Added two tests to verify userdata and password service after ip alias 
> creation on VR.
> 2.Tests will deploy vm in new CIDR,checks alias creation and verify services 
> on alias ip addresses on VR.
> 
> 
> Diffs
> -
> 
>   test/integration/component/maint/test_multiple_ip_ranges.py 68b5979 
> 
> Diff: https://reviews.apache.org/r/14531/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> sanjeev n
> 
>

Re: Review Request 14459: CLOUDSTACK-2243: Add automation tests for VMs base image update faclity

2013-10-17 Thread Santhosh Edukulla


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14459/#review27067
---



test/integration/component/test_base_image_updation.py


empty list check


- Santhosh Edukulla


On Oct. 10, 2013, 10:23 a.m., Ashutosh Kelkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14459/
> ---
> 
> (Updated Oct. 10, 2013, 10:23 a.m.)
> 
> 
> Review request for cloudstack and Girish Shilamkar.
> 
> 
> Bugs: CLOUDSTACK-2243
> https://issues.apache.org/jira/browse/CLOUDSTACK-2243
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Added new test class to test Base Image Updation Facility.
> 
> 
> Diffs
> -
> 
>   test/integration/component/test_base_image_updation.py PRE-CREATION 
>   tools/marvin/marvin/integration/lib/base.py 0d52224 
> 
> Diff: https://reviews.apache.org/r/14459/diff/
> 
> 
> Testing
> ---
> 
> Tested locally on KVM advanced setup:
> 
> Log:
> 
> ==> result.log <==
> test_01_deploy_instance_with_is_volatile_offering 
> (test_base_image_updation.TestBaseImageUpdate)
> Test deploy an instance with service offerings with IsVolatile set. ... ok
> test_02_reboot_instance_with_is_volatile_offering 
> (test_base_image_updation.TestBaseImageUpdate)
> Test rebooting instances created with isVolatile service offerings ...
> ==> client.log <==
> 2013-10-09 20:22:30,659 - DEBUG - 
> test_02_reboot_instance_with_is_volatile_offering 
> (test_base_image_updation.TestBaseImageUpdate) - Checking root
> th isVolatile=True
> 2013-10-09 20:22:30,695 - DEBUG - 
> test_02_reboot_instance_with_is_volatile_offering 
> (test_base_image_updation.TestBaseImageUpdate) - Checking root
> th isVolatile=False
> 
> ==> result.log <==
> ok
> test_03_restore_vm_with_new_template 
> (test_base_image_updation.TestBaseImageUpdate)
> Test restoring a vm with different template than the one it was created with 
> ...
> ==> client.log <==
> 2013-10-09 20:22:32,373 - DEBUG - test_03_restore_vm_with_new_template 
> (test_base_image_updation.TestBaseImageUpdate) - Registered a template of fo
> ith ID: 5a9190a9-8a59-487f-b90f-4a76a0f509a0
> 2013-10-09 20:22:32,373 - DEBUG - test_03_restore_vm_with_new_template 
> (test_base_image_updation.TestBaseImageUpdate) - Waiting for download of tem
> : 5a9190a9-8a59-487f-b90f-4a76a0f509a0
> 2013-10-09 20:38:25,367 - DEBUG - test_03_restore_vm_with_new_template 
> (test_base_image_updation.TestBaseImageUpdate) - Checking template id of VM
> le=True
> 2013-10-09 20:38:25,386 - DEBUG - test_03_restore_vm_with_new_template 
> (test_base_image_updation.TestBaseImageUpdate) - Checking template id of VM
> le=False
> 2013-10-09 20:38:30,541 - DEBUG - test_04_reoccuring_snapshot_rules 
> (test_base_image_updation.TestBaseImageUpdate) - Creating recurring snapshot 
> po
>  disk on vm created with IsVolatile=True
> 2013-10-09 20:38:30,541 - DEBUG - test_04_reoccuring_snapshot_rules 
> (test_base_image_updation.TestBaseImageUpdate) - Snapshot Policy - Type : 
> HOURL
> inute : 53
> 2013-10-09 20:38:30,982 - DEBUG - test_04_reoccuring_snapshot_rules 
> (test_base_image_updation.TestBaseImageUpdate) - Sleeping for 25 minutes till 
> t
> snapshoted
> 
> ==> result.log <==
> ok
> test_04_reoccuring_snapshot_rules 
> (test_base_image_updation.TestBaseImageUpdate) ...
> ==> client.log <==
> 2013-10-09 21:11:51,780 - DEBUG - test_04_reoccuring_snapshot_rules 
> (test_base_image_updation.TestBaseImageUpdate) - Checking whether root disk of
> atile=True was destroyed
> 2013-10-09 21:11:51,827 - DEBUG - test_04_reoccuring_snapshot_rules 
> (test_base_image_updation.TestBaseImageUpdate) - Checking whether snapshot 
> rule
> isVolatile=True was destroyed
> 
> ==> result.log <==
> ok
> 
> --
> Ran 4 tests in 3534.241s
> 
> OK
> 
> 
> Thanks,
> 
> Ashutosh Kelkar
> 
>

hackathons so far

Users and developers,

So far the list of hackathons on CCC Europe has taken a serious shape.
Don't forget to check and make sure your personal or company interest
is represented at
https://cwiki.apache.org/confluence/display/CLOUDSTACK/CCC+Europe:


h2. hackathon proposals
|| chair || subject || description ||
| Daan/? | automated testing | Improve the quality of releases by
adding more unit and integration tests to our automated test-suites.
Discuss whether the present suites of unit tests and bvt are enough or
we might need more classes of tests (i.e. component level and a
seperation of integration from functional tests) |
| Daan/? | modularization and plug-ability | make cloudstack more
plug-able and maintainable by 3rd party vendors. At the moment several
mudularization efforts are on the way. The UI guys are working on
splitting css, the storare guys are driving towards a better
plug-model for drivers and at a system level work is being done on
better organizing the injection framework. These might be worth
separate hackathons but a measure of communication is no luxury. |
|wido| Review Ceph integration | Further review and improve the Ceph
RBD integration for KVM and also look at Xen |
|Sebastien ? | API Interfaces | (AWS refactor, GCE, OCCI&CIMI
standard): Discuss the state of our interfaces, plans for future,
needs etc. AWS interface might need a refactor, GCE is a new
interface, OCCI is a standard and Isaac Chiang has developed an
interface. We are missing a CIMI interface. |
|Sebastien ? | DOCs | We are having lots of talks about docs, they
have been split in a separate repo, we need to discuss format, release
life cycle, format etc.
 |
|Sebastien /Wido ? | KVM Agent re-factor | There has been
discussions/wishes to re-write the KVM agent in something else than
Java. Review architecture, define a plan, find developers :)
 |
|Sebastien ? | Ecosystem support | The are lots of tools in the cloud
ecosystem, we should talk about docker, ansible, cloud
foundry/bosh…etc.and define a plan to have great cloudstack support in
all of those.
|
|Abhinandan/Jayapal  | Virtual Router | Hacking various services
running on the virtual router and the services configuration via
management server.
|
|Kishan Kavala  | Usage | Creating a flexible usage framework so  that
new usage entities could be added easily.
|

regards,
Daan

Re: Review Request 14591: Fixng the issue of wrong netmask value is allowed in public traffic wizard while creating a zone

2013-10-17 Thread Jayapal Reddy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14591/#review27122
---



utils/src/com/cloud/utils/net/NetUtils.java


This patch is failed to apply on master.



utils/src/com/cloud/utils/net/NetUtils.java


Please remove the tabs here


- Jayapal Reddy


On Oct. 11, 2013, 10:53 a.m., Damodar Reddy Talakanti wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14591/
> ---
> 
> (Updated Oct. 11, 2013, 10:53 a.m.)
> 
> 
> Review request for cloudstack, Abhinandan Prateek, Jayapal Reddy, and Murali 
> Reddy.
> 
> 
> Bugs: https://issues.apache.org/jira/browse/CLOUDSTACK-4811
> 
> https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/CLOUDSTACK-4811
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> The UI Wizard was allowing Invalid CIDR in public traffic page while creating 
> a zone and back end was also not checking the correctness of the CIDR before 
> creating a zone.
> 
> 
> Diffs
> -
> 
>   server/src/com/cloud/configuration/ConfigurationManagerImpl.java b09b8ca 
>   utils/pom.xml f4ac12b 
>   utils/src/com/cloud/utils/net/NetUtils.java 05b485b 
> 
> Diff: https://reviews.apache.org/r/14591/diff/
> 
> 
> Testing
> ---
> 
> Testing is done through UI
> For Example : Given wrong NetMask value 255.255.25.0 which is wrong net mask 
> value
>   Given Gateway as 10.147.30.1 and  NetMask as 255.255.255.192 
> and public IP Range as 10.147.30.150-10.147.30.159..which is a wrong ip range 
> for the given gateway and net mask values.
> 
> 
> Thanks,
> 
> Damodar Reddy Talakanti
> 
>

internal LB provider (from the thread: [VOTE] Apache CloudStack 4.2.0 (third round))

Marcus.

I ran into this on today's master. Did you follow up on this in 4.2?
Or do you have a ticket or wiki page for it?

tahnks
Daan

On Thu, Aug 29, 2013 at 1:57 AM, Marcus Sorensen  wrote:
> 2. can't figure out how to deploy VPC. In 4.2 it tells me to jump
> through a few extra hoops regarding setting up internal loadbalancer
> service first, "Provider InternalLbVm should be enabled in at least
> one physical network of the zone specified", so I look at my physical
> network -> network service providers and see "Internal LB VM" is
> disabled, so I try to enable it and get "Provider is not ready, cannot
> Enable the provider, please configure the provider first". I'm stuck,
> I don't see anything in the UI that would provide a configuration
> dialog for the internal lb provider, nor do I see anything in the
> global options. It makes me wonder how upgrades will fare if VPCs are
> being used.

[GSoC] Congratulations and Wrap up

2013-10-17 Thread sebgoa

Hi,

This week end is the Google Summer of Code summit. This year we had the chance 
to get five students in the program: Dharmesh, Meng, Ian, Nguyen and Shiva and 
I'd be remiss not to acknowledge their hard work.

I would like to congratulate all of them on working hard this past summer 
(program finished late september), and challenging themselves to learn the open 
source way through our project at the Apache Software Foundation.

Thanks also to the mentors: Kelcey, Hugo and Abhi who took time out of their 
busy schedule and $dayjob to help these students.

-Ian Duffy made committer during the GSoC period, he developed a new LDAP/AD 
plugin and did an outstanding job doing his development in a complete software 
development pipeline with 90% unit test coverage, automatic building of rpms 
through jenkins etc. Check it out:
http://youtu.be/-3LG8wP7Zac
http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.2.0/html/Admin_Guide/LDAPserver-for-user-authentication.html
Lately Ian has worked with me on a Google compute engine (GCE) interface but 
this was not part of his charter for GSoC, just a bonus:
https://pypi.python.org/pypi/gcloud/0.0.1

-Nguyen worked on extending our GRE SDN controller to support Xen and KVM 
hypervisor. 
All his code is in the sdnextensions branch, and once Hugo is done reviewing it 
(or any other volunteer), we will merge it for 4.3
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=tree;h=refs/heads/sdnextensions;hb=sdnextensions
Nguyen has done a terrific job tracking his progress in JIRA and dividing work 
in sub-tasks:
https://issues.apache.org/jira/browse/CLOUDSTACK-1779

Both Nguyen and Ian will be at the CCC conference in Amsterdam, Nguyen will 
talk about 'Autoscaling without netscaler' a patch he submitted recently, and 
Ian will talk about gcloud.

-Meng worked on integration of whirr in CloudStack. The goal was to be able to 
provision Hadoop clusters in an elastic manner and use Yarn to expose a 
map-reduce API compatible with AWS EMR. This was a very ambitious project that 
required learning whirr, working across projects at the ASF and learning how to 
create new APIs. Her final patch is on review board:
https://reviews.apache.org/r/14347/
This will need additional work and more architectural thinking before we can 
use it, but is a solid base to think through how to expose an AWS EMR api with 
CloudStack.

-Shiva worked on a new UI framework using bootstrap and angular.js, he 
progressed well and his code is now in master under tools/ngui , he also has a 
final patch in my inbox. Bad practice but review board is finicky with large 
patches :( This UI is very interesting because it can operate as standalone UI, 
meaning that you can run locally to manage your cloud remotely.

Unfortunately Dharmesh was busy and did not progress well enough to pass the 
mid-term mark, he did try to tackle the big task of moving us from com.cloud to 
org.apache:
https://issues.apache.org/jira/browse/CLOUDSTACK-212 something that we still 
need to do and requires some planning/coordination and should be done very 
early in the release cycle (like now…)

Early on they all worked on a great document that is built on jenkins but not 
released as part of our docs, it served the purpose of getting everyone through 
the "bonding period" and have all students learn how to submit a patch:
http://jenkins.buildacloud.org/view/All/job/docs-4.3-gsoc-guide/
I will try to wrap it up to have a final version of our GSoC 2013 for Apache 
CloudStack.

Again congratulations to all, I am thrilled to have had you in the project and 
I hope you will continue participating in the community, we will hopefully do 
this again next year with another set of students.

Cheers,

-Sebastien

Re: internal LB provider (from the thread: [VOTE] Apache CloudStack 4.2.0 (third round))

Yes, it should supposedly never happen under normal circumstances. The
issue I had was because I was using a Marvin zone config that needed
to be updated for the new feature. Added:

{

"broadcastdomainrange": "ZONE",

"name": "InternalLbVm"

 }

This should be configured on zone setup as well. To fix it manually, I did:

# list internalloadbalancerelements

# configure internalloadbalancerelement enabled=true id=

On Thu, Oct 17, 2013 at 8:03 AM, Daan Hoogland  wrote:
> Marcus.
>
> I ran into this on today's master. Did you follow up on this in 4.2?
> Or do you have a ticket or wiki page for it?
>
> tahnks
> Daan
>
> On Thu, Aug 29, 2013 at 1:57 AM, Marcus Sorensen  wrote:
>> 2. can't figure out how to deploy VPC. In 4.2 it tells me to jump
>> through a few extra hoops regarding setting up internal loadbalancer
>> service first, "Provider InternalLbVm should be enabled in at least
>> one physical network of the zone specified", so I look at my physical
>> network -> network service providers and see "Internal LB VM" is
>> disabled, so I try to enable it and get "Provider is not ready, cannot
>> Enable the provider, please configure the provider first". I'm stuck,
>> I don't see anything in the UI that would provide a configuration
>> dialog for the internal lb provider, nor do I see anything in the
>> global options. It makes me wonder how upgrades will fare if VPCs are
>> being used.

Re: Wiki access

Added edit permissions for Harm Boertien (CCC EU Organizer)

Cheers,

Hugo


On 16 okt. 2013, at 16:28, Hugo Trippaers  wrote:

> Kishan,
> 
> look like you have the permissions already on account kishan. Can you check?
> 
> Cheers,
> 
> Hugo
> 
> 
> On Oct 16, 2013, at 3:45 PM, Kishan Kavala  wrote:
> 
>> Please grant me edit access.
>> Email: kis...@cloud.com
>> 
>> ~kishan
>> 
>>> -Original Message-
>>> From: Radhika Puthiyetath [mailto:radhika.puthiyet...@citrix.com]
>>> Sent: Wednesday, 16 October 2013 3:22 PM
>>> To: dev@cloudstack.apache.org
>>> Subject: RE: Wiki access
>>> 
>>> Hi,
>>> 
>>> I do not have the edit permissions. My username is radhikap.
>>> 
>>> Kindly provide me with the necessary permissions.
>>> 
>>> Thanks
>>> -Radhika
>>> 
>>> -Original Message-
>>> From: Daan Hoogland [mailto:daan.hoogl...@gmail.com]
>>> Sent: Friday, October 04, 2013 3:26 PM
>>> To: dev
>>> Subject: Re: Wiki access
>>> 
>>> anshulg is in
>>> 
>>> 
>>> On Fri, Oct 4, 2013 at 11:51 AM, Anshul Gangwar
>>> wrote:
>>> 
 need edit permissions , username: anshulg
 
 Regards,
 Anshul
 
 On Friday 04 October 2013 02:47 PM, Saksham Srivastava wrote:
> Please provide me edit permissions, username: saksham
> 
> Regards,
> Saksham
> 
> -Original Message-
> From: Chiradeep Vittal [mailto:chiradeep.vit...@citrix.com]
> Sent: Friday, October 04, 2013 10:46 AM
> To: dev@cloudstack.apache.org
> Subject: Re: Wiki access
> 
> Mike, Tuna, Srinivas, Go, done.
> 
> On 10/3/13 9:12 PM, "Mike Tutkowski" 
 wrote:
> 
>> I seem to need edit access, as well.
>> 
>> I'm usually logged in by default, but I believe my username is
>> mike-tutkowski (else please try mtutkowski).
>> 
>> Thanks!
>> 
>> 
>> On Thu, Oct 3, 2013 at 7:07 PM, Go Chiba  wrote:
>> 
>>> Hi Chip,
>>> 
>>> Please grant me to edit pages. My id is "gochiba".
>>> 
>>> 
>>> On Thu, Oct 3, 2013 at 6:01 AM, Chip Childers
>>> >>> wrote:
 Not all committers were easy to find, but those that I did should
 be
>>> all
 set.
 
 So committers / contributors = provide your confluence ID's if
 you
>>> can't
 add / edit pages.  PMC folks - help add them please.
 
 
 On Wed, Oct 2, 2013 at 4:48 PM, Chip Childers
>>>  wrote:
> There were major issues with Spam on the wiki, so infra changed
> the permissions.
> 
> I've just gone and added all PMC members to the space as space
>>> admins.
> That means all PMC members can help deal with this issue (or
> at
>>> least
> those that I could find in the wiki user list).
> 
> I'm going to deal with committers next.  That permission level
>>> needs to
> exclude space admin.  I'm using alena1108 as the example for
>>> committer
>>> /
> contributor permissions.
> 
> So if you are not a committer, give your confluence ID in this
>>> thread
>>> and
> PMC folks can help get them added (make them look like alena's
>>> perms).
> 
> 
> 
> On Wed, Oct 2, 2013 at 4:35 PM, Chiradeep Vittal <
> chiradeep.vit...@citrix.com> wrote:
> 
>> Apparently, edit access has been revoked for all. Who do we
>> contact
>>> for
>> edit permissions?
>> 
>> TIA
>> 
> 
>>> 
>>> 
>>> --
>>> 千葉 豪  Go Chiba
>>> E-mail:go.ch...@gmail.com
>>> 
>> 
>> 
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkow...@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the
>> cloud
>> *?*
 
>

templates being deleted after download

In writing some full buildout tests I'm running into some intermittent
issues.  If I register a template after zone is created, but before
secondary storage vm starts, the template is pretty much guaranteed to
be deleted.  If I watch for the SSVM to come up, and its agent to
start, then there's maybe a 50% chance that the template will be
deleted, even if it successfully downloads. The odds may go down if I
wait awhile after the SSVM comes up.

I noticed this ticket, but it talks about template from snapshot. I'm
not sure if it's the same, but the root cause may be. I notice there's
no fix release set. If it only ever happens when the zone is first
being created, then it's not such a big deal, but if it happens
whenever the SSVM is stopped or just started (or randomly as the issue
implies), then it should be a blocker. If any template I register will
get successfully downloaded and then deleted if my SSVM ever happens
to be down, that's not good.

https://issues.apache.org/jira/browse/CLOUDSTACK-3305

Re: Review Request 14522: [CLOUDSTACK-4771] Support Revert VM Disk from Snapshot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14522/#review27126
---



engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/DataMotionServiceImpl.java


Add context information to error message to help operators and users 
determine what couldn't be snapshotted.



engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/DataMotionServiceImpl.java


DRY out this code (e.g. extract to a private method).  This rules needs to 
be consolidated + add the context information mentioned above.



ui/scripts/storage.js


How will a corrupted snapshot failure be surfaced to the user?


- John Burwell


On Oct. 16, 2013, 10:50 a.m., Chris Suich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14522/
> ---
> 
> (Updated Oct. 16, 2013, 10:50 a.m.)
> 
> 
> Review request for cloudstack, Brian Federle, edison su, John Burwell, and 
> Mike Tutkowski.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> After the last batch of work to the revertSnapshot API, SnapshotServiceImpl 
> was not tied into the workflow to be used by storage providers. I have added 
> the logic in a similar fashion to takeSnapshot(), backupSnapshot() and 
> deleteSnapshot().
> 
> I have also added a 'Revert to Snapshot' action to the volume snapshots list 
> in the UI.
> 
> 
> Diffs
> -
> 
>   api/src/org/apache/cloudstack/api/ApiConstants.java 62eed09 
>   api/src/org/apache/cloudstack/api/response/SnapshotResponse.java e9cb109 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/PrimaryDataStoreDriver.java
>  b124d83 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/SnapshotInfo.java
>  8d6b760 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/StrategyPriority.java
>  81034b1 
>   
> engine/api/test/org/apache/cloudstack/engine/subsystem/api/storage/StrategyPriorityTest.java
>  3d75279 
>   
> engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/DataMotionServiceImpl.java
>  2d31320 
>   
> engine/storage/integration-test/test/org/apache/cloudstack/storage/test/FakePrimaryDataStoreDriver.java
>  810afd1 
>   
> engine/storage/integration-test/test/org/apache/cloudstack/storage/test/SnapshotTest.java
>  81f77d6 
>   
> engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/SnapshotObject.java
>  3f35e1d 
>   
> engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/XenserverSnapshotStrategy.java
>  6a874d6 
>   
> plugins/storage/volume/default/src/org/apache/cloudstack/storage/datastore/driver/CloudStackPrimaryDataStoreDriverImpl.java
>  3eaeb1f 
>   
> plugins/storage/volume/sample/src/org/apache/cloudstack/storage/datastore/driver/SamplePrimaryDataStoreDriverImpl.java
>  ece7b26 
>   
> plugins/storage/volume/solidfire/src/org/apache/cloudstack/storage/datastore/driver/SolidfirePrimaryDataStoreDriver.java
>  8046b6c 
>   server/src/com/cloud/api/ApiResponseHelper.java f4ca112 
>   server/src/com/cloud/storage/snapshot/SnapshotManagerImpl.java dade983 
>   ui/scripts/storage.js b16f4d4 
> 
> Diff: https://reviews.apache.org/r/14522/diff/
> 
> 
> Testing
> ---
> 
> I have tested all of this locally with a custom storage provider.
> 
> Unfortunately, I'm still in the middle of figuring out how to properly unit 
> test this type of code. If anyone has any recommendations, please let me know.
> 
> 
> Thanks,
> 
> Chris Suich
> 
>

Re: Cloudstack collab Hackathons

I'm not at all against it, I just haven't heard anyone give any reason
as to why.  Keep in mind that we'd essentially be asking anyone who
has written code for the agent to re-do their work (midonet, vxlan,
storage plugins).  My impression is that people just like the 'feel'
of it being in Python, and for good reason, but that ignores the fact
that 1) we've already done the work, and 2) features still require
java knowledge since the agent doesn't do anything the mgmt server
doesn't ask for.  Without anyone explaining why, it just sort of feels
like rearranging deck chairs on a boat with serious leaks going on
below deck. Worse, people brought their own chairs, and we threw them
overboard because they're now the wrong color.

On Wed, Oct 16, 2013 at 6:54 PM, Darren Shepherd
 wrote:
> +1 for kvm in python.  I'd like to take it a step further.  I'd like to 
> create a framework for adding compute and storage with python drivers. The 
> first implementation should just happen to be libvirt/kvm.  So yeah, I'll be 
> interested in this.  I'll be landing around 10am so don't know what time I'll 
> get to the hackathon.
>
> Darren
>
>> On Oct 14, 2013, at 9:29 AM, Marcus Sorensen  wrote:
>>
>>> On Mon, Oct 14, 2013 at 8:22 AM, Sebastien Goasguen  
>>> wrote:
>>>
 On Oct 11, 2013, at 4:43 AM, Hugo Trippaers  wrote:

 Hey Guys,

 The CloudStack collaboration conference is right around the corner. The 
 first day of the conference will be dedicated to workshops and a hackathon.

 I'm curious which developers are planning to attend the hackthon and what 
 subjects you are interested in. In Santa Clara we had a short list of some 
 subjects to discuss and some tables and discussions were already prepared 
 in advance. In the upcoming conference we can do the same, so if you have 
 a discussion or project idea that you want to work on at the hackthon let 
 us know with a reply in this thread.

 I'm also curious if there are any aspiring developers that would like to 
 have a sort of introduction into cloudstack development during the 
 hackathon.

 If you have any other ideas, just shout.

 Cheers,

 Hugo
>>>
>>> I am adding couple topics that I would like to see:
>>>
>>> -API interfaces (AWS refactor, GCE, OCCI&CIMI standard): Discuss the state 
>>> of our interfaces, plans for future, needs etc. AWS interface might need a 
>>> refactor, GCE is a new interface, OCCI is a standard and Isaac Chiang has 
>>> developed an interface. We are missing a CIMI interface.
>>>
>>> -DOCs: We are having lots of talks about docs, they have been split in a 
>>> separate repo, we need to discuss format, release life cycle, format etc.
>>>
>>> -KVM agent: There has been discussions/wishes to re-write the KVM agent in 
>>> something else than Java. Review architecture, define a plan, find 
>>> developers :)
>>
>> What? First I've heard of it!  I'm not sure how I feel about that. On
>> one side, it seems reasonable to do it in something like python and
>> remove the java dependency on the KVM host. On the other hand, there's
>> a lot of technical debt in what's been built already, while it would
>> probably be almost trivial to set up a basic working agent, it seems
>> like a significant amount of work to transfer all of the
>> functionality, special code that works around libvirt bugs, etc. Not
>> to mention agent plugins that have already had a significant amount of
>> effort put into them. On top of that, I'm not sure there's a whole lot
>> of value if the motive is to attract non-java guys or admins, when the
>> agent's features rely on what the mgmt server can do (new
>> features/capabilities would require java code anyway, or at the very
>> least coordinated effort between multiple devs who want the same
>> feature).
>>
>>>
>>> -Ecosystem: The are lots of tools in the cloud ecosystem, we should talk 
>>> about docker, ansible, cloud foundry/bosh…etc.and define a plan to have 
>>> great cloudstack support in all of those.
>>>
>>> -sebastien
>>>
>>>

Re: Review Request 14522: [CLOUDSTACK-4771] Support Revert VM Disk from Snapshot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14522/#review27127
---


Overall, this patch is very close to ready.  I really like the way the 
condition logic is being simplified out with stronger error handling and 
detection.  Namely, it needs to add more information to the error reporting, 
and ensure that it is surfaced to the end user.  My concerns regarding the 
hypervisor vs. storage strategy selection are larger architectural issues that 
need to be addressed separately.

- John Burwell


On Oct. 16, 2013, 10:50 a.m., Chris Suich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14522/
> ---
> 
> (Updated Oct. 16, 2013, 10:50 a.m.)
> 
> 
> Review request for cloudstack, Brian Federle, edison su, John Burwell, and 
> Mike Tutkowski.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> After the last batch of work to the revertSnapshot API, SnapshotServiceImpl 
> was not tied into the workflow to be used by storage providers. I have added 
> the logic in a similar fashion to takeSnapshot(), backupSnapshot() and 
> deleteSnapshot().
> 
> I have also added a 'Revert to Snapshot' action to the volume snapshots list 
> in the UI.
> 
> 
> Diffs
> -
> 
>   api/src/org/apache/cloudstack/api/ApiConstants.java 62eed09 
>   api/src/org/apache/cloudstack/api/response/SnapshotResponse.java e9cb109 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/PrimaryDataStoreDriver.java
>  b124d83 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/SnapshotInfo.java
>  8d6b760 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/StrategyPriority.java
>  81034b1 
>   
> engine/api/test/org/apache/cloudstack/engine/subsystem/api/storage/StrategyPriorityTest.java
>  3d75279 
>   
> engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/DataMotionServiceImpl.java
>  2d31320 
>   
> engine/storage/integration-test/test/org/apache/cloudstack/storage/test/FakePrimaryDataStoreDriver.java
>  810afd1 
>   
> engine/storage/integration-test/test/org/apache/cloudstack/storage/test/SnapshotTest.java
>  81f77d6 
>   
> engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/SnapshotObject.java
>  3f35e1d 
>   
> engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/XenserverSnapshotStrategy.java
>  6a874d6 
>   
> plugins/storage/volume/default/src/org/apache/cloudstack/storage/datastore/driver/CloudStackPrimaryDataStoreDriverImpl.java
>  3eaeb1f 
>   
> plugins/storage/volume/sample/src/org/apache/cloudstack/storage/datastore/driver/SamplePrimaryDataStoreDriverImpl.java
>  ece7b26 
>   
> plugins/storage/volume/solidfire/src/org/apache/cloudstack/storage/datastore/driver/SolidfirePrimaryDataStoreDriver.java
>  8046b6c 
>   server/src/com/cloud/api/ApiResponseHelper.java f4ca112 
>   server/src/com/cloud/storage/snapshot/SnapshotManagerImpl.java dade983 
>   ui/scripts/storage.js b16f4d4 
> 
> Diff: https://reviews.apache.org/r/14522/diff/
> 
> 
> Testing
> ---
> 
> I have tested all of this locally with a custom storage provider.
> 
> Unfortunately, I'm still in the middle of figuring out how to properly unit 
> test this type of code. If anyone has any recommendations, please let me know.
> 
> 
> Thanks,
> 
> Chris Suich
> 
>

Re: Review Request 14522: [CLOUDSTACK-4771] Support Revert VM Disk from Snapshot



> On Oct. 17, 2013, 3:18 p.m., John Burwell wrote:
> > engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/DataMotionServiceImpl.java,
> >  line 62
> > 
> >
> > Add context information to error message to help operators and users 
> > determine what couldn't be snapshotted.

Fair enough - I just copied these existing error messages. What kind of context 
would you like to see in this error message? The src & dest datastores, files, 
etc.? I agree that there needs to be context, but this would potentially be a 
user facing error, so I also don't want to overload them with context that 
means nothing to them.


> On Oct. 17, 2013, 3:18 p.m., John Burwell wrote:
> > engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/DataMotionServiceImpl.java,
> >  line 79
> > 
> >
> > DRY out this code (e.g. extract to a private method).  This rules needs 
> > to be consolidated + add the context information mentioned above.

The two invocations of pickStrategy() are actually calling an overloaded 
function. I'm not sure how we could DRY this out since they method 
calls/parameter lists are actually different.


> On Oct. 17, 2013, 3:18 p.m., John Burwell wrote:
> > ui/scripts/storage.js, line 1963
> > 
> >
> > How will a corrupted snapshot failure be surfaced to the user?

It depends on what you mean by corrupted. If we can no longer find the snapshot 
we expect from the DB, then the API will throw an exception, per your other 
review, and the list of snapshots will never populate. If the file exists but 
is corrupted, then that wouldn't be discovered until either the attempt to 
revert the snapshot or after the revert operation and the user attempts to use 
the volume.


- Chris


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14522/#review27126
---


On Oct. 16, 2013, 2:50 p.m., Chris Suich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14522/
> ---
> 
> (Updated Oct. 16, 2013, 2:50 p.m.)
> 
> 
> Review request for cloudstack, Brian Federle, edison su, John Burwell, and 
> Mike Tutkowski.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> After the last batch of work to the revertSnapshot API, SnapshotServiceImpl 
> was not tied into the workflow to be used by storage providers. I have added 
> the logic in a similar fashion to takeSnapshot(), backupSnapshot() and 
> deleteSnapshot().
> 
> I have also added a 'Revert to Snapshot' action to the volume snapshots list 
> in the UI.
> 
> 
> Diffs
> -
> 
>   api/src/org/apache/cloudstack/api/ApiConstants.java 62eed09 
>   api/src/org/apache/cloudstack/api/response/SnapshotResponse.java e9cb109 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/PrimaryDataStoreDriver.java
>  b124d83 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/SnapshotInfo.java
>  8d6b760 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/StrategyPriority.java
>  81034b1 
>   
> engine/api/test/org/apache/cloudstack/engine/subsystem/api/storage/StrategyPriorityTest.java
>  3d75279 
>   
> engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/DataMotionServiceImpl.java
>  2d31320 
>   
> engine/storage/integration-test/test/org/apache/cloudstack/storage/test/FakePrimaryDataStoreDriver.java
>  810afd1 
>   
> engine/storage/integration-test/test/org/apache/cloudstack/storage/test/SnapshotTest.java
>  81f77d6 
>   
> engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/SnapshotObject.java
>  3f35e1d 
>   
> engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/XenserverSnapshotStrategy.java
>  6a874d6 
>   
> plugins/storage/volume/default/src/org/apache/cloudstack/storage/datastore/driver/CloudStackPrimaryDataStoreDriverImpl.java
>  3eaeb1f 
>   
> plugins/storage/volume/sample/src/org/apache/cloudstack/storage/datastore/driver/SamplePrimaryDataStoreDriverImpl.java
>  ece7b26 
>   
> plugins/storage/volume/solidfire/src/org/apache/cloudstack/storage/datastore/driver/SolidfirePrimaryDataStoreDriver.java
>  8046b6c 
>   server/src/com/cloud/api/ApiResponseHelper.java f4ca112 
>   server/src/com/cloud/storage/snapshot/SnapshotManagerImpl.java dade983 
>   ui/scripts/storage.js b16f4d4 
> 
> Diff: https://reviews.apache.org/r/14522/diff/
> 
> 
> Testing
> ---
> 
> I have tested all of this locally with a custom storage provider.
> 
> Unfortunately, I'm still in the middle of figuring out how

Re: Cloudstack collab Hackathons

Marcus,

Will you be at CCC?  I think it's immensely important and can give plenty of 
good reasons, but too lazy to type at the moment.  And I need to do a bit more 
analysis on the scope.

Darren

> On Oct 17, 2013, at 8:19 AM, Marcus Sorensen  wrote:
> 
> I'm not at all against it, I just haven't heard anyone give any reason
> as to why.  Keep in mind that we'd essentially be asking anyone who
> has written code for the agent to re-do their work (midonet, vxlan,
> storage plugins).  My impression is that people just like the 'feel'
> of it being in Python, and for good reason, but that ignores the fact
> that 1) we've already done the work, and 2) features still require
> java knowledge since the agent doesn't do anything the mgmt server
> doesn't ask for.  Without anyone explaining why, it just sort of feels
> like rearranging deck chairs on a boat with serious leaks going on
> below deck. Worse, people brought their own chairs, and we threw them
> overboard because they're now the wrong color.
> 
> On Wed, Oct 16, 2013 at 6:54 PM, Darren Shepherd
>  wrote:
>> +1 for kvm in python.  I'd like to take it a step further.  I'd like to 
>> create a framework for adding compute and storage with python drivers. The 
>> first implementation should just happen to be libvirt/kvm.  So yeah, I'll be 
>> interested in this.  I'll be landing around 10am so don't know what time 
>> I'll get to the hackathon.
>> 
>> Darren
>> 
 On Oct 14, 2013, at 9:29 AM, Marcus Sorensen  wrote:
 
> On Mon, Oct 14, 2013 at 8:22 AM, Sebastien Goasguen  
> wrote:
> 
> On Oct 11, 2013, at 4:43 AM, Hugo Trippaers  wrote:
> 
> Hey Guys,
> 
> The CloudStack collaboration conference is right around the corner. The 
> first day of the conference will be dedicated to workshops and a 
> hackathon.
> 
> I'm curious which developers are planning to attend the hackthon and what 
> subjects you are interested in. In Santa Clara we had a short list of 
> some subjects to discuss and some tables and discussions were already 
> prepared in advance. In the upcoming conference we can do the same, so if 
> you have a discussion or project idea that you want to work on at the 
> hackthon let us know with a reply in this thread.
> 
> I'm also curious if there are any aspiring developers that would like to 
> have a sort of introduction into cloudstack development during the 
> hackathon.
> 
> If you have any other ideas, just shout.
> 
> Cheers,
> 
> Hugo
 
 I am adding couple topics that I would like to see:
 
 -API interfaces (AWS refactor, GCE, OCCI&CIMI standard): Discuss the state 
 of our interfaces, plans for future, needs etc. AWS interface might need a 
 refactor, GCE is a new interface, OCCI is a standard and Isaac Chiang has 
 developed an interface. We are missing a CIMI interface.
 
 -DOCs: We are having lots of talks about docs, they have been split in a 
 separate repo, we need to discuss format, release life cycle, format etc.
 
 -KVM agent: There has been discussions/wishes to re-write the KVM agent in 
 something else than Java. Review architecture, define a plan, find 
 developers :)
>>> 
>>> What? First I've heard of it!  I'm not sure how I feel about that. On
>>> one side, it seems reasonable to do it in something like python and
>>> remove the java dependency on the KVM host. On the other hand, there's
>>> a lot of technical debt in what's been built already, while it would
>>> probably be almost trivial to set up a basic working agent, it seems
>>> like a significant amount of work to transfer all of the
>>> functionality, special code that works around libvirt bugs, etc. Not
>>> to mention agent plugins that have already had a significant amount of
>>> effort put into them. On top of that, I'm not sure there's a whole lot
>>> of value if the motive is to attract non-java guys or admins, when the
>>> agent's features rely on what the mgmt server can do (new
>>> features/capabilities would require java code anyway, or at the very
>>> least coordinated effort between multiple devs who want the same
>>> feature).
>>> 
 
 -Ecosystem: The are lots of tools in the cloud ecosystem, we should talk 
 about docker, ansible, cloud foundry/bosh…etc.and define a plan to have 
 great cloudstack support in all of those.
 
 -sebastien

Re: [JENKINS] systemvm build jobs

Heya,

The systemvm builds are working again on master. The created templates can be 
downloaded from:

http://jenkins.buildacloud.org/job/build-systemvm-master/
http://jenkins.buildacloud.org/job/build-systemvm64-master/


Cheers,

Hugo

On 17 okt. 2013, at 12:30, Hugo Trippaers  wrote:

> Heya,
> 
> The systemvm jobs on master are failing because the netinst iso for 7.0.0 is 
> no longer available from the debian mirrors. I've added the images to the 
> userContent on the master so we can use the copy-to-slave plugin to get them. 
> I'm updating and testing the jobs now.
> 
> Cheers,
> 
> Hugo

Re: Cloudstack collab Hackathons

I'd like to throw in the topic of API clients.  So specifically what's the 
state of clients in python, ruby, java, and go.  With the api discovery feature 
it should be simple to write dynamic clients that are always up to date with 
the api.

Darren

> On Oct 14, 2013, at 7:22 AM, Sebastien Goasguen  wrote:
> 
> 
>> On Oct 11, 2013, at 4:43 AM, Hugo Trippaers  wrote:
>> 
>> Hey Guys,
>> 
>> The CloudStack collaboration conference is right around the corner. The 
>> first day of the conference will be dedicated to workshops and a hackathon.
>> 
>> I'm curious which developers are planning to attend the hackthon and what 
>> subjects you are interested in. In Santa Clara we had a short list of some 
>> subjects to discuss and some tables and discussions were already prepared in 
>> advance. In the upcoming conference we can do the same, so if you have a 
>> discussion or project idea that you want to work on at the hackthon let us 
>> know with a reply in this thread.
>> 
>> I'm also curious if there are any aspiring developers that would like to 
>> have a sort of introduction into cloudstack development during the hackathon.
>> 
>> If you have any other ideas, just shout.
>> 
>> Cheers,
>> 
>> Hugo
> 
> I am adding couple topics that I would like to see:
> 
> -API interfaces (AWS refactor, GCE, OCCI&CIMI standard): Discuss the state of 
> our interfaces, plans for future, needs etc. AWS interface might need a 
> refactor, GCE is a new interface, OCCI is a standard and Isaac Chiang has 
> developed an interface. We are missing a CIMI interface.
> 
> -DOCs: We are having lots of talks about docs, they have been split in a 
> separate repo, we need to discuss format, release life cycle, format etc.
> 
> -KVM agent: There has been discussions/wishes to re-write the KVM agent in 
> something else than Java. Review architecture, define a plan, find developers 
> :)
> 
> -Ecosystem: The are lots of tools in the cloud ecosystem, we should talk 
> about docker, ansible, cloud foundry/bosh…etc.and define a plan to have great 
> cloudstack support in all of those.
> 
> -sebastien
> 
>

Re: Cloudstack collab Hackathons

2013-10-17 Thread Koushik Das

Agree with Marcus here.
If we are planning to rewrite, I think that there should be some discussion 
around the possibility of making it a direct agent like XS/Vmware. If this is 
possible then installing the KVM agent on individual hosts will be eliminated.


On 17-Oct-2013, at 8:49 PM, Marcus Sorensen  wrote:

> I'm not at all against it, I just haven't heard anyone give any reason
> as to why.  Keep in mind that we'd essentially be asking anyone who
> has written code for the agent to re-do their work (midonet, vxlan,
> storage plugins).  My impression is that people just like the 'feel'
> of it being in Python, and for good reason, but that ignores the fact
> that 1) we've already done the work, and 2) features still require
> java knowledge since the agent doesn't do anything the mgmt server
> doesn't ask for.  Without anyone explaining why, it just sort of feels
> like rearranging deck chairs on a boat with serious leaks going on
> below deck. Worse, people brought their own chairs, and we threw them
> overboard because they're now the wrong color.
> 
> On Wed, Oct 16, 2013 at 6:54 PM, Darren Shepherd
>  wrote:
>> +1 for kvm in python.  I'd like to take it a step further.  I'd like to 
>> create a framework for adding compute and storage with python drivers. The 
>> first implementation should just happen to be libvirt/kvm.  So yeah, I'll be 
>> interested in this.  I'll be landing around 10am so don't know what time 
>> I'll get to the hackathon.
>> 
>> Darren
>> 
>>> On Oct 14, 2013, at 9:29 AM, Marcus Sorensen  wrote:
>>> 
 On Mon, Oct 14, 2013 at 8:22 AM, Sebastien Goasguen  
 wrote:
 
> On Oct 11, 2013, at 4:43 AM, Hugo Trippaers  wrote:
> 
> Hey Guys,
> 
> The CloudStack collaboration conference is right around the corner. The 
> first day of the conference will be dedicated to workshops and a 
> hackathon.
> 
> I'm curious which developers are planning to attend the hackthon and what 
> subjects you are interested in. In Santa Clara we had a short list of 
> some subjects to discuss and some tables and discussions were already 
> prepared in advance. In the upcoming conference we can do the same, so if 
> you have a discussion or project idea that you want to work on at the 
> hackthon let us know with a reply in this thread.
> 
> I'm also curious if there are any aspiring developers that would like to 
> have a sort of introduction into cloudstack development during the 
> hackathon.
> 
> If you have any other ideas, just shout.
> 
> Cheers,
> 
> Hugo
 
 I am adding couple topics that I would like to see:
 
 -API interfaces (AWS refactor, GCE, OCCI&CIMI standard): Discuss the state 
 of our interfaces, plans for future, needs etc. AWS interface might need a 
 refactor, GCE is a new interface, OCCI is a standard and Isaac Chiang has 
 developed an interface. We are missing a CIMI interface.
 
 -DOCs: We are having lots of talks about docs, they have been split in a 
 separate repo, we need to discuss format, release life cycle, format etc.
 
 -KVM agent: There has been discussions/wishes to re-write the KVM agent in 
 something else than Java. Review architecture, define a plan, find 
 developers :)
>>> 
>>> What? First I've heard of it!  I'm not sure how I feel about that. On
>>> one side, it seems reasonable to do it in something like python and
>>> remove the java dependency on the KVM host. On the other hand, there's
>>> a lot of technical debt in what's been built already, while it would
>>> probably be almost trivial to set up a basic working agent, it seems
>>> like a significant amount of work to transfer all of the
>>> functionality, special code that works around libvirt bugs, etc. Not
>>> to mention agent plugins that have already had a significant amount of
>>> effort put into them. On top of that, I'm not sure there's a whole lot
>>> of value if the motive is to attract non-java guys or admins, when the
>>> agent's features rely on what the mgmt server can do (new
>>> features/capabilities would require java code anyway, or at the very
>>> least coordinated effort between multiple devs who want the same
>>> feature).
>>> 
 
 -Ecosystem: The are lots of tools in the cloud ecosystem, we should talk 
 about docker, ansible, cloud foundry/bosh…etc.and define a plan to have 
 great cloudstack support in all of those.
 
 -sebastien

Re: Cloudstack collab Hackathons

I won't be this time.

On Thu, Oct 17, 2013 at 9:35 AM, Darren Shepherd
 wrote:
> Marcus,
>
> Will you be at CCC?  I think it's immensely important and can give plenty of 
> good reasons, but too lazy to type at the moment.  And I need to do a bit 
> more analysis on the scope.
>
> Darren
>
>> On Oct 17, 2013, at 8:19 AM, Marcus Sorensen  wrote:
>>
>> I'm not at all against it, I just haven't heard anyone give any reason
>> as to why.  Keep in mind that we'd essentially be asking anyone who
>> has written code for the agent to re-do their work (midonet, vxlan,
>> storage plugins).  My impression is that people just like the 'feel'
>> of it being in Python, and for good reason, but that ignores the fact
>> that 1) we've already done the work, and 2) features still require
>> java knowledge since the agent doesn't do anything the mgmt server
>> doesn't ask for.  Without anyone explaining why, it just sort of feels
>> like rearranging deck chairs on a boat with serious leaks going on
>> below deck. Worse, people brought their own chairs, and we threw them
>> overboard because they're now the wrong color.
>>
>> On Wed, Oct 16, 2013 at 6:54 PM, Darren Shepherd
>>  wrote:
>>> +1 for kvm in python.  I'd like to take it a step further.  I'd like to 
>>> create a framework for adding compute and storage with python drivers. The 
>>> first implementation should just happen to be libvirt/kvm.  So yeah, I'll 
>>> be interested in this.  I'll be landing around 10am so don't know what time 
>>> I'll get to the hackathon.
>>>
>>> Darren
>>>
> On Oct 14, 2013, at 9:29 AM, Marcus Sorensen  wrote:
>
>> On Mon, Oct 14, 2013 at 8:22 AM, Sebastien Goasguen  
>> wrote:
>>
>> On Oct 11, 2013, at 4:43 AM, Hugo Trippaers  wrote:
>>
>> Hey Guys,
>>
>> The CloudStack collaboration conference is right around the corner. The 
>> first day of the conference will be dedicated to workshops and a 
>> hackathon.
>>
>> I'm curious which developers are planning to attend the hackthon and 
>> what subjects you are interested in. In Santa Clara we had a short list 
>> of some subjects to discuss and some tables and discussions were already 
>> prepared in advance. In the upcoming conference we can do the same, so 
>> if you have a discussion or project idea that you want to work on at the 
>> hackthon let us know with a reply in this thread.
>>
>> I'm also curious if there are any aspiring developers that would like to 
>> have a sort of introduction into cloudstack development during the 
>> hackathon.
>>
>> If you have any other ideas, just shout.
>>
>> Cheers,
>>
>> Hugo
>
> I am adding couple topics that I would like to see:
>
> -API interfaces (AWS refactor, GCE, OCCI&CIMI standard): Discuss the 
> state of our interfaces, plans for future, needs etc. AWS interface might 
> need a refactor, GCE is a new interface, OCCI is a standard and Isaac 
> Chiang has developed an interface. We are missing a CIMI interface.
>
> -DOCs: We are having lots of talks about docs, they have been split in a 
> separate repo, we need to discuss format, release life cycle, format etc.
>
> -KVM agent: There has been discussions/wishes to re-write the KVM agent 
> in something else than Java. Review architecture, define a plan, find 
> developers :)

 What? First I've heard of it!  I'm not sure how I feel about that. On
 one side, it seems reasonable to do it in something like python and
 remove the java dependency on the KVM host. On the other hand, there's
 a lot of technical debt in what's been built already, while it would
 probably be almost trivial to set up a basic working agent, it seems
 like a significant amount of work to transfer all of the
 functionality, special code that works around libvirt bugs, etc. Not
 to mention agent plugins that have already had a significant amount of
 effort put into them. On top of that, I'm not sure there's a whole lot
 of value if the motive is to attract non-java guys or admins, when the
 agent's features rely on what the mgmt server can do (new
 features/capabilities would require java code anyway, or at the very
 least coordinated effort between multiple devs who want the same
 feature).

>
> -Ecosystem: The are lots of tools in the cloud ecosystem, we should talk 
> about docker, ansible, cloud foundry/bosh…etc.and define a plan to have 
> great cloudstack support in all of those.
>
> -sebastien
>
>

Re: templates being deleted after download

I haven't looked at this at all, but could it be related to an issue I brought 
up before that commands go to the "LocalEndpoint" if the SSVM is unavailable.  
I've seen this too where before the ssvm starts the management stack tries to 
do the download instead of waiting for the ssvm to become available and sending 
the request there.  

Darren

> On Oct 17, 2013, at 8:07 AM, Marcus Sorensen  wrote:
> 
> In writing some full buildout tests I'm running into some intermittent
> issues.  If I register a template after zone is created, but before
> secondary storage vm starts, the template is pretty much guaranteed to
> be deleted.  If I watch for the SSVM to come up, and its agent to
> start, then there's maybe a 50% chance that the template will be
> deleted, even if it successfully downloads. The odds may go down if I
> wait awhile after the SSVM comes up.
> 
> I noticed this ticket, but it talks about template from snapshot. I'm
> not sure if it's the same, but the root cause may be. I notice there's
> no fix release set. If it only ever happens when the zone is first
> being created, then it's not such a big deal, but if it happens
> whenever the SSVM is stopped or just started (or randomly as the issue
> implies), then it should be a blocker. If any template I register will
> get successfully downloaded and then deleted if my SSVM ever happens
> to be down, that's not good.
> 
> https://issues.apache.org/jira/browse/CLOUDSTACK-3305

Re: Cloudstack collab Hackathons

2013-10-17 Thread sebgoa


On Oct 17, 2013, at 5:19 PM, Marcus Sorensen  wrote:

> I'm not at all against it, I just haven't heard anyone give any reason
> as to why.  Keep in mind that we'd essentially be asking anyone who
> has written code for the agent to re-do their work (midonet, vxlan,
> storage plugins).  My impression is that people just like the 'feel'
> of it being in Python, and for good reason, but that ignores the fact
> that 1) we've already done the work, and 2) features still require
> java knowledge since the agent doesn't do anything the mgmt server
> doesn't ask for.  Without anyone explaining why, it just sort of feels
> like rearranging deck chairs on a boat with serious leaks going on
> below deck. Worse, people brought their own chairs, and we threw them
> overboard because they're now the wrong color.
> 

To me it's merely an issue of being able to work on it, python or another 
scripting language would make it easy, it would also open the door for sys 
admins to contribute.

That said it's probably not a valid reason in itself, figuring out if there are 
valid reasons to re-write this would be worthwhile IMHO. We are not saying that 
we are doing it, just raising the issue.

-sebastien

> On Wed, Oct 16, 2013 at 6:54 PM, Darren Shepherd
>  wrote:
>> +1 for kvm in python.  I'd like to take it a step further.  I'd like to 
>> create a framework for adding compute and storage with python drivers. The 
>> first implementation should just happen to be libvirt/kvm.  So yeah, I'll be 
>> interested in this.  I'll be landing around 10am so don't know what time 
>> I'll get to the hackathon.
>> 
>> Darren
>> 
>>> On Oct 14, 2013, at 9:29 AM, Marcus Sorensen  wrote:
>>> 
 On Mon, Oct 14, 2013 at 8:22 AM, Sebastien Goasguen  
 wrote:
 
> On Oct 11, 2013, at 4:43 AM, Hugo Trippaers  wrote:
> 
> Hey Guys,
> 
> The CloudStack collaboration conference is right around the corner. The 
> first day of the conference will be dedicated to workshops and a 
> hackathon.
> 
> I'm curious which developers are planning to attend the hackthon and what 
> subjects you are interested in. In Santa Clara we had a short list of 
> some subjects to discuss and some tables and discussions were already 
> prepared in advance. In the upcoming conference we can do the same, so if 
> you have a discussion or project idea that you want to work on at the 
> hackthon let us know with a reply in this thread.
> 
> I'm also curious if there are any aspiring developers that would like to 
> have a sort of introduction into cloudstack development during the 
> hackathon.
> 
> If you have any other ideas, just shout.
> 
> Cheers,
> 
> Hugo
 
 I am adding couple topics that I would like to see:
 
 -API interfaces (AWS refactor, GCE, OCCI&CIMI standard): Discuss the state 
 of our interfaces, plans for future, needs etc. AWS interface might need a 
 refactor, GCE is a new interface, OCCI is a standard and Isaac Chiang has 
 developed an interface. We are missing a CIMI interface.
 
 -DOCs: We are having lots of talks about docs, they have been split in a 
 separate repo, we need to discuss format, release life cycle, format etc.
 
 -KVM agent: There has been discussions/wishes to re-write the KVM agent in 
 something else than Java. Review architecture, define a plan, find 
 developers :)
>>> 
>>> What? First I've heard of it!  I'm not sure how I feel about that. On
>>> one side, it seems reasonable to do it in something like python and
>>> remove the java dependency on the KVM host. On the other hand, there's
>>> a lot of technical debt in what's been built already, while it would
>>> probably be almost trivial to set up a basic working agent, it seems
>>> like a significant amount of work to transfer all of the
>>> functionality, special code that works around libvirt bugs, etc. Not
>>> to mention agent plugins that have already had a significant amount of
>>> effort put into them. On top of that, I'm not sure there's a whole lot
>>> of value if the motive is to attract non-java guys or admins, when the
>>> agent's features rely on what the mgmt server can do (new
>>> features/capabilities would require java code anyway, or at the very
>>> least coordinated effort between multiple devs who want the same
>>> feature).
>>> 
 
 -Ecosystem: The are lots of tools in the cloud ecosystem, we should talk 
 about docker, ansible, cloud foundry/bosh…etc.and define a plan to have 
 great cloudstack support in all of those.
 
 -sebastien

Re: Cloudstack collab Hackathons

Marcus,

Okay, well I'll give a fully articulated argument for this eventually.  
Regardless, no reason to discourage a hackathon on the topic.  Might as well 
talk and hack on it a bit and then we will know if it's a terrible idea.

Darren

> On Oct 17, 2013, at 8:39 AM, Marcus Sorensen  wrote:
> 
> I won't be this time.
> 
> On Thu, Oct 17, 2013 at 9:35 AM, Darren Shepherd
>  wrote:
>> Marcus,
>> 
>> Will you be at CCC?  I think it's immensely important and can give plenty of 
>> good reasons, but too lazy to type at the moment.  And I need to do a bit 
>> more analysis on the scope.
>> 
>> Darren
>> 
>>> On Oct 17, 2013, at 8:19 AM, Marcus Sorensen  wrote:
>>> 
>>> I'm not at all against it, I just haven't heard anyone give any reason
>>> as to why.  Keep in mind that we'd essentially be asking anyone who
>>> has written code for the agent to re-do their work (midonet, vxlan,
>>> storage plugins).  My impression is that people just like the 'feel'
>>> of it being in Python, and for good reason, but that ignores the fact
>>> that 1) we've already done the work, and 2) features still require
>>> java knowledge since the agent doesn't do anything the mgmt server
>>> doesn't ask for.  Without anyone explaining why, it just sort of feels
>>> like rearranging deck chairs on a boat with serious leaks going on
>>> below deck. Worse, people brought their own chairs, and we threw them
>>> overboard because they're now the wrong color.
>>> 
>>> On Wed, Oct 16, 2013 at 6:54 PM, Darren Shepherd
>>>  wrote:
 +1 for kvm in python.  I'd like to take it a step further.  I'd like to 
 create a framework for adding compute and storage with python drivers. The 
 first implementation should just happen to be libvirt/kvm.  So yeah, I'll 
 be interested in this.  I'll be landing around 10am so don't know what 
 time I'll get to the hackathon.
 
 Darren
 
>>> On Oct 14, 2013, at 9:29 AM, Marcus Sorensen  
>>> wrote:
>>> 
>>> On Mon, Oct 14, 2013 at 8:22 AM, Sebastien Goasguen  
>>> wrote:
>>> 
>>> On Oct 11, 2013, at 4:43 AM, Hugo Trippaers  wrote:
>>> 
>>> Hey Guys,
>>> 
>>> The CloudStack collaboration conference is right around the corner. The 
>>> first day of the conference will be dedicated to workshops and a 
>>> hackathon.
>>> 
>>> I'm curious which developers are planning to attend the hackthon and 
>>> what subjects you are interested in. In Santa Clara we had a short list 
>>> of some subjects to discuss and some tables and discussions were 
>>> already prepared in advance. In the upcoming conference we can do the 
>>> same, so if you have a discussion or project idea that you want to work 
>>> on at the hackthon let us know with a reply in this thread.
>>> 
>>> I'm also curious if there are any aspiring developers that would like 
>>> to have a sort of introduction into cloudstack development during the 
>>> hackathon.
>>> 
>>> If you have any other ideas, just shout.
>>> 
>>> Cheers,
>>> 
>>> Hugo
>> 
>> I am adding couple topics that I would like to see:
>> 
>> -API interfaces (AWS refactor, GCE, OCCI&CIMI standard): Discuss the 
>> state of our interfaces, plans for future, needs etc. AWS interface 
>> might need a refactor, GCE is a new interface, OCCI is a standard and 
>> Isaac Chiang has developed an interface. We are missing a CIMI interface.
>> 
>> -DOCs: We are having lots of talks about docs, they have been split in a 
>> separate repo, we need to discuss format, release life cycle, format etc.
>> 
>> -KVM agent: There has been discussions/wishes to re-write the KVM agent 
>> in something else than Java. Review architecture, define a plan, find 
>> developers :)
> 
> What? First I've heard of it!  I'm not sure how I feel about that. On
> one side, it seems reasonable to do it in something like python and
> remove the java dependency on the KVM host. On the other hand, there's
> a lot of technical debt in what's been built already, while it would
> probably be almost trivial to set up a basic working agent, it seems
> like a significant amount of work to transfer all of the
> functionality, special code that works around libvirt bugs, etc. Not
> to mention agent plugins that have already had a significant amount of
> effort put into them. On top of that, I'm not sure there's a whole lot
> of value if the motive is to attract non-java guys or admins, when the
> agent's features rely on what the mgmt server can do (new
> features/capabilities would require java code anyway, or at the very
> least coordinated effort between multiple devs who want the same
> feature).
> 
>> 
>> -Ecosystem: The are lots of tools in the cloud ecosystem, we should talk 
>> about docker, ansible, cloud foundry/bosh…etc.and define a plan to have 
>

RE: Cloudstack collab Hackathons

2013-10-17 Thread Santhosh Edukulla

Marcus,

1/ For me, its not about the debate of Java VS python, or supporting anything 
to re-write existing already. Its an effort, learning,idea and thought of 
contribution being new.  Having said that, we worked earlier on  building an 
agent  prototype where we had to deploy agents and push policies, to multiple 
agents residing on multiple machines. While working on this, we chose python 
and it went smooth. I agree to your point though, that we should not redo for 
the sake of doing some thing. 

Thanks!
Santhosh

From: Marcus Sorensen [shadow...@gmail.com]
Sent: Thursday, October 17, 2013 11:19 AM
To: dev@cloudstack.apache.org
Subject: Re: Cloudstack collab Hackathons

I'm not at all against it, I just haven't heard anyone give any reason
as to why.  Keep in mind that we'd essentially be asking anyone who
has written code for the agent to re-do their work (midonet, vxlan,
storage plugins).  My impression is that people just like the 'feel'
of it being in Python, and for good reason, but that ignores the fact
that 1) we've already done the work, and 2) features still require
java knowledge since the agent doesn't do anything the mgmt server
doesn't ask for.  Without anyone explaining why, it just sort of feels
like rearranging deck chairs on a boat with serious leaks going on
below deck. Worse, people brought their own chairs, and we threw them
overboard because they're now the wrong color.

On Wed, Oct 16, 2013 at 6:54 PM, Darren Shepherd
 wrote:
> +1 for kvm in python.  I'd like to take it a step further.  I'd like to 
> create a framework for adding compute and storage with python drivers. The 
> first implementation should just happen to be libvirt/kvm.  So yeah, I'll be 
> interested in this.  I'll be landing around 10am so don't know what time I'll 
> get to the hackathon.
>
> Darren
>
>> On Oct 14, 2013, at 9:29 AM, Marcus Sorensen  wrote:
>>
>>> On Mon, Oct 14, 2013 at 8:22 AM, Sebastien Goasguen  
>>> wrote:
>>>
 On Oct 11, 2013, at 4:43 AM, Hugo Trippaers  wrote:

 Hey Guys,

 The CloudStack collaboration conference is right around the corner. The 
 first day of the conference will be dedicated to workshops and a hackathon.

 I'm curious which developers are planning to attend the hackthon and what 
 subjects you are interested in. In Santa Clara we had a short list of some 
 subjects to discuss and some tables and discussions were already prepared 
 in advance. In the upcoming conference we can do the same, so if you have 
 a discussion or project idea that you want to work on at the hackthon let 
 us know with a reply in this thread.

 I'm also curious if there are any aspiring developers that would like to 
 have a sort of introduction into cloudstack development during the 
 hackathon.

 If you have any other ideas, just shout.

 Cheers,

 Hugo
>>>
>>> I am adding couple topics that I would like to see:
>>>
>>> -API interfaces (AWS refactor, GCE, OCCI&CIMI standard): Discuss the state 
>>> of our interfaces, plans for future, needs etc. AWS interface might need a 
>>> refactor, GCE is a new interface, OCCI is a standard and Isaac Chiang has 
>>> developed an interface. We are missing a CIMI interface.
>>>
>>> -DOCs: We are having lots of talks about docs, they have been split in a 
>>> separate repo, we need to discuss format, release life cycle, format etc.
>>>
>>> -KVM agent: There has been discussions/wishes to re-write the KVM agent in 
>>> something else than Java. Review architecture, define a plan, find 
>>> developers :)
>>
>> What? First I've heard of it!  I'm not sure how I feel about that. On
>> one side, it seems reasonable to do it in something like python and
>> remove the java dependency on the KVM host. On the other hand, there's
>> a lot of technical debt in what's been built already, while it would
>> probably be almost trivial to set up a basic working agent, it seems
>> like a significant amount of work to transfer all of the
>> functionality, special code that works around libvirt bugs, etc. Not
>> to mention agent plugins that have already had a significant amount of
>> effort put into them. On top of that, I'm not sure there's a whole lot
>> of value if the motive is to attract non-java guys or admins, when the
>> agent's features rely on what the mgmt server can do (new
>> features/capabilities would require java code anyway, or at the very
>> least coordinated effort between multiple devs who want the same
>> feature).
>>
>>>
>>> -Ecosystem: The are lots of tools in the cloud ecosystem, we should talk 
>>> about docker, ansible, cloud foundry/bosh…etc.and define a plan to have 
>>> great cloudstack support in all of those.
>>>
>>> -sebastien
>>>
>>>

Re: Possible bug in DeploymentPlanner?

2013-10-17 Thread Koushik Das

Prachi,

I am planning to add the following code in findSuitablePoolsForVolumes() 
method. If the VM has a local disk and the cluster doesn't have a local pool 
then the cluster will be added to avoid set. Let me know if you see any issues 
with this.

if (useLocalStorage) {
List allPoolsInCluster = 
_storagePoolDao.findPoolsByTags(plan.getDataCenterId(), plan.getPodId(), 
plan.getClusterId(), null);
boolean localStoragePoolExists = false;
for (StoragePoolVO pool : allPoolsInCluster) {
if (pool.isLocal()) {
localStoragePoolExists = true;
break;
}
}
if (!localStoragePoolExists) {
avoid.addCluster(plan.getClusterId());
}
}

-Koushik

On 17-Oct-2013, at 3:21 AM, Prachi Damle 
mailto:prachi.da...@citrix.com>> wrote:



-Original Message-
From: Koushik Das [mailto:koushik@citrix.com]
Sent: Wednesday, October 16, 2013 5:21 AM
To: mailto:dev@cloudstack.apache.org>>
Subject: Re: Possible bug in DeploymentPlanner?


On 16-Oct-2013, at 3:12 AM, Prachi Damle 
mailto:prachi.da...@citrix.com>> wrote:


-Original Message-
From: Koushik Das [mailto:koushik@citrix.com]
Sent: Tuesday, October 15, 2013 11:43 AM
To: mailto:dev@cloudstack.apache.org>>
Subject: Re: Possible bug in DeploymentPlanner?


Thanks for the explanation Prachi.

Here is one issue I see with what you explained below. If an allocator is not 
suppose to handle a particular scenario how should it deal with the avoid set. 
Should it put all available pools in the avoid set. In that case subsequent 
allocators won't get a chance to do anything as all pools are in avoid set. If 
more than one allocator can operate on a particular pool then the situation 
becomes tricky.

To give an example, if there is a request for volume on shared storage and 
first allocator to get called is the local allocator. Since it is not going to 
handle this case it will return empty/null list. Now should it put all the 
other shared pools in avoid set or only the local pools in avoid set? Similarly 
if the request is for a local volume and the request first goes to zone or 
cluster-wide storage allocator what should be there in the avoid set?


[Prachi] I see the bug you are thinking about - deploymentPlanningManager :: 
canAvoidCluster  is only checking the shared storages and thus does not put the 
cluster in avoid set in case the local storage is not a match for the host.

I think the storage allocators should put the resources not considered, in 
avoid set, that are within their scope. Also they should do so only when they 
are supposed to handle the request. Thus local allocator sees only local pools, 
so it should place them into avoid set if not considered and only when this 
allocator is supposed to handle the scenario. While the zone-wide/cluster-wide 
should set those within their scope.
Also, the allocators have a defined order in which they get invoked and we need 
to maintain the order defined in the componentContext.xml

Now to solve the bug, the deploymentPlanningManager :: canAvoidCluster  needs 
to consider which scenario applies for the given vmprofile - i.e whether its 
local storage or shared to correctly figure out if cluster can be avoided by 
considering applicable pools only.
Does this work fine for your scenario?


[Koushik] Not sure if canAvoidCluster() is the right place. Shared and local 
pools can belong to the same cluster, so the cluster cannot be put in avoid 
set. Only the pools needs to be selectively added to avoid set.

[Prachi] Allocators add  the resources to avoid set and then DPM can figure out 
if a cluster is done searching looking at the avoid set, so that the planner 
need not choose that cluster on next try.
So two changes are needed: Storage Allocators should set resources in avoid set 
within their scope and DPM should  check if a cluster is done accordingly.
Can you let me know the specific bug you want to change allocators for? I will 
add the above change in a separate ticket if already not done for your bug.


On 15-Oct-2013, at 11:31 PM, Prachi Damle 
mailto:prachi.da...@citrix.com>>
wrote:

Koushik,

The deployment planning process is divided between DeploymentPlanners and 
Allocators. The planners are supposed to feed a list of clusters the allocators 
should look into to find a potential destination. While the allocators should 
provide a avoid set back to the planners consisting of any resource not 
considered.

Reasoning how the planners work:
Planners can set/reset clusters into avoid set depending on the heuristics they 
implement and the order in which they want the clusters to be traversed.
When all options of the planner are exhausted they should return an empty 
cluster list halting the search.

What should the allocators do?
Iterate

Resource Configuration Failure in Simulator mode (v4.2)

2013-10-17 Thread Alex Ough

All,

After getting the latest codes of v4.2, I'm trying to configure the
resources using 'setup/dev/advanced.cfg', but it seems to be failing with
this error.
"Could not find the selected project in the reactor: :cloud-marvin"

I'm using this command and it was working ok before getting the latest.
"mvn -Pdeveloper,marvin.setup -Dmarvin.config=setup/dev/advanced.cfg -pl
:cloud-marvin integration-test"

I'm wondering if this is happening only to me or something is missing.
Your help will be very appreciated.
Thanks
Alex Ough

[Discuss] Donation of RDP client code

2013-10-17 Thread Donal Lafferty

This donation addresses our need for an RDP client free of IP restrictions that 
can be included with Apache CloudStack's console VM.



RDP is of interest, because Hyper-V exposes VM consoles over RDP.  However, 
Apache CloudStack does not have an RDP client that is easy to build, can easily 
run in the console VM, and has no IP dependencies.



The donated code addresses this gap.  The source is for an all-Java solution, 
which meets the console VM's expectations of a Java-based renderer.  A Maven 
pom.xml is included that can be incorporated in the Apache CloudStack build.  
Finally, the code has no significant IP dependencies.  Only JUnit 3.8.1 (Common 
Public License Version 1.0) and tomcat-embed-core 7.0.30 (Apache License, 
Version 2.03) are referenced by the pom.xml.  No third party Java code is used 
or included.



The client's source is in the diff attached to the Review Board submission 
https://reviews.apache.org/r/14701/  This submission includes technical details 
in the description field.  The code was developed for Citrix Systems by a 
contractor.



Note that the client still has to be integrated with console system VM.  
However, it makes more sense to do this after we're certain that the client 
will go into ACS.



Please have a look, all opinions welcome!

Re: [GSoC] Congratulations and Wrap up

2013-10-17 Thread Abhinandan Prateek

I would like to thank Ian who did a fabulous job on ldap plugin and also
Sebastian who was always around to guide the students and mentors.
It would be good to see them at Amsterdam as I am also going to be around.

-abhi


On 17/10/13 7:49 pm, "sebgoa"  wrote:

>Hi,
>
>This week end is the Google Summer of Code summit. This year we had the
>chance to get five students in the program: Dharmesh, Meng, Ian, Nguyen
>and Shiva and I'd be remiss not to acknowledge their hard work.
>
>I would like to congratulate all of them on working hard this past summer
>(program finished late september), and challenging themselves to learn
>the open source way through our project at the Apache Software Foundation.
>
>Thanks also to the mentors: Kelcey, Hugo and Abhi who took time out of
>their busy schedule and $dayjob to help these students.
>
>-Ian Duffy made committer during the GSoC period, he developed a new
>LDAP/AD plugin and did an outstanding job doing his development in a
>complete software development pipeline with 90% unit test coverage,
>automatic building of rpms through jenkins etc. Check it out:
>http://youtu.be/-3LG8wP7Zac
>http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.2.0/html/Admin
>_Guide/LDAPserver-for-user-authentication.html
>Lately Ian has worked with me on a Google compute engine (GCE) interface
>but this was not part of his charter for GSoC, just a bonus:
>https://pypi.python.org/pypi/gcloud/0.0.1
>
>-Nguyen worked on extending our GRE SDN controller to support Xen and KVM
>hypervisor. 
>All his code is in the sdnextensions branch, and once Hugo is done
>reviewing it (or any other volunteer), we will merge it for 4.3
>https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=tree;h=refs/hea
>ds/sdnextensions;hb=sdnextensions
>Nguyen has done a terrific job tracking his progress in JIRA and dividing
>work in sub-tasks:
>https://issues.apache.org/jira/browse/CLOUDSTACK-1779
>
>Both Nguyen and Ian will be at the CCC conference in Amsterdam, Nguyen
>will talk about 'Autoscaling without netscaler' a patch he submitted
>recently, and Ian will talk about gcloud.
>
>-Meng worked on integration of whirr in CloudStack. The goal was to be
>able to provision Hadoop clusters in an elastic manner and use Yarn to
>expose a map-reduce API compatible with AWS EMR. This was a very
>ambitious project that required learning whirr, working across projects
>at the ASF and learning how to create new APIs. Her final patch is on
>review board:
>https://reviews.apache.org/r/14347/
>This will need additional work and more architectural thinking before we
>can use it, but is a solid base to think through how to expose an AWS EMR
>api with CloudStack.
>
>-Shiva worked on a new UI framework using bootstrap and angular.js, he
>progressed well and his code is now in master under tools/ngui , he also
>has a final patch in my inbox. Bad practice but review board is finicky
>with large patches :( This UI is very interesting because it can operate
>as standalone UI, meaning that you can run locally to manage your cloud
>remotely.
>
>Unfortunately Dharmesh was busy and did not progress well enough to pass
>the mid-term mark, he did try to tackle the big task of moving us from
>com.cloud to org.apache:
>https://issues.apache.org/jira/browse/CLOUDSTACK-212 something that we
>still need to do and requires some planning/coordination and should be
>done very early in the release cycle (like nowŠ)
>
>Early on they all worked on a great document that is built on jenkins but
>not released as part of our docs, it served the purpose of getting
>everyone through the "bonding period" and have all students learn how to
>submit a patch:
>http://jenkins.buildacloud.org/view/All/job/docs-4.3-gsoc-guide/
>I will try to wrap it up to have a final version of our GSoC 2013 for
>Apache CloudStack.
>
>Again congratulations to all, I am thrilled to have had you in the
>project and I hope you will continue participating in the community, we
>will hopefully do this again next year with another set of students.
>
>Cheers,
>
>-Sebastien
>

Re: [GSoC] Congratulations and Wrap up

2013-10-17 Thread Ian Duffy

Thank you Sebastien and Abhi along with the Cloudstack Community for making
this opportunity possible.
It was a very interesting and educational experience.

I hope to keep up work with you guys. I will be a bit quieter on here due
to college work over the next few months.

Looking forward to meeting you guys in Amsterdam! :-)


On 17 October 2013 15:19, sebgoa  wrote:

> Hi,
>
> This week end is the Google Summer of Code summit. This year we had the
> chance to get five students in the program: Dharmesh, Meng, Ian, Nguyen and
> Shiva and I'd be remiss not to acknowledge their hard work.
>
> I would like to congratulate all of them on working hard this past summer
> (program finished late september), and challenging themselves to learn the
> open source way through our project at the Apache Software Foundation.
>
> Thanks also to the mentors: Kelcey, Hugo and Abhi who took time out of
> their busy schedule and $dayjob to help these students.
>
> -Ian Duffy made committer during the GSoC period, he developed a new
> LDAP/AD plugin and did an outstanding job doing his development in a
> complete software development pipeline with 90% unit test coverage,
> automatic building of rpms through jenkins etc. Check it out:
> http://youtu.be/-3LG8wP7Zac
>
> http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.2.0/html/Admin_Guide/LDAPserver-for-user-authentication.html
> Lately Ian has worked with me on a Google compute engine (GCE) interface
> but this was not part of his charter for GSoC, just a bonus:
> https://pypi.python.org/pypi/gcloud/0.0.1
>
> -Nguyen worked on extending our GRE SDN controller to support Xen and KVM
> hypervisor.
> All his code is in the sdnextensions branch, and once Hugo is done
> reviewing it (or any other volunteer), we will merge it for 4.3
>
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=tree;h=refs/heads/sdnextensions;hb=sdnextensions
> Nguyen has done a terrific job tracking his progress in JIRA and dividing
> work in sub-tasks:
> https://issues.apache.org/jira/browse/CLOUDSTACK-1779
>
> Both Nguyen and Ian will be at the CCC conference in Amsterdam, Nguyen
> will talk about 'Autoscaling without netscaler' a patch he submitted
> recently, and Ian will talk about gcloud.
>
> -Meng worked on integration of whirr in CloudStack. The goal was to be
> able to provision Hadoop clusters in an elastic manner and use Yarn to
> expose a map-reduce API compatible with AWS EMR. This was a very ambitious
> project that required learning whirr, working across projects at the ASF
> and learning how to create new APIs. Her final patch is on review board:
> https://reviews.apache.org/r/14347/
> This will need additional work and more architectural thinking before we
> can use it, but is a solid base to think through how to expose an AWS EMR
> api with CloudStack.
>
> -Shiva worked on a new UI framework using bootstrap and angular.js, he
> progressed well and his code is now in master under tools/ngui , he also
> has a final patch in my inbox. Bad practice but review board is finicky
> with large patches :( This UI is very interesting because it can operate as
> standalone UI, meaning that you can run locally to manage your cloud
> remotely.
>
> Unfortunately Dharmesh was busy and did not progress well enough to pass
> the mid-term mark, he did try to tackle the big task of moving us from
> com.cloud to org.apache:
> https://issues.apache.org/jira/browse/CLOUDSTACK-212 something that we
> still need to do and requires some planning/coordination and should be done
> very early in the release cycle (like now…)
>
> Early on they all worked on a great document that is built on jenkins but
> not released as part of our docs, it served the purpose of getting everyone
> through the "bonding period" and have all students learn how to submit a
> patch:
> http://jenkins.buildacloud.org/view/All/job/docs-4.3-gsoc-guide/
> I will try to wrap it up to have a final version of our GSoC 2013 for
> Apache CloudStack.
>
> Again congratulations to all, I am thrilled to have had you in the project
> and I hope you will continue participating in the community, we will
> hopefully do this again next year with another set of students.
>
> Cheers,
>
> -Sebastien
>
>

Re: [GSoC] Congratulations and Wrap up

2013-10-17 Thread kel...@backbonetechnology.com

Thank you to all the students and mentors for bringing new and interesting 
features into cloudstack. With all the hard work and dedication, we ended up 
with some great additions.

See you in Amsterdam.

Sent from my HTC

- Reply message -
From: "sebgoa" 
To: "dev@cloudstack.apache.org" 
Subject: [GSoC] Congratulations and Wrap up
Date: Thu, Oct 17, 2013 7:19 AM

Hi,

This week end is the Google Summer of Code summit. This year we had the chance 
to get five students in the program: Dharmesh, Meng, Ian, Nguyen and Shiva and 
I'd be remiss not to acknowledge their hard work.

I would like to congratulate all of them on working hard this past summer 
(program finished late september), and challenging themselves to learn the open 
source way through our project at the Apache Software Foundation.

Thanks also to the mentors: Kelcey, Hugo and Abhi who took time out of their 
busy schedule and $dayjob to help these students.

-Ian Duffy made committer during the GSoC period, he developed a new LDAP/AD 
plugin and did an outstanding job doing his development in a complete software 
development pipeline with 90% unit test coverage, automatic building of rpms 
through jenkins etc. Check it out:
http://youtu.be/-3LG8wP7Zac
http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.2.0/html/Admin_Guide/LDAPserver-for-user-authentication.html
Lately Ian has worked with me on a Google compute engine (GCE) interface but 
this was not part of his charter for GSoC, just a bonus:
https://pypi.python.org/pypi/gcloud/0.0.1

-Nguyen worked on extending our GRE SDN controller to support Xen and KVM 
hypervisor. 
All his code is in the sdnextensions branch, and once Hugo is done reviewing it 
(or any other volunteer), we will merge it for 4.3
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=tree;h=refs/heads/sdnextensions;hb=sdnextensions
Nguyen has done a terrific job tracking his progress in JIRA and dividing work 
in sub-tasks:
https://issues.apache.org/jira/browse/CLOUDSTACK-1779

Both Nguyen and Ian will be at the CCC conference in Amsterdam, Nguyen will 
talk about 'Autoscaling without netscaler' a patch he submitted recently, and 
Ian will talk about gcloud.

-Meng worked on integration of whirr in CloudStack. The goal was to be able to 
provision Hadoop clusters in an elastic manner and use Yarn to expose a 
map-reduce API compatible with AWS EMR. This was a very ambitious project that 
required learning whirr, working across projects at the ASF and learning how to 
create new APIs. Her final patch is on review board:
https://reviews.apache.org/r/14347/
This will need additional work and more architectural thinking before we can 
use it, but is a solid base to think through how to expose an AWS EMR api with 
CloudStack.

-Shiva worked on a new UI framework using bootstrap and angular.js, he 
progressed well and his code is now in master under tools/ngui , he also has a 
final patch in my inbox. Bad practice but review board is finicky with large 
patches :( This UI is very interesting because it can operate as standalone UI, 
meaning that you can run locally to manage your cloud remotely.

Unfortunately Dharmesh was busy and did not progress well enough to pass the 
mid-term mark, he did try to tackle the big task of moving us from com.cloud to 
org.apache:
https://issues.apache.org/jira/browse/CLOUDSTACK-212 something that we still 
need to do and requires some planning/coordination and should be done very 
early in the release cycle (like now…)

Early on they all worked on a great document that is built on jenkins but not 
released as part of our docs, it served the purpose of getting everyone through 
the "bonding period" and have all students learn how to submit a patch:
http://jenkins.buildacloud.org/view/All/job/docs-4.3-gsoc-guide/
I will try to wrap it up to have a final version of our GSoC 2013 for Apache 
CloudStack.

Again congratulations to all, I am thrilled to have had you in the project and 
I hope you will continue participating in the community, we will hopefully do 
this again next year with another set of students.

Cheers,

-Sebastien

Re: [Discuss] Donation of RDP client code

2013-10-17 Thread Wei ZHOU

It looks very nice. Thanks!

Hugo are working on the refactor of systemvm. Maybe it need to be modified.


2013/10/17 Donal Lafferty 

> This donation addresses our need for an RDP client free of IP restrictions
> that can be included with Apache CloudStack's console VM.
>
>
>
> RDP is of interest, because Hyper-V exposes VM consoles over RDP.
>  However, Apache CloudStack does not have an RDP client that is easy to
> build, can easily run in the console VM, and has no IP dependencies.
>
>
>
> The donated code addresses this gap.  The source is for an all-Java
> solution, which meets the console VM's expectations of a Java-based
> renderer.  A Maven pom.xml is included that can be incorporated in the
> Apache CloudStack build.  Finally, the code has no significant IP
> dependencies.  Only JUnit 3.8.1 (Common Public License Version 1.0) and
> tomcat-embed-core 7.0.30 (Apache License, Version 2.03) are referenced by
> the pom.xml.  No third party Java code is used or included.
>
>
>
> The client's source is in the diff attached to the Review Board submission
> https://reviews.apache.org/r/14701/  This submission includes technical
> details in the description field.  The code was developed for Citrix
> Systems by a contractor.
>
>
>
> Note that the client still has to be integrated with console system VM.
>  However, it makes more sense to do this after we're certain that the
> client will go into ACS.
>
>
>
> Please have a look, all opinions welcome!
>
>

Re: [GSoC] Congratulations and Wrap up

2013-10-17 Thread Wei ZHOU

Thanks all!


2013/10/17 kel...@backbonetechnology.com 

> Thank you to all the students and mentors for bringing new and interesting
> features into cloudstack. With all the hard work and dedication, we ended
> up with some great additions.
>
> See you in Amsterdam.
>
> Sent from my HTC
>
> - Reply message -
> From: "sebgoa" 
> To: "dev@cloudstack.apache.org" 
> Subject: [GSoC] Congratulations and Wrap up
> Date: Thu, Oct 17, 2013 7:19 AM
>
> Hi,
>
> This week end is the Google Summer of Code summit. This year we had the
> chance to get five students in the program: Dharmesh, Meng, Ian, Nguyen and
> Shiva and I'd be remiss not to acknowledge their hard work.
>
> I would like to congratulate all of them on working hard this past summer
> (program finished late september), and challenging themselves to learn the
> open source way through our project at the Apache Software Foundation.
>
> Thanks also to the mentors: Kelcey, Hugo and Abhi who took time out of
> their busy schedule and $dayjob to help these students.
>
> -Ian Duffy made committer during the GSoC period, he developed a new
> LDAP/AD plugin and did an outstanding job doing his development in a
> complete software development pipeline with 90% unit test coverage,
> automatic building of rpms through jenkins etc. Check it out:
> http://youtu.be/-3LG8wP7Zac
>
> http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.2.0/html/Admin_Guide/LDAPserver-for-user-authentication.html
> Lately Ian has worked with me on a Google compute engine (GCE) interface
> but this was not part of his charter for GSoC, just a bonus:
> https://pypi.python.org/pypi/gcloud/0.0.1
>
> -Nguyen worked on extending our GRE SDN controller to support Xen and KVM
> hypervisor.
> All his code is in the sdnextensions branch, and once Hugo is done
> reviewing it (or any other volunteer), we will merge it for 4.3
>
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=tree;h=refs/heads/sdnextensions;hb=sdnextensions
> Nguyen has done a terrific job tracking his progress in JIRA and dividing
> work in sub-tasks:
> https://issues.apache.org/jira/browse/CLOUDSTACK-1779
>
> Both Nguyen and Ian will be at the CCC conference in Amsterdam, Nguyen
> will talk about 'Autoscaling without netscaler' a patch he submitted
> recently, and Ian will talk about gcloud.
>
> -Meng worked on integration of whirr in CloudStack. The goal was to be
> able to provision Hadoop clusters in an elastic manner and use Yarn to
> expose a map-reduce API compatible with AWS EMR. This was a very ambitious
> project that required learning whirr, working across projects at the ASF
> and learning how to create new APIs. Her final patch is on review board:
> https://reviews.apache.org/r/14347/
> This will need additional work and more architectural thinking before we
> can use it, but is a solid base to think through how to expose an AWS EMR
> api with CloudStack.
>
> -Shiva worked on a new UI framework using bootstrap and angular.js, he
> progressed well and his code is now in master under tools/ngui , he also
> has a final patch in my inbox. Bad practice but review board is finicky
> with large patches :( This UI is very interesting because it can operate as
> standalone UI, meaning that you can run locally to manage your cloud
> remotely.
>
> Unfortunately Dharmesh was busy and did not progress well enough to pass
> the mid-term mark, he did try to tackle the big task of moving us from
> com.cloud to org.apache:
> https://issues.apache.org/jira/browse/CLOUDSTACK-212 something that we
> still need to do and requires some planning/coordination and should be done
> very early in the release cycle (like now…)
>
> Early on they all worked on a great document that is built on jenkins but
> not released as part of our docs, it served the purpose of getting everyone
> through the "bonding period" and have all students learn how to submit a
> patch:
> http://jenkins.buildacloud.org/view/All/job/docs-4.3-gsoc-guide/
> I will try to wrap it up to have a final version of our GSoC 2013 for
> Apache CloudStack.
>
> Again congratulations to all, I am thrilled to have had you in the project
> and I hope you will continue participating in the community, we will
> hopefully do this again next year with another set of students.
>
> Cheers,
>
> -Sebastien
>

Re: internal LB provider (from the thread: [VOTE] Apache CloudStack 4.2.0 (third round))

Thanks Marcus,

this helps, I can now enable it in the interface after configuring it
as enabled. Somehow I still have to enable the InternalLbVm provider
in the physical network after configuring it as enabled and can't find
a api call to do this.

On Thu, Oct 17, 2013 at 4:21 PM, Marcus Sorensen  wrote:
> Yes, it should supposedly never happen under normal circumstances. The
> issue I had was because I was using a Marvin zone config that needed
> to be updated for the new feature. Added:
>
> {
>
> "broadcastdomainrange": "ZONE",
>
> "name": "InternalLbVm"
>
>  }
>
> This should be configured on zone setup as well. To fix it manually, I did:
>
> # list internalloadbalancerelements
>
> # configure internalloadbalancerelement enabled=true id=
>
> On Thu, Oct 17, 2013 at 8:03 AM, Daan Hoogland  
> wrote:
>> Marcus.
>>
>> I ran into this on today's master. Did you follow up on this in 4.2?
>> Or do you have a ticket or wiki page for it?
>>
>> tahnks
>> Daan
>>
>> On Thu, Aug 29, 2013 at 1:57 AM, Marcus Sorensen  wrote:
>>> 2. can't figure out how to deploy VPC. In 4.2 it tells me to jump
>>> through a few extra hoops regarding setting up internal loadbalancer
>>> service first, "Provider InternalLbVm should be enabled in at least
>>> one physical network of the zone specified", so I look at my physical
>>> network -> network service providers and see "Internal LB VM" is
>>> disabled, so I try to enable it and get "Provider is not ready, cannot
>>> Enable the provider, please configure the provider first". I'm stuck,
>>> I don't see anything in the UI that would provide a configuration
>>> dialog for the internal lb provider, nor do I see anything in the
>>> global options. It makes me wonder how upgrades will fare if VPCs are
>>> being used.

Re: internal LB provider (from the thread: [VOTE] Apache CloudStack 4.2.0 (third round))

2013-10-17 Thread Alena Prokharchyk

Daan, the API is "updateNetworkServiceProvider"

-Alena.

From: Daan Hoogland mailto:daan.hoogl...@gmail.com>>
Reply-To: "dev@cloudstack.apache.org" 
mailto:dev@cloudstack.apache.org>>
Date: Thursday, October 17, 2013 10:17 AM
To: dev mailto:dev@cloudstack.apache.org>>
Subject: Re: internal LB provider (from the thread: [VOTE] Apache CloudStack 
4.2.0 (third round))

Thanks Marcus,

this helps, I can now enable it in the interface after configuring it
as enabled. Somehow I still have to enable the InternalLbVm provider
in the physical network after configuring it as enabled and can't find
a api call to do this.

On Thu, Oct 17, 2013 at 4:21 PM, Marcus Sorensen 
mailto:shadow...@gmail.com>> wrote:
Yes, it should supposedly never happen under normal circumstances. The
issue I had was because I was using a Marvin zone config that needed
to be updated for the new feature. Added:

 {

 "broadcastdomainrange": "ZONE",

 "name": "InternalLbVm"

  }

This should be configured on zone setup as well. To fix it manually, I did:

# list internalloadbalancerelements

# configure internalloadbalancerelement enabled=true id=

On Thu, Oct 17, 2013 at 8:03 AM, Daan Hoogland 
mailto:daan.hoogl...@gmail.com>> wrote:
Marcus.

I ran into this on today's master. Did you follow up on this in 4.2?
Or do you have a ticket or wiki page for it?

tahnks
Daan

On Thu, Aug 29, 2013 at 1:57 AM, Marcus Sorensen 
mailto:shadow...@gmail.com>> wrote:
2. can't figure out how to deploy VPC. In 4.2 it tells me to jump
through a few extra hoops regarding setting up internal loadbalancer
service first, "Provider InternalLbVm should be enabled in at least
one physical network of the zone specified", so I look at my physical
network -> network service providers and see "Internal LB VM" is
disabled, so I try to enable it and get "Provider is not ready, cannot
Enable the provider, please configure the provider first". I'm stuck,
I don't see anything in the UI that would provide a configuration
dialog for the internal lb provider, nor do I see anything in the
global options. It makes me wonder how upgrades will fare if VPCs are
being used.

Re: internal LB provider (from the thread: [VOTE] Apache CloudStack 4.2.0 (third round))

On Thu, Oct 17, 2013 at 7:22 PM, Alena Prokharchyk
 wrote:
> updateNetworkServiceProvider

ah, so this works even if the uuid is not returned in
listNetworkServiceProvidersCmd?

Re: internal LB provider (from the thread: [VOTE] Apache CloudStack 4.2.0 (third round))

2013-10-17 Thread Alena Prokharchyk

I do see the UUID being returned as a part of listNetworkServiceProvidersCmd 
call ("id" field):


{ "listnetworkserviceprovidersresponse" : { "count":8 ,"networkserviceprovider" 
: [  
{"name":"InternalLbVm","physicalnetworkid":"235cd520-b73c-458b-aa69-14df2f4fe871","state":"Disabled","id":"83011ee3-e498-4a31-8ff0-ba44d3b571c3","servicelist":["Lb"]},
 
{"name":"VpcVirtualRouter","physicalnetworkid":"235cd520-b73c-458b-aa69-14df2f4fe871","state":"Enabled","id":"8204b150-bb74-4cce-b0f3-3624d978a7f1","servicelist":["Vpn","Dh

Btw, the UI should enable all in-built providers (VR, VPCVR, Internal LB) by 
default as a part of the zone creation. If UI doesn't do it, its a bug. If you 
use some API client, you have to make the calls explicitly.



-alena.

From: Daan Hoogland mailto:daan.hoogl...@gmail.com>>
Reply-To: "dev@cloudstack.apache.org" 
mailto:dev@cloudstack.apache.org>>
Date: Thursday, October 17, 2013 10:29 AM
To: dev mailto:dev@cloudstack.apache.org>>
Subject: Re: internal LB provider (from the thread: [VOTE] Apache CloudStack 
4.2.0 (third round))

On Thu, Oct 17, 2013 at 7:22 PM, Alena Prokharchyk
mailto:alena.prokharc...@citrix.com>> wrote:
updateNetworkServiceProvider

ah, so this works even if the uuid is not returned in
listNetworkServiceProvidersCmd?

Re: Review Request 14522: [CLOUDSTACK-4771] Support Revert VM Disk from Snapshot



> On Oct. 17, 2013, 11:18 a.m., John Burwell wrote:
> > ui/scripts/storage.js, line 1963
> > 
> >
> > How will a corrupted snapshot failure be surfaced to the user?
> 
> Chris Suich wrote:
> It depends on what you mean by corrupted. If we can no longer find the 
> snapshot we expect from the DB, then the API will throw an exception, per 
> your other review, and the list of snapshots will never populate. If the file 
> exists but is corrupted, then that wouldn't be discovered until either the 
> attempt to revert the snapshot or after the revert operation and the user 
> attempts to use the volume.

As I mentioned earlier, the user will get the idea that something bad happened, 
but will be left with little to no information to pass onto the 
operator/administrator.  We need to surface enough breadcrumb information so 
that the user a) understands that the snapshot metadata has become corrupted 
and b) provide them with enough identifying information to support the forensic 
work of their operator/administrator.


> On Oct. 17, 2013, 11:18 a.m., John Burwell wrote:
> > engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/DataMotionServiceImpl.java,
> >  line 62
> > 
> >
> > Add context information to error message to help operators and users 
> > determine what couldn't be snapshotted.
> 
> Chris Suich wrote:
> Fair enough - I just copied these existing error messages. What kind of 
> context would you like to see in this error message? The src & dest 
> datastores, files, etc.? I agree that there needs to be context, but this 
> would potentially be a user facing error, so I also don't want to overload 
> them with context that means nothing to them.

Enough details so an operator/admin could look up the associated 
volume/snapshot and debug the issue.  Likely a good start would be to include 
the identifying information of the srcData and destData parameters.  Ask 
yourself, "What I need to know as an operator/administrator to debug this 
problem?" and put that information in the error message.


> On Oct. 17, 2013, 11:18 a.m., John Burwell wrote:
> > engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/DataMotionServiceImpl.java,
> >  line 79
> > 
> >
> > DRY out this code (e.g. extract to a private method).  This rules needs 
> > to be consolidated + add the context information mentioned above.
> 
> Chris Suich wrote:
> The two invocations of pickStrategy() are actually calling an overloaded 
> function. I'm not sure how we could DRY this out since they method 
> calls/parameter lists are actually different.

I missed the overload.  The actual issue, in my mind, is that pickStrategy 
needs to be DRY'ed out.  This likely requires a new interface which is outside 
the bounds of this patch.  However, it is technical debt that we somehow need 
to surface.  As such, please open a ticket to describing issue and put a TODO 
on pickStrategy to consolidate the implementations with a reference to the 
ticket.  Also, place a TODO on this lines to reference the future refactoring.


- John


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14522/#review27126
---


On Oct. 16, 2013, 10:50 a.m., Chris Suich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14522/
> ---
> 
> (Updated Oct. 16, 2013, 10:50 a.m.)
> 
> 
> Review request for cloudstack, Brian Federle, edison su, John Burwell, and 
> Mike Tutkowski.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> After the last batch of work to the revertSnapshot API, SnapshotServiceImpl 
> was not tied into the workflow to be used by storage providers. I have added 
> the logic in a similar fashion to takeSnapshot(), backupSnapshot() and 
> deleteSnapshot().
> 
> I have also added a 'Revert to Snapshot' action to the volume snapshots list 
> in the UI.
> 
> 
> Diffs
> -
> 
>   api/src/org/apache/cloudstack/api/ApiConstants.java 62eed09 
>   api/src/org/apache/cloudstack/api/response/SnapshotResponse.java e9cb109 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/PrimaryDataStoreDriver.java
>  b124d83 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/SnapshotInfo.java
>  8d6b760 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/StrategyPriority.java
>  81034b1 
>   
> engine/api/test/org/apache/cloudstack/engine/subsystem/api/storage/StrategyPriorityTest.java
>  3d75279 
>   
> engine/st

Re: Snapshot Strategy Review

Chris,

I have updated my comments in https://reviews.apache.org/r/14522/.   I will 
likely be offline the remainder of the day due to two sick kids + my own 
sickness.

Thanks,
-John

P.S.  I just noticed that this email thread was private to Edison and I.  All 
dev discussions should occur on the dev@ list.  Therefore, I have CC'ed this 
chain to the list, and please ensure that future replies to the email chain 
also include the dev@ list.

On Oct 17, 2013, at 1:28 PM, "SuichII, Christopher"  
wrote:

> John,
> 
> I'm glad to hear you think these changes look better. It sounds like 
> everything else you mentioned is fairly minor, so I have no problems making 
> some changes. However, I have made a couple comments on those issues 
> regarding the direction you want to take with the context in error messages, 
> how to DRY out the code (since the code is not actually duplicated) and I'm 
> not sure if the UI portion I modified should be handling the 'corrupted' 
> snapshots you're talking about.
> 
> Thanks!
> -Chris
> -- 
> Chris Suich
> chris.su...@netapp.com
> NetApp Software Engineer
> Data Center Platforms – Cloud Solutions
> Citrix, Cisco & Red Hat
> 
> On Oct 16, 2013, at 2:16 PM, Chris Suich  wrote:
> 
>> No worries - I know I'm trying to push this quick so thanks for working with 
>> me on this.
>> 
>> -Chris
>> -- 
>> Chris Suich
>> chris.su...@netapp.com
>> NetApp Software Engineer
>> Data Center Platforms – Cloud Solutions
>> Citrix, Cisco & Red Hat
>> 
>> On Oct 16, 2013, at 2:02 PM, John Burwell  wrote:
>> 
>>> Chris,
>>> 
>>> I saw it, but I have $dayjob priority ATM.  I will re-review late this
>>> evening or first thing tomorrow.  Dependent on the outcome, we can
>>> setup a call for late morning/early afternoon if necessary.
>>> 
>>> I apologize for the delay,
>>> -John
>>> 
>>> 
>>> 
>>> 
>>> On Oct 16, 2013, at 1:58 PM, "SuichII, Christopher"
>>>  wrote:
>>> 
 Hey John,

 I know it hasn't been up long so you probably haven't seen it, but I 
 posted another revision this morning. Maybe we could talk on the phone 
 about this review so we can get this resolved ASAP? I'm thinking over the 
 phone it might be quicker to clear up the objective of the changes and 
 discuss what can be done now and what should be done in a later release.

 Thanks,
 Chris
 --
 Chris Suich
 chris.su...@netapp.com
 NetApp Software Engineer
 Data Center Platforms – Cloud Solutions
 Citrix, Cisco & Red Hat

>> 
> 

signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Review Request 14334: CLOUDSTACK 4705: Fixed domain memory limits test cases

Gaurav, I will but it doesn't apply to master. Can you update it?

regards,

On Wed, Oct 16, 2013 at 6:43 PM, Girish Shilamkar  wrote:
>
>
>> On Oct. 8, 2013, 11:14 a.m., abhinav roy wrote:
>> > Ship It!
>
> Can someone commit this patch.
>
>
> - Girish
>
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14334/#review26770
> ---
>
>
> On Sept. 25, 2013, 11:06 a.m., Gaurav Aradhye wrote:
>>
>> ---
>> This is an automatically generated e-mail. To reply, visit:
>> https://reviews.apache.org/r/14334/
>> ---
>>
>> (Updated Sept. 25, 2013, 11:06 a.m.)
>>
>>
>> Review request for cloudstack, sailaja mada and Prasanna Santhanam.
>>
>>
>> Repository: cloudstack-git
>>
>>
>> Description
>> ---
>>
>> Fixed CLOUDSTACK 4705:
>> Removed attribute error and fixed indentation issues.
>>
>> Added update_resource_count method to update the resource count after 
>> upgrading and downgrading the service offering so as to get latest count. 
>> Issue was found that it was showing old resource count without calling this 
>> API.
>>
>>
>> Diffs
>> -
>>
>>   test/integration/component/memory_limits/test_domain_limits.py 479ec0b
>>
>> Diff: https://reviews.apache.org/r/14334/diff/
>>
>>
>> Testing
>> ---
>>
>>
>> Thanks,
>>
>> Gaurav Aradhye
>>
>>
>

RE: Possible bug in DeploymentPlanner?

2013-10-17 Thread Prachi Damle

Koushik,

I don't think you should add this check before calling allocators. It is the 
allocators output that should tell that local storage pools were not found. 
For your case where you need to put cluster in avoid set when no local pool is 
present:
a) Only local storage allocator should get invoked, all other allocators need 
to have a check at the start saying they can process only 'shared' storage 
request - This will make sure you do not get any pools from any other allocator 
for your local storage request.
b) canAvoidCluster() need to be changed to compare only 'local' or 'shared' 
pool set to the avoid output of the allocators to see if all pools are in avoid 
set - Or if there are no pools then  directly avoid the cluster.

Prachi

-Original Message-
From: Koushik Das [mailto:koushik@citrix.com] 
Sent: Thursday, October 17, 2013 9:23 AM
To: 
Subject: Re: Possible bug in DeploymentPlanner?

Prachi,

I am planning to add the following code in findSuitablePoolsForVolumes() 
method. If the VM has a local disk and the cluster doesn't have a local pool 
then the cluster will be added to avoid set. Let me know if you see any issues 
with this.

if (useLocalStorage) {
List allPoolsInCluster = 
_storagePoolDao.findPoolsByTags(plan.getDataCenterId(), plan.getPodId(), 
plan.getClusterId(), null);
boolean localStoragePoolExists = false;
for (StoragePoolVO pool : allPoolsInCluster) {
if (pool.isLocal()) {
localStoragePoolExists = true;
break;
}
}
if (!localStoragePoolExists) {
avoid.addCluster(plan.getClusterId());
}
}

-Koushik

On 17-Oct-2013, at 3:21 AM, Prachi Damle 
mailto:prachi.da...@citrix.com>> wrote:



-Original Message-
From: Koushik Das [mailto:koushik@citrix.com]
Sent: Wednesday, October 16, 2013 5:21 AM
To: mailto:dev@cloudstack.apache.org>>
Subject: Re: Possible bug in DeploymentPlanner?


On 16-Oct-2013, at 3:12 AM, Prachi Damle 
mailto:prachi.da...@citrix.com>> wrote:


-Original Message-
From: Koushik Das [mailto:koushik@citrix.com]
Sent: Tuesday, October 15, 2013 11:43 AM
To: mailto:dev@cloudstack.apache.org>>
Subject: Re: Possible bug in DeploymentPlanner?


Thanks for the explanation Prachi.

Here is one issue I see with what you explained below. If an allocator is not 
suppose to handle a particular scenario how should it deal with the avoid set. 
Should it put all available pools in the avoid set. In that case subsequent 
allocators won't get a chance to do anything as all pools are in avoid set. If 
more than one allocator can operate on a particular pool then the situation 
becomes tricky.

To give an example, if there is a request for volume on shared storage and 
first allocator to get called is the local allocator. Since it is not going to 
handle this case it will return empty/null list. Now should it put all the 
other shared pools in avoid set or only the local pools in avoid set? Similarly 
if the request is for a local volume and the request first goes to zone or 
cluster-wide storage allocator what should be there in the avoid set?


[Prachi] I see the bug you are thinking about - deploymentPlanningManager :: 
canAvoidCluster  is only checking the shared storages and thus does not put the 
cluster in avoid set in case the local storage is not a match for the host.

I think the storage allocators should put the resources not considered, in 
avoid set, that are within their scope. Also they should do so only when they 
are supposed to handle the request. Thus local allocator sees only local pools, 
so it should place them into avoid set if not considered and only when this 
allocator is supposed to handle the scenario. While the zone-wide/cluster-wide 
should set those within their scope.
Also, the allocators have a defined order in which they get invoked and we need 
to maintain the order defined in the componentContext.xml

Now to solve the bug, the deploymentPlanningManager :: canAvoidCluster  needs 
to consider which scenario applies for the given vmprofile - i.e whether its 
local storage or shared to correctly figure out if cluster can be avoided by 
considering applicable pools only.
Does this work fine for your scenario?


[Koushik] Not sure if canAvoidCluster() is the right place. Shared and local 
pools can belong to the same cluster, so the cluster cannot be put in avoid 
set. Only the pools needs to be selectively added to avoid set.

[Prachi] Allocators add  the resources to avoid set and then DPM can figure out 
if a cluster is done searching looking at the avoid set, so that the planner 
need not choose that cluster on next try.
So two changes are needed: Storage Allocators should set resources in avoid set 
within their scope and DPM should  check if a cluster is

Fwd: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

2013-10-17 Thread Alex Ough

All,

Currently, under the environment of cloudstack with multiple regions, each
region has its own management server running with a separate database. So
if we want to support multiple regions and provide one point of entry for a
customer, we need to duplicate domain/account/user information of that
customer to all of the databases of regions the customer accesses, which
will cause data discrepancies when users update those data independently in
each management server.

So I'd like to provide a way to sync up the data using the messaging system
introduced in 4.1.0. Using the events from each management server, updates
from each region can be propagated to the rest regions and they can be
executed accordingly.

I hope you guys have a chance to think about this and give some feedbacks
if interested.
Thanks in advance.
Alex Ough

4.2.1 package issues

2013-10-17 Thread Bjoern Teipel


Hi all,

did somebody encounter issues during RPM packaging ?
I got jar file read errors and I can't tell where they are coming from.
It seems maven tries to download those jars during the compile from an 
invalid url, because those just contain a webpage


Internet2 Shibboleth Project has moved

How can I fix the dependency urls ?

Bjoern

[INFO] 


[INFO] BUILD FAILURE
[INFO] 


[INFO] Total time: 44:39.903s
[INFO] Finished at: Thu Oct 17 10:12:50 PDT 2013
[INFO] Final Memory: 57M/393M
[INFO] 

[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile 
(default-compile) on project cloud-awsapi: Compilation failure: 
Compilation failure:
[ERROR] error: error reading 
/home/bteipel/.m2/repository/org/apache/axis2/mex/1.5.4/mex-1.5.4-impl.jar; 
error in opening zip file
[ERROR] error: error reading 
/home/bteipel/.m2/repository/org/apache/axis2/axis2-mtompolicy/1.5.4/axis2-mtompolicy-1.5.4.jar; 
error in opening zip file
[ERROR] error: error reading 
/home/bteipel/.m2/repository/org/apache/ws/commons/axiom/axiom-dom/1.2.10/axiom-dom-1.2.10.jar; 
error in opening zip file
[ERROR] error: error reading 
/home/bteipel/.m2/repository/org/opensaml/opensaml1/1.1/opensaml1-1.1.jar; 
error in opening zip file
[ERROR] error: error reading 
/home/bteipel/.m2/repository/commons-lang/commons-lang/2.3/commons-lang-2.3.jar; 
error in opening zip file

[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the 
-e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, 
please read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

[ERROR]
[ERROR] After correcting the problems, you can resume the build with the 
command

[ERROR]   mvn  -rf :cloud-awsapi
error: Bad exit status from /var/tmp/rpm-tmp.4NppiB (%build)

Re: [MERGE] txn-refactor


On 10/17/2013 01:53 AM, Hugo Trippaers wrote:

Hey Darren,

Looking through the code it looks like this more an API change than an actual 
redesign of the transaction code? I like the resulting code a lot better than 
the existing way of doing it. As far as i can see you wrapped the existing 
TransactionLegacy way of doing it (txn.start / txn.commit) inside of the new 
Transaction functions execute and executeWithException. So if i understand it 
correctly, nothing changed in how transactions are actually handled, except 
that the code can now be easily changed to use spring TX.

Yes the purpose of this is to introduce a new API that will be 
compatible with Spring TX.  The internals are all still custom ACS 
transaction.  This is why I'm comfortable with this change, because 
nothing changed too much.


But know I do need to make this change because I found that people were 
inconsistently using the old API and it was causing problems with the 
spring modularization branch.  In the spring modularization branch I've 
moved the AOP logic for the transactions from custom ACS AOP to Spring 
AOP which has a slightly different semantics.


Also note I found tons of bugs in the transaction handling while doing 
this.  Having the anonymous class style has made error handling more 
consistent.  But, there a tons of places that people are catching 
Throwable/Exception where they shouldn't.  That's a different 
discussion, I need to put together some thoughts on more consistent 
error handling in ACS.



Also you made that changes to a couple of classes to use the new api, but the 
majority of the classes still need to be done. It might be nice to annotate the 
TransactionLegacy class with @Deprecated so we can easily identify what needs 
to be done?


I updated all of the management non-DAO code.  AWS, Usage and DAOs are 
still using the old API.  I didn't touch DAOs because its not worth it 
in my mind.  DAOs should move to declarative transaction managment which 
will mean basically the whole method will be under a transactions.  So I 
didn't want to convert the code to the anonymous class style, and then 
when I do declarative transactions go and change all the DAOs again.


I don't know if I want to actually put @Deprecated on TransactionLegacy 
yet because there are things you can do with it that you can't do in the 
new API.  Namely, prepared statements and switching databases.




The new code is not covered by any new unit test yet? I couldn't check the 
cobertura result yet as there are some build and test failures. Can you have a 
look at this build : 
http://jenkins.buildacloud.org/job/cloudstack-maven-build-with-branch-parameter/3/.
  You can kick off that job yourself from jenkins if you like to do the full 
test. Didn't do the noredist build test yet as the normal build failed.



I can write a unit test that tests the new code.  I started by altering 
the existing transaction unit test, but then later figured out they 
don't work and are not being ran as part of the build.  So I just did a 
bunch of manual testing.


Darren

Re: 4.2.1 package issues


On 10/17/2013 11:06 AM, Bjoern Teipel wrote:

Compilation failure:
[ERROR] error: error reading
/home/bteipel/.m2/repository/org/apache/axis2/mex/1.5.4/mex-1.5.4-impl.jar;
error in opening zip file
[ERROR] error: error reading
/home/bteipel/.m2/repository/org/apache/axis2/axis2-mtompolicy/1.5.4/axis2-mtompolicy-1.5.4.jar;
error in opening zip file
[ERROR] error: error reading
/home/bteipel/.m2/repository/org/apache/ws/commons/axiom/axiom-dom/1.2.10/axiom-dom-1.2.10.jar;
error in opening zip file
[ERROR] error: error reading
/home/bteipel/.m2/repository/org/opensaml/opensaml1/1.1/opensaml1-1.1.jar;
error in opening zip file
[ERROR] error: error reading
/home/bteipel/.m2/repository/commons-lang/commons-lang/2.3/commons-lang-2.3.jar;
error in opening zip file


Typically this happens when a maven mirror doesn't exist any more, 
moved, is having issues, etc.  Actually open up one of those bad jars in 
a text editor and good chance there is some HTML error document in there.


You can try just deleteing the folders that contain the bad jars and 
hopefully it will download them from a good location.  In general its a 
swell idea to use a maven proxy, Nexus or Artifactory, they usually do a 
better job of avoiding these errors.


Darren

Re: Review Request 14522: [CLOUDSTACK-4771] Support Revert VM Disk from Snapshot

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14522/
---

(Updated Oct. 17, 2013, 6:46 p.m.)

Review request for cloudstack, Brian Federle, edison su, John Burwell, and Mike
Tutkowski.

Changes
---

-Added context to the error messages for not finding DataMotionStrategies.
However, it appears that the place where pickStrategy() is called for
SnapshotStrategies is not able to actually throw error messages. I will open a
separate issue for this.
-Added TODOs for DRYing out the overloaded pickStrategy() methods.
-Added context to the error message for failing to find the image store
snapshot info for a snapshot response. However, this also is unable to throw a
meaning full error message as the exception would be caught and replaced with a
generic error. I will create a separate issue for this as well.

Repository: cloudstack-git

Description
---

After the last batch of work to the revertSnapshot API, SnapshotServiceImpl was
not tied into the workflow to be used by storage providers. I have added the
logic in a similar fashion to takeSnapshot(), backupSnapshot() and
deleteSnapshot().

I have also added a 'Revert to Snapshot' action to the volume snapshots list in
the UI.

Diffs (updated)
-

engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/StrategyPriority.java
81034b1

engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/DataMotionServiceImpl.java
2d31320
server/src/com/cloud/api/ApiResponseHelper.java f4ca112
server/src/com/cloud/storage/snapshot/SnapshotManagerImpl.java dade983

Diff: https://reviews.apache.org/r/14522/diff/

Testing
---

I have tested all of this locally with a custom storage provider.

Unfortunately, I'm still in the middle of figuring out how to properly unit
test this type of code. If anyone has any recommendations, please let me know.

Thanks,

Chris Suich

Re: Snapshot Strategy Review

My apologies. Everything in the private email was also posted to the dev list, 
I just wanted to send a personal email to make sure you saw the update in your 
inbox.

-Chris
--
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Oct 17, 2013, at 1:45 PM, John Burwell 
mailto:jburw...@basho.com>> wrote:

Chris,

I have updated my comments in https://reviews.apache.org/r/14522/.   I will 
likely be offline the remainder of the day due to two sick kids + my own 
sickness.

Thanks,
-John


P.S.  I just noticed that this email thread was private to Edison and I.  All 
dev discussions should occur on the dev@ list.  Therefore, I have CC'ed this 
chain to the list, and please ensure that future replies to the email chain 
also include the dev@ list.

On Oct 17, 2013, at 1:28 PM, "SuichII, Christopher" 
mailto:chris.su...@netapp.com>> wrote:

John,

I'm glad to hear you think these changes look better. It sounds like everything 
else you mentioned is fairly minor, so I have no problems making some changes. 
However, I have made a couple comments on those issues regarding the direction 
you want to take with the context in error messages, how to DRY out the code 
(since the code is not actually duplicated) and I'm not sure if the UI portion 
I modified should be handling the 'corrupted' snapshots you're talking about.

Thanks!
-Chris
--
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Oct 16, 2013, at 2:16 PM, Chris Suich 
mailto:chris.su...@netapp.com>> wrote:

No worries - I know I'm trying to push this quick so thanks for working with me 
on this.

-Chris
--
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Oct 16, 2013, at 2:02 PM, John Burwell  wrote:

Chris,

I saw it, but I have $dayjob priority ATM.  I will re-review late this
evening or first thing tomorrow.  Dependent on the outcome, we can
setup a call for late morning/early afternoon if necessary.

I apologize for the delay,
-John




On Oct 16, 2013, at 1:58 PM, "SuichII, Christopher"
 wrote:

Hey John,

I know it hasn't been up long so you probably haven't seen it, but I posted 
another revision this morning. Maybe we could talk on the phone about this 
review so we can get this resolved ASAP? I'm thinking over the phone it might 
be quicker to clear up the objective of the changes and discuss what can be 
done now and what should be done in a later release.

Thanks,
Chris
--
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

Re: Possible bug in DeploymentPlanner?

2013-10-17 Thread Koushik Das


On 17-Oct-2013, at 11:29 PM, Prachi Damle 
 wrote:

> Koushik,
> 
> I don't think you should add this check before calling allocators. It is the 
> allocators output that should tell that local storage pools were not found. 
> For your case where you need to put cluster in avoid set when no local pool 
> is present:
> a) Only local storage allocator should get invoked, all other allocators need 
> to have a check at the start saying they can process only 'shared' storage 
> request - This will make sure you do not get any pools from any other 
> allocator for your local storage request.

[Koushik] I made the changes for this and after this the planner bug got 
uncovered where it went in a infinite loop trying to deploy VM with local disk 
when no local pool is available.
 
> b) canAvoidCluster() need to be changed to compare only 'local' or 'shared' 
> pool set to the avoid output of the allocators to see if all pools are in 
> avoid set - Or if there are no pools then  directly avoid the cluster.

[Koushik] Ok. I will make the necessary changes for this but this closely ties 
the allocators and the assumption in canAvoidCluster(). The scenario I am 
seeing is the latter where there is no pools in the allocator avoid set. So as 
you suggested in this case as well the cluster needs to be in avoid set.

> 
> Prachi
> 
> -Original Message-
> From: Koushik Das [mailto:koushik@citrix.com] 
> Sent: Thursday, October 17, 2013 9:23 AM
> To: 
> Subject: Re: Possible bug in DeploymentPlanner?
> 
> Prachi,
> 
> I am planning to add the following code in findSuitablePoolsForVolumes() 
> method. If the VM has a local disk and the cluster doesn't have a local pool 
> then the cluster will be added to avoid set. Let me know if you see any 
> issues with this.
> 
>if (useLocalStorage) {
>List allPoolsInCluster = 
> _storagePoolDao.findPoolsByTags(plan.getDataCenterId(), plan.getPodId(), 
> plan.getClusterId(), null);
>boolean localStoragePoolExists = false;
>for (StoragePoolVO pool : allPoolsInCluster) {
>if (pool.isLocal()) {
>localStoragePoolExists = true;
>break;
>}
>}
>if (!localStoragePoolExists) {
>avoid.addCluster(plan.getClusterId());
>}
>}
> 
> -Koushik
> 
> On 17-Oct-2013, at 3:21 AM, Prachi Damle 
> mailto:prachi.da...@citrix.com>> wrote:
> 
> 
> 
> -Original Message-
> From: Koushik Das [mailto:koushik@citrix.com]
> Sent: Wednesday, October 16, 2013 5:21 AM
> To: mailto:dev@cloudstack.apache.org>>
> Subject: Re: Possible bug in DeploymentPlanner?
> 
> 
> On 16-Oct-2013, at 3:12 AM, Prachi Damle 
> mailto:prachi.da...@citrix.com>> wrote:
> 
> 
> -Original Message-
> From: Koushik Das [mailto:koushik@citrix.com]
> Sent: Tuesday, October 15, 2013 11:43 AM
> To: mailto:dev@cloudstack.apache.org>>
> Subject: Re: Possible bug in DeploymentPlanner?
> 
> 
> Thanks for the explanation Prachi.
> 
> Here is one issue I see with what you explained below. If an allocator is not 
> suppose to handle a particular scenario how should it deal with the avoid 
> set. Should it put all available pools in the avoid set. In that case 
> subsequent allocators won't get a chance to do anything as all pools are in 
> avoid set. If more than one allocator can operate on a particular pool then 
> the situation becomes tricky.
> 
> To give an example, if there is a request for volume on shared storage and 
> first allocator to get called is the local allocator. Since it is not going 
> to handle this case it will return empty/null list. Now should it put all the 
> other shared pools in avoid set or only the local pools in avoid set? 
> Similarly if the request is for a local volume and the request first goes to 
> zone or cluster-wide storage allocator what should be there in the avoid set?
> 
> 
> [Prachi] I see the bug you are thinking about - deploymentPlanningManager :: 
> canAvoidCluster  is only checking the shared storages and thus does not put 
> the cluster in avoid set in case the local storage is not a match for the 
> host.
> 
> I think the storage allocators should put the resources not considered, in 
> avoid set, that are within their scope. Also they should do so only when they 
> are supposed to handle the request. Thus local allocator sees only local 
> pools, so it should place them into avoid set if not considered and only when 
> this allocator is supposed to handle the scenario. While the 
> zone-wide/cluster-wide should set those within their scope.
> Also, the allocators have a defined order in which they get invoked and we 
> need to maintain the order defined in the componentContext.xml
> 
> Now to solve the bug, the deploymentPlanningManager :: canAvoidCluster  needs 
> to consider which scenario applies for the

CLVM broken on master

Just posting this to dev for visibility.

I think commit 180cfa19 broke CLVM primary storage for KVM. I'm
failing VM deploy from template. I've been building a 'sanity check'
test that focuses on the KVM specific suff (tests storage types and
supported host OS for now), and this bubbled up.

Read more at:  https://issues.apache.org/jira/browse/CLOUDSTACK-4887

Re: Review Request 14522: [CLOUDSTACK-4771] Support Revert VM Disk from Snapshot



> On Oct. 17, 2013, 3:18 p.m., John Burwell wrote:
> > engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/DataMotionServiceImpl.java,
> >  line 62
> > 
> >
> > Add context information to error message to help operators and users 
> > determine what couldn't be snapshotted.
> 
> Chris Suich wrote:
> Fair enough - I just copied these existing error messages. What kind of 
> context would you like to see in this error message? The src & dest 
> datastores, files, etc.? I agree that there needs to be context, but this 
> would potentially be a user facing error, so I also don't want to overload 
> them with context that means nothing to them.
> 
> John Burwell wrote:
> Enough details so an operator/admin could look up the associated 
> volume/snapshot and debug the issue.  Likely a good start would be to include 
> the identifying information of the srcData and destData parameters.  Ask 
> yourself, "What I need to know as an operator/administrator to debug this 
> problem?" and put that information in the error message.


> On Oct. 17, 2013, 3:18 p.m., John Burwell wrote:
> > engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/DataMotionServiceImpl.java,
> >  line 79
> > 
> >
> > DRY out this code (e.g. extract to a private method).  This rules needs 
> > to be consolidated + add the context information mentioned above.
> 
> Chris Suich wrote:
> The two invocations of pickStrategy() are actually calling an overloaded 
> function. I'm not sure how we could DRY this out since they method 
> calls/parameter lists are actually different.
> 
> John Burwell wrote:
> I missed the overload.  The actual issue, in my mind, is that 
> pickStrategy needs to be DRY'ed out.  This likely requires a new interface 
> which is outside the bounds of this patch.  However, it is technical debt 
> that we somehow need to surface.  As such, please open a ticket to describing 
> issue and put a TODO on pickStrategy to consolidate the implementations with 
> a reference to the ticket.  Also, place a TODO on this lines to reference the 
> future refactoring.


- Chris


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14522/#review27126
---


On Oct. 17, 2013, 6:46 p.m., Chris Suich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14522/
> ---
> 
> (Updated Oct. 17, 2013, 6:46 p.m.)
> 
> 
> Review request for cloudstack, Brian Federle, edison su, John Burwell, and 
> Mike Tutkowski.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> After the last batch of work to the revertSnapshot API, SnapshotServiceImpl 
> was not tied into the workflow to be used by storage providers. I have added 
> the logic in a similar fashion to takeSnapshot(), backupSnapshot() and 
> deleteSnapshot().
> 
> I have also added a 'Revert to Snapshot' action to the volume snapshots list 
> in the UI.
> 
> 
> Diffs
> -
> 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/StrategyPriority.java
>  81034b1 
>   
> engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/DataMotionServiceImpl.java
>  2d31320 
>   server/src/com/cloud/api/ApiResponseHelper.java f4ca112 
>   server/src/com/cloud/storage/snapshot/SnapshotManagerImpl.java dade983 
> 
> Diff: https://reviews.apache.org/r/14522/diff/
> 
> 
> Testing
> ---
> 
> I have tested all of this locally with a custom storage provider.
> 
> Unfortunately, I'm still in the middle of figuring out how to properly unit 
> test this type of code. If anyone has any recommendations, please let me know.
> 
> 
> Thanks,
> 
> Chris Suich
> 
>

Re: CLVM broken on master

Oh, I noticed this and created a fix, which I thought I already had submitted 
since it was a part of the storage refactoring a couple weeks back. I'll post 
the patch for review now.

-- 
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Oct 17, 2013, at 2:51 PM, Marcus Sorensen  wrote:

> Just posting this to dev for visibility.
> 
> I think commit 180cfa19 broke CLVM primary storage for KVM. I'm
> failing VM deploy from template. I've been building a 'sanity check'
> test that focuses on the KVM specific suff (tests storage types and
> supported host OS for now), and this bubbled up.
> 
> Read more at:  https://issues.apache.org/jira/browse/CLOUDSTACK-4887

Review Request 14715: [CLOUDSTACK-4887] CLVM broken


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14715/
---

Review request for cloudstack, edison su and Marcus Sorensen.


Repository: cloudstack-git


Description
---

Fixed a bug with backup snapshot condition logic causing snapshots to not be 
backed up and the API to fail.


Diffs
-

  core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java 
002143f 

Diff: https://reviews.apache.org/r/14715/diff/


Testing
---


Thanks,

Chris Suich

RE: Possible bug in DeploymentPlanner?

2013-10-17 Thread Prachi Damle


-Original Message-
From: Koushik Das [mailto:koushik@citrix.com] 
Sent: Thursday, October 17, 2013 11:50 AM
To: 
Subject: Re: Possible bug in DeploymentPlanner?


On 17-Oct-2013, at 11:29 PM, Prachi Damle 
 wrote:

> Koushik,
> 
> I don't think you should add this check before calling allocators. It is the 
> allocators output that should tell that local storage pools were not found. 
> For your case where you need to put cluster in avoid set when no local pool 
> is present:
> a) Only local storage allocator should get invoked, all other allocators need 
> to have a check at the start saying they can process only 'shared' storage 
> request - This will make sure you do not get any pools from any other 
> allocator for your local storage request.

[Koushik] I made the changes for this and after this the planner bug got 
uncovered where it went in a infinite loop trying to deploy VM with local disk 
when no local pool is available.
 
> b) canAvoidCluster() need to be changed to compare only 'local' or 'shared' 
> pool set to the avoid output of the allocators to see if all pools are in 
> avoid set - Or if there are no pools then  directly avoid the cluster.

[Koushik] Ok. I will make the necessary changes for this but this closely ties 
the allocators and the assumption in canAvoidCluster(). The scenario I am 
seeing is the latter where there is no pools in the allocator avoid set. So as 
you suggested in this case as well the cluster needs to be in avoid set.

[Prachi] Cool thanks Koushik, this way all the logic to figure out a cluster 
should be avoided gets handled by canAvoidCluster(), and not in various places. 
 
> Prachi
> 
> -Original Message-
> From: Koushik Das [mailto:koushik@citrix.com] 
> Sent: Thursday, October 17, 2013 9:23 AM
> To: 
> Subject: Re: Possible bug in DeploymentPlanner?
> 
> Prachi,
> 
> I am planning to add the following code in findSuitablePoolsForVolumes() 
> method. If the VM has a local disk and the cluster doesn't have a local pool 
> then the cluster will be added to avoid set. Let me know if you see any 
> issues with this.
> 
>if (useLocalStorage) {
>List allPoolsInCluster = 
> _storagePoolDao.findPoolsByTags(plan.getDataCenterId(), plan.getPodId(), 
> plan.getClusterId(), null);
>boolean localStoragePoolExists = false;
>for (StoragePoolVO pool : allPoolsInCluster) {
>if (pool.isLocal()) {
>localStoragePoolExists = true;
>break;
>}
>}
>if (!localStoragePoolExists) {
>avoid.addCluster(plan.getClusterId());
>}
>}
> 
> -Koushik
> 
> On 17-Oct-2013, at 3:21 AM, Prachi Damle 
> mailto:prachi.da...@citrix.com>> wrote:
> 
> 
> 
> -Original Message-
> From: Koushik Das [mailto:koushik@citrix.com]
> Sent: Wednesday, October 16, 2013 5:21 AM
> To: mailto:dev@cloudstack.apache.org>>
> Subject: Re: Possible bug in DeploymentPlanner?
> 
> 
> On 16-Oct-2013, at 3:12 AM, Prachi Damle 
> mailto:prachi.da...@citrix.com>> wrote:
> 
> 
> -Original Message-
> From: Koushik Das [mailto:koushik@citrix.com]
> Sent: Tuesday, October 15, 2013 11:43 AM
> To: mailto:dev@cloudstack.apache.org>>
> Subject: Re: Possible bug in DeploymentPlanner?
> 
> 
> Thanks for the explanation Prachi.
> 
> Here is one issue I see with what you explained below. If an allocator is not 
> suppose to handle a particular scenario how should it deal with the avoid 
> set. Should it put all available pools in the avoid set. In that case 
> subsequent allocators won't get a chance to do anything as all pools are in 
> avoid set. If more than one allocator can operate on a particular pool then 
> the situation becomes tricky.
> 
> To give an example, if there is a request for volume on shared storage and 
> first allocator to get called is the local allocator. Since it is not going 
> to handle this case it will return empty/null list. Now should it put all the 
> other shared pools in avoid set or only the local pools in avoid set? 
> Similarly if the request is for a local volume and the request first goes to 
> zone or cluster-wide storage allocator what should be there in the avoid set?
> 
> 
> [Prachi] I see the bug you are thinking about - deploymentPlanningManager :: 
> canAvoidCluster  is only checking the shared storages and thus does not put 
> the cluster in avoid set in case the local storage is not a match for the 
> host.
> 
> I think the storage allocators should put the resources not considered, in 
> avoid set, that are within their scope. Also they should do so only when they 
> are supposed to handle the request. Thus local allocator sees only local 
> pools, so it should place them into avoid set if not considered and only when 
> this allocator is supposed to handle the scenario. While the 
>

Re: CLVM broken on master

Ok, let me test it.

On Thu, Oct 17, 2013 at 12:56 PM, SuichII, Christopher
 wrote:
> Oh, I noticed this and created a fix, which I thought I already had submitted 
> since it was a part of the storage refactoring a couple weeks back. I'll post 
> the patch for review now.
>
> --
> Chris Suich
> chris.su...@netapp.com
> NetApp Software Engineer
> Data Center Platforms – Cloud Solutions
> Citrix, Cisco & Red Hat
>
> On Oct 17, 2013, at 2:51 PM, Marcus Sorensen  wrote:
>
>> Just posting this to dev for visibility.
>>
>> I think commit 180cfa19 broke CLVM primary storage for KVM. I'm
>> failing VM deploy from template. I've been building a 'sanity check'
>> test that focuses on the KVM specific suff (tests storage types and
>> supported host OS for now), and this bubbled up.
>>
>> Read more at:  https://issues.apache.org/jira/browse/CLOUDSTACK-4887
>

Re: CLVM broken on master

Actually, I don't think that will fix this (though it probably fixes
something :-)

The issue I'm having is that we went from 'if source is a template on
nfs and destination is primary storage' to 'if source is a template
and destination is a template on primary storage'. We aren't copying
'template on secondary' -> 'template on primary', with CLVM we copy
'template on secondary' -> 'root disk on primary', since it's wasteful
and slow to copy a thin template (say a 50G img of size 500MB) to a
template on primary that's 50G, and then copy that 50G primary
template to another 50G primary root disk, since the primary storage
is neither thin nor clone-able.

On Thu, Oct 17, 2013 at 1:16 PM, Marcus Sorensen  wrote:
> Ok, let me test it.
>
> On Thu, Oct 17, 2013 at 12:56 PM, SuichII, Christopher
>  wrote:
>> Oh, I noticed this and created a fix, which I thought I already had 
>> submitted since it was a part of the storage refactoring a couple weeks 
>> back. I'll post the patch for review now.
>>
>> --
>> Chris Suich
>> chris.su...@netapp.com
>> NetApp Software Engineer
>> Data Center Platforms – Cloud Solutions
>> Citrix, Cisco & Red Hat
>>
>> On Oct 17, 2013, at 2:51 PM, Marcus Sorensen  wrote:
>>
>>> Just posting this to dev for visibility.
>>>
>>> I think commit 180cfa19 broke CLVM primary storage for KVM. I'm
>>> failing VM deploy from template. I've been building a 'sanity check'
>>> test that focuses on the KVM specific suff (tests storage types and
>>> supported host OS for now), and this bubbled up.
>>>
>>> Read more at:  https://issues.apache.org/jira/browse/CLOUDSTACK-4887
>>

Re: CLVM broken on master

Hm, interesting.

Since nothing else in the if/else if series there depends on the src being a 
template, I'd imagine it would be safe to just have the check be:

} else if (srcData.getObjectType() == DataObjectType.TEMPLATE && 
destDataStore.getRole() == DataStoreRole.Primary) {

In hindsight, adding the check for the destination being a template was just 
overkill and shouldn't have been added. So, if that fixes your problem, I 
believe it is in line with that Edison and I were doing with the storage 
subsystem, however, we should check with him as well.

--
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Oct 17, 2013, at 3:29 PM, Marcus Sorensen 
mailto:shadow...@gmail.com>> wrote:

Actually, I don't think that will fix this (though it probably fixes
something :-)

The issue I'm having is that we went from 'if source is a template on
nfs and destination is primary storage' to 'if source is a template
and destination is a template on primary storage'. We aren't copying
'template on secondary' -> 'template on primary', with CLVM we copy
'template on secondary' -> 'root disk on primary', since it's wasteful
and slow to copy a thin template (say a 50G img of size 500MB) to a
template on primary that's 50G, and then copy that 50G primary
template to another 50G primary root disk, since the primary storage
is neither thin nor clone-able.

On Thu, Oct 17, 2013 at 1:16 PM, Marcus Sorensen 
mailto:shadow...@gmail.com>> wrote:
Ok, let me test it.

On Thu, Oct 17, 2013 at 12:56 PM, SuichII, Christopher
mailto:chris.su...@netapp.com>> wrote:
Oh, I noticed this and created a fix, which I thought I already had submitted 
since it was a part of the storage refactoring a couple weeks back. I'll post 
the patch for review now.

--
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Oct 17, 2013, at 2:51 PM, Marcus Sorensen  wrote:

Just posting this to dev for visibility.

I think commit 180cfa19 broke CLVM primary storage for KVM. I'm
failing VM deploy from template. I've been building a 'sanity check'
test that focuses on the KVM specific suff (tests storage types and
supported host OS for now), and this bubbled up.

Read more at:  https://issues.apache.org/jira/browse/CLOUDSTACK-4887

Re: [MERGE] txn-refactor



Sent from my iPhone

> On 17 okt. 2013, at 20:12, Darren Shepherd  
> wrote:
> 
>> On 10/17/2013 01:53 AM, Hugo Trippaers wrote:
>> Hey Darren,
>> 
>> Looking through the code it looks like this more an API change than an 
>> actual redesign of the transaction code? I like the resulting code a lot 
>> better than the existing way of doing it. As far as i can see you wrapped 
>> the existing TransactionLegacy way of doing it (txn.start / txn.commit) 
>> inside of the new Transaction functions execute and executeWithException. So 
>> if i understand it correctly, nothing changed in how transactions are 
>> actually handled, except that the code can now be easily changed to use 
>> spring TX.
>> 
> Yes the purpose of this is to introduce a new API that will be compatible 
> with Spring TX.  The internals are all still custom ACS transaction.  This is 
> why I'm comfortable with this change, because nothing changed too much.

Nice, that also makes it a lot easier to review the changes!
> 
> But know I do need to make this change because I found that people were 
> inconsistently using the old API and it was causing problems with the spring 
> modularization branch.  In the spring modularization branch I've moved the 
> AOP logic for the transactions from custom ACS AOP to Spring AOP which has a 
> slightly different semantics.
> 
> Also note I found tons of bugs in the transaction handling while doing this.  
> Having the anonymous class style has made error handling more consistent.  
> But, there a tons of places that people are catching Throwable/Exception 
> where they shouldn't.  That's a different discussion, I need to put together 
> some thoughts on more consistent error handling in ACS.
> 
>> Also you made that changes to a couple of classes to use the new api, but 
>> the majority of the classes still need to be done. It might be nice to 
>> annotate the TransactionLegacy class with @Deprecated so we can easily 
>> identify what needs to be done?
> 
> I updated all of the management non-DAO code.  AWS, Usage and DAOs are still 
> using the old API.  I didn't touch DAOs because its not worth it in my mind.  
> DAOs should move to declarative transaction managment which will mean 
> basically the whole method will be under a transactions.  So I didn't want to 
> convert the code to the anonymous class style, and then when I do declarative 
> transactions go and change all the DAOs again.

Sounds like a solid approach. Works for me.
> 
> I don't know if I want to actually put @Deprecated on TransactionLegacy yet 
> because there are things you can do with it that you can't do in the new API. 
>  Namely, prepared statements and switching databases.

Maybe just mark start() as deprecated then. Would at least put a marker for 
anybody writing new code that they should think again about using it.

> 
>> 
>> The new code is not covered by any new unit test yet? I couldn't check the 
>> cobertura result yet as there are some build and test failures. Can you have 
>> a look at this build : 
>> http://jenkins.buildacloud.org/job/cloudstack-maven-build-with-branch-parameter/3/.
>>   You can kick off that job yourself from jenkins if you like to do the full 
>> test. Didn't do the noredist build test yet as the normal build failed.
>> 
> 
> I can write a unit test that tests the new code.  I started by altering the 
> existing transaction unit test, but then later figured out they don't work 
> and are not being ran as part of the build.  So I just did a bunch of manual 
> testing.

A unit test would be really nice to have for this piece of code. Especially now 
we know there will be changes is this area for some time to come. The DB layer 
is at the core of CloudStack so a test is a real requirement here. I know there 
is a bunch of stuff disabled, we decided long ago to fix those tests when we 
would touch that bit of code, and you just hit the jackpot ;-)

Did you have a look at the build link? The current build for your branch 
appears broken. One test failure and a compile error as far as I can tell.

> 
> Darren

Cheers,

Hugo

Re: Cloudstack collab Hackathons

That's a shame, but I could check if I can setup some teleconference gear. If 
you are interested. Though the time might be a serious bother depending on your 
timezone.

Cheers,

Hugo 

Sent from my iPhone

> On 17 okt. 2013, at 17:39, Marcus Sorensen  wrote:
> 
> I won't be this time.
> 
> On Thu, Oct 17, 2013 at 9:35 AM, Darren Shepherd
>  wrote:
>> Marcus,
>> 
>> Will you be at CCC?  I think it's immensely important and can give plenty of 
>> good reasons, but too lazy to type at the moment.  And I need to do a bit 
>> more analysis on the scope.
>> 
>> Darren
>> 
>>> On Oct 17, 2013, at 8:19 AM, Marcus Sorensen  wrote:
>>> 
>>> I'm not at all against it, I just haven't heard anyone give any reason
>>> as to why.  Keep in mind that we'd essentially be asking anyone who
>>> has written code for the agent to re-do their work (midonet, vxlan,
>>> storage plugins).  My impression is that people just like the 'feel'
>>> of it being in Python, and for good reason, but that ignores the fact
>>> that 1) we've already done the work, and 2) features still require
>>> java knowledge since the agent doesn't do anything the mgmt server
>>> doesn't ask for.  Without anyone explaining why, it just sort of feels
>>> like rearranging deck chairs on a boat with serious leaks going on
>>> below deck. Worse, people brought their own chairs, and we threw them
>>> overboard because they're now the wrong color.
>>> 
>>> On Wed, Oct 16, 2013 at 6:54 PM, Darren Shepherd
>>>  wrote:
 +1 for kvm in python.  I'd like to take it a step further.  I'd like to 
 create a framework for adding compute and storage with python drivers. The 
 first implementation should just happen to be libvirt/kvm.  So yeah, I'll 
 be interested in this.  I'll be landing around 10am so don't know what 
 time I'll get to the hackathon.
 
 Darren
 
>>> On Oct 14, 2013, at 9:29 AM, Marcus Sorensen  
>>> wrote:
>>> 
>>> On Mon, Oct 14, 2013 at 8:22 AM, Sebastien Goasguen  
>>> wrote:
>>> 
>>> On Oct 11, 2013, at 4:43 AM, Hugo Trippaers  wrote:
>>> 
>>> Hey Guys,
>>> 
>>> The CloudStack collaboration conference is right around the corner. The 
>>> first day of the conference will be dedicated to workshops and a 
>>> hackathon.
>>> 
>>> I'm curious which developers are planning to attend the hackthon and 
>>> what subjects you are interested in. In Santa Clara we had a short list 
>>> of some subjects to discuss and some tables and discussions were 
>>> already prepared in advance. In the upcoming conference we can do the 
>>> same, so if you have a discussion or project idea that you want to work 
>>> on at the hackthon let us know with a reply in this thread.
>>> 
>>> I'm also curious if there are any aspiring developers that would like 
>>> to have a sort of introduction into cloudstack development during the 
>>> hackathon.
>>> 
>>> If you have any other ideas, just shout.
>>> 
>>> Cheers,
>>> 
>>> Hugo
>> 
>> I am adding couple topics that I would like to see:
>> 
>> -API interfaces (AWS refactor, GCE, OCCI&CIMI standard): Discuss the 
>> state of our interfaces, plans for future, needs etc. AWS interface 
>> might need a refactor, GCE is a new interface, OCCI is a standard and 
>> Isaac Chiang has developed an interface. We are missing a CIMI interface.
>> 
>> -DOCs: We are having lots of talks about docs, they have been split in a 
>> separate repo, we need to discuss format, release life cycle, format etc.
>> 
>> -KVM agent: There has been discussions/wishes to re-write the KVM agent 
>> in something else than Java. Review architecture, define a plan, find 
>> developers :)
> 
> What? First I've heard of it!  I'm not sure how I feel about that. On
> one side, it seems reasonable to do it in something like python and
> remove the java dependency on the KVM host. On the other hand, there's
> a lot of technical debt in what's been built already, while it would
> probably be almost trivial to set up a basic working agent, it seems
> like a significant amount of work to transfer all of the
> functionality, special code that works around libvirt bugs, etc. Not
> to mention agent plugins that have already had a significant amount of
> effort put into them. On top of that, I'm not sure there's a whole lot
> of value if the motive is to attract non-java guys or admins, when the
> agent's features rely on what the mgmt server can do (new
> features/capabilities would require java code anyway, or at the very
> least coordinated effort between multiple devs who want the same
> feature).
> 
>> 
>> -Ecosystem: The are lots of tools in the cloud ecosystem, we should talk 
>> about docker, ansible, cloud foundry/bosh…etc.and define a plan to have 
>> great cloudstack support in all of

RE: Incomplete API docs?

2013-10-17 Thread Animesh Chaturvedi


+1

Radhika on the 4.3 release page I have solicited volunteers for different 
release activities. Looks like the high level tasks could use a better 
breakdown, do you want to take a stab at breaking doc into sub tasks and folks 
interested can fill in their name.

Animesh

> -Original Message-
> From: Radhika Puthiyetath [mailto:radhika.puthiyet...@citrix.com]
> Sent: Wednesday, October 16, 2013 3:24 AM
> To: dev@cloudstack.apache.org
> Subject: RE: Incomplete API docs?
> 
> Hi All,
> 
> Our API documentation requires a serious refurbishment.
> 
> How about creating a doc sprint for enhancing API documentation ?
> 
> I am in the process of setting up a Wiki page at
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Documentation+Spr
> int+for+Enhancing+API+Documentation to track this activity. Feel free to
> add/ edit/ remove the page content.
> 
> I request all of you to volunteer so that we will have a comprehensive
> API doc set by next release.
> 
> Appreciate your feedback.
> 
> Regards
> -Radhika
> 
> 
> 
> -Original Message-
> From: Animesh Chaturvedi [mailto:animesh.chaturv...@citrix.com]
> Sent: Wednesday, October 16, 2013 7:36 AM
> To: dev@cloudstack.apache.org
> Subject: RE: Incomplete API docs?
> 
> Demetrius do you mind creating an issue in JIRA?
> 
> > -Original Message-
> > From: Alena Prokharchyk [mailto:alena.prokharc...@citrix.com]
> > Sent: Tuesday, October 15, 2013 5:15 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: Incomplete API docs?
> >
> > The commands.xml generated as a result of mvn build (mvn clean install
> > - P developer,systemvm -DskipTests), doesn't include the commands
> > mentioned by Demetrius. Looks like some regression bug in
> > ApiXmlDocWriter - it used to include all the commands in 3.0.x version
> > of the code.
> >
> > -Alena.
> >
> > From: Demetrius Tsitrelis
> > mailto:dtsitre...@live.com>>
> > Reply-To:
> "dev@cloudstack.apache.org"
> > mailto:dev@cloudstack.apache.org>>
> > Date: Tuesday, October 15, 2013 4:33 PM
> > To: "dev@cloudstack.apache.org"
> > mailto:dev@cloudstack.apache.org>>
> > Subject: Incomplete API docs?
> >
> >
> >
> > I grep'ed the source code and came up with a list of the APIs which
> > the UI uses.  That list is at the end of this message.
> > You can see that many of them (addNetscalerLoadBalancer, addVmwareDc,
> > etc.) are not in the generated API documentation which appears at
> > http://cloudstack.apache.org/docs/api/apidocs-4.2/TOC_Root_Admin.html
> >
> >
> > Is this something wrong with the input files or is a tool at fault?
> >
> >
> > Here's the list of APIs used by the UI:
> >
> > activateProject
> >
> > addAccountToProject
> >
> > addBaremetalDhcp
> >
> > addBaremetalPxeKickStartServer
> >
> > addCluster
> >
> > addHost
> >
> > addImageStore
> >
> > addIpToNic
> >
> > addLdapConfiguration
> >
> > addNetscalerLoadBalancer
> >
> > addNetworkServiceProvider
> >
> > addNicToVirtualMachine
> >
> > addRegion
> >
> > addTrafficType
> >
> > addUcsManager
> >
> > addVmwareDc
> >
> > addVpnUser
> >
> > archiveAlerts
> >
> > archiveEvents
> >
> > assignToGlobalLoadBalancerRule
> >
> > assignToLoadBalancerRule
> >
> > assignVirtualMachine
> >
> > associateIpAddress
> >
> > associateUcsProfileToBlade
> >
> > attachIso
> >
> > attachVolume
> >
> > authorizeSecurityGroupEgress
> >
> > authorizeSecurityGroupIngress
> >
> > cancelHostMaintenance
> >
> > cancelStorageMaintenance
> >
> > configureInternalLoadBalancerElement
> >
> > configureVirtualRouterElement
> >
> > copyIso
> >
> > copyTemplate
> >
> > createAccount
> >
> > createAffinityGroup
> >
> > createAutoScalePolicy
> >
> > createAutoScaleVmGroup
> >
> > createAutoScaleVmProfile
> >
> > createCondition
> >
> > createDiskOffering
> >
> > createDomain
> >
> > createEgressFirewallRule
> >
> > createFirewallRule
> >
> > createGlobalLoadBalancerRule
> >
> > createIpForwardingRule
> >
> > createLBHealthCheckPolicy
> >
> > createLBStickinessPolicy
> >
> > createLoadBalancer
> >
> > createLoadBalancerRule
> >
> > createNetwork
> >
> > createNetworkACL
> >
> > createNetworkACLList
> >
> > createNetworkOffering
> >
> > createPhysicalNetwork
> >
> > createPod
> >
> > createPortableIpRange
> >
> > createPortForwardingRule
> >
> > createPrivateGateway
> >
> > createProject
> >
> > createRemoteAccessVpn
> >
> > createSecondaryStagingStore
> >
> > createSecurityGroup
> >
> > createServiceOffering
> >
> > createSnapshot
> >
> > createSnapshotPolicy
> >
> > createStaticRoute
> >
> > createStorageNetworkIpRange
> >
> > createStoragePool
> >
> > createTags
> >
> > createTemplate
> >
> > createUser
> >
> > createVlanIpRange
> >
> > createVMSnapshot
> >
> > createVolume
> >
> > createVPC
> >
> > createVpnConnection
> >
> > createVpnCustomerGateway
> >
> > createVpnGateway
> >
> > createZone
> >
> > dedicateCluster
> >
> > dedicateGuestVlanRange
> >
> > dedicateHost
> >
> > dedicatePod
> >
> > dedicateP

Issues deploying with marvin

2013-10-17 Thread Will Stevens

When I run:

I am getting the following error:
[DEBUG] Executing command line: python ../marvin/marvin/deployDataCenter.py
-i devcloud.cfg
5Traceback (most recent call last):
  File "../marvin/marvin/deployDataCenter.py", line 612, in 
deploy.deploy()
  File "../marvin/marvin/deployDataCenter.py", line 597, in deploy
self.createZones(self.config.zones)
  File "../marvin/marvin/deployDataCenter.py", line 466, in createZones
self.createSecondaryStorages(zone.secondaryStorages, zoneId)
  File "../marvin/marvin/deployDataCenter.py", line 176, in
createSecondaryStorages
secondarycmd = addImageStore.addImageStoreCmd()
NameError: global name 'addImageStore' is not defined
[INFO]

[INFO] BUILD FAILURE
[INFO]


I have looked and I can not find where 'addImageStore' is defined.  I was
expecting it to be imported via the 'from cloudstackAPI import *' line, but
'addImageStore.py' is not in that directory.

I have run 'python setup.py install' again in 'tools/marvin' just to make
sure it had not changed.

I ran the following:
# python -c "import pkg_resources;
print(pkg_resources.get_distribution('marvin').version)"

Which returned the version number of: 0.1.0

What do I need to do to get my environment up to speed?

Thanks...

CLOUDSTACK-4828 remove nic fails if dhcp wasn't enabled in network offering

2013-10-17 Thread Patrick Miller

Hi all,

   Is CLOUDSTACK-4828  slated for the 4.2.1 update.

 We currently do not use the Virtual Routers, Our ACS world share a common
network infrastructure with the rest of lab servers.

 Having to run a clean up sql job each night is not elegant.


 Thanks for all the great work.

 Patrick

Re: Issues deploying with marvin

2013-10-17 Thread Will Stevens

Note: I pulled from master yesterday...


On Thu, Oct 17, 2013 at 4:28 PM, Will Stevens  wrote:

> When I run:
>
> I am getting the following error:
> [DEBUG] Executing command line: python
> ../marvin/marvin/deployDataCenter.py -i devcloud.cfg
> 5Traceback (most recent call last):
>   File "../marvin/marvin/deployDataCenter.py", line 612, in 
> deploy.deploy()
>   File "../marvin/marvin/deployDataCenter.py", line 597, in deploy
> self.createZones(self.config.zones)
>   File "../marvin/marvin/deployDataCenter.py", line 466, in createZones
> self.createSecondaryStorages(zone.secondaryStorages, zoneId)
>   File "../marvin/marvin/deployDataCenter.py", line 176, in
> createSecondaryStorages
> secondarycmd = addImageStore.addImageStoreCmd()
> NameError: global name 'addImageStore' is not defined
> [INFO]
> 
> [INFO] BUILD FAILURE
> [INFO]
> 
>
> I have looked and I can not find where 'addImageStore' is defined.  I was
> expecting it to be imported via the 'from cloudstackAPI import *' line, but
> 'addImageStore.py' is not in that directory.
>
> I have run 'python setup.py install' again in 'tools/marvin' just to make
> sure it had not changed.
>
> I ran the following:
> # python -c "import pkg_resources;
> print(pkg_resources.get_distribution('marvin').version)"
>
> Which returned the version number of: 0.1.0
>
> What do I need to do to get my environment up to speed?
>
> Thanks...
>

RE: Incomplete API docs?

2013-10-17 Thread Demetrius Tsitrelis

Do you still think there needs to be a bug filed for the missing APIs?

-Original Message-
From: Animesh Chaturvedi [mailto:animesh.chaturv...@citrix.com] 
Sent: Thursday, October 17, 2013 1:21 PM
To: dev@cloudstack.apache.org
Subject: RE: Incomplete API docs?


+1

Radhika on the 4.3 release page I have solicited volunteers for different 
release activities. Looks like the high level tasks could use a better 
breakdown, do you want to take a stab at breaking doc into sub tasks and folks 
interested can fill in their name.

Animesh

> -Original Message-
> From: Radhika Puthiyetath [mailto:radhika.puthiyet...@citrix.com]
> Sent: Wednesday, October 16, 2013 3:24 AM
> To: dev@cloudstack.apache.org
> Subject: RE: Incomplete API docs?
> 
> Hi All,
> 
> Our API documentation requires a serious refurbishment.
> 
> How about creating a doc sprint for enhancing API documentation ?
> 
> I am in the process of setting up a Wiki page at 
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Documentation+S
> pr
> int+for+Enhancing+API+Documentation to track this activity. Feel free 
> int+for+Enhancing+API+to
> add/ edit/ remove the page content.
> 
> I request all of you to volunteer so that we will have a comprehensive 
> API doc set by next release.
> 
> Appreciate your feedback.
> 
> Regards
> -Radhika
> 
> 
> 
> -Original Message-
> From: Animesh Chaturvedi [mailto:animesh.chaturv...@citrix.com]
> Sent: Wednesday, October 16, 2013 7:36 AM
> To: dev@cloudstack.apache.org
> Subject: RE: Incomplete API docs?
> 
> Demetrius do you mind creating an issue in JIRA?
> 
> > -Original Message-
> > From: Alena Prokharchyk [mailto:alena.prokharc...@citrix.com]
> > Sent: Tuesday, October 15, 2013 5:15 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: Incomplete API docs?
> >
> > The commands.xml generated as a result of mvn build (mvn clean 
> > install
> > - P developer,systemvm -DskipTests), doesn't include the commands 
> > mentioned by Demetrius. Looks like some regression bug in 
> > ApiXmlDocWriter - it used to include all the commands in 3.0.x 
> > version of the code.
> >
> > -Alena.
> >
> > From: Demetrius Tsitrelis
> > mailto:dtsitre...@live.com>>
> > Reply-To:
> "dev@cloudstack.apache.org"
> > mailto:dev@cloudstack.apache.org>>
> > Date: Tuesday, October 15, 2013 4:33 PM
> > To: "dev@cloudstack.apache.org"
> > mailto:dev@cloudstack.apache.org>>
> > Subject: Incomplete API docs?
> >
> >
> >
> > I grep'ed the source code and came up with a list of the APIs which 
> > the UI uses.  That list is at the end of this message.
> > You can see that many of them (addNetscalerLoadBalancer, 
> > addVmwareDc,
> > etc.) are not in the generated API documentation which appears at 
> > http://cloudstack.apache.org/docs/api/apidocs-4.2/TOC_Root_Admin.htm
> > l
> >
> >
> > Is this something wrong with the input files or is a tool at fault?
> >
> >
> > Here's the list of APIs used by the UI:
> >
> > activateProject
> >
> > addAccountToProject
> >
> > addBaremetalDhcp
> >
> > addBaremetalPxeKickStartServer
> >
> > addCluster
> >
> > addHost
> >
> > addImageStore
> >
> > addIpToNic
> >
> > addLdapConfiguration
> >
> > addNetscalerLoadBalancer
> >
> > addNetworkServiceProvider
> >
> > addNicToVirtualMachine
> >
> > addRegion
> >
> > addTrafficType
> >
> > addUcsManager
> >
> > addVmwareDc
> >
> > addVpnUser
> >
> > archiveAlerts
> >
> > archiveEvents
> >
> > assignToGlobalLoadBalancerRule
> >
> > assignToLoadBalancerRule
> >
> > assignVirtualMachine
> >
> > associateIpAddress
> >
> > associateUcsProfileToBlade
> >
> > attachIso
> >
> > attachVolume
> >
> > authorizeSecurityGroupEgress
> >
> > authorizeSecurityGroupIngress
> >
> > cancelHostMaintenance
> >
> > cancelStorageMaintenance
> >
> > configureInternalLoadBalancerElement
> >
> > configureVirtualRouterElement
> >
> > copyIso
> >
> > copyTemplate
> >
> > createAccount
> >
> > createAffinityGroup
> >
> > createAutoScalePolicy
> >
> > createAutoScaleVmGroup
> >
> > createAutoScaleVmProfile
> >
> > createCondition
> >
> > createDiskOffering
> >
> > createDomain
> >
> > createEgressFirewallRule
> >
> > createFirewallRule
> >
> > createGlobalLoadBalancerRule
> >
> > createIpForwardingRule
> >
> > createLBHealthCheckPolicy
> >
> > createLBStickinessPolicy
> >
> > createLoadBalancer
> >
> > createLoadBalancerRule
> >
> > createNetwork
> >
> > createNetworkACL
> >
> > createNetworkACLList
> >
> > createNetworkOffering
> >
> > createPhysicalNetwork
> >
> > createPod
> >
> > createPortableIpRange
> >
> > createPortForwardingRule
> >
> > createPrivateGateway
> >
> > createProject
> >
> > createRemoteAccessVpn
> >
> > createSecondaryStagingStore
> >
> > createSecurityGroup
> >
> > createServiceOffering
> >
> > createSnapshot
> >
> > createSnapshotPolicy
> >
> > createStaticRoute
> >
> > createStorageNetworkIpRange
> >
> > createStoragePool
> >
> > createTags
> >
> > createTemplate
> >
> >

RE: Cloudstack collab Hackathons

2013-10-17 Thread Donal Lafferty

An example of how to avoid Java can be found in the Hyper-V agent.  

Hyper-V experts tend to know C# better than Java, so hypervisor control is 
through an agent on the hypervisor written in C#.  To avoid porting the message 
bus, communications are over HTTP Request.  To avoid porting the command and 
answer objects, the agent uses weak typing.  Data is left in JSON container 
objects.

DL

 
> -Original Message-
> From: Koushik Das [mailto:koushik@citrix.com]
> Sent: 17 October 2013 16:39
> To: 
> Subject: Re: Cloudstack collab Hackathons
> 
> Agree with Marcus here.
> If we are planning to rewrite, I think that there should be some discussion
> around the possibility of making it a direct agent like XS/Vmware. If this is
> possible then installing the KVM agent on individual hosts will be eliminated.
> 
> 
> On 17-Oct-2013, at 8:49 PM, Marcus Sorensen 
> wrote:
> 
> > I'm not at all against it, I just haven't heard anyone give any reason
> > as to why.  Keep in mind that we'd essentially be asking anyone who
> > has written code for the agent to re-do their work (midonet, vxlan,
> > storage plugins).  My impression is that people just like the 'feel'
> > of it being in Python, and for good reason, but that ignores the fact
> > that 1) we've already done the work, and 2) features still require
> > java knowledge since the agent doesn't do anything the mgmt server
> > doesn't ask for.  Without anyone explaining why, it just sort of feels
> > like rearranging deck chairs on a boat with serious leaks going on
> > below deck. Worse, people brought their own chairs, and we threw them
> > overboard because they're now the wrong color.
> >
> > On Wed, Oct 16, 2013 at 6:54 PM, Darren Shepherd
> >  wrote:
> >> +1 for kvm in python.  I'd like to take it a step further.  I'd like to 
> >> create a
> framework for adding compute and storage with python drivers. The first
> implementation should just happen to be libvirt/kvm.  So yeah, I'll be
> interested in this.  I'll be landing around 10am so don't know what time I'll 
> get
> to the hackathon.
> >>
> >> Darren
> >>
> >>> On Oct 14, 2013, at 9:29 AM, Marcus Sorensen 
> wrote:
> >>>
>  On Mon, Oct 14, 2013 at 8:22 AM, Sebastien Goasguen
>  wrote:
> 
> > On Oct 11, 2013, at 4:43 AM, Hugo Trippaers 
> wrote:
> >
> > Hey Guys,
> >
> > The CloudStack collaboration conference is right around the corner.
> The first day of the conference will be dedicated to workshops and a
> hackathon.
> >
> > I'm curious which developers are planning to attend the hackthon and
> what subjects you are interested in. In Santa Clara we had a short list of 
> some
> subjects to discuss and some tables and discussions were already prepared in
> advance. In the upcoming conference we can do the same, so if you have a
> discussion or project idea that you want to work on at the hackthon let us
> know with a reply in this thread.
> >
> > I'm also curious if there are any aspiring developers that would like to
> have a sort of introduction into cloudstack development during the
> hackathon.
> >
> > If you have any other ideas, just shout.
> >
> > Cheers,
> >
> > Hugo
> 
>  I am adding couple topics that I would like to see:
> 
>  -API interfaces (AWS refactor, GCE, OCCI&CIMI standard): Discuss the
> state of our interfaces, plans for future, needs etc. AWS interface might
> need a refactor, GCE is a new interface, OCCI is a standard and Isaac Chiang
> has developed an interface. We are missing a CIMI interface.
> 
>  -DOCs: We are having lots of talks about docs, they have been split in a
> separate repo, we need to discuss format, release life cycle, format etc.
> 
>  -KVM agent: There has been discussions/wishes to re-write the KVM
>  agent in something else than Java. Review architecture, define a
>  plan, find developers :)
> >>>
> >>> What? First I've heard of it!  I'm not sure how I feel about that.
> >>> On one side, it seems reasonable to do it in something like python
> >>> and remove the java dependency on the KVM host. On the other hand,
> >>> there's a lot of technical debt in what's been built already, while
> >>> it would probably be almost trivial to set up a basic working agent,
> >>> it seems like a significant amount of work to transfer all of the
> >>> functionality, special code that works around libvirt bugs, etc. Not
> >>> to mention agent plugins that have already had a significant amount
> >>> of effort put into them. On top of that, I'm not sure there's a
> >>> whole lot of value if the motive is to attract non-java guys or
> >>> admins, when the agent's features rely on what the mgmt server can
> >>> do (new features/capabilities would require java code anyway, or at
> >>> the very least coordinated effort between multiple devs who want the
> >>> same feature).
> >>>
> 
>  -Ecosystem: The are lots of tools in the cloud ecosystem, we should t

Re: CLVM broken on master

I ran that through my tester, it didn't like it.  That actually kept
the system vms from starting. Since CopyCommand is used for both
template to template and template to primary, it seems that the
original template copy is fine but now this catches the case where the
source template is on primary and we are making a root disk.
copyTemplateToPrimaryStorage has:

if (!(imageStore instanceof NfsTO)) {
return new CopyCmdAnswer("unsupported protocol");
}

we should be calling 'cloneVolumeFromBaseTemplate', but the original
if statement is now too loose.  I'll play with it a bit and see if I
can suggest a solution that works.

2013-09-17 17:58:07,178 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-2:null) Request:Seq 1-829816935:  { Cmd ,
MgmtId: 52241639751, via: 1, Ver: v1, Flags: 100011,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"bf53a7c6-1fed-11e3-a1ff-000c29d82947","origUrl":"http://download.cloud.com/templates/4.2/systemvmtemplate-2013-06-12-master-kvm.qcow2.bz2","uuid":"bf53a7c6-1fed-11e3-a1ff-000c29d82947","id":3,"format":"QCOW2","accountId":1,"checksum":"6cea42b2633841648040becb588bd8f0","hvm":false,"displayText":"SystemVM
Template 
(KVM)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"8932daaf-272c-45c9-a078-d601dfc5ca56","id":1,"poolType":"Filesystem","host":"172.17.10.10","path":"/var/lib/libvirt/images","port":0}},"name":"routing-3","hypervisorType":"KVM"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0c15b340-228b-48f1-88c4-b717ad08d4e3","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"8932daaf-272c-45c9-a078-d601dfc5ca56","id":1,"poolType":"Filesystem","host":"172.17.10.10","path":"/var/lib/libvirt/images","port":0}},"name":"ROOT-1","size":0,"volumeId":2,"vmName":"s-1-VM","accountId":1,"format":"QCOW2","id":2,"hypervisorType":"KVM"}},"executeInSequence":false,"wait":0}}]
}

2013-09-17 17:58:07,179 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-2:null) Processing command:
org.apache.cloudstack.storage.command.CopyCommand
2013-09-17 17:58:07,179 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-2:null) Seq 1-829816935:  { Ans: , MgmtId:
52241639751, via: 1, Ver: v1, Flags: 10,
[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"unsupported
protocol","wait":0}}] }

On Thu, Oct 17, 2013 at 1:45 PM, SuichII, Christopher
 wrote:
> Hm, interesting.
>
> Since nothing else in the if/else if series there depends on the src being a 
> template, I'd imagine it would be safe to just have the check be:
>
> } else if (srcData.getObjectType() == DataObjectType.TEMPLATE && 
> destDataStore.getRole() == DataStoreRole.Primary) {
>
> In hindsight, adding the check for the destination being a template was just 
> overkill and shouldn't have been added. So, if that fixes your problem, I 
> believe it is in line with that Edison and I were doing with the storage 
> subsystem, however, we should check with him as well.
>
> --
> Chris Suich
> chris.su...@netapp.com
> NetApp Software Engineer
> Data Center Platforms – Cloud Solutions
> Citrix, Cisco & Red Hat
>
> On Oct 17, 2013, at 3:29 PM, Marcus Sorensen 
> mailto:shadow...@gmail.com>> wrote:
>
> Actually, I don't think that will fix this (though it probably fixes
> something :-)
>
> The issue I'm having is that we went from 'if source is a template on
> nfs and destination is primary storage' to 'if source is a template
> and destination is a template on primary storage'. We aren't copying
> 'template on secondary' -> 'template on primary', with CLVM we copy
> 'template on secondary' -> 'root disk on primary', since it's wasteful
> and slow to copy a thin template (say a 50G img of size 500MB) to a
> template on primary that's 50G, and then copy that 50G primary
> template to another 50G primary root disk, since the primary storage
> is neither thin nor clone-able.
>
> On Thu, Oct 17, 2013 at 1:16 PM, Marcus Sorensen 
> mailto:shadow...@gmail.com>> wrote:
> Ok, let me test it.
>
> On Thu, Oct 17, 2013 at 12:56 PM, SuichII, Christopher
> mailto:chris.su...@netapp.com>> wrote:
> Oh, I noticed this and created a fix, which I thought I already had submitted 
> since it was a part of the storage refactoring a couple weeks back. I'll post 
> the patch for review now.
>
> --
> Chris Suich
> chris.su...@netapp.com
> NetApp Software Engineer
> Data Center Platforms – Cloud Solutions
> Citrix, Cisco & Red Hat
>
> On Oct 17, 2013, at 2:51 PM, Marcus Sorensen  wrote:
>
> Just posting this to dev for visibility.
>
> I think commit 180cfa19 broke CLVM primary storage for KVM. I'm
> failing VM deploy from template. I've been building a 'sanity check'
> test that focuses on the KVM specific suff (tests storage types and
> supported host OS for now), and this bubbled up.
>
> Read mor

Re: CLVM broken on master

I think if we can change this line:

if ((srcData.getObjectType() == DataObjectType.TEMPLATE) &&
(destData.getObjectType() == DataObjectType.TEMPLATE &&
destData.getDataStore().getRole() == DataStoreRole.Primary)) {

to something like:

if (srcData.getObjectType() == DataObjectType.TEMPLATE &&
srcData.getDataStore().getRole() == DataStoreRole.Image &&
destData.getDataStore().getRole() == DataStoreRole.Primary) {

Maybe that will work? That way it's strictly secondary -> primary
templates, not primary->primary templates.

Alternatively we could put it back to where it was:

if (srcData.getObjectType() == DataObjectType.TEMPLATE && srcDataStore
instanceof NfsTO && destData.getDataStore().getRole() ==
DataStoreRole.Primary) {

But your patch on the reviewboard removes NfsTO, and I'm assuming the
idea was to work towards getting away from NFS-specific secondary
storage.

On Thu, Oct 17, 2013 at 2:57 PM, Marcus Sorensen  wrote:
> I ran that through my tester, it didn't like it.  That actually kept
> the system vms from starting. Since CopyCommand is used for both
> template to template and template to primary, it seems that the
> original template copy is fine but now this catches the case where the
> source template is on primary and we are making a root disk.
> copyTemplateToPrimaryStorage has:
>
> if (!(imageStore instanceof NfsTO)) {
> return new CopyCmdAnswer("unsupported protocol");
> }
>
> we should be calling 'cloneVolumeFromBaseTemplate', but the original
> if statement is now too loose.  I'll play with it a bit and see if I
> can suggest a solution that works.
>
> 2013-09-17 17:58:07,178 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) Request:Seq 1-829816935:  { Cmd ,
> MgmtId: 52241639751, via: 1, Ver: v1, Flags: 100011,
> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"bf53a7c6-1fed-11e3-a1ff-000c29d82947","origUrl":"http://download.cloud.com/templates/4.2/systemvmtemplate-2013-06-12-master-kvm.qcow2.bz2","uuid":"bf53a7c6-1fed-11e3-a1ff-000c29d82947","id":3,"format":"QCOW2","accountId":1,"checksum":"6cea42b2633841648040becb588bd8f0","hvm":false,"displayText":"SystemVM
> Template 
> (KVM)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"8932daaf-272c-45c9-a078-d601dfc5ca56","id":1,"poolType":"Filesystem","host":"172.17.10.10","path":"/var/lib/libvirt/images","port":0}},"name":"routing-3","hypervisorType":"KVM"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0c15b340-228b-48f1-88c4-b717ad08d4e3","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"8932daaf-272c-45c9-a078-d601dfc5ca56","id":1,"poolType":"Filesystem","host":"172.17.10.10","path":"/var/lib/libvirt/images","port":0}},"name":"ROOT-1","size":0,"volumeId":2,"vmName":"s-1-VM","accountId":1,"format":"QCOW2","id":2,"hypervisorType":"KVM"}},"executeInSequence":false,"wait":0}}]
> }
>
> 2013-09-17 17:58:07,179 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) Processing command:
> org.apache.cloudstack.storage.command.CopyCommand
> 2013-09-17 17:58:07,179 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) Seq 1-829816935:  { Ans: , MgmtId:
> 52241639751, via: 1, Ver: v1, Flags: 10,
> [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"unsupported
> protocol","wait":0}}] }
>
> On Thu, Oct 17, 2013 at 1:45 PM, SuichII, Christopher
>  wrote:
>> Hm, interesting.
>>
>> Since nothing else in the if/else if series there depends on the src being a 
>> template, I'd imagine it would be safe to just have the check be:
>>
>> } else if (srcData.getObjectType() == DataObjectType.TEMPLATE && 
>> destDataStore.getRole() == DataStoreRole.Primary) {
>>
>> In hindsight, adding the check for the destination being a template was just 
>> overkill and shouldn't have been added. So, if that fixes your problem, I 
>> believe it is in line with that Edison and I were doing with the storage 
>> subsystem, however, we should check with him as well.
>>
>> --
>> Chris Suich
>> chris.su...@netapp.com
>> NetApp Software Engineer
>> Data Center Platforms – Cloud Solutions
>> Citrix, Cisco & Red Hat
>>
>> On Oct 17, 2013, at 3:29 PM, Marcus Sorensen 
>> mailto:shadow...@gmail.com>> wrote:
>>
>> Actually, I don't think that will fix this (though it probably fixes
>> something :-)
>>
>> The issue I'm having is that we went from 'if source is a template on
>> nfs and destination is primary storage' to 'if source is a template
>> and destination is a template on primary storage'. We aren't copying
>> 'template on secondary' -> 'template on primary', with CLVM we copy
>> 'template on secondary' -> 'root disk on primary', since it's wasteful
>> and slow to copy a thin template (say a 50G img of size 500MB) to a
>> template on primary that's 50G, and then copy that 50G primary
>> template to another

RE: CLVM broken on master

2013-10-17 Thread Edison Su

For CLVM, the copy template from secondary to primary and create volume from 
template logic is handled by CloudStackPrimaryDataStoreDriverImpl->copyAsync, 
not in AncientDataMotionStrategy
You can check the code: 4fb459355337c874a10f47c0224af72d6fef1ff2.

> -Original Message-
> From: Marcus Sorensen [mailto:shadow...@gmail.com]
> Sent: Thursday, October 17, 2013 2:07 PM
> To: dev@cloudstack.apache.org
> Subject: Re: CLVM broken on master
> 
> I think if we can change this line:
> 
> if ((srcData.getObjectType() == DataObjectType.TEMPLATE) &&
> (destData.getObjectType() == DataObjectType.TEMPLATE &&
> destData.getDataStore().getRole() == DataStoreRole.Primary)) {
> 
> to something like:
> 
> if (srcData.getObjectType() == DataObjectType.TEMPLATE &&
> srcData.getDataStore().getRole() == DataStoreRole.Image &&
> destData.getDataStore().getRole() == DataStoreRole.Primary) {
> 
> Maybe that will work? That way it's strictly secondary -> primary templates,
> not primary->primary templates.
> 
> Alternatively we could put it back to where it was:
> 
> if (srcData.getObjectType() == DataObjectType.TEMPLATE && srcDataStore
> instanceof NfsTO && destData.getDataStore().getRole() ==
> DataStoreRole.Primary) {
> 
> But your patch on the reviewboard removes NfsTO, and I'm assuming the
> idea was to work towards getting away from NFS-specific secondary storage.
> 
> On Thu, Oct 17, 2013 at 2:57 PM, Marcus Sorensen 
> wrote:
> > I ran that through my tester, it didn't like it.  That actually kept
> > the system vms from starting. Since CopyCommand is used for both
> > template to template and template to primary, it seems that the
> > original template copy is fine but now this catches the case where the
> > source template is on primary and we are making a root disk.
> > copyTemplateToPrimaryStorage has:
> >
> > if (!(imageStore instanceof NfsTO)) {
> > return new CopyCmdAnswer("unsupported protocol");
> > }
> >
> > we should be calling 'cloneVolumeFromBaseTemplate', but the original
> > if statement is now too loose.  I'll play with it a bit and see if I
> > can suggest a solution that works.
> >
> > 2013-09-17 17:58:07,178 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-2:null) Request:Seq 1-829816935:  { Cmd ,
> > MgmtId: 52241639751, via: 1, Ver: v1, Flags: 100011,
> >
> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.
> a
> > pache.cloudstack.storage.to.TemplateObjectTO":{"path":"bf53a7c6-1fed-1
> > 1e3-a1ff-000c29d82947","origUrl":"http://download.cloud.com/templates/
> > 4.2/systemvmtemplate-2013-06-12-master-
> kvm.qcow2.bz2","uuid":"bf53a7c6
> > -1fed-11e3-a1ff-000c29d82947","id":3,"format":"QCOW2","accountId":1,"c
> >
> hecksum":"6cea42b2633841648040becb588bd8f0","hvm":false,"displayText":
> > "SystemVM Template
> > (KVM)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryData
> > StoreTO":{"uuid":"8932daaf-272c-45c9-a078-d601dfc5ca56","id":1,"poolTy
> > pe":"Filesystem","host":"172.17.10.10","path":"/var/lib/libvirt/images
> > ","port":0}},"name":"routing-3","hypervisorType":"KVM"}},"destTO":{"or
> > g.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0c15b340-228b-
> > 48f1-88c4-
> b717ad08d4e3","volumeType":"ROOT","dataStore":{"org.apache.c
> > loudstack.storage.to.PrimaryDataStoreTO":{"uuid":"8932daaf-272c-45c9-a
> > 078-d601dfc5ca56","id":1,"poolType":"Filesystem","host":"172.17.10.10"
> > ,"path":"/var/lib/libvirt/images","port":0}},"name":"ROOT-1","size":0,
> > "volumeId":2,"vmName":"s-1-
> VM","accountId":1,"format":"QCOW2","id":2,"
> > hypervisorType":"KVM"}},"executeInSequence":false,"wait":0}}]
> > }
> >
> > 2013-09-17 17:58:07,179 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-2:null) Processing command:
> > org.apache.cloudstack.storage.command.CopyCommand
> > 2013-09-17 17:58:07,179 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-2:null) Seq 1-829816935:  { Ans: , MgmtId:
> > 52241639751, via: 1, Ver: v1, Flags: 10,
> >
> [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":fals
> > e,"details":"unsupported
> > protocol","wait":0}}] }
> >
> > On Thu, Oct 17, 2013 at 1:45 PM, SuichII, Christopher
> >  wrote:
> >> Hm, interesting.
> >>
> >> Since nothing else in the if/else if series there depends on the src being 
> >> a
> template, I'd imagine it would be safe to just have the check be:
> >>
> >> } else if (srcData.getObjectType() == DataObjectType.TEMPLATE &&
> >> destDataStore.getRole() == DataStoreRole.Primary) {
> >>
> >> In hindsight, adding the check for the destination being a template was
> just overkill and shouldn't have been added. So, if that fixes your problem, I
> believe it is in line with that Edison and I were doing with the storage
> subsystem, however, we should check with him as well.
> >>
> >> --
> >> Chris Suich
> >> chris.su...@netapp.com
> >> NetApp Software Engineer
> >> Data Center Platforms - Cloud Solutions Citrix, Cisco & Red Hat
> >>
> >> On

Re: CLVM broken on master

Sure, but CopyCommand is being triggered in this code. I've tested
several variations to this one line, some work, some don't.

On Thu, Oct 17, 2013 at 3:38 PM, Edison Su  wrote:
> For CLVM, the copy template from secondary to primary and create volume from 
> template logic is handled by CloudStackPrimaryDataStoreDriverImpl->copyAsync, 
> not in AncientDataMotionStrategy
> You can check the code: 4fb459355337c874a10f47c0224af72d6fef1ff2.
>
>> -Original Message-
>> From: Marcus Sorensen [mailto:shadow...@gmail.com]
>> Sent: Thursday, October 17, 2013 2:07 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: CLVM broken on master
>>
>> I think if we can change this line:
>>
>> if ((srcData.getObjectType() == DataObjectType.TEMPLATE) &&
>> (destData.getObjectType() == DataObjectType.TEMPLATE &&
>> destData.getDataStore().getRole() == DataStoreRole.Primary)) {
>>
>> to something like:
>>
>> if (srcData.getObjectType() == DataObjectType.TEMPLATE &&
>> srcData.getDataStore().getRole() == DataStoreRole.Image &&
>> destData.getDataStore().getRole() == DataStoreRole.Primary) {
>>
>> Maybe that will work? That way it's strictly secondary -> primary templates,
>> not primary->primary templates.
>>
>> Alternatively we could put it back to where it was:
>>
>> if (srcData.getObjectType() == DataObjectType.TEMPLATE && srcDataStore
>> instanceof NfsTO && destData.getDataStore().getRole() ==
>> DataStoreRole.Primary) {
>>
>> But your patch on the reviewboard removes NfsTO, and I'm assuming the
>> idea was to work towards getting away from NFS-specific secondary storage.
>>
>> On Thu, Oct 17, 2013 at 2:57 PM, Marcus Sorensen 
>> wrote:
>> > I ran that through my tester, it didn't like it.  That actually kept
>> > the system vms from starting. Since CopyCommand is used for both
>> > template to template and template to primary, it seems that the
>> > original template copy is fine but now this catches the case where the
>> > source template is on primary and we are making a root disk.
>> > copyTemplateToPrimaryStorage has:
>> >
>> > if (!(imageStore instanceof NfsTO)) {
>> > return new CopyCmdAnswer("unsupported protocol");
>> > }
>> >
>> > we should be calling 'cloneVolumeFromBaseTemplate', but the original
>> > if statement is now too loose.  I'll play with it a bit and see if I
>> > can suggest a solution that works.
>> >
>> > 2013-09-17 17:58:07,178 DEBUG [cloud.agent.Agent]
>> > (agentRequest-Handler-2:null) Request:Seq 1-829816935:  { Cmd ,
>> > MgmtId: 52241639751, via: 1, Ver: v1, Flags: 100011,
>> >
>> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.
>> a
>> > pache.cloudstack.storage.to.TemplateObjectTO":{"path":"bf53a7c6-1fed-1
>> > 1e3-a1ff-000c29d82947","origUrl":"http://download.cloud.com/templates/
>> > 4.2/systemvmtemplate-2013-06-12-master-
>> kvm.qcow2.bz2","uuid":"bf53a7c6
>> > -1fed-11e3-a1ff-000c29d82947","id":3,"format":"QCOW2","accountId":1,"c
>> >
>> hecksum":"6cea42b2633841648040becb588bd8f0","hvm":false,"displayText":
>> > "SystemVM Template
>> > (KVM)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryData
>> > StoreTO":{"uuid":"8932daaf-272c-45c9-a078-d601dfc5ca56","id":1,"poolTy
>> > pe":"Filesystem","host":"172.17.10.10","path":"/var/lib/libvirt/images
>> > ","port":0}},"name":"routing-3","hypervisorType":"KVM"}},"destTO":{"or
>> > g.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0c15b340-228b-
>> > 48f1-88c4-
>> b717ad08d4e3","volumeType":"ROOT","dataStore":{"org.apache.c
>> > loudstack.storage.to.PrimaryDataStoreTO":{"uuid":"8932daaf-272c-45c9-a
>> > 078-d601dfc5ca56","id":1,"poolType":"Filesystem","host":"172.17.10.10"
>> > ,"path":"/var/lib/libvirt/images","port":0}},"name":"ROOT-1","size":0,
>> > "volumeId":2,"vmName":"s-1-
>> VM","accountId":1,"format":"QCOW2","id":2,"
>> > hypervisorType":"KVM"}},"executeInSequence":false,"wait":0}}]
>> > }
>> >
>> > 2013-09-17 17:58:07,179 DEBUG [cloud.agent.Agent]
>> > (agentRequest-Handler-2:null) Processing command:
>> > org.apache.cloudstack.storage.command.CopyCommand
>> > 2013-09-17 17:58:07,179 DEBUG [cloud.agent.Agent]
>> > (agentRequest-Handler-2:null) Seq 1-829816935:  { Ans: , MgmtId:
>> > 52241639751, via: 1, Ver: v1, Flags: 10,
>> >
>> [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":fals
>> > e,"details":"unsupported
>> > protocol","wait":0}}] }
>> >
>> > On Thu, Oct 17, 2013 at 1:45 PM, SuichII, Christopher
>> >  wrote:
>> >> Hm, interesting.
>> >>
>> >> Since nothing else in the if/else if series there depends on the src 
>> >> being a
>> template, I'd imagine it would be safe to just have the check be:
>> >>
>> >> } else if (srcData.getObjectType() == DataObjectType.TEMPLATE &&
>> >> destDataStore.getRole() == DataStoreRole.Primary) {
>> >>
>> >> In hindsight, adding the check for the destination being a template was
>> just overkill and shouldn't have been added. So, if that fixes your problem, 
>> I
>> believe it is in line with that Edison and I

Re: CLVM broken on master

All of the above mentioned is in
core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java
, by the way.

On Thu, Oct 17, 2013 at 4:02 PM, Marcus Sorensen  wrote:
> Sure, but CopyCommand is being triggered in this code. I've tested
> several variations to this one line, some work, some don't.
>
> On Thu, Oct 17, 2013 at 3:38 PM, Edison Su  wrote:
>> For CLVM, the copy template from secondary to primary and create volume from 
>> template logic is handled by 
>> CloudStackPrimaryDataStoreDriverImpl->copyAsync, not in 
>> AncientDataMotionStrategy
>> You can check the code: 4fb459355337c874a10f47c0224af72d6fef1ff2.
>>
>>> -Original Message-
>>> From: Marcus Sorensen [mailto:shadow...@gmail.com]
>>> Sent: Thursday, October 17, 2013 2:07 PM
>>> To: dev@cloudstack.apache.org
>>> Subject: Re: CLVM broken on master
>>>
>>> I think if we can change this line:
>>>
>>> if ((srcData.getObjectType() == DataObjectType.TEMPLATE) &&
>>> (destData.getObjectType() == DataObjectType.TEMPLATE &&
>>> destData.getDataStore().getRole() == DataStoreRole.Primary)) {
>>>
>>> to something like:
>>>
>>> if (srcData.getObjectType() == DataObjectType.TEMPLATE &&
>>> srcData.getDataStore().getRole() == DataStoreRole.Image &&
>>> destData.getDataStore().getRole() == DataStoreRole.Primary) {
>>>
>>> Maybe that will work? That way it's strictly secondary -> primary templates,
>>> not primary->primary templates.
>>>
>>> Alternatively we could put it back to where it was:
>>>
>>> if (srcData.getObjectType() == DataObjectType.TEMPLATE && srcDataStore
>>> instanceof NfsTO && destData.getDataStore().getRole() ==
>>> DataStoreRole.Primary) {
>>>
>>> But your patch on the reviewboard removes NfsTO, and I'm assuming the
>>> idea was to work towards getting away from NFS-specific secondary storage.
>>>
>>> On Thu, Oct 17, 2013 at 2:57 PM, Marcus Sorensen 
>>> wrote:
>>> > I ran that through my tester, it didn't like it.  That actually kept
>>> > the system vms from starting. Since CopyCommand is used for both
>>> > template to template and template to primary, it seems that the
>>> > original template copy is fine but now this catches the case where the
>>> > source template is on primary and we are making a root disk.
>>> > copyTemplateToPrimaryStorage has:
>>> >
>>> > if (!(imageStore instanceof NfsTO)) {
>>> > return new CopyCmdAnswer("unsupported protocol");
>>> > }
>>> >
>>> > we should be calling 'cloneVolumeFromBaseTemplate', but the original
>>> > if statement is now too loose.  I'll play with it a bit and see if I
>>> > can suggest a solution that works.
>>> >
>>> > 2013-09-17 17:58:07,178 DEBUG [cloud.agent.Agent]
>>> > (agentRequest-Handler-2:null) Request:Seq 1-829816935:  { Cmd ,
>>> > MgmtId: 52241639751, via: 1, Ver: v1, Flags: 100011,
>>> >
>>> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.
>>> a
>>> > pache.cloudstack.storage.to.TemplateObjectTO":{"path":"bf53a7c6-1fed-1
>>> > 1e3-a1ff-000c29d82947","origUrl":"http://download.cloud.com/templates/
>>> > 4.2/systemvmtemplate-2013-06-12-master-
>>> kvm.qcow2.bz2","uuid":"bf53a7c6
>>> > -1fed-11e3-a1ff-000c29d82947","id":3,"format":"QCOW2","accountId":1,"c
>>> >
>>> hecksum":"6cea42b2633841648040becb588bd8f0","hvm":false,"displayText":
>>> > "SystemVM Template
>>> > (KVM)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryData
>>> > StoreTO":{"uuid":"8932daaf-272c-45c9-a078-d601dfc5ca56","id":1,"poolTy
>>> > pe":"Filesystem","host":"172.17.10.10","path":"/var/lib/libvirt/images
>>> > ","port":0}},"name":"routing-3","hypervisorType":"KVM"}},"destTO":{"or
>>> > g.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"0c15b340-228b-
>>> > 48f1-88c4-
>>> b717ad08d4e3","volumeType":"ROOT","dataStore":{"org.apache.c
>>> > loudstack.storage.to.PrimaryDataStoreTO":{"uuid":"8932daaf-272c-45c9-a
>>> > 078-d601dfc5ca56","id":1,"poolType":"Filesystem","host":"172.17.10.10"
>>> > ,"path":"/var/lib/libvirt/images","port":0}},"name":"ROOT-1","size":0,
>>> > "volumeId":2,"vmName":"s-1-
>>> VM","accountId":1,"format":"QCOW2","id":2,"
>>> > hypervisorType":"KVM"}},"executeInSequence":false,"wait":0}}]
>>> > }
>>> >
>>> > 2013-09-17 17:58:07,179 DEBUG [cloud.agent.Agent]
>>> > (agentRequest-Handler-2:null) Processing command:
>>> > org.apache.cloudstack.storage.command.CopyCommand
>>> > 2013-09-17 17:58:07,179 DEBUG [cloud.agent.Agent]
>>> > (agentRequest-Handler-2:null) Seq 1-829816935:  { Ans: , MgmtId:
>>> > 52241639751, via: 1, Ver: v1, Flags: 10,
>>> >
>>> [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":fals
>>> > e,"details":"unsupported
>>> > protocol","wait":0}}] }
>>> >
>>> > On Thu, Oct 17, 2013 at 1:45 PM, SuichII, Christopher
>>> >  wrote:
>>> >> Hm, interesting.
>>> >>
>>> >> Since nothing else in the if/else if series there depends on the src 
>>> >> being a
>>> template, I'd imagine it would be safe to just have the check be:
>>> >>
>>> >> } else if (srcData.getObjectType() == DataObject

Re: CLVM broken on master

The following seems to fix the issue, by the way, but again since I
didn't initially change the code I'd like someone else to
review/handle it.

diff --git 
a/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java
b/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java
index 002143f..3ac82e3 100644
--- 
a/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java
+++ 
b/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java
@@ -68,7 +68,7 @@ public class StorageSubsystemCommandHandlerBase
implements StorageSubsystemComma
 DataStoreTO srcDataStore = srcData.getDataStore();
 DataStoreTO destDataStore = destData.getDataStore();

-if ((srcData.getObjectType() == DataObjectType.TEMPLATE) &&
(destData.getObjectType() == DataObjectType.TEMPLATE &&
destData.getDataStore().getRole() == DataStoreRole.Primary)) {
+if (srcData.getObjectType() == DataObjectType.TEMPLATE &&
srcData.getDataStore().getRole() == DataStoreRole.Image &&
destData.getDataStore().getRole() == DataStoreRole.Primary) {
 //copy template to primary storage
 return processor.copyTemplateToPrimaryStorage(cmd);
 } else if (srcData.getObjectType() == DataObjectType.TEMPLATE
&& srcDataStore.getRole() == DataStoreRole.Primary &&
destDataStore.getRole() == DataStoreRole.Primary) {

On Thu, Oct 17, 2013 at 4:03 PM, Marcus Sorensen  wrote:
> All of the above mentioned is in
> core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java
> , by the way.
>
> On Thu, Oct 17, 2013 at 4:02 PM, Marcus Sorensen  wrote:
>> Sure, but CopyCommand is being triggered in this code. I've tested
>> several variations to this one line, some work, some don't.
>>
>> On Thu, Oct 17, 2013 at 3:38 PM, Edison Su  wrote:
>>> For CLVM, the copy template from secondary to primary and create volume 
>>> from template logic is handled by 
>>> CloudStackPrimaryDataStoreDriverImpl->copyAsync, not in 
>>> AncientDataMotionStrategy
>>> You can check the code: 4fb459355337c874a10f47c0224af72d6fef1ff2.
>>>
 -Original Message-
 From: Marcus Sorensen [mailto:shadow...@gmail.com]
 Sent: Thursday, October 17, 2013 2:07 PM
 To: dev@cloudstack.apache.org
 Subject: Re: CLVM broken on master

 I think if we can change this line:

 if ((srcData.getObjectType() == DataObjectType.TEMPLATE) &&
 (destData.getObjectType() == DataObjectType.TEMPLATE &&
 destData.getDataStore().getRole() == DataStoreRole.Primary)) {

 to something like:

 if (srcData.getObjectType() == DataObjectType.TEMPLATE &&
 srcData.getDataStore().getRole() == DataStoreRole.Image &&
 destData.getDataStore().getRole() == DataStoreRole.Primary) {

 Maybe that will work? That way it's strictly secondary -> primary 
 templates,
 not primary->primary templates.

 Alternatively we could put it back to where it was:

 if (srcData.getObjectType() == DataObjectType.TEMPLATE && srcDataStore
 instanceof NfsTO && destData.getDataStore().getRole() ==
 DataStoreRole.Primary) {

 But your patch on the reviewboard removes NfsTO, and I'm assuming the
 idea was to work towards getting away from NFS-specific secondary storage.

 On Thu, Oct 17, 2013 at 2:57 PM, Marcus Sorensen 
 wrote:
 > I ran that through my tester, it didn't like it.  That actually kept
 > the system vms from starting. Since CopyCommand is used for both
 > template to template and template to primary, it seems that the
 > original template copy is fine but now this catches the case where the
 > source template is on primary and we are making a root disk.
 > copyTemplateToPrimaryStorage has:
 >
 > if (!(imageStore instanceof NfsTO)) {
 > return new CopyCmdAnswer("unsupported protocol");
 > }
 >
 > we should be calling 'cloneVolumeFromBaseTemplate', but the original
 > if statement is now too loose.  I'll play with it a bit and see if I
 > can suggest a solution that works.
 >
 > 2013-09-17 17:58:07,178 DEBUG [cloud.agent.Agent]
 > (agentRequest-Handler-2:null) Request:Seq 1-829816935:  { Cmd ,
 > MgmtId: 52241639751, via: 1, Ver: v1, Flags: 100011,
 >
 [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.
 a
 > pache.cloudstack.storage.to.TemplateObjectTO":{"path":"bf53a7c6-1fed-1
 > 1e3-a1ff-000c29d82947","origUrl":"http://download.cloud.com/templates/
 > 4.2/systemvmtemplate-2013-06-12-master-
 kvm.qcow2.bz2","uuid":"bf53a7c6
 > -1fed-11e3-a1ff-000c29d82947","id":3,"format":"QCOW2","accountId":1,"c
 >
 hecksum":"6cea42b2633841648040becb588bd8f0","hvm":false,"displayText":
 > "SystemVM Template
 > (KVM)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryData
 > StoreTO":{"uuid":"8932daaf-272c-45c9-a078-d601dfc5ca56"

Re: [MERGE] txn-refactor

On Thu, Oct 17, 2013 at 12:59 PM, Hugo Trippaers  wrote:
> Maybe just mark start() as deprecated then. Would at least put a marker for 
> anybody writing new code that they should think again about using it.

Good idea.

>
> A unit test would be really nice to have for this piece of code. Especially 
> now we know there will be changes is this area for some time to come. The DB 
> layer is at the core of CloudStack so a test is a real requirement here. I 
> know there is a bunch of stuff disabled, we decided long ago to fix those 
> tests when we would touch that bit of code, and you just hit the jackpot ;-)
>

I wrote some tests, I'll commit them in a bit.

> Did you have a look at the build link? The current build for your branch 
> appears broken. One test failure and a compile error as far as I can tell.

I had no clue that build-with-branch jenkins jobs existed!  That is
very useful.  How do I get an account to kick off jobs?  I do feel
stupid now looking at the build error.  I had some maven projects
disabled, so I forgot to update about 10 different projects.  I'll fix
those real quick and commit.

Darren

regions

I'm looking for more information on use cases for regions. I've been through:

https://cwiki.apache.org/confluence/display/CLOUDSTACK/AWS-Style+Regions+Functional+Spec

And I've set up two management servers as separate regions. From what
I can tell, I'm not sure why I'd want to use it.  1) accounts need to
be replicated manually, and 2)I can't make an API call to one MS to
deploy in another region (or at least I don't see a documented way to
do this). Between those two limitations, it means I could also set up
two standalone management servers and they'd function the same, aside
from the slight UI change of having another region listed in the UI. I
realize there are GSLB and portable IP features, but they're not
mentioned in the functional spec. I'm just looking for the differences
from a management perspective and I don't see much.

Re: CLVM broken on master

I've done some testing on our plugin and that appears to work fine for me.

-- 
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Oct 17, 2013, at 6:05 PM, Marcus Sorensen  wrote:

> The following seems to fix the issue, by the way, but again since I
> didn't initially change the code I'd like someone else to
> review/handle it.
> 
> diff --git 
> a/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java
> b/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java
> index 002143f..3ac82e3 100644
> --- 
> a/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java
> +++ 
> b/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java
> @@ -68,7 +68,7 @@ public class StorageSubsystemCommandHandlerBase
> implements StorageSubsystemComma
> DataStoreTO srcDataStore = srcData.getDataStore();
> DataStoreTO destDataStore = destData.getDataStore();
> 
> -if ((srcData.getObjectType() == DataObjectType.TEMPLATE) &&
> (destData.getObjectType() == DataObjectType.TEMPLATE &&
> destData.getDataStore().getRole() == DataStoreRole.Primary)) {
> +if (srcData.getObjectType() == DataObjectType.TEMPLATE &&
> srcData.getDataStore().getRole() == DataStoreRole.Image &&
> destData.getDataStore().getRole() == DataStoreRole.Primary) {
> //copy template to primary storage
> return processor.copyTemplateToPrimaryStorage(cmd);
> } else if (srcData.getObjectType() == DataObjectType.TEMPLATE
> && srcDataStore.getRole() == DataStoreRole.Primary &&
> destDataStore.getRole() == DataStoreRole.Primary) {
> 
> On Thu, Oct 17, 2013 at 4:03 PM, Marcus Sorensen  wrote:
>> All of the above mentioned is in
>> core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java
>> , by the way.
>> 
>> On Thu, Oct 17, 2013 at 4:02 PM, Marcus Sorensen  wrote:
>>> Sure, but CopyCommand is being triggered in this code. I've tested
>>> several variations to this one line, some work, some don't.
>>> 
>>> On Thu, Oct 17, 2013 at 3:38 PM, Edison Su  wrote:
 For CLVM, the copy template from secondary to primary and create volume 
 from template logic is handled by 
 CloudStackPrimaryDataStoreDriverImpl->copyAsync, not in 
 AncientDataMotionStrategy
 You can check the code: 4fb459355337c874a10f47c0224af72d6fef1ff2.
 
> -Original Message-
> From: Marcus Sorensen [mailto:shadow...@gmail.com]
> Sent: Thursday, October 17, 2013 2:07 PM
> To: dev@cloudstack.apache.org
> Subject: Re: CLVM broken on master
> 
> I think if we can change this line:
> 
> if ((srcData.getObjectType() == DataObjectType.TEMPLATE) &&
> (destData.getObjectType() == DataObjectType.TEMPLATE &&
> destData.getDataStore().getRole() == DataStoreRole.Primary)) {
> 
> to something like:
> 
> if (srcData.getObjectType() == DataObjectType.TEMPLATE &&
> srcData.getDataStore().getRole() == DataStoreRole.Image &&
> destData.getDataStore().getRole() == DataStoreRole.Primary) {
> 
> Maybe that will work? That way it's strictly secondary -> primary 
> templates,
> not primary->primary templates.
> 
> Alternatively we could put it back to where it was:
> 
> if (srcData.getObjectType() == DataObjectType.TEMPLATE && srcDataStore
> instanceof NfsTO && destData.getDataStore().getRole() ==
> DataStoreRole.Primary) {
> 
> But your patch on the reviewboard removes NfsTO, and I'm assuming the
> idea was to work towards getting away from NFS-specific secondary storage.
> 
> On Thu, Oct 17, 2013 at 2:57 PM, Marcus Sorensen 
> wrote:
>> I ran that through my tester, it didn't like it.  That actually kept
>> the system vms from starting. Since CopyCommand is used for both
>> template to template and template to primary, it seems that the
>> original template copy is fine but now this catches the case where the
>> source template is on primary and we are making a root disk.
>> copyTemplateToPrimaryStorage has:
>> 
>>if (!(imageStore instanceof NfsTO)) {
>>return new CopyCmdAnswer("unsupported protocol");
>>}
>> 
>> we should be calling 'cloneVolumeFromBaseTemplate', but the original
>> if statement is now too loose.  I'll play with it a bit and see if I
>> can suggest a solution that works.
>> 
>> 2013-09-17 17:58:07,178 DEBUG [cloud.agent.Agent]
>> (agentRequest-Handler-2:null) Request:Seq 1-829816935:  { Cmd ,
>> MgmtId: 52241639751, via: 1, Ver: v1, Flags: 100011,
>> 
> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.
> a
>> pache.cloudstack.storage.to.TemplateObjectTO":{"path":"bf53a7c6-1fed-1
>> 1e3-a1ff-000c29d82947","origUrl":"http://download.cloud.com/templates/
>> 4.2/systemvmtemplate-2013-0

RE: CLVM broken on master

2013-10-17 Thread Edison Su

Marcus, would you mind to check in your fix into master? Thanks.

> -Original Message-
> From: SuichII, Christopher [mailto:chris.su...@netapp.com]
> Sent: Thursday, October 17, 2013 4:35 PM
> To: 
> Subject: Re: CLVM broken on master
> 
> I've done some testing on our plugin and that appears to work fine for me.
> 
> --
> Chris Suich
> chris.su...@netapp.com
> NetApp Software Engineer
> Data Center Platforms - Cloud Solutions
> Citrix, Cisco & Red Hat
> 
> On Oct 17, 2013, at 6:05 PM, Marcus Sorensen 
> wrote:
> 
> > The following seems to fix the issue, by the way, but again since I
> > didn't initially change the code I'd like someone else to
> > review/handle it.
> >
> > diff --git
> >
> a/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandl
> erBa
> > se.java
> >
> b/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandl
> erBa
> > se.java
> > index 002143f..3ac82e3 100644
> > ---
> >
> a/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandl
> erBa
> > se.java
> > +++
> b/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandl
> > +++ erBase.java
> > @@ -68,7 +68,7 @@ public class StorageSubsystemCommandHandlerBase
> > implements StorageSubsystemComma
> > DataStoreTO srcDataStore = srcData.getDataStore();
> > DataStoreTO destDataStore = destData.getDataStore();
> >
> > -if ((srcData.getObjectType() == DataObjectType.TEMPLATE) &&
> > (destData.getObjectType() == DataObjectType.TEMPLATE &&
> > destData.getDataStore().getRole() == DataStoreRole.Primary)) {
> > +if (srcData.getObjectType() == DataObjectType.TEMPLATE &&
> > srcData.getDataStore().getRole() == DataStoreRole.Image &&
> > destData.getDataStore().getRole() == DataStoreRole.Primary) {
> > //copy template to primary storage
> > return processor.copyTemplateToPrimaryStorage(cmd);
> > } else if (srcData.getObjectType() == DataObjectType.TEMPLATE
> > && srcDataStore.getRole() == DataStoreRole.Primary &&
> > destDataStore.getRole() == DataStoreRole.Primary) {
> >
> > On Thu, Oct 17, 2013 at 4:03 PM, Marcus Sorensen 
> wrote:
> >> All of the above mentioned is in
> >>
> core/src/com/cloud/storage/resource/StorageSubsystemCommandHandler
> Bas
> >> e.java
> >> , by the way.
> >>
> >> On Thu, Oct 17, 2013 at 4:02 PM, Marcus Sorensen
>  wrote:
> >>> Sure, but CopyCommand is being triggered in this code. I've tested
> >>> several variations to this one line, some work, some don't.
> >>>
> >>> On Thu, Oct 17, 2013 at 3:38 PM, Edison Su 
> wrote:
>  For CLVM, the copy template from secondary to primary and create
>  volume from template logic is handled by
> CloudStackPrimaryDataStoreDriverImpl->copyAsync, not in
> AncientDataMotionStrategy You can check the code:
> 4fb459355337c874a10f47c0224af72d6fef1ff2.
> 
> > -Original Message-
> > From: Marcus Sorensen [mailto:shadow...@gmail.com]
> > Sent: Thursday, October 17, 2013 2:07 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: CLVM broken on master
> >
> > I think if we can change this line:
> >
> > if ((srcData.getObjectType() == DataObjectType.TEMPLATE) &&
> > (destData.getObjectType() == DataObjectType.TEMPLATE &&
> > destData.getDataStore().getRole() == DataStoreRole.Primary)) {
> >
> > to something like:
> >
> > if (srcData.getObjectType() == DataObjectType.TEMPLATE &&
> > srcData.getDataStore().getRole() == DataStoreRole.Image &&
> > destData.getDataStore().getRole() == DataStoreRole.Primary) {
> >
> > Maybe that will work? That way it's strictly secondary -> primary
> > templates, not primary->primary templates.
> >
> > Alternatively we could put it back to where it was:
> >
> > if (srcData.getObjectType() == DataObjectType.TEMPLATE &&
> > srcDataStore instanceof NfsTO && destData.getDataStore().getRole()
> > ==
> > DataStoreRole.Primary) {
> >
> > But your patch on the reviewboard removes NfsTO, and I'm assuming
> > the idea was to work towards getting away from NFS-specific
> secondary storage.
> >
> > On Thu, Oct 17, 2013 at 2:57 PM, Marcus Sorensen
> > 
> > wrote:
> >> I ran that through my tester, it didn't like it.  That actually
> >> kept the system vms from starting. Since CopyCommand is used for
> >> both template to template and template to primary, it seems that
> >> the original template copy is fine but now this catches the case
> >> where the source template is on primary and we are making a root
> disk.
> >> copyTemplateToPrimaryStorage has:
> >>
> >>if (!(imageStore instanceof NfsTO)) {
> >>return new CopyCmdAnswer("unsupported protocol");
> >>}
> >>
> >> we should be calling 'cloneVolumeFromBaseTemplate', but the
> >> original if statement is now too loose.  I'll play with it a bit
> >> and see if I can suggest a solution that works

Re: Review Request 14715: [CLOUDSTACK-4887] CLVM broken


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14715/#review27157
---


Commit a504c004bf10555e5ea67ec89fe7bf6f00fe4622 in branch refs/heads/master 
from Marcus Sorensen
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=a504c00 ]

CLOUDSTACK-4887: Fix CLVM template download from storage refactoring work


- ASF Subversion and Git Services


On Oct. 17, 2013, 6:59 p.m., Chris Suich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14715/
> ---
> 
> (Updated Oct. 17, 2013, 6:59 p.m.)
> 
> 
> Review request for cloudstack, edison su and Marcus Sorensen.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Fixed a bug with backup snapshot condition logic causing snapshots to not be 
> backed up and the API to fail.
> 
> 
> Diffs
> -
> 
>   core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java 
> 002143f 
> 
> Diff: https://reviews.apache.org/r/14715/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Chris Suich
> 
>

Re: Review Request 14469: CLOUDSTACK-4637: Fix failures in test_egress_fw_rules.py


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14469/#review27158
---


Commit a65f1ebefca7b22512762faf1832291153782f58 in branch refs/heads/master 
from Gaurav Aradhye
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=a65f1eb ]

CLOUDSTACK-4637: Fix failures in test_egress_fw_rules.py

Removed log_test_exceptions which did not add any value.
Skipped few tests which are incomplete. Added timeout logic
and to wait for router to boot.


- ASF Subversion and Git Services


On Oct. 16, 2013, 2:27 p.m., Girish Shilamkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14469/
> ---
> 
> (Updated Oct. 16, 2013, 2:27 p.m.)
> 
> 
> Review request for cloudstack, Harikrishna Patnala and venkata swamy babu  
> budumuru.
> 
> 
> Bugs: CLOUDSTACK-4637
> https://issues.apache.org/jira/browse/CLOUDSTACK-4637
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Add logic to wait for router to boot.
> 
> 
> Diffs
> -
> 
>   test/integration/component/test_egress_fw_rules.py 5c18f9c 
> 
> Diff: https://reviews.apache.org/r/14469/diff/
> 
> 
> Testing
> ---
> 
> test_01_1_egress_fr1 (test_egress_fw_rules.TestEgressFWRules)
> Test By-default the communication from guest n/w to public n/w is NOT 
> allowed. ... ok
> test_01_egress_fr1 (test_egress_fw_rules.TestEgressFWRules)
> Test By-default the communication from guest n/w to public n/w is allowed. 
> ... ok
> test_02_1_egress_fr2 (test_egress_fw_rules.TestEgressFWRules)
> Test Allow Communication using Egress rule with CIDR + Port Range + Protocol. 
> ... ok
> test_02_egress_fr2 (test_egress_fw_rules.TestEgressFWRules)
> Test Allow Communication using Egress rule with CIDR + Port Range + Protocol. 
> ... ok
> test_03_1_egress_fr3 (test_egress_fw_rules.TestEgressFWRules)
> Test Communication blocked with network that is other than specified ... ok
> test_03_egress_fr3 (test_egress_fw_rules.TestEgressFWRules)
> Test Communication blocked with network that is other than specified ... ok
> test_04_1_egress_fr4 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule and check the Firewall_Rules DB table ... ok
> test_04_egress_fr4 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule and check the Firewall_Rules DB table ... ok
> test_05_1_egress_fr5 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule and check the IP tables ... ok
> test_05_egress_fr5 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule and check the IP tables ... ok
> test_06_1_egress_fr6 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule without CIDR ... ok
> test_06_egress_fr6 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule without CIDR ... ok
> test_07_1_egress_fr7 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule without End Port ... ok
> test_07_egress_fr7 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule without End Port ... ok
> test_08_1_egress_fr8 (test_egress_fw_rules.TestEgressFWRules)
> Test Port Forwarding and Egress Conflict ... ok
> test_08_egress_fr8 (test_egress_fw_rules.TestEgressFWRules)
> Test Port Forwarding and Egress Conflict ... ok
> test_09_1_egress_fr9 (test_egress_fw_rules.TestEgressFWRules)
> Test Delete Egress rule ... ok
> test_09_egress_fr9 (test_egress_fw_rules.TestEgressFWRules)
> Test Delete Egress rule ... ok
> test_10_1_egress_fr10 (test_egress_fw_rules.TestEgressFWRules)
> Test Invalid CIDR and Invalid Port ranges ... ok
> test_10_egress_fr10 (test_egress_fw_rules.TestEgressFWRules)
> Test Invalid CIDR and Invalid Port ranges ... ok
> test_11_1_egress_fr11 (test_egress_fw_rules.TestEgressFWRules)
> Test Regression on Firewall + PF + LB + SNAT ... ok
> test_11_egress_fr11 (test_egress_fw_rules.TestEgressFWRules)
> Test Regression on Firewall + PF + LB + SNAT ... ok
> test_12_1_egress_fr12 (test_egress_fw_rules.TestEgressFWRules)
> Test Reboot Router ... ok
> test_13_1_egress_fr13 (test_egress_fw_rules.TestEgressFWRules)
> Test Redundant Router : Master failover ... ok
> test_13_egress_fr13 (test_egress_fw_rules.TestEgressFWRules)
> Test Redundant Router : Master failover ... ok
> 
> --
> Ran 26 tests in 15127.540s
> 
> OK
> 
> 
> Thanks,
> 
> Girish Shilamkar
> 
>

Review Request 14729: Fix UCS HTTPS redirect

2013-10-17 Thread Amogh Vasekar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14729/
---

Review request for cloudstack and Frank Zhang.


Bugs: https://issues.apache.org/jira/browse/CLOUDSTACK-3285

https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/CLOUDSTACK-3285


Repository: cloudstack-git


Description
---

Fixing HTTPS redirect to work after moving from o.a.commons.http to 
springwork.http


Diffs
-

  plugins/hypervisors/ucs/src/com/cloud/ucs/manager/UcsHttpClient.java d3dd483 

Diff: https://reviews.apache.org/r/14729/diff/


Testing
---

Local environment
Sample response :
3ucsmanager10.x.x.x1


Thanks,

Amogh Vasekar

Re: Review Request 14469: CLOUDSTACK-4637: Fix failures in test_egress_fw_rules.py


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14469/#review27160
---


Commit 833229c416cfee3e3dec3762d2e9c8e734696f90 in branch refs/heads/4.2 from 
Gaurav Aradhye
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=833229c ]

CLOUDSTACK-4637: Fix failures in test_egress_fw_rules.py

Removed log_test_exceptions which did not add any value.
Skipped few tests which are incomplete. Added timeout logic
and to wait for router to boot.
(cherry picked from commit a65f1ebefca7b22512762faf1832291153782f58)

Signed-off-by: Sangeetha 


- ASF Subversion and Git Services


On Oct. 16, 2013, 2:27 p.m., Girish Shilamkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14469/
> ---
> 
> (Updated Oct. 16, 2013, 2:27 p.m.)
> 
> 
> Review request for cloudstack, Harikrishna Patnala and venkata swamy babu  
> budumuru.
> 
> 
> Bugs: CLOUDSTACK-4637
> https://issues.apache.org/jira/browse/CLOUDSTACK-4637
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Add logic to wait for router to boot.
> 
> 
> Diffs
> -
> 
>   test/integration/component/test_egress_fw_rules.py 5c18f9c 
> 
> Diff: https://reviews.apache.org/r/14469/diff/
> 
> 
> Testing
> ---
> 
> test_01_1_egress_fr1 (test_egress_fw_rules.TestEgressFWRules)
> Test By-default the communication from guest n/w to public n/w is NOT 
> allowed. ... ok
> test_01_egress_fr1 (test_egress_fw_rules.TestEgressFWRules)
> Test By-default the communication from guest n/w to public n/w is allowed. 
> ... ok
> test_02_1_egress_fr2 (test_egress_fw_rules.TestEgressFWRules)
> Test Allow Communication using Egress rule with CIDR + Port Range + Protocol. 
> ... ok
> test_02_egress_fr2 (test_egress_fw_rules.TestEgressFWRules)
> Test Allow Communication using Egress rule with CIDR + Port Range + Protocol. 
> ... ok
> test_03_1_egress_fr3 (test_egress_fw_rules.TestEgressFWRules)
> Test Communication blocked with network that is other than specified ... ok
> test_03_egress_fr3 (test_egress_fw_rules.TestEgressFWRules)
> Test Communication blocked with network that is other than specified ... ok
> test_04_1_egress_fr4 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule and check the Firewall_Rules DB table ... ok
> test_04_egress_fr4 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule and check the Firewall_Rules DB table ... ok
> test_05_1_egress_fr5 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule and check the IP tables ... ok
> test_05_egress_fr5 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule and check the IP tables ... ok
> test_06_1_egress_fr6 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule without CIDR ... ok
> test_06_egress_fr6 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule without CIDR ... ok
> test_07_1_egress_fr7 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule without End Port ... ok
> test_07_egress_fr7 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule without End Port ... ok
> test_08_1_egress_fr8 (test_egress_fw_rules.TestEgressFWRules)
> Test Port Forwarding and Egress Conflict ... ok
> test_08_egress_fr8 (test_egress_fw_rules.TestEgressFWRules)
> Test Port Forwarding and Egress Conflict ... ok
> test_09_1_egress_fr9 (test_egress_fw_rules.TestEgressFWRules)
> Test Delete Egress rule ... ok
> test_09_egress_fr9 (test_egress_fw_rules.TestEgressFWRules)
> Test Delete Egress rule ... ok
> test_10_1_egress_fr10 (test_egress_fw_rules.TestEgressFWRules)
> Test Invalid CIDR and Invalid Port ranges ... ok
> test_10_egress_fr10 (test_egress_fw_rules.TestEgressFWRules)
> Test Invalid CIDR and Invalid Port ranges ... ok
> test_11_1_egress_fr11 (test_egress_fw_rules.TestEgressFWRules)
> Test Regression on Firewall + PF + LB + SNAT ... ok
> test_11_egress_fr11 (test_egress_fw_rules.TestEgressFWRules)
> Test Regression on Firewall + PF + LB + SNAT ... ok
> test_12_1_egress_fr12 (test_egress_fw_rules.TestEgressFWRules)
> Test Reboot Router ... ok
> test_13_1_egress_fr13 (test_egress_fw_rules.TestEgressFWRules)
> Test Redundant Router : Master failover ... ok
> test_13_egress_fr13 (test_egress_fw_rules.TestEgressFWRules)
> Test Redundant Router : Master failover ... ok
> 
> --
> Ran 26 tests in 15127.540s
> 
> OK
> 
> 
> Thanks,
> 
> Girish Shilamkar
> 
>

Re: Review Request 14715: [CLOUDSTACK-4887] CLVM broken

2013-10-17 Thread edison su


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14715/#review27164
---

Ship it!


Ship It!

- edison su


On Oct. 17, 2013, 6:59 p.m., Chris Suich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14715/
> ---
> 
> (Updated Oct. 17, 2013, 6:59 p.m.)
> 
> 
> Review request for cloudstack, edison su and Marcus Sorensen.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Fixed a bug with backup snapshot condition logic causing snapshots to not be 
> backed up and the API to fail.
> 
> 
> Diffs
> -
> 
>   core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java 
> 002143f 
> 
> Diff: https://reviews.apache.org/r/14715/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Chris Suich
> 
>

Re: Review Request 14469: CLOUDSTACK-4637: Fix failures in test_egress_fw_rules.py

2013-10-17 Thread sangeetha hariharan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14469/#review27163
---

Ship it!


Ship It!

- sangeetha hariharan


On Oct. 16, 2013, 2:27 p.m., Girish Shilamkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14469/
> ---
> 
> (Updated Oct. 16, 2013, 2:27 p.m.)
> 
> 
> Review request for cloudstack, Harikrishna Patnala and venkata swamy babu  
> budumuru.
> 
> 
> Bugs: CLOUDSTACK-4637
> https://issues.apache.org/jira/browse/CLOUDSTACK-4637
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Add logic to wait for router to boot.
> 
> 
> Diffs
> -
> 
>   test/integration/component/test_egress_fw_rules.py 5c18f9c 
> 
> Diff: https://reviews.apache.org/r/14469/diff/
> 
> 
> Testing
> ---
> 
> test_01_1_egress_fr1 (test_egress_fw_rules.TestEgressFWRules)
> Test By-default the communication from guest n/w to public n/w is NOT 
> allowed. ... ok
> test_01_egress_fr1 (test_egress_fw_rules.TestEgressFWRules)
> Test By-default the communication from guest n/w to public n/w is allowed. 
> ... ok
> test_02_1_egress_fr2 (test_egress_fw_rules.TestEgressFWRules)
> Test Allow Communication using Egress rule with CIDR + Port Range + Protocol. 
> ... ok
> test_02_egress_fr2 (test_egress_fw_rules.TestEgressFWRules)
> Test Allow Communication using Egress rule with CIDR + Port Range + Protocol. 
> ... ok
> test_03_1_egress_fr3 (test_egress_fw_rules.TestEgressFWRules)
> Test Communication blocked with network that is other than specified ... ok
> test_03_egress_fr3 (test_egress_fw_rules.TestEgressFWRules)
> Test Communication blocked with network that is other than specified ... ok
> test_04_1_egress_fr4 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule and check the Firewall_Rules DB table ... ok
> test_04_egress_fr4 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule and check the Firewall_Rules DB table ... ok
> test_05_1_egress_fr5 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule and check the IP tables ... ok
> test_05_egress_fr5 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule and check the IP tables ... ok
> test_06_1_egress_fr6 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule without CIDR ... ok
> test_06_egress_fr6 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule without CIDR ... ok
> test_07_1_egress_fr7 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule without End Port ... ok
> test_07_egress_fr7 (test_egress_fw_rules.TestEgressFWRules)
> Test Create Egress rule without End Port ... ok
> test_08_1_egress_fr8 (test_egress_fw_rules.TestEgressFWRules)
> Test Port Forwarding and Egress Conflict ... ok
> test_08_egress_fr8 (test_egress_fw_rules.TestEgressFWRules)
> Test Port Forwarding and Egress Conflict ... ok
> test_09_1_egress_fr9 (test_egress_fw_rules.TestEgressFWRules)
> Test Delete Egress rule ... ok
> test_09_egress_fr9 (test_egress_fw_rules.TestEgressFWRules)
> Test Delete Egress rule ... ok
> test_10_1_egress_fr10 (test_egress_fw_rules.TestEgressFWRules)
> Test Invalid CIDR and Invalid Port ranges ... ok
> test_10_egress_fr10 (test_egress_fw_rules.TestEgressFWRules)
> Test Invalid CIDR and Invalid Port ranges ... ok
> test_11_1_egress_fr11 (test_egress_fw_rules.TestEgressFWRules)
> Test Regression on Firewall + PF + LB + SNAT ... ok
> test_11_egress_fr11 (test_egress_fw_rules.TestEgressFWRules)
> Test Regression on Firewall + PF + LB + SNAT ... ok
> test_12_1_egress_fr12 (test_egress_fw_rules.TestEgressFWRules)
> Test Reboot Router ... ok
> test_13_1_egress_fr13 (test_egress_fw_rules.TestEgressFWRules)
> Test Redundant Router : Master failover ... ok
> test_13_egress_fr13 (test_egress_fw_rules.TestEgressFWRules)
> Test Redundant Router : Master failover ... ok
> 
> --
> Ran 26 tests in 15127.540s
> 
> OK
> 
> 
> Thanks,
> 
> Girish Shilamkar
> 
>

how to use hashes on c.a.o?