Re: xenserver tools

[Abhinandan Prateek]

Daan,
  The info you provided is useful.
We are verifying the steps, once done will update the systemvm scripts.


On 17/09/13 6:03 pm, "Daan Hoogland"  wrote:

>Hi Abhinandan,
>
>What we have seen so far is that two files in the xe-guest-utilities
>package are missing, where CS assumes that they are there.
>
>/usr/bin/xenstore  => used in script /usr/sbin/xe-update-guest-attrs
>(this script is also included in the package but overwritten by the CS
>systemvm.iso scripts)
>And
>/etc/init.d/xe-linux-distribution => calls /usr/sbin/xe-daemon (this
>script is also included in the package but overwritten by the CS
>systemvm.iso scripts)
>
>This is the package we installed to fix it and the files included in
>that package.
>
>root@r-8691-VM:~# dpkg -L xe-guest-utilities
>/.
>/usr
>/usr/bin
>/usr/bin/xenstore-write
>/usr/bin/xenstore
>/usr/bin/xenstore-read
>/usr/bin/xenstore-exists
>/usr/bin/xenstore-list
>/usr/bin/xenstore-ls
>/usr/bin/xenstore-chmod
>/usr/bin/xenstore-rm
>/usr/share
>/usr/share/xe-guest-utilities
>/usr/share/xe-guest-utilities/citrix.list
>/usr/share/doc
>/usr/share/doc/xe-guest-utilities
>/usr/share/doc/xe-guest-utilities/COPYING.LGPL.gz
>/usr/share/doc/xe-guest-utilities/copyright
>/usr/share/doc/xe-guest-utilities/COPYING.gz
>/usr/sbin
>/usr/sbin/xe-update-guest-attrs
>/usr/sbin/xe-linux-distribution
>/usr/sbin/xe-daemon
>/etc
>/etc/udev
>/etc/udev/rules.d
>/etc/udev/rules.d/z10_xen-vcpu-hotplug.rules
>/etc/init.d
>/etc/init.d/xe-linux-distribution
>root@r-8691-VM:~# ls xe-guest-utilities_6.0.2-766_i386.deb
>
>Thanks,
>
>On Tue, Sep 17, 2013 at 1:11 PM, Abhinandan Prateek
> wrote:
>> Daan,
>>   Do you know what all packages we need to add to get the tools
>>working. I
>> see that current scripts are installing xenstore-utils.
>>
>>
>> On 17/09/13 1:58 pm, "Daan Hoogland"  wrote:
>>
>>>H,
>>>
>>>In the present template for systemvms on xen the xenserver tools are
>>>not installed due to distribution rights issues. Now that xenserver is
>>>open-sourced, can anyone say if they can be added again?
>>>
>>>regards,
>>>Daan
>>

Re: security around api.log

[Abhinandan Prateek]

We can provide a way to disable the api.log ?

On 18/09/13 11:27 am, "Rajesh Battala"  wrote:

>If anybody got access to the api.log using the session details we can do
>execute api's and cause harm.
>But the api.log is present in the mgmt server and if anybody got access
>to it, he can corrupt anything.
>Not just accessing api.log, any other services logs and get the data. I
>feel it's up to admin how to protect his system and services.
>
>Thanks
>Rajesh Battala
>
>-Original Message-
>From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com]
>Sent: Saturday, September 14, 2013 2:10 AM
>To: dev@cloudstack.apache.org
>Subject: security around api.log
>
>I just noticed api.log which seems to log all the API access in a form
>like
>
>2013-09-13 00:02:09,451 INFO  [a.c.c.a.ApiServer]
>(2011638958@qtp-657397168-0:ctx-81b1e088 ctx-174e4a62) (userId=2
>accountId=2 sessionId=7asvmtwoesbc6ia3e4kxtzrl) 127.0.0.1 -- GET
>command=listZones&response=json&sessionkey=ec6h46Om8a1y3d%2BhrdIpQ85cAfc%3
>D&_=1379055729422
>200 { "listzonesresponse" : { "count":1 ,"zone" : [
>{"id":"cdaf82f1-3b57-4aa4-b3ce-b60173ed45f2","name":"zone1","dns1":"8.8.8.
>8","dns2":"8.8.4.4","internaldns1":"8.8.4.4","networktype":"Basic","securi
>tygroupsenabled":true,"allocationstate":"Enabled","zonetoken":"6dce94e8-e8
>dc-3077-bfde-c6e8594bd449","dhcpprovider":"VirtualRouter","localstorageena
>bled":false}
>] } }
>
>The sessionId and sessionKey is logged in the file.  I haven't tried it
>yet, but can't I use that info to hijack the session?  That introduces a
>security issue in that any server operator can now hijack anybody's
>session.  So that api.log file really needs to be protected in the same
>way a file with a password in it would be.
>
>I would suggest that we just don't log the sessionId or sessionKey.
>
>Darren

Re: Call for 4.3 and 4.2.1 Release Managers!

[Abhinandan Prateek]

>
>
>> -Original Message-
>> From: Abhinandan Prateek [mailto:abhinandan.prat...@citrix.com]
>> Sent: Monday, September 16, 2013 10:18 AM
>> To: dev@cloudstack.apache.org
>> Subject: Re: Call for 4.3 and 4.2.1 Release Managers!
>> 
>> If it is fine with community I am willing to take up 4.2.1 maintenance
>> release.
>> 
>> -abhi
>[Animesh>] Abhi looks like you are the only one for 4.2.1. Let me know if
>you need any help.
>> 

Animesh, Yes your guidance will be much desired. I will start with
prioritising open tickets and 4.2 feedback we get on the mailing list.

With that I would also request community members to file issues that they
think are important and should go into the maintenance release.

-abhi


>>

RE: Call for 4.3 and 4.2.1 Release Managers!

[Daan Hoogland]

Community,

And especially our friends at Citrix, i am willing but am kind of a
dictator in this role. Im an experienced developper but not in such a big
oss project. I know i havey style and obsessions that some of you might not
like. A fixation on automated tests os one that i know is likely to be
accepted. Others will come out of the hat.

Thought i had to warm you all,
Daan

mobile biligual spell checker used
Op 18 sep. 2013 07:22 schreef "Animesh Chaturvedi" <
animesh.chaturv...@citrix.com> het volgende:

>
>
> > -Original Message-
> > From: Animesh Chaturvedi [mailto:animesh.chaturv...@citrix.com]
> > Sent: Monday, September 16, 2013 10:30 AM
> > To: dev@cloudstack.apache.org
> > Subject: RE: Call for 4.3 and 4.2.1 Release Managers!
> >
> >
> >
> > > -Original Message-
> > > From: Daan Hoogland [mailto:daan.hoogl...@gmail.com]
> > > Sent: Monday, September 16, 2013 4:35 AM
> > > To: dev
> > > Subject: Re: Call for 4.3 and 4.2.1 Release Managers!
> > >
> > > Animesh, I am passing till next time, I am willing though if really no
> > > one can be found.
> > >
> > > Dann
> >
> > [Animesh>] Daan thanks for the offer, I was getting worried that no one
> > wants to be RM. Let's see who else turns up.
> >
> [Animesh>] Daan do you want to take up 4.3.0? I have added  a 4.3.0
> release page with tentative dates for now. If you don't want to do it now,
> I can continue for 4.3.0.
> > >
> > > On Wed, Sep 11, 2013 at 7:57 AM, Animesh Chaturvedi
> > >  wrote:
> > > > Folks
> > > >
> > > > Now that 4.2.0 VOTE has passed I want to call out for any committers
> > > who are interested in taking over as the release manager for the
> > > upcoming 4.2.1 maintenance release and the 4.3.0 release.
> > > >
> > > >
> > > > Following our 4 month release cycle the feature freeze for 4.3.0
> > > should be by end of October and as community we need to figure out the
> > > timelines for 4.2.1 maintenance release.
> > > >
> > > > I will update my experience and learning from the 4.2.0 release in
> > > > the
> > > wiki shortly after the formal release announcement  later this week.
> > > >
> > > > Thanks
> > > > Animesh
> > > >
> > > >
> > > >
> > > >
>

Can I Suspend a CloudStack VM through the REST API?

[Krasimir Baylov]

Hi all,

Is there a way to hibernate a running VM in CloudStack with the REST API? I
know I can deploy/start/stop/destroy it but I couldn't find a way of
hibernating it.

By hibernate I mean - releasing the resources that the VM uses while
preserving it's state (like SLEEP in Windows). In other words, when I
resume the VM it should have the same state (running programs, opened
files,...) as it was before the suspend. (This would need to persist the
VMs RAM though).

Any hints on what can I do? Is there anything implemented I can use or I
need to develop it myself. My current setup uses KVM and CloudStack 4.1.1.

Thank you!
Krasimir

Re: xenserver tools

[Daan Hoogland]

Thanks to my colleague Joris,

I can update the scripts in the sytemvm.iso but the real question is
about copy rights. Xenserver is gpl these days isn't it. Can we
redistribute them? any legal advice. The alternative is maintaining a
nonoss systemvm.iso and maybe template.

regards,
Daan

On Wed, Sep 18, 2013 at 9:01 AM, Abhinandan Prateek
 wrote:
> Daan,
>   The info you provided is useful.
> We are verifying the steps, once done will update the systemvm scripts.
>
>
> On 17/09/13 6:03 pm, "Daan Hoogland"  wrote:
>
>>Hi Abhinandan,
>>
>>What we have seen so far is that two files in the xe-guest-utilities
>>package are missing, where CS assumes that they are there.
>>
>>/usr/bin/xenstore  => used in script /usr/sbin/xe-update-guest-attrs
>>(this script is also included in the package but overwritten by the CS
>>systemvm.iso scripts)
>>And
>>/etc/init.d/xe-linux-distribution => calls /usr/sbin/xe-daemon (this
>>script is also included in the package but overwritten by the CS
>>systemvm.iso scripts)
>>
>>This is the package we installed to fix it and the files included in
>>that package.
>>
>>root@r-8691-VM:~# dpkg -L xe-guest-utilities
>>/.
>>/usr
>>/usr/bin
>>/usr/bin/xenstore-write
>>/usr/bin/xenstore
>>/usr/bin/xenstore-read
>>/usr/bin/xenstore-exists
>>/usr/bin/xenstore-list
>>/usr/bin/xenstore-ls
>>/usr/bin/xenstore-chmod
>>/usr/bin/xenstore-rm
>>/usr/share
>>/usr/share/xe-guest-utilities
>>/usr/share/xe-guest-utilities/citrix.list
>>/usr/share/doc
>>/usr/share/doc/xe-guest-utilities
>>/usr/share/doc/xe-guest-utilities/COPYING.LGPL.gz
>>/usr/share/doc/xe-guest-utilities/copyright
>>/usr/share/doc/xe-guest-utilities/COPYING.gz
>>/usr/sbin
>>/usr/sbin/xe-update-guest-attrs
>>/usr/sbin/xe-linux-distribution
>>/usr/sbin/xe-daemon
>>/etc
>>/etc/udev
>>/etc/udev/rules.d
>>/etc/udev/rules.d/z10_xen-vcpu-hotplug.rules
>>/etc/init.d
>>/etc/init.d/xe-linux-distribution
>>root@r-8691-VM:~# ls xe-guest-utilities_6.0.2-766_i386.deb
>>
>>Thanks,
>>
>>On Tue, Sep 17, 2013 at 1:11 PM, Abhinandan Prateek
>> wrote:
>>> Daan,
>>>   Do you know what all packages we need to add to get the tools
>>>working. I
>>> see that current scripts are installing xenstore-utils.
>>>
>>>
>>> On 17/09/13 1:58 pm, "Daan Hoogland"  wrote:
>>>
H,

In the present template for systemvms on xen the xenserver tools are
not installed due to distribution rights issues. Now that xenserver is
open-sourced, can anyone say if they can be added again?

regards,
Daan
>>>
>

RE: Can I Suspend a CloudStack VM through the REST API?

[Rajesh Battala]

There is no api to suspend/hibernate the VM.
I think the reason it's not implemented is some Hypervisors might support 
suspend and some might not. Some hypervisors execute suspend/pause/hibernate 
only if the guest tools are installed.
You can create jira ticket for it and please feel free to implement it.

Thanks
Rajesh Battala

-Original Message-
From: Krasimir Baylov [mailto:kbai...@gmail.com] 
Sent: Wednesday, September 18, 2013 1:57 PM
To: dev@cloudstack.apache.org
Subject: Can I Suspend a CloudStack VM through the REST API?

Hi all,

Is there a way to hibernate a running VM in CloudStack with the REST API? I 
know I can deploy/start/stop/destroy it but I couldn't find a way of 
hibernating it.

By hibernate I mean - releasing the resources that the VM uses while preserving 
it's state (like SLEEP in Windows). In other words, when I resume the VM it 
should have the same state (running programs, opened
files,...) as it was before the suspend. (This would need to persist the VMs 
RAM though).

Any hints on what can I do? Is there anything implemented I can use or I need 
to develop it myself. My current setup uses KVM and CloudStack 4.1.1.

Thank you!
Krasimir

RE: security around api.log

[Rajesh Battala]

Disabling api log might not be a good idea, instead while logging the request 
remove the sensitive details (session details, passwords etc ) and dump it.

Thanks
Rajesh Battala

-Original Message-
From: Abhinandan Prateek [mailto:abhinandan.prat...@citrix.com] 
Sent: Wednesday, September 18, 2013 12:33 PM
To: dev@cloudstack.apache.org
Subject: Re: security around api.log

We can provide a way to disable the api.log ?

On 18/09/13 11:27 am, "Rajesh Battala"  wrote:

>If anybody got access to the api.log using the session details we can 
>do execute api's and cause harm.
>But the api.log is present in the mgmt server and if anybody got access 
>to it, he can corrupt anything.
>Not just accessing api.log, any other services logs and get the data. I 
>feel it's up to admin how to protect his system and services.
>
>Thanks
>Rajesh Battala
>
>-Original Message-
>From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com]
>Sent: Saturday, September 14, 2013 2:10 AM
>To: dev@cloudstack.apache.org
>Subject: security around api.log
>
>I just noticed api.log which seems to log all the API access in a form 
>like
>
>2013-09-13 00:02:09,451 INFO  [a.c.c.a.ApiServer]
>(2011638958@qtp-657397168-0:ctx-81b1e088 ctx-174e4a62) (userId=2
>accountId=2 sessionId=7asvmtwoesbc6ia3e4kxtzrl) 127.0.0.1 -- GET
>command=listZones&response=json&sessionkey=ec6h46Om8a1y3d%2BhrdIpQ85cAf
>c%3
>D&_=1379055729422
>200 { "listzonesresponse" : { "count":1 ,"zone" : [ 
>{"id":"cdaf82f1-3b57-4aa4-b3ce-b60173ed45f2","name":"zone1","dns1":"8.8.8.
>8","dns2":"8.8.4.4","internaldns1":"8.8.4.4","networktype":"Basic","sec
>uri
>tygroupsenabled":true,"allocationstate":"Enabled","zonetoken":"6dce94e8
>-e8 
>dc-3077-bfde-c6e8594bd449","dhcpprovider":"VirtualRouter","localstorage
>ena
>bled":false}
>] } }
>
>The sessionId and sessionKey is logged in the file.  I haven't tried it 
>yet, but can't I use that info to hijack the session?  That introduces 
>a security issue in that any server operator can now hijack anybody's 
>session.  So that api.log file really needs to be protected in the same 
>way a file with a password in it would be.
>
>I would suggest that we just don't log the sessionId or sessionKey.
>
>Darren

Re: Can I Suspend a CloudStack VM through the REST API?

[Krasimir Baylov]

So, there are no guest tools or plugins I can take advantage of. It seems
that I'll need to implementing from scratch.


On Wed, Sep 18, 2013 at 11:42 AM, Rajesh Battala
wrote:

> There is no api to suspend/hibernate the VM.
> I think the reason it's not implemented is some Hypervisors might support
> suspend and some might not. Some hypervisors execute
> suspend/pause/hibernate only if the guest tools are installed.
> You can create jira ticket for it and please feel free to implement it.
>
> Thanks
> Rajesh Battala
>
> -Original Message-
> From: Krasimir Baylov [mailto:kbai...@gmail.com]
> Sent: Wednesday, September 18, 2013 1:57 PM
> To: dev@cloudstack.apache.org
> Subject: Can I Suspend a CloudStack VM through the REST API?
>
> Hi all,
>
> Is there a way to hibernate a running VM in CloudStack with the REST API?
> I know I can deploy/start/stop/destroy it but I couldn't find a way of
> hibernating it.
>
> By hibernate I mean - releasing the resources that the VM uses while
> preserving it's state (like SLEEP in Windows). In other words, when I
> resume the VM it should have the same state (running programs, opened
> files,...) as it was before the suspend. (This would need to persist the
> VMs RAM though).
>
> Any hints on what can I do? Is there anything implemented I can use or I
> need to develop it myself. My current setup uses KVM and CloudStack 4.1.1.
>
> Thank you!
> Krasimir
>

Re: Review Request 14167: [GSoC] Adding LB, PF service to GRE controller

[Daan Hoogland]

h Nguyen,

It looks allright but I cannot judge this code without seeing it work. I
hope Hugo or Sebastien can approve it.

regards,
Daan


On Tue, Sep 17, 2013 at 12:04 PM, Nguyen Anh Tu  wrote:

> Hi guys, I made an update patch, which aim to remove commented code and
> old files. Please review it
>
>
> 2013/9/17 tuna 
>
>>This is an automatically generated e-mail. To reply, visit:
>> https://reviews.apache.org/r/14167/
>>
>> On September 17th, 2013, 8:24 a.m. UTC, *daan Hoogland* wrote:
>>
>> Do you have any (unit-)tests or #!human test scenarios to validate your code?
>>
>>  I haven't made unit tests yet. Will do asap. Just try with my own scenarios.
>>
>>
>> - tuna
>>
>> On September 17th, 2013, 3:03 a.m. UTC, tuna wrote:
>>   Review request for cloudstack, Sebastien Goasguen and Hugo Trippaers.
>> By tuna.
>>
>> *Updated Sept. 17, 2013, 3:03 a.m.*
>>  *Repository: * cloudstack-git
>> Description
>>
>> I add a final patch for supporting L3 services (staticNAT, PortForwarding, 
>> LoadBalancing) to GRE controller
>>
>>   Testing
>>
>> Testing done. I will make a screencast demo asap.
>>
>>   Diffs
>>
>>- api/src/com/cloud/network/Network.java (aea496d)
>>- api/src/com/cloud/network/Networks.java (5aede05)
>>- api/src/org/apache/cloudstack/api/ResponseGenerator.java (b8ecef3)
>>- 
>> plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java
>>(9741763)
>>- 
>> plugins/network-elements/ovs/src/com/cloud/network/element/OvsElement.java
>>(3824669)
>>- 
>> plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsTunnelManagerImpl.java
>>(6ad6e83)
>>- scripts/vm/hypervisor/xenserver/ovstunnel (2b26ed6)
>>- server/src/com/cloud/network/NetworkModelImpl.java (d7ca639)
>>- server/src/com/cloud/network/element/VirtualRouterElement.java
>>(ecf6473)
>>- ui/scripts/system.js (18c3df4)
>>
>> View Diff 
>>
>
>
>
> --
>
> N.g.U.y.e.N.A.n.H.t.U
>

Re: mido client

[Daan Hoogland]

Thanks Dave,

I still had the google docs in the proxy and it has the same id, now
both the new and the old are in our cache.

regards,
Daan

On Tue, Sep 17, 2013 at 3:21 AM, Dave Cahill  wrote:
> Daan,
>
> Yes, that's the right location. For example, the client jar file is at:
> http://cs-maven.midokura.com/releases/org/midonet/midonet-client/1.1.0/midonet-client-1.1.0.jar
>
> If you add http://cs-maven.midokura.com/releases as a 3rd-party repo in your
> Sonar repo, it should work fine.
>
> Thanks,
> Dave.
>
>
>
>
>
> On Mon, Sep 16, 2013 at 10:23 PM, Daan Hoogland 
> wrote:
>>
>> http://cs-maven.midokura.com/releases/ is the right location, right? I
>> get a blank page there (no pom or jar) or is it somewhere else?
>>
>> On Mon, Sep 16, 2013 at 3:17 PM, Dave Cahill  wrote:
>> > Copying part of the IM conversation just now in case anyone else has the
>> > same question.
>> >
>> > As Prasanna mentioned in this thread [1]:
>> > "If you are using a nexus proxy in the way of your build, you'll have
>> > to add a 3rd party repo and the jars will be automatically indexed and
>> > downloaded."
>> >
>> > Thanks,
>> > Dave.
>> >
>> > [1]
>> >
>> > http://markmail.org/message/e46n5isogps5g723#query:+page:1+mid:x37wudx5qoxop7ag+state:results
>> >
>> >
>> > On Mon, Sep 16, 2013 at 9:46 PM, Daan Hoogland 
>> > wrote:
>> >>
>> >> H Dave,
>> >>
>> >> I noticed some mails on this but did not find a workaround; I am
>> >> trying to build master and it looks for midonet-client 1.1.0. I have
>> >> removed it from the plugins pom file. Next I do want to test with
>> >> midonet however as it touches networks what I am doing. Do you have
>> >> some advice on how to build while you are uploading it somewhere?
>> >> Meaning cen you place it somewhere or mail me a copy?
>> >>
>> >> regards,
>> >
>> >
>
>

Re: mido client

[Dave Cahill]

Good to hear!


On Wed, Sep 18, 2013 at 6:10 PM, Daan Hoogland wrote:

> Thanks Dave,
>
> I still had the google docs in the proxy and it has the same id, now
> both the new and the old are in our cache.
>
> regards,
> Daan
>
> On Tue, Sep 17, 2013 at 3:21 AM, Dave Cahill  wrote:
> > Daan,
> >
> > Yes, that's the right location. For example, the client jar file is at:
> >
> http://cs-maven.midokura.com/releases/org/midonet/midonet-client/1.1.0/midonet-client-1.1.0.jar
> >
> > If you add http://cs-maven.midokura.com/releases as a 3rd-party repo in
> your
> > Sonar repo, it should work fine.
> >
> > Thanks,
> > Dave.
> >
> >
> >
> >
> >
> > On Mon, Sep 16, 2013 at 10:23 PM, Daan Hoogland  >
> > wrote:
> >>
> >> http://cs-maven.midokura.com/releases/ is the right location, right? I
> >> get a blank page there (no pom or jar) or is it somewhere else?
> >>
> >> On Mon, Sep 16, 2013 at 3:17 PM, Dave Cahill 
> wrote:
> >> > Copying part of the IM conversation just now in case anyone else has
> the
> >> > same question.
> >> >
> >> > As Prasanna mentioned in this thread [1]:
> >> > "If you are using a nexus proxy in the way of your build, you'll have
> >> > to add a 3rd party repo and the jars will be automatically indexed and
> >> > downloaded."
> >> >
> >> > Thanks,
> >> > Dave.
> >> >
> >> > [1]
> >> >
> >> >
> http://markmail.org/message/e46n5isogps5g723#query:+page:1+mid:x37wudx5qoxop7ag+state:results
> >> >
> >> >
> >> > On Mon, Sep 16, 2013 at 9:46 PM, Daan Hoogland <
> daan.hoogl...@gmail.com>
> >> > wrote:
> >> >>
> >> >> H Dave,
> >> >>
> >> >> I noticed some mails on this but did not find a workaround; I am
> >> >> trying to build master and it looks for midonet-client 1.1.0. I have
> >> >> removed it from the plugins pom file. Next I do want to test with
> >> >> midonet however as it touches networks what I am doing. Do you have
> >> >> some advice on how to build while you are uploading it somewhere?
> >> >> Meaning cen you place it somewhere or mail me a copy?
> >> >>
> >> >> regards,
> >> >
> >> >
> >
> >
>

RE: [ACS42] Resolved Defects

[Paul Angus]

Sudha,

No one has picked up CLOUDSTACK-4624 
(https://issues.apache.org/jira/browse/CLOUDSTACK-4624) which is a blocker for 
anyone using security groups in advanced zones.


Regards,

Paul Angus
S: +44 20 3603 0540 | M: +447711418784 | T: CloudyAngus
paul.an...@shapeblue.com

-Original Message-
From: Sudha Ponnaganti [mailto:sudha.ponnaga...@citrix.com]
Sent: 18 September 2013 07:57
To: dev@cloudstack.apache.org
Subject: [ACS42] Resolved Defects

Please validate and close the following as release is getting closed now

Reporter  Total
Abhinandan Prateek  2
Ahmad Emneina   2
Alena Prokharchyk  12
angeline shen5
Anthony Xu4
Ashutosk Kelkar   1
Bharat Kumar12
Brian Angus1
Brian Federle 2
Brian Spindler2
Chandan Purushothama   16
Chiradeep Vittal   1
Dave Brosius  1
Dave Cahill  1
David Noland 1
david vz1
Devdeep Singh 3
Diogo Monteiro1
Fang Wang  4
Francois Gaudreault   1
Gaurav Aradhye   1
Harikrishna Patnala 7
Hiroaki Kawai 2
Hongtu Zang  2
Hugo Trippaers 1
ilya musayev  3
Isaac Chiang   3
Jayapal Reddy   4
Jessica Tomechak2
Jessica Wang  2
John Burwell  3
Kelven Yang   2
Kishan Kavala 9
Koushik Das2
Likitha Shetty 4
Max Clark1
Mice Xia   1
Milamber2
Min Chen1
Minying Bao   2
Murali Reddy 8
Nick Wales  1
Nitin Mehta18
Parth Jagirdar1
Paul Angus  1
Prachi Damle  9
Pranav Saxena  1
Prasanna Santhanam 4
Radhika Nair   8
Rajesh Battala   3
Ram Ganesh  2
Rayees Namathponnan6
roxanne chang  1
sadhu suresh 4
Sailaja Mada   4
Saksham Srivastava 8
Sangeetha Hariharan  2
Sanjay Tripathi  1
Sanjeev N   5
Sateesh Chodapuneedi 11
sebastien goasguen1
Shane Witbeck  1
Shanker Balan   1
Sheng Yang 3
shweta agarwal1
Simon Waterhouse 1
Simon Weller 1
Sowmya Krishnan3
Sreekanth Challa  1
Srikanteswararao Talluri2
Sudha Ponnaganti   1
Toshiaki Hatano2
Venkata Siva Vijayendra Bhamidipati  4
venkata swamybabu budumuru   2
XxYton  1
Grand Total250
This email and any attachments to it may be confidential and are intended 
solely for the use of the individual to whom it is addressed. Any views or 
opinions expressed are solely those of the author and do not necessarily 
represent those of Shape Blue Ltd or related companies. If you are not the 
intended recipient of this email, you must neither take any action based upon 
its contents, nor copy or show it to anyone. Please contact the sender if you 
believe you have received this email in error. Shape Blue Ltd is a company 
incorporated in England & Wales. ShapeBlue Services India LLP is operated under 
license from Shape Blue Ltd. ShapeBlue is a registered trademark.

Review Request 14199: Adding missing test cases: Snapshots Improvement

[Gaurav Aradhye]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14199/
---

Review request for cloudstack and Prasanna Santhanam.


Repository: cloudstack-git


Description
---

Adding 5 missing test cases.
Change in asyncJobMgr.py >> the method "make_request" in cloudstackConnection 
does not exist now. Replaced it by "marvin_request". Checked running async jobs 
with this change. Works correctly.


Diffs
-

  test/integration/component/test_snapshots_improvement.py PRE-CREATION 
  tools/marvin/marvin/asyncJobMgr.py 25818a6 

Diff: https://reviews.apache.org/r/14199/diff/


Testing
---

Tested locally.


Thanks,

Gaurav Aradhye

Re: [PROPOSAL] Modularize Spring

[Daan Hoogland]

sounds great Darren,

By component, you mean maven project or some larger chunk like
distribution package? (did i miss this definition somewhere or do we
define the components now?)

regards,
Daan

On Wed, Sep 18, 2013 at 12:10 AM, Darren Shepherd
 wrote:
> Currently ACS code is fairly modular in that you can add plug-ins to ACS to
> extend most functionality.  Unfortunately ACS is not packaged in a modular
> way.  It is still delivered essentially as one large unit.  There are many
> reason for this but one large barrier to modularizing ACS is that the
> Spring configuration is managed as one large unit.
>
> I propose that we break apart the Spring XML configuration such that each
> component contributes its own configuration.  Additionally each component
> will be loaded into its own Spring ApplicationContext such that its beans
> will not conflict with the wiring of other beans in ACS.  This change will
> lay the foundation for a richer plugin model and additionally a more
> distributed architecture.
>
> The technical details for this proposal can be found on the wiki [1].
>
> Darren
>
> [1] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Spring

RE: [ACS42] Resolved Defects

[Sudha Ponnaganti]

Thanks for reminding about this one. I have changed priority and target version 
so it would show up in proper queries to be picked up by community members

Thanks
/sudha 


-Original Message-
From: Paul Angus [mailto:paul.an...@shapeblue.com] 
Sent: Wednesday, September 18, 2013 3:45 AM
To: dev@cloudstack.apache.org
Subject: RE: [ACS42] Resolved Defects 

Sudha,

No one has picked up CLOUDSTACK-4624 
(https://issues.apache.org/jira/browse/CLOUDSTACK-4624) which is a blocker for 
anyone using security groups in advanced zones.


Regards,

Paul Angus
S: +44 20 3603 0540 | M: +447711418784 | T: CloudyAngus paul.an...@shapeblue.com

-Original Message-
From: Sudha Ponnaganti [mailto:sudha.ponnaga...@citrix.com]
Sent: 18 September 2013 07:57
To: dev@cloudstack.apache.org
Subject: [ACS42] Resolved Defects

Please validate and close the following as release is getting closed now

Reporter  Total
Abhinandan Prateek  2
Ahmad Emneina   2
Alena Prokharchyk  12
angeline shen5
Anthony Xu4
Ashutosk Kelkar   1
Bharat Kumar12
Brian Angus1
Brian Federle 2
Brian Spindler2
Chandan Purushothama   16
Chiradeep Vittal   1
Dave Brosius  1
Dave Cahill  1
David Noland 1
david vz1
Devdeep Singh 3
Diogo Monteiro1
Fang Wang  4
Francois Gaudreault   1
Gaurav Aradhye   1
Harikrishna Patnala 7
Hiroaki Kawai 2
Hongtu Zang  2
Hugo Trippaers 1
ilya musayev  3
Isaac Chiang   3
Jayapal Reddy   4
Jessica Tomechak2
Jessica Wang  2
John Burwell  3
Kelven Yang   2
Kishan Kavala 9
Koushik Das2
Likitha Shetty 4
Max Clark1
Mice Xia   1
Milamber2
Min Chen1
Minying Bao   2
Murali Reddy 8
Nick Wales  1
Nitin Mehta18
Parth Jagirdar1
Paul Angus  1
Prachi Damle  9
Pranav Saxena  1
Prasanna Santhanam 4
Radhika Nair   8
Rajesh Battala   3
Ram Ganesh  2
Rayees Namathponnan6
roxanne chang  1
sadhu suresh 4
Sailaja Mada   4
Saksham Srivastava 8
Sangeetha Hariharan  2
Sanjay Tripathi  1
Sanjeev N   5
Sateesh Chodapuneedi 11
sebastien goasguen1
Shane Witbeck  1
Shanker Balan   1
Sheng Yang 3
shweta agarwal1
Simon Waterhouse 1
Simon Weller 1
Sowmya Krishnan3
Sreekanth Challa  1
Srikanteswararao Talluri2
Sudha Ponnaganti   1
Toshiaki Hatano2
Venkata Siva Vijayendra Bhamidipati  4
venkata swamybabu budumuru   2
XxYton  1
Grand Total250
This email and any attachments to it may be confidential and are intended 
solely for the use of the individual to whom it is addressed. Any views or 
opinions expressed are solely those of the author and do not necessarily 
represent those of Shape Blue Ltd or related companies. If you are not the 
intended recipient of this email, you must neither take any action based upon 
its contents, nor copy or show it to anyone. Please contact the sender if you 
believe you have received this email in error. Shape Blue Ltd is a company 
incorporated in England & Wales. ShapeBlue Services India LLP is operated under 
license from Shape Blue Ltd. ShapeBlue is a registered trademark.

Re: [PROPOSAL] Modularize Spring

[Daan Hoogland]

Darren,

I had a quick look at your four submissions, Is there no dependency
between them? if so, can you add those to the review requests?

thanks,
Daan

On Wed, Sep 18, 2013 at 1:25 PM, Daan Hoogland  wrote:
> sounds great Darren,
>
> By component, you mean maven project or some larger chunk like
> distribution package? (did i miss this definition somewhere or do we
> define the components now?)
>
> regards,
> Daan
>
> On Wed, Sep 18, 2013 at 12:10 AM, Darren Shepherd
>  wrote:
>> Currently ACS code is fairly modular in that you can add plug-ins to ACS to
>> extend most functionality.  Unfortunately ACS is not packaged in a modular
>> way.  It is still delivered essentially as one large unit.  There are many
>> reason for this but one large barrier to modularizing ACS is that the
>> Spring configuration is managed as one large unit.
>>
>> I propose that we break apart the Spring XML configuration such that each
>> component contributes its own configuration.  Additionally each component
>> will be loaded into its own Spring ApplicationContext such that its beans
>> will not conflict with the wiring of other beans in ACS.  This change will
>> lay the foundation for a richer plugin model and additionally a more
>> distributed architecture.
>>
>> The technical details for this proposal can be found on the wiki [1].
>>
>> Darren
>>
>> [1] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Spring

Review Request 14200: CLOUDSTACK-4686: Fixed volume limit for domain

[Girish Shilamkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14200/
---

Review request for cloudstack and sailaja mada.


Bugs: CLOUDSTACK-4686


Repository: cloudstack-git


Description
---

CLOUDSTACK-4686: Fixed volume limit for domain


Diffs
-

  test/integration/component/test_resource_limits.py 833723c 

Diff: https://reviews.apache.org/r/14200/diff/


Testing
---


Thanks,

Girish Shilamkar

Re: [DISCUSS] Java 7, tomcat 7 and further upgrades

[Hugo Trippaers]

Hey all,

Sorry for the threadomancy, but the discussion have become relevant again with 
the current issues with the libvirt library. Of course this could also be 
solved by updating the libvirt library with a jdk6 version. Still it might be 
good to revisit this topic.

It appears not to be possible to switch code style to 1.7 and produce a 1.6 
compatible binary. I remember this working with olders versions, but didn't dig 
to much into this.

So the new question in this thread will be, should we switch CloudStack to jdk 
1.7?

Cheers,

Hugo

On Jul 1, 2013, at 12:45 AM, Prasanna Santhanam  wrote:

> On Thu, Jun 27, 2013 at 04:18:40PM -0400, John Burwell wrote:
>> All,
>> 
>> I am +1 for Java7.  However, I would like to propose ridding
>> ourselves of Tomcat entirely and embedding a network stack such as
>> Netty (http://netty.io) with a servlet bridge.  We have one JSP in
>> the system that generates JSON resources.  It could be easily
>> eliminated with a simple servlet that generates JSON from a
>> ResourceBundle.  Outside of this JSP. I don't see any other
>> requirements for a JEE container besides hosting a servlet.  We
>> would gain a far simpler, self-contained deployment model (a single
>> jar).  Additionally, we would gain greater control of the startup
>> and shutdown lifecycle, as well as, threading dynamics.  If there is
>> interest in this approach, I have thoughts on how to achieve this
>> embedding and create a lightweight daemon framework that could be
>> used for all CloudStack daemons.
>> 
>> As an aside, I also think we should replace our hand-rolled NIO code
>> with Netty as well.
>> 
> 
> John - could you break this and other thoughts down a little more in
> what's involved?  Perhaps into its own thread. I don't know Netty. And
> my J2EE is shaky at best. 
> 
> It's been a previous wish on this list to have the packaging of
> cloudstack into a single easily deployable war instead of all the
> complicated packaging we do. So I'd like to hear more of that and
> other issues you describe.
> 
> -- 
> Prasanna.,
> 
> 
> Powered by BigRock.com
> 

[PROPOSAL] Storage Subsystem API Interface Additions

[SuichII, Christopher]

I would like to raise for discussion the idea of adding a couple methods to the 
Storage Subsystem API interface. Currently, takeSnapshot() and revertSnapshot() 
only support single VM volumes. We have a use case for snapshotting multiple VM 
volumes at the same time. For us, it is more efficient to snapshot them all at 
once rather than snapshot VM Volumes individually and this seems like a more 
elegant solution than queueing the requests within our plugin.

Base on my investigation, this should require:
-Two additional API to be invoked from the UI
-Two additional methods added to the Storage Subsystem API interface
-Changes in between the API level and invoking the Storage Subsystem API 
implementations (I know this is broad and vague), mainly around the 
SnapshotManger/Impl

There are a couple topics we would like discussion on:
-Would this be beneficial/detrimental/neutral to other storage providers?
-How should we handle the addition of new methods to the Storage Subsystem API 
interface? Default them to throw an UnsupportedOperationException? Default to 
calling the single VM volume version multiple times?
-Does anyone see any issues with allowing multiple snapshots to be taken at the 
same time or letting storage providers have a list of all the requested volumes 
to backup?

Please let me know if I've missed any major topics for discussion or if 
anything needs clarification.

Thanks,
Chris
-- 
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

Removal of cloud-plugin-netapp

[SuichII, Christopher]

I remember seeing/hearing some discussion about removing the 
cloud-plugin-netapp component since it is not used or really managed any more. 
Is this still in the works? One of the dependencies (manageontap.jar) will 
conflict with one of the dependencies of our (in development) plugin. The 
version of manageontap.jar used by CloudStack is different than what we require 
in our plugin and would likely cause runtime issues if still included in the MS 
classpath.

Any thoughts?

Thanks,
Chris
-- 
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

Re: [DISCUSS] CSS framework for CloudStack UI

[Chip Childers]

On Wed, Sep 18, 2013 at 11:53:18AM +0530, Amit Das wrote:
> Hi Brian,
> 
> I agree with Edison on usage of grunt & using maven-exec to call grunt.
> 
> Will wait for your repository that has your experiments.
> I believe setting up the Maven tasks will be a one-time setting & should
> work without issues.

IIRC, Grunt is installed via NPM.  So does that pull in a bunch of new
developer requirements to build the project?  Is there a standalone
installation for Grunt to lighten the build dependency chain?

How about using SassC? [1]

Let's be sure to use the scss spec, not the sass older style (HAML
inspired)!  That appears to be Hampton's focus these days [2].

-chip

[1] https://github.com/hcatlin/sassc
[2] Per intro on http://sass-lang.com/

quality control/test automation meeting

[Daan Hoogland]

H,

I kept 19:00 cet-daylight saving time open for the meeting discussed
in some threads. Are we going on with it and if so on what medium. For
now I am assuming #cloudstack-meeting but I remember Sebastien writing
about a teleconference.

Who calls it?

regards.
Daan

Re: Review Request 13934: CLOUDSTACK-4347 provisioning of a nicira based vpc router

[daan Hoogland]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13934/
---

(Updated Sept. 18, 2013, 1:45 p.m.)


Review request for cloudstack, Chiradeep Vittal, Hugo Trippaers, and Sheng Yang.


Changes
---

tested with regular and bridged networks


Bugs: 4347


Repository: cloudstack-git


Description
---

provisioning of a nicira based vpc router


Diffs (updated)
-

  api/src/com/cloud/agent/api/to/IpAddressTO.java 82c7d99 
  api/src/com/cloud/agent/api/to/NetworkTO.java 3edd4c0 
  api/src/com/cloud/network/NetworkService.java 87fecb0 
  api/src/com/cloud/network/Networks.java f8166c6 
  api/src/com/cloud/network/vpc/PrivateIp.java eb68433 
  api/src/com/cloud/network/vpc/StaticRouteProfile.java 54aa6e4 
  api/src/com/cloud/network/vpc/VpcGateway.java 9652b4b 
  api/src/com/cloud/network/vpc/VpcService.java f772879 
  
api/src/org/apache/cloudstack/api/command/admin/vpc/CreatePrivateGatewayCmd.java
 be7e784 
  api/src/org/apache/cloudstack/api/response/PrivateGatewayResponse.java 
c5c7df5 
  api/test/com/cloud/network/NetworksTest.java 07b55d2 
  core/src/com/cloud/agent/resource/virtualnetwork/VirtualRoutingResource.java 
1fb86e0 
  engine/schema/src/com/cloud/network/dao/NetworkDao.java d0a1a25 
  engine/schema/src/com/cloud/network/dao/NetworkDaoImpl.java 0f83815 
  engine/schema/src/com/cloud/network/vpc/VpcGatewayVO.java 2c592cd 
  engine/schema/src/com/cloud/upgrade/DatabaseUpgradeChecker.java eb987ea 
  
plugins/hypervisors/baremetal/src/com/cloud/baremetal/networkservice/BaremetaNetworkGuru.java
 56bbb28 
  
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/BridgeVifDriver.java
 c94856d 
  
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
 203587a 
  
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/OvsVifDriver.java 
119f117 
  plugins/hypervisors/ovm/src/com/cloud/ovm/hypervisor/OvmResourceBase.java 
8f21c13 
  
plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
 2253586 
  
plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java
 3cc60db 
  
plugins/network-elements/bigswitch-vns/src/com/cloud/network/guru/BigSwitchVnsGuestNetworkGuru.java
 daf65a1 
  
plugins/network-elements/cisco-vnmc/src/com/cloud/network/element/CiscoVnmcElement.java
 c053856 
  
plugins/network-elements/f5/src/com/cloud/network/resource/F5BigIpResource.java 
ffddd30 
  
plugins/network-elements/juniper-srx/src/com/cloud/network/resource/JuniperSrxResource.java
 46ef332 
  
plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java
 58541c6 
  
plugins/network-elements/nicira-nvp/src/com/cloud/network/element/NiciraNvpElement.java
 686f79c 
  
plugins/network-elements/nicira-nvp/src/com/cloud/network/guru/NiciraNvpGuestNetworkGuru.java
 fc1ecd0 
  
plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsTunnelManagerImpl.java
 e8ff1a0 
  server/src/com/cloud/api/ApiResponseHelper.java 1ad0eab 
  server/src/com/cloud/configuration/ConfigurationManagerImpl.java 17ef6bf 
  server/src/com/cloud/network/ExternalDeviceUsageManagerImpl.java 3db5111 
  server/src/com/cloud/network/ExternalFirewallDeviceManagerImpl.java e4625a4 
  server/src/com/cloud/network/ExternalLoadBalancerDeviceManagerImpl.java 
b86f4ae 
  server/src/com/cloud/network/NetworkServiceImpl.java eb63fe0 
  server/src/com/cloud/network/guru/DirectPodBasedNetworkGuru.java 67ebef1 
  server/src/com/cloud/network/guru/ExternalGuestNetworkGuru.java 55a33cc 
  server/src/com/cloud/network/guru/GuestNetworkGuru.java e90b84b 
  server/src/com/cloud/network/guru/PrivateNetworkGuru.java 12dce85 
  
server/src/com/cloud/network/router/VpcVirtualNetworkApplianceManagerImpl.java 
566f1e4 
  server/src/com/cloud/network/vpc/PrivateGatewayProfile.java 74ce002 
  server/src/com/cloud/network/vpc/PrivateIpAddress.java 2f3cf53 
  server/src/com/cloud/network/vpc/VpcManagerImpl.java 6cace7e 
  server/test/com/cloud/network/CreatePrivateNetworkTest.java ed9efd6 
  server/test/com/cloud/vpc/MockNetworkManagerImpl.java 003b774 
  server/test/com/cloud/vpc/dao/MockNetworkDaoImpl.java ec1a017 
  utils/src/com/cloud/utils/net/NetUtils.java 7a89103 

Diff: https://reviews.apache.org/r/13934/diff/


Testing
---

This is a rebase/port of code that has been tested intensively. It is here for 
review only at this time. re-tests are needed.


Thanks,

daan Hoogland

Re: Review Request 13992: (CLOUDSTACK-4405) additional patch for bridge name and firewall rules issues after KVM upgrade to 4.2

[Wei Zhou]

> On Sept. 9, 2013, 2:34 p.m., Wei Zhou wrote:
> > please ignore the r2 version for issue CLOUDSTACK-3565

CLOUDSTACK-3565 is fixed, so I will commit r2 version if nobody objects.


- Wei


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13992/#review25989
---


On Sept. 9, 2013, 8:09 a.m., Wei Zhou wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13992/
> ---
> 
> (Updated Sept. 9, 2013, 8:09 a.m.)
> 
> 
> Review request for cloudstack and edison su.
> 
> 
> Bugs: CLOUDSTACK-4405
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> There still exist two issues after Edison's commits.
> (1) Migration from new hosts to old hosts failed.
> The bridge name on old host is set to cloudVirBr* if 
> network.bridge.name.schema is set to 3.0 in 
> /etc/cloudstack/agent/agent.properties, but the actual bridge name is 
> breth*-* after running cloudstack-agent-upgrade.
> 
> (2) all ports of vms (Basic zone, or Advanced zone with security groups) on 
> old hosts are open, because the iptables rules are binding to device (bridge) 
> name which is changed by cloudstack-agent-upgrade.
> 
> 
> Diffs
> -
> 
>   agent/bindir/cloudstack-agent-upgrade.in 4972d39 
>   debian/cloudstack-agent.postinst 499ae6a 
>   debian/rules 5e3d58c 
>   packaging/centos63/cloud.spec 2b814f8 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/BridgeVifDriver.java
>  e3779a7 
>   scripts/vm/network/security_group.py 0ac8b74 
> 
> Diff: https://reviews.apache.org/r/13992/diff/
> 
> 
> Testing
> ---
> 
> tested ok on my environment.
> 
> After this, the KVM upgrade steps :
> a. Install 4.2 cloudstack agent on each kvm host 
> b. Run "cloudstack-agent-upgrade". This script will upgrade all the existing 
> bridge name to new bridge name, and update related firewall rules.
>c. install a libvirt hook:
> c1. mkdir /etc/libvirt/hooks
> c2. cp /usr/share/cloudstack-agent/lib/libvirtqemuhook 
> /etc/libvirt/hooks/qemu
> c3. chmod +x /etc/libvirt/hooks/qemu
> c4. service libvirtd restart
> c5. service cloudstack-agent restart
> 
> 
> Thanks,
> 
> Wei Zhou
> 
>

Re: Review Request 13992: (CLOUDSTACK-4405) additional patch for bridge name and firewall rules issues after KVM upgrade to 4.2

[Daan Hoogland]

please go ahead, I will change BridgeVifDriver as per
https://reviews.apache.org/r/13934. i see no conflicts

On Wed, Sep 18, 2013 at 3:48 PM, Wei Zhou  wrote:
>
>
>> On Sept. 9, 2013, 2:34 p.m., Wei Zhou wrote:
>> > please ignore the r2 version for issue CLOUDSTACK-3565
>
> CLOUDSTACK-3565 is fixed, so I will commit r2 version if nobody objects.
>
>
> - Wei
>
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13992/#review25989
> ---
>
>
> On Sept. 9, 2013, 8:09 a.m., Wei Zhou wrote:
>>
>> ---
>> This is an automatically generated e-mail. To reply, visit:
>> https://reviews.apache.org/r/13992/
>> ---
>>
>> (Updated Sept. 9, 2013, 8:09 a.m.)
>>
>>
>> Review request for cloudstack and edison su.
>>
>>
>> Bugs: CLOUDSTACK-4405
>>
>>
>> Repository: cloudstack-git
>>
>>
>> Description
>> ---
>>
>> There still exist two issues after Edison's commits.
>> (1) Migration from new hosts to old hosts failed.
>> The bridge name on old host is set to cloudVirBr* if 
>> network.bridge.name.schema is set to 3.0 in 
>> /etc/cloudstack/agent/agent.properties, but the actual bridge name is 
>> breth*-* after running cloudstack-agent-upgrade.
>>
>> (2) all ports of vms (Basic zone, or Advanced zone with security groups) on 
>> old hosts are open, because the iptables rules are binding to device 
>> (bridge) name which is changed by cloudstack-agent-upgrade.
>>
>>
>> Diffs
>> -
>>
>>   agent/bindir/cloudstack-agent-upgrade.in 4972d39
>>   debian/cloudstack-agent.postinst 499ae6a
>>   debian/rules 5e3d58c
>>   packaging/centos63/cloud.spec 2b814f8
>>   
>> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/BridgeVifDriver.java
>>  e3779a7
>>   scripts/vm/network/security_group.py 0ac8b74
>>
>> Diff: https://reviews.apache.org/r/13992/diff/
>>
>>
>> Testing
>> ---
>>
>> tested ok on my environment.
>>
>> After this, the KVM upgrade steps :
>> a. Install 4.2 cloudstack agent on each kvm host
>> b. Run "cloudstack-agent-upgrade". This script will upgrade all the existing 
>> bridge name to new bridge name, and update related firewall rules.
>>c. install a libvirt hook:
>> c1. mkdir /etc/libvirt/hooks
>> c2. cp /usr/share/cloudstack-agent/lib/libvirtqemuhook 
>> /etc/libvirt/hooks/qemu
>> c3. chmod +x /etc/libvirt/hooks/qemu
>> c4. service libvirtd restart
>> c5. service cloudstack-agent restart
>>
>>
>> Thanks,
>>
>> Wei Zhou
>>
>>
>

Re: [jira] [Commented] (CLOUDSTACK-4690) KVM Router - to many ethernet devices created

[Marcus Sorensen]

Agent needs debug enabled, all lines are INFO.
On Sep 18, 2013 3:36 AM, "Wei Zhou (JIRA)"  wrote:

>
> [
> https://issues.apache.org/jira/browse/CLOUDSTACK-4690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13770615#comment-13770615]
>
> Wei Zhou commented on CLOUDSTACK-4690:
> --
>
> I met this issue once in 4.0.0.
> It is because I built the jars in Windows, and replaced one jar in
> management server with it.
> After I built source code in CentOS with maven/ant, it was fixed.
>
> If you installed the packages from official packages (on both mgt and
> hosts), it could be an issue.
>
> > KVM Router - to many ethernet devices created
> > -
> >
> > Key: CLOUDSTACK-4690
> > URL:
> https://issues.apache.org/jira/browse/CLOUDSTACK-4690
> > Project: CloudStack
> >  Issue Type: Bug
> >  Security Level: Public(Anyone can view this level - this is the
> default.)
> >  Components: KVM
> >Affects Versions: 4.1.1
> > Environment: Centos 6.4
> >Reporter: Paul Edwards
> >Priority: Critical
> > Attachments: management-server.log, router.log
> >
> >
> > We have setup cloudstack with advanced networking. We have a network
> created 172.16.24.0/23, called news preprod. This has a number of vms
> created under it. When we initially set this up, we noticed that the router
> we created with 4 ethernet interfaces. This was unexpected, but didn't seem
> to be effecting the running of the router, so we didn't worry about it. The
> interfaces were: eth0 172.16.24.14, eth1 169.254.1.91, and both eth2 and
> eth2 were given the external ip (xxx.xxx.245.59). We investigated why we
> were getting 2 externals, but couldn't figure it out. We also configured
> the router to have a redundant, with keepalived. They were working fine.
> > Now I needed to add another external ip to the network. Did that, and
> restarted the network. I now have 8 (eight) eth interfaces. The new
> external ip is NOT one of them. Heres the ifconfig output:
> > root@r-235-VM:~# ifconfig
> > eth0  Link encap:Ethernet  HWaddr 02:00:68:e1:00:1a
> >   inet addr:172.16.24.14  Bcast:172.16.25.255  Mask:255.255.254.0
> >   inet6 addr: fe80::68ff:fee1:1a/64 Scope:Link
> >   UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >   RX packets:5485 errors:0 dropped:0 overruns:0 frame:0
> >   TX packets:21101 errors:0 dropped:0 overruns:0 carrier:0
> >   collisions:0 txqueuelen:1000
> >   RX bytes:325618 (317.9 KiB)  TX bytes:1581746 (1.5 MiB)
> > eth1  Link encap:Ethernet  HWaddr 0e:00:a9:fe:01:5b
> >   inet addr:169.254.1.91  Bcast:169.254.255.255  Mask:255.255.0.0
> >   inet6 addr: fe80::c00:a9ff:fefe:15b/64 Scope:Link
> >   UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >   RX packets:22118 errors:0 dropped:0 overruns:0 frame:0
> >   TX packets:21148 errors:0 dropped:0 overruns:0 carrier:0
> >   collisions:0 txqueuelen:1000
> >   RX bytes:3942785 (3.7 MiB)  TX bytes:4287970 (4.0 MiB)
> > eth2  Link encap:Ethernet  HWaddr 06:16:06:00:01:03
> >   inet addr:xxx.xxx.245.59  Bcast:xxx.xxx.245.255
>  Mask:255.255.255.0
> >   inet6 addr: fe80::416:6ff:fe00:103/64 Scope:Link
> >   UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >   RX packets:17331 errors:0 dropped:0 overruns:0 frame:0
> >   TX packets:3930 errors:0 dropped:0 overruns:0 carrier:0
> >   collisions:0 txqueuelen:1000
> >   RX bytes:1063053 (1.0 MiB)  TX bytes:234546 (229.0 KiB)
> > eth3  Link encap:Ethernet  HWaddr 06:d7:ec:00:01:03
> >   inet addr:xxx.xxx.245.59  Bcast:xxx.xxx.245.255
>  Mask:255.255.255.0
> >   inet6 addr: fe80::4d7:ecff:fe00:103/64 Scope:Link
> >   UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >   RX packets:13603 errors:0 dropped:0 overruns:0 frame:0
> >   TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
> >   collisions:0 txqueuelen:1000
> >   RX bytes:792907 (774.3 KiB)  TX bytes:704 (704.0 B)
> > eth4  Link encap:Ethernet  HWaddr 06:8b:5a:00:01:03
> >   inet addr:xxx.xxx.245.59  Bcast:xxx.xxx.245.255
>  Mask:255.255.255.0
> >   inet6 addr: fe80::48b:5aff:fe00:103/64 Scope:Link
> >   UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >   RX packets:13319 errors:0 dropped:0 overruns:0 frame:0
> >   TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
> >   collisions:0 txqueuelen:1000
> >   RX bytes:779753 (761.4 KiB)  TX bytes:704 (704.0 B)
> > eth5  Link encap:Ethernet  HWaddr 06:7e:68:00:01:03
> >   inet addr:xxx.xxx.245.59  Bcast:xxx.xxx.245.255
>  Mask:255.255.255.0
> >   inet6 addr: fe80::47e:68ff:fe00:103/64 Scope:Lin

Re: Review Request 14167: [GSoC] Adding LB, PF service to GRE controller

[Daan Hoogland]

/me know your pain, I am feeling it on a long standing change right
now. The pain needs to be felt and should lead to better
modularization of the code. Having some regression tests in place that
will force other committers to take into account the code you depend
on is one thing. This won't help you now. I'm afraid you are going to
have to byte the bullet.

On Wed, Sep 18, 2013 at 4:00 PM, Nguyen Anh Tu  wrote:
>
> 2013/9/18 Daan Hoogland 
>>
>> It looks allright but I cannot judge this code without seeing it work. I
>> hope Hugo or Sebastien can approve it.
>
>
> Yah. About rebase code, I tried to rebase my sdnextensions with 4.1 branch,
> but there are so many conflict need to fix manually. Do you have any tips to
> do it easier?
>
>
> --
>
> N.g.U.y.e.N.A.n.H.t.U

Re: Review Request 14167: [GSoC] Adding LB, PF service to GRE controller

[Nguyen Anh Tu]

2013/9/18 Daan Hoogland 

> It looks allright but I cannot judge this code without seeing it work. I
> hope Hugo or Sebastien can approve it.


Yah. About rebase code, I tried to rebase my sdnextensions with 4.1 branch,
but there are so many conflict need to fix manually. Do you have any tips
to do it easier?


-- 

N.g.U.y.e.N.A.n.H.t.U

Re: Review Request 14058: Including tests for VPC VM Lifecycle on Tagged hosts

[Ashutosh Kelkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14058/
---

(Updated Sept. 18, 2013, 2:28 p.m.)


Review request for cloudstack, Girish Shilamkar, venkata swamy babu  budumuru, 
and Sheng Yang.


Repository: cloudstack-git


Description
---

Added 10 tests for VPC VM liftcycle on tagged hosts

New class added  : TestVMLifeCycleDiffHosts

def test_01_deploy_instance_in_network(self):
def test_02_stop_instance_in_network(self):
def test_03_start_instance_in_network(self):
def test_04_reboot_instance_in_network(self):
def test_05_destroy_instance_in_network(self):
def test_06_recover_instance_in_network(self):
def test_07_migrate_instance_in_network(self):
def test_08_user_data(self):
def test_09_meta_data(self):
def test_10_expunge_instance_in_network(self):

This set of tests requires a multi host tagged setup (3 hosts - 2 hosts with 
tag 'host1' and  1 host with tag 'host2')


Diffs
-

  test/integration/component/test_vpc_vm_life_cycle.py 9844c1f 

Diff: https://reviews.apache.org/r/14058/diff/


Testing
---


Thanks,

Ashutosh Kelkar

Re: Review Request 14075: Added new test for Load balancing rules in a VPC

[Ashutosh Kelkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14075/
---

(Updated Sept. 18, 2013, 2:28 p.m.)


Review request for cloudstack, Girish Shilamkar, venkata swamy babu  budumuru, 
and Prasanna Santhanam.


Repository: cloudstack-git


Description
---

Added missing test for VPC load balancing rule from old QA repo to cloudstack 
repo

def test_04_VPC_CreateLBRuleInMultipleNetworksVRStoppedState

Since VPC now only allows load balancing on a single tier the test case
has been updated to check that condition


Diffs
-

  test/integration/component/test_vpc_network_lbrules.py de29ce1 

Diff: https://reviews.apache.org/r/14075/diff/


Testing
---


Thanks,

Ashutosh Kelkar

Re: Review Request 13841: Missing tests from QA repo to ASF - 3 tests from test_vmware_drs.py

[Ashutosh Kelkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13841/
---

(Updated Sept. 18, 2013, 2:28 p.m.)


Review request for cloudstack, Girish Shilamkar, Harikrishna Patnala, and 
Prasanna Santhanam.


Repository: cloudstack-git


Description
---

New File added: test_vmware_drs.py
Tests added   : def test_vm_creation_in_fully_automated_mode(self):
def test_vmware_anti_affinity(self):
def test_vmware_affinity(self):

The tests need manual setup and therefore have been marked as WIP and
skipped for the moment


Diffs
-

  test/integration/component/test_vmware_drs.py PRE-CREATION 

Diff: https://reviews.apache.org/r/13841/diff/


Testing
---


Thanks,

Ashutosh Kelkar

Re: Review Request 14076: New test added for template copy feature

[Ashutosh Kelkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14076/
---

(Updated Sept. 18, 2013, 2:29 p.m.)


Review request for cloudstack, Girish Shilamkar, Harikrishna Patnala, and 
Prasanna Santhanam.


Repository: cloudstack-git


Description
---

Added one missing test for test_templates.py from old QA repo to Cloudstack 

def test_02_copy_template


Diffs
-

  test/integration/component/test_templates.py e4599d4 

Diff: https://reviews.apache.org/r/14076/diff/


Testing
---


Thanks,

Ashutosh Kelkar

workshops at CCC Europe

[Daan Hoogland]

H,

In november in Amterdam there are going to be hackathons and other
workshops held.  Last time David (N.) started a page on this so I
think it would be a good idea to do this again.

https://cwiki.apache.org/confluence/display/CLOUDSTACK/CCC+Europe

please comment and add to it

Re: Review Request 13934: CLOUDSTACK-4347 provisioning of a nicira based vpc router

[Daan Hoogland]

I want to apply this now, any objections?


On Wed, Sep 18, 2013 at 3:45 PM, daan Hoogland wrote:

>This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13934/
>   Review request for cloudstack, Chiradeep Vittal, Hugo Trippaers, and
> Sheng Yang.
> By daan Hoogland.
>
> *Updated Sept. 18, 2013, 1:45 p.m.*
> Changes
>
> tested with regular and bridged networks
>
>   *Bugs: * 4347
>  *Repository: * cloudstack-git
> Description
>
> provisioning of a nicira based vpc router
>
>   Testing
>
> This is a rebase/port of code that has been tested intensively. It is here 
> for review only at this time. re-tests are needed.
>
>   Diffs (updated)
>
>- api/src/com/cloud/agent/api/to/IpAddressTO.java (82c7d99)
>- api/src/com/cloud/agent/api/to/NetworkTO.java (3edd4c0)
>- api/src/com/cloud/network/NetworkService.java (87fecb0)
>- api/src/com/cloud/network/Networks.java (f8166c6)
>- api/src/com/cloud/network/vpc/PrivateIp.java (eb68433)
>- api/src/com/cloud/network/vpc/StaticRouteProfile.java (54aa6e4)
>- api/src/com/cloud/network/vpc/VpcGateway.java (9652b4b)
>- api/src/com/cloud/network/vpc/VpcService.java (f772879)
>- 
> api/src/org/apache/cloudstack/api/command/admin/vpc/CreatePrivateGatewayCmd.java
>(be7e784)
>- api/src/org/apache/cloudstack/api/response/PrivateGatewayResponse.java
>(c5c7df5)
>- api/test/com/cloud/network/NetworksTest.java (07b55d2)
>- 
> core/src/com/cloud/agent/resource/virtualnetwork/VirtualRoutingResource.java
>(1fb86e0)
>- engine/schema/src/com/cloud/network/dao/NetworkDao.java (d0a1a25)
>- engine/schema/src/com/cloud/network/dao/NetworkDaoImpl.java (0f83815)
>- engine/schema/src/com/cloud/network/vpc/VpcGatewayVO.java (2c592cd)
>- engine/schema/src/com/cloud/upgrade/DatabaseUpgradeChecker.java
>(eb987ea)
>- 
> plugins/hypervisors/baremetal/src/com/cloud/baremetal/networkservice/BaremetaNetworkGuru.java
>(56bbb28)
>- 
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/BridgeVifDriver.java
>(c94856d)
>- 
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
>(203587a)
>- 
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/OvsVifDriver.java
>(119f117)
>- plugins/hypervisors/ovm/src/com/cloud/ovm/hypervisor/OvmResourceBase.java
>(8f21c13)
>- 
> plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
>(2253586)
>- 
> plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java
>(3cc60db)
>- 
> plugins/network-elements/bigswitch-vns/src/com/cloud/network/guru/BigSwitchVnsGuestNetworkGuru.java
>(daf65a1)
>- 
> plugins/network-elements/cisco-vnmc/src/com/cloud/network/element/CiscoVnmcElement.java
>(c053856)
>- 
> plugins/network-elements/f5/src/com/cloud/network/resource/F5BigIpResource.java
>(ffddd30)
>- 
> plugins/network-elements/juniper-srx/src/com/cloud/network/resource/JuniperSrxResource.java
>(46ef332)
>- 
> plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java
>(58541c6)
>- 
> plugins/network-elements/nicira-nvp/src/com/cloud/network/element/NiciraNvpElement.java
>(686f79c)
>- 
> plugins/network-elements/nicira-nvp/src/com/cloud/network/guru/NiciraNvpGuestNetworkGuru.java
>(fc1ecd0)
>- 
> plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsTunnelManagerImpl.java
>(e8ff1a0)
>- server/src/com/cloud/api/ApiResponseHelper.java (1ad0eab)
>- server/src/com/cloud/configuration/ConfigurationManagerImpl.java
>(17ef6bf)
>- server/src/com/cloud/network/ExternalDeviceUsageManagerImpl.java
>(3db5111)
>- server/src/com/cloud/network/ExternalFirewallDeviceManagerImpl.java
>(e4625a4)
>- server/src/com/cloud/network/ExternalLoadBalancerDeviceManagerImpl.java
>(b86f4ae)
>- server/src/com/cloud/network/NetworkServiceImpl.java (eb63fe0)
>- server/src/com/cloud/network/guru/DirectPodBasedNetworkGuru.java
>(67ebef1)
>- server/src/com/cloud/network/guru/ExternalGuestNetworkGuru.java
>(55a33cc)
>- server/src/com/cloud/network/guru/GuestNetworkGuru.java (e90b84b)
>- server/src/com/cloud/network/guru/PrivateNetworkGuru.java (12dce85)
>- 
> server/src/com/cloud/network/router/VpcVirtualNetworkApplianceManagerImpl.java
>(566f1e4)
>- server/src/com/cloud/network/vpc/PrivateGatewayProfile.java (74ce002)
>- server/src/com/cloud/network/vpc/PrivateIpAddress.java (2f3cf53)
>- server/src/com/cloud/network/vpc/VpcManagerImpl.java (6cace7e)
>- server/test/com/cloud/network/CreatePrivateNetworkTest.java (ed9efd6)
>- server/test/com/cloud/vpc/MockNetworkManagerImpl.java (003b774)
>- server/test/com/cloud/vpc/dao/MockNetworkDaoImpl.java (ec1a017)
>- utils/src/com/cloud/utils/net/NetUtils.java (7a89103)
>
> View Diff 

Re: quality control/test automation meeting

[Daan Hoogland]

there is a devops meetup with clouds as subject in Amsterdam during
this time, so if no reactions I am cancelling my attendance

On Wed, Sep 18, 2013 at 3:24 PM, Daan Hoogland  wrote:
> H,
>
> I kept 19:00 cet-daylight saving time open for the meeting discussed
> in some threads. Are we going on with it and if so on what medium. For
> now I am assuming #cloudstack-meeting but I remember Sebastien writing
> about a teleconference.
>
> Who calls it?
>
> regards.
> Daan

A question on vm migrations when hosts are set into a maintenance mode.

[Alex Ough]

I checked the vm migration when their host is set to a maintenance mode and
found that even if the orchestration layer fires the each vm migration at
the same time using a ha_worker thread, the actual migration seems to be
executed serially.

Is this what we expect? And if so, any chance to make the actual migrations
in parallel?

Thanks
Alex Ough

Review Request 14190: Switch to setter injection for extensibility

[Darren Shepherd]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14190/
---

Review request for cloudstack, Alex Huang and Kelven Yang.


Repository: cloudstack-git


Description
---

Various classes are using member injection to inject extensible objects.  
Really those object should come from an AdapterList that is injected in.  This 
patch switches the code to use setter injection that will later allow spring to 
inject an AdapterList or something similar to allow extensibility


Diffs
-

  engine/orchestration/src/com/cloud/vm/VirtualMachineManagerImpl.java 24f0795 
  
engine/orchestration/src/org/apache/cloudstack/engine/cloud/entity/api/VMEntityManagerImpl.java
 204b832 
  
plugins/acl/static-role-based/src/org/apache/cloudstack/acl/StaticRoleBasedAPIAccessChecker.java
 d4d73d1 
  server/src/com/cloud/api/ApiServer.java 550626f 
  server/src/com/cloud/configuration/ConfigurationManagerImpl.java 17ef6bf 
  server/src/com/cloud/consoleproxy/ConsoleProxyManagerImpl.java 90273f7 
  server/src/com/cloud/hypervisor/HypervisorGuruManagerImpl.java 4d1e1b5 
  server/src/com/cloud/network/NetworkServiceImpl.java eb63fe0 
  server/src/com/cloud/server/ManagementServerImpl.java c0a52f7 
  server/src/com/cloud/storage/VolumeApiServiceImpl.java cc99589 
  server/src/com/cloud/storage/secondary/SecondaryStorageManagerImpl.java 
d4463d9 
  server/src/com/cloud/template/TemplateManagerImpl.java e11ac0d 

Diff: https://reviews.apache.org/r/14190/diff/


Testing
---


Thanks,

Darren Shepherd

Re: [PROPOSAL] Storage Subsystem API Interface Additions

[Darren Shepherd]

Here's my general concern about multiple volume snapshots at once.  Giving
such a feature leads the user to believe that snapshotting multiple volumes
at once will give them consistency across the volumes in the snapshot.
This is not true, and difficult to do with many hypervisors, and typically
requires an agent in the VM.  A single snapshot, as exists today, is really
crash consistent, meaning that there is may exist unsync'd data.  To do a
true multi volume snapshot requires a "quiesce" functionality in the VM.
So you do pause I/O queues, fsync, fsync, snapshot, snapshot, unpause I/O.

I'm might be fine with the option of allowing multiple volumeId's to be
specified in the snapshot API, but it needs to be clear that those
snapshots may be taken sequentially and they are all independently crash
consistent.  But, if you make that clear, then why even have the API.
Essentially it is the same as doing multiple snapshot API commands.

So really I would lean towards having the multiple snapshotting supported
in the driver or storage subsystem, but not exposed to the user.  You can
easy accomplish it by having a timed window on snapshotting.  So every 10
seconds you do snapshots, if 5 requests have queued in the last 10 seconds,
you do them all at once.  This could be implemented as a framework thing.
If your provider implements "SnapshotBatching" interface and that has a
getBatchWindowTime(), then the framework can detect that it should try to
queue up some snapshot requests and send them to the driver in a batch.  Or
that could be implemented in the driver itself.  I would lean toward doing
it in the driver and if that goes well, we look at pulling the
functionality into core ACS.

Darren


On Wed, Sep 18, 2013 at 5:22 AM, SuichII, Christopher <
chris.su...@netapp.com> wrote:

> I would like to raise for discussion the idea of adding a couple methods
> to the Storage Subsystem API interface. Currently, takeSnapshot() and
> revertSnapshot() only support single VM volumes. We have a use case for
> snapshotting multiple VM volumes at the same time. For us, it is more
> efficient to snapshot them all at once rather than snapshot VM Volumes
> individually and this seems like a more elegant solution than queueing the
> requests within our plugin.
>
> Base on my investigation, this should require:
> -Two additional API to be invoked from the UI
> -Two additional methods added to the Storage Subsystem API interface
> -Changes in between the API level and invoking the Storage Subsystem API
> implementations (I know this is broad and vague), mainly around the
> SnapshotManger/Impl
>
> There are a couple topics we would like discussion on:
> -Would this be beneficial/detrimental/neutral to other storage providers?
> -How should we handle the addition of new methods to the Storage Subsystem
> API interface? Default them to throw an UnsupportedOperationException?
> Default to calling the single VM volume version multiple times?
> -Does anyone see any issues with allowing multiple snapshots to be taken
> at the same time or letting storage providers have a list of all the
> requested volumes to backup?
>
> Please let me know if I've missed any major topics for discussion or if
> anything needs clarification.
>
> Thanks,
> Chris
> --
> Chris Suich
> chris.su...@netapp.com
> NetApp Software Engineer
> Data Center Platforms – Cloud Solutions
> Citrix, Cisco & Red Hat
>
>

RE: [DISCUSS] CSS framework for CloudStack UI

[Brian Federle]

Yeah, I'm definitely thinking the newer spec, which is better for us anyway 
since it is backwards-compatible with existing CSS.

What I'll do is setup a dummy branch, which basically renames cloudstack3.css 
to cloudstack.scss or something like that, without much modification right now, 
and then see if it can be converted to the .css.

Re: NPM,  -- that is actually why I suggested the SASS plugin instead of the 
vanilla version of sass (installed via gem), since it would prevent people from 
having to install yet another dependency on their system, since I believe all 
required libs (including jRuby) are packaged in the jar, which may eliminate 
the need for Grunt for now?

-Brian

From: Chip Childers [chip.child...@sungard.com]
Sent: Wednesday, September 18, 2013 6:01 AM
To: dev@cloudstack.apache.org
Cc: Rayees Namathponnan; Frank Zhang; Animesh Chaturvedi
Subject: Re: [DISCUSS] CSS framework for CloudStack UI

On Wed, Sep 18, 2013 at 11:53:18AM +0530, Amit Das wrote:
> Hi Brian,
>
> I agree with Edison on usage of grunt & using maven-exec to call grunt.
>
> Will wait for your repository that has your experiments.
> I believe setting up the Maven tasks will be a one-time setting & should
> work without issues.

IIRC, Grunt is installed via NPM.  So does that pull in a bunch of new
developer requirements to build the project?  Is there a standalone
installation for Grunt to lighten the build dependency chain?

How about using SassC? [1]

Let's be sure to use the scss spec, not the sass older style (HAML
inspired)!  That appears to be Hampton's focus these days [2].

-chip

[1] https://github.com/hcatlin/sassc
[2] Per intro on http://sass-lang.com/

Re: Review Request 14143: CLOUDSTACK-4637: Add 30sec sleep before router is ssh'd

[ASF Subversion and Git Services]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14143/#review26219
---


Commit d4dc4f7e70159609f51cf66a032073e31b23eb97 in branch refs/heads/master 
from Girish Shilamkar
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=d4dc4f7 ]

CLOUDSTACK-4637: Add 30sec sleep before router is ssh'd

Egress rules testcases access vm via router. Sleep before
accessing router else the expect fails since router is not
accessible. Also use router.hostid instead of vm.hostid
to identify the host.

Signed-off-by: venkataswamybabu budumuru 
(cherry picked from commit 7d06e77ed9bfacf6d3b74f396c705567d4bf8ba2)


- ASF Subversion and Git Services


On Sept. 14, 2013, 7:43 a.m., Girish Shilamkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14143/
> ---
> 
> (Updated Sept. 14, 2013, 7:43 a.m.)
> 
> 
> Review request for cloudstack and venkata swamy babu  budumuru.
> 
> 
> Bugs: CLOUDSTACK-4637
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> CLOUDSTACK-4637: Add 30sec sleep before router is ssh'd
> 
> Egress rules testcases access vm via router. Sleep before
> accessing router else the expect fails since router is not
> accessible. Also use router.hostid instead of vm.hostid
> to identify the host.
> 
> 
> Diffs
> -
> 
>   test/integration/component/test_egress_fw_rules.py ef0fc5a 
> 
> Diff: https://reviews.apache.org/r/14143/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Girish Shilamkar
> 
>

Re: [PROPOSAL] Storage Subsystem API Interface Additions

[Marcus Sorensen]

Perhaps he needs to elaborate on the use case and what he means by
more efficient.  He may be referring to multiple volumes in the sense
of snapshotting the ROOT disks for 10 different VMs.

On Wed, Sep 18, 2013 at 10:10 AM, Darren Shepherd
 wrote:
> Here's my general concern about multiple volume snapshots at once.  Giving
> such a feature leads the user to believe that snapshotting multiple volumes
> at once will give them consistency across the volumes in the snapshot.
> This is not true, and difficult to do with many hypervisors, and typically
> requires an agent in the VM.  A single snapshot, as exists today, is really
> crash consistent, meaning that there is may exist unsync'd data.  To do a
> true multi volume snapshot requires a "quiesce" functionality in the VM.
> So you do pause I/O queues, fsync, fsync, snapshot, snapshot, unpause I/O.
>
> I'm might be fine with the option of allowing multiple volumeId's to be
> specified in the snapshot API, but it needs to be clear that those
> snapshots may be taken sequentially and they are all independently crash
> consistent.  But, if you make that clear, then why even have the API.
> Essentially it is the same as doing multiple snapshot API commands.
>
> So really I would lean towards having the multiple snapshotting supported
> in the driver or storage subsystem, but not exposed to the user.  You can
> easy accomplish it by having a timed window on snapshotting.  So every 10
> seconds you do snapshots, if 5 requests have queued in the last 10 seconds,
> you do them all at once.  This could be implemented as a framework thing.
> If your provider implements "SnapshotBatching" interface and that has a
> getBatchWindowTime(), then the framework can detect that it should try to
> queue up some snapshot requests and send them to the driver in a batch.  Or
> that could be implemented in the driver itself.  I would lean toward doing
> it in the driver and if that goes well, we look at pulling the
> functionality into core ACS.
>
> Darren
>
>
> On Wed, Sep 18, 2013 at 5:22 AM, SuichII, Christopher <
> chris.su...@netapp.com> wrote:
>
>> I would like to raise for discussion the idea of adding a couple methods
>> to the Storage Subsystem API interface. Currently, takeSnapshot() and
>> revertSnapshot() only support single VM volumes. We have a use case for
>> snapshotting multiple VM volumes at the same time. For us, it is more
>> efficient to snapshot them all at once rather than snapshot VM Volumes
>> individually and this seems like a more elegant solution than queueing the
>> requests within our plugin.
>>
>> Base on my investigation, this should require:
>> -Two additional API to be invoked from the UI
>> -Two additional methods added to the Storage Subsystem API interface
>> -Changes in between the API level and invoking the Storage Subsystem API
>> implementations (I know this is broad and vague), mainly around the
>> SnapshotManger/Impl
>>
>> There are a couple topics we would like discussion on:
>> -Would this be beneficial/detrimental/neutral to other storage providers?
>> -How should we handle the addition of new methods to the Storage Subsystem
>> API interface? Default them to throw an UnsupportedOperationException?
>> Default to calling the single VM volume version multiple times?
>> -Does anyone see any issues with allowing multiple snapshots to be taken
>> at the same time or letting storage providers have a list of all the
>> requested volumes to backup?
>>
>> Please let me know if I've missed any major topics for discussion or if
>> anything needs clarification.
>>
>> Thanks,
>> Chris
>> --
>> Chris Suich
>> chris.su...@netapp.com
>> NetApp Software Engineer
>> Data Center Platforms – Cloud Solutions
>> Citrix, Cisco & Red Hat
>>
>>

RE: [PROPOSAL] Storage Subsystem API Interface Additions

[Alex Huang]

That's my read on the proposal also but, Chris, please clarify.  I don't think 
the end user will see the change.  It's an optimization for interfacing with 
the storage backend.

--Alex

> -Original Message-
> From: Marcus Sorensen [mailto:shadow...@gmail.com]
> Sent: Wednesday, September 18, 2013 9:22 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [PROPOSAL] Storage Subsystem API Interface Additions
> 
> Perhaps he needs to elaborate on the use case and what he means by more
> efficient.  He may be referring to multiple volumes in the sense of
> snapshotting the ROOT disks for 10 different VMs.
> 
> On Wed, Sep 18, 2013 at 10:10 AM, Darren Shepherd
>  wrote:
> > Here's my general concern about multiple volume snapshots at once.
> > Giving such a feature leads the user to believe that snapshotting
> > multiple volumes at once will give them consistency across the volumes in
> the snapshot.
> > This is not true, and difficult to do with many hypervisors, and
> > typically requires an agent in the VM.  A single snapshot, as exists
> > today, is really crash consistent, meaning that there is may exist
> > unsync'd data.  To do a true multi volume snapshot requires a "quiesce"
> functionality in the VM.
> > So you do pause I/O queues, fsync, fsync, snapshot, snapshot, unpause I/O.
> >
> > I'm might be fine with the option of allowing multiple volumeId's to
> > be specified in the snapshot API, but it needs to be clear that those
> > snapshots may be taken sequentially and they are all independently
> > crash consistent.  But, if you make that clear, then why even have the API.
> > Essentially it is the same as doing multiple snapshot API commands.
> >
> > So really I would lean towards having the multiple snapshotting
> > supported in the driver or storage subsystem, but not exposed to the
> > user.  You can easy accomplish it by having a timed window on
> > snapshotting.  So every 10 seconds you do snapshots, if 5 requests
> > have queued in the last 10 seconds, you do them all at once.  This could be
> implemented as a framework thing.
> > If your provider implements "SnapshotBatching" interface and that has
> > a getBatchWindowTime(), then the framework can detect that it should
> > try to queue up some snapshot requests and send them to the driver in
> > a batch.  Or that could be implemented in the driver itself.  I would
> > lean toward doing it in the driver and if that goes well, we look at
> > pulling the functionality into core ACS.
> >
> > Darren
> >
> >
> > On Wed, Sep 18, 2013 at 5:22 AM, SuichII, Christopher <
> > chris.su...@netapp.com> wrote:
> >
> >> I would like to raise for discussion the idea of adding a couple
> >> methods to the Storage Subsystem API interface. Currently,
> >> takeSnapshot() and
> >> revertSnapshot() only support single VM volumes. We have a use case
> >> for snapshotting multiple VM volumes at the same time. For us, it is
> >> more efficient to snapshot them all at once rather than snapshot VM
> >> Volumes individually and this seems like a more elegant solution than
> >> queueing the requests within our plugin.
> >>
> >> Base on my investigation, this should require:
> >> -Two additional API to be invoked from the UI -Two additional methods
> >> added to the Storage Subsystem API interface -Changes in between the
> >> API level and invoking the Storage Subsystem API implementations (I
> >> know this is broad and vague), mainly around the SnapshotManger/Impl
> >>
> >> There are a couple topics we would like discussion on:
> >> -Would this be beneficial/detrimental/neutral to other storage providers?
> >> -How should we handle the addition of new methods to the Storage
> >> Subsystem API interface? Default them to throw an
> UnsupportedOperationException?
> >> Default to calling the single VM volume version multiple times?
> >> -Does anyone see any issues with allowing multiple snapshots to be
> >> taken at the same time or letting storage providers have a list of
> >> all the requested volumes to backup?
> >>
> >> Please let me know if I've missed any major topics for discussion or
> >> if anything needs clarification.
> >>
> >> Thanks,
> >> Chris
> >> --
> >> Chris Suich
> >> chris.su...@netapp.com
> >> NetApp Software Engineer
> >> Data Center Platforms - Cloud Solutions Citrix, Cisco & Red Hat
> >>
> >>

RE: [DISCUSS] Java 7, tomcat 7 and further upgrades

[Alex Huang]

I don't have much problem with switching to jdk1.7.  My eclipse is running with 
jdk1.7 as the builder and it can't find any problems in cs code.  The main 
question I think will come from the Linux variants.  Are all of them shipping 
with jdk1.7 now?

--Alex

> -Original Message-
> From: Trippie [mailto:trip...@gmail.com] On Behalf Of Hugo Trippaers
> Sent: Wednesday, September 18, 2013 5:10 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Java 7, tomcat 7 and further upgrades
> 
> Hey all,
> 
> Sorry for the threadomancy, but the discussion have become relevant again
> with the current issues with the libvirt library. Of course this could also be
> solved by updating the libvirt library with a jdk6 version. Still it might be 
> good
> to revisit this topic.
> 
> It appears not to be possible to switch code style to 1.7 and produce a 1.6
> compatible binary. I remember this working with olders versions, but didn't
> dig to much into this.
> 
> So the new question in this thread will be, should we switch CloudStack to
> jdk 1.7?
> 
> Cheers,
> 
> Hugo
> 
> On Jul 1, 2013, at 12:45 AM, Prasanna Santhanam  wrote:
> 
> > On Thu, Jun 27, 2013 at 04:18:40PM -0400, John Burwell wrote:
> >> All,
> >>
> >> I am +1 for Java7.  However, I would like to propose ridding
> >> ourselves of Tomcat entirely and embedding a network stack such as
> >> Netty (http://netty.io) with a servlet bridge.  We have one JSP in
> >> the system that generates JSON resources.  It could be easily
> >> eliminated with a simple servlet that generates JSON from a
> >> ResourceBundle.  Outside of this JSP. I don't see any other
> >> requirements for a JEE container besides hosting a servlet.  We would
> >> gain a far simpler, self-contained deployment model (a single jar).
> >> Additionally, we would gain greater control of the startup and
> >> shutdown lifecycle, as well as, threading dynamics.  If there is
> >> interest in this approach, I have thoughts on how to achieve this
> >> embedding and create a lightweight daemon framework that could be
> >> used for all CloudStack daemons.
> >>
> >> As an aside, I also think we should replace our hand-rolled NIO code
> >> with Netty as well.
> >>
> >
> > John - could you break this and other thoughts down a little more in
> > what's involved?  Perhaps into its own thread. I don't know Netty. And
> > my J2EE is shaky at best.
> >
> > It's been a previous wish on this list to have the packaging of
> > cloudstack into a single easily deployable war instead of all the
> > complicated packaging we do. So I'd like to hear more of that and
> > other issues you describe.
> >
> > --
> > Prasanna.,
> >
> > 
> > Powered by BigRock.com
> >

Re: [DISCUSS] CSS framework for CloudStack UI

[Shiva Teja]

Another positive side of using grunt would be to minimize and package
javascript. Currently, we load a huge number of javascript files
separately. It'd be great if we can minimize them into a single file during
build. Also, if we were to add any UI tests using libraries like jasmine,
grunt makes it easy automate them.

Shiva Teja


On Wed, Sep 18, 2013 at 9:49 PM, Brian Federle wrote:

> Yeah, I'm definitely thinking the newer spec, which is better for us
> anyway since it is backwards-compatible with existing CSS.
>
> What I'll do is setup a dummy branch, which basically renames
> cloudstack3.css to cloudstack.scss or something like that, without much
> modification right now, and then see if it can be converted to the .css.
>
> Re: NPM,  -- that is actually why I suggested the SASS plugin instead of
> the vanilla version of sass (installed via gem), since it would prevent
> people from having to install yet another dependency on their system, since
> I believe all required libs (including jRuby) are packaged in the jar,
> which may eliminate the need for Grunt for now?
>
> -Brian
> 
> From: Chip Childers [chip.child...@sungard.com]
> Sent: Wednesday, September 18, 2013 6:01 AM
> To: dev@cloudstack.apache.org
> Cc: Rayees Namathponnan; Frank Zhang; Animesh Chaturvedi
> Subject: Re: [DISCUSS] CSS framework for CloudStack UI
>
> On Wed, Sep 18, 2013 at 11:53:18AM +0530, Amit Das wrote:
> > Hi Brian,
> >
> > I agree with Edison on usage of grunt & using maven-exec to call grunt.
> >
> > Will wait for your repository that has your experiments.
> > I believe setting up the Maven tasks will be a one-time setting & should
> > work without issues.
>
> IIRC, Grunt is installed via NPM.  So does that pull in a bunch of new
> developer requirements to build the project?  Is there a standalone
> installation for Grunt to lighten the build dependency chain?
>
> How about using SassC? [1]
>
> Let's be sure to use the scss spec, not the sass older style (HAML
> inspired)!  That appears to be Hampton's focus these days [2].
>
> -chip
>
> [1] https://github.com/hcatlin/sassc
> [2] Per intro on http://sass-lang.com/
>

Re: Removal of cloud-plugin-netapp

[Darren Shepherd]

I really question why the current netapp plugin exists in ACS at all to
begin with at all.  It basically just manages LUNs and such in really has
nothing to do with ACS or cloud functionality (that I can see).  I'd be all
for trashing it.

Is there any chance whatever you're working on will have only open source
(and non-GPL) build dependencies?  Netapp integration to OpenStack seems to
be all xml/http based.  Is there any chance yours can be the same?  Or at
least the java binding be open source, as they just call the xml/http API?

Darren


On Wed, Sep 18, 2013 at 5:25 AM, SuichII, Christopher <
chris.su...@netapp.com> wrote:

> I remember seeing/hearing some discussion about removing the
> cloud-plugin-netapp component since it is not used or really managed any
> more. Is this still in the works? One of the dependencies (manageontap.jar)
> will conflict with one of the dependencies of our (in development) plugin.
> The version of manageontap.jar used by CloudStack is different than what we
> require in our plugin and would likely cause runtime issues if still
> included in the MS classpath.
>
> Any thoughts?
>
> Thanks,
> Chris
> --
> Chris Suich
> chris.su...@netapp.com
> NetApp Software Engineer
> Data Center Platforms – Cloud Solutions
> Citrix, Cisco & Red Hat
>
>

RE: Apache CloudStack 4.2.0 (fifth round)

[Alex Huang]

+1(binding) followed release procedure and tested on devCloud.

--Alex

> -Original Message-
> From: Animesh Chaturvedi [mailto:animesh.chaturv...@citrix.com]
> Sent: Friday, September 13, 2013 4:13 PM
> To: dev@cloudstack.apache.org
> Subject: Apache CloudStack 4.2.0 (fifth round)
> 
> 
> I've created a 4.2.0 release, with the following artifacts up for a vote:
> 
> Git Branch and Commit SH:
> https://git-wip-
> us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/4.2
> Commit: c1e24ff89f6d14d6ae74d12dbca108c35449030f
> 
> List of changes:
> https://git-wip-
> us.apache.org/repos/asf?p=cloudstack.git;a=blob_plain;f=CHANGES;hb=4.2
> 
> Source release (checksums and signatures are available at the same
> location):
> https://dist.apache.org/repos/dist/dev/cloudstack/4.2.0/
> 
> PGP release keys (signed using 94BE0D7C):
> https://dist.apache.org/repos/dist/release/cloudstack/KEYS
> 
> Testing instructions are here:
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Release+test+pr
> ocedure
> 
> Vote will be open for 72 hours (Wednesday 9/18 End of Day PST).
> 
> For sanity in tallying the vote, can PMC members please be sure to indicate
> "(binding)" with their vote?
> 
> [ ] +1  approve
> [ ] +0  no opinion
> [ ] -1  disapprove (and reason why)

Re: Review Request 14143: CLOUDSTACK-4637: Add 30sec sleep before router is ssh'd

[ASF Subversion and Git Services]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14143/#review26218
---


Commit 7d06e77ed9bfacf6d3b74f396c705567d4bf8ba2 in branch 
refs/heads/4.2-forward from Girish Shilamkar
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=7d06e77 ]

CLOUDSTACK-4637: Add 30sec sleep before router is ssh'd

Egress rules testcases access vm via router. Sleep before
accessing router else the expect fails since router is not
accessible. Also use router.hostid instead of vm.hostid
to identify the host.

Signed-off-by: venkataswamybabu budumuru 


- ASF Subversion and Git Services


On Sept. 14, 2013, 7:43 a.m., Girish Shilamkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14143/
> ---
> 
> (Updated Sept. 14, 2013, 7:43 a.m.)
> 
> 
> Review request for cloudstack and venkata swamy babu  budumuru.
> 
> 
> Bugs: CLOUDSTACK-4637
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> CLOUDSTACK-4637: Add 30sec sleep before router is ssh'd
> 
> Egress rules testcases access vm via router. Sleep before
> accessing router else the expect fails since router is not
> accessible. Also use router.hostid instead of vm.hostid
> to identify the host.
> 
> 
> Diffs
> -
> 
>   test/integration/component/test_egress_fw_rules.py ef0fc5a 
> 
> Diff: https://reviews.apache.org/r/14143/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Girish Shilamkar
> 
>

RE: Removal of cloud-plugin-netapp

[Alex Huang]

I think it should be removed.

--Alex

> -Original Message-
> From: SuichII, Christopher [mailto:chris.su...@netapp.com]
> Sent: Wednesday, September 18, 2013 5:26 AM
> To: 
> Subject: Removal of cloud-plugin-netapp
> 
> I remember seeing/hearing some discussion about removing the cloud-
> plugin-netapp component since it is not used or really managed any more. Is
> this still in the works? One of the dependencies (manageontap.jar) will
> conflict with one of the dependencies of our (in development) plugin. The
> version of manageontap.jar used by CloudStack is different than what we
> require in our plugin and would likely cause runtime issues if still included 
> in
> the MS classpath.
> 
> Any thoughts?
> 
> Thanks,
> Chris
> --
> Chris Suich
> chris.su...@netapp.com
> NetApp Software Engineer
> Data Center Platforms - Cloud Solutions
> Citrix, Cisco & Red Hat

Re: [DISCUSS] Java 7, tomcat 7 and further upgrades

[Darren Shepherd]

I'd be relatively opposed to switching everything to jdk7.  Java 6 is the
current lowest common denominator and using java 7 at the moment will just
alienate somebody  I don't know the back story.  Is libvirt compiled for
jdk7?  I'd be perfectly fine with compiling just the KVM driver with jdk7
as any distro that has a usable KVM implementation also has java 7.

So here is what I propose (which I think is what is already being proposed,
but I'm just being explicit).  We keep the maven settings to keep building
1.6 byte code.  Only the KVM driver do we set it to 1.7.  This means that
we would then have to switch to using JDK7 for jenkins, eclipse, mvn cli,
etc.  As the majority of development is done in eclipse (i think), eclipse
still sets up the projects to compile against the JRE 6 library (except KVM
will be JRE 7), so that will help prevent people from accidentally using a
JRE7 API in a 1.6 byte code format.

And as a totally random note, OpenJDK7 on CentOS is way faster than
OpenJDK6 on CentOS.  Regardless of whatever we compile against, everybody
should try use Java 7 at runtime.

Darren


On Wed, Sep 18, 2013 at 9:29 AM, Alex Huang  wrote:

> I don't have much problem with switching to jdk1.7.  My eclipse is running
> with jdk1.7 as the builder and it can't find any problems in cs code.  The
> main question I think will come from the Linux variants.  Are all of them
> shipping with jdk1.7 now?
>
> --Alex
>
> > -Original Message-
> > From: Trippie [mailto:trip...@gmail.com] On Behalf Of Hugo Trippaers
> > Sent: Wednesday, September 18, 2013 5:10 AM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [DISCUSS] Java 7, tomcat 7 and further upgrades
> >
> > Hey all,
> >
> > Sorry for the threadomancy, but the discussion have become relevant again
> > with the current issues with the libvirt library. Of course this could
> also be
> > solved by updating the libvirt library with a jdk6 version. Still it
> might be good
> > to revisit this topic.
> >
> > It appears not to be possible to switch code style to 1.7 and produce a
> 1.6
> > compatible binary. I remember this working with olders versions, but
> didn't
> > dig to much into this.
> >
> > So the new question in this thread will be, should we switch CloudStack
> to
> > jdk 1.7?
> >
> > Cheers,
> >
> > Hugo
> >
> > On Jul 1, 2013, at 12:45 AM, Prasanna Santhanam  wrote:
> >
> > > On Thu, Jun 27, 2013 at 04:18:40PM -0400, John Burwell wrote:
> > >> All,
> > >>
> > >> I am +1 for Java7.  However, I would like to propose ridding
> > >> ourselves of Tomcat entirely and embedding a network stack such as
> > >> Netty (http://netty.io) with a servlet bridge.  We have one JSP in
> > >> the system that generates JSON resources.  It could be easily
> > >> eliminated with a simple servlet that generates JSON from a
> > >> ResourceBundle.  Outside of this JSP. I don't see any other
> > >> requirements for a JEE container besides hosting a servlet.  We would
> > >> gain a far simpler, self-contained deployment model (a single jar).
> > >> Additionally, we would gain greater control of the startup and
> > >> shutdown lifecycle, as well as, threading dynamics.  If there is
> > >> interest in this approach, I have thoughts on how to achieve this
> > >> embedding and create a lightweight daemon framework that could be
> > >> used for all CloudStack daemons.
> > >>
> > >> As an aside, I also think we should replace our hand-rolled NIO code
> > >> with Netty as well.
> > >>
> > >
> > > John - could you break this and other thoughts down a little more in
> > > what's involved?  Perhaps into its own thread. I don't know Netty. And
> > > my J2EE is shaky at best.
> > >
> > > It's been a previous wish on this list to have the packaging of
> > > cloudstack into a single easily deployable war instead of all the
> > > complicated packaging we do. So I'd like to hear more of that and
> > > other issues you describe.
> > >
> > > --
> > > Prasanna.,
> > >
> > > 
> > > Powered by BigRock.com
> > >
>
>

Re: Review Request 14148: Cleanup DefaultUserAuthenticator and removed masking _name variable

[Darren Shepherd]

> On Sept. 18, 2013, 5:37 a.m., Abhinandan Prateek wrote:
> > In client/tomcatconf/applicationContext.xml.in we are referring passing 
> > name properties to the authentication adapters e.g.:  
> > 
> >  > class="org.apache.cloudstack.ldap.LdapAuthenticator">
> > 
> > 
> > 
> > There adapters never make use of this property. We should just remove the 
> > property from applicationContext.xml.in too ?

This change removes the code that hardcoded it to LDAP.  So by also removing 
the XML the name will become LdapAuthenticator (class.getSimpleName()).  Either 
way it doesn't really matter.  Really shortly I'm going to submit code to 
modularize spring config which will introduce new XML.  The new XML will set 
the name to LDAP, because the way in which I'm doing the modularization, the 
names of the adapters matter.  This is so that you can reorder the Adapters (as 
the order of authenticators is really important).  So there will be a 
configuration parameter 
user.authenticators.order="SHA256SALT,MD5,LDAP,PLAINTEXT".  So after saying all 
that, I'd just leave the XML as is, not reason to change it.


- Darren


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14148/#review26209
---


On Sept. 18, 2013, 5:39 a.m., Darren Shepherd wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14148/
> ---
> 
> (Updated Sept. 18, 2013, 5:39 a.m.)
> 
> 
> Review request for cloudstack, Abhinandan Prateek and Kelven Yang.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> DefaultUserAuthenticator maskes the _name varible in ComponentLifecycleBase 
> making the setName() method not work as expected.  This patch cleans up the 
> code such that getName() will be getClass().getSimpleName() unless overridden 
> in the Spring configuration.
> 
> 
> Diffs
> -
> 
>   
> plugins/user-authenticators/ldap/src/org/apache/cloudstack/ldap/LdapAuthenticator.java
>  e62a3d8 
>   
> plugins/user-authenticators/md5/src/com/cloud/server/auth/MD5UserAuthenticator.java
>  e5b169f 
>   
> plugins/user-authenticators/plain-text/src/com/cloud/server/auth/PlainTextUserAuthenticator.java
>  f102275 
>   
> plugins/user-authenticators/sha256salted/src/com/cloud/server/auth/SHA256SaltedUserAuthenticator.java
>  91be922 
>   server/src/com/cloud/server/auth/DefaultUserAuthenticator.java 952f724 
> 
> Diff: https://reviews.apache.org/r/14148/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Darren Shepherd
> 
>

Re: conflicting dependencies between CloudStack and Whirr

[Andrew Bayer]

Good question. The gson dependency in Whirr is coming from jclouds, and
jclouds is very particular about its gson dependencies. Any possibility of
using isolated classloaders in some way?

A.


On Tue, Sep 17, 2013 at 7:14 PM, Han,Meng  wrote:

> Dear all,
>
> I am adding an API to CloudStack which utilizes Whirr to launch various
> clusters on CloudStack. Now I am facing a dependency conflicting issue.
>
> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires gson 1.7.1.
> If I use gson 1.7.1 for Whirr, the following error will happen:
>
> com.google.common.util.**concurrent.ExecutionError: 
> java.lang.**NoClassDefFoundError:
> com/google/gson/TypeAdapter
>
> This TypeAdapter class can be found inside gson-2.2.2.jar. However if I
> modify CloudStack to use gson 2.2.2 CloudStack will not build successfully
> due to the following test error.
>
> [INFO] Building Apache CloudStack Core 4.1.1
> [INFO] --**--**
> 
> [INFO]
> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-core ---
> [INFO] Deleting /home/meng/cloudstack/core/**target
> [INFO]
> [INFO] --- maven-remote-resources-plugin:**1.3:process (default) @
> cloud-core ---
> [INFO]
> [INFO] --- maven-resources-plugin:2.5:**resources (default-resources) @
> cloud-core ---
> [debug] execute contextualize
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
> src/main/resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:**compile (default-compile) @
> cloud-core ---
> [INFO] Compiling 156 source files to /home/meng/cloudstack/core/**
> target/classes
> [INFO]
> [INFO] --- maven-resources-plugin:2.5:**testResources
> (default-testResources) @ cloud-core ---
> [debug] execute contextualize
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
> src/test/resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:**testCompile
> (default-testCompile) @ cloud-core ---
> [INFO] Compiling 1 source file to /home/meng/cloudstack/core/**
> target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @ cloud-core
> ---
> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
> target/surefire-reports
>
> --**-
>  T E S T S
> --**-
> Running com.cloud.agent.transport.**RequestTest
> log4j:WARN No appenders could be found for logger
> (com.cloud.agent.transport.**RequestTest).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See 
> http://logging.apache.org/**log4j/1.2/faq.html#noconfigfor
>  more info.
> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 3.054 sec
> <<< FAILURE!
>
> Results :
>
> Failed tests:   testSerDeser(com.cloud.agent.**transport.RequestTest)
>
> Tests in error:
>   testLogging(com.cloud.agent.**transport.RequestTest)
>
> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>
>
> I ran "mvn -P developer clean install -DskipTests" , the CloudStack
> building process went through. But when I issued an API --launchCluster to
> CloudStack, the following error related to gson popped up on CloudStack
> Management Server:
>
> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught problem
> with [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
> snapshot","pool":"/root","**hypervisorType":"KVM","**
> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy":"**
> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
> "a7320748-6346-3c9a-975e-**90ac4ae4a986-**LibvirtComputingResource","**
> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
> 0","publicMacAddress":"00:23:**ae:94:f7:22","**
> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
> LibvirtComputingResource","**gatewayIpAddress":"!
> 10.244.18
> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
> images/","poolType":"**Filesystem","capacityBytes":**
> 528446

Re: [PROPOSAL] Modularize Spring

[Darren Shepherd]

Are you referring to what I have on review board?  Any requests I have on
review board at the moment have no interdependencies.  They are just a
bunch of random things I found while doing analysis and work for this
proposal.

Darren


On Wed, Sep 18, 2013 at 4:41 AM, Daan Hoogland wrote:

> Darren,
>
> I had a quick look at your four submissions, Is there no dependency
> between them? if so, can you add those to the review requests?
>
> thanks,
> Daan
>
> On Wed, Sep 18, 2013 at 1:25 PM, Daan Hoogland 
> wrote:
> > sounds great Darren,
> >
> > By component, you mean maven project or some larger chunk like
> > distribution package? (did i miss this definition somewhere or do we
> > define the components now?)
> >
> > regards,
> > Daan
> >
> > On Wed, Sep 18, 2013 at 12:10 AM, Darren Shepherd
> >  wrote:
> >> Currently ACS code is fairly modular in that you can add plug-ins to
> ACS to
> >> extend most functionality.  Unfortunately ACS is not packaged in a
> modular
> >> way.  It is still delivered essentially as one large unit.  There are
> many
> >> reason for this but one large barrier to modularizing ACS is that the
> >> Spring configuration is managed as one large unit.
> >>
> >> I propose that we break apart the Spring XML configuration such that
> each
> >> component contributes its own configuration.  Additionally each
> component
> >> will be loaded into its own Spring ApplicationContext such that its
> beans
> >> will not conflict with the wiring of other beans in ACS.  This change
> will
> >> lay the foundation for a richer plugin model and additionally a more
> >> distributed architecture.
> >>
> >> The technical details for this proposal can be found on the wiki [1].
> >>
> >> Darren
> >>
> >> [1]
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Spring
>

Re: Removal of cloud-plugin-netapp

[Chiradeep Vittal]

It could enable a volume service for baremetal, but currently it is too
tied to NetApp.


On 9/18/13 11:35 AM, "Alex Huang"  wrote:

>I think it should be removed.
>
>--Alex
>
>> -Original Message-
>> From: SuichII, Christopher [mailto:chris.su...@netapp.com]
>> Sent: Wednesday, September 18, 2013 5:26 AM
>> To: 
>> Subject: Removal of cloud-plugin-netapp
>> 
>> I remember seeing/hearing some discussion about removing the cloud-
>> plugin-netapp component since it is not used or really managed any
>>more. Is
>> this still in the works? One of the dependencies (manageontap.jar) will
>> conflict with one of the dependencies of our (in development) plugin.
>>The
>> version of manageontap.jar used by CloudStack is different than what we
>> require in our plugin and would likely cause runtime issues if still
>>included in
>> the MS classpath.
>> 
>> Any thoughts?
>> 
>> Thanks,
>> Chris
>> --
>> Chris Suich
>> chris.su...@netapp.com
>> NetApp Software Engineer
>> Data Center Platforms - Cloud Solutions
>> Citrix, Cisco & Red Hat
>

Re: [PROPOSAL] Modularize Spring

[Darren Shepherd]

Right, component isn't a thing.  I probably shouldn't use that term.  I
want to standarize on the naming convention of plugin, module, and
extension.  It is explained a bit on the wiki [1] but I'll try to do a
little better job here.  So a plugin is basically a jar.  A jar contains
multiple modules.  A modules ends up being a spring application context
that composes multiple configuration files.  Modules are assembled into a
hierarchy at runtime.  Extensions are implementations of interfaces that
exist in a module.  A maven project produces a jar, so a plugin ends up
being a maven project also.

So currently today we don't have a strong definition of Plugin and I hope
to address that.

Darren

[1]
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-ins%2C+Modules%2C+and+Extensions


On Wed, Sep 18, 2013 at 4:25 AM, Daan Hoogland wrote:

> sounds great Darren,
>
> By component, you mean maven project or some larger chunk like
> distribution package? (did i miss this definition somewhere or do we
> define the components now?)
>
> regards,
> Daan
>
> On Wed, Sep 18, 2013 at 12:10 AM, Darren Shepherd
>  wrote:
> > Currently ACS code is fairly modular in that you can add plug-ins to ACS
> to
> > extend most functionality.  Unfortunately ACS is not packaged in a
> modular
> > way.  It is still delivered essentially as one large unit.  There are
> many
> > reason for this but one large barrier to modularizing ACS is that the
> > Spring configuration is managed as one large unit.
> >
> > I propose that we break apart the Spring XML configuration such that each
> > component contributes its own configuration.  Additionally each component
> > will be loaded into its own Spring ApplicationContext such that its beans
> > will not conflict with the wiring of other beans in ACS.  This change
> will
> > lay the foundation for a richer plugin model and additionally a more
> > distributed architecture.
> >
> > The technical details for this proposal can be found on the wiki [1].
> >
> > Darren
> >
> > [1]
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Spring
>

Re: conflicting dependencies between CloudStack and Whirr

[Darren Shepherd]

You know what would be really swell is to just switch to jackson.  The gson
we use is antiquated.  I have no idea what the impact of moving to a modern
version would be.  Jackson, IMO, is a far better framework that has a lot
of momentum.  Additionally it allows you to use JAXB annotations so that
you don't have to use framework dependent annotation in your code.  Plus it
does tons and tons of more stuff.

I really don't expect isolated classloaders will come anytime soon.  That
actually relates to things I'm working on right now, but I have a hard time
justifying adding the overhead and complexity of isolated classloaders at
the moment.

If we really, really, really, really, really think isolating classloaders
is a good thing, we can talk about it, but it will be a can of worms.

Darren


On Tue, Sep 17, 2013 at 7:14 PM, Han,Meng  wrote:

> Dear all,
>
> I am adding an API to CloudStack which utilizes Whirr to launch various
> clusters on CloudStack. Now I am facing a dependency conflicting issue.
>
> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires gson 1.7.1.
> If I use gson 1.7.1 for Whirr, the following error will happen:
>
> com.google.common.util.**concurrent.ExecutionError: 
> java.lang.**NoClassDefFoundError:
> com/google/gson/TypeAdapter
>
> This TypeAdapter class can be found inside gson-2.2.2.jar. However if I
> modify CloudStack to use gson 2.2.2 CloudStack will not build successfully
> due to the following test error.
>
> [INFO] Building Apache CloudStack Core 4.1.1
> [INFO] --**--**
> 
> [INFO]
> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-core ---
> [INFO] Deleting /home/meng/cloudstack/core/**target
> [INFO]
> [INFO] --- maven-remote-resources-plugin:**1.3:process (default) @
> cloud-core ---
> [INFO]
> [INFO] --- maven-resources-plugin:2.5:**resources (default-resources) @
> cloud-core ---
> [debug] execute contextualize
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
> src/main/resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:**compile (default-compile) @
> cloud-core ---
> [INFO] Compiling 156 source files to /home/meng/cloudstack/core/**
> target/classes
> [INFO]
> [INFO] --- maven-resources-plugin:2.5:**testResources
> (default-testResources) @ cloud-core ---
> [debug] execute contextualize
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
> src/test/resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:**testCompile
> (default-testCompile) @ cloud-core ---
> [INFO] Compiling 1 source file to /home/meng/cloudstack/core/**
> target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @ cloud-core
> ---
> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
> target/surefire-reports
>
> --**-
>  T E S T S
> --**-
> Running com.cloud.agent.transport.**RequestTest
> log4j:WARN No appenders could be found for logger
> (com.cloud.agent.transport.**RequestTest).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See 
> http://logging.apache.org/**log4j/1.2/faq.html#noconfigfor
>  more info.
> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 3.054 sec
> <<< FAILURE!
>
> Results :
>
> Failed tests:   testSerDeser(com.cloud.agent.**transport.RequestTest)
>
> Tests in error:
>   testLogging(com.cloud.agent.**transport.RequestTest)
>
> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>
>
> I ran "mvn -P developer clean install -DskipTests" , the CloudStack
> building process went through. But when I issued an API --launchCluster to
> CloudStack, the following error related to gson popped up on CloudStack
> Management Server:
>
> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught problem
> with [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
> snapshot","pool":"/root","**hypervisorType":"KVM","**
> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy":"**
> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
> "a7320748-6346-3c9a-975e-**90ac4ae4a986-**LibvirtComputingResource","**
> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
> 0","publicMacAddress":"00:23:**ae:94:f7:22","**
> privateIpAddres

Re: security around api.log

[Darren Shepherd]

Lets just not log the session info.  You can't expect somebody to protect
the box, it sort of defeats the purpose of the log.  The whole idea of the
log is to know what is going on and troubleshoot things.  So you would be
inclined to give a level 1 support tech person access to that log to see
whats going on at the moment, but that same person you probably don't want
them to have full admin access to ACS.

Darren


On Wed, Sep 18, 2013 at 1:43 AM, Rajesh Battala
wrote:

> Disabling api log might not be a good idea, instead while logging the
> request remove the sensitive details (session details, passwords etc ) and
> dump it.
>
> Thanks
> Rajesh Battala
>
> -Original Message-
> From: Abhinandan Prateek [mailto:abhinandan.prat...@citrix.com]
> Sent: Wednesday, September 18, 2013 12:33 PM
> To: dev@cloudstack.apache.org
> Subject: Re: security around api.log
>
> We can provide a way to disable the api.log ?
>
> On 18/09/13 11:27 am, "Rajesh Battala"  wrote:
>
> >If anybody got access to the api.log using the session details we can
> >do execute api's and cause harm.
> >But the api.log is present in the mgmt server and if anybody got access
> >to it, he can corrupt anything.
> >Not just accessing api.log, any other services logs and get the data. I
> >feel it's up to admin how to protect his system and services.
> >
> >Thanks
> >Rajesh Battala
> >
> >-Original Message-
> >From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com]
> >Sent: Saturday, September 14, 2013 2:10 AM
> >To: dev@cloudstack.apache.org
> >Subject: security around api.log
> >
> >I just noticed api.log which seems to log all the API access in a form
> >like
> >
> >2013-09-13 00:02:09,451 INFO  [a.c.c.a.ApiServer]
> >(2011638958@qtp-657397168-0:ctx-81b1e088 ctx-174e4a62) (userId=2
> >accountId=2 sessionId=7asvmtwoesbc6ia3e4kxtzrl) 127.0.0.1 -- GET
> >command=listZones&response=json&sessionkey=ec6h46Om8a1y3d%2BhrdIpQ85cAf
> >c%3
> >D&_=1379055729422
> >200 { "listzonesresponse" : { "count":1 ,"zone" : [
> >{"id":"cdaf82f1-3b57-4aa4-b3ce-b60173ed45f2","name":"zone1","dns1":"8.8.8.
> >8","dns2":"8.8.4.4","internaldns1":"8.8.4.4","networktype":"Basic","sec
> >uri
> >tygroupsenabled":true,"allocationstate":"Enabled","zonetoken":"6dce94e8
> >-e8
> >dc-3077-bfde-c6e8594bd449","dhcpprovider":"VirtualRouter","localstorage
> >ena
> >bled":false}
> >] } }
> >
> >The sessionId and sessionKey is logged in the file.  I haven't tried it
> >yet, but can't I use that info to hijack the session?  That introduces
> >a security issue in that any server operator can now hijack anybody's
> >session.  So that api.log file really needs to be protected in the same
> >way a file with a password in it would be.
> >
> >I would suggest that we just don't log the sessionId or sessionKey.
> >
> >Darren
>
>

Re: Review Request 14187: Framework to discover and load spring contexts in a hierarchy

[Darren Shepherd]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14187/
---

(Updated Sept. 18, 2013, 5:51 p.m.)


Review request for cloudstack, Alex Huang and Kelven Yang.


Changes
---

If you want to build this module you need rb14186 also as it has the parent pom 
in it


Repository: cloudstack-git


Description
---

This is a small framework to discover Spring XML files and then load then in a 
hierarchy of application contexts.  This patch adds the new project but does 
not add it to the build.  That will come later.


Diffs
-

  framework/spring/module/pom.xml PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/context/ResourceApplicationContext.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/factory/CloudStackSpringContext.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/factory/ModuleBasedContextFactory.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/locator/ModuleDefinitionLocator.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/locator/impl/ClasspathModuleDefinitionLocator.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/model/ModuleDefinition.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/model/ModuleDefinitionSet.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/model/impl/DefaultModuleDefinition.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/model/impl/DefaultModuleDefinitionSet.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/util/Main.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/util/ModuleLocationUtils.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/web/CloudStackContextLoaderListener.java
 PRE-CREATION 
  
framework/spring/module/src/main/resources/org/apache/cloudstack/spring/module/model/impl/defaults-context.xml
 PRE-CREATION 
  
framework/spring/module/src/test/java/org/apache/cloudstack/spring/module/factory/ModuleBasedContextFactoryTest.java
 PRE-CREATION 
  
framework/spring/module/src/test/java/org/apache/cloudstack/spring/module/locator/impl/ClasspathModuleDefinitionSetLocatorTest.java
 PRE-CREATION 
  
framework/spring/module/src/test/java/org/apache/cloudstack/spring/module/model/impl/DefaultModuleDefinitionTest.java
 PRE-CREATION 
  framework/spring/module/src/test/resources/testfiles/all/defaults.properties 
PRE-CREATION 
  
framework/spring/module/src/test/resources/testfiles/all/empty-context-inheritable.xml
 PRE-CREATION 
  framework/spring/module/src/test/resources/testfiles/all/empty-context.xml 
PRE-CREATION 
  
framework/spring/module/src/test/resources/testfiles/all/empty2-context-inheritable.xml
 PRE-CREATION 
  framework/spring/module/src/test/resources/testfiles/all/empty2-context.xml 
PRE-CREATION 
  framework/spring/module/src/test/resources/testfiles/all/module.properties 
PRE-CREATION 
  
framework/spring/module/src/test/resources/testfiles/all/test2-defaults.properties
 PRE-CREATION 
  
framework/spring/module/src/test/resources/testfiles/badname/module.properties 
PRE-CREATION 
  
framework/spring/module/src/test/resources/testfiles/blankname/module.properties
 PRE-CREATION 
  framework/spring/module/src/test/resources/testfiles/good/empty-context.xml 
PRE-CREATION 
  framework/spring/module/src/test/resources/testfiles/good/module.properties 
PRE-CREATION 
  
framework/spring/module/src/test/resources/testfiles/missingname/module.properties
 PRE-CREATION 
  
framework/spring/module/src/test/resources/testfiles/wrongname/module.properties
 PRE-CREATION 
  
framework/spring/module/src/test/resources/testhierarchy/base/module.properties 
PRE-CREATION 
  
framework/spring/module/src/test/resources/testhierarchy/base/test-context-inheritable.xml
 PRE-CREATION 
  
framework/spring/module/src/test/resources/testhierarchy/base/test-context.xml 
PRE-CREATION 
  
framework/spring/module/src/test/resources/testhierarchy/child1-1/module.properties
 PRE-CREATION 
  
framework/spring/module/src/test/resources/testhierarchy/child1-1/test-context.xml
 PRE-CREATION 
  
framework/spring/module/src/test/resources/testhierarchy/child1/module.properties
 PRE-CREATION 
  
framework/spring/module/src/test/resources/testhierarchy/child1/test-context.xml
 PRE-CREATION 
  
framework/spring/module/src/test/resources/testhierarchy/child2/module.properties
 PRE-CREATION 
  
framework/spring/module/src/test/resources/testhierarchy/child2/test-context.xml
 PRE-CREATION 
 

Re: Removal of cloud-plugin-netapp

[SuichII, Christopher]

Darren,

The integration we are working on will be closed source. The NetApp OpenStack 
integrations are GPL due to the nature of OpenStack. They may be simply doing 
xml/http request because that is the lowest level of communicating with our 
storage controllers. Our work uses an internal library that abstracts the 
xml/http work away (which transitively required the managontap library).

-Chris
-- 
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Sep 18, 2013, at 1:31 PM, Chiradeep Vittal  
wrote:

> It could enable a volume service for baremetal, but currently it is too
> tied to NetApp.
> 
> 
> On 9/18/13 11:35 AM, "Alex Huang"  wrote:
> 
>> I think it should be removed.
>> 
>> --Alex
>> 
>>> -Original Message-
>>> From: SuichII, Christopher [mailto:chris.su...@netapp.com]
>>> Sent: Wednesday, September 18, 2013 5:26 AM
>>> To: 
>>> Subject: Removal of cloud-plugin-netapp
>>> 
>>> I remember seeing/hearing some discussion about removing the cloud-
>>> plugin-netapp component since it is not used or really managed any
>>> more. Is
>>> this still in the works? One of the dependencies (manageontap.jar) will
>>> conflict with one of the dependencies of our (in development) plugin.
>>> The
>>> version of manageontap.jar used by CloudStack is different than what we
>>> require in our plugin and would likely cause runtime issues if still
>>> included in
>>> the MS classpath.
>>> 
>>> Any thoughts?
>>> 
>>> Thanks,
>>> Chris
>>> --
>>> Chris Suich
>>> chris.su...@netapp.com
>>> NetApp Software Engineer
>>> Data Center Platforms - Cloud Solutions
>>> Citrix, Cisco & Red Hat
>> 
> 

Review Request 14211: Add ability to programatically disable CloudStack ComponentConext intialization

[Darren Shepherd]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14211/
---

Review request for cloudstack and Kelven Yang.


Repository: cloudstack-git


Description
---

As we moved to a more Spring managed lifecycle, the lifecycle management in 
ComponentContext is not needed.  The transition to fully spring managed will 
not happen all at once.  The management server will be done first and then 
AWSAPI and UsageServer will come later.  This change allows the 
ComponentContext in the context of the management server to be bypassed.  Note, 
by default, the ComponentContext will run its initialization.


Diffs
-

  utils/src/com/cloud/utils/component/ComponentContext.java 065fd3b 

Diff: https://reviews.apache.org/r/14211/diff/


Testing
---


Thanks,

Darren Shepherd

Re: [PROPOSAL] Storage Subsystem API Interface Additions

[SuichII, Christopher]

First, let me apologize for the confusing terms, because some words here are 
overloaded:
A volume…
In CloudStack terms is a disk attached to a VM.
In NetApp terms is an NFS volume, analogous to CloudStack primary storage, 
where all the CloudStack volumes are stored.

A snapshot…
In CloudStack terms is a backup of a VM.
In NetApp terms is a copy of all the contents of a NetApp volume, taken at a 
point in time to create an analogous CloudStack snapshot for (up to) every 
CloudStack volume on that primary storage.

There are several reasons that an API for snapshotting multiple volumes is more 
attractive to us than calling a single volume API over and over. A lot of it 
has to do with how we actually create the snapshots. Unlike a hypervisor 
snapshot, when we create a vm snapshot, the entire primary storage is backed up 
(but only the requested volume has an entry added to the db). To add on to 
this, our hardware has a hard limit of 255 storage volume level snapshots. So, 
if there were 255 vms on a single primary storage and each one of them 
performed a backup, no more backups could be taken before we start removing the 
oldest backup (without some trickery that we are currently working on). Some 
might say a solution to this would be queueing the requests and waiting till 
they're all finished, but that seems much more error prone and like hackish 
design compared to simply allowing multiple VM volumes to be specified.

This is both a request for optimizing the backend and optimizing the experience 
for users. What happens when a user says they want to backup 30 vm volumes at 
the same time? Is it not a cleaner experience to simply select all the volumes 
they want to back up, then click backup once? This way, the storage provider is 
given all the volumes at once and if they have some way of optimizing the 
request based on their hardware or software, they can take advantage of that. 
It can even be designed in such a way that if storage providers don't want to 
be given all the volumes at once, they can be called with each one 
individually, as to remain backwards compatible.

Now, I'm also not saying that these two solutions can't co-exist. Even if we 
have the ability to backup multiple volumes at once, nothing is stopping users 
from backing them up one by one, so queueing is still something we may have to 
implement. However, I think extending the subsystem API to grant storage 
providers the ability to leverage any optimization they can without having to 
queue is a cleaner solution. If the concern is how users interpret what is 
going on in the backend, I think we can find some way to make that clear to 
them.

-Chris
-- 
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Sep 18, 2013, at 12:26 PM, Alex Huang  wrote:

> That's my read on the proposal also but, Chris, please clarify.  I don't 
> think the end user will see the change.  It's an optimization for interfacing 
> with the storage backend.
> 
> --Alex
> 
>> -Original Message-
>> From: Marcus Sorensen [mailto:shadow...@gmail.com]
>> Sent: Wednesday, September 18, 2013 9:22 AM
>> To: dev@cloudstack.apache.org
>> Subject: Re: [PROPOSAL] Storage Subsystem API Interface Additions
>> 
>> Perhaps he needs to elaborate on the use case and what he means by more
>> efficient.  He may be referring to multiple volumes in the sense of
>> snapshotting the ROOT disks for 10 different VMs.
>> 
>> On Wed, Sep 18, 2013 at 10:10 AM, Darren Shepherd
>>  wrote:
>>> Here's my general concern about multiple volume snapshots at once.
>>> Giving such a feature leads the user to believe that snapshotting
>>> multiple volumes at once will give them consistency across the volumes in
>> the snapshot.
>>> This is not true, and difficult to do with many hypervisors, and
>>> typically requires an agent in the VM.  A single snapshot, as exists
>>> today, is really crash consistent, meaning that there is may exist
>>> unsync'd data.  To do a true multi volume snapshot requires a "quiesce"
>> functionality in the VM.
>>> So you do pause I/O queues, fsync, fsync, snapshot, snapshot, unpause I/O.
>>> 
>>> I'm might be fine with the option of allowing multiple volumeId's to
>>> be specified in the snapshot API, but it needs to be clear that those
>>> snapshots may be taken sequentially and they are all independently
>>> crash consistent.  But, if you make that clear, then why even have the API.
>>> Essentially it is the same as doing multiple snapshot API commands.
>>> 
>>> So really I would lean towards having the multiple snapshotting
>>> supported in the driver or storage subsystem, but not exposed to the
>>> user.  You can easy accomplish it by having a timed window on
>>> snapshotting.  So every 10 seconds you do snapshots, if 5 requests
>>> have queued in the last 10 seconds, you do them all at once.  This could be
>> implemented as a framework thing.
>>> If your provider implemen

RE: [PROPOSAL] Modularize Spring

[Frank Zhang]

What's the point in using separate spring context per plugin?
Separate class loader is the thing I hate most in OSGI, I am afraid we are on 
the same way.
Frankly speaking, I never see benefits of this *separate* model,  our 
project(or most projects) is not like Chrome which has to create sandbox for 
extensions 
in order to avoid bad plugin screws up the whole browser(however, I still see 
bad plugins screw up my Chrome well).
Projects like CloudStack using plugin to decouple architecture should not 
introduce many isolations to plugin writer, the point preventing wrong use of 
some components
is not making much sense to me. If a plugin not following guide(if we have it) 
we should kick it out, instead of making obstacles for 99% good people. 



> -Original Message-
> From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com]
> Sent: Wednesday, September 18, 2013 10:33 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [PROPOSAL] Modularize Spring
> 
> Right, component isn't a thing.  I probably shouldn't use that term.  I want 
> to
> standarize on the naming convention of plugin, module, and extension.  It is
> explained a bit on the wiki [1] but I'll try to do a little better job here.  
> So a
> plugin is basically a jar.  A jar contains multiple modules.  A modules ends 
> up
> being a spring application context that composes multiple configuration files.
> Modules are assembled into a hierarchy at runtime.  Extensions are
> implementations of interfaces that exist in a module.  A maven project
> produces a jar, so a plugin ends up being a maven project also.
> 
> So currently today we don't have a strong definition of Plugin and I hope to
> address that.
> 
> Darren
> 
> [1]
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-
> ins%2C+Modules%2C+and+Extensions
> 
> 
> On Wed, Sep 18, 2013 at 4:25 AM, Daan Hoogland
> wrote:
> 
> > sounds great Darren,
> >
> > By component, you mean maven project or some larger chunk like
> > distribution package? (did i miss this definition somewhere or do we
> > define the components now?)
> >
> > regards,
> > Daan
> >
> > On Wed, Sep 18, 2013 at 12:10 AM, Darren Shepherd
> >  wrote:
> > > Currently ACS code is fairly modular in that you can add plug-ins to
> > > ACS
> > to
> > > extend most functionality.  Unfortunately ACS is not packaged in a
> > modular
> > > way.  It is still delivered essentially as one large unit.  There
> > > are
> > many
> > > reason for this but one large barrier to modularizing ACS is that
> > > the Spring configuration is managed as one large unit.
> > >
> > > I propose that we break apart the Spring XML configuration such that
> > > each component contributes its own configuration.  Additionally each
> > > component will be loaded into its own Spring ApplicationContext such
> > > that its beans will not conflict with the wiring of other beans in
> > > ACS.  This change
> > will
> > > lay the foundation for a richer plugin model and additionally a more
> > > distributed architecture.
> > >
> > > The technical details for this proposal can be found on the wiki [1].
> > >
> > > Darren
> > >
> > > [1]
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Spri
> > ng
> >

Review Request 14209: Add Named interface

[Darren Shepherd]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14209/
---

Review request for cloudstack.


Repository: cloudstack-git


Description
---

Add Named interface that can be used to identify objects that have a getName() 
like adapters and managers.


Diffs
-

  utils/src/com/cloud/utils/component/ComponentLifecycle.java ea671af 
  utils/src/com/cloud/utils/component/Named.java PRE-CREATION 

Diff: https://reviews.apache.org/r/14209/diff/


Testing
---


Thanks,

Darren Shepherd

Re: [PROPOSAL] Storage Subsystem API Interface Additions

[Darren Shepherd]

Given this explanation.  Would the following not work?

1) Enhance UI to allow multi select.  There is no API change, UI will just
call snapshot a bunch of time
2) Enhance storage framework or driver to detect that 20 requests just came
in within a window of X seconds and send them to the driver all at once.

I know you said queuing on the backend is hackish, but having the user send
multiple volumeIds in the API is just as hackish to me.  We can only
guarentee to the user that the multiple snapshots taken are as consistent
as if they called snapshot API individually.  The user won't really know
exactly what NetApp volume they exist on and really neither will the
storage framework, as export != volume.  Only the driver knows if batching
is really possible.  So I'm not exactly saying queue, its short batches.

In short, I'm seeing this as a bit more of a NetApp optimization than a
general thing.  I'm all for using storage device level snapshotting but it
seems like its going to be implementation specific.  Its interesting, if
you look at digital ocean they have snapshots and backups as two different
concepts.  You can see that they ran into this specific issue.  Full
storage volume snapshots are really difficult to expose to the user.  So
digital ocean does "backups" which are live but on a schedule and seem to
be a full volume backup.  And then there are snapshots which are on demand,
but require you to stop your VM (so they can essentially copy the qcow or
lv somewhere).

Darren




On Wed, Sep 18, 2013 at 11:12 AM, SuichII, Christopher <
chris.su...@netapp.com> wrote:

> First, let me apologize for the confusing terms, because some words here
> are overloaded:
> A volume…
> In CloudStack terms is a disk attached to a VM.
> In NetApp terms is an NFS volume, analogous to CloudStack primary storage,
> where all the CloudStack volumes are stored.
>
> A snapshot…
> In CloudStack terms is a backup of a VM.
> In NetApp terms is a copy of all the contents of a NetApp volume, taken at
> a point in time to create an analogous CloudStack snapshot for (up to)
> every CloudStack volume on that primary storage.
>
> There are several reasons that an API for snapshotting multiple volumes is
> more attractive to us than calling a single volume API over and over. A lot
> of it has to do with how we actually create the snapshots. Unlike a
> hypervisor snapshot, when we create a vm snapshot, the entire primary
> storage is backed up (but only the requested volume has an entry added to
> the db). To add on to this, our hardware has a hard limit of 255 storage
> volume level snapshots. So, if there were 255 vms on a single primary
> storage and each one of them performed a backup, no more backups could be
> taken before we start removing the oldest backup (without some trickery
> that we are currently working on). Some might say a solution to this would
> be queueing the requests and waiting till they're all finished, but that
> seems much more error prone and like hackish design compared to simply
> allowing multiple VM volumes to be specified.
>
> This is both a request for optimizing the backend and optimizing the
> experience for users. What happens when a user says they want to backup 30
> vm volumes at the same time? Is it not a cleaner experience to simply
> select all the volumes they want to back up, then click backup once? This
> way, the storage provider is given all the volumes at once and if they have
> some way of optimizing the request based on their hardware or software,
> they can take advantage of that. It can even be designed in such a way that
> if storage providers don't want to be given all the volumes at once, they
> can be called with each one individually, as to remain backwards compatible.
>
> Now, I'm also not saying that these two solutions can't co-exist. Even if
> we have the ability to backup multiple volumes at once, nothing is stopping
> users from backing them up one by one, so queueing is still something we
> may have to implement. However, I think extending the subsystem API to
> grant storage providers the ability to leverage any optimization they can
> without having to queue is a cleaner solution. If the concern is how users
> interpret what is going on in the backend, I think we can find some way to
> make that clear to them.
>
> -Chris
> --
> Chris Suich
> chris.su...@netapp.com
> NetApp Software Engineer
> Data Center Platforms – Cloud Solutions
> Citrix, Cisco & Red Hat
>
> On Sep 18, 2013, at 12:26 PM, Alex Huang  wrote:
>
> > That's my read on the proposal also but, Chris, please clarify.  I don't
> think the end user will see the change.  It's an optimization for
> interfacing with the storage backend.
> >
> > --Alex
> >
> >> -Original Message-
> >> From: Marcus Sorensen [mailto:shadow...@gmail.com]
> >> Sent: Wednesday, September 18, 2013 9:22 AM
> >> To: dev@cloudstack.apache.org
> >> Subject: Re: [PROPOSAL] Storage Subsystem API Interface Additions
> >>
> >> Perhaps he 

Review Request 14210: Add Registry interface

[Darren Shepherd]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14210/
---

Review request for cloudstack.


Repository: cloudstack-git


Description
---

Add Registry interface.  Implementations of this class will represent 
registries of things like adapters and managers.  This is to more generalize 
the concept in ACS currently used with AdapterList.  The extension registry 
will implement this interface and extensions on intialization will be added to 
the registry


Diffs
-

  utils/src/com/cloud/utils/component/Registry.java PRE-CREATION 

Diff: https://reviews.apache.org/r/14210/diff/


Testing
---


Thanks,

Darren Shepherd

Re: [PROPOSAL] Modularize Spring

[Darren Shepherd]

I'm not for OSGi either, but contexts are quite useful and will lead to
better things.  First off, we don't want one gigantic spring XML config
file like we have today.  I think we can all agree on that.  So each plugin
will have to supply its own XML.  So the obstacles you mention, will
largely be just that for people.

With Spring it is really simple to just inject dependencies and cross
architectural boundaries.  Its currently everywhere in ACS.  You can't just
say we should review code and make sure nobody doesn't do bad things.  A
little bit of framework to enforce separation is a good thing.  But I'm
guessing you will disagree with me there.

Here are some random points on why contexts are good.  Say I want to use
Spring AOP or Spring TX in my plugin.  With your own context you can ensure
that you won't screw with anybody else code by accidentally having you
pointcut match their bean.  You don't have to worry about bean name
conflicts.  If two config files specify bean X, Spring will gladly just use
the last one.  I've already found multiply defined beans in ACS, but things
still just happen to work.  Having multiple contexts also defines
initialization order.  We can ensure that the framework is loaded and ready
before child context are loaded and started.  (we kind of do this today
with ComponentLifeCycle, but its a hack in my mind).  Additionally, when
things start you will know, loading context "crapping plug X".  If spring
fails to initialize, the issue it there.  Today, if spring fails to start,
it could be one of over 500 beans causing the weird problem.  The list goes
on and on.

Finally, this is the big one and why I really want contexts.  I have some
notes on the wiki [1] that you might want to read through.  Basically I
want to get to a more flexible deployment model that allows both a single
monolithic JVM as today and also a fully distributed system.  Having
contexts in hierarchy will enable me to accomplish this.  By selecting
which contexts are loaded at runtime will determine what role the JVM takes
on.  The contexts help people better understand how the distributed
architecture will work too, when we get there.

Frank, trust me, I hate complex things.  I don't want OSGi, classloader
magic, etc.  But I do like organization and a little bit of framework so
that people don't accidentally shoot themselves in the foot.  I personally
like knowing that I couldn't have screwed something up, because the
framework won't even allow it.  If we separate everything as I want today,
and then tomorrow we say this is way too much overhead, moving to a flat
context is simple.  Don't think we are on some slippery slope to
classloaders and dependency hell.

Darren

[1]
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Nothing+to+see+here...#Nothingtoseehere...-DeploymentModels



On Wed, Sep 18, 2013 at 11:22 AM, Frank Zhang wrote:

> What's the point in using separate spring context per plugin?
> Separate class loader is the thing I hate most in OSGI, I am afraid we are
> on the same way.
> Frankly speaking, I never see benefits of this *separate* model,  our
> project(or most projects) is not like Chrome which has to create sandbox
> for extensions
> in order to avoid bad plugin screws up the whole browser(however, I still
> see bad plugins screw up my Chrome well).
> Projects like CloudStack using plugin to decouple architecture should not
> introduce many isolations to plugin writer, the point preventing wrong use
> of some components
> is not making much sense to me. If a plugin not following guide(if we have
> it) we should kick it out, instead of making obstacles for 99% good people.
>
>
>
> > -Original Message-
> > From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com]
> > Sent: Wednesday, September 18, 2013 10:33 AM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [PROPOSAL] Modularize Spring
> >
> > Right, component isn't a thing.  I probably shouldn't use that term.  I
> want to
> > standarize on the naming convention of plugin, module, and extension.
>  It is
> > explained a bit on the wiki [1] but I'll try to do a little better job
> here.  So a
> > plugin is basically a jar.  A jar contains multiple modules.  A modules
> ends up
> > being a spring application context that composes multiple configuration
> files.
> > Modules are assembled into a hierarchy at runtime.  Extensions are
> > implementations of interfaces that exist in a module.  A maven project
> > produces a jar, so a plugin ends up being a maven project also.
> >
> > So currently today we don't have a strong definition of Plugin and I
> hope to
> > address that.
> >
> > Darren
> >
> > [1]
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-
> > ins%2C+Modules%2C+and+Extensions
> >
> >
> > On Wed, Sep 18, 2013 at 4:25 AM, Daan Hoogland
> > wrote:
> >
> > > sounds great Darren,
> > >
> > > By component, you mean maven project or some larger chunk like
> > > distribution package? (did i miss this de

Re: [PROPOSAL] Storage Subsystem API Interface Additions

[SuichII, Christopher]

That certainly would work, but I don't see how it is a better design. Can you 
elaborate on how sending multiple volumeIds is hackish? Look at the existing 
API framework. We have several APIs that accept lists as parameters. Normally, 
they're used for things like querying or deleting. Take a look at some of these 
commands:
-ArchiveEventsCmd
-DeleteEventsCmd
-DeleteSnapshotPoliciesCmd

This kind of API is simply a shorthand for invoking another API many times.

I think it is only an NetApp optimization in the sense that we're the only ones 
who need it right now. What we're asking for has nothing specific to do to 
NetApp. We would just like the shorthand ability to do things all at once 
rather than one at a time. I think other vendors could utilize this just as 
easily.

-Chris
-- 
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Sep 18, 2013, at 2:32 PM, Darren Shepherd  
wrote:

> Given this explanation.  Would the following not work?
> 
> 1) Enhance UI to allow multi select.  There is no API change, UI will just
> call snapshot a bunch of time
> 2) Enhance storage framework or driver to detect that 20 requests just came
> in within a window of X seconds and send them to the driver all at once.
> 
> I know you said queuing on the backend is hackish, but having the user send
> multiple volumeIds in the API is just as hackish to me.  We can only
> guarentee to the user that the multiple snapshots taken are as consistent
> as if they called snapshot API individually.  The user won't really know
> exactly what NetApp volume they exist on and really neither will the
> storage framework, as export != volume.  Only the driver knows if batching
> is really possible.  So I'm not exactly saying queue, its short batches.
> 
> In short, I'm seeing this as a bit more of a NetApp optimization than a
> general thing.  I'm all for using storage device level snapshotting but it
> seems like its going to be implementation specific.  Its interesting, if
> you look at digital ocean they have snapshots and backups as two different
> concepts.  You can see that they ran into this specific issue.  Full
> storage volume snapshots are really difficult to expose to the user.  So
> digital ocean does "backups" which are live but on a schedule and seem to
> be a full volume backup.  And then there are snapshots which are on demand,
> but require you to stop your VM (so they can essentially copy the qcow or
> lv somewhere).
> 
> Darren
> 
> 
> 
> 
> On Wed, Sep 18, 2013 at 11:12 AM, SuichII, Christopher <
> chris.su...@netapp.com> wrote:
> 
>> First, let me apologize for the confusing terms, because some words here
>> are overloaded:
>> A volume…
>> In CloudStack terms is a disk attached to a VM.
>> In NetApp terms is an NFS volume, analogous to CloudStack primary storage,
>> where all the CloudStack volumes are stored.
>> 
>> A snapshot…
>> In CloudStack terms is a backup of a VM.
>> In NetApp terms is a copy of all the contents of a NetApp volume, taken at
>> a point in time to create an analogous CloudStack snapshot for (up to)
>> every CloudStack volume on that primary storage.
>> 
>> There are several reasons that an API for snapshotting multiple volumes is
>> more attractive to us than calling a single volume API over and over. A lot
>> of it has to do with how we actually create the snapshots. Unlike a
>> hypervisor snapshot, when we create a vm snapshot, the entire primary
>> storage is backed up (but only the requested volume has an entry added to
>> the db). To add on to this, our hardware has a hard limit of 255 storage
>> volume level snapshots. So, if there were 255 vms on a single primary
>> storage and each one of them performed a backup, no more backups could be
>> taken before we start removing the oldest backup (without some trickery
>> that we are currently working on). Some might say a solution to this would
>> be queueing the requests and waiting till they're all finished, but that
>> seems much more error prone and like hackish design compared to simply
>> allowing multiple VM volumes to be specified.
>> 
>> This is both a request for optimizing the backend and optimizing the
>> experience for users. What happens when a user says they want to backup 30
>> vm volumes at the same time? Is it not a cleaner experience to simply
>> select all the volumes they want to back up, then click backup once? This
>> way, the storage provider is given all the volumes at once and if they have
>> some way of optimizing the request based on their hardware or software,
>> they can take advantage of that. It can even be designed in such a way that
>> if storage providers don't want to be given all the volumes at once, they
>> can be called with each one individually, as to remain backwards compatible.
>> 
>> Now, I'm also not saying that these two solutions can't co-exist. Even if
>> we have the ability to backup multiple volumes at once, nothing is

RE: [PROPOSAL] Modularize Spring

[Frank Zhang]

I am not against boundary, I am just against making things unnecessary complex 
to enable boundary.
If we are going this way, I hope we can make it as much as transparent to 
developers. That means, as
a developer, all a plugin I need to do is 1) provide my separate spring xml 2) 
inject beans I want (valid beans)
in my bean and code business logic 3) compile to a jar and put to some place 
designated by CloudStack. That's it.

I raise this topic because I have seen some projects to create boundary making 
things horrible complex. And
sometimes developers are hurt  by wrong boundaries, as a result, to overcome 
these limitations people write
lots of ugly code which makes thing even worse. 

However, I am still worry about if we can make things so simpler. 
For example, we may have an orchestration context that contains major beans 
needed by almost every
plugin,  this context can be easily set as parent context for all plugin 
contexts when bootstrap. However, if a plugin A needs
to access some bean defined in plugin B, given they are sibling, how plugin 
framework resolves the dependency ?

> -Original Message-
> From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com]
> Sent: Wednesday, September 18, 2013 11:53 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [PROPOSAL] Modularize Spring
> 
> I'm not for OSGi either, but contexts are quite useful and will lead to better
> things.  First off, we don't want one gigantic spring XML config file like we 
> have
> today.  I think we can all agree on that.  So each plugin will have to supply 
> its
> own XML.  So the obstacles you mention, will largely be just that for people.
> 
> With Spring it is really simple to just inject dependencies and cross
> architectural boundaries.  Its currently everywhere in ACS.  You can't just 
> say
> we should review code and make sure nobody doesn't do bad things.  A little 
> bit
> of framework to enforce separation is a good thing.  But I'm guessing you will
> disagree with me there.
> 
> Here are some random points on why contexts are good.  Say I want to use
> Spring AOP or Spring TX in my plugin.  With your own context you can ensure
> that you won't screw with anybody else code by accidentally having you
> pointcut match their bean.  You don't have to worry about bean name conflicts.
> If two config files specify bean X, Spring will gladly just use the last one. 
>  I've
> already found multiply defined beans in ACS, but things still just happen to 
> work.
> Having multiple contexts also defines initialization order.  We can ensure 
> that
> the framework is loaded and ready before child context are loaded and started.
> (we kind of do this today with ComponentLifeCycle, but its a hack in my mind).
> Additionally, when things start you will know, loading context "crapping plug 
> X".
> If spring fails to initialize, the issue it there.  Today, if spring fails to 
> start, it
> could be one of over 500 beans causing the weird problem.  The list goes on
> and on.
> 
> Finally, this is the big one and why I really want contexts.  I have some 
> notes on
> the wiki [1] that you might want to read through.  Basically I want to get to 
> a
> more flexible deployment model that allows both a single monolithic JVM as
> today and also a fully distributed system.  Having contexts in hierarchy will
> enable me to accomplish this.  By selecting which contexts are loaded at
> runtime will determine what role the JVM takes on.  The contexts help people
> better understand how the distributed architecture will work too, when we get
> there.
> 
> Frank, trust me, I hate complex things.  I don't want OSGi, classloader magic,
> etc.  But I do like organization and a little bit of framework so that people 
> don't
> accidentally shoot themselves in the foot.  I personally like knowing that I
> couldn't have screwed something up, because the framework won't even allow
> it.  If we separate everything as I want today, and then tomorrow we say this 
> is
> way too much overhead, moving to a flat context is simple.  Don't think we are
> on some slippery slope to classloaders and dependency hell.
> 
> Darren
> 
> [1]
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Nothing+to+see+her
> e...#Nothingtoseehere...-DeploymentModels
> 
> 
> 
> On Wed, Sep 18, 2013 at 11:22 AM, Frank Zhang
> wrote:
> 
> > What's the point in using separate spring context per plugin?
> > Separate class loader is the thing I hate most in OSGI, I am afraid we
> > are on the same way.
> > Frankly speaking, I never see benefits of this *separate* model,  our
> > project(or most projects) is not like Chrome which has to create
> > sandbox for extensions in order to avoid bad plugin screws up the
> > whole browser(however, I still see bad plugins screw up my Chrome
> > well).
> > Projects like CloudStack using plugin to decouple architecture should
> > not introduce many isolations to plugin writer, the point preventing
> > wrong use of some components 

Re: conflicting dependencies between CloudStack and Whirr

[Andrei Savu]

It's easy to usr jclouds and whirr inside an OSGi container - just add the
feature url. Bonus: you can also use jclouds shell interface (part of
jclouds cli).

Another option is to upgrade the CloudStack API to use the new version.
On Sep 18, 2013 5:14 AM, "Han,Meng"  wrote:

> Dear all,
>
> I am adding an API to CloudStack which utilizes Whirr to launch various
> clusters on CloudStack. Now I am facing a dependency conflicting issue.
>
> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires gson 1.7.1.
> If I use gson 1.7.1 for Whirr, the following error will happen:
>
> com.google.common.util.**concurrent.ExecutionError: 
> java.lang.**NoClassDefFoundError:
> com/google/gson/TypeAdapter
>
> This TypeAdapter class can be found inside gson-2.2.2.jar. However if I
> modify CloudStack to use gson 2.2.2 CloudStack will not build successfully
> due to the following test error.
>
> [INFO] Building Apache CloudStack Core 4.1.1
> [INFO] --**--**
> 
> [INFO]
> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-core ---
> [INFO] Deleting /home/meng/cloudstack/core/**target
> [INFO]
> [INFO] --- maven-remote-resources-plugin:**1.3:process (default) @
> cloud-core ---
> [INFO]
> [INFO] --- maven-resources-plugin:2.5:**resources (default-resources) @
> cloud-core ---
> [debug] execute contextualize
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
> src/main/resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:**compile (default-compile) @
> cloud-core ---
> [INFO] Compiling 156 source files to /home/meng/cloudstack/core/**
> target/classes
> [INFO]
> [INFO] --- maven-resources-plugin:2.5:**testResources
> (default-testResources) @ cloud-core ---
> [debug] execute contextualize
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
> src/test/resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:**testCompile
> (default-testCompile) @ cloud-core ---
> [INFO] Compiling 1 source file to /home/meng/cloudstack/core/**
> target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @ cloud-core
> ---
> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
> target/surefire-reports
>
> --**-
>  T E S T S
> --**-
> Running com.cloud.agent.transport.**RequestTest
> log4j:WARN No appenders could be found for logger
> (com.cloud.agent.transport.**RequestTest).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See 
> http://logging.apache.org/**log4j/1.2/faq.html#noconfigfor
>  more info.
> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 3.054 sec
> <<< FAILURE!
>
> Results :
>
> Failed tests:   testSerDeser(com.cloud.agent.**transport.RequestTest)
>
> Tests in error:
>   testLogging(com.cloud.agent.**transport.RequestTest)
>
> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>
>
> I ran "mvn -P developer clean install -DskipTests" , the CloudStack
> building process went through. But when I issued an API --launchCluster to
> CloudStack, the following error related to gson popped up on CloudStack
> Management Server:
>
> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught problem
> with [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
> snapshot","pool":"/root","**hypervisorType":"KVM","**
> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy":"**
> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
> "a7320748-6346-3c9a-975e-**90ac4ae4a986-**LibvirtComputingResource","**
> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
> 0","publicMacAddress":"00:23:**ae:94:f7:22","**
> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
> LibvirtComputingResource","**gatewayIpAddress":"!
> 10.244.18
> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
> images/","poolType":"**Filesys

Hotels for CCC Amsterdam

[La Motta, David]

Hey everybody, I think I saw something about hotels not being decided for
CCC in Amsterdam this November.  Has that been squared out?

I know this is not a dev question per seŠ but hey, what better group to
ask that question but this one?  :-)

Thanks!

--
David La Motta
Technical Marketing Engineer - Citrix Solutions | NetApp

Direct: 1.919.476.5042
david.lamo...@netapp.com

Re: [PROPOSAL] Modularize Spring

[Darren Shepherd]

If you want to see this all working you can just fetch the "no-at-db4"
branch at https://github.com/ibuildthecloud/cloudstack.git

Plugin composes multiple modules.  If modules are siblings they can't
inject from each other.  But a plugin can augment another module if it
chooses to.  The reality is that the core cloudstack is a tangled mess of
dependencies such that most of the core code can't be modularized as it
stands.  So there exists a context towards the top of the hierarchy called
"core" that a lot of jars contribute to it.  Here is the full hierarchy
right now.  I'll probably rename a bunch of things, but this gives you an
idea.

bootstrap
  system
core
  allocator
allocator-server
planner
  api-planner
  baremetal-planner
  explicit-dedication
  host-anti-affinity
  implicit-dedication-planner
  server-planner
  user-concentrated-pod-planner
random-allocator
  api
acl-static-role-based
rate-limit
server-api
user-authenticator-ldap
user-authenticator-md5
user-authenticator-plaintext
user-authenticator-sha256salted
  backend
alert-adapter-server-backend
compute
  alert-adapter-server-compute
  baremetal-compute
  fencer-server
  investigator-server
  kvm-compute
  ovm-compute
  server-compute
  xenserver-compute
network
  baremetal-network
  elb
  midonet
  nvp
  ovs
  server-network
  ssp
storage
  alert-adapter-server-storage
  allocator-storage
  baremetal-storage
  secondary-storage
  server-storage
  storage-image-default
  storage-image-s3
  storage-image-swift
  storage-volume-default
  storage-volume-solidfire
  template-adapter-server
  discoverer
baremetal-discoverer
discoverer-server
ovm-discoverer
xenserver-discoverer



If you look at the baremetal hypervisor plugin that is pretty cross cutting
to most of ACS.  So the modules it contributes to are as follows

resources/META-INF/cloudstack/baremetal-storage/spring-context.xml
resources/META-INF/cloudstack/baremetal-compute/spring-context.xml
resources/META-INF/cloudstack/baremetal-discoverer/spring-context.xml
resources/META-INF/cloudstack/core/spring-baremetal-core-context.xml
resources/META-INF/cloudstack/baremetal-planner/spring-context.xml
resources/META-INF/cloudstack/baremetal-network/spring-context.xml

So it creates child contextes of storage, compute, network, planner, and
discoverer to add its extensions where it needs to be.  Additionally,
you'll notice, it adds some beans to the core context from the file
resources/META-INF/cloudstack/core/spring-baremetal-core-context.xml.  This
is because it has some manager class that is used by multiple contexts.

Frank, I understand the scare that we are going too complex, but do you
have some other suggestion?  I don't like the idea of one gigantic spring
context.  So I feel I am making it as simple as I can while maintaining
some order.  People just need to create one or more spring xml files and a
properties files that says where to put it in the hierarchy.

Additionally, by putting forcing people to put beans in certains modules it
forces them to think about what is the role of the code.  For example,
today in ACS the *ManagerImpl classes are a huge mess.  They implement too
many interfaces and the code crosses to many architectural boundaries.  Its
about time we start splitting things up to be more maintainable.

If you have some time, please check out what I have on github.

Darren


On Wed, Sep 18, 2013 at 1:56 PM, Frank Zhang  wrote:

> I am not against boundary, I am just against making things unnecessary
> complex to enable boundary.
> If we are going this way, I hope we can make it as much as transparent to
> developers. That means, as
> a developer, all a plugin I need to do is 1) provide my separate spring
> xml 2) inject beans I want (valid beans)
> in my bean and code business logic 3) compile to a jar and put to some
> place designated by CloudStack. That's it.
>
> I raise this topic because I have seen some projects to create boundary
> making things horrible complex. And
> sometimes developers are hurt  by wrong boundaries, as a result, to
> overcome these limitations people write
> lots of ugly code which makes thing even worse.
>
> However, I am still worry about if we can make things so simpler.
> For example, we may have an orchestration context that contains major
> beans needed by almost every
> plugin,  this context can be easily set as parent context for all plugin
> contexts when bootstrap. However, if a plugin A needs
> to access some bean defined in plugin B, given they are sibling, how
> plugin framework resolves the dependency ?
>
> > --

Review Request 14217: Make ConfigDepot capable of incrementally adding Configurables

[Darren Shepherd]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14217/
---

Review request for cloudstack and Alex Huang.


Repository: cloudstack-git


Description
---

In order to modularize ACS all extension points need to support incrementally 
adding the extensible items.  The Configurables will be loaded in a different 
context and then added to the ConfigDepotAdmin.  This means ConfigDepotImpl 
can't just rely on injection to find all Configurables as injection will only 
find the configurables in the current and parent contexts but not in child 
contexts.


Diffs
-

  
framework/config/src/org/apache/cloudstack/framework/config/ConfigDepotAdmin.java
 b4d3773 
  
framework/config/src/org/apache/cloudstack/framework/config/impl/ConfigDepotImpl.java
 b516596 
  
framework/config/test/org/apache/cloudstack/framework/config/impl/ConfigDepotAdminTest.java
 1c5fbe5 

Diff: https://reviews.apache.org/r/14217/diff/


Testing
---


Thanks,

Darren Shepherd

Review Request 14218: Add spring lifecycle framework to handle modularization

[Darren Shepherd]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14218/
---

Review request for cloudstack, Alex Huang and Kelven Yang.


Repository: cloudstack-git


Description
---

This maven project will handle all the extended lifecycle needed in Spring to 
find and register extensions in the their corresponding registries.  This patch 
adds the project but it is not added to the main build.


Diffs
-

  framework/spring/lifecycle/pom.xml PRE-CREATION 
  
framework/spring/lifecycle/src/main/java/org/apache/cloudstack/spring/lifecycle/AbstractBeanCollector.java
 PRE-CREATION 
  
framework/spring/lifecycle/src/main/java/org/apache/cloudstack/spring/lifecycle/AbstractSmartLifeCycle.java
 PRE-CREATION 
  
framework/spring/lifecycle/src/main/java/org/apache/cloudstack/spring/lifecycle/CloudStackExtendedLifeCycle.java
 PRE-CREATION 
  
framework/spring/lifecycle/src/main/java/org/apache/cloudstack/spring/lifecycle/CloudStackExtendedLifeCycleStart.java
 PRE-CREATION 
  
framework/spring/lifecycle/src/main/java/org/apache/cloudstack/spring/lifecycle/CloudStackLog4jSetup.java
 PRE-CREATION 
  
framework/spring/lifecycle/src/main/java/org/apache/cloudstack/spring/lifecycle/ConfigDepotLifeCycle.java
 PRE-CREATION 
  
framework/spring/lifecycle/src/main/java/org/apache/cloudstack/spring/lifecycle/registry/DumpRegistry.java
 PRE-CREATION 
  
framework/spring/lifecycle/src/main/java/org/apache/cloudstack/spring/lifecycle/registry/ExtensionRegistry.java
 PRE-CREATION 
  
framework/spring/lifecycle/src/main/java/org/apache/cloudstack/spring/lifecycle/registry/RegistryLifecycle.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/14218/diff/


Testing
---


Thanks,

Darren Shepherd

RE: [PROPOSAL] Modularize Spring

[Frank Zhang]

Well. The codes explain more than words.
It seems the only extra work is adding a property file that specifies parent 
context and current context name, it's not much complex.
BTW: any reason for working on repo outside ACS?

> -Original Message-
> From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com]
> Sent: Wednesday, September 18, 2013 2:43 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [PROPOSAL] Modularize Spring
> 
> If you want to see this all working you can just fetch the "no-at-db4"
> branch at https://github.com/ibuildthecloud/cloudstack.git
> 
> Plugin composes multiple modules.  If modules are siblings they can't inject
> from each other.  But a plugin can augment another module if it chooses to.
> The reality is that the core cloudstack is a tangled mess of dependencies such
> that most of the core code can't be modularized as it stands.  So there 
> exists a
> context towards the top of the hierarchy called "core" that a lot of jars
> contribute to it.  Here is the full hierarchy right now.  I'll probably 
> rename a
> bunch of things, but this gives you an idea.
> 
> bootstrap
>   system
> core
>   allocator
> allocator-server
> planner
>   api-planner
>   baremetal-planner
>   explicit-dedication
>   host-anti-affinity
>   implicit-dedication-planner
>   server-planner
>   user-concentrated-pod-planner
> random-allocator
>   api
> acl-static-role-based
> rate-limit
> server-api
> user-authenticator-ldap
> user-authenticator-md5
> user-authenticator-plaintext
> user-authenticator-sha256salted
>   backend
> alert-adapter-server-backend
> compute
>   alert-adapter-server-compute
>   baremetal-compute
>   fencer-server
>   investigator-server
>   kvm-compute
>   ovm-compute
>   server-compute
>   xenserver-compute
> network
>   baremetal-network
>   elb
>   midonet
>   nvp
>   ovs
>   server-network
>   ssp
> storage
>   alert-adapter-server-storage
>   allocator-storage
>   baremetal-storage
>   secondary-storage
>   server-storage
>   storage-image-default
>   storage-image-s3
>   storage-image-swift
>   storage-volume-default
>   storage-volume-solidfire
>   template-adapter-server
>   discoverer
> baremetal-discoverer
> discoverer-server
> ovm-discoverer
> xenserver-discoverer
> 
> 
> 
> If you look at the baremetal hypervisor plugin that is pretty cross cutting to
> most of ACS.  So the modules it contributes to are as follows
> 
> resources/META-INF/cloudstack/baremetal-storage/spring-context.xml
> resources/META-INF/cloudstack/baremetal-compute/spring-context.xml
> resources/META-INF/cloudstack/baremetal-discoverer/spring-context.xml
> resources/META-INF/cloudstack/core/spring-baremetal-core-context.xml
> resources/META-INF/cloudstack/baremetal-planner/spring-context.xml
> resources/META-INF/cloudstack/baremetal-network/spring-context.xml
> 
> So it creates child contextes of storage, compute, network, planner, and
> discoverer to add its extensions where it needs to be.  Additionally, you'll 
> notice,
> it adds some beans to the core context from the file resources/META-
> INF/cloudstack/core/spring-baremetal-core-context.xml.  This is because it has
> some manager class that is used by multiple contexts.
> 
> Frank, I understand the scare that we are going too complex, but do you have
> some other suggestion?  I don't like the idea of one gigantic spring context. 
>  So I
> feel I am making it as simple as I can while maintaining some order.  People
> just need to create one or more spring xml files and a properties files that 
> says
> where to put it in the hierarchy.
> 
> Additionally, by putting forcing people to put beans in certains modules it
> forces them to think about what is the role of the code.  For example, today 
> in
> ACS the *ManagerImpl classes are a huge mess.  They implement too many
> interfaces and the code crosses to many architectural boundaries.  Its about
> time we start splitting things up to be more maintainable.
> 
> If you have some time, please check out what I have on github.
> 
> Darren
> 
> 
> On Wed, Sep 18, 2013 at 1:56 PM, Frank Zhang 
> wrote:
> 
> > I am not against boundary, I am just against making things unnecessary
> > complex to enable boundary.
> > If we are going this way, I hope we can make it as much as transparent
> > to developers. That means, as a developer, all a plugin I need to do
> > is 1) provide my separate spring xml 2) inject beans I want (valid
> > beans) in my bean and code business logic 3) compile to a jar and put
> > to some place designated by CloudStack. That's it.
> >
> > I raise t

Re: [PROPOSAL] Modularize Spring

[Darren Shepherd]

I'm not a committer


On Wed, Sep 18, 2013 at 3:24 PM, Frank Zhang  wrote:

> Well. The codes explain more than words.
> It seems the only extra work is adding a property file that specifies
> parent context and current context name, it's not much complex.
> BTW: any reason for working on repo outside ACS?
>
> > -Original Message-
> > From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com]
> > Sent: Wednesday, September 18, 2013 2:43 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [PROPOSAL] Modularize Spring
> >
> > If you want to see this all working you can just fetch the "no-at-db4"
> > branch at https://github.com/ibuildthecloud/cloudstack.git
> >
> > Plugin composes multiple modules.  If modules are siblings they can't
> inject
> > from each other.  But a plugin can augment another module if it chooses
> to.
> > The reality is that the core cloudstack is a tangled mess of
> dependencies such
> > that most of the core code can't be modularized as it stands.  So there
> exists a
> > context towards the top of the hierarchy called "core" that a lot of jars
> > contribute to it.  Here is the full hierarchy right now.  I'll probably
> rename a
> > bunch of things, but this gives you an idea.
> >
> > bootstrap
> >   system
> > core
> >   allocator
> > allocator-server
> > planner
> >   api-planner
> >   baremetal-planner
> >   explicit-dedication
> >   host-anti-affinity
> >   implicit-dedication-planner
> >   server-planner
> >   user-concentrated-pod-planner
> > random-allocator
> >   api
> > acl-static-role-based
> > rate-limit
> > server-api
> > user-authenticator-ldap
> > user-authenticator-md5
> > user-authenticator-plaintext
> > user-authenticator-sha256salted
> >   backend
> > alert-adapter-server-backend
> > compute
> >   alert-adapter-server-compute
> >   baremetal-compute
> >   fencer-server
> >   investigator-server
> >   kvm-compute
> >   ovm-compute
> >   server-compute
> >   xenserver-compute
> > network
> >   baremetal-network
> >   elb
> >   midonet
> >   nvp
> >   ovs
> >   server-network
> >   ssp
> > storage
> >   alert-adapter-server-storage
> >   allocator-storage
> >   baremetal-storage
> >   secondary-storage
> >   server-storage
> >   storage-image-default
> >   storage-image-s3
> >   storage-image-swift
> >   storage-volume-default
> >   storage-volume-solidfire
> >   template-adapter-server
> >   discoverer
> > baremetal-discoverer
> > discoverer-server
> > ovm-discoverer
> > xenserver-discoverer
> >
> >
> >
> > If you look at the baremetal hypervisor plugin that is pretty cross
> cutting to
> > most of ACS.  So the modules it contributes to are as follows
> >
> > resources/META-INF/cloudstack/baremetal-storage/spring-context.xml
> > resources/META-INF/cloudstack/baremetal-compute/spring-context.xml
> > resources/META-INF/cloudstack/baremetal-discoverer/spring-context.xml
> > resources/META-INF/cloudstack/core/spring-baremetal-core-context.xml
> > resources/META-INF/cloudstack/baremetal-planner/spring-context.xml
> > resources/META-INF/cloudstack/baremetal-network/spring-context.xml
> >
> > So it creates child contextes of storage, compute, network, planner, and
> > discoverer to add its extensions where it needs to be.  Additionally,
> you'll notice,
> > it adds some beans to the core context from the file resources/META-
> > INF/cloudstack/core/spring-baremetal-core-context.xml.  This is because
> it has
> > some manager class that is used by multiple contexts.
> >
> > Frank, I understand the scare that we are going too complex, but do you
> have
> > some other suggestion?  I don't like the idea of one gigantic spring
> context.  So I
> > feel I am making it as simple as I can while maintaining some order.
>  People
> > just need to create one or more spring xml files and a properties files
> that says
> > where to put it in the hierarchy.
> >
> > Additionally, by putting forcing people to put beans in certains modules
> it
> > forces them to think about what is the role of the code.  For example,
> today in
> > ACS the *ManagerImpl classes are a huge mess.  They implement too many
> > interfaces and the code crosses to many architectural boundaries.  Its
> about
> > time we start splitting things up to be more maintainable.
> >
> > If you have some time, please check out what I have on github.
> >
> > Darren
> >
> >
> > On Wed, Sep 18, 2013 at 1:56 PM, Frank Zhang 
> > wrote:
> >
> > > I am not against boundary, I am just against making things unnecessary
> > > complex to enable boundary.
> > > If we are going this way, I hope we can make i

Re: [PROPOSAL] Storage Subsystem API Interface Additions

[Darren Shepherd]

Nah, I guess its not so bad to have a volumeIds param.  Just as long as its
really clear that means nothing about consistency.

I would be a little concerned about how this will be implemented by other
storage providers though.  Currently if you do 5 snapshot API calls that
will launch 5 threads and they will happen in parallel.  If they get
batched and sent in one thread, how is the framework/driver going to handle
the snapshots for drivers that don't supporting batching?  Sequential would
be bad as sometimes it takes awhile to snapshot.

The APIs you mentioned that takes lists really just manipulate data in the
DB, so they can easily batch and transactionally do a bunch at once.

Maybe someone who's more familiar with the storage implementation can
comment?

Darren


On Wed, Sep 18, 2013 at 12:46 PM, SuichII, Christopher <
chris.su...@netapp.com> wrote:

> That certainly would work, but I don't see how it is a better design. Can
> you elaborate on how sending multiple volumeIds is hackish? Look at the
> existing API framework. We have several APIs that accept lists as
> parameters. Normally, they're used for things like querying or deleting.
> Take a look at some of these commands:
> -ArchiveEventsCmd
> -DeleteEventsCmd
> -DeleteSnapshotPoliciesCmd
>
> This kind of API is simply a shorthand for invoking another API many times.
>
> I think it is only an NetApp optimization in the sense that we're the only
> ones who need it right now. What we're asking for has nothing specific to
> do to NetApp. We would just like the shorthand ability to do things all at
> once rather than one at a time. I think other vendors could utilize this
> just as easily.
>
> -Chris
> --
> Chris Suich
> chris.su...@netapp.com
> NetApp Software Engineer
> Data Center Platforms – Cloud Solutions
> Citrix, Cisco & Red Hat
>
> On Sep 18, 2013, at 2:32 PM, Darren Shepherd 
> wrote:
>
> > Given this explanation.  Would the following not work?
> >
> > 1) Enhance UI to allow multi select.  There is no API change, UI will
> just
> > call snapshot a bunch of time
> > 2) Enhance storage framework or driver to detect that 20 requests just
> came
> > in within a window of X seconds and send them to the driver all at once.
> >
> > I know you said queuing on the backend is hackish, but having the user
> send
> > multiple volumeIds in the API is just as hackish to me.  We can only
> > guarentee to the user that the multiple snapshots taken are as consistent
> > as if they called snapshot API individually.  The user won't really know
> > exactly what NetApp volume they exist on and really neither will the
> > storage framework, as export != volume.  Only the driver knows if
> batching
> > is really possible.  So I'm not exactly saying queue, its short batches.
> >
> > In short, I'm seeing this as a bit more of a NetApp optimization than a
> > general thing.  I'm all for using storage device level snapshotting but
> it
> > seems like its going to be implementation specific.  Its interesting, if
> > you look at digital ocean they have snapshots and backups as two
> different
> > concepts.  You can see that they ran into this specific issue.  Full
> > storage volume snapshots are really difficult to expose to the user.  So
> > digital ocean does "backups" which are live but on a schedule and seem to
> > be a full volume backup.  And then there are snapshots which are on
> demand,
> > but require you to stop your VM (so they can essentially copy the qcow or
> > lv somewhere).
> >
> > Darren
> >
> >
> >
> >
> > On Wed, Sep 18, 2013 at 11:12 AM, SuichII, Christopher <
> > chris.su...@netapp.com> wrote:
> >
> >> First, let me apologize for the confusing terms, because some words here
> >> are overloaded:
> >> A volume…
> >> In CloudStack terms is a disk attached to a VM.
> >> In NetApp terms is an NFS volume, analogous to CloudStack primary
> storage,
> >> where all the CloudStack volumes are stored.
> >>
> >> A snapshot…
> >> In CloudStack terms is a backup of a VM.
> >> In NetApp terms is a copy of all the contents of a NetApp volume, taken
> at
> >> a point in time to create an analogous CloudStack snapshot for (up to)
> >> every CloudStack volume on that primary storage.
> >>
> >> There are several reasons that an API for snapshotting multiple volumes
> is
> >> more attractive to us than calling a single volume API over and over. A
> lot
> >> of it has to do with how we actually create the snapshots. Unlike a
> >> hypervisor snapshot, when we create a vm snapshot, the entire primary
> >> storage is backed up (but only the requested volume has an entry added
> to
> >> the db). To add on to this, our hardware has a hard limit of 255 storage
> >> volume level snapshots. So, if there were 255 vms on a single primary
> >> storage and each one of them performed a backup, no more backups could
> be
> >> taken before we start removing the oldest backup (without some trickery
> >> that we are currently working on). Some might say a solution to this
> would
> 

Re: [DISCUSS] Java 7, tomcat 7 and further upgrades

[Wido den Hollander]

On 09/18/2013 06:51 PM, Darren Shepherd wrote:

I'd be relatively opposed to switching everything to jdk7.  Java 6 is the
current lowest common denominator and using java 7 at the moment will just
alienate somebody  I don't know the back story.  Is libvirt compiled for
jdk7?  I'd be perfectly fine with compiling just the KVM driver with jdk7
as any distro that has a usable KVM implementation also has java 7.



A new version of libvirt-java was released, but the RedHat guys compiled 
it with jdk 7.


I already got a patch in at libvirt-java to set the target version to 
1.6: 
http://www.libvirt.org/git/?p=libvirt-java.git;a=commitdiff;h=26efdef08b0d34b75fcd5930b6510dacc14f6637


I hope to get the 0.5.1 release soon which is 1.6 compatible.


So here is what I propose (which I think is what is already being proposed,
but I'm just being explicit).  We keep the maven settings to keep building
1.6 byte code.  Only the KVM driver do we set it to 1.7.  This means that
we would then have to switch to using JDK7 for jenkins, eclipse, mvn cli,
etc.  As the majority of development is done in eclipse (i think), eclipse
still sets up the projects to compile against the JRE 6 library (except KVM
will be JRE 7), so that will help prevent people from accidentally using a
JRE7 API in a 1.6 byte code format.



As far as I know both RHEL/CentOS 6 and Ubuntu 12.04 have JRE 7 in their 
repos, so it's no problem on the packaging side if we target those two 
platforms.


Wido


And as a totally random note, OpenJDK7 on CentOS is way faster than
OpenJDK6 on CentOS.  Regardless of whatever we compile against, everybody
should try use Java 7 at runtime.

Darren


On Wed, Sep 18, 2013 at 9:29 AM, Alex Huang  wrote:


I don't have much problem with switching to jdk1.7.  My eclipse is running
with jdk1.7 as the builder and it can't find any problems in cs code.  The
main question I think will come from the Linux variants.  Are all of them
shipping with jdk1.7 now?

--Alex


-Original Message-
From: Trippie [mailto:trip...@gmail.com] On Behalf Of Hugo Trippaers
Sent: Wednesday, September 18, 2013 5:10 AM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Java 7, tomcat 7 and further upgrades

Hey all,

Sorry for the threadomancy, but the discussion have become relevant again
with the current issues with the libvirt library. Of course this could

also be

solved by updating the libvirt library with a jdk6 version. Still it

might be good

to revisit this topic.

It appears not to be possible to switch code style to 1.7 and produce a

1.6

compatible binary. I remember this working with olders versions, but

didn't

dig to much into this.

So the new question in this thread will be, should we switch CloudStack

to

jdk 1.7?

Cheers,

Hugo

On Jul 1, 2013, at 12:45 AM, Prasanna Santhanam  wrote:


On Thu, Jun 27, 2013 at 04:18:40PM -0400, John Burwell wrote:

All,

I am +1 for Java7.  However, I would like to propose ridding
ourselves of Tomcat entirely and embedding a network stack such as
Netty (http://netty.io) with a servlet bridge.  We have one JSP in
the system that generates JSON resources.  It could be easily
eliminated with a simple servlet that generates JSON from a
ResourceBundle.  Outside of this JSP. I don't see any other
requirements for a JEE container besides hosting a servlet.  We would
gain a far simpler, self-contained deployment model (a single jar).
Additionally, we would gain greater control of the startup and
shutdown lifecycle, as well as, threading dynamics.  If there is
interest in this approach, I have thoughts on how to achieve this
embedding and create a lightweight daemon framework that could be
used for all CloudStack daemons.

As an aside, I also think we should replace our hand-rolled NIO code
with Netty as well.



John - could you break this and other thoughts down a little more in
what's involved?  Perhaps into its own thread. I don't know Netty. And
my J2EE is shaky at best.

It's been a previous wish on this list to have the packaging of
cloudstack into a single easily deployable war instead of all the
complicated packaging we do. So I'd like to hear more of that and
other issues you describe.

--
Prasanna.,


Powered by BigRock.com

RE: conflicting dependencies between CloudStack and Whirr

[Alex Huang]

I actually did a quick try to update cloudstack to use newest gson version 
about 3 months back.  Had to roll it back but I didn't try very hard though.  
Part of the reason why I decided to rollback is due to gson is used differently 
by various components in CloudStack and I didn't have time to get into each 
component to see if I break anything.  The other is also I thought using 
Jackson would be much better and thought our next attempt should be with 
Jackson.  

Not that any of these helps you.  Just know updating CloudStack to latest gson 
is not simple. 

--Alex

> -Original Message-
> From: Andrei Savu [mailto:savu.and...@gmail.com]
> Sent: Wednesday, September 18, 2013 10:53 AM
> To: d...@whirr.apache.org
> Cc: dev@cloudstack.apache.org
> Subject: Re: conflicting dependencies between CloudStack and Whirr
> 
> It's easy to usr jclouds and whirr inside an OSGi container - just add the
> feature url. Bonus: you can also use jclouds shell interface (part of jclouds 
> cli).
> 
> Another option is to upgrade the CloudStack API to use the new version.
> On Sep 18, 2013 5:14 AM, "Han,Meng"  wrote:
> 
> > Dear all,
> >
> > I am adding an API to CloudStack which utilizes Whirr to launch
> > various clusters on CloudStack. Now I am facing a dependency conflicting
> issue.
> >
> > Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires gson 1.7.1.
> > If I use gson 1.7.1 for Whirr, the following error will happen:
> >
> > com.google.common.util.**concurrent.ExecutionError:
> java.lang.**NoClassDefFoundError:
> > com/google/gson/TypeAdapter
> >
> > This TypeAdapter class can be found inside gson-2.2.2.jar. However if
> > I modify CloudStack to use gson 2.2.2 CloudStack will not build
> > successfully due to the following test error.
> >
> > [INFO] Building Apache CloudStack Core 4.1.1 [INFO]
> > --**--**
> > 
> > [INFO]
> > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-core
> > --- [INFO] Deleting /home/meng/cloudstack/core/**target
> > [INFO]
> > [INFO] --- maven-remote-resources-plugin:**1.3:process (default) @
> > cloud-core --- [INFO] [INFO] ---
> > maven-resources-plugin:2.5:**resources (default-resources) @
> > cloud-core --- [debug] execute contextualize [INFO] Using 'UTF-8'
> > encoding to copy filtered resources.
> > [INFO] skip non existing resourceDirectory
> > /home/meng/cloudstack/core/** src/main/resources [INFO] Copying 3
> > resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**compile
> > (default-compile) @ cloud-core --- [INFO] Compiling 156 source files
> > to /home/meng/cloudstack/core/** target/classes [INFO] [INFO] ---
> > maven-resources-plugin:2.5:**testResources
> > (default-testResources) @ cloud-core --- [debug] execute contextualize
> > [INFO] Using 'UTF-8' encoding to copy filtered resources.
> > [INFO] skip non existing resourceDirectory
> > /home/meng/cloudstack/core/** src/test/resources [INFO] Copying 3
> > resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**testCompile
> > (default-testCompile) @ cloud-core --- [INFO] Compiling 1 source file
> > to /home/meng/cloudstack/core/** target/test-classes [INFO] [INFO] ---
> > maven-surefire-plugin:2.12:**test (default-test) @ cloud-core
> > ---
> > [INFO] Surefire report directory: /home/meng/cloudstack/core/**
> > target/surefire-reports
> >
> > --**-
> >  T E S T S
> > --**-
> > Running com.cloud.agent.transport.**RequestTest
> > log4j:WARN No appenders could be found for logger
> > (com.cloud.agent.transport.**RequestTest).
> > log4j:WARN Please initialize the log4j system properly.
> > log4j:WARN See
> http://logging.apache.org/**log4j/1.2/faq.html#noconfig che.org/log4j/1.2/faq.html#noconfig>for more info.
> > Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 3.054
> > sec <<< FAILURE!
> >
> > Results :
> >
> > Failed tests:   testSerDeser(com.cloud.agent.**transport.RequestTest)
> >
> > Tests in error:
> >   testLogging(com.cloud.agent.**transport.RequestTest)
> >
> > Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
> >
> >
> > I ran "mvn -P developer clean install -DskipTests" , the CloudStack
> > building process went through. But when I issued an API
> > --launchCluster to CloudStack, the following error related to gson
> > popped up on CloudStack Management Server:
> >
> > ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught
> > problem with
> > [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
> > 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
> > vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
> > Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
> > snapshot","pool":"/root","**hypervisorType":"KVM","**
> >
> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy"
> > :"**
> > HostLocal","Host.OS":"CentOS

Re: workshops at CCC Europe

[Wido den Hollander]

On 09/18/2013 04:40 PM, Daan Hoogland wrote:

H,

In november in Amterdam there are going to be hackathons and other
workshops held.  Last time David (N.) started a page on this so I
think it would be a good idea to do this again.

https://cwiki.apache.org/confluence/display/CLOUDSTACK/CCC+Europe

please comment and add to it



Cool! I added the Ceph integration. I'd like to look at it with some 
storage minded people and also take a look at how we can do the Xen 
integration.


Wido

Master nonoss is broken?

[Kelven Yang]

In JuniperSrxResource.java, someone to look at it?

Kelven

Re: Master nonoss is broken?

[Alena Prokharchyk]

Looking

From: Kelven Yang mailto:kelven.y...@citrix.com>>
Reply-To: "dev@cloudstack.apache.org" 
mailto:dev@cloudstack.apache.org>>
Date: Wednesday, September 18, 2013 4:14 PM
To: "dev@cloudstack.apache.org" 
mailto:dev@cloudstack.apache.org>>
Subject: Master nonoss is broken?

In JuniperSrxResource.java, someone to look at it?

Kelven

[VOTE][RESULTS] Release Apache CloudStack 4.2.0 (fifth round)

[Animesh Chaturvedi]

The vote has *passed* with the following results (binding PMC votes indicated 
with a "*" next to their name:

+1 : Alex*, Chip*, Sebastien*, Prasanna*, Hugo*, Marcus*, Wido*,  Sebastien, 
Rajesh Batala, Sheng, Vijay, Abhi, Likitha, Ian, Gavin,  Daan, Amogh, Simon 
Weller, 

I'm going to proceed with moving the release into the distribution repo now and 
work on release notes and other documentation tasks.


Thanks
Animesh



On Fri, Sep 13, 2013 at 4:12 PM, Animesh Chaturvedi < 
animesh.chaturv...@citrix.com> wrote:

>
> I've created a 4.2.0 release, with the following artifacts up for a vote:
>
> Git Branch and Commit SH:
>
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=
> refs/heads/4.2
> Commit: c1e24ff89f6d14d6ae74d12dbca108c35449030f
>
> List of changes:
>
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob_plain;
> f=CHANGES;hb=4.2
>
> Source release (checksums and signatures are available at the same
> location):
> https://dist.apache.org/repos/dist/dev/cloudstack/4.2.0/
>
> PGP release keys (signed using 94BE0D7C):
> https://dist.apache.org/repos/dist/release/cloudstack/KEYS
>
> Testing instructions are here:
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Release+test+pr
> ocedure
>
> Vote will be open for 72 hours (Wednesday 9/18 End of Day PST).
>
> For sanity in tallying the vote, can PMC members please be sure to 
> indicate "(binding)" with their vote?
>
> [ ] +1  approve
> [ ] +0  no opinion
> [ ] -1  disapprove (and reason why)
>
>

Re: Master nonoss is broken?

[Darren Shepherd]

It was the below that broke it.  The Juniper code was not updated to
reflect the refactor from vlanId to broadcastUri

commit 2614b00c513734ce6b1c29e572fbd7a37d4059fc
Author: Daan Hoogland 
Date:   Thu Aug 1 16:25:27 2013 +0200

sdn hosted vpc gateways (using lswitch)



On Wed, Sep 18, 2013 at 4:19 PM, Alena Prokharchyk <
alena.prokharc...@citrix.com> wrote:

> Looking
>
> From: Kelven Yang mailto:kelven.y...@citrix.com>>
> Reply-To: "dev@cloudstack.apache.org" <
> dev@cloudstack.apache.org>
> Date: Wednesday, September 18, 2013 4:14 PM
> To: "dev@cloudstack.apache.org" <
> dev@cloudstack.apache.org>
> Subject: Master nonoss is broken?
>
> In JuniperSrxResource.java, someone to look at it?
>
> Kelven
>
>

Re: conflicting dependencies between CloudStack and Whirr

[Kelven Yang]

I almost faced that there is no choice but to go for it for VMware
recently, we found that VMware vSphere 5.1 SDK we are currently using has
some backwards compatibility issue with VMware vSphere 4.x systems.
Therefore it makes a valid business case that CloudStack may have to
support different versions of VMware SDK side-by-side at runtime.

Kelven



On 9/18/13 10:42 AM, "Darren Shepherd"  wrote:

>You know what would be really swell is to just switch to jackson.  The
>gson
>we use is antiquated.  I have no idea what the impact of moving to a
>modern
>version would be.  Jackson, IMO, is a far better framework that has a lot
>of momentum.  Additionally it allows you to use JAXB annotations so that
>you don't have to use framework dependent annotation in your code.  Plus
>it
>does tons and tons of more stuff.
>
>I really don't expect isolated classloaders will come anytime soon.  That
>actually relates to things I'm working on right now, but I have a hard
>time
>justifying adding the overhead and complexity of isolated classloaders at
>the moment.
>
>If we really, really, really, really, really think isolating classloaders
>is a good thing, we can talk about it, but it will be a can of worms.
>
>Darren
>
>
>On Tue, Sep 17, 2013 at 7:14 PM, Han,Meng  wrote:
>
>> Dear all,
>>
>> I am adding an API to CloudStack which utilizes Whirr to launch various
>> clusters on CloudStack. Now I am facing a dependency conflicting issue.
>>
>> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires gson
>>1.7.1.
>> If I use gson 1.7.1 for Whirr, the following error will happen:
>>
>> com.google.common.util.**concurrent.ExecutionError:
>>java.lang.**NoClassDefFoundError:
>> com/google/gson/TypeAdapter
>>
>> This TypeAdapter class can be found inside gson-2.2.2.jar. However if I
>> modify CloudStack to use gson 2.2.2 CloudStack will not build
>>successfully
>> due to the following test error.
>>
>> [INFO] Building Apache CloudStack Core 4.1.1
>> [INFO] --**--**
>> 
>> [INFO]
>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-core ---
>> [INFO] Deleting /home/meng/cloudstack/core/**target
>> [INFO]
>> [INFO] --- maven-remote-resources-plugin:**1.3:process (default) @
>> cloud-core ---
>> [INFO]
>> [INFO] --- maven-resources-plugin:2.5:**resources (default-resources) @
>> cloud-core ---
>> [debug] execute contextualize
>> [INFO] Using 'UTF-8' encoding to copy filtered resources.
>> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
>> src/main/resources
>> [INFO] Copying 3 resources
>> [INFO]
>> [INFO] --- maven-compiler-plugin:2.5.1:**compile (default-compile) @
>> cloud-core ---
>> [INFO] Compiling 156 source files to /home/meng/cloudstack/core/**
>> target/classes
>> [INFO]
>> [INFO] --- maven-resources-plugin:2.5:**testResources
>> (default-testResources) @ cloud-core ---
>> [debug] execute contextualize
>> [INFO] Using 'UTF-8' encoding to copy filtered resources.
>> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
>> src/test/resources
>> [INFO] Copying 3 resources
>> [INFO]
>> [INFO] --- maven-compiler-plugin:2.5.1:**testCompile
>> (default-testCompile) @ cloud-core ---
>> [INFO] Compiling 1 source file to /home/meng/cloudstack/core/**
>> target/test-classes
>> [INFO]
>> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @ cloud-core
>> ---
>> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
>> target/surefire-reports
>>
>> --**-
>>  T E S T S
>> --**-
>> Running com.cloud.agent.transport.**RequestTest
>> log4j:WARN No appenders could be found for logger
>> (com.cloud.agent.transport.**RequestTest).
>> log4j:WARN Please initialize the log4j system properly.
>> log4j:WARN See 
>>http://logging.apache.org/**log4j/1.2/faq.html#noconfig>ache.org/log4j/1.2/faq.html#noconfig>for more info.
>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 3.054
>>sec
>> <<< FAILURE!
>>
>> Results :
>>
>> Failed tests:   testSerDeser(com.cloud.agent.**transport.RequestTest)
>>
>> Tests in error:
>>   testLogging(com.cloud.agent.**transport.RequestTest)
>>
>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>>
>>
>> I ran "mvn -P developer clean install -DskipTests" , the CloudStack
>> building process went through. But when I issued an API --launchCluster
>>to
>> CloudStack, the following error related to gson popped up on CloudStack
>> Management Server:
>>
>> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught problem
>> with [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
>> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
>> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
>> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
>> snapshot","pool":"/root","**hypervisorType":"KVM","**
>> 
>>hos

Re: Master nonoss is broken?

[Alena Prokharchyk]

 Thank you Darren, just fixed with 3ab8d8d8f20c453fdc684f177a612922eae5f415

-Alena.

From: Darren Shepherd 
mailto:darren.s.sheph...@gmail.com>>
Reply-To: "dev@cloudstack.apache.org" 
mailto:dev@cloudstack.apache.org>>
Date: Wednesday, September 18, 2013 4:29 PM
To: "dev@cloudstack.apache.org" 
mailto:dev@cloudstack.apache.org>>
Cc: "dhoogl...@schubergphilis.com" 
mailto:dhoogl...@schubergphilis.com>>
Subject: Re: Master nonoss is broken?

It was the below that broke it.  The Juniper code was not updated to
reflect the refactor from vlanId to broadcastUri

commit 2614b00c513734ce6b1c29e572fbd7a37d4059fc
Author: Daan Hoogland 
mailto:dhoogl...@schubergphilis.com>>
Date:   Thu Aug 1 16:25:27 2013 +0200

sdn hosted vpc gateways (using lswitch)



On Wed, Sep 18, 2013 at 4:19 PM, Alena Prokharchyk <
alena.prokharc...@citrix.com> wrote:

Looking

From: Kelven Yang 
mailto:kelven.y...@citrix.com>>
Reply-To: 
"dev@cloudstack.apache.org"
 <
dev@cloudstack.apache.org>
Date: Wednesday, September 18, 2013 4:14 PM
To: 
"dev@cloudstack.apache.org"
 <
dev@cloudstack.apache.org>
Subject: Master nonoss is broken?

In JuniperSrxResource.java, someone to look at it?

Kelven

Re: Master nonoss is broken?

[Kelven Yang]

Thanks Alena.

Kelven

On 9/18/13 4:38 PM, "Alena Prokharchyk" 
wrote:

> Thank you Darren, just fixed with
>3ab8d8d8f20c453fdc684f177a612922eae5f415
>
>-Alena.
>
>From: Darren Shepherd
>mailto:darren.s.sheph...@gmail.com>>
>Reply-To: "dev@cloudstack.apache.org"
>mailto:dev@cloudstack.apache.org>>
>Date: Wednesday, September 18, 2013 4:29 PM
>To: "dev@cloudstack.apache.org"
>mailto:dev@cloudstack.apache.org>>
>Cc: "dhoogl...@schubergphilis.com"
>mailto:dhoogl...@schubergphilis.com>>
>Subject: Re: Master nonoss is broken?
>
>It was the below that broke it.  The Juniper code was not updated to
>reflect the refactor from vlanId to broadcastUri
>
>commit 2614b00c513734ce6b1c29e572fbd7a37d4059fc
>Author: Daan Hoogland
>mailto:dhoogl...@schubergphilis.com>>
>Date:   Thu Aug 1 16:25:27 2013 +0200
>
>sdn hosted vpc gateways (using lswitch)
>
>
>
>On Wed, Sep 18, 2013 at 4:19 PM, Alena Prokharchyk <
>alena.prokharc...@citrix.com> wrote:
>
>Looking
>
>From: Kelven Yang 
>mailto:kelven.y...@citrix.com>citrix.com>>
>Reply-To: 
>"dev@cloudstack.apache.orgoudstack.apache.org>" <
>dev@cloudstack.apache.orgudstack.apache.org>>
>Date: Wednesday, September 18, 2013 4:14 PM
>To: 
>"dev@cloudstack.apache.orgoudstack.apache.org>" <
>dev@cloudstack.apache.orgudstack.apache.org>>
>Subject: Master nonoss is broken?
>
>In JuniperSrxResource.java, someone to look at it?
>
>Kelven
>
>
>

Re: conflicting dependencies between CloudStack and Whirr

[Darren Shepherd]

I can do some analysis on this.  I'm always up for a terribly painful
refactor :)

Darren


On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang  wrote:

> I actually did a quick try to update cloudstack to use newest gson version
> about 3 months back.  Had to roll it back but I didn't try very hard
> though.  Part of the reason why I decided to rollback is due to gson is
> used differently by various components in CloudStack and I didn't have time
> to get into each component to see if I break anything.  The other is also I
> thought using Jackson would be much better and thought our next attempt
> should be with Jackson.
>
> Not that any of these helps you.  Just know updating CloudStack to latest
> gson is not simple.
>
> --Alex
>
> > -Original Message-
> > From: Andrei Savu [mailto:savu.and...@gmail.com]
> > Sent: Wednesday, September 18, 2013 10:53 AM
> > To: d...@whirr.apache.org
> > Cc: dev@cloudstack.apache.org
> > Subject: Re: conflicting dependencies between CloudStack and Whirr
> >
> > It's easy to usr jclouds and whirr inside an OSGi container - just add
> the
> > feature url. Bonus: you can also use jclouds shell interface (part of
> jclouds cli).
> >
> > Another option is to upgrade the CloudStack API to use the new version.
> > On Sep 18, 2013 5:14 AM, "Han,Meng"  wrote:
> >
> > > Dear all,
> > >
> > > I am adding an API to CloudStack which utilizes Whirr to launch
> > > various clusters on CloudStack. Now I am facing a dependency
> conflicting
> > issue.
> > >
> > > Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires gson
> 1.7.1.
> > > If I use gson 1.7.1 for Whirr, the following error will happen:
> > >
> > > com.google.common.util.**concurrent.ExecutionError:
> > java.lang.**NoClassDefFoundError:
> > > com/google/gson/TypeAdapter
> > >
> > > This TypeAdapter class can be found inside gson-2.2.2.jar. However if
> > > I modify CloudStack to use gson 2.2.2 CloudStack will not build
> > > successfully due to the following test error.
> > >
> > > [INFO] Building Apache CloudStack Core 4.1.1 [INFO]
> > > --**--**
> > > 
> > > [INFO]
> > > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-core
> > > --- [INFO] Deleting /home/meng/cloudstack/core/**target
> > > [INFO]
> > > [INFO] --- maven-remote-resources-plugin:**1.3:process (default) @
> > > cloud-core --- [INFO] [INFO] ---
> > > maven-resources-plugin:2.5:**resources (default-resources) @
> > > cloud-core --- [debug] execute contextualize [INFO] Using 'UTF-8'
> > > encoding to copy filtered resources.
> > > [INFO] skip non existing resourceDirectory
> > > /home/meng/cloudstack/core/** src/main/resources [INFO] Copying 3
> > > resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**compile
> > > (default-compile) @ cloud-core --- [INFO] Compiling 156 source files
> > > to /home/meng/cloudstack/core/** target/classes [INFO] [INFO] ---
> > > maven-resources-plugin:2.5:**testResources
> > > (default-testResources) @ cloud-core --- [debug] execute contextualize
> > > [INFO] Using 'UTF-8' encoding to copy filtered resources.
> > > [INFO] skip non existing resourceDirectory
> > > /home/meng/cloudstack/core/** src/test/resources [INFO] Copying 3
> > > resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**testCompile
> > > (default-testCompile) @ cloud-core --- [INFO] Compiling 1 source file
> > > to /home/meng/cloudstack/core/** target/test-classes [INFO] [INFO] ---
> > > maven-surefire-plugin:2.12:**test (default-test) @ cloud-core
> > > ---
> > > [INFO] Surefire report directory: /home/meng/cloudstack/core/**
> > > target/surefire-reports
> > >
> > > --**-
> > >  T E S T S
> > > --**-
> > > Running com.cloud.agent.transport.**RequestTest
> > > log4j:WARN No appenders could be found for logger
> > > (com.cloud.agent.transport.**RequestTest).
> > > log4j:WARN Please initialize the log4j system properly.
> > > log4j:WARN See
> > http://logging.apache.org/**log4j/1.2/faq.html#noconfig<
> http://logging.apa
> > che.org/log4j/1.2/faq.html#noconfig>for more info.
> > > Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 3.054
> > > sec <<< FAILURE!
> > >
> > > Results :
> > >
> > > Failed tests:   testSerDeser(com.cloud.agent.**transport.RequestTest)
> > >
> > > Tests in error:
> > >   testLogging(com.cloud.agent.**transport.RequestTest)
> > >
> > > Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
> > >
> > >
> > > I ran "mvn -P developer clean install -DskipTests" , the CloudStack
> > > building process went through. But when I issued an API
> > > --launchCluster to CloudStack, the following error related to gson
> > > popped up on CloudStack Management Server:
> > >
> > > ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught
> > > problem with
> > > [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
> > > 3844370432,"d

Re: [VOTE][RESULTS] Release Apache CloudStack 4.2.0 (fifth round)

[Wido den Hollander]

On 09/19/2013 01:28 AM, Animesh Chaturvedi wrote:


The vote has *passed* with the following results (binding PMC votes indicated with a 
"*" next to their name:

+1 : Alex*, Chip*, Sebastien*, Prasanna*, Hugo*, Marcus*, Wido*,  Sebastien, 
Rajesh Batala, Sheng, Vijay, Abhi, Likitha, Ian, Gavin,  Daan, Amogh, Simon 
Weller,

I'm going to proceed with moving the release into the distribution repo now and 
work on release notes and other documentation tasks.


Who is going to build the RPM and Deb packages?

I think I should build the .deb packages since I'm the one in 
debian/changelog who set the version to 4.2


Give me a sign and I'll put the packages online in the DEB repo.

Wido



Thanks
Animesh



On Fri, Sep 13, 2013 at 4:12 PM, Animesh Chaturvedi < 
animesh.chaturv...@citrix.com> wrote:



I've created a 4.2.0 release, with the following artifacts up for a vote:

Git Branch and Commit SH:

https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=
refs/heads/4.2
Commit: c1e24ff89f6d14d6ae74d12dbca108c35449030f

List of changes:

https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob_plain;
f=CHANGES;hb=4.2

Source release (checksums and signatures are available at the same
location):
https://dist.apache.org/repos/dist/dev/cloudstack/4.2.0/

PGP release keys (signed using 94BE0D7C):
https://dist.apache.org/repos/dist/release/cloudstack/KEYS

Testing instructions are here:

https://cwiki.apache.org/confluence/display/CLOUDSTACK/Release+test+pr
ocedure

Vote will be open for 72 hours (Wednesday 9/18 End of Day PST).

For sanity in tallying the vote, can PMC members please be sure to
indicate "(binding)" with their vote?

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)

Re: [PROPOSAL] Storage Subsystem API Interface Additions

[SuichII, Christopher]

That's a good point and I'm not sure. Maybe we can have the drivers indicate 
whether they support batched/multiple volume snapshotting. If not, then we can 
spin up a thread per volume to snapshot. I completely agree that simply calling 
them in sequence could end up badly.

You're right though, the examples I provided are only partial examples as they 
really do just edit something in the db.

Edison, Alex or anyone else knowledgable of the storage system - Do you have 
any input. Was my clarification helpful?

For what it's worth, after some further investigation, it sounds like we won't 
actually be performing NetApp volume level snapshots to backup vm volumes - 
either when requested or as a batch. We believe we have come up with a solution 
that lets us create individual backups while avoiding our 255 snapshot limit. 
However, it would still be beneficial to let users request backups of multiple 
volumes at once for the other reasons I explained earlier.

-Chris
-- 
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Sep 18, 2013, at 6:56 PM, Darren Shepherd  
wrote:

> Nah, I guess its not so bad to have a volumeIds param.  Just as long as its
> really clear that means nothing about consistency.
> 
> I would be a little concerned about how this will be implemented by other
> storage providers though.  Currently if you do 5 snapshot API calls that
> will launch 5 threads and they will happen in parallel.  If they get
> batched and sent in one thread, how is the framework/driver going to handle
> the snapshots for drivers that don't supporting batching?  Sequential would
> be bad as sometimes it takes awhile to snapshot.
> 
> The APIs you mentioned that takes lists really just manipulate data in the
> DB, so they can easily batch and transactionally do a bunch at once.
> 
> Maybe someone who's more familiar with the storage implementation can
> comment?
> 
> Darren
> 
> 
> On Wed, Sep 18, 2013 at 12:46 PM, SuichII, Christopher <
> chris.su...@netapp.com> wrote:
> 
>> That certainly would work, but I don't see how it is a better design. Can
>> you elaborate on how sending multiple volumeIds is hackish? Look at the
>> existing API framework. We have several APIs that accept lists as
>> parameters. Normally, they're used for things like querying or deleting.
>> Take a look at some of these commands:
>> -ArchiveEventsCmd
>> -DeleteEventsCmd
>> -DeleteSnapshotPoliciesCmd
>> 
>> This kind of API is simply a shorthand for invoking another API many times.
>> 
>> I think it is only an NetApp optimization in the sense that we're the only
>> ones who need it right now. What we're asking for has nothing specific to
>> do to NetApp. We would just like the shorthand ability to do things all at
>> once rather than one at a time. I think other vendors could utilize this
>> just as easily.
>> 
>> -Chris
>> --
>> Chris Suich
>> chris.su...@netapp.com
>> NetApp Software Engineer
>> Data Center Platforms – Cloud Solutions
>> Citrix, Cisco & Red Hat
>> 
>> On Sep 18, 2013, at 2:32 PM, Darren Shepherd 
>> wrote:
>> 
>>> Given this explanation.  Would the following not work?
>>> 
>>> 1) Enhance UI to allow multi select.  There is no API change, UI will
>> just
>>> call snapshot a bunch of time
>>> 2) Enhance storage framework or driver to detect that 20 requests just
>> came
>>> in within a window of X seconds and send them to the driver all at once.
>>> 
>>> I know you said queuing on the backend is hackish, but having the user
>> send
>>> multiple volumeIds in the API is just as hackish to me.  We can only
>>> guarentee to the user that the multiple snapshots taken are as consistent
>>> as if they called snapshot API individually.  The user won't really know
>>> exactly what NetApp volume they exist on and really neither will the
>>> storage framework, as export != volume.  Only the driver knows if
>> batching
>>> is really possible.  So I'm not exactly saying queue, its short batches.
>>> 
>>> In short, I'm seeing this as a bit more of a NetApp optimization than a
>>> general thing.  I'm all for using storage device level snapshotting but
>> it
>>> seems like its going to be implementation specific.  Its interesting, if
>>> you look at digital ocean they have snapshots and backups as two
>> different
>>> concepts.  You can see that they ran into this specific issue.  Full
>>> storage volume snapshots are really difficult to expose to the user.  So
>>> digital ocean does "backups" which are live but on a schedule and seem to
>>> be a full volume backup.  And then there are snapshots which are on
>> demand,
>>> but require you to stop your VM (so they can essentially copy the qcow or
>>> lv somewhere).
>>> 
>>> Darren
>>> 
>>> 
>>> 
>>> 
>>> On Wed, Sep 18, 2013 at 11:12 AM, SuichII, Christopher <
>>> chris.su...@netapp.com> wrote:
>>> 
 First, let me apologize for the confusing terms, because some words here
 are overloaded:
 A volume…

Re: [PROPOSAL] Modularize Spring

[SuichII, Christopher]

I've been following this conversation somewhat and would like to throw in some 
background as a plugin writer.

One thing that concerns me in the current plugin model is the number of 
XML/text files that need to be edited to deploy my plugin. 
-applicationContext must be edited to add our PluginMangerImpl.
-commands.properties file must be edited to included the permissions for the 
APIs we contributed.
-componentContext.xml & nonossComponentContext.xml must be edited to add our 
Storage Subsystem Provider API.
-log4j-cloud.xml must be edited to ensure that our logger is enabled and 
logging to our necessary default level.

I know our situation is a bit different than the current plug in model, but I 
think it is something we, as a community, are going to begin seeing more of. 
For a variety of reasons that I won't get in to right now, our plugin will be 
closed source and kept separate from the ACS source tree. We want our users to 
be able to simply drop in our jar file to the CS directory or run and installer 
and have it picked up by the MS upon a restart.

It sounds like what you are proposing here would be very beneficial to plugins 
that are targeting a deployment model like this.

Is this something you're looking/hoping/would like to solve, Darren?

-Chris
-- 
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Sep 18, 2013, at 6:44 PM, Darren Shepherd  
wrote:

> I'm not a committer
> 
> 
> On Wed, Sep 18, 2013 at 3:24 PM, Frank Zhang  wrote:
> 
>> Well. The codes explain more than words.
>> It seems the only extra work is adding a property file that specifies
>> parent context and current context name, it's not much complex.
>> BTW: any reason for working on repo outside ACS?
>> 
>>> -Original Message-
>>> From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com]
>>> Sent: Wednesday, September 18, 2013 2:43 PM
>>> To: dev@cloudstack.apache.org
>>> Subject: Re: [PROPOSAL] Modularize Spring
>>> 
>>> If you want to see this all working you can just fetch the "no-at-db4"
>>> branch at https://github.com/ibuildthecloud/cloudstack.git
>>> 
>>> Plugin composes multiple modules.  If modules are siblings they can't
>> inject
>>> from each other.  But a plugin can augment another module if it chooses
>> to.
>>> The reality is that the core cloudstack is a tangled mess of
>> dependencies such
>>> that most of the core code can't be modularized as it stands.  So there
>> exists a
>>> context towards the top of the hierarchy called "core" that a lot of jars
>>> contribute to it.  Here is the full hierarchy right now.  I'll probably
>> rename a
>>> bunch of things, but this gives you an idea.
>>> 
>>> bootstrap
>>>  system
>>>core
>>>  allocator
>>>allocator-server
>>>planner
>>>  api-planner
>>>  baremetal-planner
>>>  explicit-dedication
>>>  host-anti-affinity
>>>  implicit-dedication-planner
>>>  server-planner
>>>  user-concentrated-pod-planner
>>>random-allocator
>>>  api
>>>acl-static-role-based
>>>rate-limit
>>>server-api
>>>user-authenticator-ldap
>>>user-authenticator-md5
>>>user-authenticator-plaintext
>>>user-authenticator-sha256salted
>>>  backend
>>>alert-adapter-server-backend
>>>compute
>>>  alert-adapter-server-compute
>>>  baremetal-compute
>>>  fencer-server
>>>  investigator-server
>>>  kvm-compute
>>>  ovm-compute
>>>  server-compute
>>>  xenserver-compute
>>>network
>>>  baremetal-network
>>>  elb
>>>  midonet
>>>  nvp
>>>  ovs
>>>  server-network
>>>  ssp
>>>storage
>>>  alert-adapter-server-storage
>>>  allocator-storage
>>>  baremetal-storage
>>>  secondary-storage
>>>  server-storage
>>>  storage-image-default
>>>  storage-image-s3
>>>  storage-image-swift
>>>  storage-volume-default
>>>  storage-volume-solidfire
>>>  template-adapter-server
>>>  discoverer
>>>baremetal-discoverer
>>>discoverer-server
>>>ovm-discoverer
>>>xenserver-discoverer
>>> 
>>> 
>>> 
>>> If you look at the baremetal hypervisor plugin that is pretty cross
>> cutting to
>>> most of ACS.  So the modules it contributes to are as follows
>>> 
>>> resources/META-INF/cloudstack/baremetal-storage/spring-context.xml
>>> resources/META-INF/cloudstack/baremetal-compute/spring-context.xml
>>> resources/META-INF/cloudstack/baremetal-discoverer/spring-context.xml
>>> resources/META-INF/cloudstack/core/spring-baremetal-core-context.xml
>>> resources/META-INF/cloudstack/baremetal-planner/spring-context.xml
>>> resources/META-INF/cloudstack/baremetal-network/spring-context.xml
>>> 
>>> So it creates child contextes of stora

Re: Review Request 14186: Create parent pom to allow maven standard paths in project

[Kelven Yang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14186/#review26249
---

Ship it!


Ship It!

- Kelven Yang


On Sept. 17, 2013, 11:08 p.m., Darren Shepherd wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14186/
> ---
> 
> (Updated Sept. 17, 2013, 11:08 p.m.)
> 
> 
> Review request for cloudstack, Alex Huang and Kelven Yang.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Currently all cloudstack poms use non-standard maven paths for artifacts.  
> The overrides are defined in the parent pom.  This parent pom extends the 
> cloudstack top level POM but then overrides the non-standard paths back to 
> the standard paths.  This patch adds the parent pom but does not add it to 
> the build.  That will come in another patch.
> 
> 
> Diffs
> -
> 
>   maven-standard/pom.xml PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/14186/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Darren Shepherd
> 
>

Re: [PROPOSAL] Modularize Spring

[SuichII, Christopher]

s/PluginManagerImpl/PluggableService/

Along with a storage provider plug in, we are also creating a generic API 
plugin to support the UI plugin we are developing (too many plugins….).
-- 
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Sep 18, 2013, at 8:46 PM, "SuichII, Christopher"  
wrote:

> I've been following this conversation somewhat and would like to throw in 
> some background as a plugin writer.
> 
> One thing that concerns me in the current plugin model is the number of 
> XML/text files that need to be edited to deploy my plugin. 
> -applicationContext must be edited to add our PluginMangerImpl.
> -commands.properties file must be edited to included the permissions for the 
> APIs we contributed.
> -componentContext.xml & nonossComponentContext.xml must be edited to add our 
> Storage Subsystem Provider API.
> -log4j-cloud.xml must be edited to ensure that our logger is enabled and 
> logging to our necessary default level.
> 
> I know our situation is a bit different than the current plug in model, but I 
> think it is something we, as a community, are going to begin seeing more of. 
> For a variety of reasons that I won't get in to right now, our plugin will be 
> closed source and kept separate from the ACS source tree. We want our users 
> to be able to simply drop in our jar file to the CS directory or run and 
> installer and have it picked up by the MS upon a restart.
> 
> It sounds like what you are proposing here would be very beneficial to 
> plugins that are targeting a deployment model like this.
> 
> Is this something you're looking/hoping/would like to solve, Darren?
> 
> -Chris
> -- 
> Chris Suich
> chris.su...@netapp.com
> NetApp Software Engineer
> Data Center Platforms – Cloud Solutions
> Citrix, Cisco & Red Hat
> 
> On Sep 18, 2013, at 6:44 PM, Darren Shepherd  
> wrote:
> 
>> I'm not a committer
>> 
>> 
>> On Wed, Sep 18, 2013 at 3:24 PM, Frank Zhang  wrote:
>> 
>>> Well. The codes explain more than words.
>>> It seems the only extra work is adding a property file that specifies
>>> parent context and current context name, it's not much complex.
>>> BTW: any reason for working on repo outside ACS?
>>> 
 -Original Message-
 From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com]
 Sent: Wednesday, September 18, 2013 2:43 PM
 To: dev@cloudstack.apache.org
 Subject: Re: [PROPOSAL] Modularize Spring
 
 If you want to see this all working you can just fetch the "no-at-db4"
 branch at https://github.com/ibuildthecloud/cloudstack.git
 
 Plugin composes multiple modules.  If modules are siblings they can't
>>> inject
 from each other.  But a plugin can augment another module if it chooses
>>> to.
 The reality is that the core cloudstack is a tangled mess of
>>> dependencies such
 that most of the core code can't be modularized as it stands.  So there
>>> exists a
 context towards the top of the hierarchy called "core" that a lot of jars
 contribute to it.  Here is the full hierarchy right now.  I'll probably
>>> rename a
 bunch of things, but this gives you an idea.
 
 bootstrap
 system
   core
 allocator
   allocator-server
   planner
 api-planner
 baremetal-planner
 explicit-dedication
 host-anti-affinity
 implicit-dedication-planner
 server-planner
 user-concentrated-pod-planner
   random-allocator
 api
   acl-static-role-based
   rate-limit
   server-api
   user-authenticator-ldap
   user-authenticator-md5
   user-authenticator-plaintext
   user-authenticator-sha256salted
 backend
   alert-adapter-server-backend
   compute
 alert-adapter-server-compute
 baremetal-compute
 fencer-server
 investigator-server
 kvm-compute
 ovm-compute
 server-compute
 xenserver-compute
   network
 baremetal-network
 elb
 midonet
 nvp
 ovs
 server-network
 ssp
   storage
 alert-adapter-server-storage
 allocator-storage
 baremetal-storage
 secondary-storage
 server-storage
 storage-image-default
 storage-image-s3
 storage-image-swift
 storage-volume-default
 storage-volume-solidfire
 template-adapter-server
 discoverer
   baremetal-discoverer
   discoverer-server
   ovm-discoverer
   xenserver-discoverer
 
 
 
 If you look at the baremetal hypervisor plugin that is pretty cross
>>> cutting to
 most of ACS.  So the modules it contributes to are as fol

Re: Review Request 14189: UserVmDomRInvestigator and ManagementIPSystemVMInvestigator mask adapter name

[Kelven Yang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14189/#review26250
---

Ship it!


Ship It!

- Kelven Yang


On Sept. 18, 2013, midnight, Darren Shepherd wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14189/
> ---
> 
> (Updated Sept. 18, 2013, midnight)
> 
> 
> Review request for cloudstack, Alex Huang and Kelven Yang.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> UserVmDomRInvestigator and ManagementIPSystemVMInvestigator mask _name from 
> ComponentLifeCycleBase making set/getName() not work as expected
> 
> 
> Diffs
> -
> 
>   server/src/com/cloud/ha/ManagementIPSystemVMInvestigator.java 2b6d261 
>   server/src/com/cloud/ha/UserVmDomRInvestigator.java 195deff 
> 
> Diff: https://reviews.apache.org/r/14189/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Darren Shepherd
> 
>

Re: [PROPOSAL] Modularize Spring

[Darren Shepherd]

Yes this is one of the many things this is trying to address.  Adding a
plugin should be plopping your jar in a directory and restarting.  You
pointed out two things I didn't think about though, command.properties and
log4j xml.  Let me think about those twos as they should be address also.
Basically you should never have to edit a file that is packaged as part of
ACS.  Only add your artifacts to some directory, ideally just a jar.

Darren


On Wed, Sep 18, 2013 at 5:46 PM, SuichII, Christopher <
chris.su...@netapp.com> wrote:

> I've been following this conversation somewhat and would like to throw in
> some background as a plugin writer.
>
> One thing that concerns me in the current plugin model is the number of
> XML/text files that need to be edited to deploy my plugin.
> -applicationContext must be edited to add our PluginMangerImpl.
> -commands.properties file must be edited to included the permissions for
> the APIs we contributed.
> -componentContext.xml & nonossComponentContext.xml must be edited to add
> our Storage Subsystem Provider API.
> -log4j-cloud.xml must be edited to ensure that our logger is enabled and
> logging to our necessary default level.
>
> I know our situation is a bit different than the current plug in model,
> but I think it is something we, as a community, are going to begin seeing
> more of. For a variety of reasons that I won't get in to right now, our
> plugin will be closed source and kept separate from the ACS source tree. We
> want our users to be able to simply drop in our jar file to the CS
> directory or run and installer and have it picked up by the MS upon a
> restart.
>
> It sounds like what you are proposing here would be very beneficial to
> plugins that are targeting a deployment model like this.
>
> Is this something you're looking/hoping/would like to solve, Darren?
>
> -Chris
> --
> Chris Suich
> chris.su...@netapp.com
> NetApp Software Engineer
> Data Center Platforms – Cloud Solutions
> Citrix, Cisco & Red Hat
>
> On Sep 18, 2013, at 6:44 PM, Darren Shepherd 
> wrote:
>
> > I'm not a committer
> >
> >
> > On Wed, Sep 18, 2013 at 3:24 PM, Frank Zhang 
> wrote:
> >
> >> Well. The codes explain more than words.
> >> It seems the only extra work is adding a property file that specifies
> >> parent context and current context name, it's not much complex.
> >> BTW: any reason for working on repo outside ACS?
> >>
> >>> -Original Message-
> >>> From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com]
> >>> Sent: Wednesday, September 18, 2013 2:43 PM
> >>> To: dev@cloudstack.apache.org
> >>> Subject: Re: [PROPOSAL] Modularize Spring
> >>>
> >>> If you want to see this all working you can just fetch the "no-at-db4"
> >>> branch at https://github.com/ibuildthecloud/cloudstack.git
> >>>
> >>> Plugin composes multiple modules.  If modules are siblings they can't
> >> inject
> >>> from each other.  But a plugin can augment another module if it chooses
> >> to.
> >>> The reality is that the core cloudstack is a tangled mess of
> >> dependencies such
> >>> that most of the core code can't be modularized as it stands.  So there
> >> exists a
> >>> context towards the top of the hierarchy called "core" that a lot of
> jars
> >>> contribute to it.  Here is the full hierarchy right now.  I'll probably
> >> rename a
> >>> bunch of things, but this gives you an idea.
> >>>
> >>> bootstrap
> >>>  system
> >>>core
> >>>  allocator
> >>>allocator-server
> >>>planner
> >>>  api-planner
> >>>  baremetal-planner
> >>>  explicit-dedication
> >>>  host-anti-affinity
> >>>  implicit-dedication-planner
> >>>  server-planner
> >>>  user-concentrated-pod-planner
> >>>random-allocator
> >>>  api
> >>>acl-static-role-based
> >>>rate-limit
> >>>server-api
> >>>user-authenticator-ldap
> >>>user-authenticator-md5
> >>>user-authenticator-plaintext
> >>>user-authenticator-sha256salted
> >>>  backend
> >>>alert-adapter-server-backend
> >>>compute
> >>>  alert-adapter-server-compute
> >>>  baremetal-compute
> >>>  fencer-server
> >>>  investigator-server
> >>>  kvm-compute
> >>>  ovm-compute
> >>>  server-compute
> >>>  xenserver-compute
> >>>network
> >>>  baremetal-network
> >>>  elb
> >>>  midonet
> >>>  nvp
> >>>  ovs
> >>>  server-network
> >>>  ssp
> >>>storage
> >>>  alert-adapter-server-storage
> >>>  allocator-storage
> >>>  baremetal-storage
> >>>  secondary-storage
> >>>  server-storage
> >>>  storage-image-default
> >>>  storage-image-s3
> >>>  storage-image-swift
> >>>  storage-volume-default
> >>>  storage-volume-solidfire
> >>>  template-adapter-server
> >>>  discoverer
> >>>

Re: [DISCUSS] Java 7, tomcat 7 and further upgrades

[Darren Shepherd]

I know rhel6 has jdk6 as the default and jdk7 is also available.  Seems
silly to target 1.7 for a redhat project.  If libvirt can just target 1.6 I
think that would be ideal and then we can just not change anything on our
side.  Somebody can correct me if I'm wrong, but I see the vast majority of
projects targeting 1.6 unless they really need to use a 1.7 API, which
isn't all that much.  Java 8 is going to be the next real jump that most
projects move to as, similar to 1.5, it has real language features people
want.

Darren


On Wed, Sep 18, 2013 at 4:02 PM, Wido den Hollander  wrote:

>
>
> On 09/18/2013 06:51 PM, Darren Shepherd wrote:
>
>> I'd be relatively opposed to switching everything to jdk7.  Java 6 is the
>> current lowest common denominator and using java 7 at the moment will just
>> alienate somebody  I don't know the back story.  Is libvirt compiled for
>> jdk7?  I'd be perfectly fine with compiling just the KVM driver with jdk7
>> as any distro that has a usable KVM implementation also has java 7.
>>
>>
> A new version of libvirt-java was released, but the RedHat guys compiled
> it with jdk 7.
>
> I already got a patch in at libvirt-java to set the target version to 1.6:
> http://www.libvirt.org/git/?p=**libvirt-java.git;a=commitdiff;**h=**
> 26efdef08b0d34b75fcd5930b6510d**acc14f6637
>
> I hope to get the 0.5.1 release soon which is 1.6 compatible.
>
>
>  So here is what I propose (which I think is what is already being
>> proposed,
>> but I'm just being explicit).  We keep the maven settings to keep building
>> 1.6 byte code.  Only the KVM driver do we set it to 1.7.  This means that
>> we would then have to switch to using JDK7 for jenkins, eclipse, mvn cli,
>> etc.  As the majority of development is done in eclipse (i think), eclipse
>> still sets up the projects to compile against the JRE 6 library (except
>> KVM
>> will be JRE 7), so that will help prevent people from accidentally using a
>> JRE7 API in a 1.6 byte code format.
>>
>>
> As far as I know both RHEL/CentOS 6 and Ubuntu 12.04 have JRE 7 in their
> repos, so it's no problem on the packaging side if we target those two
> platforms.
>
> Wido
>
>
>  And as a totally random note, OpenJDK7 on CentOS is way faster than
>> OpenJDK6 on CentOS.  Regardless of whatever we compile against, everybody
>> should try use Java 7 at runtime.
>>
>> Darren
>>
>>
>> On Wed, Sep 18, 2013 at 9:29 AM, Alex Huang 
>> wrote:
>>
>>  I don't have much problem with switching to jdk1.7.  My eclipse is
>>> running
>>> with jdk1.7 as the builder and it can't find any problems in cs code.
>>>  The
>>> main question I think will come from the Linux variants.  Are all of them
>>> shipping with jdk1.7 now?
>>>
>>> --Alex
>>>
>>>  -Original Message-
 From: Trippie [mailto:trip...@gmail.com] On Behalf Of Hugo Trippaers
 Sent: Wednesday, September 18, 2013 5:10 AM
 To: dev@cloudstack.apache.org
 Subject: Re: [DISCUSS] Java 7, tomcat 7 and further upgrades

 Hey all,

 Sorry for the threadomancy, but the discussion have become relevant
 again
 with the current issues with the libvirt library. Of course this could

>>> also be
>>>
 solved by updating the libvirt library with a jdk6 version. Still it

>>> might be good
>>>
 to revisit this topic.

 It appears not to be possible to switch code style to 1.7 and produce a

>>> 1.6
>>>
 compatible binary. I remember this working with olders versions, but

>>> didn't
>>>
 dig to much into this.

 So the new question in this thread will be, should we switch CloudStack

>>> to
>>>
 jdk 1.7?

 Cheers,

 Hugo

 On Jul 1, 2013, at 12:45 AM, Prasanna Santhanam  wrote:

  On Thu, Jun 27, 2013 at 04:18:40PM -0400, John Burwell wrote:
>
>> All,
>>
>> I am +1 for Java7.  However, I would like to propose ridding
>> ourselves of Tomcat entirely and embedding a network stack such as
>> Netty (http://netty.io) with a servlet bridge.  We have one JSP in
>> the system that generates JSON resources.  It could be easily
>> eliminated with a simple servlet that generates JSON from a
>> ResourceBundle.  Outside of this JSP. I don't see any other
>> requirements for a JEE container besides hosting a servlet.  We would
>> gain a far simpler, self-contained deployment model (a single jar).
>> Additionally, we would gain greater control of the startup and
>> shutdown lifecycle, as well as, threading dynamics.  If there is
>> interest in this approach, I have thoughts on how to achieve this
>> embedding and create a lightweight daemon framework that could be
>> used for all CloudStack daemons.
>>
>> As an aside, I also think we should replace our hand-rolled NIO code
>> with Netty as well.
>>
>>
> John - could you break this

Re: xenserver tools

[Abhinandan Prateek]

Here is the list of Xenserver products and related licenses
http://www.xenserver.org/about-xenserver-open-source/gplv2-license.html.

On 18/09/13 2:06 pm, "Daan Hoogland"  wrote:

>Thanks to my colleague Joris,
>
>I can update the scripts in the sytemvm.iso but the real question is
>about copy rights. Xenserver is gpl these days isn't it. Can we
>redistribute them? any legal advice. The alternative is maintaining a
>nonoss systemvm.iso and maybe template.
>
>regards,
>Daan
>
>On Wed, Sep 18, 2013 at 9:01 AM, Abhinandan Prateek
> wrote:
>> Daan,
>>   The info you provided is useful.
>> We are verifying the steps, once done will update the systemvm scripts.
>>
>>
>> On 17/09/13 6:03 pm, "Daan Hoogland"  wrote:
>>
>>>Hi Abhinandan,
>>>
>>>What we have seen so far is that two files in the xe-guest-utilities
>>>package are missing, where CS assumes that they are there.
>>>
>>>/usr/bin/xenstore  => used in script /usr/sbin/xe-update-guest-attrs
>>>(this script is also included in the package but overwritten by the CS
>>>systemvm.iso scripts)
>>>And
>>>/etc/init.d/xe-linux-distribution => calls /usr/sbin/xe-daemon (this
>>>script is also included in the package but overwritten by the CS
>>>systemvm.iso scripts)
>>>
>>>This is the package we installed to fix it and the files included in
>>>that package.
>>>
>>>root@r-8691-VM:~# dpkg -L xe-guest-utilities
>>>/.
>>>/usr
>>>/usr/bin
>>>/usr/bin/xenstore-write
>>>/usr/bin/xenstore
>>>/usr/bin/xenstore-read
>>>/usr/bin/xenstore-exists
>>>/usr/bin/xenstore-list
>>>/usr/bin/xenstore-ls
>>>/usr/bin/xenstore-chmod
>>>/usr/bin/xenstore-rm
>>>/usr/share
>>>/usr/share/xe-guest-utilities
>>>/usr/share/xe-guest-utilities/citrix.list
>>>/usr/share/doc
>>>/usr/share/doc/xe-guest-utilities
>>>/usr/share/doc/xe-guest-utilities/COPYING.LGPL.gz
>>>/usr/share/doc/xe-guest-utilities/copyright
>>>/usr/share/doc/xe-guest-utilities/COPYING.gz
>>>/usr/sbin
>>>/usr/sbin/xe-update-guest-attrs
>>>/usr/sbin/xe-linux-distribution
>>>/usr/sbin/xe-daemon
>>>/etc
>>>/etc/udev
>>>/etc/udev/rules.d
>>>/etc/udev/rules.d/z10_xen-vcpu-hotplug.rules
>>>/etc/init.d
>>>/etc/init.d/xe-linux-distribution
>>>root@r-8691-VM:~# ls xe-guest-utilities_6.0.2-766_i386.deb
>>>
>>>Thanks,
>>>
>>>On Tue, Sep 17, 2013 at 1:11 PM, Abhinandan Prateek
>>> wrote:
 Daan,
   Do you know what all packages we need to add to get the tools
working. I
 see that current scripts are installing xenstore-utils.


 On 17/09/13 1:58 pm, "Daan Hoogland"  wrote:

>H,
>
>In the present template for systemvms on xen the xenserver tools are
>not installed due to distribution rights issues. Now that xenserver is
>open-sourced, can anyone say if they can be added again?
>
>regards,
>Daan

>>

4.2 - Critical DB upgrade bug for VPC

[Alena Prokharchyk]

Kishan, I was testing VPC on upgraded 4.2 system (the upgrade is done from 4.1 
build), and found Critical DB upgrade bug caused by your commit 836ce6c1. This 
is MUST fix for 4.2 as a result of it, all existing VPCs will stop functioning 
after restart/router destroy.

https://issues.apache.org/jira/browse/CLOUDSTACK-4704

Can you please take a look.

-Alena.

Re: Review Request 14148: Cleanup DefaultUserAuthenticator and removed masking _name variable

[Abhinandan Prateek]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14148/#review26261
---

Ship it!


Ship It!

- Abhinandan Prateek


On Sept. 18, 2013, 5:39 a.m., Darren Shepherd wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14148/
> ---
> 
> (Updated Sept. 18, 2013, 5:39 a.m.)
> 
> 
> Review request for cloudstack, Abhinandan Prateek and Kelven Yang.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> DefaultUserAuthenticator maskes the _name varible in ComponentLifecycleBase 
> making the setName() method not work as expected.  This patch cleans up the 
> code such that getName() will be getClass().getSimpleName() unless overridden 
> in the Spring configuration.
> 
> 
> Diffs
> -
> 
>   
> plugins/user-authenticators/ldap/src/org/apache/cloudstack/ldap/LdapAuthenticator.java
>  e62a3d8 
>   
> plugins/user-authenticators/md5/src/com/cloud/server/auth/MD5UserAuthenticator.java
>  e5b169f 
>   
> plugins/user-authenticators/plain-text/src/com/cloud/server/auth/PlainTextUserAuthenticator.java
>  f102275 
>   
> plugins/user-authenticators/sha256salted/src/com/cloud/server/auth/SHA256SaltedUserAuthenticator.java
>  91be922 
>   server/src/com/cloud/server/auth/DefaultUserAuthenticator.java 952f724 
> 
> Diff: https://reviews.apache.org/r/14148/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Darren Shepherd
> 
>

Re: Review Request 14148: Cleanup DefaultUserAuthenticator and removed masking _name variable

[Abhinandan Prateek]

> On Sept. 19, 2013, 4:20 a.m., Abhinandan Prateek wrote:
> > Ship It!

when i try to apply the patch git am says "Patch format detection failed.". 


- Abhinandan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14148/#review26261
---


On Sept. 18, 2013, 5:39 a.m., Darren Shepherd wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14148/
> ---
> 
> (Updated Sept. 18, 2013, 5:39 a.m.)
> 
> 
> Review request for cloudstack, Abhinandan Prateek and Kelven Yang.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> DefaultUserAuthenticator maskes the _name varible in ComponentLifecycleBase 
> making the setName() method not work as expected.  This patch cleans up the 
> code such that getName() will be getClass().getSimpleName() unless overridden 
> in the Spring configuration.
> 
> 
> Diffs
> -
> 
>   
> plugins/user-authenticators/ldap/src/org/apache/cloudstack/ldap/LdapAuthenticator.java
>  e62a3d8 
>   
> plugins/user-authenticators/md5/src/com/cloud/server/auth/MD5UserAuthenticator.java
>  e5b169f 
>   
> plugins/user-authenticators/plain-text/src/com/cloud/server/auth/PlainTextUserAuthenticator.java
>  f102275 
>   
> plugins/user-authenticators/sha256salted/src/com/cloud/server/auth/SHA256SaltedUserAuthenticator.java
>  91be922 
>   server/src/com/cloud/server/auth/DefaultUserAuthenticator.java 952f724 
> 
> Diff: https://reviews.apache.org/r/14148/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Darren Shepherd
> 
>

Re: [PROPOSAL] Storage Subsystem API Interface Additions

[Mike Tutkowski]

We might want to bring John Burwell into this discussion as he has
documented ideas on how storage drivers might be able to advertise to the
outside CloudStack world their capabilities.

For example, my SolidFire driver could advertise that the volumes it
creates support min, max, and burst IOPS. This info could be leveraged by
input dialogs in the GUI to display certain applicable fields.

Right now my feeling is that storage tagging is a bit of a mess.

When you encounter dialogs like Add Disk Offering, the user can enter one
or more storage tags.

That's great, but it doesn't let the dialog know (easily) what features
primary storage that is tagged with those storage tags support.

That being the case, the dialog ends up displaying all sorts of options
that users can fill in that may or may not be supported by the storage that
CloudStack ends up placing the newly created volume on. It's really up to
the admin who is creating the Disk Offering (for example) to make sure if
he fills in options that they can be supported by the storage that is
tagged.

OpenStack has a completely different paradigm here.

It has the concept of a Volume Type. A Volume Type is backed by one and
only one storage driver. You can fill in what they call Extra Specs to pass
in driver-specific info (key/value pairs, ex: min_iops=300).

If we had something like this, these dialogs would know what data is
applicable to collect.


On Wed, Sep 18, 2013 at 6:29 PM, SuichII, Christopher <
chris.su...@netapp.com> wrote:

> That's a good point and I'm not sure. Maybe we can have the drivers
> indicate whether they support batched/multiple volume snapshotting. If not,
> then we can spin up a thread per volume to snapshot. I completely agree
> that simply calling them in sequence could end up badly.
>
> You're right though, the examples I provided are only partial examples as
> they really do just edit something in the db.
>
> Edison, Alex or anyone else knowledgable of the storage system - Do you
> have any input. Was my clarification helpful?
>
> For what it's worth, after some further investigation, it sounds like we
> won't actually be performing NetApp volume level snapshots to backup vm
> volumes - either when requested or as a batch. We believe we have come up
> with a solution that lets us create individual backups while avoiding our
> 255 snapshot limit. However, it would still be beneficial to let users
> request backups of multiple volumes at once for the other reasons I
> explained earlier.
>
> -Chris
> --
> Chris Suich
> chris.su...@netapp.com
> NetApp Software Engineer
> Data Center Platforms – Cloud Solutions
> Citrix, Cisco & Red Hat
>
> On Sep 18, 2013, at 6:56 PM, Darren Shepherd 
> wrote:
>
> > Nah, I guess its not so bad to have a volumeIds param.  Just as long as
> its
> > really clear that means nothing about consistency.
> >
> > I would be a little concerned about how this will be implemented by other
> > storage providers though.  Currently if you do 5 snapshot API calls that
> > will launch 5 threads and they will happen in parallel.  If they get
> > batched and sent in one thread, how is the framework/driver going to
> handle
> > the snapshots for drivers that don't supporting batching?  Sequential
> would
> > be bad as sometimes it takes awhile to snapshot.
> >
> > The APIs you mentioned that takes lists really just manipulate data in
> the
> > DB, so they can easily batch and transactionally do a bunch at once.
> >
> > Maybe someone who's more familiar with the storage implementation can
> > comment?
> >
> > Darren
> >
> >
> > On Wed, Sep 18, 2013 at 12:46 PM, SuichII, Christopher <
> > chris.su...@netapp.com> wrote:
> >
> >> That certainly would work, but I don't see how it is a better design.
> Can
> >> you elaborate on how sending multiple volumeIds is hackish? Look at the
> >> existing API framework. We have several APIs that accept lists as
> >> parameters. Normally, they're used for things like querying or deleting.
> >> Take a look at some of these commands:
> >> -ArchiveEventsCmd
> >> -DeleteEventsCmd
> >> -DeleteSnapshotPoliciesCmd
> >>
> >> This kind of API is simply a shorthand for invoking another API many
> times.
> >>
> >> I think it is only an NetApp optimization in the sense that we're the
> only
> >> ones who need it right now. What we're asking for has nothing specific
> to
> >> do to NetApp. We would just like the shorthand ability to do things all
> at
> >> once rather than one at a time. I think other vendors could utilize this
> >> just as easily.
> >>
> >> -Chris
> >> --
> >> Chris Suich
> >> chris.su...@netapp.com
> >> NetApp Software Engineer
> >> Data Center Platforms – Cloud Solutions
> >> Citrix, Cisco & Red Hat
> >>
> >> On Sep 18, 2013, at 2:32 PM, Darren Shepherd <
> darren.s.sheph...@gmail.com>
> >> wrote:
> >>
> >>> Given this explanation.  Would the following not work?
> >>>
> >>> 1) Enhance UI to allow multi select.  There is no API change, UI will
> >> just
> >>> call snapshot a