Re: Hypervisor Questions

[Mike Tutkowski]

Right, Marcus, what you say makes sense.

You brought up a point I was going to ask about: If you can migrate between
clusters, what's the point of a cluster?

That point aside, I was really more interested (for the purposes of this
e-mail thread) to find out how this works for KVM today (not to recommend
adding a new feature if cross-cluster support is not currently in
CloudStack for KVM).

By your response, can I assume you can "only" migrate a VM from one KVM
host to another in the same cluster?

Thanks


On Fri, Oct 4, 2013 at 9:48 PM, Marcus Sorensen  wrote:

> From a kvm standpoint, a cluster has admin-defined meaning. Its not always
> going to map to some externally defined cluster.  I can make my whole data
> center a cluster, and not use zone-wide storage, or still use zone wide
> storage. If you have zone-wide storage, maybe you want to keep VMs within
> small defined clusters for some non-storage reason. It makes sense to me to
> keep the cluster as an entity that confines where a vm can run, regardless
> of zone wide storage, otherwise is there a purpose to it at all? Maybe we
> can offer a migrate vm between cluster function if zone-wide is in use?
> Also keep in mind that there's a pod level. You'd still need to be in the
> same pod if you wanted to jump between clusters.
> On Oct 4, 2013 5:05 PM, "Mike Tutkowski" 
> wrote:
>
>> Perhaps Marcus can answer this from a KVM standpoint?
>>
>>
>> On Fri, Oct 4, 2013 at 5:03 PM, Mike Tutkowski <
>> mike.tutkow...@solidfire.com> wrote:
>>
>>> This implies live migration between VMware clusters is supported in
>>> CloudStack:
>>>
>>> https://issues.apache.org/jira/browse/CLOUDSTACK-4265
>>>
>>>
>>> On Fri, Oct 4, 2013 at 4:56 PM, Mike Tutkowski <
>>> mike.tutkow...@solidfire.com> wrote:
>>>
 For anyone who is interested in the outcome of this thread, this seems
 to answer the XenServer part:


 https://cwiki.apache.org/confluence/display/CLOUDSTACK/Enabling+Storage+XenMotion+for+XenServer

 "CloudStack currently allows live migration of a virtual machine from
 one host to another only within a cluster."

 Document was last updated June 28, 2013.


 On Fri, Oct 4, 2013 at 12:41 PM, Mike Tutkowski <
 mike.tutkow...@solidfire.com> wrote:

> Yeah, that's kind of what I was interested in learning about.
>
> Now that we have zone-wide primary storage, does that mean CS is able
> to issue the live migration of a VM from one cluster to another (or are we
> still confined to clusters)?
>
>
> On Fri, Oct 4, 2013 at 11:55 AM, Travis Graham wrote:
>
>> Was that a limitation caused by the primary storage only being
>> available to a single cluster and not zone wide like 4.2.0 provides?
>>
>> Travis
>>
>> On Oct 4, 2013, at 1:52 PM, Mike Tutkowski <
>> mike.tutkow...@solidfire.com> wrote:
>>
>> > Maybe this is a silly question, but if CS handles Live Migrations,
>> are we
>> > still constrained to migrating VMs from one host to another in the
>> same
>> > cluster?
>> >
>> > Same question for HA.
>> >
>> >
>> > On Wed, Oct 2, 2013 at 8:42 AM, Clayton Weise 
>> wrote:
>> >
>> >> AFAIK, no, but it's a great RFE that I would vote for.
>> >>
>> >> -Original Message-
>> >> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
>> >> Sent: Tuesday, October 01, 2013 9:26 PM
>> >> To: dev@cloudstack.apache.org
>> >> Subject: Re: Hypervisor Questions
>> >>
>> >> Oh, and, yes, when I referred to HA, it was (as you said) with the
>> meaning
>> >> of a host going offline and VMs being restarted on other hosts
>> (perhaps in
>> >> a prioritized order if there are an insufficient number of
>> resources to
>> >> support all of the VMs that were running on the downed host).
>> >>
>> >> Does CS support assigning a priority to a VM in case not all VMs
>> can be
>> >> restarted on the remaining resources?
>> >>
>> >>
>> >> On Tue, Oct 1, 2013 at 6:35 PM, Mike Tutkowski <
>> >> mike.tutkow...@solidfire.com
>> >>> wrote:
>> >>
>> >>> Thanks, Clayton!
>> >>>
>> >>> Yeah, copy/paste mistake there. :) I meant it as you said.
>> >>>
>> >>>
>> >>> On Tue, Oct 1, 2013 at 4:54 PM, Clayton Weise > >
>> >> wrote:
>> >>>
>>  First, I think you meant to put XenServer, KVM, and VMware and
>> not
>>  XenServer 3 times in a row.  That being said I think in all cases
>>  (somebody correct me if I'm wrong here) it goes something like
>> this:
>> 
>>  Live Migration: Request is made by CS but carried out by the HV.
>>  High Availability: More accurately it's "recovery after host
>> failure"
>>  because it's still a disruptive action when a host goes
>> sideways, but
>>  by default this is handled by CS.  I 

Re: [DISCUSS] Pluggable VM snapshot related operations?

[Mike Tutkowski]

Well, my first thought on this is that the storage driver should not be
telling the hypervisor to do anything. It should be responsible for
creating/deleting volumes, snapshots, etc. on its storage system only.


On Fri, Oct 4, 2013 at 5:57 PM, Edison Su  wrote:

> In 4.2, we added VM snapshot for Vmware/Xenserver. The current workflow
> will be like the following:
> createVMSnapshot api -> VMSnapshotManagerImpl: creatVMSnapshot -> send
> CreateVMSnapshotCommand to hypervisor to create vm snapshot.
>
> If anybody wants to change the workflow, then need to either change
> VMSnapshotManagerImpl directly or subclass VMSnapshotManagerImpl. Both are
> not the ideal choice, as VMSnapshotManagerImpl should be able to handle
> different ways to take vm snapshot, instead of hard code.
>
> The requirements for the pluggable VM snapshot coming from:
> Storage vendor may have their optimization, such as NetApp.
> VM snapshot can be implemented in a totally different way(For example, I
> could just send a command to guest VM, to tell my application to flush disk
> and hold disk write, then come to hypervisor to take a volume snapshot).
>
> If we agree on enable pluggable VM snapshot, then we can move on discuss
> how to implement it.
>
> The possible options:
> 1. coarse grained interface. Add a VMSnapshotStrategy interface, which has
> the following interfaces:
> VMSnapshot takeVMSnapshot(VMSnapshot vmSnapshot);
> Boolean revertVMSnapshot(VMSnapshot vmSnapshot);
> Boolean DeleteVMSnapshot(VMSnapshot vmSnapshot);
>
>The work flow will be: createVMSnapshot api -> VMSnapshotManagerImpl:
> creatVMSnapshot -> VMSnapshotStrategy: takeVMSnapshot
>VMSnapshotManagerImpl will manage VM state, do the sanity check, then
> will handle over to VMSnapshotStrategy.
>In VMSnapshotStrategy implementation, it may just send a
> Create/revert/delete VMSnapshotCommand to hypervisor host, or do anything
> special operations.
>
> 2. fine-grained interface. Not only add a VMSnapshotStrategy interface,
> but also add certain methods on the storage driver.
> The VMSnapshotStrategy interface will be the same as option 1.
> Will add the following methods on storage driver:
>/* volumesBelongToVM  is the list of volumes of the VM that created on
> this storage, storage vendor can either take one snapshot for this volumes
> in one shot, or take snapshot for each volume separately
>The pre-condition: vm is unquiesced.
>It will return a Boolean to indicate, do need unquiesce vm or not.
>In the default storage driver, it will return false.
> */
> boolean takeVMSnapshot(List volumesBelongToVM, VMSnapshot
> vmSnapshot);
> Boolean revertVMSnapshot(List volumesBelongToVM,
> VMSnapshot vmSnapshot);
>Boolean deleteVMSnapshot(List volumesBelongToVM, VMSnapshot
> vmSNapshot);
>
> The work flow will be: createVMSnapshot api -> VMSnapshotManagerImpl:
> creatVMSnapshot -> VMSnapshotStrategy: takeVMSnapshot -> storage
> driver:takeVMSnapshot
>  In the implementation of VMSnapshotStrategy's takeVMSnapshot, the pseudo
> code looks like:
>HypervisorHelper.quiesceVM(vm);
>val volumes = vm.getVolumes();
>val maps = new Map[driver, list[VolumeInfo]]();
>Volumes.foreach(volume => maps.put(volume.getDriver, volume ::
> maps.get(volume.getdriver(
>val needUnquiesce = true;
> maps.foreach((driver, volumes) => needUnquiesce  = needUnquiesce
>  && driver.takeVMSnapshot(volumes))
>   if (needUnquiesce ) {
>HypervisorHelper.unquiesce(vm);
> }
>
> By default, the quiesceVM in HypervisorHelper will actually take vm
> snapshot through hypervisor.
> Does above logic makes senesce?
>
> The pros of option 1 is that: it's simple, no need to change storage
> driver interfaces. The cons is that each storage vendor need to implement a
> strategy, maybe they will do the same thing.
> The pros of option 2 is that, storage driver won't need to worry about how
> to quiesce/unquiesce vm. The cons is that, it will add these methods on
> each storage drivers, so it assumes that this work flow will work for
> everybody.
>
> So which option we should take? Or if you have other options, please let's
> know.
>
>
>
>
>
>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

RE: System VM

[Radhika Puthiyetath]

Hi All,

Before setting up the cloudstack repo, we argued that doc should be in a 
different repo. At that time, nobody listened to our argument of doc having a 
different life cycle.

I am wondering why this back and forth decisions changes now for docs.

-Original Message-
From: Chip Childers [mailto:chip.child...@sungard.com] 
Sent: Friday, October 04, 2013 11:25 PM
To: dev@cloudstack.apache.org
Subject: Re: System VM

On Fri, Oct 04, 2013 at 05:43:16PM +, Jessica Tomechak wrote:
> Hi guys,
> Not arguing against it, but I would be very much interested in your reasoning 
> behind why having docs in a separate repo makes them easier to work on. What 
> have we experienced since this time last year which has led us to reverse the 
> original decision to keep docs in the same repo with code?
> 
> And having mentioned this, also thanks to y'all for taking care of doing the 
> actual split and setting up the new repo.
> 
> Jessica T.

Documentation has a different lifecycle from the code, since docs aren't 
usually complete anywhere near feature complete.

Also, having it in a different repo will help contributors more easily work 
with the documentation.  We are seeing a number of new folks in the community 
that want to help on that front.  

-chip

Re: Hypervisor Questions

[Marcus Sorensen]

>From a kvm standpoint, a cluster has admin-defined meaning. Its not always
going to map to some externally defined cluster.  I can make my whole data
center a cluster, and not use zone-wide storage, or still use zone wide
storage. If you have zone-wide storage, maybe you want to keep VMs within
small defined clusters for some non-storage reason. It makes sense to me to
keep the cluster as an entity that confines where a vm can run, regardless
of zone wide storage, otherwise is there a purpose to it at all? Maybe we
can offer a migrate vm between cluster function if zone-wide is in use?
Also keep in mind that there's a pod level. You'd still need to be in the
same pod if you wanted to jump between clusters.
On Oct 4, 2013 5:05 PM, "Mike Tutkowski" 
wrote:

> Perhaps Marcus can answer this from a KVM standpoint?
>
>
> On Fri, Oct 4, 2013 at 5:03 PM, Mike Tutkowski <
> mike.tutkow...@solidfire.com> wrote:
>
>> This implies live migration between VMware clusters is supported in
>> CloudStack:
>>
>> https://issues.apache.org/jira/browse/CLOUDSTACK-4265
>>
>>
>> On Fri, Oct 4, 2013 at 4:56 PM, Mike Tutkowski <
>> mike.tutkow...@solidfire.com> wrote:
>>
>>> For anyone who is interested in the outcome of this thread, this seems
>>> to answer the XenServer part:
>>>
>>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Enabling+Storage+XenMotion+for+XenServer
>>>
>>> "CloudStack currently allows live migration of a virtual machine from
>>> one host to another only within a cluster."
>>>
>>> Document was last updated June 28, 2013.
>>>
>>>
>>> On Fri, Oct 4, 2013 at 12:41 PM, Mike Tutkowski <
>>> mike.tutkow...@solidfire.com> wrote:
>>>
 Yeah, that's kind of what I was interested in learning about.

 Now that we have zone-wide primary storage, does that mean CS is able
 to issue the live migration of a VM from one cluster to another (or are we
 still confined to clusters)?


 On Fri, Oct 4, 2013 at 11:55 AM, Travis Graham wrote:

> Was that a limitation caused by the primary storage only being
> available to a single cluster and not zone wide like 4.2.0 provides?
>
> Travis
>
> On Oct 4, 2013, at 1:52 PM, Mike Tutkowski <
> mike.tutkow...@solidfire.com> wrote:
>
> > Maybe this is a silly question, but if CS handles Live Migrations,
> are we
> > still constrained to migrating VMs from one host to another in the
> same
> > cluster?
> >
> > Same question for HA.
> >
> >
> > On Wed, Oct 2, 2013 at 8:42 AM, Clayton Weise 
> wrote:
> >
> >> AFAIK, no, but it's a great RFE that I would vote for.
> >>
> >> -Original Message-
> >> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
> >> Sent: Tuesday, October 01, 2013 9:26 PM
> >> To: dev@cloudstack.apache.org
> >> Subject: Re: Hypervisor Questions
> >>
> >> Oh, and, yes, when I referred to HA, it was (as you said) with the
> meaning
> >> of a host going offline and VMs being restarted on other hosts
> (perhaps in
> >> a prioritized order if there are an insufficient number of
> resources to
> >> support all of the VMs that were running on the downed host).
> >>
> >> Does CS support assigning a priority to a VM in case not all VMs
> can be
> >> restarted on the remaining resources?
> >>
> >>
> >> On Tue, Oct 1, 2013 at 6:35 PM, Mike Tutkowski <
> >> mike.tutkow...@solidfire.com
> >>> wrote:
> >>
> >>> Thanks, Clayton!
> >>>
> >>> Yeah, copy/paste mistake there. :) I meant it as you said.
> >>>
> >>>
> >>> On Tue, Oct 1, 2013 at 4:54 PM, Clayton Weise 
> >> wrote:
> >>>
>  First, I think you meant to put XenServer, KVM, and VMware and not
>  XenServer 3 times in a row.  That being said I think in all cases
>  (somebody correct me if I'm wrong here) it goes something like
> this:
> 
>  Live Migration: Request is made by CS but carried out by the HV.
>  High Availability: More accurately it's "recovery after host
> failure"
>  because it's still a disruptive action when a host goes sideways,
> but
>  by default this is handled by CS.  I _think_ there's an option to
> let
>  the HV handle this but I'm not totally sure.
>  DRS: Managed by CS through one of several methods with the global
>  setting vm.allocation.algorithm (see below)
> 
>  'random', 'firstfit', 'userdispersing',
> 'userconcentratedpod_random',
>  'userconcentratedpod_firstfit' : Order in which hosts within a
>  cluster will be considered for VM/volume allocation.
> 
>  That being said, after deployment there isn't any further DRS
>  monitoring; it's only done at the time an instance is
> instantiated.
> 
>  -Clayton
> 
>  -Original Message-

Re: [ACS4.2.1] UI tickets

[Abhinandan Prateek]

Done ! https://issues.apache.org/jira/browse/INFRA-6840

-abhi

On 04/10/13 10:37 pm, "Animesh Chaturvedi" 
wrote:

>Abhi
>
>It is best to set up a shared filter. You will need to open a INFRA
>ticket like this one https://issues.apache.org/jira/i#browse/INFRA-6145
>to get relevant permissions for the JIRA
>
>
>Thanks
>Animesh
>
>
>> -Original Message-
>> From: Abhinandan Prateek [mailto:abhinandan.prat...@citrix.com]
>> Sent: Friday, October 04, 2013 6:14 AM
>> To: dev@cloudstack.apache.org
>> Cc: Brian Federle; Jessica Wang; Sonny Chhen
>> Subject: Re: [ACS4.2.1] UI tickets
>> 
>> Yes, specially the one in CC.
>> 
>> On 04/10/13 6:16 pm, "Sudha Ponnaganti" 
>> wrote:
>> 
>> >Do you mean UI guys??
>> >
>> >-Original Message-
>> >From: Abhinandan Prateek [mailto:abhinandan.prat...@citrix.com]
>> >Sent: Friday, October 04, 2013 4:54 AM
>> >To: CloudStack Dev
>> >Cc: Brian Federle; Jessica Wang; Sonny Chhen
>> >Subject: [ACS4.2.1] UI tickets
>> >
>> >
>> >There are around ~72 UI tickets open. Most of them are major, can
>> >documentation guys go thru the tickets and start picking them.
>> >
>> >Following Jira query will give the tickets that are open.
>> >
>> >project = CLOUDSTACK AND resolution = Unresolved AND fixVersion in
>> >("4.2.0", "4.2.1") AND text ~ "UI"
>> >
>> >-abhi
>

[DISCUSS] Pluggable VM snapshot related operations?

[Edison Su]

In 4.2, we added VM snapshot for Vmware/Xenserver. The current workflow will be 
like the following:
createVMSnapshot api -> VMSnapshotManagerImpl: creatVMSnapshot -> send 
CreateVMSnapshotCommand to hypervisor to create vm snapshot.

If anybody wants to change the workflow, then need to either change 
VMSnapshotManagerImpl directly or subclass VMSnapshotManagerImpl. Both are not 
the ideal choice, as VMSnapshotManagerImpl should be able to handle different 
ways to take vm snapshot, instead of hard code.

The requirements for the pluggable VM snapshot coming from:
Storage vendor may have their optimization, such as NetApp.
VM snapshot can be implemented in a totally different way(For example, I could 
just send a command to guest VM, to tell my application to flush disk and hold 
disk write, then come to hypervisor to take a volume snapshot). 

If we agree on enable pluggable VM snapshot, then we can move on discuss how to 
implement it.

The possible options:
1. coarse grained interface. Add a VMSnapshotStrategy interface, which has the 
following interfaces:
VMSnapshot takeVMSnapshot(VMSnapshot vmSnapshot);
Boolean revertVMSnapshot(VMSnapshot vmSnapshot);
Boolean DeleteVMSnapshot(VMSnapshot vmSnapshot);

   The work flow will be: createVMSnapshot api -> VMSnapshotManagerImpl: 
creatVMSnapshot -> VMSnapshotStrategy: takeVMSnapshot
   VMSnapshotManagerImpl will manage VM state, do the sanity check, then will 
handle over to VMSnapshotStrategy. 
   In VMSnapshotStrategy implementation, it may just send a 
Create/revert/delete VMSnapshotCommand to hypervisor host, or do anything 
special operations.

2. fine-grained interface. Not only add a VMSnapshotStrategy interface, but 
also add certain methods on the storage driver.
The VMSnapshotStrategy interface will be the same as option 1.
Will add the following methods on storage driver:
   /* volumesBelongToVM  is the list of volumes of the VM that created on this 
storage, storage vendor can either take one snapshot for this volumes in one 
shot, or take snapshot for each volume separately
   The pre-condition: vm is unquiesced. 
   It will return a Boolean to indicate, do need unquiesce vm or not.
   In the default storage driver, it will return false.
*/
boolean takeVMSnapshot(List volumesBelongToVM, VMSnapshot 
vmSnapshot);  
Boolean revertVMSnapshot(List volumesBelongToVM, VMSnapshot 
vmSnapshot);
   Boolean deleteVMSnapshot(List volumesBelongToVM, VMSnapshot 
vmSNapshot);

The work flow will be: createVMSnapshot api -> VMSnapshotManagerImpl: 
creatVMSnapshot -> VMSnapshotStrategy: takeVMSnapshot -> storage 
driver:takeVMSnapshot
 In the implementation of VMSnapshotStrategy's takeVMSnapshot, the pseudo code 
looks like:
   HypervisorHelper.quiesceVM(vm);
   val volumes = vm.getVolumes();
   val maps = new Map[driver, list[VolumeInfo]]();
   Volumes.foreach(volume => maps.put(volume.getDriver, volume :: 
maps.get(volume.getdriver(
   val needUnquiesce = true;
maps.foreach((driver, volumes) => needUnquiesce  = needUnquiesce  && 
driver.takeVMSnapshot(volumes))
  if (needUnquiesce ) {
   HypervisorHelper.unquiesce(vm);
} 

By default, the quiesceVM in HypervisorHelper will actually take vm snapshot 
through hypervisor. 
Does above logic makes senesce?

The pros of option 1 is that: it's simple, no need to change storage driver 
interfaces. The cons is that each storage vendor need to implement a strategy, 
maybe they will do the same thing.
The pros of option 2 is that, storage driver won't need to worry about how to 
quiesce/unquiesce vm. The cons is that, it will add these methods on each 
storage drivers, so it assumes that this work flow will work for everybody.

So which option we should take? Or if you have other options, please let's 
know. 



   

Re: Hypervisor Questions

[Mike Tutkowski]

Perhaps Marcus can answer this from a KVM standpoint?


On Fri, Oct 4, 2013 at 5:03 PM, Mike Tutkowski  wrote:

> This implies live migration between VMware clusters is supported in
> CloudStack:
>
> https://issues.apache.org/jira/browse/CLOUDSTACK-4265
>
>
> On Fri, Oct 4, 2013 at 4:56 PM, Mike Tutkowski <
> mike.tutkow...@solidfire.com> wrote:
>
>> For anyone who is interested in the outcome of this thread, this seems to
>> answer the XenServer part:
>>
>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Enabling+Storage+XenMotion+for+XenServer
>>
>> "CloudStack currently allows live migration of a virtual machine from
>> one host to another only within a cluster."
>>
>> Document was last updated June 28, 2013.
>>
>>
>> On Fri, Oct 4, 2013 at 12:41 PM, Mike Tutkowski <
>> mike.tutkow...@solidfire.com> wrote:
>>
>>> Yeah, that's kind of what I was interested in learning about.
>>>
>>> Now that we have zone-wide primary storage, does that mean CS is able to
>>> issue the live migration of a VM from one cluster to another (or are we
>>> still confined to clusters)?
>>>
>>>
>>> On Fri, Oct 4, 2013 at 11:55 AM, Travis Graham wrote:
>>>
 Was that a limitation caused by the primary storage only being
 available to a single cluster and not zone wide like 4.2.0 provides?

 Travis

 On Oct 4, 2013, at 1:52 PM, Mike Tutkowski <
 mike.tutkow...@solidfire.com> wrote:

 > Maybe this is a silly question, but if CS handles Live Migrations,
 are we
 > still constrained to migrating VMs from one host to another in the
 same
 > cluster?
 >
 > Same question for HA.
 >
 >
 > On Wed, Oct 2, 2013 at 8:42 AM, Clayton Weise 
 wrote:
 >
 >> AFAIK, no, but it's a great RFE that I would vote for.
 >>
 >> -Original Message-
 >> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
 >> Sent: Tuesday, October 01, 2013 9:26 PM
 >> To: dev@cloudstack.apache.org
 >> Subject: Re: Hypervisor Questions
 >>
 >> Oh, and, yes, when I referred to HA, it was (as you said) with the
 meaning
 >> of a host going offline and VMs being restarted on other hosts
 (perhaps in
 >> a prioritized order if there are an insufficient number of resources
 to
 >> support all of the VMs that were running on the downed host).
 >>
 >> Does CS support assigning a priority to a VM in case not all VMs can
 be
 >> restarted on the remaining resources?
 >>
 >>
 >> On Tue, Oct 1, 2013 at 6:35 PM, Mike Tutkowski <
 >> mike.tutkow...@solidfire.com
 >>> wrote:
 >>
 >>> Thanks, Clayton!
 >>>
 >>> Yeah, copy/paste mistake there. :) I meant it as you said.
 >>>
 >>>
 >>> On Tue, Oct 1, 2013 at 4:54 PM, Clayton Weise 
 >> wrote:
 >>>
  First, I think you meant to put XenServer, KVM, and VMware and not
  XenServer 3 times in a row.  That being said I think in all cases
  (somebody correct me if I'm wrong here) it goes something like
 this:
 
  Live Migration: Request is made by CS but carried out by the HV.
  High Availability: More accurately it's "recovery after host
 failure"
  because it's still a disruptive action when a host goes sideways,
 but
  by default this is handled by CS.  I _think_ there's an option to
 let
  the HV handle this but I'm not totally sure.
  DRS: Managed by CS through one of several methods with the global
  setting vm.allocation.algorithm (see below)
 
  'random', 'firstfit', 'userdispersing',
 'userconcentratedpod_random',
  'userconcentratedpod_firstfit' : Order in which hosts within a
  cluster will be considered for VM/volume allocation.
 
  That being said, after deployment there isn't any further DRS
  monitoring; it's only done at the time an instance is instantiated.
 
  -Clayton
 
  -Original Message-
  From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
  Sent: Tuesday, October 01, 2013 3:00 PM
  To: dev@cloudstack.apache.org
  Subject: Hypervisor Questions
 
  Hi,
 
  I was wondering if people could clarify for me what CloudStack
  manages versus what the hypervisor manages in terms of live
  migration, high availability, and distributed resource scheduling?
 
  I know it is probably different for XenServer, VMware, and KVM.
 
  Can people fill in the info below (managed by the management
 server,
  the hypervisor, or some combination of both)?
 
  XenServer
    Live migration:
    High availability:
    Distributed Resource Scheduling:
 
  XenServer
    Live migration:
    High availability:
    Dist

Re: Hypervisor Questions

[Mike Tutkowski]

This implies live migration between VMware clusters is supported in
CloudStack:

https://issues.apache.org/jira/browse/CLOUDSTACK-4265


On Fri, Oct 4, 2013 at 4:56 PM, Mike Tutkowski  wrote:

> For anyone who is interested in the outcome of this thread, this seems to
> answer the XenServer part:
>
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Enabling+Storage+XenMotion+for+XenServer
>
> "CloudStack currently allows live migration of a virtual machine from one
> host to another only within a cluster."
>
> Document was last updated June 28, 2013.
>
>
> On Fri, Oct 4, 2013 at 12:41 PM, Mike Tutkowski <
> mike.tutkow...@solidfire.com> wrote:
>
>> Yeah, that's kind of what I was interested in learning about.
>>
>> Now that we have zone-wide primary storage, does that mean CS is able to
>> issue the live migration of a VM from one cluster to another (or are we
>> still confined to clusters)?
>>
>>
>> On Fri, Oct 4, 2013 at 11:55 AM, Travis Graham wrote:
>>
>>> Was that a limitation caused by the primary storage only being available
>>> to a single cluster and not zone wide like 4.2.0 provides?
>>>
>>> Travis
>>>
>>> On Oct 4, 2013, at 1:52 PM, Mike Tutkowski 
>>> wrote:
>>>
>>> > Maybe this is a silly question, but if CS handles Live Migrations, are
>>> we
>>> > still constrained to migrating VMs from one host to another in the same
>>> > cluster?
>>> >
>>> > Same question for HA.
>>> >
>>> >
>>> > On Wed, Oct 2, 2013 at 8:42 AM, Clayton Weise 
>>> wrote:
>>> >
>>> >> AFAIK, no, but it's a great RFE that I would vote for.
>>> >>
>>> >> -Original Message-
>>> >> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
>>> >> Sent: Tuesday, October 01, 2013 9:26 PM
>>> >> To: dev@cloudstack.apache.org
>>> >> Subject: Re: Hypervisor Questions
>>> >>
>>> >> Oh, and, yes, when I referred to HA, it was (as you said) with the
>>> meaning
>>> >> of a host going offline and VMs being restarted on other hosts
>>> (perhaps in
>>> >> a prioritized order if there are an insufficient number of resources
>>> to
>>> >> support all of the VMs that were running on the downed host).
>>> >>
>>> >> Does CS support assigning a priority to a VM in case not all VMs can
>>> be
>>> >> restarted on the remaining resources?
>>> >>
>>> >>
>>> >> On Tue, Oct 1, 2013 at 6:35 PM, Mike Tutkowski <
>>> >> mike.tutkow...@solidfire.com
>>> >>> wrote:
>>> >>
>>> >>> Thanks, Clayton!
>>> >>>
>>> >>> Yeah, copy/paste mistake there. :) I meant it as you said.
>>> >>>
>>> >>>
>>> >>> On Tue, Oct 1, 2013 at 4:54 PM, Clayton Weise 
>>> >> wrote:
>>> >>>
>>>  First, I think you meant to put XenServer, KVM, and VMware and not
>>>  XenServer 3 times in a row.  That being said I think in all cases
>>>  (somebody correct me if I'm wrong here) it goes something like this:
>>> 
>>>  Live Migration: Request is made by CS but carried out by the HV.
>>>  High Availability: More accurately it's "recovery after host
>>> failure"
>>>  because it's still a disruptive action when a host goes sideways,
>>> but
>>>  by default this is handled by CS.  I _think_ there's an option to
>>> let
>>>  the HV handle this but I'm not totally sure.
>>>  DRS: Managed by CS through one of several methods with the global
>>>  setting vm.allocation.algorithm (see below)
>>> 
>>>  'random', 'firstfit', 'userdispersing',
>>> 'userconcentratedpod_random',
>>>  'userconcentratedpod_firstfit' : Order in which hosts within a
>>>  cluster will be considered for VM/volume allocation.
>>> 
>>>  That being said, after deployment there isn't any further DRS
>>>  monitoring; it's only done at the time an instance is instantiated.
>>> 
>>>  -Clayton
>>> 
>>>  -Original Message-
>>>  From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
>>>  Sent: Tuesday, October 01, 2013 3:00 PM
>>>  To: dev@cloudstack.apache.org
>>>  Subject: Hypervisor Questions
>>> 
>>>  Hi,
>>> 
>>>  I was wondering if people could clarify for me what CloudStack
>>>  manages versus what the hypervisor manages in terms of live
>>>  migration, high availability, and distributed resource scheduling?
>>> 
>>>  I know it is probably different for XenServer, VMware, and KVM.
>>> 
>>>  Can people fill in the info below (managed by the management server,
>>>  the hypervisor, or some combination of both)?
>>> 
>>>  XenServer
>>>    Live migration:
>>>    High availability:
>>>    Distributed Resource Scheduling:
>>> 
>>>  XenServer
>>>    Live migration:
>>>    High availability:
>>>    Distributed Resource Scheduling:
>>> 
>>>  XenServer
>>>    Live migration:
>>>    High availability:
>>>    Distributed Resource Scheduling:
>>> 
>>>  Thanks!
>>> 
>>>  --
>>>  *Mike Tutkowski*
>>>  *Senior CloudStack Developer, SolidFire Inc.*
>>>  e: mike.tutkow...@solidfire.com
>>>

Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]

[Chiradeep Vittal]

I've checked the Netscaler and HAProxy docs, this appears to be artifact
of the HAProxy implementation (inability to support keep-alive).
For a cloud operator that chooses Netscaler or F5 for load balancing, this
won't make any sense.

On 10/3/13 12:55 PM, "Daan Hoogland"  wrote:

>On Thu, Oct 3, 2013 at 9:02 PM, Chip Childers
>wrote:
>
>> a model for extensions like that makes perfect sense.
>
>
>
>This model sound fine indeed. It makes no sense for httpClose however.
>
>Here's my concern:
>So when an early adapter is implemented and the rest of the market comes
>to
>their senses, how do we migrate without running into migration/upgrade
>problems?
>httpClose is a flag controlling connection pooling. I probably choose the
>wrong name. It is something that any implementation will support or should
>have supported already. Am I going to implement it as a key/value now to
>later implemented as I have done anyway? I don't like this idea.
>
>Don't get me wrong the pattern described by you guys is fine in some
>situations. I don't think it is applicable to this feature.
>
>regards,
>Daan

Re: Hypervisor Questions

[Mike Tutkowski]

For anyone who is interested in the outcome of this thread, this seems to
answer the XenServer part:

https://cwiki.apache.org/confluence/display/CLOUDSTACK/Enabling+Storage+XenMotion+for+XenServer

"CloudStack currently allows live migration of a virtual machine from one
host to another only within a cluster."

Document was last updated June 28, 2013.


On Fri, Oct 4, 2013 at 12:41 PM, Mike Tutkowski <
mike.tutkow...@solidfire.com> wrote:

> Yeah, that's kind of what I was interested in learning about.
>
> Now that we have zone-wide primary storage, does that mean CS is able to
> issue the live migration of a VM from one cluster to another (or are we
> still confined to clusters)?
>
>
> On Fri, Oct 4, 2013 at 11:55 AM, Travis Graham  wrote:
>
>> Was that a limitation caused by the primary storage only being available
>> to a single cluster and not zone wide like 4.2.0 provides?
>>
>> Travis
>>
>> On Oct 4, 2013, at 1:52 PM, Mike Tutkowski 
>> wrote:
>>
>> > Maybe this is a silly question, but if CS handles Live Migrations, are
>> we
>> > still constrained to migrating VMs from one host to another in the same
>> > cluster?
>> >
>> > Same question for HA.
>> >
>> >
>> > On Wed, Oct 2, 2013 at 8:42 AM, Clayton Weise 
>> wrote:
>> >
>> >> AFAIK, no, but it's a great RFE that I would vote for.
>> >>
>> >> -Original Message-
>> >> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
>> >> Sent: Tuesday, October 01, 2013 9:26 PM
>> >> To: dev@cloudstack.apache.org
>> >> Subject: Re: Hypervisor Questions
>> >>
>> >> Oh, and, yes, when I referred to HA, it was (as you said) with the
>> meaning
>> >> of a host going offline and VMs being restarted on other hosts
>> (perhaps in
>> >> a prioritized order if there are an insufficient number of resources to
>> >> support all of the VMs that were running on the downed host).
>> >>
>> >> Does CS support assigning a priority to a VM in case not all VMs can be
>> >> restarted on the remaining resources?
>> >>
>> >>
>> >> On Tue, Oct 1, 2013 at 6:35 PM, Mike Tutkowski <
>> >> mike.tutkow...@solidfire.com
>> >>> wrote:
>> >>
>> >>> Thanks, Clayton!
>> >>>
>> >>> Yeah, copy/paste mistake there. :) I meant it as you said.
>> >>>
>> >>>
>> >>> On Tue, Oct 1, 2013 at 4:54 PM, Clayton Weise 
>> >> wrote:
>> >>>
>>  First, I think you meant to put XenServer, KVM, and VMware and not
>>  XenServer 3 times in a row.  That being said I think in all cases
>>  (somebody correct me if I'm wrong here) it goes something like this:
>> 
>>  Live Migration: Request is made by CS but carried out by the HV.
>>  High Availability: More accurately it's "recovery after host failure"
>>  because it's still a disruptive action when a host goes sideways, but
>>  by default this is handled by CS.  I _think_ there's an option to let
>>  the HV handle this but I'm not totally sure.
>>  DRS: Managed by CS through one of several methods with the global
>>  setting vm.allocation.algorithm (see below)
>> 
>>  'random', 'firstfit', 'userdispersing', 'userconcentratedpod_random',
>>  'userconcentratedpod_firstfit' : Order in which hosts within a
>>  cluster will be considered for VM/volume allocation.
>> 
>>  That being said, after deployment there isn't any further DRS
>>  monitoring; it's only done at the time an instance is instantiated.
>> 
>>  -Clayton
>> 
>>  -Original Message-
>>  From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
>>  Sent: Tuesday, October 01, 2013 3:00 PM
>>  To: dev@cloudstack.apache.org
>>  Subject: Hypervisor Questions
>> 
>>  Hi,
>> 
>>  I was wondering if people could clarify for me what CloudStack
>>  manages versus what the hypervisor manages in terms of live
>>  migration, high availability, and distributed resource scheduling?
>> 
>>  I know it is probably different for XenServer, VMware, and KVM.
>> 
>>  Can people fill in the info below (managed by the management server,
>>  the hypervisor, or some combination of both)?
>> 
>>  XenServer
>>    Live migration:
>>    High availability:
>>    Distributed Resource Scheduling:
>> 
>>  XenServer
>>    Live migration:
>>    High availability:
>>    Distributed Resource Scheduling:
>> 
>>  XenServer
>>    Live migration:
>>    High availability:
>>    Distributed Resource Scheduling:
>> 
>>  Thanks!
>> 
>>  --
>>  *Mike Tutkowski*
>>  *Senior CloudStack Developer, SolidFire Inc.*
>>  e: mike.tutkow...@solidfire.com
>>  o: 303.746.7302
>>  Advancing the way the world uses the
>>  cloud
>>  *(tm)*
>> 
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> *Mike Tutkowski*
>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>> e: mike.tutkow...@solidfire.com
>> >>> o: 303.746.7302
>> >>> Advancing the way the worl

Re: [PROPOSAL] Service monitoring tool in virtual router

[Chiradeep Vittal]

Well just make sure that your script is resilient to its own crashes as
well.

On 10/4/13 1:59 AM, "Jayapal Reddy Uradi" 
wrote:

>Hi,
>
>I am planning to write script utility to monitor processes and restart on
>the event of failure. It will also logs the events.
>
>Thanks,
>Jayapal
>
>On 02-Oct-2013, at 3:25 AM, Simon Weller  wrote:
>
>> supervisord maybe?
>> 
>> - Original Message -
>> 
>> From: "Chiradeep Vittal" 
>> To: dev@cloudstack.apache.org
>> Sent: Tuesday, October 1, 2013 4:45:56 PM
>> Subject: Re: [PROPOSAL] Service monitoring tool in virtual router
>> 
>> Got it. Any other OSS tool out there similar to monit?
>> 
>> On 10/1/13 8:24 AM, "David Nalley"  wrote:
>> 
>>> On Thu, Sep 26, 2013 at 1:27 AM, Chiradeep Vittal
>>>  wrote:
 SNMP wouldn't restart a failed process nor would it generate alerts.
It 
 is 
 simply too generic for the requirements outlined here. The proposal
does 
 not talk about modifying monit, just using it. That wouldn't trigger
the 
 AGPL. 
>>> 
>>> Let me restate my objection to anything AGPL.
>>> People are largely comfortable with GPLv2 software - Linux is
>>> ubiquitous. Many legal departments routinely prohibit GPLv3 software
>>> (we actually saw this when CS was GPLv3 licensed.) But the Affero GPL
>>> license is anathema in many corporate environments, and by forcing it
>>> on folks in the default System VM I fear it will hurt adoption of
>>> CloudStack. 
>>> 
>>> --David 
>> 
>> 
>

[ACS4.3] Tasks in JIRA for ACS 4.3

[Sudha Ponnaganti]

Hi,

I see the date to propose features has passed [1]. When would the tasks be 
created to see full scope for 4.3??
[1] 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Cloudstack+4.3+Release

Thanks
/sudha

Re: [PROPOSAL] Modularize Spring

[Darren Shepherd]

Sure, I'm open to suggestions.  Basically I think we've discussed

1) Global Setting
2) canHandle() returns an int
3) Strategy has an enum type assigned

I'm open to all three, I don't have much vested interest in this.

Darren

On Fri, Oct 4, 2013 at 3:00 PM, SuichII, Christopher
 wrote:
> Well, it seems OK, but I think we should keep on discussing our options. One 
> concern I have with the global config approach is that it adds manual steps 
> for 'installing' extensions. Each extension must have installation 
> instructions to indicate which global configurations it must be included in 
> and where in that list it should be put (and of course, many extension are 
> going to say that they should be at the front of the list).
>
> -Chris
> --
> Chris Suich
> chris.su...@netapp.com
> NetApp Software Engineer
> Data Center Platforms – Cloud Solutions
> Citrix, Cisco & Red Hat
>
> On Oct 4, 2013, at 12:12 PM, Darren Shepherd  
> wrote:
>
>> On 10/04/2013 11:58 AM, SuichII, Christopher wrote:
>>> Darren,
>>>
>>> I think one of the benefits of allowing the priority to be specified in the 
>>> xml is that it can be configured after deployment. If for some reason two 
>>> strategies or providers conflict, then their priorities can be changed in 
>>> XML to resolve the conflict. I believe the Spring @Order annotation an be 
>>> specified in XML, not just as an annotation.
>>>
>>> -Chris
>>>
>>
>> I would *prefer* extensions to be order independent, but if we determine 
>> they are order dependant, then that is fine too.  So if we conclude that the 
>> simplest way to address this is to order the Strategies based on 
>> configuration, then I will add an ordering "global configuration" as 
>> described at 
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions.
>>
>> Does the order configuration setting approach seem fine?
>>
>> Darren
>

Re: Hi

[Steve Wilson]

Thanks!

On 10/3/13 4:37 AM, "Wei ZHOU"  wrote:

>Welcome Steve!
>
>
>2013/10/2 Steve Wilson 
>
>> Hi Everyone,
>>
>> I just wanted to take a second and introduce myself.  I recently joined
>> Citrix as the VP of Cloud Engineering, and I'm responsible for the
>>Citrix
>> CloudPlatofrm (powered by Apache CloudStack).  I just joined the list
>>and
>> am looking forward to working with everyone in the community.  Feel
>>free to
>> drop me a note if you have suggestions for me on how Citrix can best
>>work
>> with the Apache CloudStack community.
>>
>> Thanks,
>>
>> -Steve
>> twitter: @virtualsteve
>> email: steve.wil...@citrix.com
>>

Re: Hi

[Steve Wilson]

Thanks!  Looking forward to working with you too!

On 10/3/13 4:35 AM, "Bharat Kumar"  wrote:

>Hi Steve,
>
>Welcome to CloudStack. I look forward to working with you in the
>community.
>
>
>On Oct 3, 2013, at 2:07 PM, Daan Hoogland 
> wrote:
>
>> H Steve,
>> 
>> I wish you luck with managing the Citrix - Community interaction and
>>given
>> the continued huge importance of Citrix to Apache CloudStack I am taking
>> the liberty of calling you our friend.
>> 
>> Welcome,
>> Daan
>> 
>> 
>> On Thu, Oct 3, 2013 at 7:33 AM, Koushik Das 
>>wrote:
>> 
>>> Hi Steve,
>>> 
>>> Welcome to CloudStack. Looking forward to working with you.
>>> 
>>> 
>>> On 02-Oct-2013, at 6:10 PM, Chip Childers 
>>> wrote:
>>> 
 Welcome Steve!
 
 
 On Wed, Oct 2, 2013 at 12:38 AM, Steve Wilson >>> wrote:
 
> Hi Everyone,
> 
> I just wanted to take a second and introduce myself.  I recently
>joined
> Citrix as the VP of Cloud Engineering, and I'm responsible for the
>>> Citrix
> CloudPlatofrm (powered by Apache CloudStack).  I just joined the list
>>> and
> am looking forward to working with everyone in the community.  Feel
>>> free to
> drop me a note if you have suggestions for me on how Citrix can best
>>> work
> with the Apache CloudStack community.
> 
> Thanks,
> 
> -Steve
> twitter: @virtualsteve
> email: steve.wil...@citrix.com
> 
>>> 
>>> 
>

Re: Hi

[Steve Wilson]

I'll try to live up to that!

Thanks for the welcome!

On 10/3/13 1:37 AM, "Daan Hoogland"  wrote:

>H Steve,
>
>I wish you luck with managing the Citrix - Community interaction and given
>the continued huge importance of Citrix to Apache CloudStack I am taking
>the liberty of calling you our friend.
>
>Welcome,
>Daan
>
>
>On Thu, Oct 3, 2013 at 7:33 AM, Koushik Das 
>wrote:
>
>> Hi Steve,
>>
>> Welcome to CloudStack. Looking forward to working with you.
>>
>>
>> On 02-Oct-2013, at 6:10 PM, Chip Childers 
>> wrote:
>>
>> > Welcome Steve!
>> >
>> >
>> > On Wed, Oct 2, 2013 at 12:38 AM, Steve Wilson > >wrote:
>> >
>> >> Hi Everyone,
>> >>
>> >> I just wanted to take a second and introduce myself.  I recently
>>joined
>> >> Citrix as the VP of Cloud Engineering, and I'm responsible for the
>> Citrix
>> >> CloudPlatofrm (powered by Apache CloudStack).  I just joined the list
>> and
>> >> am looking forward to working with everyone in the community.  Feel
>> free to
>> >> drop me a note if you have suggestions for me on how Citrix can best
>> work
>> >> with the Apache CloudStack community.
>> >>
>> >> Thanks,
>> >>
>> >> -Steve
>> >> twitter: @virtualsteve
>> >> email: steve.wil...@citrix.com
>> >>
>>
>>

Re: Hi

[Steve Wilson]

Thanks!

On 10/2/13 10:33 PM, "Koushik Das"  wrote:

>Hi Steve,
>
>Welcome to CloudStack. Looking forward to working with you.
>
>
>On 02-Oct-2013, at 6:10 PM, Chip Childers 
>wrote:
>
>> Welcome Steve!
>> 
>> 
>> On Wed, Oct 2, 2013 at 12:38 AM, Steve Wilson
>>wrote:
>> 
>>> Hi Everyone,
>>> 
>>> I just wanted to take a second and introduce myself.  I recently joined
>>> Citrix as the VP of Cloud Engineering, and I'm responsible for the
>>>Citrix
>>> CloudPlatofrm (powered by Apache CloudStack).  I just joined the list
>>>and
>>> am looking forward to working with everyone in the community.  Feel
>>>free to
>>> drop me a note if you have suggestions for me on how Citrix can best
>>>work
>>> with the Apache CloudStack community.
>>> 
>>> Thanks,
>>> 
>>> -Steve
>>> twitter: @virtualsteve
>>> email: steve.wil...@citrix.com
>>> 
>

Re: [PROPOSAL] Modularize Spring

[SuichII, Christopher]

Well, it seems OK, but I think we should keep on discussing our options. One 
concern I have with the global config approach is that it adds manual steps for 
'installing' extensions. Each extension must have installation instructions to 
indicate which global configurations it must be included in and where in that 
list it should be put (and of course, many extension are going to say that they 
should be at the front of the list).

-Chris
-- 
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Oct 4, 2013, at 12:12 PM, Darren Shepherd  
wrote:

> On 10/04/2013 11:58 AM, SuichII, Christopher wrote:
>> Darren,
>> 
>> I think one of the benefits of allowing the priority to be specified in the 
>> xml is that it can be configured after deployment. If for some reason two 
>> strategies or providers conflict, then their priorities can be changed in 
>> XML to resolve the conflict. I believe the Spring @Order annotation an be 
>> specified in XML, not just as an annotation.
>> 
>> -Chris
>> 
> 
> I would *prefer* extensions to be order independent, but if we determine they 
> are order dependant, then that is fine too.  So if we conclude that the 
> simplest way to address this is to order the Strategies based on 
> configuration, then I will add an ordering "global configuration" as 
> described at 
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions.
> 
> Does the order configuration setting approach seem fine?
> 
> Darren

Re: [HOW-TO] Custom REST call on Destroy and Deploy

[Darren Shepherd]

Once the Spring work gets merged to master this will change as
componentContext.xml or the nonoss version is gone now.  You would
need to create a file like
META-INF/cloudstack/core/spring-eventbus-context.xml on the classpath,
so if this was production, that would probably be
/etc/cloudstack/management/META-INF/cloudstack/core/spring-eventbus-context.xml.

Really we should properly package this in its own optional RPM, but
that's a different discussion.

Darren

On Fri, Oct 4, 2013 at 1:19 PM, Chip Childers  wrote:
> On Fri, Oct 4, 2013 at 4:15 PM, Musayev, Ilya  wrote:
>
>> > On Fri, Oct 04, 2013 at 07:21:03PM +, Musayev, Ilya wrote:
>> > > We would like to add additional operations in ACS when we do deploy and
>> > destroy.
>> > >
>> > > For example, when VM is destroyed, we want to make create a JIRA ticket
>> > (rest call) to track the decomision process.
>> > > When VM is deployed, we may need to do something else, but also rest
>> > based?
>> > >
>> > > I can see 2 ways of doing this,
>> > >
>> > >
>> > > 1)  Extend the UI and initiate extra REST calls via java script
>>  (low risk)
>> > >
>> > > 2)  Add more functionality to API Command (extra work and
>> maintenance
>> > burden)
>> > >
>> > > 3)  Create a Custom front end (seems like a lot of work)
>> > >
>> > >
>> > > How can this be done most efficiently and hopefully with least
>> intrusion?
>> >
>> > Have you considered using the RabbitMQ eventbus plugin, and tying into
>> the
>> > events that you want to act on that way?
>>
>> I see you made an awesome post on integrating RabbitMQ with ACS.
>>
>>
>> http://www.chipchilders.com/blog/2013/7/16/tapping-into-apache-cloudstack-events-via-amqp.html
>>
>> You've made it a part of simulatorComponentContext.xml.in, however, in my
>> case its going to be a production system with real events, where would you
>> recommend this go?
>>
>> Thanks
>> ilya
>>
>>
>>
> In the Component Context file that you use in prod.  The different build
> targets pull in one of the different component contexts as the version to
> use in the specific build.  I suppose that means (because I know you use
> Vmware), that you'd put it in nonossComponentContext.xml.in.

RE: Master doesn't build

[Edison Su]

Sorry, forget to check in some files.

> -Original Message-
> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
> Sent: Friday, October 04, 2013 2:25 PM
> To: dev@cloudstack.apache.org
> Subject: Re: Master doesn't build
> 
> Looks like the missing files were just added.
> 
> We should be OK now.
> 
> 
> On Fri, Oct 4, 2013 at 3:23 PM, Mike Tutkowski
>  > wrote:
> 
> > Here is how I tried to build it:
> >
> > mvn -P developer,systemvm clean install -Dnoredist
> >
> >
> > On Fri, Oct 4, 2013 at 3:22 PM, Mike Tutkowski <
> > mike.tutkow...@solidfire.com> wrote:
> >
> >> [INFO] BUILD FAILURE
> >> [INFO]
> >> -
> >> ---
> >> [INFO] Total time: 55.164s
> >> [INFO] Finished at: Fri Oct 04 15:20:55 MDT 2013 [INFO] Final Memory:
> >> 40M/483M [INFO]
> >> -
> >> ---
> >> [ERROR] Failed to execute goal
> >> org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile
> >> (default-compile) on project cloud-core: Compilation failure:
> >> Compilation
> >> failure:
> >> [ERROR]
> >>
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cl
> >>
> oud/storage/resource/StorageSubsystemCommandHandlerBase.java:[27,44]
> >> error: cannot find symbol
> >> [ERROR] symbol:   class IntroduceObjectCmd
> >> [ERROR] location: package org.apache.cloudstack.storage.command
> >> [ERROR]
> >>
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cl
> >> oud/storage/resource/StorageProcessor.java:[26,44]
> >> error: cannot find symbol
> >> [ERROR] symbol:   class ForgetObjectCmd
> >> [ERROR] location: package org.apache.cloudstack.storage.command
> >> [ERROR]
> >>
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cl
> >> oud/storage/resource/StorageProcessor.java:[27,44]
> >> error: cannot find symbol
> >> [ERROR] symbol:   class IntroduceObjectCmd
> >> [ERROR] location: package org.apache.cloudstack.storage.command
> >> [ERROR]
> >>
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cl
> >> oud/storage/resource/StorageProcessor.java:[30,44]
> >> error: cannot find symbol
> >> [ERROR] symbol:   class ForgetObjectCmd
> >> [ERROR] location: package org.apache.cloudstack.storage.command
> >> [ERROR]
> >>
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cl
> >> oud/storage/resource/StorageProcessor.java:[31,44]
> >> error: cannot find symbol
> >> [ERROR] symbol:   class IntroduceObjectCmd
> >> [ERROR] location: package org.apache.cloudstack.storage.command
> >> [ERROR]
> >>
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cl
> >> oud/storage/resource/StorageProcessor.java:[50,27]
> >> error: cannot find symbol
> >> [ERROR] symbol:   class IntroduceObjectCmd
> >> [ERROR] location: interface StorageProcessor [ERROR]
> >>
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cl
> >> oud/storage/resource/StorageProcessor.java:[51,24]
> >> error: cannot find symbol
> >> [ERROR] symbol:   class ForgetObjectCmd
> >> [ERROR] location: interface StorageProcessor [ERROR]
> >>
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cl
> >>
> oud/storage/resource/StorageSubsystemCommandHandlerBase.java:[59,38]
> >> error: cannot find symbol
> >> [ERROR] symbol:   class IntroduceObjectCmd
> >> [ERROR] location: class StorageSubsystemCommandHandlerBase
> >> [ERROR]
> >>
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cl
> >>
> oud/storage/resource/StorageSubsystemCommandHandlerBase.java:[60,46]
> >> error: cannot find symbol
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkow...@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the
> >> cloud
> >> *(tm)*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkow...@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud
> > *(tm)*
> >
> 
> 
> 
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud
> *(tm)*

Re: Review Request 14477: Refactor Storage Related Resource Code

[edison su]

> On Oct. 4, 2013, 8:23 p.m., Chip Childers wrote:
> > api/src/org/apache/cloudstack/api/command/user/snapshot/RevertSnapshotCmd.java,
> >  line 36
> > 
> >
> > Are there any unit or integration tests for this new API call? I can't 
> > find any in this diff.

There is no implementation for revertsnapshot yet in the current storage 
drivers. If any storage vendor wants to implement this feature, they can add 
their implementation in their storage driver. Also, if anybody is interested in 
implementing this feature in the default storage driver, for example, for Ceph, 
then it's the place to start with. 


> On Oct. 4, 2013, 8:23 p.m., Chip Childers wrote:
> > core/src/org/apache/cloudstack/storage/command/ForgetObjectCmd.java, line 24
> > 
> >
> > Are there any unit or integration tests for this new API call? I can't 
> > find any in this diff.

It's not an api, it's just a simple command will send to hypervisor host, will 
be used by HypervisorHelper to introduce/forget an object on hypervisor host.


> On Oct. 4, 2013, 8:23 p.m., Chip Childers wrote:
> > engine/storage/src/org/apache/cloudstack/storage/helper/HypervisorHelperImpl.java,
> >  lines 35-76
> > 
> >
> > I feel like this new code should have unit test for the if logic trees.

The logic here is very simple: just send a command to hypervisor host. Not sure 
need unit test here or not.


> On Oct. 4, 2013, 8:23 p.m., Chip Childers wrote:
> > plugins/hypervisors/simulator/src/com/cloud/resource/SimulatorStorageProcessor.java,
> >  lines 226-236
> > 
> >
> > Can we get rid of these TODOs?

Yes, I can remove the TODO


> On Oct. 4, 2013, 8:23 p.m., Chip Childers wrote:
> > server/src/com/cloud/storage/snapshot/SnapshotManagerImpl.java, lines 
> > 264-283
> > 
> >
> > Unit tests for this if logic?

Again, straight forward code, not sure need add unit test here.


- edison


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14477/#review26694
---


On Oct. 4, 2013, 12:57 a.m., Chris Suich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14477/
> ---
> 
> (Updated Oct. 4, 2013, 12:57 a.m.)
> 
> 
> Review request for cloudstack.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> These changes are a joint effort between Edison and I to refactor some of the 
> code around snapshotting VM volumes and creating templates/volumes from VM 
> volume snapshots. In general, we were working towards allowing 
> PrimaryDataStoreDrivers to create snapshots on primary storage and not 
> requiring the snapshots to be transferred to secondary storage.
> 
> High level changes:
> -Added uuid to NfsTO, SwiftTO & S3TO to cut down on the requirement of 
> PrimaryDataStoreTO and ImageStoreTO which don't really serve much of a purpose
> -Initial work towards enable reverting VM volume from snapshots
> -Added hypervisor commands for introducing and forgetting new hypervisor 
> objects (snapshots, templates & volumes)
> 
> 
> Diffs
> -
> 
>   api/src/com/cloud/agent/api/to/DataStoreTO.java 9014f8e 
>   api/src/com/cloud/agent/api/to/NfsTO.java 415c95c 
>   api/src/com/cloud/agent/api/to/SwiftTO.java 7349d77 
>   api/src/com/cloud/event/EventTypes.java ec9604e 
>   api/src/com/cloud/storage/snapshot/SnapshotApiService.java 23e6522 
>   
> api/src/org/apache/cloudstack/api/command/admin/storage/ListStoragePoolsCmd.java
>  26351bb 
>   
> api/src/org/apache/cloudstack/api/command/user/snapshot/RevertSnapshotCmd.java
>  PRE-CREATION 
>   core/src/com/cloud/storage/resource/StorageProcessor.java 5fa9f8a 
>   core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java 
> ab9aa2a 
>   core/src/org/apache/cloudstack/storage/command/ForgetObjectCmd.java 
> PRE-CREATION 
>   core/src/org/apache/cloudstack/storage/command/IntroduceObjectAnswer.java 
> PRE-CREATION 
>   core/src/org/apache/cloudstack/storage/command/IntroduceObjectCmd.java 
> PRE-CREATION 
>   core/src/org/apache/cloudstack/storage/to/ImageStoreTO.java 0037ea5 
>   core/src/org/apache/cloudstack/storage/to/PrimaryDataStoreTO.java 5e870df 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/EndPointSelector.java
>  ca0cc2c 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/SnapshotService.java
>  d594a07 
>   
> engine/api/src/org/apache/clo

Re: Master doesn't build

[Mike Tutkowski]

Looks like the missing files were just added.

We should be OK now.


On Fri, Oct 4, 2013 at 3:23 PM, Mike Tutkowski  wrote:

> Here is how I tried to build it:
>
> mvn -P developer,systemvm clean install -Dnoredist
>
>
> On Fri, Oct 4, 2013 at 3:22 PM, Mike Tutkowski <
> mike.tutkow...@solidfire.com> wrote:
>
>> [INFO] BUILD FAILURE
>> [INFO]
>> 
>> [INFO] Total time: 55.164s
>> [INFO] Finished at: Fri Oct 04 15:20:55 MDT 2013
>> [INFO] Final Memory: 40M/483M
>> [INFO]
>> 
>> [ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile
>> (default-compile) on project cloud-core: Compilation failure: Compilation
>> failure:
>> [ERROR]
>> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java:[27,44]
>> error: cannot find symbol
>> [ERROR] symbol:   class IntroduceObjectCmd
>> [ERROR] location: package org.apache.cloudstack.storage.command
>> [ERROR]
>> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageProcessor.java:[26,44]
>> error: cannot find symbol
>> [ERROR] symbol:   class ForgetObjectCmd
>> [ERROR] location: package org.apache.cloudstack.storage.command
>> [ERROR]
>> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageProcessor.java:[27,44]
>> error: cannot find symbol
>> [ERROR] symbol:   class IntroduceObjectCmd
>> [ERROR] location: package org.apache.cloudstack.storage.command
>> [ERROR]
>> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageProcessor.java:[30,44]
>> error: cannot find symbol
>> [ERROR] symbol:   class ForgetObjectCmd
>> [ERROR] location: package org.apache.cloudstack.storage.command
>> [ERROR]
>> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageProcessor.java:[31,44]
>> error: cannot find symbol
>> [ERROR] symbol:   class IntroduceObjectCmd
>> [ERROR] location: package org.apache.cloudstack.storage.command
>> [ERROR]
>> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageProcessor.java:[50,27]
>> error: cannot find symbol
>> [ERROR] symbol:   class IntroduceObjectCmd
>> [ERROR] location: interface StorageProcessor
>> [ERROR]
>> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageProcessor.java:[51,24]
>> error: cannot find symbol
>> [ERROR] symbol:   class ForgetObjectCmd
>> [ERROR] location: interface StorageProcessor
>> [ERROR]
>> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java:[59,38]
>> error: cannot find symbol
>> [ERROR] symbol:   class IntroduceObjectCmd
>> [ERROR] location: class StorageSubsystemCommandHandlerBase
>> [ERROR]
>> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java:[60,46]
>> error: cannot find symbol
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkow...@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the 
>> cloud
>> *™*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the 
> cloud
> *™*
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

Re: Master doesn't build

[Mike Tutkowski]

Here is how I tried to build it:

mvn -P developer,systemvm clean install -Dnoredist


On Fri, Oct 4, 2013 at 3:22 PM, Mike Tutkowski  wrote:

> [INFO] BUILD FAILURE
> [INFO]
> 
> [INFO] Total time: 55.164s
> [INFO] Finished at: Fri Oct 04 15:20:55 MDT 2013
> [INFO] Final Memory: 40M/483M
> [INFO]
> 
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile
> (default-compile) on project cloud-core: Compilation failure: Compilation
> failure:
> [ERROR]
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java:[27,44]
> error: cannot find symbol
> [ERROR] symbol:   class IntroduceObjectCmd
> [ERROR] location: package org.apache.cloudstack.storage.command
> [ERROR]
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageProcessor.java:[26,44]
> error: cannot find symbol
> [ERROR] symbol:   class ForgetObjectCmd
> [ERROR] location: package org.apache.cloudstack.storage.command
> [ERROR]
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageProcessor.java:[27,44]
> error: cannot find symbol
> [ERROR] symbol:   class IntroduceObjectCmd
> [ERROR] location: package org.apache.cloudstack.storage.command
> [ERROR]
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageProcessor.java:[30,44]
> error: cannot find symbol
> [ERROR] symbol:   class ForgetObjectCmd
> [ERROR] location: package org.apache.cloudstack.storage.command
> [ERROR]
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageProcessor.java:[31,44]
> error: cannot find symbol
> [ERROR] symbol:   class IntroduceObjectCmd
> [ERROR] location: package org.apache.cloudstack.storage.command
> [ERROR]
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageProcessor.java:[50,27]
> error: cannot find symbol
> [ERROR] symbol:   class IntroduceObjectCmd
> [ERROR] location: interface StorageProcessor
> [ERROR]
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageProcessor.java:[51,24]
> error: cannot find symbol
> [ERROR] symbol:   class ForgetObjectCmd
> [ERROR] location: interface StorageProcessor
> [ERROR]
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java:[59,38]
> error: cannot find symbol
> [ERROR] symbol:   class IntroduceObjectCmd
> [ERROR] location: class StorageSubsystemCommandHandlerBase
> [ERROR]
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java:[60,46]
> error: cannot find symbol
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the 
> cloud
> *™*
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

Master doesn't build

[Mike Tutkowski]

[INFO] BUILD FAILURE
[INFO]

[INFO] Total time: 55.164s
[INFO] Finished at: Fri Oct 04 15:20:55 MDT 2013
[INFO] Final Memory: 40M/483M
[INFO]

[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile
(default-compile) on project cloud-core: Compilation failure: Compilation
failure:
[ERROR]
/Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java:[27,44]
error: cannot find symbol
[ERROR] symbol:   class IntroduceObjectCmd
[ERROR] location: package org.apache.cloudstack.storage.command
[ERROR]
/Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageProcessor.java:[26,44]
error: cannot find symbol
[ERROR] symbol:   class ForgetObjectCmd
[ERROR] location: package org.apache.cloudstack.storage.command
[ERROR]
/Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageProcessor.java:[27,44]
error: cannot find symbol
[ERROR] symbol:   class IntroduceObjectCmd
[ERROR] location: package org.apache.cloudstack.storage.command
[ERROR]
/Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageProcessor.java:[30,44]
error: cannot find symbol
[ERROR] symbol:   class ForgetObjectCmd
[ERROR] location: package org.apache.cloudstack.storage.command
[ERROR]
/Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageProcessor.java:[31,44]
error: cannot find symbol
[ERROR] symbol:   class IntroduceObjectCmd
[ERROR] location: package org.apache.cloudstack.storage.command
[ERROR]
/Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageProcessor.java:[50,27]
error: cannot find symbol
[ERROR] symbol:   class IntroduceObjectCmd
[ERROR] location: interface StorageProcessor
[ERROR]
/Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageProcessor.java:[51,24]
error: cannot find symbol
[ERROR] symbol:   class ForgetObjectCmd
[ERROR] location: interface StorageProcessor
[ERROR]
/Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java:[59,38]
error: cannot find symbol
[ERROR] symbol:   class IntroduceObjectCmd
[ERROR] location: class StorageSubsystemCommandHandlerBase
[ERROR]
/Users/mtutkowski/Documents/CloudStack/src/CloudStack/core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java:[60,46]
error: cannot find symbol

-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

Re: [DISCUSS] Breaking out Marvin from CloudStack

[Chip Childers]

Fair warning - some of this is a straw man argument to explore the
situation, and a little bit of ranting at the end.

On Fri, Oct 04, 2013 at 05:57:58PM +0530, Prasanna Santhanam wrote:
> I'll summarize and address the concerns raised so far.
> 
> Marvin has been in this repo for a long time for us to start writing
> tests. The only tests I've seen coming are from a specific set of
> people focussed on QA efforts. 

I agree - and that's a problem.  New features should *ALL* have tests
before they merge into master.  I think that assuming that the only test
writers are a group of folks that write the tests today is actually a
larger problem.

> I want to reduce the impediment for
> people who are writing tests *today*. Those looking to get started in
> the near future won't have any new learning to do, just that their
> code goes in an alternate repo that is pointed to right
> infrastructure. 
> 
> Automated testing also works in a push-to-production style very often.
> Testers need to run their tests on a deployed environment(s) quickly
> to be able to ensure it is valid and passes. By making them go through
> reviewboard each time for each test we massively slow down the
> process. (tons of fixes to tests are on rb today, not just new tests).
> We don't know if they run until they run on the environment.

I want to be clear about this part - a different repo doesn't change the
need for someone to be a committer to commit.

There's a thread on private@ that you should weigh in on here.  If
we have more people that should be committers, then let's get *that*
done.

> 
> Reason for tests and framework to go together is simple.  If I go look
> at the jclouds repository today I find tests for rackspace cloud,
> openstack cloud, cloudstack cloud, euca clouds in the jclouds
> repository and not in the respective provider/project repository. A
> newcomer to the marvin repository will be someone interested in
> writing tests and he will also thus be able to find tests in the
> marvin repository.
> 
> This also allows for more heterogenous testing of cloudstack. No one
> needs to be tied down to a framework / tool to write integration
> tests. If python is not your forte, use Chip's ruby client, or perhaps
> in the near future Chiradeep's stackmate to write your test, or even
> jclouds. 

But that's actually true today, right?  I mean if I wanted to write an
integration test using some other method, I'd do that...  but would it
be useful for others?  Probably not!  That's because the way that we do
testing of this type is via Marvin.  The Citrix infra wouldn't be setup
for whatever other framework I used, and the community as a whole would
get less benefit than if I was consistent.

> 
> Now the question of supporting older version of marvin against newer
> versions of cloudstack. Marvin now fully auto-generates itself (see
> the design in the proposal) based on endpoint. So you have the
> marvin version that will work with your endpoint only. As for being
> backwards compatible (also addressed in the design doc) - no old tests
> are broken, they will still run perfectly fine. 
> 

That's marvin, not the tests, right?  If I add a new feature for 4.3,
should that test appear in a 4.2 run?  Aren't we aiming for (I know
we're not there) a situation where *all* tests pass before we release?
IMO not versioning the tests (not the Marvin framework) with the target
of the tests is confusing.

> The infrastructure (currently) only looks at the changes in the test
> directory before performing a run. It doesn't care whether server/ was
> changed or plugins/x/y/z was changed. That's because the tests are
> unrelated to what is in the rest of the repository. In fact you can't
> even run them without a deployed cloud. So I don't see why idle code
> should lie in the repo.
> 
> Integration tests are essential, they will keep coming as long as
> Citrix QA is invested in the effort, but they need to come faster into
> the repos and that will be addressed by the separation IMO. 

How?

> Managing
> the feature submitted to cloudstack against tests submitted to marvin
> is not a hard thing to do. We simply mirror the release branches in
> marvin and submit tests there. In fact I wonder why we didn't have
> this question when docs were separated? It doesn't work any
> differently really.

Well, it turns out that docs are absolutely not being done before code
comes into master.  I dislike this fact, but live with it.

> 
> What I would like to see provided by CloudStack is the ability to
> upgrade all our test environments, staging environments, UATs, what
> have you in continuous integration and have tests run on the
> upgraded setup. That allows incrementally testing CloudStack the way
> users do it. The current design of installing everything from scratch,
> redoing the testbed for each automated test run is mostly a workaround
> for that inability. If we had this ability tests written in marvin can
> be run against live setups a

Re: Review Request 12228: wip: static resource compression

[Laszlo Hornyak]

> On Oct. 4, 2013, 6:52 p.m., Darren Shepherd wrote:
> > I would prefer not to do this.  From a technical perspective the code uses 
> > getRealPath and Files which are not compatible across all serlvet 
> > containers and this implementation forces a lot of file system stat() 
> > calls, but that isn't my real complaint.  This change adds yet another step 
> > to the already long ACS build and I don't think it address the root issue.  
> > The root issue, if one was to really care about performance is that the js 
> > is not cachable or consolidated.  ts=$now is added to every js file, 
> > meaning it must be downloaded every time.  Additionally to load the login 
> > screen there are 66 requests for JS files.  Even if you compress 2mb to 
> > 800k, on a slow connection it won't matter that much because you are going 
> > to get killed regardless by the round trip latency of making 66 requests.
> 
> Laszlo Hornyak wrote:
> Hi Darren,
> 
> I share your concerns about the build process and I do not want to 
> further increase that. What I had in my mind is an optional step (possibly in 
> a profile) acivated only for making releases and by default not in a 
> developement environment. Not that much to save 2 mb of bandwidth, but to 
> save the time the user have to wait.
> 
> The difficulties with the non cacheable js files is indeed a problem, and 
> I believe that should be addressed separately. (I do not quite understand yet 
> what is that for)
> 
> Also, the dynamic informations can be compressed if it makes sense, e.g. 
> over a given size.
> 
> I added the wip prefix to this to make it clear that this patch is 
> "preview" version and more discussion is needed.
> 
> Brian Federle wrote:
> re: caching of JS files, I believe the reason ts=$now is appended to the 
> JS files was because of issues during upgrade testing, where the old JS files 
> would not be cleared from the browser cache. If we can consolidate into 1 JS 
> file then we don't need that functionality anymore, as it was more a problem 
> due to the large # of scripts being loaded at once.

what was wrong with the http cache control headers?


- Laszlo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12228/#review26690
---


On Oct. 4, 2013, 8:19 p.m., Laszlo Hornyak wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12228/
> ---
> 
> (Updated Oct. 4, 2013, 8:19 p.m.)
> 
> 
> Review request for cloudstack, Brian Federle and Prasanna Santhanam.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> CloudStack at first use downloads some 3.5 MB of css and javascript to the 
> client. With a weak internet connection, this might take a long time. With 
> gzip compression content can be compressed to 850 KB.
> 
> This version of the patch uses a custom plugin to compress static resources, 
> so that no dynamic compression is needed at runtime. When the static resource 
> servlet notices that there is gzipped version of the resource and the client 
> accepts gzipped content, then it is going to send the gziped version, while 
> still respects http caching.
> 
> 
> Diffs
> -
> 
>   client/WEB-INF/web.xml e5c05d3 
>   client/pom.xml 119c96e 
>   server/src/com/cloud/servlet/StaticResourceServlet.java PRE-CREATION 
>   server/test/com/cloud/servlet/StaticResourceServletTest.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/12228/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Laszlo Hornyak
> 
>

Re: Review Request 12228: wip: static resource compression

[Brian Federle]

> On Oct. 4, 2013, 6:52 p.m., Darren Shepherd wrote:
> > I would prefer not to do this.  From a technical perspective the code uses 
> > getRealPath and Files which are not compatible across all serlvet 
> > containers and this implementation forces a lot of file system stat() 
> > calls, but that isn't my real complaint.  This change adds yet another step 
> > to the already long ACS build and I don't think it address the root issue.  
> > The root issue, if one was to really care about performance is that the js 
> > is not cachable or consolidated.  ts=$now is added to every js file, 
> > meaning it must be downloaded every time.  Additionally to load the login 
> > screen there are 66 requests for JS files.  Even if you compress 2mb to 
> > 800k, on a slow connection it won't matter that much because you are going 
> > to get killed regardless by the round trip latency of making 66 requests.
> 
> Laszlo Hornyak wrote:
> Hi Darren,
> 
> I share your concerns about the build process and I do not want to 
> further increase that. What I had in my mind is an optional step (possibly in 
> a profile) acivated only for making releases and by default not in a 
> developement environment. Not that much to save 2 mb of bandwidth, but to 
> save the time the user have to wait.
> 
> The difficulties with the non cacheable js files is indeed a problem, and 
> I believe that should be addressed separately. (I do not quite understand yet 
> what is that for)
> 
> Also, the dynamic informations can be compressed if it makes sense, e.g. 
> over a given size.
> 
> I added the wip prefix to this to make it clear that this patch is 
> "preview" version and more discussion is needed.

re: caching of JS files, I believe the reason ts=$now is appended to the JS 
files was because of issues during upgrade testing, where the old JS files 
would not be cleared from the browser cache. If we can consolidate into 1 JS 
file then we don't need that functionality anymore, as it was more a problem 
due to the large # of scripts being loaded at once. 


- Brian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12228/#review26690
---


On Oct. 4, 2013, 8:19 p.m., Laszlo Hornyak wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12228/
> ---
> 
> (Updated Oct. 4, 2013, 8:19 p.m.)
> 
> 
> Review request for cloudstack, Brian Federle and Prasanna Santhanam.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> CloudStack at first use downloads some 3.5 MB of css and javascript to the 
> client. With a weak internet connection, this might take a long time. With 
> gzip compression content can be compressed to 850 KB.
> 
> This version of the patch uses a custom plugin to compress static resources, 
> so that no dynamic compression is needed at runtime. When the static resource 
> servlet notices that there is gzipped version of the resource and the client 
> accepts gzipped content, then it is going to send the gziped version, while 
> still respects http caching.
> 
> 
> Diffs
> -
> 
>   client/WEB-INF/web.xml e5c05d3 
>   client/pom.xml 119c96e 
>   server/src/com/cloud/servlet/StaticResourceServlet.java PRE-CREATION 
>   server/test/com/cloud/servlet/StaticResourceServletTest.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/12228/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Laszlo Hornyak
> 
>

RE: [DISCUSS] Components in JIRA and bug assignment

[Animesh Chaturvedi]

> -Original Message-
> From: Musayev, Ilya [mailto:imusa...@webmd.net]
> Sent: Friday, October 04, 2013 12:10 PM
> To: dev@cloudstack.apache.org
> Subject: RE: [DISCUSS] Components in JIRA and bug assignment
> 
> Response in line:
> 
> > -Original Message-
> > From: Santhosh Edukulla [mailto:santhosh.eduku...@citrix.com]
> > Sent: Friday, October 04, 2013 2:02 PM
> > To: dev@cloudstack.apache.org
> > Subject: RE: [DISCUSS] Components in JIRA and bug assignment
> >
> > Not Sure, if it falls under similar lines\requires a separate thread.
> > I am trying to see if there is a way we can add  a component field,
> > primary owner fields for component  available to be assigned for a
> > given review under Http://reviews.apache.org.
> 
> While we want to extend JIRA and add various components to address
> current limitations, I'm not certain if this is in the same scope. I
> also have to admit, that I only use JIRA as basic user, so I don't know
> if what you are asking is doable. Perhaps someone else can?
> 
[Animesh>] Reviews are managed in separate tool called ReviewBoard
> 
> >
> > This way i will select component as "Automation", branch as "4.1" and
> > so etc , save that query on my dashboard and everytime i login, i can
> > see reviews based upon my query saved.
> 
> We can create a separate component Automation, so you can create filters
> as described.
> 
[Animesh>] JIRA already has a component for Automation. Santhosh you should 
open a separate thread for review board enhancements

> >
> > If this facility is already there, then i will search more to find it
> out.
> >
> > Regards,
> > Santhosh
> > 
> > From: Chip Childers [chip.child...@sungard.com]
> > Sent: Friday, October 04, 2013 1:45 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [DISCUSS] Components in JIRA and bug assignment
> >
> > On Fri, Oct 04, 2013 at 05:40:06PM +, Animesh Chaturvedi wrote:
> > >
> > > [Animesh>] Chip does that mean you are open to folks assigning
> > > tickets to others? Should this be posted in the original thread
> > > http://markmail.org/thread/btovj6t6opqxge5q. because the threads
> > where
> > > Ilya reopened the discussion has received few responses from Alex,
> > > Sheng, Alena, Sudha, RamG
> > >
> >
> > No, that just means that I was explaining the technical capabilities
> of Jira.
> >
> > I'm -0, so move forward if everyone else continues to agree.  I still
> > don't think it's the right way to operate, but I'm not interested in
> > blocking it if everyone else wants to do it this way.
> >
> > I won't reiterate my previous reasoning, since you pointed to that
> thread.
> >
> > Cheers!
> >
> > -chip
> 

Re: [1/2] Refactor Storage Related Resource Code These changes are a joint effort between Edison and I to refactor some of the code around snapshotting VM volumes and creating templates/volumes from V

[Chip Childers]

I some outstanding questions I just posted to the review.

Any thoughts on those questions?

https://reviews.apache.org/r/14477/


On Fri, Oct 4, 2013 at 4:07 PM,  wrote:

> Updated Branches:
>   refs/heads/master c9f41d404 -> 180cfa19e
>
>
>
> http://git-wip-us.apache.org/repos/asf/cloudstack/blob/180cfa19/plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/XenServerStorageProcessor.java
> --
> diff --git
> a/plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/XenServerStorageProcessor.java
> b/plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/XenServerStorageProcessor.java
> index 739b974..5da0571 100644
> ---
> a/plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/XenServerStorageProcessor.java
> +++
> b/plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/XenServerStorageProcessor.java
> @@ -18,6 +18,36 @@
>   */
>  package com.cloud.hypervisor.xen.resource;
>
> +import java.io.File;
> +import java.net.URI;
> +import java.util.Arrays;
> +import java.util.HashMap;
> +import java.util.List;
> +import java.util.Map;
> +import java.util.Set;
> +import java.util.UUID;
> +
> +import org.apache.cloudstack.storage.command.AttachAnswer;
> +import org.apache.cloudstack.storage.command.AttachCommand;
> +import org.apache.cloudstack.storage.command.AttachPrimaryDataStoreAnswer;
> +import org.apache.cloudstack.storage.command.AttachPrimaryDataStoreCmd;
> +import org.apache.cloudstack.storage.command.CopyCmdAnswer;
> +import org.apache.cloudstack.storage.command.CopyCommand;
> +import org.apache.cloudstack.storage.command.CreateObjectAnswer;
> +import org.apache.cloudstack.storage.command.CreateObjectCommand;
> +import org.apache.cloudstack.storage.command.DeleteCommand;
> +import org.apache.cloudstack.storage.command.DettachAnswer;
> +import org.apache.cloudstack.storage.command.DettachCommand;
> +import org.apache.cloudstack.storage.command.ForgetObjectCmd;
> +import org.apache.cloudstack.storage.command.IntroduceObjectAnswer;
> +import org.apache.cloudstack.storage.command.IntroduceObjectCmd;
> +import org.apache.cloudstack.storage.datastore.protocol.DataStoreProtocol;
> +import org.apache.cloudstack.storage.to.SnapshotObjectTO;
> +import org.apache.cloudstack.storage.to.TemplateObjectTO;
> +import org.apache.cloudstack.storage.to.VolumeObjectTO;
> +import org.apache.log4j.Logger;
> +import org.apache.xmlrpc.XmlRpcException;
> +
>  import com.cloud.agent.api.Answer;
>  import com.cloud.agent.api.CreateStoragePoolCommand;
>  import com.cloud.agent.api.to.DataObjectType;
> @@ -51,33 +81,6 @@ import com.xensource.xenapi.VBD;
>  import com.xensource.xenapi.VDI;
>  import com.xensource.xenapi.VM;
>  import com.xensource.xenapi.VMGuestMetrics;
> -import org.apache.cloudstack.storage.command.AttachAnswer;
> -import org.apache.cloudstack.storage.command.AttachCommand;
> -import org.apache.cloudstack.storage.command.AttachPrimaryDataStoreAnswer;
> -import org.apache.cloudstack.storage.command.AttachPrimaryDataStoreCmd;
> -import org.apache.cloudstack.storage.command.CopyCmdAnswer;
> -import org.apache.cloudstack.storage.command.CopyCommand;
> -import org.apache.cloudstack.storage.command.CreateObjectAnswer;
> -import org.apache.cloudstack.storage.command.CreateObjectCommand;
> -import org.apache.cloudstack.storage.command.DeleteCommand;
> -import org.apache.cloudstack.storage.command.DettachAnswer;
> -import org.apache.cloudstack.storage.command.DettachCommand;
> -import org.apache.cloudstack.storage.datastore.protocol.DataStoreProtocol;
> -import org.apache.cloudstack.storage.to.PrimaryDataStoreTO;
> -import org.apache.cloudstack.storage.to.SnapshotObjectTO;
> -import org.apache.cloudstack.storage.to.TemplateObjectTO;
> -import org.apache.cloudstack.storage.to.VolumeObjectTO;
> -import org.apache.log4j.Logger;
> -import org.apache.xmlrpc.XmlRpcException;
> -
> -import java.io.File;
> -import java.net.URI;
> -import java.util.Arrays;
> -import java.util.HashMap;
> -import java.util.List;
> -import java.util.Map;
> -import java.util.Set;
> -import java.util.UUID;
>
>  import static com.cloud.utils.ReflectUtil.flattenProperties;
>  import static com.google.common.collect.Lists.newArrayList;
> @@ -841,8 +844,7 @@ public class XenServerStorageProcessor implements
> StorageProcessor {
>
>  URI uri = new URI(storeUrl);
>  String tmplpath = uri.getHost() + ":" + uri.getPath() +
> "/" + srcData.getPath();
> -PrimaryDataStoreTO destStore =
> (PrimaryDataStoreTO)destData.getDataStore();
> -String poolName = destStore.getUuid();
> +String poolName = destData.getDataStore().getUuid();
>  Connection conn = hypervisorResource.getConnection();
>
>  SR poolsr = null;
> @@ -892,8 +894,7 @@ public class XenServerStorageProcessor implements
> StorageProcessor {
>
>  try {
>  Connection conn = h

Re: Review Request 14477: Refactor Storage Related Resource Code

[Chip Childers]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14477/#review26694
---


Some questions and comments that probably need to be answered before this is 
committed.  Thanks Chris!


api/src/org/apache/cloudstack/api/command/user/snapshot/RevertSnapshotCmd.java


Are there any unit or integration tests for this new API call? I can't find 
any in this diff.



core/src/org/apache/cloudstack/storage/command/ForgetObjectCmd.java


Are there any unit or integration tests for this new API call? I can't find 
any in this diff.



core/src/org/apache/cloudstack/storage/command/IntroduceObjectCmd.java


Are there any unit or integration tests for this new API call? I can't find 
any in this diff.



engine/storage/src/org/apache/cloudstack/storage/helper/HypervisorHelperImpl.java


I feel like this new code should have unit test for the if logic trees.



plugins/hypervisors/simulator/src/com/cloud/resource/SimulatorStorageProcessor.java


Can we get rid of these TODOs?



server/src/com/cloud/storage/snapshot/SnapshotManagerImpl.java


Unit tests for this if logic?


- Chip Childers


On Oct. 4, 2013, 12:57 a.m., Chris Suich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14477/
> ---
> 
> (Updated Oct. 4, 2013, 12:57 a.m.)
> 
> 
> Review request for cloudstack.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> These changes are a joint effort between Edison and I to refactor some of the 
> code around snapshotting VM volumes and creating templates/volumes from VM 
> volume snapshots. In general, we were working towards allowing 
> PrimaryDataStoreDrivers to create snapshots on primary storage and not 
> requiring the snapshots to be transferred to secondary storage.
> 
> High level changes:
> -Added uuid to NfsTO, SwiftTO & S3TO to cut down on the requirement of 
> PrimaryDataStoreTO and ImageStoreTO which don't really serve much of a purpose
> -Initial work towards enable reverting VM volume from snapshots
> -Added hypervisor commands for introducing and forgetting new hypervisor 
> objects (snapshots, templates & volumes)
> 
> 
> Diffs
> -
> 
>   api/src/com/cloud/agent/api/to/DataStoreTO.java 9014f8e 
>   api/src/com/cloud/agent/api/to/NfsTO.java 415c95c 
>   api/src/com/cloud/agent/api/to/SwiftTO.java 7349d77 
>   api/src/com/cloud/event/EventTypes.java ec9604e 
>   api/src/com/cloud/storage/snapshot/SnapshotApiService.java 23e6522 
>   
> api/src/org/apache/cloudstack/api/command/admin/storage/ListStoragePoolsCmd.java
>  26351bb 
>   
> api/src/org/apache/cloudstack/api/command/user/snapshot/RevertSnapshotCmd.java
>  PRE-CREATION 
>   core/src/com/cloud/storage/resource/StorageProcessor.java 5fa9f8a 
>   core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java 
> ab9aa2a 
>   core/src/org/apache/cloudstack/storage/command/ForgetObjectCmd.java 
> PRE-CREATION 
>   core/src/org/apache/cloudstack/storage/command/IntroduceObjectAnswer.java 
> PRE-CREATION 
>   core/src/org/apache/cloudstack/storage/command/IntroduceObjectCmd.java 
> PRE-CREATION 
>   core/src/org/apache/cloudstack/storage/to/ImageStoreTO.java 0037ea5 
>   core/src/org/apache/cloudstack/storage/to/PrimaryDataStoreTO.java 5e870df 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/EndPointSelector.java
>  ca0cc2c 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/SnapshotService.java
>  d594a07 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/SnapshotStrategy.java
>  86ae532 
>   
> engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/AncientDataMotionStrategy.java
>  96d1f5a 
>   
> engine/storage/image/src/org/apache/cloudstack/storage/image/store/ImageStoreImpl.java
>  855d8cb 
>   
> engine/storage/integration-test/test/org/apache/cloudstack/storage/test/SnapshotTestWithFakeData.java
>  2aaabda 
>   
> engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/SnapshotServiceImpl.java
>  3ead93f 
>   
> engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/SnapshotStrategyBase.java
>  1b57922 
>   
> engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/XenserverSnapshotStrategy.java
>  60d9407 
>   
> engine/storage/src/org/apache/cloudstack/storage/endpoint/DefaultEndPointSelector.java
>  fdc12bf 
>   
> engine/storage/src/org/apache/cloudstack

Re: Review Request 12228: wip: static resource compression

[Laszlo Hornyak]

> On Oct. 4, 2013, 6:52 p.m., Darren Shepherd wrote:
> > I would prefer not to do this.  From a technical perspective the code uses 
> > getRealPath and Files which are not compatible across all serlvet 
> > containers and this implementation forces a lot of file system stat() 
> > calls, but that isn't my real complaint.  This change adds yet another step 
> > to the already long ACS build and I don't think it address the root issue.  
> > The root issue, if one was to really care about performance is that the js 
> > is not cachable or consolidated.  ts=$now is added to every js file, 
> > meaning it must be downloaded every time.  Additionally to load the login 
> > screen there are 66 requests for JS files.  Even if you compress 2mb to 
> > 800k, on a slow connection it won't matter that much because you are going 
> > to get killed regardless by the round trip latency of making 66 requests.

Hi Darren,

I share your concerns about the build process and I do not want to further 
increase that. What I had in my mind is an optional step (possibly in a 
profile) acivated only for making releases and by default not in a developement 
environment. Not that much to save 2 mb of bandwidth, but to save the time the 
user have to wait.

The difficulties with the non cacheable js files is indeed a problem, and I 
believe that should be addressed separately. (I do not quite understand yet 
what is that for)

Also, the dynamic informations can be compressed if it makes sense, e.g. over a 
given size.

I added the wip prefix to this to make it clear that this patch is "preview" 
version and more discussion is needed.


- Laszlo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12228/#review26690
---


On Oct. 4, 2013, 8:19 p.m., Laszlo Hornyak wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12228/
> ---
> 
> (Updated Oct. 4, 2013, 8:19 p.m.)
> 
> 
> Review request for cloudstack, Brian Federle and Prasanna Santhanam.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> CloudStack at first use downloads some 3.5 MB of css and javascript to the 
> client. With a weak internet connection, this might take a long time. With 
> gzip compression content can be compressed to 850 KB.
> 
> This version of the patch uses a custom plugin to compress static resources, 
> so that no dynamic compression is needed at runtime. When the static resource 
> servlet notices that there is gzipped version of the resource and the client 
> accepts gzipped content, then it is going to send the gziped version, while 
> still respects http caching.
> 
> 
> Diffs
> -
> 
>   client/WEB-INF/web.xml e5c05d3 
>   client/pom.xml 119c96e 
>   server/src/com/cloud/servlet/StaticResourceServlet.java PRE-CREATION 
>   server/test/com/cloud/servlet/StaticResourceServletTest.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/12228/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Laszlo Hornyak
> 
>

Re: [HOW-TO] Custom REST call on Destroy and Deploy

[Chip Childers]

On Fri, Oct 4, 2013 at 4:15 PM, Musayev, Ilya  wrote:

> > On Fri, Oct 04, 2013 at 07:21:03PM +, Musayev, Ilya wrote:
> > > We would like to add additional operations in ACS when we do deploy and
> > destroy.
> > >
> > > For example, when VM is destroyed, we want to make create a JIRA ticket
> > (rest call) to track the decomision process.
> > > When VM is deployed, we may need to do something else, but also rest
> > based?
> > >
> > > I can see 2 ways of doing this,
> > >
> > >
> > > 1)  Extend the UI and initiate extra REST calls via java script
>  (low risk)
> > >
> > > 2)  Add more functionality to API Command (extra work and
> maintenance
> > burden)
> > >
> > > 3)  Create a Custom front end (seems like a lot of work)
> > >
> > >
> > > How can this be done most efficiently and hopefully with least
> intrusion?
> >
> > Have you considered using the RabbitMQ eventbus plugin, and tying into
> the
> > events that you want to act on that way?
>
> I see you made an awesome post on integrating RabbitMQ with ACS.
>
>
> http://www.chipchilders.com/blog/2013/7/16/tapping-into-apache-cloudstack-events-via-amqp.html
>
> You've made it a part of simulatorComponentContext.xml.in, however, in my
> case its going to be a production system with real events, where would you
> recommend this go?
>
> Thanks
> ilya
>
>
>
In the Component Context file that you use in prod.  The different build
targets pull in one of the different component contexts as the version to
use in the specific build.  I suppose that means (because I know you use
Vmware), that you'd put it in nonossComponentContext.xml.in.

RE: [HOW-TO] Custom REST call on Destroy and Deploy

[Musayev, Ilya]

> > On Fri, Oct 04, 2013 at 07:21:03PM +, Musayev, Ilya wrote:
> > > We would like to add additional operations in ACS when we do deploy
> > > and
> > destroy.
> > >
> > > For example, when VM is destroyed, we want to make create a JIRA
> > > ticket
> > (rest call) to track the decomision process.
> > > When VM is deployed, we may need to do something else, but also rest
> > based?
> > >
> > > I can see 2 ways of doing this,
> > >
> > >
> > > 1)  Extend the UI and initiate extra REST calls via java script  (low 
> > > risk)
> > >
> > > 2)  Add more functionality to API Command (extra work and
> maintenance
> > burden)
> > >
> > > 3)  Create a Custom front end (seems like a lot of work)
> > >
> > >
> > > How can this be done most efficiently and hopefully with least intrusion?
> >
> > Have you considered using the RabbitMQ eventbus plugin, and tying into
> > the events that you want to act on that way?
> 
> I see you made an awesome post on integrating RabbitMQ with ACS.
> 
> http://www.chipchilders.com/blog/2013/7/16/tapping-into-apache-
> cloudstack-events-via-amqp.html
> 
> You've made it a part of simulatorComponentContext.xml.in, however, in my
> case its going to be a production system with real events, where would you
> recommend this go?
> 
> Thanks
> ilya
> 
I'd think nonossComponentContext.xml, do I need to rebuild ACS? Or can I just 
plugin without rebuild?

Re: Review Request 12228: wip: static resource compression

[Laszlo Hornyak]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12228/
---

(Updated Oct. 4, 2013, 8:19 p.m.)


Review request for cloudstack, Brian Federle and Prasanna Santhanam.


Summary (updated)
-

wip: static resource compression


Repository: cloudstack-git


Description
---

CloudStack at first use downloads some 3.5 MB of css and javascript to the 
client. With a weak internet connection, this might take a long time. With gzip 
compression content can be compressed to 850 KB.

This version of the patch uses a custom plugin to compress static resources, so 
that no dynamic compression is needed at runtime. When the static resource 
servlet notices that there is gzipped version of the resource and the client 
accepts gzipped content, then it is going to send the gziped version, while 
still respects http caching.


Diffs
-

  client/WEB-INF/web.xml e5c05d3 
  client/pom.xml 119c96e 
  server/src/com/cloud/servlet/StaticResourceServlet.java PRE-CREATION 
  server/test/com/cloud/servlet/StaticResourceServletTest.java PRE-CREATION 

Diff: https://reviews.apache.org/r/12228/diff/


Testing
---


Thanks,

Laszlo Hornyak

RE: [HOW-TO] Custom REST call on Destroy and Deploy

[Musayev, Ilya]

> On Fri, Oct 04, 2013 at 07:21:03PM +, Musayev, Ilya wrote:
> > We would like to add additional operations in ACS when we do deploy and
> destroy.
> >
> > For example, when VM is destroyed, we want to make create a JIRA ticket
> (rest call) to track the decomision process.
> > When VM is deployed, we may need to do something else, but also rest
> based?
> >
> > I can see 2 ways of doing this,
> >
> >
> > 1)  Extend the UI and initiate extra REST calls via java script  (low 
> > risk)
> >
> > 2)  Add more functionality to API Command (extra work and maintenance
> burden)
> >
> > 3)  Create a Custom front end (seems like a lot of work)
> >
> >
> > How can this be done most efficiently and hopefully with least intrusion?
> 
> Have you considered using the RabbitMQ eventbus plugin, and tying into the
> events that you want to act on that way?

I see you made an awesome post on integrating RabbitMQ with ACS.

http://www.chipchilders.com/blog/2013/7/16/tapping-into-apache-cloudstack-events-via-amqp.html

You've made it a part of simulatorComponentContext.xml.in, however, in my case 
its going to be a production system with real events, where would you recommend 
this go?

Thanks
ilya

Re: Review Request 14477: Refactor Storage Related Resource Code

[edison su]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14477/#review26693
---

Ship it!


Ship It!

- edison su


On Oct. 4, 2013, 12:57 a.m., Chris Suich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14477/
> ---
> 
> (Updated Oct. 4, 2013, 12:57 a.m.)
> 
> 
> Review request for cloudstack.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> These changes are a joint effort between Edison and I to refactor some of the 
> code around snapshotting VM volumes and creating templates/volumes from VM 
> volume snapshots. In general, we were working towards allowing 
> PrimaryDataStoreDrivers to create snapshots on primary storage and not 
> requiring the snapshots to be transferred to secondary storage.
> 
> High level changes:
> -Added uuid to NfsTO, SwiftTO & S3TO to cut down on the requirement of 
> PrimaryDataStoreTO and ImageStoreTO which don't really serve much of a purpose
> -Initial work towards enable reverting VM volume from snapshots
> -Added hypervisor commands for introducing and forgetting new hypervisor 
> objects (snapshots, templates & volumes)
> 
> 
> Diffs
> -
> 
>   api/src/com/cloud/agent/api/to/DataStoreTO.java 9014f8e 
>   api/src/com/cloud/agent/api/to/NfsTO.java 415c95c 
>   api/src/com/cloud/agent/api/to/SwiftTO.java 7349d77 
>   api/src/com/cloud/event/EventTypes.java ec9604e 
>   api/src/com/cloud/storage/snapshot/SnapshotApiService.java 23e6522 
>   
> api/src/org/apache/cloudstack/api/command/admin/storage/ListStoragePoolsCmd.java
>  26351bb 
>   
> api/src/org/apache/cloudstack/api/command/user/snapshot/RevertSnapshotCmd.java
>  PRE-CREATION 
>   core/src/com/cloud/storage/resource/StorageProcessor.java 5fa9f8a 
>   core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java 
> ab9aa2a 
>   core/src/org/apache/cloudstack/storage/command/ForgetObjectCmd.java 
> PRE-CREATION 
>   core/src/org/apache/cloudstack/storage/command/IntroduceObjectAnswer.java 
> PRE-CREATION 
>   core/src/org/apache/cloudstack/storage/command/IntroduceObjectCmd.java 
> PRE-CREATION 
>   core/src/org/apache/cloudstack/storage/to/ImageStoreTO.java 0037ea5 
>   core/src/org/apache/cloudstack/storage/to/PrimaryDataStoreTO.java 5e870df 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/EndPointSelector.java
>  ca0cc2c 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/SnapshotService.java
>  d594a07 
>   
> engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/SnapshotStrategy.java
>  86ae532 
>   
> engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/AncientDataMotionStrategy.java
>  96d1f5a 
>   
> engine/storage/image/src/org/apache/cloudstack/storage/image/store/ImageStoreImpl.java
>  855d8cb 
>   
> engine/storage/integration-test/test/org/apache/cloudstack/storage/test/SnapshotTestWithFakeData.java
>  2aaabda 
>   
> engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/SnapshotServiceImpl.java
>  3ead93f 
>   
> engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/SnapshotStrategyBase.java
>  1b57922 
>   
> engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/XenserverSnapshotStrategy.java
>  60d9407 
>   
> engine/storage/src/org/apache/cloudstack/storage/endpoint/DefaultEndPointSelector.java
>  fdc12bf 
>   
> engine/storage/src/org/apache/cloudstack/storage/helper/HypervisorHelper.java 
> PRE-CREATION 
>   
> engine/storage/src/org/apache/cloudstack/storage/helper/HypervisorHelperImpl.java
>  PRE-CREATION 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java
>  82fd2ce 
>   
> plugins/hypervisors/simulator/src/com/cloud/resource/SimulatorStorageProcessor.java
>  c7768aa 
>   
> plugins/hypervisors/vmware/src/com/cloud/storage/resource/VmwareStorageProcessor.java
>  4982d87 
>   
> plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/XenServerStorageProcessor.java
>  739b974 
>   server/src/com/cloud/storage/snapshot/SnapshotManagerImpl.java 2297e6a 
>   
> services/secondary-storage/src/org/apache/cloudstack/storage/resource/NfsSecondaryStorageResource.java
>  3ef950b 
> 
> Diff: https://reviews.apache.org/r/14477/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Chris Suich
> 
>

Re: Review Request 14471: CloudStack Plugin guide for VXLAN

[Yoshikazu Nojima]

Hi Prasanna,
Thank you for let me know.
I'll resubmit the patch, but I cannot find the repository
cloudstack-docs.git in the ReviewBoard.
Do you know how to send the patch to the repository?

Regards,
Yoshikazu


2013/10/3 Prasanna Santhanam :
> Hi,
>
> The docs were broken out into a separate repo 2 days ago. So this
> should be based on the repo cloudstack-docs.git. Can you resubmit this
> for that repo?
>
> See:
> http://markmail.org/message/w44nioqt5usechhc
>
> Thanks,
>
>
> On Thu, Oct 03, 2013 at 08:20:52PM -, Yoshikazu Nojima wrote:
>>
>> ---
>> This is an automatically generated e-mail. To reply, visit:
>> https://reviews.apache.org/r/14471/
>> ---
>>
>> Review request for cloudstack and Toshiaki Hatano.
>>
>>
>> Repository: cloudstack-git
>>
>>
>> Description
>> ---
>>
>> This patch adds a Plugin guide for VXLAN.
>> It describes system requirements for VXLAN plugin and how to setup it.
>>
>> This is the ToC:
>> 1. The VXLAN Plugin  1
>> 1.1. Introduction to the VXLAN Plugin  1
>> 1.2. Features of the VXLAN Plugin  1
>> 2. System Requirements for VXLAN 3
>> 2.1. System Requirements for VXLAN  3
>> 2.2. Linux Distributions that meet the requirements  3
>> 2.3. Check the capability of your system  3
>> 2.4. Advanced: Build kernel and iproute2  3
>> 2.4.1. Build kernel  4
>> 2.4.2. Build iproute2  4
>> 3. Configure CloudStack to use VXLAN Plugin 7
>> 3.1. Configure hypervisor  7
>> 3.1.1. Configure hypervisor: KVM  7
>> 3.2. Setup zone using VXLAN  9
>> 3.2.1. Configure the physical network  10
>>
>>
>> Diffs
>> -
>>
>>   docs/en-US/CloudStack_VXLAN_Guide.ent PRE-CREATION
>>   docs/en-US/CloudStack_VXLAN_Guide.xml PRE-CREATION
>>   docs/en-US/images/vxlan-physicalnetwork.png PRE-CREATION
>>   docs/en-US/images/vxlan-trafficlabel.png PRE-CREATION
>>   docs/en-US/plugin-vxlan-about.xml PRE-CREATION
>>   docs/en-US/plugin-vxlan-config-hypervisor.xml PRE-CREATION
>>   docs/en-US/plugin-vxlan-config-management.xml PRE-CREATION
>>   docs/en-US/plugin-vxlan-configuration.xml PRE-CREATION
>>   docs/en-US/plugin-vxlan-features.xml PRE-CREATION
>>   docs/en-US/plugin-vxlan-introduction.xml PRE-CREATION
>>   docs/en-US/plugin-vxlan-requirements.xml PRE-CREATION
>>   docs/en-US/plugin-vxlan-revision-history.xml PRE-CREATION
>>   docs/publican-plugin-vxlan.cfg PRE-CREATION
>>
>> Diff: https://reviews.apache.org/r/14471/diff/
>>
>>
>> Testing
>> ---
>>
>>
>> Thanks,
>>
>> Yoshikazu Nojima
>>
>
> --
> Prasanna.,
>
> 
> Powered by BigRock.com
>

Re: Review Request 14289: unit test for KVMFencer

[Darren Shepherd]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14289/#review26691
---

Ship it!


master: c9f41d40465e3cc8bd775dcfdeb3fa1907fd1e7f

- Darren Shepherd


On Sept. 24, 2013, 7:52 p.m., Laszlo Hornyak wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14289/
> ---
> 
> (Updated Sept. 24, 2013, 7:52 p.m.)
> 
> 
> Review request for cloudstack, FrankXH FrankXH and Kelven Yang.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> A few cases covered with unit tests.
> 
> 
> Diffs
> -
> 
>   server/test/com/cloud/ha/KVMFencerTest.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/14289/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> Laszlo Hornyak
> 
>

Re: [HOW-TO] Custom REST call on Destroy and Deploy

[Chip Childers]

On Fri, Oct 04, 2013 at 07:21:03PM +, Musayev, Ilya wrote:
> We would like to add additional operations in ACS when we do deploy and 
> destroy.
> 
> For example, when VM is destroyed, we want to make create a JIRA ticket (rest 
> call) to track the decomision process.
> When VM is deployed, we may need to do something else, but also rest based?
> 
> I can see 2 ways of doing this,
> 
> 
> 1)  Extend the UI and initiate extra REST calls via java script  (low 
> risk)
> 
> 2)  Add more functionality to API Command (extra work and maintenance 
> burden)
> 
> 3)  Create a Custom front end (seems like a lot of work)
> 
> 
> How can this be done most efficiently and hopefully with least intrusion?

Have you considered using the RabbitMQ eventbus plugin, and tying into
the events that you want to act on that way?

[HOW-TO] Custom REST call on Destroy and Deploy

[Musayev, Ilya]

We would like to add additional operations in ACS when we do deploy and destroy.

For example, when VM is destroyed, we want to make create a JIRA ticket (rest 
call) to track the decomision process.
When VM is deployed, we may need to do something else, but also rest based?

I can see 2 ways of doing this,


1)  Extend the UI and initiate extra REST calls via java script  (low risk)

2)  Add more functionality to API Command (extra work and maintenance 
burden)

3)  Create a Custom front end (seems like a lot of work)


How can this be done most efficiently and hopefully with least intrusion?

Re: [DISCUSS] Components in JIRA and bug assignment

[Alena Prokharchyk]

On 10/4/13 10:37 AM, "Musayev, Ilya"  wrote:

>> On Fri, Oct 04, 2013 at 05:11:32PM +, Musayev, Ilya wrote:
>> > Question to JIRA experienced admins, we can preserve "assign to me"
>> option, and if unassigned goto "component" maintainer?
>> 
>> Absolutely.  Initial assignment does not equal the actual assignee.
>> Component-based assignment is just a way to skip the unassigned phase,
>> but people can reassign to themselves or others.
>> 
>> -chip
>
>Chip, thanks for the answer.
>
>So far, I've yet to see someone speaking negatively on this proposal. We
>do need  better structure - that will also help us being productive.
>
>Please kindly respond with +1, 0 or -1
>
>If -1, please explain why.
>
>Thanks
>ilya
>
>


+1

-Alena.

Re: [PROPOSAL] Modularize Spring

[Darren Shepherd]

On 10/04/2013 11:58 AM, SuichII, Christopher wrote:

Darren,

I think one of the benefits of allowing the priority to be specified in the xml 
is that it can be configured after deployment. If for some reason two 
strategies or providers conflict, then their priorities can be changed in XML 
to resolve the conflict. I believe the Spring @Order annotation an be specified 
in XML, not just as an annotation.

-Chris



I would *prefer* extensions to be order independent, but if we determine 
they are order dependant, then that is fine too.  So if we conclude that 
the simplest way to address this is to order the Strategies based on 
configuration, then I will add an ordering "global configuration" as 
described at 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions.


Does the order configuration setting approach seem fine?

Darren

RE: [DISCUSS] Components in JIRA and bug assignment

[Musayev, Ilya]

Response in line:

> -Original Message-
> From: Santhosh Edukulla [mailto:santhosh.eduku...@citrix.com]
> Sent: Friday, October 04, 2013 2:02 PM
> To: dev@cloudstack.apache.org
> Subject: RE: [DISCUSS] Components in JIRA and bug assignment
> 
> Not Sure, if it falls under similar lines\requires a separate thread. I am 
> trying
> to see if there is a way we can add  a component field, primary owner fields
> for component  available to be assigned for a given review under
> Http://reviews.apache.org.

While we want to extend JIRA and add various components to address current 
limitations, I'm not certain if this is in the same scope. I also have to 
admit, that I only use JIRA as basic user, so I don't know if what you are 
asking is doable. Perhaps someone else can?


> 
> This way i will select component as "Automation", branch as "4.1" and so etc ,
> save that query on my dashboard and everytime i login, i can see reviews
> based upon my query saved.

We can create a separate component Automation, so you can create filters as 
described.

> 
> If this facility is already there, then i will search more to find it out.
> 
> Regards,
> Santhosh
> 
> From: Chip Childers [chip.child...@sungard.com]
> Sent: Friday, October 04, 2013 1:45 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Components in JIRA and bug assignment
> 
> On Fri, Oct 04, 2013 at 05:40:06PM +, Animesh Chaturvedi wrote:
> >
> > [Animesh>] Chip does that mean you are open to folks assigning tickets
> > to others? Should this be posted in the original thread
> > http://markmail.org/thread/btovj6t6opqxge5q. because the threads
> where
> > Ilya reopened the discussion has received few responses from Alex,
> > Sheng, Alena, Sudha, RamG
> >
> 
> No, that just means that I was explaining the technical capabilities of Jira.
> 
> I'm -0, so move forward if everyone else continues to agree.  I still don't 
> think
> it's the right way to operate, but I'm not interested in blocking it if 
> everyone
> else wants to do it this way.
> 
> I won't reiterate my previous reasoning, since you pointed to that thread.
> 
> Cheers!
> 
> -chip

Re: [PROPOSAL] Modularize Spring

[Darren Shepherd]

On 10/04/2013 10:36 AM, Edison Su wrote:


How to use the way spring  4.0 did:
https://jira.springsource.org/browse/SPR-5574?
http://stackoverflow.com/questions/16967971/spring-ordered-list-of-beans
Add order in bean.



That is a spring specific API/interface.  In general we should never 
using Spring APIs.  Only bootstrap, initialization, or naturally Spring 
specific code should use Spring APIs.  Currently ACS does not use Spring 
APIs with the exception of @Component which is only there until I get 
around to cleaning up AWSAPI and Usage Server which use component 
scanning still.


Regardless, that is Spring 4.0, which we don't use, and the extension 
discovery now is not based on Injection in a way that the code in Spring 
4.0 would actually sort the beans.


Darren

Re: [PROPOSAL] Modularize Spring

[SuichII, Christopher]

Darren,

I think one of the benefits of allowing the priority to be specified in the xml 
is that it can be configured after deployment. If for some reason two 
strategies or providers conflict, then their priorities can be changed in XML 
to resolve the conflict. I believe the Spring @Order annotation an be specified 
in XML, not just as an annotation.

-Chris
-- 
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Oct 4, 2013, at 10:36 AM, Edison Su  wrote:

> 
> 
>> -Original Message-
>> From: SuichII, Christopher [mailto:chris.su...@netapp.com]
>> Sent: Thursday, October 03, 2013 9:02 PM
>> To: 
>> Subject: Re: [PROPOSAL] Modularize Spring
>> 
>> Sure - I could see that working. Anyone have thoughts whether an enum
>> could be used instead of an integer? That way we can provide categories or a
>> well defined scale (like 0-5)? If we give them free range of 1-100 (or any
>> integer range), I imagine people will likely go to the extremes and just use >> 0
>> for can't handle, 1 for low priority (default) and 100 for high priority 
>> (storage
>> providers). We still have the problem of handling conflicts, or implementers
>> who return the same value. However, I'm not sure there is much we can do
>> beyond selecting the first implementation that we come across with the
>> highest priority. We should document and suggest that implementations
>> ensure their canHandle() method is as intelligent as possible and only takes
>> control of operations they are truly authorities on.
> 
> How to use the way spring  4.0 did: 
> https://jira.springsource.org/browse/SPR-5574?
> http://stackoverflow.com/questions/16967971/spring-ordered-list-of-beans
> Add order in bean.
> 
>> 
>> -Chris
>> --
>> Chris Suich
>> chris.su...@netapp.com
>> NetApp Software Engineer
>> Data Center Platforms - Cloud Solutions
>> Citrix, Cisco & Red Hat
>> 
>> On Oct 3, 2013, at 7:44 PM, Darren Shepherd 
>> wrote:
>> 
>>> Could it be just as simple as the canhandle() returns an int and not 
>>> Boolean.
>> So the ancient would return 1 but if the netapp matches it returns 100.  If 
>> it
>> does not handle at all you return -1.  This effectively gives you 
>> prioritization.
>> So the calling code would still loop through all strategies each time 
>> looking for
>> the best match.  I don't want the priority to be determined at load time as
>> that is less flexible.
>>> 
>>> Darren
>>> 
 On Oct 3, 2013, at 5:42 PM, "SuichII, Christopher"
>>  wrote:
 
 Thanks for the quick reply.
 
 After talking with Edison, I think what we could do is allow the strategies
>> to either specify a priority or 'type' and then order them by that when they
>> are loaded. For example, we could have an enum of types like 'PLUGIN,
>> HYPERVISOR and DEFAULT' so that we can make sure plugins are asked
>> canHandle() first, then hypervisor implementations, then finally resort to 
>> the
>> default/ancient implementation. This 'type' or 'category' could be specified
>> as a bean parameter or as part of the strategy/provider interface. Any
>> thoughts on which are better?
 
 The problem with just making canHandle() more intelligent is that we do
>> need some kind of ordering. Ideally, the non-default implementations
>> should be asked first, then fall back onto the stock implementations. You saw
>> the problem yourself - the XenServerSnapshotStrategy just says it will handle
>> all requests, so if a non-standard strategy wants to be given a chance, it 
>> must
>> be asked before the hypervisor or ancient implementation.
 
 Alternatively if this matches the same usage of the global configuration
>> ordering system, we could add in the storage subsystem strategies and
>> providers.
 
 The reason for log4j changes is that we may want to enable a different
>> level of verbosity by default. Our support organization likes to have very
>> verbose logs to assist them in triaging support issues. The lowest level 
>> log4j
>> categories are 'com.cloud', 'org' and 'net' and those are set to DEBUG and
>> INFO. Even if we add a line for 'com', the default value should not be 
>> 'TRACE'
>> like we would like ours to be. I'm not all that great with log4j, though, so
>> maybe I'm missing a simpler solution.
 
 I'll try to keep an eye on the commands.properties/rbac stuff - that is
>> good to know.
 
 Thanks,
 Chris
 --
 Chris Suich
 chris.su...@netapp.com
 NetApp Software Engineer
 Data Center Platforms - Cloud Solutions Citrix, Cisco & Red Hat
 
> On Oct 3, 2013, at 4:30 PM, Darren Shepherd
>>  wrote:
> 
> I forgot to mention about canHandle() ordering.  For extensions, it
> should be preferred that we do not rely on loaded or configured order.
> The canHandle() implementation should assume that they're may be any
> order.  So having said that, I looked at XenServerSnapshotStrategy
> and it

Re: Review Request 12228: static resource compression

[Darren Shepherd]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12228/#review26690
---


I would prefer not to do this.  From a technical perspective the code uses 
getRealPath and Files which are not compatible across all serlvet containers 
and this implementation forces a lot of file system stat() calls, but that 
isn't my real complaint.  This change adds yet another step to the already long 
ACS build and I don't think it address the root issue.  The root issue, if one 
was to really care about performance is that the js is not cachable or 
consolidated.  ts=$now is added to every js file, meaning it must be downloaded 
every time.  Additionally to load the login screen there are 66 requests for JS 
files.  Even if you compress 2mb to 800k, on a slow connection it won't matter 
that much because you are going to get killed regardless by the round trip 
latency of making 66 requests.

- Darren Shepherd


On Sept. 24, 2013, 6:57 a.m., Laszlo Hornyak wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12228/
> ---
> 
> (Updated Sept. 24, 2013, 6:57 a.m.)
> 
> 
> Review request for cloudstack, Brian Federle and Prasanna Santhanam.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> CloudStack at first use downloads some 3.5 MB of css and javascript to the 
> client. With a weak internet connection, this might take a long time. With 
> gzip compression content can be compressed to 850 KB.
> 
> This version of the patch uses a custom plugin to compress static resources, 
> so that no dynamic compression is needed at runtime. When the static resource 
> servlet notices that there is gzipped version of the resource and the client 
> accepts gzipped content, then it is going to send the gziped version, while 
> still respects http caching.
> 
> 
> Diffs
> -
> 
>   client/WEB-INF/web.xml e5c05d3 
>   client/pom.xml 119c96e 
>   server/src/com/cloud/servlet/StaticResourceServlet.java PRE-CREATION 
>   server/test/com/cloud/servlet/StaticResourceServletTest.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/12228/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Laszlo Hornyak
> 
>

Re: Hypervisor Questions

[Mike Tutkowski]

Yeah, that's kind of what I was interested in learning about.

Now that we have zone-wide primary storage, does that mean CS is able to
issue the live migration of a VM from one cluster to another (or are we
still confined to clusters)?


On Fri, Oct 4, 2013 at 11:55 AM, Travis Graham  wrote:

> Was that a limitation caused by the primary storage only being available
> to a single cluster and not zone wide like 4.2.0 provides?
>
> Travis
>
> On Oct 4, 2013, at 1:52 PM, Mike Tutkowski 
> wrote:
>
> > Maybe this is a silly question, but if CS handles Live Migrations, are we
> > still constrained to migrating VMs from one host to another in the same
> > cluster?
> >
> > Same question for HA.
> >
> >
> > On Wed, Oct 2, 2013 at 8:42 AM, Clayton Weise 
> wrote:
> >
> >> AFAIK, no, but it's a great RFE that I would vote for.
> >>
> >> -Original Message-
> >> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
> >> Sent: Tuesday, October 01, 2013 9:26 PM
> >> To: dev@cloudstack.apache.org
> >> Subject: Re: Hypervisor Questions
> >>
> >> Oh, and, yes, when I referred to HA, it was (as you said) with the
> meaning
> >> of a host going offline and VMs being restarted on other hosts (perhaps
> in
> >> a prioritized order if there are an insufficient number of resources to
> >> support all of the VMs that were running on the downed host).
> >>
> >> Does CS support assigning a priority to a VM in case not all VMs can be
> >> restarted on the remaining resources?
> >>
> >>
> >> On Tue, Oct 1, 2013 at 6:35 PM, Mike Tutkowski <
> >> mike.tutkow...@solidfire.com
> >>> wrote:
> >>
> >>> Thanks, Clayton!
> >>>
> >>> Yeah, copy/paste mistake there. :) I meant it as you said.
> >>>
> >>>
> >>> On Tue, Oct 1, 2013 at 4:54 PM, Clayton Weise 
> >> wrote:
> >>>
>  First, I think you meant to put XenServer, KVM, and VMware and not
>  XenServer 3 times in a row.  That being said I think in all cases
>  (somebody correct me if I'm wrong here) it goes something like this:
> 
>  Live Migration: Request is made by CS but carried out by the HV.
>  High Availability: More accurately it's "recovery after host failure"
>  because it's still a disruptive action when a host goes sideways, but
>  by default this is handled by CS.  I _think_ there's an option to let
>  the HV handle this but I'm not totally sure.
>  DRS: Managed by CS through one of several methods with the global
>  setting vm.allocation.algorithm (see below)
> 
>  'random', 'firstfit', 'userdispersing', 'userconcentratedpod_random',
>  'userconcentratedpod_firstfit' : Order in which hosts within a
>  cluster will be considered for VM/volume allocation.
> 
>  That being said, after deployment there isn't any further DRS
>  monitoring; it's only done at the time an instance is instantiated.
> 
>  -Clayton
> 
>  -Original Message-
>  From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
>  Sent: Tuesday, October 01, 2013 3:00 PM
>  To: dev@cloudstack.apache.org
>  Subject: Hypervisor Questions
> 
>  Hi,
> 
>  I was wondering if people could clarify for me what CloudStack
>  manages versus what the hypervisor manages in terms of live
>  migration, high availability, and distributed resource scheduling?
> 
>  I know it is probably different for XenServer, VMware, and KVM.
> 
>  Can people fill in the info below (managed by the management server,
>  the hypervisor, or some combination of both)?
> 
>  XenServer
>    Live migration:
>    High availability:
>    Distributed Resource Scheduling:
> 
>  XenServer
>    Live migration:
>    High availability:
>    Distributed Resource Scheduling:
> 
>  XenServer
>    Live migration:
>    High availability:
>    Distributed Resource Scheduling:
> 
>  Thanks!
> 
>  --
>  *Mike Tutkowski*
>  *Senior CloudStack Developer, SolidFire Inc.*
>  e: mike.tutkow...@solidfire.com
>  o: 303.746.7302
>  Advancing the way the world uses the
>  cloud
>  *(tm)*
> 
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkow...@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud
> >>> *(tm)*
> >>>
> >>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkow...@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the
> >> cloud
> >> *(tm)*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkow...@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud

Re: Review Request 14405: ConstantTimeBackoff test and cleanup

[Darren Shepherd]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14405/#review26688
---

Ship it!


master: 826c69fd29bb98d3f631596f72a6eb3525d83aa2

- Darren Shepherd


On Oct. 1, 2013, 5:32 p.m., Laszlo Hornyak wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14405/
> ---
> 
> (Updated Oct. 1, 2013, 5:32 p.m.)
> 
> 
> Review request for cloudstack.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> - javadoc changed - the old one was copy-pasted from AgentShell
> - start and stop method removed - they did the same as the overridden methods
> - _counter removed as it was only written, but never read
> - remove from _asleep map was moved to a finally block, to make sure it is 
> removed even in case of the thread gets interrupted
> - Tests created for the above scenarios.
> 
> 
> Diffs
> -
> 
>   agent/src/com/cloud/agent/AgentShell.java bf1e818 
>   utils/src/com/cloud/utils/backoff/impl/ConstantTimeBackoff.java 976e369 
>   utils/test/com/cloud/utils/backoff/impl/ConstantTimeBackoffTest.java 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/14405/diff/
> 
> 
> Testing
> ---
> 
> test included
> 
> 
> Thanks,
> 
> Laszlo Hornyak
> 
>

Re: Review Request 14290: KVMFencer cleanup

[Darren Shepherd]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14290/#review26687
---


I'm fine with dropping the methods that are not needed, but I don't see any 
reason to really change fenceOff().  The changes seems to be more of a 
stylistic preference.  I feel both the original and the changed code are just 
as readable, and each have the same basic characteristic.  So the change for 
fenceOff() to me just seems like choosing one style over another.  Is there a 
specific bug or issue that is address with your changes that I'm not noticing.

- Darren Shepherd


On Sept. 24, 2013, 7:51 p.m., Laszlo Hornyak wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14290/
> ---
> 
> (Updated Sept. 24, 2013, 7:51 p.m.)
> 
> 
> Review request for cloudstack, FrankXH FrankXH and Kelven Yang.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> - start and stop removed, they do the same as the original method
> - host filtering logic moved to a the condition of the if statement
> - single default constructor removed
> 
> 
> Diffs
> -
> 
>   server/src/com/cloud/ha/KVMFencer.java 517209e 
> 
> Diff: https://reviews.apache.org/r/14290/diff/
> 
> 
> Testing
> ---
> 
> see https://reviews.apache.org/r/14289/
> 
> 
> Thanks,
> 
> Laszlo Hornyak
> 
>

Re: Review Request 14451: Some test for NetUtils

[Darren Shepherd]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14451/#review26684
---


It seems this patch includes a lot of line length based reformatting in the 
unit tests.  Can you narrow the patch down to just what is really needed to 
change.

- Darren Shepherd


On Oct. 4, 2013, 11:12 a.m., Laszlo Hornyak wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14451/
> ---
> 
> (Updated Oct. 4, 2013, 11:12 a.m.)
> 
> 
> Review request for cloudstack.
> 
> 
> Bugs: CLOUDSTACK-4807
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> - tests added for mac2Long, byte2Mac, long2Mac, ip2Long and long2Ip
> - isWindows static method removed - it was not used
> - minor changes in the implementations of the tested methods
> 
> 
> Diffs
> -
> 
>   utils/src/com/cloud/utils/net/NetUtils.java 1e72e22 
>   utils/test/com/cloud/utils/net/NetUtilsTest.java 3cfc98f 
> 
> Diff: https://reviews.apache.org/r/14451/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> Laszlo Hornyak
> 
>

Re: System VM

[Travis Graham]

From the perspective of a new community member being able to contribute to a 
separate repo that doesn't touch the app code is nice because that would open 
up the possibility of being a committer for the doc repo and separate that silo 
of work. Also less volatile in the fact that people, like me, who don't want to 
or can't mess with the actual Java side of the house and potentially having to 
deal with funky rebase issues and keeping up to date in the fast moving target 
state that the main code stays in.

There's talk of splitting the Release Manager out into separate roles with one 
head RM and sub RM's within their area of responsibility. I think this would 
help on that front as well. Having a lead Docs person (I think that's David?) 
who can oversee the whole process but can delegate things out to community 
members who are willing to take on the work.

That's my initial thoughts, I'm sure I could think of more advantages later 
once I work through more of the docs process. It's been interesting so far.

Travis

On Oct 4, 2013, at 1:55 PM, Chip Childers  wrote:

> On Fri, Oct 04, 2013 at 05:43:16PM +, Jessica Tomechak wrote:
>> Hi guys,
>> Not arguing against it, but I would be very much interested in your 
>> reasoning behind why having docs in a separate repo makes them easier to 
>> work on. What have we experienced since this time last year which has led us 
>> to reverse the original decision to keep docs in the same repo with code?
>> 
>> And having mentioned this, also thanks to y'all for taking care of doing the 
>> actual split and setting up the new repo.
>> 
>> Jessica T.
> 
> Documentation has a different lifecycle from the code, since docs aren't
> usually complete anywhere near feature complete.
> 
> Also, having it in a different repo will help contributors more easily
> work with the documentation.  We are seeing a number of new folks in the
> community that want to help on that front.  
> 
> -chip

Re: Review Request 13939: String join simplification

[Darren Shepherd]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13939/#review26682
---


I believe this is being blocked by 13938, can you please update to reflect 
that.  Once 13938 is resolved, I can commit this

- Darren Shepherd


On Sept. 3, 2013, 7:21 a.m., Laszlo Hornyak wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13939/
> ---
> 
> (Updated Sept. 3, 2013, 7:21 a.m.)
> 
> 
> Review request for cloudstack and David Nalley.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> - use commons-lang stringutils
> - added test to check compatibility
> 
> 
> Diffs
> -
> 
>   utils/src/com/cloud/utils/StringUtils.java 948c0ac 
>   utils/test/com/cloud/utils/StringUtilsTest.java ae37c24 
> 
> Diff: https://reviews.apache.org/r/13939/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Laszlo Hornyak
> 
>

Re: [DISCUSS] Components in JIRA and bug assignment

[Chip Childers]

On Fri, Oct 04, 2013 at 05:55:58PM +, Animesh Chaturvedi wrote:
> [Animesh>] Chip  I get the principle behind the reasoning in  thread[2] which 
> should be maintained but at the same time not being able to assign the issues 
> is a big hindrance in running a release. I am not sure if you saw my response 
> in this thread [1] started by Ilya which was based on my experience as RM for 
> 4.2.0 
>

I did see it.  I'm agreeing to disagree but not be a hindrance to
progress.  ;-)

If everyone else agrees with the approach, then rock on with it!

Re: Review Request 12134: removed txn attribute from DB annotation

[Darren Shepherd]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12134/#review26678
---

Ship it!


master: bd8536739c012e1025d35271ce82dbc9e5e66a8d

- Darren Shepherd


On Sept. 4, 2013, 4:55 p.m., Laszlo Hornyak wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12134/
> ---
> 
> (Updated Sept. 4, 2013, 4:55 p.m.)
> 
> 
> Review request for cloudstack, Alex Huang and Darren Shepherd.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> The txn attribute was used often with the DB annotation, but it does not have 
> an effect on the behavior of the code since it is never read.
> - All of this references to txn removed
> - Attribute removed from annotation
> - Documentation references removed
> 
> 
> Diffs
> -
> 
>   
> engine/orchestration/src/org/apache/cloudstack/engine/datacenter/entity/api/db/dao/EngineHostDaoImpl.java
>  5ba65cc 
>   engine/schema/src/com/cloud/certificate/dao/CertificateDaoImpl.java f071cea 
>   engine/schema/src/com/cloud/cluster/agentlb/dao/HostTransferMapDaoImpl.java 
> cff4cfc 
>   engine/schema/src/com/cloud/dc/dao/ClusterVSMMapDaoImpl.java b12fa9d 
>   engine/schema/src/com/cloud/dc/dao/DataCenterIpAddressDaoImpl.java 353402d 
>   engine/schema/src/com/cloud/dc/dao/DataCenterLinkLocalIpAddressDaoImpl.java 
> b52e373 
>   engine/schema/src/com/cloud/dc/dao/DataCenterVnetDaoImpl.java d3a2409 
>   engine/schema/src/com/cloud/dc/dao/StorageNetworkIpAddressDaoImpl.java 
> 782ee0d 
>   engine/schema/src/com/cloud/dc/dao/StorageNetworkIpRangeDaoImpl.java 
> d732e6f 
>   engine/schema/src/com/cloud/host/dao/HostDaoImpl.java 41ee7a7 
>   engine/schema/src/com/cloud/network/dao/AccountGuestVlanMapDaoImpl.java 
> e7a7b34 
>   engine/schema/src/com/cloud/network/dao/ExternalFirewallDeviceDaoImpl.java 
> 01f8861 
>   
> engine/schema/src/com/cloud/network/dao/ExternalLoadBalancerDeviceDaoImpl.java
>  ea6437d 
>   engine/schema/src/com/cloud/network/dao/FirewallRulesDaoImpl.java 41f911c 
>   engine/schema/src/com/cloud/network/dao/NetworkDaoImpl.java 0f83815 
>   engine/schema/src/com/cloud/network/dao/NetworkDomainDaoImpl.java bbb9203 
>   engine/schema/src/com/cloud/network/dao/NetworkExternalFirewallDaoImpl.java 
> b176760 
>   
> engine/schema/src/com/cloud/network/dao/NetworkExternalLoadBalancerDaoImpl.java
>  c29c164 
>   engine/schema/src/com/cloud/network/dao/NetworkServiceMapDaoImpl.java 
> 3cdd738 
>   engine/schema/src/com/cloud/network/dao/PhysicalNetworkDaoImpl.java 1e26a51 
>   
> engine/schema/src/com/cloud/network/dao/PhysicalNetworkServiceProviderDaoImpl.java
>  16a23dd 
>   
> engine/schema/src/com/cloud/network/dao/PhysicalNetworkTrafficTypeDaoImpl.java
>  7e47239 
>   engine/schema/src/com/cloud/network/dao/PortProfileDaoImpl.java 61fe52a 
>   engine/schema/src/com/cloud/network/dao/VirtualRouterProviderDaoImpl.java 
> dba835f 
>   engine/schema/src/com/cloud/network/vpc/dao/PrivateIpDaoImpl.java fe435c0 
>   engine/schema/src/com/cloud/network/vpc/dao/StaticRouteDaoImpl.java 518237d 
>   engine/schema/src/com/cloud/network/vpc/dao/VpcDaoImpl.java 6560b90 
>   engine/schema/src/com/cloud/network/vpc/dao/VpcGatewayDaoImpl.java 13c37c4 
>   engine/schema/src/com/cloud/network/vpc/dao/VpcOfferingDaoImpl.java 2cda547 
>   
> engine/schema/src/com/cloud/network/vpc/dao/VpcOfferingServiceMapDaoImpl.java 
> 4b5f1b9 
>   engine/schema/src/com/cloud/network/vpc/dao/VpcServiceMapDaoImpl.java 
> a992181 
>   engine/schema/src/com/cloud/offerings/dao/NetworkOfferingDaoImpl.java 
> ef8237a 
>   
> engine/schema/src/com/cloud/offerings/dao/NetworkOfferingServiceMapDaoImpl.java
>  7282443 
>   engine/schema/src/com/cloud/service/dao/ServiceOfferingDaoImpl.java 14b2abf 
>   engine/schema/src/com/cloud/storage/dao/StoragePoolWorkDaoImpl.java 052bae4 
>   engine/schema/src/com/cloud/storage/dao/VolumeDaoImpl.java bf28410 
>   engine/schema/src/com/cloud/upgrade/dao/VersionDaoImpl.java 7c5c9cc 
>   engine/schema/src/com/cloud/vm/dao/UserVmCloneSettingDaoImpl.java 174f283 
>   
> engine/schema/src/org/apache/cloudstack/region/gslb/GlobalLoadBalancerLbRuleMapDaoImpl.java
>  2a6e72b 
>   
> engine/schema/src/org/apache/cloudstack/storage/datastore/db/PrimaryDataStoreDaoImpl.java
>  b39f844 
>   framework/db/src/com/cloud/utils/db/DB.java f83a7ea 
>   framework/db/src/com/cloud/utils/db/GenericDaoBase.java 94e5374 
>   
> plugins/hypervisors/baremetal/src/com/cloud/baremetal/database/BaremetalDhcpDaoImpl.java
>  8123ee0 
>   
> plugins/hypervisors/baremetal/src/com/cloud/baremetal/database/BaremetalPxeDaoImpl.java
>  acd7f13 
>   plugins/hypervisors/ucs/src/com/cloud/ucs/database/UcsBladeDaoImpl.java 
> 5cb9b02 
>   plugins/hypervisors/ucs/src/com/cloud/ucs/database/U

RE: [DISCUSS] Components in JIRA and bug assignment

[Santhosh Edukulla]

Not Sure, if it falls under similar lines\requires a separate thread. I am 
trying to see if there is a way we can add  a component field, primary owner 
fields for component  available to be assigned for a given review under 
Http://reviews.apache.org. 

This way i will select component as "Automation", branch as "4.1" and so etc , 
save that query on my dashboard and everytime i login, i can see reviews based 
upon my query saved.

If this facility is already there, then i will search more to find it out.

Regards,
Santhosh

From: Chip Childers [chip.child...@sungard.com]
Sent: Friday, October 04, 2013 1:45 PM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Components in JIRA and bug assignment

On Fri, Oct 04, 2013 at 05:40:06PM +, Animesh Chaturvedi wrote:
>
> [Animesh>] Chip does that mean you are open to folks assigning tickets to 
> others? Should this be posted in the original thread 
> http://markmail.org/thread/btovj6t6opqxge5q. because the threads where Ilya 
> reopened the discussion has received few responses from Alex, Sheng, Alena, 
> Sudha, RamG
>

No, that just means that I was explaining the technical capabilities of
Jira.

I'm -0, so move forward if everyone else continues to agree.  I still
don't think it's the right way to operate, but I'm not interested in
blocking it if everyone else wants to do it this way.

I won't reiterate my previous reasoning, since you pointed to that
thread.

Cheers!

-chip

Re: Hypervisor Questions

[Travis Graham]

Was that a limitation caused by the primary storage only being available to a 
single cluster and not zone wide like 4.2.0 provides?

Travis

On Oct 4, 2013, at 1:52 PM, Mike Tutkowski  wrote:

> Maybe this is a silly question, but if CS handles Live Migrations, are we
> still constrained to migrating VMs from one host to another in the same
> cluster?
> 
> Same question for HA.
> 
> 
> On Wed, Oct 2, 2013 at 8:42 AM, Clayton Weise  wrote:
> 
>> AFAIK, no, but it's a great RFE that I would vote for.
>> 
>> -Original Message-
>> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
>> Sent: Tuesday, October 01, 2013 9:26 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: Hypervisor Questions
>> 
>> Oh, and, yes, when I referred to HA, it was (as you said) with the meaning
>> of a host going offline and VMs being restarted on other hosts (perhaps in
>> a prioritized order if there are an insufficient number of resources to
>> support all of the VMs that were running on the downed host).
>> 
>> Does CS support assigning a priority to a VM in case not all VMs can be
>> restarted on the remaining resources?
>> 
>> 
>> On Tue, Oct 1, 2013 at 6:35 PM, Mike Tutkowski <
>> mike.tutkow...@solidfire.com
>>> wrote:
>> 
>>> Thanks, Clayton!
>>> 
>>> Yeah, copy/paste mistake there. :) I meant it as you said.
>>> 
>>> 
>>> On Tue, Oct 1, 2013 at 4:54 PM, Clayton Weise 
>> wrote:
>>> 
 First, I think you meant to put XenServer, KVM, and VMware and not
 XenServer 3 times in a row.  That being said I think in all cases
 (somebody correct me if I'm wrong here) it goes something like this:
 
 Live Migration: Request is made by CS but carried out by the HV.
 High Availability: More accurately it's "recovery after host failure"
 because it's still a disruptive action when a host goes sideways, but
 by default this is handled by CS.  I _think_ there's an option to let
 the HV handle this but I'm not totally sure.
 DRS: Managed by CS through one of several methods with the global
 setting vm.allocation.algorithm (see below)
 
 'random', 'firstfit', 'userdispersing', 'userconcentratedpod_random',
 'userconcentratedpod_firstfit' : Order in which hosts within a
 cluster will be considered for VM/volume allocation.
 
 That being said, after deployment there isn't any further DRS
 monitoring; it's only done at the time an instance is instantiated.
 
 -Clayton
 
 -Original Message-
 From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
 Sent: Tuesday, October 01, 2013 3:00 PM
 To: dev@cloudstack.apache.org
 Subject: Hypervisor Questions
 
 Hi,
 
 I was wondering if people could clarify for me what CloudStack
 manages versus what the hypervisor manages in terms of live
 migration, high availability, and distributed resource scheduling?
 
 I know it is probably different for XenServer, VMware, and KVM.
 
 Can people fill in the info below (managed by the management server,
 the hypervisor, or some combination of both)?
 
 XenServer
   Live migration:
   High availability:
   Distributed Resource Scheduling:
 
 XenServer
   Live migration:
   High availability:
   Distributed Resource Scheduling:
 
 XenServer
   Live migration:
   High availability:
   Distributed Resource Scheduling:
 
 Thanks!
 
 --
 *Mike Tutkowski*
 *Senior CloudStack Developer, SolidFire Inc.*
 e: mike.tutkow...@solidfire.com
 o: 303.746.7302
 Advancing the way the world uses the
 cloud
 *(tm)*
 
>>> 
>>> 
>>> 
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkow...@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud
>>> *(tm)*
>>> 
>> 
>> 
>> 
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkow...@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the
>> cloud
>> *(tm)*
>> 
> 
> 
> 
> -- 
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud
> *™*

Re: System VM

[Chip Childers]

On Fri, Oct 04, 2013 at 05:43:16PM +, Jessica Tomechak wrote:
> Hi guys,
> Not arguing against it, but I would be very much interested in your reasoning 
> behind why having docs in a separate repo makes them easier to work on. What 
> have we experienced since this time last year which has led us to reverse the 
> original decision to keep docs in the same repo with code?
> 
> And having mentioned this, also thanks to y'all for taking care of doing the 
> actual split and setting up the new repo.
> 
> Jessica T.

Documentation has a different lifecycle from the code, since docs aren't
usually complete anywhere near feature complete.

Also, having it in a different repo will help contributors more easily
work with the documentation.  We are seeing a number of new folks in the
community that want to help on that front.  

-chip

RE: [DISCUSS] Components in JIRA and bug assignment

[Animesh Chaturvedi]

> -Original Message-
> From: Chip Childers [mailto:chip.child...@sungard.com]
> Sent: Friday, October 04, 2013 10:45 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Components in JIRA and bug assignment
> 
> On Fri, Oct 04, 2013 at 05:40:06PM +, Animesh Chaturvedi wrote:
> >
> > [Animesh>] Chip does that mean you are open to folks assigning tickets
> > to others? Should this be posted in the original thread
> > http://markmail.org/thread/btovj6t6opqxge5q. because the threads where
> > Ilya reopened the discussion has received few responses from Alex,
> > Sheng, Alena, Sudha, RamG
> >
> 
> No, that just means that I was explaining the technical capabilities of
> Jira.
> 
> I'm -0, so move forward if everyone else continues to agree.  I still
> don't think it's the right way to operate, but I'm not interested in
> blocking it if everyone else wants to do it this way.
> 
> I won't reiterate my previous reasoning, since you pointed to that
> thread.
> 
> Cheers!
> 
> -chip
[Animesh>] Chip  I get the principle behind the reasoning in  thread[2] which 
should be maintained but at the same time not being able to assign the issues 
is a big hindrance in running a release. I am not sure if you saw my response 
in this thread [1] started by Ilya which was based on my experience as RM for 
4.2.0 


I am +1 on the proposal

[1] http://markmail.org/thread/cwxxxjnkor6trq4s 
[2] http://markmail.org/thread/btovj6t6opqxge5q

FWD: [IP CLEARANCE] Contrail Plugin for CloudStack

[Chip Childers]

FYI - We have received the SGA for this plugin, and the 72 hours of lazy
consensus over at general@incubator.a.o has started.

I'll update this thread when we have cleared that hurdle, allowing us to
accept the patch into the repo.

-chip

- Forwarded message from Chip Childers  -

Date: Fri, 4 Oct 2013 13:50:47 -0400
From: Chip Childers 
To: gene...@incubator.apache.org
Subject: [IP CLEARANCE] Contrail Plugin for CloudStack
Message-ID: <20131004175047.gg88...@uslt-205755.sungardas.corp>

Juniper Networks, Inc has dontated a plugin for Apache CloudStack to
work with their Contrail networking software.

IP Clearance Document:

http://incubator.apache.org/ip-clearance/cloudstack-contrail-plugin.html

Please vote to approve this contribution.  Lazy consensus applies: if no
-1 votes are cast within the next 72 hours, the vote passes.

Thanks.

- End forwarded message -

Re: Hypervisor Questions

[Mike Tutkowski]

Maybe this is a silly question, but if CS handles Live Migrations, are we
still constrained to migrating VMs from one host to another in the same
cluster?

Same question for HA.


On Wed, Oct 2, 2013 at 8:42 AM, Clayton Weise  wrote:

> AFAIK, no, but it's a great RFE that I would vote for.
>
> -Original Message-
> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
> Sent: Tuesday, October 01, 2013 9:26 PM
> To: dev@cloudstack.apache.org
> Subject: Re: Hypervisor Questions
>
> Oh, and, yes, when I referred to HA, it was (as you said) with the meaning
> of a host going offline and VMs being restarted on other hosts (perhaps in
> a prioritized order if there are an insufficient number of resources to
> support all of the VMs that were running on the downed host).
>
> Does CS support assigning a priority to a VM in case not all VMs can be
> restarted on the remaining resources?
>
>
> On Tue, Oct 1, 2013 at 6:35 PM, Mike Tutkowski <
> mike.tutkow...@solidfire.com
> > wrote:
>
> > Thanks, Clayton!
> >
> > Yeah, copy/paste mistake there. :) I meant it as you said.
> >
> >
> > On Tue, Oct 1, 2013 at 4:54 PM, Clayton Weise 
> wrote:
> >
> >> First, I think you meant to put XenServer, KVM, and VMware and not
> >> XenServer 3 times in a row.  That being said I think in all cases
> >> (somebody correct me if I'm wrong here) it goes something like this:
> >>
> >> Live Migration: Request is made by CS but carried out by the HV.
> >> High Availability: More accurately it's "recovery after host failure"
> >> because it's still a disruptive action when a host goes sideways, but
> >> by default this is handled by CS.  I _think_ there's an option to let
> >> the HV handle this but I'm not totally sure.
> >> DRS: Managed by CS through one of several methods with the global
> >> setting vm.allocation.algorithm (see below)
> >>
> >> 'random', 'firstfit', 'userdispersing', 'userconcentratedpod_random',
> >> 'userconcentratedpod_firstfit' : Order in which hosts within a
> >> cluster will be considered for VM/volume allocation.
> >>
> >> That being said, after deployment there isn't any further DRS
> >> monitoring; it's only done at the time an instance is instantiated.
> >>
> >> -Clayton
> >>
> >> -Original Message-
> >> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
> >> Sent: Tuesday, October 01, 2013 3:00 PM
> >> To: dev@cloudstack.apache.org
> >> Subject: Hypervisor Questions
> >>
> >> Hi,
> >>
> >> I was wondering if people could clarify for me what CloudStack
> >> manages versus what the hypervisor manages in terms of live
> >> migration, high availability, and distributed resource scheduling?
> >>
> >> I know it is probably different for XenServer, VMware, and KVM.
> >>
> >> Can people fill in the info below (managed by the management server,
> >> the hypervisor, or some combination of both)?
> >>
> >> XenServer
> >>Live migration:
> >>High availability:
> >>Distributed Resource Scheduling:
> >>
> >> XenServer
> >>Live migration:
> >>High availability:
> >>Distributed Resource Scheduling:
> >>
> >> XenServer
> >>Live migration:
> >>High availability:
> >>Distributed Resource Scheduling:
> >>
> >> Thanks!
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkow...@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the
> >> cloud
> >> *(tm)*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkow...@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud
> > *(tm)*
> >
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud
> *(tm)*
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

Re: [DISCUSS] Components in JIRA and bug assignment

[Chip Childers]

On Fri, Oct 04, 2013 at 05:40:06PM +, Animesh Chaturvedi wrote:
> 
> [Animesh>] Chip does that mean you are open to folks assigning tickets to 
> others? Should this be posted in the original thread 
> http://markmail.org/thread/btovj6t6opqxge5q. because the threads where Ilya 
> reopened the discussion has received few responses from Alex, Sheng, Alena, 
> Sudha, RamG
>

No, that just means that I was explaining the technical capabilities of
Jira.

I'm -0, so move forward if everyone else continues to agree.  I still
don't think it's the right way to operate, but I'm not interested in
blocking it if everyone else wants to do it this way.

I won't reiterate my previous reasoning, since you pointed to that
thread.

Cheers!

-chip

RE: System VM

[Jessica Tomechak]

Hi guys,
Not arguing against it, but I would be very much interested in your reasoning 
behind why having docs in a separate repo makes them easier to work on. What 
have we experienced since this time last year which has led us to reverse the 
original decision to keep docs in the same repo with code?

And having mentioned this, also thanks to y'all for taking care of doing the 
actual split and setting up the new repo.

Jessica T.

From: David Nalley [da...@gnsa.us]
Sent: Wednesday, October 02, 2013 4:38 PM
To: dev@cloudstack.apache.org
Subject: Re: System VM

On Wed, Oct 2, 2013 at 7:34 PM, Travis Graham  wrote:
> Hey Marcus,
>
> The docs were split out today to make them easier to work on. I'll be going
> through them later tonight to fix things like this and sending out a diff
> to get them corrected as soon as possible.
>
> Travis

Hi Travis:

Lets try and make the patches small - perhaps one per file, or one per
several files, but not omnibus. It will be much easier to parse, and
apply, especially if something doesn't build cleanly.

--David

RE: [DISCUSS] Components in JIRA and bug assignment

[Animesh Chaturvedi]

> -Original Message-
> From: Chip Childers [mailto:chip.child...@sungard.com]
> Sent: Friday, October 04, 2013 10:28 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Components in JIRA and bug assignment
> 
> On Fri, Oct 04, 2013 at 05:11:32PM +, Musayev, Ilya wrote:
> > Question to JIRA experienced admins, we can preserve "assign to me"
> option, and if unassigned goto "component" maintainer?
> 
> Absolutely.  Initial assignment does not equal the actual assignee.
> Component-based assignment is just a way to skip the unassigned phase,
> but people can reassign to themselves or others.
> 
> -chip
[Animesh>] Chip does that mean you are open to folks assigning tickets to 
others? Should this be posted in the original thread 
http://markmail.org/thread/btovj6t6opqxge5q. because the threads where Ilya 
reopened the discussion has received few responses from Alex, Sheng, Alena, 
Sudha, RamG

RE: [DISCUSS] Components in JIRA and bug assignment

[Musayev, Ilya]

> On Fri, Oct 04, 2013 at 05:11:32PM +, Musayev, Ilya wrote:
> > Question to JIRA experienced admins, we can preserve "assign to me"
> option, and if unassigned goto "component" maintainer?
> 
> Absolutely.  Initial assignment does not equal the actual assignee.
> Component-based assignment is just a way to skip the unassigned phase,
> but people can reassign to themselves or others.
> 
> -chip

Chip, thanks for the answer.

So far, I've yet to see someone speaking negatively on this proposal. We do 
need  better structure - that will also help us being productive.

Please kindly respond with +1, 0 or -1

If -1, please explain why. 

Thanks
ilya

RE: [PROPOSAL] Modularize Spring

[Edison Su]

> -Original Message-
> From: SuichII, Christopher [mailto:chris.su...@netapp.com]
> Sent: Thursday, October 03, 2013 9:02 PM
> To: 
> Subject: Re: [PROPOSAL] Modularize Spring
> 
> Sure - I could see that working. Anyone have thoughts whether an enum
> could be used instead of an integer? That way we can provide categories or a
> well defined scale (like 0-5)? If we give them free range of 1-100 (or any
> integer range), I imagine people will likely go to the extremes and just use 0
> for can't handle, 1 for low priority (default) and 100 for high priority 
> (storage
> providers). We still have the problem of handling conflicts, or implementers
> who return the same value. However, I'm not sure there is much we can do
> beyond selecting the first implementation that we come across with the
> highest priority. We should document and suggest that implementations
> ensure their canHandle() method is as intelligent as possible and only takes
> control of operations they are truly authorities on.

How to use the way spring  4.0 did: 
https://jira.springsource.org/browse/SPR-5574?
http://stackoverflow.com/questions/16967971/spring-ordered-list-of-beans
Add order in bean.

> 
> -Chris
> --
> Chris Suich
> chris.su...@netapp.com
> NetApp Software Engineer
> Data Center Platforms - Cloud Solutions
> Citrix, Cisco & Red Hat
> 
> On Oct 3, 2013, at 7:44 PM, Darren Shepherd 
> wrote:
> 
> > Could it be just as simple as the canhandle() returns an int and not 
> > Boolean.
> So the ancient would return 1 but if the netapp matches it returns 100.  If it
> does not handle at all you return -1.  This effectively gives you 
> prioritization.
> So the calling code would still loop through all strategies each time looking 
> for
> the best match.  I don't want the priority to be determined at load time as
> that is less flexible.
> >
> > Darren
> >
> >> On Oct 3, 2013, at 5:42 PM, "SuichII, Christopher"
>  wrote:
> >>
> >> Thanks for the quick reply.
> >>
> >> After talking with Edison, I think what we could do is allow the strategies
> to either specify a priority or 'type' and then order them by that when they
> are loaded. For example, we could have an enum of types like 'PLUGIN,
> HYPERVISOR and DEFAULT' so that we can make sure plugins are asked
> canHandle() first, then hypervisor implementations, then finally resort to the
> default/ancient implementation. This 'type' or 'category' could be specified
> as a bean parameter or as part of the strategy/provider interface. Any
> thoughts on which are better?
> >>
> >> The problem with just making canHandle() more intelligent is that we do
> need some kind of ordering. Ideally, the non-default implementations
> should be asked first, then fall back onto the stock implementations. You saw
> the problem yourself - the XenServerSnapshotStrategy just says it will handle
> all requests, so if a non-standard strategy wants to be given a chance, it 
> must
> be asked before the hypervisor or ancient implementation.
> >>
> >> Alternatively if this matches the same usage of the global configuration
> ordering system, we could add in the storage subsystem strategies and
> providers.
> >>
> >> The reason for log4j changes is that we may want to enable a different
> level of verbosity by default. Our support organization likes to have very
> verbose logs to assist them in triaging support issues. The lowest level log4j
> categories are 'com.cloud', 'org' and 'net' and those are set to DEBUG and
> INFO. Even if we add a line for 'com', the default value should not be 'TRACE'
> like we would like ours to be. I'm not all that great with log4j, though, so
> maybe I'm missing a simpler solution.
> >>
> >> I'll try to keep an eye on the commands.properties/rbac stuff - that is
> good to know.
> >>
> >> Thanks,
> >> Chris
> >> --
> >> Chris Suich
> >> chris.su...@netapp.com
> >> NetApp Software Engineer
> >> Data Center Platforms - Cloud Solutions Citrix, Cisco & Red Hat
> >>
> >>> On Oct 3, 2013, at 4:30 PM, Darren Shepherd
>  wrote:
> >>>
> >>> I forgot to mention about canHandle() ordering.  For extensions, it
> >>> should be preferred that we do not rely on loaded or configured order.
> >>> The canHandle() implementation should assume that they're may be any
> >>> order.  So having said that, I looked at XenServerSnapshotStrategy
> >>> and its canHandle() is brilliantly implemented as "return true;"
> >>>
> >>> Can we look at making the strategies order independent and not
> >>> having another order configuration parameter?
> >>>
> >>> Darren
> >>>
> >>> On Thu, Oct 3, 2013 at 4:26 PM, Darren Shepherd
> >>>  wrote:
>  Chris,
> 
>  I've updated the wiki [1].  Copying from the wiki
> 
>  "Extensions are automatically discovered based on the interfaces
>  they implement and which module is their parent. For example, if
>  you place a storage extension in a child module of the network
>  module, it will not be discovered. Additionally, depending on t

[PROPOSAL] Remove Setters from *JoinVO

[SuichII, Christopher]

*JoinVOs are used to store entries from MySQL views, which are not editable. I 
think removing setters from the *JoinVOs may help avoid some potential 
confusion as setters seem to imply that the fields are editable, which they 
really aren't.

I started looking around and it looks like most setters in *JoinVOs aren't 
actually used since the creation of *VOs is handled by java reflection. Please 
let me know if this is not the case or if I'm misunderstanding the way the 
MySQL views work.

-Chris
-- 
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

Re: [DISCUSS] Components in JIRA and bug assignment

[Chip Childers]

On Fri, Oct 04, 2013 at 05:11:32PM +, Musayev, Ilya wrote:
> Question to JIRA experienced admins, we can preserve "assign to me" option, 
> and if unassigned goto "component" maintainer?

Absolutely.  Initial assignment does not equal the actual assignee.
Component-based assignment is just a way to skip the unassigned phase,
but people can reassign to themselves or others.

-chip

Re: Removal of cloud-plugin-netapp

[SuichII, Christopher]

Unfortunately, that is out of scope for our plans right now. If anyone knows 
who originally asked for this functionality or who is still using it, we would 
be interested in having discussions around the use cases these APIs are 
addressing and then looking into bringing that functionality into our current 
integration effort (for a future release).

-Chris
-- 
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Oct 2, 2013, at 7:14 AM, Chip Childers  wrote:

> As an alternative to removing the code, Christopher - would you be
> willing / able to consider fixing it up / updating it's dependencies /
> finding a way to help get support for maintaining it?
> 
> On Mon, Sep 23, 2013 at 05:42:03PM +, Chiradeep Vittal wrote:
>> If you remove it, it will break the following APIs:
>> 
>> createPool
>> createVolumeOnFiler
>> createLunOnFiler
>> AssociateLun
>> 
>> Here's how it is used:
>> The steps are
>> 
>> 1. create a volume pool using the createPool
>> 
>> API (algorithm can be "leastfull" or "roundrobin")
>> 2. add volumes to the pool using the createVolumeOnFiler
>> > r.html> API. This requires the IP address of the NetApp filer and its
>> credentials
>> 3. create LUNs in the volume pool using createLunOnFiler
>> > tm>  API. Depending on the algorithm for the pool, CloudStack will choose
>> the NetApp Volume to create the LUN.
>> 
>> 
>> Once a bare metal host / vm has booted up, you need to determine its ISCSI
>> initiator iqn and then use the
>> AssociateLUN 
>> 
>> API set the LUN mask on the filer for that LUN. Then the ISCSI initiator
>> on the bare metal host / vm can login to the LUN.
>> 
>> 
>> 
>> On 9/23/13 10:32 AM, "Chip Childers"  wrote:
>> 
>>> On Mon, Sep 23, 2013 at 04:44:51PM +, SuichII, Christopher wrote:
 I'd like to get a final word on this - is it OK to be removed? Is there
 someone who is knowledgable with what it will take to remove it and/or
 is comfortable removing it? I don't know how tied into the rest of the
 codebase it is.
>>> 
>>> I'm +1 to remove, but there are artifacts all over the place (UI, DB,
>>> code).  It might require some careful work to extract it cleanly.
>>> 
>>> Any takers?
>>> 
>>> -chip
>> 
>> 

Re: Review Request 13938: StringUtils csv methods simplified

[Darren Shepherd]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13938/#review26676
---



utils/src/com/cloud/utils/StringUtils.java


This is not actually correct, the previous code would do a trim() and the 
new code also can introduce a NPE.  So check for null and return empty list, 
also the following snippet deals well with white space

tags.trim().split("\\s*,\\s*")


- Darren Shepherd


On Sept. 3, 2013, 7:22 a.m., Laszlo Hornyak wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13938/
> ---
> 
> (Updated Sept. 3, 2013, 7:22 a.m.)
> 
> 
> Review request for cloudstack and Prachi Damle.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> - longer method bodies replaced with oneliners
> - tests added to ensure compatibility
> 
> 
> Diffs
> -
> 
>   utils/src/com/cloud/utils/StringUtils.java 948c0ac 
>   utils/test/com/cloud/utils/StringUtilsTest.java ae37c24 
> 
> Diff: https://reviews.apache.org/r/13938/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Laszlo Hornyak
> 
>

Re: XenServer Question

[Nitin Mehta]

What is your storage over provisioning factor ? Even that could be the
reason.
Regarding allocated vs used capacity I had written a wiki[1] sometime
back. Please see if it helps.

[1] 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Understanding+Dashbo
ard


Thanks,
-Nitin

On 04/10/13 8:08 AM, "benoit lair"  wrote:

>Effectively, it's very surprising.
>
>I checked my differents storage (local as iscsi sr) on xcp 1.0, xcp 1.1,
>xcp 1.4.90, xcp 1.6, xenserver 6.0 and xenserver 6.2, i observe that i can
>have a 'used' storage more important that the 'allocated' one
>
>I verified connected to my hypervisors shells and verified the data
>displayed in xencenter, the value 'used' is the good one.
>
>However, i can have allocated 200GB on my local SR to one vm (so it's
>provisionning). I can have 33GB 'used' on my local SR, but i have in
>reality less storage available into my vm.
>
>More clearly, if 200GB allocated on one vm, if xencenter says me 33GB are
>used, i can retrieve a vhd file corresponding to my vm of a size of 33GB,
>but when i'm connected to the vm's shell, i don't retrieve my 33gbs of
>storage.
>
>I observe the same phenomenon with my others xenserver hypervisors.
>
>It seems that some metadatas could be present into the vhd file + the real
>vm storage.
>
>
>
>Is there any citrix xenserver specialist in the room to confirm (or not)
>what i said ?
>
>
>Thanks.
>
>Regards, Benoit.
>
>
>2013/10/4 Mike Tutkowski 
>
>> Doh! I guess that would have been helpful info. :)
>>
>> It is in XenCenter. If you click on an SR and look at the General tab.
>>
>> http://i.imgur.com/z5F1Ct0.png
>>
>>
>> On Fri, Oct 4, 2013 at 4:34 AM, benoit lair 
>>wrote:
>>
>> > Hi Mike,
>> >
>> > Where do you see this information ? On the CS dashboard ? Or with
>> > xencenter, listing the sr with the windows gui ?
>> >
>> > Regards, Benoit.
>> >
>> >
>> > 2013/10/3 Mike Tutkowski 
>> >
>> > > Hi,
>> > >
>> > > On one of my SRs, I see the following information:
>> > >
>> > > 100.2 GB used of 200 GB total (100 GB allocated)
>> > >
>> > > Can someone explain how 'used' can be larger than 'allocated'?
>> > >
>> > > On storage systems I've worked on before, 'allocated' is generally
>> > > synonymous with 'reserved' or 'provisioned'. On these systems,
>>'used'
>> is
>> > > always less than or equal to 'allocated'. Typically 'used' is less
>>than
>> > > 'allocated' because as 'used' approaches 'allocated', 'allocated' is
>> > > expanded (until it runs into 'total').
>> > >
>> > > Thanks!
>> > >
>> > > --
>> > > *Mike Tutkowski*
>> > > *Senior CloudStack Developer, SolidFire Inc.*
>> > > e: mike.tutkow...@solidfire.com
>> > > o: 303.746.7302
>> > > Advancing the way the world uses the
>> > > cloud
>> > > **
>> > >
>> >
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkow...@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the
>> cloud
>> **
>>

[SOLVED] Re: capacity of secondaey stor us eq 0. bug ir known ussue?

[Nitin Mehta]

That¹s great Benoit. You could have also found this out running the ssvm
health check script.
Thumb rule - In case of ssvm/secondary storage related issues first
troubleshooting step is to run ssvm health check script

On 04/10/13 1:37 AM, "benoit lair"  wrote:

>Hello,
>
>Issue resolved for me.
>
>In my case, my dashboard said me, i have 0kb of storage on my secondary
>storage.
>
>The reason was not due to the cs 4.2, but it was entirely my fault due to
>a
>misconfiguration :
>
>My ssvm was correctly deploying, but it can't communicate with my nfs
>server.
>
>The reason was because i misconfigured my storage traffic label for
>xenserver, so my storage traffic was routed on the bad nic on the
>hypervisor, so the storage vlan was not available et so the ssvm cloudn't
>ping to the nfs server.
>
>If the ssvm can't ping to the nfs server, it can't mount it, if can't
>mount, it does not recognize the right size of the nfs server volume, so
>0kb.
>
>So, if dashboard says secondary storage at 0kb, there is probably a
>network
>issue with my nfs server.
>
>
>Regards, Benoit.
>
>
>2013/10/4 Nitin Mehta 
>
>> This generally should happen because of management server not able to
>> communicate with the SSVM agent.
>> Could you run the ssvm health check script to see if the connection is
>> healthy btwn ssvm and CS ?
>>
>> On 03/10/13 3:39 AM, "benoit lair"  wrote:
>>
>> >Hello,
>> >
>> >I have the same issue with cs 4.2 installed from the cloudstack rpm
>> >repositories (the stable ones), found no way to get an ssvm
>>operationnal.
>> >
>> >Thanks.
>> >
>> >
>> >2013/9/28 Netmaster 
>> >
>> >> Hallo,
>> >> we have several installation of cloudstack 4.2
>> >> In dashboard the capacity of secondary storage equal 0kb
>> >> We have standart nfs.
>> >> Api backed the same error, capacity of resource type 6 is 0
>> >> The secondary stor vm (system vm) shows normall capacity of 10tb,
>> >> as it have to be.
>> >> Is it bug or known issue?
>> >> Thanks
>>

RE: [DISCUSS] Components in JIRA and bug assignment

[Musayev, Ilya]

As per thread below with Animesh, we would like to resurrect the issue.

As per my conversation with Chip, the issue for not letting component 
maintainers - get assignments by default - was because we wanted to maintain a 
balance of "getting things done" vs "letting community grow"

Question to JIRA experienced admins, we can preserve "assign to me" option, and 
if unassigned goto "component" maintainer?



> -Original Message-
> From: Animesh Chaturvedi [mailto:animesh.chaturv...@citrix.com]
> Sent: Wednesday, October 02, 2013 6:48 PM
> To: dev@cloudstack.apache.org
> Subject: RE: [DISCUSS] Components in JIRA and bug assignment
> 
> 
> 
> > -Original Message-
> > From: Musayev, Ilya [mailto:imusa...@webmd.net]
> > Sent: Wednesday, October 02, 2013 3:15 PM
> > To: dev@cloudstack.apache.org
> > Subject: [DISCUSS] Components in JIRA and bug assignment
> >
> > I would like to separate this thread on tickets assignments in JIRA.
> >
> > Previously it was mentioned that we can add additional components into
> > JIRA CLOUDSTACK project and have ticket default to maintainer of the
> > component - who can then assign it to a proper developer.
> >
> [Animesh>] Yes JIRA allows adding a component lead. I can add components
> and the primary lead if the community agrees to it. This will be based on our
> maintainers list [1]
> 
> > What do you think?
> [Animesh>]+1,  I had brought up the same idea in the thread [2] and that
> time it was not approved by the community. With my experience from 4.2.0 I
> would like it to be reconsidered.
> 
> >
> > Any other suggestions?
> >
> [Animesh>] In the thread [3] I had also called out the email subscriptions for
> each of the components based on JIRA filter, but at that time I did not get
> much response.
> 
> > If agreed, who has the karma in JIRA to do this?
> [Animesh>] There are few folks that have Admin access to JIRA including me.
> Alex, David, Chip are ofcourse administrators.
> >
> > What components we need to add or remove to lower the confusion?
> [Animesh>] We should review the current component list and refine them.
> Some of them are over used like "management Server" as Sheng mentioned
> in the other thread.
> >
> > Last but not least, if someone can explain how bug reporter would
> > identify the proper component, we can then post it on WIKI and
> > minimize the error rate caused by improper bug filling.
> [Animesh>] We can add a description column in [1] to provide description for
> the components.
> 
> [1]
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Current+Maintai
> ners+Per+Component
> [2] http://markmail.org/thread/btovj6t6opqxge5q
> [3] http://markmail.org/thread/2gm4lq3lgfdzeo53
> 

RE: [ACS4.2.1] UI tickets

[Animesh Chaturvedi]

Abhi

It is best to set up a shared filter. You will need to open a INFRA ticket like 
this one https://issues.apache.org/jira/i#browse/INFRA-6145 to get relevant 
permissions for the JIRA


Thanks
Animesh


> -Original Message-
> From: Abhinandan Prateek [mailto:abhinandan.prat...@citrix.com]
> Sent: Friday, October 04, 2013 6:14 AM
> To: dev@cloudstack.apache.org
> Cc: Brian Federle; Jessica Wang; Sonny Chhen
> Subject: Re: [ACS4.2.1] UI tickets
> 
> Yes, specially the one in CC.
> 
> On 04/10/13 6:16 pm, "Sudha Ponnaganti" 
> wrote:
> 
> >Do you mean UI guys??
> >
> >-Original Message-
> >From: Abhinandan Prateek [mailto:abhinandan.prat...@citrix.com]
> >Sent: Friday, October 04, 2013 4:54 AM
> >To: CloudStack Dev
> >Cc: Brian Federle; Jessica Wang; Sonny Chhen
> >Subject: [ACS4.2.1] UI tickets
> >
> >
> >There are around ~72 UI tickets open. Most of them are major, can
> >documentation guys go thru the tickets and start picking them.
> >
> >Following Jira query will give the tickets that are open.
> >
> >project = CLOUDSTACK AND resolution = Unresolved AND fixVersion in
> >("4.2.0", "4.2.1") AND text ~ "UI"
> >
> >-abhi

Re: Scaling up cpu and memory of user vm above host capacity

[Nitin Mehta]

changeServiceForVirtualMachine API was the old API to change the service
offering for a stopped vm only.
I think it shouldn't have succeeded for a running vm. Please file a bug if
this is the case

scaleVirtualMachine is the new API introduced in 4.2 for scaling a
running/stopped vm. Do read the link I pointed below when you get a chance.

Thanks,
-Nitin

On 03/10/13 11:50 PM, "Gaurav Aradhye"  wrote:

>Hi Nitin,
>
>I was trying on running vm only, but I was
>using changeServiceForVirtualMachine API instead of scaleVirtualMachine
>API.
>But I wonder why changeServiceForVirtualMachine API succeeded in
>allocating
>more than host capacity.
>
>What is the basic difference between these two operations?
>
>Regards,
>Gaurav
>
>
>On Tue, Oct 1, 2013 at 10:45 PM, Nitin Mehta 
>wrote:
>
>> Gaurav - Were you trying this on a stopped vm ? If you try and start it
>> with an offering
>> above the host capacity (including over provisioning ) then it shouldn't
>> start.  Let me know how it goes.
>>
>> More details on scale vm feature @
>> 
>>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+scaling+of
>>+C
>> PU+and+RAM
>>
>> On 01/10/13 12:02 AM, "Gaurav Aradhye" 
>>wrote:
>>
>> >Thanks David. That disabuses my confusion about the CPU provisioning. I
>> >was
>> >using the wrong API to scale up the virtual machine, so above
>>observations
>> >stand invalid till I get the same results with the right API.
>> >
>> >About over-provisioning, I have the over provisioning factor set as 1
>>both
>> >in case of CPU and memory.
>> >
>> >Regards,
>> >Gaurav
>> >
>> >
>> >On Mon, Sep 30, 2013 at 10:55 PM, David Ortiz 
>> wrote:
>> >
>> >> A machine won't be able to support more cores on a VM than the
>>physical
>> >> processor.  That should result in problems trying to deploy it.  I'm
>> >> guessing the service offering is still valid since you could add a
>>host
>> >> later which has a hex core or two cpus in it.  As far as RAM goes, do
>> >>you
>> >> have overprovisioning enabled?
>> >>
>> >> > From: gaurav.arad...@clogeny.com
>> >> > Date: Mon, 30 Sep 2013 14:00:04 +0530
>> >> > Subject: Scaling up cpu and memory of user vm above host capacity
>> >> > To: dev@cloudstack.apache.org
>> >> >
>> >> > Hi,
>> >> >
>> >> > I am trying to automate a scenario here. I have only one host in
>> >>cluster
>> >> > with 4 CPU cores and 15 GB total memory. When I try to scale up cpu
>> >>and
>> >> RAM
>> >> > of a running user vm above the host capacity, it doesn't throw any
>> >>error
>> >> > and I can see the updated values in VM statistics too.
>> >> >
>> >> > For CPU, I am able to change the service offering of user vm as  5
>> >>cores
>> >> *
>> >> > 100 MHz (even though host has 4 cores). I am not sure how this
>> >> calculation
>> >> > is done. Definitely many no. of virtual cores can be formed on host
>> >>(more
>> >> > than 4), but is it possible to allocate 5 cores to single VM ?
>>When I
>> >>try
>> >> > to deploy new VM with 5 core CPU service offering, then in this
>>case
>> >>it
>> >> > fails saying not enough server capacity.
>> >> >
>> >> > Also, For memory, I am able to create 17 GB memory service offering
>> >>and
>> >> > allocate it to any running user vm (although the total memory on
>>host
>> >>is
>> >> 15
>> >> > GB).
>> >> >
>> >> > Any directions? Is this an issue or am I missing something here?
>> >> >
>> >> > Regards,
>> >> > Gaurav
>> >>
>> >>
>>
>>

Re: VmWare SDK to vijava

[Darren Shepherd]

I created https://issues.apache.org/jira/browse/LEGAL-180 for this.

Darren

Re: [MERGE] spring-modularization to master - Spring Modularization

[Darren Shepherd]

I just cherry picked that commit, so it should be fixed now.

Darren

On Fri, Oct 4, 2013 at 9:38 AM, Darren Shepherd
 wrote:
> Yes unfortunately when I did the rebase and pushed the branch, master
> was broken.  So that failing test is not related to Spring and was
> removed in commit 765f56a02ced44b7d95574f5901e834c799a56fe on master.
> Is there any way you can ignore the failure.
>
> Darren
>
> On Fri, Oct 4, 2013 at 4:13 AM, Prasanna Santhanam  wrote:
>> Hey Darren, the build appears to be broken on spring-modularization.
>> Actually a test is broken. If fixed I could generate some packages for
>> testing.
>>
>> http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-yumrepo-refresh/3028/console
>>
>> On Fri, Oct 04, 2013 at 10:27:55AM +0530, Prasanna Santhanam wrote:
>>> Thanks. I will first run a baseline against master since we don't have
>>> one and then one on spring modularization. The baseline against 4.2
>>> shows this result:
>>>
>>> http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-regression-matrix/252/testReport/
>>>
>>> On Thu, Oct 03, 2013 at 09:19:23AM -0700, Darren Shepherd wrote:
>>> > This should be fixed on the spring-modularization branch.
>>> >
>>> > Darren
>>> >
>>> > On Wed, Oct 2, 2013 at 10:05 PM, Prasanna Santhanam  
>>> > wrote:
>>> > > I switched the test infrastructure on jenkins.buildacloud.org to run
>>> > > the bvts [1] against master last week. Couple of weeks before that the
>>> > > simulator [2] tests were switched to run against master. Both are
>>> > > broken unfortunately and the bvt and checkin tests aren't running.
>>> > >
>>> > > I've filed a bug here:
>>> > > https://issues.apache.org/jira//browse/CLOUDSTACK-4791
>>> > >
>>> > > [1] http://jenkins.buildacloud.org/view/cloudstack-qa/
>>> > > [2] http://jenkins.buildacloud.org/view/simulator/
>>> > >
>>> > > Anyone who has access to the jenkins can run the bvts on their desired
>>> > > branch. Simply login and change the test-yumrepo-refresh job to point
>>> > > to your branch. Build that to refresh the remote repository with the
>>> > > packages made from your branch. Then switch test-matrix to point to
>>> > > the same development branch and fire a build. That's about it.
>>> > >
>>> > > On Wed, Oct 02, 2013 at 05:42:54PM -0700, Darren Shepherd wrote:
>>> > >> Yes agreed.  I've extensively tested this, but that is never enough.
>>> > >> How do I get the BVTs ran against this.  Due to the cross cutting
>>> > >> nature of this I want to get this merged as fast as possible.
>>> > >>
>>> > >> Darren
>>> > >>
>>> > >> > On Oct 2, 2013, at 4:43 PM, Alex Huang  wrote:
>>> > >> >
>>> > >> > +1 on running the BVT on it.  We've been through this one once 
>>> > >> > before.  Should be careful.
>>> > >> >
>>> > >> > --Alex
>>> > >> >
>>> > >> >> -Original Message-
>>> > >> >> From: Kelven Yang [mailto:kelven.y...@citrix.com]
>>> > >> >> Sent: Wednesday, October 2, 2013 4:39 PM
>>> > >> >> To: dev@cloudstack.apache.org
>>> > >> >> Subject: Re: [MERGE] spring-modularization to master - Spring
>>> > >> >> Modularization
>>> > >> >>
>>> > >> >> Darren,
>>> > >> >>
>>> > >> >> This looks really nice. A few questions on Spring AOP replacement.
>>> > >> >>
>>> > >> >> 1) Spring AOP is proxy-based, the reason we ended up of using 
>>> > >> >> customized
>>> > >> >> AOP is mainly due to that inside existing CloudStack codebase, we 
>>> > >> >> have many
>>> > >> >> places that are doing run-time type-casting, the code in these 
>>> > >> >> places
>>> > >> >> assumes a real object that implements all interfaces in the 
>>> > >> >> semantics context.
>>> > >> >> At the time when I initially converted to Spring, I couldn't ensure 
>>> > >> >> that
>>> > >> >> switching to proxy-based AOP can have 100% coverage for these 
>>> > >> >> run-time
>>> > >> >> cases. What is your approach to address this run-time type-casting 
>>> > >> >> issue?
>>> > >> >>
>>> > >> >> 2) We've run into a huge-memory footprint issue that may be caused 
>>> > >> >> by
>>> > >> >> conflicts of CGLIB usage in Spring AOP and the CGLIB usage in 
>>> > >> >> CloudStack Dao
>>> > >> >> layer. Do you have a chance to run a memory analysis in the heap 
>>> > >> >> after
>>> > >> >> management server is started.
>>> > >> >>
>>> > >> >> I might be good to run BVT test on the branch before the merge, 
>>> > >> >> could
>>> > >> >> someone initiate the effort?
>>> > >> >>
>>> > >> >> kelven
>>> > >> >>
>>> > >> >>
>>> > >> >>
>>> > >> >> On 10/2/13 3:48 PM, "Darren Shepherd" 
>>> > >> >> wrote:
>>> > >> >>
>>> > >> >>> Not sure how this works...  I would like to merge in the new
>>> > >> >>> modularized Spring setup to master. There is info on the wiki 
>>> > >> >>> about it
>>> > >> >>> [1] [2] [3].  The primary change is to break apart the monolithic
>>> > >> >>> applicationContext.xml and componentContext.xml files such that 
>>> > >> >>> each
>>> > >> >>> plugin can maintain and contribute its own configuration.
>>> > >> >>>
>>> > >> >>> In addit

Re: [MERGE] spring-modularization to master - Spring Modularization

[Darren Shepherd]

Yes unfortunately when I did the rebase and pushed the branch, master
was broken.  So that failing test is not related to Spring and was
removed in commit 765f56a02ced44b7d95574f5901e834c799a56fe on master.
Is there any way you can ignore the failure.

Darren

On Fri, Oct 4, 2013 at 4:13 AM, Prasanna Santhanam  wrote:
> Hey Darren, the build appears to be broken on spring-modularization.
> Actually a test is broken. If fixed I could generate some packages for
> testing.
>
> http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-yumrepo-refresh/3028/console
>
> On Fri, Oct 04, 2013 at 10:27:55AM +0530, Prasanna Santhanam wrote:
>> Thanks. I will first run a baseline against master since we don't have
>> one and then one on spring modularization. The baseline against 4.2
>> shows this result:
>>
>> http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-regression-matrix/252/testReport/
>>
>> On Thu, Oct 03, 2013 at 09:19:23AM -0700, Darren Shepherd wrote:
>> > This should be fixed on the spring-modularization branch.
>> >
>> > Darren
>> >
>> > On Wed, Oct 2, 2013 at 10:05 PM, Prasanna Santhanam  
>> > wrote:
>> > > I switched the test infrastructure on jenkins.buildacloud.org to run
>> > > the bvts [1] against master last week. Couple of weeks before that the
>> > > simulator [2] tests were switched to run against master. Both are
>> > > broken unfortunately and the bvt and checkin tests aren't running.
>> > >
>> > > I've filed a bug here:
>> > > https://issues.apache.org/jira//browse/CLOUDSTACK-4791
>> > >
>> > > [1] http://jenkins.buildacloud.org/view/cloudstack-qa/
>> > > [2] http://jenkins.buildacloud.org/view/simulator/
>> > >
>> > > Anyone who has access to the jenkins can run the bvts on their desired
>> > > branch. Simply login and change the test-yumrepo-refresh job to point
>> > > to your branch. Build that to refresh the remote repository with the
>> > > packages made from your branch. Then switch test-matrix to point to
>> > > the same development branch and fire a build. That's about it.
>> > >
>> > > On Wed, Oct 02, 2013 at 05:42:54PM -0700, Darren Shepherd wrote:
>> > >> Yes agreed.  I've extensively tested this, but that is never enough.
>> > >> How do I get the BVTs ran against this.  Due to the cross cutting
>> > >> nature of this I want to get this merged as fast as possible.
>> > >>
>> > >> Darren
>> > >>
>> > >> > On Oct 2, 2013, at 4:43 PM, Alex Huang  wrote:
>> > >> >
>> > >> > +1 on running the BVT on it.  We've been through this one once 
>> > >> > before.  Should be careful.
>> > >> >
>> > >> > --Alex
>> > >> >
>> > >> >> -Original Message-
>> > >> >> From: Kelven Yang [mailto:kelven.y...@citrix.com]
>> > >> >> Sent: Wednesday, October 2, 2013 4:39 PM
>> > >> >> To: dev@cloudstack.apache.org
>> > >> >> Subject: Re: [MERGE] spring-modularization to master - Spring
>> > >> >> Modularization
>> > >> >>
>> > >> >> Darren,
>> > >> >>
>> > >> >> This looks really nice. A few questions on Spring AOP replacement.
>> > >> >>
>> > >> >> 1) Spring AOP is proxy-based, the reason we ended up of using 
>> > >> >> customized
>> > >> >> AOP is mainly due to that inside existing CloudStack codebase, we 
>> > >> >> have many
>> > >> >> places that are doing run-time type-casting, the code in these places
>> > >> >> assumes a real object that implements all interfaces in the 
>> > >> >> semantics context.
>> > >> >> At the time when I initially converted to Spring, I couldn't ensure 
>> > >> >> that
>> > >> >> switching to proxy-based AOP can have 100% coverage for these 
>> > >> >> run-time
>> > >> >> cases. What is your approach to address this run-time type-casting 
>> > >> >> issue?
>> > >> >>
>> > >> >> 2) We've run into a huge-memory footprint issue that may be caused by
>> > >> >> conflicts of CGLIB usage in Spring AOP and the CGLIB usage in 
>> > >> >> CloudStack Dao
>> > >> >> layer. Do you have a chance to run a memory analysis in the heap 
>> > >> >> after
>> > >> >> management server is started.
>> > >> >>
>> > >> >> I might be good to run BVT test on the branch before the merge, could
>> > >> >> someone initiate the effort?
>> > >> >>
>> > >> >> kelven
>> > >> >>
>> > >> >>
>> > >> >>
>> > >> >> On 10/2/13 3:48 PM, "Darren Shepherd" 
>> > >> >> wrote:
>> > >> >>
>> > >> >>> Not sure how this works...  I would like to merge in the new
>> > >> >>> modularized Spring setup to master. There is info on the wiki about 
>> > >> >>> it
>> > >> >>> [1] [2] [3].  The primary change is to break apart the monolithic
>> > >> >>> applicationContext.xml and componentContext.xml files such that each
>> > >> >>> plugin can maintain and contribute its own configuration.
>> > >> >>>
>> > >> >>> In addition to breaking up the configuration we no longer use ACS
>> > >> >>> custom AOP and it is now fully Spring AOP.
>> > >> >>>
>> > >> >>> Now adding/removing a plug-in is a matter of just adding a jar to 
>> > >> >>> the
>> > >> >>> classpath (exception being commands.properties, I'll address t

Re: XenServer Question

[benoit lair]

Effectively, it's very surprising.

I checked my differents storage (local as iscsi sr) on xcp 1.0, xcp 1.1,
xcp 1.4.90, xcp 1.6, xenserver 6.0 and xenserver 6.2, i observe that i can
have a 'used' storage more important that the 'allocated' one

I verified connected to my hypervisors shells and verified the data
displayed in xencenter, the value 'used' is the good one.

However, i can have allocated 200GB on my local SR to one vm (so it's
provisionning). I can have 33GB 'used' on my local SR, but i have in
reality less storage available into my vm.

More clearly, if 200GB allocated on one vm, if xencenter says me 33GB are
used, i can retrieve a vhd file corresponding to my vm of a size of 33GB,
but when i'm connected to the vm's shell, i don't retrieve my 33gbs of
storage.

I observe the same phenomenon with my others xenserver hypervisors.

It seems that some metadatas could be present into the vhd file + the real
vm storage.



Is there any citrix xenserver specialist in the room to confirm (or not)
what i said ?


Thanks.

Regards, Benoit.


2013/10/4 Mike Tutkowski 

> Doh! I guess that would have been helpful info. :)
>
> It is in XenCenter. If you click on an SR and look at the General tab.
>
> http://i.imgur.com/z5F1Ct0.png
>
>
> On Fri, Oct 4, 2013 at 4:34 AM, benoit lair  wrote:
>
> > Hi Mike,
> >
> > Where do you see this information ? On the CS dashboard ? Or with
> > xencenter, listing the sr with the windows gui ?
> >
> > Regards, Benoit.
> >
> >
> > 2013/10/3 Mike Tutkowski 
> >
> > > Hi,
> > >
> > > On one of my SRs, I see the following information:
> > >
> > > 100.2 GB used of 200 GB total (100 GB allocated)
> > >
> > > Can someone explain how 'used' can be larger than 'allocated'?
> > >
> > > On storage systems I've worked on before, 'allocated' is generally
> > > synonymous with 'reserved' or 'provisioned'. On these systems, 'used'
> is
> > > always less than or equal to 'allocated'. Typically 'used' is less than
> > > 'allocated' because as 'used' approaches 'allocated', 'allocated' is
> > > expanded (until it runs into 'total').
> > >
> > > Thanks!
> > >
> > > --
> > > *Mike Tutkowski*
> > > *Senior CloudStack Developer, SolidFire Inc.*
> > > e: mike.tutkow...@solidfire.com
> > > o: 303.746.7302
> > > Advancing the way the world uses the
> > > cloud
> > > *™*
> > >
> >
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud
> *™*
>

Re: 2 questions regarding building docs

[Daan Hoogland]

4.2 in the new repo, yes. It doesn't fail. I happend to notice the severe
message as it ran by

mobile biligual spell checker used
Op 4 okt. 2013 16:22 schreef "David Nalley" :

> Bah - I hit send without dealing with the second question.
>
> On Fri, Oct 4, 2013 at 9:15 AM, Daan Hoogland 
> wrote:
> > H,
> >
> > I started building the docs of the 4.2 tree, using the cfg
> publican-all.cfg
> > I fails. Is there a defined set of targets that must/should build in the
> > docs?
> >
> >
> > Processing file tmp/en-US/xml_tmp/zone-add.xml ->
> > tmp/en-US/xml/zone-add.xml
> > Beginning work on en-US
> > Validation failed:
> > hypervisor-support-for-primarystorage.xml:6: validity error : ID
> > hypervisor-support-for-primarystorage already defined
> > add-remove-nic.xml:31: validity error : ID prereq-addnic already defined
> > api-throttling.xml:6: validity error : ID api-throttling already defined
> > api-throttling.xml:30: validity error : ID api-throttling-configure
> already
> > defined
> > api-throttling.xml:61: validity error : ID api-throttling-limitations
> > already defined
> > globally-configured-limits.xml:6: validity error : ID
> > globally-configured-limits already defined
> > configure-package-repository.xml:28: validity error : IDREF attribute
> > linkend references an unknown ID "sect-source-buildrpm"
> > management-server-install-multi-node.xml:30: validity error : IDREF
> > attribute linkend references an unknown ID "sect-source-buildrpm"
> > using-netscaler-load-balancers.xml:41: validity error : IDREF attribute
> > linkend references an unknown ID "external-guest-lb-integration"
> > configure-package-repository.xml:28: validity error : IDREF attribute
> > linkend references an unknown ID "sect-source-builddebs"
> > management-server-install-multi-node.xml:30: validity error : IDREF
> > attribute linkend references an unknown ID "sect-source-builddebs"
> >
> > also the other targets flash by the following
> >
> > Oct 4, 2013 3:12:03 PM org.apache.fop.hyphenation.Hyphenator
> > getHyphenationTree
> > SEVERE: Couldn't find hyphenation pattern en
> >
> > I read I should ignore warnings and this one says severe. How important
> is
> > hyphenation in english?
> >
> > thanks,
> > Daan
>
>
> You are doing this in the new cloudstack-docs repo on the 4.2 branch?
> I have been building admin and install guides, so I know they aren't
> broken and don't have validity errors. If it's only publican-all that
> is broken I am happy to git rm that file :)
>
> Re FOP warnings, if it builds, ignore them, FOP complains loudly about
> lots of things that might matter to people doing magazine layout, but
> not to tech manuals.
>

Re: docs split of only to 4.2

[Daan Hoogland]

And i will hold my peace thee rest of the week.

mobile biligual spell checker used
Op 4 okt. 2013 16:28 schreef "David Nalley" :

> On Fri, Oct 4, 2013 at 8:30 AM, Daan Hoogland 
> wrote:
> > H,
> >
> > Is there a reason the split of docs repo does not have anything in master
> > but only in 4.2? Should I cherrypick the entire tree or do we want to
> wait
> > with that for some reason?
> >
>
> There are two reasons that it's that way. First, some of the updated
> docs content never made it into master, so just copying over master
> from the main repo wouldn't work.
> Second, I am doing a bit of restructuring, and don't want to do it
> twice, so as soon as I get that done, I'll merge 4.2 into master and
> we'll have a place to start on 4.3 docs. I hope that I get this all
> done by CoB today.
>
> --David
>

Re: [jira] [Updated] (CLOUDSTACK-4799) Update Deployment Architecture Overview section

[David Nalley]

Travis:

Why are we using linkend on some, and urls on others? I am reticent to
put version specific urls in documents as that becomes a maintenance
nightmare.



--David

On Thu, Oct 3, 2013 at 5:20 PM, Travis Graham (JIRA)  wrote:
>
>  [ 
> https://issues.apache.org/jira/browse/CLOUDSTACK-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>  ]
>
> Travis Graham updated CLOUDSTACK-4799:
> --
>
> Attachment: deployment-architecture-overview.patch
>
>> Update Deployment Architecture Overview section
>> ---
>>
>> Key: CLOUDSTACK-4799
>> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4799
>> Project: CloudStack
>>  Issue Type: Improvement
>>  Security Level: Public(Anyone can view this level - this is the 
>> default.)
>>  Components: Doc
>>Affects Versions: 4.2.0
>>Reporter: Travis Graham
>>Priority: Critical
>>  Labels: documentaion
>> Fix For: 4.2.0
>>
>> Attachments: deployment-architecture-overview.patch
>>
>>
>> Adds in links to other docs it only loosely references by name and adds the 
>> Primary Storage info about being zone wide for 4.2.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.1#6144)

Re: docs split of only to 4.2

[David Nalley]

On Fri, Oct 4, 2013 at 8:30 AM, Daan Hoogland  wrote:
> H,
>
> Is there a reason the split of docs repo does not have anything in master
> but only in 4.2? Should I cherrypick the entire tree or do we want to wait
> with that for some reason?
>

There are two reasons that it's that way. First, some of the updated
docs content never made it into master, so just copying over master
from the main repo wouldn't work.
Second, I am doing a bit of restructuring, and don't want to do it
twice, so as soon as I get that done, I'll merge 4.2 into master and
we'll have a place to start on 4.3 docs. I hope that I get this all
done by CoB today.

--David

Re: 2 questions regarding building docs

[David Nalley]

Bah - I hit send without dealing with the second question.

On Fri, Oct 4, 2013 at 9:15 AM, Daan Hoogland  wrote:
> H,
>
> I started building the docs of the 4.2 tree, using the cfg publican-all.cfg
> I fails. Is there a defined set of targets that must/should build in the
> docs?
>
>
> Processing file tmp/en-US/xml_tmp/zone-add.xml ->
> tmp/en-US/xml/zone-add.xml
> Beginning work on en-US
> Validation failed:
> hypervisor-support-for-primarystorage.xml:6: validity error : ID
> hypervisor-support-for-primarystorage already defined
> add-remove-nic.xml:31: validity error : ID prereq-addnic already defined
> api-throttling.xml:6: validity error : ID api-throttling already defined
> api-throttling.xml:30: validity error : ID api-throttling-configure already
> defined
> api-throttling.xml:61: validity error : ID api-throttling-limitations
> already defined
> globally-configured-limits.xml:6: validity error : ID
> globally-configured-limits already defined
> configure-package-repository.xml:28: validity error : IDREF attribute
> linkend references an unknown ID "sect-source-buildrpm"
> management-server-install-multi-node.xml:30: validity error : IDREF
> attribute linkend references an unknown ID "sect-source-buildrpm"
> using-netscaler-load-balancers.xml:41: validity error : IDREF attribute
> linkend references an unknown ID "external-guest-lb-integration"
> configure-package-repository.xml:28: validity error : IDREF attribute
> linkend references an unknown ID "sect-source-builddebs"
> management-server-install-multi-node.xml:30: validity error : IDREF
> attribute linkend references an unknown ID "sect-source-builddebs"
>
> also the other targets flash by the following
>
> Oct 4, 2013 3:12:03 PM org.apache.fop.hyphenation.Hyphenator
> getHyphenationTree
> SEVERE: Couldn't find hyphenation pattern en
>
> I read I should ignore warnings and this one says severe. How important is
> hyphenation in english?
>
> thanks,
> Daan


You are doing this in the new cloudstack-docs repo on the 4.2 branch?
I have been building admin and install guides, so I know they aren't
broken and don't have validity errors. If it's only publican-all that
is broken I am happy to git rm that file :)

Re FOP warnings, if it builds, ignore them, FOP complains loudly about
lots of things that might matter to people doing magazine layout, but
not to tech manuals.

Re: XenServer Question

[Mike Tutkowski]

Doh! I guess that would have been helpful info. :)

It is in XenCenter. If you click on an SR and look at the General tab.

http://i.imgur.com/z5F1Ct0.png


On Fri, Oct 4, 2013 at 4:34 AM, benoit lair  wrote:

> Hi Mike,
>
> Where do you see this information ? On the CS dashboard ? Or with
> xencenter, listing the sr with the windows gui ?
>
> Regards, Benoit.
>
>
> 2013/10/3 Mike Tutkowski 
>
> > Hi,
> >
> > On one of my SRs, I see the following information:
> >
> > 100.2 GB used of 200 GB total (100 GB allocated)
> >
> > Can someone explain how 'used' can be larger than 'allocated'?
> >
> > On storage systems I've worked on before, 'allocated' is generally
> > synonymous with 'reserved' or 'provisioned'. On these systems, 'used' is
> > always less than or equal to 'allocated'. Typically 'used' is less than
> > 'allocated' because as 'used' approaches 'allocated', 'allocated' is
> > expanded (until it runs into 'total').
> >
> > Thanks!
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkow...@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud
> > *™*
> >
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

Re: 2 questions regarding building docs

[David Nalley]

On Fri, Oct 4, 2013 at 9:15 AM, Daan Hoogland  wrote:
> H,
>
> I started building the docs of the 4.2 tree, using the cfg publican-all.cfg
> I fails. Is there a defined set of targets that must/should build in the
> docs?
>

I personally don't like the -all target. I generally consider the
minimum set of documents to be release notes (cd release-notes;
publican build --langs en-US --formats html )
Install guide (publican build --langs en-US --formats html --config
publican-installation.cfg) and the Admin guide ((publican build
--langs en-US --formats html --config publican-adminguide.cfg)

There is also the allocator implementation guide, the quick install
guide, and the guides for NVP and Midonet that we could deploy -
though I am honestly loathe to do so when our more basic docs have
such major issues.

--David


>
> Processing file tmp/en-US/xml_tmp/zone-add.xml ->
> tmp/en-US/xml/zone-add.xml
> Beginning work on en-US
> Validation failed:
> hypervisor-support-for-primarystorage.xml:6: validity error : ID
> hypervisor-support-for-primarystorage already defined
> add-remove-nic.xml:31: validity error : ID prereq-addnic already defined
> api-throttling.xml:6: validity error : ID api-throttling already defined
> api-throttling.xml:30: validity error : ID api-throttling-configure already
> defined
> api-throttling.xml:61: validity error : ID api-throttling-limitations
> already defined
> globally-configured-limits.xml:6: validity error : ID
> globally-configured-limits already defined
> configure-package-repository.xml:28: validity error : IDREF attribute
> linkend references an unknown ID "sect-source-buildrpm"
> management-server-install-multi-node.xml:30: validity error : IDREF
> attribute linkend references an unknown ID "sect-source-buildrpm"
> using-netscaler-load-balancers.xml:41: validity error : IDREF attribute
> linkend references an unknown ID "external-guest-lb-integration"
> configure-package-repository.xml:28: validity error : IDREF attribute
> linkend references an unknown ID "sect-source-builddebs"
> management-server-install-multi-node.xml:30: validity error : IDREF
> attribute linkend references an unknown ID "sect-source-builddebs"
>
> also the other targets flash by the following
>
> Oct 4, 2013 3:12:03 PM org.apache.fop.hyphenation.Hyphenator
> getHyphenationTree
> SEVERE: Couldn't find hyphenation pattern en
>
> I read I should ignore warnings and this one says severe. How important is
> hyphenation in english?
>
> thanks,
> Daan

Re: Review Request 14451: Some test for NetUtils

[Laszlo Hornyak]

> On Oct. 4, 2013, 6:43 a.m., Saksham Srivastava wrote:
> > utils/test/com/cloud/utils/net/NetUtilsTest.java, line 225
> > 
> >
> > What I am suggesting is that add a method something like
> > boolean isEqual(var1, var2) {
> > if var1 == var2 
> > return true
> > return false
> > }
> > 
> > Now you can add cases like
> > assertFalse(isEqual(15, NetUtils.mac2Long("00:00:00:00:00:ff")))
> > 
> > This is just one implementation as there is no assertNotEquals() method 
> > provided by Junit. You could certainly add better implementations. 
> > We generally use assertFalse for boolean methods.
> > You can have a look at testIsSameIpRange():
> >  // Check for 2 different CIDRs and different IP Range
> > assertFalse(NetUtils.isSameIpRange(cidrFirst, cidrThird));
> > 
> > // Check for Incorrect format of CIDR
> > assertFalse(NetUtils.isSameIpRange(cidrFirst, "10.3.6.5/50"));
> > 
> > Just a few such cases will suffice.
> >
> 
> Laszlo Hornyak wrote:
> Ok... what I do not quite understand is what the purpose of this negative 
> test is. There is already test for mac2Long("...") must be equal long 
> , so a negative test seems to be a test that proves "15 != 16 or any 
> other long value, 15 only equals to 15" :)
> 
> Saksham Srivastava wrote:
> Actually the reason I suggested to add the test case is because we found 
> an interesting scenario, 2 CIDRs 10.0.151.0/20 and 10.0.144.0/20 look 
> different but have the same IP range, so isSameIpRange() will return true, 
> however isNetworkAWithinNetworkB() will hint you an different thing.
> 
> I just gave you a basic example of implementing assertNotEquals(), the 
> patch looks good to me even without adding these changes.
> You already have added ip2LongBadIp() which tests for a incorrect format 
> of IP address :)

Ok, thanks for the info, I will look into it!


- Laszlo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14451/#review26660
---


On Oct. 4, 2013, 11:12 a.m., Laszlo Hornyak wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14451/
> ---
> 
> (Updated Oct. 4, 2013, 11:12 a.m.)
> 
> 
> Review request for cloudstack.
> 
> 
> Bugs: CLOUDSTACK-4807
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> - tests added for mac2Long, byte2Mac, long2Mac, ip2Long and long2Ip
> - isWindows static method removed - it was not used
> - minor changes in the implementations of the tested methods
> 
> 
> Diffs
> -
> 
>   utils/src/com/cloud/utils/net/NetUtils.java 1e72e22 
>   utils/test/com/cloud/utils/net/NetUtilsTest.java 3cfc98f 
> 
> Diff: https://reviews.apache.org/r/14451/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> Laszlo Hornyak
> 
>

Re: Review Request 14467: fix for Invalid SMTP breaks HA

[Chip Childers]

On Fri, Oct 4, 2013 at 2:17 AM, Anshul Gangwar wrote:

> On Thursday 03 October 2013 08:44 PM, Chip Childers wrote:
> > On Thu, Oct 03, 2013 at 11:31:50AM +, Anshul Gangwar wrote:
> >> Description
> >> ---
> >>
> >> added the connection and socket timeout parameters for SMTP and sending
> message in new thread so that HA doesn't get blocked beacause of hang in
> sending email alert
> > Wouldn't a better approach be to ensure that email sending runs async to
> > critical operations like HA?
>
> This patch is making email sending  async. Currently agent thread first
> send the alert( where emails are sent) and then calls the HA. Can you
> please elaborate more? Is it logic change or something else?
>
> Thanks,
> Anshul
>

You're right.  I was commenting on the description of the review, but
obviously should have looked at the code.

LGTM!

-chip

2 questions regarding building docs

[Daan Hoogland]

H,

I started building the docs of the 4.2 tree, using the cfg publican-all.cfg
I fails. Is there a defined set of targets that must/should build in the
docs?


Processing file tmp/en-US/xml_tmp/zone-add.xml ->
tmp/en-US/xml/zone-add.xml
Beginning work on en-US
Validation failed:
hypervisor-support-for-primarystorage.xml:6: validity error : ID
hypervisor-support-for-primarystorage already defined
add-remove-nic.xml:31: validity error : ID prereq-addnic already defined
api-throttling.xml:6: validity error : ID api-throttling already defined
api-throttling.xml:30: validity error : ID api-throttling-configure already
defined
api-throttling.xml:61: validity error : ID api-throttling-limitations
already defined
globally-configured-limits.xml:6: validity error : ID
globally-configured-limits already defined
configure-package-repository.xml:28: validity error : IDREF attribute
linkend references an unknown ID "sect-source-buildrpm"
management-server-install-multi-node.xml:30: validity error : IDREF
attribute linkend references an unknown ID "sect-source-buildrpm"
using-netscaler-load-balancers.xml:41: validity error : IDREF attribute
linkend references an unknown ID "external-guest-lb-integration"
configure-package-repository.xml:28: validity error : IDREF attribute
linkend references an unknown ID "sect-source-builddebs"
management-server-install-multi-node.xml:30: validity error : IDREF
attribute linkend references an unknown ID "sect-source-builddebs"

also the other targets flash by the following

Oct 4, 2013 3:12:03 PM org.apache.fop.hyphenation.Hyphenator
getHyphenationTree
SEVERE: Couldn't find hyphenation pattern en

I read I should ignore warnings and this one says severe. How important is
hyphenation in english?

thanks,
Daan

Re: [ACS4.2.1] UI tickets

[Abhinandan Prateek]

Yes, specially the one in CC.

On 04/10/13 6:16 pm, "Sudha Ponnaganti" 
wrote:

>Do you mean UI guys??
>
>-Original Message-
>From: Abhinandan Prateek [mailto:abhinandan.prat...@citrix.com]
>Sent: Friday, October 04, 2013 4:54 AM
>To: CloudStack Dev
>Cc: Brian Federle; Jessica Wang; Sonny Chhen
>Subject: [ACS4.2.1] UI tickets
>
>
>There are around ~72 UI tickets open. Most of them are major, can
>documentation guys go thru the tickets and start picking them.
>
>Following Jira query will give the tickets that are open.
>
>project = CLOUDSTACK AND resolution = Unresolved AND fixVersion in
>("4.2.0", "4.2.1") AND text ~ "UI"
>
>-abhi

Re: [DOCS] CS-docs patch

[Travis Graham]

Hello Prasanna,

After sending this out yesterday I realized it would probably get stripped so 
Chip recommended opening issues in Jira and attaching the patches there.

I've created two issues and attached patches as well as attaching a patch to 
another issue that came in about the docs:

* CLOUDSTACK-4799
* CLOUDSTACK-4800
* CLOUDSTACK-4797

Until there's an easier way to get the patches out I'll stick to Jira and send 
out an email to dev@ about the patch so anyone can review and apply the changes.

Thanks,
Travis

On Oct 4, 2013, at 12:37 AM, Prasanna Santhanam  wrote:

> On Thu, Oct 03, 2013 at 02:10:26PM -0400, Travis Graham wrote:
>> Here is a patch for the "Deployment Architecture Overview" section
>> until things get setup in Review Board for the new repo or there's a
>> GitHub mirror that pull requests can be sent to for review. Will the
>> github mirror and PR route be supported in tandem with the Review
>> Board process or not at all?
>> 
>> It adds in links to other docs it only loosely referenced by name
>> and adds the Primary Storage info about being zone wide for 4.2.
>> 
>> Travis
> Attachments are stripped
> 
> Please paste the contents of your patch in plaintext with the subject
> [PATCH]. Also ensure you've created your patch using git format-patch
> so I can apply it readily and attribute credit to you.
> 
> I'd love to have PRs via github but that's upto asf infra and not
> project specific to answer.
> 
> 
> Thanks,
> 
> 
>> 
> 
> 
> -- 
> Prasanna.,
> 
> 
> Powered by BigRock.com
> 

Re: [DISCUSS] Breaking out Marvin from CloudStack

[Daan Hoogland]

Prasanna,

I share Chips concern. But let me start by saying that given the effort
Citrix puts in the integration testing, making it easy for your guys should
be somewhat a priority. However, can we do the separation is such a way
that integration tests as part of plugin and other enhancements is still
trivial? for instance make the maven scripts checkout a stable marvin
version and run tests using this framework. What Chip wants, i think, is
that we enforce a integration test py in every contribution not only to
validate the feature but also to ensure its continued functioning while the
contributors may have wondered off in their $dayjob.

I followed the reviews submitted and know that the ones in the marvin
corner of the world are a big chunk of it ans have few reviewers/committers
in comparison. Can your refactor be organized so that integration tests can
go in the marvin and in the cloudstack repo?

thanks,
Daan


On Fri, Oct 4, 2013 at 2:27 PM, Prasanna Santhanam  wrote:

> I'll summarize and address the concerns raised so far.
>
> Marvin has been in this repo for a long time for us to start writing
> tests. The only tests I've seen coming are from a specific set of
> people focussed on QA efforts. I want to reduce the impediment for
> people who are writing tests *today*. Those looking to get started in
> the near future won't have any new learning to do, just that their
> code goes in an alternate repo that is pointed to right
> infrastructure.
>
> Automated testing also works in a push-to-production style very often.
> Testers need to run their tests on a deployed environment(s) quickly
> to be able to ensure it is valid and passes. By making them go through
> reviewboard each time for each test we massively slow down the
> process. (tons of fixes to tests are on rb today, not just new tests).
> We don't know if they run until they run on the environment.
>
> Reason for tests and framework to go together is simple.  If I go look
> at the jclouds repository today I find tests for rackspace cloud,
> openstack cloud, cloudstack cloud, euca clouds in the jclouds
> repository and not in the respective provider/project repository. A
> newcomer to the marvin repository will be someone interested in
> writing tests and he will also thus be able to find tests in the
> marvin repository.
>
> This also allows for more heterogenous testing of cloudstack. No one
> needs to be tied down to a framework / tool to write integration
> tests. If python is not your forte, use Chip's ruby client, or perhaps
> in the near future Chiradeep's stackmate to write your test, or even
> jclouds.
>
> Now the question of supporting older version of marvin against newer
> versions of cloudstack. Marvin now fully auto-generates itself (see
> the design in the proposal) based on endpoint. So you have the
> marvin version that will work with your endpoint only. As for being
> backwards compatible (also addressed in the design doc) - no old tests
> are broken, they will still run perfectly fine.
>
> The infrastructure (currently) only looks at the changes in the test
> directory before performing a run. It doesn't care whether server/ was
> changed or plugins/x/y/z was changed. That's because the tests are
> unrelated to what is in the rest of the repository. In fact you can't
> even run them without a deployed cloud. So I don't see why idle code
> should lie in the repo.
>
> Integration tests are essential, they will keep coming as long as
> Citrix QA is invested in the effort, but they need to come faster into
> the repos and that will be addressed by the separation IMO. Managing
> the feature submitted to cloudstack against tests submitted to marvin
> is not a hard thing to do. We simply mirror the release branches in
> marvin and submit tests there. In fact I wonder why we didn't have
> this question when docs were separated? It doesn't work any
> differently really.
>
> What I would like to see provided by CloudStack is the ability to
> upgrade all our test environments, staging environments, UATs, what
> have you in continuous integration and have tests run on the
> upgraded setup. That allows incrementally testing CloudStack the way
> users do it. The current design of installing everything from scratch,
> redoing the testbed for each automated test run is mostly a workaround
> for that inability. If we had this ability tests written in marvin can
> be run against live setups at all times as and when features merge to
> master.
>
> It has always been my goal to reduce the barrier to writing tests for
> cloudstack and that's entirely the goal of the refactor as well. So I
> hope I have spent my time well. Since I can't be after people to write
> tests, and setup mandates, I aim to address the technical difficulty
> in writing tests through this refactor and separation.
>
> PS: David, by hosted docs I mean something like
> http://pythonhosted.org/cloudmonkey. Lives in the repo but read from
> the webpage.
>
> --
> Prasanna.,
>
>

RE: [ACS4.2.1] UI tickets

[Sudha Ponnaganti]

Do you mean UI guys??

-Original Message-
From: Abhinandan Prateek [mailto:abhinandan.prat...@citrix.com] 
Sent: Friday, October 04, 2013 4:54 AM
To: CloudStack Dev
Cc: Brian Federle; Jessica Wang; Sonny Chhen
Subject: [ACS4.2.1] UI tickets


There are around ~72 UI tickets open. Most of them are major, can documentation 
guys go thru the tickets and start picking them.

Following Jira query will give the tickets that are open.

project = CLOUDSTACK AND resolution = Unresolved AND fixVersion in ("4.2.0", 
"4.2.1") AND text ~ "UI"

-abhi

Re: unable to get tiny linux template in ready state, DOWNLOAD ERROR

[Punith s]

hi guys ,
finally i was able to add the instance,
https://cwiki.apache.org/confluence/display/CLOUDSTACK/SSVM%2C+templates%2C+Secondary+storage+troubleshooting

the above like helped to find the problem.

i ssh'd into SSVM using sh -i /root/.ssh/id_rsa.cloud -p 3922
root@LinkLocaip
ran the script to check the SSVM's connection with agent
/usr/local/cloud/systemvm/ssvm-check.sh

the problem was that the SSVM was not able to resolve the DNS , hence it
was failing to download the template from the url,

* i made my localhost static from dhcp and
* enabled PAE mode is activein devcloud *VM* settings, System > Processor
Tab, make sure 'Enable PAE/NX' mode is checked.

thanks for the troubleshooting doc guys ,

regards,
punith s
cloudbyte


On Thu, Oct 3, 2013 at 10:12 PM, Min Chen  wrote:

> Punith,
>
> Your log shows that your ssvm cannot connect back to MS. As Wei
> mentioned, your host configuration may not be correct, and need to be
> changed to correct Ip. Do you see an entry in your db host table with type
> SecondaryStorageVM?
>
> Thanks
> -min
>
> On 10/3/13 7:36 AM, "Wei ZHOU"  wrote:
>
> >It is normal.
> >You can get more information in vmops.log
>
>

docs split of only to 4.2

[Daan Hoogland]

H,

Is there a reason the split of docs repo does not have anything in master
but only in 4.2? Should I cherrypick the entire tree or do we want to wait
with that for some reason?

Daan

Re: [DISCUSS] Breaking out Marvin from CloudStack

[Prasanna Santhanam]

I'll summarize and address the concerns raised so far.

Marvin has been in this repo for a long time for us to start writing
tests. The only tests I've seen coming are from a specific set of
people focussed on QA efforts. I want to reduce the impediment for
people who are writing tests *today*. Those looking to get started in
the near future won't have any new learning to do, just that their
code goes in an alternate repo that is pointed to right
infrastructure. 

Automated testing also works in a push-to-production style very often.
Testers need to run their tests on a deployed environment(s) quickly
to be able to ensure it is valid and passes. By making them go through
reviewboard each time for each test we massively slow down the
process. (tons of fixes to tests are on rb today, not just new tests).
We don't know if they run until they run on the environment.

Reason for tests and framework to go together is simple.  If I go look
at the jclouds repository today I find tests for rackspace cloud,
openstack cloud, cloudstack cloud, euca clouds in the jclouds
repository and not in the respective provider/project repository. A
newcomer to the marvin repository will be someone interested in
writing tests and he will also thus be able to find tests in the
marvin repository.

This also allows for more heterogenous testing of cloudstack. No one
needs to be tied down to a framework / tool to write integration
tests. If python is not your forte, use Chip's ruby client, or perhaps
in the near future Chiradeep's stackmate to write your test, or even
jclouds. 

Now the question of supporting older version of marvin against newer
versions of cloudstack. Marvin now fully auto-generates itself (see
the design in the proposal) based on endpoint. So you have the
marvin version that will work with your endpoint only. As for being
backwards compatible (also addressed in the design doc) - no old tests
are broken, they will still run perfectly fine. 

The infrastructure (currently) only looks at the changes in the test
directory before performing a run. It doesn't care whether server/ was
changed or plugins/x/y/z was changed. That's because the tests are
unrelated to what is in the rest of the repository. In fact you can't
even run them without a deployed cloud. So I don't see why idle code
should lie in the repo.

Integration tests are essential, they will keep coming as long as
Citrix QA is invested in the effort, but they need to come faster into
the repos and that will be addressed by the separation IMO. Managing
the feature submitted to cloudstack against tests submitted to marvin
is not a hard thing to do. We simply mirror the release branches in
marvin and submit tests there. In fact I wonder why we didn't have
this question when docs were separated? It doesn't work any
differently really.

What I would like to see provided by CloudStack is the ability to
upgrade all our test environments, staging environments, UATs, what
have you in continuous integration and have tests run on the
upgraded setup. That allows incrementally testing CloudStack the way
users do it. The current design of installing everything from scratch,
redoing the testbed for each automated test run is mostly a workaround
for that inability. If we had this ability tests written in marvin can
be run against live setups at all times as and when features merge to
master.

It has always been my goal to reduce the barrier to writing tests for
cloudstack and that's entirely the goal of the refactor as well. So I
hope I have spent my time well. Since I can't be after people to write
tests, and setup mandates, I aim to address the technical difficulty
in writing tests through this refactor and separation.  

PS: David, by hosted docs I mean something like
http://pythonhosted.org/cloudmonkey. Lives in the repo but read from
the webpage.

-- 
Prasanna.,


Powered by BigRock.com

[ACS4.2.1] UI tickets

[Abhinandan Prateek]

There are around ~72 UI tickets open. Most of them are major, can documentation 
guys go thru the tickets and start picking them.

Following Jira query will give the tickets that are open.

project = CLOUDSTACK AND resolution = Unresolved AND fixVersion in ("4.2.0", 
"4.2.1") AND text ~ “UI"

-abhi

Re: Review Request 14451: Some test for NetUtils

[Saksham Srivastava]

> On Oct. 4, 2013, 6:43 a.m., Saksham Srivastava wrote:
> > utils/test/com/cloud/utils/net/NetUtilsTest.java, line 225
> > 
> >
> > What I am suggesting is that add a method something like
> > boolean isEqual(var1, var2) {
> > if var1 == var2 
> > return true
> > return false
> > }
> > 
> > Now you can add cases like
> > assertFalse(isEqual(15, NetUtils.mac2Long("00:00:00:00:00:ff")))
> > 
> > This is just one implementation as there is no assertNotEquals() method 
> > provided by Junit. You could certainly add better implementations. 
> > We generally use assertFalse for boolean methods.
> > You can have a look at testIsSameIpRange():
> >  // Check for 2 different CIDRs and different IP Range
> > assertFalse(NetUtils.isSameIpRange(cidrFirst, cidrThird));
> > 
> > // Check for Incorrect format of CIDR
> > assertFalse(NetUtils.isSameIpRange(cidrFirst, "10.3.6.5/50"));
> > 
> > Just a few such cases will suffice.
> >
> 
> Laszlo Hornyak wrote:
> Ok... what I do not quite understand is what the purpose of this negative 
> test is. There is already test for mac2Long("...") must be equal long 
> , so a negative test seems to be a test that proves "15 != 16 or any 
> other long value, 15 only equals to 15" :)

Actually the reason I suggested to add the test case is because we found an 
interesting scenario, 2 CIDRs 10.0.151.0/20 and 10.0.144.0/20 look different 
but have the same IP range, so isSameIpRange() will return true, however 
isNetworkAWithinNetworkB() will hint you an different thing.

I just gave you a basic example of implementing assertNotEquals(), the patch 
looks good to me even without adding these changes.
You already have added ip2LongBadIp() which tests for a incorrect format of IP 
address :)


- Saksham


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14451/#review26660
---


On Oct. 4, 2013, 11:12 a.m., Laszlo Hornyak wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14451/
> ---
> 
> (Updated Oct. 4, 2013, 11:12 a.m.)
> 
> 
> Review request for cloudstack.
> 
> 
> Bugs: CLOUDSTACK-4807
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> - tests added for mac2Long, byte2Mac, long2Mac, ip2Long and long2Ip
> - isWindows static method removed - it was not used
> - minor changes in the implementations of the tested methods
> 
> 
> Diffs
> -
> 
>   utils/src/com/cloud/utils/net/NetUtils.java 1e72e22 
>   utils/test/com/cloud/utils/net/NetUtilsTest.java 3cfc98f 
> 
> Diff: https://reviews.apache.org/r/14451/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> Laszlo Hornyak
> 
>

[ACS4.2.1] Doc tickets

[Abhinandan Prateek]

There are around ~47 Documentation tickets open. Most of them are major, can 
documentation guys go thru the tickets and start picking them.

Following Jira query will give the tickets that are open.

project = CLOUDSTACK AND resolution = Unresolved AND fixVersion in ("4.2.0", 
"4.2.1") AND text ~ “Doc"

-abhi

Re: docs build with publican

[Daan Hoogland]

works slow but like a charme. no tinkering needed for creating the devguide
in pdf format. I am about to try the rest in pdf+html

thanks Sebastien


On Fri, Oct 4, 2013 at 12:58 PM, sebgoa  wrote:

>
> On Oct 4, 2013, at 12:50 PM, Daan Hoogland 
> wrote:
>
> > Does anybody have a link to a howto that is useful for building the docs
> on
> > a mac (osx 10.6)?
> >
>
>
> http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.1.1/html/Developers_Guide/installing-publican.html
>
> installing publican on OSX can be a bit tricky:
> https://fedorahosted.org/publican/wiki/Installing_OSX
>
> I have had to modify the perl script a bit.
>
> > thanks,
> > Daan
>
>

[ACS4.2.1] Automation tickets

[Abhinandan Prateek]

There are around ~110 Automation tickets open. Most of them are major, can 
automation guys go thru the tickets and start picking them.

Following Jira query will give the tickets that are open.

project = CLOUDSTACK AND resolution = Unresolved AND fixVersion in ("4.2.0", 
"4.2.1") AND text ~ “Automation"

-abhi