Re: Libvirt-java 0.5.0 has been released

[Wei ZHOU]

try backup and remove the jna.jar in /usr/share/java

2013/9/29, Mike Tutkowski :
> Hi Wei,
>
> So, re-installing cloudstack-common allowed cloudstack-agent to install
> successfuly. Thanks
>
> I'm still having what appears to be Libvirt-related issues on Ubuntu 12.04
> with master.
>
> Any thoughts on this?
>
> log4j:WARN No appenders could be found for logger
> (org.apache.commons.httpclient.params.DefaultHttpParams).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
> more info.
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:243)
> Caused by: java.lang.NoSuchMethodError: com.sun.jna.Native.free(J)V
> at org.libvirt.Library.free(Unknown Source)
> at org.libvirt.Connect.getCapabilities(Unknown Source)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.IsHVMEnabled(LibvirtComputingResource.java:4533)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.configure(LibvirtComputingResource.java:747)
> at com.cloud.agent.Agent.(Agent.java:161)
> at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:421)
> at com.cloud.agent.AgentShell.launchAgentFromClassInfo(AgentShell.java:376)
> at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:357)
> at com.cloud.agent.AgentShell.start(AgentShell.java:454)
> ... 5 more
> Cannot start daemon
> Service exit with a return value of 5
>
>
> On Fri, Sep 27, 2013 at 1:45 PM, Mike Tutkowski <
> mike.tutkow...@solidfire.com> wrote:
>
>> Looks like listNetworks() is called a couple times in
>> cloudstack-agent-upgrade:
>>
>> if __name__ == '__main__':
>>netlib = networkConfig()
>>bridges = netlib.listNetworks()
>>bridges = filter(isOldStyleBridge, bridges)
>>for br in bridges:
>>enslavedDev = netlib.getEnslavedDev(br, 1)
>>if enslavedDev is not None:
>>upgradeBridgeName(br, enslavedDev)
>>
>>bridges = netlib.listNetworks()
>>bridges = filter(isOldStyleBridge, bridges)
>>if len(bridges) > 0:
>>print("Warning: upgrade is not finished, still some bridges have
>> the old style name:" + str(bridges))
>>else:
>>print("Upgrade succeed")
>>
>>
>> On Fri, Sep 27, 2013 at 1:43 PM, Mike Tutkowski <
>> mike.tutkow...@solidfire.com> wrote:
>>
>>> Kind of made progress here. :) Looks like we have another problem. Is it
>>> possible another item is missing from the scripts?
>>>
>>> mtutkowski@ubuntu:~$ sudo apt-get install cloudstack-agent
>>> Reading package lists... Done
>>> Building dependency tree
>>> Reading state information... Done
>>> The following NEW packages will be installed:
>>>   cloudstack-agent
>>> 0 upgraded, 1 newly installed, 0 to remove and 468 not upgraded.
>>> Need to get 39.1 MB of archives.
>>> After this operation, 43.6 MB of additional disk space will be used.
>>> WARNING: The following packages cannot be authenticated!
>>>   cloudstack-agent
>>> Install these packages without verification [y/N]? y
>>> Get:1 http://localhost/cloudstack/repo/ binary/ cloudstack-agent 4.3.0
>>> [39.1 MB]
>>> Fetched 39.1 MB in 0s (50.5 MB/s)
>>> Selecting previously unselected package cloudstack-agent.
>>> (Reading database ... 168800 files and directories currently installed.)
>>> Unpacking cloudstack-agent (from .../cloudstack-agent_4.3.0_all.deb) ...
>>> Processing triggers for ureadahead ...
>>> Setting up cloudstack-agent (4.3.0) ...
>>> Traceback (most recent call last):
>>>   File "/usr/bin/cloudstack-agent-upgrade", line 51, in 
>>> bridges = netlib.listNetworks()
>>> AttributeError: networkConfig instance has no attribute 'listNetworks'
>>> dpkg: error processing cloudstack-agent (--configure):
>>>  subprocess installed post-installation script returned error exit
>>> status
>>> 1
>>> Errors were encountered while processing:
>>>  cloudstack-agent
>>> E: Sub-process /usr/bin/dpkg returned an error code (1)
>>>
>>>
>>> On Fri, Sep 27, 2013 at 8:26 AM, Mike Tutkowski <
>>> mike.tutkow...@solidfire.com> wrote:
>>>
 Great - thanks, Wei!


 On Fri, Sep 27, 2013 at 2:34 AM, Wei ZHOU 
 wrote:

> Mike,
> cloudstack-agent-upgrade is missing in cloudstack-agent installation.
> I added it in commit a6bfd9602129d8ae308ba58f36623c04826e15ca
>
> [root@weizhou-centos master(4.2)]# git show
> a6bfd9602129d8ae308ba58f36623c04826e15ca
> commit a6bfd9602129d8ae308ba58f36623c04826e15ca
> Author: Wei Zhou 
> Date:   Fri Sep 27 10:31:54 2013 +0200
> add missing cloudstack-agent-upgrade in cloudstack-agent debian
> packages
> diff --git a/debian/cloudstack-agent.i

Re: Libvirt-java 0.5.0 has been released

[Mike Tutkowski]

Hi Wei,

So, re-installing cloudstack-common allowed cloudstack-agent to install
successfuly. Thanks

I'm still having what appears to be Libvirt-related issues on Ubuntu 12.04
with master.

Any thoughts on this?

log4j:WARN No appenders could be found for logger
(org.apache.commons.httpclient.params.DefaultHttpParams).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:243)
Caused by: java.lang.NoSuchMethodError: com.sun.jna.Native.free(J)V
at org.libvirt.Library.free(Unknown Source)
at org.libvirt.Connect.getCapabilities(Unknown Source)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.IsHVMEnabled(LibvirtComputingResource.java:4533)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.configure(LibvirtComputingResource.java:747)
at com.cloud.agent.Agent.(Agent.java:161)
at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:421)
at com.cloud.agent.AgentShell.launchAgentFromClassInfo(AgentShell.java:376)
at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:357)
at com.cloud.agent.AgentShell.start(AgentShell.java:454)
... 5 more
Cannot start daemon
Service exit with a return value of 5


On Fri, Sep 27, 2013 at 1:45 PM, Mike Tutkowski <
mike.tutkow...@solidfire.com> wrote:

> Looks like listNetworks() is called a couple times in
> cloudstack-agent-upgrade:
>
> if __name__ == '__main__':
>netlib = networkConfig()
>bridges = netlib.listNetworks()
>bridges = filter(isOldStyleBridge, bridges)
>for br in bridges:
>enslavedDev = netlib.getEnslavedDev(br, 1)
>if enslavedDev is not None:
>upgradeBridgeName(br, enslavedDev)
>
>bridges = netlib.listNetworks()
>bridges = filter(isOldStyleBridge, bridges)
>if len(bridges) > 0:
>print("Warning: upgrade is not finished, still some bridges have
> the old style name:" + str(bridges))
>else:
>print("Upgrade succeed")
>
>
> On Fri, Sep 27, 2013 at 1:43 PM, Mike Tutkowski <
> mike.tutkow...@solidfire.com> wrote:
>
>> Kind of made progress here. :) Looks like we have another problem. Is it
>> possible another item is missing from the scripts?
>>
>> mtutkowski@ubuntu:~$ sudo apt-get install cloudstack-agent
>> Reading package lists... Done
>> Building dependency tree
>> Reading state information... Done
>> The following NEW packages will be installed:
>>   cloudstack-agent
>> 0 upgraded, 1 newly installed, 0 to remove and 468 not upgraded.
>> Need to get 39.1 MB of archives.
>> After this operation, 43.6 MB of additional disk space will be used.
>> WARNING: The following packages cannot be authenticated!
>>   cloudstack-agent
>> Install these packages without verification [y/N]? y
>> Get:1 http://localhost/cloudstack/repo/ binary/ cloudstack-agent 4.3.0
>> [39.1 MB]
>> Fetched 39.1 MB in 0s (50.5 MB/s)
>> Selecting previously unselected package cloudstack-agent.
>> (Reading database ... 168800 files and directories currently installed.)
>> Unpacking cloudstack-agent (from .../cloudstack-agent_4.3.0_all.deb) ...
>> Processing triggers for ureadahead ...
>> Setting up cloudstack-agent (4.3.0) ...
>> Traceback (most recent call last):
>>   File "/usr/bin/cloudstack-agent-upgrade", line 51, in 
>> bridges = netlib.listNetworks()
>> AttributeError: networkConfig instance has no attribute 'listNetworks'
>> dpkg: error processing cloudstack-agent (--configure):
>>  subprocess installed post-installation script returned error exit status
>> 1
>> Errors were encountered while processing:
>>  cloudstack-agent
>> E: Sub-process /usr/bin/dpkg returned an error code (1)
>>
>>
>> On Fri, Sep 27, 2013 at 8:26 AM, Mike Tutkowski <
>> mike.tutkow...@solidfire.com> wrote:
>>
>>> Great - thanks, Wei!
>>>
>>>
>>> On Fri, Sep 27, 2013 at 2:34 AM, Wei ZHOU  wrote:
>>>
 Mike,
 cloudstack-agent-upgrade is missing in cloudstack-agent installation.
 I added it in commit a6bfd9602129d8ae308ba58f36623c04826e15ca

 [root@weizhou-centos master(4.2)]# git show
 a6bfd9602129d8ae308ba58f36623c04826e15ca
 commit a6bfd9602129d8ae308ba58f36623c04826e15ca
 Author: Wei Zhou 
 Date:   Fri Sep 27 10:31:54 2013 +0200
 add missing cloudstack-agent-upgrade in cloudstack-agent debian
 packages
 diff --git a/debian/cloudstack-agent.install
 b/debian/cloudstack-agent.install
 index a3cc869..d708514 100644
 --- a/debian/cloudstack-agent.install
 +++ b/debian/cloudstack-agent.install
 @@ -21,6 +21,7 @@
  /etc/init.d/cloudstack-agent
  /usr/bin/cloudstack-

Re: [ANNOUNCE] New Committer: Darren Shepherd

[Alena Prokharchyk]

Congrats, Darren!

-Alena

From: Chip Childers 
mailto:chip.child...@sungard.com>>
Reply-To: "dev@cloudstack.apache.org" 
mailto:dev@cloudstack.apache.org>>
Date: Saturday, September 28, 2013 5:24 AM
To: "dev@cloudstack.apache.org" 
mailto:dev@cloudstack.apache.org>>
Subject: [ANNOUNCE] New Committer: Darren Shepherd

The Project Management Committee (PMC) for Apache CloudStack
has asked Darren Shepherd to become a committer and we are pleased to
announce that he has accepted.

Being a committer allows many contributors to contribute more
autonomously. For developers, it makes it easier to submit changes and
eliminates the need to have contributions reviewed via the patch
submission process. Whether contributions are development-related or
otherwise, it is a recognition of a contributor's participation in the
project and commitment to the project and the Apache Way.

Please join me in congratulating Darren!

-chip
on behalf of the CloudStack PMC

Re: Scalable Backup and Recovery

[Darren Shepherd]

Based on your use cases it sounds like what your asking for is the
ability to create a selection criteria for scheduled snapshots.  So as
long as your VM/Volume matches that criteria it will be backed up at
some given time.  I think that would be useful because a user could
say something like "all volumes in network 'production' should be
backed up every night."

So if such a thing was implemented it seems like if the storage
provider implemented some capability to handle multiple snapshots at
once, then it could be passed all volumes at once and it would do
something intelligent.

Now reading between the lines, it seems like you're looking for some
use case to exploit some functionality in netapp.  I'll tell you what
I'd like to see implemented.  Having ran netapp in the past with CS we
ran into this conundrum.  We would take snapshots on the filter, but
in reality they were pretty useless.  If you ever needed to rollback
to the snapshot, your screwed.  Things have changed since the
snapshot, VMs were created, deleted, VM snapshots had occurred, etc.
So if you rollback the metadata in CS is completely out of sync now.
What I think would be a great feature is to be able to do StoragePool
snapshots.  Now doing the snapshot is simple, the really complex thing
is how to implement the "StoragePool revert to snapshot"
functionality.  If somebody could do that, that would be awesome.

Darren

On Sat, Sep 28, 2013 at 5:41 PM, SuichII, Christopher
 wrote:
> Well, yes, in part. By scalable I mean that if CloudStack is expected to be 
> able to manage such a large number of vms, it should be able to backup and 
> recover those vms with minimal effort. Doing things one at a time does not 
> necessarily scale well when you're talking about a cloud infrastructure.
>
>> Also certain hypervisors have various quirks which stand in the way of an
>> efficient solution.
>
> I absolutely agree. This is where the storage subsystem API comes in. 
> Creating backups for some storage providers can be much faster, easier and 
> more efficient than hypervisor. As the storage subsystem API gains more 
> traction and true backup and recovery becomes available, I think we'll begin 
> to see people asking why things must be done one at a time. The use cases I 
> listed below would help us get ahead of the curve and have these features I 
> predict people will be asking for (and it sounds like Kelcey is asking for it 
> now!).
>
> --
> Chris Suich
> chris.su...@netapp.com
> NetApp Software Engineer
> Data Center Platforms – Cloud Solutions
> Citrix, Cisco & Red Hat
>
> On Sep 27, 2013, at 6:38 PM, Chiradeep Vittal  
> wrote:
>
>> Ah I see. You mean a "scalable user experience".
>>
>> The actual scalability of the snapshot process itself is limited by
>> available disk and network bandwidth.
>> Also certain hypervisors have various quirks which stand in the way of an
>> efficient solution.
>>
>> On 9/27/13 10:27 AM, "SuichII, Christopher"  wrote:
>>
>>> I'd like to start a discussion around the direction of scalable backup
>>> and recovery in CloudStack. Currently, the only want to backup and
>>> recover vms is by setting up a schedule or manually snapshotting up
>>> individual vm disks or manually snapshotting vms. Unfortunately, I don't
>>> believe this is a very scalable solution. What if a user wants all of
>>> their vm disks to be backed up on the same schedule? What if a domain
>>> administrator wants all of the vms in their domain to be backed up on the
>>> same schedule or to manually backup every vm in their domain?
>>>
>>> Here are some use cases I see for helping to scale things up:
>>> -Scheduled and manual backup of 1 to all of a user's vms and vm disks
>>> -Scheduled and manual backup of 1 to all of a domain's vms and vm disks
>>> (by a domain admin)
>>> -Scheduled and manual backup of 1 to all vms and vm disks on primary
>>> storage (by a cloud admin) - this one is tougher to find a valid use case
>>> for
>>> -Backup schedules attached to service offerings
>>>
>>> I know I previously started a discussion about backing up multiple vm
>>> disks at once, but I think these use cases, broken down by user type
>>> (user, domain admin and admin), should help clear things up and show the
>>> utility of being able to backup multiple objects at once.
>>>
>>> Thanks!
>>> Chris
>>> --
>>> Chris Suich
>>> chris.su...@netapp.com
>>> NetApp Software Engineer
>>> Data Center Platforms ­ Cloud Solutions
>>> Citrix, Cisco & Red Hat
>>>
>>
>

my outstanding patches on review board

[Darren Shepherd]

I have a bunch of outstanding patches on review board.  Now that I'm a
committer I'm going to commit a bunch of then that I consider small
and non-impacting.  I know this would normally be bad form, so I hope
you'll excuse me this one time.  I really need to get all those
committed because they are blocking me from getting all the Spring
stuff prepared for review.  The longer I wait to get the spring stuff
in, the more maintenance I have to do as people change stuff (like
hyper-v, contrail, vxlan).

Darren

my two cents on RBAC/authorization

[Darren Shepherd]

I've noticed there's a rbac branch and things are being committed
there.  I didn't see any documentation about the design or anything
(maybe it exists and I looked in the wrong place), so I'm just going
to give you my two cents on authorization systems.  Hopefully this
falls in line with what is being implemented, if not, at least we'll
avoid the awkward conversation when its finish when I say the code is
marginally useful and should be rewritten.

When talking about authorization there's a bunch of terms like
principal, permission, subject, action, policy, etc.  I want to focus
on policy.  Policy is central to an authorization system.  The policy
is the collection of permissions that grant or deny access to some
resource or action for a given subject.  RBAC is a really just a means
to generate a policy.  Once you know the user, group, roles, and the
permissions of those entities that aggregation of information forms
the policy.  You then take that policy and use it determine if the
given resource/action is granted/denied to a particular subject.

It is really important that policy is a first class object in an
authorization system.  This is important to understand because usually
in a big fat enterprise-y company, they really want you to enforce the
policy, but not necessarily maintain it.  For example, you'll go to
your fortune 500 company and they'll tell you they need RBAC.  So you
go and create an RBAC system.  The problem is that the fortune 500
company probably already has a RBAC system, and its probably AD based.
 So when they said they need RBAC, the really meant you need to
enforce RBAC.  If you implemented RBAC -> Policy -> Authorization,
your good, if you implemented RBAC - > Authorization, your kinda
screwed. Now you need to create a system to sync the two RBACs.  And
keeping data in two places and trying to sync them is never a good
idea.  Now if you implemented your system as having a policy as a
first class object, you can just swap your RBAC for theirs and all is
still swell.

So if I was to implement this, this is how I'd do it.  (And if this
sounds a lot like IAM, its because it is.  If Amazon got anything
right, it's IAM).  The authenticator should be able to implement
another interface that allows it to supply a Policy object during
authentication.  This is logical in that the authentication systems
quite often hold authorization information too.  If the authenticator
doesn't implement the interface we fall back to generating the policy
ourself.  The policy is then consulted to see if the API command and
the resulting entities are granted/denied.  So far none of this has
anything to do with RBAC.  So the RBAC is implemented in that default
fallback implemenation that generates the policy.  You map the current
user/account to groups and roles and get the policies of those
entities to construct the policy.

Now for storing the policies I wouldn't do it in a traditional
normalized form.  All you need is tables for user/group/role and the
mappings for each.  The for user, group, and role you can specify a
policy JSON blob and that gets stored in the database as a mediumtext
field on the user/group/role row.  From an API perspective (just like
IAM), you just let people upload the JSON blobs for each.

So if we do it this way, we can have our own simple RBAC but then be
able to plug into far more complex and powerful authorization systems.
 Hopefully that all made sense.

Darren

Re: Review Request 14360: Make getName() match the constant names for Network Capability, and Service

[Darren Shepherd]

> On Sept. 27, 2013, 8:21 p.m., Alena Prokharchyk wrote:
> > Darren, changes in Network.java are going to break API compatibility as 
> > capabilities parameters are used in createNetworkOffering API call 
> > (serviceCapabilitystList param). CS UI and third party UI will get impacted 
> > as well.
> > If we have to change the name, its gotta be the capability parameter name, 
> > not its name value.
> 
> daan Hoogland wrote:
> This in turn, could have impact on other uses, like in a db could it?
> 
> @Darren: you will need a lot more then just a (unit)test for the getname()
> 
> Alena Prokharchyk wrote:
> Won't impact the DB as there is no direct ref between capability name and 
> DB field name /value.

I already discarded this.  I didn't realize the current string values are what 
we desire and the Java constant names that are out of sync are really the 
"typos."  So I have no complaint with the current implementation and changing 
the constant and all references to it would be a waste of time.


- Darren


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14360/#review26447
---


On Sept. 27, 2013, 5:43 p.m., Darren Shepherd wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14360/
> ---
> 
> (Updated Sept. 27, 2013, 5:43 p.m.)
> 
> 
> Review request for cloudstack, Alena Prokharchyk and Chiradeep Vittal.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> VPCVirtualRouter is still left as inconsitent as changing that name will be 
> impactful.  The unit test checked in will still ensure that future providers 
> match though
> 
> 
> Diffs
> -
> 
>   api/src/com/cloud/network/Network.java 49f380b 
>   api/test/com/cloud/network/NetworkTest.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/14360/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Darren Shepherd
> 
>

Re: Review Request 14381: KVM: add connect/disconnect capabilities to StorageAdaptors so that external storage services can attach/detach devices on-demand

[Marcus Sorensen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14381/
---

(Updated Sept. 29, 2013, 3:40 a.m.)


Review request for cloudstack, edison su and Mike Tutkowski.


Changes
---

Updated notes


Repository: cloudstack-git


Description (updated)
---

With custom storage plugins comes the need to prep the KVM host prior to 
utilizing the disks. e.g. an iscsi initiator needs to log into the target and 
scan for the lun before it can be used on the host. This patch is an example I 
developed against 4.2, minor changes may be necessary to apply to master, but I 
want to share with others who are working on storage so they can ensure it 
works for them. Please tweak as you see fit.

MigrateCommand: pass vmTO object so we can see which disks/storage pool types 
belong to the vm when migrating a VM. This facilitates being able to call 
disconnectPhysicalDisksViaVmSpec

VirtualMachineManagerImpl: pass VirtualMachineTO when migrating so that we can 
see which disks belong to the VM and what storage pools/adaptors should be used

LibvirtComputingResource: add calls KVMStoragePoolManager's 
connectPhysicalDiskViaVmSpec and disconnectPhysicalDiskViaVmSpec calls where 
appropriate (when starting a vm, migrating a vm). Ensure that we create 'raw' 
format XML disk definitions when the storage format is RAW. Move cleanupDisk 
logic to storage adaptors so that each adaptor type can clean up its disks in 
is own way.

KVMStoragePoolManager:  add connectPhysicalDisk, disconnectPhysicalDisk, 
connectPhysicalDiskViaVmSpec, disconnectPhysicalDiskViaVmSpec, 
disconnectPhysicalDiskByPath. These all call the specific StorageAdaptor's 
connectPhysicalDisk, disconnectPhysicalDisk, or disconnectPhysicalDiskByPath 
calls.

KVMStorageProcessor: Call connectPhysicalDisk/disconnectPhysicalDisk on the 
storage adaptor. Whether or not this is implemented is up to the storage 
adaptor.

LibvirtStorageAdaptor: implement dummy 
connectPhysicalDisk/disconnectPhysicalDisk, move cleanupDisk logic from 
LibvirtComputingResource to disconnectPhysicalDiskByPath

StorageAdaptor: define 
connectPhysicalDisk/disconnectPhysicalDisk/disconnectPhysicalDiskByPath in the 
interface


Diffs
-

  core/src/com/cloud/agent/api/MigrateCommand.java 5042b8c 
  
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
 3ee811f 
  
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/KVMStoragePoolManager.java
 e09c9ba 
  
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java
 c69f9b0 
  
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java
 123a9f1 
  
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/StorageAdaptor.java
 4956d8d 
  server/src/com/cloud/vm/VirtualMachineManagerImpl.java d46bbb0 

Diff: https://reviews.apache.org/r/14381/diff/


Testing
---

Basic testing with my storage adaptor


Thanks,

Marcus Sorensen

Re: Review Request 14360: Make getName() match the constant names for Network Capability, and Service

[Alena Prokharchyk]

> On Sept. 27, 2013, 8:21 p.m., Alena Prokharchyk wrote:
> > Darren, changes in Network.java are going to break API compatibility as 
> > capabilities parameters are used in createNetworkOffering API call 
> > (serviceCapabilitystList param). CS UI and third party UI will get impacted 
> > as well.
> > If we have to change the name, its gotta be the capability parameter name, 
> > not its name value.
> 
> daan Hoogland wrote:
> This in turn, could have impact on other uses, like in a db could it?
> 
> @Darren: you will need a lot more then just a (unit)test for the getname()

Won't impact the DB as there is no direct ref between capability name and DB 
field name /value.


- Alena


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14360/#review26447
---


On Sept. 27, 2013, 5:43 p.m., Darren Shepherd wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14360/
> ---
> 
> (Updated Sept. 27, 2013, 5:43 p.m.)
> 
> 
> Review request for cloudstack, Alena Prokharchyk and Chiradeep Vittal.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> VPCVirtualRouter is still left as inconsitent as changing that name will be 
> impactful.  The unit test checked in will still ensure that future providers 
> match though
> 
> 
> Diffs
> -
> 
>   api/src/com/cloud/network/Network.java 49f380b 
>   api/test/com/cloud/network/NetworkTest.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/14360/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Darren Shepherd
> 
>

Re: Scalable Backup and Recovery

[SuichII, Christopher]

Well, yes, in part. By scalable I mean that if CloudStack is expected to be 
able to manage such a large number of vms, it should be able to backup and 
recover those vms with minimal effort. Doing things one at a time does not 
necessarily scale well when you're talking about a cloud infrastructure.

> Also certain hypervisors have various quirks which stand in the way of an
> efficient solution.

I absolutely agree. This is where the storage subsystem API comes in. Creating 
backups for some storage providers can be much faster, easier and more 
efficient than hypervisor. As the storage subsystem API gains more traction and 
true backup and recovery becomes available, I think we'll begin to see people 
asking why things must be done one at a time. The use cases I listed below 
would help us get ahead of the curve and have these features I predict people 
will be asking for (and it sounds like Kelcey is asking for it now!).

-- 
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Sep 27, 2013, at 6:38 PM, Chiradeep Vittal  
wrote:

> Ah I see. You mean a "scalable user experience".
> 
> The actual scalability of the snapshot process itself is limited by
> available disk and network bandwidth.
> Also certain hypervisors have various quirks which stand in the way of an
> efficient solution.
> 
> On 9/27/13 10:27 AM, "SuichII, Christopher"  wrote:
> 
>> I'd like to start a discussion around the direction of scalable backup
>> and recovery in CloudStack. Currently, the only want to backup and
>> recover vms is by setting up a schedule or manually snapshotting up
>> individual vm disks or manually snapshotting vms. Unfortunately, I don't
>> believe this is a very scalable solution. What if a user wants all of
>> their vm disks to be backed up on the same schedule? What if a domain
>> administrator wants all of the vms in their domain to be backed up on the
>> same schedule or to manually backup every vm in their domain?
>> 
>> Here are some use cases I see for helping to scale things up:
>> -Scheduled and manual backup of 1 to all of a user's vms and vm disks
>> -Scheduled and manual backup of 1 to all of a domain's vms and vm disks
>> (by a domain admin)
>> -Scheduled and manual backup of 1 to all vms and vm disks on primary
>> storage (by a cloud admin) - this one is tougher to find a valid use case
>> for
>> -Backup schedules attached to service offerings
>> 
>> I know I previously started a discussion about backing up multiple vm
>> disks at once, but I think these use cases, broken down by user type
>> (user, domain admin and admin), should help clear things up and show the
>> utility of being able to backup multiple objects at once.
>> 
>> Thanks!
>> Chris
>> -- 
>> Chris Suich
>> chris.su...@netapp.com
>> NetApp Software Engineer
>> Data Center Platforms ­ Cloud Solutions
>> Citrix, Cisco & Red Hat
>> 
> 

Re: Review Request 14360: Make getName() match the constant names for Network Capability, and Service

[Darren Shepherd]

I discarded the patch.  The purpose of throwing it up on review board was so 
that people could comment on the impact.  And the impact is bad.  

Darren

> On Sep 28, 2013, at 4:43 AM, "daan Hoogland"  wrote:
> 
> 
> This is an automatically generated e-mail. To reply, visit: 
> https://reviews.apache.org/r/14360/
> 
> On September 27th, 2013, 8:21 p.m. UTC, Alena Prokharchyk wrote:
> 
> Darren, changes in Network.java are going to break API compatibility as 
> capabilities parameters are used in createNetworkOffering API call 
> (serviceCapabilitystList param). CS UI and third party UI will get impacted 
> as well.
> If we have to change the name, its gotta be the capability parameter name, 
> not its name value.
> This in turn, could have impact on other uses, like in a db could it?
> 
> @Darren: you will need a lot more then just a (unit)test for the getname()
> 
> - daan
> 
> 
> On September 27th, 2013, 5:43 p.m. UTC, Darren Shepherd wrote:
> 
> Review request for cloudstack, Alena Prokharchyk and Chiradeep Vittal.
> By Darren Shepherd.
> Updated Sept. 27, 2013, 5:43 p.m.
> 
> Repository: cloudstack-git
> Description
> 
> VPCVirtualRouter is still left as inconsitent as changing that name will be 
> impactful.  The unit test checked in will still ensure that future providers 
> match though
> Diffs
> 
> api/src/com/cloud/network/Network.java (49f380b)
> api/test/com/cloud/network/NetworkTest.java (PRE-CREATION)
> View Diff

Re: Review Request 14398: NumbersUtil cleanup

[daan Hoogland]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14398/#review26462
---


Looks alright, please add spark404 to reviewers. just in case I missed 
something.

- daan Hoogland


On Sept. 28, 2013, 4:38 p.m., Laszlo Hornyak wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14398/
> ---
> 
> (Updated Sept. 28, 2013, 4:38 p.m.)
> 
> 
> Review request for cloudstack.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> - removed methods that were not used
> - parseLong, parseInt and parseFloat replaced with the commons-lang 
> NumberUtils call
> - Test for the remaining methods
> 
> 
> Diffs
> -
> 
>   utils/src/com/cloud/utils/NumbersUtil.java 035239a 
>   utils/test/com/cloud/utils/NumbersUtilTest.java f60f58a 
> 
> Diff: https://reviews.apache.org/r/14398/diff/
> 
> 
> Testing
> ---
> 
> tests included
> 
> 
> Thanks,
> 
> Laszlo Hornyak
> 
>

Re: [ANNOUNCE] New Committer: Darren Shepherd

[Daan Hoogland]

welcome Darren, best


On Sat, Sep 28, 2013 at 5:14 PM, Rajesh Battala
wrote:

> Hearty Congratulations Darren.! :)
>
> -Original Message-
> From: Chip Childers [mailto:chip.child...@sungard.com]
> Sent: Saturday, September 28, 2013 5:54 PM
> To: dev@cloudstack.apache.org
> Subject: [ANNOUNCE] New Committer: Darren Shepherd
>
> The Project Management Committee (PMC) for Apache CloudStack has asked
> Darren Shepherd to become a committer and we are pleased to announce that
> he has accepted.
>
> Being a committer allows many contributors to contribute more
> autonomously. For developers, it makes it easier to submit changes and
> eliminates the need to have contributions reviewed via the patch submission
> process. Whether contributions are development-related or otherwise, it is
> a recognition of a contributor's participation in the project and
> commitment to the project and the Apache Way.
>
> Please join me in congratulating Darren!
>
> -chip
> on behalf of the CloudStack PMC
>

Re: [jira] [Created] (CLOUDSTACK-4756) Can't install DEBs properly on Ubuntu 12.04

[Wei ZHOU]

Mike,
did you reinstall cloudstack-common?

2013/9/28, Mike Tutkowski (JIRA) :
> Mike Tutkowski created CLOUDSTACK-4756:
> --
>
>  Summary: Can't install DEBs properly on Ubuntu 12.04
>  Key: CLOUDSTACK-4756
>  URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4756
>  Project: CloudStack
>   Issue Type: Bug
>   Security Level: Public (Anyone can view this level - this is the
> default.)
>   Components: KVM
> Affects Versions: Future
>  Environment: Ubuntu 12.04
> Reporter: Mike Tutkowski
>  Fix For: Future
>
>
> A script is missing the listNetworks() method as is seen below:
>
> Kind of made progress here. :) Looks like we have another problem. Is it
> possible another item is missing from the scripts?
>
> mtutkowski@ubuntu:~$ sudo apt-get install cloudstack-agent
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> The following NEW packages will be installed:
>   cloudstack-agent
> 0 upgraded, 1 newly installed, 0 to remove and 468 not upgraded.
> Need to get 39.1 MB of archives.
> After this operation, 43.6 MB of additional disk space will be used.
> WARNING: The following packages cannot be authenticated!
>   cloudstack-agent
> Install these packages without verification [y/N]? y
> Get:1 http://localhost/cloudstack/repo/ binary/ cloudstack-agent 4.3.0 [39.1
> MB]
> Fetched 39.1 MB in 0s (50.5 MB/s)
> Selecting previously unselected package cloudstack-agent.
> (Reading database ... 168800 files and directories currently installed.)
> Unpacking cloudstack-agent (from .../cloudstack-agent_4.3.0_all.deb) ...
> Processing triggers for ureadahead ...
> Setting up cloudstack-agent (4.3.0) ...
> Traceback (most recent call last):
>   File "/usr/bin/cloudstack-agent-upgrade", line 51, in 
> bridges = netlib.listNetworks()
> AttributeError: networkConfig instance has no attribute 'listNetworks'
>
> Looks like listNetworks() is called a couple times in
> cloudstack-agent-upgrade:
>
> if __name__ == '__main__':
>netlib = networkConfig()
>bridges = netlib.listNetworks()
>bridges = filter(isOldStyleBridge, bridges)
>for br in bridges:
>enslavedDev = netlib.getEnslavedDev(br, 1)
>if enslavedDev is not None:
>upgradeBridgeName(br, enslavedDev)
>
>bridges = netlib.listNetworks()
>bridges = filter(isOldStyleBridge, bridges)
>if len(bridges) > 0:
>print("Warning: upgrade is not finished, still some bridges have the
> old style name:" + str(bridges))
>else:
>print("Upgrade succeed")
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>

Review Request 14398: NumbersUtil cleanup

[Laszlo Hornyak]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14398/
---

Review request for cloudstack.


Repository: cloudstack-git


Description
---

- removed methods that were not used
- parseLong, parseInt and parseFloat replaced with the commons-lang NumberUtils 
call
- Test for the remaining methods


Diffs
-

  utils/src/com/cloud/utils/NumbersUtil.java 035239a 
  utils/test/com/cloud/utils/NumbersUtilTest.java f60f58a 

Diff: https://reviews.apache.org/r/14398/diff/


Testing
---

tests included


Thanks,

Laszlo Hornyak

RE: [ANNOUNCE] New committer: Rajesh Battala

[Rajesh Battala]

Thanks a lot Everybody for your Kind wishes. :)

Thanks
Rajesh Battala

-Original Message-
From: Animesh Chaturvedi [mailto:animesh.chaturv...@citrix.com] 
Sent: Thursday, September 26, 2013 10:22 AM
To: dev@cloudstack.apache.org
Subject: RE: [ANNOUNCE] New committer: Rajesh Battala

Congrats Rajesh

> -Original Message-
> From: Chip Childers [mailto:chip.child...@sungard.com]
> Sent: Tuesday, September 24, 2013 8:30 AM
> To: dev@cloudstack.apache.org
> Subject: [ANNOUNCE] New committer: Rajesh Battala
> 
> The Project Management Committee (PMC) for Apache CloudStack has asked 
> Rajesh Battala to become a committer and we are pleased to announce 
> that they have accepted.
> 
> Being a committer allows many contributors to contribute more 
> autonomously. For developers, it makes it easier to submit changes and 
> eliminates the need to have contributions reviewed via the patch 
> submission process. Whether contributions are development-related or 
> otherwise, it is a recognition of a contributor's participation in the 
> project and commitment to the project and the Apache Way.
> 
> Please join me in congratulating Rajesh!
> 
> -chip
> on behalf of the CloudStack PMC

RE: [Merge] Minimal Hyper-V Plugin

[Rajesh Battala]

+1
I had used the code and Agent code works fine.
Adding HyperV host, discovering local storage and Advance Networking for user 
VMs works fine ( in mixed environment ).

Thanks
Rajesh Battala

-Original Message-
From: Sebastien Goasguen [mailto:run...@gmail.com] 
Sent: Thursday, September 26, 2013 12:29 PM
To: dev@cloudstack.apache.org
Subject: Re: [Merge] Minimal Hyper-V Plugin

+1

I would love to see a screencast of how this works, once it's put in its own 
branch (and you back from paternity leave...congratulations :) )

On Sep 25, 2013, at 5:25 PM, Chiradeep Vittal  
wrote:

> +1 to put this into a branch off of master. Can merge into master 
> +after
> unit tests for the agent API.
> 
> 
> On 9/24/13 12:37 PM, "Donal Lafferty"  wrote:
> 
>> On paternity leave, so I don't get to these emails right away...
>> 
>>> -Original Message-
>>> From: Chiradeep Vittal [mailto:chiradeep.vit...@citrix.com]
>>> Sent: 20 September 2013 06:40
>>> To: dev@cloudstack.apache.org
>>> Subject: Re: [Merge] Minimal Hyper-V Plugin
>>> 
>>> Thanks for the preliminary testing.
>>> Questions:
>>> 1. More for the community: should the C# code be in a separate repo?
>>> According to the merge request, mono and maven can be used to build 
>>> the agent.
>> 
>> Silence == acceptance?
>> 
>>> 2. Packaging: how is the C# agent installed?
>> 
>> The agent is implemented to as a self-contained Windows Service, 
>> which is the Microsoft Windows equivalent of a Linux daemon.
>> 
>> To make the agent distributable, package the agent and an app.config 
>> consistent with your data center in an MSI.  WiX is the preferred 
>> tool (http://en.wikipedia.org/wiki/WiX ).  When executed, the MSI 
>> will add the agent to set of Windows Services.
>> 
>> To distribute and run this MSI, use Active Directory's GPO (global 
>> policy
>> object) service.  In typical deployments machines running Hyper-V 
>> will be domain joined.  Where machines are not domain joined, look at 
>> something like puppet.
>> 
>>> 3. What does minimal mean? What works? What doesn't? Local storage?
>>> Shared storage? Networking modes? Are the hypervisors supposed to be 
>>> clustered?
>> 
>> Minimal = create / start / stop / destroy a local storage VM in a 
>> QuickCloud network offering and CIFS secondary storage.
>> 
>> No clustering required.
>> 
>>> 4. How does one extend the "minimal" plugin?
>> 
>> Each CloudStack command has a corresponding an HTTP URI served by the 
>> agent.  These are written in ASP.NET MVC4.  Data received by the 
>> agent is kept in a JSON object graph.
>> 
>> E.g.
>> 
>>   // POST api/HypervResource/ReadyCommand
>>   [HttpPost]
>>   [ActionName(CloudStackTypes.ReadyCommand)]
>>   public JContainer ReadyCommand([FromBody]dynamic cmd)
>>   {
>>   using (log4net.NDC.Push(Guid.NewGuid().ToString()))
>>   {
>>   logger.Info(CloudStackTypes.ReadyCommand + 
>> cmd.ToString());
>>   object ansContent = new
>>   {
>>   result = true,
>>   details = (string)null
>>   };
>>   return ReturnCloudStackTypedJArray(ansContent,
>> CloudStackTypes.ReadyAnswer);
>>   }
>> 
>>   }
>> 
>> Therefore, to extend the plugin, add new HTTP URIs corresponding to 
>> missing commands, or extend the capabilities of existing commands.
>> 
>> I can follow up with an explanation in a blog entry.
>> 
>>> 5. Can the unit tests (at least those that test the agent API) be 
>>> run in a non- hyper-v environment?
>> 
>> Unit tests start the agent in a local process.  Provided Mono is 
>> installed on your system, the unit tests will run.  However, they 
>> will complain of bad output.
>> 
>>> 6. Is the RDP console you had earlier mentioned included in the merge?
>> 
>> Yes, but it serves no purpose at the moment.
>> 
>> If there is an IP clearance protocol to follow for this console, I 
>> would prefer to remove the console from the submission.
>> 
>>> 7. Any known issues?
>>> 
>> 
>> There seems to be a bug with local paths that include spaces.  I've 
>> asked Rajesh to provide a bug report, but it's unclear where to put 
>> this.  Can we use JIRA for code not merged, or should the bug appear in the 
>> comments.
>> 
>> 
>>> On 9/11/13 8:00 AM, "Donal Lafferty"  wrote:
>>> 
 Hi Rajesh,
 
 Thanks for spotting this problem with the Hyper-V Agent.  Sounds 
 like it should first URL decode the field.
 
 Can you update the review with details of your testing?
 
 I would need to know the command and which incoming field is 
 causing problems.  Also, can you add a serialised example of the 
 instruction that fails?  There should be an example in the agent's 
 log file.  By default, the log file is in the same folder as the 
 agent executable.  I will use this to update the automated tests.
 
 If you want to go ahead and made the fixes from a git clone, send a 
 Pull Request

RE: [ANNOUNCE] New Committer: Darren Shepherd

[Rajesh Battala]

Hearty Congratulations Darren.! :) 

-Original Message-
From: Chip Childers [mailto:chip.child...@sungard.com] 
Sent: Saturday, September 28, 2013 5:54 PM
To: dev@cloudstack.apache.org
Subject: [ANNOUNCE] New Committer: Darren Shepherd

The Project Management Committee (PMC) for Apache CloudStack has asked Darren 
Shepherd to become a committer and we are pleased to announce that he has 
accepted.

Being a committer allows many contributors to contribute more autonomously. For 
developers, it makes it easier to submit changes and eliminates the need to 
have contributions reviewed via the patch submission process. Whether 
contributions are development-related or otherwise, it is a recognition of a 
contributor's participation in the project and commitment to the project and 
the Apache Way.

Please join me in congratulating Darren!

-chip
on behalf of the CloudStack PMC

[ANNOUNCE] New Committer: Darren Shepherd

[Chip Childers]

The Project Management Committee (PMC) for Apache CloudStack
has asked Darren Shepherd to become a committer and we are pleased to
announce that he has accepted.

Being a committer allows many contributors to contribute more
autonomously. For developers, it makes it easier to submit changes and
eliminates the need to have contributions reviewed via the patch
submission process. Whether contributions are development-related or
otherwise, it is a recognition of a contributor's participation in the
project and commitment to the project and the Apache Way.

Please join me in congratulating Darren!

-chip
on behalf of the CloudStack PMC

Re: Review Request 14360: Make getName() match the constant names for Network Capability, and Service

[daan Hoogland]

> On Sept. 27, 2013, 8:21 p.m., Alena Prokharchyk wrote:
> > Darren, changes in Network.java are going to break API compatibility as 
> > capabilities parameters are used in createNetworkOffering API call 
> > (serviceCapabilitystList param). CS UI and third party UI will get impacted 
> > as well.
> > If we have to change the name, its gotta be the capability parameter name, 
> > not its name value.

This in turn, could have impact on other uses, like in a db could it?

@Darren: you will need a lot more then just a (unit)test for the getname()


- daan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14360/#review26447
---


On Sept. 27, 2013, 5:43 p.m., Darren Shepherd wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14360/
> ---
> 
> (Updated Sept. 27, 2013, 5:43 p.m.)
> 
> 
> Review request for cloudstack, Alena Prokharchyk and Chiradeep Vittal.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> VPCVirtualRouter is still left as inconsitent as changing that name will be 
> impactful.  The unit test checked in will still ensure that future providers 
> match though
> 
> 
> Diffs
> -
> 
>   api/src/com/cloud/network/Network.java 49f380b 
>   api/test/com/cloud/network/NetworkTest.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/14360/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Darren Shepherd
> 
>

Re: Upgrade failed because system VM was created using old template?

[Daan Hoogland]

afaik not, you can define the new one with the documented name and you're
done.


On Fri, Sep 27, 2013 at 5:49 AM, Indra Pramana  wrote:

> Dear all,
>
> Would like to confirm on this:
>
> On Wed, Sep 25, 2013 at 7:40 PM, Indra Pramana  wrote:
>
> > 2. Do I need to delete the old system VM template from the template list,
> > prior to the upgrade? If yes, I think this should be inside the
> > documentation.
> >
>
> Prior to the upgrade from 4.1.1 to 4.2.0, we need to register the new
> system VM template for 4.2.0. My question, do we also need to delete the
> old system VM template?
>
> Looking forward to your reply, thank you.
>
> Cheers.
>

Re: [PROPOSAL] Site-to-site VPN between two virtual routers

[Daan Hoogland]

I think this is badly needed Sheng, so +1. I hope you are not running into
conflict with nicira base private vpc gateways :)

please keep me in the loop,
Daan


On Thu, Sep 26, 2013 at 11:42 PM, Sheng Yang  wrote:

> Hi,
>
> I'd like to propose "site-to-site VPN between VRs" as a new feature for
> CloudStack 4.3.
>
> Basically the VPC's site-to-site VPN currently only support to access
> remote peer which is a hardware VPN gateway. This feature would extend the
> support to other CloudStack VRs, which enabled CloudStack networks to
> communicate with each other through VPN.
>
> Original requirement at:
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Site-to-Site+VPN+2.0
> https://issues.apache.org/jira/browse/CLOUDSTACK-730
>
> FS available at:
>
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Site-to-site+VPN+VR-to-VR+functional+spec
>
> --Sheng
>

Re: Review Request 14320: add boolean option httpModeEnabled to the service offering for use in haproxy conf

[Daan Hoogland]

Chiradeep,

the network offerings are created by the cloud operator aren't they? The
netscaler  en f5 modules will have to implement it's own behavior on
httpClose. in case of haproxy it means no mode http and option httpclose
(and some other things)

If you define it zone wide every tenant has the same setting whilst you
want this to tune setting (like with maxConnections) for a tenant.

regards,
Daan


On Thu, Sep 26, 2013 at 10:57 PM, Chiradeep Vittal wrote:

>This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14320/
>
> Not sure if this should be in the API since it is a HAProxy-specific 
> configuration. This wouldn't apply to Netscaler or F5.
> After all the end user has no idea if he is using HAProxy of Netscaler or F5.
>
> Likely this flag is of interest to the cloud operator only, so why not put it 
> in zone-wide config instead of the network offering.
> Do you really see someone creating 2 offerings: one with HttpClose and one 
> without HttpClose?
>
>
> - Chiradeep Vittal
>
> On September 26th, 2013, 7:01 p.m. UTC, daan Hoogland wrote:
>   Review request for cloudstack and Wei Zhou.
> By daan Hoogland.
>
> *Updated Sept. 26, 2013, 7:01 p.m.*
>  *Bugs: * CLOUDSTACK-4328
>  *Repository: * cloudstack-git
> Description
>
> add boolean option httpModeEnabled to the service offering for use in haproxy 
> conf
>
>   Testing
>
> created unit test.
> instantiated a network with some loadbalancer rule based on a netoffer with 
> the option to true/false and maxconnections to a non default value -> checked 
> haproxy.cfg on the router
>
>   Diffs
>
>- api/src/com/cloud/offering/NetworkOffering.java (6c5573e)
>- api/src/org/apache/cloudstack/api/ApiConstants.java (f85784b)
>- 
> api/src/org/apache/cloudstack/api/command/admin/network/CreateNetworkOfferingCmd.java
>(bdad904)
>- 
> api/src/org/apache/cloudstack/api/command/admin/network/UpdateNetworkOfferingCmd.java
>(c9c4c8a)
>- core/src/com/cloud/agent/api/routing/LoadBalancerConfigCommand.java
>(ee29290)
>- core/src/com/cloud/network/HAProxyConfigurator.java (2309125)
>- core/test/com/cloud/network/HAProxyConfiguratorTest.java
>(PRE-CREATION)
>- 
> engine/components-api/src/com/cloud/configuration/ConfigurationManager.java
>(5e1b9b5)
>- 
> engine/orchestration/src/org/apache/cloudstack/engine/orchestration/NetworkOrchestrator.java
>(53f64fd)
>- engine/schema/src/com/cloud/offerings/NetworkOfferingVO.java
>(eefdc94)
>- 
> plugins/network-elements/elastic-loadbalancer/src/com/cloud/network/lb/ElasticLoadBalancerManagerImpl.java
>(ecd6006)
>- 
> plugins/network-elements/internal-loadbalancer/src/org/apache/cloudstack/network/lb/InternalLoadBalancerVMManagerImpl.java
>(587ae99)
>- server/src/com/cloud/configuration/ConfigurationManagerImpl.java
>(8a0f7a6)
>- 
> server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java
>(7c026a4)
>- server/test/com/cloud/vpc/MockConfigurationManagerImpl.java (c9a0480)
>- 
> server/test/org/apache/cloudstack/networkoffering/CreateNetworkOfferingTest.java
>(1f1fb75)
>- setup/db/db/schema-420to430.sql (44a884d)
>
> View Diff 
>

Re: [PROPOSAL] Service monitoring tool in virtual router

[Daan Hoogland]

Even when always restarting on every glitch we need to monitor the inside
of the vr to know when to restart/respin a new vr. There is much
functionality present on the vr an for us it is not possible to say for
sure what is important to a customer installation so the admin should be
able to define the minimal reqs that will stop us from spinning up a new
vr. And there must be tools present for monitoring these reqs.

makes sense?


On Thu, Sep 26, 2013 at 10:01 PM, David Nalley  wrote:

> For what it's worth we created an ACS-specific MIB (beneath the
> org.apache MIB) so really this is just a matter of defining and
> publishing it.
>
> But lets think about monit being used to restart services - with HA,
> Redundant VR, are we sure that we want to inject yet another point of
> control into things? Is it better to just respawn an instance since
> they are essentially stateless? I don't know, but management server,
> local daemons, and other SysVMs making decisions seems like we are
> increasing complexity.
>
> --David
>
> On Thu, Sep 26, 2013 at 10:31 AM, Chiradeep Vittal
>  wrote:
> > In this case you would have to invent another enterprise MIB. Not too
> > hard, but I'd argue that it needs to be proxied through some other
> service
> > anyway and it represents a different integration point with ACS. Depends
> > on whether you consider the system vm part of the ACS deployment, or an
> > entity like a host.
> >
> > On 9/26/13 10:27 AM, "Alex Huang"  wrote:
> >
> >>Using SNMP for alert notification is not a bad idea though.  I don't see
> >>why we can't do that instead of posting to the management server.  This
> >>is specifically referring to the second part of the proposal.  Why
> >>reinvent that part of it?
> >>
> >>--Alex
> >>
> >>> -Original Message-
> >>> From: Chiradeep Vittal [mailto:chiradeep.vit...@citrix.com]
> >>> Sent: Wednesday, September 25, 2013 10:28 PM
> >>> To: dev@cloudstack.apache.org
> >>> Subject: Re: [PROPOSAL] Service monitoring tool in virtual router
> >>>
> >>> SNMP wouldn't restart a failed process nor would it generate alerts. It
> >>>is
> >>> simply too generic for the requirements outlined here. The proposal
> does
> >>> not talk about modifying monit, just using it. That wouldn't trigger
> >>>the AGPL.
> >>> I think the idea is to have a tight monitoring loop that scales: so
> >>>executing the
> >>> monitoring loop in-situ makes sense.
> >>>
> >>>
> >>> On 9/25/13 9:53 PM, "David Nalley"  wrote:
> >>>
> >>> >On Wed, Sep 25, 2013 at 9:30 AM, Jayapal Reddy Uradi
> >>> > wrote:
> >>> >> Hi,
> >>> >>
> >>> >> Currently in virtual router there is no way to recover and notify if
> >>> >>some service goes down unexpectedly.
> >>> >>
> >>> >> This feature is about monitoring all the services rendered by the
> >>> >>virtual router, ensure that the services are running through the life
> >>> >>time of the VR.
> >>> >>
> >>> >> On service failure:
> >>> >> 1. Generate an alert and event indicating failure 2. Restart the
> >>> >> service
> >>> >>
> >>> >> Services to be monitored:
> >>> >> DHCP, DNS, haproxy, password server etc.
> >>> >>
> >>> >> As part of monitoring there are two activities
> >>> >>
> >>> >> 1. One is monitoring the services in VR and log the events. Using
> >>> >>monit for monitoring services  2. Second part is pushing alerts from
> >>> >>router to  MS server. Thinking on POST the logs to web server in MS.
> >>> >>
> >>> >> I will be updating more details and FS in this thread.
> >>> >>
> >>> >> I created enhancement bug for this.
> >>> >> https://issues.apache.org/jira/browse/CLOUDSTACK-4736
> >>> >>
> >>> >> Thanks,
> >>> >> Jayapal
> >>> >
> >>> >So several things - why not make this via SNMP? Query processes, and
> >>> >many other things. This should be relatively simple, is well known,
> can
> >>> >be locked down (or could be monitored for many other things by
> external
> >>> >monitoring packages) and is the defacto standard for monitoring hosts.
> >>> >Second - monit is Affero GPL licensed - which is a cat-x license.
> >>> >While I expect that we would merely use this and not do any hacking on
> >>> >it - I think its inclusion might be a surprise (and forbidden in many
> >>> >environments) to our users
> >>> >
> >>> >--David
> >>
> >
>

Re: [PROPOSAL] move away from time-based releases and/or revamp release process

[Daan Hoogland]

Ilya,

I think you are in line with my 'on the side' notice. It is a promise I
like with one downside. You risk a successful 3rd party plugin writer still
and only supporting 5.6.5 whilst we are already at 7.3.x, hence the last
few lines of that remark.

So does not allowing a plugin 4.3 on a 4.2 instance also mean not allowing
a 4.2 on 4.3?
Can a developer build for 10.3 and up?
And how are we ensuring that at version 13 we disallow this plugin before
it is revisited by the author because we know of some quirk in our plugin
mechs that we created or fixed?
Rhetoric question, that last one. The answer being of course some extra
numbering scheme or table under the hood. It must be devised in my opinion
as allowing the plugin to load will hurt the user and the rep of ACS.
So answering the first two questions defines how much work this will be.

Daan


On Thu, Sep 26, 2013 at 8:15 PM, Musayev, Ilya  wrote:

> Dan and David,
>
> I see your concerns. Let me explain what I mean by modular and how we can
> address dependency.
>
> Assume you have base ACS 4.2
> Then you would have plugins like Nicira 4.2.x, VMware 4.2.x, NetScaler
> 4.2.x etc.. Goes without saying, but you cannot use a plugin from 4.3.x
> family on ACS 4.2.x base...
> We can aslo introduce dependcy checks, such that if Nicira plugin is
> enabled, we need VMware plugin enabled as well.
>
> Why would we consider this? You can update modules more rapidly that
> include bug fixes (and perhaps minor non impacting enhacements) without
> waiting for the entire release cycle, you testing should be less as well,
> as you would only be testing affected modules, unless they strongly tie
> with another plugin.
>
> If properly architected, CloudStack wont suffer OpenStack's faith - which
> is what they go through now - of only specific version x.y.z module can
> work with another x.y.z module.
>
> Regards
> ilya
>
> > -Original Message-
> > From: Daan Hoogland [mailto:daan.hoogl...@gmail.com]
> > Sent: Thursday, September 26, 2013 4:57 AM
> > To: dev
> > Subject: Re: [PROPOSAL] move away from time-based releases and/or
> > revamp release process
> >
> > Some more remarks on this:
> >
> > Another model is of keeping quality high is controlling what gets in per
> > release. This would go against the opensource model somewhat. You would
> > need e release manager like me and I strongly advice against that.
> >
> > Giving users more control on what goes into their system is a good
> thing. it
> > must not become a 'university degree required' hurdle though. It would
> > allow them to have a narrower view on what is good software,
> functionality -
> > and quality wise. That would lead to better quality of quality control.
> I know I
> > am no good at that aspect right now because of the load of code I need to
> > know about.
> >
> > 
> > hear hear, David. You are pointing at some of the challenges OSGi deals
> with.
> > Ours is a little simpler; As we don't need run-time control, our plugins
> can
> > have a strict hierarchical requirements tree which is somewhat easier to
> > handle. Still you might end up with a system with two hypervisor
> (versions)
> > that require a different storage or network version to work. or vice
> versa. I
> > think we should be happy at first if we get a static system where 3rd
> party
> > plugins work with version x till y. Note that then the plugin interface
> is to be
> > stable through a (semantically versioned) major release.
> >
> > challenge i like, :)
> > 
> >
> > On Thu, Sep 26, 2013 at 6:21 AM, David Nalley  wrote:
> > > On Wed, Sep 25, 2013 at 11:26 AM, Musayev, Ilya
> >  wrote:
> > >> We can still use scheduled release as we do now and yet have some
> > agility.
> > >>
> > >> An idea was passed around before if we can modularize ACS in future
> > releases VS being it monolithic.
> > >>
> > >> We can still have a core as is, but additional components/plugins can
> be
> > loaded adhoc pm the need base. As an example, there is no need to bake in
> > vmware support into release if a customer has not need for it. You cant
> just
> > upgrade vmware code unless you upgrade from version X.y.1 to X.y.2.
> > >>
> > >> Same applies to all other "plugins", that are not truly pluggable.
> > >>
> > >> Splitting components as separate "plugins" (or whatever the proper
> term
> > is) would ease the release cycle and give us flexibility in my opinion.
> > >>
> > >
> > > Can you imagine the complexity of that model? Core version 4.3.x has
> > > to work with VMware plugin 7.2.3, 7.1.5, and 7.0.2 and also has to
> > > work with Nicira plugin version 3.4.1, 3.5.2, and 3.6.0 - except that
> > > both of those plugins interact with each other as well as the core
> > > orchestration platform. We struggle to test well now, the additional
> > > combinations there boggle the mind.
> > >
> > > --David
>
>
>

capacity of secondaey stor us eq 0. bug ir known ussue?

[Netmaster]

Hallo,
we have several installation of cloudstack 4.2
In dashboard the capacity of secondary storage equal 0kb
We have standart nfs.
Api backed the same error, capacity of resource type 6 is 0
The secondary stor vm (system vm) shows normall capacity of 10tb, 
as it have to be.
Is it bug or known issue?
Thanks