Re: Scaling up cpu and memory of user vm above host capacity

[Gaurav Aradhye]

Hi Nitin,

I was trying on running vm only, but I was
using changeServiceForVirtualMachine API instead of scaleVirtualMachine API.
But I wonder why changeServiceForVirtualMachine API succeeded in allocating
more than host capacity.

What is the basic difference between these two operations?

Regards,
Gaurav


On Tue, Oct 1, 2013 at 10:45 PM, Nitin Mehta  wrote:

> Gaurav - Were you trying this on a stopped vm ? If you try and start it
> with an offering
> above the host capacity (including over provisioning ) then it shouldn't
> start.  Let me know how it goes.
>
> More details on scale vm feature @
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+scaling+of+C
> PU+and+RAM
>
> On 01/10/13 12:02 AM, "Gaurav Aradhye"  wrote:
>
> >Thanks David. That disabuses my confusion about the CPU provisioning. I
> >was
> >using the wrong API to scale up the virtual machine, so above observations
> >stand invalid till I get the same results with the right API.
> >
> >About over-provisioning, I have the over provisioning factor set as 1 both
> >in case of CPU and memory.
> >
> >Regards,
> >Gaurav
> >
> >
> >On Mon, Sep 30, 2013 at 10:55 PM, David Ortiz 
> wrote:
> >
> >> A machine won't be able to support more cores on a VM than the physical
> >> processor.  That should result in problems trying to deploy it.  I'm
> >> guessing the service offering is still valid since you could add a host
> >> later which has a hex core or two cpus in it.  As far as RAM goes, do
> >>you
> >> have overprovisioning enabled?
> >>
> >> > From: gaurav.arad...@clogeny.com
> >> > Date: Mon, 30 Sep 2013 14:00:04 +0530
> >> > Subject: Scaling up cpu and memory of user vm above host capacity
> >> > To: dev@cloudstack.apache.org
> >> >
> >> > Hi,
> >> >
> >> > I am trying to automate a scenario here. I have only one host in
> >>cluster
> >> > with 4 CPU cores and 15 GB total memory. When I try to scale up cpu
> >>and
> >> RAM
> >> > of a running user vm above the host capacity, it doesn't throw any
> >>error
> >> > and I can see the updated values in VM statistics too.
> >> >
> >> > For CPU, I am able to change the service offering of user vm as  5
> >>cores
> >> *
> >> > 100 MHz (even though host has 4 cores). I am not sure how this
> >> calculation
> >> > is done. Definitely many no. of virtual cores can be formed on host
> >>(more
> >> > than 4), but is it possible to allocate 5 cores to single VM ? When I
> >>try
> >> > to deploy new VM with 5 core CPU service offering, then in this case
> >>it
> >> > fails saying not enough server capacity.
> >> >
> >> > Also, For memory, I am able to create 17 GB memory service offering
> >>and
> >> > allocate it to any running user vm (although the total memory on host
> >>is
> >> 15
> >> > GB).
> >> >
> >> > Any directions? Is this an issue or am I missing something here?
> >> >
> >> > Regards,
> >> > Gaurav
> >>
> >>
>
>

Re: Review Request 14451: Some test for NetUtils

[Saksham Srivastava]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14451/#review26660
---



utils/src/com/cloud/utils/net/NetUtils.java


I have created https://issues.apache.org/jira/browse/CLOUDSTACK-4807. You 
could assign the bug to yourself and resolve it when your patch is merged.
Just add the bug id in the bug section in the review board, also add the 
branch(master) in the review board, it helps people applying the patch.



utils/src/com/cloud/utils/net/NetUtils.java


Thanks for the explanation.
Can you keep this variable with all other static variables at the beginning 
of the class and not at this place.



utils/test/com/cloud/utils/net/NetUtilsTest.java


What I am suggesting is that add a method something like
boolean isEqual(var1, var2) {
if var1 == var2 
return true
return false
}

Now you can add cases like
assertFalse(isEqual(15, NetUtils.mac2Long("00:00:00:00:00:ff")))

This is just one implementation as there is no assertNotEquals() method 
provided by Junit. You could certainly add better implementations. 
We generally use assertFalse for boolean methods.
You can have a look at testIsSameIpRange():
 // Check for 2 different CIDRs and different IP Range
assertFalse(NetUtils.isSameIpRange(cidrFirst, cidrThird));

// Check for Incorrect format of CIDR
assertFalse(NetUtils.isSameIpRange(cidrFirst, "10.3.6.5/50"));

Just a few such cases will suffice.



- Saksham Srivastava


On Oct. 2, 2013, 9:19 p.m., Laszlo Hornyak wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14451/
> ---
> 
> (Updated Oct. 2, 2013, 9:19 p.m.)
> 
> 
> Review request for cloudstack.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> - tests added for mac2Long, byte2Mac, long2Mac, ip2Long and long2Ip
> - isWindows static method removed - it was not used
> - minor changes in the implementations of the tested methods
> 
> 
> Diffs
> -
> 
>   utils/src/com/cloud/utils/net/NetUtils.java 1e72e22 
>   utils/test/com/cloud/utils/net/NetUtilsTest.java 3cfc98f 
> 
> Diff: https://reviews.apache.org/r/14451/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> Laszlo Hornyak
> 
>

Re: Review Request 14467: fix for Invalid SMTP breaks HA

[Anshul Gangwar]

On Thursday 03 October 2013 08:44 PM, Chip Childers wrote:
> On Thu, Oct 03, 2013 at 11:31:50AM +, Anshul Gangwar wrote:
>> Description
>> ---
>>
>> added the connection and socket timeout parameters for SMTP and sending 
>> message in new thread so that HA doesn't get blocked beacause of hang in 
>> sending email alert
> Wouldn't a better approach be to ensure that email sending runs async to
> critical operations like HA?

This patch is making email sending  async. Currently agent thread first 
send the alert( where emails are sent) and then calls the HA. Can you 
please elaborate more? Is it logic change or something else?

Thanks,
Anshul

Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]

[Murali Reddy]

On 04/10/13 12:32 AM, "Chip Childers"  wrote:

>> On Oct 3, 2013, at 3:01 PM, Chiradeep Vittal
>> wrote:
>>
>> I was thinking along the same lines that a offering could have a generic
>> key/value map (where the value could be a string/json string/whatever).
>> CloudStack core wouldn't have any idea about the semantics of the keys
>>or
>> values, but the specific hypervisor/provider might decide to interpret
>>it.
>
>+1 - a model for extensions like that makes perfect sense.

We already have api's (add/list/removeResourceDetails) in 4.2 [1] to add
key-value metadata to first class objects of CloudStack. At present it
does not have semantic meaning to neither core nor plug-ins, but should
not be hard to extend so that plug-ins can access meta-data and interpret
it.

[1] http://s.apache.org/ReK

>
>>
>>
>>> On 10/3/13 11:15 AM, "Darren Shepherd" 
>>>wrote:
>>>
>>> We are always going to run into the issue where they're implementation
>>> specific features that can't be represented in a generic way in the
>>> API.  First class attributes in the API need to essentially be the
>>> lowest common denominator.  ACS already has this pattern of *_details
>>> tables.  I honestly don't know completely how they are used, but I
>>> would hope that pattern could be used to address these types of
>>> issues.  By allowing the admin to add arbitrary key/value pairs to the
>>> various offerings/entities we can use that to tap into provider
>>> specific or advanced configuration.  For the http close proposal you
>>> could just add haproxy.httpClose=true on the network offering and the
>>> haproxy impl can respect that and use it.  So implementations can
>>> document what additional key/value parameters they support to access
>>> advanced features.
>>>
>>> Review 13896 adds an otherOptions field to the service offering.
>>> Conceptually that is fine in my mind, but I just don't understand why
>>> the *_details tables can't be used instead.  Just like XenServer has
>>> other_config on all types, I'd like it if ACS had a similar thing
>>> (which I kinda, sorta think it does?).  So we should be able to
>>> arbitrary data onto any type.  Can somebody comment on how this idea
>>> may relate with the existing implementation of "details" and "tags" in
>>> ACS?
>>>
>>> Darren
>>>
>>> On Thu, Oct 3, 2013 at 10:21 AM, Daan Hoogland
>>>
>>> wrote:
 Murali,

 What our admins need is the ability to instantiate an abstract thing
 called
 a virtual redundant high available router. They need be able to tune
it
 with all sorts of features is such a way that other routers redundant
or
 not and high availible or not, may but do not have to have the same
 tuning
 parameters. This 'feature' of httpClose is just one of the many things
 they
 need to be able to tune. Others include a more fine grained control
over
 the iptables/firewall rules and monitoring of the
functionality/daemons
 running on the machines. I am not biased as to the way how to
 do/implement
 this control. The networkoffering seemed like the way to do it.

 Having said that I didn't think that httpClose would be considered
 haproxy
 specific. It seems worrying that someone could device a proxy or a
 loadbalancer that does not support either one of the features of
 maximizing - or pooling connections. I'm not into that market recently
 so I
 might be mistaking. This maybe besides the point but it discomforts me
 somewhat.

 regards,
 Daan


 On Thu, Oct 3, 2013 at 2:22 PM, murali reddy
 wrote:

> On Thu, Oct 3, 2013 at 4:18 PM, Daan Hoogland
>> wrote:
>
>> Chiradeep,
>>
>> I have been thinking about your concern and there is something
> bothering
> me
>> about it. The issue CLOUDSTACK-4328 of which
>> https://reviews.apache.org/r/14320/ is an implementation. This issue
> has
>> been brought up by the engineers at Schuberg Philis. These are cloud
>> operators and domain admins. So these are the people that need to be
>> bothered by this tuning and they are certainly haproxy and xen
> experts to
>> an extend. For them not being able to use connection caching was a
> bug.
> And
>> the proposed solution still seems reasonable to me. It is exposing
> the
>> abstract feature only to those instantiating a vpc, isn't it? This
> last
>> question probably touches on the reason you are addressing my
> submission
>> together with the ones from Nicolas  I see only people instantiating
> a
> vpc
>> having to be bothered by which netoffering to use (with respect to
>> httpClose functionality) If this is not the case how should I
>>isolate
> this
>> concern from other , more mondain users?
>>
>> btw I did not create an FS page as the issue was created as a jira
> ticket.
>> I am willing to do that in hind sigh

4.2.1 rpm build failing - CLOUDSTACK-4808

[Rayees Namathponnan]

Hi Hugo,

4.2.1 rpm build failing; submitted patch for review; can you please review and 
push to 4.2.1 branch ?

https://reviews.apache.org/r/14482/
https://issues.apache.org/jira/browse/CLOUDSTACK-4808

Regards,
Rayees

Re: Wiki access

[Chiradeep Vittal]

Mike, Tuna, Srinivas, Go, done.

On 10/3/13 9:12 PM, "Mike Tutkowski"  wrote:

>I seem to need edit access, as well.
>
>I'm usually logged in by default, but I believe my username is
>mike-tutkowski (else please try mtutkowski).
>
>Thanks!
>
>
>On Thu, Oct 3, 2013 at 7:07 PM, Go Chiba  wrote:
>
>> Hi Chip,
>>
>> Please grant me to edit pages. My id is "gochiba".
>>
>>
>> On Thu, Oct 3, 2013 at 6:01 AM, Chip Childers > >wrote:
>>
>> > Not all committers were easy to find, but those that I did should be
>>all
>> > set.
>> >
>> > So committers / contributors = provide your confluence ID's if you
>>can't
>> > add / edit pages.  PMC folks - help add them please.
>> >
>> >
>> > On Wed, Oct 2, 2013 at 4:48 PM, Chip Childers
>>> > >wrote:
>> >
>> > > There were major issues with Spam on the wiki, so infra changed the
>> > > permissions.
>> > >
>> > > I've just gone and added all PMC members to the space as space
>>admins.
>> > >  That means all PMC members can help deal with this issue (or at
>>least
>> > > those that I could find in the wiki user list).
>> > >
>> > > I'm going to deal with committers next.  That permission level
>>needs to
>> > > exclude space admin.  I'm using alena1108 as the example for
>>committer
>> /
>> > > contributor permissions.
>> > >
>> > > So if you are not a committer, give your confluence ID in this
>>thread
>> and
>> > > PMC folks can help get them added (make them look like alena's
>>perms).
>> > >
>> > >
>> > >
>> > >
>> > > On Wed, Oct 2, 2013 at 4:35 PM, Chiradeep Vittal <
>> > > chiradeep.vit...@citrix.com> wrote:
>> > >
>> > >> Apparently, edit access has been revoked for all. Who do we contact
>> for
>> > >> edit permissions?
>> > >>
>> > >> TIA
>> > >>
>> > >
>> > >
>> >
>>
>>
>>
>> --
>> 千葉 豪  Go Chiba
>> E-mail:go.ch...@gmail.com
>>
>
>
>
>-- 
>*Mike Tutkowski*
>*Senior CloudStack Developer, SolidFire Inc.*
>e: mike.tutkow...@solidfire.com
>o: 303.746.7302
>Advancing the way the world uses the
>cloud
>*?*

Review Request 14482: CLOUDSTACK-4808 - Fix for build failure in 4.2.1. branch

[Rayees Namathponnan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14482/
---

Review request for cloudstack, Frank Zhang and Hugo Trippaers.


Bugs: CLOUDSTACK-4808


Repository: cloudstack-git


Description
---

4.2.1 RPM build failing, we need to remove below two check-ins from 4.2.1 
branch; these changes are made during 4.2 release build 

https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commit;h=8689f7f504acee7ccbf6870d409cbb47b177d598
 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commit;h=db9f5243c31cf00dbf3ed4a81585671c27bab469
 


Diffs
-

  packaging/centos63/package.sh f30a0e7 

Diff: https://reviews.apache.org/r/14482/diff/


Testing
---

tested with 4.2.1 RPM build


Thanks,

Rayees Namathponnan

Re: [MERGE] spring-modularization to master - Spring Modularization

[Prasanna Santhanam]

Thanks. I will first run a baseline against master since we don't have
one and then one on spring modularization. The baseline against 4.2
shows this result: 

http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-regression-matrix/252/testReport/

On Thu, Oct 03, 2013 at 09:19:23AM -0700, Darren Shepherd wrote:
> This should be fixed on the spring-modularization branch.
> 
> Darren
> 
> On Wed, Oct 2, 2013 at 10:05 PM, Prasanna Santhanam  wrote:
> > I switched the test infrastructure on jenkins.buildacloud.org to run
> > the bvts [1] against master last week. Couple of weeks before that the
> > simulator [2] tests were switched to run against master. Both are
> > broken unfortunately and the bvt and checkin tests aren't running.
> >
> > I've filed a bug here:
> > https://issues.apache.org/jira//browse/CLOUDSTACK-4791
> >
> > [1] http://jenkins.buildacloud.org/view/cloudstack-qa/
> > [2] http://jenkins.buildacloud.org/view/simulator/
> >
> > Anyone who has access to the jenkins can run the bvts on their desired
> > branch. Simply login and change the test-yumrepo-refresh job to point
> > to your branch. Build that to refresh the remote repository with the
> > packages made from your branch. Then switch test-matrix to point to
> > the same development branch and fire a build. That's about it.
> >
> > On Wed, Oct 02, 2013 at 05:42:54PM -0700, Darren Shepherd wrote:
> >> Yes agreed.  I've extensively tested this, but that is never enough.
> >> How do I get the BVTs ran against this.  Due to the cross cutting
> >> nature of this I want to get this merged as fast as possible.
> >>
> >> Darren
> >>
> >> > On Oct 2, 2013, at 4:43 PM, Alex Huang  wrote:
> >> >
> >> > +1 on running the BVT on it.  We've been through this one once before.  
> >> > Should be careful.
> >> >
> >> > --Alex
> >> >
> >> >> -Original Message-
> >> >> From: Kelven Yang [mailto:kelven.y...@citrix.com]
> >> >> Sent: Wednesday, October 2, 2013 4:39 PM
> >> >> To: dev@cloudstack.apache.org
> >> >> Subject: Re: [MERGE] spring-modularization to master - Spring
> >> >> Modularization
> >> >>
> >> >> Darren,
> >> >>
> >> >> This looks really nice. A few questions on Spring AOP replacement.
> >> >>
> >> >> 1) Spring AOP is proxy-based, the reason we ended up of using customized
> >> >> AOP is mainly due to that inside existing CloudStack codebase, we have 
> >> >> many
> >> >> places that are doing run-time type-casting, the code in these places
> >> >> assumes a real object that implements all interfaces in the semantics 
> >> >> context.
> >> >> At the time when I initially converted to Spring, I couldn't ensure that
> >> >> switching to proxy-based AOP can have 100% coverage for these run-time
> >> >> cases. What is your approach to address this run-time type-casting 
> >> >> issue?
> >> >>
> >> >> 2) We've run into a huge-memory footprint issue that may be caused by
> >> >> conflicts of CGLIB usage in Spring AOP and the CGLIB usage in 
> >> >> CloudStack Dao
> >> >> layer. Do you have a chance to run a memory analysis in the heap after
> >> >> management server is started.
> >> >>
> >> >> I might be good to run BVT test on the branch before the merge, could
> >> >> someone initiate the effort?
> >> >>
> >> >> kelven
> >> >>
> >> >>
> >> >>
> >> >> On 10/2/13 3:48 PM, "Darren Shepherd" 
> >> >> wrote:
> >> >>
> >> >>> Not sure how this works...  I would like to merge in the new
> >> >>> modularized Spring setup to master. There is info on the wiki about it
> >> >>> [1] [2] [3].  The primary change is to break apart the monolithic
> >> >>> applicationContext.xml and componentContext.xml files such that each
> >> >>> plugin can maintain and contribute its own configuration.
> >> >>>
> >> >>> In addition to breaking up the configuration we no longer use ACS
> >> >>> custom AOP and it is now fully Spring AOP.
> >> >>>
> >> >>> Now adding/removing a plug-in is a matter of just adding a jar to the
> >> >>> classpath (exception being commands.properties, I'll address that in a
> >> >>> different thread).  Unfortunately this branch does not have the changes
> >> >>> to package things in different RPMs.  So it would be great if somebody
> >> >>> could take up the packaging effort to split out all the plugins into
> >> >>> different RPMs.
> >> >>>
> >> >>> Darren
> >> >>>
> >> >>> [1]
> >> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Sp
> >> >> rin
> >> >>> g
> >> >>> [2]
> >> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-
> >> >> ins%2C+Modu
> >> >>> les
> >> >>> %2C+and+Extensions
> >> >>> [3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
> >> >
> >
> > --
> > Prasanna.,
> >
> > 
> > Powered by BigRock.com
> >

-- 
Prasanna.,


Powered by BigRock.com

Re: [MERGE] spring-modularization to master - Spring Modularization

[Prasanna Santhanam]

On Thu, Oct 03, 2013 at 02:46:40PM +, Rayees Namathponnan wrote:
> Hi Prasanna,
> 
> If we are running BVT or regression on specific branch, marvin
> should also from same branch right ? but we always getting marvin
> from master branch  job "cloudstack-marvin"
> 

Yes, but marvin's core is the same in master and 4.2 at the moment.
That will change if and when the merge of the refactoring happens. Can
you help fix the jenkins job to fetch marvin from the repo the test is
targetted towards?

-- 
Prasanna.,


Powered by BigRock.com

Re: Review Request 14471: CloudStack Plugin guide for VXLAN

[Prasanna Santhanam]

Hi,

The docs were broken out into a separate repo 2 days ago. So this
should be based on the repo cloudstack-docs.git. Can you resubmit this
for that repo?

See:
http://markmail.org/message/w44nioqt5usechhc

Thanks,


On Thu, Oct 03, 2013 at 08:20:52PM -, Yoshikazu Nojima wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14471/
> ---
> 
> Review request for cloudstack and Toshiaki Hatano.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> This patch adds a Plugin guide for VXLAN.
> It describes system requirements for VXLAN plugin and how to setup it.
> 
> This is the ToC:
> 1. The VXLAN Plugin  1
> 1.1. Introduction to the VXLAN Plugin  1
> 1.2. Features of the VXLAN Plugin  1
> 2. System Requirements for VXLAN 3
> 2.1. System Requirements for VXLAN  3
> 2.2. Linux Distributions that meet the requirements  3
> 2.3. Check the capability of your system  3
> 2.4. Advanced: Build kernel and iproute2  3
> 2.4.1. Build kernel  4
> 2.4.2. Build iproute2  4
> 3. Configure CloudStack to use VXLAN Plugin 7
> 3.1. Configure hypervisor  7
> 3.1.1. Configure hypervisor: KVM  7
> 3.2. Setup zone using VXLAN  9
> 3.2.1. Configure the physical network  10
> 
> 
> Diffs
> -
> 
>   docs/en-US/CloudStack_VXLAN_Guide.ent PRE-CREATION 
>   docs/en-US/CloudStack_VXLAN_Guide.xml PRE-CREATION 
>   docs/en-US/images/vxlan-physicalnetwork.png PRE-CREATION 
>   docs/en-US/images/vxlan-trafficlabel.png PRE-CREATION 
>   docs/en-US/plugin-vxlan-about.xml PRE-CREATION 
>   docs/en-US/plugin-vxlan-config-hypervisor.xml PRE-CREATION 
>   docs/en-US/plugin-vxlan-config-management.xml PRE-CREATION 
>   docs/en-US/plugin-vxlan-configuration.xml PRE-CREATION 
>   docs/en-US/plugin-vxlan-features.xml PRE-CREATION 
>   docs/en-US/plugin-vxlan-introduction.xml PRE-CREATION 
>   docs/en-US/plugin-vxlan-requirements.xml PRE-CREATION 
>   docs/en-US/plugin-vxlan-revision-history.xml PRE-CREATION 
>   docs/publican-plugin-vxlan.cfg PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/14471/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Yoshikazu Nojima
> 

-- 
Prasanna.,


Powered by BigRock.com

Re: [DOCS] CS-docs patch

[Prasanna Santhanam]

On Thu, Oct 03, 2013 at 02:10:26PM -0400, Travis Graham wrote:
> Here is a patch for the "Deployment Architecture Overview" section
> until things get setup in Review Board for the new repo or there's a
> GitHub mirror that pull requests can be sent to for review. Will the
> github mirror and PR route be supported in tandem with the Review
> Board process or not at all?
> 
> It adds in links to other docs it only loosely referenced by name
> and adds the Primary Storage info about being zone wide for 4.2.
> 
> Travis
Attachments are stripped

Please paste the contents of your patch in plaintext with the subject
[PATCH]. Also ensure you've created your patch using git format-patch
so I can apply it readily and attribute credit to you.

I'd love to have PRs via github but that's upto asf infra and not
project specific to answer.


Thanks,


> 


-- 
Prasanna.,


Powered by BigRock.com

Re: Wiki access

[Mike Tutkowski]

I seem to need edit access, as well.

I'm usually logged in by default, but I believe my username is
mike-tutkowski (else please try mtutkowski).

Thanks!


On Thu, Oct 3, 2013 at 7:07 PM, Go Chiba  wrote:

> Hi Chip,
>
> Please grant me to edit pages. My id is "gochiba".
>
>
> On Thu, Oct 3, 2013 at 6:01 AM, Chip Childers  >wrote:
>
> > Not all committers were easy to find, but those that I did should be all
> > set.
> >
> > So committers / contributors = provide your confluence ID's if you can't
> > add / edit pages.  PMC folks - help add them please.
> >
> >
> > On Wed, Oct 2, 2013 at 4:48 PM, Chip Childers  > >wrote:
> >
> > > There were major issues with Spam on the wiki, so infra changed the
> > > permissions.
> > >
> > > I've just gone and added all PMC members to the space as space admins.
> > >  That means all PMC members can help deal with this issue (or at least
> > > those that I could find in the wiki user list).
> > >
> > > I'm going to deal with committers next.  That permission level needs to
> > > exclude space admin.  I'm using alena1108 as the example for committer
> /
> > > contributor permissions.
> > >
> > > So if you are not a committer, give your confluence ID in this thread
> and
> > > PMC folks can help get them added (make them look like alena's perms).
> > >
> > >
> > >
> > >
> > > On Wed, Oct 2, 2013 at 4:35 PM, Chiradeep Vittal <
> > > chiradeep.vit...@citrix.com> wrote:
> > >
> > >> Apparently, edit access has been revoked for all. Who do we contact
> for
> > >> edit permissions?
> > >>
> > >> TIA
> > >>
> > >
> > >
> >
>
>
>
> --
> 千葉 豪  Go Chiba
> E-mail:go.ch...@gmail.com
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

Re: [PROPOSAL] Modularize Spring

[SuichII, Christopher]

Sure - I could see that working. Anyone have thoughts whether an enum could be 
used instead of an integer? That way we can provide categories or a well 
defined scale (like 0-5)? If we give them free range of 1-100 (or any integer 
range), I imagine people will likely go to the extremes and just use 0 for 
can't handle, 1 for low priority (default) and 100 for high priority (storage 
providers). We still have the problem of handling conflicts, or implementers 
who return the same value. However, I'm not sure there is much we can do beyond 
selecting the first implementation that we come across with the highest 
priority. We should document and suggest that implementations ensure their 
canHandle() method is as intelligent as possible and only takes control of 
operations they are truly authorities on.

-Chris
-- 
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Oct 3, 2013, at 7:44 PM, Darren Shepherd  wrote:

> Could it be just as simple as the canhandle() returns an int and not Boolean. 
>  So the ancient would return 1 but if the netapp matches it returns 100.  If 
> it does not handle at all you return -1.  This effectively gives you 
> prioritization.  So the calling code would still loop through all strategies 
> each time looking for the best match.  I don't want the priority to be 
> determined at load time as that is less flexible. 
> 
> Darren
> 
>> On Oct 3, 2013, at 5:42 PM, "SuichII, Christopher"  
>> wrote:
>> 
>> Thanks for the quick reply.
>> 
>> After talking with Edison, I think what we could do is allow the strategies 
>> to either specify a priority or 'type' and then order them by that when they 
>> are loaded. For example, we could have an enum of types like 'PLUGIN, 
>> HYPERVISOR and DEFAULT' so that we can make sure plugins are asked 
>> canHandle() first, then hypervisor implementations, then finally resort to 
>> the default/ancient implementation. This 'type' or 'category' could be 
>> specified as a bean parameter or as part of the strategy/provider interface. 
>> Any thoughts on which are better?
>> 
>> The problem with just making canHandle() more intelligent is that we do need 
>> some kind of ordering. Ideally, the non-default implementations should be 
>> asked first, then fall back onto the stock implementations. You saw the 
>> problem yourself - the XenServerSnapshotStrategy just says it will handle 
>> all requests, so if a non-standard strategy wants to be given a chance, it 
>> must be asked before the hypervisor or ancient implementation.
>> 
>> Alternatively if this matches the same usage of the global configuration 
>> ordering system, we could add in the storage subsystem strategies and 
>> providers.
>> 
>> The reason for log4j changes is that we may want to enable a different level 
>> of verbosity by default. Our support organization likes to have very verbose 
>> logs to assist them in triaging support issues. The lowest level log4j 
>> categories are 'com.cloud', 'org' and 'net' and those are set to DEBUG and 
>> INFO. Even if we add a line for 'com', the default value should not be 
>> 'TRACE' like we would like ours to be. I'm not all that great with log4j, 
>> though, so maybe I'm missing a simpler solution.
>> 
>> I'll try to keep an eye on the commands.properties/rbac stuff - that is good 
>> to know.
>> 
>> Thanks,
>> Chris
>> -- 
>> Chris Suich
>> chris.su...@netapp.com
>> NetApp Software Engineer
>> Data Center Platforms – Cloud Solutions
>> Citrix, Cisco & Red Hat
>> 
>>> On Oct 3, 2013, at 4:30 PM, Darren Shepherd  
>>> wrote:
>>> 
>>> I forgot to mention about canHandle() ordering.  For extensions, it
>>> should be preferred that we do not rely on loaded or configured order.
>>> The canHandle() implementation should assume that they're may be any
>>> order.  So having said that, I looked at XenServerSnapshotStrategy and
>>> its canHandle() is brilliantly implemented as "return true;"
>>> 
>>> Can we look at making the strategies order independent and not having
>>> another order configuration parameter?
>>> 
>>> Darren
>>> 
>>> On Thu, Oct 3, 2013 at 4:26 PM, Darren Shepherd
>>>  wrote:
 Chris,
 
 I've updated the wiki [1].  Copying from the wiki
 
 "Extensions are automatically discovered based on the interfaces they
 implement and which module is their parent. For example, if you place
 a storage extension in a child module of the network module, it will
 not be discovered. Additionally, depending on the extension, the
 ordering may be important in how they extension is used. For the
 extensions that ordering is important there is also a "Global
 Configuration" setting for the ordering. The value of the setting is a
 comma seperated list like SHA256SALT,MD5,LDAP,PLAINTEXT. The values
 are the names of the extension. The name will be the result of
 getName(), if the bean implements the Named interface, or the sor

Slow network speed to VM instances after upgrade to CloudStack 4.2.0

[Indra Pramana]

Dear all,

After CloudStack upgrade to 4.2.0, the network speed to our VM instances is
very slow and seems to be limited to 2 Mbps for both ingress and egress
traffic for each instance. It seems that there's some network rate limiting
which didn't take effect on CloudStack version 4.1.1 or below, but now is
taking effect on CloudStack 4.2.0.

I read this article on how CloudStack manages the network traffic for the
VMs:

http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.1.1/html/Admin_Guide/network-rate.html

For guest VMs, the network rate-limiting is set on the compute offering
associate with each VM. Please CMIIW.

All our guest VMs are created under different accounts and have their own
compute offering, which is auto-created under that specific account when
the VM instance is created.

The issue is, from admin account, I can't seem to modify the compute
offering for the guest VMs. If I go to GUI > Service Offerings > Compute
Offering, I can only see the compute offering for admin account only. How
can I see and modify the compute offering under different accounts?

Furthermore, it seems that I cannot modify the network rate field of an
existing compute offering. How do I modify it so that I can set higher
network rate limit or unlimited bandwidth for an existing guest VM?

Looking forward to your reply, thank you.

Cheers.

Re: [PROPOSAL] Modularize Spring

[Darren Shepherd]

Could it be just as simple as the canhandle() returns an int and not Boolean.  
So the ancient would return 1 but if the netapp matches it returns 100.  If it 
does not handle at all you return -1.  This effectively gives you 
prioritization.  So the calling code would still loop through all strategies 
each time looking for the best match.  I don't want the priority to be 
determined at load time as that is less flexible. 

Darren

> On Oct 3, 2013, at 5:42 PM, "SuichII, Christopher"  
> wrote:
> 
> Thanks for the quick reply.
> 
> After talking with Edison, I think what we could do is allow the strategies 
> to either specify a priority or 'type' and then order them by that when they 
> are loaded. For example, we could have an enum of types like 'PLUGIN, 
> HYPERVISOR and DEFAULT' so that we can make sure plugins are asked 
> canHandle() first, then hypervisor implementations, then finally resort to 
> the default/ancient implementation. This 'type' or 'category' could be 
> specified as a bean parameter or as part of the strategy/provider interface. 
> Any thoughts on which are better?
> 
> The problem with just making canHandle() more intelligent is that we do need 
> some kind of ordering. Ideally, the non-default implementations should be 
> asked first, then fall back onto the stock implementations. You saw the 
> problem yourself - the XenServerSnapshotStrategy just says it will handle all 
> requests, so if a non-standard strategy wants to be given a chance, it must 
> be asked before the hypervisor or ancient implementation.
> 
> Alternatively if this matches the same usage of the global configuration 
> ordering system, we could add in the storage subsystem strategies and 
> providers.
> 
> The reason for log4j changes is that we may want to enable a different level 
> of verbosity by default. Our support organization likes to have very verbose 
> logs to assist them in triaging support issues. The lowest level log4j 
> categories are 'com.cloud', 'org' and 'net' and those are set to DEBUG and 
> INFO. Even if we add a line for 'com', the default value should not be 
> 'TRACE' like we would like ours to be. I'm not all that great with log4j, 
> though, so maybe I'm missing a simpler solution.
> 
> I'll try to keep an eye on the commands.properties/rbac stuff - that is good 
> to know.
> 
> Thanks,
> Chris
> -- 
> Chris Suich
> chris.su...@netapp.com
> NetApp Software Engineer
> Data Center Platforms – Cloud Solutions
> Citrix, Cisco & Red Hat
> 
>> On Oct 3, 2013, at 4:30 PM, Darren Shepherd  
>> wrote:
>> 
>> I forgot to mention about canHandle() ordering.  For extensions, it
>> should be preferred that we do not rely on loaded or configured order.
>> The canHandle() implementation should assume that they're may be any
>> order.  So having said that, I looked at XenServerSnapshotStrategy and
>> its canHandle() is brilliantly implemented as "return true;"
>> 
>> Can we look at making the strategies order independent and not having
>> another order configuration parameter?
>> 
>> Darren
>> 
>> On Thu, Oct 3, 2013 at 4:26 PM, Darren Shepherd
>>  wrote:
>>> Chris,
>>> 
>>> I've updated the wiki [1].  Copying from the wiki
>>> 
>>> "Extensions are automatically discovered based on the interfaces they
>>> implement and which module is their parent. For example, if you place
>>> a storage extension in a child module of the network module, it will
>>> not be discovered. Additionally, depending on the extension, the
>>> ordering may be important in how they extension is used. For the
>>> extensions that ordering is important there is also a "Global
>>> Configuration" setting for the ordering. The value of the setting is a
>>> comma seperated list like SHA256SALT,MD5,LDAP,PLAINTEXT. The values
>>> are the names of the extension. The name will be the result of
>>> getName(), if the bean implements the Named interface, or the sort
>>> class name (ie getClass().getSimpleName())."
>>> 
>>> I setup all the extensible types based on what was in
>>> componentContext.xml and also trial an error.  The types you have
>>> mentioned were not in componetContext.xml so I missed them.  It works
>>> today, but is not properly extensible.  I will update the
>>> configuration and wiki today.
>>> 
>>> I've haven't made progress on the commands.properties as I realized it
>>> will overlap with the rbac work being done.  I will start a thread
>>> regarding that so maybe it will be address in the rbac work.
>>> 
>>> Regarding log4j.  First, why do you have to change that?  Regardless,
>>> at the end of the day you are limited to whatever log4j can provide.
>>> So log4j AFAIK does not support multiple log4j config files so the
>>> best we can do is allow somebody to supply a different log4j config
>>> file and configure cloudstack to use that.  One of the things I don't
>>> particularily like is that ACS forces log4j to its own configuration
>>> file and will not allow the regular log4j config discovery.  So what I
>>> could 

Re: Wiki access

[Nguyen Anh Tu]

Hi Chip,

Please add me to grant list. My username is: tuna

Sent from my GT-N7000
On Oct 4, 2013 8:07 AM, "Go Chiba"  wrote:

> Hi Chip,
>
> Please grant me to edit pages. My id is "gochiba".
>
>
> On Thu, Oct 3, 2013 at 6:01 AM, Chip Childers  >wrote:
>
> > Not all committers were easy to find, but those that I did should be all
> > set.
> >
> > So committers / contributors = provide your confluence ID's if you can't
> > add / edit pages.  PMC folks - help add them please.
> >
> >
> > On Wed, Oct 2, 2013 at 4:48 PM, Chip Childers  > >wrote:
> >
> > > There were major issues with Spam on the wiki, so infra changed the
> > > permissions.
> > >
> > > I've just gone and added all PMC members to the space as space admins.
> > >  That means all PMC members can help deal with this issue (or at least
> > > those that I could find in the wiki user list).
> > >
> > > I'm going to deal with committers next.  That permission level needs to
> > > exclude space admin.  I'm using alena1108 as the example for committer
> /
> > > contributor permissions.
> > >
> > > So if you are not a committer, give your confluence ID in this thread
> and
> > > PMC folks can help get them added (make them look like alena's perms).
> > >
> > >
> > >
> > >
> > > On Wed, Oct 2, 2013 at 4:35 PM, Chiradeep Vittal <
> > > chiradeep.vit...@citrix.com> wrote:
> > >
> > >> Apparently, edit access has been revoked for all. Who do we contact
> for
> > >> edit permissions?
> > >>
> > >> TIA
> > >>
> > >
> > >
> >
>
>
>
> --
> 千葉 豪  Go Chiba
> E-mail:go.ch...@gmail.com
>

Re: Wiki access

[Go Chiba]

Hi Chip,

Please grant me to edit pages. My id is "gochiba".


On Thu, Oct 3, 2013 at 6:01 AM, Chip Childers wrote:

> Not all committers were easy to find, but those that I did should be all
> set.
>
> So committers / contributors = provide your confluence ID's if you can't
> add / edit pages.  PMC folks - help add them please.
>
>
> On Wed, Oct 2, 2013 at 4:48 PM, Chip Childers  >wrote:
>
> > There were major issues with Spam on the wiki, so infra changed the
> > permissions.
> >
> > I've just gone and added all PMC members to the space as space admins.
> >  That means all PMC members can help deal with this issue (or at least
> > those that I could find in the wiki user list).
> >
> > I'm going to deal with committers next.  That permission level needs to
> > exclude space admin.  I'm using alena1108 as the example for committer /
> > contributor permissions.
> >
> > So if you are not a committer, give your confluence ID in this thread and
> > PMC folks can help get them added (make them look like alena's perms).
> >
> >
> >
> >
> > On Wed, Oct 2, 2013 at 4:35 PM, Chiradeep Vittal <
> > chiradeep.vit...@citrix.com> wrote:
> >
> >> Apparently, edit access has been revoked for all. Who do we contact for
> >> edit permissions?
> >>
> >> TIA
> >>
> >
> >
>



-- 
千葉 豪  Go Chiba
E-mail:go.ch...@gmail.com

Review Request 14477: Refactor Storage Related Resource Code

[Chris Suich]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14477/
---

Review request for cloudstack.


Repository: cloudstack-git


Description
---

These changes are a joint effort between Edison and I to refactor some of the 
code around snapshotting VM volumes and creating templates/volumes from VM 
volume snapshots. In general, we were working towards allowing 
PrimaryDataStoreDrivers to create snapshots on primary storage and not 
requiring the snapshots to be transferred to secondary storage.

High level changes:
-Added uuid to NfsTO, SwiftTO & S3TO to cut down on the requirement of 
PrimaryDataStoreTO and ImageStoreTO which don't really serve much of a purpose
-Initial work towards enable reverting VM volume from snapshots
-Added hypervisor commands for introducing and forgetting new hypervisor 
objects (snapshots, templates & volumes)


Diffs
-

  api/src/com/cloud/agent/api/to/DataStoreTO.java 9014f8e 
  api/src/com/cloud/agent/api/to/NfsTO.java 415c95c 
  api/src/com/cloud/agent/api/to/SwiftTO.java 7349d77 
  api/src/com/cloud/event/EventTypes.java ec9604e 
  api/src/com/cloud/storage/snapshot/SnapshotApiService.java 23e6522 
  
api/src/org/apache/cloudstack/api/command/admin/storage/ListStoragePoolsCmd.java
 26351bb 
  
api/src/org/apache/cloudstack/api/command/user/snapshot/RevertSnapshotCmd.java 
PRE-CREATION 
  core/src/com/cloud/storage/resource/StorageProcessor.java 5fa9f8a 
  core/src/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java 
ab9aa2a 
  core/src/org/apache/cloudstack/storage/command/ForgetObjectCmd.java 
PRE-CREATION 
  core/src/org/apache/cloudstack/storage/command/IntroduceObjectAnswer.java 
PRE-CREATION 
  core/src/org/apache/cloudstack/storage/command/IntroduceObjectCmd.java 
PRE-CREATION 
  core/src/org/apache/cloudstack/storage/to/ImageStoreTO.java 0037ea5 
  core/src/org/apache/cloudstack/storage/to/PrimaryDataStoreTO.java 5e870df 
  
engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/EndPointSelector.java
 ca0cc2c 
  
engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/SnapshotService.java
 d594a07 
  
engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/SnapshotStrategy.java
 86ae532 
  
engine/storage/datamotion/src/org/apache/cloudstack/storage/motion/AncientDataMotionStrategy.java
 96d1f5a 
  
engine/storage/image/src/org/apache/cloudstack/storage/image/store/ImageStoreImpl.java
 855d8cb 
  
engine/storage/integration-test/test/org/apache/cloudstack/storage/test/SnapshotTestWithFakeData.java
 2aaabda 
  
engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/SnapshotServiceImpl.java
 3ead93f 
  
engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/SnapshotStrategyBase.java
 1b57922 
  
engine/storage/snapshot/src/org/apache/cloudstack/storage/snapshot/XenserverSnapshotStrategy.java
 60d9407 
  
engine/storage/src/org/apache/cloudstack/storage/endpoint/DefaultEndPointSelector.java
 fdc12bf 
  engine/storage/src/org/apache/cloudstack/storage/helper/HypervisorHelper.java 
PRE-CREATION 
  
engine/storage/src/org/apache/cloudstack/storage/helper/HypervisorHelperImpl.java
 PRE-CREATION 
  
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java
 82fd2ce 
  
plugins/hypervisors/simulator/src/com/cloud/resource/SimulatorStorageProcessor.java
 c7768aa 
  
plugins/hypervisors/vmware/src/com/cloud/storage/resource/VmwareStorageProcessor.java
 4982d87 
  
plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/XenServerStorageProcessor.java
 739b974 
  server/src/com/cloud/storage/snapshot/SnapshotManagerImpl.java 2297e6a 
  
services/secondary-storage/src/org/apache/cloudstack/storage/resource/NfsSecondaryStorageResource.java
 3ef950b 

Diff: https://reviews.apache.org/r/14477/diff/


Testing
---


Thanks,

Chris Suich

Re: Wiki access

[Srinivas Avasarala]

Need edit access. Username: savasarala

Re: [PROPOSAL] Modularize Spring

[SuichII, Christopher]

Thanks for the quick reply.

After talking with Edison, I think what we could do is allow the strategies to 
either specify a priority or 'type' and then order them by that when they are 
loaded. For example, we could have an enum of types like 'PLUGIN, HYPERVISOR 
and DEFAULT' so that we can make sure plugins are asked canHandle() first, then 
hypervisor implementations, then finally resort to the default/ancient 
implementation. This 'type' or 'category' could be specified as a bean 
parameter or as part of the strategy/provider interface. Any thoughts on which 
are better?

The problem with just making canHandle() more intelligent is that we do need 
some kind of ordering. Ideally, the non-default implementations should be asked 
first, then fall back onto the stock implementations. You saw the problem 
yourself - the XenServerSnapshotStrategy just says it will handle all requests, 
so if a non-standard strategy wants to be given a chance, it must be asked 
before the hypervisor or ancient implementation.

Alternatively if this matches the same usage of the global configuration 
ordering system, we could add in the storage subsystem strategies and providers.

The reason for log4j changes is that we may want to enable a different level of 
verbosity by default. Our support organization likes to have very verbose logs 
to assist them in triaging support issues. The lowest level log4j categories 
are 'com.cloud', 'org' and 'net' and those are set to DEBUG and INFO. Even if 
we add a line for 'com', the default value should not be 'TRACE' like we would 
like ours to be. I'm not all that great with log4j, though, so maybe I'm 
missing a simpler solution.

I'll try to keep an eye on the commands.properties/rbac stuff - that is good to 
know.

Thanks,
Chris
-- 
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Oct 3, 2013, at 4:30 PM, Darren Shepherd  wrote:

> I forgot to mention about canHandle() ordering.  For extensions, it
> should be preferred that we do not rely on loaded or configured order.
> The canHandle() implementation should assume that they're may be any
> order.  So having said that, I looked at XenServerSnapshotStrategy and
> its canHandle() is brilliantly implemented as "return true;"
> 
> Can we look at making the strategies order independent and not having
> another order configuration parameter?
> 
> Darren
> 
> On Thu, Oct 3, 2013 at 4:26 PM, Darren Shepherd
>  wrote:
>> Chris,
>> 
>> I've updated the wiki [1].  Copying from the wiki
>> 
>> "Extensions are automatically discovered based on the interfaces they
>> implement and which module is their parent. For example, if you place
>> a storage extension in a child module of the network module, it will
>> not be discovered. Additionally, depending on the extension, the
>> ordering may be important in how they extension is used. For the
>> extensions that ordering is important there is also a "Global
>> Configuration" setting for the ordering. The value of the setting is a
>> comma seperated list like SHA256SALT,MD5,LDAP,PLAINTEXT. The values
>> are the names of the extension. The name will be the result of
>> getName(), if the bean implements the Named interface, or the sort
>> class name (ie getClass().getSimpleName())."
>> 
>> I setup all the extensible types based on what was in
>> componentContext.xml and also trial an error.  The types you have
>> mentioned were not in componetContext.xml so I missed them.  It works
>> today, but is not properly extensible.  I will update the
>> configuration and wiki today.
>> 
>> I've haven't made progress on the commands.properties as I realized it
>> will overlap with the rbac work being done.  I will start a thread
>> regarding that so maybe it will be address in the rbac work.
>> 
>> Regarding log4j.  First, why do you have to change that?  Regardless,
>> at the end of the day you are limited to whatever log4j can provide.
>> So log4j AFAIK does not support multiple log4j config files so the
>> best we can do is allow somebody to supply a different log4j config
>> file and configure cloudstack to use that.  One of the things I don't
>> particularily like is that ACS forces log4j to its own configuration
>> file and will not allow the regular log4j config discovery.  So what I
>> could do is add a JVM property like -Dcloudstack.log4jstandardconfig
>> that will disable the ACS behavior of configuring log4j and then you
>> can use the log4j standard approach [2].
>> 
>> Darren
>> 
>> [1] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
>> [2] http://logging.apache.org/log4j/1.2/manual.html
>> 
>> On Thu, Oct 3, 2013 at 3:29 PM, SuichII, Christopher
>>  wrote:
>>> Darren,
>>> 
>>> I read through this thread and your docs on the wiki, but I'd appreciate it 
>>> if you could answer a couple questions for me:
>>> 
>>> -When creating an extension, such as a DataStoreProvider, those items are 
>>> currently add

Re: [PROPOSAL] Modularize Spring

[Darren Shepherd]

I forgot to mention about canHandle() ordering.  For extensions, it
should be preferred that we do not rely on loaded or configured order.
 The canHandle() implementation should assume that they're may be any
order.  So having said that, I looked at XenServerSnapshotStrategy and
its canHandle() is brilliantly implemented as "return true;"

Can we look at making the strategies order independent and not having
another order configuration parameter?

Darren

On Thu, Oct 3, 2013 at 4:26 PM, Darren Shepherd
 wrote:
> Chris,
>
> I've updated the wiki [1].  Copying from the wiki
>
> "Extensions are automatically discovered based on the interfaces they
> implement and which module is their parent. For example, if you place
> a storage extension in a child module of the network module, it will
> not be discovered. Additionally, depending on the extension, the
> ordering may be important in how they extension is used. For the
> extensions that ordering is important there is also a "Global
> Configuration" setting for the ordering. The value of the setting is a
> comma seperated list like SHA256SALT,MD5,LDAP,PLAINTEXT. The values
> are the names of the extension. The name will be the result of
> getName(), if the bean implements the Named interface, or the sort
> class name (ie getClass().getSimpleName())."
>
> I setup all the extensible types based on what was in
> componentContext.xml and also trial an error.  The types you have
> mentioned were not in componetContext.xml so I missed them.  It works
> today, but is not properly extensible.  I will update the
> configuration and wiki today.
>
> I've haven't made progress on the commands.properties as I realized it
> will overlap with the rbac work being done.  I will start a thread
> regarding that so maybe it will be address in the rbac work.
>
> Regarding log4j.  First, why do you have to change that?  Regardless,
> at the end of the day you are limited to whatever log4j can provide.
> So log4j AFAIK does not support multiple log4j config files so the
> best we can do is allow somebody to supply a different log4j config
> file and configure cloudstack to use that.  One of the things I don't
> particularily like is that ACS forces log4j to its own configuration
> file and will not allow the regular log4j config discovery.  So what I
> could do is add a JVM property like -Dcloudstack.log4jstandardconfig
> that will disable the ACS behavior of configuring log4j and then you
> can use the log4j standard approach [2].
>
> Darren
>
> [1] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
> [2] http://logging.apache.org/log4j/1.2/manual.html
>
> On Thu, Oct 3, 2013 at 3:29 PM, SuichII, Christopher
>  wrote:
>> Darren,
>>
>> I read through this thread and your docs on the wiki, but I'd appreciate it 
>> if you could answer a couple questions for me:
>>
>> -When creating an extension, such as a DataStoreProvider, those items are 
>> currently added into the list of providers for the appropriate bean, such as:
>> > 
>> class="org.apache.cloudstack.storage.datastore.provider.DataStoreProviderManagerImpl">
>> 
>>   
>> 
>> 
>> 
>> 
>> 
>>   
>> 
>>   
>>
>> So, how do we add our bean to that list?
>>
>> -There are a number of extensions that are not currently listed, such as 
>> DataMotionStrategy, SnapshotStrategy, etc. Is it a problem that those are 
>> omitted from 
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions?
>>
>> -I know somewhere in this thread you talked about the order of beans, but 
>> can you document how the ordering or precedence works in the wiki? For 
>> example, if I create a DataMotionStrategy, how do I ensure that my 
>> strategy's canHandle() method is invoked before the AncientDataStoreStrategy?
>>
>> -Is there any progress on modularizing commands.properties and the log4j 
>> configuration?
>>
>> Thanks,
>> Chris
>> --
>> Chris Suich
>> chris.su...@netapp.com
>> NetApp Software Engineer
>> Data Center Platforms – Cloud Solutions
>> Citrix, Cisco & Red Hat
>>

Re: [PROPOSAL] Modularize Spring

[Darren Shepherd]

Chris,

I've updated the wiki [1].  Copying from the wiki

"Extensions are automatically discovered based on the interfaces they
implement and which module is their parent. For example, if you place
a storage extension in a child module of the network module, it will
not be discovered. Additionally, depending on the extension, the
ordering may be important in how they extension is used. For the
extensions that ordering is important there is also a "Global
Configuration" setting for the ordering. The value of the setting is a
comma seperated list like SHA256SALT,MD5,LDAP,PLAINTEXT. The values
are the names of the extension. The name will be the result of
getName(), if the bean implements the Named interface, or the sort
class name (ie getClass().getSimpleName())."

I setup all the extensible types based on what was in
componentContext.xml and also trial an error.  The types you have
mentioned were not in componetContext.xml so I missed them.  It works
today, but is not properly extensible.  I will update the
configuration and wiki today.

I've haven't made progress on the commands.properties as I realized it
will overlap with the rbac work being done.  I will start a thread
regarding that so maybe it will be address in the rbac work.

Regarding log4j.  First, why do you have to change that?  Regardless,
at the end of the day you are limited to whatever log4j can provide.
So log4j AFAIK does not support multiple log4j config files so the
best we can do is allow somebody to supply a different log4j config
file and configure cloudstack to use that.  One of the things I don't
particularily like is that ACS forces log4j to its own configuration
file and will not allow the regular log4j config discovery.  So what I
could do is add a JVM property like -Dcloudstack.log4jstandardconfig
that will disable the ACS behavior of configuring log4j and then you
can use the log4j standard approach [2].

Darren

[1] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
[2] http://logging.apache.org/log4j/1.2/manual.html

On Thu, Oct 3, 2013 at 3:29 PM, SuichII, Christopher
 wrote:
> Darren,
>
> I read through this thread and your docs on the wiki, but I'd appreciate it 
> if you could answer a couple questions for me:
>
> -When creating an extension, such as a DataStoreProvider, those items are 
> currently added into the list of providers for the appropriate bean, such as:
>  
> class="org.apache.cloudstack.storage.datastore.provider.DataStoreProviderManagerImpl">
> 
>   
> 
> 
> 
> 
> 
>   
> 
>   
>
> So, how do we add our bean to that list?
>
> -There are a number of extensions that are not currently listed, such as 
> DataMotionStrategy, SnapshotStrategy, etc. Is it a problem that those are 
> omitted from 
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions?
>
> -I know somewhere in this thread you talked about the order of beans, but can 
> you document how the ordering or precedence works in the wiki? For example, 
> if I create a DataMotionStrategy, how do I ensure that my strategy's 
> canHandle() method is invoked before the AncientDataStoreStrategy?
>
> -Is there any progress on modularizing commands.properties and the log4j 
> configuration?
>
> Thanks,
> Chris
> --
> Chris Suich
> chris.su...@netapp.com
> NetApp Software Engineer
> Data Center Platforms – Cloud Solutions
> Citrix, Cisco & Red Hat
>

Re: Resize data-disk doesn't work after upgrade

[Marcus Sorensen]

What primary storage are you using? Any errors in agent log?
On Oct 3, 2013 3:16 PM, "Indra Pramana"  wrote:

> Hi Marcus,
>
> Good day to you, and thank you for your e-mail.
>
> I have tried restarting the VM and even stop and start the VM, but after
> logging in to the VM, I still see the hard drive's size as 20 GB instead of
> 60 GB.
>
> I tried to check /var/log/libvirt/libvirtd.log file on the KVM host where
> the VM is hosted, and can't find any messages related to volBlockResize.
>
> Any other troubleshooting steps you can recommend, i.e. any other area I
> can look into?
>
> Looking forward to your reply, thank you.
>
> Cheers.
>
>
>
> On Fri, Oct 4, 2013 at 4:46 AM, Marcus Sorensen 
> wrote:
>
> > I just tested local storage qcow2 and CLVM resize on 4.2, they both
> worked.
> >
> > Resize works like this:
> >
> > 1. Do sanity checks
> > 2. Send resize command to the agent
> > 3. Resize the disk/lun/file
> > 4. Inform the VM instance that the disk has changed by making a
> > libvirt volBlockResize call (this is not fatal, some guest types can't
> > resize online and need to be restarted)
> > 5. Update the database
> >
> > You can check #3 looking at the disks themselves on storage to see if
> > they've grown. You can check #4 by restarting the VM to see if it
> > picks up the change.
> >
> > It may be that libvirt was unable to inform the VM of the change (for
> > example if you haven't upgraded to a supported version of Ubuntu or
> > CentOS and it has an old libvirt that doesn't support volBlockResize).
> >  The way to know for sure is stop/start the VM if you can.
> >
> > Look at those two things and let us know
> >
> > On Thu, Oct 3, 2013 at 2:33 PM, Indra Pramana  wrote:
> > > Dear all,
> > >
> > > After upgrading to 4.2.0, I tried to resize a data disk of a VM
> instance
> > > from 20 GB to 60 GB, through the Cloudstack GUI. The UI reports that
> the
> > > resize was successful, and that the data disk is now showing 60 GB
> > instead
> > > of 20 GB. However, when I check the actual disk on the VM, it seems
> that
> > > it's still 20 GB.
> > >
> > > Any reason what might have been the cause of the problem? I even tried
> to
> > > re-partition it to see if the size changed, but it wasn't and still at
> 20
> > > GB. Which logs I need to look into?
> > >
> > > Any help on this is greatly appreciated.
> > >
> > > Looking forward to your reply, thank you.
> > >
> > > Cheers.
> >
>

Re: capacity of secondaey stor us eq 0. bug ir known ussue?

[Nitin Mehta]

This generally should happen because of management server not able to
communicate with the SSVM agent.
Could you run the ssvm health check script to see if the connection is
healthy btwn ssvm and CS ?

On 03/10/13 3:39 AM, "benoit lair"  wrote:

>Hello,
>
>I have the same issue with cs 4.2 installed from the cloudstack rpm
>repositories (the stable ones), found no way to get an ssvm operationnal.
>
>Thanks.
>
>
>2013/9/28 Netmaster 
>
>> Hallo,
>> we have several installation of cloudstack 4.2
>> In dashboard the capacity of secondary storage equal 0kb
>> We have standart nfs.
>> Api backed the same error, capacity of resource type 6 is 0
>> The secondary stor vm (system vm) shows normall capacity of 10tb,
>> as it have to be.
>> Is it bug or known issue?
>> Thanks

Re: [PROPOSAL] Modularize Spring

[SuichII, Christopher]

Darren,

I read through this thread and your docs on the wiki, but I'd appreciate it if 
you could answer a couple questions for me:

-When creating an extension, such as a DataStoreProvider, those items are 
currently added into the list of providers for the appropriate bean, such as:


  





  

  

So, how do we add our bean to that list?

-There are a number of extensions that are not currently listed, such as 
DataMotionStrategy, SnapshotStrategy, etc. Is it a problem that those are 
omitted from https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions?

-I know somewhere in this thread you talked about the order of beans, but can 
you document how the ordering or precedence works in the wiki? For example, if 
I create a DataMotionStrategy, how do I ensure that my strategy's canHandle() 
method is invoked before the AncientDataStoreStrategy?

-Is there any progress on modularizing commands.properties and the log4j 
configuration?

Thanks,
Chris
--
Chris Suich
chris.su...@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Sep 24, 2013, at 2:35 AM, Daan Hoogland 
mailto:daan.hoogl...@gmail.com>> wrote:

touching on thread hijack but; How does this work relate to the css
modularization going on at the moment as well? It is proposed there to
do merging at build time. Try to beat me if I am to much off topic,
Daan

On Tue, Sep 24, 2013 at 12:43 AM, Darren Shepherd
mailto:darren.s.sheph...@gmail.com>> wrote:
Ah right, okay.  So your talking about the order of the adapters.
Currently that is maintained as the order in the AdapterList in the
componentContext.xml.  So what I've done is that extensions get added
to "Registries."  Registries that need to be ordered can specify an
ordering configuration variable so that when the extensions are found
they are added to the list in a specific order.  So the registry
definition for the auth stuff looks something like


   
   
   
   

So one can use user.authenticators.order to change the order and
user.authenticators.exclude to exclude certain extensions from being
used.  The default value is also specified in that example.

Darren

On Mon, Sep 23, 2013 at 3:28 PM, Kelven Yang 
mailto:kelven.y...@citrix.com>> wrote:
I understand that loading order can be completely solved by Spring with
dependency injection, either within flat context or hierarchy context
structure. But some sibling plugins in CloudStack do require a flexible
way to declare ordering for runtime. For example, allocator plugins or
authenticator plugins. The order itself is currently designated in their
parent manager to reference to a ordered adapter list.

This order semantics has nothing to do with dependency injection, but
unfortunately in previous version of CloudStack, it does mix the
requirement into injection framework and there are business logic relying
on it. The fact for parent manager to compose ordering has made an
assumption at compile/load time binding to subject plugins, which we don't
want to see in the future since we want drop-in jar for plugins, what is
our answer for this?

Kelven


On 9/23/13 1:40 PM, "Darren Shepherd" 
mailto:darren.s.sheph...@gmail.com>> wrote:

Siblings have no relationship to each other really.  The load order
doesn't matter as one sibling has no visibility to another.  Child
contexts are pretty much for plug-ins.  Core components of ACS will
live in the core context as they are all interdependent.

I will explain the whole Spring lifecycle though and how it works in the
model.

Contexts are initialized from parent to child.  So the top most parent
context is initialized first, then its child, then its grandchildren.
When a context is initialized the following happens in the below order

1) All beans are instantiated
2) Dependencies are wire up (@Inject)
3) @PostConstruct is call for all beans in dependency graph order
4) Extensions are discovered an registered (so NetworkElement for
example will be discovered an registered as a NetworkElement)
5) configure() on all ComponentLifecycle beans will be called in the
getRunLevel() order

Once all modules have been initialized in this fashion then in a
parent first child last order start() on all ComponentLifecycle beans
is called following the getRunLevel() order for the beans in the
context.

Darren


On Mon, Sep 23, 2013 at 11:34 AM, Kelven Yang 
mailto:kelven.y...@citrix.com>>
wrote:
Darren,

Due to internal release work, I haven't got a chance to review it but
I'm
planning to do so later today and tomorrow. Before that, I have one
question about hierarchy-orginzated context structure, could you
elaborate
an example to the ML on how two sibling plugins to declare their runtime
load order? I'd like to get a feeling on how hard or easy for developers
to do things that involve with structural change under the new hierarchy
mode.

Kelven


On 9/23/13 12:

Re: System offerings mixed in with Disk Offerings

[Mike Tutkowski]

It appears to be an API issue.

I've opened CLOUDSTACK-4801 for this.


On Wed, Oct 2, 2013 at 2:35 PM, Mike Tutkowski  wrote:

> Sure.
>
> Preparing for a meeting now, but I should be able to do this later today.
>
> By the way, when you go to the Add Volume dialog and drop down the list of
> what is supposed to be Disk Offerings, you also get the System Offerings.
> They show blank descriptions, but I believe it's the System Offerings.
>
>
> On Wed, Oct 2, 2013 at 2:17 PM, Alena Prokharchyk <
> alena.prokharc...@citrix.com> wrote:
>
>>  Mike, can you check what Api call the UI makes (use firebug for that).
>> If all it does – call listDiskOfferings, then its an API bug. If the UI
>> does 2 calls – listServiceOfferings and listDiskOfferings, then its a UI
>> bug.
>>
>>  -alena.
>>
>>   From: Mike Tutkowski 
>> Reply-To: "dev@cloudstack.apache.org" 
>> Date: Wednesday, October 2, 2013 12:38 PM
>> To: "dev@cloudstack.apache.org" 
>> Subject: System offerings mixed in with Disk Offerings
>>
>>   Hi,
>>
>>  I assume this is a mistake:
>>
>>  http://imgur.com/XVBeNRf
>>
>>  System offerings are mixed in with Disk Offerings (in master).
>>
>>  --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkow...@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the
>> cloud
>> *™*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the 
> cloud
> *™*
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

4.2-forward branch is now merged into 4.2 branch

[Animesh Chaturvedi]

Folks

Last night I merged 4.2-forward branch into 4.2 branch. So from now on 4.2 
branch is available for bug fixes and will be the branch from which 4.2.1 will 
be released. 

Do not make any new checkins into 4.2-forward branch. I will keep 4.2-forward 
around for a few days just in case and will be deleted after that.

Thanks
Animesh

Re: Resize data-disk doesn't work after upgrade

[Indra Pramana]

Hi Marcus,

Good day to you, and thank you for your e-mail.

I have tried restarting the VM and even stop and start the VM, but after
logging in to the VM, I still see the hard drive's size as 20 GB instead of
60 GB.

I tried to check /var/log/libvirt/libvirtd.log file on the KVM host where
the VM is hosted, and can't find any messages related to volBlockResize.

Any other troubleshooting steps you can recommend, i.e. any other area I
can look into?

Looking forward to your reply, thank you.

Cheers.



On Fri, Oct 4, 2013 at 4:46 AM, Marcus Sorensen  wrote:

> I just tested local storage qcow2 and CLVM resize on 4.2, they both worked.
>
> Resize works like this:
>
> 1. Do sanity checks
> 2. Send resize command to the agent
> 3. Resize the disk/lun/file
> 4. Inform the VM instance that the disk has changed by making a
> libvirt volBlockResize call (this is not fatal, some guest types can't
> resize online and need to be restarted)
> 5. Update the database
>
> You can check #3 looking at the disks themselves on storage to see if
> they've grown. You can check #4 by restarting the VM to see if it
> picks up the change.
>
> It may be that libvirt was unable to inform the VM of the change (for
> example if you haven't upgraded to a supported version of Ubuntu or
> CentOS and it has an old libvirt that doesn't support volBlockResize).
>  The way to know for sure is stop/start the VM if you can.
>
> Look at those two things and let us know
>
> On Thu, Oct 3, 2013 at 2:33 PM, Indra Pramana  wrote:
> > Dear all,
> >
> > After upgrading to 4.2.0, I tried to resize a data disk of a VM instance
> > from 20 GB to 60 GB, through the Cloudstack GUI. The UI reports that the
> > resize was successful, and that the data disk is now showing 60 GB
> instead
> > of 20 GB. However, when I check the actual disk on the VM, it seems that
> > it's still 20 GB.
> >
> > Any reason what might have been the cause of the problem? I even tried to
> > re-partition it to see if the size changed, but it wasn't and still at 20
> > GB. Which logs I need to look into?
> >
> > Any help on this is greatly appreciated.
> >
> > Looking forward to your reply, thank you.
> >
> > Cheers.
>

System VM | Default Template

[Maurice Lawler]

Could I please inquire as to why when installing CS4.2 that the template 
it automatically installs is still CentOS 5.x, one would think this too 
would have been updated.


Is there information why this is the case?

- M.

Re: Resize data-disk doesn't work after upgrade

[Marcus Sorensen]

I just tested local storage qcow2 and CLVM resize on 4.2, they both worked.

Resize works like this:

1. Do sanity checks
2. Send resize command to the agent
3. Resize the disk/lun/file
4. Inform the VM instance that the disk has changed by making a
libvirt volBlockResize call (this is not fatal, some guest types can't
resize online and need to be restarted)
5. Update the database

You can check #3 looking at the disks themselves on storage to see if
they've grown. You can check #4 by restarting the VM to see if it
picks up the change.

It may be that libvirt was unable to inform the VM of the change (for
example if you haven't upgraded to a supported version of Ubuntu or
CentOS and it has an old libvirt that doesn't support volBlockResize).
 The way to know for sure is stop/start the VM if you can.

Look at those two things and let us know

On Thu, Oct 3, 2013 at 2:33 PM, Indra Pramana  wrote:
> Dear all,
>
> After upgrading to 4.2.0, I tried to resize a data disk of a VM instance
> from 20 GB to 60 GB, through the Cloudstack GUI. The UI reports that the
> resize was successful, and that the data disk is now showing 60 GB instead
> of 20 GB. However, when I check the actual disk on the VM, it seems that
> it's still 20 GB.
>
> Any reason what might have been the cause of the problem? I even tried to
> re-partition it to see if the size changed, but it wasn't and still at 20
> GB. Which logs I need to look into?
>
> Any help on this is greatly appreciated.
>
> Looking forward to your reply, thank you.
>
> Cheers.

Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]

[Daan Hoogland]

I know and I was underlining that in the part of my mail you didn't quote.
:p


On Thu, Oct 3, 2013 at 10:26 PM, Chip Childers wrote:

> On Thu, Oct 03, 2013 at 09:55:41PM +0200, Daan Hoogland wrote:
> > On Thu, Oct 3, 2013 at 9:02 PM, Chip Childers  >wrote:
> >
> > > a model for extensions like that makes perfect sense.
> >
> >
> >
> > This model sound fine indeed. It makes no sense for httpClose however.
>
> My personal comment was only about the usefulness of the approach, not
> the specific function in question. ;-)
>
> I don't have an opinion either way on this one, since I don't have
> enough information to weigh in.
>
> >
> > Here's my concern:
> > So when an early adapter is implemented and the rest of the market comes
> to
> > their senses, how do we migrate without running into migration/upgrade
> > problems?
> > httpClose is a flag controlling connection pooling. I probably choose the
> > wrong name. It is something that any implementation will support or
> should
> > have supported already. Am I going to implement it as a key/value now to
> > later implemented as I have done anyway? I don't like this idea.
> >
> > Don't get me wrong the pattern described by you guys is fine in some
> > situations. I don't think it is applicable to this feature.
> >
> > regards,
> > Daan
>

Re: [Proposal] Improve VR upgrades

[Daan Hoogland]

At Schuberg Philis we are looking at supporting HA redundant virtual
routers to be able to upgrade without any downtime.  I don't think there is
any other way to go in the future, for multi-tenant corporate environments.
Upgrading will then mean destroying the redundant pair one by one while
waiting with destroying the second one till the first is back up.

Of course this is not usefull for other systemvms then the router vms. Do
you see a place for this in your design, Kishan?

As for Chip's concern; another implementation would be to allow for a
versioned vm/ms interface, instead of a backwards compatible one. Any way
it is a matter to deal with somehow.

Daan



On Thu, Oct 3, 2013 at 5:21 PM, Chip Childers wrote:

> On Thu, Oct 03, 2013 at 11:47:57AM +, Kishan Kavala wrote:
> > During CS upgrade, VRs are required to be upgraded to use newer systemVm
> template .
> > The current VR upgrade procedure has following limitations:
> >  - takes 'long' time and the time exponentially increases with the size
> of the cloud
> > - no way to sequence upgrade of different parts of the cloud, i.e.,
> specific clusters or pods or even zones
> > - there is no way to determine when a particular customer's services
> (e.g. VR) will be upgraded with the upgrade interval
> >
> > Goals for this feature are to address the above issues
> >
> > 1. Give admin control to sequence the upgrade of the cloud by:
> >- Infrastructure hierarchy: by Cluster, Pod, and Zone etc.
> >- Administrative hierarchy: by Tenant or Domain
> > 2. Minimize service interruption to users
> > 3. Improve the speed of the upgrade time by making as many upgrade
> operations in parallel as possible
> >
> > I've created JIRA ticket:
> > https://issues.apache.org/jira/browse/CLOUDSTACK-4793
> >
> > thanks,
> > Kishan
> >
>
> This proposal sounds great, but the devil will be in the implementation
> details.  To do this as rolling, we'd need to ensure backward compat
> with agent>MS communications, right?
>
> -chip
>

Resize data-disk doesn't work after upgrade

[Indra Pramana]

Dear all,

After upgrading to 4.2.0, I tried to resize a data disk of a VM instance
from 20 GB to 60 GB, through the Cloudstack GUI. The UI reports that the
resize was successful, and that the data disk is now showing 60 GB instead
of 20 GB. However, when I check the actual disk on the VM, it seems that
it's still 20 GB.

Any reason what might have been the cause of the problem? I even tried to
re-partition it to see if the size changed, but it wasn't and still at 20
GB. Which logs I need to look into?

Any help on this is greatly appreciated.

Looking forward to your reply, thank you.

Cheers.

Re: [URGENT] Mounting issues from KVM hosts to secondary storage

[Indra Pramana]

Hi Marcus,

Good day to you, and thank you for your e-mail.

We will try to do what you have suggested and will see if we can fix the
problem. I am looking into increasing the filesystem of the KVM hosts to
see if it can resolve the problem for now.

Thank you very much for your help!

Cheers.



On Fri, Oct 4, 2013 at 2:32 AM, Marcus Sorensen  wrote:

> You'll have to go back further in the log then, and/or go to the agent
> log, because the issue of the ghost mount is due to the filesystem
> being full, and if that's your secondary storage I'm not sure why it
> would be written to from the kvm host unless you were doing backups or
> snapshots.
>
> On Thu, Oct 3, 2013 at 12:03 PM, Indra Pramana  wrote:
> > Hi Marcus,
> >
> > It happens randomly, mounting fails, and then / gets filled. I suspect
> this
> > occurs when the mounting fails, any files which is supposed to be
> > transferred to the mount point (e.g. templates used to create VMs) will
> > fill-up the / partition of the KVM host instead. This doesn't happen in
> > 4.1.1, this only happens after we upgraded to 4.2.0.
> >
> > And how about virsh pool-list result which is not reflecting the actual
> IDs
> > of the secondary storage we see from Cloudstack GUI, is it supposed to be
> > the case? If not, how to rectify it?
> >
> > Looking forward to your reply, thank you.
> >
> > Cheers.
> >
> >
> >
> >
> > On Fri, Oct 4, 2013 at 1:51 AM, Marcus Sorensen 
> wrote:
> >
> >> It sort of looks like the out of space triggered the issue. Libvirt
> shows
> >>
> >> 2013-10-03 15:38:57.414+: 2710: error : virCommandWait:2348 :
> >> internal error Child process (/bin/umount
> >> /mnt/5230667e-9c58-3ff6-983c-5fc2a72df669) unexpected exit status 32:
> >> error writing /etc/mtab.tmp: No space left on device
> >>
> >> So that entry of the filesystem being mounted is orphaned in
> >> /etc/mtab, since it can't be removed. That seems to be the source of
> >> why 'df' shows the thing mounted when it isn't, at least.
> >>
> >>
> >> On Thu, Oct 3, 2013 at 11:45 AM, Indra Pramana  wrote:
> >> > Hi Marcus and all,
> >> >
> >> > I also find some strange and interesting error messages from the
> libvirt
> >> > logs:
> >> >
> >> > http://pastebin.com/5ByfNpAf
> >> >
> >> > Looking forward to your reply, thank you.
> >> >
> >> > Cheers.
> >> >
> >> >
> >> >
> >> > On Fri, Oct 4, 2013 at 1:38 AM, Indra Pramana  wrote:
> >> >
> >> >> Hi Marcus,
> >> >>
> >> >> Good day to you, and thank you for your e-mail. See my reply inline.
> >> >>
> >> >> On Fri, Oct 4, 2013 at 12:29 AM, Marcus Sorensen <
> shadow...@gmail.com
> >> >wrote:
> >> >>
> >> >>> Are you using the release artifacts, or your own 4.2 build?
> >> >>>
> >> >>
> >> >> [Indra:] We are using the release artifacts from below repo since we
> are
> >> >> using Ubuntu:
> >> >>
> >> >> deb http://cloudstack.apt-get.eu/ubuntu precise 4.2
> >> >>
> >> >>
> >> >>> When you rebooted the host, did the problem go away or come back the
> >> >>> same?
> >> >>
> >> >>
> >> >> [Indra:] When I rebooted the host, the problem go away for a while,
> but
> >> it
> >> >> will come back again after some time. It will come randomly at the
> time
> >> >> when we need to create a new instance on that host, or start an
> existing
> >> >> stopped instance.
> >> >>
> >> >>
> >> >>> You may want to look at 'virsh pool-list' to see if libvirt is
> >> >>> mounting/registering the secondary storage.
> >> >>>
> >> >>
> >> >> [Indra:] This is the result of the virsh pool-list command:
> >> >>
> >> >> root@hv-kvm-02:/var/log/libvirt# virsh pool-list
> >> >> Name State  Autostart
> >> >> -
> >> >> 301071ac-4c1d-4eac-855b-124126da0a38 active no
> >> >> 5230667e-9c58-3ff6-983c-5fc2a72df669 active no
> >> >> d433809b-01ea-3947-ba0f-48077244e4d6 active no
> >> >>
> >> >> Strange thing is that none of my secondary storage IDs are there.
> Could
> >> it
> >> >> be that the ID might have changed during Cloudstack upgrade? Here is
> the
> >> >> list of my secondary storage (there are two of them) even though they
> >> are
> >> >> on the same NFS server:
> >> >>   c02da448-b9f4-401b-b8d5-83e8ead5cfde nfs://
> >> >> 10.237.11.31/mnt/vol1/sec-storage NFS
> >> >> 5937edb6-2e95-4ae2-907b-80fe4599ed87 nfs://
> >> >> 10.237.11.31/mnt/vol1/sec-storage2 NFS
> >> >>
> >> >>> Is this happening on multiple hosts, the same way?
> >> >>
> >> >>
> >> >> [Indra:] Yes, it is happening on all the two hosts that I have, the
> same
> >> >> way.
> >> >>
> >> >>
> >> >>> You may want to
> >> >>> look at /etc/mtab, if the system reports it's mounted, though it's
> >> >>> not, it might be in there. Look at /proc/mounts as well.
> >> >>>
> >> >>
> >> >> [Indra:] Please find result of df, /etc/mtab and /proc/mounts below.
> The
> >> >> "ghost" mount point is on df and /etc/mtab, but not on /proc/mounts.
> >> >>
> >> >> root@hv-kvm-02:/etc# df
> >> >>
> >> >> Filesystem  1K

Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]

[Chip Childers]

On Thu, Oct 03, 2013 at 09:55:41PM +0200, Daan Hoogland wrote:
> On Thu, Oct 3, 2013 at 9:02 PM, Chip Childers 
> wrote:
> 
> > a model for extensions like that makes perfect sense.
> 
> 
> 
> This model sound fine indeed. It makes no sense for httpClose however.

My personal comment was only about the usefulness of the approach, not
the specific function in question. ;-)

I don't have an opinion either way on this one, since I don't have
enough information to weigh in.

> 
> Here's my concern:
> So when an early adapter is implemented and the rest of the market comes to
> their senses, how do we migrate without running into migration/upgrade
> problems?
> httpClose is a flag controlling connection pooling. I probably choose the
> wrong name. It is something that any implementation will support or should
> have supported already. Am I going to implement it as a key/value now to
> later implemented as I have done anyway? I don't like this idea.
> 
> Don't get me wrong the pattern described by you guys is fine in some
> situations. I don't think it is applicable to this feature.
> 
> regards,
> Daan

Review Request 14471: CloudStack Plugin guide for VXLAN

[Yoshikazu Nojima]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14471/
---

Review request for cloudstack and Toshiaki Hatano.


Repository: cloudstack-git


Description
---

This patch adds a Plugin guide for VXLAN.
It describes system requirements for VXLAN plugin and how to setup it.

This is the ToC:
1. The VXLAN Plugin  1
1.1. Introduction to the VXLAN Plugin  1
1.2. Features of the VXLAN Plugin  1
2. System Requirements for VXLAN 3
2.1. System Requirements for VXLAN  3
2.2. Linux Distributions that meet the requirements  3
2.3. Check the capability of your system  3
2.4. Advanced: Build kernel and iproute2  3
2.4.1. Build kernel  4
2.4.2. Build iproute2  4
3. Configure CloudStack to use VXLAN Plugin 7
3.1. Configure hypervisor  7
3.1.1. Configure hypervisor: KVM  7
3.2. Setup zone using VXLAN  9
3.2.1. Configure the physical network  10


Diffs
-

  docs/en-US/CloudStack_VXLAN_Guide.ent PRE-CREATION 
  docs/en-US/CloudStack_VXLAN_Guide.xml PRE-CREATION 
  docs/en-US/images/vxlan-physicalnetwork.png PRE-CREATION 
  docs/en-US/images/vxlan-trafficlabel.png PRE-CREATION 
  docs/en-US/plugin-vxlan-about.xml PRE-CREATION 
  docs/en-US/plugin-vxlan-config-hypervisor.xml PRE-CREATION 
  docs/en-US/plugin-vxlan-config-management.xml PRE-CREATION 
  docs/en-US/plugin-vxlan-configuration.xml PRE-CREATION 
  docs/en-US/plugin-vxlan-features.xml PRE-CREATION 
  docs/en-US/plugin-vxlan-introduction.xml PRE-CREATION 
  docs/en-US/plugin-vxlan-requirements.xml PRE-CREATION 
  docs/en-US/plugin-vxlan-revision-history.xml PRE-CREATION 
  docs/publican-plugin-vxlan.cfg PRE-CREATION 

Diff: https://reviews.apache.org/r/14471/diff/


Testing
---


Thanks,

Yoshikazu Nojima

Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]

[Daan Hoogland]

On Thu, Oct 3, 2013 at 9:02 PM, Chip Childers wrote:

> a model for extensions like that makes perfect sense.



This model sound fine indeed. It makes no sense for httpClose however.

Here's my concern:
So when an early adapter is implemented and the rest of the market comes to
their senses, how do we migrate without running into migration/upgrade
problems?
httpClose is a flag controlling connection pooling. I probably choose the
wrong name. It is something that any implementation will support or should
have supported already. Am I going to implement it as a key/value now to
later implemented as I have done anyway? I don't like this idea.

Don't get me wrong the pattern described by you guys is fine in some
situations. I don't think it is applicable to this feature.

regards,
Daan

Re: Wiki access

[Manan Shah]

Thanks Chip.

Regards,
Manan Shah




On 10/3/13 12:51 PM, "Chip Childers"  wrote:

>On Thu, Oct 03, 2013 at 07:48:34PM +, Manan Shah wrote:
>> My id is manans. Can someone grant me access?
>
>Done

Re: Wiki access

[Chip Childers]

On Thu, Oct 03, 2013 at 07:48:34PM +, Manan Shah wrote:
> My id is manans. Can someone grant me access?

Done

Re: Wiki access

[Manan Shah]

My id is manans. Can someone grant me access?

Regards,
Manan Shah




On 10/3/13 12:36 PM, "Daan Hoogland"  wrote:

>wow, i feel overpowered.
>
>
>On Thu, Oct 3, 2013 at 7:52 PM, Chip Childers
>wrote:
>
>> On Thu, Oct 03, 2013 at 05:47:15PM +, Chiradeep Vittal wrote:
>> > Daan, it is at the 'Browse' drop down at the upper right. Heh
>>
>> Yeah - not sure what UX genius decided to put it there...  ;-)
>>

Re: Wiki access

[Daan Hoogland]

wow, i feel overpowered.


On Thu, Oct 3, 2013 at 7:52 PM, Chip Childers wrote:

> On Thu, Oct 03, 2013 at 05:47:15PM +, Chiradeep Vittal wrote:
> > Daan, it is at the 'Browse' drop down at the upper right. Heh
>
> Yeah - not sure what UX genius decided to put it there...  ;-)
>

Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]

[Chip Childers]

> On Oct 3, 2013, at 3:01 PM, Chiradeep Vittal  
> wrote:
>
> I was thinking along the same lines that a offering could have a generic
> key/value map (where the value could be a string/json string/whatever).
> CloudStack core wouldn't have any idea about the semantics of the keys or
> values, but the specific hypervisor/provider might decide to interpret it.

+1 - a model for extensions like that makes perfect sense.

>
>
>> On 10/3/13 11:15 AM, "Darren Shepherd"  wrote:
>>
>> We are always going to run into the issue where they're implementation
>> specific features that can't be represented in a generic way in the
>> API.  First class attributes in the API need to essentially be the
>> lowest common denominator.  ACS already has this pattern of *_details
>> tables.  I honestly don't know completely how they are used, but I
>> would hope that pattern could be used to address these types of
>> issues.  By allowing the admin to add arbitrary key/value pairs to the
>> various offerings/entities we can use that to tap into provider
>> specific or advanced configuration.  For the http close proposal you
>> could just add haproxy.httpClose=true on the network offering and the
>> haproxy impl can respect that and use it.  So implementations can
>> document what additional key/value parameters they support to access
>> advanced features.
>>
>> Review 13896 adds an otherOptions field to the service offering.
>> Conceptually that is fine in my mind, but I just don't understand why
>> the *_details tables can't be used instead.  Just like XenServer has
>> other_config on all types, I'd like it if ACS had a similar thing
>> (which I kinda, sorta think it does?).  So we should be able to
>> arbitrary data onto any type.  Can somebody comment on how this idea
>> may relate with the existing implementation of "details" and "tags" in
>> ACS?
>>
>> Darren
>>
>> On Thu, Oct 3, 2013 at 10:21 AM, Daan Hoogland 
>> wrote:
>>> Murali,
>>>
>>> What our admins need is the ability to instantiate an abstract thing
>>> called
>>> a virtual redundant high available router. They need be able to tune it
>>> with all sorts of features is such a way that other routers redundant or
>>> not and high availible or not, may but do not have to have the same
>>> tuning
>>> parameters. This 'feature' of httpClose is just one of the many things
>>> they
>>> need to be able to tune. Others include a more fine grained control over
>>> the iptables/firewall rules and monitoring of the functionality/daemons
>>> running on the machines. I am not biased as to the way how to
>>> do/implement
>>> this control. The networkoffering seemed like the way to do it.
>>>
>>> Having said that I didn't think that httpClose would be considered
>>> haproxy
>>> specific. It seems worrying that someone could device a proxy or a
>>> loadbalancer that does not support either one of the features of
>>> maximizing - or pooling connections. I'm not into that market recently
>>> so I
>>> might be mistaking. This maybe besides the point but it discomforts me
>>> somewhat.
>>>
>>> regards,
>>> Daan
>>>
>>>
>>> On Thu, Oct 3, 2013 at 2:22 PM, murali reddy
>>> wrote:
>>>
 On Thu, Oct 3, 2013 at 4:18 PM, Daan Hoogland  wrote:

> Chiradeep,
>
> I have been thinking about your concern and there is something
 bothering
 me
> about it. The issue CLOUDSTACK-4328 of which
> https://reviews.apache.org/r/14320/ is an implementation. This issue
 has
> been brought up by the engineers at Schuberg Philis. These are cloud
> operators and domain admins. So these are the people that need to be
> bothered by this tuning and they are certainly haproxy and xen
 experts to
> an extend. For them not being able to use connection caching was a
 bug.
 And
> the proposed solution still seems reasonable to me. It is exposing
 the
> abstract feature only to those instantiating a vpc, isn't it? This
 last
> question probably touches on the reason you are addressing my
 submission
> together with the ones from Nicolas  I see only people instantiating
 a
 vpc
> having to be bothered by which netoffering to use (with respect to
> httpClose functionality) If this is not the case how should I isolate
 this
> concern from other , more mondain users?
>
> btw I did not create an FS page as the issue was created as a jira
 ticket.
> I am willing to do that in hind sight but want to have a path to
 follow
> first.
>
> regards,
> Daan
>
> On Tue, Oct 1, 2013 at 11:06 PM, Chiradeep Vittal <
> chiradeep.vit...@citrix.com> wrote:
>>
>> We have a couple of people trying to expose the advanced
 capabilities
 of
> the underlying physical resources to the end-user. In the case of
 Nicolas
> FOATA, he is trying to expose some of the advanced functions of
> XenServer/XCP, and in the case of Daan, he is trying to expose so

Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]

[Chiradeep Vittal]

I was thinking along the same lines that a offering could have a generic
key/value map (where the value could be a string/json string/whatever).
CloudStack core wouldn't have any idea about the semantics of the keys or
values, but the specific hypervisor/provider might decide to interpret it.


On 10/3/13 11:15 AM, "Darren Shepherd"  wrote:

>We are always going to run into the issue where they're implementation
>specific features that can't be represented in a generic way in the
>API.  First class attributes in the API need to essentially be the
>lowest common denominator.  ACS already has this pattern of *_details
>tables.  I honestly don't know completely how they are used, but I
>would hope that pattern could be used to address these types of
>issues.  By allowing the admin to add arbitrary key/value pairs to the
>various offerings/entities we can use that to tap into provider
>specific or advanced configuration.  For the http close proposal you
>could just add haproxy.httpClose=true on the network offering and the
>haproxy impl can respect that and use it.  So implementations can
>document what additional key/value parameters they support to access
>advanced features.
>
>Review 13896 adds an otherOptions field to the service offering.
>Conceptually that is fine in my mind, but I just don't understand why
>the *_details tables can't be used instead.  Just like XenServer has
>other_config on all types, I'd like it if ACS had a similar thing
>(which I kinda, sorta think it does?).  So we should be able to
>arbitrary data onto any type.  Can somebody comment on how this idea
>may relate with the existing implementation of "details" and "tags" in
>ACS?
>
>Darren
>
>On Thu, Oct 3, 2013 at 10:21 AM, Daan Hoogland 
>wrote:
>> Murali,
>>
>> What our admins need is the ability to instantiate an abstract thing
>>called
>> a virtual redundant high available router. They need be able to tune it
>> with all sorts of features is such a way that other routers redundant or
>> not and high availible or not, may but do not have to have the same
>>tuning
>> parameters. This 'feature' of httpClose is just one of the many things
>>they
>> need to be able to tune. Others include a more fine grained control over
>> the iptables/firewall rules and monitoring of the functionality/daemons
>> running on the machines. I am not biased as to the way how to
>>do/implement
>> this control. The networkoffering seemed like the way to do it.
>>
>> Having said that I didn't think that httpClose would be considered
>>haproxy
>> specific. It seems worrying that someone could device a proxy or a
>> loadbalancer that does not support either one of the features of
>> maximizing - or pooling connections. I'm not into that market recently
>>so I
>> might be mistaking. This maybe besides the point but it discomforts me
>> somewhat.
>>
>> regards,
>> Daan
>>
>>
>> On Thu, Oct 3, 2013 at 2:22 PM, murali reddy
>>wrote:
>>
>>> On Thu, Oct 3, 2013 at 4:18 PM, Daan Hoogland >> >wrote:
>>>
>>> > Chiradeep,
>>> >
>>> > I have been thinking about your concern and there is something
>>>bothering
>>> me
>>> > about it. The issue CLOUDSTACK-4328 of which
>>> > https://reviews.apache.org/r/14320/ is an implementation. This issue
>>>has
>>> > been brought up by the engineers at Schuberg Philis. These are cloud
>>> > operators and domain admins. So these are the people that need to be
>>> > bothered by this tuning and they are certainly haproxy and xen
>>>experts to
>>> > an extend. For them not being able to use connection caching was a
>>>bug.
>>> And
>>> > the proposed solution still seems reasonable to me. It is exposing
>>>the
>>> > abstract feature only to those instantiating a vpc, isn't it? This
>>>last
>>> > question probably touches on the reason you are addressing my
>>>submission
>>> > together with the ones from Nicolas  I see only people instantiating
>>>a
>>> vpc
>>> > having to be bothered by which netoffering to use (with respect to
>>> > httpClose functionality) If this is not the case how should I isolate
>>> this
>>> > concern from other , more mondain users?
>>> >
>>> > btw I did not create an FS page as the issue was created as a jira
>>> ticket.
>>> > I am willing to do that in hind sight but want to have a path to
>>>follow
>>> > first.
>>> >
>>> > regards,
>>> > Daan
>>> >
>>> > On Tue, Oct 1, 2013 at 11:06 PM, Chiradeep Vittal <
>>> > chiradeep.vit...@citrix.com> wrote:
>>> > >
>>> > > We have a couple of people trying to expose the advanced
>>>capabilities
>>> of
>>> > the underlying physical resources to the end-user. In the case of
>>>Nicolas
>>> > FOATA, he is trying to expose some of the advanced functions of
>>> > XenServer/XCP, and in the case of Daan, he is trying to expose some
>>> feature
>>> > of HAProxy.
>>> > >
>>> > > Users are ideally abstracted from these details and shouldn't have
>>>to
>>> > wonder which offering to choose [because they are not Xen/HAProxy
>>> experts].
>>> > > After all one of the goals of CS is to 

RE: Post-release work

[Animesh Chaturvedi]

I merged 4.2-forward into 4.2 last night. I will keep 4.2-forward around for 
couple of days just in case something is missed. I will send a separate email 
to mailing list that 4.2 is now open for checkin and do not checkin into 
4.2-forward.

> -Original Message-
> From: Animesh Chaturvedi [mailto:animesh.chaturv...@citrix.com]
> Sent: Wednesday, October 02, 2013 12:13 PM
> To: dev@cloudstack.apache.org
> Subject: RE: Post-release work
> 
> 
> 
> > -Original Message-
> > From: David Nalley [mailto:da...@gnsa.us]
> > Sent: Wednesday, October 02, 2013 7:22 AM
> > To: dev@cloudstack.apache.org
> > Subject: Post-release work
> >
> > As previously discussed, now that 4.2 is released, I moving docs to
> > their own repo.
> >
> > https://git-wip-us.apache.org/repos/cloudstack-docs.git
> >
> > For present, I am pushing the 4.2 docs into a 4.2 branch, and will
> > deal with master in a bit.
> >
> >
> > We also need to merge anything in 4.2-forward into 4.2 and purge 4.2-
> > forward as we prepare for 4.2.1.
> [Animesh>] Yes I plan to do it later today.

> >
> > --David

Re: [URGENT] Mounting issues from KVM hosts to secondary storage

[Marcus Sorensen]

You'll have to go back further in the log then, and/or go to the agent
log, because the issue of the ghost mount is due to the filesystem
being full, and if that's your secondary storage I'm not sure why it
would be written to from the kvm host unless you were doing backups or
snapshots.

On Thu, Oct 3, 2013 at 12:03 PM, Indra Pramana  wrote:
> Hi Marcus,
>
> It happens randomly, mounting fails, and then / gets filled. I suspect this
> occurs when the mounting fails, any files which is supposed to be
> transferred to the mount point (e.g. templates used to create VMs) will
> fill-up the / partition of the KVM host instead. This doesn't happen in
> 4.1.1, this only happens after we upgraded to 4.2.0.
>
> And how about virsh pool-list result which is not reflecting the actual IDs
> of the secondary storage we see from Cloudstack GUI, is it supposed to be
> the case? If not, how to rectify it?
>
> Looking forward to your reply, thank you.
>
> Cheers.
>
>
>
>
> On Fri, Oct 4, 2013 at 1:51 AM, Marcus Sorensen  wrote:
>
>> It sort of looks like the out of space triggered the issue. Libvirt shows
>>
>> 2013-10-03 15:38:57.414+: 2710: error : virCommandWait:2348 :
>> internal error Child process (/bin/umount
>> /mnt/5230667e-9c58-3ff6-983c-5fc2a72df669) unexpected exit status 32:
>> error writing /etc/mtab.tmp: No space left on device
>>
>> So that entry of the filesystem being mounted is orphaned in
>> /etc/mtab, since it can't be removed. That seems to be the source of
>> why 'df' shows the thing mounted when it isn't, at least.
>>
>>
>> On Thu, Oct 3, 2013 at 11:45 AM, Indra Pramana  wrote:
>> > Hi Marcus and all,
>> >
>> > I also find some strange and interesting error messages from the libvirt
>> > logs:
>> >
>> > http://pastebin.com/5ByfNpAf
>> >
>> > Looking forward to your reply, thank you.
>> >
>> > Cheers.
>> >
>> >
>> >
>> > On Fri, Oct 4, 2013 at 1:38 AM, Indra Pramana  wrote:
>> >
>> >> Hi Marcus,
>> >>
>> >> Good day to you, and thank you for your e-mail. See my reply inline.
>> >>
>> >> On Fri, Oct 4, 2013 at 12:29 AM, Marcus Sorensen > >wrote:
>> >>
>> >>> Are you using the release artifacts, or your own 4.2 build?
>> >>>
>> >>
>> >> [Indra:] We are using the release artifacts from below repo since we are
>> >> using Ubuntu:
>> >>
>> >> deb http://cloudstack.apt-get.eu/ubuntu precise 4.2
>> >>
>> >>
>> >>> When you rebooted the host, did the problem go away or come back the
>> >>> same?
>> >>
>> >>
>> >> [Indra:] When I rebooted the host, the problem go away for a while, but
>> it
>> >> will come back again after some time. It will come randomly at the time
>> >> when we need to create a new instance on that host, or start an existing
>> >> stopped instance.
>> >>
>> >>
>> >>> You may want to look at 'virsh pool-list' to see if libvirt is
>> >>> mounting/registering the secondary storage.
>> >>>
>> >>
>> >> [Indra:] This is the result of the virsh pool-list command:
>> >>
>> >> root@hv-kvm-02:/var/log/libvirt# virsh pool-list
>> >> Name State  Autostart
>> >> -
>> >> 301071ac-4c1d-4eac-855b-124126da0a38 active no
>> >> 5230667e-9c58-3ff6-983c-5fc2a72df669 active no
>> >> d433809b-01ea-3947-ba0f-48077244e4d6 active no
>> >>
>> >> Strange thing is that none of my secondary storage IDs are there. Could
>> it
>> >> be that the ID might have changed during Cloudstack upgrade? Here is the
>> >> list of my secondary storage (there are two of them) even though they
>> are
>> >> on the same NFS server:
>> >>   c02da448-b9f4-401b-b8d5-83e8ead5cfde nfs://
>> >> 10.237.11.31/mnt/vol1/sec-storage NFS
>> >> 5937edb6-2e95-4ae2-907b-80fe4599ed87 nfs://
>> >> 10.237.11.31/mnt/vol1/sec-storage2 NFS
>> >>
>> >>> Is this happening on multiple hosts, the same way?
>> >>
>> >>
>> >> [Indra:] Yes, it is happening on all the two hosts that I have, the same
>> >> way.
>> >>
>> >>
>> >>> You may want to
>> >>> look at /etc/mtab, if the system reports it's mounted, though it's
>> >>> not, it might be in there. Look at /proc/mounts as well.
>> >>>
>> >>
>> >> [Indra:] Please find result of df, /etc/mtab and /proc/mounts below. The
>> >> "ghost" mount point is on df and /etc/mtab, but not on /proc/mounts.
>> >>
>> >> root@hv-kvm-02:/etc# df
>> >>
>> >> Filesystem  1K-blocks
>>  Used
>> >> Available Use% Mounted on
>> >> /dev/sda1 4195924
>> >> 4192372 0 100% /
>> >>
>> >> udev132053356
>> 4
>> >> 132053352   1% /dev
>> >> tmpfs52825052
>> 704
>> >> 52824348   1% /run
>> >>
>> >> none 5120
>> >> 0  5120   0% /run/lock
>> >> none132062620
>> 0
>> >> 132062620   0% /run/shm
>> >> cgroup  132062620

Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]

[Kelven Yang]

*_details table is currently used as the generic way of passing key/value
pairs that are transparent to API layer and core CloudStack orchestration
flow. When I say transparent, the scope is only limited to API layer and
core generic orchestration flow, specific plugin elements or Gurus still
have the chance to further interpret these key/value data with current
Guru/Element pattern in the flow.

So, we do have such generic facility, but how to present in a more
domain-specific and friendly way is another problem.

Kelven

On 10/3/13 11:15 AM, "Darren Shepherd"  wrote:

>We are always going to run into the issue where they're implementation
>specific features that can't be represented in a generic way in the
>API.  First class attributes in the API need to essentially be the
>lowest common denominator.  ACS already has this pattern of *_details
>tables.  I honestly don't know completely how they are used, but I
>would hope that pattern could be used to address these types of
>issues.  By allowing the admin to add arbitrary key/value pairs to the
>various offerings/entities we can use that to tap into provider
>specific or advanced configuration.  For the http close proposal you
>could just add haproxy.httpClose=true on the network offering and the
>haproxy impl can respect that and use it.  So implementations can
>document what additional key/value parameters they support to access
>advanced features.
>
>Review 13896 adds an otherOptions field to the service offering.
>Conceptually that is fine in my mind, but I just don't understand why
>the *_details tables can't be used instead.  Just like XenServer has
>other_config on all types, I'd like it if ACS had a similar thing
>(which I kinda, sorta think it does?).  So we should be able to
>arbitrary data onto any type.  Can somebody comment on how this idea
>may relate with the existing implementation of "details" and "tags" in
>ACS?
>
>Darren
>
>On Thu, Oct 3, 2013 at 10:21 AM, Daan Hoogland 
>wrote:
>> Murali,
>>
>> What our admins need is the ability to instantiate an abstract thing
>>called
>> a virtual redundant high available router. They need be able to tune it
>> with all sorts of features is such a way that other routers redundant or
>> not and high availible or not, may but do not have to have the same
>>tuning
>> parameters. This 'feature' of httpClose is just one of the many things
>>they
>> need to be able to tune. Others include a more fine grained control over
>> the iptables/firewall rules and monitoring of the functionality/daemons
>> running on the machines. I am not biased as to the way how to
>>do/implement
>> this control. The networkoffering seemed like the way to do it.
>>
>> Having said that I didn't think that httpClose would be considered
>>haproxy
>> specific. It seems worrying that someone could device a proxy or a
>> loadbalancer that does not support either one of the features of
>> maximizing - or pooling connections. I'm not into that market recently
>>so I
>> might be mistaking. This maybe besides the point but it discomforts me
>> somewhat.
>>
>> regards,
>> Daan
>>
>>
>> On Thu, Oct 3, 2013 at 2:22 PM, murali reddy
>>wrote:
>>
>>> On Thu, Oct 3, 2013 at 4:18 PM, Daan Hoogland >> >wrote:
>>>
>>> > Chiradeep,
>>> >
>>> > I have been thinking about your concern and there is something
>>>bothering
>>> me
>>> > about it. The issue CLOUDSTACK-4328 of which
>>> > https://reviews.apache.org/r/14320/ is an implementation. This issue
>>>has
>>> > been brought up by the engineers at Schuberg Philis. These are cloud
>>> > operators and domain admins. So these are the people that need to be
>>> > bothered by this tuning and they are certainly haproxy and xen
>>>experts to
>>> > an extend. For them not being able to use connection caching was a
>>>bug.
>>> And
>>> > the proposed solution still seems reasonable to me. It is exposing
>>>the
>>> > abstract feature only to those instantiating a vpc, isn't it? This
>>>last
>>> > question probably touches on the reason you are addressing my
>>>submission
>>> > together with the ones from Nicolas  I see only people instantiating
>>>a
>>> vpc
>>> > having to be bothered by which netoffering to use (with respect to
>>> > httpClose functionality) If this is not the case how should I isolate
>>> this
>>> > concern from other , more mondain users?
>>> >
>>> > btw I did not create an FS page as the issue was created as a jira
>>> ticket.
>>> > I am willing to do that in hind sight but want to have a path to
>>>follow
>>> > first.
>>> >
>>> > regards,
>>> > Daan
>>> >
>>> > On Tue, Oct 1, 2013 at 11:06 PM, Chiradeep Vittal <
>>> > chiradeep.vit...@citrix.com> wrote:
>>> > >
>>> > > We have a couple of people trying to expose the advanced
>>>capabilities
>>> of
>>> > the underlying physical resources to the end-user. In the case of
>>>Nicolas
>>> > FOATA, he is trying to expose some of the advanced functions of
>>> > XenServer/XCP, and in the case of Daan, he is trying to expose some
>>> feature
>>>

Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]

[Darren Shepherd]

We are always going to run into the issue where they're implementation
specific features that can't be represented in a generic way in the
API.  First class attributes in the API need to essentially be the
lowest common denominator.  ACS already has this pattern of *_details
tables.  I honestly don't know completely how they are used, but I
would hope that pattern could be used to address these types of
issues.  By allowing the admin to add arbitrary key/value pairs to the
various offerings/entities we can use that to tap into provider
specific or advanced configuration.  For the http close proposal you
could just add haproxy.httpClose=true on the network offering and the
haproxy impl can respect that and use it.  So implementations can
document what additional key/value parameters they support to access
advanced features.

Review 13896 adds an otherOptions field to the service offering.
Conceptually that is fine in my mind, but I just don't understand why
the *_details tables can't be used instead.  Just like XenServer has
other_config on all types, I'd like it if ACS had a similar thing
(which I kinda, sorta think it does?).  So we should be able to
arbitrary data onto any type.  Can somebody comment on how this idea
may relate with the existing implementation of "details" and "tags" in
ACS?

Darren

On Thu, Oct 3, 2013 at 10:21 AM, Daan Hoogland  wrote:
> Murali,
>
> What our admins need is the ability to instantiate an abstract thing called
> a virtual redundant high available router. They need be able to tune it
> with all sorts of features is such a way that other routers redundant or
> not and high availible or not, may but do not have to have the same tuning
> parameters. This 'feature' of httpClose is just one of the many things they
> need to be able to tune. Others include a more fine grained control over
> the iptables/firewall rules and monitoring of the functionality/daemons
> running on the machines. I am not biased as to the way how to do/implement
> this control. The networkoffering seemed like the way to do it.
>
> Having said that I didn't think that httpClose would be considered haproxy
> specific. It seems worrying that someone could device a proxy or a
> loadbalancer that does not support either one of the features of
> maximizing - or pooling connections. I'm not into that market recently so I
> might be mistaking. This maybe besides the point but it discomforts me
> somewhat.
>
> regards,
> Daan
>
>
> On Thu, Oct 3, 2013 at 2:22 PM, murali reddy wrote:
>
>> On Thu, Oct 3, 2013 at 4:18 PM, Daan Hoogland > >wrote:
>>
>> > Chiradeep,
>> >
>> > I have been thinking about your concern and there is something bothering
>> me
>> > about it. The issue CLOUDSTACK-4328 of which
>> > https://reviews.apache.org/r/14320/ is an implementation. This issue has
>> > been brought up by the engineers at Schuberg Philis. These are cloud
>> > operators and domain admins. So these are the people that need to be
>> > bothered by this tuning and they are certainly haproxy and xen experts to
>> > an extend. For them not being able to use connection caching was a bug.
>> And
>> > the proposed solution still seems reasonable to me. It is exposing the
>> > abstract feature only to those instantiating a vpc, isn't it? This last
>> > question probably touches on the reason you are addressing my submission
>> > together with the ones from Nicolas  I see only people instantiating a
>> vpc
>> > having to be bothered by which netoffering to use (with respect to
>> > httpClose functionality) If this is not the case how should I isolate
>> this
>> > concern from other , more mondain users?
>> >
>> > btw I did not create an FS page as the issue was created as a jira
>> ticket.
>> > I am willing to do that in hind sight but want to have a path to follow
>> > first.
>> >
>> > regards,
>> > Daan
>> >
>> > On Tue, Oct 1, 2013 at 11:06 PM, Chiradeep Vittal <
>> > chiradeep.vit...@citrix.com> wrote:
>> > >
>> > > We have a couple of people trying to expose the advanced capabilities
>> of
>> > the underlying physical resources to the end-user. In the case of Nicolas
>> > FOATA, he is trying to expose some of the advanced functions of
>> > XenServer/XCP, and in the case of Daan, he is trying to expose some
>> feature
>> > of HAProxy.
>> > >
>> > > Users are ideally abstracted from these details and shouldn't have to
>> > wonder which offering to choose [because they are not Xen/HAProxy
>> experts].
>> > > After all one of the goals of CS is to hide these messy details and let
>> > users focus on their apps.
>> > >
>> > > Is there a possibility of a generic way of leaking abstractions for
>> > sufficiently advanced users?
>> > >
>> >
>>
>> Generally as a principle core API's abstract and expose functionality that
>> is commonly available across the hypervisors and network service providers
>> etc.. I believe we should continue to enforce it.
>>
>> But we do have a pattern of API's configure* (configurevirtualrouter,
>> configure

[DOCS] CS-docs patch

[Travis Graham]

Here is a patch for the "Deployment Architecture Overview" section until things 
get setup in Review Board for the new repo or there's a GitHub mirror that pull 
requests can be sent to for review. Will the github mirror and PR route be 
supported in tandem with the Review Board process or not at all?

It adds in links to other docs it only loosely referenced by name and adds the 
Primary Storage info about being zone wide for 4.2.

Travis

Re: [URGENT] Mounting issues from KVM hosts to secondary storage

[Indra Pramana]

Hi Marcus,

It happens randomly, mounting fails, and then / gets filled. I suspect this
occurs when the mounting fails, any files which is supposed to be
transferred to the mount point (e.g. templates used to create VMs) will
fill-up the / partition of the KVM host instead. This doesn't happen in
4.1.1, this only happens after we upgraded to 4.2.0.

And how about virsh pool-list result which is not reflecting the actual IDs
of the secondary storage we see from Cloudstack GUI, is it supposed to be
the case? If not, how to rectify it?

Looking forward to your reply, thank you.

Cheers.




On Fri, Oct 4, 2013 at 1:51 AM, Marcus Sorensen  wrote:

> It sort of looks like the out of space triggered the issue. Libvirt shows
>
> 2013-10-03 15:38:57.414+: 2710: error : virCommandWait:2348 :
> internal error Child process (/bin/umount
> /mnt/5230667e-9c58-3ff6-983c-5fc2a72df669) unexpected exit status 32:
> error writing /etc/mtab.tmp: No space left on device
>
> So that entry of the filesystem being mounted is orphaned in
> /etc/mtab, since it can't be removed. That seems to be the source of
> why 'df' shows the thing mounted when it isn't, at least.
>
>
> On Thu, Oct 3, 2013 at 11:45 AM, Indra Pramana  wrote:
> > Hi Marcus and all,
> >
> > I also find some strange and interesting error messages from the libvirt
> > logs:
> >
> > http://pastebin.com/5ByfNpAf
> >
> > Looking forward to your reply, thank you.
> >
> > Cheers.
> >
> >
> >
> > On Fri, Oct 4, 2013 at 1:38 AM, Indra Pramana  wrote:
> >
> >> Hi Marcus,
> >>
> >> Good day to you, and thank you for your e-mail. See my reply inline.
> >>
> >> On Fri, Oct 4, 2013 at 12:29 AM, Marcus Sorensen  >wrote:
> >>
> >>> Are you using the release artifacts, or your own 4.2 build?
> >>>
> >>
> >> [Indra:] We are using the release artifacts from below repo since we are
> >> using Ubuntu:
> >>
> >> deb http://cloudstack.apt-get.eu/ubuntu precise 4.2
> >>
> >>
> >>> When you rebooted the host, did the problem go away or come back the
> >>> same?
> >>
> >>
> >> [Indra:] When I rebooted the host, the problem go away for a while, but
> it
> >> will come back again after some time. It will come randomly at the time
> >> when we need to create a new instance on that host, or start an existing
> >> stopped instance.
> >>
> >>
> >>> You may want to look at 'virsh pool-list' to see if libvirt is
> >>> mounting/registering the secondary storage.
> >>>
> >>
> >> [Indra:] This is the result of the virsh pool-list command:
> >>
> >> root@hv-kvm-02:/var/log/libvirt# virsh pool-list
> >> Name State  Autostart
> >> -
> >> 301071ac-4c1d-4eac-855b-124126da0a38 active no
> >> 5230667e-9c58-3ff6-983c-5fc2a72df669 active no
> >> d433809b-01ea-3947-ba0f-48077244e4d6 active no
> >>
> >> Strange thing is that none of my secondary storage IDs are there. Could
> it
> >> be that the ID might have changed during Cloudstack upgrade? Here is the
> >> list of my secondary storage (there are two of them) even though they
> are
> >> on the same NFS server:
> >>   c02da448-b9f4-401b-b8d5-83e8ead5cfde nfs://
> >> 10.237.11.31/mnt/vol1/sec-storage NFS
> >> 5937edb6-2e95-4ae2-907b-80fe4599ed87 nfs://
> >> 10.237.11.31/mnt/vol1/sec-storage2 NFS
> >>
> >>> Is this happening on multiple hosts, the same way?
> >>
> >>
> >> [Indra:] Yes, it is happening on all the two hosts that I have, the same
> >> way.
> >>
> >>
> >>> You may want to
> >>> look at /etc/mtab, if the system reports it's mounted, though it's
> >>> not, it might be in there. Look at /proc/mounts as well.
> >>>
> >>
> >> [Indra:] Please find result of df, /etc/mtab and /proc/mounts below. The
> >> "ghost" mount point is on df and /etc/mtab, but not on /proc/mounts.
> >>
> >> root@hv-kvm-02:/etc# df
> >>
> >> Filesystem  1K-blocks
>  Used
> >> Available Use% Mounted on
> >> /dev/sda1 4195924
> >> 4192372 0 100% /
> >>
> >> udev132053356
> 4
> >> 132053352   1% /dev
> >> tmpfs52825052
> 704
> >> 52824348   1% /run
> >>
> >> none 5120
> >> 0  5120   0% /run/lock
> >> none132062620
> 0
> >> 132062620   0% /run/shm
> >> cgroup  132062620
> 0
> >> 132062620   0% /sys/fs/cgroup
> >> /dev/sda610650544
> >> 2500460   7609056  25% /home
> >> 10.237.11.31:/mnt/vol1/sec-storage2/template/tmpl/2/288   4195924
> >> 4192372 0 100% /mnt/5230667e-9c58-3ff6-983c-5fc2a72df669
> >>
> >> root@hv-kvm-02:/etc# cat /etc/mtab
> >> /dev/sda1 / ext4 rw,errors=remount-ro 0 0
> >> proc /proc proc rw,noexec,nosuid,nodev 0 0
> >> sysfs /sys sysfs rw,noexec,nosuid,nodev 0 0
> >> none /sys/fs/f

Re: [URGENT] Mounting issues from KVM hosts to secondary storage

[Marcus Sorensen]

It sort of looks like the out of space triggered the issue. Libvirt shows

2013-10-03 15:38:57.414+: 2710: error : virCommandWait:2348 :
internal error Child process (/bin/umount
/mnt/5230667e-9c58-3ff6-983c-5fc2a72df669) unexpected exit status 32:
error writing /etc/mtab.tmp: No space left on device

So that entry of the filesystem being mounted is orphaned in
/etc/mtab, since it can't be removed. That seems to be the source of
why 'df' shows the thing mounted when it isn't, at least.


On Thu, Oct 3, 2013 at 11:45 AM, Indra Pramana  wrote:
> Hi Marcus and all,
>
> I also find some strange and interesting error messages from the libvirt
> logs:
>
> http://pastebin.com/5ByfNpAf
>
> Looking forward to your reply, thank you.
>
> Cheers.
>
>
>
> On Fri, Oct 4, 2013 at 1:38 AM, Indra Pramana  wrote:
>
>> Hi Marcus,
>>
>> Good day to you, and thank you for your e-mail. See my reply inline.
>>
>> On Fri, Oct 4, 2013 at 12:29 AM, Marcus Sorensen wrote:
>>
>>> Are you using the release artifacts, or your own 4.2 build?
>>>
>>
>> [Indra:] We are using the release artifacts from below repo since we are
>> using Ubuntu:
>>
>> deb http://cloudstack.apt-get.eu/ubuntu precise 4.2
>>
>>
>>> When you rebooted the host, did the problem go away or come back the
>>> same?
>>
>>
>> [Indra:] When I rebooted the host, the problem go away for a while, but it
>> will come back again after some time. It will come randomly at the time
>> when we need to create a new instance on that host, or start an existing
>> stopped instance.
>>
>>
>>> You may want to look at 'virsh pool-list' to see if libvirt is
>>> mounting/registering the secondary storage.
>>>
>>
>> [Indra:] This is the result of the virsh pool-list command:
>>
>> root@hv-kvm-02:/var/log/libvirt# virsh pool-list
>> Name State  Autostart
>> -
>> 301071ac-4c1d-4eac-855b-124126da0a38 active no
>> 5230667e-9c58-3ff6-983c-5fc2a72df669 active no
>> d433809b-01ea-3947-ba0f-48077244e4d6 active no
>>
>> Strange thing is that none of my secondary storage IDs are there. Could it
>> be that the ID might have changed during Cloudstack upgrade? Here is the
>> list of my secondary storage (there are two of them) even though they are
>> on the same NFS server:
>>   c02da448-b9f4-401b-b8d5-83e8ead5cfde nfs://
>> 10.237.11.31/mnt/vol1/sec-storage NFS
>> 5937edb6-2e95-4ae2-907b-80fe4599ed87 nfs://
>> 10.237.11.31/mnt/vol1/sec-storage2 NFS
>>
>>> Is this happening on multiple hosts, the same way?
>>
>>
>> [Indra:] Yes, it is happening on all the two hosts that I have, the same
>> way.
>>
>>
>>> You may want to
>>> look at /etc/mtab, if the system reports it's mounted, though it's
>>> not, it might be in there. Look at /proc/mounts as well.
>>>
>>
>> [Indra:] Please find result of df, /etc/mtab and /proc/mounts below. The
>> "ghost" mount point is on df and /etc/mtab, but not on /proc/mounts.
>>
>> root@hv-kvm-02:/etc# df
>>
>> Filesystem  1K-blocksUsed
>> Available Use% Mounted on
>> /dev/sda1 4195924
>> 4192372 0 100% /
>>
>> udev132053356   4
>> 132053352   1% /dev
>> tmpfs52825052 704
>> 52824348   1% /run
>>
>> none 5120
>> 0  5120   0% /run/lock
>> none132062620   0
>> 132062620   0% /run/shm
>> cgroup  132062620   0
>> 132062620   0% /sys/fs/cgroup
>> /dev/sda610650544
>> 2500460   7609056  25% /home
>> 10.237.11.31:/mnt/vol1/sec-storage2/template/tmpl/2/288   4195924
>> 4192372 0 100% /mnt/5230667e-9c58-3ff6-983c-5fc2a72df669
>>
>> root@hv-kvm-02:/etc# cat /etc/mtab
>> /dev/sda1 / ext4 rw,errors=remount-ro 0 0
>> proc /proc proc rw,noexec,nosuid,nodev 0 0
>> sysfs /sys sysfs rw,noexec,nosuid,nodev 0 0
>> none /sys/fs/fuse/connections fusectl rw 0 0
>> none /sys/kernel/debug debugfs rw 0 0
>> none /sys/kernel/security securityfs rw 0 0
>> udev /dev devtmpfs rw,mode=0755 0 0
>> devpts /dev/pts devpts rw,noexec,nosuid,gid=5,mode=0620 0 0
>> tmpfs /run tmpfs rw,noexec,nosuid,size=10%,mode=0755 0 0
>> none /run/lock tmpfs rw,noexec,nosuid,nodev,size=5242880 0 0
>> none /run/shm tmpfs rw,nosuid,nodev 0 0
>> cgroup /sys/fs/cgroup tmpfs rw,relatime,mode=755 0 0
>> cgroup /sys/fs/cgroup/cpuset cgroup rw,relatime,cpuset 0 0
>> cgroup /sys/fs/cgroup/cpu cgroup rw,relatime,cpu 0 0
>> cgroup /sys/fs/cgroup/cpuacct cgroup rw,relatime,cpuacct 0 0
>> cgroup /sys/fs/cgroup/memory cgroup rw,relatime,memory 0 0
>> cgroup /sys/fs/cgroup/devices cgroup rw,relatime,devices 0 0
>> cgroup /sys/fs/cgroup/freezer cgroup rw,relatime,freezer 0 0
>> cgroup /sys/fs/cgroup/blkio cgroup rw,relatime,blkio 0 0
>>

Re: Wiki access

[Chip Childers]

On Thu, Oct 03, 2013 at 05:47:15PM +, Chiradeep Vittal wrote:
> Daan, it is at the 'Browse' drop down at the upper right. Heh

Yeah - not sure what UX genius decided to put it there...  ;-)

Re: [DISCUSS] Breaking out Marvin from CloudStack

[Chip Childers]

On Thu, Oct 03, 2013 at 05:27:27PM +, Santhosh Edukulla wrote:
> >The problem with tests being broken out from the main repo is that we
> > end up having a really hard time confirming that new features include
> > appropriate tests.  Plus, tests are tied to features, which are tied to
> > the code.  They should be versioned together.
> 
> I believe this makes more sense for unit tests though, where tests need to be 
> there with the code and both go hand in hand with release. 
> 
> As mentioned earlier, keeping or separating automation code\tests with the 
> main product will not make much of an impact as a whole. 

Right, it makes absolutely 100% sense for unit tests (in fact, we would
be idiots to break them out).

My point still stands.  Integration tests should be part of building a
feature.  No tests, no commit to master.  I don't want to have to cross
reference between two different repos, when a simple feature branch diff
(with an up-to-date feature branch) will make it clear that a merge
includes good changes *and* integration tests.

-chip

XenServer Question

[Mike Tutkowski]

Hi,

On one of my SRs, I see the following information:

100.2 GB used of 200 GB total (100 GB allocated)

Can someone explain how 'used' can be larger than 'allocated'?

On storage systems I've worked on before, 'allocated' is generally
synonymous with 'reserved' or 'provisioned'. On these systems, 'used' is
always less than or equal to 'allocated'. Typically 'used' is less than
'allocated' because as 'used' approaches 'allocated', 'allocated' is
expanded (until it runs into 'total').

Thanks!

-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

Re: Wiki access

[Chip Childers]

On Thu, Oct 03, 2013 at 07:33:14PM +0200, Daan Hoogland wrote:
> I'm not seeing any links for editting permissions either. id == dahn

You have space admin rights...

Do this:

1) go to any page in the space (this will do:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Home )

2) click on Browse (upper right hand corner)

3) click on Space Admin in the Browse dropdown (should be the last in
first part of the menu list, right before the )

4) Click on Security / Permissions in the left hand menu

5) Enter your creds again to elevate permissions

6) From there, you'll see the table of groups and individual users.
Click the "Edit Permissions" button at the *bottom* of the individual
users table.

7) Have fun editing!

Re: Wiki access

[Chiradeep Vittal]

Daan, it is at the 'Browse' drop down at the upper right. Heh


On 10/3/13 10:33 AM, "Daan Hoogland"  wrote:

>I'm not seeing any links for editting permissions either. id == dahn
>
>
>On Thu, Oct 3, 2013 at 7:26 PM, Ian Duffy  wrote:
>
>> Hi,
>>
>> May I get access to edit/add pages as well?
>> My username is imduffy15.
>>
>> Thanks,
>> Ian
>>
>> On 3 October 2013 16:13, Chip Childers 
>>wrote:
>> > On Thu, Oct 03, 2013 at 09:58:03AM +, Bharat Kumar wrote:
>> >> Hi All,
>> >> any one please grant me the edit or add page permission to cwiki.
>> my user name is bharat.kumar
>> >
>> > Done
>>

Re: [URGENT] Mounting issues from KVM hosts to secondary storage

[Indra Pramana]

Hi Marcus and all,

I also find some strange and interesting error messages from the libvirt
logs:

http://pastebin.com/5ByfNpAf

Looking forward to your reply, thank you.

Cheers.



On Fri, Oct 4, 2013 at 1:38 AM, Indra Pramana  wrote:

> Hi Marcus,
>
> Good day to you, and thank you for your e-mail. See my reply inline.
>
> On Fri, Oct 4, 2013 at 12:29 AM, Marcus Sorensen wrote:
>
>> Are you using the release artifacts, or your own 4.2 build?
>>
>
> [Indra:] We are using the release artifacts from below repo since we are
> using Ubuntu:
>
> deb http://cloudstack.apt-get.eu/ubuntu precise 4.2
>
>
>> When you rebooted the host, did the problem go away or come back the
>> same?
>
>
> [Indra:] When I rebooted the host, the problem go away for a while, but it
> will come back again after some time. It will come randomly at the time
> when we need to create a new instance on that host, or start an existing
> stopped instance.
>
>
>> You may want to look at 'virsh pool-list' to see if libvirt is
>> mounting/registering the secondary storage.
>>
>
> [Indra:] This is the result of the virsh pool-list command:
>
> root@hv-kvm-02:/var/log/libvirt# virsh pool-list
> Name State  Autostart
> -
> 301071ac-4c1d-4eac-855b-124126da0a38 active no
> 5230667e-9c58-3ff6-983c-5fc2a72df669 active no
> d433809b-01ea-3947-ba0f-48077244e4d6 active no
>
> Strange thing is that none of my secondary storage IDs are there. Could it
> be that the ID might have changed during Cloudstack upgrade? Here is the
> list of my secondary storage (there are two of them) even though they are
> on the same NFS server:
>   c02da448-b9f4-401b-b8d5-83e8ead5cfde nfs://
> 10.237.11.31/mnt/vol1/sec-storage NFS
> 5937edb6-2e95-4ae2-907b-80fe4599ed87 nfs://
> 10.237.11.31/mnt/vol1/sec-storage2 NFS
>
>> Is this happening on multiple hosts, the same way?
>
>
> [Indra:] Yes, it is happening on all the two hosts that I have, the same
> way.
>
>
>> You may want to
>> look at /etc/mtab, if the system reports it's mounted, though it's
>> not, it might be in there. Look at /proc/mounts as well.
>>
>
> [Indra:] Please find result of df, /etc/mtab and /proc/mounts below. The
> "ghost" mount point is on df and /etc/mtab, but not on /proc/mounts.
>
> root@hv-kvm-02:/etc# df
>
> Filesystem  1K-blocksUsed
> Available Use% Mounted on
> /dev/sda1 4195924
> 4192372 0 100% /
>
> udev132053356   4
> 132053352   1% /dev
> tmpfs52825052 704
> 52824348   1% /run
>
> none 5120
> 0  5120   0% /run/lock
> none132062620   0
> 132062620   0% /run/shm
> cgroup  132062620   0
> 132062620   0% /sys/fs/cgroup
> /dev/sda610650544
> 2500460   7609056  25% /home
> 10.237.11.31:/mnt/vol1/sec-storage2/template/tmpl/2/288   4195924
> 4192372 0 100% /mnt/5230667e-9c58-3ff6-983c-5fc2a72df669
>
> root@hv-kvm-02:/etc# cat /etc/mtab
> /dev/sda1 / ext4 rw,errors=remount-ro 0 0
> proc /proc proc rw,noexec,nosuid,nodev 0 0
> sysfs /sys sysfs rw,noexec,nosuid,nodev 0 0
> none /sys/fs/fuse/connections fusectl rw 0 0
> none /sys/kernel/debug debugfs rw 0 0
> none /sys/kernel/security securityfs rw 0 0
> udev /dev devtmpfs rw,mode=0755 0 0
> devpts /dev/pts devpts rw,noexec,nosuid,gid=5,mode=0620 0 0
> tmpfs /run tmpfs rw,noexec,nosuid,size=10%,mode=0755 0 0
> none /run/lock tmpfs rw,noexec,nosuid,nodev,size=5242880 0 0
> none /run/shm tmpfs rw,nosuid,nodev 0 0
> cgroup /sys/fs/cgroup tmpfs rw,relatime,mode=755 0 0
> cgroup /sys/fs/cgroup/cpuset cgroup rw,relatime,cpuset 0 0
> cgroup /sys/fs/cgroup/cpu cgroup rw,relatime,cpu 0 0
> cgroup /sys/fs/cgroup/cpuacct cgroup rw,relatime,cpuacct 0 0
> cgroup /sys/fs/cgroup/memory cgroup rw,relatime,memory 0 0
> cgroup /sys/fs/cgroup/devices cgroup rw,relatime,devices 0 0
> cgroup /sys/fs/cgroup/freezer cgroup rw,relatime,freezer 0 0
> cgroup /sys/fs/cgroup/blkio cgroup rw,relatime,blkio 0 0
> cgroup /sys/fs/cgroup/perf_event cgroup rw,relatime,perf_event 0 0
> /dev/sda6 /home ext4 rw 0 0
> rpc_pipefs /run/rpc_pipefs rpc_pipefs rw 0 0
> binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,noexec,nosuid,nodev 0 0
> 10.237.11.31:/mnt/vol1/sec-storage2/template/tmpl/2/288
> /mnt/5230667e-9c58-3ff6-983c-5fc2a72df669 nfs rw,addr=10.237.11.31 0 0
>
> rootfs / rootfs rw 0 0
> sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
> proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
> udev /dev devtmpfs rw,relatime,size=132053356k,nr_inodes=33013339,mode=755
> 0 0
> devpts /dev/pts devpts
> rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
> tmpfs /run tmpfs rw,

Re: Wiki access

[Chip Childers]

On Thu, Oct 03, 2013 at 06:26:19PM +0100, Ian Duffy wrote:
> Hi,
> 
> May I get access to edit/add pages as well?
> My username is imduffy15.

Added

Re: [URGENT] Mounting issues from KVM hosts to secondary storage

[Indra Pramana]

Hi Dean,

Good day to you, and thank you for your e-mail.

The current setting (value) of storage.overprovisioning.factor is 2. Do you
think it matters? I can't remember whether the value was the default value,
or is it something that we changed earlier. Do you think changing this to 1
will help?

Looking forward to your reply, thank you.

Cheers.



On Fri, Oct 4, 2013 at 12:24 AM, Dean Kamali  wrote:

> have you looked at storage over-provisioning settings ??
>
>
> On Thu, Oct 3, 2013 at 11:53 AM, Indra Pramana  wrote:
>
> > Dear all,
> >
> > We face a major problem after upgrading to 4.2.0. Mounting from KVM hosts
> > to secondary storage seems to fail, every time a new VM instance is
> > created, it will use up the / (root) partition of the KVM hosts instead.
> >
> > Here is the df result:
> >
> > 
> > root@hv-kvm-02:/home/indra# df
> > Filesystem  1K-blocksUsed
> > Available Use% Mounted on
> > /dev/sda1 4195924
> > 4195924 0 100% /
> > udev132053356   4
> > 132053352   1% /dev
> > tmpfs52825052 440
> > 52824612   1% /run
> > none 5120
> > 0  5120   0% /run/lock
> > none132062620   0
> > 132062620   0% /run/shm
> > cgroup  132062620   0
> > 132062620   0% /sys/fs/cgroup
> > /dev/sda610650544 2500424
> > 7609092  25% /home
> > 10.237.11.31:/mnt/vol1/sec-storage2/template/tmpl/2/288   4195924
> > 4195924 0 100% /mnt/5230667e-9c58-3ff6-983c-5fc2a72df669
> > 
> >
> > The strange thing is that df shows that it seems to be mounted, but it's
> > actually not mounted. If you noticed, the total capacity of the mount
> point
> > is exactly the same as the capacity of the / (root) partition. By right
> it
> > should show 7 TB instead of just 4 GB.
> >
> > This caused VM creation to be error due to out of disk space. This also
> > affect the KVM operations since the / (root) partition becomes full, and
> we
> > can only release the space after we reboot the KVM host.
> >
> > Anyone experience this problem before? We are at loss on how to resolve
> the
> > problem.
> >
> > Looking forward to your reply, thank you.
> >
> > Cheers.
> >
>

Re: A question on vm migrations when hosts are set into a maintenance mode.

[Chip Childers]

On Thu, Oct 03, 2013 at 11:44:46AM -0500, Alex Ough wrote:
> I'm not sure what else commands 'MigrateCommand' actually execute in
> addition to 'Start/Stop/CopyCommand', but can we include 'MigrateCommand'
> if it consists of only those 3 commands?
> 
> Thanks
> Alex Ough

In the case of VMware, the migrate command is executed via the
MigrateVMTask that's part of the VMware SDK (see
vmware-base/src/com/cloud/hypervisor/vmware/mo/VirtualMachineMO.java).

For VMware, I know that vCenter will queue and process concurrent
requests for migrations.  Specifically, it will throttle the migrations
happening, based on it's internal concurrency constraints, but the task
queue will still accept more connections.  Obviously the risk are the
VMware layer tasks timing out if it takes too long for the task queue to
complete.

As for XenServer, it's happening in what appears to be a similar way
(although the source host is the target for the migration API call).

Check
plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java.

I'm not familiar enough with XenServer's concurrency model for
migrations.  Any experts know the answer to if it can handle concurrency
in a stable way?

With KVM, it's obviously executing via the agent.  Similarly to
XenServer, I'm not familiar enough to know about concurrent operations.

So do the HV experts on the list have any opinions about XenServer and
KVM migration concurrency?

-chip

Re: [URGENT] Mounting issues from KVM hosts to secondary storage

[Indra Pramana]

Hi Marcus,

Good day to you, and thank you for your e-mail. See my reply inline.

On Fri, Oct 4, 2013 at 12:29 AM, Marcus Sorensen wrote:

> Are you using the release artifacts, or your own 4.2 build?
>

[Indra:] We are using the release artifacts from below repo since we are
using Ubuntu:

deb http://cloudstack.apt-get.eu/ubuntu precise 4.2


> When you rebooted the host, did the problem go away or come back the
> same?


[Indra:] When I rebooted the host, the problem go away for a while, but it
will come back again after some time. It will come randomly at the time
when we need to create a new instance on that host, or start an existing
stopped instance.


> You may want to look at 'virsh pool-list' to see if libvirt is
> mounting/registering the secondary storage.
>

[Indra:] This is the result of the virsh pool-list command:

root@hv-kvm-02:/var/log/libvirt# virsh pool-list
Name State  Autostart
-
301071ac-4c1d-4eac-855b-124126da0a38 active no
5230667e-9c58-3ff6-983c-5fc2a72df669 active no
d433809b-01ea-3947-ba0f-48077244e4d6 active no

Strange thing is that none of my secondary storage IDs are there. Could it
be that the ID might have changed during Cloudstack upgrade? Here is the
list of my secondary storage (there are two of them) even though they are
on the same NFS server:
c02da448-b9f4-401b-b8d5-83e8ead5cfdenfs://10.237.11.31/mnt/vol1/sec-storage
NFS 5937edb6-2e95-4ae2-907b-80fe4599ed87nfs://
10.237.11.31/mnt/vol1/sec-storage2NFS

> Is this happening on multiple hosts, the same way?


[Indra:] Yes, it is happening on all the two hosts that I have, the same
way.


> You may want to
> look at /etc/mtab, if the system reports it's mounted, though it's
> not, it might be in there. Look at /proc/mounts as well.
>

[Indra:] Please find result of df, /etc/mtab and /proc/mounts below. The
"ghost" mount point is on df and /etc/mtab, but not on /proc/mounts.

root@hv-kvm-02:/etc# df
Filesystem  1K-blocksUsed
Available Use% Mounted on
/dev/sda1 4195924
4192372 0 100% /
udev132053356   4
132053352   1% /dev
tmpfs52825052 704
52824348   1% /run
none 5120
0  5120   0% /run/lock
none132062620   0
132062620   0% /run/shm
cgroup  132062620   0
132062620   0% /sys/fs/cgroup
/dev/sda610650544 2500460
7609056  25% /home
10.237.11.31:/mnt/vol1/sec-storage2/template/tmpl/2/288   4195924
4192372 0 100% /mnt/5230667e-9c58-3ff6-983c-5fc2a72df669

root@hv-kvm-02:/etc# cat /etc/mtab
/dev/sda1 / ext4 rw,errors=remount-ro 0 0
proc /proc proc rw,noexec,nosuid,nodev 0 0
sysfs /sys sysfs rw,noexec,nosuid,nodev 0 0
none /sys/fs/fuse/connections fusectl rw 0 0
none /sys/kernel/debug debugfs rw 0 0
none /sys/kernel/security securityfs rw 0 0
udev /dev devtmpfs rw,mode=0755 0 0
devpts /dev/pts devpts rw,noexec,nosuid,gid=5,mode=0620 0 0
tmpfs /run tmpfs rw,noexec,nosuid,size=10%,mode=0755 0 0
none /run/lock tmpfs rw,noexec,nosuid,nodev,size=5242880 0 0
none /run/shm tmpfs rw,nosuid,nodev 0 0
cgroup /sys/fs/cgroup tmpfs rw,relatime,mode=755 0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,relatime,cpuset 0 0
cgroup /sys/fs/cgroup/cpu cgroup rw,relatime,cpu 0 0
cgroup /sys/fs/cgroup/cpuacct cgroup rw,relatime,cpuacct 0 0
cgroup /sys/fs/cgroup/memory cgroup rw,relatime,memory 0 0
cgroup /sys/fs/cgroup/devices cgroup rw,relatime,devices 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,relatime,freezer 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,relatime,blkio 0 0
cgroup /sys/fs/cgroup/perf_event cgroup rw,relatime,perf_event 0 0
/dev/sda6 /home ext4 rw 0 0
rpc_pipefs /run/rpc_pipefs rpc_pipefs rw 0 0
binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,noexec,nosuid,nodev 0 0
10.237.11.31:/mnt/vol1/sec-storage2/template/tmpl/2/288
/mnt/5230667e-9c58-3ff6-983c-5fc2a72df669 nfs rw,addr=10.237.11.31 0 0

rootfs / rootfs rw 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
udev /dev devtmpfs rw,relatime,size=132053356k,nr_inodes=33013339,mode=755
0 0
devpts /dev/pts devpts
rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /run tmpfs rw,nosuid,relatime,size=52825052k,mode=755 0 0
/dev/disk/by-uuid/d21f4676-075d-44be-a378-92ef6aaf2496 / ext4
rw,relatime,errors=remount-ro,data=ordered 0 0
none /sys/fs/fuse/connections fusectl rw,relatime 0 0
none /sys/kernel/debug debugfs rw,relatime 0 0
cgroup /sys/fs/cgroup tmpfs rw,relatime,mode=755 0 0
none /sys/kernel/security securityfs rw,relatime 0 0
none /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
none /run/shm tmpfs rw,n

Re: Can't compile cloudstack 4.2 rpms from source

[Wei ZHOU]

I downloaded and built cloudstack 4.2.0 source code just now. It is ok.



[INFO]

[INFO] Building Apache CloudStack Plugin - Hypervisor Xen 4.2.0
[INFO]

[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @
cloud-plugin-hypervisor-xen ---
[INFO]
[INFO] --- maven-remote-resources-plugin:1.3:process (default) @
cloud-plugin-hypervisor-xen ---
[INFO]
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @
cloud-plugin-hypervisor-xen ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/root/apache-cloudstack-4.2.0-src/plugins/hypervisors/xen/src/main/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @
cloud-plugin-hypervisor-xen ---
[INFO] Compiling 19 source files to
/root/apache-cloudstack-4.2.0-src/plugins/hypervisors/xen/target/classes
[INFO]
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources)
@ cloud-plugin-hypervisor-xen ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/root/apache-cloudstack-4.2.0-src/plugins/hypervisors/xen/test/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @
cloud-plugin-hypervisor-xen ---
[INFO] Compiling 1 source file to
/root/apache-cloudstack-4.2.0-src/plugins/hypervisors/xen/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.12:test (default-test) @
cloud-plugin-hypervisor-xen ---
[INFO] Surefire report directory:
/root/apache-cloudstack-4.2.0-src/plugins/hypervisors/xen/target/surefire-reports
---
 T E S T S
---
Running com.cloud.hypervisor.xen.resource.CitrixResourceBaseTest
log4j:WARN No appenders could be found for logger
(com.cloud.hypervisor.xen.resource.XenServerConnectionPool).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.174 sec
Results :
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @
cloud-plugin-hypervisor-xen ---
[INFO] Building jar:
/root/apache-cloudstack-4.2.0-src/plugins/hypervisors/xen/target/cloud-plugin-hypervisor-xen-4.2.0.jar
[INFO]
[INFO] --- maven-site-plugin:3.1:attach-descriptor (attach-descriptor) @
cloud-plugin-hypervisor-xen ---
[INFO]
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @
cloud-plugin-hypervisor-xen ---
[INFO] Installing
/root/apache-cloudstack-4.2.0-src/plugins/hypervisors/xen/target/cloud-plugin-hypervisor-xen-4.2.0.jar
to
/root/.m2/repository/org/apache/cloudstack/cloud-plugin-hypervisor-xen/4.2.0/cloud-plugin-hypervisor-xen-4.2.0.jar
[INFO] Installing
/root/apache-cloudstack-4.2.0-src/plugins/hypervisors/xen/pom.xml to
/root/.m2/repository/org/apache/cloudstack/cloud-plugin-hypervisor-xen/4.2.0/cloud-plugin-hypervisor-xen-4.2.0.pom
[INFO]


2013/10/3 benoit lair 

> Hi Wei,
>
> What is the result expected with "mvn -P developer" ?
>
> It will intent to give me rpms usable for a production environment ?
>
> Thanks for your reponse.
>
> Regards, Benoit.
>
>
> 2013/10/3 Wei ZHOU 
>
> > mvn -P developer,systemvm clean install
> >
>

Re: Wiki access

[Daan Hoogland]

I'm not seeing any links for editting permissions either. id == dahn


On Thu, Oct 3, 2013 at 7:26 PM, Ian Duffy  wrote:

> Hi,
>
> May I get access to edit/add pages as well?
> My username is imduffy15.
>
> Thanks,
> Ian
>
> On 3 October 2013 16:13, Chip Childers  wrote:
> > On Thu, Oct 03, 2013 at 09:58:03AM +, Bharat Kumar wrote:
> >> Hi All,
> >> any one please grant me the edit or add page permission to cwiki.
> my user name is bharat.kumar
> >
> > Done
>

RE: [DISCUSS] Breaking out Marvin from CloudStack

[Santhosh Edukulla]

>The problem with tests being broken out from the main repo is that we
end up having a really hard time confirming that new features include
appropriate tests.  Plus, tests are tied to features, which are tied to
the code.  They should be versioned together.

I believe this makes more sense for unit tests though, where tests need to be 
there with the code and both go hand in hand with release. 

As mentioned earlier, keeping or separating automation code\tests with the main 
product will not make much of an impact as a whole. 

Regards,
Santhosh

From: Chip Childers [chip.child...@sungard.com]
Sent: Thursday, October 03, 2013 11:07 AM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Breaking out Marvin from CloudStack

On Thu, Oct 03, 2013 at 05:55:14AM +, Santhosh Edukulla wrote:
> I believe both can exist independently. The framework should have more 
> flexibility and facilitation to run features\test cases based upon a  given 
> release and version on a given test bed. It should be intelligent enough to 
> run those cases  and only those cases pertaining to that release with few 
> config variables. Keeping aside branch\repo terminology, for me all marvin 
> both framework and tests should be at one place and separate from main 
> product code.

The problem with tests being broken out from the main repo is that we
end up having a really hard time confirming that new features include
appropriate tests.  Plus, tests are tied to features, which are tied to
the code.  They should be versioned together.

As you said though, the framework is the framework...  that's a tool.
Frankly, with auto-discovery going forward, Marvin becomes no different
from any other testing framework software that has to be installed to do
useful things with the code in the main repo.  Ex: maven!

-chip

Re: Wiki access

[Ian Duffy]

Hi,

May I get access to edit/add pages as well?
My username is imduffy15.

Thanks,
Ian

On 3 October 2013 16:13, Chip Childers  wrote:
> On Thu, Oct 03, 2013 at 09:58:03AM +, Bharat Kumar wrote:
>> Hi All,
>> any one please grant me the edit or add page permission to cwiki. my 
>> user name is bharat.kumar
>
> Done

Re: [PROPOSAL] Revert to VM disk Snapshot

[Daan Hoogland]

go go go


On Wed, Oct 2, 2013 at 5:07 PM, David Ortiz  wrote:

> This sounds like a great idea.
>
> > From: chiradeep.vit...@citrix.com
> > To: dev@cloudstack.apache.org
> > Subject: Re: [PROPOSAL] Revert to VM disk Snapshot
> > Date: Mon, 30 Sep 2013 21:53:35 +
> >
> > +1.
> >
> > On 9/30/13 2:31 PM, "SuichII, Christopher" 
> wrote:
> >
> > >The storage subsystem API currently has an interface for takeSnapshot()
> > >and an associated externally facing API for takeSnapshot. There is also
> a
> > >method on the primary data store interface for revertSnapshot().
> However,
> > >this method is unused. We would like this storage subsystem interface
> > >method be hooked up to an externally facing API so that we can provide
> > >true VM disk backup and recovery.
> > >
> > >I will work on formalizing a full functional spec but wanted to get this
> > >up for discussion ASAP.
> > >
> > >I have created a JIRA ticket:
> > >https://issues.apache.org/jira/browse/CLOUDSTACK-4771
> > >
> > >Thanks,
> > >Chris
> > >--
> > >Chris Suich
> > >chris.su...@netapp.com
> > >NetApp Software Engineer
> > >Data Center Platforms ­ Cloud Solutions
> > >Citrix, Cisco & Red Hat
> > >
> >
>
>

Re: Can't compile cloudstack 4.2 rpms from source

[Chip Childers]

On Thu, Oct 03, 2013 at 06:24:57PM +0200, benoit lair wrote:
> Hi Wei,
> 
> What is the result expected with "mvn -P developer" ?
> 
> It will intent to give me rpms usable for a production environment ?

No, it's intended to test that maven builds are functional for you.

If you just want the RPMs, have you considered pulling them down from
our repo?

http://cloudstack.apt-get.eu/rhel/4.2/

> 
> Thanks for your reponse.
> 
> Regards, Benoit.
> 
> 
> 2013/10/3 Wei ZHOU 
> 
> > mvn -P developer,systemvm clean install
> >

Re: [DISCUSS] Leaky abstractions [was review requests 13238, 13896, 14320]

[Daan Hoogland]

Murali,

What our admins need is the ability to instantiate an abstract thing called
a virtual redundant high available router. They need be able to tune it
with all sorts of features is such a way that other routers redundant or
not and high availible or not, may but do not have to have the same tuning
parameters. This 'feature' of httpClose is just one of the many things they
need to be able to tune. Others include a more fine grained control over
the iptables/firewall rules and monitoring of the functionality/daemons
running on the machines. I am not biased as to the way how to do/implement
this control. The networkoffering seemed like the way to do it.

Having said that I didn't think that httpClose would be considered haproxy
specific. It seems worrying that someone could device a proxy or a
loadbalancer that does not support either one of the features of
maximizing - or pooling connections. I'm not into that market recently so I
might be mistaking. This maybe besides the point but it discomforts me
somewhat.

regards,
Daan


On Thu, Oct 3, 2013 at 2:22 PM, murali reddy wrote:

> On Thu, Oct 3, 2013 at 4:18 PM, Daan Hoogland  >wrote:
>
> > Chiradeep,
> >
> > I have been thinking about your concern and there is something bothering
> me
> > about it. The issue CLOUDSTACK-4328 of which
> > https://reviews.apache.org/r/14320/ is an implementation. This issue has
> > been brought up by the engineers at Schuberg Philis. These are cloud
> > operators and domain admins. So these are the people that need to be
> > bothered by this tuning and they are certainly haproxy and xen experts to
> > an extend. For them not being able to use connection caching was a bug.
> And
> > the proposed solution still seems reasonable to me. It is exposing the
> > abstract feature only to those instantiating a vpc, isn't it? This last
> > question probably touches on the reason you are addressing my submission
> > together with the ones from Nicolas  I see only people instantiating a
> vpc
> > having to be bothered by which netoffering to use (with respect to
> > httpClose functionality) If this is not the case how should I isolate
> this
> > concern from other , more mondain users?
> >
> > btw I did not create an FS page as the issue was created as a jira
> ticket.
> > I am willing to do that in hind sight but want to have a path to follow
> > first.
> >
> > regards,
> > Daan
> >
> > On Tue, Oct 1, 2013 at 11:06 PM, Chiradeep Vittal <
> > chiradeep.vit...@citrix.com> wrote:
> > >
> > > We have a couple of people trying to expose the advanced capabilities
> of
> > the underlying physical resources to the end-user. In the case of Nicolas
> > FOATA, he is trying to expose some of the advanced functions of
> > XenServer/XCP, and in the case of Daan, he is trying to expose some
> feature
> > of HAProxy.
> > >
> > > Users are ideally abstracted from these details and shouldn't have to
> > wonder which offering to choose [because they are not Xen/HAProxy
> experts].
> > > After all one of the goals of CS is to hide these messy details and let
> > users focus on their apps.
> > >
> > > Is there a possibility of a generic way of leaking abstractions for
> > sufficiently advanced users?
> > >
> >
>
> Generally as a principle core API's abstract and expose functionality that
> is commonly available across the hypervisors and network service providers
> etc.. I believe we should continue to enforce it.
>
> But we do have a pattern of API's configure* (configurevirtualrouter,
> configurenetscaler etc) where a hypervisor/network element plug-in can let
> admin to configure params specific to plug-in. Perhaps we can use this
> API's fine tune a plugin for advanced configurations. For e.g both HaProxy
> max connections, httpModeEnabled can be parameters that can be configured
> with configureVirtualRouter api.
>
> Daan,
>
> does this approach works for you?
>
> -Murali
>
>
> > > https://reviews.apache.org/r/13238/
> > > https://reviews.apache.org/r/14320/
> > > https://reviews.apache.org/r/13896/
> > >
> > > BTW, I really prefer that these changes are discussed by putting up an
> FS
> > on the wiki rather than submitting patch requests.
> > > If it touches more than a few files, it is probably worth discussing
> with
> > a [DISCUSS] tag line.
> > > Also, it requires tests.
> > >
> > >
> > >
> >
>

Re: unable to get tiny linux template in ready state, DOWNLOAD ERROR

[Min Chen]

Punith,

Your log shows that your ssvm cannot connect back to MS. As Wei
mentioned, your host configuration may not be correct, and need to be
changed to correct Ip. Do you see an entry in your db host table with type
SecondaryStorageVM?

Thanks
-min

On 10/3/13 7:36 AM, "Wei ZHOU"  wrote:

>It is normal.
>You can get more information in vmops.log

Re: A question on vm migrations when hosts are set into a maintenance mode.

[Alex Ough]

I'm not sure what else commands 'MigrateCommand' actually execute in
addition to 'Start/Stop/CopyCommand', but can we include 'MigrateCommand'
if it consists of only those 3 commands?

Thanks
Alex Ough


On Thu, Oct 3, 2013 at 11:31 AM, Alena Prokharchyk <
alena.prokharc...@citrix.com> wrote:

> On 10/3/13 9:25 AM, "Koushik Das"  wrote:
>
> >Alena,
> >I see that you had added this config setting. Any specific reason that
> >this only for Start/Stop/Copy and not for all the commands as Alex
> >mentioned? The name of the setting looks generic.
>
> Because these are the commands for which hypervisors support parallel
> execution for sure (tested by QA on Xen/KVM/VmWare).
>
> As all our commands are generic, and not hypervisor specific, you have to
> be very careful if decide to enable parallel execution for MigrateCommand
> and test it on all the hypervisors CS supports.
>
> -Alena.
>
> >
> >-Koushik
> >
> >On 03-Oct-2013, at 8:10 PM, Alex Ough  wrote:
> >
> >> Hi Koushik,
> >>
> >> Thanks for your reply, but unfortunately, this setting does NOT cover
> >> 'MigrateCommand'.
> >> As you specified, it seems to be effective only in
> >>Start/Stop/CopyCommand.
> >>
> >> So can we include 'MigrateCommand' in that setting?
> >>
> >> Thanks
> >> Alex Ough
> >>
> >>
> >> On Wed, Oct 2, 2013 at 11:44 PM, Koushik Das 
> >>wrote:
> >>
> >>> It is already a global setting in 4.2. The name of the setting is
> >>> "execute.in.sequence.hypervisor.commands".
> >>>
> >>> Check out Config.java
> >>>
> >>>ExecuteInSequence("Advanced", ManagementServer.class, Boolean.class,
> >>> "execute.in.sequence.hypervisor.commands", "true", "If set to true,
> >>> StartCommand, StopCommand, CopyCommand will be synchronized on the
> >>>agent
> >>> side." +
> >>>" If set to false, these commands become asynchronous. Default value
> >>> is true.", null),
> >>>
> >>>
> >>> -Koushik
> >>>
> >>> On 02-Oct-2013, at 10:43 PM, Alex Ough  >>> alex.o...@sungard.com>> wrote:
> >>>
> >>> Thanks for the reply, Marcus.
> >>>
> >>> What about the option #3, which is to make it as a global setting?
> >>> I think it can prevent side effects if exist.
> >>>
> >>> Alex Ough
> >>>
> >>>
> >>> On Wed, Oct 2, 2013 at 11:37 AM, Marcus Sorensen  >>> >wrote:
> >>>
> >>> Not sure. I don't know the history well enough to know if there were
> >>> issues in the past, it might be that some hypervisors were fine and
> >>> others weren't.
> >>>
> >>> On Wed, Oct 2, 2013 at 9:21 AM, Alex Ough
> >>> >>> alex.o...@sungard.com>> wrote:
> >>> Marcus/Kelven,
> >>>
> >>> Any thoughts on my suggestions?
> >>>
> >>> Thanks
> >>> Alex Ough
> >>>
> >>>
> >>> On Tue, Sep 24, 2013 at 12:09 PM, Alex Ough
> >>> >>> alex.o...@sungard.com>>
> >>> wrote:
> >>>
> >>> Oh, sorry for the confusion. I must have reversed the flags.
> >>> As Kelven pointed, it is set as 'TRUE', which makes the process as
> >>> sequential.
> >>>
> >>> So my questions are
> >>> 1. If there is any reason why the method have been defined to return
> >>> 'TRUE' always?
> >>> 2. Do we expect any side effects and/or malfunctioning if we change it
> >>> to
> >>> returning 'FALSE'?
> >>> 3. For a resolution without breaking possible flows, can we add the
> >>> value
> >>> of 'executeInSequence' to the global setting if #2 answers YES?
> >>>
> >>>
> >>> On Tue, Sep 24, 2013 at 11:19 AM, Marcus Sorensen  >>> >
> >>> wrote:
> >>>
> >>> I thought executeInSequence of 'true' made it go serially, or
> >>> sequentially. In my codebase for 4.1,4.2,master it's been 'true' since
> >>> August of 2010:
> >>>
> >>> 2010-08-11 09:13:29 -0700 19) public class MigrateCommand extends
> >>> Command
> >>> {
> >>> 2010-08-11 09:13:29 -0700 20) String vmName;
> >>> 2010-08-11 09:13:29 -0700 21) String destIp;
> >>> 2011-08-10 10:26:04 -0700 22) String hostGuid;
> >>> 2010-08-11 09:13:29 -0700 23) boolean isWindows;
> >>> 2010-08-11 09:13:29 -0700 24)
> >>> 2010-08-11 09:13:29 -0700 25)
> >>> 2010-08-11 09:13:29 -0700 26) protected MigrateCommand() {
> >>> 2010-08-11 09:13:29 -0700 27) }
> >>> 2012-12-03 22:06:41 -0800 28)
> >>> 2010-08-11 09:13:29 -0700 29) public MigrateCommand(String vmName,
> >>> String destIp, boolean isWindows)
> >>> 2010-08-11 09:13:29 -0700 30) this.vmName = vmName;
> >>> 2010-08-11 09:13:29 -0700 31) this.destIp = destIp;
> >>> 2010-08-11 09:13:29 -0700 32) this.isWindows = isWindows;
> >>> 2010-08-11 09:13:29 -0700 33) }
> >>> 2012-12-03 22:06:41 -0800 34)
> >>> 2010-08-11 09:13:29 -0700 35) public boolean isWindows() {
> >>> 2010-08-11 09:13:29 -0700 36) return isWindows;
> >>> 2010-08-11 09:13:29 -0700 37) }
> >>> 2012-12-03 22:06:41 -0800 38)
> >>> 2010-08-11 09:13:29 -0700 39) public String getDestinationIp() {
> >>> 2010-08-11 09:13:29 -0700 40) return destIp;
> >>> 2010-08-11 09:13:29 -0700 41) }
> >>> 2012-12-03 22:06:41 -0800 42)
> >>> 2010-08-11 09:13:29 -0700

Re: Can't compile cloudstack 4.2 rpms from source

[benoit lair]

Hi Wei,

What is the result expected with "mvn -P developer" ?

It will intent to give me rpms usable for a production environment ?

Thanks for your reponse.

Regards, Benoit.


2013/10/3 Wei ZHOU 

> mvn -P developer,systemvm clean install
>

Re: A question on vm migrations when hosts are set into a maintenance mode.

[Alena Prokharchyk]

On 10/3/13 9:25 AM, "Koushik Das"  wrote:

>Alena,
>I see that you had added this config setting. Any specific reason that
>this only for Start/Stop/Copy and not for all the commands as Alex
>mentioned? The name of the setting looks generic.

Because these are the commands for which hypervisors support parallel
execution for sure (tested by QA on Xen/KVM/VmWare).

As all our commands are generic, and not hypervisor specific, you have to
be very careful if decide to enable parallel execution for MigrateCommand
and test it on all the hypervisors CS supports.

-Alena.

>
>-Koushik
>
>On 03-Oct-2013, at 8:10 PM, Alex Ough  wrote:
>
>> Hi Koushik,
>> 
>> Thanks for your reply, but unfortunately, this setting does NOT cover
>> 'MigrateCommand'.
>> As you specified, it seems to be effective only in
>>Start/Stop/CopyCommand.
>> 
>> So can we include 'MigrateCommand' in that setting?
>> 
>> Thanks
>> Alex Ough
>> 
>> 
>> On Wed, Oct 2, 2013 at 11:44 PM, Koushik Das 
>>wrote:
>> 
>>> It is already a global setting in 4.2. The name of the setting is
>>> "execute.in.sequence.hypervisor.commands".
>>> 
>>> Check out Config.java
>>> 
>>>ExecuteInSequence("Advanced", ManagementServer.class, Boolean.class,
>>> "execute.in.sequence.hypervisor.commands", "true", "If set to true,
>>> StartCommand, StopCommand, CopyCommand will be synchronized on the
>>>agent
>>> side." +
>>>" If set to false, these commands become asynchronous. Default value
>>> is true.", null),
>>> 
>>> 
>>> -Koushik
>>> 
>>> On 02-Oct-2013, at 10:43 PM, Alex Ough >> alex.o...@sungard.com>> wrote:
>>> 
>>> Thanks for the reply, Marcus.
>>> 
>>> What about the option #3, which is to make it as a global setting?
>>> I think it can prevent side effects if exist.
>>> 
>>> Alex Ough
>>> 
>>> 
>>> On Wed, Oct 2, 2013 at 11:37 AM, Marcus Sorensen >> >wrote:
>>> 
>>> Not sure. I don't know the history well enough to know if there were
>>> issues in the past, it might be that some hypervisors were fine and
>>> others weren't.
>>> 
>>> On Wed, Oct 2, 2013 at 9:21 AM, Alex Ough
> alex.o...@sungard.com>> wrote:
>>> Marcus/Kelven,
>>> 
>>> Any thoughts on my suggestions?
>>> 
>>> Thanks
>>> Alex Ough
>>> 
>>> 
>>> On Tue, Sep 24, 2013 at 12:09 PM, Alex Ough
> alex.o...@sungard.com>>
>>> wrote:
>>> 
>>> Oh, sorry for the confusion. I must have reversed the flags.
>>> As Kelven pointed, it is set as 'TRUE', which makes the process as
>>> sequential.
>>> 
>>> So my questions are
>>> 1. If there is any reason why the method have been defined to return
>>> 'TRUE' always?
>>> 2. Do we expect any side effects and/or malfunctioning if we change it
>>> to
>>> returning 'FALSE'?
>>> 3. For a resolution without breaking possible flows, can we add the
>>> value
>>> of 'executeInSequence' to the global setting if #2 answers YES?
>>> 
>>> 
>>> On Tue, Sep 24, 2013 at 11:19 AM, Marcus Sorensen >> >
>>> wrote:
>>> 
>>> I thought executeInSequence of 'true' made it go serially, or
>>> sequentially. In my codebase for 4.1,4.2,master it's been 'true' since
>>> August of 2010:
>>> 
>>> 2010-08-11 09:13:29 -0700 19) public class MigrateCommand extends
>>> Command
>>> {
>>> 2010-08-11 09:13:29 -0700 20) String vmName;
>>> 2010-08-11 09:13:29 -0700 21) String destIp;
>>> 2011-08-10 10:26:04 -0700 22) String hostGuid;
>>> 2010-08-11 09:13:29 -0700 23) boolean isWindows;
>>> 2010-08-11 09:13:29 -0700 24)
>>> 2010-08-11 09:13:29 -0700 25)
>>> 2010-08-11 09:13:29 -0700 26) protected MigrateCommand() {
>>> 2010-08-11 09:13:29 -0700 27) }
>>> 2012-12-03 22:06:41 -0800 28)
>>> 2010-08-11 09:13:29 -0700 29) public MigrateCommand(String vmName,
>>> String destIp, boolean isWindows)
>>> 2010-08-11 09:13:29 -0700 30) this.vmName = vmName;
>>> 2010-08-11 09:13:29 -0700 31) this.destIp = destIp;
>>> 2010-08-11 09:13:29 -0700 32) this.isWindows = isWindows;
>>> 2010-08-11 09:13:29 -0700 33) }
>>> 2012-12-03 22:06:41 -0800 34)
>>> 2010-08-11 09:13:29 -0700 35) public boolean isWindows() {
>>> 2010-08-11 09:13:29 -0700 36) return isWindows;
>>> 2010-08-11 09:13:29 -0700 37) }
>>> 2012-12-03 22:06:41 -0800 38)
>>> 2010-08-11 09:13:29 -0700 39) public String getDestinationIp() {
>>> 2010-08-11 09:13:29 -0700 40) return destIp;
>>> 2010-08-11 09:13:29 -0700 41) }
>>> 2012-12-03 22:06:41 -0800 42)
>>> 2010-08-11 09:13:29 -0700 43) public String getVmName() {
>>> 2010-08-11 09:13:29 -0700 44) return vmName;
>>> 2010-08-11 09:13:29 -0700 45) }
>>> 2012-12-03 22:06:41 -0800 46)
>>> 2011-08-10 10:26:04 -0700 47) public void setHostGuid(String guid)
>>> {
>>> 2011-08-10 10:26:04 -0700 48) this.hostGuid = guid;
>>> 2011-08-10 10:26:04 -0700 49) }
>>> 2012-12-03 22:06:41 -0800 50)
>>> 2011-08-10 10:26:04 -0700 51) public String getHostGuid() {
>>> 2011-08-10 10:26:04 -0700 52) return this.hostGuid;
>>> 2011-08-10 10:2

Re: [URGENT] Mounting issues from KVM hosts to secondary storage

[Marcus Sorensen]

Are you using the release artifacts, or your own 4.2 build?

When you rebooted the host, did the problem go away or come back the
same? You may want to look at 'virsh pool-list' to see if libvirt is
mounting/registering the secondary storage.

Is this happening on multiple hosts, the same way? You may want to
look at /etc/mtab, if the system reports it's mounted, though it's
not, it might be in there. Look at /proc/mounts as well.

On Thu, Oct 3, 2013 at 9:53 AM, Indra Pramana  wrote:
> Dear all,
>
> We face a major problem after upgrading to 4.2.0. Mounting from KVM hosts
> to secondary storage seems to fail, every time a new VM instance is
> created, it will use up the / (root) partition of the KVM hosts instead.
>
> Here is the df result:
>
> 
> root@hv-kvm-02:/home/indra# df
> Filesystem  1K-blocksUsed
> Available Use% Mounted on
> /dev/sda1 4195924
> 4195924 0 100% /
> udev132053356   4
> 132053352   1% /dev
> tmpfs52825052 440
> 52824612   1% /run
> none 5120
> 0  5120   0% /run/lock
> none132062620   0
> 132062620   0% /run/shm
> cgroup  132062620   0
> 132062620   0% /sys/fs/cgroup
> /dev/sda610650544 2500424
> 7609092  25% /home
> 10.237.11.31:/mnt/vol1/sec-storage2/template/tmpl/2/288   4195924
> 4195924 0 100% /mnt/5230667e-9c58-3ff6-983c-5fc2a72df669
> 
>
> The strange thing is that df shows that it seems to be mounted, but it's
> actually not mounted. If you noticed, the total capacity of the mount point
> is exactly the same as the capacity of the / (root) partition. By right it
> should show 7 TB instead of just 4 GB.
>
> This caused VM creation to be error due to out of disk space. This also
> affect the KVM operations since the / (root) partition becomes full, and we
> can only release the space after we reboot the KVM host.
>
> Anyone experience this problem before? We are at loss on how to resolve the
> problem.
>
> Looking forward to your reply, thank you.
>
> Cheers.

Re: A question on vm migrations when hosts are set into a maintenance mode.

[Koushik Das]

Alena,
I see that you had added this config setting. Any specific reason that this 
only for Start/Stop/Copy and not for all the commands as Alex mentioned? The 
name of the setting looks generic.

-Koushik

On 03-Oct-2013, at 8:10 PM, Alex Ough  wrote:

> Hi Koushik,
> 
> Thanks for your reply, but unfortunately, this setting does NOT cover
> 'MigrateCommand'.
> As you specified, it seems to be effective only in Start/Stop/CopyCommand.
> 
> So can we include 'MigrateCommand' in that setting?
> 
> Thanks
> Alex Ough
> 
> 
> On Wed, Oct 2, 2013 at 11:44 PM, Koushik Das  wrote:
> 
>> It is already a global setting in 4.2. The name of the setting is
>> "execute.in.sequence.hypervisor.commands".
>> 
>> Check out Config.java
>> 
>>ExecuteInSequence("Advanced", ManagementServer.class, Boolean.class,
>> "execute.in.sequence.hypervisor.commands", "true", "If set to true,
>> StartCommand, StopCommand, CopyCommand will be synchronized on the agent
>> side." +
>>" If set to false, these commands become asynchronous. Default value
>> is true.", null),
>> 
>> 
>> -Koushik
>> 
>> On 02-Oct-2013, at 10:43 PM, Alex Ough > alex.o...@sungard.com>> wrote:
>> 
>> Thanks for the reply, Marcus.
>> 
>> What about the option #3, which is to make it as a global setting?
>> I think it can prevent side effects if exist.
>> 
>> Alex Ough
>> 
>> 
>> On Wed, Oct 2, 2013 at 11:37 AM, Marcus Sorensen > >wrote:
>> 
>> Not sure. I don't know the history well enough to know if there were
>> issues in the past, it might be that some hypervisors were fine and
>> others weren't.
>> 
>> On Wed, Oct 2, 2013 at 9:21 AM, Alex Ough > alex.o...@sungard.com>> wrote:
>> Marcus/Kelven,
>> 
>> Any thoughts on my suggestions?
>> 
>> Thanks
>> Alex Ough
>> 
>> 
>> On Tue, Sep 24, 2013 at 12:09 PM, Alex Ough > alex.o...@sungard.com>>
>> wrote:
>> 
>> Oh, sorry for the confusion. I must have reversed the flags.
>> As Kelven pointed, it is set as 'TRUE', which makes the process as
>> sequential.
>> 
>> So my questions are
>> 1. If there is any reason why the method have been defined to return
>> 'TRUE' always?
>> 2. Do we expect any side effects and/or malfunctioning if we change it
>> to
>> returning 'FALSE'?
>> 3. For a resolution without breaking possible flows, can we add the
>> value
>> of 'executeInSequence' to the global setting if #2 answers YES?
>> 
>> 
>> On Tue, Sep 24, 2013 at 11:19 AM, Marcus Sorensen > >
>> wrote:
>> 
>> I thought executeInSequence of 'true' made it go serially, or
>> sequentially. In my codebase for 4.1,4.2,master it's been 'true' since
>> August of 2010:
>> 
>> 2010-08-11 09:13:29 -0700 19) public class MigrateCommand extends
>> Command
>> {
>> 2010-08-11 09:13:29 -0700 20) String vmName;
>> 2010-08-11 09:13:29 -0700 21) String destIp;
>> 2011-08-10 10:26:04 -0700 22) String hostGuid;
>> 2010-08-11 09:13:29 -0700 23) boolean isWindows;
>> 2010-08-11 09:13:29 -0700 24)
>> 2010-08-11 09:13:29 -0700 25)
>> 2010-08-11 09:13:29 -0700 26) protected MigrateCommand() {
>> 2010-08-11 09:13:29 -0700 27) }
>> 2012-12-03 22:06:41 -0800 28)
>> 2010-08-11 09:13:29 -0700 29) public MigrateCommand(String vmName,
>> String destIp, boolean isWindows)
>> 2010-08-11 09:13:29 -0700 30) this.vmName = vmName;
>> 2010-08-11 09:13:29 -0700 31) this.destIp = destIp;
>> 2010-08-11 09:13:29 -0700 32) this.isWindows = isWindows;
>> 2010-08-11 09:13:29 -0700 33) }
>> 2012-12-03 22:06:41 -0800 34)
>> 2010-08-11 09:13:29 -0700 35) public boolean isWindows() {
>> 2010-08-11 09:13:29 -0700 36) return isWindows;
>> 2010-08-11 09:13:29 -0700 37) }
>> 2012-12-03 22:06:41 -0800 38)
>> 2010-08-11 09:13:29 -0700 39) public String getDestinationIp() {
>> 2010-08-11 09:13:29 -0700 40) return destIp;
>> 2010-08-11 09:13:29 -0700 41) }
>> 2012-12-03 22:06:41 -0800 42)
>> 2010-08-11 09:13:29 -0700 43) public String getVmName() {
>> 2010-08-11 09:13:29 -0700 44) return vmName;
>> 2010-08-11 09:13:29 -0700 45) }
>> 2012-12-03 22:06:41 -0800 46)
>> 2011-08-10 10:26:04 -0700 47) public void setHostGuid(String guid)
>> {
>> 2011-08-10 10:26:04 -0700 48) this.hostGuid = guid;
>> 2011-08-10 10:26:04 -0700 49) }
>> 2012-12-03 22:06:41 -0800 50)
>> 2011-08-10 10:26:04 -0700 51) public String getHostGuid() {
>> 2011-08-10 10:26:04 -0700 52) return this.hostGuid;
>> 2011-08-10 10:26:04 -0700 53) }
>> 2010-08-11 09:13:29 -0700 54)
>> 2010-08-11 09:13:29 -0700 55) @Override
>> 2010-08-11 09:13:29 -0700 56) public boolean executeInSequence() {
>> 2010-08-11 09:13:29 -0700 57) return true;
>> 2010-08-11 09:13:29 -0700 58) }
>> 2010-08-11 09:13:29 -0700 59) }
>> 
>> On Tue, Sep 24, 2013 at 9:58 AM, Chip Childers
>> mailto:chip.child...@sungard.com>> wrote:
>> Hey Kelven - This topic was discussed briefly in the past [1].  Are
>> you
>> able to provide any thoughts on Al

Re: Can't compile cloudstack 4.2 rpms from source

[benoit lair]

Yes Chip,

Even if try to compile cs 4.2.0 with mvn -P deps or if i try to compile
with the package.sh script, i have the same errors.

here is the ouput generated with package.sh :

http://pastebin.com/TNegzXUX

Regards, Benoit Lair.


2013/10/3 Chip Childers 

> So the mvn build has the same issues as the RPM build?  I'm just
> confirming what you are saying is the condition.
>
> On Thu, Oct 03, 2013 at 05:20:19PM +0200, benoit lair wrote:
> > Hello Chip,
> >
> > Yes i do. I've tried mvn clean install, and :
> >
> > mvn
> > mvn clean install
> > mvn - P deps (as the install way of cs 4.0.0)
> > mvn clean install
> >
> >
> > I got always the same errors.
> >
> >
> > 2013/10/3 Chip Childers 
> >
> > > On Thu, Oct 03, 2013 at 11:21:57AM +0200, benoit lair wrote:
> > > > A precision, the linux centos 6.3 i used is a ghost of the centos 6.3
> > > > environment i use for compiling apache cloudstack 4.0.0-incubating,
> which
> > > > works very well, but not with cs 4.2.0
> > >
> > > Are you able to build using maven (don't use the package.sh script, do
> > > mvn clean install).
> > >
>

Re: Configuration table changes and database update

[Darren Shepherd]

I've done my best to test various DB upgrade scenarios and I believe
all this stuff should be fixed in the spring-modularization branch.  I
stated wrong before, the database upgrade stuff is in the system
context.  So the majority of ACS is in the "core" context which is
initialized after system.

Darren

On Thu, Oct 3, 2013 at 7:35 AM, Darren Shepherd
 wrote:
> Let me look at this today.  The spring changes I have will greatly improve 
> these initialization issues.  The current state of spring on master makes it 
> very difficult to deterministic ally initialize beans.  With the spring 
> changes I have in place the database upgrade stuff is in the bootstrap 
> context and will be initialized before everything else.
>
> You mentioned there's an issue with the database upgrade depending on a DAO.  
> I'll look into that and fix it today.  The fix will be on the 
> spring-modularization branch which I'd like to get merge ASAP.
>
> Darren
>
>> On Oct 3, 2013, at 2:08 AM, Hugo Trippaers  wrote:
>>
>> Heya guys,
>>
>> I'm still running into this problem. Our jenkins automagically installs and 
>> tests the latest packages based on master. With a clean database on clean 
>> machines.
>>
>> The error reported is similar to the error encountered:
>>
>> Caused by: com.cloud.utils.exception.CloudRuntimeException: DB Exception on: 
>> com.mysql.jdbc.JDBC4PreparedStatement@64726693: SELECT 
>> configuration.instance, configuration.component, configuration.name, 
>> configuration.value, configuration.default_value, configuration.description, 
>> configuration.category, configuration.is_dynamic, configuration.scope, 
>> configuration.updated FROM configuration WHERE configuration.name = 
>> _binary'storage.cache.replacement.lru.interval'  ORDER BY RAND() LIMIT 1
>>at 
>> com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:421)
>>at 
>> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>>at 
>> com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:356)
>>at 
>> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>>at 
>> com.cloud.utils.db.GenericDaoBase.findOneIncludingRemovedBy(GenericDaoBase.java:869)
>>at 
>> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>>at 
>> org.apache.cloudstack.framework.config.dao.ConfigurationDaoImpl.findByName(ConfigurationDaoImpl.java:201)
>>at 
>> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>>at 
>> org.apache.cloudstack.framework.config.dao.ConfigurationDaoImpl.getValue(ConfigurationDaoImpl.java:166)
>>at 
>> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>>at 
>> org.apache.cloudstack.storage.cache.manager.StorageCacheReplacementAlgorithmLRU.initialize(StorageCacheReplacementAlgorithmLRU.java:63)
>>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>at 
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>at 
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>at java.lang.reflect.Method.invoke(Method.java:616)
>>at 
>> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:346)
>>at 
>> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:299)
>>at 
>> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:132)
>>... 79 more
>> Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: 
>> Unknown column 'configuration.default_value' in 'field list'
>>at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>at 
>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>>at 
>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
>>at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
>>at com.mysql.jdbc.Util.getInstance(Util.java:386)
>>at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1053)
>>at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4074)
>>at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4006)
>>at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2468)
>>at com.mys

Re: [MERGE] spring-modularization to master - Spring Modularization

[Darren Shepherd]

This should be fixed on the spring-modularization branch.

Darren

On Wed, Oct 2, 2013 at 10:05 PM, Prasanna Santhanam  wrote:
> I switched the test infrastructure on jenkins.buildacloud.org to run
> the bvts [1] against master last week. Couple of weeks before that the
> simulator [2] tests were switched to run against master. Both are
> broken unfortunately and the bvt and checkin tests aren't running.
>
> I've filed a bug here:
> https://issues.apache.org/jira//browse/CLOUDSTACK-4791
>
> [1] http://jenkins.buildacloud.org/view/cloudstack-qa/
> [2] http://jenkins.buildacloud.org/view/simulator/
>
> Anyone who has access to the jenkins can run the bvts on their desired
> branch. Simply login and change the test-yumrepo-refresh job to point
> to your branch. Build that to refresh the remote repository with the
> packages made from your branch. Then switch test-matrix to point to
> the same development branch and fire a build. That's about it.
>
> On Wed, Oct 02, 2013 at 05:42:54PM -0700, Darren Shepherd wrote:
>> Yes agreed.  I've extensively tested this, but that is never enough.
>> How do I get the BVTs ran against this.  Due to the cross cutting
>> nature of this I want to get this merged as fast as possible.
>>
>> Darren
>>
>> > On Oct 2, 2013, at 4:43 PM, Alex Huang  wrote:
>> >
>> > +1 on running the BVT on it.  We've been through this one once before.  
>> > Should be careful.
>> >
>> > --Alex
>> >
>> >> -Original Message-
>> >> From: Kelven Yang [mailto:kelven.y...@citrix.com]
>> >> Sent: Wednesday, October 2, 2013 4:39 PM
>> >> To: dev@cloudstack.apache.org
>> >> Subject: Re: [MERGE] spring-modularization to master - Spring
>> >> Modularization
>> >>
>> >> Darren,
>> >>
>> >> This looks really nice. A few questions on Spring AOP replacement.
>> >>
>> >> 1) Spring AOP is proxy-based, the reason we ended up of using customized
>> >> AOP is mainly due to that inside existing CloudStack codebase, we have 
>> >> many
>> >> places that are doing run-time type-casting, the code in these places
>> >> assumes a real object that implements all interfaces in the semantics 
>> >> context.
>> >> At the time when I initially converted to Spring, I couldn't ensure that
>> >> switching to proxy-based AOP can have 100% coverage for these run-time
>> >> cases. What is your approach to address this run-time type-casting issue?
>> >>
>> >> 2) We've run into a huge-memory footprint issue that may be caused by
>> >> conflicts of CGLIB usage in Spring AOP and the CGLIB usage in CloudStack 
>> >> Dao
>> >> layer. Do you have a chance to run a memory analysis in the heap after
>> >> management server is started.
>> >>
>> >> I might be good to run BVT test on the branch before the merge, could
>> >> someone initiate the effort?
>> >>
>> >> kelven
>> >>
>> >>
>> >>
>> >> On 10/2/13 3:48 PM, "Darren Shepherd" 
>> >> wrote:
>> >>
>> >>> Not sure how this works...  I would like to merge in the new
>> >>> modularized Spring setup to master. There is info on the wiki about it
>> >>> [1] [2] [3].  The primary change is to break apart the monolithic
>> >>> applicationContext.xml and componentContext.xml files such that each
>> >>> plugin can maintain and contribute its own configuration.
>> >>>
>> >>> In addition to breaking up the configuration we no longer use ACS
>> >>> custom AOP and it is now fully Spring AOP.
>> >>>
>> >>> Now adding/removing a plug-in is a matter of just adding a jar to the
>> >>> classpath (exception being commands.properties, I'll address that in a
>> >>> different thread).  Unfortunately this branch does not have the changes
>> >>> to package things in different RPMs.  So it would be great if somebody
>> >>> could take up the packaging effort to split out all the plugins into
>> >>> different RPMs.
>> >>>
>> >>> Darren
>> >>>
>> >>> [1]
>> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Sp
>> >> rin
>> >>> g
>> >>> [2]
>> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-
>> >> ins%2C+Modu
>> >>> les
>> >>> %2C+and+Extensions
>> >>> [3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
>> >
>
> --
> Prasanna.,
>
> 
> Powered by BigRock.com
>

Re: [MERGE] spring-modularization to master - Spring Modularization

[Darren Shepherd]

Sure can switch jenkins.buildacloud.org to build off of
spring-modularization?  It will at least fix the DB upgrade issues. I
really don't understand the whole test environment.  What tests are
ran where, how to get "BVT" running etc.

Darren

On Thu, Oct 3, 2013 at 7:58 AM, Chip Childers  wrote:
> On Wed, Oct 2, 2013 at 8:59 PM, Darren Shepherd > wrote:
>
>> It's actually really easy to turn this off.  If you look at the last
>> commit of the branch you'll see what changes to turn it on.  So you can
>> just reverse that one commit to disable it.
>>
>>  I'd rather merge this and then do the BVTs on master if possible.  I
>> really want to get this committed.  Every time somebody changes a spring
>> config file I have to keep making the corresponding changes.  It seems like
>> people are gearing up and developing stuff right now for 4.3.  So the
>> longer we wait the more headache it is for everyone.
>>
>> Again, if we find out this is a complete disaster, I can just flip some
>> config settings and were back to the old style.  I had to keep
>> compatibility with the old style as Awsapi and usage still use it.
>>
>> Darren
>>
>
>
> I get the desire, but *please* let's use BVT on this.  We, unfortunately,
> have a poor track record with changes like this breaking things in master.
>  I'm not saying this as a judgement / measurement of your branch, I'm
> making a general historical statement.  We really need to hold ourselves to
> the goal of keeping master stable through clear test results in feature
> branches prior to a merge.

[RESOLVED] Re: [URGENT] Unable to stop/start system VMs after upgrade to 4.2.0

[Indra Pramana]

Dear Venkata and all,

This has been resolved earlier. Realised that the KVM host (where the CPVM
is on) is on disconnected state. Problem was resolved after the problematic
KVM host was rebooted.

Thank you every one for your help.

Cheers.


On Thu, Oct 3, 2013 at 9:42 PM, Venkata SwamyBabu Budumuru <
venkataswamybabu.budum...@citrix.com> wrote:

> What is the hypervisor you are using?
>
> On 03/10/13 6:14 PM, "David Ortiz"  wrote:
>
> >If you manually change the vm status in the database would it let you
> >destroy it then?
> >
> >> Date: Thu, 3 Oct 2013 20:22:33 +0800
> >> Subject: Re: [URGENT] Unable to stop/start system VMs after upgrade to
> >>4.2.0
> >> From: in...@sg.or.id
> >> To: dev@cloudstack.apache.org; us...@cloudstack.apache.org
> >>
> >> Dear all,
> >>
> >> I managed to fix the VRs issue by destroying the routers and restarting
> >>the
> >> networks associated to the routers.
> >>
> >> Only left is the CPVM, I am not able to stop/destroy it. Any advise?
> >>
> >> Looking forward to your reply, thank you.
> >>
> >> Cheers.
> >>
> >>
> >>
> >> On Thu, Oct 3, 2013 at 8:16 PM, Indra Pramana  wrote:
> >>
> >> > Dear all,
> >> >
> >> > I am now in the progress of upgrading my CloudStack from 4.1.1 to
> >>4.2.0.
> >> > Running cloudstack-sysvmadm can restart the SSVM, but failed to
> >>restart the
> >> > CPVM and all the VRs.
> >> >
> >> > 
> >> > [1] 3152
> >> > root@cs-mgmt-01:~/backup/20131003# tail -f sysvm.log
> >> > nohup: ignoring input
> >> > /usr/bin/cloudstack-sysvmadm: line 21: /etc/rc.d/init.d/functions: No
> >>such
> >> > file or directory
> >> >
> >> > Stopping and starting 1 secondary storage vm(s)...
> >> > Done stopping and starting secondary storage vm(s)
> >> >
> >> > Stopping and starting 1 console proxy vm(s)...
> >> > ERROR: Failed to stop console proxy vm with id 1980
> >> >
> >> > Done stopping and starting console proxy vm(s) .
> >> >
> >> > Stopping and starting 0 running routing vm(s)...
> >> > 
> >> >
> >> > SSVM seems OK, I have run health check script and everything is in
> >>order.
> >> >
> >> > VM state of CPVM is running, but state is disconnected. I can't stop
> >>or
> >> > destroy the CPVM.
> >> >
> >> > All the VRs are running now but I am not too sure if they are already
> >> > using the new 4.2 template or still using the old template, since it
> >>seems
> >> > the sysvmadm script was unable to detect the virtual routers.
> >> >
> >> > Anyone can help?
> >> >
> >> > Thank you.
> >> >
> >
>
>

Re: CS 4.2.0 - Bug with ssvm - wrong capacity recognized

[benoit lair]

Okay, i deleted my secondary storage server from the ui.
Change the ip of my secondary storage server vm, switched to the good vlan
from xencenter
Declared the new secondary storage server from the UI
Seen that i do not configured the good xenserver storage traffic label, so
i got the storage traffic routed on a bad nic on the hypervisor (it's my
fault so, you can't throw the tomatoes)
Rebooted the ssvm vm from the UI, got a clear ssvm fresh install pointing
on the nfs server.
So ssh into the ssvm, seen that the nfs server is well mounted.
I just tried to upload a template, my template is uploading into my nfs
server into templates directory.


Thanks a lot Wei !


2013/10/3 benoit lair 

> Okay, i changed the ip of the secondary storage server into the ip range
> of the pod, it doesn't work better.
>
> I change now the network of the secondary storage server.
>
>
> 2013/10/3 benoit lair 
>
>> I thought i could have my secondary storage nfs server into the mgmt cidr
>> ? It were okay with a cs 4.0.0
>>
>> So i need to move my secondary storage nfs server to the 10.32.0.0 cidr ?
>>
>>
>> 2013/10/3 Wei ZHOU 
>>
>>> I guess the root cause could be that your nfs server ip is not in storage
>>> network.
>>>
>>
>>
>

[URGENT] Mounting issues from KVM hosts to secondary storage

[Indra Pramana]

Dear all,

We face a major problem after upgrading to 4.2.0. Mounting from KVM hosts
to secondary storage seems to fail, every time a new VM instance is
created, it will use up the / (root) partition of the KVM hosts instead.

Here is the df result:


root@hv-kvm-02:/home/indra# df
Filesystem  1K-blocksUsed
Available Use% Mounted on
/dev/sda1 4195924
4195924 0 100% /
udev132053356   4
132053352   1% /dev
tmpfs52825052 440
52824612   1% /run
none 5120
0  5120   0% /run/lock
none132062620   0
132062620   0% /run/shm
cgroup  132062620   0
132062620   0% /sys/fs/cgroup
/dev/sda610650544 2500424
7609092  25% /home
10.237.11.31:/mnt/vol1/sec-storage2/template/tmpl/2/288   4195924
4195924 0 100% /mnt/5230667e-9c58-3ff6-983c-5fc2a72df669


The strange thing is that df shows that it seems to be mounted, but it's
actually not mounted. If you noticed, the total capacity of the mount point
is exactly the same as the capacity of the / (root) partition. By right it
should show 7 TB instead of just 4 GB.

This caused VM creation to be error due to out of disk space. This also
affect the KVM operations since the / (root) partition becomes full, and we
can only release the space after we reboot the KVM host.

Anyone experience this problem before? We are at loss on how to resolve the
problem.

Looking forward to your reply, thank you.

Cheers.

Re: Can't compile cloudstack 4.2 rpms from source

[Chip Childers]

So the mvn build has the same issues as the RPM build?  I'm just
confirming what you are saying is the condition.

On Thu, Oct 03, 2013 at 05:20:19PM +0200, benoit lair wrote:
> Hello Chip,
> 
> Yes i do. I've tried mvn clean install, and :
> 
> mvn
> mvn clean install
> mvn - P deps (as the install way of cs 4.0.0)
> mvn clean install
> 
> 
> I got always the same errors.
> 
> 
> 2013/10/3 Chip Childers 
> 
> > On Thu, Oct 03, 2013 at 11:21:57AM +0200, benoit lair wrote:
> > > A precision, the linux centos 6.3 i used is a ghost of the centos 6.3
> > > environment i use for compiling apache cloudstack 4.0.0-incubating, which
> > > works very well, but not with cs 4.2.0
> >
> > Are you able to build using maven (don't use the package.sh script, do
> > mvn clean install).
> >

Re: Can't compile cloudstack 4.2 rpms from source

[Wei ZHOU]

mvn -P developer,systemvm clean install

Re: Can't compile cloudstack 4.2 rpms from source

[benoit lair]

In order to be sure that there weren't something wrong with my environment,
i've retried to create rpms for apache cloudstack 4.0.0 and it worked well.
I've deleted the cs 4.2.0 src dir, tar xvfz of the tarbal file of cs 4.2
sources, retried to launch package.sh, same error. Retried a mvn clean and
so a mvn -P deps as in the cs 4.0.0 docs, and i got the same error.

Thanks for your help.

Regards, Benoit


2013/10/3 benoit lair 

> Hello Chip,
>
> Yes i do. I've tried mvn clean install, and :
>
> mvn
> mvn clean install
> mvn - P deps (as the install way of cs 4.0.0)
> mvn clean install
>
>
> I got always the same errors.
>
>
> 2013/10/3 Chip Childers 
>
>> On Thu, Oct 03, 2013 at 11:21:57AM +0200, benoit lair wrote:
>> > A precision, the linux centos 6.3 i used is a ghost of the centos 6.3
>> > environment i use for compiling apache cloudstack 4.0.0-incubating,
>> which
>> > works very well, but not with cs 4.2.0
>>
>> Are you able to build using maven (don't use the package.sh script, do
>> mvn clean install).
>>
>
>

Re: [Proposal] Improve VR upgrades

[Chip Childers]

On Thu, Oct 03, 2013 at 11:47:57AM +, Kishan Kavala wrote:
> During CS upgrade, VRs are required to be upgraded to use newer systemVm 
> template . 
> The current VR upgrade procedure has following limitations:
>  - takes 'long' time and the time exponentially increases with the size of 
> the cloud
> - no way to sequence upgrade of different parts of the cloud, i.e., specific 
> clusters or pods or even zones
> - there is no way to determine when a particular customer's services (e.g. 
> VR) will be upgraded with the upgrade interval
> 
> Goals for this feature are to address the above issues
> 
> 1. Give admin control to sequence the upgrade of the cloud by:
>- Infrastructure hierarchy: by Cluster, Pod, and Zone etc.
>- Administrative hierarchy: by Tenant or Domain 
> 2. Minimize service interruption to users
> 3. Improve the speed of the upgrade time by making as many upgrade operations 
> in parallel as possible
> 
> I've created JIRA ticket:
> https://issues.apache.org/jira/browse/CLOUDSTACK-4793
> 
> thanks,
> Kishan
>

This proposal sounds great, but the devil will be in the implementation
details.  To do this as rolling, we'd need to ensure backward compat
with agent>MS communications, right?

-chip

Re: Can't compile cloudstack 4.2 rpms from source

[benoit lair]

Hello Chip,

Yes i do. I've tried mvn clean install, and :

mvn
mvn clean install
mvn - P deps (as the install way of cs 4.0.0)
mvn clean install


I got always the same errors.


2013/10/3 Chip Childers 

> On Thu, Oct 03, 2013 at 11:21:57AM +0200, benoit lair wrote:
> > A precision, the linux centos 6.3 i used is a ghost of the centos 6.3
> > environment i use for compiling apache cloudstack 4.0.0-incubating, which
> > works very well, but not with cs 4.2.0
>
> Are you able to build using maven (don't use the package.sh script, do
> mvn clean install).
>

Re: Review Request 14467: fix for Invalid SMTP breaks HA

[Chip Childers]

On Thu, Oct 03, 2013 at 11:31:50AM +, Anshul Gangwar wrote:
> Description
> ---
> 
> added the connection and socket timeout parameters for SMTP and sending 
> message in new thread so that HA doesn't get blocked beacause of hang in 
> sending email alert

Wouldn't a better approach be to ensure that email sending runs async to
critical operations like HA?

Re: Wiki access

[Chip Childers]

On Thu, Oct 03, 2013 at 09:58:03AM +, Bharat Kumar wrote:
> Hi All,
> any one please grant me the edit or add page permission to cwiki. my user 
> name is bharat.kumar

Done

Re: Can't compile cloudstack 4.2 rpms from source

[Chip Childers]

On Thu, Oct 03, 2013 at 11:21:57AM +0200, benoit lair wrote:
> A precision, the linux centos 6.3 i used is a ghost of the centos 6.3
> environment i use for compiling apache cloudstack 4.0.0-incubating, which
> works very well, but not with cs 4.2.0

Are you able to build using maven (don't use the package.sh script, do
mvn clean install).

Re: [DISCUSS] Breaking out Marvin from CloudStack

[Chip Childers]

On Thu, Oct 03, 2013 at 05:55:14AM +, Santhosh Edukulla wrote:
> I believe both can exist independently. The framework should have more 
> flexibility and facilitation to run features\test cases based upon a  given 
> release and version on a given test bed. It should be intelligent enough to 
> run those cases  and only those cases pertaining to that release with few 
> config variables. Keeping aside branch\repo terminology, for me all marvin 
> both framework and tests should be at one place and separate from main 
> product code.

The problem with tests being broken out from the main repo is that we
end up having a really hard time confirming that new features include
appropriate tests.  Plus, tests are tied to features, which are tied to
the code.  They should be versioned together.

As you said though, the framework is the framework...  that's a tool.
Frankly, with auto-discovery going forward, Marvin becomes no different
from any other testing framework software that has to be installed to do
useful things with the code in the main repo.  Ex: maven!

-chip

Re: [MERGE] spring-modularization to master - Spring Modularization

[Chip Childers]

On Wed, Oct 2, 2013 at 8:59 PM, Darren Shepherd  wrote:

> It's actually really easy to turn this off.  If you look at the last
> commit of the branch you'll see what changes to turn it on.  So you can
> just reverse that one commit to disable it.
>
>  I'd rather merge this and then do the BVTs on master if possible.  I
> really want to get this committed.  Every time somebody changes a spring
> config file I have to keep making the corresponding changes.  It seems like
> people are gearing up and developing stuff right now for 4.3.  So the
> longer we wait the more headache it is for everyone.
>
> Again, if we find out this is a complete disaster, I can just flip some
> config settings and were back to the old style.  I had to keep
> compatibility with the old style as Awsapi and usage still use it.
>
> Darren
>


I get the desire, but *please* let's use BVT on this.  We, unfortunately,
have a poor track record with changes like this breaking things in master.
 I'm not saying this as a judgement / measurement of your branch, I'm
making a general historical statement.  We really need to hold ourselves to
the goal of keeping master stable through clear test results in feature
branches prior to a merge.

RE: [MERGE] spring-modularization to master - Spring Modularization

[Rayees Namathponnan]

Hi Prasanna,



If we are running BVT or regression on specific branch, marvin should also from 
same branch right ? but we always getting marvin from master branch  job 
"cloudstack-marvin"



Regards,

Rayees



-Original Message-
From: Prasanna Santhanam [mailto:t...@apache.org]
Sent: Wednesday, October 02, 2013 10:06 PM
To: dev@cloudstack.apache.org
Subject: Re: [MERGE] spring-modularization to master - Spring Modularization



I switched the test infrastructure on jenkins.buildacloud.org to run the bvts 
[1] against master last week. Couple of weeks before that the simulator [2] 
tests were switched to run against master. Both are broken unfortunately and 
the bvt and checkin tests aren't running.



I've filed a bug here:

https://issues.apache.org/jira//browse/CLOUDSTACK-4791



[1] http://jenkins.buildacloud.org/view/cloudstack-qa/

[2] http://jenkins.buildacloud.org/view/simulator/



Anyone who has access to the jenkins can run the bvts on their desired branch. 
Simply login and change the test-yumrepo-refresh job to point to your branch. 
Build that to refresh the remote repository with the packages made from your 
branch. Then switch test-matrix to point to the same development branch and 
fire a build. That's about it.



On Wed, Oct 02, 2013 at 05:42:54PM -0700, Darren Shepherd wrote:

> Yes agreed.  I've extensively tested this, but that is never enough.

> How do I get the BVTs ran against this.  Due to the cross cutting

> nature of this I want to get this merged as fast as possible.

>

> Darren

>

> > On Oct 2, 2013, at 4:43 PM, Alex Huang 
> > mailto:alex.hu...@citrix.com>> wrote:

> >

> > +1 on running the BVT on it.  We've been through this one once before.  
> > Should be careful.

> >

> > --Alex

> >

> >> -Original Message-

> >> From: Kelven Yang [mailto:kelven.y...@citrix.com]

> >> Sent: Wednesday, October 2, 2013 4:39 PM

> >> To: dev@cloudstack.apache.org

> >> Subject: Re: [MERGE] spring-modularization to master - Spring

> >> Modularization

> >>

> >> Darren,

> >>

> >> This looks really nice. A few questions on Spring AOP replacement.

> >>

> >> 1) Spring AOP is proxy-based, the reason we ended up of using

> >> customized AOP is mainly due to that inside existing CloudStack

> >> codebase, we have many places that are doing run-time type-casting,

> >> the code in these places assumes a real object that implements all 
> >> interfaces in the semantics context.

> >> At the time when I initially converted to Spring, I couldn't ensure

> >> that switching to proxy-based AOP can have 100% coverage for these

> >> run-time cases. What is your approach to address this run-time 
> >> type-casting issue?

> >>

> >> 2) We've run into a huge-memory footprint issue that may be caused

> >> by conflicts of CGLIB usage in Spring AOP and the CGLIB usage in

> >> CloudStack Dao layer. Do you have a chance to run a memory analysis

> >> in the heap after management server is started.

> >>

> >> I might be good to run BVT test on the branch before the merge,

> >> could someone initiate the effort?

> >>

> >> kelven

> >>

> >>

> >>

> >> On 10/2/13 3:48 PM, "Darren Shepherd" 
> >> mailto:darren.s.sheph...@gmail.com>>

> >> wrote:

> >>

> >>> Not sure how this works...  I would like to merge in the new

> >>> modularized Spring setup to master. There is info on the wiki

> >>> about it [1] [2] [3].  The primary change is to break apart the

> >>> monolithic applicationContext.xml and componentContext.xml files

> >>> such that each plugin can maintain and contribute its own configuration.

> >>>

> >>> In addition to breaking up the configuration we no longer use ACS

> >>> custom AOP and it is now fully Spring AOP.

> >>>

> >>> Now adding/removing a plug-in is a matter of just adding a jar to

> >>> the classpath (exception being commands.properties, I'll address

> >>> that in a different thread).  Unfortunately this branch does not

> >>> have the changes to package things in different RPMs.  So it would

> >>> be great if somebody could take up the packaging effort to split

> >>> out all the plugins into different RPMs.

> >>>

> >>> Darren

> >>>

> >>> [1]

> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+

> >>> Sp

> >> rin

> >>> g

> >>> [2]

> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-

> >> ins%2C+Modu

> >>> les

> >>> %2C+and+Extensions

> >>> [3]

> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions

> >



--

Prasanna.,





Powered by BigRock.com

Re: A question on vm migrations when hosts are set into a maintenance mode.

[Alex Ough]

Hi Koushik,

Thanks for your reply, but unfortunately, this setting does NOT cover
'MigrateCommand'.
As you specified, it seems to be effective only in Start/Stop/CopyCommand.

So can we include 'MigrateCommand' in that setting?

Thanks
Alex Ough


On Wed, Oct 2, 2013 at 11:44 PM, Koushik Das  wrote:

> It is already a global setting in 4.2. The name of the setting is
> "execute.in.sequence.hypervisor.commands".
>
> Check out Config.java
>
> ExecuteInSequence("Advanced", ManagementServer.class, Boolean.class,
> "execute.in.sequence.hypervisor.commands", "true", "If set to true,
> StartCommand, StopCommand, CopyCommand will be synchronized on the agent
> side." +
> " If set to false, these commands become asynchronous. Default value
> is true.", null),
>
>
> -Koushik
>
> On 02-Oct-2013, at 10:43 PM, Alex Ough  alex.o...@sungard.com>> wrote:
>
> Thanks for the reply, Marcus.
>
> What about the option #3, which is to make it as a global setting?
> I think it can prevent side effects if exist.
>
> Alex Ough
>
>
> On Wed, Oct 2, 2013 at 11:37 AM, Marcus Sorensen  >wrote:
>
> Not sure. I don't know the history well enough to know if there were
> issues in the past, it might be that some hypervisors were fine and
> others weren't.
>
> On Wed, Oct 2, 2013 at 9:21 AM, Alex Ough  alex.o...@sungard.com>> wrote:
> Marcus/Kelven,
>
> Any thoughts on my suggestions?
>
> Thanks
> Alex Ough
>
>
> On Tue, Sep 24, 2013 at 12:09 PM, Alex Ough  alex.o...@sungard.com>>
> wrote:
>
> Oh, sorry for the confusion. I must have reversed the flags.
> As Kelven pointed, it is set as 'TRUE', which makes the process as
> sequential.
>
> So my questions are
> 1. If there is any reason why the method have been defined to return
> 'TRUE' always?
> 2. Do we expect any side effects and/or malfunctioning if we change it
> to
> returning 'FALSE'?
> 3. For a resolution without breaking possible flows, can we add the
> value
> of 'executeInSequence' to the global setting if #2 answers YES?
>
>
> On Tue, Sep 24, 2013 at 11:19 AM, Marcus Sorensen  >
> wrote:
>
> I thought executeInSequence of 'true' made it go serially, or
> sequentially. In my codebase for 4.1,4.2,master it's been 'true' since
> August of 2010:
>
> 2010-08-11 09:13:29 -0700 19) public class MigrateCommand extends
> Command
> {
> 2010-08-11 09:13:29 -0700 20) String vmName;
> 2010-08-11 09:13:29 -0700 21) String destIp;
> 2011-08-10 10:26:04 -0700 22) String hostGuid;
> 2010-08-11 09:13:29 -0700 23) boolean isWindows;
> 2010-08-11 09:13:29 -0700 24)
> 2010-08-11 09:13:29 -0700 25)
> 2010-08-11 09:13:29 -0700 26) protected MigrateCommand() {
> 2010-08-11 09:13:29 -0700 27) }
> 2012-12-03 22:06:41 -0800 28)
> 2010-08-11 09:13:29 -0700 29) public MigrateCommand(String vmName,
> String destIp, boolean isWindows)
> 2010-08-11 09:13:29 -0700 30) this.vmName = vmName;
> 2010-08-11 09:13:29 -0700 31) this.destIp = destIp;
> 2010-08-11 09:13:29 -0700 32) this.isWindows = isWindows;
> 2010-08-11 09:13:29 -0700 33) }
> 2012-12-03 22:06:41 -0800 34)
> 2010-08-11 09:13:29 -0700 35) public boolean isWindows() {
> 2010-08-11 09:13:29 -0700 36) return isWindows;
> 2010-08-11 09:13:29 -0700 37) }
> 2012-12-03 22:06:41 -0800 38)
> 2010-08-11 09:13:29 -0700 39) public String getDestinationIp() {
> 2010-08-11 09:13:29 -0700 40) return destIp;
> 2010-08-11 09:13:29 -0700 41) }
> 2012-12-03 22:06:41 -0800 42)
> 2010-08-11 09:13:29 -0700 43) public String getVmName() {
> 2010-08-11 09:13:29 -0700 44) return vmName;
> 2010-08-11 09:13:29 -0700 45) }
> 2012-12-03 22:06:41 -0800 46)
> 2011-08-10 10:26:04 -0700 47) public void setHostGuid(String guid)
> {
> 2011-08-10 10:26:04 -0700 48) this.hostGuid = guid;
> 2011-08-10 10:26:04 -0700 49) }
> 2012-12-03 22:06:41 -0800 50)
> 2011-08-10 10:26:04 -0700 51) public String getHostGuid() {
> 2011-08-10 10:26:04 -0700 52) return this.hostGuid;
> 2011-08-10 10:26:04 -0700 53) }
> 2010-08-11 09:13:29 -0700 54)
> 2010-08-11 09:13:29 -0700 55) @Override
> 2010-08-11 09:13:29 -0700 56) public boolean executeInSequence() {
> 2010-08-11 09:13:29 -0700 57) return true;
> 2010-08-11 09:13:29 -0700 58) }
> 2010-08-11 09:13:29 -0700 59) }
>
> On Tue, Sep 24, 2013 at 9:58 AM, Chip Childers
> mailto:chip.child...@sungard.com>> wrote:
> Hey Kelven - This topic was discussed briefly in the past [1].  Are
> you
> able to provide any thoughts on Alex's ideas below?
>
> -chip
>
>
> [1] http://markmail.org/message/fznrszaswruvlmuy
>
>
>
> On Tue, Sep 24, 2013 at 10:53:04AM -0500, Alex Ough wrote:
> For a resolution without breaking possible flows, I'd like to add
> the
> value
> of 'executeInSequence' to the global setting.
> Is there any reason not to do this?
>
> Thanks
> Alex Ough
>
>
> On Mon, Sep 23, 2013 at 12:57 PM, Alex Ough  alex.o...@sungard.com>>
> wrote:

Re: unable to get tiny linux template in ready state, DOWNLOAD ERROR

[Wei ZHOU]

It is normal.
You can get more information in vmops.log

Re: Configuration table changes and database update

[Darren Shepherd]

Let me look at this today.  The spring changes I have will greatly improve 
these initialization issues.  The current state of spring on master makes it 
very difficult to deterministic ally initialize beans.  With the spring changes 
I have in place the database upgrade stuff is in the bootstrap context and will 
be initialized before everything else.

You mentioned there's an issue with the database upgrade depending on a DAO.  
I'll look into that and fix it today.  The fix will be on the 
spring-modularization branch which I'd like to get merge ASAP.

Darren

> On Oct 3, 2013, at 2:08 AM, Hugo Trippaers  wrote:
> 
> Heya guys,
> 
> I'm still running into this problem. Our jenkins automagically installs and 
> tests the latest packages based on master. With a clean database on clean 
> machines.
> 
> The error reported is similar to the error encountered:
> 
> Caused by: com.cloud.utils.exception.CloudRuntimeException: DB Exception on: 
> com.mysql.jdbc.JDBC4PreparedStatement@64726693: SELECT 
> configuration.instance, configuration.component, configuration.name, 
> configuration.value, configuration.default_value, configuration.description, 
> configuration.category, configuration.is_dynamic, configuration.scope, 
> configuration.updated FROM configuration WHERE configuration.name = 
> _binary'storage.cache.replacement.lru.interval'  ORDER BY RAND() LIMIT 1
>at 
> com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:421)
>at 
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>at 
> com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:356)
>at 
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>at 
> com.cloud.utils.db.GenericDaoBase.findOneIncludingRemovedBy(GenericDaoBase.java:869)
>at 
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>at 
> org.apache.cloudstack.framework.config.dao.ConfigurationDaoImpl.findByName(ConfigurationDaoImpl.java:201)
>at 
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>at 
> org.apache.cloudstack.framework.config.dao.ConfigurationDaoImpl.getValue(ConfigurationDaoImpl.java:166)
>at 
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>at 
> org.apache.cloudstack.storage.cache.manager.StorageCacheReplacementAlgorithmLRU.initialize(StorageCacheReplacementAlgorithmLRU.java:63)
>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>at java.lang.reflect.Method.invoke(Method.java:616)
>at 
> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:346)
>at 
> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:299)
>at 
> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:132)
>... 79 more
> Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown 
> column 'configuration.default_value' in 'field list'
>at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
>at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
>at com.mysql.jdbc.Util.getInstance(Util.java:386)
>at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1053)
>at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4074)
>at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4006)
>at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2468)
>at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2629)
>at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2719)
>at 
> com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155)
>at 
> com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2318)
>at 
> org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
>at 
> org.apache.commons.dbcp.De

Review Request 14469: CLOUDSTACK-4637: Fix failures in test_egress_fw_rules.py

[Girish Shilamkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14469/
---

Review request for cloudstack, Harikrishna Patnala and venkata swamy babu  
budumuru.


Bugs: CLOUDSTACK-4637


Repository: cloudstack-git


Description
---

Add logic to wait for router to boot.


Diffs
-

  test/integration/component/test_egress_fw_rules.py 5c18f9c 

Diff: https://reviews.apache.org/r/14469/diff/


Testing
---


Thanks,

Girish Shilamkar

Re: unable to get tiny linux template in ready state, DOWNLOAD ERROR

[Punith s]

hi wei,

i tried to re deploy the db ran the MS again , in parallel i ran the
devcloud python script , this is my MS console log , i saw some xen
resources missing ,

WARN  [cloud.cluster.ClusterManagerImpl] (Cluster-Notification-1:)
Notifying management server join event took 175 ms
INFO  [cloud.configuration.ConfigurationManagerImpl]
(1927634425@qtp-938693467-3:) adding a new subnet to the network 204
INFO  [cloud.resource.ResourceManagerImpl] (1927634425@qtp-938693467-3:)
Trying to add a new host at http://192.168.56.10/ in data center 1
INFO  [xen.discoverer.XcpServerDiscoverer] (1927634425@qtp-938693467-3:)
Found host devcloud ip=192.168.56.10 product version=1.6.0
INFO  [xen.resource.CitrixResourceBase] (1927634425@qtp-938693467-3:)
Private Network is Pool-wide network associated with eth0 for host
192.168.56.10
INFO  [xen.resource.CitrixResourceBase] (1927634425@qtp-938693467-3:) Guest
Network is Pool-wide network associated with eth0 for host 192.168.56.10
INFO  [xen.resource.CitrixResourceBase] (1927634425@qtp-938693467-3:)
Public Network is Pool-wide network associated with eth0 for host
192.168.56.10
INFO  [xen.discoverer.XcpServerDiscoverer] (1927634425@qtp-938693467-3:)
Host: devcloud connected with hypervisor type: XenServer. Checking CIDR...
INFO  [agent.manager.DirectAgentAttache] (1927634425@qtp-938693467-3:)
StartupAnswer received 1 Interval = 60
*WARN  [xen.resource.CitrixResourceBase] (DirectAgent-1:) We cannot locate
/home/punith/Desktop/apache-cloudstack-4.2.0-src/client/target/cloud-client-ui-4.2.0/WEB-INF/classes/scripts/vm/hypervisor/xenserver/xcposs/../ovsgre
*
*WARN  [xen.resource.CitrixResourceBase] (DirectAgent-1:) We cannot locate
/home/punith/Desktop/apache-cloudstack-4.2.0-src/client/target/cloud-client-ui-4.2.0/WEB-INF/classes/scripts/vm/hypervisor/xenserver/xcposs/../../../../network/domr//ipassoc.sh
*
*WARN  [xen.resource.CitrixResourceBase] (DirectAgent-1:) We cannot locate
/home/punith/Desktop/apache-cloudstack-4.2.0-src/client/target/cloud-client-ui-4.2.0/WEB-INF/classes/scripts/vm/hypervisor/xenserver/xcposs/../../../../network/domr//getDomRVersion.sh
*
*WARN  [xen.resource.CitrixResourceBase] (DirectAgent-1:) We cannot locate
/home/punith/Desktop/apache-cloudstack-4.2.0-src/client/target/cloud-client-ui-4.2.0/WEB-INF/classes/scripts/vm/hypervisor/xenserver/xcposs/../../../../network/domr//getRouterStatus.sh
*
*WARN  [xen.resource.CitrixResourceBase] (DirectAgent-1:) We cannot locate
/home/punith/Desktop/apache-cloudstack-4.2.0-src/client/target/cloud-client-ui-4.2.0/WEB-INF/classes/scripts/vm/hypervisor/xenserver/xcposs/../../../../network/domr//networkUsage.sh
*
*WARN  [xen.resource.CitrixResourceBase] (DirectAgent-1:) We cannot locate
/home/punith/Desktop/apache-cloudstack-4.2.0-src/client/target/cloud-client-ui-4.2.0/WEB-INF/classes/scripts/vm/hypervisor/xenserver/xcposs/../../../../network/domr//l2tp_vpn.sh
*
*WARN  [xen.resource.CitrixResourceBase] (DirectAgent-1:) We cannot locate
/home/punith/Desktop/apache-cloudstack-4.2.0-src/client/target/cloud-client-ui-4.2.0/WEB-INF/classes/scripts/vm/hypervisor/xenserver/xcposs/../vhd-util
*
INFO  [network.security.SecurityGroupListener] (1927634425@qtp-938693467-3:)
Received a host startup notification
INFO  [network.security.SecurityGroupListener] (1927634425@qtp-938693467-3:)
Scheduled network rules cleanup, interval=2669
INFO  [network.security.SecurityGroupListener] (1927634425@qtp-938693467-3:)
Received a host startup notification
INFO  [cloud.resource.ResourceManagerImpl] (1927634425@qtp-938693467-3:)
server resources successfully discovered by XCP Agent
INFO  [datastore.lifecycle.CloudStackImageStoreLifeCycleImpl]
(1927634425@qtp-938693467-3:) Trying to add a new data store at
nfs://192.168.56.10/opt/storage/secondary to data center 1
INFO  [cloud.configuration.ConfigurationManagerImpl]
(1927634425@qtp-938693467-3:) No storage traffic type was specified by
admin, create default storage traffic on physical network 200 with same
configure of management traffic type
INFO  [storage.endpoint.DefaultEndPointSelector] (StatsCollector-3:) No
running ssvm is found, so command will be sent to LocalHostEndPoint
INFO  [cloud.secstorage.PremiumSecondaryStorageManagerImpl] (secstorage-1:)
No running secondary storage vms found in datacenter id=1, starting one
INFO  [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:) No
stopped secondary storage vm is available, need to allocate a new secondary
storage vm
INFO  [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:) No
stopped console proxy is available, need to allocate a new console proxy
INFO  [storage.volume.VolumeServiceImpl] (consoleproxy-1:) lock is acquired
for VMTemplateStoragePool 1
INFO  [storage.volume.VolumeServiceImpl] (consoleproxy-1:) releasing lock
for VMTemplateStoragePool 1
WARN  [utils.db.Merovingian2] (consoleproxy-1:) Was unable to find lock for
the key template_spool_ref1 and thread id 566503218
INFO  [storage.volume.VolumeServiceImpl] 

Re: CS 4.2.0 - Bug with ssvm - wrong capacity recognized

[Punith s]

i cannot access the console of SSVM ,
i ran a python ../marvin/marvin/deployDataCenter.py -i devcloud.cfg python
script to setup my basic zone.
but my virtual router shown 0 in the infrastructure even after restarting
my MS, any idea on why on getting my virtual router up ?? thanks.

regards,
punith s


On Thu, Oct 3, 2013 at 7:04 PM, Wei ZHOU  wrote:

> Can you run ssvm_check.sh in SSVM ?
> The template is in devcloud2, so it should because of an unknown
> misconfiguration.
>
>
> 2013/10/3 Punith s 
>
> > yea i'm also finding issue with the sysvm in devcloud at host only mode.
> > the tiny linux template is not getting downloaded even though the
> secondary
> > storage sys vm is shown running in the UI , but i can't access its
> console.
> > i even tried ssh into 192.168.56.10(xenbr0) and looked into
> >  root@devcloud:/opt/storage/secondary/template/tmpl/1/5# ls
> > ce5b212e-215a-3461-94fb-814a635b2215.vhd  template.properties where the
> > .vhd is present.
> >
> > i think The Secondary Storage VM is failing to  retrieve templates and
> make
> > the ready to use.
> > i looked into vmops.log and
> > found com.cloud.storage.download.DownloadListener.java doing this job ,
> > perhaps need to debug to know who is calling this.
> >
> >
> >
> >
> > On Thu, Oct 3, 2013 at 6:38 PM, benoit lair 
> wrote:
> >
> > > Okay, i changed the ip of the secondary storage server into the ip
> range
> > of
> > > the pod, it doesn't work better.
> > >
> > > I change now the network of the secondary storage server.
> > >
> > >
> > > 2013/10/3 benoit lair 
> > >
> > > > I thought i could have my secondary storage nfs server into the mgmt
> > cidr
> > > > ? It were okay with a cs 4.0.0
> > > >
> > > > So i need to move my secondary storage nfs server to the 10.32.0.0
> > cidr ?
> > > >
> > > >
> > > > 2013/10/3 Wei ZHOU 
> > > >
> > > >> I guess the root cause could be that your nfs server ip is not in
> > > storage
> > > >> network.
> > > >>
> > > >
> > > >
> > >
> >
>

Re: unable to get tiny linux template in ready state, DOWNLOAD ERROR

[Wei ZHOU]

The virtual router will be created when you deploy the first VM.


2013/10/3 Punith s 

> hi Wei ,
>
> i found no system virtual running in my infrastructure !! , i think this
> is the main reason behind this mess ??
> any suggestions to get a virtual router up? and should i be on static ip
> in my localhost ??thanks.
>
> regards,
> punith s
>
>
>
>
>
>
> On Thu, Oct 3, 2013 at 6:47 PM, Punith s  wrote:
>
>> i have been running in the mode, till now it was just fine until my
>> system crashed , so i got the new 4.2.0 release and started building on .
>> previously i had posted which devcloud mode is best suited for
>> development , you can refer this link
>> https://mail.google.com/mail/#sent/1411177bb5a650a2
>>
>> the problem here is , i can see the system vm's shown as running in the
>> UI , but i cannot access their console from the MS, they are like half live
>> half dead .
>> and secondary system vm is failing to download the template(tiny linux)
>> since the agent is shown null in the UI.
>>
>> i shall try to register a new template or iso from a remote web server.
>>
>> regards,
>> punith s
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Thu, Oct 3, 2013 at 6:28 PM
>>
>> , Wei ZHOU  wrote:
>>
>>> The host (management server ip) in global setting is fixed as
>>> 192.168.56.1
>>> in tools/devcloud/devcloud.sql
>>> Maybe you need to change it.
>>> I use another virtualbox instance with ip 192.168.56.2 as the management
>>> server, then I change "host" in global setting to 192.168.56.2
>>>
>>>
>>> 2013/10/3 Punith s 
>>>
>>> > no i'm using devcloud2 as the HOST ONLY MODE(xen) but i'm building the
>>> MS
>>> > in my localhost which is a DHCP(10.11.1.135)
>>> > the devcloud's ip is 198.168.56.1.
>>> >
>>> > i'm building with ref's to this link
>>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/DevCloud
>>> >
>>> >
>>> >
>>> >
>>> > On Thu, Oct 3, 2013 at 5:57 PM, Wei ZHOU 
>>> wrote:
>>> >
>>> > > Is the ip of your management-server 192.168.56.1?
>>> > >
>>> > >
>>> > > 2013/10/3 Punith s 
>>> > >
>>> > > > hi wei ,
>>> > > > thanks for the reply but
>>> > > >
>>> > > > like i said i,m using devcloud2 as my host and building my basic
>>> > > > infrastructure by running the python script
>>> > > > python ../marvin/marvin/deployDataCenter.py -i devcloud.cfg which
>>> > > compised
>>> > > > the package containing sysvm's and tiny linux templates .
>>> > > >
>>> > > > all the system vm's are shown running the the MS , but the agent
>>> state
>>> > is
>>> > > > shown null(-),
>>> > > >
>>> > > > while creating the instance , i cannot add the default tiny linux
>>> > > template
>>> > > > since it is not in the ready state(not yet downloaded by the sec
>>> stor
>>> > vm)
>>> > > > even though the systemvm's are running, is this due to agent state
>>> > shown
>>> > > > null ??
>>> > > >
>>> > > > regards,
>>> > > > punith s
>>> > > > cloudbyte
>>> > > >
>>> > > >
>>> > > >
>>> > > >
>>> > > > On Thu, Oct 3, 2013 at 5:00 PM, Wei ZHOU 
>>> > wrote:
>>> > > >
>>> > > >> Please follow the link to install new systemvm template:
>>> > > >>
>>> > > >>
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/How+to+build+CloudStack#HowtobuildCloudStack-InstalltheSystemVMTemplate
>>> > > >>
>>> > > >> PS: The download url changed to
>>> > > http://download.cloud.com/templates/4.2/
>>> > > >> systemvmtemplate-2013-07-12-master-xen.vhd.bz2<
>>> > >
>>> >
>>> http://download.cloud.com/templates/4.2/systemvmtemplate-2013-07-12-master-xen.vhd.bz2
>>> > > >
>>> > > >> .
>>> > > >>
>>> > > >>
>>> > > >> 2013/10/3 Punith s 
>>> > > >>
>>> > > >> > hi everyone ,
>>> > > >> >
>>> > > >> > i just build the cloudstack realese 4.2.0 on my devcloud2 to
>>> test my
>>> > > >> plugin
>>> > > >> > ,
>>> > > >> >
>>> > > >> > but i'm unable to create the tiny linux instance here after
>>> running
>>> > > the
>>> > > >> >  python ../marvin/marvin/deployDataCenter.py -i devcloud.cfg
>>> > > >> >
>>> > > >> > the template is not getting ready , any suggestions ??
>>> > > >> >
>>> > > >> > the vmops.log file shows the following ,
>>> > > >> >
>>> > > >> > 2013-10-03 16:12:30,599 DEBUG [cloud.api.ApiServlet]
>>> > > >> > (1995997711@qtp-1547603062-3:null) ===END===  0:0:0:0:0:0:0:1
>>> --
>>> > GET
>>> > > >> >
>>> > > >> >
>>> > > >>
>>> > >
>>> >
>>>  
>>> command=listTemplates&templatefilter=selfexecutable&zoneid=a2ed9972-3f9a-4bcc-b888-d68114a42b7a&response=json&sessionkey=yCeA7XxmdJZEF47qnBCzXPrV7Go%3D&_=1380796950532
>>> > > >> > 2013-10-03 16:12:30,923 DEBUG [agent.manager.DirectAgentAttache]
>>> > > >> > (DirectAgent-7:null) Seq 1-349175812: Response Received:
>>> > > >> > 2013-10-03 16:12:30,926 DEBUG [agent.transport.Request]
>>> > > >> > (DirectAgent-7:null) Seq 1-349175812: Processing:  { Ans: ,
>>> MgmtId:
>>> > > >> > 268749071320359, via: 1, Ver: v1, Flags: 10,
>>> > > >> >
>>> > > >> >
>>> > > >>
>>> > >
>>> >
>>> [{"com.cloud.agent.api.ClusterSyncAnswer":{"_clusterId":1,"_newStates":{},"_isExecuted":false,"r

Re: [URGENT] Unable to stop/start system VMs after upgrade to 4.2.0

[Venkata SwamyBabu Budumuru]

What is the hypervisor you are using?

On 03/10/13 6:14 PM, "David Ortiz"  wrote:

>If you manually change the vm status in the database would it let you
>destroy it then?
>
>> Date: Thu, 3 Oct 2013 20:22:33 +0800
>> Subject: Re: [URGENT] Unable to stop/start system VMs after upgrade to
>>4.2.0
>> From: in...@sg.or.id
>> To: dev@cloudstack.apache.org; us...@cloudstack.apache.org
>> 
>> Dear all,
>> 
>> I managed to fix the VRs issue by destroying the routers and restarting
>>the
>> networks associated to the routers.
>> 
>> Only left is the CPVM, I am not able to stop/destroy it. Any advise?
>> 
>> Looking forward to your reply, thank you.
>> 
>> Cheers.
>> 
>> 
>> 
>> On Thu, Oct 3, 2013 at 8:16 PM, Indra Pramana  wrote:
>> 
>> > Dear all,
>> >
>> > I am now in the progress of upgrading my CloudStack from 4.1.1 to
>>4.2.0.
>> > Running cloudstack-sysvmadm can restart the SSVM, but failed to
>>restart the
>> > CPVM and all the VRs.
>> >
>> > 
>> > [1] 3152
>> > root@cs-mgmt-01:~/backup/20131003# tail -f sysvm.log
>> > nohup: ignoring input
>> > /usr/bin/cloudstack-sysvmadm: line 21: /etc/rc.d/init.d/functions: No
>>such
>> > file or directory
>> >
>> > Stopping and starting 1 secondary storage vm(s)...
>> > Done stopping and starting secondary storage vm(s)
>> >
>> > Stopping and starting 1 console proxy vm(s)...
>> > ERROR: Failed to stop console proxy vm with id 1980
>> >
>> > Done stopping and starting console proxy vm(s) .
>> >
>> > Stopping and starting 0 running routing vm(s)...
>> > 
>> >
>> > SSVM seems OK, I have run health check script and everything is in
>>order.
>> >
>> > VM state of CPVM is running, but state is disconnected. I can't stop
>>or
>> > destroy the CPVM.
>> >
>> > All the VRs are running now but I am not too sure if they are already
>> > using the new 4.2 template or still using the old template, since it
>>seems
>> > the sysvmadm script was unable to detect the virtual routers.
>> >
>> > Anyone can help?
>> >
>> > Thank you.
>> >
> 

Re: unable to get tiny linux template in ready state, DOWNLOAD ERROR

[Punith s]

hi Wei ,

i found no system virtual running in my infrastructure !! , i think this is
the main reason behind this mess ??
any suggestions to get a virtual router up? and should i be on static ip in
my localhost ??thanks.

regards,
punith s






On Thu, Oct 3, 2013 at 6:47 PM, Punith s  wrote:

> i have been running in the mode, till now it was just fine until my system
> crashed , so i got the new 4.2.0 release and started building on .
> previously i had posted which devcloud mode is best suited for development
> , you can refer this link
> https://mail.google.com/mail/#sent/1411177bb5a650a2
>
> the problem here is , i can see the system vm's shown as running in the UI
> , but i cannot access their console from the MS, they are like half live
> half dead .
> and secondary system vm is failing to download the template(tiny linux)
> since the agent is shown null in the UI.
>
> i shall try to register a new template or iso from a remote web server.
>
> regards,
> punith s
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, Oct 3, 2013 at 6:28 PM
>
> , Wei ZHOU  wrote:
>
>> The host (management server ip) in global setting is fixed as 192.168.56.1
>> in tools/devcloud/devcloud.sql
>> Maybe you need to change it.
>> I use another virtualbox instance with ip 192.168.56.2 as the management
>> server, then I change "host" in global setting to 192.168.56.2
>>
>>
>> 2013/10/3 Punith s 
>>
>> > no i'm using devcloud2 as the HOST ONLY MODE(xen) but i'm building the
>> MS
>> > in my localhost which is a DHCP(10.11.1.135)
>> > the devcloud's ip is 198.168.56.1.
>> >
>> > i'm building with ref's to this link
>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/DevCloud
>> >
>> >
>> >
>> >
>> > On Thu, Oct 3, 2013 at 5:57 PM, Wei ZHOU  wrote:
>> >
>> > > Is the ip of your management-server 192.168.56.1?
>> > >
>> > >
>> > > 2013/10/3 Punith s 
>> > >
>> > > > hi wei ,
>> > > > thanks for the reply but
>> > > >
>> > > > like i said i,m using devcloud2 as my host and building my basic
>> > > > infrastructure by running the python script
>> > > > python ../marvin/marvin/deployDataCenter.py -i devcloud.cfg which
>> > > compised
>> > > > the package containing sysvm's and tiny linux templates .
>> > > >
>> > > > all the system vm's are shown running the the MS , but the agent
>> state
>> > is
>> > > > shown null(-),
>> > > >
>> > > > while creating the instance , i cannot add the default tiny linux
>> > > template
>> > > > since it is not in the ready state(not yet downloaded by the sec
>> stor
>> > vm)
>> > > > even though the systemvm's are running, is this due to agent state
>> > shown
>> > > > null ??
>> > > >
>> > > > regards,
>> > > > punith s
>> > > > cloudbyte
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On Thu, Oct 3, 2013 at 5:00 PM, Wei ZHOU 
>> > wrote:
>> > > >
>> > > >> Please follow the link to install new systemvm template:
>> > > >>
>> > > >>
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/How+to+build+CloudStack#HowtobuildCloudStack-InstalltheSystemVMTemplate
>> > > >>
>> > > >> PS: The download url changed to
>> > > http://download.cloud.com/templates/4.2/
>> > > >> systemvmtemplate-2013-07-12-master-xen.vhd.bz2<
>> > >
>> >
>> http://download.cloud.com/templates/4.2/systemvmtemplate-2013-07-12-master-xen.vhd.bz2
>> > > >
>> > > >> .
>> > > >>
>> > > >>
>> > > >> 2013/10/3 Punith s 
>> > > >>
>> > > >> > hi everyone ,
>> > > >> >
>> > > >> > i just build the cloudstack realese 4.2.0 on my devcloud2 to
>> test my
>> > > >> plugin
>> > > >> > ,
>> > > >> >
>> > > >> > but i'm unable to create the tiny linux instance here after
>> running
>> > > the
>> > > >> >  python ../marvin/marvin/deployDataCenter.py -i devcloud.cfg
>> > > >> >
>> > > >> > the template is not getting ready , any suggestions ??
>> > > >> >
>> > > >> > the vmops.log file shows the following ,
>> > > >> >
>> > > >> > 2013-10-03 16:12:30,599 DEBUG [cloud.api.ApiServlet]
>> > > >> > (1995997711@qtp-1547603062-3:null) ===END===  0:0:0:0:0:0:0:1 --
>> > GET
>> > > >> >
>> > > >> >
>> > > >>
>> > >
>> >
>>  
>> command=listTemplates&templatefilter=selfexecutable&zoneid=a2ed9972-3f9a-4bcc-b888-d68114a42b7a&response=json&sessionkey=yCeA7XxmdJZEF47qnBCzXPrV7Go%3D&_=1380796950532
>> > > >> > 2013-10-03 16:12:30,923 DEBUG [agent.manager.DirectAgentAttache]
>> > > >> > (DirectAgent-7:null) Seq 1-349175812: Response Received:
>> > > >> > 2013-10-03 16:12:30,926 DEBUG [agent.transport.Request]
>> > > >> > (DirectAgent-7:null) Seq 1-349175812: Processing:  { Ans: ,
>> MgmtId:
>> > > >> > 268749071320359, via: 1, Ver: v1, Flags: 10,
>> > > >> >
>> > > >> >
>> > > >>
>> > >
>> >
>> [{"com.cloud.agent.api.ClusterSyncAnswer":{"_clusterId":1,"_newStates":{},"_isExecuted":false,"result":true,"wait":0}}]
>> > > >> > }
>> > > >> > 2013-10-03 16:12:31,140 DEBUG [storage.download.DownloadListener]
>> > > >> > (Timer-5:null) timeout: transitioning to download error state,
>> > > >> > currstate=NOT_DOWNLOADED, TEMPLATE: 5 at host 0
>> > > >> > 2013-10-03