Re: Cloudstack git repositories

[prasanna]

On 21 March 2013 12:39, Sateesh Chodapuneedi
 wrote:
> Unable to access Cloudstack git repositories.
> Seems infra is migrating our git repos off incubator.

Yes - this and other infra pieces are moving, the website for example
has moved. This was communicated on the IRC meeting from y'day.

Re: [ACS41] Blockers for tomorrow's release

[prasanna]

On 22 March 2013 02:57, Sudha Ponnaganti  wrote:
> Below are the issues that are blocking tomorrow's release. Pl  take a look at 
> these. I have included Developers in the TO list, I have removed bare metal 
> related issues from this list.
> Prasanna Santhanam CLOUDSTACK-1736  Ubuntu 12.04 
> cloud-setup-management Failed to configure CloudStack Management Server

Sorry missed this one. Wido is the right person to check on the deb
packaging issue. I've assigned it to him to take a look.

Re: [ACS41] Review request for the fix - failed to download default template

[prasanna]

On 22 March 2013 02:45, Sudha Ponnaganti  wrote:
> Hi,
>
> Can anyone pick up the review for defect 
> https://issues.apache.org/jira/browse/CLOUDSTACK-1252
> We need this fix today
>
There is a lot of discussion on the JIRA ticket that I couldn't
follow. It seems this is a non-oss packaging issue? Someone pass me
the review request please.

Re: [PROPOSAL] BVT for CloudStack checkins

[prasanna]

On 23 March 2013 01:49, Chip Childers  wrote:
>
> OK - Next errors:
>
> mvn -Pdeveloper,marvin -Dmarvin.config=setup/dev/advanced.cfg -pl 
> :cloud-marvin integration-test
> [INFO] Scanning for projects...
> [INFO]
> [INFO] 
> 
> [INFO] Building Apache CloudStack marvin 4.2.0-SNAPSHOT
> [INFO] 
> 
> [INFO]
> [INFO] --- exec-maven-plugin:1.2.1:exec (generate-sources) @ cloud-marvin ---
> [INFO]
> [INFO] --- maven-remote-resources-plugin:1.3:process (default) @ cloud-marvin 
> ---
> [INFO]
> [INFO] --- maven-site-plugin:3.1:attach-descriptor (attach-descriptor) @ 
> cloud-marvin ---
> [INFO]
> [INFO] --- exec-maven-plugin:1.2.1:exec (pre-integration-test) @ cloud-marvin 
> ---
> [INFO]
> [INFO] --- exec-maven-plugin:1.2.1:exec (integration-test) @ cloud-marvin ---
> Usage: nosetests [options]
>
> nosetests: error: no such option: --with-marvin
> [INFO] 
> 
> [INFO] BUILD FAILURE

Awesome! Thanks for trying Chip. I think you may have forgotten to
register the marvin-nose plugin with nose

$ cd tools/marvin/marvin
$ pip install .

I'll combine the two pip install steps so it's only one Marvin installation.

Re: Problem adding Host inside a Cluster

[prasanna]

On 22 March 2013 21:14, Kausal Malladi  wrote:
> Hi,
> Please find the logs at http://paste.cloudstack.org/KHPO/
>
Within devcloud can you check if xapi is functional? Try restarting
the service xcp-xapi?

Re: [PROPOSAL] BVT for CloudStack checkins

[prasanna]

On 23 March 2013 17:53, prasanna  wrote:
> On 23 March 2013 01:49, Chip Childers  wrote:
>>
>> OK - Next errors:
>>
>> mvn -Pdeveloper,marvin -Dmarvin.config=setup/dev/advanced.cfg -pl 
>> :cloud-marvin integration-test
>> [INFO] Scanning for projects...
>> [INFO]
>> [INFO] 
>> 
>> [INFO] Building Apache CloudStack marvin 4.2.0-SNAPSHOT
>> [INFO] 
>> 
>> [INFO]
>> [INFO] --- exec-maven-plugin:1.2.1:exec (generate-sources) @ cloud-marvin ---
>> [INFO]
>> [INFO] --- maven-remote-resources-plugin:1.3:process (default) @ 
>> cloud-marvin ---
>> [INFO]
>> [INFO] --- maven-site-plugin:3.1:attach-descriptor (attach-descriptor) @ 
>> cloud-marvin ---
>> [INFO]
>> [INFO] --- exec-maven-plugin:1.2.1:exec (pre-integration-test) @ 
>> cloud-marvin ---
>> [INFO]
>> [INFO] --- exec-maven-plugin:1.2.1:exec (integration-test) @ cloud-marvin ---
>> Usage: nosetests [options]
>>
>> nosetests: error: no such option: --with-marvin
>> [INFO] 
>> 
>> [INFO] BUILD FAILURE
>
> Awesome! Thanks for trying Chip. I think you may have forgotten to
> register the marvin-nose plugin with nose
>
> $ cd tools/marvin/marvin
> $ pip install .
>
> I'll combine the two pip install steps so it's only one Marvin installation.

This is done. You don't have to register marvin with nose anymore.
It's done as part of marvin install now. I've corrected the wiki(s)
with this information.

Also a good time to enhance marvin to use ApiDiscovery like
cloudmonkey does so that new APIs included can be tested along with
this mvn+marvin integration.

Re: [DISCUSS] NFS lockdown for SSVM

[prasanna]

On 22 March 2013 04:01, Musayev, Ilya  wrote:
> Looks like we have no port lockdown for NFS on SSVM, probably not ideal when 
> you want firewall and all port accountability.
>
> Before I cook something up and inject it into systemvm.iso and submit a 
> patch, has this been discussed?
>
> I want to get the community feel on this.  I need to justify every single 
> open port to auditors and having random ports allocated by portmapper is not 
> going to pass the audits.

Related? https://issues.apache.org/jira/browse/CLOUDSTACK-244. Needs
a resolution either ways.

Re: [ACS41] Status

[prasanna]

On 23 March 2013 00:19, Chip Childers  wrote:
> Hi all,
>
> I'm not going to cut the RC today, due to the outstanding bugs that we
> are still working on.
>
> Let's keep pushing to get things wrapped up.  I've been trying to get
> review requests *with my name on them* and / or "Patch Request" emails
> done as soon as I can... keep them coming!
>
> -chip

I went through all the Major bugs to check if we possibly mis-triaged
an important bug. The following seem important to fix but I'll let the
owners take a call:

https://issues.apache.org/jira/browse/CLOUDSTACK-155
https://issues.apache.org/jira/browse/CLOUDSTACK-431
https://issues.apache.org/jira/browse/CLOUDSTACK-1006

All of them are KVM related.

Re: NetScaler lib/sdk jars are apache licensed

[prasanna]

On 5 March 2013 00:14, Vijay Venkatachalam
 wrote:
> Thanks for the tip David on the signatures upload!
>
> I was a able to successfully release the NetScaler's Nitro jar file.
>
> Once I complete the release of NetScaler SDX's jar file as well, the stage 
> will be set to make the NetScaler's Adapter part of OSS.
>
> Will summarize and send a separate mail once NetScaler SDX's jar is also 
> uploaded.
>
> -Vijay
>

Hey VIjay - Did this get done? Are the jars available on maven central
and/or any other repo?

Re: Building nonoss

[prasanna]

Is it possible you have an older netscaler-sdk? Can you replace the
one you have with the one at:

http://community.citrix.com/download/attachments/37847122/cloud-netscaler-jars.zip

On 25 March 2013 10:38, Mike Tutkowski  wrote:
> Hey Rajesh,
>
> I ran the following:
>
> mvn clean install -Dnonoss -X
>
> here was the output (thanks!):
>
> [INFO] Apache CloudStack Plugin - Dns Notifier Example ... SUCCESS [0.906s]
> [INFO] Apache CloudStack Plugin - Storage Image S3 ... SUCCESS [0.889s]
> [INFO] Apache CloudStack Plugin - Storage Volume solidfire  SUCCESS [0.631s]
> [INFO] Apache CloudStack Plugin - Storage Volume default provider  SUCCESS
> [0.601s]
> [INFO] Apache CloudStack Plugin - SNMP Alerts  SUCCESS [1.340s]
> [INFO] Apache CloudStack Plugin - NetApp File System . SUCCESS [0.915s]
> [INFO] Apache CloudStack Plugin - F5 . SUCCESS [0.872s]
> [INFO] Apache CloudStack Plugin - Network Netscaler .. FAILURE [2.345s]
> [INFO] Apache CloudStack Plugin - Juniper SRX  SKIPPED
> [INFO] Apache CloudStack VMware Base . SKIPPED
> [INFO] Apache CloudStack Plugin - Hypervisor VMware .. SKIPPED
> [INFO] Apache CloudStack SystemVM Patches  SKIPPED
> [INFO] Apache CloudStack Cloud Services .. SKIPPED
> [INFO] Apache CloudStack Console Proxy Service ... SKIPPED
> [INFO] Apache CloudStack Console Proxy Plugin  SKIPPED
> [INFO] Apache CloudStack Console Proxy ... SKIPPED
> [INFO] Apache CloudStack Test  SKIPPED
> [INFO] Apache CloudStack Client UI ... SKIPPED
> [INFO]
> 
> [INFO] BUILD FAILURE
> [INFO]
> 
> [INFO] Total time: 1:36.937s
> [INFO] Finished at: Sun Mar 24 23:02:58 MDT 2013
> [INFO] Final Memory: 35M/95M
> [INFO]
> 
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile
> (default-compile) on project cloud-plugin-network-netscaler: Compilation
> failure: Compilation failure:
> [ERROR]
> /Users/mtutkowski/Documents/CloudStack/src/incubator-cloudstack/plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java:[32,59]
> package com.citrix.netscaler.nitro.resource.config.autoscale does not exist
> [ERROR]
> /Users/mtutkowski/Documents/CloudStack/src/incubator-cloudstack/plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java:[33,59]
> package com.citrix.netscaler.nitro.resource.config.autoscale does not exist
> [ERROR]
> /Users/mtutkowski/Documents/CloudStack/src/incubator-cloudstack/plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java:[41,52]
> cannot find symbol
> [ERROR] symbol  : class lbmonitor_metric_binding
> [ERROR] location: package com.citrix.netscaler.nitro.resource.config.lb
> [ERROR]
> /Users/mtutkowski/Documents/CloudStack/src/incubator-cloudstack/plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java:[53,52]
> cannot find symbol
> [ERROR] symbol  : class nstimer
> [ERROR] location: package com.citrix.netscaler.nitro.resource.config.ns
> [ERROR]
> /Users/mtutkowski/Documents/CloudStack/src/incubator-cloudstack/plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java:[54,52]
> cannot find symbol
> [ERROR] symbol  : class nstimer_autoscalepolicy_binding
> [ERROR] location: package com.citrix.netscaler.nitro.resource.config.ns
> [ERROR]
> /Users/mtutkowski/Documents/CloudStack/src/incubator-cloudstack/plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java:[1686,30]
> cannot find symbol
> [ERROR] symbol  : method set_monstate(java.lang.String)
> [ERROR] location: class
> com.citrix.netscaler.nitro.resource.config.basic.service_lbmonitor_binding
> [ERROR]
> /Users/mtutkowski/Documents/CloudStack/src/incubator-cloudstack/plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java:[1688,90]
> cannot find symbol
> [ERROR] symbol  : method
> add(com.citrix.netscaler.nitro.service.nitro_service,com.citrix.netscaler.nitro.resource.config.basic.service_lbmonitor_binding)
> [ERROR] location: class
> com.citrix.netscaler.nitro.resource.config.basic.service_lbmonitor_binding
> [ERROR]
> /Users/mtutkowski/Documents/CloudStack/src/incubator-cloudstack/plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java:[1713,41]
> cannot find symbol
> [ERROR] symbol  : method
> delete(com.citrix.netscaler.nitro.service.nitro_service,com.citrix.netscaler.nitro.resource.config.basic.service_lbmonitor_binding)
> [ERROR] location: class
> com.citrix.netscaler.nitro.resource.config.basic.service_lbmonitor_binding
> [ERROR]
> /Users/m

Re: New committer: Ahmad Emneina

[prasanna]

On 25 March 2013 05:05, Joe Brockmeier  wrote:
> Hi all,
>
> The Project Management Committee (PMC) for Apache CloudStack
> has asked Ahmad Emneina to become a committer and we are pleased to
> announce that he has accepted.
>
> Being a committer allows many contributors to contribute more
> autonomously. For developers, it makes it easier to submit changes and
> eliminates the need to have contributions reviewed via the patch
> submission process. Whether contributions are development-related or
> otherwise, it is a recognition of a contributor's participation in the
> project and commitment to the project and the Apache Way.
>
> Please join me in congratulating Ahmad!
>
> Joe Brockmeier, on behalf of the CloudStack PPMC.
>
> Best,
>

Congratulations Ahmad!

Re: NetScaler lib/sdk jars are apache licensed

[prasanna]

On 25 March 2013 10:49, Vijay Venkatachalam
 wrote:
> No, there are 2 jars to this, only one of them is done. I haven’t got time to 
> get the other one done. It should get done in another 10-15 days.
>

Thanks, when that's done I'll move netscaler into oss profiles so it
builds by default.

Re: New committer: Geoff Higginbottom

[prasanna]

On 25 March 2013 20:55, Chip Childers  wrote:
> The Project Management Committee (PMC) for Apache CloudStack
> has asked Geoff Higginbottom to become a committer and we are pleased to
> announce that he has accepted.
>
> Being a committer allows many contributors to contribute more
> autonomously. For developers, it makes it easier to submit changes and
> eliminates the need to have contributions reviewed via the patch
> submission process. Whether contributions are development-related or
> otherwise, it is a recognition of a contributor's participation in the
> project and commitment to the project and the Apache Way.
>
> Please join me in congratulating Geoff!
>
> -chip
> on behalf of the CloudStack PMC

Congratulations Geoff!

Re: New committer: Hiroaki Kawai

[prasanna]

On 25 March 2013 19:09, David Nalley  wrote:
> The Project Management Committee (PMC) for Apache CloudStack has asked
> Hiroaki Kawai to become a committer and we are pleased to announce
> that they have accepted.
>
> Being a committer allows many contributors to contribute more
> autonomously. For developers, it makes it easier to submit changes and
> eliminates the need to have contributions reviewed via the patch
> submission process. Whether contributions are development-related or
> otherwise, it is a recognition of a contributor's participation in the
> project and commitment to the project and the Apache Way.
>
> Please join me in congratulating Kawai-san
>
> --David
> on behalf of the CloudStack PMC

Congratulations!

Re: [DISCUSS] Hyper-V Plugin & Microsoft Compiler & IP Clearance

[prasanna]

On 27 March 2013 13:51, Sebastien Goasguen  wrote:
>
> On Mar 26, 2013, at 7:29 PM, Alex Huang  wrote:
>
>>> and the C# part is for the ServerResource, right ?
>>>
>>> I won't code this, but it would be nice to minimize languages used in
>>> CloudStack and try to be consistent across hypervisors (as much as 
>>> possible).
>>> Introducing another language and some framework that may or may not
>>> have proprietary modules/dependencies will certainly complicate the build
>>> and the testing, and maybe even the licensing.
>>>
>>
>> Actually the ServerResource part was always planned to be implemented in 
>> other languages.  Java is a terrible platform to execute scripts for 
>> example.  That's why communication between management server and the server 
>> resource was done in JSON.
>>
>> --Alex
>
> Right, and I believe that people agree that the KVM agent needs to be 
> re-written in non-Java. Could we agree on how to do this, and do the same for 
> Hyper-V ?
>


Is Iron Python a possibility? Should be supported by the .NET CLR.

Re: 4.1 build and network problems, fixed (CS-1773 - review #10150)

[prasanna]

On 27 March 2013 14:53, Funs Kessen  wrote:
> Hi Guys,
>
> After doing a fresh checkout yesterday I noticed that 4.1 would not build due 
> to the client/pom.xml still containing UCS (so I commented it out) and the 
> "plugins/network-elements/dns-notifier/resources/components-example.xml" 
> still had addDefaultBaremetalProvidersToPhysicalNetwork which will get called 
> but is missing when a VM is to be deployed, and thus breaks deploying.
>
> I've made some change so it works again, I've put them in for review under: 
> #10150.
>
> Cheers,
>
> Funs

Applied! thanks

Re: [PROPOSAL] BVT for CloudStack checkins

[prasanna]

On 25 March 2013 18:44, Prasanna Santhanam  wrote:
> I'm working to include a couple more things:
>
> a. Allow a developer to run his/her own integration test written in
> marvin.
> b. new APIs to be synced automatically without marvin reinstallation

In doing b. I realized I had to improve the response given out by the
API discovery plugin. What I'm essential trying to do is allow a
developer to sync the APIs that he/she might have introduced/modified
and run integration-tests against those newly introduced APIs. Marvin
will regenerate its libraries on sync and the mvn integration-test can
be run against such a server. Since many of our APIs return embedded
entities I've altered API discovery to include parent+child (if any)
relationships. The new response for a sample API returned by the
listApis command is shown at [1]

[1] https://gist.github.com/vogxn/5255883

I've put this change in the bvt branch and tested that cloudmonkey was
unaffected by it.

Rohit - since you wrote API discovery would you mind taking a look at
the change?

Thanks,

Re: [QuickCloud] zero to cloud in less than a minute

[prasanna]

This is a significant step no doubt!

One comment - I think the deployDataCenter.py should move out of the
 phase of the devcloud/pom.xml. When it runs the install
phase which includes the package phase the build will fail with the
following  stack:

https://gist.github.com/vogxn/5264956

In the bvt branch I work around this problem by putting the deploy
into the integration-test phase and into a profile that takes a
property which is the config file of marvin.

On 28 March 2013 05:49, Chiradeep Vittal  wrote:
> This should now be usable for production use as well
> Follow instructions here
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/QuickCloud#QuickClou
> d-howto2
>
> From:  Ahmad Emneina 
> Reply-To:  "aemne...@gmail.com" 
> Date:  Wednesday, March 27, 2013 10:34 AM
> To:  Chiradeep Vittal 
> Cc:  "dev@cloudstack.apache.org" 
> Subject:  Re: [QuickCloud] zero to cloud in less than a minute
>
>
> +1
> so awesome!
>
>
> On Wed, Mar 27, 2013 at 10:17 AM, Chiradeep Vittal
>  wrote:
>
> Yes (actually that's what the instructions say)
>
> On 3/26/13 10:46 PM, "Ahmad Emneina"  wrote:
>
>>would someone be able to fire up all those services (say for a basic zone)
>>on one host?
>>
>>
>>On Tue, Mar 26, 2013 at 10:37 PM, Chiradeep Vittal <
>>chiradeep.vit...@citrix.com> wrote:
>>
>>> Following the discussion [1], we have QuickCloud in a rough-but-ready
>>> state for developers to try out
>>> Instructions  for developers to try it out with DevCloud2 here:
>>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/QuickCloud
> 
>>>
>>> For now only Mac / Unix developers can use this workflow since NFS
>>>mounts
>>> are required.
>>>
>>> [1] http://markmail.org/thread/ajw7b6arhluqcuv2
>>>
>
>
>
>
>
>
>
>

Re: New Committer: Ilya Musayev

[prasanna]

On 29 March 2013 00:06, Chip Childers  wrote:
> The Project Management Committee (PMC) for Apache CloudStack
> has asked Ilya Musayev to become a committer and we are pleased to
> announce that they have accepted.
>
> Being a committer allows many contributors to contribute more
> autonomously. For developers, it makes it easier to submit changes and
> eliminates the need to have contributions reviewed via the patch
> submission process. Whether contributions are development-related or
> otherwise, it is a recognition of a contributor's participation in the
> project and commitment to the project and the Apache Way.
>
> Please join me in congratulating Ilya!
>
> --Chip Childers
> on behalf of the CloudStack PMC

Congratulations Ilya!

Re: New Committer: Animesh Chaturvedi

[prasanna]

Congratulations Animesh!

On 29 March 2013 01:27, Alex Huang  wrote:
> The Podling Project Management Committee (PPMC) for Apache CloudStack has 
> asked Animesh Chaturvedi to become a committer and we are pleased to announce 
> that they have accepted.
>
>
>
> Being a committer enables easier contribution to the project since there is 
> no need to go via the patch submission process. This should enable better 
> productivity.
>
>
>
> Please join me in congratulating Sateesh!
>
>
>
> --Alex
>
> on behalf of the CloudStack PPMC
>

Re: [ACS41][Patch Request]

[prasanna]

I've applied this on 4.1 so we can test upgrades -
260f6f7e5d7adae9da0d62c530e5279aeffb6f79



On 29 March 2013 07:37, Marcus Sorensen  wrote:
> commit d6e11f25ace4696ec39063f86342cd8a94bd96ee
> Author: Marcus Sorensen 
> Date:   Thu Mar 28 20:03:28 2013 -0600
>
> CLOUDSTACK-1844 - fix typo in agent pre-script to allow copy of old 
> configs
> to new config location
>
> Reported-by: Sangeetha Hariharan (JIRA)
> Signed-off-by: Marcus Sorensen  1364522608 -0600

Re: Can same Xen Server Host be used for different Management Servers?

[prasanna]

On 29 March 2013 11:11, Kausal Malladi  wrote:
> Hi,
> As part of development, I have configured a XenServer host with
> Management Server built from CloudStack 3.0 source code and have
> changed few things (resource allocation algorithm) in CloudStack 4.0
> source code (for my experiment).
> Can I use the same host for testing my cloud with management server
> built from modified CloudStack 4.0 source code, or should I create a
> new host?
>
> In short, can a XenServer host be switched between two or more
> management servers? I think giving same IP to both management servers
> and using one at a time should be okay. Correct me if I am wrong.
>

The xenserver host you add to cloudstack should be fresh. It shouldn't
have any previously running VMs that you may have deployed using the
older source. Why not use devcloud for this purpose? You can restore
to a base snapshot as you please. DevCloud isn't very different from a
real physical  xenserver.

Re: Can same Xen Server Host be used for different Management Servers?

[prasanna]

On 29 March 2013 11:37, Kausal Malladi  wrote:
> I tried that, but on my 3GB machine inside DevCloud only SSVM is
> getting started and not ConsoleProxy VM because of Insufficient Server
> Capacity Exception. Also I am not able to create a new Virtual Machine
> as template is not getting downloaded.
> My experiment needs at least 2 VMs to be created in a host on my cloud
> which is not happening inside DevCloud.

Hopefully - this will be resolved after the 'quickcloud' branch merges
into master.

>
> Also, Sebastien once suggested that DevCloud2.0 and ACS 4.0.0 won't work.
>

Sebastien is right.  Devcloud2 is for 4.1 onwards

> Do you have any suggestions in this regard?

You will need to create a new XenServer setup with 4.0 if you don't
want to affect the 3.0 setup you have.

Re: Feature Request: Password Reset / Change UI

[prasanna]

On 29 March 2013 12:49, Maurice Lawler  wrote:
> Hello Gang,
>
> I was wondering short of rendering this with the API. Is there plans to allow 
> end users on the CS UI to be permitted to easily change their login password ?
>
> I noticed at this time I (being the admin) can change them, but someone was 
> asking me; if this would be implemented in future updates.
>
> Thank you for your time.
>
> Maurice

There were several requests and JIRA tickets for this. I vaguely
recall seeing a patch on reviewboard as well.

Re: Feature Request: Password Reset / Change UI

[prasanna]

On 29 March 2013 13:39, prasanna  wrote:
> On 29 March 2013 12:49, Maurice Lawler  wrote:
>> Hello Gang,
>>
>> I was wondering short of rendering this with the API. Is there plans to 
>> allow end users on the CS UI to be permitted to easily change their login 
>> password ?
>>
>> I noticed at this time I (being the admin) can change them, but someone was 
>> asking me; if this would be implemented in future updates.
>>
>> Thank you for your time.
>>
>> Maurice
>
> There were several requests and JIRA tickets for this. I vaguely
> recall seeing a patch on reviewboard as well.

See https://issues.apache.org/jira/browse/CLOUDSTACK-648

Re: [ACS41] Today's 4.1 blocker and crit list! Go get 'em!

[prasanna]

On 29 March 2013 15:16, Sebastien Goasguen  wrote:
>
> On Mar 29, 2013, at 2:27 AM, Prasanna Santhanam  wrote:
>
>> Here's the latest Blockers and criticals (UTC +0530, Mar 29)
>>
>> Blocker:
>> CLOUDSTACK-1746
>> Cloudstack Usage Server won't start
>> Kishan KavalaFrancois Gaudreault
>>
>> CLOUDSTACK-1836
>> License header failures for ja-JP .po translation files
>> sebastien goasguen   Chandan Purushothama
>
> I fixed that one yesterday night. I did not close the ticket because I was 
> waiting for the next build to run.
> Can you tell me if the RAT passed properly ?
>

Hey Sebastien, builds.a.o was down all day y'day. I see that RAT passes now
https://builds.apache.org/view/A-F/view/Cloudstack/job/cloudstack-rat-41/205/

Re: [CI] Upgrading Jenkins

[prasanna]

The repo was pointed wrong. Fixed now.


On 29 March 2013 19:52, Prasanna Santhanam wrote:

> Shouldn't be. But not sure, will take a look shortly.
>
>
> --
> Prasanna.,
>
> - Original Message -
> From: Chip Childers [mailto:chip.child...@sungard.com]
> Sent: Friday, March 29, 2013 09:24 PM
> To: CloudStack Dev 
> Subject: Re: [CI] Upgrading Jenkins
>
> On Fri, Mar 29, 2013 at 05:07:23PM +0530, Prasanna Santhanam wrote:
> > On Fri, Mar 29, 2013 at 04:54:02PM +0530, Prasanna Santhanam wrote:
> > > Hi,
> > >
> > > I'm hitting couple of jenkins bugs [1][2] on my build jobs.
> > > Jenkins.cs.o will be upgraded.
> > >
> > > [1] https://issues.jenkins-ci.org/browse/JENKINS-13972
> > > [2] https://issues.jenkins-ci.org/browse/JENKINS-17337
> > >
> > This is done. Let me know if there's any problems.
> >
> > --
> > Prasanna.,
> >
>
> Is the master docs build failing due to this upgrade?
>
> http://jenkins.cloudstack.org/job/build-docs-admin-master/1223/console
>



-- 
Prasanna.,

Re: cloudmonkey printing enhancements proposal

[prasanna]

On 30 March 2013 09:20, Justin Grudzien  wrote:
>
> My company is building a private cloud and we are moving to cloudstack. As
> we begun investigating the cloudmonkey CLI we found that the output was
> slightly hard to read. I have begun working on some optimizations that I
> think will benefit the community and I reached out to Rohit, who
> recommended that I join this list and present my ideas. Here is what I am
> proposing:
>
> 1. Add json output to cloudmonkey
> I have accomplished this by adding a config parameter called display, which
> can be set to json, tabularize, or default. I have removed the tabularize
> parameter.
>
> Justins-MacBook-Pro:cloudmonkey grudzien$ cloudmonkey list users
> account=grudzien
> {
>   "count": 1,
>   "user": [
> {
>   "account": "grudzien",
>   "accountid": "b799783d-e5bb-460a-be0e-3966bd69edda",
>   "accounttype": 1,
>   "apikey": "*nokey*",
>   "created": "2013-03-27T16:09:17-0500",
>   "domain": "ROOT",
>   "domainid": "7e61c32f-9873-4944-947a-dcc00d3bebdc",
>   "email": "grudz...@gmail.com",
>   "firstname": "Justin",
>   "id": "265930bc-62ef-41f8-849c-e58593ca4b1f",
>   "lastname": "Grudzien",
>   "secretkey": "*nokey*",
>   "state": "enabled",
>   "username": "grudzien"
> }
>   ]
> }
>
> 2. Add filtering as a standard parameter for all output types.
> The only thing that has filtering now is the tabular output and grep breaks
> the json.
>
> Justins-MacBook-Pro:cloudmonkey grudzien$ cloudmonkey list users
> account=grudzien filter=account,email,username,state
> {
>   "count": 1,
>   "user": [
> {
>   "account": "grudzien",
>   "email": "grudz...@gmail.com",
>   "state": "enabled",
>   "username": "grudzien"
> }
>   ]
> }
>

+1 - JSON output would make it very good to consume into other api
response processing. Right now cloudmonkey actually isn't aware of the
response object model. For certain API responses that return embedded
objects (eg: listNetworks [1]) it would be good to return the JSON
appropriately formatted as well.

[1] https://gist.github.com/vogxn/5255883


> 3. Add color to the json output
> I was thinking of colorizing the keys in the key/value pairs to increase
> readability.
>
> 4. Move the color option from the config file to the command line.
> There are two reasons for this. First, I want to be able to wrap a script
> around cloudmonkey and not have to worry about colorization that will
> impede me processing the output and second I think it would be more useful
> to use the highlighting on demand rather than having to back out of the
> shell to edit a config file.
>
> 5. Standardize messaging for the output types.
> Right now certain kinds of messaging is presented differently for an output
> type. For example, if I issue an api command that doesn't exist it displays
> a generic error message, regardless of the output type selected. Ideally,
> all output would be in the specified format.
>
> I have the first two working and am planning on implementing the others as
> I flesh them out. I will submit a patch when I feel it is ready. Any early
> feedback on whether these changes will be useful to others is appreciated.

Patches most welcome!




--
Prasanna.,

Re: [DISCUSS] git rebase vs git merge in your feature branch?

[prasanna]

I've been doing a rebase of my branch ever since this thread started
and ended up in quite a lot of hell with merge conflicts. Perhaps I
shouldn't have done it on a branch that I was doing merges with. So I
want to understand the workflow of the rebase. Is it something like
this:

So I want to work on feature:

1. git co -b featureBranch
2. make my commits on day1 locally and push to remote/origin (?)
3. make my commits on day2 locally and push to remote/origin
4. decide to see the latest on master so do a git rebase master
(resolve conflicts here)
5. make more changes to featureBranch on day3 and push to remote/origin
6. send in a squashed merge request for merging with master.

How does this work if multiple people want to work on the
aforementioned featureBranch? Does it work out this easily with
everyone rebasing at will?

If the fuss is about cleaner merge history then merge --no-ff will
skip those annoying merge commits. I also find it easier to review the
feature branch in logical chunks rather than one large patch.

PS: rerere made the rebase conflicts easier but I wouldn't trust that
with code I haven't touched.

--
Prasanna.,

Re: [ACS41][QA] Unassigned Critical Upgrade defects

[prasanna]

>> > CLOUDSTACK-1856  Upgrade 4.0 -> 4.1 - Fresh install of 4.1 has 3
>> > parameters missing in db. Properties compared to an upgraded 4.0 set up.
>> >
>>
>> Properties files should be backed up or ask user to replace/keep the file, 
>> this
>> is a packaging issue.
>
> I see following commit removing these parameters - 
> http://mail-archives.apache.org/mod_mbox/incubator-cloudstack-commits/201212.mbox/%3c20121219220428.9e565324...@tyr.zones.apache.org%3E.
>  Are they obsolete now?

That's right. Those properties never come into use anywhere within our code.



--
Prasanna.,

Re: [ACS41] Ubuntu upgrade process from 4.0.X to 4.1

[prasanna]

The keys are found here : http://www.apache.org/dist/incubator/cloudstack/KEYS

On 1 April 2013 16:13, Sailaja Mada  wrote:
> Hi,
>
> I tried to upgrade 4.0 Ubuntu Management server to 4.1 with below 
> instructions :
>
> Procedure 1:
>
> 1.  Add/Modify the line "deb http://cloudstack.apt-get.eu/ubuntu precise 4.1 
> " in /etc/apt/sources.list.d/cloudstack.list
> 2.  apt-get update
>
> Observation: It failed with GPG Error.
>
> Reading package lists... Done
> W: GPG error: http://cloudstack.apt-get.eu precise Release: The following 
> signatures couldn't be verified because the public key is not available: 
> NO_PUBKEY BBFCFE5386C278E3
>
> I tried to key KEYS from 
> http://www.apache.org/dist/dev/incubator/cloudstack/KEYS   as mentioned @ 
> http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.0.1-incubating/html/Installation_Guide/sect-source-verify.html
>   .
> But I am getting 404 error as KEYS are not available at this location.
>
> Procedure 2:
>
> 1. Tried to create APT Repo with packages that are built @ 
> http://jenkins.cloudstack.org/job/package-deb-4.1/ and tried apt-get update . 
>  But there are no packages got upgraded.  Filed the JIRA ticket for this 
> issue couple of days back @  
> https://issues.apache.org/jira/browse/CLOUDSTACK-1848
>
>
> Thanks,
> Sailaja.M
>
>
> -Original Message-
> From: Wido den Hollander [mailto:w...@widodh.nl]
> Sent: Sunday, March 31, 2013 12:38 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [ACS41] Ubuntu upgrade process from 4.0.X to 4.1
>
>
>
> On 03/30/2013 04:02 PM, Marcus Sorensen wrote:
>> One of the things that helped me quite a bit With the CentOS stuff was
>> that I created a VM with agent/management server running in it. I set
>> up a 4.0 system with an advanced zone, pulled down the source code
>> into it, and then took a copy of the VM. Then I ran through the
>> upgrade process 10 or 15 times, testing, making note of what was
>> missing, rolling back and then trying again, slowly building a patch
>> for the cloud.spec. With the database upgrade and everything else that
>> needs to be done, it's really hard to roll back a real environment and
>> try the upgrade again from scratch.
>>
>
> I did the same with the Deb packaging, I have a couple of VMs where I test 
> all of this.
>
>> At this point upgrading CentOS is just installing the new RPMs along
>> with a few new dependencies, restarting the services, and restarting
>> the system vms so they get upgraded software. Your e-mail has prompted
>> me to want to create a wiki though, in case anything goes wrong,
>> unless upgrade notes will be released somewhere in particular. Just a
>> place to put upgrade tips like "Old configs will be moved from
>> /etc/cloud to /etc/cloud.rpmsave" in case something does need to be
>> moved manually by the admin.
>>
>
> The problem is that we are moving files from location to location and we 
> don't simply want to do all of that without telling the admin what is going 
> on.
>
> I'll try to do as much as I can in the packages and pre- and 
> post-installation files, but I'll leave the "scary work" to the admins.
>
> The big warning should say: BACKUP YOUR DATABASE before continuing.
>
> So yes, the packages should do all the heavy lifting, but some guidance would 
> be nice for a lot of users I think.
>
> Wido
>
>> On Sat, Mar 30, 2013 at 6:17 AM, Wido den Hollander  wrote:
>>> Hi,
>>>
>>> I've started a Wiki[0] to describe the upgrade process going from 4.0
>>> to 4.1 with Ubuntu packages.
>>>
>>> Since the packages got renamed from cloud-* to cloudstack-* a lot has
>>> changed and some manual intervention is required.
>>>
>>> The Wiki is work-in-progress while I'm upgrading one of my 4.0
>>> clusters to
>>> 4.1 and see how it all works out.
>>>
>>> Feel free to change the Wiki if you find something that needs to be
>>> adressed!
>>>
>>> Wido
>>>
>>> [0]:
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ubuntu+upgrade
>>> +process
>>>



-- 
Prasanna.,

Re: [ACS42][QA]Limit Resources - Internal Error when there are no resources available

[prasanna]

Surfacing an exception that makes sense based on the role of the user
receiving the exception will require a more fundamental change: The
error message you see is basically the cleaned version of what the
admin sees. Looking at the ApiServer you'll see that whether the user
context receives - InsufficientCapacityException,
ResourceAllocationException, ResourceUnavailableException or just any
other generic exception the api decides to throw - the user will
receive the 'internal error' message. And the class of APIs that will
see this kind of filtering is anything that implements the BaseCmd
which is just about every API in cloudstack :)

On 3 April 2013 12:39, Sailaja Mada  wrote:
> It would be good to mention failure reason with out providing the information 
> about specific Resource type and domain id .
>
> Thanks,
> Sailaja.M
>
> -Original Message-
> From: Ahmad Emneina [mailto:aemne...@gmail.com]
> Sent: Wednesday, April 03, 2013 12:32 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [ACS42][QA]Limit Resources - Internal Error when there are no 
> resources available
>
> this is done to hide potentially sensitive operational/resource information 
> from end users. Limiting exploitation vectors. probably worth a discussion 
> and/or a feature enhancement to modify the behavior.
>
>
> On Tue, Apr 2, 2013 at 11:27 PM, Sailaja Mada wrote:
>
>> Hi,
>>
>> I have set the limit to use x amount of resources for the Domain Admin
>> account. When there are not enough resources available and tried to
>> deploy VM,  this user gets the message as "Internal error executing
>> command, please contact your system administrator"  .
>>
>> Only in the Management Server log  , root user can get the reason why
>> it failed as INFO Message : 2013-04-03 11:50:59,813 INFO
>> [cloud.api.ApiServer] (catalina-exec-18:null) Maximum number of
>> resources of type 'cpu' for domain id=3 has been exceeded."
>>
>> Is there any reason why we can not provide the root cause for failure
>> to Child Domain Account ?
>>
>> Thanks,
>> Sailaja.M
>>

Re: cloudmonkey printing enhancements proposal

[prasanna]

>> >
>>
>> +1 - JSON output would make it very good to consume into other api
>> response processing. Right now cloudmonkey actually isn't aware of the
>> response object model. For certain API responses that return embedded
>> objects (eg: listNetworks [1]) it would be good to return the JSON
>> appropriately formatted as well.
>>
>> [1] https://gist.github.com/vogxn/5255883
>
> If cloudmonkey cares about json response now, it makes sense that both 
> cloudmonkey and marvin share the same code to send api command and parse json 
> response. Isn't it?
>

Yup - makes perfect sense. They do use the same make_request method.
cloudmonkey/requester.py and
tools/marvin/marvin/cloudstackConnection.py. I fix (CLOUDSTACK-1906)
that so we have one codepath for the request call. monkey->marvin or
marvin->monkey? or a cloudstack-request-util may be.

Re: [MERGE] bvt branch to master

[prasanna]

On 3 April 2013 01:18, Rohit Yadav  wrote:
> Awesome, thanks for your work Prasanna, do we have a jenkins job yet?
>
This one needs mysql db to be present. So i'll have to run it on a
fresh new node. I'll probably set it up if no one uses the feature :)

Re: [Discuss][DeltaCloud]CloudStack suppprt

[prasanna]

For python there's libcloud - I was going to work on support in Apache
Libcloud but never found bandwidth. If anyone's interested they have a
pretty active community. Would be nice to cross-contribute to their
effort.

On 3 April 2013 20:29, Musayev, Ilya  wrote:
> Team WebMD would like to write drivers for CloudStack and DeltaCloud.
>
> Reaching out to dev community to see if anyone has started this project 
> already or wants to join and we can work together or we will be the first 
> ones to write a support for it.
>
> Feedback is welcome,
>
> Regards
> Ilya

Re: Master - jetty run failure

[prasanna]

I'm not seeing this on the latest master. Perhaps something wrong with
nonOss component context? Are you running nonOss by any chance?

On 3 April 2013 23:23, Pranav Saxena  wrote:
> Possible workaround  until that issue is fixed - Revering these two  commits 
> - eddf7b9357bc18497b8cb16a6c6f3382ac52f61c  and  
> 44567453e0511e3090ac22518113b283cfa26b4b to have jetty running successfully.
>
> -Original Message-
> From: Likitha Shetty [mailto:likitha.she...@citrix.com]
> Sent: Wednesday, April 03, 2013 11:15 PM
> To: dev@cloudstack.apache.org
> Cc: Hugo Trippaers
> Subject: Master - jetty run failure
>
> On latest master when I run jetty,
>
> ERROR [utils.component.ComponentContext] (Timer-2:) Unhandled exception
> javax.naming.ConfigurationException: Could not find midonet API location in 
> config
> at 
> com.cloud.network.element.MidoNetElement.configure(MidoNetElement.java:148)
> at 
> com.cloud.utils.component.ComponentContext.initComponentsLifeCycle(ComponentContext.java:111)
> at 
> com.cloud.servlet.CloudStartupServlet$1.run(CloudStartupServlet.java:50)
> at java.util.TimerThread.mainLoop(Timer.java:534)
> at java.util.TimerThread.run(Timer.java:484)
> Exception in thread "Timer-2" java.lang.RuntimeException: Unable to configure 
> com.cloud.network.element.MidoNetElement_EnhancerByCloudStack_c688f431
> at 
> com.cloud.utils.component.ComponentContext.initComponentsLifeCycle(ComponentContext.java:114)
> at 
> com.cloud.servlet.CloudStartupServlet$1.run(CloudStartupServlet.java:50)
> at java.util.TimerThread.mainLoop(Timer.java:534)
> at java.util.TimerThread.run(Timer.java:484)
> Caused by: javax.naming.ConfigurationException: Could not find midonet API 
> location in config
> at 
> com.cloud.network.element.MidoNetElement.configure(MidoNetElement.java:148)
> at 
> com.cloud.utils.component.ComponentContext.initComponentsLifeCycle(ComponentContext.java:111)
> ... 3 more
>
> Thank you,
> Likitha

Re: Master broken

[prasanna]

On 4 April 2013 05:15, Alex Huang  wrote:
> Prasanna,
>
> This is actually a very interesting case.  In our check-in test, is there any 
> way for us to disable db access when running the unit tests?  I can see that 
> this would be one of the things that the developer can easily forget but hits 
> on the build server and breaks the Jenkins build often.
>
> Thanks.
>
> --Alex
>
That will be hard to do since the checkin test requires the DB to be
present as it starts up the server and sets-up the databases. We'd
have to turn it off for the build and turn it back on again for the
bvt. Best if we avoid doing -DskipTests when building as a general
good rule.

Re: [ACS42][QA]Test Plan for "Affinity / Anti-affinity Rules"

[prasanna]

On 4 April 2013 10:40, Sangeetha Hariharan
 wrote:
> Test cases for " Affinity / Anti-affinity Rules" is posted here - 
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Test+Paln+for+Affinity-Anti-affinity+Rules.
>
>
>
> Reference FS: - 
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/FS+-+Affinity-Anti-affinity+groups
>
>
There are planning decisions that are controlled in the global
settings that do similar things as do affinity groups.
UserConcentration and UserDispersion are two kinds of allocators that
affect planning decision very similar to affinity and anti-affinity.
Some cases surrounding the conflicts arising from these settings would
need to be added. Rest of the tests look good given that the scope of
the feature is only a single anti-affinity processor at this time.

Re: 4.1 packaging failed

[prasanna]

Hugo fixed this already: the latest packages have been passing
http://jenkins.cloudstack.org/job/package-rhel63-4.1/30/

On 5 April 2013 23:27, Rayees Namathponnan
 wrote:
> Hi Hugo,
>
> 4.1 packaging failed, is it due below check in ?
>
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commit;h=7cbd8f581f7f42cca056b82ed7cbfce8aebf974f
>
>
> Packaging failed with below error
>
>
> Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 
> rpmlib(PayloadFilesHavePrefix) <= 4.0-1
> Checking for unpackaged file(s): /usr/lib/rpm/check-files 
> /root/jenkins/build/workspace//dist/rpmbuild/BUILDROOT/cloudstack-4.1.0-SNAPSHOT.el6.x86_64
> error: Installed (but unpackaged) file(s) found:
>/usr/share/cloudstack-common/jasypt-1.9.0.jar
>
>
>
> Regards,
> Rayees
>
>



-- 
Prasanna.,

Re: [MERGE] affinity_groups branch to master

[prasanna]

On 6 April 2013 05:56, Prachi Damle  wrote:
> Hi all,
>
>
>
> I would like to merge the Affinity/Anti-Affinity feature developed in
> affinity_groups branch to master.
>
>
>
> -  Functional Specs for this feature can be found here:
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/FS+-+Affinity-Anti-affinity+groups
>

>
> -  Integration tests: Added python test using marvin here:
> cloud-tools/marvin/marvin/sandbox/demo/live/
> testDeployVMWithAffinityGroup.py
>
>
Thanks Prachi for the test. I've modified it in your branch to use
some library methods from marvin which makes it concise. When I run
the BVT on the branch I noticed an exception. Perhaps it is related to
the changes made in the branch, perhaps not:

Can you please take a look?
https://issues.apache.org/jira/browse/CLOUDSTACK-1968

Thanks,

Re: [ACS41] Outstanding work for the release

[prasanna]

On 9 April 2013 14:33, Hugo Trippaers  wrote:
>
>>
>> RPM Packaging: I believe that Hugo said we are done there, but I'd love a
>> confirmation!
>
> Yup, I'm considering the RPM packages done. I'm testing installation of all 
> the rpms every four hours using Jenkins. I'm doing limited functional checks 
> on management server (startup and zone configuration with xcp hypervisors), 
> agent (just start-up) and usage (successful start) so I'm confident that they 
> are properly installed. I'm not doing any checks on aws-api (if anybody can 
> give me some pointers I'm happy to put some tests in).
>
> As for functionality beyond this, I believe that QA is using these packages 
> for their tests as well?
>
Yup - RPM gets tested every 4 hours using Xen and KVM as hypervisors
to setup a zone.
http://jenkins.cloudstack.org/view/cloudstack-qa/job/test-packaging/

Re: [MERGE] bvt branch to master

[prasanna]

On 10 April 2013 07:05, Chiradeep Vittal  wrote:
> Don't have the logs any more, but I suspect that
> tools/marvin/pom.xml uses the same property to activate both profiles, so
> marvin.setup gets run whenever -Dmarvin.config is specified.
>

I just ran the tests on latest master and am able to see them pass:

~/workspace/cloudstack/incubator-cloudstack(branch:master) » mvn
-Pdeveloper,marvin.test -pl :cloud-marvin integration-test

 tsp@cloud-2
[INFO] Scanning for projects...
[INFO]
[INFO] 
[INFO] Building Apache CloudStack marvin 4.2.0-SNAPSHOT
[INFO] 
[INFO]
[INFO] --- exec-maven-plugin:1.2.1:exec (generate-sources) @ cloud-marvin ---
[INFO]
[INFO] --- maven-remote-resources-plugin:1.3:process (default) @
cloud-marvin ---
[INFO]
[INFO] --- maven-site-plugin:3.1:attach-descriptor (attach-descriptor)
@ cloud-marvin ---
[INFO]
[INFO] --- exec-maven-plugin:1.2.1:exec (integration-test) @ cloud-marvin ---
Test to create disk offering ... ok
Test to update existing disk offering ... ok
Test to delete disk offering ... ok
Test advanced zone virtual router ... ok
Test Deploy Virtual Machine ... ok

--
Ran 5 tests in 37.231s

OK
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 55.986s
[INFO] Finished at: Wed Apr 10 10:28:31 GMT+05:30 2013
[INFO] Final Memory: 18M/81M
[INFO] 

Re: [MERGE] bvt branch to master

[prasanna]

I see the problem now with the profiles getting activated incorrectly.
I wasn't using the config property and hence the profile selection
worked.

On 10 April 2013 10:30, prasanna  wrote:
> On 10 April 2013 07:05, Chiradeep Vittal  wrote:
>> Don't have the logs any more, but I suspect that
>> tools/marvin/pom.xml uses the same property to activate both profiles, so
>> marvin.setup gets run whenever -Dmarvin.config is specified.
>>
>
> I just ran the tests on latest master and am able to see them pass:
>
> ~/workspace/cloudstack/incubator-cloudstack(branch:master) » mvn
> -Pdeveloper,marvin.test -pl :cloud-marvin integration-test
>
>  tsp@cloud-2
> [INFO] Scanning for projects...
> [INFO]
> [INFO] 
> 
> [INFO] Building Apache CloudStack marvin 4.2.0-SNAPSHOT
> [INFO] 
> 
> [INFO]
> [INFO] --- exec-maven-plugin:1.2.1:exec (generate-sources) @ cloud-marvin ---
> [INFO]
> [INFO] --- maven-remote-resources-plugin:1.3:process (default) @
> cloud-marvin ---
> [INFO]
> [INFO] --- maven-site-plugin:3.1:attach-descriptor (attach-descriptor)
> @ cloud-marvin ---
> [INFO]
> [INFO] --- exec-maven-plugin:1.2.1:exec (integration-test) @ cloud-marvin ---
> Test to create disk offering ... ok
> Test to update existing disk offering ... ok
> Test to delete disk offering ... ok
> Test advanced zone virtual router ... ok
> Test Deploy Virtual Machine ... ok
>
> --
> Ran 5 tests in 37.231s
>
> OK
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 55.986s
> [INFO] Finished at: Wed Apr 10 10:28:31 GMT+05:30 2013
> [INFO] Final Memory: 18M/81M
> [INFO] 
> 
> 

Re: [DISCUSS] Don't assign tickets to people when triaging

[prasanna]

On 11 April 2013 15:52, Abhinandan Prateek
 wrote:
>
>>>
>>>I will start with an example: A critical bug (CLOUDSTACK-1941) that is
>>>blocking a release (4.1) is not picked up by any community member for 5
>>>days !
>>>The reason being that it is a UI issue but not that clear from the desc,
>>>the nature of the bug is known after someone spends time on it.
>>>
>>>Now is it wrong to ask the community members who have expertise on UI to
>>>fix it, in a bid to help Chip get the release out ?
>>>
>>>A set of guidelines are necessary so that this whole confusion about bugs
>>>getting assigned is cleared up. I will start by proposing some simple
>>>rules:
>>>
>>>1. Never assign bugs that are not critical or blocker unless they meet
>>>any
>>>of the below condition.
>>
>>Never would be too lenient. Maybe assign it after 7-8 days since they
>>don't need immediate attention.
>
> 7-8 days is a huge time lost. I was suggesting that this to be 3 days. Let
> other community members chime in too.
>

Just to note: contributors will enter anytime. They don't necessarily
know of our release roadmap when they come in looking to contribute.
And typically they'd be looking at low hanging fruit - ie not blocker,
criticals. If you're saying 'x days are lost' then that doesn't make
much sense to an external contributor, x days based on what? If a bug
is blocking someone else's progress then the person who is blocked
should make noise on this list so the appropriate person can fix it.
I think the issue this thread is trying to address is assignment of
bugs in the background without community participation and/or
knowledge.

Also I don't think bugs are getting fixed immediately after assignment
as you indicate. Bugs go between Triager 1 to Triager 2 to Developer 3
and then Developer 3 assigns it to the Developer 4 who fixes the bug.
Instead - Developer 4 should have got to it first or explained the
nature of the fix if he hasn't the time to fix it. Which doesn't
happen either. As Alex proposed, when someone takes up a bug they
should mark it in progress so that we know work is on going. Instead
if Triagers are just assigning bugs based on some kind of weird
LRU-cache in their head of who's (usually $dayjob stakeholder) the
rightful owner I find it exclusionary to community participation.

There is no contest on the bugs being assigned by the RM that are
essential to be fixed for a release. So I agree with you on that.

Re: Associate IP feature request (RE: CLOUDSTACK-1942)

[prasanna]

On 11 April 2013 01:08, Ryan Dietrich  wrote:
> RE: https://issues.apache.org/jira/browse/CLOUDSTACK-1942
>
> I added this feature this morning, but before I present the diff to review 
> board, I'd like to have some feedback on the approach.  I just want to make 
> sure how I did it is ok from the maintainers perspective so I'm not wasting 
> anyones time.
>
> 1. Add an optional parameter to 
> api/src/org/apache/cloudstack/api/command/user/address/AssociateIPAddrCmd.java
>  allowing you to pass in a UUID of the specific IP you want to add.
> 2. Make AssociateIPAddrCmd either use the provided IP or use the existing 
> "getEntity()" call it is using now.
> 3. Update "allocateIP" to have a new optional argument, the Long (id) of 
> IpAddress.
> 4. Update the NetworkManager and NetworkService interfaces to include the new 
> parameter
> 5. Update the MockNetworkManager and MockNetworkService implementations to 
> include the new parameter.
> 6. Update the NetworkServiceImplementation to include the new parameter and 
> pass it through.
> 7. Update NetworkManagerImpl: Use the existing functionality of 
> "fetchNewPublicIp" and pass in the String "requestedIp" based on the Long id 
> of IpAddress (I used ApiDBUtils to fetch the object out and transform it into 
> a string).
>
I don't see any problem with the approach.

Re: [DISCUSS] Don't assign tickets to people when triaging

[prasanna]

Abhi - not to gang up on you and I'm glad to see you share your
opinions, concerns about release management and such.

I see the problem you might be facing though. I think it would be
better to have your internal JIRA mirror the ASF JIRA. That way you
can triage as you please corporate style ;) in your internal JIRA and
not let that spill over into the Apache JIRA which should be left to
work community style. If a bug is assigned to someone on your internal
JIRA, then that person will come and assign the bug to themselves in
the community ASF JIRA when they have time to fix it. That, I think,
works well for both the community and corporates invested in the
project.

HTH,

On 11 April 2013 19:17, Noah Slater  wrote:
> I believe it is possible to "mention" someone in a JIRA ticket in such a
> way that they get notified. Might this be an effective way of CCing someone
> into the conversation, without prescribing who should fix it? Might there
> be some room for exploration here?
>
>
> On Thursday, 11 April 2013, Abhinandan Prateek wrote:
>
>> Yes, I think we need to space our releases further apart.
>> I had big trouble when master was unstable for a while and specially on
>> VMware it was difficult to deploy and test features. Yes for each issue I
>> could have shouted on mail list I saw people doing that but the fact is
>> that instability was around for a while. Doesn't it make sense that in such
>> scenarios we could do things in a more pro active manner. Again I donot see
>> much difference in asking someone on Jira to pick a issue vs sending a
>> email, but will agree to whatever the community decides here.
>>
>> Also community members should volunteer to own some part so that in above
>> circumstances a person looking for some fix can approach that member, once
>> again a suggestion.
>>
>>
>>
>> On 11-Apr-2013, at 5:17 PM, "Noah Slater" >
>> wrote:
>>
>> > Of course releases are important.
>> >
>> > But if our current cadence is putting too much pressure on the community,
>> > one option might be to do our releases further apart from each other. Or,
>> > we get strict about the principal of time based releases: i.e. if your
>> > feature is not ready for the freeze, then it doesn't make it in. No big
>> > deal. If it's ready for the next freeze, then we'll ship it then.
>> >
>> > Also, I may be reading your message wrong, but there's no need for this
>> to
>> > be a divisive argument. There are no "sides" to this. As a community, it
>> is
>> > up to us all to identify our problems, and figure out solutions.
>> >
>> > So what problems do you think we'll run in to if we stop assigning the
>> > majority of bugs, and how do you think we can mitigate those problems? Or
>> > do you have another idea in mind altogether?
>> >
>> >
>> >
>> >
>> > On 11 April 2013 12:40, Abhinandan Prateek <
>> abhinandan.prat...@citrix.com >wrote:
>> >
>> >> I think it will be good if we also find out a process so that the
>> release
>> >> cycle is not affected by unclaimed bugs sitting out there. Here I am
>> >> assuming the releases are important.
>> >>
>> >> I guess the discussion has turned into keeping things free without
>> >> offering solutions to problems that that system will create.
>> >>
>> >>
>> >> On 11/04/13 5:04 PM, "John Burwell" >
>> wrote:
>> >>
>> >>> +1
>> >>>
>> >>> On Apr 11, 2013, at 7:22 AM, Noah Slater 
>> >>> >
>> wrote:
>> >>>
>>  On 11 April 2013 11:22, Abhinandan Prateek
>>  >wrote:
>> 
>> >
>> > 7-8 days is a huge time lost. I was suggesting that this to be 3
>> days.
>> > Let
>> > other community members chime in too.
>> 
>> 
>>  I should have replied to this in my previous missive. But I want to
>>  reenforce how unhealthy I believe this practice is. 7-8 days, or even
>> 3
>>  days "being a huge time loss" makes absolutely no sense to me at all.
>>  Assigning a bug should not mean it gets fixed any faster. If it does,
>>  then
>>  we need to change the way we are working. (And if this means changing
>>  the
>>  JIRA ticket workflow, then so be it. If something isn't working for
>> us,
>>  we
>>  change it.)
>> 
>>  In fact, I would go so far as to say that we should think of assigning
>>  bugs
>>  as an exclusionary practice. Every time you assign a bug, you're
>>  shutting
>>  out the community. That's how we should think about it. Assign the
>> bug,
>>  shut out the community. And so, I would say we should try to avoid
>> doing
>>  it, unless it is absolutely necessary. (Such as when you're
>>  co-ordinating
>>  some release critical work, or when you, yourself, are about to start
>>  work
>>  on something. Of course, it's perfectly fine to shut out the
>> community,
>>  if
>>  you're doing that at the same time as starting work on something!)
>> 
>> 
>>  --
>>  NS
>> >
>> >
>> > --
>> > NS
>>
>
>
> --
> NS

Re: [DISCUSS] Don't assign tickets to people when triaging

[prasanna]

On 11 April 2013 20:09, Chip Childers  wrote:
> On Thu, Apr 11, 2013 at 12:51:34PM +, Abhinandan Prateek wrote:
>> Yes, I think we need to space our releases further apart.
>
> That's a different discussion, which you are free to raise if you'd like.
>
>> Also community members should volunteer to own some part so that in above 
>> circumstances a person looking for some fix can approach that member, once 
>> again a suggestion.
>
> I've been reading through this thread, and I'll pick the "owner" comment
> above as a starting point for my personal opinions.  This is a reaction
> to the whole thread really, so take a minute to read to the end please.
>
> "Owning some part" is antithetical to a healthy community approach.
> Certainly people will gravitate to certain areas, and by all means
> everyone should feel free to work on areas of the code-base that they
> feel like they want to improve or support.  This may lead to people
> effectively being the primary "do-er" for certain areas (examples: Wido
> has been working on DEB packaging, Rohit has been working on
> CloudMonkey), but we shouldn't ever consider this ownership. I feel
> personally welcome to make a change in CloudMonkey, and would certainly
> consider it important to collaborate with anyone (especially Rohit) that
> may have input and insights.
>
> The idea of ownership if a part of the software is something I'm strongly
> against.
>
> Even the idea of maintainers seems like it is problematic in
> implementation.  How do we decide who the "official" maintainer is?  How
> do we decide when someone else should do that... And frankly, doesn't a
> "maintainer" model really discourage others from working in named areas?
>
> All of these attempts to structure the community appear to be natural
> responses when you have a background in corporate development (product
> or otherwise), which is my background as well.  It doesn't work here,
> and you have to fight the urge to apply the same solutions (WRT
> structure and process) in this environment.  If you haven't read
> "Producing OSS" [1], go do that.
>

And if you don't have the patience, here's the appropriate extract:
http://www.producingoss.com/en/managing-volunteers.html#delegation-assignment

Re: Master api-docs build

[prasanna]

Oddly 453 failed again. They build on different nodes ubuntu4 and
ubuntu3 and ubuntu4 seems to fail the build.

On 12 April 2013 07:14, Prasanna Santhanam
 wrote:
> 452 has actually passed the build.
>
> --
> Prasanna.,
>
> - Original Message -
> From: Chiradeep Vittal [mailto:chiradeep.vit...@citrix.com]
> Sent: Friday, April 12, 2013 09:41 AM
> To: dev@cloudstack.apache.org 
> Subject: Master api-docs build
>
> Raising this again. Jenkins seems to not be cleaning up properly. The
> dedicatePublicIpRange patch was reverted a few days ago and the build
> should be successful.
>
> On 4/11/13 3:05 PM, "Apache Jenkins Server" 
> wrote:
>
>>See <https://builds.apache.org/job/cloudstack-apidocs-master/451/changes>
>>
>>Changes:
>>
>>[jessica.wang] CLOUDSTACK-1910: cloudstack UI - Regions menu - implement
>>create GSLB action.
>>
>>[kelveny] Fix the systemvm packaging issue
>>
>>[brian.federle] List view: Fix broken add row action
>>
>>[brian.federle] VM NICs list view: Fix 'VM name' field for VMs without
>>name
>>
>>[jessica.wang] CLOUDSTACK-1910: cloudstack UI - Regions menu - create
>>GSLB - (1) pass gslbstickysessionmethodname parameter to
>>createGlobalLoadBalancerRule API. (2) Take async Job response.
>>
>>[prachi] API changes for createAffinityGroup
>>
>>[prachi] More API changes
>>
>>[prachi] DeleteAffinityGroup API changes
>>
>>[prachi] ListAffinityGroups API changes
>>
>>[prachi] Changes to add AffinityGroupprocessor, deployVM changes
>>
>>[prachi] Separated out host anti-affinity as a plugin.
>>
>>[prachi] Added apache license header
>>
>>[prachi] Schema changes to create affinity group tables
>>
>>[prachi] DAO constructor should be lightweight to make Spring DI faster.
>>
>>[prachi] Not using entity factory
>>
>>[prachi] API to list planners and set the planner in Service offering
>>
>>[prachi] API changes to expose the commands
>>
>>[prachi] Changes to make affinity group types configurable.
>>
>>[prachi] Fixes after functional tests
>>
>>[prachi] Adding the missing header!
>>
>>[prachi] Build failure fixes after rebase.
>>
>>[prachi] Adding a unit test for the new affinity groups API
>>
>>[prachi] Integration testcase and the config file needed,  that runs with
>>marvin.
>>
>>[prachi] Correcting the rebase merge issues.
>>
>>[prachi] Added cleanup of affinitygroups when a VM is expunging and when
>>the account is deleted.
>>
>>[prachi] Changes to return affinity groups information during listVMsCmd
>>
>>[prachi] Added AffinityGroup View in order to include VM details while
>>listing AffinityGroups.
>>
>>[prachi] Fixes to de-couple the AffinityGroupResponse from
>>UserVmResponse, since ApiDiscoveryService breaks, if we nest two response
>>objects into each other.
>>
>>[prachi] More log statements to debug
>>
>>[prachi] affinity group tests moved into the test folder
>>
>>[prachi] bvt: marvin test for the affinity groups feature
>>
>>[prachi] marvin bvt: getting rid of unused keys in the test
>>
>>[prachi] CLOUDSTACK-1968: affinity_groups: Column 'deployment planner'
>>cannot be null when creating a service offering
>>
>>[prachi] affinitygroup bvt: changes to the bvt for affinity groups
>>
>>[prachi] ACl on affinity group
>>
>>[prachi] Adding pretty toString() to AffinityGroup
>>
>>[prachi] Fixes for issues found while testing after the merge
>>
>>[prachi] Fixes to unit-test dues to changes in master
>>
>>[prachi] Fixing rat. Merge with master missed the header change
>>
>>[jessica.wang] CLOUDSTACK-1065: cloudstack UI - AWS Style Regions - set
>>autocomplete off.
>>
>>[jessica.wang] CLOUDSTACK-1065: cloudstack UI - AWS Style Regions - make
>>loginCmdText local.
>>
>>[prachi]  Excluding this unit test for a while, since it fails because
>>ComponentContext.initComponentsLifeCycle(); is failing when DB is
>>unavailable
>>
>>--
>>[...truncated 3749 lines...]
>>
>>---
>> T E S T S
>>---
>>
>>[INFO] --- maven-surefire-plugin:2.12:test (default-test) @
>>cloud-client-ui ---
>>
>>Results :
>>
>>Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
>>
>>[JENKINS] Recording test results

Re: New Committer: Bruno Demion (a.k.a milamber)

[prasanna]

On 12 April 2013 21:32, Sebastien Goasguen  wrote:
> The Project Management Committee (PMC) for Apache CloudStack has asked Bruno 
> Demion (a.k.a milamber) to become a committer and we are pleased to announced 
> that they have accepted.
>
> Bruno has contributed to the french translation of the UI and the Docs, as 
> well as fixed several issues with encoding of the localization support of the 
> UI.  For information, Bruno is also a committer and PMC member of the Apache 
> Jmeter project.
> Being a committer allows many contributors to contribute more
> autonomously. For developers, it makes it easier to submit changes and
> eliminates the need to have contributions reviewed via the patch
> submission process. Whether contributions are development-related or
> otherwise, it is a recognition of a contributor's participation in the
> project and commitment to the project and the Apache Way.
> Please join me in congratulating Bruno,
>

Congratulations Milamber! Hope we can get performance metrics out of a
jmeter and cloudstack collaboration.

Re: Latest master is failing at (apidocs)

[prasanna]

Add a dictionary entry to tools/apidoc/gen_toc.py with 'Cisco' as the key

On 16 April 2013 13:26, Rajesh Battala  wrote:
> Hi All,
> Am trying to build the latest master its failing at apidocs.
>
> Check, is this api part of another build profile? Null value for key: 
> addBaremetalHost preProcessedCommand=1
> Traceback (most recent call last):
>   File "/home/rajesh/hc_fzf/tools/apidoc/gen_toc.py", line 173, in 
> category = choose_category(fn)
>   File "/home/rajesh/hc_fzf/tools/apidoc/gen_toc.py", line 153, in 
> choose_category
> (fn, __file__))
> Exception: Need to add a category for deleteCiscoAsa1000vResource.xml to 
> /home/rajesh/hc_fzf/tools/apidoc/gen_toc.py:known_categories
>
>
> Please let me know how to resolve this error?
>
> Thanks
> Rajesh   Battala

Re: Review Request: Remove 2k limitation for user data on a deployVMCmd issued as an HTTP POST request

[prasanna]

On 17 April 2013 17:10, Prasanna Santhanam  wrote:
> On Wed, Apr 17, 2013 at 11:52:08AM +0530, Prasanna Santhanam wrote:
>> On Wed, Apr 17, 2013 at 11:10:01AM +0530, Rohit Yadav wrote:
>> >
>> >
>> > If it does not cost much, let's introduce a new dependency for requesting
>> > stuff (get or post etc.) using "requests" [1] for both marvin and
>> > cloudmonkey. Vijay, you can write your own requester, as it's not much
>> > effort required to implement your own requester once you how signature is
>> > calculated.
>> >
>> > IMO, cloudstackConnection could be rewritten using latest ways and better
>> > libraries as it's not much code and is totally do-able with much less
>> > effort. A lot of flows for auth or non-auth are inter-dependent, the
>> > json->obj and obj->json parts and requesting logic is not straight forward
>> > (each method does *only* one thing). We need to make it generic by
>> > providing both raw and processed outputs, for example the result expects to
>> > return an obj and not json (maybe I need json and not an obj), one reason
>> > why I chose to write my own for cloudmonkey after I tried to reuse it.
>> > Lastly, host marvin on pypi so it's easier for any developer to get it off
>> > the shelf and start hacking some cool clients.
>> >
>> > Cheers.
>> >
>> > [1] http://docs.python-requests.org/en/latest/
>> >
>> >
>>
>> Yeah - we should simplify cloudstackConnection and make it easily
>> exportable for both cloudmonkey,marvin and any other tools that might
>> require to make connections to CS mgmt. requests certainly looks
>> elegant. I'm trying to experiment with it to see if I can cook up
>> something for marvin.
>>
>> At one point I moved from httplib to urllib2 to overcome an annoying
>> BadStatusLine issue from httplib. I still find it hard to throttle
>> multiple tests through a single connection and urllib3 seems to handle
>> that. requests appears to be based on urllib3. So it would make sense
>> to move to it.
>>
>> The json decode,encode is not done within cloudstackConnection. That
>> part is annoying and I explored json and simplejson and there doesn't
>> seem to be an easy way to accomplish marvin's case. Esp. with nested
>> json objects I find it easier to use a recursive json decoder as in
>> jsonHelper. I've switched to simplejson and find that it performs
>> better for large jsons like our listXxx calls. jsonHelper right now
>> makes plain python objects which can be improved to make it a response
>> object as returned by the API. So a createPhysicalNetworkResponse
>> looks like the physicalNetworkResponse and not a plain python dict.
>>
>
> I refactored the cloudstackconnection and pushed a wip branch
> requestsformarvin. Just need to iron out the POST data passing for
> cmds and the key signing. I think we can add the login mechanism based
> on session-based auth that the UI does with requests as well. That way
> we have a single module for all forms of auth done by CS.
>

Fixed in master and tested using the bvt. I added some basic support
for being able to handle POST but it's not yet clear to me which of
the APIs will handle POST. May be if API discovery exposes the
mechanism GET/POST for an API we can handle the POST data elegantly
for marvin and cloudmonkey. What say?

I'm trying to bring in further changes to marvin and add DSL support
after which I'll look into putting it up on pypi.


commit 167781ec0184f1332c0b45ca599f460d7d7b5c92
Author: Prasanna Santhanam 
Date:   Wed Apr 17 17:06:15 2013 +0530

refactor marvin to use requests instead of urllib2

Use python-requests [1] for Marvin. Requests enables graceful handling
of http connections. Marvin's cloudstackConnection has been refactored,
cleaned up to act as a single module for all kinds of cloudstack API
requesting.

TODO:
1. session based login mechanism of the UI should work from
cloudstackConnection
2. cloudmonkey can also reuse /import marvin.cloudstackConnection
3. More graceful handling of POST requests

[1] http://docs.python-requests.org/en/latest/

Signed-off-by: Prasanna Santhanam 

Re: Review Request: Remove 2k limitation for user data on a deployVMCmd issued as an HTTP POST request

[prasanna]

On 17 April 2013 23:13, Chiradeep Vittal  wrote:
> Would be great if it passes pep8 also.(just my nitpicky self)
>
Done - I'll clean up the rest of the marvin modules gradually.

Re: 4.1 Build Stalls

[prasanna]

 I've seen this stalling happen for me. Wild guess - Are you on
wireless by any chance?

On 18 April 2013 05:16, Mike Tutkowski  wrote:
> Actually, it seems kind of random.
>
> I'm trying to build 4.2 now and I had the same problem.  I just cleaned and
> installed and eventually it got past the stall.
>
>
> On Wed, Apr 17, 2013 at 2:35 PM, Chip Childers 
> wrote:
>
>> On Wed, Apr 17, 2013 at 02:34:29PM -0600, Mike Tutkowski wrote:
>> > Sorry...I followed up with that in a subsequent e-mail...I should have
>> put
>> > it in the first one.  :)
>> >
>> > mvn clean install -P developer,systemvm
>> >
>> > I then sent an e-mail saying this worked:
>> >
>> > mvn clean
>> > mvn install -P developer,systemvm
>>
>> It *might* be something in the developer or systemvm profiles.
>>
>> >
>> >
>> > On Wed, Apr 17, 2013 at 2:26 PM, Chip Childers <
>> chip.child...@sungard.com>wrote:
>> >
>> > > On Wed, Apr 17, 2013 at 02:23:04PM -0600, Mike Tutkowski wrote:
>> > > > Hi,
>> > > >
>> > > > Anyone know why the 4.1 build (updated from today) stalls for me
>> here?:
>> > > >
>> > > > Running com.cloud.utils.testcase.NioTest
>> > > > 2013-04-17 14:20:07,810 INFO  [utils.testcase.NioTest] (main:) Test
>> > > > 2013-04-17 14:20:07,853 INFO  [utils.nio.NioServer]
>> > > > (NioTestServer-Selector:) NioConnection started and listening on
>> > > > 0.0.0.0/0.0.0.0:
>> > > >
>> > > > Thanks!
>> > >
>> > > What build command are you using?
>> > >
>> >
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkow...@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud
>> > *™*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud
> *™*

Re: New Committer: Go Chiba

[prasanna]

Congratulations Go!

On 18 April 2013 12:26, Sebastien Goasguen  wrote:
> The Project Management Committee (PMC) for Apache CloudStack has asked Go 
> Chiba to become a committer and we are pleased to announced that they have 
> accepted.
>
> Go has contributed tremendously to the Japanese translations of the 
> documentation (runbook, UI and docs).
> Being a committer allows many contributors to contribute more autonomously. 
> For developers, it makes it easier to submit changes and eliminates the need 
> to have contributions reviewed via the patch submission process. Whether 
> contributions are development-related or otherwise, it is a recognition of a 
> contributor's participation in the project and commitment to the project and 
> the Apache Way.
> Please join me in congratulating Go,
>
> -Sebastien, on behalf of  the Apache CloudStack PMC
>

Re: Unable to add XenServer host to basic zone

[prasanna]

On 19 April 2013 16:29, Venkata SwamyBabu Budumuru
 wrote:
> Hi,
>
> I am seeing the following issue on the MASTER build. Currently I filled it 
> with priority set to "critical". If anyone else seeing the same either you 
> can apply the workaround mentioned in the bug or you can raise the priority.
>
> https://issues.apache.org/jira/browse/CLOUDSTACK-2115 - [BasicZone-XenServer] 
> Unable to add host to basic zone that is configured with bridge

Would you like to submit a patch request with the
result.lower().strip() applied to the vmops plugin? Seems like a
fairly trivial fix.

Re: [DISCUSS] ACS Release 4 month v/s 6 month

[prasanna]

On 26 April 2013 05:32, Animesh Chaturvedi
 wrote:
> Let me attempt to summarize this thread, if I missed any glaring points feel 
> free to bring them up
>
> 4 months:
> Proponents (9): Chip, Alex, David, Noah, Hugo, Joe,  Sebastian, Prasanna, 
> Rohit
> Reasoning:
> *   We have not given proper shot to 4 month cycle, this was just the 
> first time. Level of automation has increased between 4.0 to 4.1 which lays 
> groundwork for better automation
> *   Longer feature cycle will mean more features and bigger and more 
> complex release
> *   Faster feedback loop to respond and address problems and shorter wait 
> time for feature delivery
>
>
> 6 months:
> Proponents (12): Will, Animesh, Edison, Frank, Min, Ilya, Kelven, Edison, 
> Sudha, Radhika, Nitin, Mice

I see Edison's stuffed two votes in there :)

Good Summary! Thanks,

Re: Error starting management server

[Prasanna]

What's your python command on the command line? And the json file that
you passed in to run it. Put it up on apaste.info?

-- 
Prasanna.,

On Thu, Apr 25, 2013 at 06:04:14PM +0530, Dharmesh Kakadia wrote:
> Hi,
> 
> I was able to only run management server. Now stuck at step-8. I tried
> running it both way (mvn and python).
> 
> Traceback (most recent call last):
>   File "../marvin/marvin/deployDataCenter.py", line 469, in 
> deploy.deploy()
>   File "../marvin/marvin/deployDataCenter.py", line 452, in deploy
> self.loadCfg()
>   File "../marvin/marvin/deployDataCenter.py", line 409, in loadCfg
> apiKey, securityKey = self.registerApiKey()
>   File "../marvin/marvin/deployDataCenter.py", line 349, in registerApiKey
> listuserRes = self.testClient.getApiClient().listUsers(listuser)
>   File
> "/Users/GreatGod/cloudstack/cloudstack/tools/marvin/marvin/cloudstackAPI/cloudstackAPIClient.py",
> line 436, in listUsers
> response = self.connection.marvin_request(command,
> response_type=response, method=method)
>   File
> "/Users/GreatGod/cloudstack/cloudstack/tools/marvin/marvin/cloudstackConnection.py",
> line 208, in marvin_request
> response = jsonHelper.getResultObj(response.json(), response_type)
>   File "/Library/Python/2.7/site-packages/requests/models.py", line 638, in
> json
> return json.loads(self.text or self.content, **kwargs)
>   File
> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py",
> line 326, in loads
> return _default_decoder.decode(s)
>   File
> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py",
> line 360, in decode
> obj, end = self.raw_decode(s, idx=_w(s, 0).end())
>   File
> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py",
> line 378, in raw_decode
> raise ValueError("No JSON object could be decoded")
> ValueError: No JSON object could be decoded
> 
> 
> I am using python2.7 and have version 2.6 installed on system.
> 
> Also, I need help in using the management UI with devcloud. I am newbie and
> guessed most of the values, but I am not able to add devcloud host
> (192.168.56.10). Any pointers ?
> 
> Thanks,
> Dharmesh
> 
> 
> On Thu, Apr 25, 2013 at 12:36 PM, Dharmesh Kakadia wrote:
> 
> > Hi,
> >
> > Thanks. That was exactly what I was missing. Running now !!
> >
> > Thanks,
> > Dharmesh
> >
> >
> > On Thu, Apr 25, 2013 at 10:34 AM, Prasanna Santhanam wrote:
> >
> >> On Wed, Apr 24, 2013 at 11:07:38PM +0530, Dharmesh Kakadia wrote:
> >> > Hi,
> >> >
> >> > I am setting up dev environment following :
> >> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/DevCloud
> >>
> >> Dev environment specifics are here:
> >>
> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Setting+up+CloudStack+Development+Environment
> >>
> >> If you are using master branch for building:
> >>
> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/How+to+build+on+master+branch
> >>
> >> You might want to give more memory as shown in the wiki:
> >> export MAVEN_OPTS="-XX:MaxPermSize=512m -Xmx2g -Xdebug
> >> -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n"
> >>
> >> The supported tomcat version is 6.0.33. So you might want to downgrade
> >> and setup your CATALINA_HOME to point to the correct tomcat install.
> >>
> >> --
> >> Prasanna.,
> >>
> >> >
> >> > I am stuck on step 7.3 while starting the management server.
> >> >
> >> > the output of
> >> >
> >> > mvn -pl :cloud-client-ui jetty:run
> >> >
> >> > gets stuck after
> >> >
> >> > INFO  [utils.component.ComponentContext] (main:) Setup Spring
> >> Application
> >> > context
> >> >
> >> > I tried starting client-ui manually by copying the cloud-client-ui war
> >> file
> >> > into tomcat.
> >> >
> >> > Interestingly the log says
> >> >
> >> > SEVERE: Error listenerStart
> >> > Apr 24, 2013 6:32:38 PM org.apache.catalina.core.StandardContext start
> >> > SEVERE: Context [/cloud-client-ui-4.2.0-SNAPSHOT] startup failed due to
> >> > previous errors
> >> > Apr 24, 2013 6:32:38 PM org.apache.catalina.loader.WebappClassLoader
&

Re: Review Request: When associating an IP to a VPC allow admin's to specify what IP they want to associate. (CLOUDSTACK-1942)

[Prasanna]

Sorry - I never got to this in time. +1 to merge and I'll take a look
at the marvin test post-merge.

On Tue, Apr 30, 2013 at 02:48:23PM -, Marcus Sorensen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10423/#review19951
> ---
> 
> 
> The crickets have it.
> 
> This patch doesn't apply cleanly. If you could update it I'll apply it to 
> master.
> 
> Thanks!
> 
> - Marcus Sorensen
> 
> 
> On April 12, 2013, 4:12 p.m., Ryan Dietrich wrote:
> > 
> > ---
> > This is an automatically generated e-mail. To reply, visit:
> > https://reviews.apache.org/r/10423/
> > ---
> > 
> > (Updated April 12, 2013, 4:12 p.m.)
> > 
> > 
> > Review request for cloudstack, Prasanna Santhanam, Chiradeep Vittal, and 
> > Marcus Sorensen.
> > 
> > 
> > Description
> > ---
> > 
> > https://issues.apache.org/jira/browse/CLOUDSTACK-1942
> > 
> > This patch will allow you to specify an IP when associating an IP to a VPC.
> > 
> > 
> > Diffs
> > -
> > 
> >   api/src/com/cloud/network/NetworkService.java ab6d7bf 
> >   
> > api/src/org/apache/cloudstack/api/command/user/address/AssociateIPAddrCmd.java
> >  406f782 
> >   server/src/com/cloud/network/NetworkManager.java 4af716c 
> >   server/src/com/cloud/network/NetworkManagerImpl.java a97f2ce 
> >   server/src/com/cloud/network/NetworkServiceImpl.java 4eb620c 
> >   server/test/com/cloud/network/MockNetworkManagerImpl.java 18eae08 
> >   server/test/com/cloud/vpc/MockNetworkManagerImpl.java bd4fd67 
> > 
> > Diff: https://reviews.apache.org/r/10423/diff/
> > 
> > 
> > Testing
> > ---
> > 
> > Manually tested using master branch, as well as 4.1.  Tested manually using 
> > "ipid" as well as leaving it off resulting in existing functionality.
> > 
> > Also, ran new marvin test.  This probably could have been incorporated into 
> > the the existing test_network.py file, but half of those tests were failing 
> > on my system.
> > 
> > 
> > Thanks,
> > 
> > Ryan Dietrich
> > 
> >
> 

-- 
Prasanna.,

Re: Review Request 14148: Cleanup DefaultUserAuthenticator and removed masking _name variable

[Prasanna Santhanam]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14148/
---

(Updated Sept. 18, 2013, 5:38 a.m.)


Review request for cloudstack and Abhinandan Prateek.


Changes
---

adding kelven for spring changes.


Repository: cloudstack-git


Description
---

DefaultUserAuthenticator maskes the _name varible in ComponentLifecycleBase 
making the setName() method not work as expected.  This patch cleans up the 
code such that getName() will be getClass().getSimpleName() unless overridden 
in the Spring configuration.


Diffs
-

  
plugins/user-authenticators/ldap/src/org/apache/cloudstack/ldap/LdapAuthenticator.java
 e62a3d8 
  
plugins/user-authenticators/md5/src/com/cloud/server/auth/MD5UserAuthenticator.java
 e5b169f 
  
plugins/user-authenticators/plain-text/src/com/cloud/server/auth/PlainTextUserAuthenticator.java
 f102275 
  
plugins/user-authenticators/sha256salted/src/com/cloud/server/auth/SHA256SaltedUserAuthenticator.java
 91be922 
  server/src/com/cloud/server/auth/DefaultUserAuthenticator.java 952f724 

Diff: https://reviews.apache.org/r/14148/diff/


Testing
---


Thanks,

Darren Shepherd

Re: Review Request 14148: Cleanup DefaultUserAuthenticator and removed masking _name variable

[Prasanna Santhanam]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14148/
---

(Updated Sept. 18, 2013, 5:39 a.m.)


Review request for cloudstack, Abhinandan Prateek and Kelven Yang.


Repository: cloudstack-git


Description
---

DefaultUserAuthenticator maskes the _name varible in ComponentLifecycleBase 
making the setName() method not work as expected.  This patch cleans up the 
code such that getName() will be getClass().getSimpleName() unless overridden 
in the Spring configuration.


Diffs
-

  
plugins/user-authenticators/ldap/src/org/apache/cloudstack/ldap/LdapAuthenticator.java
 e62a3d8 
  
plugins/user-authenticators/md5/src/com/cloud/server/auth/MD5UserAuthenticator.java
 e5b169f 
  
plugins/user-authenticators/plain-text/src/com/cloud/server/auth/PlainTextUserAuthenticator.java
 f102275 
  
plugins/user-authenticators/sha256salted/src/com/cloud/server/auth/SHA256SaltedUserAuthenticator.java
 91be922 
  server/src/com/cloud/server/auth/DefaultUserAuthenticator.java 952f724 

Diff: https://reviews.apache.org/r/14148/diff/


Testing
---


Thanks,

Darren Shepherd

Re: Review Request 14230: Storage type was set to local due to which tests failed.

[Prasanna Santhanam]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14230/#review26268
---

Ship it!


- Prasanna Santhanam


On Sept. 19, 2013, 1:20 p.m., Girish Shilamkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14230/
> ---
> 
> (Updated Sept. 19, 2013, 1:20 p.m.)
> 
> 
> Review request for cloudstack, Harikrishna Patnala and Prasanna Santhanam.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Storage type was set to local due to which tests failed.
> 
> 
> Diffs
> -
> 
>   test/integration/component/test_reset_ssh_keypair.py 8b499d0 
> 
> Diff: https://reviews.apache.org/r/14230/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Girish Shilamkar
> 
>

[MERGE] marvin-refactor to master

[Prasanna Santhanam]

Once upon a time [1] I had propagated the idea of refactoring marvin to
make test case writing simpler. At the time, there weren't enough
people writing tests using marvin however. Now as focus on testing has
become much more important for the stability of our releases I would
like to bring back the discussion and to review the refactoring of
marvin which I've been doing in the marvin_refactor branch.

The key goal of this refactor was to simplify test case writing. In
doing so I've transformed the library from its brittle hand-written
nature to a completely auto-generated set of libraries. In that sense,
marvin is much closer to cloudmonkey now.

The two important changes in this refactor are:

1. data represented in an object-oriented fashion presented as factories
2. test case writing using entities and their operations rather than
a sequence of disconnected API calls.

To see the full nature of this proposal I've updated the spec I put up
on the wiki:
https://cwiki.apache.org/confluence/x/RI3lAQ

For a quick comparison I wrote a test for the VPC vm's lifecycle in
tools/marvin/marvin/test/test_vpc_life_cycle.py which one can compare
with the existing tests for vpc under
test/integration/component/test_vpc_vm_life_cycle.py

These changes being 'architectural' so to speak and in a way
disruptive even I would like to merge this at the beginning of the
upcoming cloudstack release.

This is only a small part of a larger change for marvin which will be
moving to a more BDD like implementation [2] where tests are written
using a gherkin-like language. But that will come later.

I've also tried to disconnect marvin from depending on CloudStack's
build and repo. This will help split marvin from CloudStack which I
will discuss in a seperate thread.

[1] http://markmail.org/message/4tsscn6zvtfsskuj
[2] http://pythonhosted.org/behave/

-- 
Prasanna.,


Powered by BigRock.com

Re: [MERGE] marvin-refactor to master

[Prasanna Santhanam]

On Wed, Oct 02, 2013 at 12:51:18PM -0400, Chip Childers wrote:
> On Wed, Oct 02, 2013 at 10:12:40PM +0530, Prasanna Santhanam wrote:
> > I've also tried to disconnect marvin from depending on CloudStack's
> > build and repo. This will help split marvin from CloudStack which I
> > will discuss in a seperate thread.
> 
> Should we just use this branch as the source for a new repo, and move on
> from there?

I haven't fully disconnected it but will start a DISCUSS thread if
there are any concerns about splitting it from the existing repo. If
there is consensus we can break it apart from the branch.

-- 
Prasanna.,


Powered by BigRock.com

[DISCUSS] Breaking out Marvin from CloudStack

[Prasanna Santhanam]

I would like to seperate marvin from the main cloudstack repo. Much of
marvin's development has little coupling with CloudStack. 

Similar to CloudMonkey, marvin undergoes rapid changes and it is
essential to provide a smooth workflow and faster releases for those
working with it. 

There are also a small set of people currently looking at marvin for
testing right now. Often, their reviews and QA effort is mixed with
those of cloudstack itself. By having a different repo I'd like to be
able to provide commit access to those working on marvin alone quickly
to help with testing.

After separating marvin
0. we will have a separate release cycle for marvin
1. we will have a new home for marvin's docs using Sphinx
2. if possible, a different criteria for providing commit access to
marvin's repos.
3. all tests of cloudstack will also move to marvin's repository

Thoughts?

-- 
Prasanna.,


Powered by BigRock.com

Re: contrail virtual network controller

[Prasanna Santhanam]

On Tue, Sep 17, 2013 at 10:18:29AM -0700, Pedro Roque Marques wrote:
> 
> We have an integration test which requires spawning a "mysql" db,
> initialize it and then go through the process of creating a zone,
> etc.  This allows us to test the plugin against the API. The
> drawback is that this test takes a bit to spawn the mysql db...
> Does anyone have a recipe to run against an in-memory database for
> testing purposes ? 
> 

Does the mysqldb contain fixture data from when contrail is deployed
on a cloudstack cloud? I presume then the integration test can run
without the contrail device added to cloudstack?

-- 
Prasanna.,


Powered by BigRock.com

Re: marvin over https

[Prasanna Santhanam]

On Thu, Sep 26, 2013 at 04:21:38PM +0200, Daan Hoogland wrote:
> H,
> 
> I have some trouble getting marvin to connect to cloudstack over https.
> I am supposing the following should work
> conn = cloudConnection(mgmtip, apiKey=apikey,
> securityKey=secretkey, logging=log, port=443, scheme="https")
> lz = listZones.listZonesCmd()
> conn.marvin_request(lz)
> 
> is this a valid assumption?
> 
> I can browse to the https:///client/ and login to retrieve the
> keys used, but on running the code above i get
> 
> requests.exceptions.ConnectionError:
> HTTPSConnectionPool(host='10.200.23.16', port=443): Max retries
> exceeded with url:
> /client/api?apiKey=JGvIQPeIVsbgEhVC3shZ51r9buYwClB4ToJZX9Cxs9e3NZbRoJLNyANnWEKgsmgt1uoF_eLdL31GHMwcss6Zyw&command=listZones&signature=KL93r9GYIr6%2FRcbNHuaOj3jUF6o%3D&response=json
> (Caused by : [Errno 111] Connection refused)

In the loglevel() method in CloudConnection.py, switch the logging to
logging.DEBUG. That will spew out more verbose logging as to what's
happening here.

I've never tried it on an https enabled cloudstack so there might be a
bug. Does cloudmonkey work for you on this endpoint? If yes, then I
don't see why marvin shouldn't. Both use the same request mechanism.

> 
> I am not sure where to look. at marvin, httprequest or the setup of my
> env. Hints?
> 
> thanks,
> Daan

-- 
Prasanna.,


Powered by BigRock.com

Re: [MERGE] marvin-refactor to master

[Prasanna Santhanam]

Copying folks that this change will affect for review.

On Wed, Oct 02, 2013 at 10:12:40PM +0530, Prasanna Santhanam wrote:
> Once upon a time [1] I had propagated the idea of refactoring marvin to
> make test case writing simpler. At the time, there weren't enough
> people writing tests using marvin however. Now as focus on testing has
> become much more important for the stability of our releases I would
> like to bring back the discussion and to review the refactoring of
> marvin which I've been doing in the marvin_refactor branch.
> 
> The key goal of this refactor was to simplify test case writing. In
> doing so I've transformed the library from its brittle hand-written
> nature to a completely auto-generated set of libraries. In that sense,
> marvin is much closer to cloudmonkey now.
> 
> The two important changes in this refactor are:
> 
> 1. data represented in an object-oriented fashion presented as factories
> 2. test case writing using entities and their operations rather than
> a sequence of disconnected API calls.
> 
> To see the full nature of this proposal I've updated the spec I put up
> on the wiki:
> https://cwiki.apache.org/confluence/x/RI3lAQ
> 
> For a quick comparison I wrote a test for the VPC vm's lifecycle in
> tools/marvin/marvin/test/test_vpc_life_cycle.py which one can compare
> with the existing tests for vpc under
> test/integration/component/test_vpc_vm_life_cycle.py
> 
> These changes being 'architectural' so to speak and in a way
> disruptive even I would like to merge this at the beginning of the
> upcoming cloudstack release.
> 
> This is only a small part of a larger change for marvin which will be
> moving to a more BDD like implementation [2] where tests are written
> using a gherkin-like language. But that will come later.
> 
> I've also tried to disconnect marvin from depending on CloudStack's
> build and repo. This will help split marvin from CloudStack which I
> will discuss in a seperate thread.
> 
> [1] http://markmail.org/message/4tsscn6zvtfsskuj
> [2] http://pythonhosted.org/behave/
> 
> -- 
> Prasanna.,
> 
> 
> Powered by BigRock.com

-- 
Prasanna.,


Powered by BigRock.com

Re: [MERGE] spring-modularization to master - Spring Modularization

[Prasanna Santhanam]

I switched the test infrastructure on jenkins.buildacloud.org to run
the bvts [1] against master last week. Couple of weeks before that the
simulator [2] tests were switched to run against master. Both are
broken unfortunately and the bvt and checkin tests aren't running.

I've filed a bug here:
https://issues.apache.org/jira//browse/CLOUDSTACK-4791

[1] http://jenkins.buildacloud.org/view/cloudstack-qa/
[2] http://jenkins.buildacloud.org/view/simulator/

Anyone who has access to the jenkins can run the bvts on their desired
branch. Simply login and change the test-yumrepo-refresh job to point
to your branch. Build that to refresh the remote repository with the
packages made from your branch. Then switch test-matrix to point to
the same development branch and fire a build. That's about it.

On Wed, Oct 02, 2013 at 05:42:54PM -0700, Darren Shepherd wrote:
> Yes agreed.  I've extensively tested this, but that is never enough.
> How do I get the BVTs ran against this.  Due to the cross cutting
> nature of this I want to get this merged as fast as possible.  
> 
> Darren
> 
> > On Oct 2, 2013, at 4:43 PM, Alex Huang  wrote:
> > 
> > +1 on running the BVT on it.  We've been through this one once before.  
> > Should be careful.
> > 
> > --Alex
> > 
> >> -Original Message-
> >> From: Kelven Yang [mailto:kelven.y...@citrix.com]
> >> Sent: Wednesday, October 2, 2013 4:39 PM
> >> To: dev@cloudstack.apache.org
> >> Subject: Re: [MERGE] spring-modularization to master - Spring
> >> Modularization
> >> 
> >> Darren,
> >> 
> >> This looks really nice. A few questions on Spring AOP replacement.
> >> 
> >> 1) Spring AOP is proxy-based, the reason we ended up of using customized
> >> AOP is mainly due to that inside existing CloudStack codebase, we have many
> >> places that are doing run-time type-casting, the code in these places
> >> assumes a real object that implements all interfaces in the semantics 
> >> context.
> >> At the time when I initially converted to Spring, I couldn't ensure that
> >> switching to proxy-based AOP can have 100% coverage for these run-time
> >> cases. What is your approach to address this run-time type-casting issue?
> >> 
> >> 2) We've run into a huge-memory footprint issue that may be caused by
> >> conflicts of CGLIB usage in Spring AOP and the CGLIB usage in CloudStack 
> >> Dao
> >> layer. Do you have a chance to run a memory analysis in the heap after
> >> management server is started.
> >> 
> >> I might be good to run BVT test on the branch before the merge, could
> >> someone initiate the effort?
> >> 
> >> kelven
> >> 
> >> 
> >> 
> >> On 10/2/13 3:48 PM, "Darren Shepherd" 
> >> wrote:
> >> 
> >>> Not sure how this works...  I would like to merge in the new
> >>> modularized Spring setup to master. There is info on the wiki about it
> >>> [1] [2] [3].  The primary change is to break apart the monolithic
> >>> applicationContext.xml and componentContext.xml files such that each
> >>> plugin can maintain and contribute its own configuration.
> >>> 
> >>> In addition to breaking up the configuration we no longer use ACS
> >>> custom AOP and it is now fully Spring AOP.
> >>> 
> >>> Now adding/removing a plug-in is a matter of just adding a jar to the
> >>> classpath (exception being commands.properties, I'll address that in a
> >>> different thread).  Unfortunately this branch does not have the changes
> >>> to package things in different RPMs.  So it would be great if somebody
> >>> could take up the packaging effort to split out all the plugins into
> >>> different RPMs.
> >>> 
> >>> Darren
> >>> 
> >>> [1]
> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Sp
> >> rin
> >>> g
> >>> [2]
> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-
> >> ins%2C+Modu
> >>> les
> >>> %2C+and+Extensions
> >>> [3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
> > 

-- 
Prasanna.,


Powered by BigRock.com

Re: Configuration table changes and database update

[Prasanna Santhanam]

t; com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:415)
>   ... 116 more
> 
> The full log is here: 
> http://nvpmadm1.nvp.strocamp.net:8080/job/cloudstack-qa-env-test/505/console
> 
> This still appears to be an issue. How can we make sure that the database is 
> properly updated before this piece of code is hit? 
> 
> Cheers,
> 
> Hugo
> 
> 
> On Sep 24, 2013, at 4:13 PM, Daan Hoogland  wrote:
> 
> > works in the latest version
> > 
> > On Tue, Sep 24, 2013 at 4:09 PM, Daan Hoogland  
> > wrote:
> >> I ran into this as well, As I did some change to schema-420-430.sql I
> >> was my primary suspect. It does not have to do with my field though.
> >> Still looking,
> >> Daan
> >> 
> >> On Tue, Sep 24, 2013 at 2:17 PM, Wei ZHOU  wrote:
> >>> I ran a fresh installation on devcloud just now, it works.
> >>> some records in configuration table are introduced by sql files, and
> >>> ConfigurationServerImpl will check and insert the records if not exist.
> >>> 
> >>> 
> >>> 2013/9/24 Hugo Trippaers 
> >>> 
> >>>> Hey all,
> >>>> 
> >>>> Noticed an interesting problem today. I was trying to start a management
> >>>> server based on the latest sources in master, but failed. The reason was
> >>>> that the configuration threw an exception that a certain column in the
> >>>> database fit not exist. This column is added during the database upgrade
> >>>> sequence, but apparently the configuration is already accessed before the
> >>>> database upgrade takes place.
> >>>> 
> >>>> Seems like a chicken and egg problem to me.
> >>>> 
> >>>> Did anybody else run into this problem?
> >>>> 
> >>>> Cheers,
> >>>> 
> >>>> Hugo

-- 
Prasanna.,


Powered by BigRock.com

Re: [DOCS] CS-docs patch

[Prasanna Santhanam]

On Thu, Oct 03, 2013 at 02:10:26PM -0400, Travis Graham wrote:
> Here is a patch for the "Deployment Architecture Overview" section
> until things get setup in Review Board for the new repo or there's a
> GitHub mirror that pull requests can be sent to for review. Will the
> github mirror and PR route be supported in tandem with the Review
> Board process or not at all?
> 
> It adds in links to other docs it only loosely referenced by name
> and adds the Primary Storage info about being zone wide for 4.2.
> 
> Travis
Attachments are stripped

Please paste the contents of your patch in plaintext with the subject
[PATCH]. Also ensure you've created your patch using git format-patch
so I can apply it readily and attribute credit to you.

I'd love to have PRs via github but that's upto asf infra and not
project specific to answer.


Thanks,


> 


-- 
Prasanna.,


Powered by BigRock.com

Re: Review Request 14471: CloudStack Plugin guide for VXLAN

[Prasanna Santhanam]

Hi,

The docs were broken out into a separate repo 2 days ago. So this
should be based on the repo cloudstack-docs.git. Can you resubmit this
for that repo?

See:
http://markmail.org/message/w44nioqt5usechhc

Thanks,


On Thu, Oct 03, 2013 at 08:20:52PM -, Yoshikazu Nojima wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14471/
> ---
> 
> Review request for cloudstack and Toshiaki Hatano.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> This patch adds a Plugin guide for VXLAN.
> It describes system requirements for VXLAN plugin and how to setup it.
> 
> This is the ToC:
> 1. The VXLAN Plugin  1
> 1.1. Introduction to the VXLAN Plugin  1
> 1.2. Features of the VXLAN Plugin  1
> 2. System Requirements for VXLAN 3
> 2.1. System Requirements for VXLAN  3
> 2.2. Linux Distributions that meet the requirements  3
> 2.3. Check the capability of your system  3
> 2.4. Advanced: Build kernel and iproute2  3
> 2.4.1. Build kernel  4
> 2.4.2. Build iproute2  4
> 3. Configure CloudStack to use VXLAN Plugin 7
> 3.1. Configure hypervisor  7
> 3.1.1. Configure hypervisor: KVM  7
> 3.2. Setup zone using VXLAN  9
> 3.2.1. Configure the physical network  10
> 
> 
> Diffs
> -
> 
>   docs/en-US/CloudStack_VXLAN_Guide.ent PRE-CREATION 
>   docs/en-US/CloudStack_VXLAN_Guide.xml PRE-CREATION 
>   docs/en-US/images/vxlan-physicalnetwork.png PRE-CREATION 
>   docs/en-US/images/vxlan-trafficlabel.png PRE-CREATION 
>   docs/en-US/plugin-vxlan-about.xml PRE-CREATION 
>   docs/en-US/plugin-vxlan-config-hypervisor.xml PRE-CREATION 
>   docs/en-US/plugin-vxlan-config-management.xml PRE-CREATION 
>   docs/en-US/plugin-vxlan-configuration.xml PRE-CREATION 
>   docs/en-US/plugin-vxlan-features.xml PRE-CREATION 
>   docs/en-US/plugin-vxlan-introduction.xml PRE-CREATION 
>   docs/en-US/plugin-vxlan-requirements.xml PRE-CREATION 
>   docs/en-US/plugin-vxlan-revision-history.xml PRE-CREATION 
>   docs/publican-plugin-vxlan.cfg PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/14471/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Yoshikazu Nojima
> 

-- 
Prasanna.,


Powered by BigRock.com

Re: [MERGE] spring-modularization to master - Spring Modularization

[Prasanna Santhanam]

On Thu, Oct 03, 2013 at 02:46:40PM +, Rayees Namathponnan wrote:
> Hi Prasanna,
> 
> If we are running BVT or regression on specific branch, marvin
> should also from same branch right ? but we always getting marvin
> from master branch  job "cloudstack-marvin"
> 

Yes, but marvin's core is the same in master and 4.2 at the moment.
That will change if and when the merge of the refactoring happens. Can
you help fix the jenkins job to fetch marvin from the repo the test is
targetted towards?

-- 
Prasanna.,


Powered by BigRock.com

Re: [MERGE] spring-modularization to master - Spring Modularization

[Prasanna Santhanam]

Thanks. I will first run a baseline against master since we don't have
one and then one on spring modularization. The baseline against 4.2
shows this result: 

http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-regression-matrix/252/testReport/

On Thu, Oct 03, 2013 at 09:19:23AM -0700, Darren Shepherd wrote:
> This should be fixed on the spring-modularization branch.
> 
> Darren
> 
> On Wed, Oct 2, 2013 at 10:05 PM, Prasanna Santhanam  wrote:
> > I switched the test infrastructure on jenkins.buildacloud.org to run
> > the bvts [1] against master last week. Couple of weeks before that the
> > simulator [2] tests were switched to run against master. Both are
> > broken unfortunately and the bvt and checkin tests aren't running.
> >
> > I've filed a bug here:
> > https://issues.apache.org/jira//browse/CLOUDSTACK-4791
> >
> > [1] http://jenkins.buildacloud.org/view/cloudstack-qa/
> > [2] http://jenkins.buildacloud.org/view/simulator/
> >
> > Anyone who has access to the jenkins can run the bvts on their desired
> > branch. Simply login and change the test-yumrepo-refresh job to point
> > to your branch. Build that to refresh the remote repository with the
> > packages made from your branch. Then switch test-matrix to point to
> > the same development branch and fire a build. That's about it.
> >
> > On Wed, Oct 02, 2013 at 05:42:54PM -0700, Darren Shepherd wrote:
> >> Yes agreed.  I've extensively tested this, but that is never enough.
> >> How do I get the BVTs ran against this.  Due to the cross cutting
> >> nature of this I want to get this merged as fast as possible.
> >>
> >> Darren
> >>
> >> > On Oct 2, 2013, at 4:43 PM, Alex Huang  wrote:
> >> >
> >> > +1 on running the BVT on it.  We've been through this one once before.  
> >> > Should be careful.
> >> >
> >> > --Alex
> >> >
> >> >> -Original Message-
> >> >> From: Kelven Yang [mailto:kelven.y...@citrix.com]
> >> >> Sent: Wednesday, October 2, 2013 4:39 PM
> >> >> To: dev@cloudstack.apache.org
> >> >> Subject: Re: [MERGE] spring-modularization to master - Spring
> >> >> Modularization
> >> >>
> >> >> Darren,
> >> >>
> >> >> This looks really nice. A few questions on Spring AOP replacement.
> >> >>
> >> >> 1) Spring AOP is proxy-based, the reason we ended up of using customized
> >> >> AOP is mainly due to that inside existing CloudStack codebase, we have 
> >> >> many
> >> >> places that are doing run-time type-casting, the code in these places
> >> >> assumes a real object that implements all interfaces in the semantics 
> >> >> context.
> >> >> At the time when I initially converted to Spring, I couldn't ensure that
> >> >> switching to proxy-based AOP can have 100% coverage for these run-time
> >> >> cases. What is your approach to address this run-time type-casting 
> >> >> issue?
> >> >>
> >> >> 2) We've run into a huge-memory footprint issue that may be caused by
> >> >> conflicts of CGLIB usage in Spring AOP and the CGLIB usage in 
> >> >> CloudStack Dao
> >> >> layer. Do you have a chance to run a memory analysis in the heap after
> >> >> management server is started.
> >> >>
> >> >> I might be good to run BVT test on the branch before the merge, could
> >> >> someone initiate the effort?
> >> >>
> >> >> kelven
> >> >>
> >> >>
> >> >>
> >> >> On 10/2/13 3:48 PM, "Darren Shepherd" 
> >> >> wrote:
> >> >>
> >> >>> Not sure how this works...  I would like to merge in the new
> >> >>> modularized Spring setup to master. There is info on the wiki about it
> >> >>> [1] [2] [3].  The primary change is to break apart the monolithic
> >> >>> applicationContext.xml and componentContext.xml files such that each
> >> >>> plugin can maintain and contribute its own configuration.
> >> >>>
> >> >>> In addition to breaking up the configuration we no longer use ACS
> >> >>> custom AOP and it is now fully Spring AOP.
> >> >>>
> >> >>> Now adding/removing a plug-in is a matter of just adding a jar to the
> >> >>> classpath (exception being commands.properties, I'll address that in a
> >> >>> different thread).  Unfortunately this branch does not have the changes
> >> >>> to package things in different RPMs.  So it would be great if somebody
> >> >>> could take up the packaging effort to split out all the plugins into
> >> >>> different RPMs.
> >> >>>
> >> >>> Darren
> >> >>>
> >> >>> [1]
> >> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Sp
> >> >> rin
> >> >>> g
> >> >>> [2]
> >> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-
> >> >> ins%2C+Modu
> >> >>> les
> >> >>> %2C+and+Extensions
> >> >>> [3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
> >> >
> >
> > --
> > Prasanna.,
> >
> > 
> > Powered by BigRock.com
> >

-- 
Prasanna.,


Powered by BigRock.com

Re: [MERGE] spring-modularization to master - Spring Modularization

[Prasanna Santhanam]

Hey Darren, the build appears to be broken on spring-modularization.
Actually a test is broken. If fixed I could generate some packages for
testing.

http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-yumrepo-refresh/3028/console

On Fri, Oct 04, 2013 at 10:27:55AM +0530, Prasanna Santhanam wrote:
> Thanks. I will first run a baseline against master since we don't have
> one and then one on spring modularization. The baseline against 4.2
> shows this result: 
> 
> http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-regression-matrix/252/testReport/
> 
> On Thu, Oct 03, 2013 at 09:19:23AM -0700, Darren Shepherd wrote:
> > This should be fixed on the spring-modularization branch.
> > 
> > Darren
> > 
> > On Wed, Oct 2, 2013 at 10:05 PM, Prasanna Santhanam  wrote:
> > > I switched the test infrastructure on jenkins.buildacloud.org to run
> > > the bvts [1] against master last week. Couple of weeks before that the
> > > simulator [2] tests were switched to run against master. Both are
> > > broken unfortunately and the bvt and checkin tests aren't running.
> > >
> > > I've filed a bug here:
> > > https://issues.apache.org/jira//browse/CLOUDSTACK-4791
> > >
> > > [1] http://jenkins.buildacloud.org/view/cloudstack-qa/
> > > [2] http://jenkins.buildacloud.org/view/simulator/
> > >
> > > Anyone who has access to the jenkins can run the bvts on their desired
> > > branch. Simply login and change the test-yumrepo-refresh job to point
> > > to your branch. Build that to refresh the remote repository with the
> > > packages made from your branch. Then switch test-matrix to point to
> > > the same development branch and fire a build. That's about it.
> > >
> > > On Wed, Oct 02, 2013 at 05:42:54PM -0700, Darren Shepherd wrote:
> > >> Yes agreed.  I've extensively tested this, but that is never enough.
> > >> How do I get the BVTs ran against this.  Due to the cross cutting
> > >> nature of this I want to get this merged as fast as possible.
> > >>
> > >> Darren
> > >>
> > >> > On Oct 2, 2013, at 4:43 PM, Alex Huang  wrote:
> > >> >
> > >> > +1 on running the BVT on it.  We've been through this one once before. 
> > >> >  Should be careful.
> > >> >
> > >> > --Alex
> > >> >
> > >> >> -Original Message-
> > >> >> From: Kelven Yang [mailto:kelven.y...@citrix.com]
> > >> >> Sent: Wednesday, October 2, 2013 4:39 PM
> > >> >> To: dev@cloudstack.apache.org
> > >> >> Subject: Re: [MERGE] spring-modularization to master - Spring
> > >> >> Modularization
> > >> >>
> > >> >> Darren,
> > >> >>
> > >> >> This looks really nice. A few questions on Spring AOP replacement.
> > >> >>
> > >> >> 1) Spring AOP is proxy-based, the reason we ended up of using 
> > >> >> customized
> > >> >> AOP is mainly due to that inside existing CloudStack codebase, we 
> > >> >> have many
> > >> >> places that are doing run-time type-casting, the code in these places
> > >> >> assumes a real object that implements all interfaces in the semantics 
> > >> >> context.
> > >> >> At the time when I initially converted to Spring, I couldn't ensure 
> > >> >> that
> > >> >> switching to proxy-based AOP can have 100% coverage for these run-time
> > >> >> cases. What is your approach to address this run-time type-casting 
> > >> >> issue?
> > >> >>
> > >> >> 2) We've run into a huge-memory footprint issue that may be caused by
> > >> >> conflicts of CGLIB usage in Spring AOP and the CGLIB usage in 
> > >> >> CloudStack Dao
> > >> >> layer. Do you have a chance to run a memory analysis in the heap after
> > >> >> management server is started.
> > >> >>
> > >> >> I might be good to run BVT test on the branch before the merge, could
> > >> >> someone initiate the effort?
> > >> >>
> > >> >> kelven
> > >> >>
> > >> >>
> > >> >>
> > >> >> On 10/2/13 3:48 PM, "Darren Shepherd" 
> > >> >> wrote:
> > >>

Re: [DISCUSS] Breaking out Marvin from CloudStack

[Prasanna Santhanam]

I'll summarize and address the concerns raised so far.

Marvin has been in this repo for a long time for us to start writing
tests. The only tests I've seen coming are from a specific set of
people focussed on QA efforts. I want to reduce the impediment for
people who are writing tests *today*. Those looking to get started in
the near future won't have any new learning to do, just that their
code goes in an alternate repo that is pointed to right
infrastructure. 

Automated testing also works in a push-to-production style very often.
Testers need to run their tests on a deployed environment(s) quickly
to be able to ensure it is valid and passes. By making them go through
reviewboard each time for each test we massively slow down the
process. (tons of fixes to tests are on rb today, not just new tests).
We don't know if they run until they run on the environment.

Reason for tests and framework to go together is simple.  If I go look
at the jclouds repository today I find tests for rackspace cloud,
openstack cloud, cloudstack cloud, euca clouds in the jclouds
repository and not in the respective provider/project repository. A
newcomer to the marvin repository will be someone interested in
writing tests and he will also thus be able to find tests in the
marvin repository.

This also allows for more heterogenous testing of cloudstack. No one
needs to be tied down to a framework / tool to write integration
tests. If python is not your forte, use Chip's ruby client, or perhaps
in the near future Chiradeep's stackmate to write your test, or even
jclouds. 

Now the question of supporting older version of marvin against newer
versions of cloudstack. Marvin now fully auto-generates itself (see
the design in the proposal) based on endpoint. So you have the
marvin version that will work with your endpoint only. As for being
backwards compatible (also addressed in the design doc) - no old tests
are broken, they will still run perfectly fine. 

The infrastructure (currently) only looks at the changes in the test
directory before performing a run. It doesn't care whether server/ was
changed or plugins/x/y/z was changed. That's because the tests are
unrelated to what is in the rest of the repository. In fact you can't
even run them without a deployed cloud. So I don't see why idle code
should lie in the repo.

Integration tests are essential, they will keep coming as long as
Citrix QA is invested in the effort, but they need to come faster into
the repos and that will be addressed by the separation IMO. Managing
the feature submitted to cloudstack against tests submitted to marvin
is not a hard thing to do. We simply mirror the release branches in
marvin and submit tests there. In fact I wonder why we didn't have
this question when docs were separated? It doesn't work any
differently really.

What I would like to see provided by CloudStack is the ability to
upgrade all our test environments, staging environments, UATs, what
have you in continuous integration and have tests run on the
upgraded setup. That allows incrementally testing CloudStack the way
users do it. The current design of installing everything from scratch,
redoing the testbed for each automated test run is mostly a workaround
for that inability. If we had this ability tests written in marvin can
be run against live setups at all times as and when features merge to
master.

It has always been my goal to reduce the barrier to writing tests for
cloudstack and that's entirely the goal of the refactor as well. So I
hope I have spent my time well. Since I can't be after people to write
tests, and setup mandates, I aim to address the technical difficulty
in writing tests through this refactor and separation.  

PS: David, by hosted docs I mean something like
http://pythonhosted.org/cloudmonkey. Lives in the repo but read from
the webpage.

-- 
Prasanna.,


Powered by BigRock.com

Re: Edit access to Jira

[Prasanna Santhanam]

On Mon, Oct 07, 2013 at 05:34:35AM +, Damoder Reddy wrote:
> Hi,
> 
> Can you please provide me the edit access to Jira. Following are my user
> details:
> 
> Username: damoder.reddy
> Mail id: damoder.re...@citrix.com<mailto:damoder.re...@citrix.com>
> 
> Please do the needful.
Make sure you register with the wiki (http://cwiki.apache.org) as well
with the same address so I can provide you with edit access to cwiki.

JIRA permissions have been given for username: damoder.reddy
> 
> Thanks
> Damodar

-- 
Prasanna.,


Powered by BigRock.com

Re: [MERGE] marvin-refactor to master

[Prasanna Santhanam]

Edison - thanks for the review! I've answered inline.

(I've brought the technical review to the right thread from the one about
marvin's repo separation)

> Few questions:
> 1. About the "more object-oriented" CloudStack API python binding: Is the
> proposed api  good enough?

As long as the cloudstack API retains its compatibility as it does now by not
altering required arguments. We are good to go. The current implementation of
VirtualMachine is bloated and does too many things, like SSH connections, NAT
creation, security group creation etc. The new method will provide such special
cases as factory hierarchies instead.

So: you'll have the regular VirtualMachine -> VpcVirtualMachine ->
VirtualMachineWithNAT -> VirtualMachineWithIngress etc

> For example, 
> The current hand written create virtual machine looks like:
> class VirtualMachine(object):
> 
> @classmethod
> def create(cls, apiclient, services, templateid=None, accountid=None,
> domainid=None, zoneid=None, networkids=None, serviceofferingid=None,
> securitygroupids=None, projectid=None, startvm=None,
> diskofferingid=None, affinitygroupnames=None, group=None,
> hostid=None, keypair=None, mode='basic', method='GET'):
> 
> the proposed api may look like:
> 
> class VirtualMachine(object):
>def create(self, apiclient, accountId, templateId, **kwargs)
> 
> The proposed api will look better than previous one, and it's automatically
> generated, so easy to maintain. But as a consumer of the api, how do people
> know what kind of parameters should be passed in? Will you have an online
> document for your api? Or you assume people will look at the api docs 
> generated
> by CloudStack? 
> Or why not make the api itself as self-contained? For example, add docs
> before create method:

All **kwargs will be spelt out as docstrings in the entity's methods. This is
something I haven't got to yet. It's in the TODO list doc on the branch 
however. I
recognize the difficulty in understanding kwargs for someone looking at the
API. I will fix before merge.  

My concern however is of factories being appropriately documented since they
are user written. Those will need to be caught via review.

> 
> 2. Regarding to data factories. From the proposed factories, in each test
> case, does test writer still need to write the code to get data, such as
> writing code to get account during the setupclass?

No. this is not required anymore. All data is represented as a factory. So to
get account data you simply import the necessary factory. You don't have to
imagine the structure of this data and json anymore.

  from marvin.factory.data import UserAccount 
  ...
  def setUp()
  account = UserAccount(apiclient)

So those crufty json headers should altogether disappear.

> With the data factories, the code will look like the following?
> 
> Class TestFoo:
>  Def setupClass():
>   Account = UserAccount(apiclient)
>VM = UserVM(apiClient)
> 
> And if I want to customize the default data factories, I should be able to
> use something like: UserAccount(apiclient, username='myfoo')?

Yes, this will create a new useraccount with an overridden username. You may
override any attribute of the data this way. This however, doesn't check for
duplicates.  So if a username 'myfoo' already exists, that account creation
will fail. If you use the factory, since it generates a random sequence you
won't have the problem of collisions

> And the data factories should be able to customized based on test
> environment, right? 
> For example, the current iso test cases are hardcoded to test against
> http://people.apache.org/~tsp/dummy.iso, but it won't work for devcloud, or
> in an internal network. The ISO data factory should be able to return an url
> based on different test environment, thus iso test cases can be reused.

Yes, we'll have to create a LocalIsoFactory which represents an ISO available
on the internal network. It is customizable. May be we can represent it to look
for a file within devcloud itself?

Thanks,

> On Wed, Oct 02, 2013 at 10:12:40PM +0530, Prasanna Santhanam wrote:
> > Once upon a time [1] I had propagated the idea of refactoring marvin to
> > make test case writing simpler. At the time, there weren't enough
> > people writing tests using marvin however. Now as focus on testing has
> > become much more important for the stability of our releases I would
> > like to bring back the discussion and to review the refactoring of
> > marvin which I've been doing in the marvin_refactor branch.
> > 
> > The key goal of this refactor was to simplify test case writing. In
> > doing so I'

Re: [DISCUSS] Breaking out Marvin from CloudStack

[Prasanna Santhanam]

All - thanks for your detailed thoughts on this so far. It seems like
this should've come with a fair bit of notice. I will hold on this
separation for now. May be a couple more releases to see the level of
participation in the QA and if it still feels like a hindrance we'll
discuss separating it later. For now I think it should suffice to
reorganize the current folder structure of marvin to be able to find
all its bits at a single place. This will be proposed separately.

I am closing this DISCUSS thread as of now by not splitting out
marvin.

I will however address some additional points which were raised so
far. Inline.

On Fri, Oct 04, 2013 at 04:52:45PM -0400, Chip Childers wrote:
> 
> Fair warning - some of this is a straw man argument to explore the
> situation, and a little bit of ranting at the end.
> 
> On Fri, Oct 04, 2013 at 05:57:58PM +0530, Prasanna Santhanam wrote:
> > I'll summarize and address the concerns raised so far.
> > 
> > Marvin has been in this repo for a long time for us to start writing
> > tests. The only tests I've seen coming are from a specific set of
> > people focussed on QA efforts. 
> 
> I agree - and that's a problem.  New features should *ALL* have tests
> before they merge into master.  I think that assuming that the only test
> writers are a group of folks that write the tests today is actually a
> larger problem.
> 

It's a bit more prevalent than test cases alone. Docs for eg are
written by doc writers and there's a whole bit of process of doc
review and someone writing the stuff in an email and then someone else
going and correcting it. If you have repo access and notice a problem,
please help fix it as well and not create additional process. I'm not
for such segmentation but there are roles in a corporate setup that
are probably causing this presumption that marvin tests are written by
QA alone.


> > I want to reduce the impediment for
> > people who are writing tests *today*. Those looking to get started in
> > the near future won't have any new learning to do, just that their
> > code goes in an alternate repo that is pointed to right
> > infrastructure. 
> > 
> > Automated testing also works in a push-to-production style very often.
> > Testers need to run their tests on a deployed environment(s) quickly
> > to be able to ensure it is valid and passes. By making them go through
> > reviewboard each time for each test we massively slow down the
> > process. (tons of fixes to tests are on rb today, not just new tests).
> > We don't know if they run until they run on the environment.
> 
> I want to be clear about this part - a different repo doesn't change the
> need for someone to be a committer to commit.

Yes - I thought this would be a problem as David also mentioned
earlier. There are no ACLs within a project for controlling repo
access. Except if it was possible, I would've looked at providing
access faster to those who are contributing.

> > 
> > Reason for tests and framework to go together is simple.  If I go look
> > at the jclouds repository today I find tests for rackspace cloud,
> > openstack cloud, cloudstack cloud, euca clouds in the jclouds
> > repository and not in the respective provider/project repository. A
> > newcomer to the marvin repository will be someone interested in
> > writing tests and he will also thus be able to find tests in the
> > marvin repository.
> > 
> > This also allows for more heterogenous testing of cloudstack. No one
> > needs to be tied down to a framework / tool to write integration
> > tests. If python is not your forte, use Chip's ruby client, or perhaps
> > in the near future Chiradeep's stackmate to write your test, or even
> > jclouds. 
> 
> But that's actually true today, right?  I mean if I wanted to write an
> integration test using some other method, I'd do that...  but would it
> be useful for others?  Probably not!  That's because the way that we do
> testing of this type is via Marvin.  The Citrix infra wouldn't be setup
> for whatever other framework I used, and the community as a whole would
> get less benefit than if I was consistent.
> 

Somehow I sense that as a problem of our test writing. The tests are
written to assume a certain infrastructure. But a lot of the API is
also admin only which requires certain infra to be in place. The other
tools jclouds live tests for example cannot assume admin access to the
cloud because they are only testing the user api. 

Tests written by other tools would only be provided an endpoint and
API/Secret keys and test whatever they can. Specific tools required
for those tests can be provisioned automatically on the machine

Re: marvin create network offering incomplete?

[Prasanna Santhanam]

On Tue, Oct 08, 2013 at 03:27:10PM +0200, Daan Hoogland wrote:
> H,
> 
> I am building an integration test with marvin and the following data:
> 
> "network_offering": {
> "name": 'Test Network offering',
> "displaytext": 'Test Network offering',
> "guestiptype": 'Isolated',
> "supportedservices": 'Connectivity',
> "traffictype": 'GUEST',
> "availability": 'Optional',
> "specifyvlan": False,
> "specifyipranges": False,
> "serviceproviderlist": {
> "Connectivity": "NiciraNVP"
> },
> "conservemode": False,
> "tags": [ "nicira-based" ]
> },

This looks fine.

> "network": {
> "name": "Test Network",
> "displaytext": "Test Network",
> "tags": "nicira-based"
> },
>  in the code I put:
> self.network_offering = NetworkOffering.create(
> self.apiclient,
> self.testdata["network_offering"]
> )
> # Enable Network offering
> self.network_offering.update(self.apiclient, state='Enabled')
> self.testdata["network"]["zoneid"] = self.zone.id
> self.testdata["network"]["networkoffering"] =
> self.network_offering.id
> self.network = Network.create(
> self.apiclient,
> self.testdata["network"]
> )
> The network doesn't get created:
>  431, errorText:More than one physical networks exist in zone id=1 and no
> tags are specified in order to make a choice
> 
> When inspecting indeed the tags are not set on the offering and neither is
> conservemode.

Is that a resource-tag for the network offering? What is the API
request that is going to the management server from marvin? How does
it compare against what you sent through via the UI?

> 
> Am I hunting a bug ar did I misconfigure my test?

I'm not sure. But it could just be that marvin created network
offering is not the same as the network offering created via the UI.

> 
> Daan

-- 
Prasanna.,


Powered by BigRock.com

Re: marvin over https

[Prasanna Santhanam]

Ok, this is a bug. requests lib is verifying SSL by default while
cloudmonkey is probably ignoring SSL. There are two options

1) Fix marvin to accept SSL while detecting your default certs in
/etc/ssl/certs? Or use an env variable
2) Ignore SSL auth from marvin.

Can you please file a bug report? It should be a simple fix, so you
can run with it or I'll get to it tomorrow.

Ref: 
http://www.python-requests.org/en/latest/user/advanced/#ssl-cert-verification


On Tue, Oct 08, 2013 at 04:14:46PM +0200, Daan Hoogland wrote:
> H Prasanna,
> 
> $ ./zoneCommand.py
> Traceback (most recent call last):
>   File "./zoneCommand.py", line 91, in 
> print "zones: " + repr(blub.listZones(conn))
>   File "./zoneCommand.py", line 42, in listZones
> resp = conn.marvin_request(lz)
>   File "/usr/lib/python2.7/site-packages/marvin/cloudstackConnection.py",
> line 218, in marvin_request
> cmdname, self.auth, payload=payload, method=method)
>   File "/usr/lib/python2.7/site-packages/marvin/cloudstackConnection.py",
> line 153, in request
> raise c
> requests.exceptions.SSLError: [Errno 1] _ssl.c:508: error:14090086:SSL
> routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
> 
> This is using the keys that are also used by cloudmonkey
> 
> any hint?
> Daan
> 
> 
> On Tue, Oct 8, 2013 at 2:31 PM, Daan Hoogland wrote:
> 
> > H Prasanne,
> >
> > I didn't get around this bit a few days.
> >
> > Cloudmonkey works throught the same connection. I will find some time the
> > coming days to test this with debug enabled marvin.
> >
> > regards,
> > Daan
> >
> >
> > On Thu, Oct 3, 2013 at 6:22 AM, Prasanna Santhanam  wrote:
> >
> >> On Thu, Sep 26, 2013 at 04:21:38PM +0200, Daan Hoogland wrote:
> >> > H,
> >> >
> >> > I have some trouble getting marvin to connect to cloudstack over https.
> >> > I am supposing the following should work
> >> > conn = cloudConnection(mgmtip, apiKey=apikey,
> >> > securityKey=secretkey, logging=log, port=443, scheme="https")
> >> > lz = listZones.listZonesCmd()
> >> > conn.marvin_request(lz)
> >> >
> >> > is this a valid assumption?
> >> >
> >> > I can browse to the https:///client/ and login to retrieve the
> >> > keys used, but on running the code above i get
> >> >
> >> > requests.exceptions.ConnectionError:
> >> > HTTPSConnectionPool(host='10.200.23.16', port=443): Max retries
> >> > exceeded with url:
> >> >
> >> /client/api?apiKey=JGvIQPeIVsbgEhVC3shZ51r9buYwClB4ToJZX9Cxs9e3NZbRoJLNyANnWEKgsmgt1uoF_eLdL31GHMwcss6Zyw&command=listZones&signature=KL93r9GYIr6%2FRcbNHuaOj3jUF6o%3D&response=json
> >> > (Caused by : [Errno 111] Connection refused)
> >>
> >> In the loglevel() method in CloudConnection.py, switch the logging to
> >> logging.DEBUG. That will spew out more verbose logging as to what's
> >> happening here.
> >>
> >> I've never tried it on an https enabled cloudstack so there might be a
> >> bug. Does cloudmonkey work for you on this endpoint? If yes, then I
> >> don't see why marvin shouldn't. Both use the same request mechanism.
> >>
> >> >
> >> > I am not sure where to look. at marvin, httprequest or the setup of my
> >> > env. Hints?
> >> >
> >> > thanks,
> >> > Daan
> >>
> >> --
> >> Prasanna.,
> >>
> >> 
> >> Powered by BigRock.com
> >>
> >>
> >

-- 
Prasanna.,


Powered by BigRock.com

Re: Contrail plugin

[Prasanna Santhanam]

On Tue, Oct 08, 2013 at 10:23:32AM -0400, Chip Childers wrote:
> As stated, I've imported the contrail plugin donation into the contrail
> branch.
> 
> I've taken the time to add the ASF license header to all of the new files
> in that branch.  I think we have to complete the following in order to
> merge into master.
> 
> 1) I'd like to see the package structure changed to match
> org.apache.cloudstack, instead of the Juniper namespace.  We only have
> com.cloud namespaces for legacy reasons, and are trying to consolidate into
> the apache ns.
> 
> 2) Folks with past experience with network plugins need to review the
> plugin's code and provide comments or +1s for a merge.  Chiradeep and Hugo,
> you've been "randomly" selected to help on this...  ;-)  Pedro, I'll assume
> that you will be happy to provide patches via reviewboard against this
> branch if changes are requested (including the package structure noted
> above).
> 
> 3) I'd love if we could get some consensus on what additional tests and /
> or changes to the test approach are needed.  Prasanna - as with Hugo and
> Chiradeep, you've been "randomly" selected to at least provide some input
> here.

I saw the thread earlier about a mysql db generated for performing an
integration test. If someone can point me to the spec/docs/readme on
how to run these presumably without the contrail device I'm happy to
take a look.

> 
> Anything I'm missing?
> 
> -chip

-- 
Prasanna.,


Powered by BigRock.com

Re: [MERGE] spring-modularization to master - Spring Modularization

[Prasanna Santhanam]

On Tue, Oct 08, 2013 at 10:20:01AM -0700, Darren Shepherd wrote:
> >From what I can gather it seems that master currently fails the BVT
> (and know when I say BVT I mean that black box that apparently exists
> somewhere doing something, but I have no clue what it really means).
> So in turn my spring modularization branch will additionally fail BVT.
>  Citrix internal QA ran some tests against my branch and they mostly
> passed but some failed.  Its quite difficult to sort through this all
> because tests are failing on master.  So I don't know what to do at
> this point.  At least my branch won't completely blow up everything.
> I just know the longer it takes to merge this the more painful it will
> be

Darren, Sorry about the frustrations. I haven't been able to keep
track these last few weeks about your work.

I have run the tests against your branch and everything looks good so
far. Whatever is failing is failing on master as well, so we'll ignore
that. I will share the report shortly.

They are the result of the job here, which I will summarize in a more
readable format and mail out later tonight once it completes:
http://jenkins.buildacloud.org/job/test-matrix/571/

Each bubble in that grid represents a hypervisor profile against which
all the tests were run.
 
> Honestly this is all quite frustrating for myself being new to
> contributing to ACS.  I feel somewhat lost in the whole process of how
> to get features in.  I'll refrain from venting my frustrations.
> 
> Darren

-- 
Prasanna.,


Powered by BigRock.com

Re: [MERGE] spring-modularization to master - Spring Modularization

[Prasanna Santhanam]

 20
:setup   
   0  1
integration.smoke.test_loadbalance.TestLoadBalance.test_01_create_lb_rule_src_nat
   1003.58   2
integration.smoke.test_loadbalance.TestLoadBalance.test_02_create_lb_rule_non_nat
   1003.45   2
integration.smoke.test_loadbalance.TestLoadBalance.test_assign_and_removal_lb   
1003.48   2
integration.smoke.test_templates.TestTemplates.test_03_delete_template  
   5.12  20
:setup  
   0 20

Fixed

name   
durationage
integration.smoke.test_templates.TestCreateTemplate.test_01_create_template 
 35.415  0



On Wed, Oct 09, 2013 at 06:16:04PM +0530, Prasanna Santhanam wrote:
> On Tue, Oct 08, 2013 at 10:20:01AM -0700, Darren Shepherd wrote:
> > >From what I can gather it seems that master currently fails the BVT
> > (and know when I say BVT I mean that black box that apparently exists
> > somewhere doing something, but I have no clue what it really means).
> > So in turn my spring modularization branch will additionally fail BVT.
> >  Citrix internal QA ran some tests against my branch and they mostly
> > passed but some failed.  Its quite difficult to sort through this all
> > because tests are failing on master.  So I don't know what to do at
> > this point.  At least my branch won't completely blow up everything.
> > I just know the longer it takes to merge this the more painful it will
> > be
> 
> Darren, Sorry about the frustrations. I haven't been able to keep
> track these last few weeks about your work.
> 
> I have run the tests against your branch and everything looks good so
> far. Whatever is failing is failing on master as well, so we'll ignore
> that. I will share the report shortly.
> 
> They are the result of the job here, which I will summarize in a more
> readable format and mail out later tonight once it completes:
> http://jenkins.buildacloud.org/job/test-matrix/571/
> 
> Each bubble in that grid represents a hypervisor profile against which
> all the tests were run.
>  
> > Honestly this is all quite frustrating for myself being new to
> > contributing to ACS.  I feel somewhat lost in the whole process of how
> > to get features in.  I'll refrain from venting my frustrations.
> > 
> > Darren
> 
> -- 
> Prasanna.,
> 
> 
> Powered by BigRock.com

-- 
Prasanna.,


Powered by BigRock.com

master simulator build broken on ManagedContext

[Prasanna Santhanam]

Hi,

on the simulator build in jenkins [1] on starting jetty I see the
following issue:

ERROR [o.s.w.c.ContextLoader] (main:null) Context initialization failed
org.springframework.beans.factory.BeanCreationException: Error creating bean
with name 'ManagedContext' defined in class path resource
[applicationContext.xml]: Cannot create inner bean
'org.apache.cloudstack.context.CallContextListener#2fd0f745' of type
[org.apache.cloudstack.context.CallContextListener] while setting bean property
'listeners' with key [0]; nested exception is
org.springframework.beans.factory.BeanCreationException: Error creating bean
with name 'org.apache.cloudstack.context.CallContextListener#2fd0f745':
Injection of autowired dependencies failed; nested exception is
org.springframework.beans.factory.BeanCreationException: Could not autowire
field: com.cloud.utils.db.EntityManager
org.apache.cloudstack.context.CallContextListener.entityMgr; nested exception
is org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find
class [org.apache.cloudstack.framework.config.ConfigDepotImpl] for bean with
name 'configDepot' defined in class path resource
[simulatorComponentContext.xml]; nested exception is
java.lang.ClassNotFoundException:
org.apache.cloudstack.framework.config.ConfigDepotImpl


Can someone throw some light on this and how I can get the simulator build to
run some basic tests per checkin again?

Here's the steps to run the simulator on the dev environment:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Simulator+integration


[1] http://jenkins.buildacloud.org/view/simulator/job/start-jetty/280/console
-- 
Prasanna.,


Powered by BigRock.com

Re: master simulator build broken on ManagedContext

[Prasanna Santhanam]

Thanks Darren, the jetty start has chugged along and I'm able to bring
up the server. I'd like to not have that orphaned context ideally
since it gets left out often this way. But I can live with that for
now.

On Wed, Oct 09, 2013 at 10:55:58AM -0700, Darren Shepherd wrote:
> Prasanna,
> 
> Try now.  I pushed 0d7aa931b4892661df733cd1ba20fe139d13e59b for this.
> The issue was that at some point the package of ConfigDepotImpl was
> changed in some refactoring.  The applicationContext was changed for
> this, but apparently somebody didn't check simulatorContext. The bean
> should never have been in simulatorContext, so I deleted it.
> 
> Darren
> 
> On Wed, Oct 9, 2013 at 10:30 AM, Darren Shepherd
>  wrote:
> > I'll look at that.  Gimme about 15 minutes.
> >
> > Darren
> >
> > On Wed, Oct 9, 2013 at 10:12 AM, Prasanna Santhanam  wrote:
> >> Hi,
> >>
> >> on the simulator build in jenkins [1] on starting jetty I see the
> >> following issue:
> >>
> >> ERROR [o.s.w.c.ContextLoader] (main:null) Context initialization failed
> >> org.springframework.beans.factory.BeanCreationException: Error creating 
> >> bean
> >> with name 'ManagedContext' defined in class path resource
> >> [applicationContext.xml]: Cannot create inner bean
> >> 'org.apache.cloudstack.context.CallContextListener#2fd0f745' of type
> >> [org.apache.cloudstack.context.CallContextListener] while setting bean 
> >> property
> >> 'listeners' with key [0]; nested exception is
> >> org.springframework.beans.factory.BeanCreationException: Error creating 
> >> bean
> >> with name 'org.apache.cloudstack.context.CallContextListener#2fd0f745':
> >> Injection of autowired dependencies failed; nested exception is
> >> org.springframework.beans.factory.BeanCreationException: Could not autowire
> >> field: com.cloud.utils.db.EntityManager
> >> org.apache.cloudstack.context.CallContextListener.entityMgr; nested 
> >> exception
> >> is org.springframework.beans.factory.CannotLoadBeanClassException: Cannot 
> >> find
> >> class [org.apache.cloudstack.framework.config.ConfigDepotImpl] for bean 
> >> with
> >> name 'configDepot' defined in class path resource
> >> [simulatorComponentContext.xml]; nested exception is
> >> java.lang.ClassNotFoundException:
> >> org.apache.cloudstack.framework.config.ConfigDepotImpl
> >>
> >>
> >> Can someone throw some light on this and how I can get the simulator build 
> >> to
> >> run some basic tests per checkin again?
> >>
> >> Here's the steps to run the simulator on the dev environment:
> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Simulator+integration
> >>
> >>
> >> [1] 
> >> http://jenkins.buildacloud.org/view/simulator/job/start-jetty/280/console
> >> --
> >> Prasanna.,
> >>
> >> 
> >> Powered by BigRock.com
> >>

-- 
Prasanna.,


Powered by BigRock.com

Re: Back end volume names in KVM

[Prasanna Santhanam]

+1 - happens in lots of places in our code where a random-uuid
associates with a physical resource's uuid.

Will this will happen only for new volumes? Old volumes can still be
listed and found using the old method? I'm specifically concerned
about upgraded systems.

On Wed, Oct 09, 2013 at 11:05:48PM -0600, Mike Tutkowski wrote:
> Yeah, I'm not really clear how the snapshot strategy works if you have
> multiple vendors that implement that interface either.
> 
> 
> On Wed, Oct 9, 2013 at 9:57 PM, Marcus Sorensen  wrote:
> 
> > Does anyone have any reservations about changing the volume identifier in
> > KVM's volume creation command to be the uuid of the volume? Currently for
> > new volumes it generates a random uuid and passes that back to be stored in
> > the database. From an admin perspective, the only way to link a volume on
> > the back end (be it a qcow2 image or an LVM volume) to one as reported is
> > to look in the DB and see what this 'secondary uuid' is and look for THAT
> > as the filename/image name on the back end. It would simply remove a layer
> > of translating uuid to another hidden uuid to get file/volume name.
> >
> > It shouldn't disrupt or change current volumes, just new ones.
> >
> > The only caveat I can think of so far is if we wanted multiple files/images
> > on the back end to map to one volume, but I don't see that as a blocker
> > since it would probably have lots of other implications to the tracking
> > volume objects.
> >
> 
> 
> 
> -- 
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *?*

-- 
Prasanna.,


Powered by BigRock.com

[INFRA] Sorely missing documentation

[Prasanna Santhanam]

Hi,

I've added some missing documentation on how our build systems
(jenkins) are setup. I think I've addressed most questions that came
up in threads on this mailing list. If you would like to understand
how all this is setup please go through the following wiki pages: I've
put in pictures and everything :)

Build systems:
https://cwiki.apache.org/confluence/x/Cy-VAQ

Test infrastructure:
https://cwiki.apache.org/confluence/x/_y7VAQ

Thanks,

-- 
Prasanna.,


Powered by BigRock.com

Re: [INFRA] Sorely missing documentation

[Prasanna Santhanam]

By latest you mean something in those pages is outdated? I haven't
documented all the jobs yet.

I think it would be nice to add links to our community repositories
etc. Is this what you mean by artifacts?

On Thu, Oct 10, 2013 at 02:17:13PM +0200, Daan Hoogland wrote:
> Thanks Prasanna,
> 
> should we publish some 'latest' docs or is this not desirable? We
> could do with links to the artifacts, i imagine.
> 
> Daan
> 
> On Thu, Oct 10, 2013 at 2:00 PM, Prasanna Santhanam  wrote:
> > Hi,
> >
> > I've added some missing documentation on how our build systems
> > (jenkins) are setup. I think I've addressed most questions that came
> > up in threads on this mailing list. If you would like to understand
> > how all this is setup please go through the following wiki pages: I've
> > put in pictures and everything :)
> >
> > Build systems:
> > https://cwiki.apache.org/confluence/x/Cy-VAQ
> >
> > Test infrastructure:
> > https://cwiki.apache.org/confluence/x/_y7VAQ
> >
> > Thanks,
> >
> > --
> > Prasanna.,
> >
> > 
> > Powered by BigRock.com
> >

-- 
Prasanna.,


Powered by BigRock.com

Re: Docs have moved

[Prasanna Santhanam]

On Wed, Oct 02, 2013 at 03:15:23PM -0400, David Nalley wrote:
> Per our earlier discussion, now that 4.2 has released, I moved the
> docs for 4.2 to their own repo, and will be doing the same for master
> in the next day or so as I clean things up.
> 
> The new repo is:
> 
> https://git-wip-us.apache.org/repos/asf/cloudstack-docs.git
> 

Fixed all the build jobs on jenkins to point to the right repo.
http://jenkins.buildacloud.org/view/master/

> The master branch is pretty desolate until I catch up, but 4.2 is
> fully populated.
> 
> --David

-- 
Prasanna.,


Powered by BigRock.com

Re: [INFRA] Sorely missing documentation

[Prasanna Santhanam]

Should be okay for the moment. Except we forget updating links on the
wiki with jenkins changing. But I wouldn't point these to the users as
'working' documentation. We should get the corrected docs on the site.

On Thu, Oct 10, 2013 at 02:26:22PM +0200, Daan Hoogland wrote:
> by latest i mean the results of jenkins doc build jobs.
> 
> and yes I mean either copy the build results to the site or the wiki
> on the one hand and just set links to the artifacts in jenkins on the
> other hand.
> 
> On Thu, Oct 10, 2013 at 2:21 PM, Prasanna Santhanam  wrote:
> > By latest you mean something in those pages is outdated? I haven't
> > documented all the jobs yet.
> >
> > I think it would be nice to add links to our community repositories
> > etc. Is this what you mean by artifacts?
> >
> > On Thu, Oct 10, 2013 at 02:17:13PM +0200, Daan Hoogland wrote:
> >> Thanks Prasanna,
> >>
> >> should we publish some 'latest' docs or is this not desirable? We
> >> could do with links to the artifacts, i imagine.
> >>
> >> Daan
> >>
> >> On Thu, Oct 10, 2013 at 2:00 PM, Prasanna Santhanam  
> >> wrote:
> >> > Hi,
> >> >
> >> > I've added some missing documentation on how our build systems
> >> > (jenkins) are setup. I think I've addressed most questions that came
> >> > up in threads on this mailing list. If you would like to understand
> >> > how all this is setup please go through the following wiki pages: I've
> >> > put in pictures and everything :)
> >> >
> >> > Build systems:
> >> > https://cwiki.apache.org/confluence/x/Cy-VAQ
> >> >
> >> > Test infrastructure:
> >> > https://cwiki.apache.org/confluence/x/_y7VAQ
> >> >
> >> > Thanks,
> >> >
> >> > --
> >> > Prasanna.,
> >> >
> >> > 
> >> > Powered by BigRock.com
> >> >
> >
> > --
> > Prasanna.,
> >
> > 
> > Powered by BigRock.com
> >

-- 
Prasanna.,


Powered by BigRock.com

Re: Docs have moved

[Prasanna Santhanam]

On Thu, Oct 10, 2013 at 05:57:58PM +0530, Prasanna Santhanam wrote:
> On Wed, Oct 02, 2013 at 03:15:23PM -0400, David Nalley wrote:
> > Per our earlier discussion, now that 4.2 has released, I moved the
> > docs for 4.2 to their own repo, and will be doing the same for master
> > in the next day or so as I clean things up.
> > 
> > The new repo is:
> > 
> > https://git-wip-us.apache.org/repos/asf/cloudstack-docs.git
> > 
> 
> Fixed all the build jobs on jenkins to point to the right repo.
> http://jenkins.buildacloud.org/view/master/

Actually I think the jobs are broken now. I'm not too clear on the
publican changes. Can someone take a look and fix the publican
arguments?

> 
> > The master branch is pretty desolate until I catch up, but 4.2 is
> > fully populated.
> > 
> > --David
> 
> -- 
> Prasanna.,
> 
> 
> Powered by BigRock.com

-- 
Prasanna.,


Powered by BigRock.com

bvt failure since on master

[Prasanna Santhanam]

Hi All - Looks like the server won't come up on master
(CLOUDSTACK-4848). It started happening since last night. Is this
something anyone else is seeing? I'm hitting it regularly on the test
environment. The DatabaseUpgradeChecker is complaining about a
duplicate column `size` in an upgrade script.



-- 
Prasanna.,


Powered by BigRock.com

Re: bvt failure since on master

[Prasanna Santhanam]

Can you tell me what job is doing that? I'm not sure which one is
emailing people. We shouldn't be emailing anything from
jenkins.buildacloud to individuals unless they ask to be.

On Thu, Oct 10, 2013 at 07:00:27PM +0200, Laszlo Hornyak wrote:
> Hi,
> 
> I am receiving tons of emails from jenkins about broken builds, but my
> local build is ok and no tests failing, Is this some environment issue?
> 
> 
> 
> 
> On Thu, Oct 10, 2013 at 5:51 PM, Prasanna Santhanam  wrote:
> 
> > Hi All - Looks like the server won't come up on master
> > (CLOUDSTACK-4848). It started happening since last night. Is this
> > something anyone else is seeing? I'm hitting it regularly on the test
> > environment. The DatabaseUpgradeChecker is complaining about a
> > duplicate column `size` in an upgrade script.
> >
> >
> >
> > --
> > Prasanna.,
> >
> > 
> > Powered by BigRock.com
> >
> >
> 
> 
> -- 
> 
> EOF

-- 
Prasanna.,


Powered by BigRock.com

Re: Global setting "host" is not set to MS IP anymore on master

[Prasanna Santhanam]

So, why does setting the 'host' to your IP not work? I do that for the
test environment on master as part of deployment via marvin. It's only
a small change in the marvin-config.

On https://github.com/vogxn/cloud-autodeploy.git

commit 8fe580e82fe4ab6dd8e124bf226c5941e72a4204
Author: Prasanna Santhanam 
Date:   Fri Oct 4 13:53:55 2013 +0530

host value in global config defaults to localhost
Signed-off-by: Prasanna Santhanam 

diff --git a/kvm.cfg b/kvm.cfg
index 43b45de..0e23db0 100644
--- a/kvm.cfg
+++ b/kvm.cfg
@@ -152,6 +152,10 @@
 ], 
 "globalConfig": [
 {
+"name": "host", 
+"value": "cloudstack-centos63"
+}, 
+{
 "name": "network.gc.wait", 
 "value": "60"
 }, 


On Thu, Oct 10, 2013 at 11:25:15PM +, Prachi Damle wrote:
> I see it on Windows too.
> 
> -Original Message-
> From: Alena Prokharchyk [mailto:alena.prokharc...@citrix.com] 
> Sent: Thursday, October 10, 2013 4:20 PM
> To: dev@cloudstack.apache.org; Min Chen
> Subject: Re: Global setting "host" is not set to MS IP anymore on master
> Importance: High
> 
> I'm facing the same problem on master (OS X v10.8.4 if the platform
> matters)
> 
> On 10/10/13 3:22 PM, "Min Chen"  wrote:
> 
> >Hmm? I think that it may be related to new ConfigDepot framework to 
> >support dynamic config value without restarting MS.
> >
> >-min
> >
> >
> >
> >On 10/10/13 3:00 PM, "Nitin Mehta"  wrote:
> >
> >>If my memory serves me right this was fixed in 4.2. Maybe the fixer 
> >>forgot to port it to master ?
> >>
> >>On 10/10/13 2:46 PM, "Min Chen"  wrote:
> >>
> >>>Hi there,
> >>>
> >>>Not sure if others experience the same issue. In 4.2, when MS starts, 
> >>>it will discover the IP address of MS and automatically set global 
> >>>setting "host" to that IP address. Now it seems broken on master, MS 
> >>>started but "host" is still set to "localhost", which caused my 
> >>>system vm cannot ping back MS and thus no agent status cannot be 
> >>>found for system vm, thus this message on my console "INFO  
> >>>[o.a.c.s.e.DefaultEndPointSelector] (StatsCollector-3:null) No 
> >>>running ssvm is found, so command will be sent to LocalHostEndPoint".
> >>>Anybody has any idea on this?
> >>>
> >>>Thanks
> >>>-min
> >>
> >
> >
> 

-- 
Prasanna.,


Powered by BigRock.com

Re: Master - BVT automation result on KVM

[Prasanna Santhanam]

Hi Rayees,

Thanks for stepping up to send this report. Can we use the
jenkins2email [1] script I wrote for this email? The format gets
messed up when you copy paste from jenkins.

Also the result here is not that of KVM. KVM is actually broken on
master, the addHost seems to be failing to bring up hosts. I'm still
troubleshooting this for a clear bug report.

You can tell which profile the results are coming from by seeing the
upstream profile that triggerd this job:

In your case the page says:

"""
Started by upstream project 
test-matrix/distro=centos63,hypervisor=xen,profile=xen62 build number 580
originally caused by: 
Started by upstream project test-matrix build number 580
originally caused by:
Started by timer
"""

Which is the Xen6.2 hypervisor.

[1] https://github.com/vogxn/jenkins2email

> 
> Here the BVT automation result on KVM,  You can  see the result @ 
> http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-smoke-matrix/898/ 
> 
> Delete template test case failed this runs, we need to look that (I didn't 
> tested manually this failure) 
> 
> Issues reported with last run still open 
> 
> https://issues.apache.org/jira/browse/CLOUDSTACK-4835   
> https://issues.apache.org/jira/browse/CLOUDSTACK-4834   
> https://issues.apache.org/jira/browse/CLOUDSTACK-4833 
> 
> Configuration Name   All???  Failed?? 
>  Defect
>
> test_volumes2   2 
> CLOUDSTACK-4834
> test_vm_snapshots3   0
> test_vm_life_cycle  101
> test_templates 8  2   
>   i)CLOUDSTACK-4833  and  ii)Failed to delete te
> test_ssvm   10
>0
> test_service_offerings  4   0
> test_scale_vm  1   0
> test_routers  9   > 0
> test_resource_detail 1   1
> test_reset_vm_on_reboot 1   0
> test_regions  1   > 0
> test_pvlan  1 
>   0
> test_public_ip_range1   0
> test_privategw_acl1   1
> test_portable_publicip 2   0
> test_non_contigiousvlan 1   0
> test_nic   1  
>  0
> test_network_acl1   0
> test_network7   0
> test_multipleips_per_nic 1   0
> test_loadbalance 3   0
> test_iso2 
>   1   CLOUDSTACK-4833
> test_internal_lb1   0
> test_guest_vlan_range  1   0
> test_global_settings2   2 
>  CLOUDSTACK-4835
> test_disk_offerings  3   0
> test_deploy_vms_with_varied_deploymentplanners  3   0
> test_deploy_vm_with_userdata 2   0
> test_deploy_vm 1   0
> test_affinity_groups 1   0
> 
> 
> 
> Regards,
> Rayees 

-- 
Prasanna.,


Powered by BigRock.com

Re: Steps in building DEB packages for 4.2.0

[Prasanna Santhanam]

On Fri, Oct 11, 2013 at 09:59:52AM +0800, Indra Pramana wrote:
> Dear all,
> 
> I am reading the documentation on the steps in building DEB packages for
> CloudStack 4.2.0:
> 
> http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.2.0/html/Installation_Guide/sect-source-builddebs.html
> 
> and compare it with the steps on the earlier version (e.g. 4.1.1):
> 
> http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.1.1/html/Installation_Guide/sect-source-builddebs.html
> 
> I noticed that there's one command missing:
> 
> ===
> mvn clean install -P developer,systemvm
> ===
> 
> Can I confirm that the above step is no longer required for 4.2.0 and we
> can go ahead to build the package using the command: "dpkg-buildpackage -uc
> -us"?

Yes, dpkg-buildpackage will take care of the mvn bits as well for you.
It is not required. BTW, the list of debs generated also differs in
4.2 and that's now been corrected. So don't be alarmed if you see
different set of debs.

> 
> Looking forward to your reply, thank you.
> 
> Cheers.

-- 
Prasanna.,


Powered by BigRock.com

Re: [MERGE] marvin-refactor to master

[Prasanna Santhanam]

On Sun, Oct 13, 2013 at 08:14:01AM +0530, Girish Shilamkar wrote:
> Hello Prasanna,
> 
> +1 to the new marvin architecture. 
> The proposed architecture will make marvin lot more stable, remove
> defining data in code and will be easier to add testcases.

Great! thanks for reviewing.

> 
> Can we have a run in jenkins.cloudstack.org to ensure that the
> existing tests won't break with new marvin ?

Absolutely. I will rebase the branch run the test suite on jenkins.
> 
> How much effort would it be to convert existing tests to use new
> marvin, sooner or later we might want to do it ?

I think it would be a reasonably big task to take up and I wasn't sure
if we were ever going to take that up. We would gradually have to
write tests using the new way and with enough tests under our belt
abandon the old tests. This is why I didn't break the compatibility or
I would've refactored it slightly differently getting rid of the
cloudstackAPI cmd objects altogether.

> Going forward it will be critical to maintain these associations via
> factory hierarchies else would beat the purpose. 
> And needs to be caught using reviews.

Yes - we need a lot of care in defining our factories to make them
reusable, well documented and carefully organized.

> 
> In the branch what is the feature/ dir for ?

I was experimenting with the behave framework for converting the new
representation of cloudstack entities into a BDD based test. But this
will be separate from the intermediary work of factories.

> 
> Could you also put up a Todo list on the wiki ? I did not find one
> in the branch.

I will paste the TODO list in the branch on to the wiki. They're in
errata.markdown in tools/marvin/doc

> 
> Regards,
> Girish
> 
> 
> On 02-Oct-2013, at 10:12 PM, Prasanna Santhanam  wrote:
> 
> > Once upon a time [1] I had propagated the idea of refactoring marvin to
> > make test case writing simpler. At the time, there weren't enough
> > people writing tests using marvin however. Now as focus on testing has
> > become much more important for the stability of our releases I would
> > like to bring back the discussion and to review the refactoring of
> > marvin which I've been doing in the marvin_refactor branch.
> > 
> > The key goal of this refactor was to simplify test case writing. In
> > doing so I've transformed the library from its brittle hand-written
> > nature to a completely auto-generated set of libraries. In that sense,
> > marvin is much closer to cloudmonkey now.
> > 
> > The two important changes in this refactor are:
> > 
> > 1. data represented in an object-oriented fashion presented as factories
> > 2. test case writing using entities and their operations rather than
> > a sequence of disconnected API calls.
> > 
> > To see the full nature of this proposal I've updated the spec I put up
> > on the wiki:
> > https://cwiki.apache.org/confluence/x/RI3lAQ
> > 
> > For a quick comparison I wrote a test for the VPC vm's lifecycle in
> > tools/marvin/marvin/test/test_vpc_life_cycle.py which one can compare
> > with the existing tests for vpc under
> > test/integration/component/test_vpc_vm_life_cycle.py
> > 
> > These changes being 'architectural' so to speak and in a way
> > disruptive even I would like to merge this at the beginning of the
> > upcoming cloudstack release.
> > 
> > This is only a small part of a larger change for marvin which will be
> > moving to a more BDD like implementation [2] where tests are written
> > using a gherkin-like language. But that will come later.
> > 
> > I've also tried to disconnect marvin from depending on CloudStack's
> > build and repo. This will help split marvin from CloudStack which I
> > will discuss in a seperate thread.
> > 
> > [1] http://markmail.org/message/4tsscn6zvtfsskuj
> > [2] http://pythonhosted.org/behave/
> > 
> > -- 
> > Prasanna.,
> > 
> > 
> > Powered by BigRock.com
> > 
> 

-- 
Prasanna.,


Powered by BigRock.com

Re: Global setting "host" is not set to MS IP anymore on master

[Prasanna Santhanam]

After deploying using marvin's config file I have to perform a
restart of mgmt server for the globals to take effect. The problem
also appears quite annoyingly in KVM hosts where the agent recieves
localhost in the mgmt server info and tries to connect back to itself.

The spring-modularization branch I believe already fixes this problem. 

On Fri, Oct 11, 2013 at 07:33:36PM +, Soheil Eizadi wrote:
> Since I synced my code with 4.3 Master, the Host and Management
> Network is getting properly for me. The storage network is empty.
> -Soheil
> 
> From: Mike Tutkowski [mike.tutkow...@solidfire.com]
> Sent: Friday, October 11, 2013 11:23 AM
> To: dev@cloudstack.apache.org
> Cc: Prasanna Santhanam
> Subject: Re: Global setting "host" is not set to MS IP anymore on master
> 
> Yeah, I tend to set up my CS environment manually.
> 
> The first thing I do when I log in via the GUI is change the host,
> secstorage, and cidr Global Settings and then re-start the management
> server.
> 
> It would be great if this were not required, of course.
> 
> 
> On Fri, Oct 11, 2013 at 11:14 AM, Min Chen  wrote:
> 
> > I am also using marvin to deploy my local environment. Do you need to
> > restart MS after adding this globalConfig section in our .cfg file? If so,
> > developer may be able to workaround this. But this issue still needs to be
> > addressed for people not using marvin framework to setup environment, then
> > they have to always remember a manual update host step before starting any
> > ssvm.
> >
> > Thanks
> > -min
> >
> > On 10/10/13 9:31 PM, "Prasanna Santhanam"  wrote:
> >
> > >So, why does setting the 'host' to your IP not work? I do that for the
> > >test environment on master as part of deployment via marvin. It's only
> > >a small change in the marvin-config.
> > >
> > >On https://github.com/vogxn/cloud-autodeploy.git
> > >
> > >commit 8fe580e82fe4ab6dd8e124bf226c5941e72a4204
> > >Author: Prasanna Santhanam 
> > >Date:   Fri Oct 4 13:53:55 2013 +0530
> > >
> > >host value in global config defaults to localhost
> > >Signed-off-by: Prasanna Santhanam 
> > >
> > >diff --git a/kvm.cfg b/kvm.cfg
> > >index 43b45de..0e23db0 100644
> > >--- a/kvm.cfg
> > >+++ b/kvm.cfg
> > >@@ -152,6 +152,10 @@
> > > ],
> > > "globalConfig": [
> > > {
> > >+"name": "host",
> > >+"value": "cloudstack-centos63"
> > >+},
> > >+{
> > > "name": "network.gc.wait",
> > > "value": "60"
> > > },
> > >
> > >
> > >On Thu, Oct 10, 2013 at 11:25:15PM +, Prachi Damle wrote:
> > >> I see it on Windows too.
> > >>
> > >> -Original Message-
> > >> From: Alena Prokharchyk [mailto:alena.prokharc...@citrix.com]
> > >> Sent: Thursday, October 10, 2013 4:20 PM
> > >> To: dev@cloudstack.apache.org; Min Chen
> > >> Subject: Re: Global setting "host" is not set to MS IP anymore on master
> > >> Importance: High
> > >>
> > >> I'm facing the same problem on master (OS X v10.8.4 if the platform
> > >> matters)
> > >>
> > >> On 10/10/13 3:22 PM, "Min Chen"  wrote:
> > >>
> > >> >Hmm? I think that it may be related to new ConfigDepot framework to
> > >> >support dynamic config value without restarting MS.
> > >> >
> > >> >-min
> > >> >
> > >> >
> > >> >
> > >> >On 10/10/13 3:00 PM, "Nitin Mehta"  wrote:
> > >> >
> > >> >>If my memory serves me right this was fixed in 4.2. Maybe the fixer
> > >> >>forgot to port it to master ?
> > >> >>
> > >> >>On 10/10/13 2:46 PM, "Min Chen"  wrote:
> > >> >>
> > >> >>>Hi there,
> > >> >>>
> > >> >>>Not sure if others experience the same issue. In 4.2, when MS
> > >>starts,
> > >> >>>it will discover the IP address of MS and automatically set global
> > >> >>>setting "host" to that IP address. Now it seems broken on master, MS
> > >> >>>started but "host" is still set to "localhost", which caused my
> > >> >>>system vm cannot ping back MS and thus no agent status cannot be
> > >> >>>found for system vm, thus this message on my console "INFO
> > >> >>>[o.a.c.s.e.DefaultEndPointSelector] (StatsCollector-3:null) No
> > >> >>>running ssvm is found, so command will be sent to LocalHostEndPoint".
> > >> >>>Anybody has any idea on this?
> > >> >>>
> > >> >>>Thanks
> > >> >>>-min
> > >> >>
> > >> >
> > >> >
> > >>
> > >
> > >--
> > >Prasanna.,
> > >
> > >
> > >Powered by BigRock.com
> > >
> >
> >
> 
> 
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *?*

-- 
Prasanna.,


Powered by BigRock.com

Re: Wiki access

[Prasanna Santhanam]

On Mon, Oct 14, 2013 at 07:55:12PM -0600, Yoshikazu Nojima wrote:
> Please provide me edit permissions, username: ynojima.
> I'd like to update the vxlan integration feature proposal.
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Feature+Proposal+-+Linux+native+VXLAN+support+on+KVM+hypervisor#
> 

Done, please check. You should have permission to assign tickets on
jira as well.

> Thanks,
> Noji
> 
> 
> 
> 2013/10/8 Chip Childers :
> > On Tue, Oct 08, 2013 at 06:14:39AM +, Rajani Karuturi wrote:
> >> May I get access to edit pages as well?
> >> username: rajanik
> >
> > Looks like this was already taken care of.

-- 
Prasanna.,


Powered by BigRock.com

Re: [DOC] Jira Issues that can be reviewed and closed

[Prasanna Santhanam]

On Mon, Oct 14, 2013 at 11:43:33PM -0400, Travis Graham wrote:
> Here is a list of Jira issues against the docs that can be reviewed and 
> closed if the outcome is acceptable:
> 
> CLOUDSTACK-4844 -- Corrected download link for MSI file, removed MSI link 
> from Linux section
> CLOUDSTACK-4805 -- Added the missing Ubuntu steps
> CLOUDSTACK-4782 -- Already corrected in 4.2.0 docs
> CLOUDSTACK-4692 -- Fix references to componentContext.xml

Hi - you should have access to assign these tickets to yourself and
mark them as fixed. Ideally, the reporter of the ticket will verify
and close the issue.

> 
> 
> Since we're still waiting on the addition of cloudstack-docs to
> Review Board I'm sending out this update as a review request to
> start closing out some of the doc issues in Jira.
> 
> 
> Travis

-- 
Prasanna.,


Powered by BigRock.com