from:"Bharat Kumar"

Re: Fault percentage value of CPU usage in Cloud Platform

2016-11-21 Thread Bharat Kumar

Hi,  

There may be a difference in what you have allocated and what is being actually 
used. The dashboard shows what is allocated.

Regards,
Bharat.

On 11/21/16, 9:44 PM, "williamstev...@gmail.com on behalf of Will Stevens" 
 wrote:

>You will have to contact Accelerite for support with ACP (previously CCP).
>We have no visibility into the ACP code or how to support you.
>
>https://support.accelerite.com/hc/en-us
>
>Best of luck...
>
>*Will STEVENS*
>Lead Developer
>
>
>
>On Mon, Nov 21, 2016 at 3:44 AM, anil lakineni <
>anilkumar459.lakin...@gmail.com> wrote:
>
>> Dear All,
>>
>> On CloudPlatform dashboard our CPU usage is showing wrong (high -91%) value
>> which in-turn not allowing us to provision new VMs. But, the fact is only
>> 40% of the available CPU is utilized and Even in the Dashboard only
>> percentage calculation is showing false metric value, But Cpu usage value
>> is showing accurate(800/2000 GHZ).
>>
>> In addition to that when we go to check the CPU status at Zones level we
>> are seeing the accurate CPU usage percentage in all Zones, Only we are
>> getting false usage percentage at dashboard level(which leads to fail the
>> new deployments).
>>
>> - Our CCP version is 4.5.0
>> - Hypervisors are Xen 6.2 & 6.5
>>
>> Please help me to sort out this issue and also let me know if any
>> additional information needed.
>>
>>
>> Best Regards,
>> Anil.
>>

DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Accelerite, a Persistent Systems business. It is intended only for 
the use of the individual or entity to which it is addressed. If you are not 
the intended recipient, you are not authorized to read, retain, copy, print, 
distribute or use this message. If you have received this communication in 
error, please notify the sender and delete all copies of this message. 
Accelerite, a Persistent Systems business does not accept any liability for 
virus infected mails.

Re: [VOTE] Split Marvin to its own repository

2016-07-19 Thread Bharat Kumar

Hi Rohit,


what are we trying to achieve by moving marvin into a separate repo.?


--Bharat.


From: Raja Pullela 
Sent: Tuesday, July 19, 2016 5:30:20 PM
To: dev@cloudstack.apache.org
Subject: Re: [VOTE] Split Marvin to its own repository

Hi Rohit,

same question as Rene has posted, impact on older releases – will have issues 
on older releases.  I know that the older releases have marvin code which can 
be used.  Also, this will require changes on the CI side to pull the correct 
repo for Marvin.

+1, if Bharat can modify CI implementation to take care of this?

best,
Raja
Senior Manager, Product Development
Accelerate, 
www.accelerite.com,@accelerite
2055, Laurelwood Road,  Santa Clara, CA 95054, USA
Phone: 1-408-216-7010

On 7/18/16, 3:14 PM, "Rohit Yadav"  wrote:
All,

Based on a recent discussion thread [1], I want to start a voting thread to
gather consensus around splitting Marvin from the CloudStack repository.

On successful voting, we would extract and maintain Marvin as a separate
library in a separate repository (example repository [2]) and various
build/test systems such as Travis [3] can install it directly for usage
with pip+git etc.

Background: During the build process, a commands.xml generated to build
apidocs is also used to generate CloudStack Cmd and Request classes are
auto-generated, which is the only dependency why we needed Marvin and
CloudStack together. The auto-generated cloudstackAPI module can be also
generated against a live running CloudStack mgmt server which has api
discovery (listApis) enabled. The integration tests will still be tied to a
branch and will remain withing the repository. A PR [3] was sent to show
that we can still execute tests using this approach, and this would finally
allow us to build, release and use Marvin as an independent library.

Vote will be open for 72 hours.

For sanity in tallying the vote, can PMC members please be sure to indicate
"(binding)" with their vote?

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)

[1] http://markmail.org/thread/kiezqhjpz44hvrau
[2] https://github.com/rhtyd/marvin
[3] https://github.com/apache/cloudstack/pull/1599

Regards,
Rohit Yadav





DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Accelerite, a Persistent Systems business. It is intended only for 
the use of the individual or entity to which it is addressed. If you are not 
the intended recipient, you are not authorized to read, retain, copy, print, 
distribute or use this message. If you have received this communication in 
error, please notify the sender and delete all copies of this message. 
Accelerite, a Persistent Systems business does not accept any liability for 
virus infected mails.



DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Accelerite, a Persistent Systems business. It is intended only for 
the use of the individual or entity to which it is addressed. If you are not 
the intended recipient, you are not authorized to read, retain, copy, print, 
distribute or use this message. If you have received this communication in 
error, please notify the sender and delete all copies of this message. 
Accelerite, a Persistent Systems business does not accept any liability for 
virus infected mails.

Incorrect system vm templates for master build.

2016-04-28 Thread Bharat Kumar

Hi All,

looks like the systemvm templates are not getting built correctly in 
http://jenkins.buildacloud.org/job/build-systemvm64-master.

The build version is in correctly set to 4.6,  Im not sure but i think the 
build version in
/etc/cloudstck-release is also getting wrongly set inside the template.

Can any one please fix this. I do not have access to jenkins.

Thanks,
Bharat.






DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Accelerite, a Persistent Systems business. It is intended only for 
the use of the individual or entity to which it is addressed. If you are not 
the intended recipient, you are not authorized to read, retain, copy, print, 
distribute or use this message. If you have received this communication in 
error, please notify the sender and delete all copies of this message. 
Accelerite, a Persistent Systems business does not accept any liability for 
virus infected mails.

Re: [DISCUSS] PR testing process

2016-03-30 Thread Bharat Kumar

Hi Sanjeev,

working other piece currently, I have not stared on it.

Thanks,
Bharat.

> On 31-Mar-2016, at 10:35 AM, Sanjeev Neelarapu 
> <sanjeev.neelar...@accelerite.com> wrote:
>
> Hi Bharat,
>
> Just wanted to check if the CI is still getting the code from the PR branch 
> or from the latest master?
>
> Best Regards,
> Sanjeev N
> Chief Product Engineer, Accelerite
> Off: +91 40 6722 9368 | EMail: sanjeev.neelar...@accelerite.com
>
>
> -Original Message-
> From: Remi Bergsma [mailto:rberg...@schubergphilis.com]
> Sent: Monday, March 28, 2016 1:24 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] PR testing process
>
> Indeed. If a merge fails then you'll not be able to merge it to master or 
> another release branch later on anyway so the author must rebase against the 
> base branch first. No need spending test cycles on that.
>
> Most likely github already reports that PR as 'unstable' so you could also 
> check for that.
>
> Regards, Remi
>
> Sent from my iPhone
>
>> On 28 Mar 2016, at 09:28, Bharat Kumar <bharat.ku...@accelerite.com> wrote:
>>
>> Hi Sanjeev,
>>
>> Thanks for bringing this up.
>>
>> This is happening because the PR code and the master have deviated since the 
>> pr has been created(assuming that PR was rebased with master at the time of 
>> creation)  and we start testing it. Ideally we should start testing as soon 
>> as the pr is created but due to hardware limitation we have to queue the PRs 
>> for testing.
>>
>> One way of fixing this would be to merge the PR with master before testing, 
>> If the merge fails we post a comment on the PR and skip testing it, until it 
>> can me merged.
>>
>> We need to make sure that all the PR we create can be merged to master 
>> without conflicts.
>>
>> Thanks,
>> Bharat.
>>
>>
>> On 28-Mar-2016, at 12:35 PM, Sanjeev Neelarapu 
>> <sanjeev.neelar...@accelerite.com<mailto:sanjeev.neelar...@accelerite.com>> 
>> wrote:
>>
>> Hi,
>>
>> Currently CI is picking the code from the PR branch, which may or may not be 
>> rebased with latest master. This is causing test failures even though they 
>> were fixed in latest master.
>> e.g.: test_vpc_site2site_vpn.
>> Error Message
>> local variable 'vm1' referenced before assignment.
>>
>> There were few issues with this test suite which were fixed in master. 
>> However, we don't see these changes in some of the PR branches.
>>
>> Any thoughts on how to overcome this?
>>
>>
>> Best Regards,
>> Sanjeev N
>> Chief Product Engineer, Accelerite
>> Off: +91 40 6722 9368 | EMail: 
>> sanjeev.neelar...@accelerite.com<mailto:sanjeev.neelar...@accelerite.com>
>>
>>
>> DISCLAIMER == This e-mail may contain privileged and confidential 
>> information which is the property of Accelerite, a Persistent Systems 
>> business. It is intended only for the use of the individual or entity to 
>> which it is addressed. If you are not the intended recipient, you are not 
>> authorized to read, retain, copy, print, distribute or use this message. If 
>> you have received this communication in error, please notify the sender and 
>> delete all copies of this message. Accelerite, a Persistent Systems business 
>> does not accept any liability for virus infected mails.
>>
>>
>>
>>
>> DISCLAIMER
>> ==
>> This e-mail may contain privileged and confidential information which is the 
>> property of Accelerite, a Persistent Systems business. It is intended only 
>> for the use of the individual or entity to which it is addressed. If you are 
>> not the intended recipient, you are not authorized to read, retain, copy, 
>> print, distribute or use this message. If you have received this 
>> communication in error, please notify the sender and delete all copies of 
>> this message. Accelerite, a Persistent Systems business does not accept any 
>> liability for virus infected mails.
>
>
>
> DISCLAIMER
> ==
> This e-mail may contain privileged and confidential information which is the 
> property of Accelerite, a Persistent Systems business. It is intended only 
> for the use of the individual or entity to which it is addressed. If you are 
> not the intended recipient, you are not authorized to read, retain, copy, 
> print, distribute or use this message. If you have received this 
> communication in error, please notify the sender and delete all copies of 
> this message. Accelerite, a Persistent Systems business does not

Re: [DISCUSS] PR testing process

2016-03-28 Thread Bharat Kumar

Hi Sanjeev,

Thanks for bringing this up.

This is happening because the PR code and the master have deviated since the pr 
has been created(assuming that PR was rebased with master at the time of 
creation)  and we start testing it. Ideally we should start testing as soon as 
the pr is created but due to hardware limitation we have to queue the PRs for 
testing.

One way of fixing this would be to merge the PR with master before testing, If 
the merge fails we post a comment on the PR and skip testing it, until it can 
me merged.

We need to make sure that all the PR we create can be merged to master without 
conflicts.

Thanks,
Bharat.


On 28-Mar-2016, at 12:35 PM, Sanjeev Neelarapu 
> 
wrote:

Hi,

Currently CI is picking the code from the PR branch, which may or may not be 
rebased with latest master. This is causing test failures even though they were 
fixed in latest master.
e.g.: test_vpc_site2site_vpn.
Error Message
local variable 'vm1' referenced before assignment.

There were few issues with this test suite which were fixed in master. However, 
we don’t see these changes in some of the PR branches.

Any thoughts on how to overcome this?


Best Regards,
Sanjeev N
Chief Product Engineer, Accelerite
Off: +91 40 6722 9368 | EMail: 
sanjeev.neelar...@accelerite.com


DISCLAIMER == This e-mail may contain privileged and confidential 
information which is the property of Accelerite, a Persistent Systems business. 
It is intended only for the use of the individual or entity to which it is 
addressed. If you are not the intended recipient, you are not authorized to 
read, retain, copy, print, distribute or use this message. If you have received 
this communication in error, please notify the sender and delete all copies of 
this message. Accelerite, a Persistent Systems business does not accept any 
liability for virus infected mails.




DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Accelerite, a Persistent Systems business. It is intended only for 
the use of the individual or entity to which it is addressed. If you are not 
the intended recipient, you are not authorized to read, retain, copy, print, 
distribute or use this message. If you have received this communication in 
error, please notify the sender and delete all copies of this message. 
Accelerite, a Persistent Systems business does not accept any liability for 
virus infected mails.

Re: introduction

2016-03-23 Thread Bharat Kumar

Welcome back Murali.
—Bharat.

> On 23-Mar-2016, at 9:50 PM, Rajesh Battala  
> wrote:
> 
> Welcome Back Murali. Good to see you again on CloudStack :) 
> 
> -Original Message-
> From: Murali Reddy [mailto:murali.re...@shapeblue.com] 
> Sent: Wednesday, March 23, 2016 5:21 PM
> To: dev@cloudstack.apache.org
> Subject: introduction
> 
> All,
> 
> Just wanted to let the community know that I have decided to work on Apache 
> CloudStack again. I have taken up a position at ShapeBlue, so you will be 
> seeing me around @dev and @users. For the new members of the community and 
> others who I have not interacted with, here is small self introduction. I 
> started my journey with CloudStack, in late 2010 as cloud.com employee. Later 
> spent 4 years till late 2014 working on various CloudStack features (EIP, 
> GSLB, NAAS, NetScaler integration, native SDN controller, distributed virtual 
> router, event bus to name few). I was working on NFV solution on OpenStack 
> over a year. I am excited to join back the community, looking forward to 
> interact with you.
> 
> Thanks,
> Murali
> 
> 
> 
> DISCLAIMER
> ==
> This e-mail may contain privileged and confidential information which is the 
> property of Accelerite, a Persistent Systems business. It is intended only 
> for the use of the individual or entity to which it is addressed. If you are 
> not the intended recipient, you are not authorized to read, retain, copy, 
> print, distribute or use this message. If you have received this 
> communication in error, please notify the sender and delete all copies of 
> this message. Accelerite, a Persistent Systems business does not accept any 
> liability for virus infected mails.




DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Accelerite, a Persistent Systems business. It is intended only for 
the use of the individual or entity to which it is addressed. If you are not 
the intended recipient, you are not authorized to read, retain, copy, print, 
distribute or use this message. If you have received this communication in 
error, please notify the sender and delete all copies of this message. 
Accelerite, a Persistent Systems business does not accept any liability for 
virus infected mails.

RE: [VOTE] Move 'apache/cloudstack' -> 'apache-cloudstack/cloudstack'

2016-03-21 Thread Bharat Kumar

+1
--Bharat

On 21 Mar 2016 11:51 am, Kishan Kavala  wrote:
+1 for the move.

Regards,
Kishan Kavala
Chief Product Engineer,
CloudPlatform Development, Accelerite.
www.accelerite.com

-Original Message-
From: chunfeng tian [mailto:tianyunqu...@gmail.com]
Sent: 20 March 2016 05:22 PM
To: dev@cloudstack.apache.org
Subject: Re: [VOTE] Move 'apache/cloudstack' -> 'apache-cloudstack/cloudstack'

+1

On Sun, Mar 20, 2016 at 1:51 PM, Mattmann, Chris A (3980) < 
chris.a.mattm...@jpl.nasa.gov> wrote:

> +1..
>
> ++
> Chris Mattmann, Ph.D.
> Chief Architect
> Instrument Software and Science Data Systems Section (398) NASA Jet
> Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 168-519, Mailstop: 168-527
> Email: chris.a.mattm...@nasa.gov
> WWW:  http://sunset.usc.edu/~mattmann/
> ++
> Director, Information Retrieval and Data Science Group (IRDS) Adjunct
> Associate Professor, Computer Science Department University of
> Southern California, Los Angeles, CA 90089 USA
> WWW: http://irds.usc.edu/
> ++
>
>
>
>
>
> -Original Message-
> From: Will Stevens 
> Reply-To: "dev@cloudstack.apache.org" 
> Date: Friday, March 18, 2016 at 3:44 PM
> To: "dev@cloudstack.apache.org" 
> Subject: [VOTE] Move 'apache/cloudstack' -> 'apache-cloudstack/cloudstack'
>
> >We are discussing this proposal in 3 or 4 threads, so I will not try
> >to recap everything.  Instead I will try to give a brief overview of
> >the problem and a proposal for solving it.
> >
> >*Problem:*
> >The Apache CloudStack community needs additional github permissions
> >in order to integrate CI for the purpose of maintaining code quality.
> >The ASF does not have enough granular control via the 'apache' github
> >organization to give the 'apache/cloudstack' repository the needed
> >permissions.
> >
> >*Proposal:*
> >Transfer ownership of the 'apache/cloudstack' mirrored repository out
> >of the 'apache' github organization into the 'apache-cloudstack'
> >github organization (which I have already setup and started inviting users 
> >to).
> >Both members of the ACS community and the ASF board will have 'owner'
> >permissions on this new organization.  This will allow for
> >permissions to be applied specifically to the 'apache-cloudstack'
> >organization and not have to be applied to the entire 'apache' organization.
> >
> >By transferring ownership, all of the PRs will be copied to the new
> >repository and redirects will be created on github from
> >'apache/cloudstack'
> >to 'apache-cloudstack/cloudstack'.
> >
> >The developer workflow and commit workflow will remain unchanged.
> >The canonical ASF repository (git://git.apache.org/cloudstack.git)
> >will
> remain
> >the source of truth and commits will be made to that repository.
> >
> >Please ask if anything is unclear or needs to be better defined in
> >order for you to cast a vote.
>
>

--
Tian ChunFeng
http://cloud.domolo.com

DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Accelerite, a Persistent Systems business. It is intended only for 
the use of the individual or entity to which it is addressed. If you are not 
the intended recipient, you are not authorized to read, retain, copy, print, 
distribute or use this message. If you have received this communication in 
error, please notify the sender and delete all copies of this message. 
Accelerite, a Persistent Systems business does not accept any liability for 
virus infected mails.

DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Accelerite, a Persistent Systems business. It is intended only for 
the use of the individual or entity to which it is addressed. If you are not 
the intended recipient, you are not authorized to read, retain, copy, print, 
distribute or use this message. If you have received this communication in 
error, please notify the sender and delete all copies of this message. 
Accelerite, a Persistent Systems business does not accept any liability for 
virus infected mails.

Re: [PROPOSAL] Minimum Viable CI Integration

2016-03-19 Thread Bharat Kumar

Hi Remi,

I have moved most of the failing tests to known issues, and modified the 
reports accordingly. Now we should see only the passed test suits.
When we see a failed test in multiple runs we can move it to known issues and 
start fixing them.

Thanks,
Bharat.

On 15-Mar-2016, at 4:36 PM, Remi Bergsma 
<rberg...@schubergphilis.com<mailto:rberg...@schubergphilis.com>> wrote:

Hi Bharat

I’d say remove the tests that fail, to get a report with only tests that pass.

Regards,
Remi

From: Bharat Kumar 
<bharat.ku...@accelerite.com<mailto:bharat.ku...@accelerite.com>>
Date: Tuesday 15 March 2016 at 10:40
To: Remi Bergsma 
<rberg...@schubergphilis.com<mailto:rberg...@schubergphilis.com>>
Cc: "dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>" 
<dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>>, Bharat Kumar 
<bharat.ku...@accelerite.com<mailto:bharat.ku...@accelerite.com>>
Subject: Re: [PROPOSAL] Minimum Viable CI Integration

Hi Remi,

Thanks for the inputs.

I agree with you, May be i need to modify the CI report and not report if any 
of the known issues fail.  We can still have the known_issues header to track 
what needs to be fixed.

Thanks,
Bharat.


On 15-Mar-2016, at 1:26 PM, Remi Bergsma 
<rberg...@schubergphilis.com<mailto:rberg...@schubergphilis.com>> wrote:

Hi Bharat,

I’d suggest removing all tests that fail for some reason, not being an error in 
the PR. This will result in a smaller set of tests that “always pass”. Once a 
test doesn’t pass anymore, we know the PR has an issue. Then your CI system 
provides value.

Right now, there are so many failures that soon people don’t even care to look 
at it. Once we have a stable set that works, we can start fixing tests and 
1-by-1 add them to the mix.

Just my $0.02

Regards,
Remi



On 15/03/16 07:06, "Bharat Kumar" 
<bharat.ku...@accelerite.com<mailto:bharat.ku...@accelerite.com>> wrote:

Hi Paul,

I totally agree with you. For now we should ignore the tests which fail all the 
time while we try to address them.

Now that we have a CI in place we need go and fix issues with the tests.

I have tried to identify a few of them and have  mentioned them in the known 
issues part of the CI test result.

Some of the test cases are not generic enough to run on different CIs. For 
example the test_vpc_vpn.py
hardcodes the test data in the test itself, which is not a good thing. 
Generally this kind of data is added the the test_data.py file and is reused .
This gives us a chance to modify it to suit the needs of the a particular CI.

So for fixing these issues we need test data to identify the issue. Each of the 
CI run gives these details, the test run info, test results and the management 
server logs( via the drop box link in the report).
With this info any one in the community can access this data and start fixing 
them. This way we can progressively track and improve the health of the release.

So i think we should continue posting these results, so that people can analyse 
the result and come back with inputs related to incorrect test cases.

—Thanks
Bharat.


On 15-Mar-2016, at 1:24 AM, Paul Angus 
<paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com><mailto:paul.an...@shapeblue.com>>
 wrote:

Cool. Thanks Will.

@Bharat - the work that you're trying to do is great, but until the Marvin 
tests have been gone through and 'fixed' the information being put into PRs is 
misleading.

No one knows why these failures are occurring, and it's very likely giving both 
false positives and false negatives.
You should probably hold off altogether or isolate the good tests and only run 
them.

I plan to set up a wiki page (or two) with all of the tests listed, their 
purpose and other info such as if there are known to work (or not) when they 
were last checked. You're welcome to make a start on it if you wish.

For example (false negative):
--
nosetests --with-marvin --marvin-config=/tmp/te2.cfg --hypervisor=KVM -a 
tags=advanced,required_hardware=true 
integration/component/test_acl_isolatednetwork.py
2016-01-30 07:36:30,488 - DEBUG - Parsing Test data successful
2016-01-30 07:36:30,488 - DEBUG - Payload: {'account': 'admin', 'command': 
'listUsers', 'response': 'json'}
2016-01-30 07:36:30,488 - DEBUG - Sending GET Cmd : listUsers===
2016-01-30 07:36:30,521 - DEBUG - Response : [{username : u'admin', account : 
u'admin', domainid : u'4895ddbe-bfac-11e5-be38-06a910012727', firstname : 
u'admin', created : u'2016-01-20T19:33:19+', lastname : u'cloud', apikey : 
u'oiD-zcjZw7dsPq8yrjarls3sCtHV0E8FQYRpNXlmEx0ssJ1fKFuCySYL4vLHVmlLA5E3PF7_hSnTND_eSRyK_A',
 domain : u'ROOT', secretkey : 
u'gLz6lDVFch6ayBAB-8SMZ-RPiOgJTEVJmNgnAMdv36O6eMUKUN6R-Q-3ZtYJUH84jye-5q6Qpxf5WOtSuG8WtA',
 iscallerchilddomain : False, state : u'enabled', accounttype : 1, id : 
u'a8b21230-bfac-1

Re: [PROPOSAL] Minimum Viable CI Integration

2016-03-15 Thread Bharat Kumar

Hi Remi,

Thanks for the inputs.

I agree with you, May be i need to modify the CI report and not report if any
of the known issues fail. We can still have the known_issues header to track
what needs to be fixed.

Thanks,
Bharat.

On 15-Mar-2016, at 1:26 PM, Remi Bergsma
<rberg...@schubergphilis.com<mailto:rberg...@schubergphilis.com>> wrote:

Hi Bharat,

I’d suggest removing all tests that fail for some reason, not being an error in
the PR. This will result in a smaller set of tests that “always pass”. Once a
test doesn’t pass anymore, we know the PR has an issue. Then your CI system
provides value.

Right now, there are so many failures that soon people don’t even care to look
at it. Once we have a stable set that works, we can start fixing tests and
1-by-1 add them to the mix.

Just my $0.02

Regards,
Remi

On 15/03/16 07:06, "Bharat Kumar"
<bharat.ku...@accelerite.com<mailto:bharat.ku...@accelerite.com>> wrote:

Hi Paul,

I totally agree with you. For now we should ignore the tests which fail all the
time while we try to address them.

Now that we have a CI in place we need go and fix issues with the tests.

I have tried to identify a few of them and have mentioned them in the known
issues part of the CI test result.

Some of the test cases are not generic enough to run on different CIs. For
example the test_vpc_vpn.py
hardcodes the test data in the test itself, which is not a good thing.
Generally this kind of data is added the the test_data.py file and is reused .
This gives us a chance to modify it to suit the needs of the a particular CI.

So for fixing these issues we need test data to identify the issue. Each of the
CI run gives these details, the test run info, test results and the management
server logs( via the drop box link in the report).
With this info any one in the community can access this data and start fixing
them. This way we can progressively track and improve the health of the release.

So i think we should continue posting these results, so that people can analyse
the result and come back with inputs related to incorrect test cases.

—Thanks
Bharat.

On 15-Mar-2016, at 1:24 AM, Paul Angus
<paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com><mailto:paul.an...@shapeblue.com>>
wrote:

Cool. Thanks Will.

@Bharat - the work that you're trying to do is great, but until the Marvin
tests have been gone through and 'fixed' the information being put into PRs is
misleading.

No one knows why these failures are occurring, and it's very likely giving both
false positives and false negatives.
You should probably hold off altogether or isolate the good tests and only run
them.

I plan to set up a wiki page (or two) with all of the tests listed, their
purpose and other info such as if there are known to work (or not) when they
were last checked. You're welcome to make a start on it if you wish.

For example (false negative):
--
nosetests --with-marvin --marvin-config=/tmp/te2.cfg --hypervisor=KVM -a
tags=advanced,required_hardware=true
integration/component/test_acl_isolatednetwork.py
2016-01-30 07:36:30,488 - DEBUG - Parsing Test data successful
2016-01-30 07:36:30,488 - DEBUG - Payload: {'account': 'admin', 'command':
'listUsers', 'response': 'json'}
2016-01-30 07:36:30,488 - DEBUG - Sending GET Cmd : listUsers===
2016-01-30 07:36:30,521 - DEBUG - Response : [{username : u'admin', account :
u'admin', domainid : u'4895ddbe-bfac-11e5-be38-06a910012727', firstname :
u'admin', created : u'2016-01-20T19:33:19+', lastname : u'cloud', apikey :
u'oiD-zcjZw7dsPq8yrjarls3sCtHV0E8FQYRpNXlmEx0ssJ1fKFuCySYL4vLHVmlLA5E3PF7_hSnTND_eSRyK_A',
domain : u'ROOT', secretkey :
u'gLz6lDVFch6ayBAB-8SMZ-RPiOgJTEVJmNgnAMdv36O6eMUKUN6R-Q-3ZtYJUH84jye-5q6Qpxf5WOtSuG8WtA',
iscallerchilddomain : False, state : u'enabled', accounttype : 1, id :
u'a8b21230-bfac-11e5-be38-06a910012727', isdefault : True, accountid :
u'a8b2063c-bfac-11e5-be38-06a910012727'}]
2016-01-30 07:36:30,521 - DEBUG - Test Client Creation Successful
---

Nothing actually happens ?!
Same with test_acl_listvm.py

Re: [PROPOSAL] Minimum Viable CI Integration

2016-03-15 Thread Bharat Kumar

Hi Paul,

I totally agree with you. For now we should ignore the tests which fail all the
time while we try to address them.

Now that we have a CI in place we need go and fix issues with the tests.

I have tried to identify a few of them and have mentioned them in the known
issues part of the CI test result.

So i think we should continue posting these results, so that people can analyse
the result and come back with inputs related to incorrect test cases.

—Thanks
Bharat.

On 15-Mar-2016, at 1:24 AM, Paul Angus
> wrote:

Cool. Thanks Will.

@Bharat - the work that you're trying to do is great, but until the Marvin
tests have been gone through and 'fixed' the information being put into PRs is
misleading.

Nothing actually happens ?!
Same with test_acl_listvm.py

Example(false positive-failure is in marvin test code):
-
nosetests --with-marvin --marvin-config=/tmp/te2.cfg --hypervisor=KVM -a
tags=advanced,required_hardware=true
/tmp/integration/component/test_browse_templates.py
2016-01-30 19:52:26,160 - CRITICAL - EXCEPTION: None: ['Traceback (most recent
call last):\n', ' File "/usr/lib/python2.7/site-packages/nose/suite.py", line
209, in run\n self.setUp()\n', ' File
"/usr/lib/python2.7/site-packages/nose/suite.py", line 292, in setUp\n
self.setupContext(ancestor)\n', ' File
"/usr/lib/python2.7/site-packages/nose/suite.py", line 315, in setupContext\n
try_run(context, names)\n', ' File
"/usr/lib/python2.7/site-packages/nose/util.py", line 471, in try_run\n return
func()\n', ' File "/tmp/integration/component/test_browse_templates.py", line
84, in setUpClass\n
cls.uploadurl=cls.testdata["configurableData"]["browser_upload_template"][cls.uploadtemplateformat]["url"]\n',
"KeyError: 'browser_upload_template'\n"]

Paul Angus
VP Technology , ShapeBlue

d: +44 203 617 0528 | s: +44 203 603
0540 |
m: +44 7711 418784

e: paul.an...@shapeblue.com | t:
@cloudyangus |
w: www.shapeblue.com

a: 53 Chandos Place, Covent Garden London WC2N 4HS UK

Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services
India LLP is a company incorporated in India and is operated under license from
Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
Brasil and is operated under

Re: [PROPOSAL] Minimum Viable CI Integration

2016-03-07 Thread Bharat Kumar

Hi Suresh,

We can only get which assertion failed, but for getting the actual reason i.e. 
what happened in cloudstack  we need to check the cloudstack log. I am 
uploading the logs to dropBox.

Thanks,
Bharat.

> On 08-Mar-2016, at 11:36 AM, Suresh Anaparti <suresh.anapa...@accelerite.com> 
> wrote:
> 
> Hi Bharat,
> 
> Good to see the list of failed and skipped ones in the report. Is it possible 
> to add some generic comment/reason to know why these test cases are 
> failed/skipped?
> 
> Thanks,
> Suresh
> 
> 
> 
> 
> On 08/03/16 11:05 am, "Bharat Kumar" <bharat.ku...@accelerite.com> wrote:
> 
>> Hi Srinivas,
>> 
>> The tests get skipped because the hardware requirement for those tests is 
>> not satisfied. I will try to add more details to
>> the skipped tests so that people can run manually if needed.
>> 
>> The details of the hardware used can be added, but I am thinking of 
>> publishing it in the wiki
>> rather than adding these in the test results.
>> 
>> Thanks for the pointers.
>> 
>> —Bharat.
>> 
>> 
>> On 08-Mar-2016, at 10:36 AM, Gandikota Srinivas 
>> <gandikota.srini...@gmail.com<mailto:gandikota.srini...@gmail.com>> wrote:
>> 
>> Hi Bharat,
>> 
>> Great job, report looks really cool.
>> Will you be able to add few more details like the setup info (number of 
>> hosts, etc)
>> 
>> Report indicates few tests are skipped. is this due to setup limitations? 
>> can the skipped tests be also listed so that some of us can run those 
>> specific tests (say manually) and post the results.
>> 
>> Thanks,
>> Srinivas
>> 
>> On Mon, Mar 7, 2016 at 11:06 PM, Bharat Kumar 
>> <bharat.ku...@accelerite.com<mailto:bharat.ku...@accelerite.com>> wrote:
>> Hi guys,
>> 
>> I am also working on the similar reporting problem, here is what i did
>> 
>> link to the report https://github.com/bvbharatk/cloud-stack/pull/1
>> 
>> I am thinking this is good enough for now, I want to start posting the 
>> results on each pr as shown in the above link.
>> please give me your comments or suggestions.
>> 
>> Thanks,
>> Bharat
>> On 05-Mar-2016, at 7:02 PM, Will Stevens 
>> <wstev...@cloudops.com<mailto:wstev...@cloudops.com><mailto:wstev...@cloudops.com<mailto:wstev...@cloudops.com>>>
>>  wrote:
>> 
>> Daan
>> 
>> Regarding the obligatory provider id.  I agree, but I am still trying to
>> figure out the details.  Creating distinct runs that have their own status
>> is done by setting the 'context'.  I think we would need to have two pieces
>> to this.  A provider id and an environment id.
>> 
>> So for example.  Lets assume that my provider id is 'CloudOps' and I have
>> two different environments, one for 'KVM' and one for 'Xen' (for example).
>> I would then the tool would produce the following two independent CI
>> statuses.
>> 'CloudOps - KVM' : with a basic description of the environment.
>> 'CloudOps - Xen' : again with a basic description of the env.
>> ... and so on ...
>> 
>> I am still sorting out the details here as well as making it easy for us to
>> integrate this into the apache/cloudstack repo with the access we currently
>> have.  Adding this is 'no biggy' for me because I am building this tool as
>> we speak, and trying to tailor it to our (ACS) needs, so this type of
>> feedback is perfect as it allows me to adapt the tool before I get too deep.
>> 
>> *Will STEVENS*
>> Lead Developer
>> 
>> *CloudOps* *| *Cloud Solutions Experts
>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>> w 
>> cloudops.com<http://cloudops.com/><http://cloudops.com<http://cloudops.com/>>
>>  *|* tw @CloudOps_
>> 
>> On Sat, Mar 5, 2016 at 4:17 AM, Daan Hoogland 
>> <daan.hoogl...@gmail.com<mailto:daan.hoogl...@gmail.com><mailto:daan.hoogl...@gmail.com<mailto:daan.hoogl...@gmail.com>>>
>> wrote:
>> 
>> Will
>> 
>> Gret work, especially the thing you are showing in link [4], I would like
>> to make an enhancement request and that is a obligatory provider id. Only
>> if it is no biggy for you!
>> Several people may decide to do a XVM on ChildrensOS for instance and so we
>> may be aware of an obscurity that is different. If one fails and the other
>> succeeds it is easily identified.
>> 
>> Ilya,
>> 
>> I have been playing with go and it is a very nice language for such a
>> simple script, though it was

Re: [PROPOSAL] Minimum Viable CI Integration

2016-03-07 Thread Bharat Kumar

Hi Srinivas,

The tests get skipped because the hardware requirement for those tests is not
satisfied. I will try to add more details to
the skipped tests so that people can run manually if needed.

The details of the hardware used can be added, but I am thinking of publishing
it in the wiki
rather than adding these in the test results.

Thanks for the pointers.

—Bharat.

On 08-Mar-2016, at 10:36 AM, Gandikota Srinivas
<gandikota.srini...@gmail.com<mailto:gandikota.srini...@gmail.com>> wrote:

Hi Bharat,

Great job, report looks really cool.
Will you be able to add few more details like the setup info (number of hosts,
etc)

Report indicates few tests are skipped. is this due to setup limitations? can
the skipped tests be also listed so that some of us can run those specific
tests (say manually) and post the results.

Thanks,
Srinivas

On Mon, Mar 7, 2016 at 11:06 PM, Bharat Kumar
<bharat.ku...@accelerite.com<mailto:bharat.ku...@accelerite.com>> wrote:
Hi guys,

I am also working on the similar reporting problem, here is what i did

link to the report https://github.com/bvbharatk/cloud-stack/pull/1

I am thinking this is good enough for now, I want to start posting the results
on each pr as shown in the above link.
please give me your comments or suggestions.

Thanks,
Bharat
On 05-Mar-2016, at 7:02 PM, Will Stevens
<wstev...@cloudops.com<mailto:wstev...@cloudops.com><mailto:wstev...@cloudops.com<mailto:wstev...@cloudops.com>>>
wrote:

Daan

Regarding the obligatory provider id. I agree, but I am still trying to
figure out the details. Creating distinct runs that have their own status
is done by setting the 'context'. I think we would need to have two pieces
to this. A provider id and an environment id.

So for example. Lets assume that my provider id is 'CloudOps' and I have
two different environments, one for 'KVM' and one for 'Xen' (for example).
I would then the tool would produce the following two independent CI
statuses.
'CloudOps - KVM' : with a basic description of the environment.
'CloudOps - Xen' : again with a basic description of the env.
... and so on ...

I am still sorting out the details here as well as making it easy for us to
integrate this into the apache/cloudstack repo with the access we currently
have. Adding this is 'no biggy' for me because I am building this tool as
we speak, and trying to tailor it to our (ACS) needs, so this type of
feedback is perfect as it allows me to adapt the tool before I get too deep.

*Will STEVENS*
Lead Developer

On Sat, Mar 5, 2016 at 4:17 AM, Daan Hoogland
<daan.hoogl...@gmail.com<mailto:daan.hoogl...@gmail.com><mailto:daan.hoogl...@gmail.com<mailto:daan.hoogl...@gmail.com>>>
wrote:

Will

Gret work, especially the thing you are showing in link [4], I would like
to make an enhancement request and that is a obligatory provider id. Only
if it is no biggy for you!
Several people may decide to do a XVM on ChildrensOS for instance and so we
may be aware of an obscurity that is different. If one fails and the other
succeeds it is easily identified.

Ilya,

I have been playing with go and it is a very nice language for such a
simple script, though it wasn't exactly designed for it. So don't read my
comment/question as an objection. But we do have
bash,python,c#,java,javascript,xslt,sql at least. That is not counting the
build system and I am sure the hyperv has some extra windows specific
stuff.
To me it is inherent to the nature of across platform orchestration and
provisioning system so it is fine. It is something to consider. On the
other hand when bringing in new tools we don't make the choice so I am
ranting, I guess.

On Sat, Mar 5, 2016 at 7:38 AM, ilya
<ilya.mailing.li...@gmail.com<mailto:ilya.mailing.li...@gmail.com><mailto:ilya.mailing.li...@gmail.com<mailto:ilya.mailing.li...@gmail.com>>>
wrote:

I see where Daan is coming from :) I thought this would be 4th, not
exactly 7ths.

I'm not against golang by any means (if anything - its my next "go" to
language these days).

Things to consider:

Would notify-pr support proxy? I've been thinking on ways of
contributing test runs, there would have to be few things i'd need to do.

1) massage the log content - such that no environment details are
exposed, i can probably handle this with sed/awk..

2) i'm behind multiple firewalls with no internet access. however, some
lab environments might have a proxy, so it would be nice to have a
support for it.

Thanks
ilya

On 3/4/16 6:56 AM, Will Stevens wrote:
Yes, I have most of it already built and will be releasing it later
today
or over the weekend. The reason I chose Golang is because it can be
cross
compiled to be run on a

Re: [PROPOSAL] Minimum Viable CI Integration

2016-03-07 Thread Bharat Kumar

Hi guys,

I am also working on the similar reporting problem, here is what i did

link to the report https://github.com/bvbharatk/cloud-stack/pull/1

I am thinking this is good enough for now, I want to start posting the results 
on each pr as shown in the above link.
please give me your comments or suggestions.

Thanks,
Bharat
On 05-Mar-2016, at 7:02 PM, Will Stevens 
> wrote:

Daan

Regarding the obligatory provider id.  I agree, but I am still trying to
figure out the details.  Creating distinct runs that have their own status
is done by setting the 'context'.  I think we would need to have two pieces
to this.  A provider id and an environment id.

So for example.  Lets assume that my provider id is 'CloudOps' and I have
two different environments, one for 'KVM' and one for 'Xen' (for example).
I would then the tool would produce the following two independent CI
statuses.
'CloudOps - KVM' : with a basic description of the environment.
'CloudOps - Xen' : again with a basic description of the env.
... and so on ...

I am still sorting out the details here as well as making it easy for us to
integrate this into the apache/cloudstack repo with the access we currently
have.  Adding this is 'no biggy' for me because I am building this tool as
we speak, and trying to tailor it to our (ACS) needs, so this type of
feedback is perfect as it allows me to adapt the tool before I get too deep.

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Sat, Mar 5, 2016 at 4:17 AM, Daan Hoogland 
>
wrote:

Will

Gret work, especially the thing you are showing in link [4], I would like
to make an enhancement request and that is a obligatory provider id. Only
if it is no biggy for you!
Several people may decide to do a XVM on ChildrensOS for instance and so we
may be aware of an obscurity that is different. If one fails and the other
succeeds it is easily identified.

Ilya,

I have been playing with go and it is a very nice language for such a
simple script, though it wasn't exactly designed for it. So don't read my
comment/question as an objection. But we do have
bash,python,c#,java,javascript,xslt,sql at least. That is not counting the
build system and I am sure the hyperv has some extra windows specific
stuff.
To me it is inherent to the nature of across platform orchestration and
provisioning system so it is fine. It is something to consider. On the
other hand when bringing in new tools we don't make the choice so I am
ranting, I guess.

On Sat, Mar 5, 2016 at 7:38 AM, ilya 
> wrote:

I see where Daan is coming from :)  I thought this would be 4th, not
exactly 7ths.

I'm not against golang by any means (if anything - its my next "go" to
language these days).

Things to consider:

Would notify-pr support proxy? I've been thinking on ways of
contributing test runs, there would have to be few things i'd need to do.

1) massage the log content - such that no environment details are
exposed, i can probably handle this with sed/awk..

2) i'm behind multiple firewalls with no internet access. however, some
lab environments might have a proxy, so it would be nice to have a
support for it.

Thanks
ilya

On 3/4/16 6:56 AM, Will Stevens wrote:
Yes, I have most of it already built and will be releasing it later
today
or over the weekend.  The reason I chose Golang is because it can be
cross
compiled to be run on any system and distributed as a single binary
with
no
dependencies.  This means that no one will have to worry about building
it
or having to change their environment at all in order to use it.  I am
trying to lower the barrier to entry and make it as easy as possible
for
people to contribute back their CI execution details.

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Fri, Mar 4, 2016 at 8:53 AM, Daan Hoogland 

wrote:

Will, Do you have an implementation of notify-pr? I am asking as you
specify it will be implemented in golang which seems odd. It is not
amongst
the 7 or so languages already in use.

On Fri, Mar 4, 2016 at 1:54 AM, Will Stevens <
williamstev...@gmail.com>
wrote:

Hey Everyone,
As I am sure most of you are aware, I have been focusing a lot on
ways
to
get CI integrated back into the community.

Today I build a little POC to validate some ideas and get a feel for
a
potential approach for getting CI integrated into the Github pull
request
workflow.

There are multiple individuals/companies focusing on CI right now
(which
is a good thing), but there has not really been much discussion
(that I

Re: [PROPOSAL] Minimum Viable CI Integration

2016-03-03 Thread Bharat Kumar

Thanks for the info Will.

I also faced a similar problem. I will get back to you if i see any further 
issues.

Thanks,
Bharat.

> On 04-Mar-2016, at 10:12 AM, Will Stevens <williamstev...@gmail.com> wrote:
>
> Awesome, thanks for the update Bharat, that is great progress.
>
> If you need a hand with the posting back to the pull requests on github,
> just let me know, I have that piece working.  My implementation is a cross
> platform binary without any dependencies so it could be easy to integrate
> if that would be helpful.  I plan to clean it up a bit and open source it.
> I have only spent a few hours on it so far, so I want to do a once through
> to get it ready to release.
>
> Just an FYI, it is not very easy to post back to github using the 'Pull
> Requests' API because of the code reference requirements.  Checkout the
> 'Issues' API which is basically the same thing, but enables you to create
> comments without having to specify a specific location in the PRs code
> which you are commenting on.  This tripped me up for a while today and once
> I realized I could do it through the Issues instead of the Pull Requests,
> it made things infinitely easier.
>
> Let me know if you have other questions.
>
> Cheers,
>
> Will
>
>
> On Thu, Mar 3, 2016 at 11:29 PM, Bharat Kumar <bharat.ku...@accelerite.com>
> wrote:
>
>> Hi will,
>>
>> we have a solution to post the results to the community by email. We also
>> have a github integration to fetch the Prs , run tests against them and post
>> the consolidated results by email and share the logs using dropbox.
>>
>> We are facing some setup delays to get this up and running. I am sure we
>> will be able to do this shortly.
>>
>> I am also working on posting the results in PR as comments.
>>
>> Thanks,
>> Bharat.
>>
>>
>>
>>> On 04-Mar-2016, at 7:36 AM, Will Stevens <williamstev...@gmail.com>
>> wrote:
>>>
>>> Last I knew Bharat did not have a solution for posting results back to
>> the
>>> community. I could be wrong though, I don't really know how complete a
>>> solution Bharat has at this point.
>>>
>>> There are two other CI implementations in various states of completeness
>>> and I think it is important to have a common mechanism which will work to
>>> post results back to the community regardless of the CI implementation.
>>>
>>> "bubble" from SBP seems to be the most complete so far and has been used
>> to
>>> test over 100 PRs, but I don't think the results have formally
>> contributed
>>> back for the community to review.
>>>
>>> "trillian" is a project shapeblue is working on, but I am still getting
>>> details for when it is expected to be operational.
>>>
>>> This may not be a final solution, but I feel it is a good first step
>> which
>>> will be easy for everyone to work with.
>>> On Mar 3, 2016 8:35 PM, "Gandikota Srinivas" <
>> gandikota.srini...@gmail.com>
>>> wrote:
>>>
>>>> Will,
>>>> I guess Bharat has something similar in working.
>>>>
>>>> Bharat,
>>>> Can you please elaborate your approach for sharing the CI results with
>>>> community ?
>>>>
>>>> Thanks,
>>>> Srinivas
>>>>
>>>> On Fri, Mar 4, 2016 at 6:28 AM, Will Stevens <williamstev...@gmail.com>
>>>> wrote:
>>>>
>>>>> Apparently attached files don't work when sending to the mailing list.
>>>>>
>>>>> Find the screenshot here:
>>>>>
>>>>>
>>>>
>> https://objects-east.cloud.ca/v1/5ef827605f884961b94881e928e7a250/swill/Screen%20Shot%202016-03-03%20at%207.53.42%20PM.png
>>>>>
>>>>> On Thu, Mar 3, 2016 at 7:54 PM, Will Stevens <williamstev...@gmail.com
>>>
>>>>> wrote:
>>>>>
>>>>>> Hey Everyone,
>>>>>> As I am sure most of you are aware, I have been focusing a lot on ways
>>>> to
>>>>>> get CI integrated back into the community.
>>>>>>
>>>>>> Today I build a little POC to validate some ideas and get a feel for a
>>>>>> potential approach for getting CI integrated into the Github pull
>>>> request
>>>>>> workflow.
>>>>>>
>>>>>> There are multiple individuals/companies focusing on CI right now
>>>> (whic

Re: [PROPOSAL] Minimum Viable CI Integration

2016-03-03 Thread Bharat Kumar

Hi will,

we have a solution to post the results to the community by email. We also have 
a github integration to fetch the Prs , run tests against them and post
the consolidated results by email and share the logs using dropbox.

We are facing some setup delays to get this up and running. I am sure we will 
be able to do this shortly.

I am also working on posting the results in PR as comments.

Thanks,
Bharat.



> On 04-Mar-2016, at 7:36 AM, Will Stevens  wrote:
>
> Last I knew Bharat did not have a solution for posting results back to the
> community. I could be wrong though, I don't really know how complete a
> solution Bharat has at this point.
>
> There are two other CI implementations in various states of completeness
> and I think it is important to have a common mechanism which will work to
> post results back to the community regardless of the CI implementation.
>
> "bubble" from SBP seems to be the most complete so far and has been used to
> test over 100 PRs, but I don't think the results have formally contributed
> back for the community to review.
>
> "trillian" is a project shapeblue is working on, but I am still getting
> details for when it is expected to be operational.
>
> This may not be a final solution, but I feel it is a good first step which
> will be easy for everyone to work with.
> On Mar 3, 2016 8:35 PM, "Gandikota Srinivas" 
> wrote:
>
>> Will,
>> I guess Bharat has something similar in working.
>>
>> Bharat,
>> Can you please elaborate your approach for sharing the CI results with
>> community ?
>>
>> Thanks,
>> Srinivas
>>
>> On Fri, Mar 4, 2016 at 6:28 AM, Will Stevens 
>> wrote:
>>
>>> Apparently attached files don't work when sending to the mailing list.
>>>
>>> Find the screenshot here:
>>>
>>>
>> https://objects-east.cloud.ca/v1/5ef827605f884961b94881e928e7a250/swill/Screen%20Shot%202016-03-03%20at%207.53.42%20PM.png
>>>
>>> On Thu, Mar 3, 2016 at 7:54 PM, Will Stevens 
>>> wrote:
>>>
 Hey Everyone,
 As I am sure most of you are aware, I have been focusing a lot on ways
>> to
 get CI integrated back into the community.

 Today I build a little POC to validate some ideas and get a feel for a
 potential approach for getting CI integrated into the Github pull
>> request
 workflow.

 There are multiple individuals/companies focusing on CI right now
>> (which
 is a good thing), but there has not really been much discussion (that I
>>> am
 aware of) for how these different CI runs will push back results to the
 community.  I want to make sure that nobody's work on this topic goes
>> to
 waste, so my goal is to provide a simple and consistent way for
>> everyone
>>> to
 push their results back to the community.

 Here is the basic idea (please give feedback):
 - A simple cross platform command line tool with zero dependencies will
>>> be
 provided (and open sourced) which will handle the submission of the CI
 results back to the community.  It is written in Golang and is
>> currently
 called 'notify_pr'.
 - At the end of a CI execution, the CI should automate the execution of
 this tool to handle updating the appropriate PR with the results.

 Configuration can be done via the command line or through an INI file.
 Here is an example of the configuration details.  The commit is the
 commit that the CI just executed against.

 token  = 
 owner  = apache
 repo   = cloudstack
 commit = c8443496d3fad78a4b1a48a0992ce545bde299e8

 summary_file = 
 full_detail_dir = >>> object store>
 full_detail_files = > object
 store>
 store_api = 
 store_endpoint = 
 store_identity = 
 store_secret = 

 I have not yet implemented the object storage endpoints, but I have
>> code
 to do it from a different project, so I just need to add it.  I will be
 able to host my CI output in a swift object store, but others may need
>> to
 use AWS S3.  Maybe we can get sponsorship for this storage.  We will
>> only
 keep the logs for a window of like a week or so on the object store so
>>> the
 storage usage will not be ever growing.

 Basically, the tool takes the details of the repository you are
>>> validating
 a Pull Request for and the commit you are building.  It also takes the
 files you would like to push to the pull request.  The summary file
>> will
>>> be
 shown as text in the pull request comment and the other files will be
 uploaded to an object store and will be publically available for a
>> period
 of time (probably about a week and then get cleaned up, details TBD).
>>> The
 files to be uploaded to object store could be either specified as
 individual files, OR a target directory and all the files in that
>>> directory
 will be recursively uploaded to the object

Re: PR validation using proposed CI.

2016-03-02 Thread Bharat Kumar

Hi Will,

I was not having  email access since few days, so could’ t  reply in time.

Yes, The idea is to run this on each PR.

Once i am done with this, it should be easy to replicate this locally if needed.

Thanks,
Bharat.

> On 27-Feb-2016, at 2:47 AM, Will Stevens <wstev...@cloudops.com> wrote:
> 
> I am also interested in setting this up locally.
> 
> Bharat, once you are happy with the state of this implementation, is the
> idea to set it up to run against every pull request branch submitted in
> github?   Or is there a different implementation in mind?
> 
> Thanks,
> 
> *Will STEVENS*
> Lead Developer
> 
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
> 
> On Fri, Feb 26, 2016 at 2:27 PM, Sebastien Goasguen <run...@gmail.com>
> wrote:
> 
>> 
>>> On Feb 26, 2016, at 7:58 PM, Bharat Kumar <bharat.ku...@citrix.com>
>> wrote:
>>> 
>>> Hi ilya,
>>> 
>>> We use cloudstack marvin framework to run tests. All of the tests were
>> written in the community, you can take a look at them
>>> in the source (tests/integration)
>>> 
>> 
>> Bharat, Ilya is aware of Marvin and all the tests, I think his question is
>> more about the description of the actual CI environment, the hardware, the
>> triggers etc…did you write a wiki page for it ?
>> 
>>> Thanks,
>>> Bharat.
>>> 
>>>> On 25-Feb-2016, at 11:45 PM, ilya <ilya.mailing.li...@gmail.com> wrote:
>>>> 
>>>> Bharat,
>>>> 
>>>> Hope all is well,
>>>> 
>>>> Perhaps you can explain the workflow and extensiveness of your tests.
>>>> 
>>>> Thanks
>>>> ilya
>>>> 
>>>> 
>>>> 
>>>> On 2/24/16 9:23 PM, Bharat Kumar wrote:
>>>>> Hi,
>>>>> 
>>>>> As all of you know from earlier discussions in the community, we have
>> been working on implementing a CI to test github PRs and publish results.
>> We have implemented this
>>>>> internally and will start testing PRs and publish the results by mail
>> to dev list.
>>>>> 
>>>>> At this point if a test fails, we do not know it for sure if it is due
>> to a bug in the test, in cloudstack or in the CI environment. We do not
>> have anything to cross reference. If a test is being
>>>>> reported as failure but is passing in your local environment,  please
>> let us know we will try and fix this in the CI environment.
>>>>> 
>>>>> We also do not have a way to share the management server logs and test
>> run logs. we are working on it.
>>>>> 
>>>>> Please let us know if you guys think any improvements are needed.
>>>>> 
>>>>> Thanks,
>>>>> Bharat.
>>>>> 
>>> 
>> 
>> 




DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Accelerite, a Persistent Systems business. It is intended only for 
the use of the individual or entity to which it is addressed. If you are not 
the intended recipient, you are not authorized to read, retain, copy, print, 
distribute or use this message. If you have received this communication in 
error, please notify the sender and delete all copies of this message. 
Accelerite, a Persistent Systems business does not accept any liability for 
virus infected mails.

Re: PR validation using proposed CI.

2016-02-26 Thread Bharat Kumar

Hi ilya,

We use cloudstack marvin framework to run tests. All of the tests were written 
in the community, you can take a look at them 
in the source (tests/integration)

Thanks,
Bharat.

> On 25-Feb-2016, at 11:45 PM, ilya <ilya.mailing.li...@gmail.com> wrote:
> 
> Bharat,
> 
> Hope all is well,
> 
> Perhaps you can explain the workflow and extensiveness of your tests.
> 
> Thanks
> ilya
> 
> 
> 
> On 2/24/16 9:23 PM, Bharat Kumar wrote:
>> Hi,
>> 
>> As all of you know from earlier discussions in the community, we have been 
>> working on implementing a CI to test github PRs and publish results. We have 
>> implemented this 
>> internally and will start testing PRs and publish the results by mail to dev 
>> list. 
>> 
>> At this point if a test fails, we do not know it for sure if it is due to a 
>> bug in the test, in cloudstack or in the CI environment. We do not have 
>> anything to cross reference. If a test is being
>> reported as failure but is passing in your local environment,  please let us 
>> know we will try and fix this in the CI environment.
>> 
>> We also do not have a way to share the management server logs and test run 
>> logs. we are working on it. 
>> 
>> Please let us know if you guys think any improvements are needed.
>> 
>> Thanks,
>> Bharat.
>>

Re: PR validation using proposed CI.

2016-02-25 Thread Bharat Kumar

Hi Daan,

we can do this, but before this i wanted make the test run logs available. Once 
I am done with this i will work on posting results on PR.

Thanks,
Bharat.

> On 25-Feb-2016, at 3:52 PM, Daan Hoogland <daan.hoogl...@gmail.com> wrote:
> 
> Bharat, will you publish the results in the PR?
> 
> On Thu, Feb 25, 2016 at 6:23 AM, Bharat Kumar <bharat.ku...@citrix.com>
> wrote:
> 
>> Hi,
>> 
>> As all of you know from earlier discussions in the community, we have been
>> working on implementing a CI to test github PRs and publish results. We
>> have implemented this
>> internally and will start testing PRs and publish the results by mail to
>> dev list.
>> 
>> At this point if a test fails, we do not know it for sure if it is due to
>> a bug in the test, in cloudstack or in the CI environment. We do not have
>> anything to cross reference. If a test is being
>> reported as failure but is passing in your local environment,  please let
>> us know we will try and fix this in the CI environment.
>> 
>> We also do not have a way to share the management server logs and test run
>> logs. we are working on it.
>> 
>> Please let us know if you guys think any improvements are needed.
>> 
>> Thanks,
>> Bharat.
>> 
>> 
> 
> 
> -- 
> Daan

Re: ACS CI BVT RUN: xenserver: Pr_number: 1417

2016-02-25 Thread Bharat Kumar

Hi seboga,

i replied inline.

Thanks,
Bharat.

> On 25-Feb-2016, at 2:09 PM, sebgoa <run...@gmail.com> wrote:
> 
> Bharat, that’s great, thanks for this
> 
> But…
> 
> Since this is the first results like this sent to the list, can you explain 
> what we are looking at.
> What hardware this is running on, what configuration of cloudstack…

Currently the tests are bing run using xenserver, the test setup consists of 3 
xen hosts across 2 clusters and the networking type is advanced.
I will add some of these details to the report as well.

> 
> And I very much would like for these results to be sent as comments to the PR 
> discussion,
> if it cannot be done we need to push ASF infra to make it happen

I totally agree with you, I started on it, but before that I am thinking we 
need to somehow make the test run logs available. once this is complete 
i will enhance this to post directly on the PR.

> 
> 
> 
>> On Feb 25, 2016, at 6:27 AM, Bharat Kumar <bharat.ku...@citrix.com> wrote:
>> 
>> 
>> On 24-Feb-2016, at 10:24 PM, 
>> jenkins-...@citrix.com<mailto:jenkins-...@citrix.com> wrote:
>> 
>> [http://jenkins-ccp.citrix.com/static/e59dfe28/images/32x32/red.gif]
>> BUILD FAILURE
>> Build Start Date:   Wed Feb 24 06:29:34 2016
>> Git Repo Url:   https://github.com/apache/cloudstack.git
>> Git Commit Id:  9d89229942436d98f4897d8030f95bb8702bf
>> Git Branch Info:1417
>> Hyper Visor Info:   xenserver
>> Build No:   11
>> Health Report
>> W   Description Score
>> [http://jenkins-ccp.citrix.com//static/d7293164/images/16x16/health-00to19.png]
>>  Build stability: All recent builds failed.  0
>> [http://jenkins-ccp.citrix.com//static/d7293164/images/16x16/health-80plus.png]
>>  Test Result: 22 tests failing out of a total of 142 tests.  84
>> 
>> <http://jenkins-ccp.citrix.com/job/report_generator_11_zone-xen_XenServer.xml/1//changes>
>> Changes
>> No Changes
>> 
>> <http://jenkins-ccp.citrix.com/job/report_generator_11_zone-xen_XenServer.xml/1//testReport>
>> Test Result
>> Package Failed  Passed  Skipped Total
>> < tt=""><>  2   0   0   2
>> :teardown
>> :setup
>> integration.smoke.test_affinity_groups  0   1   0   1
>> integration.smoke.test_affinity_groups_projects 0   1   0   1
>> integration.smoke.test_deploy_vgpu_enabled_vm   0   0   1   1
>> integration.smoke.test_deploy_vm_iso0   1   0   1
>> integration.smoke.test_deploy_vm_root_resize0   3   0   3
>> integration.smoke.test_deploy_vm_with_userdata  0   2   0   2
>> integration.smoke.test_deploy_vms_with_varied_deploymentplanners0
>>3   0   3
>> integration.smoke.test_disk_offerings   0   5   0   5
>> integration.smoke.test_global_settings  0   1   0   1
>> integration.smoke.test_guest_vlan_range 1   0   0   1
>> integration.smoke.test_guest_vlan_range.TestDedicateGuestVlanRange.test_dedicateGuestVlanRange
>> integration.smoke.test_internal_lb  0   4   0   4
>> integration.smoke.test_iso  2   4   1   7
>> integration.smoke.test_iso.TestISO.test_04_extract_Iso
>> integration.smoke.test_iso.TestISO.test_07_list_default_iso
>> integration.smoke.test_loadbalance  0   3   0   3
>> integration.smoke.test_multipleips_per_nic  0   1   0   1
>> integration.smoke.test_network  0   10  0   10
>> integration.smoke.test_network_acl  0   1   0   1
>> integration.smoke.test_nic  0   1   0   1
>> integration.smoke.test_nic_adapter_type 0   0   1   1
>> integration.smoke.test_non_contigiousvlan   0   1   0   1
>> integration.smoke.test_over_provisioning0   1   0   1
>> integration.smoke.test_password_server  1   0   0   1
>> integration.smoke.test_password_server.TestIsolatedNetworksPasswdServer.test_isolate_network_password_server
>> integration.smoke.test_portable_publicip0   2   0   2
>> integration.smoke.test_primary_storage  1   1   0   2
>> integration.smoke.test_primary_storage.TestPrimaryStorageServices.test_01_primary_storage_iscsi
>> integration.smoke.test_privategw_acl3   1   0   4
>> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
>> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_vpc_privategw_restart_vpc_cleanup
>> inte

Re: ACS CI BVT RUN: xenserver: Pr_number: 1417

2016-02-24 Thread Bharat Kumar


On 24-Feb-2016, at 10:24 PM, 
jenkins-...@citrix.com wrote:

[http://jenkins-ccp.citrix.com/static/e59dfe28/images/32x32/red.gif]BUILD 
FAILURE
Build Start Date:   Wed Feb 24 06:29:34 2016
Git Repo Url:   https://github.com/apache/cloudstack.git
Git Commit Id:  9d89229942436d98f4897d8030f95bb8702bf
Git Branch Info:1417
Hyper Visor Info:   xenserver
Build No:   11
Health Report
W   Description Score
[http://jenkins-ccp.citrix.com//static/d7293164/images/16x16/health-00to19.png] 
Build stability: All recent builds failed.  0
[http://jenkins-ccp.citrix.com//static/d7293164/images/16x16/health-80plus.png] 
Test Result: 22 tests failing out of a total of 142 tests.  84


Changes
No Changes


Test Result
Package Failed  Passed  Skipped Total
< tt=""><>  2   0   0   2
:teardown
:setup
integration.smoke.test_affinity_groups  0   1   0   1
integration.smoke.test_affinity_groups_projects 0   1   0   1
integration.smoke.test_deploy_vgpu_enabled_vm   0   0   1   1
integration.smoke.test_deploy_vm_iso0   1   0   1
integration.smoke.test_deploy_vm_root_resize0   3   0   3
integration.smoke.test_deploy_vm_with_userdata  0   2   0   2
integration.smoke.test_deploy_vms_with_varied_deploymentplanners0   
3   0   3
integration.smoke.test_disk_offerings   0   5   0   5
integration.smoke.test_global_settings  0   1   0   1
integration.smoke.test_guest_vlan_range 1   0   0   1
integration.smoke.test_guest_vlan_range.TestDedicateGuestVlanRange.test_dedicateGuestVlanRange
integration.smoke.test_internal_lb  0   4   0   4
integration.smoke.test_iso  2   4   1   7
integration.smoke.test_iso.TestISO.test_04_extract_Iso
integration.smoke.test_iso.TestISO.test_07_list_default_iso
integration.smoke.test_loadbalance  0   3   0   3
integration.smoke.test_multipleips_per_nic  0   1   0   1
integration.smoke.test_network  0   10  0   10
integration.smoke.test_network_acl  0   1   0   1
integration.smoke.test_nic  0   1   0   1
integration.smoke.test_nic_adapter_type 0   0   1   1
integration.smoke.test_non_contigiousvlan   0   1   0   1
integration.smoke.test_over_provisioning0   1   0   1
integration.smoke.test_password_server  1   0   0   1
integration.smoke.test_password_server.TestIsolatedNetworksPasswdServer.test_isolate_network_password_server
integration.smoke.test_portable_publicip0   2   0   2
integration.smoke.test_primary_storage  1   1   0   2
integration.smoke.test_primary_storage.TestPrimaryStorageServices.test_01_primary_storage_iscsi
integration.smoke.test_privategw_acl3   1   0   4
integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_vpc_privategw_restart_vpc_cleanup
integration.smoke.test_privategw_acl.TestPrivateGwACL.test_04_rvpc_privategw_static_routes
integration.smoke.test_public_ip_range  0   1   0   1
integration.smoke.test_pvlan0   1   0   1
integration.smoke.test_regions  0   1   0   1
integration.smoke.test_reset_vm_on_reboot   0   1   0   1
integration.smoke.test_resource_detail  0   1   0   1
integration.smoke.test_router_dhcphosts 1   0   0   1
integration.smoke.test_router_dhcphosts.TestRouterDHCPHosts.test_router_dhcphosts
integration.smoke.test_routers  0   9   0   9
integration.smoke.test_routers_iptables_default_policy  0   2   0   
2
integration.smoke.test_routers_network_ops  3   2   0   5
integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_01_isolate_network_FW_PF_default_routes_egress_true
integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
integration.smoke.test_scale_vm 0   1   0   1
integration.smoke.test_secondary_storage0   2   0   2
integration.smoke.test_service_offerings0   4   0   4
integration.smoke.test_snapshots0   1   0   1
integration.smoke.test_ssvm 0   10  0   10
integration.smoke.test_templates1   7   1   9
integration.smoke.test_templates.TestTemplates.test_04_extract_template
integration.smoke.test_vm_life_cycle1   10

PR validation using proposed CI.

2016-02-24 Thread Bharat Kumar

Hi,

As all of you know from earlier discussions in the community, we have been 
working on implementing a CI to test github PRs and publish results. We have 
implemented this 
internally and will start testing PRs and publish the results by mail to dev 
list. 

At this point if a test fails, we do not know it for sure if it is due to a bug 
in the test, in cloudstack or in the CI environment. We do not have anything to 
cross reference. If a test is being
reported as failure but is passing in your local environment,  please let us 
know we will try and fix this in the CI environment.

We also do not have a way to share the management server logs and test run 
logs. we are working on it. 

Please let us know if you guys think any improvements are needed.

Thanks,
Bharat.

Re: Important Pending Items

2016-02-11 Thread Bharat Kumar

at range is marked as 'inuse' in the database. When 
creating a test environment, Trillian looks in the database for the next 
available VLAN range, the next available public IP range and so on. The 
returned values are used to populate a Marvin cfg file which in turn will be 
used to both build the environment and when running the Marvin testing. When 
the virtualised infra is cleaned up, the database will be updated to reflect 
that the used ranges are available again.
This initiative has only recently been started, and as stated earlier we are 
currently figuring out the requirements (and quirks) of the individual pieces 
and looking for the most suitable wrapper to glue it all together.
Also I have found that Marvin requires a little work to make the output more 
meaningful/readable (especially in the case of errors and exceptions) and to 
make it a little more intelligent about the tests it can/can't run based on the 
chosen infrastructure components. I have also found unreachable or very slow 
ISO and template paths hardcoded into Marvin or individual tests.
We plan to enhance tests to address these issues and also reduce runtimes where 
possible.

<http://www.shapeblue.com/>
Paul Angus
VP Technology   ,   ShapeBlue

d:  +44 203 617 0528 | s: +44 203 603 
0540<tel:+44%20203%20617%200528%20|%20s:%20+44%20203%20603%200540> |  
m:  +44 7711 418784<tel:+44%207711%20418784>

e:  paul.an...@shapeblue.com | t: 
@cloudyangus<mailto:paul.an...@shapeblue.com%20|%20t:%20@cloudyangus>  |
  w:  www.shapeblue.com<http://www.shapeblue.com/>

a:  53 Chandos Place, Covent Garden London WC2N 4HS UK

Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services 
India LLP is a company incorporated in India and is operated under license from 
Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in 
Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd 
is a company registered by The Republic of South Africa and is traded under 
license from Shape Blue Ltd. ShapeBlue is a registered trademark.
This email and any attachments to it may be confidential and are intended 
solely for the use of the individual to whom it is addressed. Any views or 
opinions expressed are solely those of the author and do not necessarily 
represent those of Shape Blue Ltd or related companies. If you are not the 
intended recipient of this email, you must neither take any action based upon 
its contents, nor copy or show it to anyone. Please contact the sender if you 
believe you have received this email in error.

-Original Message-
From: Bharat Kumar [mailto:bharat.ku...@citrix.com]
Sent: Thursday, February 11, 2016 9:57 AM
To: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>
Subject: Re: Important Pending Items

Hi Sebastien,

As Raja said, we are actively working on it and we will share the code on 
github soon.

Thanks,
Bharat.

> On 11-Feb-2016, at 2:41 PM, Raja Pullela 
> <raja.pull...@citrix.com<mailto:raja.pull...@citrix.com>> wrote:
>
> Hi Sebastien,
>
> On item (3) - the BVTs have been running consistently (with same passrates) 
> and haven't had the time to automate the process of reporting to an external 
> site.
> Bharat and Sanjeev are working on CI - running BVTs/Regression, working on 
> getting it run automatically and consistently.
> Hope to see it ready soon.
>
> Best,
> Raja
> -Original Message-
> From: Sebastien Goasguen [mailto:run...@gmail.com]
> Sent: Wednesday, February 10, 2016 2:29 PM
> To: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>
> Subject: Important Pending Items
>
> Morning folks,
>
> We have several crucial pending items, that we need to resolve before moving 
> on:
>
> 1- We need an RM for master ( just saw some commits there that should be 
> reverted or merged properly in other branches).
>
> 2- We need to automate writing release notes, pushing/tagging new docs when 
> release come out and announcing releases on website. Currently neither 4.7 
> nor 4.8 have been announced.
>
> 3- CI is still almost inexistent
>
> -Sebastien

Find out more about ShapeBlue and our range of CloudStack related services:
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> | 
CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | 
CloudStack Software 
Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure 
Support<http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack 
Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>

Re: Important Pending Items

2016-02-11 Thread Bharat Kumar

Hi Sebastien,

As Raja said, we are actively working on it and we will share the code on 
github soon.

Thanks,
Bharat.


> On 11-Feb-2016, at 2:41 PM, Raja Pullela  wrote:
> 
> Hi Sebastien,
> 
> On item (3) - the BVTs have been running consistently (with same passrates) 
> and haven't had the time to automate the process of reporting to an external 
> site.
> Bharat and Sanjeev are working on CI - running BVTs/Regression, working on 
> getting it run automatically and consistently.  
> Hope to see it ready soon.
> 
> Best,
> Raja
> -Original Message-
> From: Sebastien Goasguen [mailto:run...@gmail.com] 
> Sent: Wednesday, February 10, 2016 2:29 PM
> To: dev@cloudstack.apache.org
> Subject: Important Pending Items
> 
> Morning folks,
> 
> We have several crucial pending items, that we need to resolve before moving 
> on:
> 
> 1- We need an RM for master ( just saw some commits there that should be 
> reverted or merged properly in other branches).
> 
> 2- We need to automate writing release notes, pushing/tagging new docs when 
> release come out and announcing releases on website. Currently neither 4.7 
> nor 4.8 have been announced.
> 
> 3- CI is still almost inexistent
> 
> -Sebastien

Re: [PROPOSAL] Freeze everything until we get CI

2016-02-01 Thread Bharat Kumar

Sure Remi,

Once I am done consolidating the scripts i will put them in a separate repo on 
github.

Thanks,
Bharat.


> On 30-Jan-2016, at 5:14 PM, Remi Bergsma <rberg...@schubergphilis.com> wrote:
> 
> Please put it in a separate repo. There’s way too much stuff in the 
> cloudstack repo already, IMHO and we should be splitting out, not adding more 
> :-)
> 
> Regards,
> Remi
> 
> 
> 
> 
> 
> On 29/01/16 08:22, "Erik Weber" <terbol...@gmail.com> wrote:
> 
>> I'd love to see this in the cloudstack repository.
>> Others might have an easier time getting access to hardware, and could use
>> it to help test releases/PR
>> 
>> 
>> Erik
>> 
>> Den fredag 29. januar 2016 skrev Bharat Kumar <bharat.ku...@citrix.com>
>> følgende:
>> 
>>> yes, we would be sharing it with the community and get this running in the
>>> ACS infra.
>>> Currently it can create a cloudstack test bed, runs tests and email the
>>> results.
>>> 
>>> Here are some details on how this works and what is needed to set this up.
>>> 
>>>  *   we use jenkins, cobbler, puppet and marvin to create cloudstack
>>> setup.
>>>  *   jenkins triggers the test runs, collects the test results and mails
>>> them.
>>>  *   cobbler is use to image the hosts and create Management server.
>>>  *   The management server is a VM and each time a test run is triggered
>>> we pull the latest code, build (dev setup) the MS and run it.
>>>  *   Need IPMI enabled servers to uses and Hosts in cloudstack setup.
>>> Cobbler installs the required OS on these hosts.
>>>  *   We use a XenServer to create management server VMs.
>>> 
>>> The resources required to set this up.
>>> 
>>>  *   We need two servers to host the VMs used in CI, one XenServer to
>>> host the Cloustack management servers and at least 3 IPMI enabled servers
>>> per cloudstack setup to run the BVTs.
>>>  *   some set of IPs (public and private IPs) and vlans.
>>> 
>>> Once we have the resources in ACS infra we can start setting this up. But
>>> some work needs to
>>> be done to integrate this with the github to test and post the results in
>>> the PRs instead of mailing them.
>>> 
>>> I think the best way to share it will be by implementing this in the ACS
>>> infra. Once we do this every one can pitch in, replicate and further
>>> contribute to this.
>>> 
>>> Meanwhile i will commit the scripts to set this up and keep this going.
>>> 
>>> Thanks,
>>> Bharat.
>>> 
>>> 
>>> On 28-Jan-2016, at 7:37 PM, Erik Weber <terbol...@gmail.com <javascript:;>
>>> <mailto:terbol...@gmail.com <javascript:;>>> wrote:
>>> 
>>> Why not share it as is, then the community could help improving this,
>>> rather than this being a single company effort?
>>> 
>>> --
>>> Erik
>>> 
>>> On Thu, Jan 28, 2016 at 1:49 PM, Bharat Kumar <bharat.ku...@citrix.com
>>> <javascript:;><mailto:bharat.ku...@citrix.com <javascript:;>>>
>>> wrote:
>>> 
>>> Hi All,
>>> 
>>> I agree that we need to have a CI to deal with the large volume of PRs.
>>> The current travis CI is not good enough as it runs only simulator tests.
>>> We identified this issue and came up with a effective CI for automating
>>> test runs for a each PR. This is already functional, with few github
>>> integration aspects pending. We are internally stabilizing it before
>>> sharing it.
>>> 
>>> We have been in touch with David Nalley ( CC’ed )  in making this
>>> operational for entire community using ACS infra.
>>> 
>>> 
>>> For your reference, here is the FS I have shared with the community
>>> earlier and also in this thread before, your feedback is welcome.
>>> (
>>> 
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Test+bed+orchestrator+and+test+runner+to+enable+continuos+integration
>>> ).
>>> 
>>> Thanks,
>>> Bharat.
>>> 
>>> 
>>> 
>>> 
>>> On 28-Jan-2016, at 4:26 PM, Rohit Yadav <rohit.ya...@shapeblue.com
>>> <javascript:;>>> rohit.ya...@shapeblue.com <javascript:;>>> wrote:
>>> 
>>> All,
>>> 
>>> I’m sorry to get to have the PRs merged without adhering to the strict
>>> testing requirements. Wh

Re: [PROPOSAL] Freeze everything until we get CI

2016-01-28 Thread Bharat Kumar

yes, we would be sharing it with the community and get this running in the ACS
infra.
Currently it can create a cloudstack test bed, runs tests and email the results.

Here are some details on how this works and what is needed to set this up.

* we use jenkins, cobbler, puppet and marvin to create cloudstack setup.
* jenkins triggers the test runs, collects the test results and mails them.
* cobbler is use to image the hosts and create Management server.
* The management server is a VM and each time a test run is triggered we
pull the latest code, build (dev setup) the MS and run it.
* Need IPMI enabled servers to uses and Hosts in cloudstack setup. Cobbler
installs the required OS on these hosts.
* We use a XenServer to create management server VMs.

The resources required to set this up.

* We need two servers to host the VMs used in CI, one XenServer to host the
Cloustack management servers and at least 3 IPMI enabled servers per cloudstack
setup to run the BVTs.
* some set of IPs (public and private IPs) and vlans.

Once we have the resources in ACS infra we can start setting this up. But some
work needs to
be done to integrate this with the github to test and post the results in the
PRs instead of mailing them.

I think the best way to share it will be by implementing this in the ACS infra.
Once we do this every one can pitch in, replicate and further contribute to
this.

Meanwhile i will commit the scripts to set this up and keep this going.

Thanks,
Bharat.

On 28-Jan-2016, at 7:37 PM, Erik Weber
<terbol...@gmail.com<mailto:terbol...@gmail.com>> wrote:

Why not share it as is, then the community could help improving this,
rather than this being a single company effort?

--
Erik

On Thu, Jan 28, 2016 at 1:49 PM, Bharat Kumar
<bharat.ku...@citrix.com<mailto:bharat.ku...@citrix.com>>
wrote:

Hi All,

I agree that we need to have a CI to deal with the large volume of PRs.
The current travis CI is not good enough as it runs only simulator tests.
We identified this issue and came up with a effective CI for automating
test runs for a each PR. This is already functional, with few github
integration aspects pending. We are internally stabilizing it before
sharing it.

We have been in touch with David Nalley ( CC’ed ) in making this
operational for entire community using ACS infra.

For your reference, here is the FS I have shared with the community
earlier and also in this thread before, your feedback is welcome.
(
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Test+bed+orchestrator+and+test+runner+to+enable+continuos+integration
).

Thanks,
Bharat.

On 28-Jan-2016, at 4:26 PM, Rohit Yadav <rohit.ya...@shapeblue.com> wrote:

All,

I’m sorry to get to have the PRs merged without adhering to the strict
testing requirements. While I think PRs were alright and it did not break
anything, the way it was merged made people uncomfortable that there is
some sort of haste in doing this fast which there is none.

I’ll not repeat this and hope you understand that I never had any hidden
agenda but to simply help people with some PRs.

Regards.

On 28-Jan-2016, at 11:36 AM, Sebastien Goasguen <run...@gmail.com> wrote:

Hi Folks,

My proposal to freeze until we get CI was indeed due to seeing Rohit’s
commit but was by no means a personal attack or judgment.

We have lots of PR pending (as mentioned before by Remi) and we need
people to help review and test.
So thanks to Rohit.

My only concerns were two fold:

1- We need to keep to adhere to our release principles:

https://cwiki.apache.org/confluence/display/CLOUDSTACK/Release+principles+for+Apache+CloudStack+4.6+and+up

Hence I replied to some PR asking if they needed to be merged directly in
master or not and wondered about the release branches.

With so many releases in flight it is not yet clear to me where we start
to apply a PR ?

2- We need to keep testing and post results of tests.

Currently it is manual and but there has been a strong guarantee in the
last releases that the PR where not going to break things.
While I agree that some PR are small and *should* not break things,
history has shown that even small unrelated things *somehow* can affect the
behavior of cloudstack.

So I proposed a freeze because:

- Remi stepped down as RM and we don’t have an official RM yet.
- The code has reached a solid state and we don’t want to do anything that
changes that
- We have a proposal for LTS on the floor
- We still don’t have CI.

So my standpoint is that we focused in the last 6 months on getting our
release principles right (pending LTS principles), code has stabilized and
we can release. Awesome.

Now is probably a good time to concentrate our limited resources on
figuring out automated CI.

- For instance as far as I know Travis is bonkers…(reports green but does
not do anything)
- And with citrix stepping out, we need to take control of the jenkins
slaves (some of which

Re: [PROPOSAL] Freeze everything until we get CI

2016-01-28 Thread Bharat Kumar

Hi All,

I agree that we need to have a CI to deal with the large volume of PRs. The
current travis CI is not good enough as it runs only simulator tests.
We identified this issue and came up with a effective CI for automating test
runs for a each PR. This is already functional, with few github integration
aspects pending. We are internally stabilizing it before sharing it.

We have been in touch with David Nalley ( CC’ed ) in making this operational
for entire community using ACS infra.

For your reference, here is the FS I have shared with the community earlier and
also in this thread before, your feedback is welcome.
(https://cwiki.apache.org/confluence/display/CLOUDSTACK/Test+bed+orchestrator+and+test+runner+to+enable+continuos+integration).

Thanks,
Bharat.

On 28-Jan-2016, at 4:26 PM, Rohit Yadav
> wrote:

All,

I’m sorry to get to have the PRs merged without adhering to the strict testing
requirements. While I think PRs were alright and it did not break anything, the
way it was merged made people uncomfortable that there is some sort of haste in
doing this fast which there is none.

I’ll not repeat this and hope you understand that I never had any hidden agenda
but to simply help people with some PRs.

Regards.

On 28-Jan-2016, at 11:36 AM, Sebastien Goasguen
> wrote:

Hi Folks,

My proposal to freeze until we get CI was indeed due to seeing Rohit’s commit
but was by no means a personal attack or judgment.

We have lots of PR pending (as mentioned before by Remi) and we need people to
help review and test.
So thanks to Rohit.

My only concerns were two fold:

1- We need to keep to adhere to our release principles:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Release+principles+for+Apache+CloudStack+4.6+and+up

Hence I replied to some PR asking if they needed to be merged directly in
master or not and wondered about the release branches.

With so many releases in flight it is not yet clear to me where we start to
apply a PR ?

2- We need to keep testing and post results of tests.

Currently it is manual and but there has been a strong guarantee in the last
releases that the PR where not going to break things.
While I agree that some PR are small and *should* not break things, history has
shown that even small unrelated things *somehow* can affect the behavior of
cloudstack.

So I proposed a freeze because:

So my standpoint is that we focused in the last 6 months on getting our release
principles right (pending LTS principles), code has stabilized and we can
release. Awesome.

Now is probably a good time to concentrate our limited resources on figuring
out automated CI.

- For instance as far as I know Travis is bonkers…(reports green but does not
do anything)
- And with citrix stepping out, we need to take control of the jenkins slaves
(some of which are on AWS and still paid by Citrix…)

My email while triggered by seeing Rohit’s commits, was not a judgement or
critic of his actions, so let’s not get into a personal argument here.

-Sebastien

On Jan 28, 2016, at 11:00 AM, Rohit Yadav wrote:

So, since some have directly (over IM etc) or indirectly have thrown
allegations on me since I merged most of the PRs.
Here is a list of those 12 PRs and answers on why they were merged on
case-by-case basis.
Please keep any further replies technical and to the specific PR, please point
out and revert if needed:

1. https://github.com/apache/cloudstack/pull/1288

Enough LGTMs, JS related change and fix tested with UI screenshot from Remi. I
personally looked at the diff and therefore then merged.

2. https://github.com/apache/cloudstack/pull/1274/files

Enough LGTMs, a simple NPE fix one-liner. I personally thought we can cheat
here and given Travis/Jenkins passed I merged it.

3. https://github.com/apache/cloudstack/pull/1261/files

Enough LGTMs, the diff only removed unused variable leading to change in the
constructor definition. Explicit integration tests are not necessary as it’d
simply dead-code removal and as the simulator smoke tests passed with
Travis/Jenkins passed so I merged it.

4. https://github.com/apache/cloudstack/pull/1048

Enough LGTMs. This change is related to a marvin test itself where it adds 2
new test methods — so no need to run regression integration test. The
integration test result of the marvin test was shared in the comment. PR merged
on this basis.

5. https://github.com/apache/cloudstack/pull/1044

Enough LGTMs and regression tests results (shared as attachments by Daan, in
case someone missed), so merged.

6. https://github.com/apache/cloudstack/pull/969

Enough LGTMs and

Re: [PROPOSAL] Freeze everything until we get CI

2016-01-27 Thread Bharat Kumar

Hi,

I agree with Remi on the hurdles he mentioned. It is difficult to integrate 3rd
party CI, If someone wants to implement their own CI, the link below gives one
way to do it.
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Test+bed+orchestrator+and+test+runner+to+enable+continuos+integration

Thanks,
Bharat.

On 28-Jan-2016, at 2:52 AM, Wido den Hollander
> wrote:

On 01/27/2016 10:02 PM, Remi Bergsma wrote:
Hi all,

We should keep the simple approach that was used until now: one LGTM based on
code review and one LGTM based on integration tests (that’s not the same as
2xLGTM).

If we care about master stability, every change has to be tested for
regression. Period. Things may look OK, but still break something else in an
unexpected way. IMHO, giving up on it would be a shame.

In a perfect world, CI would automate this for us and run it against every PR.
Current impediments I see for proper CI on every PR:
- there is no community hardware to run integration tests on (there was
supposed to be something)
- no support from Apache Infra: need Github write access (also to hook 3rd
party CI). It’s nowhere close.
- the community has very few people that are able to run the integration tests
on their own laptop/hardware and post the results on PRs

We tried to address points 1 and 2 and I propose to give up on them.

If all devs would stop developing for a couple of days and make sure they have
a box that can run KVM, then clone the code we use [1] it should be pretty
straight forward to run the integration tests. We choose to make it all virtual
and used nested-virtualisation, but that is optional. A simple Intel Nuc or
similar will do.

Yes, that's great code. I still have to master it, but the README should
get us there.

I still need to create one which also spins up a Ceph cluster. Probably
a good thing to do now ;)

Wido

People can then test the PRs that they find interesting and post results, after
which it can be merged. Distribute it. Share the load.

This is what Wilder, Miguel, Boris, Daan an I have been doing for months.
Simply running the tests. Hundreds of times. That’s why we can run a 100%
Mission Critical Cloud close to master branch at Schuberg Philis.

Regards,
Remi

[1] https://github.com/schubergphilis/MCT-shared/

On 27/01/16 21:33, "Sebastien Goasguen"
> wrote:

On Jan 27, 2016, at 9:25 PM, Wido den Hollander
> wrote:

On 01/27/2016 09:18 PM, Sebastien Goasguen wrote:
Folks,

How about we freeze our repo entirely until we get proper CI in place.

Seems to me all the hard work from Remi and co could be lost if we start
committing again.

Now Travis is not running, Jenkins fails all the time and nobody cares…

So how about we figure out CI now ? and not do anything else.

I think forces have to be combined to make this work.

Questions which come to mind: Who runs Jenkins? Do we need a additional
slave?

I haven't figured out the Integration tests completely personally.

In an ideal case, PR should trigger tests totally distributed on everyone’s own
hardware. Then tests would report back on the PR.
Only when all are green can we merge.

there is an issue with creating triggers in github on our own, but I think
that’s what we should aspire to.

for instance, how can pcextreme automate its testing and report back on each PR
?

Wido

-Sebastien

Re: User vm password gets update only the master VR.

2015-11-05 Thread Bharat Kumar

Hi Wilder,

If we think that it is ok to have the work around for now fix this later, we 
can bring the priority down to critical.

Thanks,
Bharat.

On 05-Nov-2015, at 2:49 pm, Wilder Rodrigues <wrodrig...@schubergphilis.com> 
wrote:

> Hi Bharat,
> 
> Please check if what you just mention will work. If it does, then the issue 
> is not a blocker.
> 
> Cheers,
> Wilder
> 
> 
>> On 05 Nov 2015, at 10:09, Bharat Kumar <bharat.ku...@citrix.com> wrote:
>> 
>> I have’t checked, but form the code it looks like it will work if we reissue 
>> the reset command after the slave becomes master.
>> 
>> —Bharat.
>> 
>> On 05-Nov-2015, at 12:59 pm, Erik Weber 
>> <terbol...@gmail.com<mailto:terbol...@gmail.com>> wrote:
>> 
>> On Thu, Nov 5, 2015 at 7:15 AM, Bharat Kumar 
>> <bharat.ku...@citrix.com<mailto:bharat.ku...@citrix.com>>
>> wrote:
>> 
>> Hi,
>> 
>> In my local setup i found this issue, The user VM password is not getting
>> saved in the backup router.
>> 
>> we send the save password command to both the VRs in a rvr enabled
>> network, But the password gets saved only in the master VR. This happens
>> because the password server is not running in the backup.
>> Because of this if someone resets the password of a VM and starts it when
>> the backup becomes master. Then the password of the user VM will not
>> change, because the save password command was not successful.
>> 
>> This breaks the password reset functionality, I have a raised a Blocker
>> issue CLOUDSTACK-9035<
>> https://issues.apache.org/jira/browse/CLOUDSTACK-9035> to track this.
>> 
>> 
>> 
>> Does it work if you re-issue the password reset call after the slave vr has
>> been promoted to master?
>> If it does, then atleast you have a workaround for the issue.
>> 
>> --
>> Erik
>> 
>

Re: User vm password gets update only the master VR.

2015-11-05 Thread Bharat Kumar

I have’t checked, but form the code it looks like it will work if we reissue 
the reset command after the slave becomes master.

—Bharat.

On 05-Nov-2015, at 12:59 pm, Erik Weber 
<terbol...@gmail.com<mailto:terbol...@gmail.com>> wrote:

On Thu, Nov 5, 2015 at 7:15 AM, Bharat Kumar 
<bharat.ku...@citrix.com<mailto:bharat.ku...@citrix.com>>
wrote:

Hi,

In my local setup i found this issue, The user VM password is not getting
saved in the backup router.

we send the save password command to both the VRs in a rvr enabled
network, But the password gets saved only in the master VR. This happens
because the password server is not running in the backup.
Because of this if someone resets the password of a VM and starts it when
the backup becomes master. Then the password of the user VM will not
change, because the save password command was not successful.

This breaks the password reset functionality, I have a raised a Blocker
issue CLOUDSTACK-9035<
https://issues.apache.org/jira/browse/CLOUDSTACK-9035> to track this.



Does it work if you re-issue the password reset call after the slave vr has
been promoted to master?
If it does, then atleast you have a workaround for the issue.

--
Erik

Re: User vm password gets update only the master VR.

2015-11-05 Thread Bharat Kumar

Hi,

Wilder has already reduced the priority to critical, I found it while manually 
testing some of my changes.
I am not aware of a test which checks this.

—Bharat.

On 05-Nov-2015, at 3:03 pm, Remi Bergsma <rberg...@schubergphilis.com> wrote:

> Hi Bharat,
> 
> First of all: thanks for reporting the issue!
> 
> Would it be possible to write a Marvin test for it, so we can test this 
> automatically and prevent regression? Or did you find this by an existing 
> Marvin test that I missed?
> 
> Please downgrade to critical. We should fix this, and we will, but it 
> shouldn’t block 4.6.0 IMHO. Talked to Rajani and she agrees.
> 
> Regards,
> Remi
> 
> 
> 
> On 05/11/15 10:29, "Bharat Kumar" <bharat.ku...@citrix.com> wrote:
> 
>> Hi Wilder,
>> 
>> If we think that it is ok to have the work around for now fix this later, we 
>> can bring the priority down to critical.
>> 
>> Thanks,
>> Bharat.
>> 
>> On 05-Nov-2015, at 2:49 pm, Wilder Rodrigues <wrodrig...@schubergphilis.com> 
>> wrote:
>> 
>>> Hi Bharat,
>>> 
>>> Please check if what you just mention will work. If it does, then the issue 
>>> is not a blocker.
>>> 
>>> Cheers,
>>> Wilder
>>> 
>>> 
>>>> On 05 Nov 2015, at 10:09, Bharat Kumar <bharat.ku...@citrix.com> wrote:
>>>> 
>>>> I have’t checked, but form the code it looks like it will work if we 
>>>> reissue the reset command after the slave becomes master.
>>>> 
>>>> —Bharat.
>>>> 
>>>> On 05-Nov-2015, at 12:59 pm, Erik Weber 
>>>> <terbol...@gmail.com<mailto:terbol...@gmail.com>> wrote:
>>>> 
>>>> On Thu, Nov 5, 2015 at 7:15 AM, Bharat Kumar 
>>>> <bharat.ku...@citrix.com<mailto:bharat.ku...@citrix.com>>
>>>> wrote:
>>>> 
>>>> Hi,
>>>> 
>>>> In my local setup i found this issue, The user VM password is not getting
>>>> saved in the backup router.
>>>> 
>>>> we send the save password command to both the VRs in a rvr enabled
>>>> network, But the password gets saved only in the master VR. This happens
>>>> because the password server is not running in the backup.
>>>> Because of this if someone resets the password of a VM and starts it when
>>>> the backup becomes master. Then the password of the user VM will not
>>>> change, because the save password command was not successful.
>>>> 
>>>> This breaks the password reset functionality, I have a raised a Blocker
>>>> issue CLOUDSTACK-9035<
>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-9035> to track this.
>>>> 
>>>> 
>>>> 
>>>> Does it work if you re-issue the password reset call after the slave vr has
>>>> been promoted to master?
>>>> If it does, then atleast you have a workaround for the issue.
>>>> 
>>>> --
>>>> Erik
>>>> 
>>> 
>>

User vm password gets update only the master VR.

2015-11-04 Thread Bharat Kumar

Hi,

In my local setup i found this issue, The user VM password is not getting saved 
in the backup router.

we send the save password command to both the VRs in a rvr enabled network, But 
the password gets saved only in the master VR. This happens because the 
password server is not running in the backup.
Because of this if someone resets the password of a VM and starts it when the 
backup becomes master. Then the password of the user VM will not change, 
because the save password command was not successful.

This breaks the password reset functionality, I have a raised a Blocker issue 
CLOUDSTACK-9035 to track 
this.


Thanks,
Bharat.

Re: Blameless post mortem

2015-09-28 Thread Bharat Kumar

se then why 
> care about PRs, code reviews at all. I feel we as a community need to improve 
> the documentation, at least start with the new changes. And if the changes 
> impact the core pieces of the system it becomes even more important. If you 
> look at PR 118 you will realise that the person who submitted the PR is 
> calming that he is not fully aware of the changes. What do you expect me to 
> make out of that?
> 
> 
> Even if someone volunteers to fix an issue, the person may not be sure if 
> something else will break given the nature of code and current status of 
> tests. The existing VR scripts (may not be as appealing as the new .py 
> scripts :) ) were stabilised over multiple releases and I am sure there must 
> have been lot of manual testing done as well. To do the same on the new 
> changes will at least take similar amount of testing.
> 
> Given that 4.6 release is nearing, one option would be to fix the pending 
> ones and release with a disclaimer that VR related functionality may have 
> regressed over last release (at this point no one knows what all other 
> potential issues may crop up).
> 
> Nope…we don’t. But it is a very general statement, we don’t know what can 
> come up due to this change or any other change…mostly because we have always 
> had bad tests coverage, unit or integration or what have you, and that 3 
> years in we still don’t have a dedicated CI.
> 
> Or the other options would be to indefinitely postpone 4.6, fix the 
> documentation, fix issues and once there is consensus on the stability aspect 
> go ahead with the release.
> 
> This in my view is traditional software engineering release concept. We don’t 
> need this in an open source project like this. We can release, it takes 5 
> minutes to cut a release. We should release and fix, release and fix, release 
> and fix.
> 
> The all idea about what we are doing is that we can release everytime we find 
> and fix a bug. There is no benefit about postponing things.
> 
> 
> Theoretically we can cut any number of releases at any frequency. But since 
> concerns have been raised on the quality aspect I feel it needs to be 
> addressed appropriately. Let users take an informed decision on whether to 
> use a particular release or not. I am not saying that postponing is the only 
> option.
> 
> 
> 
> 
> -Koushik
> 
> 
> On 28-Sep-2015, at 6:20 PM, Wilder Rodrigues 
> <wrodrig...@schubergphilis.com<mailto:wrodrig...@schubergphilis.com>>
> wrote:
> 
> I agree with the docs stuff, that I said 5 emails ago.
> 
> Once things are fixed, I will take the time to understand the code as a whole 
> and write the documentation: we will need ir for release purposes anyway.
> 
> Cheers,
> Wilder
> 
> 
> On 28 Sep 2015, at 14:47, Bharat Kumar 
> <bharat.ku...@citrix.com<mailto:bharat.ku...@citrix.com>> wrote:
> 
> Hi Wilder,
> 
> I am not talking about just the vpc networks. There are many other ares 
> getting effected because of this, some of them are vpn(not implemented) , rvr 
> in isolated networks etc.
> All i am saying is the design doc will help us understand the complete impact 
> of the changes and deal with them accordingly.
> 
> 
> Regards,
> Bharat.
> 
> 
> On 28-Sep-2015, at 6:02 pm, Wilder Rodrigues 
> <wrodrig...@schubergphilis.com<mailto:wrodrig...@schubergphilis.com>> wrote:
> 
> Only few tests…. 51 tests against a real environment.
> 
> At that time Nux also tested it and we tried to get Paul Angus, Geoff and 
> Rohit from Shape Blue to test it as well. Nux found a couple of issues that 
> were reported and fixed (see email below).
> 
> When I came back from holidays, 4 weeks ago, a PR containing 360 files 
> changed and almost 4000 lines, which was not even compiling, was merged onto 
> Master. We have less than a handful of people executing tests against PRs - 
> so few that I could even name who tests and who doesn’t. But hey, that’s a 
> blames email. I’m not trying to justify anything, but that handful of people, 
> who actually care about ACS, are getting quite fedup with this whole 
> discussion.
> 
> Cheers,
> Wilder
> 
> ===
> 
> On 20 Feb 2015, at 10:03, Nux! 
> <n...@li.nux.ro<mailto:n...@li.nux.ro><mailto:n...@li.nux.ro>> wrote:
> 
> Well, it looks like we were right to test it, found some problems.
> 
> 1 - the passwords for instances are not served properly, `wget --header 
> "DomU_Request: send_my_password" $router:8080` returns blank response. I am 
> not sure why this happens, though I tried to look around the router.
> 
> 2 - in addition to the above, in

Re: Blameless post mortem

2015-09-28 Thread Bharat Kumar

Hi Remi,

Thank you for the Blame less postmortem. 

I think there is a bigger problem here than just the review process and running 
tests. Even if we run the tests we cannot be sure that every thing will work as 
intended. The tests will only give some level of confidence. The tests may not 
cover all the use cases.

I think the problem here is that the way major changes to the code base are 
dealt with. For example,  VR refactoring was done without discussing the design 
implications and the amount of changes it would bring in. I could not find any 
design document. The vr refactor changed a lot of code and the way VR used to 
work and in my opinion it was incomplete-vpn, isolated networks, basic 
networks, iptable rules and rvr in isolated case etc were not implemented. Most 
of us are still in the process of understanding this. Even before reaching this 
state we had to spend a lot of time fixing issues mentioned in the thread 
[Blocker/Critical] VR related Issues.  

When a change of this magnitude is being made, we should call out all the 
changes and document them properly. This will help people to create better 
fixes. Currently when we attempt to fix the isolated vr case it is effecting 
the vpc and vice versa. for example pr 738 fixed it for vpc networks but broke 
it for isolated case. I believe it is not too late to at least start 
documenting the changes now.

Thanks,
Bharat.

On 28-Sep-2015, at 10:52 am, Sanjeev N  wrote:

> I have a concern here. Some of us are actively involved in reviewing the
> PRs related to marvin tests(Enhancing existing tests/Adding new tests). If
> we have to test a PR it requires an environment to be created with actual
> resources and this is going to take lot of time. Some of the tests can run
> on simulator but most of the tests require real hardware to test. PR
> submitter is already testing and submitting the test results along with the
> PR. So is it require to test these PRs by reviewers?
> 
> On Sat, Sep 26, 2015 at 1:49 PM, sebgoa  wrote:
> 
>> Remi, thanks for the detailed post-mortem, it's a good read and great
>> learning.
>> I hope everyone reads it.
>> 
>> The one thing to emphasize is that we now have a very visible way to get
>> code into master, we have folks investing time to provide review (great),
>> we need the submitters to make due diligence and answer all comments in the
>> reviews.
>> 
>> In another project i work on, nothing can be added to the code without
>> unit tests. I think we could go down the route of asking for new
>> integration tests and unit tests for anything. If not, the PR does not get
>> merged. But let's digest your post-mortem and we can discuss after 4.6.0.
>> 
>> I see that you reverted one commit that was not made by you, that's great.
>> 
>> Let's focus on the blockers now, everything else can wait.
>> 
>> The big bonus of doing what we are doing is that once 4.6.0 is out, we can
>> merge PRs again (assuming they are properly rebased and tested) and we can
>> release 4.6.1 really quickly after.
>> 
>> -sebastien
>> 
>> On Sep 25, 2015, at 9:51 PM, Remi Bergsma 
>> wrote:
>> 
>>> Hi all,
>>> 
>>> This mail is intended to be blameless. We need to learn something from
>> it. That's why I left out who exactly did what because it’s not relevant.
>> There are multiple examples but it's about the why. Let's learn from this
>> without blaming anyone.
>>> 
>>> We know we need automated testing. We have integration tests, but we are
>> unable to run all of them on any Pull Request we receive. If we would have
>> that in place, it'd be much easier to spot errors, regression and so on.
>> It'd also be more rewarding to write more tests.
>>> 
>>> Unfortunately we're not there yet. So, we need to do something else
>> instead until we get there. If we do nothing, we know we have many issues
>> because a master that breaks on a regular basis is the most frustrating
>> things. We said we'd use Pull Requests with at least two humans to review
>> and give their OK for a Pull Request. In the form of LGTM: Looks Good To
>> Me. Ok, so the LGTMs are there because we have no automated testing. Keep
>> that in mind. You are supposed to replace automated testing until it's
>> there.
>>> 
>>> Since we do this, master got a lot more stable. But every now and then
>> we still have issues. Let's look at how we do manual reviews. Again, this
>> is not to blame anyone. It's to open our eyes and make us realise what
>> we're doing and what results we get out of that.
>>> 
>>> 
>>> Example Pull Request #784:
>>> Title: CLOUDSTACK-8799 fixed the default routes
>>> 
>>> That's nice, it has a Jira id and a short description (as it should be).
>>> 
>>> The first person comes along and makes a comment:
>>> "There was also an issue with VPC VRs" ... "Have you seen this issue?
>> Does your change affects the VPC VR (single/redundant)?"
>>> 
>>> Actually a good question. Unfortunaly there

Re: Blameless post mortem

2015-09-28 Thread Bharat Kumar

Hi guys,

Anyway of all the things said and done I think we all agree that we need some 
documentation related to python changes.


Regards,
Bharat.

On 28-Sep-2015, at 5:46 pm, Wilder Rodrigues <wrodrig...@schubergphilis.com> 
wrote:

> Hi Bharat,
> 
> Perhaps you haven’t been away of not reading all the email that were sent to 
> the list in the past. Why am I saying that? just based on your sentence where 
> you said  “i wonder why was this ignored when merging the VR refactor code"
> 
> Is there any particular point you want to make that we are not aware of? Remi 
> and Sebastien already exposed their thoughts concerning the importance of 
> documentation and tests. So, why to continue this whole thing? Please, help 
> us all to understand it better.
> 
> Concerning the rVPC tests, you can find some reports here:
> 
> * http://markmail.org/message/khjw4y6m57pia5pm
> * http://markmail.org/message/4yzzew6fu2rrpz2p
> 
> And if you go to markmail you will find more.
> 
> -1 for you 1st suggestion - I will fix the 2 remaining VR issues and also add 
> (and execute) tests to cover them.
> 
> About documentation, once issues are fixed, I will go through the whole code 
> and write some design about it. I’m in the same situation as you and many 
> other: I don’t know the code 100%, but I would rather fix/document it than 
> waste energy with those discussions.
> 
> Sorry to say, but you are being captain obvious with all those emails about 
> documentation/bugs when we all already know about it and are working hard to 
> get it in a better shape.
> 
> Cheers,
> Wilder
> 
>> On 28 Sep 2015, at 13:54, Bharat Kumar <bharat.ku...@citrix.com> wrote:
>> 
>> Hi Remi,
>> 
>> Whatever  ever we think  we have discovered are all well known best 
>> practices while developing code in community. 
>> I agree that tests need to be run on a new PR,  but i wonder why was this 
>> ignored when merging the VR refactor code. Perhaps we will uncover some more 
>> issues if we investigate this. I believe 
>> one of the reasons for this is the complexity and incomplete nature of the 
>> vr refactor change and failing to identify the areas which got effected. If 
>> we had a good documentation i think we cloud have understood the areas that 
>> were getting 
>> impacted early on, areas like the vpn ,  iptables, isolated networks rvr   
>> etc  and run the relevant tests. The documentation will also help us focus 
>> on these areas while reviewing  and fixing subsequent issues. Currently no 
>> one knows the areas that got effected 
>> due to the vr refactor change, we are seeing issues all over the code.  I 
>> think this is a bigger problem than what we have discussed so far.
>> 
>> I think presently we should stop fixing all the vr refactoring  bugs until 
>> we come up with a  proper document describing the VR refactoring  changes.
>> 
>> I am not suggesting that we should revert the vr refactor code, I am willing 
>> to work on this and fix the issues,  I am only asking if we can get some 
>> documentation.
>> 
>> 
>> Regards,
>> Bharat.
>> 
>> On 28-Sep-2015, at 4:59 pm, Sebastien Goasguen <run...@gmail.com> wrote:
>> 
>>> 
>>>> On Sep 28, 2015, at 1:14 PM, Remi Bergsma <rberg...@schubergphilis.com> 
>>>> wrote:
>>>> 
>>>> Hi Bharat,
>>>> 
>>>> 
>>>> There is only one way to prove a feature works: with tests. That’s why I 
>>>> say actually _running_ the tests we have today on any new PR, is the most 
>>>> important thing. Having no documentation is a problem, I agree, but it is 
>>>> not more important IMHO. If we had the documentation, we still would have 
>>>> issues if nobody runs the tests and verifies pull requests. Documentation 
>>>> that is perfect does not automatically lead to perfect implementation. So 
>>>> we need tests to verify.
>>>> 
>>>> If we don’t agree that is also fine. We need to do both anyway and I think 
>>>> we do agree on that.
>>>> 
>>> 
>>> Also we need to move forward. We should have a live chat once 4.6 is out to 
>>> discuss all issues/problems and iron out the process.
>>> 
>>> But reverting the VR refactor is not going to happen. There was ample 
>>> discussions on the PR when it was submitted, there was time to review and 
>>> raise concerns at that time. It went through quite a few reviews, tests 
>>> etc…Maybe the documentation is not good, but the time to raise this concern 
>>>

Re: Blameless post mortem

2015-09-28 Thread Bharat Kumar

Hi Remi,

Whatever  ever we think  we have discovered are all well known best practices 
while developing code in community. 
I agree that tests need to be run on a new PR,  but i wonder why was this 
ignored when merging the VR refactor code. Perhaps we will uncover some more 
issues if we investigate this. I believe 
one of the reasons for this is the complexity and incomplete nature of the vr 
refactor change and failing to identify the areas which got effected. If we had 
a good documentation i think we cloud have understood the areas that were 
getting 
impacted early on, areas like the vpn ,  iptables, isolated networks rvr   etc  
and run the relevant tests. The documentation will also help us focus on these 
areas while reviewing  and fixing subsequent issues. Currently no one knows the 
areas that got effected 
due to the vr refactor change, we are seeing issues all over the code.  I think 
this is a bigger problem than what we have discussed so far.

I think presently we should stop fixing all the vr refactoring  bugs until we 
come up with a  proper document describing the VR refactoring  changes.

I am not suggesting that we should revert the vr refactor code, I am willing to 
work on this and fix the issues,  I am only asking if we can get some 
documentation.

Regards,
Bharat.

On 28-Sep-2015, at 4:59 pm, Sebastien Goasguen <run...@gmail.com> wrote:

> 
>> On Sep 28, 2015, at 1:14 PM, Remi Bergsma <rberg...@schubergphilis.com> 
>> wrote:
>> 
>> Hi Bharat,
>> 
>> 
>> There is only one way to prove a feature works: with tests. That’s why I say 
>> actually _running_ the tests we have today on any new PR, is the most 
>> important thing. Having no documentation is a problem, I agree, but it is 
>> not more important IMHO. If we had the documentation, we still would have 
>> issues if nobody runs the tests and verifies pull requests. Documentation 
>> that is perfect does not automatically lead to perfect implementation. So we 
>> need tests to verify.
>> 
>> If we don’t agree that is also fine. We need to do both anyway and I think 
>> we do agree on that.
>> 
> 
> Also we need to move forward. We should have a live chat once 4.6 is out to 
> discuss all issues/problems and iron out the process.
> 
> But reverting the VR refactor is not going to happen. There was ample 
> discussions on the PR when it was submitted, there was time to review and 
> raise concerns at that time. It went through quite a few reviews, tests 
> etc…Maybe the documentation is not good, but the time to raise this concern I 
> am afraid was six months ago. We can learn from it, but we are not going to 
> revert it, this would not go cleanly as David mentioned.
> 
> So let’s get back to blockers for 4.6, are there still VR related issues with 
> master ?
> 
> 
> 
> 
>> Regards,
>> Remi
>> 
>> 
>> 
>> 
>> 
>> 
>> On 28/09/15 12:15, "Bharat Kumar" <bharat.ku...@citrix.com> wrote:
>> 
>>> Hi Remi,
>>> 
>>> i do not agree with “There is no bigger problem”  part of your reply. so I 
>>> had to repeat myself to make it more clear, Not because i am not aware of 
>>> what this thread is supposed to do.
>>> 
>>> Regards,
>>> Bharat.
>>> 
>>> On 28-Sep-2015, at 2:51 pm, Remi Bergsma <rberg...@schubergphilis.com> 
>>> wrote:
>>> 
>>>> Hi Bharat,
>>>> 
>>>> I understand your frustrations but we already agreed on this so no need to 
>>>> repeat. This thread is supposed to list some improvements and learn from 
>>>> it. Your point has been taken so let’s move on.
>>>> 
>>>> We need documentation first, then do a change after which all tests should 
>>>> pass. Even better is we write (missing) tests before changing stuff so you 
>>>> know they pass before and after the fact. 
>>>> 
>>>> When doing reviews, feel free to ask for design documentation if you feel 
>>>> it is needed.
>>>> 
>>>> Regards, Remi
>>>> 
>>>> 
>>>> 
>>>> On 28/09/15 11:02, "Bharat Kumar" <bharat.ku...@citrix.com> wrote:
>>>> 
>>>>> Hi Remi,
>>>>> 
>>>>> I never intended to say that we should not run tests, but even before 
>>>>> tests we should have proper documentation. My concern was if a major 
>>>>> change is being introduced it should be properly documented. All the 
>>>>> issues which we are trying to fix are majorly due to VR refactor. If 
>>>>> there w

Re: Blameless post mortem

2015-09-28 Thread Bharat Kumar

Hi Sebastien,

You are confused, we are talking about  persistent VR config changes. below is 
the pr related to it.
https://github.com/apache/cloudstack/pull/118

If you look at it you will notice that there are more than 250 commits and only 
a few tests that were run.

Regards,
Bharat.

On 28-Sep-2015, at 5:24 pm, Bharat Kumar 
<bharat.ku...@citrix.com<mailto:bharat.ku...@citrix.com>> wrote:

Hi Remi,

Whatever  ever we think  we have discovered are all well known best practices 
while developing code in community.
I agree that tests need to be run on a new PR,  but i wonder why was this 
ignored when merging the VR refactor code. Perhaps we will uncover some more 
issues if we investigate this. I believe
one of the reasons for this is the complexity and incomplete nature of the vr 
refactor change and failing to identify the areas which got effected. If we had 
a good documentation i think we cloud have understood the areas that were 
getting
impacted early on, areas like the vpn ,  iptables, isolated networks rvr   etc  
and run the relevant tests. The documentation will also help us focus on these 
areas while reviewing  and fixing subsequent issues. Currently no one knows the 
areas that got effected
due to the vr refactor change, we are seeing issues all over the code.  I think 
this is a bigger problem than what we have discussed so far.

I think presently we should stop fixing all the vr refactoring  bugs until we 
come up with a  proper document describing the VR refactoring  changes.

I am not suggesting that we should revert the vr refactor code, I am willing to 
work on this and fix the issues,  I am only asking if we can get some 
documentation.


Regards,
Bharat.

On 28-Sep-2015, at 4:59 pm, Sebastien Goasguen 
<run...@gmail.com<mailto:run...@gmail.com>> wrote:


On Sep 28, 2015, at 1:14 PM, Remi Bergsma 
<rberg...@schubergphilis.com<mailto:rberg...@schubergphilis.com>> wrote:

Hi Bharat,


There is only one way to prove a feature works: with tests. That’s why I say 
actually _running_ the tests we have today on any new PR, is the most important 
thing. Having no documentation is a problem, I agree, but it is not more 
important IMHO. If we had the documentation, we still would have issues if 
nobody runs the tests and verifies pull requests. Documentation that is perfect 
does not automatically lead to perfect implementation. So we need tests to 
verify.

If we don’t agree that is also fine. We need to do both anyway and I think we 
do agree on that.


Also we need to move forward. We should have a live chat once 4.6 is out to 
discuss all issues/problems and iron out the process.

But reverting the VR refactor is not going to happen. There was ample 
discussions on the PR when it was submitted, there was time to review and raise 
concerns at that time. It went through quite a few reviews, tests etc…Maybe the 
documentation is not good, but the time to raise this concern I am afraid was 
six months ago. We can learn from it, but we are not going to revert it, this 
would not go cleanly as David mentioned.

So let’s get back to blockers for 4.6, are there still VR related issues with 
master ?




Regards,
Remi






On 28/09/15 12:15, "Bharat Kumar" 
<bharat.ku...@citrix.com<mailto:bharat.ku...@citrix.com>> wrote:

Hi Remi,

i do not agree with “There is no bigger problem”  part of your reply. so I had 
to repeat myself to make it more clear, Not because i am not aware of what this 
thread is supposed to do.

Regards,
Bharat.

On 28-Sep-2015, at 2:51 pm, Remi Bergsma 
<rberg...@schubergphilis.com<mailto:rberg...@schubergphilis.com>> wrote:

Hi Bharat,

I understand your frustrations but we already agreed on this so no need to 
repeat. This thread is supposed to list some improvements and learn from it. 
Your point has been taken so let’s move on.

We need documentation first, then do a change after which all tests should 
pass. Even better is we write (missing) tests before changing stuff so you know 
they pass before and after the fact.

When doing reviews, feel free to ask for design documentation if you feel it is 
needed.

Regards, Remi



On 28/09/15 11:02, "Bharat Kumar" 
<bharat.ku...@citrix.com<mailto:bharat.ku...@citrix.com>> wrote:

Hi Remi,

I never intended to say that we should not run tests, but even before tests we 
should have proper documentation. My concern was if a major change is being 
introduced it should be properly documented. All the issues which we are trying 
to fix are majorly due to VR refactor. If there was a proper documentation for 
this we could
have fixed this in a better way.  Even to add tests we need to understand how a 
particular thing works and what data dose it expect. I think while fixing the 
python based code changes this is where most of the people are facing issues. A 
proper documentation will help in understanding these in a better way.

Thanks,

Re: Blameless post mortem

2015-09-28 Thread Bharat Kumar

Dude,

There was nothing friendly about the postmortem you did, It was only partial, 
we should do a complete postmortem and then draw conclusions.

I think the post-mortems like this are of no use if we do not do them 
completely.

Regards,
Bharat.

On 28-Sep-2015, at 5:39 pm, Remi Bergsma <rberg...@schubergphilis.com> wrote:

> Dude, this is the final friendly email about his. All points have been made 
> in previous mails. This has nothing to do with ‘blameless’ and ‘learning’ 
> anymore.
> 
> Read Seb’s mail. We will move on now.
> 
> 
> Regards, Remi
> 
> 
> 
> On 28/09/15 13:54, "Bharat Kumar" <bharat.ku...@citrix.com> wrote:
> 
>> Hi Remi,
>> 
>> Whatever  ever we think  we have discovered are all well known best 
>> practices while developing code in community. 
>> I agree that tests need to be run on a new PR,  but i wonder why was this 
>> ignored when merging the VR refactor code. Perhaps we will uncover some more 
>> issues if we investigate this. I believe 
>> one of the reasons for this is the complexity and incomplete nature of the 
>> vr refactor change and failing to identify the areas which got effected. If 
>> we had a good documentation i think we cloud have understood the areas that 
>> were getting 
>> impacted early on, areas like the vpn ,  iptables, isolated networks rvr   
>> etc  and run the relevant tests. The documentation will also help us focus 
>> on these areas while reviewing  and fixing subsequent issues. Currently no 
>> one knows the areas that got effected 
>> due to the vr refactor change, we are seeing issues all over the code.  I 
>> think this is a bigger problem than what we have discussed so far.
>> 
>> I think presently we should stop fixing all the vr refactoring  bugs until 
>> we come up with a  proper document describing the VR refactoring  changes.
>> 
>> I am not suggesting that we should revert the vr refactor code, I am willing 
>> to work on this and fix the issues,  I am only asking if we can get some 
>> documentation.
>> 
>> 
>> Regards,
>> Bharat.
>> 
>> On 28-Sep-2015, at 4:59 pm, Sebastien Goasguen <run...@gmail.com> wrote:
>> 
>>> 
>>>> On Sep 28, 2015, at 1:14 PM, Remi Bergsma <rberg...@schubergphilis.com> 
>>>> wrote:
>>>> 
>>>> Hi Bharat,
>>>> 
>>>> 
>>>> There is only one way to prove a feature works: with tests. That’s why I 
>>>> say actually _running_ the tests we have today on any new PR, is the most 
>>>> important thing. Having no documentation is a problem, I agree, but it is 
>>>> not more important IMHO. If we had the documentation, we still would have 
>>>> issues if nobody runs the tests and verifies pull requests. Documentation 
>>>> that is perfect does not automatically lead to perfect implementation. So 
>>>> we need tests to verify.
>>>> 
>>>> If we don’t agree that is also fine. We need to do both anyway and I think 
>>>> we do agree on that.
>>>> 
>>> 
>>> Also we need to move forward. We should have a live chat once 4.6 is out to 
>>> discuss all issues/problems and iron out the process.
>>> 
>>> But reverting the VR refactor is not going to happen. There was ample 
>>> discussions on the PR when it was submitted, there was time to review and 
>>> raise concerns at that time. It went through quite a few reviews, tests 
>>> etc…Maybe the documentation is not good, but the time to raise this concern 
>>> I am afraid was six months ago. We can learn from it, but we are not going 
>>> to revert it, this would not go cleanly as David mentioned.
>>> 
>>> So let’s get back to blockers for 4.6, are there still VR related issues 
>>> with master ?
>>> 
>>> 
>>> 
>>> 
>>>> Regards,
>>>> Remi
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On 28/09/15 12:15, "Bharat Kumar" <bharat.ku...@citrix.com> wrote:
>>>> 
>>>>> Hi Remi,
>>>>> 
>>>>> i do not agree with “There is no bigger problem”  part of your reply. so 
>>>>> I had to repeat myself to make it more clear, Not because i am not aware 
>>>>> of what this thread is supposed to do.
>>>>> 
>>>>> Regards,
>>>>> Bharat.
>>>>> 
>>>>> On 28-Sep-2015, at 2:51 pm, Remi Bergsma <rberg...@schuberg

Re: Blameless post mortem

2015-09-28 Thread Bharat Kumar

Hi Remi,

I never intended to say that we should not run tests, but even before tests we 
should have proper documentation. My concern was if a major change is being 
introduced it should be properly documented. All the issues which we are trying 
to fix are majorly due to VR refactor. If there was a proper documentation for 
this we could
have fixed this in a better way.  Even to add tests we need to understand how a 
particular thing works and what data dose it expect. I think while fixing the 
python based code changes this is where most of the people are facing issues. A 
proper documentation will help in understanding these in a better way.

Thanks,
Bharat.

On 28-Sep-2015, at 1:57 pm, Remi Bergsma <rberg...@schubergphilis.com> wrote:

> Hi Bharat,
> 
> There is no bigger problem. We should always run the tests and if we find a 
> case that isn’t currently covered by the tests we should simply add tests for 
> it. There’s no way we’ll get a stable master without them. The fact that they 
> may not cover everything, is no reason to not rely on them. If a feature is 
> not important enough to write a test for, then the feature is probably not 
> important anyway. And if it is, then add a test :-)
> 
> I do agree on the design documentation requirement for any (major?) change. I 
> found some design documentations on the subject you mention, but it should 
> have been more detailed. 
> 
> Regards,
> Remi
> 
> 
> 
> 
> 
> 
> On 28/09/15 09:58, "Bharat Kumar" <bharat.ku...@citrix.com> wrote:
> 
>> Hi Remi,
>> 
>> Thank you for the Blame less postmortem. 
>> 
>> I think there is a bigger problem here than just the review process and 
>> running tests. Even if we run the tests we cannot be sure that every thing 
>> will work as intended. The tests will only give some level of confidence. 
>> The tests may not cover all the use cases.
>> 
>> I think the problem here is that the way major changes to the code base are 
>> dealt with. For example,  VR refactoring was done without discussing the 
>> design implications and the amount of changes it would bring in. I could not 
>> find any design document. The vr refactor changed a lot of code and the way 
>> VR used to work and in my opinion it was incomplete-vpn, isolated networks, 
>> basic networks, iptable rules and rvr in isolated case etc were not 
>> implemented. Most of us are still in the process of understanding this. Even 
>> before reaching this state we had to spend a lot of time fixing issues 
>> mentioned in the thread [Blocker/Critical] VR related Issues.  
>> 
>> When a change of this magnitude is being made, we should call out all the 
>> changes and document them properly. This will help people to create better 
>> fixes. Currently when we attempt to fix the isolated vr case it is effecting 
>> the vpc and vice versa. for example pr 738 fixed it for vpc networks but 
>> broke it for isolated case. I believe it is not too late to at least start 
>> documenting the changes now.
>> 
>> Thanks,
>> Bharat.
>> 
>> On 28-Sep-2015, at 10:52 am, Sanjeev N <sanj...@apache.org> wrote:
>> 
>>> I have a concern here. Some of us are actively involved in reviewing the
>>> PRs related to marvin tests(Enhancing existing tests/Adding new tests). If
>>> we have to test a PR it requires an environment to be created with actual
>>> resources and this is going to take lot of time. Some of the tests can run
>>> on simulator but most of the tests require real hardware to test. PR
>>> submitter is already testing and submitting the test results along with the
>>> PR. So is it require to test these PRs by reviewers?
>>> 
>>> On Sat, Sep 26, 2015 at 1:49 PM, sebgoa <run...@gmail.com> wrote:
>>> 
>>>> Remi, thanks for the detailed post-mortem, it's a good read and great
>>>> learning.
>>>> I hope everyone reads it.
>>>> 
>>>> The one thing to emphasize is that we now have a very visible way to get
>>>> code into master, we have folks investing time to provide review (great),
>>>> we need the submitters to make due diligence and answer all comments in the
>>>> reviews.
>>>> 
>>>> In another project i work on, nothing can be added to the code without
>>>> unit tests. I think we could go down the route of asking for new
>>>> integration tests and unit tests for anything. If not, the PR does not get
>>>> merged. But let's digest your post-mortem and we can discuss after 4.6.0.
>>>> 
>>>> I see that you reverted one commit that

Re: Blameless post mortem

2015-09-28 Thread Bharat Kumar

Hi Remi,

 i do not agree with “There is no bigger problem”  part of your reply. so I had 
to repeat myself to make it more clear, Not because i am not aware of what this 
thread is supposed to do.
 
Regards,
Bharat.

On 28-Sep-2015, at 2:51 pm, Remi Bergsma <rberg...@schubergphilis.com> wrote:

> Hi Bharat,
> 
> I understand your frustrations but we already agreed on this so no need to 
> repeat. This thread is supposed to list some improvements and learn from it. 
> Your point has been taken so let’s move on.
> 
> We need documentation first, then do a change after which all tests should 
> pass. Even better is we write (missing) tests before changing stuff so you 
> know they pass before and after the fact. 
> 
> When doing reviews, feel free to ask for design documentation if you feel it 
> is needed.
> 
> Regards, Remi
> 
> 
> 
> On 28/09/15 11:02, "Bharat Kumar" <bharat.ku...@citrix.com> wrote:
> 
>> Hi Remi,
>> 
>> I never intended to say that we should not run tests, but even before tests 
>> we should have proper documentation. My concern was if a major change is 
>> being introduced it should be properly documented. All the issues which we 
>> are trying to fix are majorly due to VR refactor. If there was a proper 
>> documentation for this we could
>> have fixed this in a better way.  Even to add tests we need to understand 
>> how a particular thing works and what data dose it expect. I think while 
>> fixing the python based code changes this is where most of the people are 
>> facing issues. A proper documentation will help in understanding these in a 
>> better way.
>> 
>> Thanks,
>> Bharat.
>> 
>> On 28-Sep-2015, at 1:57 pm, Remi Bergsma <rberg...@schubergphilis.com> wrote:
>> 
>>> Hi Bharat,
>>> 
>>> There is no bigger problem. We should always run the tests and if we find a 
>>> case that isn’t currently covered by the tests we should simply add tests 
>>> for it. There’s no way we’ll get a stable master without them. The fact 
>>> that they may not cover everything, is no reason to not rely on them. If a 
>>> feature is not important enough to write a test for, then the feature is 
>>> probably not important anyway. And if it is, then add a test :-)
>>> 
>>> I do agree on the design documentation requirement for any (major?) change. 
>>> I found some design documentations on the subject you mention, but it 
>>> should have been more detailed. 
>>> 
>>> Regards,
>>> Remi
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On 28/09/15 09:58, "Bharat Kumar" <bharat.ku...@citrix.com> wrote:
>>> 
>>>> Hi Remi,
>>>> 
>>>> Thank you for the Blame less postmortem. 
>>>> 
>>>> I think there is a bigger problem here than just the review process and 
>>>> running tests. Even if we run the tests we cannot be sure that every thing 
>>>> will work as intended. The tests will only give some level of confidence. 
>>>> The tests may not cover all the use cases.
>>>> 
>>>> I think the problem here is that the way major changes to the code base 
>>>> are dealt with. For example,  VR refactoring was done without discussing 
>>>> the design implications and the amount of changes it would bring in. I 
>>>> could not find any design document. The vr refactor changed a lot of code 
>>>> and the way VR used to work and in my opinion it was incomplete-vpn, 
>>>> isolated networks, basic networks, iptable rules and rvr in isolated case 
>>>> etc were not implemented. Most of us are still in the process of 
>>>> understanding this. Even before reaching this state we had to spend a lot 
>>>> of time fixing issues mentioned in the thread [Blocker/Critical] VR 
>>>> related Issues.  
>>>> 
>>>> When a change of this magnitude is being made, we should call out all the 
>>>> changes and document them properly. This will help people to create better 
>>>> fixes. Currently when we attempt to fix the isolated vr case it is 
>>>> effecting the vpc and vice versa. for example pr 738 fixed it for vpc 
>>>> networks but broke it for isolated case. I believe it is not too late to 
>>>> at least start documenting the changes now.
>>>> 
>>>> Thanks,
>>>> Bharat.
>>>> 
>>>> On 28-Sep-2015, at 10:52 am, Sanjeev N <sanj...@apache.org> wrote:
>>>> 
>>

Re: Blameless post mortem

2015-09-28 Thread Bharat Kumar

Hi Wilder,

I am not talking about just the vpc networks. There are many other ares getting 
effected because of this, some of them are vpn(not implemented) , rvr in 
isolated networks etc. 
All i am saying is the design doc will help us understand the complete impact 
of the changes and deal with them accordingly.


Regards,
Bharat.


On 28-Sep-2015, at 6:02 pm, Wilder Rodrigues <wrodrig...@schubergphilis.com> 
wrote:

> Only few tests…. 51 tests against a real environment.
> 
> At that time Nux also tested it and we tried to get Paul Angus, Geoff and 
> Rohit from Shape Blue to test it as well. Nux found a couple of issues that 
> were reported and fixed (see email below).
> 
> When I came back from holidays, 4 weeks ago, a PR containing 360 files 
> changed and almost 4000 lines, which was not even compiling, was merged onto 
> Master. We have less than a handful of people executing tests against PRs - 
> so few that I could even name who tests and who doesn’t. But hey, that’s a 
> blames email. I’m not trying to justify anything, but that handful of people, 
> who actually care about ACS, are getting quite fedup with this whole 
> discussion.
> 
> Cheers,
> Wilder
> 
> ===
> 
> On 20 Feb 2015, at 10:03, Nux! <n...@li.nux.ro<mailto:n...@li.nux.ro>> wrote:
> 
> Well, it looks like we were right to test it, found some problems.
> 
> 1 - the passwords for instances are not served properly, `wget --header 
> "DomU_Request: send_my_password" $router:8080` returns blank response. I am 
> not sure why this happens, though I tried to look around the router.
> 
> 2 - in addition to the above, in a redundant VPC the SNAT does not work. 
> >From an instance I can ping the router(s), but not any further than that. 
> SNAT works fine in a normal/non-vpc network.
> I'll try to look more into it later today.
> 
> Have a nice day :)
> 
> Lucian
> 
> --
> Sent from the Delta quadrant using Borg technology!
> 
> Nux!
> www.nux.ro<http://www.nux.ro>
> 
> 
> 
> 
> On 28 Sep 2015, at 14:13, Bharat Kumar 
> <bharat.ku...@citrix.com<mailto:bharat.ku...@citrix.com>> wrote:
> 
> Hi Sebastien,
> 
> You are confused, we are talking about  persistent VR config changes. below 
> is the pr related to it.
> https://github.com/apache/cloudstack/pull/118
> 
> If you look at it you will notice that there are more than 250 commits and 
> only a few tests that were run.
> 
> Regards,
> Bharat.
> 
> On 28-Sep-2015, at 5:24 pm, Bharat Kumar 
> <bharat.ku...@citrix.com<mailto:bharat.ku...@citrix.com><mailto:bharat.ku...@citrix.com>>
>  wrote:
> 
> Hi Remi,
> 
> Whatever  ever we think  we have discovered are all well known best practices 
> while developing code in community.
> I agree that tests need to be run on a new PR,  but i wonder why was this 
> ignored when merging the VR refactor code. Perhaps we will uncover some more 
> issues if we investigate this. I believe
> one of the reasons for this is the complexity and incomplete nature of the vr 
> refactor change and failing to identify the areas which got effected. If we 
> had a good documentation i think we cloud have understood the areas that were 
> getting
> impacted early on, areas like the vpn ,  iptables, isolated networks rvr   
> etc  and run the relevant tests. The documentation will also help us focus on 
> these areas while reviewing  and fixing subsequent issues. Currently no one 
> knows the areas that got effected
> due to the vr refactor change, we are seeing issues all over the code.  I 
> think this is a bigger problem than what we have discussed so far.
> 
> I think presently we should stop fixing all the vr refactoring  bugs until we 
> come up with a  proper document describing the VR refactoring  changes.
> 
> I am not suggesting that we should revert the vr refactor code, I am willing 
> to work on this and fix the issues,  I am only asking if we can get some 
> documentation.
> 
> 
> Regards,
> Bharat.
> 
> On 28-Sep-2015, at 4:59 pm, Sebastien Goasguen 
> <run...@gmail.com<mailto:run...@gmail.com><mailto:run...@gmail.com>> wrote:
> 
> 
> On Sep 28, 2015, at 1:14 PM, Remi Bergsma 
> <rberg...@schubergphilis.com<mailto:rberg...@schubergphilis.com><mailto:rberg...@schubergphilis.com>>
>  wrote:
> 
> Hi Bharat,
> 
> 
> There is only one way to prove a feature works: with tests. That’s why I say 
> actually _running_ the tests we have today on any new PR, is the most 
> important thing. Having no documentation is a problem, I agree, but it is not 
> more important IMHO. If we had the

Re: Proposal to get to 100% passrate on BVTs

2015-09-22 Thread Bharat Kumar

Hi,

I am sharing the link to the FS, it is not completely solid yet, but i intend 
to add more details as we progress further. Please go through this and let me 
know if someone wants to
help me on this. As i said earlier i want some help in generating, publishing 
and archiving the test results.

The link to the FS 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Test+bed+orchestrator+and+test+runner+to+enable+continuos+integration.


Thanks,
Bharat.


On 22-Sep-2015, at 3:35 pm, Raja Pullela 
<raja.pull...@citrix.com<mailto:raja.pull...@citrix.com>> wrote:

Bharat,  can you please repost if there is any specific ask ?

Thanks,
Raja
-Original Message-
From: Bharat Kumar [mailto:bharat.ku...@citrix.com]
Sent: Tuesday, September 22, 2015 10:03 AM
To: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>
Subject: Re: Proposal to get to 100% passrate on BVTs

Hi,

I am already working towards this. We can already replicate the test infra 
using puppet. But i need some help in generating reports, sharing the test 
results and archiving them.
particularly archiving them since we need some space which is accessible by 
everyone.

I have also made a proposal to do this in the acs, It did not get much 
attention though.

Thanks,
Bharat.


On 22-Sep-2015, at 9:43 am, Sanjeev N 
<sanj...@apache.org<mailto:sanj...@apache.org>> wrote:

I would be happy to lend my hand in generalizing the BVT Infra.

-Sanjeev

On Mon, Sep 21, 2015 at 4:34 PM, Raja Pullela
<raja.pull...@citrix.com<mailto:raja.pull...@citrix.com>>
wrote:

Rohit, not sure about the efforts required to do this.  Can certainly
work with you or someone to do this.  Please let me know,

best,
Raja
From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com]
Sent: Monday, September 21, 2015 2:53 PM
To: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>
Subject: Re: Proposal to get to 100% passrate on BVTs

Great initiative Raja!

Any effort in generalizing the BVT infra so the infra can be
reproduced using Puppet/Ansible?
Regards,
Rohit Yadav
Software Architect, ShapeBlue


[cid:9DD97B41-04C5-45F0-92A7-951F3E962F7A]


M. +91 88 262 30892 | 
rohit.ya...@shapeblue.com<mailto:rohit.ya...@shapeblue.com>mailto:rohit.ya...@shapeblue.com>>
Blog: bhaisaab.org<http://bhaisaab.org><http://bhaisaab.org> | Twitter: 
@_bhaisaab





Find out more about ShapeBlue and our range of CloudStack related
services

IaaS Cloud Design & Build<
http://shapeblue.com/iaas-cloud-design-and-build/>
CSForge - rapid IaaS deployment
framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Software Engineering<
http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<
http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training Courses<
http://shapeblue.com/cloudstack-training/>

This email and any attachments to it may be confidential and are
intended solely for the use of the individual to whom it is
addressed. Any views or opinions expressed are solely those of the
author and do not necessarily represent those of Shape Blue Ltd or
related companies. If you are not the intended recipient of this
email, you must neither take any action based upon its contents, nor
copy or show it to anyone. Please contact the sender if you believe
you have received this email in error. Shape Blue Ltd is a company
incorporated in England & Wales. ShapeBlue Services India LLP is a
company incorporated in India and is operated under license from
Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company
incorporated in Brasil and is operated under license from Shape Blue
Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South 
Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a 
registered trademark.

Re: Proposal to get to 100% passrate on BVTs

2015-09-21 Thread Bharat Kumar

Hi, 

I am already working towards this. We can already replicate the test infra 
using puppet. But i need some help in generating reports, sharing the test 
results and archiving them.
particularly archiving them since we need some space which is accessible by 
everyone.

I have also made a proposal to do this in the acs, It did not get much 
attention though. 

Thanks,
Bharat.


On 22-Sep-2015, at 9:43 am, Sanjeev N  wrote:

> I would be happy to lend my hand in generalizing the BVT Infra.
> 
> -Sanjeev
> 
> On Mon, Sep 21, 2015 at 4:34 PM, Raja Pullela 
> wrote:
> 
>> Rohit, not sure about the efforts required to do this.  Can certainly work
>> with you or someone to do this.  Please let me know,
>> 
>> best,
>> Raja
>> From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com]
>> Sent: Monday, September 21, 2015 2:53 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: Proposal to get to 100% passrate on BVTs
>> 
>> Great initiative Raja!
>> 
>> Any effort in generalizing the BVT infra so the infra can be reproduced
>> using Puppet/Ansible?
>> Regards,
>> Rohit Yadav
>> Software Architect, ShapeBlue
>> 
>> 
>> [cid:9DD97B41-04C5-45F0-92A7-951F3E962F7A]
>> 
>> 
>> M. +91 88 262 30892 | rohit.ya...@shapeblue.com> rohit.ya...@shapeblue.com>
>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>> 
>> 
>> 
>> 
>> 
>> Find out more about ShapeBlue and our range of CloudStack related services
>> 
>> IaaS Cloud Design & Build<
>> http://shapeblue.com/iaas-cloud-design-and-build/>
>> CSForge - rapid IaaS deployment framework
>> CloudStack Consulting
>> CloudStack Software Engineering<
>> http://shapeblue.com/cloudstack-software-engineering/>
>> CloudStack Infrastructure Support<
>> http://shapeblue.com/cloudstack-infrastructure-support/>
>> CloudStack Bootcamp Training Courses<
>> http://shapeblue.com/cloudstack-training/>
>> 
>> This email and any attachments to it may be confidential and are intended
>> solely for the use of the individual to whom it is addressed. Any views or
>> opinions expressed are solely those of the author and do not necessarily
>> represent those of Shape Blue Ltd or related companies. If you are not the
>> intended recipient of this email, you must neither take any action based
>> upon its contents, nor copy or show it to anyone. Please contact the sender
>> if you believe you have received this email in error. Shape Blue Ltd is a
>> company incorporated in England & Wales. ShapeBlue Services India LLP is a
>> company incorporated in India and is operated under license from Shape Blue
>> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
>> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is
>> a company registered by The Republic of South Africa and is traded under
>> license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>>

Re: [Blocker/Critical] VR related Issues

2015-09-03 Thread Bharat Kumar

Hi,

found few more issues related to rvr in isolated networks.
There seems to be a problem with the keepalived config and setting up of 
default routes when rvr changes states.

created bugs for these issues.
CLOUDSTACK-8798<https://issues.apache.org/jira/browse/CLOUDSTACK-8798>
CLOUDSTACK-8799<https://issues.apache.org/jira/browse/CLOUDSTACK-8799>

Thanks,
Bharat.

On 12-Aug-2015, at 10:52 am, Bharat Kumar 
<bharat.ku...@citrix.com<mailto:bharat.ku...@citrix.com>> wrote:

Hi,

looks like  there is  one more issue. Conntrackd fails to start in case of rvr 
enabled isolated networks.
created a bug to track this. 
https://issues.apache.org/jira/browse/CLOUDSTACK-8725

Thanks,
Bharat.

On 11-Aug-2015, at 3:03 pm, Kishan Kavala 
<kishan.kav...@citrix.com<mailto:kishan.kav...@citrix.com>> wrote:

Below VR related issues currently open. Most of these issues did not exist in 
4.5.x and are related to VR refactor (persistent VR).

Blocker
https://issues.apache.org/jira/browse/CLOUDSTACK-8690 - Remote Access VPN not 
working

Critical
https://issues.apache.org/jira/browse/CLOUDSTACK-8688 - Default policy for 
INPUT and FORWARD chain is ACCEPT in VR filter table (Wilder is working on this)
https://issues.apache.org/jira/browse/CLOUDSTACK-8681 - CS does not honor the 
default deny egress policy in isolated network
https://issues.apache.org/jira/browse/CLOUDSTACK-8710 - site2site vpn iptables 
rules are not configured on VR
https://issues.apache.org/jira/browse/CLOUDSTACK-8685 - Default route is not 
configured on VPC VR
https://issues.apache.org/jira/browse/CLOUDSTACK-8694  - Monitor service cron 
job not visible

Re: [Proposal] Minimise VR downtime during network update.

2015-08-11 Thread Bharat Kumar

Hi Daan,

I am not aware of  the work done for rVPC networks.  I am still in the POC 
stage, will update the FS or scope it down depending on 
the changes related to rvpc, If there are too many changes may be i will 
restrict this to non vpc networks.

I will create a apache ticket fro this work. 

Thanks,
Bharat.

On 11-Aug-2015, at 9:16 pm, Daan Hoogland daan.hoogl...@gmail.com wrote:

 Can you make a ticket on apache instead of referring an internal ticket?
 
 Can you describe how this work relates to the rvpc refactor work done for
 4.6?
 
 
 
 On Tue, Aug 11, 2015 at 5:41 PM, Bharat Kumar bharat.ku...@citrix.com
 wrote:
 
 Hi,
 
 Currently we use redundant virtual router enabled networks to prevent
 network outage even if one of the virtual router fails. This is achieved by
 using redundant virtual routers along with VRRP protocol. This has
 increased the network robustness. However  updating a network with new
 network offering involves some downtime due to Vr restarts. In case of
 redundant VR based networks we can sequence the network update operations
 to update the backup first and then update the master while master's
 operations are being handled by backup, this way i think we can minimise
 the downtime.
 
 link to FS.
 
 https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61310469
 
 Please review the FS and give suggestion or comments.
 
 Thanks,
 Bharat.
 
 
 
 
 -- 
 Daan

[Proposal] Minimise VR downtime during network update.

2015-08-11 Thread Bharat Kumar

Hi,

Currently we use redundant virtual router enabled networks to prevent network 
outage even if one of the virtual router fails. This is achieved by using 
redundant virtual routers along with VRRP protocol. This has increased the 
network robustness. However  updating a network with new network offering 
involves some downtime due to Vr restarts. In case of redundant VR based 
networks we can sequence the network update operations to update the backup 
first and then update the master while master's operations are being handled by 
backup, this way i think we can minimise the downtime.

link to FS.

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61310469

Please review the FS and give suggestion or comments.

Thanks,
Bharat.

Re: [Blocker/Critical] VR related Issues

2015-08-11 Thread Bharat Kumar

Hi,

looks like  there is  one more issue. Conntrackd fails to start in case of rvr 
enabled isolated networks.
created a bug to track this. 
https://issues.apache.org/jira/browse/CLOUDSTACK-8725

Thanks,
Bharat.

On 11-Aug-2015, at 3:03 pm, Kishan Kavala 
kishan.kav...@citrix.commailto:kishan.kav...@citrix.com wrote:

Below VR related issues currently open. Most of these issues did not exist in 
4.5.x and are related to VR refactor (persistent VR).

Blocker
https://issues.apache.org/jira/browse/CLOUDSTACK-8690 - Remote Access VPN not 
working

Critical
https://issues.apache.org/jira/browse/CLOUDSTACK-8688 - Default policy for 
INPUT and FORWARD chain is ACCEPT in VR filter table (Wilder is working on this)
https://issues.apache.org/jira/browse/CLOUDSTACK-8681 - CS does not honor the 
default deny egress policy in isolated network
https://issues.apache.org/jira/browse/CLOUDSTACK-8710 - site2site vpn iptables 
rules are not configured on VR
https://issues.apache.org/jira/browse/CLOUDSTACK-8685 - Default route is not 
configured on VPC VR
https://issues.apache.org/jira/browse/CLOUDSTACK-8694  - Monitor service cron 
job not visible

Re: [Proposal] Continuos Integration Testing on ACS master.

2015-07-23 Thread Bharat Kumar

Hi Kishan,

I am thinking we will use Citrix hardware until we find something in the acs 
lab. We will publish the results to acs by mails. However until we find 
something publicly accessible people can only see the list of tests that failed,
for providing more info we will need some place to host the logs and other 
related info of the test runs.

long term plan is to move everything into acs infra.

Thanks,
Bharat.

On 23-Jul-2015, at 4:19 pm, Kishan Kavala kishan.kav...@citrix.com wrote:

 Bharat,
 I hope this will go a long way in improving product quality. How are you 
 going to handle the handle the hardware requirement? What is the plan to 
 publish the results to ML?
 
 -Original Message-
 From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] 
 Sent: 23 July 2015 03:46 PM
 To: dev
 Subject: Re: [Proposal] Continuos Integration Testing on ACS master.
 
 Good work Bharat, Keep in mind that many people work on this in many ways and 
 many attempts have failed due to nothing but lack of consensus. I have only 
 glanced but will take my time with it soon.
 
 On Thu, Jul 23, 2015 at 10:50 AM, Bharat Kumar bharat.ku...@citrix.com 
 wrote:
 Hi All,
 
 Currently we do sanity checks on master using Travis, But travis runs 
 only simulator tests (AFAIK). I think it will be better if we can run both 
 BVTs and regression tests on real hardware. I am woking on implementing this.
 
 Functional spec and design details.
 https://cwiki.apache.org/confluence/display/CLOUDSTACK/Test+bed+orches
 trator+and+test+runner+to+enable+continuos+integration
 
 Please read the FS and give your comments or suggestions.
 
 Thanks,
 Bharat.
 
 
 
 --
 Daan

[Proposal] Continuos Integration Testing on ACS master.

2015-07-23 Thread Bharat Kumar

Hi All,

Currently we do sanity checks on master using Travis, But travis runs only 
simulator tests (AFAIK). I think it will be better if
we can run both BVTs and regression tests on real hardware. I am woking on 
implementing this.

Functional spec and design details.
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Test+bed+orchestrator+and+test+runner+to+enable+continuos+integration

Please read the FS and give your comments or suggestions.

Thanks,
Bharat.

Re: file /var/log/cloudstack/agent from install of cloudstack-agent-4.5.1-1.el7.centos.x86_64 conflicts with file from package cloudstack-management-4.5.1-1.el7.centos.x86_64

2015-07-15 Thread Bharat Kumar

Hi,

I totally agree with Wido, This should be fixed in the ACS repo for every one 
to use, Instead of sharing this using non ACS private builds.

Thanks,
Bharat.

On 15-Jul-2015, at 2:41 pm, Rohit Yadav 
rohit.ya...@shapeblue.commailto:rohit.ya...@shapeblue.com wrote:


On 14-Jul-2015, at 9:23 pm, Wido den Hollander 
w...@widodh.nlmailto:w...@widodh.nl wrote:

Although that will help the end-user, we shouldn't want that. This
should be fixed in the ACS source releases as well rather then in
packages.

It seems to be fixed in the 4.5 branch, but 4.5.2 hasn't been released
which has this fix.

This was identified just around 4.5.1 release; when building the 4.5.1 rpm 
packages I fixed the spec file (so mgmt rpm won’t create the agent log 
directory) therefore the packages on 
packages.shapeblue.comhttp://packages.shapeblue.com/ I shared won’t cause 
this conflict.

Regards,
Rohit Yadav
Software Architect, ShapeBlue




M. +91 88 262 30892 | 
rohit.ya...@shapeblue.commailto:rohit.ya...@shapeblue.com
Blog: bhaisaab.orghttp://bhaisaab.org/ | Twitter: @_bhaisaab




Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design  Buildhttp://shapeblue.com/iaas-cloud-design-and-build//
CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/
CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/
CloudStack Software 
Engineeringhttp://shapeblue.com/cloudstack-software-engineering/
CloudStack Infrastructure 
Supporthttp://shapeblue.com/cloudstack-infrastructure-support/
CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/

This email and any attachments to it may be confidential and are intended 
solely for the use of the individual to whom it is addressed. Any views or 
opinions expressed are solely those of the author and do not necessarily 
represent those of Shape Blue Ltd or related companies. If you are not the 
intended recipient of this email, you must neither take any action based upon 
its contents, nor copy or show it to anyone. Please contact the sender if you 
believe you have received this email in error. Shape Blue Ltd is a company 
incorporated in England  Wales. ShapeBlue Services India LLP is a company 
incorporated in India and is operated under license from Shape Blue Ltd. Shape 
Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is 
operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company 
registered by The Republic of South Africa and is traded under license from 
Shape Blue Ltd. ShapeBlue is a registered trademark.

Re: 4.5 clustered mgmt server

2015-04-15 Thread Bharat Kumar

Hi,

I guss you are correct. we need to add the newly created channel ch1 to the map 
after the handshake is complete.

_peers.put(peerName, ch) should be changed to _peers.put(peerName, ch1); 

and also return ch1 instead of ch.

Thanks,
Bharat.


On 15-Apr-2015, at 12:21 pm, Marcus shadow...@gmail.com wrote:

 Is anyone running 4.5 mgmt server in cluster setup? I've run into an
 issue where the channel between the two servers is null, and I think
 I've tracked it down to coverity fixes from
 78bfaa79cf3eaa170ef9422bf8fb1825c0cecfc1.
 
 The method returns 'ch', which is null in the error. The coverity fix
 changes ch for ch1 and seems to initialize ch1 and then drop it on the
 floor. If someone is familiar with this code I'd appreciate a review
 of it.
 
 diff --git 
 a/engine/orchestration/src/com/cloud/agent/manager/ClusteredAgentManagerImpl.java
 b/engine/orchestration/src/com/cloud/agent/manager/ClusteredAgentManagerImpl.java
 index 600dca2..e38489b 100755
 --- 
 a/engine/orchestration/src/com/cloud/agent/manager/ClusteredAgentManagerImpl.java
 +++ 
 b/engine/orchestration/src/com/cloud/agent/manager/ClusteredAgentManagerImpl.java
 @@ -498,18 +498,17 @@ public class ClusteredAgentManagerImpl extends
 AgentManagerImpl implements Clust
 } catch (UnknownHostException e) {
 throw new CloudRuntimeException(Unable to resolve  + 
 ip);
 }
 -try {
 -ch = SocketChannel.open(new
 InetSocketAddress(addr, Port.value()));
 -ch.configureBlocking(true); // make sure we are
 working at blocking mode
 -ch.socket().setKeepAlive(true);
 -ch.socket().setSoTimeout(60 * 1000);
 +try (SocketChannel ch1 = SocketChannel.open(new
 InetSocketAddress(addr, Port.value()));){
 +ch1.configureBlocking(true); // make sure we are
 working at blocking mode
 +ch1.socket().setKeepAlive(true);
 +ch1.socket().setSoTimeout(60 * 1000);
 try {
 SSLContext sslContext = Link.initSSLContext(true);
 sslEngine = sslContext.createSSLEngine(ip,
 Port.value());
 sslEngine.setUseClientMode(true);
 
 sslEngine.setEnabledProtocols(SSLUtils.getSupportedProtocols(sslEngine.getEnabledProtocols()));
 
 -Link.doHandshake(ch, sslEngine, true);
 +Link.doHandshake(ch1, sslEngine, true);
 s_logger.info(SSL: Handshake done);
 } catch (Exception e) {
 throw new IOException(SSL: Fail to init SSL!  + e);

Regarding Cloudstack shutdown operations.

2015-04-13 Thread Bharat Kumar

Hi, 

Recently i noticed that when we stop cloudstack we do not mark the state of the 
management server as down in the ms_host table.

On further investigation I came to know that the stop methods of the beans are 
not getting executed during shutdown. I could not see any logs related to 
cloudstack shutdown. 
I cloud find some code in the CloudStackExtendedLifeCycle.java which calls the 
stop methods of the beans, But this code doesn’t seem to run at shutdown.

I am not sure about how to fix this.  One way would be to call the stop methods 
of bean instances in the registry explicitly.

Any other ideas ?.

Thanks,
Bharat.

Review Request 29364: [Dashboard] Primary Storage Used(type tag with value 2) related tag is not showing in listCapacity api response.

2014-12-23 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29364/
---

Review request for cloudstack and Kishan Kavala.


Repository: cloudstack-git


Description
---

https://issues.citrite.net/browse/CS-30086
[Dashboard] Primary Storage Used(type tag with value 2) related tag is not 
showing in listCapacity api response.


Diffs
-

  server/src/com/cloud/server/ManagementServerImpl.java deb2424 

Diff: https://reviews.apache.org/r/29364/diff/


Testing
---

tested on 4.5


Thanks,

bharat kumar

Re: Review Request 29365: [Overcommit]Static Min/Max memory is not getting configured according to overcommit factor for vms on xen 6.2

2014-12-23 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29365/
---

(Updated Dec. 23, 2014, 1:05 p.m.)


Review request for cloudstack and Kishan Kavala.


Repository: cloudstack-git


Description
---

[Overcommit]Static Min/Max memory is not getting configured according to 
overcommit factor for vms on xen 6.2


Diffs
-

  
plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resource/CitrixResourceBase.java
 a090b1106ac4fb1acf3680991e990931d951fb65 

Diff: https://reviews.apache.org/r/29365/diff/


Testing
---


Thanks,

bharat kumar

Re: Review Request 29364: [Dashboard] Primary Storage Used(type tag with value 2) related tag is not showing in listCapacity api response.

2014-12-23 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29364/
---

(Updated Dec. 23, 2014, 1:05 p.m.)


Review request for cloudstack and Kishan Kavala.


Repository: cloudstack-git


Description (updated)
---


[Dashboard] Primary Storage Used(type tag with value 2) related tag is not 
showing in listCapacity api response.


Diffs
-

  server/src/com/cloud/server/ManagementServerImpl.java deb2424 

Diff: https://reviews.apache.org/r/29364/diff/


Testing
---

tested on 4.5


Thanks,

bharat kumar

Review Request 28217: when a template is deleted and then copied over again , it is still marked as Removed in template_zone_ref table

2014-11-18 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28217/
---

Review request for cloudstack and Kishan Kavala.


Repository: cloudstack-git


Description
---

https://issues.apache.org/jira/browse/CLOUDSTACK-7939


Diffs
-

  engine/schema/src/com/cloud/storage/dao/VMTemplateDaoImpl.java 401a4a2 
  engine/schema/src/com/cloud/storage/dao/VMTemplateZoneDao.java 67f7c3f 
  engine/schema/src/com/cloud/storage/dao/VMTemplateZoneDaoImpl.java 8cadf61 

Diff: https://reviews.apache.org/r/28217/diff/


Testing
---


Thanks,

bharat kumar

Review Request 28085: Fixed some coverity issues.

2014-11-14 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28085/
---

Review request for cloudstack and Kishan Kavala.


Repository: cloudstack-git


Description
---

Fixed some coverity issues.

Added the prepared statement close statements.
https://issues.apache.org/jira/browse/CLOUDSTACK-7710


Diffs
-

  engine/schema/src/com/cloud/upgrade/dao/Upgrade410to420.java 
dab2b46a5759819951a0bf6b7d9f06df78b11e88 

Diff: https://reviews.apache.org/r/28085/diff/


Testing
---


Thanks,

bharat kumar

Review Request 27868: InvalidParameter Exception with stacktrace in MS log wile executing scale vm.

2014-11-11 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27868/
---

Review request for cloudstack and Kishan Kavala.


Repository: cloudstack-git


Description
---

InvalidParameter Exception with stacktrace in  MS log
https://issues.apache.org/jira/browse/CLOUDSTACK-7348


Diffs
-

  server/src/com/cloud/vm/UserVmManagerImpl.java 
77ace7a5f6c7f6e423a7e97bbf4c4bb5fdcb730a 

Diff: https://reviews.apache.org/r/27868/diff/


Testing
---


Thanks,

bharat kumar

Re: Review Request 27868: InvalidParameter Exception with stacktrace in MS log wile executing scale vm.

2014-11-11 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27868/
---

(Updated Nov. 11, 2014, 2:19 p.m.)


Review request for cloudstack and Kishan Kavala.


Repository: cloudstack-git


Description (updated)
---

InvalidParameter Exception with stacktrace in  MS log
https://issues.apache.org/jira/browse/CLOUDSTACK-7348

we do not throw the exceptions now. Added some info in the log, to say what 
happend to the command.


Diffs
-

  server/src/com/cloud/vm/UserVmManagerImpl.java 
77ace7a5f6c7f6e423a7e97bbf4c4bb5fdcb730a 

Diff: https://reviews.apache.org/r/27868/diff/


Testing
---


Thanks,

bharat kumar

Re: Review Request 27868: InvalidParameter Exception with stacktrace in MS log wile executing scale vm.

2014-11-11 Thread bharat kumar



 On Nov. 12, 2014, 4:02 a.m., Koushik Das wrote:
  server/src/com/cloud/vm/UserVmManagerImpl.java, line 1348
  https://reviews.apache.org/r/27868/diff/1/?file=757725#file757725line1348
 
  What operation, which hypervisor and VM? Also the info log is not 
  required as the exception message will be present in logs.

I can see the exception messages at debug log level, Think it will be good if 
we say something at the info level as well. 

updated the log with relavent info.


- bharat


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27868/#review60943
---


On Nov. 11, 2014, 2:19 p.m., bharat kumar wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27868/
 ---
 
 (Updated Nov. 11, 2014, 2:19 p.m.)
 
 
 Review request for cloudstack and Kishan Kavala.
 
 
 Repository: cloudstack-git
 
 
 Description
 ---
 
 InvalidParameter Exception with stacktrace in  MS log
 https://issues.apache.org/jira/browse/CLOUDSTACK-7348
 
 we do not throw the exceptions now. Added some info in the log, to say what 
 happend to the command.
 
 
 Diffs
 -
 
   server/src/com/cloud/vm/UserVmManagerImpl.java 
 77ace7a5f6c7f6e423a7e97bbf4c4bb5fdcb730a 
 
 Diff: https://reviews.apache.org/r/27868/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 bharat kumar

Re: Review Request 27868: InvalidParameter Exception with stacktrace in MS log wile executing scale vm.

2014-11-11 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27868/
---

(Updated Nov. 12, 2014, 6:26 a.m.)


Review request for cloudstack and Kishan Kavala.


Repository: cloudstack-git


Description
---

InvalidParameter Exception with stacktrace in  MS log
https://issues.apache.org/jira/browse/CLOUDSTACK-7348

we do not throw the exceptions now. Added some info in the log, to say what 
happend to the command.


Diffs (updated)
-

  server/src/com/cloud/vm/UserVmManagerImpl.java 
77ace7a5f6c7f6e423a7e97bbf4c4bb5fdcb730a 

Diff: https://reviews.apache.org/r/27868/diff/


Testing
---


Thanks,

bharat kumar

Review Request 27021: Reservations for VMware VMs remain after dynamic scaling

2014-10-22 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27021/
---

Review request for cloudstack and Kishan Kavala.


Bugs: CLOUDSTACK-7763
https://issues.apache.org/jira/browse/CLOUDSTACK-7763


Repository: cloudstack-git


Description
---

Reservations for VMware VMs remain after dynamic scaling
https://issues.apache.org/jira/browse/CLOUDSTACK-7763


Diffs
-

  
plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
 085b6bbed4918be3155fd44c8bed785983526e11 

Diff: https://reviews.apache.org/r/27021/diff/


Testing
---


Thanks,

bharat kumar

Review Request 26969: During VM deployment and while choosing CUSTOM disk offering for data disk, the size of the data disk is not considered for checking resource limit of primary storage.

2014-10-21 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26969/
---

Review request for cloudstack and Kishan Kavala.


Bugs: CLOUDSTACK-7760
https://issues.apache.org/jira/browse/CLOUDSTACK-7760


Repository: cloudstack-git


Description
---

https://issues.apache.org/jira/browse/CLOUDSTACK-7760
During VM deployment and while choosing CUSTOM disk offering for data disk, the 
size of the data disk is not considered for checking resource limit of primary 
storage.


Diffs
-

  server/src/com/cloud/vm/UserVmManagerImpl.java 
2636096d03d7cfa097605ac066b8bd66b24972b7 

Diff: https://reviews.apache.org/r/26969/diff/


Testing
---

tested on 4.5


Thanks,

bharat kumar

Review Request 25991: user vm can get a gateway ip when gateway ip is a part of the guest ip range.

2014-09-24 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25991/
---

Review request for cloudstack and Jayapal Reddy.


Bugs: CLOUDSTACK-7536
https://issues.apache.org/jira/browse/CLOUDSTACK-7536


Repository: cloudstack-git


Description
---

user vm can get a gateway ip when gateway ip is a part of the guest ip range.
https://issues.apache.org/jira/browse/CLOUDSTACK-7536


Diffs
-

  server/src/com/cloud/configuration/ConfigurationManagerImpl.java df6ce45 

Diff: https://reviews.apache.org/r/25991/diff/


Testing
---


Thanks,

bharat kumar

Review Request 25768: changing value of cpu/mem.overprovisioning.factor for xen cluster is not affecting total memory at zone level

2014-09-17 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25768/
---

Review request for cloudstack and Kishan Kavala.


Bugs: CLOUDSTACK-7571
https://issues.apache.org/jira/browse/CLOUDSTACK-7571


Repository: cloudstack-git


Description
---

https://issues.apache.org/jira/browse/CLOUDSTACK-7571


Diffs
-

  engine/schema/src/com/cloud/capacity/dao/CapacityDaoImpl.java 9cae045 

Diff: https://reviews.apache.org/r/25768/diff/


Testing
---

Tested on master.


Thanks,

bharat kumar

Regarding gateway ip getting allocated to user VMs in shared network.

2014-09-11 Thread Bharat Kumar

Hi All,

I have found that the gateway ip can get allocated to the uservms in case of 
shared networks. This is a corner case and might happen accidentally. 

This happens because we allow addition guest ip ranges containing the gateway 
ip, while creating a shared network and internally there is no check 
to see if the ip getting allocated is a gateway ip. 

Simple way to fix this would be to fail the network creation if the supplied 
guest ip range contains the gateway ip.

example
gateway=172.16.88.1
netmask=255.255.255.0
guestIpRange=172.16.88.1 to 172.16.88.20.

we fail the network creation in this case as the gateway ip is a part of the 
guest ip range.


Any concerns or suggestions regarding this fix ?

bug id CLOUDSTACK-7536

Review Request 25430: live migration is failing for vm deployed using dynaic compute offerings with NPE

2014-09-08 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25430/
---

Review request for cloudstack and Koushik Das.


Bugs: CLOUDSTACK-6099
https://issues.apache.org/jira/browse/CLOUDSTACK-6099


Repository: cloudstack-git


Description
---

live migration is failing for vm deployed using dynaic compute offerings with 
NPE
https://issues.apache.org/jira/browse/CLOUDSTACK-6099


Diffs
-

  api/src/com/cloud/vm/VirtualMachineProfile.java 29f3164 
  engine/components-api/src/com/cloud/vm/VirtualMachineProfileImpl.java a1e2528 
  engine/orchestration/src/com/cloud/vm/VirtualMachineManagerImpl.java 8edef77 
  server/src/com/cloud/server/ManagementServerImpl.java 697d1c4 

Diff: https://reviews.apache.org/r/25430/diff/


Testing
---

Tested live migration on master using xenserver.


Thanks,

bharat kumar

Re: Review Request 25430: live migration is failing for vm deployed using dynaic compute offerings with NPE

2014-09-08 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25430/
---

(Updated Sept. 8, 2014, 10:52 a.m.)


Review request for cloudstack and Koushik Das.


Bugs: CLOUDSTACK-6099
https://issues.apache.org/jira/browse/CLOUDSTACK-6099


Repository: cloudstack-git


Description
---

live migration is failing for vm deployed using dynaic compute offerings with 
NPE
https://issues.apache.org/jira/browse/CLOUDSTACK-6099


Diffs (updated)
-

  engine/orchestration/src/com/cloud/vm/VirtualMachineManagerImpl.java 8edef77 

Diff: https://reviews.apache.org/r/25430/diff/


Testing
---

Tested live migration on master using xenserver.


Thanks,

bharat kumar

Re: Review Request 25430: live migration is failing for vm deployed using dynaic compute offerings with NPE

2014-09-08 Thread bharat kumar



 On Sept. 8, 2014, 9:27 a.m., Rohit Yadav wrote:
  server/src/com/cloud/server/ManagementServerImpl.java, lines 1160-1161
  https://reviews.apache.org/r/25430/diff/1/?file=682473#file682473line1160
 
  Why not fix VirtualMachineProfileImpl constructor where we're passing 
  the offering?
  
  VirtualMachineProfile vmProfile = new VirtualMachineProfileImpl(vm, 
  null, _offeringDao.findById(vm.getId(), vm.getServiceOfferingId()), null, 
  null);

it is a typo, fixed this.


- bharat


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25430/#review52575
---


On Sept. 8, 2014, 9:16 a.m., bharat kumar wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/25430/
 ---
 
 (Updated Sept. 8, 2014, 9:16 a.m.)
 
 
 Review request for cloudstack and Koushik Das.
 
 
 Bugs: CLOUDSTACK-6099
 https://issues.apache.org/jira/browse/CLOUDSTACK-6099
 
 
 Repository: cloudstack-git
 
 
 Description
 ---
 
 live migration is failing for vm deployed using dynaic compute offerings with 
 NPE
 https://issues.apache.org/jira/browse/CLOUDSTACK-6099
 
 
 Diffs
 -
 
   api/src/com/cloud/vm/VirtualMachineProfile.java 29f3164 
   engine/components-api/src/com/cloud/vm/VirtualMachineProfileImpl.java 
 a1e2528 
   engine/orchestration/src/com/cloud/vm/VirtualMachineManagerImpl.java 
 8edef77 
   server/src/com/cloud/server/ManagementServerImpl.java 697d1c4 
 
 Diff: https://reviews.apache.org/r/25430/diff/
 
 
 Testing
 ---
 
 Tested live migration on master using xenserver.
 
 
 Thanks,
 
 bharat kumar

Re: Review Request 25430: live migration is failing for vm deployed using dynaic compute offerings with NPE

2014-09-08 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25430/
---

(Updated Sept. 8, 2014, 1:55 p.m.)


Review request for cloudstack, Alena Prokharchyk, Devdeep Singh, edison su, 
Koushik Das, Mike Tutkowski, and Nitin Mehta.


Bugs: CLOUDSTACK-6099
https://issues.apache.org/jira/browse/CLOUDSTACK-6099


Repository: cloudstack-git


Description
---

live migration is failing for vm deployed using dynaic compute offerings with 
NPE
https://issues.apache.org/jira/browse/CLOUDSTACK-6099


Diffs
-

  engine/orchestration/src/com/cloud/vm/VirtualMachineManagerImpl.java 8edef77 

Diff: https://reviews.apache.org/r/25430/diff/


Testing
---

Tested live migration on master using xenserver.


Thanks,

bharat kumar

Re: Review Request 23806: Re-copying templates to other zones doesn't work

2014-08-20 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23806/
---

(Updated Aug. 20, 2014, 11:59 a.m.)


Review request for cloudstack and Kishan Kavala.


Bugs: CLOUDSTACK-7155 and CLOUDSTACK-7156
https://issues.apache.org/jira/browse/CLOUDSTACK-7155
https://issues.apache.org/jira/browse/CLOUDSTACK-7156


Repository: cloudstack-git


Description
---

https://issues.apache.org/jira/browse/CLOUDSTACK-7155
Re-copying templates to other zones doesn't work 

List templates API returns destroyed templates when recopying the template.
https://issues.apache.org/jira/browse/CLOUDSTACK-7156


Diffs (updated)
-

  
engine/schema/src/org/apache/cloudstack/storage/datastore/db/TemplateDataStoreDao.java
 cb15949 
  
engine/storage/src/org/apache/cloudstack/storage/image/db/TemplateDataStoreDaoImpl.java
 50334f4 
  server/src/com/cloud/template/HypervisorTemplateAdapter.java 72aaff5 
  server/src/com/cloud/template/TemplateManagerImpl.java f5ad97f 

Diff: https://reviews.apache.org/r/23806/diff/


Testing
---


Thanks,

bharat kumar

Re: Review Request 24892: passwd_server attempts to start but terminates with the exit code 137

2014-08-20 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24892/
---

(Updated Aug. 20, 2014, 2:25 p.m.)


Review request for cloudstack and Sheng Yang.


Bugs: CLOUDSTACK-7376
https://issues.apache.org/jira/browse/CLOUDSTACK-7376


Repository: cloudstack-git


Description
---

https://issues.apache.org/jira/browse/CLOUDSTACK-7376
passwd_server attempts to start but terminates with the exit code 137


Diffs
-

  systemvm/patches/debian/config/opt/cloud/bin/passwd_server 0f4a772 

Diff: https://reviews.apache.org/r/24892/diff/


Testing (updated)
---

I was able to reproduce this in the older version but not on master.
but iam putting this patch anyway as this will not effect the regular runs.


Thanks,

bharat kumar

Re: Review Request 23806: Re-copying templates to other zones doesn't work

2014-08-19 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23806/
---

(Updated Aug. 19, 2014, 2:23 p.m.)


Review request for cloudstack and Kishan Kavala.


Bugs: CLOUDSTACK-7155 and CLOUDSTACK-7156
https://issues.apache.org/jira/browse/CLOUDSTACK-7155
https://issues.apache.org/jira/browse/CLOUDSTACK-7156


Repository: cloudstack-git


Description
---

https://issues.apache.org/jira/browse/CLOUDSTACK-7155
Re-copying templates to other zones doesn't work 

List templates API returns destroyed templates when recopying the template.
https://issues.apache.org/jira/browse/CLOUDSTACK-7156


Diffs (updated)
-

  
engine/api/src/org/apache/cloudstack/storage/datastore/db/TemplateDataStoreDao.java
 PRE-CREATION 
  
engine/storage/src/org/apache/cloudstack/storage/image/db/TemplateDataStoreDaoImpl.java
 50334f4 
  server/src/com/cloud/template/HypervisorTemplateAdapter.java 72aaff5 
  server/src/com/cloud/template/TemplateManagerImpl.java f5ad97f 

Diff: https://reviews.apache.org/r/23806/diff/


Testing
---


Thanks,

bharat kumar

Re: Review Request 23806: Re-copying templates to other zones doesn't work

2014-08-19 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23806/
---

(Updated Aug. 19, 2014, 2:39 p.m.)


Review request for cloudstack and Kishan Kavala.


Bugs: CLOUDSTACK-7155 and CLOUDSTACK-7156
https://issues.apache.org/jira/browse/CLOUDSTACK-7155
https://issues.apache.org/jira/browse/CLOUDSTACK-7156


Repository: cloudstack-git


Description
---

https://issues.apache.org/jira/browse/CLOUDSTACK-7155
Re-copying templates to other zones doesn't work 

List templates API returns destroyed templates when recopying the template.
https://issues.apache.org/jira/browse/CLOUDSTACK-7156


Diffs (updated)
-

  
engine/schema/src/org/apache/cloudstack/storage/datastore/db/TemplateDataStoreDao.java
 cb15949 
  
engine/storage/src/org/apache/cloudstack/storage/image/db/TemplateDataStoreDaoImpl.java
 50334f4 
  server/src/com/cloud/template/HypervisorTemplateAdapter.java 72aaff5 
  server/src/com/cloud/template/TemplateManagerImpl.java f5ad97f 

Diff: https://reviews.apache.org/r/23806/diff/


Testing
---


Thanks,

bharat kumar

Review Request 24645: Disabling failed test cases.

2014-08-13 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24645/
---

Review request for cloudstack and Santhosh Edukulla.


Repository: cloudstack-git


Description
---

Adding tag BugId to disable the test cases until they are fixed.

integration.smoke.test_snapshots.TestSnapshotRootDisk.test_01_snapshot_root_disk
  CLOUDSTACK-7336
integration.smoke.test_templates.TestTemplates.test_06_copy_template  
CLOUDSTACK-7335


Diffs
-

  test/integration/smoke/test_snapshots.py 5db3e40 
  test/integration/smoke/test_templates.py a757c2a 

Diff: https://reviews.apache.org/r/24645/diff/


Testing
---


Thanks,

bharat kumar

Review Request 23806: Re-copying templates to other zones doesn't work

2014-07-22 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23806/
---

Review request for cloudstack and Kishan Kavala.


Bugs: CLOUDSTACK-7155 and CLOUDSTACK-7156
https://issues.apache.org/jira/browse/CLOUDSTACK-7155
https://issues.apache.org/jira/browse/CLOUDSTACK-7156


Repository: cloudstack-git


Description
---

https://issues.apache.org/jira/browse/CLOUDSTACK-7155
Re-copying templates to other zones doesn't work 

List templates API returns destroyed templates when recopying the template.
https://issues.apache.org/jira/browse/CLOUDSTACK-7156


Diffs
-

  engine/schema/src/com/cloud/capacity/dao/CapacityDaoImpl.java b7784f2 
  
engine/schema/src/org/apache/cloudstack/storage/datastore/db/TemplateDataStoreDao.java
 cb15949 
  
engine/storage/src/org/apache/cloudstack/storage/image/db/TemplateDataStoreDaoImpl.java
 50334f4 
  server/src/com/cloud/template/HypervisorTemplateAdapter.java 20309d6 
  server/src/com/cloud/template/TemplateManagerImpl.java f5ad97f 

Diff: https://reviews.apache.org/r/23806/diff/


Testing
---


Thanks,

bharat kumar

Review Request 23807: List templates API returns destroyed templates when recopying the template.

2014-07-22 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23807/
---

Review request for cloudstack and Koushik Das.


Bugs: CLOUDSTACK-7156
https://issues.apache.org/jira/browse/CLOUDSTACK-7156


Repository: cloudstack-git


Description
---

List templates API returns destroyed templates when recopying the template.
https://issues.apache.org/jira/browse/CLOUDSTACK-7156


Diffs
-

  server/src/com/cloud/api/query/dao/TemplateJoinDao.java 298be4d 
  server/src/com/cloud/api/query/dao/TemplateJoinDaoImpl.java 4c8ada1 

Diff: https://reviews.apache.org/r/23807/diff/


Testing
---


Thanks,

bharat kumar

Re: Review Request 23806: Re-copying templates to other zones doesn't work

2014-07-22 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23806/
---

(Updated July 22, 2014, 1:20 p.m.)


Review request for cloudstack and Kishan Kavala.


Bugs: CLOUDSTACK-7155 and CLOUDSTACK-7156
https://issues.apache.org/jira/browse/CLOUDSTACK-7155
https://issues.apache.org/jira/browse/CLOUDSTACK-7156


Repository: cloudstack-git


Description
---

https://issues.apache.org/jira/browse/CLOUDSTACK-7155
Re-copying templates to other zones doesn't work 

List templates API returns destroyed templates when recopying the template.
https://issues.apache.org/jira/browse/CLOUDSTACK-7156


Diffs (updated)
-

  
engine/schema/src/org/apache/cloudstack/storage/datastore/db/TemplateDataStoreDao.java
 cb15949 
  
engine/storage/src/org/apache/cloudstack/storage/image/db/TemplateDataStoreDaoImpl.java
 50334f4 
  server/src/com/cloud/template/HypervisorTemplateAdapter.java 20309d6 
  server/src/com/cloud/template/TemplateManagerImpl.java f5ad97f 

Diff: https://reviews.apache.org/r/23806/diff/


Testing
---


Thanks,

bharat kumar

Re: Review Request 23808: listCapacity API missing types for certain zones

2014-07-22 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23808/
---

(Updated July 22, 2014, 1:24 p.m.)


Review request for cloudstack and Kishan Kavala.


Bugs: CLOUDSTACK-7158
https://issues.apache.org/jira/browse/CLOUDSTACK-7158


Repository: cloudstack-git


Description
---

listCapacity API missing types for certain zones
https://issues.apache.org/jira/browse/CLOUDSTACK-7158


Diffs (updated)
-

  engine/schema/src/com/cloud/capacity/dao/CapacityDaoImpl.java b7784f2 
  server/src/com/cloud/server/ManagementServerImpl.java 99b12732 

Diff: https://reviews.apache.org/r/23808/diff/


Testing
---


Thanks,

bharat kumar

Review Request 23193: mark the test_01_primary_storage_iscsi as test that requires hardware

2014-07-01 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23193/
---

Review request for cloudstack and Abhinandan Prateek.


Bugs: CLOUDSTACK-7026
https://issues.apache.org/jira/browse/CLOUDSTACK-7026


Repository: cloudstack-git


Description
---

The test_01_primary_storage_iscsi is failing on simulator runs. It is because 
simulator cannot handle ISCSi type of storage. so we need to tag this test as a 
required hardware test.


Diffs
-

  test/integration/smoke/test_primary_storage.py 0312ad6 

Diff: https://reviews.apache.org/r/23193/diff/


Testing
---


Thanks,

bharat kumar

Re: [ACS4.5] Can't access volumes on storage tab in GUI

2014-06-30 Thread Bharat Kumar

Hi Mike,

I think the volume_view did not get created properly for some reason. I think 
recreating the volume_view will fix the problem.

below commas should fix the issue.

DROP VIEW IF EXISTS `cloud`.`volume_view`;
CREATE VIEW `cloud`.`volume_view` AS
select
volumes.id,
volumes.uuid,
volumes.name,
volumes.device_id,
volumes.volume_type,
volumes.size,
volumes.min_iops,
volumes.max_iops,
volumes.created,
volumes.state,
volumes.attached,
volumes.removed,
volumes.pod_id,
volumes.display_volume,
volumes.format,
volumes.path,
volumes.chain_info,
account.id account_id,
account.uuid account_uuid,
account.account_name account_name,
account.type account_type,
domain.id domain_id,
domain.uuid domain_uuid,
domain.name domain_name,
domain.path domain_path,
projects.id project_id,
projects.uuid project_uuid,
projects.name project_name,
data_center.id data_center_id,
data_center.uuid data_center_uuid,
data_center.name data_center_name,
data_center.networktype data_center_type,
vm_instance.id vm_id,
vm_instance.uuid vm_uuid,
vm_instance.name vm_name,
vm_instance.state vm_state,
vm_instance.vm_type,
user_vm.display_name vm_display_name,
volume_store_ref.size volume_store_size,
volume_store_ref.download_pct,
volume_store_ref.download_state,
volume_store_ref.error_str,
volume_store_ref.created created_on_store,
disk_offering.id disk_offering_id,
disk_offering.uuid disk_offering_uuid,
disk_offering.name disk_offering_name,
disk_offering.display_text disk_offering_display_text,
disk_offering.use_local_storage,
disk_offering.system_use,
disk_offering.bytes_read_rate,
disk_offering.bytes_write_rate,
disk_offering.iops_read_rate,
disk_offering.iops_write_rate,
disk_offering.cache_mode,
storage_pool.id pool_id,
storage_pool.uuid pool_uuid,
storage_pool.name pool_name,
cluster.hypervisor_type,
vm_template.id template_id,
vm_template.uuid template_uuid,
vm_template.extractable,
vm_template.type template_type,
vm_template.name template_name,
vm_template.display_text template_display_text,
iso.id iso_id,
iso.uuid iso_uuid,
iso.name iso_name,
iso.display_text iso_display_text,
resource_tags.id tag_id,
resource_tags.uuid tag_uuid,
resource_tags.key tag_key,
resource_tags.value tag_value,
resource_tags.domain_id tag_domain_id,
resource_tags.account_id tag_account_id,
resource_tags.resource_id tag_resource_id,
resource_tags.resource_uuid tag_resource_uuid,
resource_tags.resource_type tag_resource_type,
resource_tags.customer tag_customer,
async_job.id job_id,
async_job.uuid job_uuid,
async_job.job_status job_status,
async_job.account_id job_account_id
from
`cloud`.`volumes`
inner join
`cloud`.`account` ON volumes.account_id = account.id
inner join
`cloud`.`domain` ON volumes.domain_id = domain.id
left join
`cloud`.`projects` ON projects.project_account_id = account.id
left join
`cloud`.`data_center` ON volumes.data_center_id = data_center.id
left join
`cloud`.`vm_instance` ON volumes.instance_id = vm_instance.id
left join
`cloud`.`user_vm` ON user_vm.id = vm_instance.id
left join
`cloud`.`volume_store_ref` ON volumes.id = volume_store_ref.volume_id
left join
`cloud`.`disk_offering` ON volumes.disk_offering_id = disk_offering.id
left join
`cloud`.`storage_pool` ON volumes.pool_id = storage_pool.id
left join
`cloud`.`cluster` ON storage_pool.cluster_id = cluster.id
left join
`cloud`.`vm_template` ON volumes.template_id = vm_template.id
left join
`cloud`.`vm_template` iso ON iso.id = volumes.iso_id
left join
`cloud`.`resource_tags` ON resource_tags.resource_id = volumes.id
and resource_tags.resource_type = 'Volume'
left join
`cloud`.`async_job` ON async_job.instance_id = volumes.id
and async_job.instance_type = 'Volume'
and async_job.job_status = 0;

Thanks,
Bharat.

On 28-Jun-2014, at 3:30 am, Mike Tutkowski mike.tutkow...@solidfire.com wrote:

 Hi,
 
 If you click on the Storage tab in the GUI when you have one or more
 volumes, you receive the following exception (is this something someone is
 already working on?):
 
 Caused by:

Review Request 22376: tagging the test case test_09_delete_detached_volume with the bugId.

2014-06-09 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22376/
---

Review request for cloudstack and Abhinandan Prateek.


Bugs: CLOUDSTACK-6875
https://issues.apache.org/jira/browse/CLOUDSTACK-6875


Repository: cloudstack-git


Description
---

The test case test_09_delete_detached_volume is failing. we have logged a bug 
CLOUDSTACK-6875 to track this.
this patch tags the test case with the above mentioned bugId.


Diffs
-

  test/integration/smoke/test_volumes.py 73c2722 

Diff: https://reviews.apache.org/r/22376/diff/


Testing
---


Thanks,

bharat kumar

Review Request 22377: tagging the test case test_01_sys_vm_start with the bugId.

2014-06-09 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22377/
---

Review request for cloudstack and Abhinandan Prateek.


Bugs: CLOUDSTACK-6877
https://issues.apache.org/jira/browse/CLOUDSTACK-6877


Repository: cloudstack-git


Description
---

The test case test_01_sys_vm_start is failing. we have logged a bug 
CLOUDSTACK-6877 to track this.
this patch tags the test case with the above mentioned bugId.


Diffs
-

  test/integration/smoke/test_secondary_storage.py 481d907 

Diff: https://reviews.apache.org/r/22377/diff/


Testing
---


Thanks,

bharat kumar

Review Request 22378: tagging the test case test_01_snapshot_root_disk with the bugId.

2014-06-09 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22378/
---

Review request for cloudstack and Abhinandan Prateek.


Bugs: CLOUDSTACK-6878
https://issues.apache.org/jira/browse/CLOUDSTACK-6878


Repository: cloudstack-git


Description
---

The test case test_01_snapshot_root_disk is failing. we have logged a bug 
CLOUDSTACK-6876 to track this.
this patch tags the test case with the above mentioned bugId.


Diffs
-

  test/integration/smoke/test_snapshots.py a960e70 

Diff: https://reviews.apache.org/r/22378/diff/


Testing
---


Thanks,

bharat kumar

Review Request 22379: tagging the test case test_vpc_site2site_vpn with the bugId.

2014-06-09 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22379/
---

Review request for cloudstack and Abhinandan Prateek.


Bugs: CLOUDSTACK-6879
https://issues.apache.org/jira/browse/CLOUDSTACK-6879


Repository: cloudstack-git


Description
---

The test case test_vpc_site2site_vpn is failing. we have logged a bug 
CLOUDSTACK-6876 to track this.
this patch tags the test case with the above mentioned bugId.


Diffs
-

  test/integration/smoke/test_vpc_vpn.py 03826e9 

Diff: https://reviews.apache.org/r/22379/diff/


Testing
---


Thanks,

bharat kumar

Review Request 22375: tagging the test case test_deploy_vgpu_enabled_vm with the bugId.

2014-06-09 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22375/
---

Review request for cloudstack and Abhinandan Prateek.


Bugs: CLOUDSTACK-6876
https://issues.apache.org/jira/browse/CLOUDSTACK-6876


Repository: cloudstack-git


Description
---

The test case test_deploy_vgpu_enabled_vm is failing. we have logged a bug 
CLOUDSTACK-6876 to track this.
this patch tags the test case with the above mentioned bugId.


Diffs
-

  test/integration/smoke/test_deploy_vgpu_enabled_vm.py 13fad61 

Diff: https://reviews.apache.org/r/22375/diff/


Testing
---


Thanks,

bharat kumar

Review Request 22229: Test case failure in test_iso.py

2014-06-04 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9/
---

Review request for cloudstack, Abhinandan Prateek and Koushik Das.


Bugs: CLOUDSTACK-6769
https://issues.apache.org/jira/browse/CLOUDSTACK-6769


Repository: cloudstack-git


Description
---

Hi,
there was a environment issue because of which the test suite was failing at 
setup. I fixed the env issue,
now the test suite runs on Xenserver but not on KVM (it fails at setup on KVM).

Also out of the 5 tests running on xen only 3 are passing. the failed tests are
integration.smoke.test_iso.TestISO.test_04_extract_Iso
integration.smoke.test_iso.TestISO.test_06_copy_iso

So disabling this suite until the above mentioned issues are fixed.


Diffs
-

  test/integration/smoke/test_iso.py a3e42f1 

Diff: https://reviews.apache.org/r/9/diff/


Testing
---


Thanks,

bharat kumar

[ACS 4.4] cherry-pick request for CLOUDSTACK-5077

2014-05-05 Thread Bharat Kumar

Hi Daan,

please cherry pick the following commit  to  4.4 branch.
commit 489bb0c7edf6ab011895d5f082328eb5fe48aac1 in cloudstack 4.4-forward 
branch.
link to diff  Cloudstack-5077: reserve cpu and memory only when 
vmware.reserve.cpu/mem are 
…http://git-ccp.citrix.com/cgit/internal-cloudstack.git/commit/?h=4.4-forwardid=489bb0c7edf6ab011895d5f082328eb5fe48aac1

Thanks,
Bharat.

Re: [ACS 4.4] cherry-pick request for CLOUDSTACK-5077

2014-05-05 Thread Bharat Kumar

Hi Daan,

updating the link to diff.
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commit;h=99b4cf788eded0f6f155ff8cda51aeb2e505f764

Thanks
Bharat.
On 05-May-2014, at 3:36 pm, Bharat Kumar 
bharat.ku...@citrix.commailto:bharat.ku...@citrix.com wrote:

Hi Daan,

please cherry pick the following commit  to  4.4 branch.
commit 489bb0c7edf6ab011895d5f082328eb5fe48aac1 in cloudstack 4.4-forward 
branch.
link to diff  Cloudstack-5077: reserve cpu and memory only when 
vmware.reserve.cpu/mem are 
…http://git-ccp.citrix.com/cgit/internal-cloudstack.git/commit/?h=4.4-forwardid=489bb0c7edf6ab011895d5f082328eb5fe48aac1

Thanks,
Bharat.

Re: Review Request 20210: List templates returns destroyed temples when copying the same template after deleting.

2014-04-16 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20210/
---

(Updated April 17, 2014, 4:15 a.m.)


Review request for cloudstack, Kishan Kavala and Min Chen.


Changes
---

we don't modify the existing entries anymore. we remove all the entries in the 
template_store ref when a template is deleted.


Bugs: CLOUDSTACK-6377
https://issues.apache.org/jira/browse/CLOUDSTACK-6377


Repository: cloudstack-git


Description
---

List templates returns destroyed temples when copying the same template after 
deleting.
https://issues.apache.org/jira/browse/CLOUDSTACK-6377


Diffs (updated)
-

  
engine/api/src/org/apache/cloudstack/storage/datastore/db/TemplateDataStoreDao.java
 048ce22 
  
engine/storage/src/org/apache/cloudstack/storage/image/db/TemplateDataStoreDaoImpl.java
 f5140e0 
  server/src/com/cloud/template/HypervisorTemplateAdapter.java 963aec9 
  server/src/com/cloud/template/TemplateManagerImpl.java 57f7ff6 

Diff: https://reviews.apache.org/r/20210/diff/


Testing
---

Tested on 4.2


Thanks,

bharat kumar

Re: Review Request 15349: Overprivisoning changes needed

2014-04-16 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15349/
---

(Updated April 17, 2014, 4:31 a.m.)


Review request for cloudstack, Kishan Kavala and Nitin Mehta.


Bugs: Cloudstack-5077.
https://issues.apache.org/jira/browse/Cloudstack-5077.


Repository: cloudstack-git


Description
---


1.) This includes the change to reserve cpu and memory only when 
vmware.reserve.cpu/mem is true, regardless of overcommit. 
2.) populate the default value of the cpu overcommit at cluster level form the 
global setting when up grading. 


Diffs (updated)
-

  engine/schema/src/com/cloud/upgrade/dao/Upgrade410to420.java 82b6d84 
  engine/schema/src/com/cloud/upgrade/dao/Upgrade420to421.java c3a6b06 
  
plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
 e89777d 
  setup/db/db/schema-420to421.sql 42e9c1e 

Diff: https://reviews.apache.org/r/15349/diff/


Testing
---


Thanks,

bharat kumar

Re: Review Request 20210: List templates returns destroyed temples when copying the same template after deleting.

2014-04-11 Thread bharat kumar

On April 10, 2014, 5:19 p.m., Nitin Mehta wrote:
engine/storage/src/org/apache/cloudstack/storage/datastore/ObjectInDataStoreManagerImpl.java,
line 134
https://reviews.apache.org/r/20210/diff/2/?file=554427#file554427line134

Why do we need to update the old entry and not create a new one ? We
should handle listTemplate not returning the destroyed result for the
template ?

Hi Min,

in case of template deletion and recopying we update the same entry in template
zone ref, So i wanted to make the behaviour consistent even in case of entries
in the template_store_ref when dealing with re copying of templates. I am not
sure which one to follow as both have advantages and disadvantages. but what
ever is done i think it should be consistent.

so since the idea is to create new entries always. i will create a bug for
template_zone_ref case and change the current patch to do the same.

On April 10, 2014, 5:19 p.m., Nitin Mehta wrote:
engine/storage/src/org/apache/cloudstack/storage/datastore/ObjectInDataStoreManagerImpl.java,
line 141
https://reviews.apache.org/r/20210/diff/2/?file=554427#file554427line141

Changing the state not using state machine is bad practice.

Hi Min,

I was updating a already destroyed template so had to set it manually as state
machine will not come into picture for cases like these.

On April 10, 2014, 5:19 p.m., Nitin Mehta wrote:
engine/storage/src/org/apache/cloudstack/storage/image/db/TemplateDataStoreDaoImpl.java,
line 286
https://reviews.apache.org/r/20210/diff/2/?file=554428#file554428line286

method name says its zone but then you are passing the image store ?

Hi Min,

will change this to markAsDestroyedByTemplateStore.

On April 10, 2014, 5:19 p.m., Nitin Mehta wrote:
server/src/com/cloud/template/HypervisorTemplateAdapter.java, line 341
https://reviews.apache.org/r/20210/diff/2/?file=554429#file554429line341

The comment doesnt match what you are doing.

The comment was not about what the function is doing, it was about the intent
of that snippet of the code. will change the comment to be more explicit about
this.

On April 10, 2014, 5:19 p.m., Nitin Mehta wrote:
server/src/com/cloud/template/HypervisorTemplateAdapter.java, line 342
https://reviews.apache.org/r/20210/diff/2/?file=554429#file554429line342

Why is this required ?
This should happen via state machine and not like this. Are you seeing
the destroyed not being set ?

Hi Min,

There are two fields which can be set to destroyed int the template_store_ref.
one is the state and the other is the destroyed flag. Again this is confusing
as I am not sure why do we have a destroyed flag and a destroyed state. I think
one should be enough.

- bharat

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20210/#review40034
---

On April 10, 2014, 3:04 p.m., bharat kumar wrote:

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20210/
---

(Updated April 10, 2014, 3:04 p.m.)

Review request for cloudstack, Kishan Kavala and Min Chen.

Bugs: CLOUDSTACK-6377
https://issues.apache.org/jira/browse/CLOUDSTACK-6377

Repository: cloudstack-git

Description
---

List templates returns destroyed temples when copying the same template after
deleting.
https://issues.apache.org/jira/browse/CLOUDSTACK-6377

Diffs
-

engine/api/src/org/apache/cloudstack/storage/datastore/db/TemplateDataStoreDao.java
048ce22

engine/storage/src/org/apache/cloudstack/storage/datastore/ObjectInDataStoreManagerImpl.java
d9a5164

engine/storage/src/org/apache/cloudstack/storage/image/db/TemplateDataStoreDaoImpl.java
f5140e0
server/src/com/cloud/template/HypervisorTemplateAdapter.java 963aec9

Diff: https://reviews.apache.org/r/20210/diff/

Testing
---

Tested on 4.2

Thanks,

bharat kumar

Re: Review Request 20210: List templates returns destroyed temples when copying the same template after deleting.

2014-04-11 Thread bharat kumar



 On April 10, 2014, 5:25 p.m., Min Chen wrote:
  The root cause for this issue is that we didn't remove template_store_ref 
  entry from db when a template is deleted from a zone and only mark them as 
  'Destroyed'. In addition, template_view db view is created by joining with 
  those destroyed entries in template_store_ref as well, which causing it to 
  only randomly pick the entry that may be 'Destroyed'.  The much simpler fix 
  to me is to remove entries from template_store_ref table at the end of 
  deleting template flow when the deletion is successful (since they have 
  already been deleted from secondary storage) instead of finding and then 
  manually updating entry done in the patch.

Hi Min,

The template_view dose the correct thing here. This problem occurs because we 
set the template state as destroyed but we do not mark the destroyed flag to 
true. The template view picks all the entries where destroyed flag is false, 
since even the deleted templates have this set to false they are also getting 
populated in the template_view table. 


- bharat


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20210/#review40039
---


On April 10, 2014, 3:04 p.m., bharat kumar wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/20210/
 ---
 
 (Updated April 10, 2014, 3:04 p.m.)
 
 
 Review request for cloudstack, Kishan Kavala and Min Chen.
 
 
 Bugs: CLOUDSTACK-6377
 https://issues.apache.org/jira/browse/CLOUDSTACK-6377
 
 
 Repository: cloudstack-git
 
 
 Description
 ---
 
 List templates returns destroyed temples when copying the same template after 
 deleting.
 https://issues.apache.org/jira/browse/CLOUDSTACK-6377
 
 
 Diffs
 -
 
   
 engine/api/src/org/apache/cloudstack/storage/datastore/db/TemplateDataStoreDao.java
  048ce22 
   
 engine/storage/src/org/apache/cloudstack/storage/datastore/ObjectInDataStoreManagerImpl.java
  d9a5164 
   
 engine/storage/src/org/apache/cloudstack/storage/image/db/TemplateDataStoreDaoImpl.java
  f5140e0 
   server/src/com/cloud/template/HypervisorTemplateAdapter.java 963aec9 
 
 Diff: https://reviews.apache.org/r/20210/diff/
 
 
 Testing
 ---
 
 Tested on 4.2
 
 
 Thanks,
 
 bharat kumar

Re: Review Request 20210: List templates returns destroyed temples when copying the same template after deleting.

2014-04-10 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20210/
---

(Updated April 10, 2014, 3:04 p.m.)


Review request for cloudstack, Kishan Kavala and Min Chen.


Changes
---

Adding Min to the reviewer's list.


Bugs: CLOUDSTACK-6377
https://issues.apache.org/jira/browse/CLOUDSTACK-6377


Repository: cloudstack-git


Description
---

List templates returns destroyed temples when copying the same template after 
deleting.
https://issues.apache.org/jira/browse/CLOUDSTACK-6377


Diffs
-

  
engine/api/src/org/apache/cloudstack/storage/datastore/db/TemplateDataStoreDao.java
 048ce22 
  
engine/storage/src/org/apache/cloudstack/storage/datastore/ObjectInDataStoreManagerImpl.java
 d9a5164 
  
engine/storage/src/org/apache/cloudstack/storage/image/db/TemplateDataStoreDaoImpl.java
 f5140e0 
  server/src/com/cloud/template/HypervisorTemplateAdapter.java 963aec9 

Diff: https://reviews.apache.org/r/20210/diff/


Testing
---

Tested on 4.2


Thanks,

bharat kumar

Review Request 20210: List templates returns destroyed temples when copying the same template after deleting.

2014-04-10 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20210/
---

Review request for cloudstack and Kishan Kavala.


Summary (updated)
-

List templates returns destroyed temples when copying the same template after 
deleting.


Bugs: CLOUDSTACK-6377
https://issues.apache.org/jira/browse/CLOUDSTACK-6377


Repository: cloudstack-git


Description (updated)
---

List templates returns destroyed temples when copying the same template after 
deleting.
https://issues.apache.org/jira/browse/CLOUDSTACK-6377


Diffs (updated)
-

  
engine/api/src/org/apache/cloudstack/storage/datastore/db/TemplateDataStoreDao.java
 048ce22 
  
engine/storage/src/org/apache/cloudstack/storage/datastore/ObjectInDataStoreManagerImpl.java
 d9a5164 
  
engine/storage/src/org/apache/cloudstack/storage/image/db/TemplateDataStoreDaoImpl.java
 f5140e0 
  server/src/com/cloud/template/HypervisorTemplateAdapter.java 963aec9 

Diff: https://reviews.apache.org/r/20210/diff/


Testing (updated)
---

Tested on 4.2


Thanks,

bharat kumar

Re: [PROPOSAL][QUESTION] Map parameters in API Commands

2014-03-10 Thread Bharat Kumar

Hi All,
the roodiskresize is no longer valid. as there is no code which is using 
rootdiskresize currently.

As a part of the custom service offering we had to enable specifying custom 
values to parameters cpu, memory and cpuCores. 
instead of adding a parameter for each of these values we changed it use a 
details map.  This will also not require any further 
changes in the API if we need to add some more custom values in future.
  
On 08-Mar-2014, at 1:42 am, Marcus shadow...@gmail.com wrote:

 Any suggestion? Do we go forward assuming that the correct parameter
 for resize on deploy is:
 
 deployVirtualMachinedetails[0].rootdisksize=3
 
 or do we change it to
 
 deployVirtualMachinerootdisksize=3
 
 On Tue, Mar 4, 2014 at 4:14 PM, Marcus shadow...@gmail.com wrote:
 Ok, sounds like there needs to be some work done to make these more
 consistent, perhaps. Can you comment on why rootdisksize was made from
 a parameter into a part of the details map?
 
 On Tue, Mar 4, 2014 at 3:12 AM, Bharat Kumar bharat.ku...@citrix.com wrote:
 Hi ALL,
 There are many other APIs that use Map like createNetworkOffering,
 updateZone, createTemplate, in most of the cases we do not
 say how to use maps, one way would be to write this in the description or to
 use the same way to access maps of all APIs.
 
 BTW the way to use details in deploy vm API is
 details[0].foo=bardetails[1].baz=12 where foo and baz are keys.
 
 Also  if we want to use the regix protected static final String
 MAP_KEY_PATTERN_EXPRESSION = ^([^\\[^\\]]+)\\[(\\d+)\\]\\.key$;
   protected static final
 String MAP_VALUE_PATTERN_EXPRESSION = ^[^\\[^\\]]+\\[\\d+\\]\\.value$;
 
 wil this work in the following case. I believe service is the key here which
 repeats.
 http://10.147.59.119:8080/client/api?command=createNetworkOfferingresponse=jsonsessionkey=/kGFJDXFmMQU8JZnnC7QFfj3tV8=name=bharatdisplayText=bharatguestIpType=IsolatedlbType=publicLb;
 servicecapabilitylist[0].service=SourceNatservicecapabilitylist[0].capabilitytype=SupportedSourceNatTypes
 servicecapabilitylist[0].capabilityvalue=peraccount
 servicecapabilitylist[1].service=lbservicecapabilitylist[1].capabilitytype=SupportedLbIsolation
 servicecapabilitylist[1].capabilityvalue=dedicatedavailability=Optionalegresspolicy=ALLOWstate=Creatingstatus=Creatingallocationstate=CreatingsupportedServices=Vpn,Dhcp,Dns,Firewall,Lb,UserData,SourceNat,StaticNat,PortForwardingspecifyIpRanges=falsespecifyVlan=falseisPersistent=falseconservemode=falseserviceProviderList[0].service=VpnserviceProviderList[0].provider=VirtualRouterserviceProviderList[1].service=DhcpserviceProviderList[1].provider=VirtualRouterserviceProviderList[2].service=DnsserviceProviderList[2].provider=VirtualRouterserviceProviderList[3].service=FirewallserviceProviderList[3].provider=VirtualRouterserviceProviderList[4].service=LbserviceProviderList[4].provider=VirtualRouterserviceProviderList[5].service=UserDataserviceProviderList[5].provider=VirtualRouterserviceProviderList[6].service=SourceNatserviceProviderList[6].provider=JuniperSRXserviceProviderList[7].service=StaticNatserviceProviderList[7].provider=JuniperSRXserviceProviderList[8].service=PortForwardingserviceProviderList[8].provider=JuniperSRXegressdefaultpolicy=truetraffictype=GUEST_=1393925230248
 
 
 
 On 04-Mar-2014, at 2:30 am, Marcus shadow...@gmail.com wrote:
 
 Along these lines, the details parameter in deployVirtualMachine seems
 broken. If I call details[0].key=foo,details[0].value=bar, it stores
 entries in the database like this:
 
 id | vmid | name | value | display
 
 12 | 25   |  value | bar   | 1
 13 | 25   |  key   | foo   | 1
 
 It seems as though this might be correct per Alena's email, but I
 don't understand how this can be reconstructed into foo=bar when
 there's no binding between the two rows. Perhaps details are supposed
 to be passed differently than the resource tags, because if I do
 details[0].foo=bardetails[1].baz=12, I get:
 
 id | vmid | name | value | display
 
 12 | 25   |  foo| bar| 1
 13 | 25   |  baz   | 12 | 1
 
 And indeed there is code utilizing these details already committed
 that expects this format, as deployVirtualMachines getDetails() only
 seems to pass a correct MapString, String with Key, Value if I use
 this format.
 
 On Mon, Mar 3, 2014 at 11:48 AM, Antonio Fornié Casarrubios
 antonio.for...@gmail.com wrote:
 
 Hi Alena,
 
 Of course, the API will not have any changes. This is not a functional
 change, just some refactoring. The problem is there are many things in CS
 that really need some refactoring otherwise the problem will continue
 growing more and more, but doing the change and above all making sure it
 all works afterwards is not simple.
 
 I will make sure that everything works exactly the same way and that the
 data returned is also the same.
 
 Thanks. Cheers
 Antonio
 
 
 2014-03-03 18:48 GMT+01:00 Alena

Re: [PROPOSAL][QUESTION] Map parameters in API Commands

2014-03-10 Thread Bharat Kumar

Hi,
i don’t  know if this is still valid, but there was one more parameter disksize 
which which specifies the disk size for data disks. 
I would prefer adding both the root disk size and disk size to the details map. 
ideally we should also change the name disksize to 
dataDisksize to remove confusion but this might break backward compatibility.

Adding them to a map will be intuitive as we already use a map to specify any 
custom parameters related to VM. 

Thanks
Bharat.
  
On 10-Mar-2014, at 10:57 pm, Marcus shadow...@gmail.com wrote:

 It is valid, as I've implemented it. So we need to decide if we're
 using 'details' or rootdisksize as an api param. That's why I'm
 asking.
 
 On Mon, Mar 10, 2014 at 1:43 AM, Bharat Kumar bharat.ku...@citrix.com wrote:
 Hi All,
 the roodiskresize is no longer valid. as there is no code which is using 
 rootdiskresize currently.
 
 As a part of the custom service offering we had to enable specifying custom 
 values to parameters cpu, memory and cpuCores.
 instead of adding a parameter for each of these values we changed it use a 
 details map.  This will also not require any further
 changes in the API if we need to add some more custom values in future.
 
 On 08-Mar-2014, at 1:42 am, Marcus shadow...@gmail.com wrote:
 
 Any suggestion? Do we go forward assuming that the correct parameter
 for resize on deploy is:
 
 deployVirtualMachinedetails[0].rootdisksize=3
 
 or do we change it to
 
 deployVirtualMachinerootdisksize=3
 
 On Tue, Mar 4, 2014 at 4:14 PM, Marcus shadow...@gmail.com wrote:
 Ok, sounds like there needs to be some work done to make these more
 consistent, perhaps. Can you comment on why rootdisksize was made from
 a parameter into a part of the details map?
 
 On Tue, Mar 4, 2014 at 3:12 AM, Bharat Kumar bharat.ku...@citrix.com 
 wrote:
 Hi ALL,
 There are many other APIs that use Map like createNetworkOffering,
 updateZone, createTemplate, in most of the cases we do not
 say how to use maps, one way would be to write this in the description or 
 to
 use the same way to access maps of all APIs.
 
 BTW the way to use details in deploy vm API is
 details[0].foo=bardetails[1].baz=12 where foo and baz are keys.
 
 Also  if we want to use the regix protected static final String
 MAP_KEY_PATTERN_EXPRESSION = ^([^\\[^\\]]+)\\[(\\d+)\\]\\.key$;
  protected static final
 String MAP_VALUE_PATTERN_EXPRESSION = ^[^\\[^\\]]+\\[\\d+\\]\\.value$;
 
 wil this work in the following case. I believe service is the key here 
 which
 repeats.
 http://10.147.59.119:8080/client/api?command=createNetworkOfferingresponse=jsonsessionkey=/kGFJDXFmMQU8JZnnC7QFfj3tV8=name=bharatdisplayText=bharatguestIpType=IsolatedlbType=publicLb;
 servicecapabilitylist[0].service=SourceNatservicecapabilitylist[0].capabilitytype=SupportedSourceNatTypes
 servicecapabilitylist[0].capabilityvalue=peraccount
 servicecapabilitylist[1].service=lbservicecapabilitylist[1].capabilitytype=SupportedLbIsolation
 servicecapabilitylist[1].capabilityvalue=dedicatedavailability=Optionalegresspolicy=ALLOWstate=Creatingstatus=Creatingallocationstate=CreatingsupportedServices=Vpn,Dhcp,Dns,Firewall,Lb,UserData,SourceNat,StaticNat,PortForwardingspecifyIpRanges=falsespecifyVlan=falseisPersistent=falseconservemode=falseserviceProviderList[0].service=VpnserviceProviderList[0].provider=VirtualRouterserviceProviderList[1].service=DhcpserviceProviderList[1].provider=VirtualRouterserviceProviderList[2].service=DnsserviceProviderList[2].provider=VirtualRouterserviceProviderList[3].service=FirewallserviceProviderList[3].provider=VirtualRouterserviceProviderList[4].service=LbserviceProviderList[4].provider=VirtualRouterserviceProviderList[5].service=UserDataserviceProviderList[5].provider=VirtualRouterserviceProviderList[6].service=SourceNatserviceProviderList[6].provider=JuniperSRXserviceProviderList[7].service=StaticNatserviceProviderList[7].provider=JuniperSRXserviceProviderList[8].service=PortForwardingserviceProviderList[8].provider=JuniperSRXegressdefaultpolicy=truetraffictype=GUEST_=1393925230248
 
 
 
 On 04-Mar-2014, at 2:30 am, Marcus shadow...@gmail.com wrote:
 
 Along these lines, the details parameter in deployVirtualMachine seems
 broken. If I call details[0].key=foo,details[0].value=bar, it stores
 entries in the database like this:
 
 id | vmid | name | value | display
 
 12 | 25   |  value | bar   | 1
 13 | 25   |  key   | foo   | 1
 
 It seems as though this might be correct per Alena's email, but I
 don't understand how this can be reconstructed into foo=bar when
 there's no binding between the two rows. Perhaps details are supposed
 to be passed differently than the resource tags, because if I do
 details[0].foo=bardetails[1].baz=12, I get:
 
 id | vmid | name | value | display
 
 12 | 25   |  foo| bar| 1
 13 | 25   |  baz   | 12 | 1
 
 And indeed there is code utilizing these details already

Re: [PROPOSAL][QUESTION] Map parameters in API Commands

2014-03-04 Thread Bharat Kumar

Hi ALL,
There are many other APIs that use Map like createNetworkOffering, updateZone, 
createTemplate, in most of the cases we do not
say how to use maps, one way would be to write this in the description or to 
use the same way to access maps of all APIs.

BTW the way to use details in deploy vm API is 
details[0].foo=bardetails[1].baz=12 where foo and baz are keys.

Also  if we want to use the regix protected static final String 
MAP_KEY_PATTERN_EXPRESSION = 
^([^\\[^\\]]+)\\[(\\d+)\\]\\.key$smb://[(//d+)//]//.key$;
   protected static final 
String MAP_VALUE_PATTERN_EXPRESSION = ^[^\\[^\\]]+\\[\\d+\\]\\.value$”;

wil this work in the following case. I believe service is the key here which 
repeats.
http://10.147.59.119:8080/client/api?command=createNetworkOfferingresponse=jsonsessionkey=/kGFJDXFmMQU8JZnnC7QFfj3tV8=name=bharatdisplayText=bharatguestIpType=IsolatedlbType=publicLb;
servicecapabilitylist[0].service=SourceNatservicecapabilitylist[0].capabilitytype=SupportedSourceNatTypes
servicecapabilitylist[0].capabilityvalue=peraccount
servicecapabilitylist[1].service=lbservicecapabilitylist[1].capabilitytype=SupportedLbIsolation
servicecapabilitylist[1].capabilityvalue=dedicatedavailability=Optionalegresspolicy=ALLOWstate=Creatingstatus=Creatingallocationstate=CreatingsupportedServices=Vpn,Dhcp,Dns,Firewall,Lb,UserData,SourceNat,StaticNat,PortForwardingspecifyIpRanges=falsespecifyVlan=falseisPersistent=falseconservemode=falseserviceProviderList[0].service=VpnserviceProviderList[0].provider=VirtualRouterserviceProviderList[1].service=DhcpserviceProviderList[1].provider=VirtualRouterserviceProviderList[2].service=DnsserviceProviderList[2].provider=VirtualRouterserviceProviderList[3].service=FirewallserviceProviderList[3].provider=VirtualRouterserviceProviderList[4].service=LbserviceProviderList[4].provider=VirtualRouterserviceProviderList[5].service=UserDataserviceProviderList[5].provider=VirtualRouterserviceProviderList[6].service=SourceNatserviceProviderList[6].provider=JuniperSRXserviceProviderList[7].service=StaticNatserviceProviderList[7].provider=JuniperSRXserviceProviderList[8].service=PortForwardingserviceProviderList[8].provider=JuniperSRXegressdefaultpolicy=truetraffictype=GUEST_=1393925230248



On 04-Mar-2014, at 2:30 am, Marcus 
shadow...@gmail.commailto:shadow...@gmail.com wrote:

Along these lines, the details parameter in deployVirtualMachine seems
broken. If I call details[0].key=foo,details[0].value=bar, it stores
entries in the database like this:

id | vmid | name | value | display

12 | 25   |  value | bar   | 1
13 | 25   |  key   | foo   | 1

It seems as though this might be correct per Alena's email, but I
don't understand how this can be reconstructed into foo=bar when
there's no binding between the two rows. Perhaps details are supposed
to be passed differently than the resource tags, because if I do
details[0].foo=bardetails[1].baz=12, I get:

id | vmid | name | value | display

12 | 25   |  foo| bar| 1
13 | 25   |  baz   | 12 | 1

And indeed there is code utilizing these details already committed
that expects this format, as deployVirtualMachines getDetails() only
seems to pass a correct MapString, String with Key, Value if I use
this format.

On Mon, Mar 3, 2014 at 11:48 AM, Antonio Fornié Casarrubios
antonio.for...@gmail.commailto:antonio.for...@gmail.com wrote:
Hi Alena,

Of course, the API will not have any changes. This is not a functional
change, just some refactoring. The problem is there are many things in CS
that really need some refactoring otherwise the problem will continue
growing more and more, but doing the change and above all making sure it
all works afterwards is not simple.

I will make sure that everything works exactly the same way and that the
data returned is also the same.

Thanks. Cheers
Antonio


2014-03-03 18:48 GMT+01:00 Alena Prokharchyk 
alena.prokharc...@citrix.commailto:alena.prokharc...@citrix.com:

Antonio, sure I will review the patch. But please make sure that API
backwards compatibly is intact, otherwise the fix won¹t be accepted.

-Alena.


On 3/2/14, 4:31 PM, Antonio Fornié Casarrubios
antonio.for...@gmail.commailto:antonio.for...@gmail.com wrote:

Hi Alena,

The reasons for this strange format? I don't know. There doesn't seem to
be
one. After asking on my team and in the dev list I thought perhaps you
could know. It seems we all see it strange and nobody knows why. But of
course, if it is for reasons I will stop the change.



And about the DB, you are right, in the DB is not like I said. But you can
have this in a table row field:
{0={value=Toronto,key=City}}
for some tables. I think there are two cases:

1- params in wich the get method fixes the params on the fly. In these of
course the strange format is not propagated anymore. But this is still
wrong: the format itself before the get is invoked, the time spent on
fixing

Review Request 18571: live migration is failing for vm deployed using dynaic compute offerings

2014-02-27 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18571/
---

Review request for cloudstack and Kishan Kavala.


Bugs: CLOUDSTACK-6099
https://issues.apache.org/jira/browse/CLOUDSTACK-6099


Repository: cloudstack-git


Description
---

CLOUDSTACK-6099 live migration is failing for vm deployed using dynaic compute 
offerings
https://issues.apache.org/jira/browse/CLOUDSTACK-6099


Diffs
-

  server/src/com/cloud/hypervisor/HypervisorGuruBase.java e504e81 

Diff: https://reviews.apache.org/r/18571/diff/


Testing
---

TESTED ON 4.3


Thanks,

bharat kumar

Re: Review Request 18571: live migration is failing for vm deployed using dynaic compute offerings

2014-02-27 Thread bharat kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18571/
---

(Updated Feb. 27, 2014, 12:13 p.m.)


Review request for cloudstack and Kishan Kavala.


Changes
---

this is the parent diff 


Bugs: CLOUDSTACK-6099
https://issues.apache.org/jira/browse/CLOUDSTACK-6099


Repository: cloudstack-git


Description
---

CLOUDSTACK-6099 live migration is failing for vm deployed using dynaic compute 
offerings
https://issues.apache.org/jira/browse/CLOUDSTACK-6099


Diffs
-

  server/src/com/cloud/hypervisor/HypervisorGuruBase.java e504e81 

Diff: https://reviews.apache.org/r/18571/diff/


Testing
---

TESTED ON 4.3


File Attachments (updated)


parent diff
  
https://reviews.apache.org/media/uploaded/files/2014/02/27/237012ce-e1a5-4852-9a2e-93927e7e60bd__0001-CLOUDSTACK-60993-fix-migration-when-using-custom-com.patch


Thanks,

bharat kumar

Re: KVM memory overprovision breaking me

2014-01-28 Thread Bharat Kumar

Hi Marcus, 

if you want to get what is assigned in the service offering then we can simply 
disable the overcommit feature.

Thanks,
Bharat.

On 28-Jan-2014, at 12:37 pm, Marcus Sorensen shadow...@gmail.com wrote:

 They're both very specific. As mentioned, ballooning only works if
 you're basically just making vms for yourself. And even then you have
 to add your own script to make it work, hence my suggestion to enable
 it via agent.properties.
 
 Again, my main concern is that it messes with the service offering,
 you don't get what you see in the offering. It's downright painful for
 system vms.
 
 On Mon, Jan 27, 2014 at 11:44 PM, Bharat Kumar bharat.ku...@citrix.com 
 wrote:
 Hi Marcus,
 
 KSM or memory de-duplication on KVM can only be used when the memory pages 
 are identical.
 IMO this is a huge constraint which is true only for specific use cases. 
 using KSM to implement overprovisioning
 will limit this feature to a specific use case and hence memory ballooning 
 was chosen which is more generic.
 
 Thanks,
 Bharat.
 
 On 28-Jan-2014, at 11:58 am, Marcus Sorensen shadow...@gmail.com wrote:
 
 Yeah, I'm a little disappointed that the functional spec doesn't
 really address memory deduplication, which is the real version of
 overcommit, IMO.  Since it looks like the feature is already fully
 implemented, I'm not sure I have much of a leg to stand on in trying
 to change it. I'll just patch it out of our builds. Thanks for the
 input.
 
 On Mon, Jan 27, 2014 at 11:13 PM, Bharat Kumar bharat.ku...@citrix.com 
 wrote:
 Hi Marcus,
 
 in case of KVM the guest memory is not dynamically adjusted by hypervisor, 
 this is a hypervisor limitation.  we have documented this in the FS in 
 prerequisites for KVM.
 https://cwiki.apache.org/confluence/display/CLOUDSTACK/CPU+and+RAM+Overcommit
 
 One way to make this work automatically is to use a script to monitor the 
 memory pressure in the guest VM and adjust the memory using cgroups.
 one such script is Memory over commit manager.
 more info here 
 http://aglitke.wordpress.com/2011/03/03/automatic-memory-ballooning-with-mom/
 
 Thanks,
 Bharat.
 
 
 
 On 28-Jan-2014, at 11:23 am, Marcus Sorensen 
 shadow...@gmail.commailto:shadow...@gmail.com wrote:
 
 Yeah, that's not overprovisioning, though. That's scaling. The way
 it's implemented now is just ... provisioning. It allocates exactly
 what's available at the host.
 
 Also, the vm in your example can't go to 4GB. There is nothing that
 changes it's 'currentMemory' setting. Without a scaling feature it's
 simply always stuck at 2GB.
 
 On Mon, Jan 27, 2014 at 10:32 PM, Harikrishna Patnala
 harikrishna.patn...@citrix.commailto:harikrishna.patn...@citrix.com 
 wrote:
 Hi,
 I think the way it was done is to guarantee minimum memory to that VM and 
 upon demand it can get upto the memory defined in service offering.
 Say a vm with service offering 4Gb is deployed with overprovisioing factor 
 2, we guarantee that vm should get minimum of 2GB (4GB/2) and if that VM 
 is overloaded then it can get memory unto max 4GB.
 This is what it is showing vm definition
 memory unit='KiB’4GB/memory
 currentMemory unit='KiB’2GB/currentMemory
 memory unit” is what it can get maximum.
 
 -Harikrishna
 
 
 On 28-Jan-2014, at 7:46 am, Marcus Sorensen 
 shadow...@gmail.commailto:shadow...@gmail.com wrote:
 
 Its an easy fix on the KVM side, just waiting to hear any objections.
 On Jan 27, 2014 6:11 PM, Nux! n...@li.nux.romailto:n...@li.nux.ro 
 wrote:
 
 On 28.01.2014 00:49, Marcus Sorensen wrote:
 
 So... I tried to use memory overcommit on KVM this week, and it blew
 up in my face. Apparently it's configured such that if I have a
 Service Offering of 4G, and I set memory overprovisioning to 2:1, the
 guest only actually gets configured with 2G. That's not how
 overprovisioning is supposed to work, IMO.
 
 Here's a vm definition with a 3:1 mem overprovision setting, which
 ensures that system vms don't work:
 
 memory unit='KiB'262144/memory
 currentMemory unit='KiB'87040/currentMemory
 
 Note currentMemory needs to be manually tuned if I ever want the vm to
 use/see more. This is more for live scaling (which is also broken
 because the guest could just rmmod virtio-balloon and see everything).
 
 I'd like to just rip out the code that is setting ballooning feature
 based on overprovisioning factor, but perhaps there was a reason this
 was done. From my point of view, if I give someone a service offering
 that says 4G, it should provide 4G, and if I can do memory
 deduplication on the backend to overprovision that's up to me to do.
 Overprovisioning should not be a divider on all service offerings.
 
 
 Wow! I also thought, heck, KSM  thin qcows for the win! If
 overprovisioning really works as you described then it can't possibly be
 used for any commercial offering ...
 This needs to get fixed.. Too late to see this in 4.3?
 
 --
 Sent from the Delta quadrant using Borg technology!
 
 Nux!
 www.nux.rohttp://www.nux.ro

1 2 3 4 >

1 - 100 of 344 matches

Mail list logo