Hello QAers,
Just a quick note to say next week's IRC meeting (Thursdays at 17:00
UTC) will focus on planning the QA track at the upcoming Havana design
summit.
Please do spend a few cycles this week proposing or assisting with
planning design summit proposals.
Note that you can submit session i
.com.
Just kiddin. You can put me email in there :) jaypi...@gmail.com.
Best,
-jay
> Cheers,
> stef
>
> On Fri 02 Nov 2012 04:37:18 PM CET, Jay Pipes wrote:
>> On 11/01/2012 04:49 PM, David Kranz wrote:
>>> There is now a full tempest run going daily and reporting fail
On 11/01/2012 04:49 PM, David Kranz wrote:
> There is now a full tempest run going daily and reporting failures to
> this list. But that won't work because
> jenkins and gerrit cannot be launchpad members. According to the ci
> folks, others have dealt with this
> by moving their mailing lists to
On 11/01/2012 03:45 PM, Sean Dague wrote:
> As we start to file QA blueprints, which of these is the right place to
> do them in:
>
> https://blueprints.launchpad.net/openstack-qa or
No, this is the QA documentation.
> https://blueprints.launchpad.net/tempest or is there some 3rd place we
> s
On 10/25/2012 01:13 AM, Daryl Walleck wrote:
> While spinning up a new devstack tonight I noticed some very odd
> behavior. Keystone is suddenly giving me back a 3000+ character auth
> token, and the ids for flavors I'm creating are extremely large ints
> (uuids I could see, but not this). Does any
gt; I wonder what is the correct git/gerrit workflow for submitting a change
> based on someone else rejected/expired commit ?
> I would like to mark correctly the original code base.
>
> Best Regards,
> Attila
>
> ----- Original Message -
> From: "Jay Pipes&qu
Approved the blueprint.
Daryl, feel free to create work items (there is a section in the
blueprint for them) for the blueprint. I'll contribute some as well.
Best,
-jay
On 10/17/2012 02:11 AM, Daryl Walleck wrote:
> Hi everyone,
>
> After our discussions earlier today about being more vocal abo
t different
> categories of tests are -- smoke, fuzz, positive/negative, etc -- and
> put this up on the wiki - Sam from my team will be working on this first
> draft as agreed upon at the meeting
Yep, saw that and now remembered.
Best,
-jay
> Gigi
>
>
>
>
> On 10/22/
On 10/22/2012 02:10 PM, Gigi Geoffrion wrote:
> Hi Jay -
>
> Thanks for sending this update - My updates in Red below on a few
> things :)
Hmm, well, plain/text email doesn't have colors, but... see answers
inline below :)
> Great seeing you all who made the meeting that day!
Indeed, great to
On 10/22/2012 12:41 PM, Yaniv Kaul wrote:
>> 1) Tempest executes a series of HTTP calls against public REST endpoints
>> in OpenStack. It has no way of determining what code was run **on the
>> server**. It only has the ability to know what Tempest itself executed,
>> not what percentage of the tot
On 10/22/2012 12:41 PM, Yaniv Kaul wrote:
> Ok - although it's not very well documented -
> http://testtools.readthedocs.org/en/latest/py-modindex.html
http://mumak.net/testtools/apidocs/
--
Mailing list: https://launchpad.net/~openstack-qa-team
Post to : openstack-qa-team@lists.launchpad.ne
Hi Yaniv, answers inline...
On 10/22/2012 11:41 AM, Yaniv Kaul wrote:
> On 10/22/2012 05:33 PM, Jay Pipes wrote:
>> Hi Sean :)
>>
>> Here's a quick recap:
>>
>> We agreed:
>>
>> * nosetests just isn't a good foundation for our work -- e
On 10/22/2012 10:36 AM, Attila Fazekas wrote:
> Hi everyone,
>
> I am considering implementing test cases for using the EC2 and S3 API in
> tempest.
> I would like to know is anybody else working on this kind of tests ?
Hi! :) It's old, but Chuck Short (cc'd) once gave this a go and the code
rev
Hi Sean :)
Here's a quick recap:
We agreed:
* nosetests just isn't a good foundation for our work -- especially
regarding performance/parallelism
* We need to produce good, updated documentation on what different
categories of tests are -- smoke, fuzz, positive/negative, etc -- and
put this up o
empest.tests.compute.test_server_basic_ops.TestServerBasicOps) ... ok
> test_008_resume_server
> (tempest.tests.compute.test_server_basic_ops.TestServerBasicOps) ... ok
> test_099_terminate_instance
> (tempest.tests.compute.test_server_basic_ops.TestServerBasicOps) ... ok
>
>
>
> On 9/28/201
-os-password, etc?
>>
>> https://github.com/openstack/python-keystoneclient/commit/641f6123624b6ac89182c303dfcb0459b28055a2
>>
>>
>> -Dolph
>>
>>
>> On Tue, Sep 25, 2012 at 9:35 AM, Jay Pipes > <mailto:jaypi...@gmail.com>> wrote:
>>
On 09/25/2012 09:38 AM, David Kranz wrote:
> I heard from some of my team members that test_server_basic_ops and
> test_server_advanced_ops were failing and I can reproduce it with
> current devstack/tempest.
> Looking at the code it seems that the keystone Client object does not
> have a servic
On 09/14/2012 06:51 AM, Jaroslav Henner wrote:
> I think unittests may be relatively easily split between several Jenkins
> jobs just by specifying which packages to run. Following might be
> executed in different Jobs:
>
> nose -w PATH_TO_TEMPEST tempest.tests.compute
> nose -w PATH_TO_TEMPES
On 09/13/2012 06:43 AM, Sean Dague wrote:
> On 09/13/2012 12:42 AM, Vishvananda Ishaya wrote:
>> Brian Waldon and I discussed this today. We came up with an idea which is to
>> have the tempest run on stable/ during the dev cycle, and
>> after the last milestone turn it on during the bugfixing ph
On 09/07/2012 08:13 PM, Dan Smith wrote:
> DW> We also have a different problem with running tests in parallel
> DW> now. None of the newly designed basic/advanced ops test can be run
> DW> in parallel given their dependency between tests. The only way I can
> DW> think of to proceed would be to re
On 09/06/2012 02:21 PM, David Kranz wrote:
> Do we have a policy about whether bug tickets are needed for every
> change? I happened to see a silly coding error and would prefer to avoid
> the overhead of a bug ticket for such things.
Silly coding errors/typos/style cleanups do not need a bug. B
On 09/05/2012 09:03 PM, James E. Blair wrote:
> Here's what I'd like to propose instead:
>
> 1) Make the tempest gate symmetrical -- cross-project gating works best
> when all the projects gate on the same tests so that one project can't
> break another.
Hmm. Well, this really is just circumventi
On 08/29/2012 09:15 AM, David Kranz wrote:
> On 8/29/2012 8:41 AM, Jaroslav Henner wrote:
>> Hi.
>>
>> I made my first little patch:
>> https://review.openstack.org/#/c/12048/
>>
>> which failed 3 times on
>> http://logs.openstack.org/12048/1/check/gate-tempest-devstack-vm/9570
>>
>> which surely i
On 08/21/2012 05:45 PM, Dan Smith wrote:
> In other suites, I've seen an XFAIL result used to mark tests that we
> know are failing right now so that they're not SKIPped like tests that
> are missing some component, but rather just not fatal to the task at
> hand. Maybe something like that would be
Sorry for late reply... in Bulgaria on vacation until 29th... Comments
inline.
On 08/16/2012 08:13 PM, David Kranz wrote:
> I think this clearly illustrates why it is so important for QA to be
> done more in the open, and to post tempest blueprints for largish
> projects like this. I have a theo
On 07/09/2012 10:09 AM, John Garbutt wrote:
>>> - historically XenAPI had migrate, Libvirt had live migrate
>>> - But by end of Folsom we should have both having both
>>
>> Yes, but what is the difference between the two?
> Got you. I think this is right:
>
> Migration:
> - shutdown the VM
> - mov
On 07/10/2012 10:20 AM, David Kranz wrote:
> I just submitted a removal of the skips. Note that there were two user
> role tests that still failed in a way I could not decipher so I filed a
> tempest bug https://bugs.launchpad.net/tempest/+bug/1022990.
> Unfortunately, when the skip script went in
On 07/09/2012 09:35 AM, John Garbutt wrote:
> Renuka and I are looking at proposing a blueprint to try and sort out some of
> this confusion.
>
> I think this is the current situation:
> - historically XenAPI had migrate, Libvirt had live migrate
> - But by end of Folsom we should have both havin
Hey all,
I went ahead and created a script that you can run to see the status of
any Launchpad bugs that are causing Tempest tests to be skipped:
https://review.openstack.org/#/c/9503
It's output looks like this:
jpipes@uberbox:~/repos/tempest$ python tools/skip_tracker.py
INFO: Total bug skips
On 07/06/2012 05:34 AM, John Garbutt wrote:
> Hi,
>
> I am in the process of adding Live Migration support into the XenAPI driver.
> Are there any current plans on how to test Live Migration, is it done already?
> I was thinking of copying what has been done for Migration.
cc'ing Anne Gentle beca
Hi all,
Firstly, apologies to Sapan Kona. Sapan, I'm going to be using a recent
proposed set of tests from you as an example in this mailing list post.
There have been many similar patches from many authors proposed and
already accepted into Tempest. I'm using Sapan's patch as an example,
not
QAers,
There has been a pile-up of code reviews recently, and I wanted to
explain a decision that a number of core QA team members reached last
week and why some patchsets have not been reviewed.
Two of the goals of Tempest are to have a functional integration test
suite that stresses *diffe
make sure, since the semantics and tenses of
the power, VM, and task states are a bit inconsistent.
Best,
-jay
Original message
Subject: Re: [Openstack-qa-team] wait_for_server_status and Compute API
From: Jay Pipes
To: "openstack-qa-team@lists.launchpad.net
On 06/18/2012 12:01 PM, David Kranz wrote:
There are a few tempest tests, and many in the old kong suite that is
still there, that wait for a server status that is something other than
ACTIVE or VERIFY_RESIZE. These other states, such as BUILD or REBOOT,
are transient so I don't understand why it
On 06/12/2012 02:22 PM, Daryl Walleck wrote:
Due to the large number of input fuzzing tests that have been submitted, I've
been thinking of ways to reduce the amount of code needed to achieve this
(whether we should do it or not is a totally different discussion). Rather than
have x number of
On 05/29/2012 09:59 PM, Daryl Walleck wrote:
On May 29, 2012, at 6:46 PM, Jay Pipes wrote:
On 05/29/2012 07:28 PM, Daryl Walleck wrote:
Well, if by complete failure you mean not configured, you're correct.
:-) The cleanup solution has several parallel solutions in progress, and
should
On 05/29/2012 07:28 PM, Daryl Walleck wrote:
Well, if by complete failure you mean not configured, you're correct.
:-) The cleanup solution has several parallel solutions in progress, and
should be a coding standard going forward.
Perhaps, but I haven't seen any of those...
> The quotas issue
On 05/29/2012 04:33 PM, David Kranz wrote:
Can any one say what the current status is of running in parallel reliably?
Sure. It doesn't work. :)
You first run into quota issues and then you run into resource cleanup
issues. It's a complete failure...
-jay
--
Mailing list: https://launchpad
The QA team had its weekly IRC meeting yesterday. Here is a summary of
the action items and decisions coming out of the meeting.
* Online summary:
http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-05-24-17.00.html
* Status of the project
We've more than DOU
Including JimB, MontyT, and DanP for some related CI needs...
On 05/25/2012 05:12 AM, John Garbutt wrote:
Hi,
An quick update on the Citrix XenServer CI efforts.
I have got some tests (based on DevStack) running tempest for trunk and
stable/essex with XenServer based DevStack deployments.
Aw
Thanks Dan!
On 05/23/2012 09:40 PM, Dan Prince wrote:
- Original Message -
From: "Jay Pipes"
To: openstack-qa-team@lists.launchpad.net
Sent: Wednesday, May 23, 2012 9:27:07 PM
Subject: [Openstack-qa-team] Single failed test case needs fixing...
deleting a flavor
All,
The weekly QA Team meeting takes place at 17:00 UTC on IRC
(#openstack-meeting on Freenode). We invite anyone interested in
testing, quality assurance and performance engineering to attend the
weekly meeting.
The agenda for this week is as follows:
* Review of last week's action items
- Sta
All, if anyone has a few spare cycles to look into this bug, I'd
appreciate it:
https://bugs.launchpad.net/tempest/+bug/1003741
It is reproduceable in the Tempest gate job...
Best,
-jay
--
Mailing list: https://launchpad.net/~openstack-qa-team
Post to : openstack-qa-team@lists.launchpad.n
On 05/03/2012 10:54 PM, Maru Newby wrote:
The rest api is the default interface, and the client tools target that
interface. Since the clients are cli more than python api, they can be
used by any language that can use a shell. What exactly does
reimplementing the clients for the sake of testing
On 05/03/2012 03:54 PM, Daryl Walleck wrote:
So my first question is around this. So is the claim is that the client
tools are the default interface for the applications?
Sorry, perhaps a better term would have been "the most common interface
to OpenStack Compute"...
> While that works
for
On 05/04/2012 09:28 AM, David Kranz wrote:
On 5/4/2012 9:08 AM, Jay Pipes wrote:
On 05/04/2012 06:14 AM, Karajgi, Rohit wrote:
Hi,
What is the policy that we should or are following for test cases that
fail due to an existing Open Bug in Launchpad?
For eg:
https://github.com/openstack
All,
A momentous event has occurred. We now have all Tempest integration
tests passing against the devstack-deployed test environment spun up in
the dev-gate-tempest-devstack-vm:
https://jenkins.openstack.org/job/dev-gate-tempest-devstack-vm/test/?width=800&height=600
Tempest is executing 15
On 05/04/2012 06:14 AM, Karajgi, Rohit wrote:
Hi,
What is the policy that we should or are following for test cases that
fail due to an existing Open Bug in Launchpad?
For eg:
https://github.com/openstack/tempest/blob/master/tempest/tests/test_list_floating_ips.py#L64
skips the test and posts t
Hi all,
I'd like to get some alignment on the following:
1) The definition of what is a "smoke" test
2) How we envision the Tempest project's role in running such tests
First, a discussion on #1.
There seem to be very loose semantics used to describe what a "smoke
test" is. Some groups use th
On 04/30/2012 04:30 PM, David Kranz wrote:
Does any one have any new items to add to the agenda for this week's
meeting?
So far we have:
1. Getting some consensus on what precisely a "smoke test" is and
discussing some code Jay will have pushed that cleans up our "smoke
test" abilities.
Yep,
On 04/30/2012 09:24 AM, Karajgi, Rohit wrote:
The coverage can be expanded quite a bit by having tests that perform certain
'white-box' actions such as updating DB entries, and restoring them at the end
of the test.
But most of these tests would be negative, but important, scenarios.
These shou
The QA team had its weekly IRC meeting yesterday. Here is a summary of
the action items and decisions coming out of the meeting.
* Progress
- Currently up to more than 150 integration tests
- The dev-gate-tempest-devstack-vm Jenkins job [1] has run smoothly
now for a number of iterations. T
On 04/02/2012 05:45 AM, John Garbutt wrote:
Hi
Thanks for the clarifications. Being hypervisor agnostic (as much of
possible) makes a lot of sense.
Where would the Jenkins tests for all this go? I guess in here:
https://github.com/openstack/openstack-ci
From what you say, to process to get f
On 03/23/2012 11:51 AM, David Kranz wrote:
I am getting the following failure on a new essex cluster using Ubuntu
packages. It seems 404 (NotFound) is expected by Tempest. Has any one
seen this? This test was passing fairly recently...
-David
On 02/25/2012 01:32 AM, Daryl Walleck wrote:
I've heard KVM/libvert support was added in Essex. I can confirm resize is
still working with Xen Server, so the issue must be with those implementations.
Yes, I've seen this happen as well -- and so has Nati IIRC.
-jay
--
Mailing list: https://la
Nice job, David! Keep up the good work. I approved that skipper for the
bug so we should be getting close now.
Going to get to the remaining reviews now.
Cheers,
-jay
On 02/24/2012 01:55 PM, David Kranz wrote:
One failure was a nova bug I reported and just posted this workaround:
https://revi
56 matches
Mail list logo
