Re: [openstack-dev] [tripleo] pingtest vs tempest

On Thu, Apr 6, 2017 at 12:29 PM, Justin Kilpatrick wrote: > Maybe I'm getting a little off topic with this question, but why was > Tempest removed last time? > > I'm not well versed in the history of this discussion, but from what I > understand Tempest in the gate has > been

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Tue, Sep 5, 2017 at 11:35 PM, Giulio Fidente wrote: [...] > slightly OT but notable mention, scenario001/container is also the one > gating Ceph via ceph-ansible currently :D > > ... soon to convert scenario004/container as well Really good work Giulio and John to make

Re: [openstack-dev] [tripleo] pingtest vs tempest

On 09/05/2017 09:26 PM, Emilien Macchi wrote: > On Wed, Apr 5, 2017 at 1:49 PM, Emilien Macchi wrote: > [...] > >> == Solutions >> >> 1) Switch pingtest to Tempest run on some specific tests, with feature >> parity of what we had with pingtest. >> For example, we could

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Wed, Apr 5, 2017 at 1:49 PM, Emilien Macchi wrote: [...] > == Solutions > > 1) Switch pingtest to Tempest run on some specific tests, with feature > parity of what we had with pingtest. > For example, we could imagine to run the scenarios that deploys VM and > boot from

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Thursday, 4 May 2017 20:11:14 CEST Emilien Macchi wrote: > On Thu, May 4, 2017 at 9:41 AM, Dan Prince wrote: > > I like the idea of getting pingtest out of tripleo.sh as more of a > > stand alone tool. I would support an effort that re-implemented it... > > and using

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Thu, May 4, 2017 at 7:13 PM Emilien Macchi wrote: > On Thu, May 4, 2017 at 9:41 AM, Dan Prince wrote: > > On Thu, 2017-05-04 at 03:11 -0400, Luigi Toscano wrote: > >> - Original Message - > >> > On Wed, 2017-05-03 at 17:53 -0400, Emilien Macchi

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Thu, May 4, 2017 at 11:11 PM Dan Prince wrote: > On Thu, 2017-05-04 at 14:11 -0400, Emilien Macchi wrote: > > On Thu, May 4, 2017 at 9:41 AM, Dan Prince > > wrote: > > > On Thu, 2017-05-04 at 03:11 -0400, Luigi Toscano wrote: > > > > - Original

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Thu, 2017-05-04 at 14:11 -0400, Emilien Macchi wrote: > On Thu, May 4, 2017 at 9:41 AM, Dan Prince > wrote: > > On Thu, 2017-05-04 at 03:11 -0400, Luigi Toscano wrote: > > > - Original Message - > > > > On Wed, 2017-05-03 at 17:53 -0400, Emilien Macchi wrote: > > >

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Thu, May 4, 2017 at 9:41 AM, Dan Prince wrote: > On Thu, 2017-05-04 at 03:11 -0400, Luigi Toscano wrote: >> - Original Message - >> > On Wed, 2017-05-03 at 17:53 -0400, Emilien Macchi wrote: >> > > (cross-posting) >> > >> > > Instead of running the Pingtest, we

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Thursday, 4 May 2017 15:41:04 CEST Dan Prince wrote: > On Thu, 2017-05-04 at 03:11 -0400, Luigi Toscano wrote: > > - Original Message - > > > > > > > > Running a subset of Tempest tests isn't the same thing as designing > > > (and owning) your own test suite that targets the things

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Thu, 2017-05-04 at 03:11 -0400, Luigi Toscano wrote: > - Original Message - > > On Wed, 2017-05-03 at 17:53 -0400, Emilien Macchi wrote: > > > (cross-posting) > > > > > Instead of running the Pingtest, we would execute a Tempest > > > Scenario > > > that boot an instance from volume

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Thu, May 4, 2017 at 5:34 PM, Arx Cruz wrote: > > > On Wed, May 3, 2017 at 11:53 PM, Emilien Macchi wrote: >> >> (cross-posting) >> >> I've seen a bunch of interesting thoughts here. >> The most relevant feedback I've seen so far: >> >> - TripleO folks

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Wed, May 3, 2017 at 11:53 PM, Emilien Macchi wrote: > (cross-posting) > > I've seen a bunch of interesting thoughts here. > The most relevant feedback I've seen so far: > > - TripleO folks want to keep testing fast and efficient. > - Tempest folks understand this

Re: [openstack-dev] [tripleo] pingtest vs tempest

- Original Message - > On Wed, 2017-05-03 at 17:53 -0400, Emilien Macchi wrote: > > (cross-posting) > > > Instead of running the Pingtest, we would execute a Tempest Scenario > > that boot an instance from volume (like Pingstest is already doing) > > and see how it goes (in term of

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Wed, 2017-05-03 at 17:53 -0400, Emilien Macchi wrote: > (cross-posting) > > I've seen a bunch of interesting thoughts here. > The most relevant feedback I've seen so far: > > - TripleO folks want to keep testing fast and efficient. > - Tempest folks understand this problematic and is willing

Re: [openstack-dev] [tripleo] pingtest vs tempest

(cross-posting) I've seen a bunch of interesting thoughts here. The most relevant feedback I've seen so far: - TripleO folks want to keep testing fast and efficient. - Tempest folks understand this problematic and is willing to collaborate. I propose that we move forward and experiment the

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Tue, Apr 18, 2017 at 11:04 AM, Arx Cruz wrote: > > > On Tue, Apr 18, 2017 at 10:42 AM, Steven Hardy wrote: > >> On Mon, Apr 17, 2017 at 12:48:32PM -0400, Justin Kilpatrick wrote: >> > On Mon, Apr 17, 2017 at 12:28 PM, Ben Nemec

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Monday, 17 April 2017 18:28:24 CEST Ben Nemec wrote: > On 04/17/2017 10:51 AM, Emilien Macchi wrote: > > We haven't got much feedback from TripleO core reviewers, who are > > usually more involved on this topic. I'll give a chance to let them > > talk because we take some actions based on the

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Tue, Apr 18, 2017 at 10:07 AM Arx Cruz wrote: > On Tue, Apr 18, 2017 at 10:42 AM, Steven Hardy wrote: > >> On Mon, Apr 17, 2017 at 12:48:32PM -0400, Justin Kilpatrick wrote: >> > On Mon, Apr 17, 2017 at 12:28 PM, Ben Nemec >>

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Tue, Apr 18, 2017 at 10:42 AM, Steven Hardy wrote: > On Mon, Apr 17, 2017 at 12:48:32PM -0400, Justin Kilpatrick wrote: > > On Mon, Apr 17, 2017 at 12:28 PM, Ben Nemec > wrote: > > > Tempest isn't really either of those things. According to another

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Mon, Apr 17, 2017 at 12:48:32PM -0400, Justin Kilpatrick wrote: > On Mon, Apr 17, 2017 at 12:28 PM, Ben Nemec wrote: > > Tempest isn't really either of those things. According to another message > > in this thread it takes around 15 minutes to run just the smoke tests.

Re: [openstack-dev] [tripleo] pingtest vs tempest

p again, if it is done today wil ping you and >>> schedule again >>> >>> Otherwise i will schedule for Monday >>> >>> Thanks, >>> Ravi >>> - Original Message - >>> From: jtale...@redhat.com >>> To: openstack-dev

Re: [openstack-dev] [tripleo] pingtest vs tempest

iginal Message - From: jtale...@redhat.com To: openstack-dev@lists.openstack.org, jkilp...@redhat.com Sent: Thursday, April 6, 2017 4:15:58 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: Re: [openstack-dev] [tripleo] pingtest vs tempest On Thu, Apr 6, 2017 at 5:32 AM, Sagi Shnaidman

Re: [openstack-dev] [tripleo] pingtest vs tempest

gt; - Original Message - > From: jtale...@redhat.com > To: openstack-dev@lists.openstack.org, jkilp...@redhat.com > Sent: Thursday, April 6, 2017 4:15:58 PM GMT +05:30 Chennai, Kolkata, Mumbai, > New Delhi > Subject: Re: [openstack-dev] [tripleo] pingtest vs tempest >

Re: [openstack-dev] [tripleo] pingtest vs tempest

@lists.openstack.org, jkilp...@redhat.com Sent: Thursday, April 6, 2017 4:15:58 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: Re: [openstack-dev] [tripleo] pingtest vs tempest On Thu, Apr 6, 2017 at 5:32 AM, Sagi Shnaidman <sshna...@redhat.com> wrote: > HI, > > I think Ra

Re: [openstack-dev] [tripleo] pingtest vs tempest

On 06/04/17 12:32 +0300, Sagi Shnaidman wrote: HI, I think Rally or Browbeat and other performance oriented solutions won't serve our needs, because we run TripleO CI on virtualized environment with very limited resources. Actually we are pretty close to full utilizing these resources when

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Wed, Apr 5, 2017 at 4:49 PM, Emilien Macchi wrote: > Greetings dear owls, > > I would like to bring back an old topic: running tempest in the gate. > > == Context > > Right now, TripleO gate is running something called pingtest to > validate that the OpenStack cloud is

Re: [openstack-dev] [tripleo] pingtest vs tempest

I would say smoke tests, and at least the minimum scenario tests. Smoke tests takes 14 minutes (113 tests) to run, and I can check how long it takes the minimum scenario tests later. So it won't take a long time running. Kind regards, Arx Cruz On Thu, Apr 6, 2017 at 2:44 PM, Andrea Frittoli

Re: [openstack-dev] [tripleo] pingtest vs tempest

I don't really have much context in what the decision is going to be based on here, so I'll just add some random comments here and there. On Thu, Apr 6, 2017 at 12:48 PM Arx Cruz wrote: > Having tempest running will allow these jobs to appear in openstack-health > system as

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Thursday, 6 April 2017 13:29:32 CEST Justin Kilpatrick wrote: > Maybe I'm getting a little off topic with this question, but why was > Tempest removed last time? > > I'm not well versed in the history of this discussion, but from what I > understand Tempest in the gate has > been an off and on

Re: [openstack-dev] [tripleo] pingtest vs tempest

Having tempest running will allow these jobs to appear in openstack-health system as well. On Thu, Apr 6, 2017 at 1:29 PM, Justin Kilpatrick wrote: > Maybe I'm getting a little off topic with this question, but why was > Tempest removed last time? > > I'm not well versed in

Re: [openstack-dev] [tripleo] pingtest vs tempest

Maybe I'm getting a little off topic with this question, but why was Tempest removed last time? I'm not well versed in the history of this discussion, but from what I understand Tempest in the gate has been an off and on again thing for a while but I've never heard the story of why it got

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Thu, 6 Apr 2017, Sagi Shnaidman wrote: It may be useful to run a "limited edition" of API tests that maximize coverage and don't duplicate, for example just to check service working basically, without covering all its functionality. It will take very little time (i.e. 5 tests for each

Re: [openstack-dev] [tripleo] pingtest vs tempest

Sagi, I think Rally or Browbeat and other performance oriented solutions won't > serve our needs, because we run TripleO CI on virtualized environment with > very limited resources. Actually we are pretty close to full utilizing > these resources when deploying openstack, so very little is

Re: [openstack-dev] [tripleo] pingtest vs tempest

On 04/05/2017 10:49 PM, Emilien Macchi wrote: Greetings dear owls, I would like to bring back an old topic: running tempest in the gate. == Context Right now, TripleO gate is running something called pingtest to validate that the OpenStack cloud is working. It's an Heat stack, that deploys a

Re: [openstack-dev] [tripleo] pingtest vs tempest

HI, I think Rally or Browbeat and other performance oriented solutions won't serve our needs, because we run TripleO CI on virtualized environment with very limited resources. Actually we are pretty close to full utilizing these resources when deploying openstack, so very little is available for

Re: [openstack-dev] [tripleo] pingtest vs tempest

On Wed, Apr 5, 2017 at 4:49 PM, Emilien Macchi wrote: > Greetings dear owls, > > I would like to bring back an old topic: running tempest in the gate. > > == Context > > Right now, TripleO gate is running something called pingtest to > validate that the OpenStack cloud is

[openstack-dev] [tripleo] pingtest vs tempest

Greetings dear owls, I would like to bring back an old topic: running tempest in the gate. == Context Right now, TripleO gate is running something called pingtest to validate that the OpenStack cloud is working. It's an Heat stack, that deploys a Nova server, some volumes, a glance image, a