> Sean Dague wrote:
>
> To be clear, the functional tests will not be Tempest tests. This is a
> different class of testing, it's really another tox target that needs a
> devstack to run. A really good initial transition would be things like
> the CLI testing.
>
> Also, the Tempest team has gon
On Mon, Jul 28, 2014 at 02:28:56PM +0200, Thierry Carrez wrote:
> James E. Blair wrote:
> > [...]
> > Most of these bugs are not failures of the test system; they are real
> > bugs. Many of them have even been in OpenStack for a long time, but are
> > only becoming visible now due to improvements
James E. Blair wrote:
> [...]
> Most of these bugs are not failures of the test system; they are real
> bugs. Many of them have even been in OpenStack for a long time, but are
> only becoming visible now due to improvements in our tests. That's not
> much help to developers whose patches are bein
On 07/24/2014 06:36 PM, John Dickinson wrote:
On Jul 24, 2014, at 3:25 PM, Sean Dague wrote:
On 07/24/2014 06:15 PM, Angus Salkeld wrote:
We do this in Solum and I really like it. It's nice for the same
reviewers to see the functional tests and the code the implements
a feature.
One downside
On 26 July 2014 08:20, Matthew Treinish wrote:
>> This is also more of a pragmatic organic approach to figuring out the
>> interfaces we need to lock down. When one projects breaks depending on
>> an interface in another project, that should trigger this kind of
>> contract growth, which hopefull
On Thu, Jul 24, 2014 at 06:54:38PM -0400, Sean Dague wrote:
> On 07/24/2014 05:57 PM, Matthew Treinish wrote:
> > On Wed, Jul 23, 2014 at 02:39:47PM -0700, James E. Blair wrote:
> >> OpenStack has a substantial CI system that is core to its development
> >> process. The goals of the system are to
On Thu, Jul 24, 2014 at 3:54 PM, Sean Dague wrote:
> On 07/24/2014 05:57 PM, Matthew Treinish wrote:
> > On Wed, Jul 23, 2014 at 02:39:47PM -0700, James E. Blair wrote:
> >> OpenStack has a substantial CI system that is core to its development
> >> process. The goals of the system are to facilit
On 25/07/14 11:18, Sean Dague wrote:
> On 07/25/2014 10:01 AM, Steven Hardy wrote:
>> On Wed, Jul 23, 2014 at 02:39:47PM -0700, James E. Blair wrote:
>>
>>> * Put the burden for a bunch of these tests back on the projects as
>>> "functional" tests. Basically a custom devstack environment tha
On 07/25/2014 10:01 AM, Steven Hardy wrote:
> On Wed, Jul 23, 2014 at 02:39:47PM -0700, James E. Blair wrote:
>
>> * Put the burden for a bunch of these tests back on the projects as
>> "functional" tests. Basically a custom devstack environment that a
>> project can create with a set of
On 07/25/2014 10:01 AM, Steven Hardy wrote:
On Wed, Jul 23, 2014 at 02:39:47PM -0700, James E. Blair wrote:
* Put the burden for a bunch of these tests back on the projects as
"functional" tests. Basically a custom devstack environment that a
project can create with a set of servic
On Wed, Jul 23, 2014 at 02:39:47PM -0700, James E. Blair wrote:
> * Put the burden for a bunch of these tests back on the projects as
> "functional" tests. Basically a custom devstack environment that a
> project can create with a set of services that they minimally need
> to do thei
On Thu, Jul 24, 2014 at 04:01:39PM -0400, Sean Dague wrote:
> On 07/24/2014 12:40 PM, Daniel P. Berrange wrote:
> > On Wed, Jul 23, 2014 at 02:39:47PM -0700, James E. Blair wrote:
> >
> >> ==Future changes==
> >
> >> ===Fixing Faster===
> >>
> >> We introduce bugs to OpenStack at some constant ra
On 07/24/2014 05:57 PM, Matthew Treinish wrote:
> On Wed, Jul 23, 2014 at 02:39:47PM -0700, James E. Blair wrote:
>> OpenStack has a substantial CI system that is core to its development
>> process. The goals of the system are to facilitate merging good code,
>> prevent regressions, and ensure tha
On Jul 24, 2014, at 3:25 PM, Sean Dague wrote:
> On 07/24/2014 06:15 PM, Angus Salkeld wrote:
>> On Wed, 2014-07-23 at 14:39 -0700, James E. Blair wrote:
>>> OpenStack has a substantial CI system that is core to its development
>>> process. The goals of the system are to facilitate merging good
On 07/24/2014 06:15 PM, Angus Salkeld wrote:
> On Wed, 2014-07-23 at 14:39 -0700, James E. Blair wrote:
>> OpenStack has a substantial CI system that is core to its development
>> process. The goals of the system are to facilitate merging good code,
>> prevent regressions, and ensure that there is
On Wed, 2014-07-23 at 14:39 -0700, James E. Blair wrote:
> OpenStack has a substantial CI system that is core to its development
> process. The goals of the system are to facilitate merging good code,
> prevent regressions, and ensure that there is at least one configuration
> of upstream OpenStac
On Wed, Jul 23, 2014 at 02:39:47PM -0700, James E. Blair wrote:
> OpenStack has a substantial CI system that is core to its development
> process. The goals of the system are to facilitate merging good code,
> prevent regressions, and ensure that there is at least one configuration
> of upstream O
On Jul 24, 2014, at 12:54 PM, Sean Dague wrote:
> On 07/24/2014 02:51 PM, Joshua Harlow wrote:
>> A potentially brilliant idea ;-)
>>
>> Aren't all the machines the gate runs tests on VMs running via OpenStack
>> APIs?
>>
>> OpenStack supports snapshotting (last time I checked). So instead of
On 07/24/2014 12:40 PM, Daniel P. Berrange wrote:
> On Wed, Jul 23, 2014 at 02:39:47PM -0700, James E. Blair wrote:
>
>> ==Future changes==
>
>> ===Fixing Faster===
>>
>> We introduce bugs to OpenStack at some constant rate, which piles up
>> over time. Our systems currently treat all changes as
On 07/24/2014 02:51 PM, Joshua Harlow wrote:
> A potentially brilliant idea ;-)
>
> Aren't all the machines the gate runs tests on VMs running via OpenStack APIs?
>
> OpenStack supports snapshotting (last time I checked). So instead of
> providing back a whole bunch of log files, provide back a
On Jul 24, 2014, at 12:08 PM, Anita Kuno wrote:
> On 07/24/2014 12:40 PM, Daniel P. Berrange wrote:
>> On Wed, Jul 23, 2014 at 02:39:47PM -0700, James E. Blair wrote:
>>
>>> ==Future changes==
>>
>>> ===Fixing Faster===
>>>
>>> We introduce bugs to OpenStack at some constant rate, which piles
On 07/24/2014 12:40 PM, Daniel P. Berrange wrote:
> On Wed, Jul 23, 2014 at 02:39:47PM -0700, James E. Blair wrote:
>
>> ==Future changes==
>
>> ===Fixing Faster===
>>
>> We introduce bugs to OpenStack at some constant rate, which piles up
>> over time. Our systems currently treat all changes as
A potentially brilliant idea ;-)
Aren't all the machines the gate runs tests on VMs running via OpenStack APIs?
OpenStack supports snapshotting (last time I checked). So instead of providing
back a whole bunch of log files, provide back a snapshot of the machine/s that
ran the tests; let person
On Wed, Jul 23, 2014 at 02:39:47PM -0700, James E. Blair wrote:
> ==Future changes==
> ===Fixing Faster===
>
> We introduce bugs to OpenStack at some constant rate, which piles up
> over time. Our systems currently treat all changes as equally risky and
> important to the health of the system, w
On 07/23/2014 05:39 PM, James E. Blair wrote:
> ==Final thoughts==
>
> The current rate of test failures and subsequent rechecks is not
> sustainable in the long term. It's not good for contributors,
> reveiewers, or the overall project quality. While these bugs do need to
> be addressed, it's u
On 07/24/2014 06:06 AM, Chmouel Boudjnah wrote:
> Hello,
>
> Thanks for writing this summary, I like all those ideas and thanks working
> hard on fixing this.
>
>> * For all non gold standard configurations, we'll dedicate a part of
>> our infrastructure to running them in a continuous back
Hello,
Thanks for writing this summary, I like all those ideas and thanks working
hard on fixing this.
> * For all non gold standard configurations, we'll dedicate a part of
> our infrastructure to running them in a continuous background loop,
> as well as making these configs available
OpenStack has a substantial CI system that is core to its development
process. The goals of the system are to facilitate merging good code,
prevent regressions, and ensure that there is at least one configuration
of upstream OpenStack that we know works as a whole. The "project
gating" technique
28 matches
Mail list logo
