On May 31, 2012, at 11:40 AM, Sanne Grinovero wrote:
> On 31 May 2012 10:33, Galder Zamarreño wrote:
>>
>> On May 29, 2012, at 12:48 PM, Manik Surtani wrote:
>>
>>> I pretty much agree with this; and here's a bit of history.
>>>
>>> For the large part we have had a stable test suite, but the
IMO, test groups + maven integration to run this = PITA
On May 30, 2012, at 6:22 PM, Dan Berindei wrote:
> On Wed, May 30, 2012 at 5:19 PM, Manik Surtani wrote:
>>
>> On 30 May 2012, at 14:58, Sanne Grinovero wrote:
>>
>>
>> I see two issues with your plan, though:
>>
>> 1. Buildhive is lim
On May 30, 2012, at 2:55 PM, Sanne Grinovero wrote:
> On 30 May 2012 13:36, Manik Surtani wrote:
>> Well, one problem leads to another, as you are well aware. Flaky parallel
>> suite leads careless commits. Yes, we should fix what is broken at the
>> moment but that is not enough since it wi
On May 30, 2012, at 3:25 PM, Dan Berindei wrote:
> On Wed, May 30, 2012 at 4:12 PM, Manik Surtani wrote:
>>
>> On 30 May 2012, at 13:55, Sanne Grinovero wrote:
>>
>>> On 30 May 2012 13:36, Manik Surtani wrote:
Well, one problem leads to another, as you are well aware. Flaky parallel
>>
On 31 May 2012, at 15:30, Sanne Grinovero wrote:
> On 31 May 2012 14:57, Manik Surtani wrote:
>> On 31 May 2012, at 10:40, Sanne Grinovero wrote:
>>
>> Seriously, I just think we can design the tests better.
>>
>>
>> Are you referring to tests, or the test framework? In any case, what do you
On 31 May 2012 14:57, Manik Surtani wrote:
> On 31 May 2012, at 10:40, Sanne Grinovero wrote:
>
> Seriously, I just think we can design the tests better.
>
>
> Are you referring to tests, or the test framework? In any case, what do you
> have in mind?
I don't have anything specific in mind, nor
On 31 May 2012, at 10:18, Dan Berindei wrote:
> It's only targeted for 1.1, though. In the meantime, I wonder if we
> could configure just the cdi module to run tests sequentially.
Yes, that can certainly be done. Check out some of the other modules, I know
some are configured to only run seq
On 31 May 2012, at 10:40, Sanne Grinovero wrote:
> Seriously, I just think we can design the tests better.
Are you referring to tests, or the test framework? In any case, what do you
have in mind?
--
Manik Surtani
ma...@jboss.org
twitter.com/maniksurtani
Lead, Infinispan
http://www.infinispan
On Thu, May 31, 2012 at 12:40 PM, Sanne Grinovero wrote:
> On 31 May 2012 10:33, Galder Zamarreño wrote:
>>
>> On May 29, 2012, at 12:48 PM, Manik Surtani wrote:
>>
>>> I pretty much agree with this; and here's a bit of history.
>>>
>>> For the large part we have had a stable test suite, but the
On 30 May 2012, at 18:06, Manik Surtani wrote:
>
> On 30 May 2012, at 17:22, Dan Berindei wrote:
>
>>
>> Yet there are still lots of tests that have both group="manual" and
>> enabled="false", with descriptions like "Disabled until we can
>> configure Surefire to skip manual tests". You can't
On 31 May 2012 10:33, Galder Zamarreño wrote:
>
> On May 29, 2012, at 12:48 PM, Manik Surtani wrote:
>
>> I pretty much agree with this; and here's a bit of history.
>>
>> For the large part we have had a stable test suite, but the occasional
>> unpredictability in the suite came in when we intro
On May 30, 2012, at 2:28 PM, Sanne Grinovero wrote:
> On 30 May 2012 13:24, Manik Surtani wrote:
>>
>> On 30 May 2012, at 13:00, Dan Berindei wrote:
>>
>>> On Tue, May 29, 2012 at 1:48 PM, Manik Surtani wrote:
I pretty much agree with this; and here's a bit of history.
For the
On May 29, 2012, at 12:48 PM, Manik Surtani wrote:
> I pretty much agree with this; and here's a bit of history.
>
> For the large part we have had a stable test suite, but the occasional
> unpredictability in the suite came in when we introduced the parallel test
> runner, to allow us to ru
On Wed, May 30, 2012 at 8:06 PM, Manik Surtani wrote:
>
> On 30 May 2012, at 17:22, Dan Berindei wrote:
>
>
> Yet there are still lots of tests that have both group="manual" and
> enabled="false", with descriptions like "Disabled until we can
> configure Surefire to skip manual tests". You can't b
On 30 May 2012, at 17:22, Dan Berindei wrote:
>
> Yet there are still lots of tests that have both group="manual" and
> enabled="false", with descriptions like "Disabled until we can
> configure Surefire to skip manual tests". You can't blame me for
> thinking the comment is still valid :)
Find
On Wed, May 30, 2012 at 5:19 PM, Manik Surtani wrote:
>
> On 30 May 2012, at 14:58, Sanne Grinovero wrote:
>
>
> I see two issues with your plan, though:
>
> 1. Buildhive is limited to 15 mins, and a reviewer wouldn't
>
> necessarily wait for 2 hours to integrate a pull request anyway. So
>
> the
On 30 May 2012, at 14:58, Sanne Grinovero wrote:
>>
>> I see two issues with your plan, though:
>> 1. Buildhive is limited to 15 mins, and a reviewer wouldn't
>> necessarily wait for 2 hours to integrate a pull request anyway. So
>> the sequential build would be limited to Jenkins runs.
>> 2. Ho
On 30 May 2012 14:25, Dan Berindei wrote:
> On Wed, May 30, 2012 at 4:12 PM, Manik Surtani wrote:
>>
>> On 30 May 2012, at 13:55, Sanne Grinovero wrote:
>>
>>> On 30 May 2012 13:36, Manik Surtani wrote:
Well, one problem leads to another, as you are well aware. Flaky parallel
suite l
On Wed, May 30, 2012 at 4:12 PM, Manik Surtani wrote:
>
> On 30 May 2012, at 13:55, Sanne Grinovero wrote:
>
>> On 30 May 2012 13:36, Manik Surtani wrote:
>>> Well, one problem leads to another, as you are well aware. Flaky parallel
>>> suite leads careless commits. Yes, we should fix what is
On 30 May 2012, at 13:55, Sanne Grinovero wrote:
> On 30 May 2012 13:36, Manik Surtani wrote:
>> Well, one problem leads to another, as you are well aware. Flaky parallel
>> suite leads careless commits. Yes, we should fix what is broken at the
>> moment but that is not enough since it will
On 30 May 2012 13:36, Manik Surtani wrote:
> Well, one problem leads to another, as you are well aware. Flaky parallel
> suite leads careless commits. Yes, we should fix what is broken at the
> moment but that is not enough since it will get this way again unless we have
> a stable suite that
On 30 May 2012, at 13:28, Sanne Grinovero wrote:
> +1000 let's open an issue for each test that fails, even if it fails
> occasionally. If it can't be fixed, the test is doing more harm than
> any good and should be either redesigned or removed.
Let's start with identifying them.
https://comm
Well, one problem leads to another, as you are well aware. Flaky parallel
suite leads careless commits. Yes, we should fix what is broken at the moment
but that is not enough since it will get this way again unless we have a stable
suite that can be used to ensure quality moving fwd. The opti
On 30 May 2012 13:24, Manik Surtani wrote:
>
> On 30 May 2012, at 13:00, Dan Berindei wrote:
>
>> On Tue, May 29, 2012 at 1:48 PM, Manik Surtani wrote:
>>> I pretty much agree with this; and here's a bit of history.
>>>
>>> For the large part we have had a stable test suite, but the occasional
>
It's not really about "which" test mode we should use, as long as
people check their commits and have any test that gives enough
confidence, which is still not the case even during this discussion:
last build Infinispan-master-JDK6-tcp-NON_PARALLEL : 123 failures / +95 failures
We need to use non
On 30 May 2012, at 13:00, Dan Berindei wrote:
> On Tue, May 29, 2012 at 1:48 PM, Manik Surtani wrote:
>> I pretty much agree with this; and here's a bit of history.
>>
>> For the large part we have had a stable test suite, but the occasional
>> unpredictability in the suite came in when we int
On Tue, May 29, 2012 at 1:48 PM, Manik Surtani wrote:
> I pretty much agree with this; and here's a bit of history.
>
> For the large part we have had a stable test suite, but the occasional
> unpredictability in the suite came in when we introduced the parallel test
> runner, to allow us to run
I pretty much agree with this; and here's a bit of history.
For the large part we have had a stable test suite, but the occasional
unpredictability in the suite came in when we introduced the parallel test
runner, to allow us to run the (core) suite in under 5 minutes - a suite which
otherwis
On 29 May 2012, at 10:30, Galder Zamarreño wrote:
> When I handle pull reqs, I sometimes see random testsuite failures which
> often are not related to patch itself.
>
> What I do then is send an email to an author, or expert in the test with
> TRACE log and failure information, and ask to lo
On 29 May 2012 10:24, Galder Zamarreño wrote:
> BuildHive might help somewhat (when it fully works...) but the real problem
> testsuites will continue to be there:
>
> 1. Some tests fail randomly, due to concurrency issues or any kind.
> 2. Some tests fail due to differences in environments (CI v
When I handle pull reqs, I sometimes see random testsuite failures which often
are not related to patch itself.
What I do then is send an email to an author, or expert in the test with TRACE
log and failure information, and ask to look into it.
They might not look into it immediately (for seve
BuildHive might help somewhat (when it fully works...) but the real problem
testsuites will continue to be there:
1. Some tests fail randomly, due to concurrency issues or any kind.
2. Some tests fail due to differences in environments (CI vs your own machine)
The problem with this tests will co
We're actually in the process of setting up BuildHive. Galder's on it. :)
On 28 May 2012, at 17:20, Adrian Cole wrote:
> FWIW, might be a good idea trying buildhive a bit, then deciding. It is
> working pretty well for jenkins-ci projects, and so much easier than fetch,
> cherry-pick, test p
FWIW, might be a good idea trying buildhive a bit, then deciding. It is
working pretty well for jenkins-ci projects, and so much easier than fetch,
cherry-pick, test push loop.
In jclouds, we are setting this up as community members are starting to be
more brave (ex refactor things that other PRs
I don't think everyone has to handle tens of PRs a day. It's more like one per
person per day, which IMO isn't unreasonable as long as everyone does their
fair share.
On 27 May 2012, at 14:51, Bela Ban wrote:
> +1000. I completely agree that if someone has to handle tens of pull
> requests pe
+1000. I completely agree that if someone has to handle tens of pull
requests per day, he will *not* seriously look into the request, test it
etc. So IMO this is a farce, and we might as well go back to trusting
people, rather than wasting their time...
On 5/25/12 1:47 PM, Sanne Grinovero wro
On 25 May 2012, at 14:44, Mircea Markus wrote:
>>
>> I'll take over that role. One thing I'd like to propose:
>>
>> 1. The reviewer is ultimately responsible for the patch. If a broken patch
>> makes it to upstream it is the reviewer's fault.
>
> I think we should differentiate from commu
On 25 May 2012, at 14:34, Sanne Grinovero wrote:
> I don't disagree on having proper reviews, actually the opposite: my
> topic was meant to be provocative and I'm very glad of your reaction.
> But I don' t understand your "no" in this context after my comment
> which is specifically commenting o
Mircea Markus píše v Fri 25. 05. 2012 v 14:44 +0100:
> >
> > I'll take over that role. One thing I'd like to propose:
> >
> > 1. The reviewer is ultimately responsible for the patch. If a broken
> > patch makes it to upstream it is the reviewer's fault.
>
> I think we should differentiate fr
>
> I'll take over that role. One thing I'd like to propose:
>
> 1. The reviewer is ultimately responsible for the patch. If a broken patch
> makes it to upstream it is the reviewer's fault.
I think we should differentiate from community patches (i.e. external
contributors) and internal pa
On 25 May 2012 13:44, Manik Surtani wrote:
>
> On 25 May 2012, at 13:34, Sanne Grinovero wrote:
>
>> On 25 May 2012 13:16, Manik Surtani wrote:
>>>
>>> On 25 May 2012, at 12:47, Sanne Grinovero wrote:
>>>
guys, please don't take me as the one who is again complaining about
failing tests
On 25 May 2012, at 13:34, Sanne Grinovero wrote:
> On 25 May 2012 13:16, Manik Surtani wrote:
>>
>> On 25 May 2012, at 12:47, Sanne Grinovero wrote:
>>
>>> guys, please don't take me as the one who is again complaining about
>>> failing tests; I'm having doubts about the development process an
On 25 May 2012 13:16, Manik Surtani wrote:
>
> On 25 May 2012, at 12:47, Sanne Grinovero wrote:
>
>> guys, please don't take me as the one who is again complaining about
>> failing tests; I'm having doubts about the development process and the
>> amount of time this is wasting on all of us.
>>
>>
On 25 May 2012, at 12:47, Sanne Grinovero wrote:
> guys, please don't take me as the one who is again complaining about
> failing tests; I'm having doubts about the development process and the
> amount of time this is wasting on all of us.
>
> We're all humans and do mistakes, still it happens s
44 matches
Mail list logo
