Re: Hadoop testing project

2011-02-18 Thread Eric Yang
to a VM in the VLAN, but that's tractable. Packaging, VM testing, scale, then deployment automation. My goal is deployment automation, and this requires polishing each steps along the way. For hadoop testing project, it depends on the type of testing that you are focus on. It will be easier

Re: Hadoop testing project

2011-02-17 Thread Ian Holsman
I'm not sure it makes sense to all the testing packages under a different umbrella that covers the code they test. While there might be commonalities building a test harness, I would think that each testing tool would need to have deep knowledge of the tool's internals that it is testing. as

Re: Hadoop testing project

2011-02-17 Thread Allen Wittenauer
On Feb 16, 2011, at 11:50 AM, Konstantin Boudnik wrote: As Joep said this ...will reduce the effort to take any (set of ) changes from development into production. Take it one step further: when your cluster is 'assembled' you need to validate it (on top of a concrete OS, etc.); is it

Re: Hadoop testing project

2011-02-17 Thread Eric Yang
The biggest hurtle in hadoop adoption is that there is no easy way to setup a pseudo cluster on developer's machine. People are steering off course to build additional simulation tools and validation tools. In practice, those tools don't provide nearly enough insight in things that could go

Re: Hadoop testing project

2011-02-17 Thread Konstantin Boudnik
Eric. I am sure that packaging of Hadoop and other application working directly with Hadoop is a highly needed thing (although there's always a tricky question how many platforms you plan to provide packaging for, etc.). What we are discussing here is software testing, not packaging nor

Re: Hadoop testing project [Was: [VOTE] Abandon mrunit MapReduce contrib]

2011-02-17 Thread Aaron Kimball
From: c...@boudnik.org [c...@boudnik.org] On Behalf Of Konstantin Boudnik [ c...@apache.org] Sent: Tuesday, February 15, 2011 1:58 PM To: general@hadoop.apache.org Subject: Hadoop testing project [Was: [VOTE] Abandon mrunit MapReduce contrib] While MrUnit

Re: Hadoop testing project [Was: [VOTE] Abandon mrunit MapReduce contrib]

2011-02-17 Thread Konstantin Boudnik
, 2011 1:58 PM To: general@hadoop.apache.org Subject: Hadoop testing project [Was: [VOTE] Abandon mrunit MapReduce contrib] While MrUnit discussion draws to its natural conclusion I would like to bring up another point which might be well aligned with that discussion. Patrick Hunt has brought

RE: Hadoop testing project [Was: [VOTE] Abandon mrunit MapReduce contrib]

2011-02-16 Thread Rottinghuis, Joep
From: c...@boudnik.org [c...@boudnik.org] On Behalf Of Konstantin Boudnik [c...@apache.org] Sent: Tuesday, February 15, 2011 1:58 PM To: general@hadoop.apache.org Subject: Hadoop testing project [Was: [VOTE] Abandon mrunit MapReduce contrib] While MrUnit discussion draws

Re: Hadoop testing project

2011-02-16 Thread Konstantin Boudnik
Steve. If the project under discussion will provide a common harness where such a test artifact (think of a Maven artifact for example) will click and will be executed automatically with all needed tools and dependencies resolved for you - would it be appealing for end-users' cause? As Joep said

Re: Hadoop testing project [Was: [VOTE] Abandon mrunit MapReduce contrib]

2011-02-15 Thread Mattmann, Chris A (388J)
Sounds good to me, Cos. I'm fine to help/mentor with either one that ends up standing when the dust clears :) Cheers, Chris On Feb 15, 2011, at 1:58 PM, Konstantin Boudnik wrote: While MrUnit discussion draws to its natural conclusion I would like to bring up another point which might be

Re: Hadoop testing project [Was: [VOTE] Abandon mrunit MapReduce contrib]

2011-02-15 Thread Eric Sammer
I think this is a good idea. The only thing I think is that it may make sense to split such an effort into two components: one for the testing of Hadoop and the projects themselves and one to test end user applications and libraries. Performance testing tools like YCSB are probably more in the