On Sat, Dec 11, 2010 at 11:17 AM, Andrei Savu <[email protected]> wrote: > Hi guys, > > My name is Andrei Savu. Last summer I have worked on Apache ZooKeeper > as a GSoC intern (Patrick Hunt was my mentor). Right now I'm studying > for a M.Sc. and for my final thesis I have decided that I want to > build a fault injection framework for black-box testing distributed > systems (highly available systems like ZooKeeper, HDFS, HBase, > Cassandra etc.) > > The test cycle as I see it, should have three major steps: > 1. bring the SUT into a predefined initial state (this is where I want > to use Whirr) - and load some test data. > 2. run a predefined or auto-generated test scenario - it should inject > multiple kinds of failures (no more than the maximum number of > failures the SUT can theoretically handle) > 3. perform sanity checks. If a failure is detected then it should stop > the test and collect all relevant data needed for diagnostics. > > I want to build this framework on top of Apache Whirr and I would like > to to start by contributing to the project. I have managed to setup a > local development environment (I haven't run any integration tests yet > - planning to get an AWS account soon) and I want to begin by writing > some tests (WHIRR-160).
That sounds great. Using Whirr to bring up a system is a good use of it; have you seen the (fledgling) benchmark framework introduced in https://issues.apache.org/jira/browse/WHIRR-92? It's a demonstration of how to run the full testing lifecycle on a cluster. You might extend or generalize this for black box testing. > > What else should I test? Could you point me to some important open JIRAs? I think Herriot may be relevant here too. Have you looked at that? http://wiki.apache.org/hadoop/HowToUseSystemTestFramework It provides ways to inject known faults into Hadoop and run tests to check that they are handled appropriately. Using Whirr to run Herriot tests would be a good approach for running Hadoop system tests. It might be generalizable to other systems too. I think Herriot's complementary to Gremlins, which is for non-deterministic faults. (As I understand it, Cos and Todd can correct me on this.) Cheers, Tom > > Best, > > -- > Andrei Savu -- http://www.andreisavu.ro/ >
