[
https://issues.apache.org/jira/browse/HADOOP-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12759597#action_12759597
]
Steve Loughran commented on HADOOP-6248:
----------------------------------------
# I do functional testing with my smartfrog wrapper around the
lifecycle-enabled version of Hadoop. what I have not done yet -but which fully
automated test frameworks can do- is have the test framework try out different
configuration options, to *explore the configuration space*. That's fairly
bleeding edge in system testing, especially when that configuration space
includes the (virtualized) network infrastructure too, but it is a fantastic
way to find bugs.
# I do think it's critical that Hadoop tests the means by which it gets started
up. For {{hadoop.sh}} that means the shell scripts need to run. For me, that
means bring up some machines with the right RPMs installed, push out the config
to them, and then run functional tests against a live cluster.
# There's lots of scope for doing really interesting reporting here. You do
want the logs from 8 different machines all displayed in the test reports, all
in the best temporal order that Lamport will allow.
# Long term, lots of scope for datamining the test results.
# Log analysis is tricky. You don't want tests that are brittle against log
messages, leads to many false failures.
# Performance testing is not something you can do on a virtual system, tricky
w/ functional tests. Benchmarking is a separate problem. You shoudn't be using
your functional tests to assess performance, as the functional tests are
looking at the corner cases, trying to break things, not simulate well-behaved
code.
> Circus: Proposal and Preliminary Code for a Hadoop System Testing Framework
> ---------------------------------------------------------------------------
>
> Key: HADOOP-6248
> URL: https://issues.apache.org/jira/browse/HADOOP-6248
> Project: Hadoop Common
> Issue Type: New Feature
> Components: test
> Environment: Python, bash
> Reporter: Alex Loddengaard
> Attachments: HADOOP-6248.diff, HADOOP-6248_v2.diff,
> HADOOP-6248_v3.diff
>
>
> This issue contains a proposal and preliminary source code for Circus, a
> Hadoop system testing framework. At a high level, Circus will help Hadoop
> users and QA engineers to run system tests on a configurable Hadoop cluster,
> or distribution of Hadoop. See the comment below for the proposal itself.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
