[
https://issues.apache.org/jira/browse/MAPREDUCE-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13800884#comment-13800884
]
Andrey Klochkov commented on MAPREDUCE-3860:
--------------------------------------------
Ravi,
Well, to begin with, majority of Hadoop "unit" tests are not true unit tests. I
mean all these mini cluster based tests and similar ones. Still I think that
with a mature and slowly evolving codebase, and priorities shifted to maturity
rather than quick changes, it makes sense to have such kinds of tests,
especially when the alternative is not to have any. Currently Rumen code
coverage by tests is almost zero, and this has been the case for a long time.
Knowing that it is probably one of the slowest changing parts of the codebase,
I think having that old tests which are indeed based on static files generated
manually are better than having none. As I understand some of the tests like
{{TestRumenJobTraces.testHadoop20JHParser}} use static pre-generated files to
test compatibility with older versions of Hadoop, and in case of
{{TestRumenJobTraces}} there is {{testCurrentJHParser}} which does use a real
job to generate logs and then parse them, e.g. this is the test which works
with the current version of the codebase. Having said that, I agree that
improving these tests to use mini clusters as much as possible, instead of
using pre-generated files, is a proper way of improving the tests further. The
meaning of this task is to bring back old tests, which as I understand were
broken when switching to Yarn, and were removed just to fix the builds quickly.
On questions for {{TestRumentAnonymization}}.
1. It's waiting for 100 *milli*seconds, not 100 seconds. Which is tolerable.
2. No, having an instance of {{Configuration}} as the field in the class may
not affect tests in these way. JUnit creates a dedicated instance of the class
for each test execution.
3. Not sure if this is a valid anonymization to have a null in the username. Is
it?
4. Why should those temp dirs be deleted after test runs? It makes
troubleshooting more difficult. Those dirs do not interfere with each other or
other tests.
> [Rumen] Bring back the removed Rumen unit tests
> -----------------------------------------------
>
> Key: MAPREDUCE-3860
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3860
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tools/rumen
> Reporter: Ravi Gummadi
> Assignee: Andrey Klochkov
> Attachments: MAPREDUCE-3860--n2.patch, MAPREDUCE-3860.patch,
> rumen-test-data.tar.gz
>
>
> MAPREDUCE-3582 did not move some of the Rumen unit tests to the new folder
> and then MAPREDUCE-3705 deleted those unit tests. These Rumen unit tests need
> to be brought back:
> TestZombieJob.java
> TestRumenJobTraces.java
> TestRumenFolder.java
> TestRumenAnonymization.java
> TestParsedLine.java
> TestConcurrentRead.java
--
This message was sent by Atlassian JIRA
(v6.1#6144)
