Jose Luis López created MAPREDUCE-7545:
------------------------------------------
Summary: TaskId.hashCode depends on the JVM
Key: MAPREDUCE-7545
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7545
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Reporter: Jose Luis López
Assignee: Jose Luis López
TaskId.hashCode() mixes in getTaskType().hashCode(). TaskType is an enum, and an
enum's hashCode() is its identity hash. HotSpot draws identity hashes from a
sequence that shifts with whatever asked for one earlier in the JVM: which jars
are on the classpath, which classes are initialized. So every HashMap keyed by
TaskId iterates in an order that depends on the environment, not on the tasks.
TestRuntimeEstimators depends on that order. It builds five slow maps and
expects three speculations to win. In some orders only one does, and all three
tests then fail every time on that environment:
TestRuntimeEstimators.testLegacyEstimator:254->coreTestEstimator:247
We got the wrong number of successful speculations. ==> expected: <3> but
was: <1>
Seen on GitHub Actions (Java 17) on the HADOOP-19970 branch, twice, with the
surefire reruns failing identically, while the same commit passes locally.
Reproduced locally by consuming four extra identity hashes before the test
starts (new Object().hashCode() in a static block), with no other change.
Fix: hash the task type's ordinal instead of the enum itself, so the hash is
the same in every JVM. A test in TestIds pins it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
