[
https://issues.apache.org/jira/browse/PIG-5423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17521328#comment-17521328
]
Koji Noguchi commented on PIG-5423:
-----------------------------------
Two surprises in trying hadoop 2.10.1.
* HADOOP-14213 changed the preferences of config files and unit tests started
failing unless we renamed hadoop-site.xml to core-site.xml
* HADOOP-13065 changed how filesystem counter is tracked. For unit tests where
we run multiple tasks within a single process, started to see tasks including
filesystem counts from other tasks. For sequential task scheduling job, we can
still overcome this by resetting the counter at beginning of task. For job
with multiple concurrent tasks, I couldn't find a way to get the correct counts
for our unit tests.
> Upgrade hadoop/tez dependency
> ------------------------------
>
> Key: PIG-5423
> URL: https://issues.apache.org/jira/browse/PIG-5423
> Project: Pig
> Issue Type: Improvement
> Reporter: Koji Noguchi
> Assignee: Koji Noguchi
> Priority: Major
> Attachments: pig-5423-v01.patch
>
>
> We already have PIG-5253 for supporting hadoop3. Here, upgrading hadoop2
> dependency to the most recent hadoop2 version, 2.10.1.
> Also, upgrading Tez to 0.9.2. (0.10.1 showed some regressions and need
> further checking).
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
