[jira] [Commented] (MAPREDUCE-6922) MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
[
https://issues.apache.org/jira/browse/MAPREDUCE-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198041#comment-16198041
]
rangjiaheng commented on MAPREDUCE-6922:
We solve this problem simply by adding an judgment after read ordinal, but if
someone add an Enumeration value in the middle position of TaskCounter.class,
the AM will read confused Enumeration values from RPC.
I support counters should serialized through protocol buffer.
{code:java}
public void readFields(DataInput in) throws IOException {
clear();
int len = WritableUtils.readVInt(in);
T[] enums = enumClass.getEnumConstants();
for (int i = 0; i < len; ++i) {
int ord = WritableUtils.readVInt(in);
if(ord >= enums.length) {
WritableUtils.readVLong(in);
continue;
}
Counter counter = newCounter(enums[ord]);
counter.setValue(WritableUtils.readVLong(in));
counters[ord] = counter;
}
}
{code}
> MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
>
>
> Key: MAPREDUCE-6922
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6922
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Miklos Szegedi
>Assignee: Miklos Szegedi
>Priority: Blocker
> Attachments: YARN-6922.branch-2.000.patch
>
>
> MAPREDUCE-6829 should be reverted from branch-2 because rolling upgrade fails.
> {code}
> 2017-06-08 17:43:37,173 WARN [Socket Reader #1 for port 41187]
> org.apache.hadoop.ipc.Server: Unable to read call parameters for client
> 10.17.242.22on connection protocol
> org.apache.hadoop.mapred.TaskUmbilicalProtocol for rpcKind RPC_WRITABLE
> java.lang.ArrayIndexOutOfBoundsException: 23
> at
> org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.readFields(FrameworkCounterGroup.java:261)
> at org.apache.hadoop.mapred.Counters$Group.readFields(Counters.java:324)
> at
> org.apache.hadoop.mapreduce.counters.AbstractCounters.readFields(AbstractCounters.java:306)
> at org.apache.hadoop.mapred.TaskStatus.readFields(TaskStatus.java:489)
> at
> org.apache.hadoop.mapred.MapTaskStatus.readFields(MapTaskStatus.java:88)
> at
> org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:285)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invocation.readFields(WritableRpcEngine.java:160)
> at
> org.apache.hadoop.ipc.Server$Connection.processRpcRequest(Server.java:1909)
> at
> org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1841)
> at
> org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1600)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:820)
> at
> org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:693)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:664)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6922) MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
[
https://issues.apache.org/jira/browse/MAPREDUCE-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108033#comment-16108033
]
Miklos Szegedi commented on MAPREDUCE-6922:
---
Thank you, [~jlowe]!
> MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
>
>
> Key: MAPREDUCE-6922
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6922
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Miklos Szegedi
>Assignee: Miklos Szegedi
>Priority: Blocker
> Attachments: YARN-6922.branch-2.000.patch
>
>
> MAPREDUCE-6829 should be reverted from branch-2 because rolling upgrade fails.
> {code}
> 2017-06-08 17:43:37,173 WARN [Socket Reader #1 for port 41187]
> org.apache.hadoop.ipc.Server: Unable to read call parameters for client
> 10.17.242.22on connection protocol
> org.apache.hadoop.mapred.TaskUmbilicalProtocol for rpcKind RPC_WRITABLE
> java.lang.ArrayIndexOutOfBoundsException: 23
> at
> org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.readFields(FrameworkCounterGroup.java:261)
> at org.apache.hadoop.mapred.Counters$Group.readFields(Counters.java:324)
> at
> org.apache.hadoop.mapreduce.counters.AbstractCounters.readFields(AbstractCounters.java:306)
> at org.apache.hadoop.mapred.TaskStatus.readFields(TaskStatus.java:489)
> at
> org.apache.hadoop.mapred.MapTaskStatus.readFields(MapTaskStatus.java:88)
> at
> org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:285)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invocation.readFields(WritableRpcEngine.java:160)
> at
> org.apache.hadoop.ipc.Server$Connection.processRpcRequest(Server.java:1909)
> at
> org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1841)
> at
> org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1600)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:820)
> at
> org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:693)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:664)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6922) MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
[
https://issues.apache.org/jira/browse/MAPREDUCE-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108014#comment-16108014
]
Jason Lowe commented on MAPREDUCE-6922:
---
OK I'll revert MAPREDUCE-6829 from branch-2 and update that JIRA accordingly.
> MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
>
>
> Key: MAPREDUCE-6922
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6922
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Miklos Szegedi
>Assignee: Miklos Szegedi
>Priority: Blocker
> Attachments: YARN-6922.branch-2.000.patch
>
>
> MAPREDUCE-6829 should be reverted from branch-2 because rolling upgrade fails.
> {code}
> 2017-06-08 17:43:37,173 WARN [Socket Reader #1 for port 41187]
> org.apache.hadoop.ipc.Server: Unable to read call parameters for client
> 10.17.242.22on connection protocol
> org.apache.hadoop.mapred.TaskUmbilicalProtocol for rpcKind RPC_WRITABLE
> java.lang.ArrayIndexOutOfBoundsException: 23
> at
> org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.readFields(FrameworkCounterGroup.java:261)
> at org.apache.hadoop.mapred.Counters$Group.readFields(Counters.java:324)
> at
> org.apache.hadoop.mapreduce.counters.AbstractCounters.readFields(AbstractCounters.java:306)
> at org.apache.hadoop.mapred.TaskStatus.readFields(TaskStatus.java:489)
> at
> org.apache.hadoop.mapred.MapTaskStatus.readFields(MapTaskStatus.java:88)
> at
> org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:285)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invocation.readFields(WritableRpcEngine.java:160)
> at
> org.apache.hadoop.ipc.Server$Connection.processRpcRequest(Server.java:1909)
> at
> org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1841)
> at
> org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1600)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:820)
> at
> org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:693)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:664)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6922) MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
[
https://issues.apache.org/jira/browse/MAPREDUCE-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108011#comment-16108011
]
Miklos Szegedi commented on MAPREDUCE-6922:
---
Thank you, Jason. I agree the distributed cache deployment solves this the
issue, however I am not sure how many users use the other option. I think we
should revert from branch-2. My understanding is that there is no rolling
upgrade requirement from 2.x to 3.0.
> MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
>
>
> Key: MAPREDUCE-6922
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6922
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Miklos Szegedi
>Assignee: Miklos Szegedi
>Priority: Blocker
> Attachments: YARN-6922.branch-2.000.patch
>
>
> MAPREDUCE-6829 should be reverted from branch-2 because rolling upgrade fails.
> {code}
> 2017-06-08 17:43:37,173 WARN [Socket Reader #1 for port 41187]
> org.apache.hadoop.ipc.Server: Unable to read call parameters for client
> 10.17.242.22on connection protocol
> org.apache.hadoop.mapred.TaskUmbilicalProtocol for rpcKind RPC_WRITABLE
> java.lang.ArrayIndexOutOfBoundsException: 23
> at
> org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.readFields(FrameworkCounterGroup.java:261)
> at org.apache.hadoop.mapred.Counters$Group.readFields(Counters.java:324)
> at
> org.apache.hadoop.mapreduce.counters.AbstractCounters.readFields(AbstractCounters.java:306)
> at org.apache.hadoop.mapred.TaskStatus.readFields(TaskStatus.java:489)
> at
> org.apache.hadoop.mapred.MapTaskStatus.readFields(MapTaskStatus.java:88)
> at
> org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:285)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invocation.readFields(WritableRpcEngine.java:160)
> at
> org.apache.hadoop.ipc.Server$Connection.processRpcRequest(Server.java:1909)
> at
> org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1841)
> at
> org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1600)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:820)
> at
> org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:693)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:664)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
