Hard to say at a glance, but this issue is happening down in the
MapReduce framework, not in Phoenix itself.
It looks similar to problems I've seen many years ago around
mapreduce.task.io.sort.mb. You can try reducing that value. It also may
be related to a bug in your Hadoop version.
Good luck!
On 3/22/18 4:37 AM, Margusja wrote:
Hi
Needed to recreate indexes over main table contains more than 2.3 x
10^10 records.
I used ASYNC and org.apache.phoenix.mapreduce.index.IndexTool
One index succeed but another gives stack:
2018-03-20 13:23:16,723 FATAL [IPC Server handler 0 on 43926]
org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
attempt_1521544097253_0004_m_000008_0 - exited :
java.lang.ArrayIndexOutOfBoundsException at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1453)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1349)
at java.io.DataOutputStream.writeInt(DataOutputStream.java:197) at
org.apache.hadoop.hbase.io.ImmutableBytesWritable.write(ImmutableBytesWritable.java:159)
at
org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:98)
at
org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:82)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1149)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:715)
at
org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
at
org.apache.phoenix.mapreduce.index.PhoenixIndexImportMapper.map(PhoenixIndexImportMapper.java:114)
at
org.apache.phoenix.mapreduce.index.PhoenixIndexImportMapper.map(PhoenixIndexImportMapper.java:48)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at
org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)
Is here any best practice how to deal with situations like this?
Br, Margus
