Hi,
I tried to copy round about 1 million records from one table to another with
this sql:
UPSERT INTO New_Table (ID, data) SELECT ID, data FROM OLD_Table WHERE id >=
'2010000000' AND ID < '2011000000';
With smaller number of records it works, but with this sql I get the following
exception:
18/03/26 07:54:02 WARN client.AsyncProcess: #1, table=New_Table, attempt=35/35
failed=28363ops, last exception:
org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.ipc.RpcServer$CallQueueTooBigException):
Call queue is full on /x.x.x.x:16020, is hbase.ipc.server.max.callqueue.size
too small? on dn13,16020,1521542835853, tracking started Mon Mar 26 07:44:16
CEST 2018; not retrying 28363 - final failure
Error: org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
Failed 28363 actions: RemoteWithExtrasException: 28363 times, (state=,code=0)
org.apache.phoenix.execute.CommitException:
org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed
28363 actions: RemoteWithExtrasException: 28363 times,
at org.apache.phoenix.execute.MutationState.send(MutationState.java:985)
at
org.apache.phoenix.execute.MutationState.send(MutationState.java:1327)
at
org.apache.phoenix.execute.MutationState.commit(MutationState.java:1159)
at
org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnection.java:528)
at
org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnection.java:525)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at
org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:525)
at
org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:346)
at
org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:331)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at
org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:329)
at
org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1440)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:292)
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
Failed 28363 actions: RemoteWithExtrasException: 28363 times,
at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:234)
at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1700(AsyncProcess.java:214)
at
org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.getErrors(AsyncProcess.java:1623)
at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:909)
at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:923)
at org.apache.phoenix.execute.MutationState.send(MutationState.java:952)
... 17 more
I have searched a lot for correct / better values for
hbase.ipc.server.max.callqueue.size but I don't find anything. And in our
configuration there is no entry for that.
I tried this on a Hortonworks 2.5.3 with Phoenix 4.7.
Any ideas where I must tune to get this sql running?
Thank's in advance
Martin
------------------------------------------------------------------------------
FIZ Karlsruhe - Leibniz-Institut für Informationsinfrastruktur GmbH.
Sitz der Gesellschaft: Eggenstein-Leopoldshafen, Amtsgericht Mannheim HRB
101892.
Geschäftsführerin: Sabine Brünger-Weilandt.
Vorsitzender des Aufsichtsrats: MinDirig Dr. Stefan Luther.
