[
https://issues.apache.org/jira/browse/IMPALA-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-6971:
----------------------------------
Component/s: Backend
> Log flooding with unhelpful parsing error message
> -------------------------------------------------
>
> Key: IMPALA-6971
> URL: https://issues.apache.org/jira/browse/IMPALA-6971
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 3.0, Impala 2.12.0
> Reporter: Zoram Thanga
> Priority: Minor
> Labels: supportability
>
> Once in a while I've seen Impala flood the log file with messages similar to
> the following:
> {noformat}
> I0318 08:20:28.566603 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.566643 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.566676 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.566709 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.566742 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.566787 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.566820 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.566856 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.566890 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.566923 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.566959 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.566998 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567039 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567070 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567101 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567132 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567173 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567209 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567245 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567276 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567312 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567350 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567386 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567433 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567473 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567507 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567538 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567574 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567606 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567638 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567674 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567710 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567749 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567785 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567821 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567857 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567893 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567924 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567961 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.567993 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568038 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568078 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568117 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568150 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568186 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568222 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568255 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568291 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568327 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568367 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568399 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568430 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568470 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568506 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568542 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568578 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568619 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568662 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568699 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568738 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568771 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568807 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568840 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568883 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> I0318 08:20:28.568919 984564 runtime-state.cc:168] Error from query
> cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
> hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
> offset: 2122317824
> {noformat}
> Two main issues with this log entry are:
> # The line number printed is wrong. This line has nothing to do with
> processing rows. We should try harder to show the real location of the
> problem.
> # It is unnecessarily repeated, filling log files.
> In this specific instance, the same entry is repeated more than 1,740,663
> times!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
