[ https://issues.apache.org/jira/browse/DRILL-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782266#comment-15782266 ]
Khurram Faraaz commented on DRILL-4831: --------------------------------------- Run concurrent SELECT statements over parquet file. >From another sqlline session run REFRESH TABLE METADATA command, while the >above SELECT statements are executing concurrently. We see an Exception as >reported in this JIRA. The below stack trace is seen in drillbit.log. Drill 1.10.0 git commit ID: bbcf4b7 {noformat} 2016-12-28 07:00:06,403 [279c9d09-5640-c401-a132-d85e0e0f71ac:foreman] INFO o.a.d.exec.store.parquet.Metadata - Creating metadata files recursively took 33 ms 2016-12-28 07:00:06,403 [279c9d09-5640-c401-a132-d85e0e0f71ac:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, numFiles: 1 2016-12-28 07:00:06,405 [279c9d08-a348-3eac-55de-940a60cb6975:foreman] ERROR o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR: FileNotFoundException: Requested file maprfs:/tmp/typeall_l/.drill.parquet_metadata does not exist. [Error Id: ff9a2816-d27f-401e-b612-a22c6e6bdf5a on centos-01.qa.lab:31010] org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: FileNotFoundException: Requested file maprfs:/tmp/typeall_l/.drill.parquet_metadata does not exist. [Error Id: ff9a2816-d27f-401e-b612-a22c6e6bdf5a on centos-01.qa.lab:31010] at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:544) ~[drill-common-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:824) [drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:944) [drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:281) [drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91] Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception during fragment initialization: Internal error: Error while applying rule DrillPushProjIntoScan, args [rel#93190:LogicalProject.NONE.ANY([]).[](input=rel#93189:Subset#0.ENUMERABLE.ANY([]).[],col_intrvl_day=$1), rel#93160:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[dfs, tmp, typeall_l])] ... 4 common frames omitted Caused by: java.lang.AssertionError: Internal error: Error while applying rule DrillPushProjIntoScan, args [rel#93190:LogicalProject.NONE.ANY([]).[](input=rel#93189:Subset#0.ENUMERABLE.ANY([]).[],col_intrvl_day=$1), rel#93160:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[dfs, tmp, typeall_l])] at org.apache.calcite.util.Util.newInternal(Util.java:792) ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19] at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:251) ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19] at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:808) ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19] at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:303) ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:404) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:343) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:240) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:290) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:168) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan(DrillSqlWorker.java:122) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:96) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1017) [drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:264) [drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] ... 3 common frames omitted Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: java.io.FileNotFoundException: Requested file maprfs:/tmp/typeall_l/.drill.parquet_metadata does not exist. at org.apache.drill.exec.planner.logical.DrillPushProjIntoScan.onMatch(DrillPushProjIntoScan.java:95) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228) ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19] ... 14 common frames omitted Caused by: java.io.FileNotFoundException: Requested file maprfs:/tmp/typeall_l/.drill.parquet_metadata does not exist. at com.mapr.fs.MapRFileSystem.getMapRFileStatus(MapRFileSystem.java:1374) ~[maprfs-5.2.0.40963-mapr.jar:5.2.0.40963-mapr] at com.mapr.fs.MapRFileSystem.getFileStatus(MapRFileSystem.java:1031) ~[maprfs-5.2.0.40963-mapr.jar:5.2.0.40963-mapr] at org.apache.drill.exec.store.dfs.DrillFileSystem.getFileStatus(DrillFileSystem.java:205) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.store.parquet.Metadata.tableModified(Metadata.java:673) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.store.parquet.Metadata.readBlockMeta(Metadata.java:625) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.store.parquet.Metadata.readBlockMeta(Metadata.java:149) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.store.parquet.ParquetGroupScan.initFromMetadataCache(ParquetGroupScan.java:646) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.store.parquet.ParquetGroupScan.expandIfNecessary(ParquetGroupScan.java:278) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.store.parquet.ParquetGroupScan.<init>(ParquetGroupScan.java:211) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.store.parquet.ParquetGroupScan.<init>(ParquetGroupScan.java:190) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.store.parquet.ParquetFormatPlugin.getGroupScan(ParquetFormatPlugin.java:169) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.store.parquet.ParquetFormatPlugin.getGroupScan(ParquetFormatPlugin.java:67) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.store.dfs.FileSystemPlugin.getPhysicalScan(FileSystemPlugin.java:145) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan(AbstractStoragePlugin.java:103) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.planner.logical.DrillTable.getGroupScan(DrillTable.java:85) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] at org.apache.drill.exec.planner.logical.DrillPushProjIntoScan.onMatch(DrillPushProjIntoScan.java:63) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT] ... 15 common frames omitted {noformat} > Running refresh table metadata concurrently randomly fails with > JsonParseException > ---------------------------------------------------------------------------------- > > Key: DRILL-4831 > URL: https://issues.apache.org/jira/browse/DRILL-4831 > Project: Apache Drill > Issue Type: Bug > Components: Metadata > Affects Versions: 1.8.0 > Reporter: Rahul Challapalli > Assignee: Padma Penumarthy > Attachments: error.log, l_3level.tgz > > > git.commit.id.abbrev=f476eb5 > Just run the below command concurrently from 10 different JDBC connections. > There is a likelihood that you might encounter the below error > Extracts from the log > {code} > Caused By (java.lang.AssertionError) Internal error: Error while applying > rule DrillPushProjIntoScan, args > [rel#189411:LogicalProject.NONE.ANY([]).[](input=rel#189289:Subset#3.ENUMERABLE.ANY([]).[],l_orderkey=$1,dir0=$2,dir1=$3,dir2=$4,l_shipdate=$5,l_extendedprice=$6,l_discount=$7), > rel#189233:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[dfs, > metadata_caching_pp, l_3level])] > org.apache.calcite.util.Util.newInternal():792 > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():251 > ..... > ..... > java.lang.Thread.run():745 > Caused By (org.apache.drill.common.exceptions.DrillRuntimeException) > com.fasterxml.jackson.core.JsonParseException: Illegal character ((CTRL-CHAR, > code 0)): only regular white space (\r, \n, \t) is allowed between tokens > at [Source: com.mapr.fs.MapRFsDataInputStream@57a574a8; line: 1, column: 2] > org.apache.drill.exec.planner.logical.DrillPushProjIntoScan.onMatch():95 > {code} > Attached the complete log message and the data set -- This message was sent by Atlassian JIRA (v6.3.4#6332)