I am using Oracle storage cloud service. I did not run refresh table metadata. Even refresh table metadata fails with this message.
I think it depends on external network . One another day, the same query works fine.. Regards, Projjwal On Mon, Oct 9, 2017 at 8:23 PM, Padma Penumarthy <ppenumar...@mapr.com> wrote: > which cloud service is this ? > It is not able to read parquet metadata. Did you run refresh table > metadata to > generate parquet metadata ? > Can you manually check if there is parquet metadata file > (.drill.parquet_metadata) > in the directory you used in the query i.e. `data25Goct6/websales` ? > > Thanks > Padma > > > On Oct 9, 2017, at 5:50 AM, PROJJWAL SAHA <proj.s...@gmail.com<mailto:pr > oj.s...@gmail.com>> wrote: > > Hello all, > > I am getting the below exception when querying parquet data stored in > storage cloud service.What does this exception point to ? > The query on the same parquet files works when they are stored in > alluxio.which means the data is fine. > I am using drill 11.1 > > Any help is appreciated ! > > Regards, > Projjwal > > 2017-10-09 08:11:10,221 [262498a1-4fc1-608e-a7bc-ab2c6ddc09c9:foreman] > INFO o.a.drill.exec.work.foreman.Foreman - Query text for query id > 262498a1-4fc1-608e-a7bc-ab2c6ddc09c9: select count(*) from > `data25Goct6/websales` > 2017-10-09 08:11:38,117 [262498a1-4fc1-608e-a7bc-ab2c6ddc09c9:foreman] > INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() > took 0 ms, numFiles: 1 > 2017-10-09 08:11:58,362 [262498a1-4fc1-608e-a7bc-ab2c6ddc09c9:foreman] > INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() > took 0 ms, numFiles: 1 > 2017-10-09 08:15:28,459 [262498a1-4fc1-608e-a7bc-ab2c6ddc09c9:foreman] > INFO o.a.d.exec.store.parquet.Metadata - Took 105962 ms to get file > statuses > 2017-10-09 08:16:00,651 [262498a1-4fc1-608e-a7bc-ab2c6ddc09c9:foreman] > ERROR o.a.d.exec.store.parquet.Metadata - Waited for 27187ms, but > tasks for 'Fetch parquet metadata' are not complete. Total runnable > size 29, parallelism 16. > 2017-10-09 08:16:00,652 [262498a1-4fc1-608e-a7bc-ab2c6ddc09c9:foreman] > INFO o.a.d.exec.store.parquet.Metadata - User Error Occurred: Waited > for 27187ms, but tasks for 'Fetch parquet metadata' are not complete. > Total runnable size 29, parallelism > 16.org.apache.drill.common.exceptions.UserException: RESOURCE ERROR: > Waited for 27187ms, but tasks for 'Fetch parquet metadata' are not > complete. Total runnable size 29, parallelism 16. > > > [Error Id: d9b6ee72-2e81-49ae-846c-61a14931b7ab ] > at org.apache.drill.common.exceptions.UserException$ > Builder.build(UserException.java:550) > ~[drill-common-1.11.0.jar:1.11.0] > at org.apache.drill.exec.store.TimedRunnable.run(TimedRunnable.java:151) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.store.parquet.Metadata.getParquetFileMetadata_v3( > Metadata.java:293) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.store.parquet.Metadata.getParquetTableMetadata( > Metadata.java:270) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.store.parquet.Metadata.getParquetTableMetadata( > Metadata.java:255) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.store.parquet.Metadata.getParquetTableMetadata( > Metadata.java:117) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.store.parquet.ParquetGroupScan.init( > ParquetGroupScan.java:730) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.store.parquet.ParquetGroupScan.< > init>(ParquetGroupScan.java:226) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.store.parquet.ParquetGroupScan.< > init>(ParquetGroupScan.java:186) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.store.parquet.ParquetFormatPlugin.getGroupScan( > ParquetFormatPlugin.java:170) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.store.parquet.ParquetFormatPlugin.getGroupScan( > ParquetFormatPlugin.java:66) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.store.dfs.FileSystemPlugin.getPhysicalScan( > FileSystemPlugin.java:144) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan( > AbstractStoragePlugin.java:100) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.planner.logical.DrillTable. > getGroupScan(DrillTable.java:85) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.planner.logical.DrillPushProjIntoScan. > onMatch(DrillPushProjIntoScan.java:63) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.calcite.plan.volcano.VolcanoRuleCall. > onMatch(VolcanoRuleCall.java:228) > [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21] > at org.apache.calcite.plan.volcano.VolcanoPlanner. > findBestExp(VolcanoPlanner.java:811) > [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21] > at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:310) > [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21] > at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform( > DefaultSqlHandler.java:401) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform( > DefaultSqlHandler.java:343) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler. > convertToRawDrel(DefaultSqlHandler.java:242) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler. > convertToDrel(DefaultSqlHandler.java:292) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan( > DefaultSqlHandler.java:169) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.planner.sql.DrillSqlWorker. > getQueryPlan(DrillSqlWorker.java:131) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan( > DrillSqlWorker.java:79) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1050) > [drill-java-exec-1.11.0.jar:1.11.0] > at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:280) > [drill-java-exec-1.11.0.jar:1.11.0] > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > [na:1.8.0_121] > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > [na:1.8.0_121] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] > >
