[
https://issues.apache.org/jira/browse/DRILL-405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949117#comment-13949117
]
Bhallamudi Venkata Siva Kamesh commented on DRILL-405:
------------------------------------------------------
I think the problem is BasicFormatMatcher does not support directory reads
where as ParquetFormatMatcher supports directory reads. So, it reads all the
*.parquet* extension files. But if the directory contains other extension files
than *.parquet*, it fails to read those files.
I think we have to support directory reads in BasicFormatMatcher only.
Please comment on this.
> Not able to query directory containing json files
> -------------------------------------------------
>
> Key: DRILL-405
> URL: https://issues.apache.org/jira/browse/DRILL-405
> Project: Apache Drill
> Issue Type: Bug
> Reporter: Ramana Inukonda Nagaraj
>
> Trying to query a directory which contains json files fails. The same test
> works for parquet files. Also if entire path upto the file is specified the
> json file can be queried successfully.
> 0: jdbc:drill:schema=dfs> select * from dfs.`/user/root/testJsonDir`;
> Query failed: org.apache.drill.exec.rpc.RpcException: Remote failure while
> running query.[error_id: "afb20373-1010-4d2d-8a88-d89015db5801"
> endpoint {
> address: "perfnode166.perf.lab"
> user_port: 31010
> control_port: 31011
> data_port: 31012
> }
> error_type: 0
> message: "Failure while parsing sql. < ValidationException:[
> org.eigenbase.util.EigenbaseContextException: From line 1, column 15 to line
> 1, column 42 ] < EigenbaseContextException:[ From line 1, column 15 to line
> 1, column 42 ] < SqlValidatorException:[ Table \'dfs./user/root/testJsonDir\'
> not found ]"
> ]
> Error: exception while executing query (state=,code=0)
> Log file has
> 13:16:57.398 [WorkManager-2] DEBUG o.a.d.e.w.fragment.FragmentExecutor -
> Fragment runner complete. 0:0
> 13:17:03.797 [WorkManager Event Thread] DEBUG
> o.apache.drill.exec.work.WorkManager - Starting pending task
> org.apache.drill.exec.work.foreman.Foreman@771fa2d1
> 13:17:03.806 [WorkManager-3] DEBUG o.a.d.e.s.dfs.WorkspaceSchemaFactory -
> File read failed.
> java.io.IOException: Open failed for file: /user/root/testJsonDir, error:
> Invalid argument (22)
> at com.mapr.fs.MapRClientImpl.open(MapRClientImpl.java:141)
> ~[maprfs-core-1.0.3-mapr-3.0.2.jar:na]
> at com.mapr.fs.MapRFileSystem.open(MapRFileSystem.java:499)
> ~[maprfs-core-1.0.3-mapr-3.0.2.jar:na]
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:463)
> ~[hadoop-0.20.2-dev-core.jar:na]
> at
> org.apache.drill.exec.store.dfs.shim.fallback.FallbackFileSystem.open(FallbackFileSystem.java:89)
>
> ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
> at
> org.apache.drill.exec.store.dfs.BasicFormatMatcher$MagicStringMatcher.matches(BasicFormatMatcher.java:104)
>
> ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
> at
> org.apache.drill.exec.store.dfs.BasicFormatMatcher.isReadable(BasicFormatMatcher.java:77)
>
> ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
> at
> org.apache.drill.exec.store.dfs.BasicFormatMatcher.isReadable(BasicFormatMatcher.java:64)
>
> ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
> at
> org.apache.drill.exec.store.parquet.ParquetFormatPlugin$ParquetFormatMatcher.isReadable(ParquetFormatPlugin.java:163)
>
> ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
> at
> org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory.create(WorkspaceSchemaFactory.java:77)
>
> [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
> at
> org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory.create(WorkspaceSchemaFactory.java:35)
>
> [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
> at
> org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.getNewEntry(ExpandingConcurrentMap.java:83)
>
> [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
> at
> org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.containsKey(ExpandingConcurrentMap.java:64)
>
> [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
> at
> org.apache.drill.exec.planner.sql.ExpandingConcurrentMap$DelegatingKeySet.contains(ExpandingConcurrentMap.java:160)
>
> [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
> at com.google.common.collect.Maps$AsMapView.containsKey(Maps.java:788)
> [guava-14.0.1.jar:na]
> at net.hydromatic.optiq.util.CompositeMap.get(CompositeMap.java:91)
> [optiq-core-0.4.18.jar:na]
> at
> net.hydromatic.optiq.prepare.OptiqCatalogReader.getTableFrom(OptiqCatalogReader.java:89)
> [optiq-core-0.4.18.jar:na]
> at
> net.hydromatic.optiq.prepare.OptiqCatalogReader.getTable(OptiqCatalogReader.java:77)
> [optiq-core-0.4.18.jar:na]
> at
> net.hydromatic.optiq.prepare.OptiqCatalogReader.getTable(OptiqCatalogReader.java:43)
> [optiq-core-0.4.18.jar:na]
> at
> org.eigenbase.sql.validate.IdentifierNamespace.validateImpl(IdentifierNamespace.java:67)
> [optiq-core-0.4.18.jar:na]
> at
> org.eigenbase.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:80)
> [optiq-core-0.4.18.jar:na]
> at
> org.eigenbase.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:753)
> [optiq-core-0.4.18.jar:na]
> at
> org.eigenbase.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:742)
> [optiq-core-0.4.18.jar:na]
> at
> org.eigenbase.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:2672)
> [optiq-core-0.4.18.jar:na]
> at
> org.eigenbase.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:2899)
> [optiq-core-0.4.18.jar:na]
> at
> org.eigenbase.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
> [optiq-core-0.4.18.jar:na]
> at
> org.eigenbase.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:80)
> [optiq-core-0.4.18.jar:na]
> at
> org.eigenbase.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:753)
> [optiq-core-0.4.18.jar:na]
> at
> org.eigenbase.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:742)
> [optiq-core-0.4.18.jar:na]
> at org.eigenbase.sql.SqlSelect.validate(SqlSelect.java:135)
> [optiq-core-0.4.18.jar:na]
> at
> org.eigenbase.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:707)
> [optiq-core-0.4.18.jar:na]
> at
> org.eigenbase.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:423)
> [optiq-core-0.4.18.jar:na]
> at
> net.hydromatic.optiq.prepare.PlannerImpl.validate(PlannerImpl.java:153)
> [optiq-core-0.4.18.jar:na]
> at
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:76)
>
> [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
> at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:350)
> [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:175)
> [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
--
This message was sent by Atlassian JIRA
(v6.2#6252)
