I should mention that this is a kerberized HDFS cluster. I'm still not sure
why the SHOW FILES would work but the query would not--but it could be behind
the issue somehow.
On 7/23/20, 2:18 PM, "Updike, Clark" <clark.upd...@jhuapl.edu> wrote:
No change unfortunately:
apache drill> select * from hdfs.`root`.`/tmp/employee.json`;
Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 18:
Object '/tmp/employee.json' not found within 'hdfs.root'
On 7/23/20, 2:11 PM, "Paul Rogers" <par0...@gmail.com> wrote:
Hi Clark,
Try using `hdfs`.`root` rather than `hdfs.root`. Calcite wants to walk
down
`hdfs` then `root`. There is no workspace called `hdfs.root`.
Thanks,
- Paul
On Thu, Jul 23, 2020 at 8:58 AM Updike, Clark <clark.upd...@jhuapl.edu>
wrote:
> Oops, sorry. No luck there either unfortunately:
>
> apache drill> SELECT * FROM hdfs.`/tmp/employee.json`;
> Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 18:
> Object '/tmp/employee.json' not found within 'hdfs'
>
>
> On 7/23/20, 11:52 AM, "Charles Givre" <cgi...@gmail.com> wrote:
>
> Oh.. I meant:
>
> SELECT *
> FROM hdfs.`/tmp/employee.json`
>
> > On Jul 23, 2020, at 11:41 AM, Updike, Clark
<clark.upd...@jhuapl.edu>
> wrote:
> >
> > No change unfortunately...
> >
> > $ hdfs dfs -ls hdfs://nn01:8020/tmp/employee.json
> > -rw-r--r-- 2 me supergroup 474630 2020-07-23 10:53
> hdfs://nn01:8020/tmp/employee.json
> >
> > apache drill> select * from
> hdfs.root.`hdfs://nn01:8020/tmp/employee.json`;
> > Error: VALIDATION ERROR: From line 1, column 15 to line 1,
column
> 18: Object 'hdfs://nn01:8020/tmp/employee.json' not found within
'hdfs.root'
> >
> >
> > On 7/23/20, 11:30 AM, "Charles Givre" <cgi...@gmail.com> wrote:
> >
> > Hi Clark,
> > That's strange. My initial thought is that this could be a
> permission issue. However, it might also be that Drill isn't finding
the
> file for some reason.
> >
> > Could you try:
> >
> > SELECT *
> > FROM hdfs.`<full hdfs path to file>`
> >
> > Best,
> > --- C
> >
> >
> >> On Jul 23, 2020, at 11:23 AM, Updike, Clark <
> clark.upd...@jhuapl.edu> wrote:
> >>
> >> This is in 1.17. I can use SHOW FILES to list the file I'm
> targeting, but I cannot query it:
> >>
> >> apache drill> show files in hdfs.root.`/tmp/employee.json`;
> >>
>
+---------------+-------------+--------+--------+----------+------------+-------------+-------------------------+-------------------------+
> >> | name | isDirectory | isFile | length | owner |
> group | permissions | accessTime |
modificationTime
> |
> >>
>
+---------------+-------------+--------+--------+----------+------------+-------------+-------------------------+-------------------------+
> >> | employee.json | false | true | 474630 | me |
> supergroup | rw-r--r-- | 2020-07-23 10:53:15.055 | 2020-07-23
> 10:53:15.387 |
> >>
>
+---------------+-------------+--------+--------+----------+------------+-------------+-------------------------+-------------------------+
> >> 1 row selected (3.039 seconds)
> >>
> >>
> >> apache drill> select * from hdfs.root.`/tmp/employee.json`;
> >> Error: VALIDATION ERROR: From line 1, column 15 to line 1,
column
> 18: Object '/tmp/employee.json' not found within 'hdfs.root'
> >> [Error Id: 3b833622-4fac-4ecc-becd-118291cd8560 ]
(state=,code=0)
> >>
> >> The storage plugin uses the standard json config:
> >>
> >> "json": {
> >> "type": "json",
> >> "extensions": [
> >> "json"
> >> ]
> >> },
> >>
> >> I can't see any problems on the HDFS side. Full stack trace is
> below.
> >>
> >> Any ideas what could be causing this behavior?
> >>
> >> Thanks, Clark
> >>
> >>
> >>
> >> FULL STACKTRACE:
> >>
> >> apache drill> select * from hdfs.root.`/tmp/employee.json`;
> >> Error: VALIDATION ERROR: From line 1, column 15 to line 1,
column
> 18: Object '/tmp/employee.json' not found within 'hdfs.root'
> >>
> >>
> >> [Error Id: 69c8ffc0-4933-4008-a786-85ad623578ea ]
> >>
> >> (org.apache.calcite.runtime.CalciteContextException) From line
1,
> column 15 to line 1, column 18: Object '/tmp/employee.json' not found
> within 'hdfs.root'
> >> sun.reflect.NativeConstructorAccessorImpl.newInstance0():-2
> >> sun.reflect.NativeConstructorAccessorImpl.newInstance():62
> >>
sun.reflect.DelegatingConstructorAccessorImpl.newInstance():45
> >> java.lang.reflect.Constructor.newInstance():423
> >> org.apache.calcite.runtime.Resources$ExInstWithCause.ex():463
> >> org.apache.calcite.sql.SqlUtil.newContextException():824
> >> org.apache.calcite.sql.SqlUtil.newContextException():809
> >>
>
org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError():4805
> >>
> org.apache.calcite.sql.validate.IdentifierNamespace.resolveImpl():127
> >>
>
org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl():177
> >>
org.apache.calcite.sql.validate.AbstractNamespace.validate():84
> >>
>
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():995
> >>
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():955
> >>
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():3109
> >>
>
org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():298
> >>
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():3091
> >>
>
org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():298
> >>
>
org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3363
> >>
org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60
> >>
org.apache.calcite.sql.validate.AbstractNamespace.validate():84
> >>
>
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():995
> >>
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():955
> >> org.apache.calcite.sql.SqlSelect.validate():216
> >>
>
org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression():930
> >>
org.apache.calcite.sql.validate.SqlValidatorImpl.validate():637
> >> org.apache.drill.exec.planner.sql.SqlConverter.validate():218
> >>
>
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode():665
> >>
>
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert():199
> >>
>
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():172
> >>
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():282
> >>
>
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan():162
> >>
org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan():127
> >> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():92
> >> org.apache.drill.exec.work.foreman.Foreman.runSQL():590
> >> org.apache.drill.exec.work.foreman.Foreman.run():275
> >> java.util.concurrent.ThreadPoolExecutor.runWorker():1142
> >> java.util.concurrent.ThreadPoolExecutor$Worker.run():617
> >> java.lang.Thread.run():745
> >> Caused By
(org.apache.calcite.sql.validate.SqlValidatorException)
> Object '/tmp/employee.json' not found within 'hdfs.root'
> >> sun.reflect.NativeConstructorAccessorImpl.newInstance0():-2
> >> sun.reflect.NativeConstructorAccessorImpl.newInstance():62
> >>
sun.reflect.DelegatingConstructorAccessorImpl.newInstance():45
> >> java.lang.reflect.Constructor.newInstance():423
> >> org.apache.calcite.runtime.Resources$ExInstWithCause.ex():463
> >> org.apache.calcite.runtime.Resources$ExInst.ex():572
> >> org.apache.calcite.sql.SqlUtil.newContextException():824
> >> org.apache.calcite.sql.SqlUtil.newContextException():809
> >>
>
org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError():4805
> >>
> org.apache.calcite.sql.validate.IdentifierNamespace.resolveImpl():127
> >>
>
org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl():177
> >>
org.apache.calcite.sql.validate.AbstractNamespace.validate():84
> >>
>
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():995
> >>
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():955
> >>
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():3109
> >>
>
org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():298
> >>
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():3091
> >>
>
org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():298
> >>
>
org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3363
> >>
org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60
> >>
org.apache.calcite.sql.validate.AbstractNamespace.validate():84
> >>
>
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():995
> >>
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():955
> >> org.apache.calcite.sql.SqlSelect.validate():216
> >>
>
org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression():930
> >>
org.apache.calcite.sql.validate.SqlValidatorImpl.validate():637
> >> org.apache.drill.exec.planner.sql.SqlConverter.validate():218
> >>
>
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode():665
> >>
>
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert():199
> >>
>
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():172
> >>
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():282
> >>
>
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan():162
> >>
org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan():127
> >> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():92
> >> org.apache.drill.exec.work.foreman.Foreman.runSQL():590
> >> org.apache.drill.exec.work.foreman.Foreman.run():275
> >> java.util.concurrent.ThreadPoolExecutor.runWorker():1142
> >> java.util.concurrent.ThreadPoolExecutor$Worker.run():617
> >> java.lang.Thread.run():745 (state=,code=0)
> >
> >
> >
>
>
>
>
