[jira] [Created] (DRILL-1881) Test o.a.d.TestBugFixes#testVersionTable fails while building from source tarball
Aditya Kishore created DRILL-1881: - Summary: Test o.a.d.TestBugFixes#testVersionTable fails while building from source tarball Key: DRILL-1881 URL: https://issues.apache.org/jira/browse/DRILL-1881 Project: Apache Drill Issue Type: Bug Components: Storage - Information Schema Affects Versions: 0.7.0 Reporter: Aditya Kishore Assignee: Aditya Kishore Fix For: 0.7.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 29129: DRILL-1385: bytecode rewriting to move holder values to locals
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29129/ --- (Updated Dec. 17, 2014, 2:27 a.m.) Review request for drill and Jacques Nadeau. Changes --- added github link Bugs: DRILL-1385 https://issues.apache.org/jira/browse/DRILL-1385 Repository: drill-git Description (updated) --- DRILL-1385: bytecode rewriting to move holder values to locals; see diff for longer list of work To see this in my github fork, see the sr1 branch https://github.com/cwestin/incubator-drill/tree/sr1 Diffs - Diff: https://reviews.apache.org/r/29129/diff/ Testing --- Maven Install TPCH SF100 Functional Passing - New (but got a verification failure in joinCSVParquet.q - not sure about this one) File Attachments patch - reviewboard refuses to upload diff https://reviews.apache.org/media/uploaded/files/2014/12/17/ccbd0221-4947-43ee-880f-f6dd501b5b50__drill-1385.1.patch.txt Thanks, Chris Westin
[jira] [Resolved] (DRILL-1880) Query with three where conditions returned wrong results
[
https://issues.apache.org/jira/browse/DRILL-1880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chun Chang resolved DRILL-1880.
---
Resolution: Invalid
> Query with three where conditions returned wrong results
>
>
> Key: DRILL-1880
> URL: https://issues.apache.org/jira/browse/DRILL-1880
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
>Affects Versions: 0.7.0
>Reporter: Chun Chang
>
> #Mon Dec 15 11:37:23 EST 2014
> git.commit.id.abbrev=3b0ff5d
> The following query containing three where conditions returned wrong results.
> (data is too big so not included here.)
> {code}
> SELECT t.gbyi,
>Count(t.id)
> FROM `complex.json` t
> WHERE t.gbyi <= 5
> OR t.gbyi >= 11
>AND t.gbyt <> 'ooof'
> GROUP BY t.gbyi
> ORDER BY t.gbyi;
> {code}
> Wrong result, the count column is mostly wrong:
> {code}
> 0: jdbc:drill:schema=dfs.drillTestDirComplexJ> select t.gbyi, count(t.id)
> from `complex.json` t where t.gbyi <= 5 or t.gbyi >= 11 and t.gbyt <> 'ooof'
> group by t.gbyi order by t.gbyi;
> +++
> |gbyi| EXPR$1 |
> +++
> | 0 | 66943 |
> | 1 | 66318 |
> | 2 | 66994 |
> | 3 | 66683 |
> | 4 | 66638 |
> | 5 | 66439 |
> | 11 | 63172 |
> | 12 | 63008 |
> | 13 | 62685 |
> | 14 | 62970 |
> +++
> {code}
> Reduce the where condition to just two gives the correct result:
> {code}
> SELECT t.gbyi,
>Count(t.id)
> FROM `complex.json` t
> WHERE t.gbyi <= 5
>AND t.gbyt <> 'ooof'
> GROUP BY t.gbyi
> ORDER BY t.gbyi;
> {code}
> {code}
> 0: jdbc:drill:schema=dfs.drillTestDirComplexJ> select t.gbyi, count(t.id)
> from `complex.json` t where t.gbyi <= 5 and t.gbyt <> 'ooof' group by t.gbyi
> order by t.gbyi;
> +++
> |gbyi| EXPR$1 |
> +++
> | 0 | 63305 |
> | 1 | 62671 |
> | 2 | 63249 |
> | 3 | 63070 |
> | 4 | 62967 |
> | 5 | 62737 |
> +++
> {code}
> physical plan for the query returned wrong result:
> {code}
> 0: jdbc:drill:schema=dfs.drillTestDirComplexJ> explain plan for select
> t.gbyi, count(t.id) from `complex.json` t where t.gbyi <= 5 or t.gbyi >= 11
> and t.gbyt <> 'ooof' group by t.gbyi order by t.gbyi;
> +++
> |text|json|
> +++
> | 00-00Screen
> 00-01 Project(gbyi=[$0], EXPR$1=[$1])
> 00-02SelectionVectorRemover
> 00-03 Sort(sort0=[$0], dir0=[ASC])
> 00-04HashAgg(group=[{0}], EXPR$1=[$SUM0($1)])
> 00-05 HashAgg(group=[{0}], EXPR$1=[COUNT($1)])
> 00-06Project(gbyi=[$0], id=[$2])
> 00-07 SelectionVectorRemover
> 00-08Filter(condition=[OR(<=($0, 5), AND(>=($0, 11),
> <>($1, 'ooof')))])
> 00-09 Project(gbyi=[$1], gbyt=[$2], id=[$0])
> 00-10Scan(groupscan=[EasyGroupScan
> [selectionRoot=/drill/testdata/complex_type/json/complex.json, numFiles=1,
> columns=[`gbyi`, `gbyt`, `id`],
> files=[maprfs:/drill/testdata/complex_type/json/complex.json]]])
> | {
> "head" : {
> "version" : 1,
> "generator" : {
> "type" : "ExplainHandler",
> "info" : ""
> },
> "type" : "APACHE_DRILL_PHYSICAL",
> "options" : [ ],
> "queue" : 0,
> "resultMode" : "EXEC"
> },
> "graph" : [ {
> "pop" : "fs-scan",
> "@id" : 10,
> "files" : [ "maprfs:/drill/testdata/complex_type/json/complex.json" ],
> "storage" : {
> "type" : "file",
> "enabled" : true,
> "connection" : "maprfs:///",
> "workspaces" : {
> "root" : {
> "location" : "/",
> "writable" : false,
> "defaultInputFormat" : null
> },
> "tmp" : {
> "location" : "/tmp",
> "writable" : true,
> "defaultInputFormat" : "csv"
> },
> "drillTestDir" : {
> "location" : "/drill/testdata/",
> "writable" : true,
> "defaultInputFormat" : "parquet"
> },
> "drillTestDirComplexJson" : {
> "location" : "/drill/testdata/complex_type/json",
> "writable" : true,
> "defaultInputFormat" : "json"
> },
> "drillTestDirAmplab" : {
> "location" : "/drill/testdata/amplab",
> "writable" : true,
> "defaultInputFormat" : "parquet"
> },
> "drillTestDirInformationSchema" : {
> "location" : "/drill/testdata/information-schema",
>
Re: [VOTE] Release Apache Drill 0.7.0
Sounds good. That sinks this release. Let's see if we can spin another soon. thanks! On Tue, Dec 16, 2014 at 4:07 PM, Aditya wrote: > > -1 (binding) > > I ran into failures while > > 1) Running Drill unit tests on Windows, and > 2) Starting Drillbit on Windows in distributed mode. > > So I am -1 on this. > > Will create JIRAs for individual issues and put patches in a few hours. > > On Tue, Dec 16, 2014 at 9:45 AM, Norris Lee wrote: > > > > +1 > > > > - Built from source > > - Queried against Hive, HBase, CSV, JSON, Parquet files/tables > > > > Norris > > > > -Original Message- > > From: Neeraja Rentachintala [mailto:nrentachint...@maprtech.com] > > Sent: Tuesday, December 16, 2014 6:34 AM > > To: dev@drill.apache.org > > Subject: Re: [VOTE] Release Apache Drill 0.7.0 > > > > +1(non-binding) > > > > On Tue, Dec 16, 2014 at 12:15 AM, Mehdi Chitforoosh < > > mehdichitforo...@gmail.com> wrote: > > > > > > +1 > > > > > > On Tue, Dec 16, 2014 at 10:14 AM, Yash Sharma > wrote: > > > > > > > > +1 > > > > > > > > - Source and binaries present > > > > - Source and Binaries contain README, NOTICE, LICENSE Files. > > > > - Source contains INSTALL.md > > > > - Verified Checksums for src and binaries > > > > - Able to launch drill from binary distribution (Embeded mode). > > > > - Able to build from source > > > > - Able to fire sample queries on sqlline > > > > > > > > On Mon, Dec 15, 2014 at 11:56 PM, Tomer Shiran > > > wrote: > > > > > > > > > > +1 > > > > > > > > > > On Mon, Dec 15, 2014 at 9:39 AM, Jacques Nadeau > > > > > > > > > > wrote: > > > > > > > > > > > > Good morning, > > > > > > > > > > > > I would like to propose the release of Apache Drill, version > 0.7.0. > > > > > > > > > > > > This release includes 223 resolved JIRAs [1]. > > > > > > > > > > > > The artifacts are hosted at [2]. > > > > > > > > > > > > The vote will be open for 72 hours, ending 10 AM Pacific, > > > > > > December > > > 18, > > > > > > 2014. > > > > > > > > > > > > [ ] +1 > > > > > > [ ] +0 > > > > > > [ ] -1 > > > > > > > > > > > > > > > > > > Thank you, > > > > > > Jacques > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313 > > > 820&version=12327473 > > > > > > [2] http://people.apache.org/~jacques/apache-drill-0.7.0.rc0/ > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Mehdi Chitforoosh > > > > > >
[jira] [Created] (DRILL-1880) Query with three where conditions returned wrong results
Chun Chang created DRILL-1880:
-
Summary: Query with three where conditions returned wrong results
Key: DRILL-1880
URL: https://issues.apache.org/jira/browse/DRILL-1880
Project: Apache Drill
Issue Type: Bug
Components: Execution - Flow
Affects Versions: 0.7.0
Reporter: Chun Chang
#Mon Dec 15 11:37:23 EST 2014
git.commit.id.abbrev=3b0ff5d
The following query containing three where conditions returned wrong results.
(data is too big so not included here.)
{code}
SELECT t.gbyi,
Count(t.id)
FROM `complex.json` t
WHERE t.gbyi <= 5
OR t.gbyi >= 11
AND t.gbyt <> 'ooof'
GROUP BY t.gbyi
ORDER BY t.gbyi;
{code}
Wrong result, the count column is mostly wrong:
{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ> select t.gbyi, count(t.id) from
`complex.json` t where t.gbyi <= 5 or t.gbyi >= 11 and t.gbyt <> 'ooof' group
by t.gbyi order by t.gbyi;
+++
|gbyi| EXPR$1 |
+++
| 0 | 66943 |
| 1 | 66318 |
| 2 | 66994 |
| 3 | 66683 |
| 4 | 66638 |
| 5 | 66439 |
| 11 | 63172 |
| 12 | 63008 |
| 13 | 62685 |
| 14 | 62970 |
+++
{code}
Reduce the where condition to just two gives the correct result:
{code}
SELECT t.gbyi,
Count(t.id)
FROM `complex.json` t
WHERE t.gbyi <= 5
AND t.gbyt <> 'ooof'
GROUP BY t.gbyi
ORDER BY t.gbyi;
{code}
{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ> select t.gbyi, count(t.id) from
`complex.json` t where t.gbyi <= 5 and t.gbyt <> 'ooof' group by t.gbyi order
by t.gbyi;
+++
|gbyi| EXPR$1 |
+++
| 0 | 63305 |
| 1 | 62671 |
| 2 | 63249 |
| 3 | 63070 |
| 4 | 62967 |
| 5 | 62737 |
+++
{code}
physical plan for the query returned wrong result:
{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ> explain plan for select t.gbyi,
count(t.id) from `complex.json` t where t.gbyi <= 5 or t.gbyi >= 11 and t.gbyt
<> 'ooof' group by t.gbyi order by t.gbyi;
+++
|text|json|
+++
| 00-00Screen
00-01 Project(gbyi=[$0], EXPR$1=[$1])
00-02SelectionVectorRemover
00-03 Sort(sort0=[$0], dir0=[ASC])
00-04HashAgg(group=[{0}], EXPR$1=[$SUM0($1)])
00-05 HashAgg(group=[{0}], EXPR$1=[COUNT($1)])
00-06Project(gbyi=[$0], id=[$2])
00-07 SelectionVectorRemover
00-08Filter(condition=[OR(<=($0, 5), AND(>=($0, 11), <>($1,
'ooof')))])
00-09 Project(gbyi=[$1], gbyt=[$2], id=[$0])
00-10Scan(groupscan=[EasyGroupScan
[selectionRoot=/drill/testdata/complex_type/json/complex.json, numFiles=1,
columns=[`gbyi`, `gbyt`, `id`],
files=[maprfs:/drill/testdata/complex_type/json/complex.json]]])
| {
"head" : {
"version" : 1,
"generator" : {
"type" : "ExplainHandler",
"info" : ""
},
"type" : "APACHE_DRILL_PHYSICAL",
"options" : [ ],
"queue" : 0,
"resultMode" : "EXEC"
},
"graph" : [ {
"pop" : "fs-scan",
"@id" : 10,
"files" : [ "maprfs:/drill/testdata/complex_type/json/complex.json" ],
"storage" : {
"type" : "file",
"enabled" : true,
"connection" : "maprfs:///",
"workspaces" : {
"root" : {
"location" : "/",
"writable" : false,
"defaultInputFormat" : null
},
"tmp" : {
"location" : "/tmp",
"writable" : true,
"defaultInputFormat" : "csv"
},
"drillTestDir" : {
"location" : "/drill/testdata/",
"writable" : true,
"defaultInputFormat" : "parquet"
},
"drillTestDirComplexJson" : {
"location" : "/drill/testdata/complex_type/json",
"writable" : true,
"defaultInputFormat" : "json"
},
"drillTestDirAmplab" : {
"location" : "/drill/testdata/amplab",
"writable" : true,
"defaultInputFormat" : "parquet"
},
"drillTestDirInformationSchema" : {
"location" : "/drill/testdata/information-schema",
"writable" : true,
"defaultInputFormat" : "parquet"
},
"drillTestDirUdfs" : {
"location" : "/drill/testdata/udfs/",
"writable" : true,
"defaultInputFormat" : "parquet"
},
"drillTestDirP1" : {
"location" : "/drill/testdata/p1tests",
"writable" : true,
"defaultInputFormat" : "parquet"
},
"drillTestDirTpch10Parquet" : {
"l
[jira] [Created] (DRILL-1879) Unit test failures on Windows
Aditya Kishore created DRILL-1879:
-
Summary: Unit test failures on Windows
Key: DRILL-1879
URL: https://issues.apache.org/jira/browse/DRILL-1879
Project: Apache Drill
Issue Type: Bug
Components: Tools, Build & Test
Affects Versions: 0.7.0
Reporter: Aditya Kishore
Assignee: Aditya Kishore
Fix For: 0.7.0
TestJsonReader#schemaChange failed with the following error:
{noformat}
org.apache.drill.exec.rpc.RpcException: Query failed: Failure validating SQL.
org.eigenbase.util.EigenbaseContextException:
>From line 1, column 15 to line 1, column 17:
Table
'dfs.X:gitapache-drillexecjava-exec/src/test/resources/vector/complex/writer/schemaChange/'
not found
at
org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:79)
~[classes/:na]
at
org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:93)
~[classes/:na]
at
org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:52)
~[classes/:na]
at
org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:1)
~[classes/:na]
at org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:58)
~[classes/:na]
at
org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:194)
~[classes/:na]
at
org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:1)
~[classes/:na]
{20 more}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-1878) Unable to start Drillbit in distributed mode on Windows
Aditya Kishore created DRILL-1878:
-
Summary: Unable to start Drillbit in distributed mode on Windows
Key: DRILL-1878
URL: https://issues.apache.org/jira/browse/DRILL-1878
Project: Apache Drill
Issue Type: Bug
Components: Metadata
Affects Versions: 0.7.0
Reporter: Aditya Kishore
Assignee: Aditya Kishore
Priority: Critical
Fix For: 0.7.0
Attempts to run Drillbit on Windows fails and Drillbit shutdowns with the
following error.
{noformat}
2014-12-16 13:26:07,808 [main] INFO o.a.d.exec.store.sys.PStoreRegistry -
Using the configured PStoreProvider class:
'org.apache.drill.exec.store.sys.zk.ZkPStoreProvider'.
2014-12-16 13:26:08,289 [main] ERROR o.a.d.exec.store.sys.PStoreRegistry - null
java.lang.reflect.InvocationTargetException: null
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method) ~[na:1.7.0_67]
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
~[na:1.7.0_67]
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
~[na:1.7.0_67]
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
~[na:1.7.0_67]
at
org.apache.drill.exec.store.sys.PStoreRegistry.newPStoreProvider(PStoreRegistry.java:56)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at org.apache.drill.exec.server.Drillbit.(Drillbit.java:113)
[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:66)
[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:59)
[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at org.apache.drill.exec.server.Drillbit.main(Drillbit.java:82)
[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
Caused by: java.lang.IllegalArgumentException: Wrong FS:
file://C:\cygwin\var\log\drill, expected: file:///
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:643)
~[hadoop-common-2.4.1.jar:na]
at
org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:79)
~[hadoop-common-2.4.1.jar:na]
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:405)
~[hadoop-common-2.4.1.jar:na]
at
org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:584)
~[hadoop-common-2.4.1.jar:na]
at
org.apache.drill.exec.store.sys.local.FilePStore.getFileSystem(FilePStore.java:88)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.store.sys.zk.ZkPStoreProvider.(ZkPStoreProvider.java:64)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
... 9 common frames omitted
2014-12-16 13:26:08,299 [ShutdownHook] INFO
o.apache.drill.exec.server.Drillbit - Received shutdown request.
{noformat}
The manually built URI at
[FilePStore.java:76|https://github.com/apache/drill/blob/fc58c693ae0cd9234e5e62a7602b4cc216b8549d/exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/FilePStore.java#L76]
can not be parsed correctly by Hadoop FileSystem class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Review Request 29129: DRILL-1385: bytecode rewriting to move holder values to locals
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29129/ --- Review request for drill and Jacques Nadeau. Bugs: DRILL-1385 https://issues.apache.org/jira/browse/DRILL-1385 Repository: drill-git Description --- DRILL-1385: bytecode rewriting to move holder values to locals; see diff for longer list of work Diffs - Diff: https://reviews.apache.org/r/29129/diff/ Testing --- Maven Install TPCH SF100 Functional Passing - New (but got a verification failure in joinCSVParquet.q - not sure about this one) File Attachments patch - reviewboard refuses to upload diff https://reviews.apache.org/media/uploaded/files/2014/12/17/ccbd0221-4947-43ee-880f-f6dd501b5b50__drill-1385.1.patch.txt Thanks, Chris Westin
Re: [VOTE] Release Apache Drill 0.7.0
-1 (binding) I ran into failures while 1) Running Drill unit tests on Windows, and 2) Starting Drillbit on Windows in distributed mode. So I am -1 on this. Will create JIRAs for individual issues and put patches in a few hours. On Tue, Dec 16, 2014 at 9:45 AM, Norris Lee wrote: > > +1 > > - Built from source > - Queried against Hive, HBase, CSV, JSON, Parquet files/tables > > Norris > > -Original Message- > From: Neeraja Rentachintala [mailto:nrentachint...@maprtech.com] > Sent: Tuesday, December 16, 2014 6:34 AM > To: dev@drill.apache.org > Subject: Re: [VOTE] Release Apache Drill 0.7.0 > > +1(non-binding) > > On Tue, Dec 16, 2014 at 12:15 AM, Mehdi Chitforoosh < > mehdichitforo...@gmail.com> wrote: > > > > +1 > > > > On Tue, Dec 16, 2014 at 10:14 AM, Yash Sharma wrote: > > > > > > +1 > > > > > > - Source and binaries present > > > - Source and Binaries contain README, NOTICE, LICENSE Files. > > > - Source contains INSTALL.md > > > - Verified Checksums for src and binaries > > > - Able to launch drill from binary distribution (Embeded mode). > > > - Able to build from source > > > - Able to fire sample queries on sqlline > > > > > > On Mon, Dec 15, 2014 at 11:56 PM, Tomer Shiran > > wrote: > > > > > > > > +1 > > > > > > > > On Mon, Dec 15, 2014 at 9:39 AM, Jacques Nadeau > > > > > > > > wrote: > > > > > > > > > > Good morning, > > > > > > > > > > I would like to propose the release of Apache Drill, version 0.7.0. > > > > > > > > > > This release includes 223 resolved JIRAs [1]. > > > > > > > > > > The artifacts are hosted at [2]. > > > > > > > > > > The vote will be open for 72 hours, ending 10 AM Pacific, > > > > > December > > 18, > > > > > 2014. > > > > > > > > > > [ ] +1 > > > > > [ ] +0 > > > > > [ ] -1 > > > > > > > > > > > > > > > Thank you, > > > > > Jacques > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313 > > 820&version=12327473 > > > > > [2] http://people.apache.org/~jacques/apache-drill-0.7.0.rc0/ > > > > > > > > > > > > > > > > > > -- > > Mehdi Chitforoosh > > >
[jira] [Created] (DRILL-1877) Throw unsupported error message for || operator early in the parsing stage
Victoria Markman created DRILL-1877: --- Summary: Throw unsupported error message for || operator early in the parsing stage Key: DRILL-1877 URL: https://issues.apache.org/jira/browse/DRILL-1877 Project: Apache Drill Issue Type: Bug Components: SQL Parser Affects Versions: 0.7.0 Reporter: Victoria Markman If we don't plan to support concat operator (||), we should throw a decent error message early. The error is cryptic: 0: jdbc:drill:schema=dfs> select cast(l_orderkey as varchar(10)) || '-' || cast(l_partkey as varchar(10)) from cp.`tpch/lineitem.parquet`; Query failed: Query stopped., Line 65, Column 29: "value" is neither a method, a field, nor a member class of "org.apache.drill.exec.expr.holders.VarCharHolder" [ bc85685d-e08a-494b-aa9f-5b3fa4ee4033 on atsqa4-133.qa.lab:3 1010 ] Error: exception while executing query: Failure while executing query. (state=,code=0) I believe that error that I'm getting from the query below is the result of not throwing "Unsupported" message earlier. Please correct me if I'm wrong. 0: jdbc:drill:schema=dfs> select . . . . . . . . . . . . > cast(l_orderkey as varchar(10)) || '-' || cast(l_partkey as varchar(10)) . . . . . . . . . . . . > from . . . . . . . . . . . . > ( . . . . . . . . . . . . > select sum(l_quantity), . . . . . . . . . . . . > l_orderkey, . . . . . . . . . . . . > l_partkey . . . . . . . . . . . . > fromcp.`tpch/lineitem.parquet` . . . . . . . . . . . . > group by . . . . . . . . . . . . > l_orderkey, . . . . . . . . . . . . > l_partkey . . . . . . . . . . . . > ) . . . . . . . . . . . . > group by . . . . . . . . . . . . > cast(l_orderkey as varchar(10)) || '-' || cast(l_partkey as varchar(10)); Query failed: Query failed: Failure while running fragment., You tried to do a batch data read operation when you were in a state of STOP. You can only do this type of operation when you are in a state of OK or OK_NEW_SCHEMA. [ f5cb68a8-7c05-4e02-a104-56081e6d34ce on atsqa4-133.qa.lab:31010 ] [ f5cb68a8-7c05-4e02-a104-56081e6d34ce on atsqa4-133.qa.lab:31010 ] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-1876) Remove || (concatenation operator) from documentation
Victoria Markman created DRILL-1876: --- Summary: Remove || (concatenation operator) from documentation Key: DRILL-1876 URL: https://issues.apache.org/jira/browse/DRILL-1876 Project: Apache Drill Issue Type: Bug Components: Documentation Affects Versions: 0.7.0 Reporter: Victoria Markman It is documented, but it does not work at all (DRILL-779) Workaround exists: concat function As of version 0.7.0 (git.commit.id.abbrev=3b0ff5d) (1) 0: jdbc:drill:schema=dfs> select 'abc' || 'def' from cp.`tpch/nation.parquet`; Query failed: Query failed: Failure while running fragment.[ dc4727ed-4ac3-42bb-a60b-f8f370dd4fcf on atsqa4-133.qa.lab:31010 ] [ dc4727ed-4ac3-42bb-a60b-f8f370dd4fcf on atsqa4-133.qa.lab:31010 ] Error: exception while executing query: Failure while executing query. (state=,code=0) (2) 0: jdbc:drill:schema=dfs> select n_name || 'xyz' from cp.`tpch/nation.parquet`; Query failed: Query failed: Unexpected exception during fragment initialization: Internal error: pre-condition failed: SqlTypeUtil.sameNamedType(argTypes[0], argTypes[1]) Error: exception while executing query: Failure while executing query. (state=,code=0) (3) 0: jdbc:drill:schema=dfs> select cast(n_nationkey as varchar(10)) || n_name from cp.`tpch/nation.parquet`; Query failed: Query failed: Unexpected exception during fragment initialization: Internal error: pre-condition failed: SqlTypeUtil.sameNamedType(argTypes[0], argTypes[1]) Error: exception while executing query: Failure while executing query. (state=,code=0) (4) 0: jdbc:drill:schema=dfs> select 'abc' || 'cde' from `test.json`; Query failed: Query failed: Failure while running fragment.[ 01194cdf-746c-45db-ae4e-014590bfbffd on atsqa4-133.qa.lab:31010 ] [ 01194cdf-746c-45db-ae4e-014590bfbffd on atsqa4-133.qa.lab:31010 ] Error: exception while executing query: Failure while executing query. (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-1799) kvgen() row limit
[ https://issues.apache.org/jira/browse/DRILL-1799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved DRILL-1799. --- Resolution: Fixed > kvgen() row limit > - > > Key: DRILL-1799 > URL: https://issues.apache.org/jira/browse/DRILL-1799 > Project: Apache Drill > Issue Type: Bug > Components: Functions - Drill >Affects Versions: 0.7.0 >Reporter: Chun Chang > > #Mon Dec 01 11:15:02 PST 2014 > git.commit.id.abbrev=a60e1db > Running the following query that contains kvgen() appears running out of > memory and fails. There appears to be a 50 row limit. If I add a limit to the > query, limit 50 or less works, with limit 51, the query fails. > 0: jdbc:drill:schema=dfs.drillTestDir> select kvgen(sub.esr) from > `es/esr-part` sub; > The query returns result but fails in the middle of displaying the result. No > other error message except the following stack: > 18:06:08.158 [2b82e176-0aeb-9b05-15f2-520eea269a1c:frag:0:0] WARN > o.a.d.e.w.fragment.FragmentExecutor - Error while initializing or executing > fragment > java.lang.IllegalStateException: Attempted to close accountor with 151 > buffer(s) still allocatedfor QueryId: 2b82e176-0aeb-9b05-15f2-520eea269a1c, > MajorFragmentId: 0, MinorFragmentId: 0. > Total 1 allocation(s) of byte size(s): 4096, at stack location: > > org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.buffer(TopLevelAllocator.java:212) > > org.apache.drill.exec.vector.UInt1Vector.allocateNewSafe(UInt1Vector.java:137) > > org.apache.drill.exec.vector.NullableBigIntVector.allocateNewSafe(NullableBigIntVector.java:173) > > org.apache.drill.exec.vector.complex.MapVector.allocateNewSafe(MapVector.java:165) > > org.apache.drill.exec.vector.complex.RepeatedMapVector.allocateNewSafe(RepeatedMapVector.java:236) > > org.apache.drill.exec.vector.complex.MapVector.allocateNewSafe(MapVector.java:165) > > org.apache.drill.exec.vector.complex.MapVector.allocateNewSafe(MapVector.java:165) > > org.apache.drill.exec.vector.complex.impl.SingleMapWriter.allocate(SingleMapWriter.java:135) > > org.apache.drill.exec.vector.complex.impl.RepeatedMapWriter.allocate(RepeatedMapWriter.java:137) > > org.apache.drill.exec.vector.complex.impl.SingleListWriter.allocate(SingleListWriter.java:116) > > org.apache.drill.exec.vector.complex.impl.ComplexWriterImpl.allocate(ComplexWriterImpl.java:157) > > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doAlloc(ProjectRecordBatch.java:217) > > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork(ProjectRecordBatch.java:144) > > org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:89) > > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:129) > > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:106) > > org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:124) > > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:86) > > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:76) > > org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:52) > > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:129) > > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:106) > > org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:124) > > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:67) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 29064: DRILL-1869 - CPP Client does not handle record batches that contain nullable varchars with only null values.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29064/#review65229 --- Ship it! Ship It! - Hanifi Gunes On Dec. 16, 2014, 6:28 p.m., Parth Chandra wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/29064/ > --- > > (Updated Dec. 16, 2014, 6:28 p.m.) > > > Review request for drill and Hanifi Gunes. > > > Repository: drill-git > > > Description > --- > > CPP client asserts when the value vactor returned by the server contains only > nulls because the underlying sliced bytebuf class did not handle zero length > buffers. Also in some cases the server sends back an entire column with a > value count of zero instead of a nullable vector. > > > Diffs > - > > contrib/native/client/src/clientlib/recordBatch.cpp 2a679fa > contrib/native/client/src/include/drill/recordBatch.hpp 28377cb > contrib/native/client/src/protobuf/BitControl.pb.h 798da33 > contrib/native/client/src/protobuf/BitControl.pb.cc 64ab783 > contrib/native/client/src/protobuf/UserBitShared.pb.h cfe6938 > contrib/native/client/src/protobuf/UserBitShared.pb.cc 17b3463 > > Diff: https://reviews.apache.org/r/29064/diff/ > > > Testing > --- > > > Thanks, > > Parth Chandra > >
Re: Review Request 29064: DRILL-1869 - CPP Client does not handle record batches that contain nullable varchars with only null values.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29064/ --- (Updated Dec. 16, 2014, 6:28 p.m.) Review request for drill and Hanifi Gunes. Changes --- removed unused code from loadNull Repository: drill-git Description --- CPP client asserts when the value vactor returned by the server contains only nulls because the underlying sliced bytebuf class did not handle zero length buffers. Also in some cases the server sends back an entire column with a value count of zero instead of a nullable vector. Diffs (updated) - contrib/native/client/src/clientlib/recordBatch.cpp 2a679fa contrib/native/client/src/include/drill/recordBatch.hpp 28377cb contrib/native/client/src/protobuf/BitControl.pb.h 798da33 contrib/native/client/src/protobuf/BitControl.pb.cc 64ab783 contrib/native/client/src/protobuf/UserBitShared.pb.h cfe6938 contrib/native/client/src/protobuf/UserBitShared.pb.cc 17b3463 Diff: https://reviews.apache.org/r/29064/diff/ Testing --- Thanks, Parth Chandra
[jira] [Created] (DRILL-1875) VectorContainder.getTransferClone() doesn't seem to handle ignoreWrappers properly
Deneche A. Hakim created DRILL-1875:
---
Summary: VectorContainder.getTransferClone() doesn't seem to
handle ignoreWrappers properly
Key: DRILL-1875
URL: https://issues.apache.org/jira/browse/DRILL-1875
Project: Apache Drill
Issue Type: Task
Reporter: Deneche A. Hakim
Assignee: Timothy Chen
Priority: Critical
Looking at org.apache.drill.exec.record.VectorContainer:
{code}
public static VectorContainer getTransferClone(VectorAccessible incoming,
VectorWrapper[] ignoreWrappers) {
VectorContainer vc = new VectorContainer();
for (VectorWrapper w : incoming) {
if(ignoreWrappers != null) {
for(VectorWrapper wrapper : ignoreWrappers) {
if (w == wrapper) {
continue;
}
}
}
vc.cloneAndTransfer(w);
}
return vc;
}
{code}
The inner loop doesn't seem to do anything. Shouldn't the continue statement
actually continue the outer loop instead ? like this:
{code}
outerloop:
for (VectorWrapper w : incoming) {
if(ignoreWrappers != null) {
for(VectorWrapper wrapper : ignoreWrappers) {
if (w == wrapper) {
continue outerloop;
}
}
}
vc.cloneAndTransfer(w);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Hangout happening now
Hey guys, Feel free to jump in on the hangout if you would like to discuss the unit test changes, 0.7 features and roadmap for drill. https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc -Jason
RE: [VOTE] Release Apache Drill 0.7.0
+1 - Built from source - Queried against Hive, HBase, CSV, JSON, Parquet files/tables Norris -Original Message- From: Neeraja Rentachintala [mailto:nrentachint...@maprtech.com] Sent: Tuesday, December 16, 2014 6:34 AM To: dev@drill.apache.org Subject: Re: [VOTE] Release Apache Drill 0.7.0 +1(non-binding) On Tue, Dec 16, 2014 at 12:15 AM, Mehdi Chitforoosh < mehdichitforo...@gmail.com> wrote: > > +1 > > On Tue, Dec 16, 2014 at 10:14 AM, Yash Sharma wrote: > > > > +1 > > > > - Source and binaries present > > - Source and Binaries contain README, NOTICE, LICENSE Files. > > - Source contains INSTALL.md > > - Verified Checksums for src and binaries > > - Able to launch drill from binary distribution (Embeded mode). > > - Able to build from source > > - Able to fire sample queries on sqlline > > > > On Mon, Dec 15, 2014 at 11:56 PM, Tomer Shiran > wrote: > > > > > > +1 > > > > > > On Mon, Dec 15, 2014 at 9:39 AM, Jacques Nadeau > > > > > > wrote: > > > > > > > > Good morning, > > > > > > > > I would like to propose the release of Apache Drill, version 0.7.0. > > > > > > > > This release includes 223 resolved JIRAs [1]. > > > > > > > > The artifacts are hosted at [2]. > > > > > > > > The vote will be open for 72 hours, ending 10 AM Pacific, > > > > December > 18, > > > > 2014. > > > > > > > > [ ] +1 > > > > [ ] +0 > > > > [ ] -1 > > > > > > > > > > > > Thank you, > > > > Jacques > > > > > > > > [1] > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313 > 820&version=12327473 > > > > [2] http://people.apache.org/~jacques/apache-drill-0.7.0.rc0/ > > > > > > > > > > > > -- > Mehdi Chitforoosh >
[jira] [Created] (DRILL-1874) Text reader should allow treating empty fields as NULLs instead of empty string
Aman Sinha created DRILL-1874:
-
Summary: Text reader should allow treating empty fields as NULLs
instead of empty string
Key: DRILL-1874
URL: https://issues.apache.org/jira/browse/DRILL-1874
Project: Apache Drill
Issue Type: Bug
Affects Versions: 0.7.0
Reporter: Aman Sinha
The text reader treats a record as a repeated varchar and if there is an empty
field it produces an empty string. This prevents CASTing the field to a
numeric type because one can get a NumberFormatException. This is a pain
point for many use cases because empty fields occur frequently in text files.
Ideally, based on some configuration setting, we should allow treating empty
fields as NULL values instead of empty string. This will allow cast to work
correctly.
In the meantime, here's a rewrite that can be used as workaround:
{code}
0: jdbc:drill:zk=local> select case when columns[17] = '' then cast(null as
int) else cast(columns[17] as int) end from dfs.tmp.`sample2.tbl`;
++
| EXPR$0 |
++
| 5 |
| null |
++
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: [VOTE] Release Apache Drill 0.7.0
+1(non-binding) On Tue, Dec 16, 2014 at 12:15 AM, Mehdi Chitforoosh < mehdichitforo...@gmail.com> wrote: > > +1 > > On Tue, Dec 16, 2014 at 10:14 AM, Yash Sharma wrote: > > > > +1 > > > > - Source and binaries present > > - Source and Binaries contain README, NOTICE, LICENSE Files. > > - Source contains INSTALL.md > > - Verified Checksums for src and binaries > > - Able to launch drill from binary distribution (Embeded mode). > > - Able to build from source > > - Able to fire sample queries on sqlline > > > > On Mon, Dec 15, 2014 at 11:56 PM, Tomer Shiran > wrote: > > > > > > +1 > > > > > > On Mon, Dec 15, 2014 at 9:39 AM, Jacques Nadeau > > > wrote: > > > > > > > > Good morning, > > > > > > > > I would like to propose the release of Apache Drill, version 0.7.0. > > > > > > > > This release includes 223 resolved JIRAs [1]. > > > > > > > > The artifacts are hosted at [2]. > > > > > > > > The vote will be open for 72 hours, ending 10 AM Pacific, December > 18, > > > > 2014. > > > > > > > > [ ] +1 > > > > [ ] +0 > > > > [ ] -1 > > > > > > > > > > > > Thank you, > > > > Jacques > > > > > > > > [1] > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313820&version=12327473 > > > > [2] http://people.apache.org/~jacques/apache-drill-0.7.0.rc0/ > > > > > > > > > > > > -- > Mehdi Chitforoosh >
Re: [VOTE] Release Apache Drill 0.7.0
+1 On Tue, Dec 16, 2014 at 10:14 AM, Yash Sharma wrote: > > +1 > > - Source and binaries present > - Source and Binaries contain README, NOTICE, LICENSE Files. > - Source contains INSTALL.md > - Verified Checksums for src and binaries > - Able to launch drill from binary distribution (Embeded mode). > - Able to build from source > - Able to fire sample queries on sqlline > > On Mon, Dec 15, 2014 at 11:56 PM, Tomer Shiran wrote: > > > > +1 > > > > On Mon, Dec 15, 2014 at 9:39 AM, Jacques Nadeau > > wrote: > > > > > > Good morning, > > > > > > I would like to propose the release of Apache Drill, version 0.7.0. > > > > > > This release includes 223 resolved JIRAs [1]. > > > > > > The artifacts are hosted at [2]. > > > > > > The vote will be open for 72 hours, ending 10 AM Pacific, December 18, > > > 2014. > > > > > > [ ] +1 > > > [ ] +0 > > > [ ] -1 > > > > > > > > > Thank you, > > > Jacques > > > > > > [1] > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313820&version=12327473 > > > [2] http://people.apache.org/~jacques/apache-drill-0.7.0.rc0/ > > > > > > -- Mehdi Chitforoosh
