[jira] [Assigned] (DRILL-2798) Suppress log location message from sqlline
[ https://issues.apache.org/jira/browse/DRILL-2798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra reassigned DRILL-2798: Assignee: Parth Chandra (was: DrillCommitter) Suppress log location message from sqlline -- Key: DRILL-2798 URL: https://issues.apache.org/jira/browse/DRILL-2798 Project: Apache Drill Issue Type: Bug Components: Client - CLI Affects Versions: 0.8.0 Reporter: Parth Chandra Assignee: Parth Chandra Fix For: 0.9.0 Attachments: DRILL-2798.1.patch.txt sqlline is now printing a message with the location of the log file that is breaking external scripts to extract data using Drill. We need to add an option to suppress sqlline shell script messages (or remove them). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2798) Suppress log location message from sqlline
[ https://issues.apache.org/jira/browse/DRILL-2798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496994#comment-14496994 ] Parth Chandra commented on DRILL-2798: -- +!. Looks good to me. Suppress log location message from sqlline -- Key: DRILL-2798 URL: https://issues.apache.org/jira/browse/DRILL-2798 Project: Apache Drill Issue Type: Bug Components: Client - CLI Affects Versions: 0.8.0 Reporter: Parth Chandra Assignee: Parth Chandra Fix For: 0.9.0 Attachments: DRILL-2798.1.patch.txt sqlline is now printing a message with the location of the log file that is breaking external scripts to extract data using Drill. We need to add an option to suppress sqlline shell script messages (or remove them). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-2671) C++ Client Authentication API passing std::string across DLL boundaries
[ https://issues.apache.org/jira/browse/DRILL-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra resolved DRILL-2671. -- Resolution: Fixed Fixed in e4e88cc C++ Client Authentication API passing std::string across DLL boundaries --- Key: DRILL-2671 URL: https://issues.apache.org/jira/browse/DRILL-2671 Project: Apache Drill Issue Type: Bug Components: Client - C++ Reporter: Norris Lee Assignee: Norris Lee Fix For: 0.9.0 DrillUserProperty::setProperty is taking std::string as parameters. Memory gets allocated in the client yet Drill Client tries to clean it up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2802) Projecting dir[n] by itself, results in projecting of all columns
Victoria Markman created DRILL-2802:
---
Summary: Projecting dir[n] by itself, results in projecting of all
columns
Key: DRILL-2802
URL: https://issues.apache.org/jira/browse/DRILL-2802
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 0.9.0
Reporter: Victoria Markman
Assignee: Jinfeng Ni
{code}
0: jdbc:drill:schema=dfs select dir1 from bigtable limit 1;
+++++
| a1 | b1 | c1 |dir1|
+++++
| 1 | a | 2015-01-01 | 01 |
+++++
1 row selected (0.189 seconds)
0: jdbc:drill:schema=dfs select dir0 from bigtable limit 1;
+++++
| a1 | b1 | c1 |dir0|
+++++
| 1 | a | 2015-01-01 | 2015 |
+++++
1 row selected (0.193 seconds)
{code}
In explain plan, I don't see project:
{code}
0: jdbc:drill:schema=dfs explain plan for select dir0 from bigtable;
+++
|text|json|
+++
| 00-00Screen
00-01 Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath
[path=maprfs:/test/bigtable/2015/01/4_0_0.parquet], ReadEntryWithPath
[path=maprfs:/test/bigtable/2015/01/3_0_0.parquet], ReadEntryWithPath
[path=maprfs:/test/bigtable/2015/01/5_0_0.parquet], ReadEntryWithPath
[path=maprfs:/test/bigtable/2015/01/1_0_0.parquet], ReadEntryWithPath
[path=maprfs:/test/bigtable/2015/01/2_0_0.parquet], ReadEntryWithPath
[path=maprfs:/test/bigtable/2015/01/0_0_0.parquet], ReadEntryWithPath
[path=maprfs:/test/bigtable/2015/02/0_0_0.parquet], ReadEntryWithPath
[path=maprfs:/test/bigtable/2015/03/0_0_0.parquet], ReadEntryWithPath
[path=maprfs:/test/bigtable/2015/04/0_0_0.parquet], ReadEntryWithPath
[path=maprfs:/test/bigtable/2016/01/parquet.file], ReadEntryWithPath
[path=maprfs:/test/bigtable/2016/parquet.file]], selectionRoot=/test/bigtable,
numFiles=11, columns=[`dir0`]]])
{code}
If you project both dir0 and dir1, both columns are projected with the correct
result:
{code}
0: jdbc:drill:schema=dfs select dir0, dir1 from bigtable;
+++
|dir0|dir1|
+++
| 2015 | 01 |
| 2015 | 01 |
| 2015 | 01 |
| 2015 | 01 |
| 2015 | 01 |
| 2015 | 01 |
| 2015 | 01 |
| 2015 | 01 |
| 2015 | 01 |
{code}
{code}
[Wed Apr 15 14:09:47 root@/mapr/vmarkman.cluster.com/test/bigtable ] # ls -R
.:
2015 2016
./2015:
01 02 03 04
./2015/01:
0_0_0.parquet 1_0_0.parquet 2_0_0.parquet 3_0_0.parquet 4_0_0.parquet
5_0_0.parquet
./2015/02:
0_0_0.parquet
./2015/03:
0_0_0.parquet
./2015/04:
0_0_0.parquet
./2016:
01 parquet.file
./2016/01:
parquet.file
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-1330) String aggregate function - string_agg(expression, delimiter)
[ https://issues.apache.org/jira/browse/DRILL-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-1330: - Target Version/s: Future String aggregate function - string_agg(expression, delimiter) - Key: DRILL-1330 URL: https://issues.apache.org/jira/browse/DRILL-1330 Project: Apache Drill Issue Type: Improvement Components: Functions - Drill Reporter: Yash Sharma Assignee: Yash Sharma Priority: Minor Fix For: Future -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-1330) String aggregate function - string_agg(expression, delimiter)
[ https://issues.apache.org/jira/browse/DRILL-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-1330: - Fix Version/s: Future String aggregate function - string_agg(expression, delimiter) - Key: DRILL-1330 URL: https://issues.apache.org/jira/browse/DRILL-1330 Project: Apache Drill Issue Type: Improvement Components: Functions - Drill Reporter: Yash Sharma Assignee: Yash Sharma Priority: Minor Fix For: Future -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-1330) String aggregate function - string_agg(expression, delimiter)
[ https://issues.apache.org/jira/browse/DRILL-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-1330: - Fix Version/s: (was: 0.9.0) String aggregate function - string_agg(expression, delimiter) - Key: DRILL-1330 URL: https://issues.apache.org/jira/browse/DRILL-1330 Project: Apache Drill Issue Type: Improvement Components: Functions - Drill Reporter: Yash Sharma Assignee: Yash Sharma Priority: Minor Fix For: Future -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2554) Incorrect results for repeated values when using jdbc
[
https://issues.apache.org/jira/browse/DRILL-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-2554:
-
Target Version/s: 1.0.0
Fix Version/s: (was: 0.9.0)
1.0.0
Incorrect results for repeated values when using jdbc
-
Key: DRILL-2554
URL: https://issues.apache.org/jira/browse/DRILL-2554
Project: Apache Drill
Issue Type: Bug
Components: Client - JDBC, Execution - Data Types
Affects Versions: 0.8.0
Reporter: Khurram Faraaz
Assignee: Parth Chandra
Priority: Critical
Fix For: 1.0.0
Data is missing from the output of select * from JSON data file statement.
Data pertaining to key2 and key3 and key4 is missing from the output of the
below select statement. I had enabled `store.json.all_text_mode`=true for
that session.
{code}
0: jdbc:drill: alter session set `store.json.all_text_mode`=true;
+++
| ok | summary |
+++
| true | store.json.all_text_mode updated. |
+++
1 row selected (0.022 seconds)
0: jdbc:drill: select * from `testJsnData02.json`;
++++++
|key |key1|key2|key3|key4|
++++++
| 12345 | {} | [] | {} | [] |
| -123456| {} | [] | {} | null |
| 0 | {} | [] | {} | null |
| -9.999 | {} | [] | {} | null |
| .9876 | {} | [] | {} | null |
| Hello World! | {} | [] | {} | null |
| this is a long string, not very long though! | {} | [] | {}
| null |
| true | {} | [] | {} | null |
| false | {} | [] | {} | null |
| null | {} | [] | {} | null |
| 2147483647 | {} | [] | {} | null |
| 1100110010101010100101010101010101 | {} | [] | {} |
null |
| 2008-1-23 14:24:23 | {} | [] | {} | null |
| 2008-2-23 | {} | [] | {} | null |
| 10:20:30.123 | {} | null | {} | null |
| -1 | {} | null | {} | null |
| 3.147 | {} | null | {} | null |
| null | {id:1000.997} | null | {} | null |
| null | {} | null | {} | null |
| null | {} | null | {} | null |
| null | {} | null | {} | null |
| abcdefghijklmnopqrstuvwxyz1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ12345
aeiou | {} | null | {} | null |
++++++
22 rows selected (0.069 seconds)
0: jdbc:drill: select * from sys.version;
+++-+-++
| commit_id | commit_message | commit_time | build_email | build_time |
+++-+-++
| f658a3c513ddf7f2d1b0ad7aa1f3f65049a594fe | DRILL-2209 Insert
ProjectOperator with MuxExchange | 09.03.2015 @ 01:49:18 EDT | Unknown |
09.03.2015 @ 04:52:49 EDT |
+++-+-++
1 row selected (0.041 seconds)
{code}
The data that I used in my test was
{code}
{key:12345}
{key:-123456}
{key:0}
{key:-9.999}
{key:.9876}
{key:Hello World!}
{key:this is a long string, not very long though!}
{key:true}
{key:false}
{key:null}
{key:2147483647}
{key:1100110010101010100101010101010101}
{key:2008-1-23 14:24:23}
{key:2008-2-23}
{key:10:20:30.123}
{key:-1}
{key:3.147}
{key1:{id:1000.997}}
{key2:[1,2,3,4,-1,0,135.987,9,-.876,2147483647,test
string,null,true,false]}
{key3:{id:null}}
{key4:[null]}
{key:abcdefghijklmnopqrstuvwxyz1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ
12345 aeiou}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2647) NullPointerException from CONVERT_FROM given a NULL
[ https://issues.apache.org/jira/browse/DRILL-2647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2647: - Target Version/s: 1.0.0 Fix Version/s: (was: 0.9.0) 1.0.0 NullPointerException from CONVERT_FROM given a NULL --- Key: DRILL-2647 URL: https://issues.apache.org/jira/browse/DRILL-2647 Project: Apache Drill Issue Type: Bug Components: Functions - Drill Reporter: Daniel Barclay (Drill) Assignee: Parth Chandra Fix For: 1.0.0 CONVERT_FROM crashes when given a null value like this: SELECT CONVERT_FROM(CAST(NULL AS VARCHAR), 'JSON') FROM INFORMATION_SCHEMA.CATALOGS; This fails similarly SELECT CONVERT_FROM(CAST(NULL AS INTEGER), 'JSON') FROM INFORMATION_SCHEMA.CATALOGS; -- 0: jdbc:drill:zk=local SELECT CONVERT_FROM(CAST(NULL AS VARCHAR), 'JSON') FROM INFORMATION_SCHEMA.CATALOGS; Exception in thread 2ae48af0-c497-8b98-d9eb-f64353f79065:frag:0:0 java.lang.RuntimeException: Error closing fragment context. at org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:224) at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:187) at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Error while converting from JSON. at org.apache.drill.exec.test.generated.ProjectorGen4.doEval(ProjectorTemplate.java:38) Query failed: RemoteRpcException: Failure while running fragment., Error while converting from JSON. [ f0c043d2-f86f-4e4d-a864-74df93f6c79f on dev-linux2:31010 ] [ f0c043d2-f86f-4e4d-a864-74df93f6c79f on dev-linux2:31010 ] at org.apache.drill.exec.test.generated.ProjectorGen4.projectRecords(ProjectorTemplate.java:62) at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork(ProjectRecordBatch.java:174) at org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93) at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:134) at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142) at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118) at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:68) at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:96) at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:58) at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:163) ... 4 more Caused by: java.lang.NullPointerException at org.apache.drill.exec.vector.complex.fn.DrillBufInputStream.getStream(DrillBufInputStream.java:56) at org.apache.drill.exec.vector.complex.fn.JsonReader.setSource(JsonReader.java:114) at org.apache.drill.exec.test.generated.ProjectorGen4.doEval(ProjectorTemplate.java:34) ... 14 more Error: exception while executing query: Failure while executing query. (state=,code=0) 0: jdbc:drill:zk=local -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2658) Add ilike and regex substring functions
[ https://issues.apache.org/jira/browse/DRILL-2658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14495729#comment-14495729 ] Steven Phillips commented on DRILL-2658: Created reviewboard https://reviews.apache.org/r/33212/ Add ilike and regex substring functions --- Key: DRILL-2658 URL: https://issues.apache.org/jira/browse/DRILL-2658 Project: Apache Drill Issue Type: New Feature Components: Functions - Drill Reporter: Steven Phillips Assignee: Deneche A. Hakim Fix For: 1.0.0 Attachments: DRILL-2658.patch, DRILL-2658.patch This will not modify the parser, so postgress syntax such as: ... where c ILIKE '%ABC%' will not be currently supported. It will simply be a function: ... where ILIKE(c, '%ABC%') Same for substring: select substr(c, 'abc')... will be equivalent to postgress select substr(c from 'abc'), but 'abc' will be treated as a java regex pattern. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-1512) Avro Record Reader
[ https://issues.apache.org/jira/browse/DRILL-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steven Phillips updated DRILL-1512: --- Attachment: DRILL-1512.4b.patch DRILL-1512.4a.patch I applied this patch, and made some minor changes to resolve some conflicts. This is patch 4a. I then made some additional changes on top and added them as 4b. The main thing I did with my changes is refactor to extend EasyFormatPlugin. By doing this, we automatically get the assignment and affinity functionality that the original patch had not yet implemented. Avro Record Reader -- Key: DRILL-1512 URL: https://issues.apache.org/jira/browse/DRILL-1512 Project: Apache Drill Issue Type: New Feature Reporter: Andrew Priority: Minor Labels: avro, drill Fix For: 0.9.0 Attachments: DRILL-1512.1.patch.txt, DRILL-1512.2.patch.txt, DRILL-1512.3.patch.txt, DRILL-1512.4a.patch, DRILL-1512.4b.patch Record reader implementation for Avro data files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (DRILL-1512) Avro Record Reader
[ https://issues.apache.org/jira/browse/DRILL-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steven Phillips reassigned DRILL-1512: -- Assignee: Steven Phillips Avro Record Reader -- Key: DRILL-1512 URL: https://issues.apache.org/jira/browse/DRILL-1512 Project: Apache Drill Issue Type: New Feature Reporter: Andrew Assignee: Steven Phillips Priority: Minor Labels: avro, drill Fix For: 0.9.0 Attachments: DRILL-1512.1.patch.txt, DRILL-1512.2.patch.txt, DRILL-1512.3.patch.txt, DRILL-1512.4a.patch, DRILL-1512.4b.patch Record reader implementation for Avro data files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-2577) Parquet scan fails when directory contains _SUCCESS or _logs
[ https://issues.apache.org/jira/browse/DRILL-2577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steven Phillips resolved DRILL-2577. Resolution: Fixed fixed in 96943de Parquet scan fails when directory contains _SUCCESS or _logs Key: DRILL-2577 URL: https://issues.apache.org/jira/browse/DRILL-2577 Project: Apache Drill Issue Type: New Feature Components: Storage - Parquet Affects Versions: 0.8.0 Reporter: Steven Phillips Assignee: Steven Phillips Fix For: 0.9.0 Attachments: DRILL-2577.patch _SUCCESS and _logs are often created from map reduce jobs, and typically ignored. This is commonly done using the OutputFilesFilter in hadoop. The new FooterGatherer class, which is used to read parquet footers in parallel, does not use this filter. So it attempts to read these as parquet files and fails. The fix is to use the DrillPathFilter, which extends OutputFilesFilter, in the FooterGather code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
