[jira] [Updated] (DRILL-3496) Augment logging in DrillConfig and classpath scanning.
[ https://issues.apache.org/jira/browse/DRILL-3496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Barclay (Drill) updated DRILL-3496: -- Assignee: Jason Altekruse (was: Daniel Barclay (Drill)) Augment logging in DrillConfig and classpath scanning. -- Key: DRILL-3496 URL: https://issues.apache.org/jira/browse/DRILL-3496 Project: Apache Drill Issue Type: Bug Reporter: Daniel Barclay (Drill) Assignee: Jason Altekruse -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-3412) Projections are not getting push down below Window operator
[
https://issues.apache.org/jira/browse/DRILL-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637550#comment-14637550
]
Victoria Markman commented on DRILL-3412:
-
This bug is blocking performance testing of window functions.
Projections are not getting push down below Window operator
---
Key: DRILL-3412
URL: https://issues.apache.org/jira/browse/DRILL-3412
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Reporter: Aman Sinha
Assignee: Jinfeng Ni
Priority: Blocker
Fix For: 1.2.0
The plan below shows that the 'star' column is being produced by the Scan and
subsequent Project. This indicates projection pushdown is not working as
desired when window function is present. The query produces correct results.
{code}
explain plan for select min(n_nationkey) over (partition by n_regionkey) from
cp.`tpch/nation.parquet` ;
00-00Screen
00-01 Project(EXPR$0=[$0])
00-02Project(w0$o0=[$3])
00-03 Window(window#0=[window(partition {2} order by [] range
between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [MIN($1)])])
00-04SelectionVectorRemover
00-05 Sort(sort0=[$2], dir0=[ASC])
00-06Project(T1¦¦*=[$0], n_nationkey=[$1], n_regionkey=[$2])
00-07 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=classpath:/tpch/nation.parquet]],
selectionRoot=/tpch/nation.parquet, numFiles=1, columns=[`*`]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-3412) Projections are not getting push down below Window operator
[
https://issues.apache.org/jira/browse/DRILL-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-3412:
Priority: Blocker (was: Major)
Projections are not getting push down below Window operator
---
Key: DRILL-3412
URL: https://issues.apache.org/jira/browse/DRILL-3412
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Reporter: Aman Sinha
Assignee: Jinfeng Ni
Priority: Blocker
Fix For: 1.2.0
The plan below shows that the 'star' column is being produced by the Scan and
subsequent Project. This indicates projection pushdown is not working as
desired when window function is present. The query produces correct results.
{code}
explain plan for select min(n_nationkey) over (partition by n_regionkey) from
cp.`tpch/nation.parquet` ;
00-00Screen
00-01 Project(EXPR$0=[$0])
00-02Project(w0$o0=[$3])
00-03 Window(window#0=[window(partition {2} order by [] range
between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [MIN($1)])])
00-04SelectionVectorRemover
00-05 Sort(sort0=[$2], dir0=[ASC])
00-06Project(T1¦¦*=[$0], n_nationkey=[$1], n_regionkey=[$2])
00-07 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=classpath:/tpch/nation.parquet]],
selectionRoot=/tpch/nation.parquet, numFiles=1, columns=[`*`]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3412) Projections are not getting push down below Window operator
[
https://issues.apache.org/jira/browse/DRILL-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637580#comment-14637580
]
Victoria Markman commented on DRILL-3412:
-
Jinfeng, this bug was reported this morning as a blocking issue for testing.
We are using TPCDS schema. Should we proceed with views on top of TPCDS tables
that project fewer columns ?
Projections are not getting push down below Window operator
---
Key: DRILL-3412
URL: https://issues.apache.org/jira/browse/DRILL-3412
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Reporter: Aman Sinha
Assignee: Jinfeng Ni
Priority: Blocker
Fix For: 1.2.0
The plan below shows that the 'star' column is being produced by the Scan and
subsequent Project. This indicates projection pushdown is not working as
desired when window function is present. The query produces correct results.
{code}
explain plan for select min(n_nationkey) over (partition by n_regionkey) from
cp.`tpch/nation.parquet` ;
00-00Screen
00-01 Project(EXPR$0=[$0])
00-02Project(w0$o0=[$3])
00-03 Window(window#0=[window(partition {2} order by [] range
between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [MIN($1)])])
00-04SelectionVectorRemover
00-05 Sort(sort0=[$2], dir0=[ASC])
00-06Project(T1¦¦*=[$0], n_nationkey=[$1], n_regionkey=[$2])
00-07 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=classpath:/tpch/nation.parquet]],
selectionRoot=/tpch/nation.parquet, numFiles=1, columns=[`*`]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2815) Some PathScanner logging, misc. cleanup.
[ https://issues.apache.org/jira/browse/DRILL-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Barclay (Drill) updated DRILL-2815: -- Assignee: Jason Altekruse (was: Daniel Barclay (Drill)) Some PathScanner logging, misc. cleanup. Key: DRILL-2815 URL: https://issues.apache.org/jira/browse/DRILL-2815 Project: Apache Drill Issue Type: Bug Reporter: Daniel Barclay (Drill) Assignee: Jason Altekruse Priority: Minor Fix For: 1.2.0 Attachments: DRILL-2815.5.patch.txt, DRILL-2815.6.patch.txt Add a little more logging to PathScanner; clean up a little. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-3533) null values in a sub-structure in Parquet returns unexpected/misleading results
[
https://issues.apache.org/jira/browse/DRILL-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637484#comment-14637484
]
Jinfeng Ni commented on DRILL-3533:
---
Can you check the content of /var/www/as/drill/distribution/target/rat.txt, to
see which file(s) failed at the rat check?
On my machine, I run mvn clean install successfully.
null values in a sub-structure in Parquet returns unexpected/misleading
results
---
Key: DRILL-3533
URL: https://issues.apache.org/jira/browse/DRILL-3533
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 1.1.0
Reporter: Stefán Baxter
Assignee: Parth Chandra
Priority: Critical
With this minimal dataset as /tmp/test.json:
{dimensions:{adults:A}}
select lower(p.dimensions.budgetLevel) as `field1`,
lower(p.dimensions.adults) as `field2` from dfs.tmp.`/test.json` as p;
Returns this:
+-+-+
| field1 | field2 |
+-+-+
| null| a |
+-+-+
With the same data as a Parquet file
CREATE TABLE dfs.tmp.`/test` AS SELECT * FROM dfs.tmp.`/test.json`;
The same query:
select lower(p.dimensions.budgetLevel) as `field1`,
lower(p.dimensions.adults) as `field2` from dfs.tmp.`/test/0_0_0.parquet` as
p;
Return this:
+-+-+
| field1 | field2 |
+-+-+
| a | null|
+-+-+
After some more testing it appears that this has nothing to do with trim.
(any non existing nested-value will be pushed aside)
select p.dimensions.budgetLevel as `field1`, lower(p.dimensions.adults) as
`field2` from dfs.tmp.`/test/0_0_0.parquet` as p;
also returns:
+-+-+
| field1 | field2 |
+-+-+
| a | null|
+-+-+
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3412) Projections are not getting push down below Window operator
[
https://issues.apache.org/jira/browse/DRILL-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637569#comment-14637569
]
Jinfeng Ni commented on DRILL-3412:
---
Although there is big impact for window functions without project pushdown, I
do not think it would be a deciding factor for window function performance,
especially if the table does not have too many columns which are not referenced
in the query.
In stead, in many cases, the performance bottleneck for window function would
be in the SORT operator, not in the SCAN operator.
Projections are not getting push down below Window operator
---
Key: DRILL-3412
URL: https://issues.apache.org/jira/browse/DRILL-3412
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Reporter: Aman Sinha
Assignee: Jinfeng Ni
Priority: Blocker
Fix For: 1.2.0
The plan below shows that the 'star' column is being produced by the Scan and
subsequent Project. This indicates projection pushdown is not working as
desired when window function is present. The query produces correct results.
{code}
explain plan for select min(n_nationkey) over (partition by n_regionkey) from
cp.`tpch/nation.parquet` ;
00-00Screen
00-01 Project(EXPR$0=[$0])
00-02Project(w0$o0=[$3])
00-03 Window(window#0=[window(partition {2} order by [] range
between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [MIN($1)])])
00-04SelectionVectorRemover
00-05 Sort(sort0=[$2], dir0=[ASC])
00-06Project(T1¦¦*=[$0], n_nationkey=[$1], n_regionkey=[$2])
00-07 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=classpath:/tpch/nation.parquet]],
selectionRoot=/tpch/nation.parquet, numFiles=1, columns=[`*`]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (DRILL-3528) Apache drill plugin for solr
[ https://issues.apache.org/jira/browse/DRILL-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudip Mukherjee resolved DRILL-3528. Resolution: Fixed Apache drill plugin for solr Key: DRILL-3528 URL: https://issues.apache.org/jira/browse/DRILL-3528 Project: Apache Drill Issue Type: New Feature Components: Storage - Other Environment: windows, eclipse ide,java 8 ,maven 3x Reporter: Sudip Mukherjee Assignee: Jacques Nadeau Attachments: storage-solr.rar I am trying to explore apache drill project and able to set it in my local enviroment with the help of documentation. However, i was looking for some code flow document to get started with writing a custom plug-in for trial. I am trying add a type of apache solr to have a sql like interface on top of it. when i added the project plugin under /jars folder and tried to create new storage of type solr got an exception Error : Invalid JSON mapping. Could you please help me with some debugging points. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2818) Error message must be updated when query fails with FileNotFoundException
[
https://issues.apache.org/jira/browse/DRILL-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637692#comment-14637692
]
ASF GitHub Bot commented on DRILL-2818:
---
Github user jaltekruse commented on a diff in the pull request:
https://github.com/apache/drill/pull/93#discussion_r35268073
--- Diff:
common/src/main/java/org/apache/drill/common/config/DrillConfig.java ---
@@ -138,34 +154,83 @@ public static DrillConfig create(Properties
testConfigurations) {
return create(null, testConfigurations, true);
}
- public static DrillConfig create(String overrideFileName, boolean
enableServerConfigs) {
-return create(overrideFileName, null, enableServerConfigs);
+ /**
+ * ...
+ * @param overrideFileResourcePathname
+ * see {@link #create(String)}'s {@code
overrideFileResourcePathname}
+ */
+ public static DrillConfig create(String overrideFileResourcePathname,
boolean enableServerConfigs) {
+return create(overrideFileResourcePathname, null, enableServerConfigs);
}
- private static DrillConfig create(String overrideFileName, Properties
overriderProps, boolean enableServerConfigs) {
-overrideFileName = overrideFileName == null ?
CommonConstants.CONFIG_OVERRIDE : overrideFileName;
-
-// first we load defaults.
+ /**
+ * ...
--- End diff --
I think if you should just put in explicit todo. However I'm not even sure
that it makes sense to document these methods individually. There is a comment
that is very descriptive about what all of these do, it's just above the wrong
version of the method. If you move it above the one that takes all of the
parameters, I think it would be fine to even leave out the parameter
descriptions on the other versions and have a common message directing people
to the one descriptive comment (which also describes all possible parameters).
No need for a todo, not a bunch of work up front.
Error message must be updated when query fails with FileNotFoundException
-
Key: DRILL-2818
URL: https://issues.apache.org/jira/browse/DRILL-2818
Project: Apache Drill
Issue Type: Bug
Components: SQL Parser
Affects Versions: 0.9.0
Environment: exception branch
Reporter: Abhishek Girish
Assignee: Deneche A. Hakim
Priority: Minor
Labels: error_message_must_fix
Fix For: 1.3.0
When user specifies a non-existent file/directory in a query, the following
error is being thrown:
{code:sql}
show files from dfs.tmp.`tpch`;
Query failed: SYSTEM ERROR: Failure handling SQL.
[9184097e-8339-42d3-96ce-1fba51c6bc78 on 192.168.158.107:31010]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
This should be updated to
{code:sql}
show files from dfs.tmp.`tpch`;
Query failed: File /tmp/tpch does not exist.
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-3540) SQLline calls toString to convert complex columns into JSON. This translation doesn't work for some types
[
https://issues.apache.org/jira/browse/DRILL-3540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacques Nadeau updated DRILL-3540:
--
Summary: SQLline calls toString to convert complex columns into JSON. This
translation doesn't work for some types (was: Joda module not added to jackson
for default serialization of DateTime objects (enhanced json))
SQLline calls toString to convert complex columns into JSON. This
translation doesn't work for some types
--
Key: DRILL-3540
URL: https://issues.apache.org/jira/browse/DRILL-3540
Project: Apache Drill
Issue Type: Bug
Components: Storage - JSON
Affects Versions: 1.1.0
Reporter: Stefán Baxter
Assignee: Steven Phillips
test-datetime.json
{
occurred_at : {
$date : 2015-04-01T00:14:02.309Z
},
sub:{
occurred_at:{$date:2015-04-01T00:14:02.309Z}
}
}
correctly results in this (without timezone though):
select t.occurred_at, t.sub.occurred_at from dfs.tmp.`/test-datetime.json`
as t;
+--+--+
| occurred_at| EXPR$1 |
+--+--+
| 2015-04-01 00:14:02.309 | 2015-04-01 00:14:02.309 |
+--+--+
1 row selected (0.067 seconds)
here we select the substructure, not the field
select t.occurred_at, t.sub from dfs.tmp.`/test-datetime.json` as t;
incorrectly results in this:
+-+-+
| occurred_at | sub |
+-+-+
| 2015-04-01 00:14:02.309 |
{occurred_at:{year:2015,dayOfMonth:1,dayOfWeek:3,era:1,dayOfYear:91,weekyear:2015,weekOfWeekyear:14,monthOfYear:4,yearOfEra:2015,yearOfCentury:15,centuryOfEra:20,millisOfSecond:309,millisOfDay:842309,secondOfMinute:2,secondOfDay:842,minuteOfHour:14,minuteOfDay:14,hourOfDay:0,millis:1427847242309,zone:{fixed:false,uncachedZone:{cachable:true,fixed:false,id:Atlantic/Reykjavik},id:Atlantic/Reykjavik},chronology:{zone:{fixed:false,uncachedZone:{cachable:true,fixed:false,id:Atlantic/Reykjavik},id:Atlantic/Reykjavik}},afterNow:false,beforeNow:true,equalNow:false}}
|
+-+-+
1 row selected (0.066 seconds)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-3545) Need documentation on BINARY_STRING and STRING_BINARY functions
Ted Dunning created DRILL-3545: -- Summary: Need documentation on BINARY_STRING and STRING_BINARY functions Key: DRILL-3545 URL: https://issues.apache.org/jira/browse/DRILL-3545 Project: Apache Drill Issue Type: Bug Reporter: Ted Dunning These are darn handy but we need to document them so the community at large can find out about them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-3545) Need documentation on BINARY_STRING and STRING_BINARY functions
[ https://issues.apache.org/jira/browse/DRILL-3545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated DRILL-3545: -- Component/s: Documentation Need documentation on BINARY_STRING and STRING_BINARY functions --- Key: DRILL-3545 URL: https://issues.apache.org/jira/browse/DRILL-3545 Project: Apache Drill Issue Type: Improvement Components: Documentation Reporter: Ted Dunning These are darn handy but we need to document them so the community at large can find out about them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-3545) Need documentation on BINARY_STRING and STRING_BINARY functions
[ https://issues.apache.org/jira/browse/DRILL-3545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated DRILL-3545: -- Issue Type: Improvement (was: Bug) Need documentation on BINARY_STRING and STRING_BINARY functions --- Key: DRILL-3545 URL: https://issues.apache.org/jira/browse/DRILL-3545 Project: Apache Drill Issue Type: Improvement Components: Documentation Reporter: Ted Dunning These are darn handy but we need to document them so the community at large can find out about them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-3544) Need better error messages when convert_to is given a bad type
Ted Dunning created DRILL-3544:
--
Summary: Need better error messages when convert_to is given a bad
type
Key: DRILL-3544
URL: https://issues.apache.org/jira/browse/DRILL-3544
Project: Apache Drill
Issue Type: Bug
Reporter: Ted Dunning
The first query below fails because I used UTF-8 instead of UTF8. This should
have a decent error message.
{code}
0: jdbc:drill:zk=local SELECT CONVERT_TO('[ [1, 2], [3, 4], [5]]' ,'UTF-8') AS
MYCOL1 FROM sys.version;
Error: SYSTEM ERROR: org.apache.drill.exec.work.foreman.ForemanException:
Unexpected exception during fragment initialization: null
[Error Id: 899207da-2338-4b09-bdc8-8e12e320b661 on 172.16.0.61:31010]
(state=,code=0)
0: jdbc:drill:zk=local SELECT CONVERT_TO('[ [1, 2], [3, 4], [5]]' ,'UTF8') AS
MYCOL1 FROM sys.version;
+-+
| MYCOL1|
+-+
| [B@71f3d3a |
+-+
1 row selected (0.108 seconds)
0: jdbc:drill:zk=local
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (DRILL-3544) Need better error messages when convert_to is given a bad type
[
https://issues.apache.org/jira/browse/DRILL-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacques Nadeau resolved DRILL-3544.
---
Resolution: Duplicate
Need better error messages when convert_to is given a bad type
--
Key: DRILL-3544
URL: https://issues.apache.org/jira/browse/DRILL-3544
Project: Apache Drill
Issue Type: Bug
Reporter: Ted Dunning
The first query below fails because I used UTF-8 instead of UTF8. This
should have a decent error message.
{code}
0: jdbc:drill:zk=local SELECT CONVERT_TO('[ [1, 2], [3, 4], [5]]' ,'UTF-8')
AS MYCOL1 FROM sys.version;
Error: SYSTEM ERROR: org.apache.drill.exec.work.foreman.ForemanException:
Unexpected exception during fragment initialization: null
[Error Id: 899207da-2338-4b09-bdc8-8e12e320b661 on 172.16.0.61:31010]
(state=,code=0)
0: jdbc:drill:zk=local SELECT CONVERT_TO('[ [1, 2], [3, 4], [5]]' ,'UTF8')
AS MYCOL1 FROM sys.version;
+-+
| MYCOL1|
+-+
| [B@71f3d3a |
+-+
1 row selected (0.108 seconds)
0: jdbc:drill:zk=local
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3533) null values in a sub-structure in Parquet returns unexpected/misleading results
[
https://issues.apache.org/jira/browse/DRILL-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637713#comment-14637713
]
Stefán Baxter commented on DRILL-3533:
--
Yeah, it's working fine now!
thnx again!
null values in a sub-structure in Parquet returns unexpected/misleading
results
---
Key: DRILL-3533
URL: https://issues.apache.org/jira/browse/DRILL-3533
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 1.1.0
Reporter: Stefán Baxter
Assignee: Parth Chandra
Priority: Critical
With this minimal dataset as /tmp/test.json:
{dimensions:{adults:A}}
select lower(p.dimensions.budgetLevel) as `field1`,
lower(p.dimensions.adults) as `field2` from dfs.tmp.`/test.json` as p;
Returns this:
+-+-+
| field1 | field2 |
+-+-+
| null| a |
+-+-+
With the same data as a Parquet file
CREATE TABLE dfs.tmp.`/test` AS SELECT * FROM dfs.tmp.`/test.json`;
The same query:
select lower(p.dimensions.budgetLevel) as `field1`,
lower(p.dimensions.adults) as `field2` from dfs.tmp.`/test/0_0_0.parquet` as
p;
Return this:
+-+-+
| field1 | field2 |
+-+-+
| a | null|
+-+-+
After some more testing it appears that this has nothing to do with trim.
(any non existing nested-value will be pushed aside)
select p.dimensions.budgetLevel as `field1`, lower(p.dimensions.adults) as
`field2` from dfs.tmp.`/test/0_0_0.parquet` as p;
also returns:
+-+-+
| field1 | field2 |
+-+-+
| a | null|
+-+-+
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3497) Throw UserException#validationError for errors when modifying options
[ https://issues.apache.org/jira/browse/DRILL-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637990#comment-14637990 ] ASF GitHub Bot commented on DRILL-3497: --- Github user sudheeshkatkam commented on the pull request: https://github.com/apache/drill/pull/98#issuecomment-123917913 Addressed review comments mentioned on #91 Throw UserException#validationError for errors when modifying options - Key: DRILL-3497 URL: https://issues.apache.org/jira/browse/DRILL-3497 Project: Apache Drill Issue Type: Bug Components: Execution - Flow Reporter: Sudheesh Katkam Assignee: Sudheesh Katkam Priority: Minor Fix For: 1.2.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2650) Cancelled queries json profile shows query end time occurs before fragments end time
[ https://issues.apache.org/jira/browse/DRILL-2650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637999#comment-14637999 ] ASF GitHub Bot commented on DRILL-2650: --- Github user sudheeshkatkam commented on the pull request: https://github.com/apache/drill/pull/80#issuecomment-123918414 Downgraded to .debug() Cancelled queries json profile shows query end time occurs before fragments end time - Key: DRILL-2650 URL: https://issues.apache.org/jira/browse/DRILL-2650 Project: Apache Drill Issue Type: Bug Components: Client - HTTP Affects Versions: 0.9.0 Reporter: Krystal Assignee: Jacques Nadeau Fix For: 1.2.0 Attachments: DRILL-2650.1.patch.txt I have a query that was successfully cancelled. The query start and end time is as follows: type: 1, start: 1427839192049, end: 1427839194966, This translates to a query duration of about 3 seconds. However, the duration of the query's fragments are much longer up to more than 6 seconds. Here is an entry for majorFragmentId=0 with a duration of 6.6 seconds: startTime: 1427839192796, endTime: 1427839199408, 8 out of 11 major fragments have duration greater than the query itself. To an end user, this is confusing and does not make sense. We should wait for all of the major fragments to be completely cancelled before updating the the end time of the query. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-3497) Throw UserException#validationError for errors when modifying options
[ https://issues.apache.org/jira/browse/DRILL-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637962#comment-14637962 ] ASF GitHub Bot commented on DRILL-3497: --- GitHub user sudheeshkatkam opened a pull request: https://github.com/apache/drill/pull/98 DRILL-3497: Throw UserException#validationError for errors when modifying options Mistakenly closed https://github.com/apache/drill/pull/91. Please review 4c2b698. You can merge this pull request into a Git repository by running: $ git pull https://github.com/sudheeshkatkam/drill DRILL-3497 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/drill/pull/98.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #98 commit 9dc056ec9f60b0e91acd345ca496a7e6fc077c05 Author: Sudheesh Katkam skat...@maprtech.com Date: 2015-07-22T23:32:02Z DRILL-2304: Manage options with lowercase names Major refactoring: + Removed unnecessary methods from OptionManagers, and refactored implementations + Removed OptionAdmin interface, which was a delegate to validators Other fixes: + Added CaseInsensitiveMap + Moved OptionValue creation from TypeValidators to SetOptionHandler + Fixed error message in BaseOptionManager + Added documentation for all OptionManagers, and OptionValidator commit 4c2b698fe5c059c7987c808e7e90f55659b74ba5 Author: Sudheesh Katkam skat...@maprtech.com Date: 2015-07-14T22:33:30Z DRILL-3497: Throw UserException#validationError instead of SetOptionException and ExpressionParsingException for options + Fixed bug in PositiveLongValidator (0 is not positive) + Added UserExceptionMatcher for unit tests Throw UserException#validationError for errors when modifying options - Key: DRILL-3497 URL: https://issues.apache.org/jira/browse/DRILL-3497 Project: Apache Drill Issue Type: Bug Components: Execution - Flow Reporter: Sudheesh Katkam Assignee: Sudheesh Katkam Priority: Minor Fix For: 1.2.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2304) Case sensitivity - system and session options are case sensitive
[
https://issues.apache.org/jira/browse/DRILL-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637997#comment-14637997
]
ASF GitHub Bot commented on DRILL-2304:
---
Github user sudheeshkatkam commented on the pull request:
https://github.com/apache/drill/pull/90#issuecomment-123918173
Addressed review comments mentioned on 313c8e3 f33654c
Case sensitivity - system and session options are case sensitive
Key: DRILL-2304
URL: https://issues.apache.org/jira/browse/DRILL-2304
Project: Apache Drill
Issue Type: Bug
Components: Storage - Information Schema
Affects Versions: 0.8.0
Reporter: Ramana Inukonda Nagaraj
Assignee: Sudheesh Katkam
Priority: Minor
Fix For: 1.2.0
Attachments: DRILL-2304.1.patch.txt, DRILL-2304.2.patch.txt
TBH I am not sure if this is a bug.
When trying to set a session option and I specify the name in a different
case the alter command fails. Considering the way we store session options
this might be an invalid bug but considering how typical Database hints and
options work this is a bug.
{code}
0: jdbc:drill: alter SESSION set `STORE.PARQUET.COMPRESSION`='GZIP';
Query failed: SetOptionException: Unknown option: STORE.PARQUET.COMPRESSION
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-3543) Add stats for external sort to a query profile
Victoria Markman created DRILL-3543:
---
Summary: Add stats for external sort to a query profile
Key: DRILL-3543
URL: https://issues.apache.org/jira/browse/DRILL-3543
Project: Apache Drill
Issue Type: Improvement
Components: Execution - Relational Operators
Affects Versions: 1.1.0
Reporter: Victoria Markman
Assignee: Chris Westin
The only indication if sort spilled to disk today is info from the drillbit.log.
It would be great if this information was displayed in the query profile.
{code}
015-07-22 23:47:29,907 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] INFO
o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to
/tmp/drill/spill/2a4fd46e-f8c3-6b96-b165-b665a41be311/major_fragment_0/minor_fragment_0/operator_7/92
2015-07-22 23:47:29,919 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] INFO
o.a.d.e.p.i.xsort.ExternalSortBatch - Merging and spilling to
/tmp/drill/spill/2a4fd46e-f8c3-6b96-b165-b665a41be311/major_fragment_0/minor_fragment_0/operator_7/93
2015-07-22 23:47:29,919 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] INFO
o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to
/tmp/drill/spill/2a4fd46e-f8c3-6b96-b165-b665a41be311/major_fragment_0/minor_fragment_0/operator_7/93
2015-07-22 23:47:29,919 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] WARN
o.a.d.e.p.i.xsort.ExternalSortBatch - Starting to merge. 7 batch groups.
Current allocated memory: 11566787
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2828) Nested WHERE clause error on a HBase view
[
https://issues.apache.org/jira/browse/DRILL-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Hsuan-Yi Chu updated DRILL-2828:
-
Fix Version/s: (was: 1.2.0)
1.4.0
Nested WHERE clause error on a HBase view
-
Key: DRILL-2828
URL: https://issues.apache.org/jira/browse/DRILL-2828
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 0.8.0
Reporter: Subhajit Ghosh
Assignee: Sean Hsuan-Yi Chu
Fix For: 1.4.0
A RexInputRef error when using a nested WHERE clause (with DOT notation)
against a view. The following can be used to reproduce the error:
create 'MyTable', 'main'
put 'MyTable', 'id1', 'main:objectBytes', '{foo:123,prop:{len:234}}'
In Drill:
{code}
create or replace view MyTableView as select
convert_from(MyTable.main.objectBytes, 'JSON') json from hbase.MyTable;
select v.json.prop.len as len from MyTableView v;
++
|len |
++
| 234|
++
{code}
This produces the error:
{code}
select v.json.prop.len as len from MyTableView v where v.json.prop.len
2;
Query failed: AssertionError: RexInputRef index 2 out of range 0..1
{code}
Detailed error below. Please let me know if you need more information.
{code}
2015-04-13 12:58:31,444 [2ad45097-c878-962c-b3be-288928a4bc79:foreman] INFO
o.a.d.e.s.hbase.TableStatsCalculator - Region size calculation disabled.
2015-04-13 12:58:31,600 [2ad45097-c878-962c-b3be-288928a4bc79:foreman] INFO
o.a.drill.exec.work.foreman.Foreman - State change requested. PENDING --
FAILED
org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception
during fragment initialization: null
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:213)
[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_71]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
Caused by: java.lang.reflect.UndeclaredThrowableException: null
at com.sun.proxy.$Proxy63.getRowCount(Unknown Source) ~[na:na]
at
org.eigenbase.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:96)
~[optiq-core-0.9-drill-r20.jar:na]
at org.eigenbase.rel.SingleRel.getRows(SingleRel.java:65)
~[optiq-core-0.9-drill-r20.jar:na]
at
org.apache.drill.exec.planner.physical.visitor.ExcessiveExchangeIdentifier$MajorFragmentStat.add(ExcessiveExchangeIdentifier.java:99)
~[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at
org.apache.drill.exec.planner.physical.visitor.ExcessiveExchangeIdentifier.visitPrel(ExcessiveExchangeIdentifier.java:74)
~[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at
org.apache.drill.exec.planner.physical.visitor.ExcessiveExchangeIdentifier.visitPrel(ExcessiveExchangeIdentifier.java:31)
~[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at
org.apache.drill.exec.planner.physical.visitor.BasePrelVisitor.visitProject(BasePrelVisitor.java:48)
~[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at
org.apache.drill.exec.planner.physical.ProjectPrel.accept(ProjectPrel.java:69)
~[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at
org.apache.drill.exec.planner.physical.visitor.ExcessiveExchangeIdentifier.visitScreen(ExcessiveExchangeIdentifier.java:61)
~[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at
org.apache.drill.exec.planner.physical.visitor.ExcessiveExchangeIdentifier.visitScreen(ExcessiveExchangeIdentifier.java:31)
~[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at
org.apache.drill.exec.planner.physical.ScreenPrel.accept(ScreenPrel.java:64)
~[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at
org.apache.drill.exec.planner.physical.visitor.ExcessiveExchangeIdentifier.removeExcessiveEchanges(ExcessiveExchangeIdentifier.java:42)
~[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToPrel(DefaultSqlHandler.java:290)
~[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:140)
~[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:145)
~[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:773)
[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:204)
[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
... 3 common frames omitted
Caused by: java.lang.reflect.InvocationTargetException: null
at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source) ~[na:na]
at
[jira] [Commented] (DRILL-3533) null values in a sub-structure in Parquet returns unexpected/misleading results
[
https://issues.apache.org/jira/browse/DRILL-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637014#comment-14637014
]
Stefán Baxter commented on DRILL-3533:
--
thanks a lot
null values in a sub-structure in Parquet returns unexpected/misleading
results
---
Key: DRILL-3533
URL: https://issues.apache.org/jira/browse/DRILL-3533
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 1.1.0
Reporter: Stefán Baxter
Assignee: Jinfeng Ni
Priority: Critical
With this minimal dataset as /tmp/test.json:
{dimensions:{adults:A}}
select lower(p.dimensions.budgetLevel) as `field1`,
lower(p.dimensions.adults) as `field2` from dfs.tmp.`/test.json` as p;
Returns this:
+-+-+
| field1 | field2 |
+-+-+
| null| a |
+-+-+
With the same data as a Parquet file
CREATE TABLE dfs.tmp.`/test` AS SELECT * FROM dfs.tmp.`/test.json`;
The same query:
select lower(p.dimensions.budgetLevel) as `field1`,
lower(p.dimensions.adults) as `field2` from dfs.tmp.`/test/0_0_0.parquet` as
p;
Return this:
+-+-+
| field1 | field2 |
+-+-+
| a | null|
+-+-+
After some more testing it appears that this has nothing to do with trim.
(any non existing nested-value will be pushed aside)
select p.dimensions.budgetLevel as `field1`, lower(p.dimensions.adults) as
`field2` from dfs.tmp.`/test/0_0_0.parquet` as p;
also returns:
+-+-+
| field1 | field2 |
+-+-+
| a | null|
+-+-+
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3533) null values in a sub-structure in Parquet returns unexpected/misleading results
[
https://issues.apache.org/jira/browse/DRILL-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637013#comment-14637013
]
Jinfeng Ni commented on DRILL-3533:
---
I will post the patch for review today. If the reviewer does not see any issue
with the patch, I'll merge the patch.
null values in a sub-structure in Parquet returns unexpected/misleading
results
---
Key: DRILL-3533
URL: https://issues.apache.org/jira/browse/DRILL-3533
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 1.1.0
Reporter: Stefán Baxter
Assignee: Jinfeng Ni
Priority: Critical
With this minimal dataset as /tmp/test.json:
{dimensions:{adults:A}}
select lower(p.dimensions.budgetLevel) as `field1`,
lower(p.dimensions.adults) as `field2` from dfs.tmp.`/test.json` as p;
Returns this:
+-+-+
| field1 | field2 |
+-+-+
| null| a |
+-+-+
With the same data as a Parquet file
CREATE TABLE dfs.tmp.`/test` AS SELECT * FROM dfs.tmp.`/test.json`;
The same query:
select lower(p.dimensions.budgetLevel) as `field1`,
lower(p.dimensions.adults) as `field2` from dfs.tmp.`/test/0_0_0.parquet` as
p;
Return this:
+-+-+
| field1 | field2 |
+-+-+
| a | null|
+-+-+
After some more testing it appears that this has nothing to do with trim.
(any non existing nested-value will be pushed aside)
select p.dimensions.budgetLevel as `field1`, lower(p.dimensions.adults) as
`field2` from dfs.tmp.`/test/0_0_0.parquet` as p;
also returns:
+-+-+
| field1 | field2 |
+-+-+
| a | null|
+-+-+
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3533) null values in a sub-structure in Parquet returns unexpected/misleading results
[
https://issues.apache.org/jira/browse/DRILL-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14636810#comment-14636810
]
Stefán Baxter commented on DRILL-3533:
--
do you think you can commit this change soon? :)
null values in a sub-structure in Parquet returns unexpected/misleading
results
---
Key: DRILL-3533
URL: https://issues.apache.org/jira/browse/DRILL-3533
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 1.1.0
Reporter: Stefán Baxter
Assignee: Jinfeng Ni
Priority: Critical
With this minimal dataset as /tmp/test.json:
{dimensions:{adults:A}}
select lower(p.dimensions.budgetLevel) as `field1`,
lower(p.dimensions.adults) as `field2` from dfs.tmp.`/test.json` as p;
Returns this:
+-+-+
| field1 | field2 |
+-+-+
| null| a |
+-+-+
With the same data as a Parquet file
CREATE TABLE dfs.tmp.`/test` AS SELECT * FROM dfs.tmp.`/test.json`;
The same query:
select lower(p.dimensions.budgetLevel) as `field1`,
lower(p.dimensions.adults) as `field2` from dfs.tmp.`/test/0_0_0.parquet` as
p;
Return this:
+-+-+
| field1 | field2 |
+-+-+
| a | null|
+-+-+
After some more testing it appears that this has nothing to do with trim.
(any non existing nested-value will be pushed aside)
select p.dimensions.budgetLevel as `field1`, lower(p.dimensions.adults) as
`field2` from dfs.tmp.`/test/0_0_0.parquet` as p;
also returns:
+-+-+
| field1 | field2 |
+-+-+
| a | null|
+-+-+
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
