[jira] [Commented] (DRILL-6983) PAM Auth Enabled on Drill-On-YARN only works on YARN user
[
https://issues.apache.org/jira/browse/DRILL-6983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758033#comment-16758033
]
Michael Dennis Uanang commented on DRILL-6983:
--
Thanks, [~kkhatua]already ask to mailing list.
[~shamirwasia], yes did follow the steps (also follow the MapR docs). Also used
both jpam and jibpam4j BUT they have the same result.
ONLY the *yarn* user is allowed to logged in when PAM is enabled.
> PAM Auth Enabled on Drill-On-YARN only works on YARN user
> -
>
> Key: DRILL-6983
> URL: https://issues.apache.org/jira/browse/DRILL-6983
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - HTTP
>Affects Versions: 1.14.0, 1.15.0
>Reporter: Michael Dennis Uanang
>Priority: Major
> Attachments: Selection_999(203).png, Selection_999(204).png,
> Selection_999(205).png
>
>
> Hi,
> I'm having problem running Drill-on-YARN with PAM authentication enabled. PAM
> auth is working, BUT only accepting login via WEBUI for YARN user.
> _drill-override.conf_
>
> {code:java}
> drill.exec: {
> cluster-id: "drillbits2",
> zk.connect: "app40:2181,app41:2181,app42:2181",
> impersonation: {
> enabled: true
> },
> security: {
> auth.mechanisms: [ "PLAIN" ],
> user.auth.enabled: true,
> user.auth.packages += "org.apache.drill.exec.rpc.user.security",
> user.auth.impl: "pam",
> user.auth.pam_profiles: [ "login", "sshd" ]
> }
> }
> {code}
>
>
> SEE errors below:
> !Selection_999(204).png!
>
> !Selection_999(203).png!
> As you can see from the screenshot, when trying to login via WEBUI using
> infra or drill user, I'm having error 'password check failed for user
> (USER)`. But you'll also notice that it's giving me authentication failure
> for UID=1018 which is YARN
> !Selection_999(205).png!
>
> Please help me to right direction or if I'm missing something.
> Thank you.
>
> MD
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6755) HashJoin should not build hash tables when probe side is empty.
[
https://issues.apache.org/jira/browse/DRILL-6755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757752#comment-16757752
]
Robert Hou commented on DRILL-6755:
---
Boaz suggested verifying this by joining with an empty file.
{noformat}
select count(*) from dfs.`/empty.json` E where E.l_orderkey in (select
L.l_orderkey from lineitem L);
{noformat}
I tested this with Drill 1.15. I had to turn off semijoins to get the desired
plan because if a semijoin is used, then the join is re-ordered so that the
empty file is on the build side (may be a bug).
I was able to verify that the hash join operator does not build a hash table
for this query.
> HashJoin should not build hash tables when probe side is empty.
> ---
>
> Key: DRILL-6755
> URL: https://issues.apache.org/jira/browse/DRILL-6755
> Project: Apache Drill
> Issue Type: Improvement
>Reporter: Timothy Farkas
>Assignee: Boaz Ben-Zvi
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.15.0
>
>
> Currently when doing an Inner or a Right join we still build hashtables when
> the probe side is empty. A performance optimization would be to not build
> them.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (DRILL-6755) HashJoin should not build hash tables when probe side is empty.
[ https://issues.apache.org/jira/browse/DRILL-6755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Hou closed DRILL-6755. - > HashJoin should not build hash tables when probe side is empty. > --- > > Key: DRILL-6755 > URL: https://issues.apache.org/jira/browse/DRILL-6755 > Project: Apache Drill > Issue Type: Improvement >Reporter: Timothy Farkas >Assignee: Boaz Ben-Zvi >Priority: Major > Labels: ready-to-commit > Fix For: 1.15.0 > > > Currently when doing an Inner or a Right join we still build hashtables when > the probe side is empty. A performance optimization would be to not build > them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-7021) HTTPD Throws NPE and Doesn't Recognize Timeformat
Charles Givre created DRILL-7021: Summary: HTTPD Throws NPE and Doesn't Recognize Timeformat Key: DRILL-7021 URL: https://issues.apache.org/jira/browse/DRILL-7021 Project: Apache Drill Issue Type: Bug Components: Storage - Other Affects Versions: 1.16.0 Reporter: Charles Givre Assignee: Charles Givre The HTTPD format plugin throws a NPE when you attempt to query all fields and enumerate them in the query. The query below will throw the NPE: ``` SELECT `request_referer_query`, . . . . . . .semicolon> `request_referer_ref`, . . . . . . .semicolon> `request_firstline_uri_port`, . . . . . . .semicolon> `request_firstline_method`, . . . . . . .semicolon> `request_firstline_uri_protocol`, . . . . . . .semicolon> `request_receive_time_millisecond`, . . . . . . .semicolon> `request_receive_time_day__utc`, . . . . . . .semicolon> `request_receive_time_year__utc`, . . . . . . .semicolon> `request_receive_time_weekofweekyear__utc`, . . . . . . .semicolon> `request_receive_time_second`, . . . . . . .semicolon> `connection_client_host`, . . . . . . .semicolon> `request_referer_userinfo`, . . . . . . .semicolon> `request_referer_path`, . . . . . . .semicolon> `request_referer_host`, . . . . . . .semicolon> `request_receive_time_monthname`, . . . . . . .semicolon> `request_receive_time_month__utc`, . . . . . . .semicolon> `request_firstline_uri_ref`, . . . . . . .semicolon> `request_firstline_protocol_version`, . . . . . . .semicolon> `request_receive_time_minute`, . . . . . . .semicolon> `request_firstline`, . . . . . . .semicolon> `request_receive_time_day`, . . . . . . .semicolon> `request_referer_port`, . . . . . . .semicolon> `request_receive_time_year`, . . . . . . .semicolon> `request_referer_query_$`, . . . . . . .semicolon> `request_firstline_uri_query_$`, . . . . . . .semicolon> `request_firstline_uri`, . . . . . . .semicolon> `request_receive_time_month`, . . . . . . .semicolon> `request_receive_time_weekofweekyear`, . . . . . . .semicolon> `request_firstline_uri_userinfo`, . . . . . . .semicolon> `request_referer`, . . . . . . .semicolon> `request_receive_time_epoch`, . . . . . . .semicolon> `request_referer_protocol`, . . . . . . .semicolon> `request_receive_time_monthname__utc`, . . . . . . .semicolon> `connection_client_logname`, . . . . . . .semicolon> `request_receive_time`, . . . . . . .semicolon> `request_firstline_protocol`, . . . . . . .semicolon> `request_receive_time_hour`, . . . . . . .semicolon> `request_firstline_uri_host`, . . . . . . .semicolon> `request_firstline_uri_path`, . . . . . . .semicolon> `request_user-agent`, . . . . . . .semicolon> `request_receive_time_hour__utc`, . . . . . . .semicolon> `request_receive_time_second__utc`, . . . . . . .semicolon> `request_receive_time_weekyear`, . . . . . . .semicolon> `request_receive_time_timezone`, . . . . . . .semicolon> `request_receive_time_weekyear__utc`, . . . . . . .semicolon> `response_body_bytesclf`, . . . . . . .semicolon> `connection_client_user`, . . . . . . .semicolon> `request_receive_time_millisecond__utc`, . . . . . . .semicolon> `request_status_last`, . . . . . . .semicolon> `request_firstline_uri_query`, . . . . . . .semicolon> `request_receive_time_minute__utc` . . . . . . .semicolon> FROM `dfs.drillclass`.`hackers-access.httpd` ``` The cause for the NPE is that several fields were missing from a type map in the format plugin. Separately, the format plugin is not recognizing the time stamp and is not parsing dates as time formats. Oh... and the unit tests suck. Sorry. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7018) Drill Query (when store.parquet.reader.int96_as_timestamp=true) on Parquet File fails with Error: SYSTEM ERROR: IndexOutOfBoundsException: readerIndex: 0, writerIndex: 37
[ https://issues.apache.org/jira/browse/DRILL-7018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] salim achouche updated DRILL-7018: -- Reviewer: Vitalii Diravka (was: Boaz Ben-Zvi) > Drill Query (when store.parquet.reader.int96_as_timestamp=true) on Parquet > File fails with Error: SYSTEM ERROR: IndexOutOfBoundsException: readerIndex: > 0, writerIndex: 372 (expected: 0 <= readerIndex <= writerIndex <= > capacity(256)) > > > Key: DRILL-7018 > URL: https://issues.apache.org/jira/browse/DRILL-7018 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Parquet >Affects Versions: 1.14.0 >Reporter: salim achouche >Assignee: salim achouche >Priority: Major > Labels: ready-to-commit > Fix For: 1.16.0 > > Original Estimate: 0h > Remaining Estimate: 0h > > alter system set `store.parquet.reader.int96_as_timestamp`= true > run query witch projects a column of type Parquet INT96 timestamp with 31 > nulls > The following exception will be thrown: > java.lang.IndexOutOfBoundsException: readerIndex: 0, writerIndex: 372 > (expected: 0 <= readerIndex <= writerIndex <= capacity(256)) > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6992) Support column histogram statistics
[ https://issues.apache.org/jira/browse/DRILL-6992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757549#comment-16757549 ] Aman Sinha commented on DRILL-6992: --- Added a link to the histogram design doc. > Support column histogram statistics > --- > > Key: DRILL-6992 > URL: https://issues.apache.org/jira/browse/DRILL-6992 > Project: Apache Drill > Issue Type: New Feature > Components: Query Planning Optimization >Affects Versions: 1.15.0 >Reporter: Aman Sinha >Assignee: Aman Sinha >Priority: Major > > As a follow-up to > [DRILL-1328|https://issues.apache.org/jira/browse/DRILL-1328] which is adding > NDV (num distinct values) support and creating the framework for statistics, > we also need Histograms. These are needed for range predicates selectivity > estimation as well as equality predicates when there is non-uniform > distribution of data. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6862) Update Calcite to 1.18.0
[ https://issues.apache.org/jira/browse/DRILL-6862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Volodymyr Vysotskyi updated DRILL-6862: --- Labels: ready-to-commit (was: ) > Update Calcite to 1.18.0 > - > > Key: DRILL-6862 > URL: https://issues.apache.org/jira/browse/DRILL-6862 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.15.0 >Reporter: Igor Guzenko >Assignee: Igor Guzenko >Priority: Major > Labels: ready-to-commit > Fix For: 1.16.0 > > > After ongoing release of the new Calcite version we will change our > dependency. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6964) Implement CREATE / DROP TABLE SCHEMA commands
[ https://issues.apache.org/jira/browse/DRILL-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Volodymyr Vysotskyi updated DRILL-6964: --- Labels: ready-to-commit (was: ) > Implement CREATE / DROP TABLE SCHEMA commands > - > > Key: DRILL-6964 > URL: https://issues.apache.org/jira/browse/DRILL-6964 > Project: Apache Drill > Issue Type: Sub-task >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva >Priority: Major > Labels: ready-to-commit > Fix For: 1.16.0 > > > Design doc - > https://docs.google.com/document/d/1mp4egSbNs8jFYRbPVbm_l0Y5GjH3HnoqCmOpMTR_g4w/edit -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-4232) Support for EXCEPT set operator
[ https://issues.apache.org/jira/browse/DRILL-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757288#comment-16757288 ] benj commented on DRILL-4232: - Duplicate ticket as DRILL-4224 (MINUS and EXCEPT are alias) But very expected and useful possibility [https://www.postgresql.org/docs/11/sql-select.html#SQL-EXCEPT] [https://www.ibm.com/support/knowledgecenter/en/SSGU8G_12.1.0/com.ibm.sqls.doc/ids_sqs_1836.htm] [https://docs.oracle.com/cd/B19306_01/server.102/b14200/queries004.htm] > Support for EXCEPT set operator > --- > > Key: DRILL-4232 > URL: https://issues.apache.org/jira/browse/DRILL-4232 > Project: Apache Drill > Issue Type: New Feature > Components: Query Planning Optimization >Reporter: Victoria Markman >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-923) Bit wise and logical Aggregate functions
[
https://issues.apache.org/jira/browse/DRILL-923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757269#comment-16757269
]
benj commented on DRILL-923:
Please updating documentation in which these functions do not appear :
https://drill.apache.org/docs/aggregate-and-aggregate-statistical/
> Bit wise and logical Aggregate functions
>
>
> Key: DRILL-923
> URL: https://issues.apache.org/jira/browse/DRILL-923
> Project: Apache Drill
> Issue Type: New Feature
> Components: Functions - Drill
>Affects Versions: 0.4.0
>Reporter: Yash Sharma
>Assignee: DrillCommitter
>Priority: Major
> Fix For: 0.4.0
>
> Attachments: DRILL-923-formatted.patch, DRILL-923.patch,
> DRILL-923.patch
>
>
> Implementing new Bitwise and Logical aggregate functions:
> {quote}bit_and(expression)
> bit_or(expression)
> bool_and(expression)
> bool_or(expression)
> every(expression)
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-7017) lz4 codec for (un)compression
[
https://issues.apache.org/jira/browse/DRILL-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757258#comment-16757258
]
benj commented on DRILL-7017:
-
lz4 is well in the list of supported compression.
Searched a lot to try to work with .csv.lz4 but don't find any way.
I encounter problems with hadoop & Lz4 due to HADOOP-12990. But I don't know if
this problem can really prevent Drill from working with .csv.lz4 files
> lz4 codec for (un)compression
> -
>
> Key: DRILL-7017
> URL: https://issues.apache.org/jira/browse/DRILL-7017
> Project: Apache Drill
> Issue Type: Wish
> Components: Storage - Text CSV
>Affects Versions: 1.15.0
>Reporter: benj
>Priority: Major
>
> I didn't find in the documentation what compression formats are supported.
> But as it's possible to use drill on compressed file, like
> {code:java}
> SELECT * FROM tmp.`myfile.csv.gz`;
> {code}
> It will be useful to have the possibility to use this functionality for lz4
> file ([https://github.com/lz4/lz4])
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-7019) Add check for redundant imports
[
https://issues.apache.org/jira/browse/DRILL-7019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vitalii Diravka updated DRILL-7019:
---
Labels: ready-to-commit (was: )
> Add check for redundant imports
> ---
>
> Key: DRILL-7019
> URL: https://issues.apache.org/jira/browse/DRILL-7019
> Project: Apache Drill
> Issue Type: Task
> Components: Tools, Build Test
>Affects Versions: 1.15.0
>Reporter: Volodymyr Vysotskyi
>Assignee: Volodymyr Vysotskyi
>Priority: Minor
> Labels: ready-to-commit
> Fix For: 1.16.0
>
>
> Currently, used only {{UnusedImports}} check which does not prevents
> duplicate imports or imports from the same package.
> The goal of this Jira is to add {{RedundantImport}} check and fix checkstyle
> errors.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-7018) Drill Query (when store.parquet.reader.int96_as_timestamp=true) on Parquet File fails with Error: SYSTEM ERROR: IndexOutOfBoundsException: readerIndex: 0, writerIndex: 37
[ https://issues.apache.org/jira/browse/DRILL-7018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vitalii Diravka updated DRILL-7018: --- Labels: ready-to-commit (was: ) > Drill Query (when store.parquet.reader.int96_as_timestamp=true) on Parquet > File fails with Error: SYSTEM ERROR: IndexOutOfBoundsException: readerIndex: > 0, writerIndex: 372 (expected: 0 <= readerIndex <= writerIndex <= > capacity(256)) > > > Key: DRILL-7018 > URL: https://issues.apache.org/jira/browse/DRILL-7018 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Parquet >Affects Versions: 1.14.0 >Reporter: salim achouche >Assignee: salim achouche >Priority: Major > Labels: ready-to-commit > Fix For: 1.16.0 > > Original Estimate: 0h > Remaining Estimate: 0h > > alter system set `store.parquet.reader.int96_as_timestamp`= true > run query witch projects a column of type Parquet INT96 timestamp with 31 > nulls > The following exception will be thrown: > java.lang.IndexOutOfBoundsException: readerIndex: 0, writerIndex: 372 > (expected: 0 <= readerIndex <= writerIndex <= capacity(256)) > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
