[jira] [Updated] (HIVE-9143) select user(), current_user()
[
https://issues.apache.org/jira/browse/HIVE-9143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-9143:
Release Note:
Adds a UDF current_user that returns current user name
Example -
{code}
hive> select current_user();
OK
thejas
Time taken: 1.786 seconds, Fetched: 1 row(s)
{code}
was:Returns current user name
[~leftylev] What would be a good place to document this.
I have updated the release notes.
> select user(), current_user()
> -
>
> Key: HIVE-9143
> URL: https://issues.apache.org/jira/browse/HIVE-9143
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 0.13.0
>Reporter: Hari Sekhon
>Assignee: Alexander Pivovarov
>Priority: Minor
> Fix For: 1.2.0
>
> Attachments: HIVE-9143.1.patch, HIVE-9143.2.patch, HIVE-9143.3.patch
>
>
> Feature request to add support for determining in HQL session which user I am
> currently connected as - an old MySQL ability:
> {code}mysql> select user(), current_user();
> +++
> | user() | current_user() |
> +++
> | root@localhost | root@localhost |
> +++
> 1 row in set (0.00 sec)
> {code}
> which doesn't seem to have a counterpart in Hive at this time:
> {code}0: jdbc:hive2://:100> select user();
> Error: Error while compiling statement: FAILED: SemanticException Line 0:-1
> Invalid function 'user' (state=42000,code=4)
> 0: jdbc:hive2://:100> select current_user();
> Error: Error while compiling statement: FAILED: SemanticException [Error
> 10011]: Line 1:7 Invalid function 'current_user'
> (state=42000,code=10011){code}
> Regards,
> Hari Sekhon
> http://www.linkedin.com/in/harisekhon
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13290) Support primary keys/foreign keys constraint as part of create table command in Hive
[ https://issues.apache.org/jira/browse/HIVE-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sankar Sivarama Subramaniyan updated HIVE-13290: - Status: Patch Available (was: Open) > Support primary keys/foreign keys constraint as part of create table command > in Hive > > > Key: HIVE-13290 > URL: https://issues.apache.org/jira/browse/HIVE-13290 > Project: Hive > Issue Type: Sub-task > Components: CBO, Logical Optimizer >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Hari Sankar Sivarama Subramaniyan > Attachments: HIVE-13290.1.patch, HIVE-13290.2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13290) Support primary keys/foreign keys constraint as part of create table command in Hive
[ https://issues.apache.org/jira/browse/HIVE-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sankar Sivarama Subramaniyan updated HIVE-13290: - Attachment: HIVE-13290.2.patch > Support primary keys/foreign keys constraint as part of create table command > in Hive > > > Key: HIVE-13290 > URL: https://issues.apache.org/jira/browse/HIVE-13290 > Project: Hive > Issue Type: Sub-task > Components: CBO, Logical Optimizer >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Hari Sankar Sivarama Subramaniyan > Attachments: HIVE-13290.1.patch, HIVE-13290.2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13290) Support primary keys/foreign keys constraint as part of create table command in Hive
[
https://issues.apache.org/jira/browse/HIVE-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hari Sankar Sivarama Subramaniyan updated HIVE-13290:
-
Description:
SUPPORT for the following statements
{code}
CREATE TABLE product
(
product_idINTEGER,
product_vendor_id INTEGER,
PRIMARY KEY (product_id),
CONSTRAINT product_fk_1 FOREIGN KEY (product_vendor_id) REFERENCES
vendor(vendor_id)
);
CREATE TABLE vendor
(
vendor_id INTEGER,
PRIMARY KEY (vendor_id)
);
{code}
In the above syntax, [CONSTRAINT constraint-Name] is optional. If this is not
specified by the user, we will use system generated constraint name. For the
purpose of simplicity, we will allow only CONSTRAINT option for foreign keys.
> Support primary keys/foreign keys constraint as part of create table command
> in Hive
>
>
> Key: HIVE-13290
> URL: https://issues.apache.org/jira/browse/HIVE-13290
> Project: Hive
> Issue Type: Sub-task
> Components: CBO, Logical Optimizer
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Attachments: HIVE-13290.1.patch, HIVE-13290.2.patch
>
>
> SUPPORT for the following statements
> {code}
> CREATE TABLE product
> (
> product_idINTEGER,
> product_vendor_id INTEGER,
> PRIMARY KEY (product_id),
> CONSTRAINT product_fk_1 FOREIGN KEY (product_vendor_id) REFERENCES
> vendor(vendor_id)
> );
> CREATE TABLE vendor
> (
> vendor_id INTEGER,
> PRIMARY KEY (vendor_id)
> );
> {code}
> In the above syntax, [CONSTRAINT constraint-Name] is optional. If this is not
> specified by the user, we will use system generated constraint name. For the
> purpose of simplicity, we will allow only CONSTRAINT option for foreign keys.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13085) Need an API / configuration parameter to find out the authenticated user from beeline
[ https://issues.apache.org/jira/browse/HIVE-13085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209893#comment-15209893 ] Thejas M Nair commented on HIVE-13085: -- Would this work in kerberos mode ? (I am not sure if it would). I think the feature ought to work consistently across all authentication modes. We already have a 'select username()' feature available via HIVE-9143. > Need an API / configuration parameter to find out the authenticated user > from beeline > -- > > Key: HIVE-13085 > URL: https://issues.apache.org/jira/browse/HIVE-13085 > Project: Hive > Issue Type: Improvement >Reporter: Oleksiy Sayankin >Assignee: Oleksiy Sayankin > Fix For: 2.0.0 > > Attachments: HIVE-13085.1.patch > > > This request has come from the customer who wanted an API / configuration > parameter to identify the authenticated user from the beeline. This is > similar to the request done in the thread below: > https://community.hortonworks.com/questions/2620/hadoop-environment-variable-or-configuration-varia.html > But this would not be a feasible option for the requestor. So here the > general ask is once the user is logged in to the beeline, they wanted to > identify who this user is and then use this information to enforce the ACLs > on the tables through the customer's custom code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13290) Support primary keys/foreign keys constraint as part of create table command in Hive
[
https://issues.apache.org/jira/browse/HIVE-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hari Sankar Sivarama Subramaniyan updated HIVE-13290:
-
Description:
SUPPORT for the following statements
{code}
CREATE TABLE product
(
product_idINTEGER,
product_vendor_id INTEGER,
PRIMARY KEY (product_id),
CONSTRAINT product_fk_1 FOREIGN KEY (product_vendor_id) REFERENCES
vendor(vendor_id)
);
CREATE TABLE vendor
(
vendor_id INTEGER,
PRIMARY KEY (vendor_id)
);
{code}
In the above syntax, [CONSTRAINT constraint-Name] is optional. If this is not
specified by the user, we will use system generated constraint name. For the
purpose of simplicity, we will allow CONSTRAINT option for foreign keys and
not primary key since there is only one primary key per table.
was:
SUPPORT for the following statements
{code}
CREATE TABLE product
(
product_idINTEGER,
product_vendor_id INTEGER,
PRIMARY KEY (product_id),
CONSTRAINT product_fk_1 FOREIGN KEY (product_vendor_id) REFERENCES
vendor(vendor_id)
);
CREATE TABLE vendor
(
vendor_id INTEGER,
PRIMARY KEY (vendor_id)
);
{code}
In the above syntax, [CONSTRAINT constraint-Name] is optional. If this is not
specified by the user, we will use system generated constraint name. For the
purpose of simplicity, we will allow only CONSTRAINT option for foreign keys.
> Support primary keys/foreign keys constraint as part of create table command
> in Hive
>
>
> Key: HIVE-13290
> URL: https://issues.apache.org/jira/browse/HIVE-13290
> Project: Hive
> Issue Type: Sub-task
> Components: CBO, Logical Optimizer
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Attachments: HIVE-13290.1.patch, HIVE-13290.2.patch
>
>
> SUPPORT for the following statements
> {code}
> CREATE TABLE product
> (
> product_idINTEGER,
> product_vendor_id INTEGER,
> PRIMARY KEY (product_id),
> CONSTRAINT product_fk_1 FOREIGN KEY (product_vendor_id) REFERENCES
> vendor(vendor_id)
> );
> CREATE TABLE vendor
> (
> vendor_id INTEGER,
> PRIMARY KEY (vendor_id)
> );
> {code}
> In the above syntax, [CONSTRAINT constraint-Name] is optional. If this is not
> specified by the user, we will use system generated constraint name. For the
> purpose of simplicity, we will allow CONSTRAINT option for foreign keys and
> not primary key since there is only one primary key per table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13130) API calls for retrieving primary keys and foreign keys information
[ https://issues.apache.org/jira/browse/HIVE-13130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sankar Sivarama Subramaniyan updated HIVE-13130: - Issue Type: Task (was: Bug) > API calls for retrieving primary keys and foreign keys information > --- > > Key: HIVE-13130 > URL: https://issues.apache.org/jira/browse/HIVE-13130 > Project: Hive > Issue Type: Task >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Hari Sankar Sivarama Subramaniyan > Attachments: HIVE-13130.1.patch, HIVE-13130.2.patch, > HIVE-13130.3.patch > > > ODBC exposes the SQLPrimaryKeys and SQLForeignKeys API calls and JDBC exposes > getPrimaryKeys and getCrossReference API calls. We need to provide these > interfaces as part of PK/FK implementation in Hive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13130) API calls for retrieving primary keys and foreign keys information
[ https://issues.apache.org/jira/browse/HIVE-13130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sankar Sivarama Subramaniyan updated HIVE-13130: - Issue Type: Sub-task (was: Task) Parent: HIVE-13076 > API calls for retrieving primary keys and foreign keys information > --- > > Key: HIVE-13130 > URL: https://issues.apache.org/jira/browse/HIVE-13130 > Project: Hive > Issue Type: Sub-task >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Hari Sankar Sivarama Subramaniyan > Attachments: HIVE-13130.1.patch, HIVE-13130.2.patch, > HIVE-13130.3.patch > > > ODBC exposes the SQLPrimaryKeys and SQLForeignKeys API calls and JDBC exposes > getPrimaryKeys and getCrossReference API calls. We need to provide these > interfaces as part of PK/FK implementation in Hive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13130) HS2 changes : API calls for retrieving primary keys and foreign keys information
[ https://issues.apache.org/jira/browse/HIVE-13130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sankar Sivarama Subramaniyan updated HIVE-13130: - Summary: HS2 changes : API calls for retrieving primary keys and foreign keys information (was: API calls for retrieving primary keys and foreign keys information) > HS2 changes : API calls for retrieving primary keys and foreign keys > information > - > > Key: HIVE-13130 > URL: https://issues.apache.org/jira/browse/HIVE-13130 > Project: Hive > Issue Type: Sub-task >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Hari Sankar Sivarama Subramaniyan > Attachments: HIVE-13130.1.patch, HIVE-13130.2.patch, > HIVE-13130.3.patch > > > ODBC exposes the SQLPrimaryKeys and SQLForeignKeys API calls and JDBC exposes > getPrimaryKeys and getCrossReference API calls. We need to provide these > interfaces as part of PK/FK implementation in Hive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-13349) Metastore Changes : HS2 changes : API calls for retrieving primary keys and foreign keys information
[ https://issues.apache.org/jira/browse/HIVE-13349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sankar Sivarama Subramaniyan reassigned HIVE-13349: Assignee: Hari Sankar Sivarama Subramaniyan > Metastore Changes : HS2 changes : API calls for retrieving primary keys and > foreign keys information > > > Key: HIVE-13349 > URL: https://issues.apache.org/jira/browse/HIVE-13349 > Project: Hive > Issue Type: Sub-task > Components: CBO, Logical Optimizer >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Hari Sankar Sivarama Subramaniyan > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HIVE-13085) Need an API / configuration parameter to find out the authenticated user from beeline
[ https://issues.apache.org/jira/browse/HIVE-13085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209893#comment-15209893 ] Thejas M Nair edited comment on HIVE-13085 at 3/24/16 7:13 AM: --- Thanks working on this and contributing to hive [~osayankin] ! I have one concern with this patch, it does not seem like it would work with kerberos mode. I think the feature ought to work consistently across all authentication modes. Also note that we already have a 'select username()' feature available via HIVE-9143. was (Author: thejas): Would this work in kerberos mode ? (I am not sure if it would). I think the feature ought to work consistently across all authentication modes. We already have a 'select username()' feature available via HIVE-9143. > Need an API / configuration parameter to find out the authenticated user > from beeline > -- > > Key: HIVE-13085 > URL: https://issues.apache.org/jira/browse/HIVE-13085 > Project: Hive > Issue Type: Improvement >Reporter: Oleksiy Sayankin >Assignee: Oleksiy Sayankin > Fix For: 2.0.0 > > Attachments: HIVE-13085.1.patch > > > This request has come from the customer who wanted an API / configuration > parameter to identify the authenticated user from the beeline. This is > similar to the request done in the thread below: > https://community.hortonworks.com/questions/2620/hadoop-environment-variable-or-configuration-varia.html > But this would not be a feasible option for the requestor. So here the > general ask is once the user is logged in to the beeline, they wanted to > identify who this user is and then use this information to enforce the ACLs > on the tables through the customer's custom code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13350) Support Alter commands for Rely/NoRely novalidate for PK/FK constraints
[ https://issues.apache.org/jira/browse/HIVE-13350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sankar Sivarama Subramaniyan updated HIVE-13350: - Summary: Support Alter commands for Rely/NoRely novalidate for PK/FK constraints (was: Support Alter commands for Rely/NoRely novalidate) > Support Alter commands for Rely/NoRely novalidate for PK/FK constraints > > > Key: HIVE-13350 > URL: https://issues.apache.org/jira/browse/HIVE-13350 > Project: Hive > Issue Type: Sub-task > Components: CBO, Logical Optimizer >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Hari Sankar Sivarama Subramaniyan > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-13351) Support drop Primary Key/Foreign Key constraints
[ https://issues.apache.org/jira/browse/HIVE-13351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sankar Sivarama Subramaniyan reassigned HIVE-13351: Assignee: Hari Sankar Sivarama Subramaniyan > Support drop Primary Key/Foreign Key constraints > > > Key: HIVE-13351 > URL: https://issues.apache.org/jira/browse/HIVE-13351 > Project: Hive > Issue Type: Sub-task > Components: CBO, Logical Optimizer >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Hari Sankar Sivarama Subramaniyan > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13290) Support primary keys/foreign keys constraint as part of create table command in Hive
[
https://issues.apache.org/jira/browse/HIVE-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209904#comment-15209904
]
Hive QA commented on HIVE-13290:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12795175/HIVE-13290.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 4 tests passed
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-METASTORE-Test/129/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-METASTORE-Test/129/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-METASTORE-Test-129/
Messages:
{noformat}
LXC derby found.
LXC derby is not started. Starting container...
Container started.
Preparing derby container...
Container prepared.
Calling /hive/testutils/metastore/dbs/derby/prepare.sh ...
Server prepared.
Calling /hive/testutils/metastore/dbs/derby/execute.sh ...
Tests executed.
LXC mysql found.
LXC mysql is not started. Starting container...
Container started.
Preparing mysql container...
Container prepared.
Calling /hive/testutils/metastore/dbs/mysql/prepare.sh ...
Server prepared.
Calling /hive/testutils/metastore/dbs/mysql/execute.sh ...
Tests executed.
LXC oracle found.
LXC oracle is not started. Starting container...
Container started.
Preparing oracle container...
Container prepared.
Calling /hive/testutils/metastore/dbs/oracle/prepare.sh ...
Server prepared.
Calling /hive/testutils/metastore/dbs/oracle/execute.sh ...
Tests executed.
LXC postgres found.
LXC postgres is not started. Starting container...
Container started.
Preparing postgres container...
Container prepared.
Calling /hive/testutils/metastore/dbs/postgres/prepare.sh ...
Server prepared.
Calling /hive/testutils/metastore/dbs/postgres/execute.sh ...
Tests executed.
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12795175 - PreCommit-HIVE-METASTORE-Test
> Support primary keys/foreign keys constraint as part of create table command
> in Hive
>
>
> Key: HIVE-13290
> URL: https://issues.apache.org/jira/browse/HIVE-13290
> Project: Hive
> Issue Type: Sub-task
> Components: CBO, Logical Optimizer
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Attachments: HIVE-13290.1.patch, HIVE-13290.2.patch
>
>
> SUPPORT for the following statements
> {code}
> CREATE TABLE product
> (
> product_idINTEGER,
> product_vendor_id INTEGER,
> PRIMARY KEY (product_id),
> CONSTRAINT product_fk_1 FOREIGN KEY (product_vendor_id) REFERENCES
> vendor(vendor_id)
> );
> CREATE TABLE vendor
> (
> vendor_id INTEGER,
> PRIMARY KEY (vendor_id)
> );
> {code}
> In the above syntax, [CONSTRAINT constraint-Name] is optional. If this is not
> specified by the user, we will use system generated constraint name. For the
> purpose of simplicity, we will allow CONSTRAINT option for foreign keys and
> not primary key since there is only one primary key per table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13111) Fix timestamp / interval_day_time wrong results with HIVE-9862
[ https://issues.apache.org/jira/browse/HIVE-13111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-13111: Status: Patch Available (was: In Progress) > Fix timestamp / interval_day_time wrong results with HIVE-9862 > --- > > Key: HIVE-13111 > URL: https://issues.apache.org/jira/browse/HIVE-13111 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-13111.01.patch, HIVE-13111.02.patch, > HIVE-13111.03.patch, HIVE-13111.04.patch, HIVE-13111.05.patch, > HIVE-13111.06.patch > > > Fix timestamp / interval_day_time issues discovered when testing the > Vectorized Text patch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13111) Fix timestamp / interval_day_time wrong results with HIVE-9862
[ https://issues.apache.org/jira/browse/HIVE-13111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-13111: Status: In Progress (was: Patch Available) > Fix timestamp / interval_day_time wrong results with HIVE-9862 > --- > > Key: HIVE-13111 > URL: https://issues.apache.org/jira/browse/HIVE-13111 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-13111.01.patch, HIVE-13111.02.patch, > HIVE-13111.03.patch, HIVE-13111.04.patch, HIVE-13111.05.patch, > HIVE-13111.06.patch > > > Fix timestamp / interval_day_time issues discovered when testing the > Vectorized Text patch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13217) Replication for HoS mapjoin small file needs to respect dfs.replication.max
[ https://issues.apache.org/jira/browse/HIVE-13217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam updated HIVE-13217: Resolution: Fixed Fix Version/s: 2.0.0 Status: Resolved (was: Patch Available) Committed to master, thanks for the review Szehon Ho > Replication for HoS mapjoin small file needs to respect dfs.replication.max > --- > > Key: HIVE-13217 > URL: https://issues.apache.org/jira/browse/HIVE-13217 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 1.2.1, 2.0.0 >Reporter: Szehon Ho >Assignee: Chinna Rao Lalam >Priority: Minor > Fix For: 2.0.0 > > Attachments: HIVE-13217.1.patch, HIVE-13217.2.patch > > > Currently Hive on Spark Mapjoin replicates small table file to a hard-coded > value of 10. See SparkHashTableSinkOperator.MIN_REPLICATION. > When dfs.replication.max is less than 10, HoS query fails. This constant > should cap at dfs.replication.max. > Normally dfs.replication.max seems set at 512. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13325) Excessive logging when ORC PPD fails type conversions
[
https://issues.apache.org/jira/browse/HIVE-13325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209968#comment-15209968
]
Hive QA commented on HIVE-13325:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12795040/HIVE-13325.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 22 failed/errored test(s), 9855 tests
executed
*Failed tests:*
{noformat}
TestSparkCliDriver-groupby3_map.q-sample2.q-auto_join14.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-groupby_map_ppr_multi_distinct.q-table_access_keys_stats.q-groupby4_noskew.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-join_rc.q-insert1.q-vectorized_rcfile_columnar.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-ppd_join4.q-join9.q-ppd_join3.q-and-12-more - did not
produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llap_udf
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.org.apache.hadoop.hive.cli.TestMiniTezCliDriver
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_auto_join1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_constprog_dpp
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_enforce_order
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_insert_values_tmp_table
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_llapdecider
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge8
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge_incompat3
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_orc_nonvec_fetchwork_table
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_smb_cache
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_stats_filemetadata
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_subquery_exists
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union4
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_4
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_outer_join1
org.apache.hive.jdbc.TestJdbcWithLocalClusterSpark.testTempTable
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7352/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7352/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7352/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 22 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12795040 - PreCommit-HIVE-TRUNK-Build
> Excessive logging when ORC PPD fails type conversions
> -
>
> Key: HIVE-13325
> URL: https://issues.apache.org/jira/browse/HIVE-13325
> Project: Hive
> Issue Type: Bug
> Components: Logging, ORC
>Affects Versions: 1.3.0, 2.1.0
>Reporter: Siddharth Seth
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-13325.1.patch, HIVE-13325.2.patch
>
>
> Timestamp was specified as "-MM-DD HH:MM:SS": 2016-01-23 00:00:00
> {code}
> 2016-02-10 02:15:43,175 [WARN] [TezChild] |orc.RecordReaderImpl|: Exception
> when evaluating predicate. Skipping ORC PPD. Exception:
> java.lang.IllegalArgumentException: ORC SARGS could not convert from String
> to TIMESTAMP
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.getBaseObjectForComparison(RecordReaderImpl.java:659)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.evaluatePredicateRange(RecordReaderImpl.java:373)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.evaluatePredicateProto(RecordReaderImpl.java:338)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$SargApplier.pickRowGroups(RecordReaderImpl.java:710)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.pickRowGroups(RecordReaderImpl.java:751)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.readStripe(RecordReaderImpl.java:777)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceStripe(RecordReaderImpl.java:986)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceToNextRow(RecordReaderImpl.java:1019)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.(RecordR
[jira] [Commented] (HIVE-12616) NullPointerException when spark session is reused to run a mapjoin
[
https://issues.apache.org/jira/browse/HIVE-12616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210116#comment-15210116
]
Hive QA commented on HIVE-12616:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12794821/HIVE-12616.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 9856 tests
executed
*Failed tests:*
{noformat}
TestSparkCliDriver-groupby3_map.q-sample2.q-auto_join14.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-groupby_map_ppr_multi_distinct.q-table_access_keys_stats.q-groupby4_noskew.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-join_rc.q-insert1.q-vectorized_rcfile_columnar.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-ppd_join4.q-join9.q-ppd_join3.q-and-12-more - did not
produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llap_udf
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager.concurrencyFalse
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager.testDDLExclusive
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager.testDelete
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager.testLockTimeout
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager.testRollback
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager.testSingleReadPartition
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager.testSingleWriteTable
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager.testUpdate
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager.testWriteDynamicPartition
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7354/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7354/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7354/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12794821 - PreCommit-HIVE-TRUNK-Build
> NullPointerException when spark session is reused to run a mapjoin
> --
>
> Key: HIVE-12616
> URL: https://issues.apache.org/jira/browse/HIVE-12616
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 1.3.0
>Reporter: Nemon Lou
>Assignee: Nemon Lou
> Attachments: HIVE-12616.1.patch, HIVE-12616.2.patch,
> HIVE-12616.3.patch, HIVE-12616.patch
>
>
> The way to reproduce:
> {noformat}
> set hive.execution.engine=spark;
> create table if not exists test(id int);
> create table if not exists test1(id int);
> insert into test values(1);
> insert into test1 values(1);
> select max(a.id) from test a ,test1 b
> where a.id = b.id;
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13027) Configuration changes to improve logging performance
[
https://issues.apache.org/jira/browse/HIVE-13027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210333#comment-15210333
]
Hive QA commented on HIVE-13027:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12794710/HIVE-13027.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 9857 tests
executed
*Failed tests:*
{noformat}
TestSparkCliDriver-groupby3_map.q-sample2.q-auto_join14.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-groupby_map_ppr_multi_distinct.q-table_access_keys_stats.q-groupby4_noskew.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-join_rc.q-insert1.q-vectorized_rcfile_columnar.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-ppd_join4.q-join9.q-ppd_join3.q-and-12-more - did not
produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriverMethods.testProcessSelectDatabase
org.apache.hadoop.hive.cli.TestCliDriverMethods.testQuit
org.apache.hadoop.hive.cli.TestCliDriverMethods.testRun
org.apache.hadoop.hive.cli.TestCliDriverMethods.testThatCliDriverPrintsHeaderForCommandsWithSchema
org.apache.hadoop.hive.cli.TestCliDriverMethods.testThatCliDriverPrintsNoHeaderForCommandsWithNoSchema
org.apache.hadoop.hive.cli.TestCliDriverMethods.testprocessInitFiles
org.apache.hadoop.hive.cli.TestContribNegativeCliDriver.testNegativeCliDriver_case_with_row_sequence
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llap_udf
org.apache.hadoop.hive.metastore.txn.TestCompactionTxnHandler.testRevokeTimedOutWorkers
org.apache.hadoop.hive.ql.exec.TestExecDriver.testMapPlan1
org.apache.hadoop.hive.ql.exec.TestExecDriver.testMapPlan2
org.apache.hadoop.hive.ql.exec.TestExecDriver.testMapRedPlan1
org.apache.hadoop.hive.ql.exec.TestExecDriver.testMapRedPlan2
org.apache.hadoop.hive.ql.exec.TestExecDriver.testMapRedPlan3
org.apache.hadoop.hive.ql.exec.TestExecDriver.testMapRedPlan4
org.apache.hadoop.hive.ql.exec.TestExecDriver.testMapRedPlan5
org.apache.hadoop.hive.ql.exec.TestExecDriver.testMapRedPlan6
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager.concurrencyFalse
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager.testLockTimeout
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager.testUpdate
org.apache.hive.hcatalog.cli.TestPermsGrp.testCustomPerms
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7355/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7355/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7355/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 25 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12794710 - PreCommit-HIVE-TRUNK-Build
> Configuration changes to improve logging performance
>
>
> Key: HIVE-13027
> URL: https://issues.apache.org/jira/browse/HIVE-13027
> Project: Hive
> Issue Type: Improvement
> Components: Logging
>Affects Versions: 2.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-13027.1.patch, HIVE-13027.2.patch
>
>
> For LLAP and HS2, some configuration changes can be made to improve logging
> performance
> 1) LOG4j2's async logger claims to have 6-68 times better performance than
> synchronous logger. https://logging.apache.org/log4j/2.x/manual/async.html
> 2) Replace File appenders with RandomAccessFileAppender that claims to be
> 20-200% more performant.
> https://logging.apache.org/log4j/2.x/manual/appenders.html#RandomAccessFileAppender
> Also make async logging configurable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13343) Need to disable hybrid grace hash join in llap mode except for dynamically partitioned hash join
[ https://issues.apache.org/jira/browse/HIVE-13343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210493#comment-15210493 ] Siddharth Seth commented on HIVE-13343: --- Can we please add an override for this. There's a parallel jira which will cause regular map joins to fail in LLAP if they exceed memory limits. In that case, the user should at least be able to run the query in LLAP - even if it is with degraded performance. > Need to disable hybrid grace hash join in llap mode except for dynamically > partitioned hash join > > > Key: HIVE-13343 > URL: https://issues.apache.org/jira/browse/HIVE-13343 > Project: Hive > Issue Type: Bug > Components: llap >Affects Versions: 2.1.0 >Reporter: Vikram Dixit K >Assignee: Vikram Dixit K > Attachments: HIVE-13343.1.patch > > > Due to performance reasons, we should disable use of hybrid grace hash join > in llap when dynamic partition hash join is not used. With dynamic partition > hash join, we need hybrid grace hash join due to the possibility of skews. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13344) port HIVE-12902 to 1.x line
[ https://issues.apache.org/jira/browse/HIVE-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-13344: -- Description: w/o this it makes it difficult to make matching checkins into 2.x and 1.x line (was: w/o this it makes it difficult to make checkins into 2.x and 1.x line) > port HIVE-12902 to 1.x line > --- > > Key: HIVE-13344 > URL: https://issues.apache.org/jira/browse/HIVE-13344 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.3.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-12902.branch-1.patch > > > w/o this it makes it difficult to make matching checkins into 2.x and 1.x line -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13295) Improvement to LDAP search queries in HS2 LDAP Authenticator
[ https://issues.apache.org/jira/browse/HIVE-13295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang updated HIVE-13295: --- Resolution: Fixed Fix Version/s: 2.1.0 Status: Resolved (was: Patch Available) Committed to 2.1.0. Thanks [~ngangam] for the patch. I think you may need to document the new LDAP configuration properties in wiki. > Improvement to LDAP search queries in HS2 LDAP Authenticator > > > Key: HIVE-13295 > URL: https://issues.apache.org/jira/browse/HIVE-13295 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 1.3.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam > Fix For: 2.1.0 > > Attachments: HIVE-13295.1.patch, HIVE-13295.2.patch > > > As more usecases, for various LDAP flavors and deployments, emerge, Hive's > LDAP authentication provider needs additional configuration properties to > make it more flexible to work with different LDAP deployments. > For example: > 1) Not every LDAP server supports a "memberOf" property on user entries that > refer to the groups the user belongs to. This attribute is used for group > filter support. So instead of relying on this attribute to be set, we can > reverse the search and find all the groups that have an attribute, that > refers to its members, set. For example "member" or "memberUid" etc. > Since this atttribute name differs from ldap to ldap, its best we make this > configurable, with a default value of "member" > 2) In HIVE-12885, a new property was introduced to make the attribute for an > user/group search key user-configurable instead of assuming its "uid" (when > baseDN is set) or "cn" (otherwise). This change was deferred from the initial > patch. > 3) LDAP Groups can have various ObjectClass'es. For example objectClass=group > or objectClass=groupOfNames or objectClass=posixGroup or > objectClass=groupOfUniqueNames etc. There could be other we dont know of. > So we need a property to make this user-configurable with a certain default. > 4) There is also a bug where the lists for groupFilter and userFilter are not > re-initialized each time init() is called. > These lists are only re-initialized if the new HiveConf has userFilter or > groupFilter set values. Otherwise, the provider will use values from previous > initialization. > I found this bug when writing some new tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13295) Improvement to LDAP search queries in HS2 LDAP Authenticator
[ https://issues.apache.org/jira/browse/HIVE-13295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210522#comment-15210522 ] Naveen Gangam commented on HIVE-13295: -- Thanks [~ctang.ma] I will add these properties to the wiki. > Improvement to LDAP search queries in HS2 LDAP Authenticator > > > Key: HIVE-13295 > URL: https://issues.apache.org/jira/browse/HIVE-13295 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 1.3.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam > Fix For: 2.1.0 > > Attachments: HIVE-13295.1.patch, HIVE-13295.2.patch > > > As more usecases, for various LDAP flavors and deployments, emerge, Hive's > LDAP authentication provider needs additional configuration properties to > make it more flexible to work with different LDAP deployments. > For example: > 1) Not every LDAP server supports a "memberOf" property on user entries that > refer to the groups the user belongs to. This attribute is used for group > filter support. So instead of relying on this attribute to be set, we can > reverse the search and find all the groups that have an attribute, that > refers to its members, set. For example "member" or "memberUid" etc. > Since this atttribute name differs from ldap to ldap, its best we make this > configurable, with a default value of "member" > 2) In HIVE-12885, a new property was introduced to make the attribute for an > user/group search key user-configurable instead of assuming its "uid" (when > baseDN is set) or "cn" (otherwise). This change was deferred from the initial > patch. > 3) LDAP Groups can have various ObjectClass'es. For example objectClass=group > or objectClass=groupOfNames or objectClass=posixGroup or > objectClass=groupOfUniqueNames etc. There could be other we dont know of. > So we need a property to make this user-configurable with a certain default. > 4) There is also a bug where the lists for groupFilter and userFilter are not > re-initialized each time init() is called. > These lists are only re-initialized if the new HiveConf has userFilter or > groupFilter set values. Otherwise, the provider will use values from previous > initialization. > I found this bug when writing some new tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13353) SHOW COMPACTIONS should support filtering options
[ https://issues.apache.org/jira/browse/HIVE-13353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210559#comment-15210559 ] Eugene Koifman commented on HIVE-13353: --- now that HIVE-11994 & HIVE-12353 produce history info as part of Show Compactions there is enough data there to make filtering options necessary > SHOW COMPACTIONS should support filtering options > - > > Key: HIVE-13353 > URL: https://issues.apache.org/jira/browse/HIVE-13353 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.3.0, 2.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > > This command should support ability to filter by db/table/partition. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HIVE-13281) Update some default configs for LLAP
[ https://issues.apache.org/jira/browse/HIVE-13281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth reopened HIVE-13281: --- Assignee: Siddharth Seth > Update some default configs for LLAP > > > Key: HIVE-13281 > URL: https://issues.apache.org/jira/browse/HIVE-13281 > Project: Hive > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > > Disable uber mode. > Enable llap.io by default -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13281) Update some default configs for LLAP
[ https://issues.apache.org/jira/browse/HIVE-13281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-13281: -- Status: Patch Available (was: Reopened) > Update some default configs for LLAP > > > Key: HIVE-13281 > URL: https://issues.apache.org/jira/browse/HIVE-13281 > Project: Hive > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-13281.1.patch > > > Disable uber mode. > Enable llap.io by default -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13281) Update some default configs for LLAP
[ https://issues.apache.org/jira/browse/HIVE-13281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-13281: -- Attachment: HIVE-13281.1.patch Turns off uber by default, and enables the lrfu cache. [~sershe] - please review. Submitting to jenkins, I believe this will end up causing a bunch of failures in the explain output related to uber. > Update some default configs for LLAP > > > Key: HIVE-13281 > URL: https://issues.apache.org/jira/browse/HIVE-13281 > Project: Hive > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-13281.1.patch > > > Disable uber mode. > Enable llap.io by default -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11424) Rule to transform OR clauses into IN clauses in CBO
[
https://issues.apache.org/jira/browse/HIVE-11424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210642#comment-15210642
]
Hive QA commented on HIVE-11424:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12794941/HIVE-11424.05.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 9858 tests
executed
*Failed tests:*
{noformat}
TestSparkCliDriver-groupby3_map.q-sample2.q-auto_join14.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-groupby_map_ppr_multi_distinct.q-table_access_keys_stats.q-groupby4_noskew.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-join_rc.q-insert1.q-vectorized_rcfile_columnar.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-ppd_join4.q-join9.q-ppd_join3.q-and-12-more - did not
produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constprog_semijoin
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dynpart_sort_optimization_acid
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_mr_diff_schema_alias
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llap_udf
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_semijoin
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_bucketpruning1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_constprog_semijoin
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_mr_diff_schema_alias
org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query13
org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query27
org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query34
org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query48
org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query68
org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query73
org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query79
org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query82
org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query85
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7356/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7356/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7356/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 21 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12794941 - PreCommit-HIVE-TRUNK-Build
> Rule to transform OR clauses into IN clauses in CBO
> ---
>
> Key: HIVE-11424
> URL: https://issues.apache.org/jira/browse/HIVE-11424
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-11424.01.patch, HIVE-11424.01.patch,
> HIVE-11424.03.patch, HIVE-11424.03.patch, HIVE-11424.04.patch,
> HIVE-11424.05.patch, HIVE-11424.2.patch, HIVE-11424.patch
>
>
> We create a rule that will transform OR clauses into IN clauses (when
> possible).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13344) port HIVE-12902 to 1.x line
[ https://issues.apache.org/jira/browse/HIVE-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-13344: -- Attachment: HIVE-13344.patch > port HIVE-12902 to 1.x line > --- > > Key: HIVE-13344 > URL: https://issues.apache.org/jira/browse/HIVE-13344 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.3.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-12902.branch-1.patch, HIVE-13344.patch > > > w/o this it makes it difficult to make matching checkins into 2.x and 1.x line -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13300) Hive on spark throws exception for multi-insert with join
[
https://issues.apache.org/jira/browse/HIVE-13300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szehon Ho updated HIVE-13300:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed to master, thanks Chao and Xuefu for review.
Created HIVE-13355 to track unrelated SparkCliDriver test timeouts.
> Hive on spark throws exception for multi-insert with join
> -
>
> Key: HIVE-13300
> URL: https://issues.apache.org/jira/browse/HIVE-13300
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 2.0.0
>Reporter: Szehon Ho
>Assignee: Szehon Ho
> Attachments: HIVE-13300.2.patch, HIVE-13300.3.patch, HIVE-13300.patch
>
>
> For certain multi-insert queries, Hive on Spark throws a deserialization
> error.
> {noformat}
> create table status_updates(userid int,status string,ds string);
> create table profiles(userid int,school string,gender int);
> drop table school_summary; create table school_summary(school string,cnt int)
> partitioned by (ds string);
> drop table gender_summary; create table gender_summary(gender int,cnt int)
> partitioned by (ds string);
> insert into status_updates values (1, "status_1", "2016-03-16");
> insert into profiles values (1, "school_1", 0);
> set hive.auto.convert.join=false;
> set hive.execution.engine=spark;
> FROM (SELECT a.status, b.school, b.gender
> FROM status_updates a JOIN profiles b
> ON (a.userid = b.userid and
> a.ds='2009-03-20' )
> ) subq1
> INSERT OVERWRITE TABLE gender_summary
> PARTITION(ds='2009-03-20')
> SELECT subq1.gender, COUNT(1) GROUP BY subq1.gender
> INSERT OVERWRITE TABLE school_summary
> PARTITION(ds='2009-03-20')
> SELECT subq1.school, COUNT(1) GROUP BY subq1.school
> {noformat}
> Error:
> {noformat}
> 16/03/17 13:29:00 [task-result-getter-3]: WARN scheduler.TaskSetManager: Lost
> task 0.0 in stage 2.0 (TID 3, localhost): java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error: Unable
> to deserialize reduce input key from x1x128x0x0 with properties
> {serialization.sort.order.null=a, columns=reducesinkkey0,
> serialization.lib=org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe,
> serialization.sort.order=+, columns.types=int}
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkReduceRecordHandler.processRow(SparkReduceRecordHandler.java:279)
> at
> org.apache.hadoop.hive.ql.exec.spark.HiveReduceFunctionResultList.processNextRecord(HiveReduceFunctionResultList.java:49)
> at
> org.apache.hadoop.hive.ql.exec.spark.HiveReduceFunctionResultList.processNextRecord(HiveReduceFunctionResultList.java:28)
> at
> org.apache.hadoop.hive.ql.exec.spark.HiveBaseFunctionResultList$ResultIterator.hasNext(HiveBaseFunctionResultList.java:95)
> at
> scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:41)
> at
> org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error: Unable to deserialize reduce input key from x1x128x0x0 with properties
> {serialization.sort.order.null=a, columns=reducesinkkey0,
> serialization.lib=org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe,
> serialization.sort.order=+, columns.types=int}
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkReduceRecordHandler.processRow(SparkReduceRecordHandler.java:251)
> ... 12 more
> Caused by: org.apache.hadoop.hive.serde2.SerDeException: java.io.EOFException
> at
> org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.deserialize(BinarySortableSerDe.java:241)
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkReduceRecordHandler.processRow(SparkReduceRecordHandler.java:249)
> ... 12 more
> Caused by: java.io.EOFException
> at
> org.apache.hadoop.hive.serde2.binarysortable.InputByteBuffer.read(InputByteBuffer.java:54)
> at
> org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.deserializeInt(BinarySortableSerDe.java:597)
> at
> org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.deserialize(BinarySortableSerDe.java:288)
> at
> org.apache.hadoop.hive.serd
[jira] [Updated] (HIVE-12616) NullPointerException when spark session is reused to run a mapjoin
[
https://issues.apache.org/jira/browse/HIVE-12616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szehon Ho updated HIVE-12616:
-
Resolution: Fixed
Fix Version/s: 2.1.0
Status: Resolved (was: Patch Available)
Committed to master, thanks Nemon for contribution and Xuefu for the review.
> NullPointerException when spark session is reused to run a mapjoin
> --
>
> Key: HIVE-12616
> URL: https://issues.apache.org/jira/browse/HIVE-12616
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 1.3.0
>Reporter: Nemon Lou
>Assignee: Nemon Lou
> Fix For: 2.1.0
>
> Attachments: HIVE-12616.1.patch, HIVE-12616.2.patch,
> HIVE-12616.3.patch, HIVE-12616.patch
>
>
> The way to reproduce:
> {noformat}
> set hive.execution.engine=spark;
> create table if not exists test(id int);
> create table if not exists test1(id int);
> insert into test values(1);
> insert into test1 values(1);
> select max(a.id) from test a ,test1 b
> where a.id = b.id;
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13300) Hive on spark throws exception for multi-insert with join
[
https://issues.apache.org/jira/browse/HIVE-13300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szehon Ho updated HIVE-13300:
-
Fix Version/s: 2.1.0
> Hive on spark throws exception for multi-insert with join
> -
>
> Key: HIVE-13300
> URL: https://issues.apache.org/jira/browse/HIVE-13300
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 2.0.0
>Reporter: Szehon Ho
>Assignee: Szehon Ho
> Fix For: 2.1.0
>
> Attachments: HIVE-13300.2.patch, HIVE-13300.3.patch, HIVE-13300.patch
>
>
> For certain multi-insert queries, Hive on Spark throws a deserialization
> error.
> {noformat}
> create table status_updates(userid int,status string,ds string);
> create table profiles(userid int,school string,gender int);
> drop table school_summary; create table school_summary(school string,cnt int)
> partitioned by (ds string);
> drop table gender_summary; create table gender_summary(gender int,cnt int)
> partitioned by (ds string);
> insert into status_updates values (1, "status_1", "2016-03-16");
> insert into profiles values (1, "school_1", 0);
> set hive.auto.convert.join=false;
> set hive.execution.engine=spark;
> FROM (SELECT a.status, b.school, b.gender
> FROM status_updates a JOIN profiles b
> ON (a.userid = b.userid and
> a.ds='2009-03-20' )
> ) subq1
> INSERT OVERWRITE TABLE gender_summary
> PARTITION(ds='2009-03-20')
> SELECT subq1.gender, COUNT(1) GROUP BY subq1.gender
> INSERT OVERWRITE TABLE school_summary
> PARTITION(ds='2009-03-20')
> SELECT subq1.school, COUNT(1) GROUP BY subq1.school
> {noformat}
> Error:
> {noformat}
> 16/03/17 13:29:00 [task-result-getter-3]: WARN scheduler.TaskSetManager: Lost
> task 0.0 in stage 2.0 (TID 3, localhost): java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error: Unable
> to deserialize reduce input key from x1x128x0x0 with properties
> {serialization.sort.order.null=a, columns=reducesinkkey0,
> serialization.lib=org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe,
> serialization.sort.order=+, columns.types=int}
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkReduceRecordHandler.processRow(SparkReduceRecordHandler.java:279)
> at
> org.apache.hadoop.hive.ql.exec.spark.HiveReduceFunctionResultList.processNextRecord(HiveReduceFunctionResultList.java:49)
> at
> org.apache.hadoop.hive.ql.exec.spark.HiveReduceFunctionResultList.processNextRecord(HiveReduceFunctionResultList.java:28)
> at
> org.apache.hadoop.hive.ql.exec.spark.HiveBaseFunctionResultList$ResultIterator.hasNext(HiveBaseFunctionResultList.java:95)
> at
> scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:41)
> at
> org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error: Unable to deserialize reduce input key from x1x128x0x0 with properties
> {serialization.sort.order.null=a, columns=reducesinkkey0,
> serialization.lib=org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe,
> serialization.sort.order=+, columns.types=int}
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkReduceRecordHandler.processRow(SparkReduceRecordHandler.java:251)
> ... 12 more
> Caused by: org.apache.hadoop.hive.serde2.SerDeException: java.io.EOFException
> at
> org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.deserialize(BinarySortableSerDe.java:241)
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkReduceRecordHandler.processRow(SparkReduceRecordHandler.java:249)
> ... 12 more
> Caused by: java.io.EOFException
> at
> org.apache.hadoop.hive.serde2.binarysortable.InputByteBuffer.read(InputByteBuffer.java:54)
> at
> org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.deserializeInt(BinarySortableSerDe.java:597)
> at
> org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.deserialize(BinarySortableSerDe.java:288)
> at
> org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.deserialize(BinarySortableSerDe.java:237)
> ... 13 more
> {noformat}
--
This message was sent
[jira] [Commented] (HIVE-13344) port HIVE-12902 to 1.x line
[ https://issues.apache.org/jira/browse/HIVE-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210704#comment-15210704 ] Eugene Koifman commented on HIVE-13344: --- HIVe-13344.patch is a branch-1 patch running here http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-BRANCH_1-Build/27/console > port HIVE-12902 to 1.x line > --- > > Key: HIVE-13344 > URL: https://issues.apache.org/jira/browse/HIVE-13344 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.3.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-12902.branch-1.patch, HIVE-13344.patch > > > w/o this it makes it difficult to make matching checkins into 2.x and 1.x line -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13281) Update some default configs for LLAP
[ https://issues.apache.org/jira/browse/HIVE-13281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210708#comment-15210708 ] Sergey Shelukhin commented on HIVE-13281: - Why turn uber off by default? Should we just remove this feature then? It's already off in all mode. > Update some default configs for LLAP > > > Key: HIVE-13281 > URL: https://issues.apache.org/jira/browse/HIVE-13281 > Project: Hive > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-13281.1.patch > > > Disable uber mode. > Enable llap.io by default -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13344) port HIVE-12902 to 1.x line
[ https://issues.apache.org/jira/browse/HIVE-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210711#comment-15210711 ] Wei Zheng commented on HIVE-13344: -- 1. buildQueryWithINClause can be removed from TxnHandler since TxnUtils has it now. 2. Missed changes for TestDbTxnManager2.testDummyTxnManagerOnAcidTable (HIVE-10632 committed slightly different for master and branch-1 due to the absence of TxnStore in branch-1 at that time. Now with this ticket, they should be consistent.) > port HIVE-12902 to 1.x line > --- > > Key: HIVE-13344 > URL: https://issues.apache.org/jira/browse/HIVE-13344 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.3.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-12902.branch-1.patch, HIVE-13344.patch > > > w/o this it makes it difficult to make matching checkins into 2.x and 1.x line -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13281) Update some default configs for LLAP
[ https://issues.apache.org/jira/browse/HIVE-13281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210713#comment-15210713 ] Siddharth Seth commented on HIVE-13281: --- Is it already off in all mode ? It's a half done feature - hence disabling it by default. > Update some default configs for LLAP > > > Key: HIVE-13281 > URL: https://issues.apache.org/jira/browse/HIVE-13281 > Project: Hive > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-13281.1.patch > > > Disable uber mode. > Enable llap.io by default -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13344) port HIVE-12902 to 1.x line
[ https://issues.apache.org/jira/browse/HIVE-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210716#comment-15210716 ] Wei Zheng commented on HIVE-13344: -- createValidReadTxnList can also be removed from TxnHandler > port HIVE-12902 to 1.x line > --- > > Key: HIVE-13344 > URL: https://issues.apache.org/jira/browse/HIVE-13344 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.3.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-12902.branch-1.patch, HIVE-13344.patch > > > w/o this it makes it difficult to make matching checkins into 2.x and 1.x line -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13344) port HIVE-12902 to 1.x line
[ https://issues.apache.org/jira/browse/HIVE-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210721#comment-15210721 ] Wei Zheng commented on HIVE-13344: -- In TestTxnHandler.testBuildQueryWithINClause() and TxnHandler.timeOutLocks(), TxnHandler.buildQueryWithINClause should be replaced with TxnUtils.buildQueryWithINClause > port HIVE-12902 to 1.x line > --- > > Key: HIVE-13344 > URL: https://issues.apache.org/jira/browse/HIVE-13344 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.3.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-12902.branch-1.patch, HIVE-13344.patch > > > w/o this it makes it difficult to make matching checkins into 2.x and 1.x line -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13355) SparkCliDriver tests frequently timeout
[ https://issues.apache.org/jira/browse/HIVE-13355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210767#comment-15210767 ] Szehon Ho commented on HIVE-13355: -- Looking but not sure yet, I'm not seeing anything interesting in logs, for example hive.log in [http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7356/failed/TestSparkCliDriver-groupby3_map.q-sample2.q-auto_join14.q-and-12-more/]. It is just take a long time of upward of 2 hours to run, at which point I believe the test timeout. > SparkCliDriver tests frequently timeout > --- > > Key: HIVE-13355 > URL: https://issues.apache.org/jira/browse/HIVE-13355 > Project: Hive > Issue Type: Bug >Reporter: Szehon Ho > > Something has changed recently to make the SparkCliDriver tests timeout > frequently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13346) LLAP doesn't update metadata priority when reusing from cache; some tweaks in LRFU policy
[ https://issues.apache.org/jira/browse/HIVE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-13346: Status: Patch Available (was: Open) > LLAP doesn't update metadata priority when reusing from cache; some tweaks in > LRFU policy > - > > Key: HIVE-13346 > URL: https://issues.apache.org/jira/browse/HIVE-13346 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13346.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13355) SparkCliDriver tests frequently timeout
[
https://issues.apache.org/jira/browse/HIVE-13355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210771#comment-15210771
]
Szehon Ho commented on HIVE-13355:
--
Spark executor logs show that it finish but somehow stays for about an hour
more before shutting down (killed by timeout in this case).
{noformat}
16/03/24 08:54:10 INFO Executor: Finished task 0.0 in stage 60.0 (TID 76). 1259
bytes result sent to driver
16/03/24 10:50:45 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL 15:
SIGTERM
16/03/24 10:50:45 INFO DiskBlockManager: Shutdown hook called
{noformat}
> SparkCliDriver tests frequently timeout
> ---
>
> Key: HIVE-13355
> URL: https://issues.apache.org/jira/browse/HIVE-13355
> Project: Hive
> Issue Type: Bug
>Reporter: Szehon Ho
>
> Something has changed recently to make the SparkCliDriver tests timeout
> frequently.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13346) LLAP doesn't update metadata priority when reusing from cache; some tweaks in LRFU policy
[ https://issues.apache.org/jira/browse/HIVE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-13346: Attachment: HIVE-13346.01.patch One more tweak - decrease the minimum allocation size. > LLAP doesn't update metadata priority when reusing from cache; some tweaks in > LRFU policy > - > > Key: HIVE-13346 > URL: https://issues.apache.org/jira/browse/HIVE-13346 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13346.01.patch, HIVE-13346.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13325) Excessive logging when ORC PPD fails type conversions
[
https://issues.apache.org/jira/browse/HIVE-13325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-13325:
-
Resolution: Fixed
Fix Version/s: 2.1.0
1.3.0
Status: Resolved (was: Patch Available)
Committed to branch-1 and master. Thanks [~gopalv]!
> Excessive logging when ORC PPD fails type conversions
> -
>
> Key: HIVE-13325
> URL: https://issues.apache.org/jira/browse/HIVE-13325
> Project: Hive
> Issue Type: Bug
> Components: Logging, ORC
>Affects Versions: 1.3.0, 2.1.0
>Reporter: Siddharth Seth
>Assignee: Prasanth Jayachandran
> Fix For: 1.3.0, 2.1.0
>
> Attachments: HIVE-13325.1.patch, HIVE-13325.2.patch
>
>
> Timestamp was specified as "-MM-DD HH:MM:SS": 2016-01-23 00:00:00
> {code}
> 2016-02-10 02:15:43,175 [WARN] [TezChild] |orc.RecordReaderImpl|: Exception
> when evaluating predicate. Skipping ORC PPD. Exception:
> java.lang.IllegalArgumentException: ORC SARGS could not convert from String
> to TIMESTAMP
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.getBaseObjectForComparison(RecordReaderImpl.java:659)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.evaluatePredicateRange(RecordReaderImpl.java:373)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.evaluatePredicateProto(RecordReaderImpl.java:338)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$SargApplier.pickRowGroups(RecordReaderImpl.java:710)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.pickRowGroups(RecordReaderImpl.java:751)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.readStripe(RecordReaderImpl.java:777)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceStripe(RecordReaderImpl.java:986)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceToNextRow(RecordReaderImpl.java:1019)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.(RecordReaderImpl.java:205)
> at
> org.apache.hadoop.hive.ql.io.orc.ReaderImpl.rowsOptions(ReaderImpl.java:598)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger$ReaderPair.(OrcRawRecordMerger.java:183)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger$OriginalReaderPair.(OrcRawRecordMerger.java:226)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.(OrcRawRecordMerger.java:437)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getReader(OrcInputFormat.java:1269)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1151)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:249)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:193)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.(TezGroupedSplitsInputFormat.java:135)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat.getRecordReader(TezGroupedSplitsInputFormat.java:101)
> at
> org.apache.tez.mapreduce.lib.MRReaderMapred.setupOldRecordReader(MRReaderMapred.java:149)
> at
> org.apache.tez.mapreduce.lib.MRReaderMapred.setSplit(MRReaderMapred.java:80)
> at
> org.apache.tez.mapreduce.input.MRInput.initFromEventInternal(MRInput.java:650)
> at
> org.apache.tez.mapreduce.input.MRInput.initFromEvent(MRInput.java:621)
> at
> org.apache.tez.mapreduce.input.MRInputLegacy.checkAndAwaitRecordReaderInitialization(MRInputLegacy.java:145)
> at
> org.apache.tez.mapreduce.input.MRInputLegacy.init(MRInputLegacy.java:109)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.getMRInput(MapRecordProcessor.java:406)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:128)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache
[jira] [Commented] (HIVE-13355) SparkCliDriver tests frequently timeout
[
https://issues.apache.org/jira/browse/HIVE-13355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210809#comment-15210809
]
Szehon Ho commented on HIVE-13355:
--
Ah I guess its related to HIVE-13223. The hanging query has 0 splits.
{noformat}
2016-03-24T08:54:14,419 INFO [stderr-redir-1[]]: client.SparkClientImpl
(SparkClientImpl.java:run(593)) - 16/03/24 08:54:14 INFO
CombineHiveInputFormat: number of splits 0
2016-03-24T08:54:14,419 INFO [stderr-redir-1[]]: client.SparkClientImpl
(SparkClientImpl.java:run(593)) - 16/03/24 08:54:14 INFO
CombineHiveInputFormat: Number of all splits 0
{noformat}
After which point it goes on forever.
> SparkCliDriver tests frequently timeout
> ---
>
> Key: HIVE-13355
> URL: https://issues.apache.org/jira/browse/HIVE-13355
> Project: Hive
> Issue Type: Bug
>Reporter: Szehon Ho
>
> Something has changed recently to make the SparkCliDriver tests timeout
> frequently.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HIVE-13314) Hive on spark mapjoin errors if spark.master is not set
[
https://issues.apache.org/jira/browse/HIVE-13314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szehon Ho resolved HIVE-13314.
--
Resolution: Duplicate
Fix Version/s: 2.1.0
> Hive on spark mapjoin errors if spark.master is not set
> ---
>
> Key: HIVE-13314
> URL: https://issues.apache.org/jira/browse/HIVE-13314
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Reporter: Szehon Ho
>Assignee: Szehon Ho
>Priority: Minor
> Fix For: 2.1.0
>
>
> There are some errors that happen if spark.master is not set.
> This is despite the code defaulting to yarn-cluster if spark.master is not
> set by user or on the config files:
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveSparkClientFactory.java#L51]
> The funny thing is that while it works the first time due to this default,
> subsequent tries will fail as the hiveConf is refreshed without that default
> being set.
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/RemoteHiveSparkClient.java#L180]
> Exception is follows:
> {noformat}
> Job aborted due to stage failure: Task 40 in stage 1.0 failed 4 times, most
> recent failure: Lost task 40.3 in stage 1.0 (TID 22,
> d2409.halxg.cloudera.com): java.lang.RuntimeException: Error processing row:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.processRow(SparkMapRecordHandler.java:154)
> at
> org.apache.hadoop.hive.ql.exec.spark.HiveMapFunctionResultList.processNextRecord(HiveMapFunctionResultList.java:48)
> at
> org.apache.hadoop.hive.ql.exec.spark.HiveMapFunctionResultList.processNextRecord(HiveMapFunctionResultList.java:27)
> at
> org.apache.hadoop.hive.ql.exec.spark.HiveBaseFunctionResultList$ResultIterator.hasNext(HiveBaseFunctionResultList.java:95)
> at
> scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:41)
> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
> at
> org.apache.spark.rdd.AsyncRDDActions$$anonfun$foreachAsync$1$$anonfun$apply$15.apply(AsyncRDDActions.scala:120)
> at
> org.apache.spark.rdd.AsyncRDDActions$$anonfun$foreachAsync$1$$anonfun$apply$15.apply(AsyncRDDActions.scala:120)
> at
> org.apache.spark.SparkContext$$anonfun$38.apply(SparkContext.scala:2003)
> at
> org.apache.spark.SparkContext$$anonfun$38.apply(SparkContext.scala:2003)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.spark.HashTableLoader.load(HashTableLoader.java:117)
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:197)
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.cleanUpInputFileChangedOp(MapJoinOperator.java:223)
> at
> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1051)
> at
> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1055)
> at
> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1055)
> at
> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1055)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:490)
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.processRow(SparkMapRecordHandler.java:141)
> ... 16 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkUtilities.isDedicatedCluster(SparkUtilities.java:108)
> at
> org.apache.hadoop.hive.ql.exec.spark.HashTableLoader.load(HashTableLoader.java:124)
> at
> org.apache.hadoop.hive.ql.exec.spark.HashTableLoader.load(HashTableLoader.java:114)
> ... 24 more
> Driver stacktrace:
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13295) Improvement to LDAP search queries in HS2 LDAP Authenticator
[ https://issues.apache.org/jira/browse/HIVE-13295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210819#comment-15210819 ] Naveen Gangam commented on HIVE-13295: -- Updated the documentation at https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties and https://cwiki.apache.org/confluence/display/Hive/User+and+Group+Filter+Support+with+LDAP+Atn+Provider+in+HiveServer2 > Improvement to LDAP search queries in HS2 LDAP Authenticator > > > Key: HIVE-13295 > URL: https://issues.apache.org/jira/browse/HIVE-13295 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 1.3.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam > Fix For: 2.1.0 > > Attachments: HIVE-13295.1.patch, HIVE-13295.2.patch > > > As more usecases, for various LDAP flavors and deployments, emerge, Hive's > LDAP authentication provider needs additional configuration properties to > make it more flexible to work with different LDAP deployments. > For example: > 1) Not every LDAP server supports a "memberOf" property on user entries that > refer to the groups the user belongs to. This attribute is used for group > filter support. So instead of relying on this attribute to be set, we can > reverse the search and find all the groups that have an attribute, that > refers to its members, set. For example "member" or "memberUid" etc. > Since this atttribute name differs from ldap to ldap, its best we make this > configurable, with a default value of "member" > 2) In HIVE-12885, a new property was introduced to make the attribute for an > user/group search key user-configurable instead of assuming its "uid" (when > baseDN is set) or "cn" (otherwise). This change was deferred from the initial > patch. > 3) LDAP Groups can have various ObjectClass'es. For example objectClass=group > or objectClass=groupOfNames or objectClass=posixGroup or > objectClass=groupOfUniqueNames etc. There could be other we dont know of. > So we need a property to make this user-configurable with a certain default. > 4) There is also a bug where the lists for groupFilter and userFilter are not > re-initialized each time init() is called. > These lists are only re-initialized if the new HiveConf has userFilter or > groupFilter set values. Otherwise, the provider will use values from previous > initialization. > I found this bug when writing some new tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13223) HoS may hang for queries that run on 0 splits
[ https://issues.apache.org/jira/browse/HIVE-13223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210820#comment-15210820 ] Szehon Ho commented on HIVE-13223: -- Ashutosh, Sergey, Is this patch ready to go in? Should have searched for this before debugging the recently hanging tests myself.. > HoS may hang for queries that run on 0 splits > --- > > Key: HIVE-13223 > URL: https://issues.apache.org/jira/browse/HIVE-13223 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 2.1.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-13223.1.patch, HIVE-13223.patch > > > Can be seen on all timed out tests after HIVE-13040 went in -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-13355) SparkCliDriver tests frequently timeout
[ https://issues.apache.org/jira/browse/HIVE-13355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho reassigned HIVE-13355: Assignee: Szehon Ho > SparkCliDriver tests frequently timeout > --- > > Key: HIVE-13355 > URL: https://issues.apache.org/jira/browse/HIVE-13355 > Project: Hive > Issue Type: Bug >Reporter: Szehon Ho >Assignee: Szehon Ho > > Something has changed recently to make the SparkCliDriver tests timeout > frequently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13223) HoS may hang for queries that run on 0 splits
[ https://issues.apache.org/jira/browse/HIVE-13223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-13223: Attachment: HIVE-13223.2.patch REeattaching the patch for HiveQA, looks like it didn't get picked up > HoS may hang for queries that run on 0 splits > --- > > Key: HIVE-13223 > URL: https://issues.apache.org/jira/browse/HIVE-13223 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 2.1.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-13223.1.patch, HIVE-13223.2.patch, HIVE-13223.patch > > > Can be seen on all timed out tests after HIVE-13040 went in -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13223) HoS may hang for queries that run on 0 splits
[ https://issues.apache.org/jira/browse/HIVE-13223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210878#comment-15210878 ] Ashutosh Chauhan commented on HIVE-13223: - This patch is not ready. I think bug is in Spark itself, which is if you submit spark job with 0 splits, spark executors just hang. This got exposed by HIVE-13040 after which we were generating such jobs which was in turn effect of not generating splits for 0-length files. Note, MR & Tez dont have this issue. In this patch, I tried to generate splits even for 0-length files (by not skipping them) but that breaks later at job execution time because ORC reader is not resilient to 0-length file. To fix this issue we need to either figure out and fix Spark hang issue or extend Orc reader to handle 0-length file more gracefully (those failures were exposed in last Hive QA run) > HoS may hang for queries that run on 0 splits > --- > > Key: HIVE-13223 > URL: https://issues.apache.org/jira/browse/HIVE-13223 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 2.1.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-13223.1.patch, HIVE-13223.2.patch, HIVE-13223.patch > > > Can be seen on all timed out tests after HIVE-13040 went in -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13356) ClassCastException: org.apache.hadoop.hive.serde2.io.DateWritable cannot be cast to org.apache.hadoop.io.IntWritable
[
https://issues.apache.org/jira/browse/HIVE-13356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vitalii Diravka updated HIVE-13356:
---
Attachment: fewtypes.parquet
> ClassCastException: org.apache.hadoop.hive.serde2.io.DateWritable cannot be
> cast to org.apache.hadoop.io.IntWritable
>
>
> Key: HIVE-13356
> URL: https://issues.apache.org/jira/browse/HIVE-13356
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.2.1
>Reporter: Vitalii Diravka
> Attachments: fewtypes.parquet
>
>
> Hive query involves cast date to int data type throughs cast exception.
> {code}
> [mapr@cs2none500 bin]$ rpm -qa |grep mapr
> mapr-hadoop-core-2.7.0.32987.GA-1.x86_64
> mapr-fileserver-5.0.0.32987.GA-1.x86_64
> mapr-zookeeper-5.0.0.32987.GA-1.x86_64
> mapr-historyserver-2.7.0.32987.GA-1.x86_64
> mapr-hiveserver2-1.2.201601281422-1.noarch
> mapr-hive-1.2.201601281422-1.noarch
> mapr-hivemetastore-1.2.201601281422-1.noarch
> mapr-core-internal-5.0.0.32987.GA-1.x86_64
> mapr-mapreduce2-2.7.0.32987.GA-1.x86_64
> mapr-core-5.0.0.32987.GA-1.x86_64
> mapr-tasktracker-5.0.0.32987.GA-1.x86_64
> mapr-jobtracker-5.0.0.32987.GA-1.x86_64
> mapr-webserver-5.0.0.32987.GA-1.x86_64
> mapr-nodemanager-2.7.0.32987.GA-1.x86_64
> mapr-zk-internal-5.0.0.32987.GA.v3.4.5-1.x86_64
> mapr-mapreduce1-0.20.2.32987.GA-1.x86_64
> mapr-cldb-5.0.0.32987.GA-1.x86_64
> mapr-resourcemanager-2.7.0.32987.GA-1.x86_64
> mapr-drill-1.4.0.201601071151-1.noarch
> {code}
> Hive schema (parquet file in attachments):
> {code}
> create external table if not exists hive_storage.fewtypes_parquet (
> int_col int,
> bigint_col bigint,
> date_col int,
> time_col int,
> timestamp_col bigint,
> interval_col string,
> varchar_col string,
> float_col float,
> double_col double,
> bool_col boolean
> )
> STORED AS PARQUET
> LOCATION '/tmp/testdata/hive_storage/fewtypes_parquet';
> {code}
> {code}
> hive> use hive_storage;
> OK
> Time taken: 0.035 seconds
> hive> show tables;
> OK
> fewtypes_parquet
> Time taken: 0.034 seconds, Fetched: 1 row(s)
> hive> select * from fewtypes_parquet;
> OK
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
> details.
> Failed with exception
> java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException: org.apache.hadoop.hive.serde2.io.DateWritable
> cannot be cast to org.apache.hadoop.io.IntWritable
> Time taken: 0.41 seconds
> {code}
> If date_col in CTAS is date the query will be executed.
> Hive1.0 hasn't this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13111) Fix timestamp / interval_day_time wrong results with HIVE-9862
[ https://issues.apache.org/jira/browse/HIVE-13111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210899#comment-15210899 ] Jason Dere commented on HIVE-13111: --- +1 pending tests > Fix timestamp / interval_day_time wrong results with HIVE-9862 > --- > > Key: HIVE-13111 > URL: https://issues.apache.org/jira/browse/HIVE-13111 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-13111.01.patch, HIVE-13111.02.patch, > HIVE-13111.03.patch, HIVE-13111.04.patch, HIVE-13111.05.patch, > HIVE-13111.06.patch > > > Fix timestamp / interval_day_time issues discovered when testing the > Vectorized Text patch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11766) LLAP: Remove MiniLlapCluster from shim layer after hadoop-1 removal
[ https://issues.apache.org/jira/browse/HIVE-11766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-11766: Assignee: (was: Sergey Shelukhin) > LLAP: Remove MiniLlapCluster from shim layer after hadoop-1 removal > --- > > Key: HIVE-11766 > URL: https://issues.apache.org/jira/browse/HIVE-11766 > Project: Hive > Issue Type: Sub-task > Components: llap >Reporter: Prasanth Jayachandran > > Remove HIVE-11732 changes after HIVE-11378 goes in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13344) port HIVE-12902 to 1.x line
[ https://issues.apache.org/jira/browse/HIVE-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-13344: -- Attachment: HIVE-13344.branch-1.patch > port HIVE-12902 to 1.x line > --- > > Key: HIVE-13344 > URL: https://issues.apache.org/jira/browse/HIVE-13344 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.3.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-12902.branch-1.patch, HIVE-13344.branch-1.patch, > HIVE-13344.patch > > > w/o this it makes it difficult to make matching checkins into 2.x and 1.x line -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11388) Allow ACID Compactor components to run in multiple metastores
[ https://issues.apache.org/jira/browse/HIVE-11388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210930#comment-15210930 ] Wei Zheng commented on HIVE-11388: -- branch-1 patch looks good. +1 > Allow ACID Compactor components to run in multiple metastores > - > > Key: HIVE-11388 > URL: https://issues.apache.org/jira/browse/HIVE-11388 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Critical > Attachments: HIVE-11388.2.patch, HIVE-11388.4.patch, > HIVE-11388.5.patch, HIVE-11388.6.patch, HIVE-11388.7.patch, > HIVE-11388.branch-1.patch, HIVE-11388.patch > > > (this description is no loner accurate; see further comments) > org.apache.hadoop.hive.ql.txn.compactor.Initiator is a thread that runs > inside the metastore service to manage compactions of ACID tables. There > should be exactly 1 instance of this thread (even with multiple Thrift > services). > This is documented in > https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration > but not enforced. > Should add enforcement, since more than 1 Initiator could cause concurrent > attempts to compact the same table/partition - which will not work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12612) beeline always exits with 0 status when reading query from standard input
[
https://issues.apache.org/jira/browse/HIVE-12612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210941#comment-15210941
]
Hive QA commented on HIVE-12612:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12794805/HIVE-12612.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 9831 tests
executed
*Failed tests:*
{noformat}
TestCliDriver-tez_union_dynamic_partition.q-udf_in_file.q-mapjoin_filter_on_outerjoin.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-groupby3_map.q-sample2.q-auto_join14.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-groupby_map_ppr_multi_distinct.q-table_access_keys_stats.q-groupby4_noskew.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-join_rc.q-insert1.q-vectorized_rcfile_columnar.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-ppd_join4.q-join9.q-ppd_join3.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llap_udf
org.apache.hive.beeline.cli.TestHiveCli.testErrOutput
org.apache.hive.beeline.cli.TestHiveCli.testInValidCmd
org.apache.hive.beeline.cli.TestHiveCli.testSourceCmd3
org.apache.hive.beeline.cli.TestHiveCli.testUseCurrentDB1
org.apache.hive.beeline.cli.TestHiveCli.testUseCurrentDB2
org.apache.hive.beeline.cli.TestHiveCli.testUseCurrentDB3
org.apache.hive.beeline.cli.TestHiveCli.testUseInvalidDB
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7357/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7357/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7357/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12794805 - PreCommit-HIVE-TRUNK-Build
> beeline always exits with 0 status when reading query from standard input
> -
>
> Key: HIVE-12612
> URL: https://issues.apache.org/jira/browse/HIVE-12612
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 1.1.0
> Environment: CDH5.5.0
>Reporter: Paulo Sequeira
>Assignee: Reuben Kuhnert
>Priority: Minor
> Attachments: HIVE-12612.01.patch
>
>
> Similar to what was reported on HIVE-6978, but now it only happens when the
> query is read from the standard input. For example, the following fails as
> expected:
> {code}
> bash$ if beeline -u "jdbc:hive2://..." -e "boo;" ; then echo "Ok?!" ; else
> echo "Failed!" ; fi
> Connecting to jdbc:hive2://...
> Connected to: Apache Hive (version 1.1.0-cdh5.5.0)
> Driver: Hive JDBC (version 1.1.0-cdh5.5.0)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Error: Error while compiling statement: FAILED: ParseException line 1:0
> cannot recognize input near 'boo' '' '' (state=42000,code=4)
> Closing: 0: jdbc:hive2://...
> Failed!
> {code}
> But the following does not:
> {code}
> bash$ if echo "boo;"|beeline -u "jdbc:hive2://..." ; then echo "Ok?!" ; else
> echo "Failed!" ; fi
> Connecting to jdbc:hive2://...
> Connected to: Apache Hive (version 1.1.0-cdh5.5.0)
> Driver: Hive JDBC (version 1.1.0-cdh5.5.0)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 1.1.0-cdh5.5.0 by Apache Hive
> 0: jdbc:hive2://...:8> Error: Error while compiling statement: FAILED:
> ParseException line 1:0 cannot recognize input near 'boo' '' ''
> (state=42000,code=4)
> 0: jdbc:hive2://...:8> Closing: 0: jdbc:hive2://...
> Ok?!
> {code}
> This was misleading our batch scripts to always believe that the execution of
> the queries succeded, when sometimes that was not the case.
> h2. Workaround
> We found we can work around the issue by always using the -e or the -f
> parameters, and even reading the standard input through the /dev/stdin device
> (this was useful because a lot of the scripts fed the queries from here
> documents), like this:
> {code:title=some-script.sh}
> #!/bin/sh
> set -o nounset -o errexit -o pipefail
> # As beeline is failing to report an error status if reading the query
> # to be executed from STDIN, check whether no -f or -e option is used
> # and, in
[jira] [Updated] (HIVE-13358) Stats state is not captured correctly: turn off stats optimizer for sampled table
[ https://issues.apache.org/jira/browse/HIVE-13358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-13358: --- Summary: Stats state is not captured correctly: turn off stats optimizer for sampled table (was: stats state is not captured correctly: turn off stats optimizer for sampled table) > Stats state is not captured correctly: turn off stats optimizer for sampled > table > - > > Key: HIVE-13358 > URL: https://issues.apache.org/jira/browse/HIVE-13358 > Project: Hive > Issue Type: Sub-task > Components: Logical Optimizer, Statistics >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13358) Stats state is not captured correctly: turn off stats optimizer for sampled table
[ https://issues.apache.org/jira/browse/HIVE-13358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-13358: --- Attachment: HIVE-13358.01.patch [~ashutoshc], could u please take a look? > Stats state is not captured correctly: turn off stats optimizer for sampled > table > - > > Key: HIVE-13358 > URL: https://issues.apache.org/jira/browse/HIVE-13358 > Project: Hive > Issue Type: Sub-task > Components: Logical Optimizer, Statistics >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-13358.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13358) Stats state is not captured correctly: turn off stats optimizer for sampled table
[ https://issues.apache.org/jira/browse/HIVE-13358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-13358: --- Status: Patch Available (was: Open) > Stats state is not captured correctly: turn off stats optimizer for sampled > table > - > > Key: HIVE-13358 > URL: https://issues.apache.org/jira/browse/HIVE-13358 > Project: Hive > Issue Type: Sub-task > Components: Logical Optimizer, Statistics >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-13358.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-11766) LLAP: Remove MiniLlapCluster from shim layer after hadoop-1 removal
[ https://issues.apache.org/jira/browse/HIVE-11766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth reassigned HIVE-11766: - Assignee: Siddharth Seth > LLAP: Remove MiniLlapCluster from shim layer after hadoop-1 removal > --- > > Key: HIVE-11766 > URL: https://issues.apache.org/jira/browse/HIVE-11766 > Project: Hive > Issue Type: Sub-task > Components: llap >Reporter: Prasanth Jayachandran >Assignee: Siddharth Seth > > Remove HIVE-11732 changes after HIVE-11378 goes in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11766) LLAP: Remove MiniLlapCluster from shim layer after hadoop-1 removal
[ https://issues.apache.org/jira/browse/HIVE-11766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-11766: -- Target Version/s: 2.1.0 > LLAP: Remove MiniLlapCluster from shim layer after hadoop-1 removal > --- > > Key: HIVE-11766 > URL: https://issues.apache.org/jira/browse/HIVE-11766 > Project: Hive > Issue Type: Sub-task > Components: llap >Reporter: Prasanth Jayachandran >Assignee: Siddharth Seth > > Remove HIVE-11732 changes after HIVE-11378 goes in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13359) NoClassFoundError hadoop configuration with jdbc-standalone JAR
[ https://issues.apache.org/jira/browse/HIVE-13359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-13359: Component/s: JDBC > NoClassFoundError hadoop configuration with jdbc-standalone JAR > --- > > Key: HIVE-13359 > URL: https://issues.apache.org/jira/browse/HIVE-13359 > Project: Hive > Issue Type: Bug > Components: JDBC >Reporter: Rohit Dholakia >Assignee: Rohit Dholakia > > When the hive-jdbc-SNAPSHOT-standalone.jar is used to run queries, it leads > to a NoClassDefFoundError for org/apache/hadoop/conf/Configuration. This > patch will resolve it by updating the jdbc/pom.xml file to not exclude > commons-configuration and org.apache.hadoop:* as part of the maven shaded > plugin. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13358) Stats state is not captured correctly: turn off stats optimizer for sampled table
[ https://issues.apache.org/jira/browse/HIVE-13358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210994#comment-15210994 ] Ashutosh Chauhan commented on HIVE-13358: - +1 pending tests. Instead of adding new test, you can set hive.compute.query.using.stats=true; in existing test. > Stats state is not captured correctly: turn off stats optimizer for sampled > table > - > > Key: HIVE-13358 > URL: https://issues.apache.org/jira/browse/HIVE-13358 > Project: Hive > Issue Type: Sub-task > Components: Logical Optimizer, Statistics >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-13358.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13262) LLAP: Remove log levels from DebugUtils
[ https://issues.apache.org/jira/browse/HIVE-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211002#comment-15211002 ] Sergey Shelukhin commented on HIVE-13262: - +1. Can you test that enabling loggers works locally? > LLAP: Remove log levels from DebugUtils > --- > > Key: HIVE-13262 > URL: https://issues.apache.org/jira/browse/HIVE-13262 > Project: Hive > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-13262.1.patch, HIVE-13262.2.patch, > HIVE-13262.2.patch > > > DebugUtils has many hardcoded log levels. To enable logging we need to > recompile code with desired value. Instead configure add loggers for these > classes with log levels via log4j properties. Also use parametrized logging > in IO elevator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13358) Stats state is not captured correctly: turn off stats optimizer for sampled table
[ https://issues.apache.org/jira/browse/HIVE-13358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211004#comment-15211004 ] Pengcheng Xiong commented on HIVE-13358: [~ashutoshc], thanks a lot for your comments. I have considered using existing test, but i decided to add two new tests to cover two different code paths in StatsOptimizer. > Stats state is not captured correctly: turn off stats optimizer for sampled > table > - > > Key: HIVE-13358 > URL: https://issues.apache.org/jira/browse/HIVE-13358 > Project: Hive > Issue Type: Sub-task > Components: Logical Optimizer, Statistics >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-13358.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13359) NoClassFoundError hadoop configuration with jdbc-standalone JAR
[ https://issues.apache.org/jira/browse/HIVE-13359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Dholakia updated HIVE-13359: -- Attachment: HIVE-13359.1.patch > NoClassFoundError hadoop configuration with jdbc-standalone JAR > --- > > Key: HIVE-13359 > URL: https://issues.apache.org/jira/browse/HIVE-13359 > Project: Hive > Issue Type: Bug > Components: JDBC >Reporter: Rohit Dholakia >Assignee: Rohit Dholakia > Attachments: HIVE-13359.1.patch > > > When the hive-jdbc-SNAPSHOT-standalone.jar is used to run queries, it leads > to a NoClassDefFoundError for org/apache/hadoop/conf/Configuration. This > patch will resolve it by updating the jdbc/pom.xml file to not exclude > commons-configuration and org.apache.hadoop:* as part of the maven shaded > plugin. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12049) Provide an option to write serialized thrift objects in final tasks
[
https://issues.apache.org/jira/browse/HIVE-12049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211011#comment-15211011
]
Vaibhav Gumashta commented on HIVE-12049:
-
[~rohitdholakia] You might want to look at the following failures, which look
related:
{code}
org.apache.hive.beeline.TestBeeLineWithArgs.testEmbeddedBeelineOutputs
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
org.apache.hive.jdbc.TestJdbcDriver2.testGetQueryLog
{code}
> Provide an option to write serialized thrift objects in final tasks
> ---
>
> Key: HIVE-12049
> URL: https://issues.apache.org/jira/browse/HIVE-12049
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2
>Reporter: Rohit Dholakia
>Assignee: Rohit Dholakia
> Attachments: HIVE-12049.1.patch, HIVE-12049.11.patch,
> HIVE-12049.12.patch, HIVE-12049.13.patch, HIVE-12049.14.patch,
> HIVE-12049.2.patch, HIVE-12049.3.patch, HIVE-12049.4.patch,
> HIVE-12049.5.patch, HIVE-12049.6.patch, HIVE-12049.7.patch, HIVE-12049.9.patch
>
>
> For each fetch request to HiveServer2, we pay the penalty of deserializing
> the row objects and translating them into a different representation suitable
> for the RPC transfer. In a moderate to high concurrency scenarios, this can
> result in significant CPU and memory wastage. By having each task write the
> appropriate thrift objects to the output files, HiveServer2 can simply stream
> a batch of rows on the wire without incurring any of the additional cost of
> deserialization and translation.
> This can be implemented by writing a new SerDe, which the FileSinkOperator
> can use to write thrift formatted row batches to the output file. Using the
> pluggable property of the {{hive.query.result.fileformat}}, we can set it to
> use SequenceFile and write a batch of thrift formatted rows as a value blob.
> The FetchTask can now simply read the blob and send it over the wire. On the
> client side, the *DBC driver can read the blob and since it is already
> formatted in the way it expects, it can continue building the ResultSet the
> way it does in the current implementation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9143) select user(), current_user()
[
https://issues.apache.org/jira/browse/HIVE-9143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211024#comment-15211024
]
Lefty Leverenz commented on HIVE-9143:
--
It's already documented in the UDFs doc under Misc. Functions (thanks,
[~apivovarov]):
* [Hive Operators and UDFs -- Misc. Functions |
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-Misc.Functions]
Perhaps it should also be mentioned in the Authorization docs:
* [Hive Default Authorization -- Users, Groups, and Roles |
https://cwiki.apache.org/confluence/display/Hive/Hive+Default+Authorization+-+Legacy+Mode#HiveDefaultAuthorization-LegacyMode-Users,Groups,andRoles]
* [SQL Standard Based Hive Authorization -- Users and Roles |
https://cwiki.apache.org/confluence/display/Hive/SQL+Standard+Based+Hive+Authorization#SQLStandardBasedHiveAuthorization-UsersandRoles]
Hmmm ... the SQL Standards Based Auth doc needs a section on user commands,
similar to Role Management Commands. If it had that, current_user() should be
documented there instead.
Ditto for Hive Default Authorization: it has Creating/Dropping/Using Roles but
no equivalent section for users.
> select user(), current_user()
> -
>
> Key: HIVE-9143
> URL: https://issues.apache.org/jira/browse/HIVE-9143
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 0.13.0
>Reporter: Hari Sekhon
>Assignee: Alexander Pivovarov
>Priority: Minor
> Fix For: 1.2.0
>
> Attachments: HIVE-9143.1.patch, HIVE-9143.2.patch, HIVE-9143.3.patch
>
>
> Feature request to add support for determining in HQL session which user I am
> currently connected as - an old MySQL ability:
> {code}mysql> select user(), current_user();
> +++
> | user() | current_user() |
> +++
> | root@localhost | root@localhost |
> +++
> 1 row in set (0.00 sec)
> {code}
> which doesn't seem to have a counterpart in Hive at this time:
> {code}0: jdbc:hive2://:100> select user();
> Error: Error while compiling statement: FAILED: SemanticException Line 0:-1
> Invalid function 'user' (state=42000,code=4)
> 0: jdbc:hive2://:100> select current_user();
> Error: Error while compiling statement: FAILED: SemanticException [Error
> 10011]: Line 1:7 Invalid function 'current_user'
> (state=42000,code=10011){code}
> Regards,
> Hari Sekhon
> http://www.linkedin.com/in/harisekhon
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13361) Orc concatenation should enforce the compression buffer size
[
https://issues.apache.org/jira/browse/HIVE-13361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-13361:
-
Description:
With HIVE-11807 buffer size estimation happens by default. This can have
undesired effect wrt file concatenation. Consider the following table with files
{code}
testtable
-- 00_0 (created before HIVE-11807 which has buffer size 256KB)
-- 01_0 (created before HIVE-11807 which has buffer size 256KB)
-- 02_0 (created after HIVE-11807 with buffer size chosen as 128KB)
-- 03_0 (created after HIVE-11807 with buffer size chosen as 128KB)
{code}
If we perform ALTER TABLE .. CONCATENATE on the above table with HIVE-11807,
then depending on the split arrangement 00_0 and 01_0 will be
concatenated together to new merged file. But this new merged file will have
128KB buffer size (estimated buffer size and not requested buffer size). Since
new ORC writer size does not honor the requested buffer size the new merged
files will have smaller buffers than the required 256KB making the file
unreadable. Following exception will be thrown when reading the table after
concatenation
{code}
2016-03-24T16:26:33,974 ERROR [a9e27a9a-37cb-411d-9708-6c58a4ce34f2 main]:
CliDriver (SessionState.java:printError(1049)) - Failed with exception
java.io.IOException:java.lang.IllegalArgumentException: Buffer size too small.
size = 131072 needed = 153187
java.io.IOException: java.lang.IllegalArgumentException: Buffer size too small.
size = 131072 needed = 153187
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:513)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:420)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:145)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1848)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:256)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:187)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:782)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:721)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:648)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
{code}
was:
With HIVE-11807 buffer size estimation happens by default. This can have
undesired effect wrt file concatenation. Consider the following table with files
{code}
testtable
-- 00_0 (created before HIVE-11807 which has buffer size 256KB)
-- 01_0 (created before HIVE-11807 which has buffer size 256KB)
-- 02_0 (created after HIVE-11807 with buffer size chosen as 128KB)
-- 03_0 (created after HIVE-11807 with buffer size chosen as 128KB)
{code}
If we perform ALTER TABLE .. CONCATENATE on the above table with HIVE-11807,
then depending on the split arrangement 00_0 and 01_0 will be
concatenated together to new merged file. But this new merged file will have
128KB buffer size (estimated buffer size and not requested buffer size). Since
new ORC writer size does not honor the requested buffer size the new merged
files will have smaller buffers than the required 256KB making the file
unreadable. Following exception will be thrown when reading the table after
concatenation
{code}
2016-03-24T16:26:33,974 ERROR [a9e27a9a-37cb-411d-9708-6c58a4ce34f2 main]:
CliDriver (SessionState.java:printError(1049)) - Failed with exception
java.io.IOException:java.lang.IllegalArgumentException: Buffer size too small.
size = 131072 needed = 153187
java.io.IOException: java.lang.IllegalArgumentException: Buffer size too small.
size = 131072 needed = 153187
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:513)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:420)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:145)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1848)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:256)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:187)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:782)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDrive
[jira] [Commented] (HIVE-9143) select user(), current_user()
[
https://issues.apache.org/jira/browse/HIVE-9143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211028#comment-15211028
]
Lefty Leverenz commented on HIVE-9143:
--
Um ... I vaguely recall something about users being defined externally so there
aren't any create/drop commands for them. Is that correct?
> select user(), current_user()
> -
>
> Key: HIVE-9143
> URL: https://issues.apache.org/jira/browse/HIVE-9143
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 0.13.0
>Reporter: Hari Sekhon
>Assignee: Alexander Pivovarov
>Priority: Minor
> Fix For: 1.2.0
>
> Attachments: HIVE-9143.1.patch, HIVE-9143.2.patch, HIVE-9143.3.patch
>
>
> Feature request to add support for determining in HQL session which user I am
> currently connected as - an old MySQL ability:
> {code}mysql> select user(), current_user();
> +++
> | user() | current_user() |
> +++
> | root@localhost | root@localhost |
> +++
> 1 row in set (0.00 sec)
> {code}
> which doesn't seem to have a counterpart in Hive at this time:
> {code}0: jdbc:hive2://:100> select user();
> Error: Error while compiling statement: FAILED: SemanticException Line 0:-1
> Invalid function 'user' (state=42000,code=4)
> 0: jdbc:hive2://:100> select current_user();
> Error: Error while compiling statement: FAILED: SemanticException [Error
> 10011]: Line 1:7 Invalid function 'current_user'
> (state=42000,code=10011){code}
> Regards,
> Hari Sekhon
> http://www.linkedin.com/in/harisekhon
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13361) Orc concatenation should enforce the compression buffer size
[
https://issues.apache.org/jira/browse/HIVE-13361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211027#comment-15211027
]
Prasanth Jayachandran commented on HIVE-13361:
--
Ideally we want, compression buffer size stored in stripe footer instead of
file footer which can enable concatenating files of different buffer sizes. For
now, we can add an option to writer to enforce the requested buffer size
instead of estimating it.
> Orc concatenation should enforce the compression buffer size
>
>
> Key: HIVE-13361
> URL: https://issues.apache.org/jira/browse/HIVE-13361
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.3.0, 2.0.0, 2.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Critical
>
> With HIVE-11807 buffer size estimation happens by default. This can have
> undesired effect wrt file concatenation. Consider the following table with
> files
> {code}
> testtable
> -- 00_0 (created before HIVE-11807 which has buffer size 256KB)
> -- 01_0 (created before HIVE-11807 which has buffer size 256KB)
> -- 02_0 (created after HIVE-11807 with buffer size chosen as 128KB)
> -- 03_0 (created after HIVE-11807 with buffer size chosen as 128KB)
> {code}
> If we perform ALTER TABLE .. CONCATENATE on the above table with HIVE-11807,
> then depending on the split arrangement 00_0 and 01_0 will be
> concatenated together to new merged file. But this new merged file will have
> 128KB buffer size (estimated buffer size and not requested buffer size).
> Since new ORC writer size does not honor the requested buffer size the new
> merged files will have smaller buffers than the required 256KB making the
> file unreadable. Following exception will be thrown when reading the table
> after concatenation
> {code}
> 2016-03-24T16:26:33,974 ERROR [a9e27a9a-37cb-411d-9708-6c58a4ce34f2 main]:
> CliDriver (SessionState.java:printError(1049)) - Failed with exception
> java.io.IOException:java.lang.IllegalArgumentException: Buffer size too
> small. size = 131072 needed = 153187
> java.io.IOException: java.lang.IllegalArgumentException: Buffer size too
> small. size = 131072 needed = 153187
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:513)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:420)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:145)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1848)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:256)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:187)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:782)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:721)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:648)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HIVE-13361) Orc concatenation should enforce the compression buffer size
[
https://issues.apache.org/jira/browse/HIVE-13361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211027#comment-15211027
]
Prasanth Jayachandran edited comment on HIVE-13361 at 3/24/16 10:03 PM:
Ideally we want, compression buffer size stored in stripe footer instead of
file footer which can enable concatenating files of different buffer sizes. For
now, we can add an option to writer to enforce the requested buffer size
instead of estimating it for concatenation use case.
was (Author: prasanth_j):
Ideally we want, compression buffer size stored in stripe footer instead of
file footer which can enable concatenating files of different buffer sizes. For
now, we can add an option to writer to enforce the requested buffer size
instead of estimating it.
> Orc concatenation should enforce the compression buffer size
>
>
> Key: HIVE-13361
> URL: https://issues.apache.org/jira/browse/HIVE-13361
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.3.0, 2.0.0, 2.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Critical
>
> With HIVE-11807 buffer size estimation happens by default. This can have
> undesired effect wrt file concatenation. Consider the following table with
> files
> {code}
> testtable
> -- 00_0 (created before HIVE-11807 which has buffer size 256KB)
> -- 01_0 (created before HIVE-11807 which has buffer size 256KB)
> -- 02_0 (created after HIVE-11807 with buffer size chosen as 128KB)
> -- 03_0 (created after HIVE-11807 with buffer size chosen as 128KB)
> {code}
> If we perform ALTER TABLE .. CONCATENATE on the above table with HIVE-11807,
> then depending on the split arrangement 00_0 and 01_0 will be
> concatenated together to new merged file. But this new merged file will have
> 128KB buffer size (estimated buffer size and not requested buffer size).
> Since new ORC writer size does not honor the requested buffer size the new
> merged files will have smaller buffers than the required 256KB making the
> file unreadable. Following exception will be thrown when reading the table
> after concatenation
> {code}
> 2016-03-24T16:26:33,974 ERROR [a9e27a9a-37cb-411d-9708-6c58a4ce34f2 main]:
> CliDriver (SessionState.java:printError(1049)) - Failed with exception
> java.io.IOException:java.lang.IllegalArgumentException: Buffer size too
> small. size = 131072 needed = 153187
> java.io.IOException: java.lang.IllegalArgumentException: Buffer size too
> small. size = 131072 needed = 153187
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:513)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:420)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:145)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1848)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:256)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:187)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:782)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:721)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:648)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13361) Orc concatenation should enforce the compression buffer size
[
https://issues.apache.org/jira/browse/HIVE-13361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-13361:
-
Reporter: Yi Zhang (was: Prasanth Jayachandran)
> Orc concatenation should enforce the compression buffer size
>
>
> Key: HIVE-13361
> URL: https://issues.apache.org/jira/browse/HIVE-13361
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.3.0, 2.0.0, 2.1.0
>Reporter: Yi Zhang
>Assignee: Prasanth Jayachandran
>Priority: Critical
>
> With HIVE-11807 buffer size estimation happens by default. This can have
> undesired effect wrt file concatenation. Consider the following table with
> files
> {code}
> testtable
> -- 00_0 (created before HIVE-11807 which has buffer size 256KB)
> -- 01_0 (created before HIVE-11807 which has buffer size 256KB)
> -- 02_0 (created after HIVE-11807 with buffer size chosen as 128KB)
> -- 03_0 (created after HIVE-11807 with buffer size chosen as 128KB)
> {code}
> If we perform ALTER TABLE .. CONCATENATE on the above table with HIVE-11807,
> then depending on the split arrangement 00_0 and 01_0 will be
> concatenated together to new merged file. But this new merged file will have
> 128KB buffer size (estimated buffer size and not requested buffer size).
> Since new ORC writer size does not honor the requested buffer size the new
> merged files will have smaller buffers than the required 256KB making the
> file unreadable. Following exception will be thrown when reading the table
> after concatenation
> {code}
> 2016-03-24T16:26:33,974 ERROR [a9e27a9a-37cb-411d-9708-6c58a4ce34f2 main]:
> CliDriver (SessionState.java:printError(1049)) - Failed with exception
> java.io.IOException:java.lang.IllegalArgumentException: Buffer size too
> small. size = 131072 needed = 153187
> java.io.IOException: java.lang.IllegalArgumentException: Buffer size too
> small. size = 131072 needed = 153187
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:513)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:420)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:145)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1848)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:256)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:187)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:782)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:721)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:648)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11766) LLAP: Remove MiniLlapCluster from shim layer after hadoop-1 removal
[ https://issues.apache.org/jira/browse/HIVE-11766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211050#comment-15211050 ] Siddharth Seth commented on HIVE-11766: --- [~prasanth_j], [~sershe] - please review. > LLAP: Remove MiniLlapCluster from shim layer after hadoop-1 removal > --- > > Key: HIVE-11766 > URL: https://issues.apache.org/jira/browse/HIVE-11766 > Project: Hive > Issue Type: Sub-task > Components: llap >Reporter: Prasanth Jayachandran >Assignee: Siddharth Seth > Attachments: HIVE-11766.1.patch > > > Remove HIVE-11732 changes after HIVE-11378 goes in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11766) LLAP: Remove MiniLlapCluster from shim layer after hadoop-1 removal
[ https://issues.apache.org/jira/browse/HIVE-11766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-11766: -- Attachment: HIVE-11766.1.patch Removes llap daemon from the shim, and gets rid of llap-daemon-site, in favor of putting the settings in hive-site.xml. MiniHS2 will end up not loading tez-site.xml - I'm not sure how to fix that and ensure it picks up the correct working dir. > LLAP: Remove MiniLlapCluster from shim layer after hadoop-1 removal > --- > > Key: HIVE-11766 > URL: https://issues.apache.org/jira/browse/HIVE-11766 > Project: Hive > Issue Type: Sub-task > Components: llap >Reporter: Prasanth Jayachandran >Assignee: Siddharth Seth > Attachments: HIVE-11766.1.patch > > > Remove HIVE-11732 changes after HIVE-11378 goes in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11766) LLAP: Remove MiniLlapCluster from shim layer after hadoop-1 removal
[ https://issues.apache.org/jira/browse/HIVE-11766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-11766: -- Status: Patch Available (was: Open) > LLAP: Remove MiniLlapCluster from shim layer after hadoop-1 removal > --- > > Key: HIVE-11766 > URL: https://issues.apache.org/jira/browse/HIVE-11766 > Project: Hive > Issue Type: Sub-task > Components: llap >Reporter: Prasanth Jayachandran >Assignee: Siddharth Seth > Attachments: HIVE-11766.1.patch > > > Remove HIVE-11732 changes after HIVE-11378 goes in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13361) Orc concatenation should enforce the compression buffer size
[
https://issues.apache.org/jira/browse/HIVE-13361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-13361:
-
Attachment: HIVE-13361.1.patch
> Orc concatenation should enforce the compression buffer size
>
>
> Key: HIVE-13361
> URL: https://issues.apache.org/jira/browse/HIVE-13361
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.3.0, 2.0.0, 2.1.0
>Reporter: Yi Zhang
>Assignee: Prasanth Jayachandran
>Priority: Critical
> Attachments: HIVE-13361.1.patch
>
>
> With HIVE-11807 buffer size estimation happens by default. This can have
> undesired effect wrt file concatenation. Consider the following table with
> files
> {code}
> testtable
> -- 00_0 (created before HIVE-11807 which has buffer size 256KB)
> -- 01_0 (created before HIVE-11807 which has buffer size 256KB)
> -- 02_0 (created after HIVE-11807 with buffer size chosen as 128KB)
> -- 03_0 (created after HIVE-11807 with buffer size chosen as 128KB)
> {code}
> If we perform ALTER TABLE .. CONCATENATE on the above table with HIVE-11807,
> then depending on the split arrangement 00_0 and 01_0 will be
> concatenated together to new merged file. But this new merged file will have
> 128KB buffer size (estimated buffer size and not requested buffer size).
> Since new ORC writer size does not honor the requested buffer size the new
> merged files will have smaller buffers than the required 256KB making the
> file unreadable. Following exception will be thrown when reading the table
> after concatenation
> {code}
> 2016-03-24T16:26:33,974 ERROR [a9e27a9a-37cb-411d-9708-6c58a4ce34f2 main]:
> CliDriver (SessionState.java:printError(1049)) - Failed with exception
> java.io.IOException:java.lang.IllegalArgumentException: Buffer size too
> small. size = 131072 needed = 153187
> java.io.IOException: java.lang.IllegalArgumentException: Buffer size too
> small. size = 131072 needed = 153187
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:513)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:420)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:145)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1848)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:256)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:187)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:782)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:721)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:648)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13361) Orc concatenation should enforce the compression buffer size
[
https://issues.apache.org/jira/browse/HIVE-13361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-13361:
-
Attachment: alltypesorc3xcols
This patch needs the attached orc file. This orc file is generated before
HIVE-11807 with around 30 columns with 256KB buffer size. Same file if created
after 11807 will have 128KB buffer size.
> Orc concatenation should enforce the compression buffer size
>
>
> Key: HIVE-13361
> URL: https://issues.apache.org/jira/browse/HIVE-13361
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.3.0, 2.0.0, 2.1.0
>Reporter: Yi Zhang
>Assignee: Prasanth Jayachandran
>Priority: Critical
> Attachments: HIVE-13361.1.patch, alltypesorc3xcols
>
>
> With HIVE-11807 buffer size estimation happens by default. This can have
> undesired effect wrt file concatenation. Consider the following table with
> files
> {code}
> testtable
> -- 00_0 (created before HIVE-11807 which has buffer size 256KB)
> -- 01_0 (created before HIVE-11807 which has buffer size 256KB)
> -- 02_0 (created after HIVE-11807 with buffer size chosen as 128KB)
> -- 03_0 (created after HIVE-11807 with buffer size chosen as 128KB)
> {code}
> If we perform ALTER TABLE .. CONCATENATE on the above table with HIVE-11807,
> then depending on the split arrangement 00_0 and 01_0 will be
> concatenated together to new merged file. But this new merged file will have
> 128KB buffer size (estimated buffer size and not requested buffer size).
> Since new ORC writer size does not honor the requested buffer size the new
> merged files will have smaller buffers than the required 256KB making the
> file unreadable. Following exception will be thrown when reading the table
> after concatenation
> {code}
> 2016-03-24T16:26:33,974 ERROR [a9e27a9a-37cb-411d-9708-6c58a4ce34f2 main]:
> CliDriver (SessionState.java:printError(1049)) - Failed with exception
> java.io.IOException:java.lang.IllegalArgumentException: Buffer size too
> small. size = 131072 needed = 153187
> java.io.IOException: java.lang.IllegalArgumentException: Buffer size too
> small. size = 131072 needed = 153187
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:513)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:420)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:145)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1848)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:256)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:187)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:782)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:721)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:648)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13361) Orc concatenation should enforce the compression buffer size
[
https://issues.apache.org/jira/browse/HIVE-13361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-13361:
-
Status: Patch Available (was: Open)
> Orc concatenation should enforce the compression buffer size
>
>
> Key: HIVE-13361
> URL: https://issues.apache.org/jira/browse/HIVE-13361
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.3.0, 2.1.0
>Reporter: Yi Zhang
>Assignee: Prasanth Jayachandran
>Priority: Critical
> Attachments: HIVE-13361.1.patch, alltypesorc3xcols
>
>
> With HIVE-11807 buffer size estimation happens by default. This can have
> undesired effect wrt file concatenation. Consider the following table with
> files
> {code}
> testtable
> -- 00_0 (created before HIVE-11807 which has buffer size 256KB)
> -- 01_0 (created before HIVE-11807 which has buffer size 256KB)
> -- 02_0 (created after HIVE-11807 with buffer size chosen as 128KB)
> -- 03_0 (created after HIVE-11807 with buffer size chosen as 128KB)
> {code}
> If we perform ALTER TABLE .. CONCATENATE on the above table with HIVE-11807,
> then depending on the split arrangement 00_0 and 01_0 will be
> concatenated together to new merged file. But this new merged file will have
> 128KB buffer size (estimated buffer size and not requested buffer size).
> Since new ORC writer size does not honor the requested buffer size the new
> merged files will have smaller buffers than the required 256KB making the
> file unreadable. Following exception will be thrown when reading the table
> after concatenation
> {code}
> 2016-03-24T16:26:33,974 ERROR [a9e27a9a-37cb-411d-9708-6c58a4ce34f2 main]:
> CliDriver (SessionState.java:printError(1049)) - Failed with exception
> java.io.IOException:java.lang.IllegalArgumentException: Buffer size too
> small. size = 131072 needed = 153187
> java.io.IOException: java.lang.IllegalArgumentException: Buffer size too
> small. size = 131072 needed = 153187
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:513)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:420)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:145)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1848)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:256)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:187)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:782)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:721)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:648)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13361) Orc concatenation should enforce the compression buffer size
[
https://issues.apache.org/jira/browse/HIVE-13361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211091#comment-15211091
]
Prasanth Jayachandran commented on HIVE-13361:
--
[~gopalv] Could you please review the patch? It's mostly test diff.
> Orc concatenation should enforce the compression buffer size
>
>
> Key: HIVE-13361
> URL: https://issues.apache.org/jira/browse/HIVE-13361
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.3.0, 2.0.0, 2.1.0
>Reporter: Yi Zhang
>Assignee: Prasanth Jayachandran
>Priority: Critical
> Attachments: HIVE-13361.1.patch, alltypesorc3xcols
>
>
> With HIVE-11807 buffer size estimation happens by default. This can have
> undesired effect wrt file concatenation. Consider the following table with
> files
> {code}
> testtable
> -- 00_0 (created before HIVE-11807 which has buffer size 256KB)
> -- 01_0 (created before HIVE-11807 which has buffer size 256KB)
> -- 02_0 (created after HIVE-11807 with buffer size chosen as 128KB)
> -- 03_0 (created after HIVE-11807 with buffer size chosen as 128KB)
> {code}
> If we perform ALTER TABLE .. CONCATENATE on the above table with HIVE-11807,
> then depending on the split arrangement 00_0 and 01_0 will be
> concatenated together to new merged file. But this new merged file will have
> 128KB buffer size (estimated buffer size and not requested buffer size).
> Since new ORC writer size does not honor the requested buffer size the new
> merged files will have smaller buffers than the required 256KB making the
> file unreadable. Following exception will be thrown when reading the table
> after concatenation
> {code}
> 2016-03-24T16:26:33,974 ERROR [a9e27a9a-37cb-411d-9708-6c58a4ce34f2 main]:
> CliDriver (SessionState.java:printError(1049)) - Failed with exception
> java.io.IOException:java.lang.IllegalArgumentException: Buffer size too
> small. size = 131072 needed = 153187
> java.io.IOException: java.lang.IllegalArgumentException: Buffer size too
> small. size = 131072 needed = 153187
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:513)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:420)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:145)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1848)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:256)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:187)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:782)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:721)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:648)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13361) Orc concatenation should enforce the compression buffer size
[
https://issues.apache.org/jira/browse/HIVE-13361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211092#comment-15211092
]
Prasanth Jayachandran commented on HIVE-13361:
--
If the approach looks good I will commit the orc binary file first so that it
will be available for precommit test run.
> Orc concatenation should enforce the compression buffer size
>
>
> Key: HIVE-13361
> URL: https://issues.apache.org/jira/browse/HIVE-13361
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.3.0, 2.0.0, 2.1.0
>Reporter: Yi Zhang
>Assignee: Prasanth Jayachandran
>Priority: Critical
> Attachments: HIVE-13361.1.patch, alltypesorc3xcols
>
>
> With HIVE-11807 buffer size estimation happens by default. This can have
> undesired effect wrt file concatenation. Consider the following table with
> files
> {code}
> testtable
> -- 00_0 (created before HIVE-11807 which has buffer size 256KB)
> -- 01_0 (created before HIVE-11807 which has buffer size 256KB)
> -- 02_0 (created after HIVE-11807 with buffer size chosen as 128KB)
> -- 03_0 (created after HIVE-11807 with buffer size chosen as 128KB)
> {code}
> If we perform ALTER TABLE .. CONCATENATE on the above table with HIVE-11807,
> then depending on the split arrangement 00_0 and 01_0 will be
> concatenated together to new merged file. But this new merged file will have
> 128KB buffer size (estimated buffer size and not requested buffer size).
> Since new ORC writer size does not honor the requested buffer size the new
> merged files will have smaller buffers than the required 256KB making the
> file unreadable. Following exception will be thrown when reading the table
> after concatenation
> {code}
> 2016-03-24T16:26:33,974 ERROR [a9e27a9a-37cb-411d-9708-6c58a4ce34f2 main]:
> CliDriver (SessionState.java:printError(1049)) - Failed with exception
> java.io.IOException:java.lang.IllegalArgumentException: Buffer size too
> small. size = 131072 needed = 153187
> java.io.IOException: java.lang.IllegalArgumentException: Buffer size too
> small. size = 131072 needed = 153187
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:513)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:420)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:145)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1848)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:256)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:187)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:782)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:721)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:648)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13295) Improvement to LDAP search queries in HS2 LDAP Authenticator
[ https://issues.apache.org/jira/browse/HIVE-13295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1523#comment-1523 ] Lefty Leverenz commented on HIVE-13295: --- The docs look good, thanks [~ngangam]! Here are links to the three new configuration parameters: * [Configuration Properties -- hive.server2.authentication.ldap.guidKey | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.server2.authentication.ldap.guidKey] * [Configuration Properties -- hive.server2.authentication.ldap.groupMembershipKey | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.server2.authentication.ldap.groupMembershipKey] * [Configuration Properties -- hive.server2.authentication.ldap.groupClassKey | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.server2.authentication.ldap.groupClassKey] > Improvement to LDAP search queries in HS2 LDAP Authenticator > > > Key: HIVE-13295 > URL: https://issues.apache.org/jira/browse/HIVE-13295 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 1.3.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam > Fix For: 2.1.0 > > Attachments: HIVE-13295.1.patch, HIVE-13295.2.patch > > > As more usecases, for various LDAP flavors and deployments, emerge, Hive's > LDAP authentication provider needs additional configuration properties to > make it more flexible to work with different LDAP deployments. > For example: > 1) Not every LDAP server supports a "memberOf" property on user entries that > refer to the groups the user belongs to. This attribute is used for group > filter support. So instead of relying on this attribute to be set, we can > reverse the search and find all the groups that have an attribute, that > refers to its members, set. For example "member" or "memberUid" etc. > Since this atttribute name differs from ldap to ldap, its best we make this > configurable, with a default value of "member" > 2) In HIVE-12885, a new property was introduced to make the attribute for an > user/group search key user-configurable instead of assuming its "uid" (when > baseDN is set) or "cn" (otherwise). This change was deferred from the initial > patch. > 3) LDAP Groups can have various ObjectClass'es. For example objectClass=group > or objectClass=groupOfNames or objectClass=posixGroup or > objectClass=groupOfUniqueNames etc. There could be other we dont know of. > So we need a property to make this user-configurable with a certain default. > 4) There is also a bug where the lists for groupFilter and userFilter are not > re-initialized each time init() is called. > These lists are only re-initialized if the new HiveConf has userFilter or > groupFilter set values. Otherwise, the provider will use values from previous > initialization. > I found this bug when writing some new tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13217) Replication for HoS mapjoin small file needs to respect dfs.replication.max
[ https://issues.apache.org/jira/browse/HIVE-13217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211139#comment-15211139 ] Lefty Leverenz commented on HIVE-13217: --- [~chinnalalam], since this was committed to master the Fix Version/s should be 2.1.0 instead of 2.0.0. > Replication for HoS mapjoin small file needs to respect dfs.replication.max > --- > > Key: HIVE-13217 > URL: https://issues.apache.org/jira/browse/HIVE-13217 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 1.2.1, 2.0.0 >Reporter: Szehon Ho >Assignee: Chinna Rao Lalam >Priority: Minor > Fix For: 2.0.0 > > Attachments: HIVE-13217.1.patch, HIVE-13217.2.patch > > > Currently Hive on Spark Mapjoin replicates small table file to a hard-coded > value of 10. See SparkHashTableSinkOperator.MIN_REPLICATION. > When dfs.replication.max is less than 10, HoS query fails. This constant > should cap at dfs.replication.max. > Normally dfs.replication.max seems set at 512. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12902) Refactor TxnHandler to be an interface
[ https://issues.apache.org/jira/browse/HIVE-12902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211149#comment-15211149 ] Lefty Leverenz commented on HIVE-12902: --- Also see HIVE-13344, which adds *hive.metastore.txn.store.impl* in branch-1 for release 1.3.0. > Refactor TxnHandler to be an interface > -- > > Key: HIVE-12902 > URL: https://issues.apache.org/jira/browse/HIVE-12902 > Project: Hive > Issue Type: Improvement > Components: Metastore, Transactions >Affects Versions: 2.0.0 >Reporter: Alan Gates >Assignee: Alan Gates > Labels: TODOC2.1 > Fix For: 2.1.0 > > Attachments: HIVE-12902.2.patch, HIVE-12902.3.patch, HIVE-12902.patch > > > TxnHandler (and it's extender CompactionTxnHandler) implement the transaction > functionality in the metastore. These need to be re-factored as interfaces > so that we can have a separate implementation for the HBase metastore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13344) port HIVE-12902 to 1.x line
[ https://issues.apache.org/jira/browse/HIVE-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-13344: -- Labels: TODOC1.3 (was: ) > port HIVE-12902 to 1.x line > --- > > Key: HIVE-13344 > URL: https://issues.apache.org/jira/browse/HIVE-13344 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.3.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Labels: TODOC1.3 > Attachments: HIVE-12902.branch-1.patch, HIVE-13344.branch-1.patch, > HIVE-13344.patch > > > w/o this it makes it difficult to make matching checkins into 2.x and 1.x line -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13281) Update some default configs for LLAP
[ https://issues.apache.org/jira/browse/HIVE-13281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211157#comment-15211157 ] Sergey Shelukhin commented on HIVE-13281: - Is there a plan to finish the other half? :) > Update some default configs for LLAP > > > Key: HIVE-13281 > URL: https://issues.apache.org/jira/browse/HIVE-13281 > Project: Hive > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-13281.1.patch > > > Disable uber mode. > Enable llap.io by default -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HIVE-13281) Update some default configs for LLAP
[ https://issues.apache.org/jira/browse/HIVE-13281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211157#comment-15211157 ] Sergey Shelukhin edited comment on HIVE-13281 at 3/24/16 11:56 PM: --- Is there a plan to finish the other half? :) Actually, what is missing? was (Author: sershe): Is there a plan to finish the other half? :) > Update some default configs for LLAP > > > Key: HIVE-13281 > URL: https://issues.apache.org/jira/browse/HIVE-13281 > Project: Hive > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-13281.1.patch > > > Disable uber mode. > Enable llap.io by default -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13344) port HIVE-12902 to 1.x line
[ https://issues.apache.org/jira/browse/HIVE-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211159#comment-15211159 ] Lefty Leverenz commented on HIVE-13344: --- Doc note: This adds *hive.metastore.txn.store.impl* to HiveConf.java, so it needs to be documented in the wiki for release 1.3.0 (TODOC1.3 label). HIVE-12902 added it for 2.1.0 -- see the doc note there for other places this should be documented: * [doc note on HIVE-12902 | https://issues.apache.org/jira/browse/HIVE-12902?focusedCommentId=15124607&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15124607] > port HIVE-12902 to 1.x line > --- > > Key: HIVE-13344 > URL: https://issues.apache.org/jira/browse/HIVE-13344 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.3.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Labels: TODOC1.3 > Attachments: HIVE-12902.branch-1.patch, HIVE-13344.branch-1.patch, > HIVE-13344.patch > > > w/o this it makes it difficult to make matching checkins into 2.x and 1.x line -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13151) Clean up UGI objects in FileSystem cache for transactions
[
https://issues.apache.org/jira/browse/HIVE-13151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211163#comment-15211163
]
Hive QA commented on HIVE-13151:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12794812/HIVE-13151.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 9821 tests executed
*Failed tests:*
{noformat}
TestParseNegative - did not produce a TEST-*.xml file
TestSparkCliDriver-groupby3_map.q-sample2.q-auto_join14.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-groupby_map_ppr_multi_distinct.q-table_access_keys_stats.q-groupby4_noskew.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-join_rc.q-insert1.q-vectorized_rcfile_columnar.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-ppd_join4.q-join9.q-ppd_join3.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llap_udf
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7358/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7358/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7358/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12794812 - PreCommit-HIVE-TRUNK-Build
> Clean up UGI objects in FileSystem cache for transactions
> -
>
> Key: HIVE-13151
> URL: https://issues.apache.org/jira/browse/HIVE-13151
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.0.0
>Reporter: Wei Zheng
>Assignee: Wei Zheng
> Attachments: HIVE-13151.1.patch, HIVE-13151.2.patch,
> HIVE-13151.3.patch, HIVE-13151.4.patch
>
>
> One issue with FileSystem.CACHE is that it does not clean itself. The key in
> that cache includes UGI object. When new UGI objects are created and used
> with the FileSystem api, new entries get added to the cache.
> We need to manually clean up those UGI objects once they are no longer in use.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9660) store end offset of compressed data for RG in RowIndex in ORC
[
https://issues.apache.org/jira/browse/HIVE-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211170#comment-15211170
]
Hive QA commented on HIVE-9660:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12794925/HIVE-9660.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7359/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7359/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7359/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]]
+ export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ export
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-7359/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
db2efe4..1787082 branch-1 -> origin/branch-1
+ git reset --hard HEAD
HEAD is now at d3a5f20 HIVE-13325: Excessive logging when ORC PPD fails type
conversions (Prasanth Jayachandran reviewed by Gopal V)
+ git clean -f -d
Removing ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java.orig
Removing ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Initiator.java.orig
Removing ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java.orig
+ git checkout master
Already on 'master'
+ git reset --hard origin/master
HEAD is now at d3a5f20 HIVE-13325: Excessive logging when ORC PPD fails type
conversions (Prasanth Jayachandran reviewed by Gopal V)
+ git merge --ff-only origin/master
Already up-to-date.
+ git gc
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12794925 - PreCommit-HIVE-TRUNK-Build
> store end offset of compressed data for RG in RowIndex in ORC
> -
>
> Key: HIVE-9660
> URL: https://issues.apache.org/jira/browse/HIVE-9660
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-9660.WIP2.patch, HIVE-9660.patch, HIVE-9660.patch
>
>
> Right now the end offset is estimated, which in some cases results in tons of
> extra data being read.
> We can add a separate array to RowIndex (positions_v2?) that stores number of
> compressed buffers for each RG, or end offset, or something, to remove this
> estimation magic
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13344) port HIVE-12902 to 1.x line
[ https://issues.apache.org/jira/browse/HIVE-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211179#comment-15211179 ] Eugene Koifman commented on HIVE-13344: --- we don't need to doc this for 1.x line. It's only expected to have 1 (default) value in 1.x. > port HIVE-12902 to 1.x line > --- > > Key: HIVE-13344 > URL: https://issues.apache.org/jira/browse/HIVE-13344 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.3.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Labels: TODOC1.3 > Attachments: HIVE-12902.branch-1.patch, HIVE-13344.branch-1.patch, > HIVE-13344.patch > > > w/o this it makes it difficult to make matching checkins into 2.x and 1.x line -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13344) port HIVE-12902 to 1.x line
[ https://issues.apache.org/jira/browse/HIVE-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-13344: -- Resolution: Fixed Fix Version/s: 1.3.0 Status: Resolved (was: Patch Available) > port HIVE-12902 to 1.x line > --- > > Key: HIVE-13344 > URL: https://issues.apache.org/jira/browse/HIVE-13344 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.3.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Labels: TODOC1.3 > Fix For: 1.3.0 > > Attachments: HIVE-12902.branch-1.patch, HIVE-13344.branch-1.patch, > HIVE-13344.patch > > > w/o this it makes it difficult to make matching checkins into 2.x and 1.x line -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13361) Orc concatenation should enforce the compression buffer size
[
https://issues.apache.org/jira/browse/HIVE-13361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211200#comment-15211200
]
Gopal V commented on HIVE-13361:
LGTM - +1.
I like the new log line, that makes sense to log the buffer sizes on the writer.
> Orc concatenation should enforce the compression buffer size
>
>
> Key: HIVE-13361
> URL: https://issues.apache.org/jira/browse/HIVE-13361
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.3.0, 2.0.0, 2.1.0
>Reporter: Yi Zhang
>Assignee: Prasanth Jayachandran
>Priority: Critical
> Attachments: HIVE-13361.1.patch, alltypesorc3xcols
>
>
> With HIVE-11807 buffer size estimation happens by default. This can have
> undesired effect wrt file concatenation. Consider the following table with
> files
> {code}
> testtable
> -- 00_0 (created before HIVE-11807 which has buffer size 256KB)
> -- 01_0 (created before HIVE-11807 which has buffer size 256KB)
> -- 02_0 (created after HIVE-11807 with buffer size chosen as 128KB)
> -- 03_0 (created after HIVE-11807 with buffer size chosen as 128KB)
> {code}
> If we perform ALTER TABLE .. CONCATENATE on the above table with HIVE-11807,
> then depending on the split arrangement 00_0 and 01_0 will be
> concatenated together to new merged file. But this new merged file will have
> 128KB buffer size (estimated buffer size and not requested buffer size).
> Since new ORC writer size does not honor the requested buffer size the new
> merged files will have smaller buffers than the required 256KB making the
> file unreadable. Following exception will be thrown when reading the table
> after concatenation
> {code}
> 2016-03-24T16:26:33,974 ERROR [a9e27a9a-37cb-411d-9708-6c58a4ce34f2 main]:
> CliDriver (SessionState.java:printError(1049)) - Failed with exception
> java.io.IOException:java.lang.IllegalArgumentException: Buffer size too
> small. size = 131072 needed = 153187
> java.io.IOException: java.lang.IllegalArgumentException: Buffer size too
> small. size = 131072 needed = 153187
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:513)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:420)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:145)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1848)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:256)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:187)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:782)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:721)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:648)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11388) Allow ACID Compactor components to run in multiple metastores
[ https://issues.apache.org/jira/browse/HIVE-11388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-11388: -- Resolution: Fixed Fix Version/s: 2.1.0 1.3.0 Status: Resolved (was: Patch Available) > Allow ACID Compactor components to run in multiple metastores > - > > Key: HIVE-11388 > URL: https://issues.apache.org/jira/browse/HIVE-11388 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Critical > Fix For: 1.3.0, 2.1.0 > > Attachments: HIVE-11388.2.patch, HIVE-11388.4.patch, > HIVE-11388.5.patch, HIVE-11388.6.patch, HIVE-11388.7.patch, > HIVE-11388.branch-1.patch, HIVE-11388.patch > > > (this description is no loner accurate; see further comments) > org.apache.hadoop.hive.ql.txn.compactor.Initiator is a thread that runs > inside the metastore service to manage compactions of ACID tables. There > should be exactly 1 instance of this thread (even with multiple Thrift > services). > This is documented in > https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration > but not enforced. > Should add enforcement, since more than 1 Initiator could cause concurrent > attempts to compact the same table/partition - which will not work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13360) Refactoring Hive Authorization
[ https://issues.apache.org/jira/browse/HIVE-13360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-13360: --- Attachment: HIVE-13360.01.patch [~ashutoshc], could you please take a look? Thanks. > Refactoring Hive Authorization > -- > > Key: HIVE-13360 > URL: https://issues.apache.org/jira/browse/HIVE-13360 > Project: Hive > Issue Type: Sub-task > Components: Security >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Fix For: 2.1.0 > > Attachments: HIVE-13360.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13360) Refactoring Hive Authorization
[ https://issues.apache.org/jira/browse/HIVE-13360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-13360: --- Status: Patch Available (was: Open) > Refactoring Hive Authorization > -- > > Key: HIVE-13360 > URL: https://issues.apache.org/jira/browse/HIVE-13360 > Project: Hive > Issue Type: Sub-task > Components: Security >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Fix For: 2.1.0 > > Attachments: HIVE-13360.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12992) Hive on tez: Bucket map join plan is incorrect
[
https://issues.apache.org/jira/browse/HIVE-12992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211222#comment-15211222
]
Jason Dere commented on HIVE-12992:
---
I believe the changes make sense - if one of the parent operators is a map join
then only the big table parent of that map join should be traversed.
Are the test failures related?
> Hive on tez: Bucket map join plan is incorrect
> --
>
> Key: HIVE-12992
> URL: https://issues.apache.org/jira/browse/HIVE-12992
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Affects Versions: 1.2.1, 2.0.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
> Labels: tez
> Attachments: HIVE-12992.1.patch
>
>
> TPCH Query 9 fails when bucket map join is enabled:
> {code}
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Reducer
> 5, vertexId=vertex_1450634494433_0007_2_06, diagnostics=[Exception in
> EdgeManager, vertex=vertex_1450634494433_0007_2_06 [Reducer 5], Fail to
> sendTezEventToDestinationTasks, event:DataMovementEvent [sourceIndex=0,
> targetIndex=-1, version=0], sourceInfo:{ producerConsumerType=OUTPUT,
> taskVertexName=Map 1, edgeVertexName=Reducer 5,
> taskAttemptId=attempt_1450634494433_0007_2_05_00_0 },
> destinationInfo:null, EdgeInfo: sourceVertexName=Map 1,
> destinationVertexName=Reducer 5, java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.CustomPartitionEdge.routeDataMovementEventToDestination(CustomPartitionEdge.java:88)
> at
> org.apache.tez.dag.app.dag.impl.Edge.sendTezEventToDestinationTasks(Edge.java:458)
> at
> org.apache.tez.dag.app.dag.impl.Edge.handleCompositeDataMovementEvent(Edge.java:386)
> at
> org.apache.tez.dag.app.dag.impl.Edge.sendTezEventToDestinationTasks(Edge.java:439)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handleRoutedTezEvents(VertexImpl.java:4382)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.access$4000(VertexImpl.java:202)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:4172)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:4164)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13008) WebHcat DDL commands in secure mode NPE when default FileSystem doesn't support delegation tokens
[
https://issues.apache.org/jira/browse/HIVE-13008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-13008:
--
Resolution: Fixed
Fix Version/s: 2.1.0
Status: Resolved (was: Patch Available)
> WebHcat DDL commands in secure mode NPE when default FileSystem doesn't
> support delegation tokens
> -
>
> Key: HIVE-13008
> URL: https://issues.apache.org/jira/browse/HIVE-13008
> Project: Hive
> Issue Type: Bug
> Components: WebHCat
>Affects Versions: 1.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Fix For: 2.1.0
>
> Attachments: HIVE-13008.patch
>
>
> {noformat}
> ERROR | 11 Jan 2016 20:19:02,781 |
> org.apache.hive.hcatalog.templeton.CatchallExceptionMapper |
> java.lang.NullPointerException
> at
> org.apache.hive.hcatalog.templeton.SecureProxySupport$2.run(SecureProxySupport.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hive.hcatalog.templeton.SecureProxySupport.writeProxyDelegationTokens(SecureProxySupport.java:168)
> at
> org.apache.hive.hcatalog.templeton.SecureProxySupport.open(SecureProxySupport.java:95)
> at
> org.apache.hive.hcatalog.templeton.HcatDelegator.run(HcatDelegator.java:63)
> at org.apache.hive.hcatalog.templeton.Server.ddl(Server.java:217)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1480)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1411)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1360)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1350)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:565)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1360)
> at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:615)
> at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:574)
> at org.apache.hadoop.hdfs.web.AuthFilter.doFilter(AuthFilter.java:88)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:477)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1031)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:965)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHa
[jira] [Updated] (HIVE-13362) Commit binary file required for HIVE-13361
[ https://issues.apache.org/jira/browse/HIVE-13362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-13362: - Attachment: alltypesorc3xcols > Commit binary file required for HIVE-13361 > -- > > Key: HIVE-13362 > URL: https://issues.apache.org/jira/browse/HIVE-13362 > Project: Hive > Issue Type: Sub-task >Affects Versions: 1.3.0, 2.0.0, 2.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: alltypesorc3xcols > > > Separate jira for committing orc binary file required for precommit test run > for HIVE-13361 > NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.3.4#6332)
