[jira] [Created] (HIVE-14560) Support exchange partition between s3 and hdfs tables
Abdullah Yousufi created HIVE-14560:
---
Summary: Support exchange partition between s3 and hdfs tables
Key: HIVE-14560
URL: https://issues.apache.org/jira/browse/HIVE-14560
Project: Hive
Issue Type: Bug
Reporter: Abdullah Yousufi
Assignee: Abdullah Yousufi
Fix For: 2.2.0
{code}
alter table s3_tbl exchange partition (country='USA', state='CA') with table
hdfs_tbl;
{code}
results in:
{code}
Error: Error while processing statement: FAILED: Execution Error, return code 1
from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got
exception: java.lang.IllegalArgumentException Wrong FS:
s3a://hive-on-s3/s3_tbl/country=USA/state=CA, expected: hdfs://localhost:9000)
(state=08S01,code=1)
{code}
because the check for whether the s3 destination table path exists occurs on
the hdfs filesystem.
Furthermore, exchanging between s3 to hdfs fails because the hdfs rename
operation is not supported across filesystems. Fix uses copy + deletion in the
case that the file systems differ.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-14165) Enable faster S3 Split Computation by listing files in blocks
Abdullah Yousufi created HIVE-14165: --- Summary: Enable faster S3 Split Computation by listing files in blocks Key: HIVE-14165 URL: https://issues.apache.org/jira/browse/HIVE-14165 Project: Hive Issue Type: Improvement Affects Versions: 2.1.0 Reporter: Abdullah Yousufi Assignee: Abdullah Yousufi During split computation when a large of files are required to be listed from S3 then instead of executing 1 API call per file, one can optimize by listing 1000 files in each API call. Thereby reducing the amount of time required for listing files. Qubole has this optimization in place as detailed here: https://www.qubole.com/blog/product/optimizing-hadoop-for-s3-part-1/?nabe=5695374637924352:0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14149) S3A connector throws an AmazonS3Exception on Hadoop3.0 with Java1.8u60 and higher
Abdullah Yousufi created HIVE-14149:
---
Summary: S3A connector throws an AmazonS3Exception on Hadoop3.0
with Java1.8u60 and higher
Key: HIVE-14149
URL: https://issues.apache.org/jira/browse/HIVE-14149
Project: Hive
Issue Type: Bug
Components: Hive
Affects Versions: 2.1.0
Reporter: Abdullah Yousufi
Assignee: Abdullah Yousufi
Java1.8u60 and higher cause Joda Time 2.5 to incorrectly format timezones,
which leads to the aws server rejecting requests with the aws sdk hadoop3.0
uses. This means any queries involving the s3a connector will return the
following AmazonS3Exception:
{code}
com.amazonaws.services.s3.model.AmazonS3Exception: AWS authentication requires
a valid Date or x-amz-date header
{code}
The fix for this is to update Joda Time from 2.5 to 2.8.1. See here for details:
https://github.com/aws/aws-sdk-java/issues/444
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-14074) RELOAD FUNCTION should update dropped functions
Abdullah Yousufi created HIVE-14074: --- Summary: RELOAD FUNCTION should update dropped functions Key: HIVE-14074 URL: https://issues.apache.org/jira/browse/HIVE-14074 Project: Hive Issue Type: Bug Affects Versions: 2.0.1 Reporter: Abdullah Yousufi Assignee: Abdullah Yousufi Fix For: 2.2.0 Due to HIVE-2573, functions are stored in a per-session registry and only loaded in from the metastore when hs2 or hive cli is started. Running RELOAD FUNCTION in the current session is a way to force a reload of the functions, so that changes that occurred in other running sessions will be reflected in the current session, without having to restart the current session. However, while functions that are created in other sessions will now appear in the current session, functions that have been dropped are not removed from the current session's registry. It seems inconsistent that created functions are updated while dropped functions are not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14049) Password prompt in Beeline is continuously printed
Abdullah Yousufi created HIVE-14049:
---
Summary: Password prompt in Beeline is continuously printed
Key: HIVE-14049
URL: https://issues.apache.org/jira/browse/HIVE-14049
Project: Hive
Issue Type: Bug
Components: Beeline
Affects Versions: 2.0.1
Reporter: Abdullah Yousufi
I'm experiencing this issue with a Mac, which was not occurring until recently.
{code}
Beeline version 2.2.0-SNAPSHOT by Apache Hive
beeline> !connect jdbc:hive2://localhost:1
Connecting to jdbc:hive2://localhost:1
Enter username for jdbc:hive2://localhost:1: hive
Enter password for jdbc:hive2://localhost:1:
Enter password for jdbc:hive2://localhost:1:
Enter password for jdbc:hive2://localhost:1:
...
{code}
The 'Enter password for jdbc:hive2://localhost:1:' line continues to print
until enter is hit. From looking at the code in Commands.java (lines
1413-1420), it's not quite clear why this happens on the second call to
readLine()) :
{code}
if (username == null) {
username = beeLine.getConsoleReader().readLine("Enter username for " +
url + ": ");
}
props.setProperty("user", username);
if (password == null) {
password = beeLine.getConsoleReader().readLine("Enter password for " +
url + ": ",
new Character('*'));
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-13987) Clarify current error shown when HS2 is down
Abdullah Yousufi created HIVE-13987:
---
Summary: Clarify current error shown when HS2 is down
Key: HIVE-13987
URL: https://issues.apache.org/jira/browse/HIVE-13987
Project: Hive
Issue Type: Bug
Components: Beeline
Affects Versions: 2.0.1
Reporter: Abdullah Yousufi
Assignee: Abdullah Yousufi
Priority: Minor
Fix For: 2.2.0
When HS2 is down and a query is run, the following error is shown in beeline:
{code}
0: jdbc:hive2://localhost:1> show tables;
Error: org.apache.thrift.transport.TTransportException (state=08S01,code=0)
{code}
It may be more helpful to also indicate that the reason for this is that HS2 is
down, such as:
{code}
0: jdbc:hive2://localhost:1> show tables;
HS2 may be unavailable, check server status
Error: org.apache.thrift.transport.TTransportException (state=08S01,code=0)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-13964) Add a parameter to beeline to allow a properties file to be passed in
Abdullah Yousufi created HIVE-13964: --- Summary: Add a parameter to beeline to allow a properties file to be passed in Key: HIVE-13964 URL: https://issues.apache.org/jira/browse/HIVE-13964 Project: Hive Issue Type: New Feature Components: Beeline Reporter: Abdullah Yousufi Assignee: Abdullah Yousufi Priority: Minor HIVE-6652 removed the ability to pass in a properties file as a beeline parameter. It may be a useful feature to be able to pass the file in is a parameter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
