[jira] [Updated] (HIVE-8206) Merge from trunk to spark 9/20/14
[ https://issues.apache.org/jira/browse/HIVE-8206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-8206: --- Attachment: HIVE-8206-spark.patch Merge from trunk to spark 9/20/14 - Key: HIVE-8206 URL: https://issues.apache.org/jira/browse/HIVE-8206 Project: Hive Issue Type: Sub-task Components: Spark Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-8206-spark.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8206) Merge from trunk to spark 9/20/14
[ https://issues.apache.org/jira/browse/HIVE-8206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-8206: --- Attachment: (was: HIVE-8206.patch) Merge from trunk to spark 9/20/14 - Key: HIVE-8206 URL: https://issues.apache.org/jira/browse/HIVE-8206 Project: Hive Issue Type: Sub-task Components: Spark Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-8206-spark.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8083) Authorization DDLs should not enforce hive identifier syntax for user or group
[ https://issues.apache.org/jira/browse/HIVE-8083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141798#comment-14141798 ] Lefty Leverenz commented on HIVE-8083: -- Does this need user doc? If so, it probably belongs here: * [Hive Default Authorization (Legacy Mode) -- Names of Users and Roles | https://cwiki.apache.org/confluence/display/Hive/Hive+Default+Authorization+-+Legacy+Mode#HiveDefaultAuthorization-LegacyMode-NamesofUsersandRoles] * [SQL Standard Based Hive Authorization -- Names of Users and Roles | https://cwiki.apache.org/confluence/display/Hive/SQL+Standard+Based+Hive+Authorization#SQLStandardBasedHiveAuthorization-NamesofUsersandRoles] A release note would also be helpful. Authorization DDLs should not enforce hive identifier syntax for user or group -- Key: HIVE-8083 URL: https://issues.apache.org/jira/browse/HIVE-8083 Project: Hive Issue Type: Bug Components: SQL, SQLStandardAuthorization Affects Versions: 0.13.0, 0.13.1 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Fix For: 0.14.0 Attachments: HIVE-8083.1.patch, HIVE-8083.2.patch, HIVE-8083.3.patch The compiler expects principals (user, group and role) as hive identifiers for authorization DDLs. The user and group are entities that belong to external namespace and we can't expect those to follow hive identifier syntax rules. For example, a userid or group can contain '-' which is not allowed by compiler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7790) Update privileges to check for update and delete
[ https://issues.apache.org/jira/browse/HIVE-7790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141806#comment-14141806 ] Lefty Leverenz commented on HIVE-7790: -- Does this need to be documented in SQL Standard Based Hive Authorization? (What about Hive Default Authorization?) * [SQL Standard Based Hive Authorization -- Privileges Required for Hive Operations | https://cwiki.apache.org/confluence/display/Hive/SQL+Standard+Based+Hive+Authorization#SQLStandardBasedHiveAuthorization-PrivilegesRequiredforHiveOperations] * [Hive Default Authorization (Legacy Mode) -- Hive Operations and Required Privileges | https://cwiki.apache.org/confluence/display/Hive/Hive+Default+Authorization+-+Legacy+Mode#HiveDefaultAuthorization-LegacyMode-HiveOperationsandRequiredPrivileges] Update privileges to check for update and delete Key: HIVE-7790 URL: https://issues.apache.org/jira/browse/HIVE-7790 Project: Hive Issue Type: Sub-task Reporter: Alan Gates Assignee: Alan Gates Fix For: 0.14.0 Attachments: HIVE-7790.2.patch, HIVE-7790.3.patch, HIVE-7790.patch In the new SQLStdAuth scheme, we need to add UPDATE and DELETE as operations and add ability check for them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8189) A select statement with a subquery is failing with HBaseSerde
[
https://issues.apache.org/jira/browse/HIVE-8189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141814#comment-14141814
]
Hive QA commented on HIVE-8189:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12670041/HIVE-8189.1.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6299 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.parse.TestParse.testParse_union
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/889/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/889/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-889/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12670041
A select statement with a subquery is failing with HBaseSerde
-
Key: HIVE-8189
URL: https://issues.apache.org/jira/browse/HIVE-8189
Project: Hive
Issue Type: Bug
Components: HBase Handler
Affects Versions: 0.12.0, 0.13.1
Reporter: Yongzhi Chen
Assignee: Yongzhi Chen
Attachments: HIVE-8189.1.patch, hbase_ppd_join.q
Hive tables in the query are hbase tables, and the subquery is a join
statement.
When
set hive.optimize.ppd=true;
and
set hive.auto.convert.join=false;
The query does not return data.
While hive.optimize.ppd=true and hive.auto.convert.join=true return values
back. See attached query file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7892) Thrift Set type not working with Hive
[
https://issues.apache.org/jira/browse/HIVE-7892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141818#comment-14141818
]
Lefty Leverenz commented on HIVE-7892:
--
Should this be documented in the wiki?
We don't have a wikidoc for the Thrift SerDe (but probably should). Here are
some places it is mentioned:
* [Developer Guide -- Hive SerDe |
https://cwiki.apache.org/confluence/display/Hive/DeveloperGuide#DeveloperGuide-HiveSerDe]
* [SerDe -- Built-in SerDes |
https://cwiki.apache.org/confluence/display/Hive/SerDe#SerDe-Built-inSerDes]
Hive complex types are documented here:
* [Hive Data Types -- Complex Types |
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types#LanguageManualTypes-ComplexTypes]
Thrift Set type not working with Hive
-
Key: HIVE-7892
URL: https://issues.apache.org/jira/browse/HIVE-7892
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Reporter: Satish Mittal
Assignee: Satish Mittal
Fix For: 0.14.0
Attachments: HIVE-7892.1.patch, HIVE-7892.patch.txt
Thrift supports List, Map and Struct complex types, which get mapped to
Array, Map and Struct complex types in Hive respectively. However thrift Set
type doesn't seem to be working.
Here is an example thrift struct:
{noformat}
namespace java sample.thrift
struct setrow {
1: required seti32 ids,
2: required string name,
}
{noformat}
A Hive table is created with ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer' WITH
SERDEPROPERTIES ('serialization.class'='sample.thrift.setrow',
'serialization.format'='org.apache.thrift.protocol.TBinaryProtocol').
Describing the table shows:
{noformat}
hive describe settable;
OK
ids structfrom deserializer
namestringfrom deserializer
{noformat}
Issuing a select query on set column throws SemanticException:
{noformat}
hive select ids from settable;
FAILED: SemanticException java.lang.IllegalArgumentException: Error: name
expected at the position 7 of 'struct' but '' is found.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8206) Merge from trunk to spark 9/20/14
[
https://issues.apache.org/jira/browse/HIVE-8206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141826#comment-14141826
]
Hive QA commented on HIVE-8206:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12670187/HIVE-8206-spark.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 6457 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample_islocalmode_hook
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_fs_default_name2
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/143/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/143/console
Test logs:
http://ec2-54-176-176-199.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-143/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12670187
Merge from trunk to spark 9/20/14
-
Key: HIVE-8206
URL: https://issues.apache.org/jira/browse/HIVE-8206
Project: Hive
Issue Type: Sub-task
Components: Spark
Reporter: Brock Noland
Assignee: Brock Noland
Attachments: HIVE-8206-spark.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7615) Beeline should have an option for user to see the query progress
[
https://issues.apache.org/jira/browse/HIVE-7615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141866#comment-14141866
]
Thejas M Nair commented on HIVE-7615:
-
Thanks for the new patch and pointing out the issue with just having a single
isRunning boolean.
I have some more comments/thoughts -
# I think we should avoid throwing exceptions in the normal code path, as Brock
pointed out. (getQueryLog throwing exception when statementHandle is not
initialized)
# getQueryLog should throw an exception when the Statement is cancelled or
closed. I think this is the state that needs to be captured (canceled/closed vs
pre-initalization). For the getQueryLog api, it not matter if the query
succeeded or failed.
# It would be useful to have a way to determine if there are not going to be
any more logs being logged. In current implementation, once execute call
returns, the execution is over and all logs have been written. The user can
stop making calls at that point.
# The current code does not guarantee that the last few lines of logs (which
indicate success) would be picked up. The logging thread could be sleeping
while query completes and interrupt might happen before it is able to make
another getQueryLog. This can be confusing to a beeline user.
# HiveQueryResultSet does not lock calls to client using the transportLock .
This means that the getQueryLog and
HiveQueryResultSet.next might end up using the client object at same time,
causing problems.
{code}
/**
* This method is a public API for usage outside of Hive, although it is not
part of the
* interface java.sql.Statement.
* @return true if query execution might be producing more logs. It does not
indicate if last log lines have been fetched by getQueryLog.
* @throws ClosedOrCancelledStatement if statement has been cancelled or closed
*/
boolean hasMoreLogs() throws ClosedOrCancelledStatement;
/**
* Get the execution logs of the given SQL statement.
* This method is a public API for usage outside of Hive, although it is not
part of the
* interface java.sql.Statement.
* @param incremental indicate getting logs either incrementally or from the
beginning,
*when it is true or false.
* @param fetchSize the number of lines to fetch
* @return a list of log messages. It can be empty if there are no new logs
to be retrieved at that time.
* @throws ClosedOrCancelledStatement if statement has been cancelled or
closed
* @throws SQLException
*/
public ListString getQueryLog(boolean incremental, int fetchSize) throws
ClosedOrCancelledStatement, SQLException
{code}
The code for retrieving the logs can simply be -
{code}
new Runnable () {
public void run() {
while(stmt.hasMoreLogs()) {
printProgress(stmt.getQueryLog(true, 50));
try {
Thread.sleep(1000);
} catch(InterruptedException e) {
return;
}
}
}
{code}
The mainline code using jdbc can on the lines of -
{code}
stmt.execute();
// get results
// before closing statement, interrupt the thread.
logThread.interrupt();
// get any last lines of log synchronously before closing statement
stmt.close(); // or resultSet.close()
{code}
volatile boolean closedStmt = false; // set this to true when
HiveStatement.closeClientOperation is called
volatile boolean isLogBeingGenerated = true; // set this to false when the
while (!operationComplete) { loop is complete, or an exception is thrown in
that loop.
in getQueryLog, the check that is there for stmtHandle == null can be changed
to throw ClosedOrCancelledStatement only if stmtHandle == null closedStmt
hasMoreLogs can return the value of isLogBeingGenerated
Sorry about the large number of comments. Creating a public api is a big
commitment, just trying to make sure it is possible to stay committed! Thats
for all the work you have done.
Beeline should have an option for user to see the query progress
Key: HIVE-7615
URL: https://issues.apache.org/jira/browse/HIVE-7615
Project: Hive
Issue Type: Improvement
Components: CLI
Reporter: Dong Chen
Assignee: Dong Chen
Attachments: HIVE-7615.1.patch, HIVE-7615.2.patch, HIVE-7615.patch,
complete_logs, simple_logs
When executing query in Beeline, user should have a option to see the
progress through the outputs.
Beeline could use the API introduced in HIVE-4629 to get and display the logs
to the client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7892) Thrift Set type not working with Hive
[
https://issues.apache.org/jira/browse/HIVE-7892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141865#comment-14141865
]
Satish Mittal commented on HIVE-7892:
-
[~leftylev] The following wiki briefly mentions various available Hive Serdes:
* [Developer Guide -- Hive SerDe |
https://cwiki.apache.org/confluence/display/Hive/DeveloperGuide#DeveloperGuide-HiveSerDe
]
We can probably mention against ThriftSerde that Thrift complex types get
mapped to Hive complex types (List-Array, Map-, Struct - Struct
respectively). With HIVE-7892, Thrift Set type gets mapped to Hive Array type.
User can setup a Thrift based table using ThriftDeserializer class and query
various Thrift fields through Hive query.
Thrift Set type not working with Hive
-
Key: HIVE-7892
URL: https://issues.apache.org/jira/browse/HIVE-7892
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Reporter: Satish Mittal
Assignee: Satish Mittal
Fix For: 0.14.0
Attachments: HIVE-7892.1.patch, HIVE-7892.patch.txt
Thrift supports List, Map and Struct complex types, which get mapped to
Array, Map and Struct complex types in Hive respectively. However thrift Set
type doesn't seem to be working.
Here is an example thrift struct:
{noformat}
namespace java sample.thrift
struct setrow {
1: required seti32 ids,
2: required string name,
}
{noformat}
A Hive table is created with ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer' WITH
SERDEPROPERTIES ('serialization.class'='sample.thrift.setrow',
'serialization.format'='org.apache.thrift.protocol.TBinaryProtocol').
Describing the table shows:
{noformat}
hive describe settable;
OK
ids structfrom deserializer
namestringfrom deserializer
{noformat}
Issuing a select query on set column throws SemanticException:
{noformat}
hive select ids from settable;
FAILED: SemanticException java.lang.IllegalArgumentException: Error: name
expected at the position 7 of 'struct' but '' is found.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 25575: HIVE-7615: Beeline should have an option for user to see the query progress
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25575/#review54058
---
beeline/src/java/org/apache/hive/beeline/Commands.java
https://reviews.apache.org/r/25575/#comment94024
It would be better to call logThread.interrupt() before stmt.close(). That
way getLog is less likely to get called when the handle is null.
I think we should also synchronously get any remaining logs before closing
the statement. That will ensure any last remaining log lines have been fetched.
beeline/src/java/org/apache/hive/beeline/Commands.java
https://reviews.apache.org/r/25575/#comment94023
I agree with Brock. It would be better to just return empty list in this
case the query has not started. This can happen in normal operation.
For the case where statement has been closed/cancelled, I think it makes
sense to throw a distinct exception. Say a 'ClosedOrCancelledStatement extends
SQLException'.
I think we should throw the exception irrespective of the query suceeding
or failing. QUery succeeding or failing is not relavent for the getLog api.
itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcDriver2.java
https://reviews.apache.org/r/25575/#comment94029
synchronously getting any last few lines would make this test case more
robust. Otherwhise, it is possible that test fails on some slow virtual
machines.
jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
https://reviews.apache.org/r/25575/#comment93996
Brock,
That order that is better is highly subjective IMO.
For me this is more natural
if(valid common case) {
} else {
}
For me if( not null) is actually checking for a valid case, and seems
more natural to me.
jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
https://reviews.apache.org/r/25575/#comment94003
In this case, the if-else looks quite readable to me.
Should we be removing else where ever possible ?
In my opinion, we can leave such very subjective options to the author, as
long as it does not violate the coding standards of hive/oracle-java (or
another well known coding guildline that is compatible with Hive's), and does
not deviate from the style followed in hive or locally in that class.
- Thejas Nair
On Sept. 19, 2014, 9:22 a.m., Dong Chen wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25575/
---
(Updated Sept. 19, 2014, 9:22 a.m.)
Review request for hive.
Repository: hive-git
Description
---
When executing query in Beeline, user should have a option to see the
progress through the outputs. Beeline could use the API introduced in
HIVE-4629 to get and display the logs to the client.
Diffs
-
beeline/pom.xml 45fa02b
beeline/src/java/org/apache/hive/beeline/Commands.java a92d69f
itests/hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java
1e66542
itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcDriver2.java
daf8e9e
jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java 2cbf58c
Diff: https://reviews.apache.org/r/25575/diff/
Testing
---
UT passed.
Thanks,
Dong Chen
[jira] [Commented] (HIVE-7883) DBTxnManager trying to close already closed metastore client connection
[
https://issues.apache.org/jira/browse/HIVE-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141873#comment-14141873
]
Hive QA commented on HIVE-7883:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12670057/HIVE-7883.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 6297 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority
org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/890/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/890/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-890/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12670057
DBTxnManager trying to close already closed metastore client connection
---
Key: HIVE-7883
URL: https://issues.apache.org/jira/browse/HIVE-7883
Project: Hive
Issue Type: Bug
Components: Metastore, Transactions
Affects Versions: 0.14.0
Reporter: Mostafa Mokhtar
Assignee: Alan Gates
Attachments: HIVE-7883.patch
You will find following log message :
{code}
ERROR hive.metastore: Unable to shutdown local metastore client
org.apache.thrift.transport.TTransportException: Cannot write to null
outputStream
at
org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:142)
at
org.apache.thrift.protocol.TBinaryProtocol.writeI32(TBinaryProtocol.java:163)
at
org.apache.thrift.protocol.TBinaryProtocol.writeMessageBegin(TBinaryProtocol.java:91)
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62)
at
com.facebook.fb303.FacebookService$Client.send_shutdown(FacebookService.java:431)
at
com.facebook.fb303.FacebookService$Client.shutdown(FacebookService.java:425)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:435)
at
org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.destruct(DbTxnManager.java:304)
at
org.apache.hadoop.hive.ql.lockmgr.HiveTxnManagerImpl.finalize(HiveTxnManagerImpl.java:44)
at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:101)
at java.lang.ref.Finalizer.access$100(Finalizer.java:32)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:190)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-7482) The execution side changes for SMB join in hive-tez
[ https://issues.apache.org/jira/browse/HIVE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-7482: - Attachment: HIVE-7482.7.patch Fix failing tests and address review comments. The execution side changes for SMB join in hive-tez --- Key: HIVE-7482 URL: https://issues.apache.org/jira/browse/HIVE-7482 Project: Hive Issue Type: Bug Components: Tez Affects Versions: tez-branch Reporter: Vikram Dixit K Assignee: Vikram Dixit K Attachments: HIVE-7482.1.patch, HIVE-7482.2.patch, HIVE-7482.3.patch, HIVE-7482.4.patch, HIVE-7482.5.patch, HIVE-7482.6.patch, HIVE-7482.7.patch, HIVE-7482.WIP.2.patch, HIVE-7482.WIP.3.patch, HIVE-7482.WIP.4.patch, HIVE-7482.WIP.patch A piece of HIVE-7430. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-7482) The execution side changes for SMB join in hive-tez
[ https://issues.apache.org/jira/browse/HIVE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-7482: - Attachment: HIVE-7482.8.patch Removed unnecessary logging from previous version. The execution side changes for SMB join in hive-tez --- Key: HIVE-7482 URL: https://issues.apache.org/jira/browse/HIVE-7482 Project: Hive Issue Type: Bug Components: Tez Affects Versions: tez-branch Reporter: Vikram Dixit K Assignee: Vikram Dixit K Attachments: HIVE-7482.1.patch, HIVE-7482.2.patch, HIVE-7482.3.patch, HIVE-7482.4.patch, HIVE-7482.5.patch, HIVE-7482.6.patch, HIVE-7482.7.patch, HIVE-7482.8.patch, HIVE-7482.WIP.2.patch, HIVE-7482.WIP.3.patch, HIVE-7482.WIP.4.patch, HIVE-7482.WIP.patch A piece of HIVE-7430. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8184) inconsistence between colList and columnExprMap when ConstantPropagate is applied to subquery
[
https://issues.apache.org/jira/browse/HIVE-8184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141901#comment-14141901
]
Hive QA commented on HIVE-8184:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12670064/HIVE-8184.2.patch
{color:red}ERROR:{color} -1 due to 34 failed/errored test(s), 6298 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_select
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_view
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_lateral_view_ppd
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_oneskew_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_repeated_alias
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_notin
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_notin_having
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_unqualcolumnrefs
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_views
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_case
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_elt
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_when
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_elt
org.apache.hadoop.hive.ql.parse.TestParse.testParse_cast1
org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby1
org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby2
org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby3
org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby4
org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby5
org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby6
org.apache.hadoop.hive.ql.parse.TestParse.testParse_join8
org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf1
org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf4
org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf6
org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf_case
org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf_when
org.apache.hadoop.hive.ql.parse.TestParse.testParse_union
org.apache.hive.hcatalog.streaming.TestStreaming.testInterleavedTransactionBatchCommits
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/891/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/891/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-891/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 34 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12670064
inconsistence between colList and columnExprMap when ConstantPropagate is
applied to subquery
--
Key: HIVE-8184
URL: https://issues.apache.org/jira/browse/HIVE-8184
Project: Hive
Issue Type: Improvement
Reporter: Pengcheng Xiong
Priority: Minor
Attachments: HIVE-8184.1.patch, HIVE-8184.2.patch
Query like
select * from (select a.key as ak, a.value as av, b.key as bk, b.value as bv
from src a join src1 b where a.key = '428' ) c;
will fail as
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8191) Update and delete on tables with non Acid output formats gives runtime error
[
https://issues.apache.org/jira/browse/HIVE-8191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141924#comment-14141924
]
Hive QA commented on HIVE-8191:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12670080/HIVE-8191.patch
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 6300 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.plan.TestReadEntityDirect.testSelectEntityDirect
org.apache.hadoop.hive.ql.plan.TestReadEntityDirect.testSelectEntityInDirect
org.apache.hadoop.hive.ql.plan.TestReadEntityDirect.testSelectEntityInDirectJoinAlias
org.apache.hadoop.hive.ql.plan.TestReadEntityDirect.testSelectEntityViewDirectJoin
org.apache.hadoop.hive.ql.plan.TestReadEntityDirect.testSelectEntityViewDirectUnion
org.apache.hive.beeline.TestBeeLineWithArgs.testNullDefault
org.apache.hive.beeline.TestBeeLineWithArgs.testNullEmpty
org.apache.hive.beeline.TestBeeLineWithArgs.testNullEmptyCmdArg
org.apache.hive.beeline.TestBeeLineWithArgs.testNullNonEmpty
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/892/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/892/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-892/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12670080
Update and delete on tables with non Acid output formats gives runtime error
Key: HIVE-8191
URL: https://issues.apache.org/jira/browse/HIVE-8191
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.14.0
Reporter: Alan Gates
Assignee: Alan Gates
Priority: Critical
Attachments: HIVE-8191.patch
{code}
create table not_an_acid_table(a int, b varchar(128));
insert into table not_an_acid_table select cint, cast(cstring1 as
varchar(128)) from alltypesorc where cint is not null order by cint limit 10;
delete from not_an_acid_table where b = '0ruyd6Y50JpdGRf6HqD';
{code}
This generates a runtime error. It should get a compile error instead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8083) Authorization DDLs should not enforce hive identifier syntax for user or group
[ https://issues.apache.org/jira/browse/HIVE-8083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-8083: -- Labels: TODOC14 (was: ) Authorization DDLs should not enforce hive identifier syntax for user or group -- Key: HIVE-8083 URL: https://issues.apache.org/jira/browse/HIVE-8083 Project: Hive Issue Type: Bug Components: SQL, SQLStandardAuthorization Affects Versions: 0.13.0, 0.13.1 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Labels: TODOC14 Fix For: 0.14.0 Attachments: HIVE-8083.1.patch, HIVE-8083.2.patch, HIVE-8083.3.patch The compiler expects principals (user, group and role) as hive identifiers for authorization DDLs. The user and group are entities that belong to external namespace and we can't expect those to follow hive identifier syntax rules. For example, a userid or group can contain '-' which is not allowed by compiler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8083) Authorization DDLs should not enforce hive identifier syntax for user or group
[ https://issues.apache.org/jira/browse/HIVE-8083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141988#comment-14141988 ] Xuefu Zhang commented on HIVE-8083: --- Thanks, Lefty. It does seem that this has doc impact, especially regarding hive.support.quoted.identifiers. [~prasadm], could you please comment on this? Authorization DDLs should not enforce hive identifier syntax for user or group -- Key: HIVE-8083 URL: https://issues.apache.org/jira/browse/HIVE-8083 Project: Hive Issue Type: Bug Components: SQL, SQLStandardAuthorization Affects Versions: 0.13.0, 0.13.1 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Labels: TODOC14 Fix For: 0.14.0 Attachments: HIVE-8083.1.patch, HIVE-8083.2.patch, HIVE-8083.3.patch The compiler expects principals (user, group and role) as hive identifiers for authorization DDLs. The user and group are entities that belong to external namespace and we can't expect those to follow hive identifier syntax rules. For example, a userid or group can contain '-' which is not allowed by compiler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-7674) Update to Spark 1.2 [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-7674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-7674: -- Description: In HIVE-8160 we added a custom repo to use Spark 1.2. Once 1.2is released we need to remove this repo. (was: In HIVE-7540 we added a custom repo to use Spark 1.1. Once 1.1 is released we need to remove this repo.) Summary: Update to Spark 1.2 [Spark Branch] (was: Update to Spark 1.1 [Spark Branch]) Updated the JIRA to reflect de status quo. Update to Spark 1.2 [Spark Branch] -- Key: HIVE-7674 URL: https://issues.apache.org/jira/browse/HIVE-7674 Project: Hive Issue Type: Sub-task Components: Spark Reporter: Brock Noland Priority: Blocker In HIVE-8160 we added a custom repo to use Spark 1.2. Once 1.2is released we need to remove this repo. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-7674) Update to Spark 1.2 [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-7674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-7674: -- Description: In HIVE-8160 we added a custom repo to use Spark 1.2. Once 1.2 is released we need to remove this repo. (was: In HIVE-8160 we added a custom repo to use Spark 1.2. Once 1.2is released we need to remove this repo.) Update to Spark 1.2 [Spark Branch] -- Key: HIVE-7674 URL: https://issues.apache.org/jira/browse/HIVE-7674 Project: Hive Issue Type: Sub-task Components: Spark Reporter: Brock Noland Priority: Blocker In HIVE-8160 we added a custom repo to use Spark 1.2. Once 1.2 is released we need to remove this repo. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8105) booleans and nulls not handled properly in insert/values
[
https://issues.apache.org/jira/browse/HIVE-8105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141998#comment-14141998
]
Hive QA commented on HIVE-8105:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12670137/HIVE-8105.3.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6298 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.parse.TestParse.testParse_union
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/894/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/894/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-894/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12670137
booleans and nulls not handled properly in insert/values
Key: HIVE-8105
URL: https://issues.apache.org/jira/browse/HIVE-8105
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.14.0
Reporter: Alan Gates
Assignee: Alan Gates
Priority: Critical
Attachments: HIVE-8105.2.patch, HIVE-8105.2.patch, HIVE-8105.3.patch,
HIVE-8105.patch
Doing an insert/values with a boolean always results in a value of true,
regardless of whether true or false is given in the query.
Doing an insert/values with a null for a column value results in a semantic
error.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7100) Users of hive should be able to specify skipTrash when dropping tables.
[
https://issues.apache.org/jira/browse/HIVE-7100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142002#comment-14142002
]
Xuefu Zhang commented on HIVE-7100:
---
{quote}
What should the behavior of drop table be for an immutable table? With and
without the purge option?
{quote}
There should be no difference in dropping table behavior regarding immutable
tables, accordingly HIVE-6406. Being immutable doesn't prevents droping. It
only blocks updating when content exisits.
Purge is an option for dropping, Thus, there shouldn't be any relationship
between immutable and purge.
Users of hive should be able to specify skipTrash when dropping tables.
---
Key: HIVE-7100
URL: https://issues.apache.org/jira/browse/HIVE-7100
Project: Hive
Issue Type: Improvement
Affects Versions: 0.13.0
Reporter: Ravi Prakash
Assignee: david serafini
Attachments: HIVE-7100.1.patch, HIVE-7100.10.patch,
HIVE-7100.2.patch, HIVE-7100.3.patch, HIVE-7100.4.patch, HIVE-7100.5.patch,
HIVE-7100.8.patch, HIVE-7100.9.patch, HIVE-7100.patch
Users of our clusters are often running up against their quota limits because
of Hive tables. When they drop tables, they have to then manually delete the
files from HDFS using skipTrash. This is cumbersome and unnecessary. We
should enable users to skipTrash directly when dropping tables.
We should also be able to provide this functionality without polluting SQL
syntax.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7100) Users of hive should be able to specify skipTrash when dropping tables.
[ https://issues.apache.org/jira/browse/HIVE-7100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142003#comment-14142003 ] Xuefu Zhang commented on HIVE-7100: --- [~dbsalti] The patch needs to be rebased as it doesn't apply to latest trunk any more. Users of hive should be able to specify skipTrash when dropping tables. --- Key: HIVE-7100 URL: https://issues.apache.org/jira/browse/HIVE-7100 Project: Hive Issue Type: Improvement Affects Versions: 0.13.0 Reporter: Ravi Prakash Assignee: david serafini Attachments: HIVE-7100.1.patch, HIVE-7100.10.patch, HIVE-7100.2.patch, HIVE-7100.3.patch, HIVE-7100.4.patch, HIVE-7100.5.patch, HIVE-7100.8.patch, HIVE-7100.9.patch, HIVE-7100.patch Users of our clusters are often running up against their quota limits because of Hive tables. When they drop tables, they have to then manually delete the files from HDFS using skipTrash. This is cumbersome and unnecessary. We should enable users to skipTrash directly when dropping tables. We should also be able to provide this functionality without polluting SQL syntax. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8105) booleans and nulls not handled properly in insert/values
[ https://issues.apache.org/jira/browse/HIVE-8105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-8105: - Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Patch 3 committed, with commented as requested by Eugene. Thanks Eugene for the review. booleans and nulls not handled properly in insert/values Key: HIVE-8105 URL: https://issues.apache.org/jira/browse/HIVE-8105 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.14.0 Reporter: Alan Gates Assignee: Alan Gates Priority: Critical Fix For: 0.14.0 Attachments: HIVE-8105.2.patch, HIVE-8105.2.patch, HIVE-8105.3.patch, HIVE-8105.patch Doing an insert/values with a boolean always results in a value of true, regardless of whether true or false is given in the query. Doing an insert/values with a null for a column value results in a semantic error. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Hive-branch-0.12-hadoop2 - Build # 35 - Still Failing
Changes for Build #26 Changes for Build #27 Changes for Build #28 Changes for Build #29 Changes for Build #30 Changes for Build #31 Changes for Build #32 Changes for Build #33 [daijy] PIG-4119: Add message at end of each testcase with timestamp in Pig system tests Changes for Build #34 Changes for Build #35 [rohini] PIG-4186: Fix e2e run against new build of pig and some enhancements (rohini) 54 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket4 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestMinimrCliDriver.runTest(TestMinimrCliDriver.java:280) at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket4(TestMinimrCliDriver.java:133) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) REGRESSION: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket5 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestMinimrCliDriver.runTest(TestMinimrCliDriver.java:280) at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket5(TestMinimrCliDriver.java:137) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) REGRESSION: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestMinimrCliDriver.runTest(TestMinimrCliDriver.java:280) at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers(TestMinimrCliDriver.java:141) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at
[jira] [Commented] (HIVE-5317) Implement insert, update, and delete in Hive with full ACID support
[ https://issues.apache.org/jira/browse/HIVE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142013#comment-14142013 ] Alan Gates commented on HIVE-5317: -- The discussion of when to branch for this has been going on on dev@hive.apache.org for a bit now, see http://mail-archives.apache.org/mod_mbox/hive-dev/201408.mbox/%3CCAKjA-pyhnHhxjaCYhWibX3o-RfQ7g2Sk9fyLYBN%3DFx6UofJ33A%40mail.gmail.com%3E The summary is, any day now. Once we branch it's usually ~4 weeks for stabilization and release mechanics before the release. Implement insert, update, and delete in Hive with full ACID support --- Key: HIVE-5317 URL: https://issues.apache.org/jira/browse/HIVE-5317 Project: Hive Issue Type: New Feature Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: InsertUpdatesinHive.pdf Many customers want to be able to insert, update and delete rows from Hive tables with full ACID support. The use cases are varied, but the form of the queries that should be supported are: * INSERT INTO tbl SELECT … * INSERT INTO tbl VALUES ... * UPDATE tbl SET … WHERE … * DELETE FROM tbl WHERE … * MERGE INTO tbl USING src ON … WHEN MATCHED THEN ... WHEN NOT MATCHED THEN ... * SET TRANSACTION LEVEL … * BEGIN/END TRANSACTION Use Cases * Once an hour, a set of inserts and updates (up to 500k rows) for various dimension tables (eg. customer, inventory, stores) needs to be processed. The dimension tables have primary keys and are typically bucketed and sorted on those keys. * Once a day a small set (up to 100k rows) of records need to be deleted for regulatory compliance. * Once an hour a log of transactions is exported from a RDBS and the fact tables need to be updated (up to 1m rows) to reflect the new data. The transactions are a combination of inserts, updates, and deletes. The table is partitioned and bucketed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8149) hive.optimize.reducededuplication should be set to false for IUD ops
[ https://issues.apache.org/jira/browse/HIVE-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-8149: - Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Test FileSinkOperator failures are unrelated and have been resolved by HIVE-6936. Patch checked in. Thank you Eugene for the review. hive.optimize.reducededuplication should be set to false for IUD ops Key: HIVE-8149 URL: https://issues.apache.org/jira/browse/HIVE-8149 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.14.0 Reporter: Eugene Koifman Assignee: Alan Gates Fix For: 0.14.0 Attachments: HIVE-8149.patch this optimizer causes both old and new rows to show up in a select after update (for tables involving few rows) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: hive unit test report question
In the past I have felt like failsafe was poorly implemented. This is shy I chose the approach we currently have. I don't remember the exact details and thus I am open to seeing what hive with failsafe looks like. FWIW - Hive tests take several days to execute outside of ptest and surefire will almost certainly run out of memory parsing the TEST XML files when run serially. On Sep 8, 2014 12:35 AM, Nick Dimiduk ndimi...@gmail.com wrote: IMHO, would be better to wire up the integration suite via failsafe plugin (surefire for IT) and link the modules correctly. This is on (admittedly, near the bottom of) my todo list. See also HBase poms for an example. -n On Saturday, September 6, 2014, wzc wzc1...@gmail.com wrote: hi all: I would like to create a jenkins job to run both hive ut and integration test. Right now it seems that I have to execute mulitple maven goals in different poms: mvn clean install surefire-report:report -Daggregate=true -Phadoop-2 cd itests mvn clean install surefire-report:report -Daggregate=true -Phadoop-2 I would like to use one maven jenkins job and right now I cant figure out how to configure job propery to execute maven goals in different poms (maybe I can add post-build step to execute another shell?). Each hive ptest2 job can run all tests and I would like to know the configure it use. Any help is appreciated. Thanks. 2014-01-14 14:05 GMT+08:00 Shanyu Zhao shz...@microsoft.com javascript:;: Thanks guys for your help! I found Eugene's comments are particularly helpful. With -Daggregate=true I now can see an aggregated unit test results. Btw, I didn't mean to run itests, I just want to run all unit tests. I think in the FAQ they made it clear that itests are disconnected from the top level pom.xml. Shanyu -Original Message- From: Eugene Koifman [mailto:ekoif...@hortonworks.com javascript:;] Sent: Monday, January 13, 2014 4:06 PM To: dev@hive.apache.org javascript:; Subject: Re: hive unit test report question I think you want to add -Daggregate=true you should then have target/site/surefire-report.html in the module where you ran the command On Mon, Jan 13, 2014 at 2:54 PM, Szehon Ho sze...@cloudera.com javascript:; wrote: Hi Shanyu, Are you running in /itests? The unit tests are in there, and are not run if you are running from the root. Thanks Szehon On Mon, Jan 13, 2014 at 1:59 PM, Shanyu Zhao shz...@microsoft.com javascript:; wrote: Hi, I was trying to build hive trunk, run all unit tests and generate reports, but I'm not sure what's the correct command line. I was using: mvn clean install -Phadoop-2 -DskipTests mvn test surefire-report:report -Phadoop-2 But the reports in the root folder and several other projects (such as metastore) are empty with no test results. And I couldn't find a summary page for all unit tests. I was trying to avoid mvn site because it seems to take forever to finish. Am I using the correct commands? How can I get a report like the one in the precommit report: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/827/testRep ort/ ? I really appreciate your help! Shanyu -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Assigned] (HIVE-8207) Add .q tests for multi-table insertion [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-8207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao reassigned HIVE-8207: -- Assignee: Chao Add .q tests for multi-table insertion [Spark Branch] - Key: HIVE-8207 URL: https://issues.apache.org/jira/browse/HIVE-8207 Project: Hive Issue Type: Test Components: Spark Reporter: Chao Assignee: Chao Now that multi-table insertion is committed to branch, we should enable those related qtests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-8207) Add .q tests for multi-table insertion [Spark Branch]
Chao created HIVE-8207: -- Summary: Add .q tests for multi-table insertion [Spark Branch] Key: HIVE-8207 URL: https://issues.apache.org/jira/browse/HIVE-8207 Project: Hive Issue Type: Test Components: Spark Reporter: Chao Now that multi-table insertion is committed to branch, we should enable those related qtests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8052) Vectorization: min() on TimeStamp datatype fails with error Vector aggregate not implemented: min for type: TIMESTAMP
[
https://issues.apache.org/jira/browse/HIVE-8052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142059#comment-14142059
]
Hive QA commented on HIVE-8052:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12670142/HIVE-8052.04.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6298 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.parse.TestParse.testParse_union
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/895/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/895/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-895/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12670142
Vectorization: min() on TimeStamp datatype fails with error Vector aggregate
not implemented: min for type: TIMESTAMP
---
Key: HIVE-8052
URL: https://issues.apache.org/jira/browse/HIVE-8052
Project: Hive
Issue Type: Bug
Reporter: Matt McCline
Assignee: Matt McCline
Priority: Critical
Attachments: HIVE-8052.01.patch, HIVE-8052.02.patch,
HIVE-8052.04.patch
Changes in HIVE-5760 to make explicit when timestamp and date can be
vectorized as Long were accidentally to strict for min, max, count, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8208) Multi-table insertion optimization #1: don't always break operator tree. [Spark Branch]
Chao created HIVE-8208: -- Summary: Multi-table insertion optimization #1: don't always break operator tree. [Spark Branch] Key: HIVE-8208 URL: https://issues.apache.org/jira/browse/HIVE-8208 Project: Hive Issue Type: Improvement Reporter: Chao Currently, with the current patch of multi-table insertion, it will break whenever there exists one TableScanOperator that can leads to multiple FileSinkOperators. Then, it identifies the lowest common ancestor (LCA), and breaks the tree there, creating same number of child SparkTasks as the number of FileSinkOperators. However, in the following situation it's better not to break the operator tree: Of all the paths from these FileSinkOperators to the LCA, if ReduceSinkOperator only exist in 0 or 1 path of them. In this case, we can do it in one spark job, and no need to break the operator tree. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-8209) Multi-table insertion optimization #2: use separate context
Chao created HIVE-8209:
--
Summary: Multi-table insertion optimization #2: use separate
context
Key: HIVE-8209
URL: https://issues.apache.org/jira/browse/HIVE-8209
Project: Hive
Issue Type: Improvement
Components: Spark
Reporter: Chao
Priority: Minor
Currently, the multi-table insertion patch uses {{GenSparkProcContext}} and
added some states of its own. It's better to use a separate context only for
the purpose of handling multi-table insertion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8094) add LIKE keyword support for SHOW FUNCTIONS
[ https://issues.apache.org/jira/browse/HIVE-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142080#comment-14142080 ] peter liu commented on HIVE-8094: - [~thejas] for review add LIKE keyword support for SHOW FUNCTIONS --- Key: HIVE-8094 URL: https://issues.apache.org/jira/browse/HIVE-8094 Project: Hive Issue Type: Improvement Affects Versions: 0.14.0, 0.13.1 Reporter: peter liu Assignee: peter liu Fix For: 0.14.0 Attachments: HIVE-8094.1.patch, HIVE-8094.2.patch It would be nice to add LIKE keyword support for SHOW FUNCTIONS as below, and keep the patterns consistent to the way as SHOW DATABASES, SHOW TABLES. bq. SHOW FUNCTIONS LIKE 'foo*'; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7842) load_dyn_part1.q fails with an assertion [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-7842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142085#comment-14142085
]
Chao commented on HIVE-7842:
With HIVE-7503, this test now passes.
load_dyn_part1.q fails with an assertion [Spark Branch]
---
Key: HIVE-7842
URL: https://issues.apache.org/jira/browse/HIVE-7842
Project: Hive
Issue Type: Sub-task
Components: Spark
Affects Versions: spark-branch
Reporter: Venki Korukanti
Assignee: Venki Korukanti
Labels: Spark-M1
Fix For: spark-branch
On spark branch, load_dyn_part1.q fails with following assertion. Looks like
SerDe is receiving invalid ByteWritable buffer.
{code}
java.lang.AssertionError
org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.deserialize(BinarySortableSerDe.java:205)
org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.deserialize(BinarySortableSerDe.java:187)
org.apache.hadoop.hive.ql.exec.spark.SparkReduceRecordHandler.processRow(SparkReduceRecordHandler.java:186)
org.apache.hadoop.hive.ql.exec.spark.HiveReduceFunctionResultList.processNextRecord(HiveReduceFunctionResultList.java:47)
org.apache.hadoop.hive.ql.exec.spark.HiveReduceFunctionResultList.processNextRecord(HiveReduceFunctionResultList.java:27)
org.apache.hadoop.hive.ql.exec.spark.HiveBaseFunctionResultList$ResultIterator.hasNext(HiveBaseFunctionResultList.java:98)
scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:41)
scala.collection.Iterator$class.foreach(Iterator.scala:727)
scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
org.apache.spark.rdd.RDD$$anonfun$foreach$1.apply(RDD.scala:759)
org.apache.spark.rdd.RDD$$anonfun$foreach$1.apply(RDD.scala:759)
org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1121)
org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1121)
org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:62)
org.apache.spark.scheduler.Task.run(Task.scala:54)
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:199)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:744)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8185) hive-jdbc-0.14.0-SNAPSHOT-standalone.jar fails verification for signatures in build
[
https://issues.apache.org/jira/browse/HIVE-8185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-8185:
---
Resolution: Fixed
Fix Version/s: 0.14.0
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks, Deepesh!
hive-jdbc-0.14.0-SNAPSHOT-standalone.jar fails verification for signatures in
build
---
Key: HIVE-8185
URL: https://issues.apache.org/jira/browse/HIVE-8185
Project: Hive
Issue Type: Bug
Components: JDBC
Affects Versions: 0.14.0
Reporter: Gopal V
Assignee: Deepesh Khandelwal
Priority: Critical
Fix For: 0.14.0
Attachments: HIVE-8185.1.patch, HIVE-8185.2.patch
In the current build, running
{code}
jarsigner --verify ./lib/hive-jdbc-0.14.0-SNAPSHOT-standalone.jar
Jar verification failed.
{code}
unless that jar is removed from the lib dir, all hive queries throw the
following error
{code}
Exception in thread main java.lang.SecurityException: Invalid signature
file digest for Manifest main attributes
at
sun.security.util.SignatureFileVerifier.processImpl(SignatureFileVerifier.java:240)
at
sun.security.util.SignatureFileVerifier.process(SignatureFileVerifier.java:193)
at java.util.jar.JarVerifier.processEntry(JarVerifier.java:305)
at java.util.jar.JarVerifier.update(JarVerifier.java:216)
at java.util.jar.JarFile.initializeVerifier(JarFile.java:345)
at java.util.jar.JarFile.getInputStream(JarFile.java:412)
at
sun.misc.URLClassPath$JarLoader$2.getInputStream(URLClassPath.java:775)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7946) CBO: Merge CBO changes to Trunk
[
https://issues.apache.org/jira/browse/HIVE-7946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142119#comment-14142119
]
Hive QA commented on HIVE-7946:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12670150/HIVE-7946.14.patch
{color:green}SUCCESS:{color} +1 6299 tests passed
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/896/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/896/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-896/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12670150
CBO: Merge CBO changes to Trunk
---
Key: HIVE-7946
URL: https://issues.apache.org/jira/browse/HIVE-7946
Project: Hive
Issue Type: Bug
Components: CBO
Reporter: Laljo John Pullokkaran
Assignee: Laljo John Pullokkaran
Attachments: HIVE-7946.1.patch, HIVE-7946.10.patch,
HIVE-7946.11.patch, HIVE-7946.12.patch, HIVE-7946.13.patch,
HIVE-7946.14.patch, HIVE-7946.2.patch, HIVE-7946.3.patch, HIVE-7946.4.patch,
HIVE-7946.5.patch, HIVE-7946.6.patch, HIVE-7946.7.patch, HIVE-7946.8.patch,
HIVE-7946.9.patch, HIVE-7946.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8201) Remove hardwiring to HiveInputFormat in acid qfile tests
[
https://issues.apache.org/jira/browse/HIVE-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142121#comment-14142121
]
Hive QA commented on HIVE-8201:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12670154/HIVE-8201.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/897/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/897/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-897/
Messages:
{noformat}
This message was trimmed, see log for full details
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as LPAREN LPAREN KW_MAP using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as LPAREN KW_NOT KW_IF using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as LPAREN KW_CASE KW_IF using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as LPAREN LPAREN KW_IF using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:115:5:
Decision can match input such as KW_CLUSTER KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:127:5:
Decision can match input such as KW_PARTITION KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:138:5:
Decision can match input such as KW_DISTRIBUTE KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:149:5:
Decision can match input such as KW_SORT KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:166:7:
Decision can match input such as STAR using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as KW_STRUCT using multiple alternatives: 4, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as KW_ARRAY using multiple alternatives: 2, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as KW_UNIONTYPE using multiple alternatives: 5,
6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as KW_NULL using multiple alternatives: 1, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as KW_TRUE using multiple alternatives: 3, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as KW_FALSE using multiple alternatives: 3, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as KW_DATE StringLiteral using multiple
alternatives: 2, 3
As a result, alternative(s) 3 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_ORDER
KW_BY using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_MAP LPAREN
using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_SORT KW_BY
using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT
KW_OVERWRITE using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_GROUP
KW_BY using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as KW_BETWEEN KW_MAP LPAREN
[jira] [Updated] (HIVE-8206) Merge from trunk to spark 9/20/14
[ https://issues.apache.org/jira/browse/HIVE-8206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-8206: --- Resolution: Fixed Fix Version/s: spark-branch Status: Resolved (was: Patch Available) Merge from trunk to spark 9/20/14 - Key: HIVE-8206 URL: https://issues.apache.org/jira/browse/HIVE-8206 Project: Hive Issue Type: Sub-task Components: Spark Reporter: Brock Noland Assignee: Brock Noland Fix For: spark-branch Attachments: HIVE-8206-spark.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-8188) ExprNodeGenericFuncEvaluator::_evaluate() loads class annotations in a tight loop
[ https://issues.apache.org/jira/browse/HIVE-8188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned HIVE-8188: - Assignee: Gopal V ExprNodeGenericFuncEvaluator::_evaluate() loads class annotations in a tight loop - Key: HIVE-8188 URL: https://issues.apache.org/jira/browse/HIVE-8188 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.14.0 Reporter: Gopal V Assignee: Gopal V Attachments: udf-deterministic.png When running a near-constant UDF, most of the CPU is burnt within the VM trying to read the class annotations for every row. !udf-deterministic.png! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8185) hive-jdbc-0.14.0-SNAPSHOT-standalone.jar fails verification for signatures in build
[
https://issues.apache.org/jira/browse/HIVE-8185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142158#comment-14142158
]
Deepesh Khandelwal commented on HIVE-8185:
--
Thanks Gopal and Ashutosh!
hive-jdbc-0.14.0-SNAPSHOT-standalone.jar fails verification for signatures in
build
---
Key: HIVE-8185
URL: https://issues.apache.org/jira/browse/HIVE-8185
Project: Hive
Issue Type: Bug
Components: JDBC
Affects Versions: 0.14.0
Reporter: Gopal V
Assignee: Deepesh Khandelwal
Priority: Critical
Fix For: 0.14.0
Attachments: HIVE-8185.1.patch, HIVE-8185.2.patch
In the current build, running
{code}
jarsigner --verify ./lib/hive-jdbc-0.14.0-SNAPSHOT-standalone.jar
Jar verification failed.
{code}
unless that jar is removed from the lib dir, all hive queries throw the
following error
{code}
Exception in thread main java.lang.SecurityException: Invalid signature
file digest for Manifest main attributes
at
sun.security.util.SignatureFileVerifier.processImpl(SignatureFileVerifier.java:240)
at
sun.security.util.SignatureFileVerifier.process(SignatureFileVerifier.java:193)
at java.util.jar.JarVerifier.processEntry(JarVerifier.java:305)
at java.util.jar.JarVerifier.update(JarVerifier.java:216)
at java.util.jar.JarFile.initializeVerifier(JarFile.java:345)
at java.util.jar.JarFile.getInputStream(JarFile.java:412)
at
sun.misc.URLClassPath$JarLoader$2.getInputStream(URLClassPath.java:775)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8179) Fetch task conversion: Remove some dependencies on AST
[
https://issues.apache.org/jira/browse/HIVE-8179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142159#comment-14142159
]
Hive QA commented on HIVE-8179:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12670156/HIVE-8179.3.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6298 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.parse.TestParse.testParse_union
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/898/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/898/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-898/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12670156
Fetch task conversion: Remove some dependencies on AST
--
Key: HIVE-8179
URL: https://issues.apache.org/jira/browse/HIVE-8179
Project: Hive
Issue Type: Bug
Reporter: Gunther Hagleitner
Assignee: Gunther Hagleitner
Attachments: HIVE-8179.1.patch, HIVE-8179.2.patch, HIVE-8179.3.patch
fetch task conversion is does some strange things:
For instance: select * from (select * from x) t, wont get converted even
though it's the exact same operator plan as: select * from x.
Or: select * from foo will get converted with minimal, but select list all
columns of foo from foo won't.
We also check the AST for group by etc, but then do the same thing in the
operator tree again.
I'm also wondering why we ship with moar as default, but test with
minimal in the unit tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8188) ExprNodeGenericFuncEvaluator::_evaluate() loads class annotations in a tight loop
[ https://issues.apache.org/jira/browse/HIVE-8188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8188: -- Labels: Performance (was: ) Status: Patch Available (was: Open) ExprNodeGenericFuncEvaluator::_evaluate() loads class annotations in a tight loop - Key: HIVE-8188 URL: https://issues.apache.org/jira/browse/HIVE-8188 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.14.0 Reporter: Gopal V Assignee: Gopal V Labels: Performance Attachments: HIVE-8188.1.patch, udf-deterministic.png When running a near-constant UDF, most of the CPU is burnt within the VM trying to read the class annotations for every row. !udf-deterministic.png! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8188) ExprNodeGenericFuncEvaluator::_evaluate() loads class annotations in a tight loop
[ https://issues.apache.org/jira/browse/HIVE-8188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8188: -- Attachment: HIVE-8188.1.patch ExprNodeGenericFuncEvaluator::_evaluate() loads class annotations in a tight loop - Key: HIVE-8188 URL: https://issues.apache.org/jira/browse/HIVE-8188 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.14.0 Reporter: Gopal V Assignee: Gopal V Labels: Performance Attachments: HIVE-8188.1.patch, udf-deterministic.png When running a near-constant UDF, most of the CPU is burnt within the VM trying to read the class annotations for every row. !udf-deterministic.png! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8188) ExprNodeGenericFuncEvaluator::_evaluate() loads class annotations in a tight loop
[ https://issues.apache.org/jira/browse/HIVE-8188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8188: -- Attachment: HIVE-8188.2.patch ExprNodeGenericFuncEvaluator::_evaluate() loads class annotations in a tight loop - Key: HIVE-8188 URL: https://issues.apache.org/jira/browse/HIVE-8188 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.14.0 Reporter: Gopal V Assignee: Gopal V Labels: Performance Attachments: HIVE-8188.1.patch, HIVE-8188.2.patch, udf-deterministic.png When running a near-constant UDF, most of the CPU is burnt within the VM trying to read the class annotations for every row. !udf-deterministic.png! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-8210) TezJobMonitor should print time spent in Application
Gopal V created HIVE-8210: - Summary: TezJobMonitor should print time spent in Application Key: HIVE-8210 URL: https://issues.apache.org/jira/browse/HIVE-8210 Project: Hive Issue Type: Bug Components: Tez Affects Versions: 0.14.0 Reporter: Gopal V Assignee: Gopal V Priority: Trivial In a fully loaded cluster, there is no easy way to distinguish between a query which takes 2 minutes to run and a query which takes 2 minutes to get an AM. Clearly log the amount of time the query spent in RUNNING state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8210) TezJobMonitor should print time spent in Application (RUNNING)
[ https://issues.apache.org/jira/browse/HIVE-8210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8210: -- Summary: TezJobMonitor should print time spent in Application (RUNNING) (was: TezJobMonitor should print time spent in Application ) TezJobMonitor should print time spent in Application (RUNNING) -- Key: HIVE-8210 URL: https://issues.apache.org/jira/browse/HIVE-8210 Project: Hive Issue Type: Bug Components: Tez Affects Versions: 0.14.0 Reporter: Gopal V Assignee: Gopal V Priority: Trivial Labels: UX In a fully loaded cluster, there is no easy way to distinguish between a query which takes 2 minutes to run and a query which takes 2 minutes to get an AM. Clearly log the amount of time the query spent in RUNNING state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8210) TezJobMonitor should print time spent in Application (RUNNING)
[ https://issues.apache.org/jira/browse/HIVE-8210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8210: -- Attachment: HIVE-8210.1.patch TezJobMonitor should print time spent in Application (RUNNING) -- Key: HIVE-8210 URL: https://issues.apache.org/jira/browse/HIVE-8210 Project: Hive Issue Type: Bug Components: Tez Affects Versions: 0.14.0 Reporter: Gopal V Assignee: Gopal V Priority: Trivial Labels: UX Attachments: HIVE-8210.1.patch In a fully loaded cluster, there is no easy way to distinguish between a query which takes 2 minutes to run and a query which takes 2 minutes to get an AM. Clearly log the amount of time the query spent in RUNNING state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8210) TezJobMonitor should print time spent in Application (RUNNING)
[ https://issues.apache.org/jira/browse/HIVE-8210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8210: -- Status: Patch Available (was: Open) TezJobMonitor should print time spent in Application (RUNNING) -- Key: HIVE-8210 URL: https://issues.apache.org/jira/browse/HIVE-8210 Project: Hive Issue Type: Bug Components: Tez Affects Versions: 0.14.0 Reporter: Gopal V Assignee: Gopal V Priority: Trivial Labels: UX Attachments: HIVE-8210.1.patch In a fully loaded cluster, there is no easy way to distinguish between a query which takes 2 minutes to run and a query which takes 2 minutes to get an AM. Clearly log the amount of time the query spent in RUNNING state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8205) Using strings in group type fails in ParquetSerDe
[
https://issues.apache.org/jira/browse/HIVE-8205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142188#comment-14142188
]
Hive QA commented on HIVE-8205:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12670171/HIVE-8205.patch
{color:green}SUCCESS:{color} +1 6298 tests passed
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/899/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/899/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-899/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12670171
Using strings in group type fails in ParquetSerDe
-
Key: HIVE-8205
URL: https://issues.apache.org/jira/browse/HIVE-8205
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Reporter: Mohit Sabharwal
Assignee: Mohit Sabharwal
Labels: parquet
Attachments: HIVE-8205.patch
In HIVE-7735, schema info was plumbed to ETypeConverter to disambiguate
between hive Char, Varchar and String types, which are all represented as
PrimitiveType binary and OriginalType utf8 in parquet.
However, this does not work for parquet nested types (that map to hive Array,
Map, etc.) containing these values, because schema lookup for nested values
was not implemented. It's also non-trivial to do that in the current parquet
serde implementation. Instead of plumbing in the schema, we should convert
these types to the same Text writeable and let the object inspectors handle
the final conversion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8052) Vectorization: min() on TimeStamp datatype fails with error Vector aggregate not implemented: min for type: TIMESTAMP
[ https://issues.apache.org/jira/browse/HIVE-8052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142190#comment-14142190 ] Jason Dere commented on HIVE-8052: -- +1 Since you found the cause of the different results between MR/Tez, does that mean that the other agg functions you had allowed to be vectorized in your first patch will also work? Vectorization: min() on TimeStamp datatype fails with error Vector aggregate not implemented: min for type: TIMESTAMP --- Key: HIVE-8052 URL: https://issues.apache.org/jira/browse/HIVE-8052 Project: Hive Issue Type: Bug Reporter: Matt McCline Assignee: Matt McCline Priority: Critical Attachments: HIVE-8052.01.patch, HIVE-8052.02.patch, HIVE-8052.04.patch Changes in HIVE-5760 to make explicit when timestamp and date can be vectorized as Long were accidentally to strict for min, max, count, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7950) StorageHandler resources aren't added to Tez Session if already Session is already Open
[
https://issues.apache.org/jira/browse/HIVE-7950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142210#comment-14142210
]
Hive QA commented on HIVE-7950:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12670172/HIVE-7950.5.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6307 tests executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.streaming.TestStreaming.testInterleavedTransactionBatchCommits
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/900/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/900/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-900/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12670172
StorageHandler resources aren't added to Tez Session if already Session is
already Open
---
Key: HIVE-7950
URL: https://issues.apache.org/jira/browse/HIVE-7950
Project: Hive
Issue Type: Bug
Components: StorageHandler, Tez
Reporter: Josh Elser
Assignee: Josh Elser
Fix For: 0.14.0
Attachments: HIVE-7950-1.diff, HIVE-7950.2.patch, HIVE-7950.3.patch,
HIVE-7950.4.patch, HIVE-7950.5.patch, hive-7950-tez-WIP.diff
Was trying to run some queries using the AccumuloStorageHandler when using
the Tez execution engine. Some things that classes which were added to
tmpjars weren't making it into the container. When a Tez Session is already
open, as is the normal case when simply using the `hive` command, the
resources aren't added.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7482) The execution side changes for SMB join in hive-tez
[ https://issues.apache.org/jira/browse/HIVE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142231#comment-14142231 ] Gunther Hagleitner commented on HIVE-7482: -- looked at the latest rb entry. some minor comments, but I'm worried about some updates to golden files that seem wrong. The execution side changes for SMB join in hive-tez --- Key: HIVE-7482 URL: https://issues.apache.org/jira/browse/HIVE-7482 Project: Hive Issue Type: Bug Components: Tez Affects Versions: tez-branch Reporter: Vikram Dixit K Assignee: Vikram Dixit K Attachments: HIVE-7482.1.patch, HIVE-7482.2.patch, HIVE-7482.3.patch, HIVE-7482.4.patch, HIVE-7482.5.patch, HIVE-7482.6.patch, HIVE-7482.7.patch, HIVE-7482.8.patch, HIVE-7482.WIP.2.patch, HIVE-7482.WIP.3.patch, HIVE-7482.WIP.4.patch, HIVE-7482.WIP.patch A piece of HIVE-7430. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7482) The execution side changes for SMB join in hive-tez
[
https://issues.apache.org/jira/browse/HIVE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142237#comment-14142237
]
Hive QA commented on HIVE-7482:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12670198/HIVE-7482.8.patch
{color:red}ERROR:{color} -1 due to 19 failed/errored test(s), 6315 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_script_env_var1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_script_env_var2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_temp_table
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testEqualOpClass
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testGreaterThanOpClass
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testGreaterThanOrEqualOpClass
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testHitLastBlock
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testHitSamePositionTwice
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testLessThanOpClass
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testLessThanOrEqualOpClass
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testNonLinearEqualTo
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testNonLinearGreaterThan
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testNonLinearLessThan
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testResetRange
org.apache.hadoop.hive.ql.parse.TestParse.testParse_union
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/902/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/902/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-902/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 19 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12670198
The execution side changes for SMB join in hive-tez
---
Key: HIVE-7482
URL: https://issues.apache.org/jira/browse/HIVE-7482
Project: Hive
Issue Type: Bug
Components: Tez
Affects Versions: tez-branch
Reporter: Vikram Dixit K
Assignee: Vikram Dixit K
Attachments: HIVE-7482.1.patch, HIVE-7482.2.patch, HIVE-7482.3.patch,
HIVE-7482.4.patch, HIVE-7482.5.patch, HIVE-7482.6.patch, HIVE-7482.7.patch,
HIVE-7482.8.patch, HIVE-7482.WIP.2.patch, HIVE-7482.WIP.3.patch,
HIVE-7482.WIP.4.patch, HIVE-7482.WIP.patch
A piece of HIVE-7430.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7842) load_dyn_part1.q fails with an assertion [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-7842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142252#comment-14142252
]
Xuefu Zhang commented on HIVE-7842:
---
[~vkorukanti], could you please verify and enable the test if it hasn't been
enabled? Thanks.
load_dyn_part1.q fails with an assertion [Spark Branch]
---
Key: HIVE-7842
URL: https://issues.apache.org/jira/browse/HIVE-7842
Project: Hive
Issue Type: Sub-task
Components: Spark
Affects Versions: spark-branch
Reporter: Venki Korukanti
Assignee: Venki Korukanti
Labels: Spark-M1
Fix For: spark-branch
On spark branch, load_dyn_part1.q fails with following assertion. Looks like
SerDe is receiving invalid ByteWritable buffer.
{code}
java.lang.AssertionError
org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.deserialize(BinarySortableSerDe.java:205)
org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.deserialize(BinarySortableSerDe.java:187)
org.apache.hadoop.hive.ql.exec.spark.SparkReduceRecordHandler.processRow(SparkReduceRecordHandler.java:186)
org.apache.hadoop.hive.ql.exec.spark.HiveReduceFunctionResultList.processNextRecord(HiveReduceFunctionResultList.java:47)
org.apache.hadoop.hive.ql.exec.spark.HiveReduceFunctionResultList.processNextRecord(HiveReduceFunctionResultList.java:27)
org.apache.hadoop.hive.ql.exec.spark.HiveBaseFunctionResultList$ResultIterator.hasNext(HiveBaseFunctionResultList.java:98)
scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:41)
scala.collection.Iterator$class.foreach(Iterator.scala:727)
scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
org.apache.spark.rdd.RDD$$anonfun$foreach$1.apply(RDD.scala:759)
org.apache.spark.rdd.RDD$$anonfun$foreach$1.apply(RDD.scala:759)
org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1121)
org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1121)
org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:62)
org.apache.spark.scheduler.Task.run(Task.scala:54)
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:199)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:744)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-7482) The execution side changes for SMB join in hive-tez
[ https://issues.apache.org/jira/browse/HIVE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-7482: - Attachment: HIVE-7482.9.patch .9 addresses some of the review comments. The execution side changes for SMB join in hive-tez --- Key: HIVE-7482 URL: https://issues.apache.org/jira/browse/HIVE-7482 Project: Hive Issue Type: Bug Components: Tez Affects Versions: tez-branch Reporter: Vikram Dixit K Assignee: Vikram Dixit K Attachments: HIVE-7482.1.patch, HIVE-7482.2.patch, HIVE-7482.3.patch, HIVE-7482.4.patch, HIVE-7482.5.patch, HIVE-7482.6.patch, HIVE-7482.7.patch, HIVE-7482.8.patch, HIVE-7482.9.patch, HIVE-7482.WIP.2.patch, HIVE-7482.WIP.3.patch, HIVE-7482.WIP.4.patch, HIVE-7482.WIP.patch A piece of HIVE-7430. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8200) Make beeline use the hive-jdbc standalone jar
[
https://issues.apache.org/jira/browse/HIVE-8200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142264#comment-14142264
]
Hive QA commented on HIVE-8200:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12670145/HIVE-8200.1.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6297 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.txn.TestCompactionTxnHandler.testRevokeTimedOutWorkers
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/903/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/903/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-903/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12670145
Make beeline use the hive-jdbc standalone jar
-
Key: HIVE-8200
URL: https://issues.apache.org/jira/browse/HIVE-8200
Project: Hive
Issue Type: Bug
Components: CLI, HiveServer2
Reporter: Deepesh Khandelwal
Assignee: Deepesh Khandelwal
Attachments: HIVE-8200.1.patch
Hiveserver2 JDBC client beeline currently generously includes all the jars
under $HIVE_HOME/lib in its invocation. With the fix from HIVE-8129 it should
only need a few. This will be a good validation of the hive-jdbc standalone
jar.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8188) ExprNodeGenericFuncEvaluator::_evaluate() loads class annotations in a tight loop
[
https://issues.apache.org/jira/browse/HIVE-8188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142300#comment-14142300
]
Hive QA commented on HIVE-8188:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12670244/HIVE-8188.2.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6298 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.parse.TestParse.testParse_union
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/904/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/904/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-904/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12670244
ExprNodeGenericFuncEvaluator::_evaluate() loads class annotations in a tight
loop
-
Key: HIVE-8188
URL: https://issues.apache.org/jira/browse/HIVE-8188
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0
Reporter: Gopal V
Assignee: Gopal V
Labels: Performance
Attachments: HIVE-8188.1.patch, HIVE-8188.2.patch,
udf-deterministic.png
When running a near-constant UDF, most of the CPU is burnt within the VM
trying to read the class annotations for every row.
!udf-deterministic.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8152) Update with expression in set fails
[
https://issues.apache.org/jira/browse/HIVE-8152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-8152:
-
Attachment: HIVE-8152.1.patch
Update with expression in set fails
---
Key: HIVE-8152
URL: https://issues.apache.org/jira/browse/HIVE-8152
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.14.0
Reporter: Alan Gates
Assignee: Eugene Koifman
Priority: Critical
Attachments: HIVE-8152.1.patch, HIVE-8152.patch
Doing: {{update orctabsmall set age = age + 1;}} results in {{FAILED:
ParseException line 1:33 missing EOF at '+' near 'age'}}
We should be able to handle simple expressions in set clauses.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8152) Update with expression in set fails
[
https://issues.apache.org/jira/browse/HIVE-8152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-8152:
-
Status: Patch Available (was: Open)
Update with expression in set fails
---
Key: HIVE-8152
URL: https://issues.apache.org/jira/browse/HIVE-8152
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.14.0
Reporter: Alan Gates
Assignee: Eugene Koifman
Priority: Critical
Attachments: HIVE-8152.1.patch, HIVE-8152.patch
Doing: {{update orctabsmall set age = age + 1;}} results in {{FAILED:
ParseException line 1:33 missing EOF at '+' near 'age'}}
We should be able to handle simple expressions in set clauses.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-7935) Support dynamic service discovery for HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-7935: - Labels: TODOC14 (was: ) Support dynamic service discovery for HiveServer2 - Key: HIVE-7935 URL: https://issues.apache.org/jira/browse/HIVE-7935 Project: Hive Issue Type: New Feature Components: HiveServer2, JDBC Affects Versions: 0.14.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Labels: TODOC14 Fix For: 0.14.0 Attachments: HIVE-7935.1.patch, HIVE-7935.2.patch, HIVE-7935.3.patch, HIVE-7935.4.patch, HIVE-7935.5.patch, HIVE-7935.6.patch, HIVE-7935.7.patch, HIVE-7935.8.patch To support Rolling Upgrade / HA, we need a mechanism by which a JDBC client can dynamically resolve an HiveServer2 to connect to. *High Level Design:* Whether, dynamic service discovery is supported or not, can be configured by setting HIVE_SERVER2_SUPPORT_DYNAMIC_SERVICE_DISCOVERY. ZooKeeper is used to support this. * When an instance of HiveServer2 comes up, it adds itself as a znode to ZooKeeper under a configurable namespace (HIVE_SERVER2_ZOOKEEPER_NAMESPACE). * A JDBC/ODBC client now specifies the ZooKeeper ensemble in its connection string, instead of pointing to a specific HiveServer2 instance. The JDBC driver, uses the ZooKeeper ensemble to pick an instance of HiveServer2 to connect for the entire session. * When an instance is removed from ZooKeeper, the existing client sessions continue till completion. When the last client session completes, the instance shuts down. * All new client connection pick one of the available HiveServer2 uris from ZooKeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-8195) Update golden files for tez
[ https://issues.apache.org/jira/browse/HIVE-8195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner resolved HIVE-8195. -- Resolution: Fixed committed to branch. Update golden files for tez --- Key: HIVE-8195 URL: https://issues.apache.org/jira/browse/HIVE-8195 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-8195.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8210) TezJobMonitor should print time spent in Application (RUNNING)
[
https://issues.apache.org/jira/browse/HIVE-8210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142318#comment-14142318
]
Hive QA commented on HIVE-8210:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12670246/HIVE-8210.1.patch
{color:green}SUCCESS:{color} +1 6298 tests passed
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/905/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/905/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-905/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12670246
TezJobMonitor should print time spent in Application (RUNNING)
--
Key: HIVE-8210
URL: https://issues.apache.org/jira/browse/HIVE-8210
Project: Hive
Issue Type: Bug
Components: Tez
Affects Versions: 0.14.0
Reporter: Gopal V
Assignee: Gopal V
Priority: Trivial
Labels: UX
Attachments: HIVE-8210.1.patch
In a fully loaded cluster, there is no easy way to distinguish between a
query which takes 2 minutes to run and a query which takes 2 minutes to get
an AM.
Clearly log the amount of time the query spent in RUNNING state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8184) inconsistence between colList and columnExprMap when ConstantPropagate is applied to subquery
[ https://issues.apache.org/jira/browse/HIVE-8184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-8184: -- Status: Open (was: Patch Available) inconsistence between colList and columnExprMap when ConstantPropagate is applied to subquery -- Key: HIVE-8184 URL: https://issues.apache.org/jira/browse/HIVE-8184 Project: Hive Issue Type: Improvement Reporter: Pengcheng Xiong Priority: Minor Attachments: HIVE-8184.1.patch, HIVE-8184.2.patch Query like select * from (select a.key as ak, a.value as av, b.key as bk, b.value as bv from src a join src1 b where a.key = '428' ) c; will fail as FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8184) inconsistence between colList and columnExprMap when ConstantPropagate is applied to subquery
[ https://issues.apache.org/jira/browse/HIVE-8184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-8184: -- Attachment: HIVE-8184.3.patch update golden files inconsistence between colList and columnExprMap when ConstantPropagate is applied to subquery -- Key: HIVE-8184 URL: https://issues.apache.org/jira/browse/HIVE-8184 Project: Hive Issue Type: Improvement Reporter: Pengcheng Xiong Priority: Minor Attachments: HIVE-8184.1.patch, HIVE-8184.2.patch, HIVE-8184.3.patch Query like select * from (select a.key as ak, a.value as av, b.key as bk, b.value as bv from src a join src1 b where a.key = '428' ) c; will fail as FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 25800: inconsistence between colList and columnExprMap when ConstantPropagate is applied to subquery
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25800/ --- (Updated Sept. 21, 2014, 4:18 a.m.) Review request for hive. Changes --- update golden files Repository: hive-git Description --- Query like select * from (select a.key as ak, a.value as av, b.key as bk, b.value as bv from src a join src1 b where a.key = '428' ) c; will fail as FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java 790a92e ql/src/test/queries/clientpositive/constantPropagateForSubQuery.q PRE-CREATION ql/src/test/results/clientpositive/annotate_stats_select.q.out 1b3b334 ql/src/test/results/clientpositive/auto_join8.q.out 2e3d1c8 ql/src/test/results/clientpositive/constantPropagateForSubQuery.q.out PRE-CREATION ql/src/test/results/clientpositive/join8.q.out 6435b92 ql/src/test/results/clientpositive/join_view.q.out 3e54177 ql/src/test/results/clientpositive/lateral_view_ppd.q.out 756679a ql/src/test/results/clientpositive/list_bucket_query_oneskew_2.q.out 221ee70 ql/src/test/results/clientpositive/ppd_repeated_alias.q.out f51b5a3 ql/src/test/results/clientpositive/select_dummy_source.q.out 651ce1d ql/src/test/results/clientpositive/subquery_multiinsert.q.out 399c4f8 ql/src/test/results/clientpositive/subquery_notin.q.out d0e50b9 ql/src/test/results/clientpositive/subquery_notin_having.q.out 91581de ql/src/test/results/clientpositive/subquery_unqualcolumnrefs.q.out fe99821 ql/src/test/results/clientpositive/subquery_views.q.out 472fa10 ql/src/test/results/clientpositive/udf4.q.out 9ea9293 ql/src/test/results/clientpositive/udf7.q.out 05f2037 ql/src/test/results/clientpositive/udf_case.q.out 7eab4ab ql/src/test/results/clientpositive/udf_elt.q.out fe6c856 ql/src/test/results/clientpositive/udf_when.q.out fb262a7 ql/src/test/results/clientpositive/vector_elt.q.out 55f63b7 ql/src/test/results/compiler/plan/cast1.q.xml fe34e5b ql/src/test/results/compiler/plan/groupby1.q.xml b895372 ql/src/test/results/compiler/plan/groupby2.q.xml 46a7a5f ql/src/test/results/compiler/plan/groupby3.q.xml 5b6af21 ql/src/test/results/compiler/plan/groupby4.q.xml 92086c3 ql/src/test/results/compiler/plan/groupby5.q.xml 26e59d9 ql/src/test/results/compiler/plan/groupby6.q.xml 87e83c9 ql/src/test/results/compiler/plan/join8.q.xml 2fd7a37 ql/src/test/results/compiler/plan/udf1.q.xml e34f4d1 ql/src/test/results/compiler/plan/udf4.q.xml d6dab9e ql/src/test/results/compiler/plan/udf6.q.xml 37b969c ql/src/test/results/compiler/plan/udf_case.q.xml 726f290 ql/src/test/results/compiler/plan/udf_when.q.xml 4179e6b ql/src/test/results/compiler/plan/union.q.xml 2808b05 Diff: https://reviews.apache.org/r/25800/diff/ Testing --- Thanks, pengcheng xiong
[jira] [Updated] (HIVE-8184) inconsistence between colList and columnExprMap when ConstantPropagate is applied to subquery
[ https://issues.apache.org/jira/browse/HIVE-8184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-8184: -- Status: Patch Available (was: Open) inconsistence between colList and columnExprMap when ConstantPropagate is applied to subquery -- Key: HIVE-8184 URL: https://issues.apache.org/jira/browse/HIVE-8184 Project: Hive Issue Type: Improvement Reporter: Pengcheng Xiong Priority: Minor Attachments: HIVE-8184.1.patch, HIVE-8184.2.patch, HIVE-8184.3.patch Query like select * from (select a.key as ak, a.value as av, b.key as bk, b.value as bv from src a join src1 b where a.key = '428' ) c; will fail as FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7935) Support dynamic service discovery for HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142328#comment-14142328 ] Lefty Leverenz commented on HIVE-7935: -- Doc note: This adds 2 new configuration parameters and changes 6 parameter descriptions, so the wiki needs to be updated for 0.14.0: * new: *hive.server2.support.dynamic.service.discovery* * new: *hive.server2.zookeeper.namespace* * redescribe: *hive.zookeeper.quorum* * redescribe: *hive.zookeeper.client.port* * redescribe: *hive.server2.max.start.attempts* * redescribe: *hive.server2.thrift.bind.host* * redescribe: *hive.server2.thrift.http.port* * redescribe: *hive.server2.thrift.port* (This also changes the definition for *hive.server2.global.init.file.location*, but that doesn't need documentation.) Dynamic service discovery for HS2 also needs to be documented in the HS2 setup doc. Should it be mentioned in JDBC/ODBC docs too? * [Setting Up HiveServer2 | https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2] * [Hive JDBC Driver | https://cwiki.apache.org/confluence/display/Hive/HiveJDBCInterface] * [Hive ODBC Driver | https://cwiki.apache.org/confluence/display/Hive/HiveODBC] Finally, we need a release note. Support dynamic service discovery for HiveServer2 - Key: HIVE-7935 URL: https://issues.apache.org/jira/browse/HIVE-7935 Project: Hive Issue Type: New Feature Components: HiveServer2, JDBC Affects Versions: 0.14.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Labels: TODOC14 Fix For: 0.14.0 Attachments: HIVE-7935.1.patch, HIVE-7935.2.patch, HIVE-7935.3.patch, HIVE-7935.4.patch, HIVE-7935.5.patch, HIVE-7935.6.patch, HIVE-7935.7.patch, HIVE-7935.8.patch To support Rolling Upgrade / HA, we need a mechanism by which a JDBC client can dynamically resolve an HiveServer2 to connect to. *High Level Design:* Whether, dynamic service discovery is supported or not, can be configured by setting HIVE_SERVER2_SUPPORT_DYNAMIC_SERVICE_DISCOVERY. ZooKeeper is used to support this. * When an instance of HiveServer2 comes up, it adds itself as a znode to ZooKeeper under a configurable namespace (HIVE_SERVER2_ZOOKEEPER_NAMESPACE). * A JDBC/ODBC client now specifies the ZooKeeper ensemble in its connection string, instead of pointing to a specific HiveServer2 instance. The JDBC driver, uses the ZooKeeper ensemble to pick an instance of HiveServer2 to connect for the entire session. * When an instance is removed from ZooKeeper, the existing client sessions continue till completion. When the last client session completes, the instance shuts down. * All new client connection pick one of the available HiveServer2 uris from ZooKeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8043) Support merging small files [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-8043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142337#comment-14142337
]
Xuefu Zhang commented on HIVE-8043:
---
Patch looks good to me. +1
Support merging small files [Spark Branch]
--
Key: HIVE-8043
URL: https://issues.apache.org/jira/browse/HIVE-8043
Project: Hive
Issue Type: Task
Components: Spark
Reporter: Xuefu Zhang
Assignee: Rui Li
Labels: Spark-M1
Attachments: HIVE-8043.1-spark.patch, HIVE-8043.2-spark.patch,
HIVE-8043.3-spark.patch
Hive currently supports merging small files with MR as the execution engine.
There are options available for this, such as
{code}
hive.merge.mapfiles
hive.merge.mapredfiles
{code}
Hive.merge.sparkfiles is already introduced in HIVE-7810. To make it work, we
might need a little more research and design on this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7482) The execution side changes for SMB join in hive-tez
[
https://issues.apache.org/jira/browse/HIVE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142340#comment-14142340
]
Hive QA commented on HIVE-7482:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12670261/HIVE-7482.9.patch
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 6315 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_smb_main
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testEqualOpClass
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testGreaterThanOpClass
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testGreaterThanOrEqualOpClass
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testHitLastBlock
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testHitSamePositionTwice
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testLessThanOpClass
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testLessThanOrEqualOpClass
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testNonLinearEqualTo
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testNonLinearGreaterThan
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testNonLinearLessThan
org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.testResetRange
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/906/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/906/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-906/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12670261
The execution side changes for SMB join in hive-tez
---
Key: HIVE-7482
URL: https://issues.apache.org/jira/browse/HIVE-7482
Project: Hive
Issue Type: Bug
Components: Tez
Affects Versions: tez-branch
Reporter: Vikram Dixit K
Assignee: Vikram Dixit K
Attachments: HIVE-7482.1.patch, HIVE-7482.2.patch, HIVE-7482.3.patch,
HIVE-7482.4.patch, HIVE-7482.5.patch, HIVE-7482.6.patch, HIVE-7482.7.patch,
HIVE-7482.8.patch, HIVE-7482.9.patch, HIVE-7482.WIP.2.patch,
HIVE-7482.WIP.3.patch, HIVE-7482.WIP.4.patch, HIVE-7482.WIP.patch
A piece of HIVE-7430.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8043) Support merging small files [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-8043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated HIVE-8043:
--
Resolution: Fixed
Fix Version/s: spark-branch
Status: Resolved (was: Patch Available)
Patch committed to Spark branch. Thanks to Rui for the great contribution.
Support merging small files [Spark Branch]
--
Key: HIVE-8043
URL: https://issues.apache.org/jira/browse/HIVE-8043
Project: Hive
Issue Type: Task
Components: Spark
Reporter: Xuefu Zhang
Assignee: Rui Li
Labels: Spark-M1
Fix For: spark-branch
Attachments: HIVE-8043.1-spark.patch, HIVE-8043.2-spark.patch,
HIVE-8043.3-spark.patch
Hive currently supports merging small files with MR as the execution engine.
There are options available for this, such as
{code}
hive.merge.mapfiles
hive.merge.mapredfiles
{code}
Hive.merge.sparkfiles is already introduced in HIVE-7810. To make it work, we
might need a little more research and design on this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
