[jira] [Commented] (HIVE-14957) HiveSortLimitPullUpConstantsRule misses branches when parent operator is Union
[
https://issues.apache.org/jira/browse/HIVE-14957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15574464#comment-15574464
]
Jesus Camacho Rodriguez commented on HIVE-14957:
[~pxiong], if I remember correctly, the reason we match the parent op is that
the rule should not be triggered at the top of the plan (see the header comment
of the rule). Let's wait for QA results, but if this is the case, it would
suffice with adding the other children when we copy the parent operator.
> HiveSortLimitPullUpConstantsRule misses branches when parent operator is Union
> --
>
> Key: HIVE-14957
> URL: https://issues.apache.org/jira/browse/HIVE-14957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-14957.01.patch
>
>
> {code}
> call.transformTo(parent.copy(parent.getTraitSet(),
> ImmutableList.of(relBuilder.build(;
> {code}
> When parent is an union operator which has 2 inputs, the parent.copy will
> only copy the one that has SortLimit and ignore the other branches.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14921) Move slow CliDriver tests to MiniLlap - part 2
[
https://issues.apache.org/jira/browse/HIVE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15574429#comment-15574429
]
Hive QA commented on HIVE-14921:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833204/HIVE-14921.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10507 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_globallimit]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[reloadJar]
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_adaptor_usage_mode]
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJarWithoutAddDriverClazz[0]
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[0]
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[1]
org.apache.hive.jdbc.authorization.TestJdbcWithSQLAuthorization.testBlackListedUdfUsage
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1557/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1557/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1557/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833204 - PreCommit-HIVE-Build
> Move slow CliDriver tests to MiniLlap - part 2
> --
>
> Key: HIVE-14921
> URL: https://issues.apache.org/jira/browse/HIVE-14921
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14921.1.patch, HIVE-14921.1.patch,
> HIVE-14921.2.patch, HIVE-14921.2.patch, HIVE-14921.3.patch
>
>
> Continuation to HIVE-14877
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14916) Reduce the memory requirements for Spark tests
[ https://issues.apache.org/jira/browse/HIVE-14916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-14916: -- Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Committed. Thanks [~dapengsun], [~Ferd] > Reduce the memory requirements for Spark tests > -- > > Key: HIVE-14916 > URL: https://issues.apache.org/jira/browse/HIVE-14916 > Project: Hive > Issue Type: Sub-task >Reporter: Ferdinand Xu >Assignee: Dapeng Sun > Fix For: 2.2.0 > > Attachments: HIVE-14916.001.patch, HIVE-14916.002.patch, > HIVE-14916.003.patch, HIVE-14916.004.patch > > > As HIVE-14887, we need to reduce the memory requirements for Spark tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12049) HiveServer2: Provide an option to write serialized thrift objects in final tasks
[
https://issues.apache.org/jira/browse/HIVE-12049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15574414#comment-15574414
]
Lefty Leverenz commented on HIVE-12049:
---
HIVE-14876 changes the default value of
hive.server2.thrift.resultset.max.fetch.size from 1000 to 1 in 2.2.0.
> HiveServer2: Provide an option to write serialized thrift objects in final
> tasks
>
>
> Key: HIVE-12049
> URL: https://issues.apache.org/jira/browse/HIVE-12049
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, JDBC
>Affects Versions: 2.0.0
>Reporter: Rohit Dholakia
>Assignee: Rohit Dholakia
> Labels: TODOC2.1
> Fix For: 2.1.0
>
> Attachments: HIVE-12049.1.patch, HIVE-12049.11.patch,
> HIVE-12049.12.patch, HIVE-12049.13.patch, HIVE-12049.14.patch,
> HIVE-12049.15.patch, HIVE-12049.16.patch, HIVE-12049.17.patch,
> HIVE-12049.18.patch, HIVE-12049.19.patch, HIVE-12049.2.patch,
> HIVE-12049.25.patch, HIVE-12049.26.patch, HIVE-12049.3.patch,
> HIVE-12049.4.patch, HIVE-12049.5.patch, HIVE-12049.6.patch,
> HIVE-12049.7.patch, HIVE-12049.9.patch, new-driver-profiles.png,
> old-driver-profiles.png
>
>
> For each fetch request to HiveServer2, we pay the penalty of deserializing
> the row objects and translating them into a different representation suitable
> for the RPC transfer. In a moderate to high concurrency scenarios, this can
> result in significant CPU and memory wastage. By having each task write the
> appropriate thrift objects to the output files, HiveServer2 can simply stream
> a batch of rows on the wire without incurring any of the additional cost of
> deserialization and translation.
> This can be implemented by writing a new SerDe, which the FileSinkOperator
> can use to write thrift formatted row batches to the output file. Using the
> pluggable property of the {{hive.query.result.fileformat}}, we can set it to
> use SequenceFile and write a batch of thrift formatted rows as a value blob.
> The FetchTask can now simply read the blob and send it over the wire. On the
> client side, the *DBC driver can read the blob and since it is already
> formatted in the way it expects, it can continue building the ResultSet the
> way it does in the current implementation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14876) make the number of rows to fetch from various HS2 clients/servers configurable
[ https://issues.apache.org/jira/browse/HIVE-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15574411#comment-15574411 ] Lefty Leverenz commented on HIVE-14876: --- Doc note: This changes the default value of *hive.server2.thrift.resultset.max.fetch.size* from 1000 to 1 and adds *hive.server2.resultset.default.fetch.size* so they need to be documented in Configuration Properties. (*hive.server2.thrift.resultset.max.fetch.size* was introduced in 2.1.0 by HIVE-12049 and isn't documented yet in the wiki.) Added a TODOC2.2 label. * [Configuration Properties -- HiveServer2 | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-HiveServer2] By the way, the most recent attached patch (01) doesn't include the change of default value. Compare with commit 47d904675d04539bdef96ed26e00b1786251e9f2. > make the number of rows to fetch from various HS2 clients/servers configurable > -- > > Key: HIVE-14876 > URL: https://issues.apache.org/jira/browse/HIVE-14876 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Labels: TODOC2.2 > Fix For: 2.2.0 > > Attachments: HIVE-14876.01.patch, HIVE-14876.patch > > > Right now, it's hardcoded to a variety of values -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-14938) Add deployed ptest properties file to repo, update to remove isolated tests
[ https://issues.apache.org/jira/browse/HIVE-14938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth resolved HIVE-14938. --- Resolution: Fixed Fix Version/s: 2.2.0 Committed. precommit not required since these files are only picked up when deployed on the ptest node. They serve as a reference in the repository at the moment. > Add deployed ptest properties file to repo, update to remove isolated tests > --- > > Key: HIVE-14938 > URL: https://issues.apache.org/jira/browse/HIVE-14938 > Project: Hive > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Fix For: 2.2.0 > > Attachments: HIVE-14938.part1.patch, HIVE-14938.part2.patch > > > The intent is to checkin the original file, and then modify it to remove > isolated tests (and move relevant ones to the skipBatching list), which > normally lead to stragglers, and sub-optimal resource utilization. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14876) make the number of rows to fetch from various HS2 clients/servers configurable
[ https://issues.apache.org/jira/browse/HIVE-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-14876: -- Labels: TODOC2.2 (was: ) > make the number of rows to fetch from various HS2 clients/servers configurable > -- > > Key: HIVE-14876 > URL: https://issues.apache.org/jira/browse/HIVE-14876 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Labels: TODOC2.2 > Fix For: 2.2.0 > > Attachments: HIVE-14876.01.patch, HIVE-14876.patch > > > Right now, it's hardcoded to a variety of values -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14237) Move stats collection logic out of transaction for create table
[ https://issues.apache.org/jira/browse/HIVE-14237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15574390#comment-15574390 ] Thejas M Nair commented on HIVE-14237: -- [~ashutoshc] The call to MetaStoreUtils.updateTableStatsFast from HiveMetaStore.create_table_core no longer there after this change. That should also move to before the openTransaction, is that right ? > Move stats collection logic out of transaction for create table > --- > > Key: HIVE-14237 > URL: https://issues.apache.org/jira/browse/HIVE-14237 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 1.0.0, 1.2.0, 1.1.0, 2.0.0, 2.1.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-14237.1.patch, HIVE-14237.patch > > > Stats collection logic interacts with FS which may take long time. There is > no need to do that inside DB transaction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14864) Distcp is not called from MoveTask when src is a directory
[
https://issues.apache.org/jira/browse/HIVE-14864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-14864:
Attachment: HIVE-14864.1.patch
> Distcp is not called from MoveTask when src is a directory
> --
>
> Key: HIVE-14864
> URL: https://issues.apache.org/jira/browse/HIVE-14864
> Project: Hive
> Issue Type: Bug
>Reporter: Vihang Karajgaonkar
>Assignee: Sahil Takiar
> Attachments: HIVE-14864.1.patch, HIVE-14864.patch
>
>
> In FileUtils.java the following code does not get executed even when src
> directory size is greater than HIVE_EXEC_COPYFILE_MAXSIZE because
> srcFS.getFileStatus(src).getLen() returns 0 when src is a directory. We
> should use srcFS.getContentSummary(src).getLength() instead.
> {noformat}
> /* Run distcp if source file/dir is too big */
> if (srcFS.getUri().getScheme().equals("hdfs") &&
> srcFS.getFileStatus(src).getLen() >
> conf.getLongVar(HiveConf.ConfVars.HIVE_EXEC_COPYFILE_MAXSIZE)) {
> LOG.info("Source is " + srcFS.getFileStatus(src).getLen() + " bytes.
> (MAX: " + conf.getLongVar(HiveConf.ConfVars.HIVE_EXEC_COPYFILE_MAXSIZE) +
> ")");
> LOG.info("Launch distributed copy (distcp) job.");
> HiveConfUtil.updateJobCredentialProviders(conf);
> copied = shims.runDistCp(src, dst, conf);
> if (copied && deleteSource) {
> srcFS.delete(src, true);
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14876) make the number of rows to fetch from various HS2 clients/servers configurable
[ https://issues.apache.org/jira/browse/HIVE-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15574338#comment-15574338 ] Siddharth Seth commented on HIVE-14876: --- No pre-commit runs? > make the number of rows to fetch from various HS2 clients/servers configurable > -- > > Key: HIVE-14876 > URL: https://issues.apache.org/jira/browse/HIVE-14876 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 2.2.0 > > Attachments: HIVE-14876.01.patch, HIVE-14876.patch > > > Right now, it's hardcoded to a variety of values -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-13557) Make interval keyword optional while specifying DAY in interval arithmetic
[
https://issues.apache.org/jira/browse/HIVE-13557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich reassigned HIVE-13557:
---
Assignee: Zoltan Haindrich
> Make interval keyword optional while specifying DAY in interval arithmetic
> --
>
> Key: HIVE-13557
> URL: https://issues.apache.org/jira/browse/HIVE-13557
> Project: Hive
> Issue Type: Sub-task
> Components: Types
>Reporter: Ashutosh Chauhan
>Assignee: Zoltan Haindrich
>
> Currently we support expressions like: {code}
> WHERE SOLD_DATE BETWEEN ((DATE('2000-01-31')) - INTERVAL '30' DAY) AND
> DATE('2000-01-31')
> {code}
> We should support:
> {code}
> WHERE SOLD_DATE BETWEEN ((DATE('2000-01-31')) + (-30) DAY) AND
> DATE('2000-01-31')
> {code}
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14940) MiniTezCliDriver - switch back to SQL metastore as default
[ https://issues.apache.org/jira/browse/HIVE-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15574289#comment-15574289 ] Siddharth Seth commented on HIVE-14940: --- +1. Doubt the test failure is related. Would be could to wait for another one - or a jira which caused the MiniLlapLocal timeouts. > MiniTezCliDriver - switch back to SQL metastore as default > -- > > Key: HIVE-14940 > URL: https://issues.apache.org/jira/browse/HIVE-14940 > Project: Hive > Issue Type: Sub-task > Components: Tests >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-14940.1.patch, HIVE-14940.1.patch, > HIVE-14940.2.patch, HIVE-14940.2.patch, HIVE-14940.3.patch, > HIVE-14940.3.patch, HIVE-14940.4.patch > > > HBase setup for metastore in MiniTez is taking around 3 mins for setup. The > actual runtime of the queries is typically much lower. To avoid the high > overhead we should be switch back to SQL metastore as default and if required > we can have dedicated set of tests that run against hbase metastore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HIVE-14940) MiniTezCliDriver - switch back to SQL metastore as default
[ https://issues.apache.org/jira/browse/HIVE-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15574289#comment-15574289 ] Siddharth Seth edited comment on HIVE-14940 at 10/14/16 5:36 AM: - +1. Doubt the test failure is related. Would be could to wait for another one - or a jira which caused the MiniLlapLocal timeouts, before committing. was (Author: sseth): +1. Doubt the test failure is related. Would be could to wait for another one - or a jira which caused the MiniLlapLocal timeouts. > MiniTezCliDriver - switch back to SQL metastore as default > -- > > Key: HIVE-14940 > URL: https://issues.apache.org/jira/browse/HIVE-14940 > Project: Hive > Issue Type: Sub-task > Components: Tests >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-14940.1.patch, HIVE-14940.1.patch, > HIVE-14940.2.patch, HIVE-14940.2.patch, HIVE-14940.3.patch, > HIVE-14940.3.patch, HIVE-14940.4.patch > > > HBase setup for metastore in MiniTez is taking around 3 mins for setup. The > actual runtime of the queries is typically much lower. To avoid the high > overhead we should be switch back to SQL metastore as default and if required > we can have dedicated set of tests that run against hbase metastore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14799) Query operation are not thread safe during its cancellation
[
https://issues.apache.org/jira/browse/HIVE-14799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15574273#comment-15574273
]
Hive QA commented on HIVE-14799:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833239/HIVE-14799.7.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10534 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_globallimit]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[order_null]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[reloadJar]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_fast_stats]
org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.testConnections
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJarWithoutAddDriverClazz[0]
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[0]
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[1]
org.apache.hive.jdbc.authorization.TestJdbcWithSQLAuthorization.testBlackListedUdfUsage
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1556/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1556/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1556/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833239 - PreCommit-HIVE-Build
> Query operation are not thread safe during its cancellation
> ---
>
> Key: HIVE-14799
> URL: https://issues.apache.org/jira/browse/HIVE-14799
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Chaoyu Tang
>Assignee: Chaoyu Tang
> Attachments: HIVE-14799.1.patch, HIVE-14799.2.patch,
> HIVE-14799.3.patch, HIVE-14799.4.patch, HIVE-14799.5.patch,
> HIVE-14799.5.patch, HIVE-14799.6.patch, HIVE-14799.6.patch,
> HIVE-14799.7.patch, HIVE-14799.patch
>
>
> When a query is cancelled either via Beeline (Ctrl-C) or API call
> TCLIService.Client.CancelOperation, SQLOperation.cancel is invoked in a
> different thread from that running the query to close/destroy its
> encapsulated Driver object. Both SQLOperation and Driver are not thread-safe
> which could sometimes result in Runtime exceptions like NPE. The errors from
> the running query are not handled properly therefore probably causing some
> stuffs (files, locks etc) not being cleaned after the query termination.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14916) Reduce the memory requirements for Spark tests
[ https://issues.apache.org/jira/browse/HIVE-14916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15574270#comment-15574270 ] Ferdinand Xu commented on HIVE-14916: - Thanks [~sseth] for the review. +1 for the diff since memory cut in half. > Reduce the memory requirements for Spark tests > -- > > Key: HIVE-14916 > URL: https://issues.apache.org/jira/browse/HIVE-14916 > Project: Hive > Issue Type: Sub-task >Reporter: Ferdinand Xu >Assignee: Dapeng Sun > Attachments: HIVE-14916.001.patch, HIVE-14916.002.patch, > HIVE-14916.003.patch, HIVE-14916.004.patch > > > As HIVE-14887, we need to reduce the memory requirements for Spark tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14887) Reduce the memory requirements for tests
[ https://issues.apache.org/jira/browse/HIVE-14887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-14887: -- Attachment: HIVE-14887.03.patch Trying with a slightly higher container size. > Reduce the memory requirements for tests > > > Key: HIVE-14887 > URL: https://issues.apache.org/jira/browse/HIVE-14887 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-14887.01.patch, HIVE-14887.02.patch, > HIVE-14887.03.patch > > > The clusters that we spin up end up requiring 16GB at times. Also the maven > arguments seem a little heavy weight. > Reducing this will allow for additional ptest drones per box, which should > bring down the runtime. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14916) Reduce the memory requirements for Spark tests
[ https://issues.apache.org/jira/browse/HIVE-14916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15574224#comment-15574224 ] Siddharth Seth commented on HIVE-14916: --- +1 for the changes to the shim etc. [~Ferd] - could your please take a look at the diff. > Reduce the memory requirements for Spark tests > -- > > Key: HIVE-14916 > URL: https://issues.apache.org/jira/browse/HIVE-14916 > Project: Hive > Issue Type: Sub-task >Reporter: Ferdinand Xu >Assignee: Dapeng Sun > Attachments: HIVE-14916.001.patch, HIVE-14916.002.patch, > HIVE-14916.003.patch, HIVE-14916.004.patch > > > As HIVE-14887, we need to reduce the memory requirements for Spark tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14955) Fix reloadJar.q
[ https://issues.apache.org/jira/browse/HIVE-14955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15574195#comment-15574195 ] Siddharth Seth commented on HIVE-14955: --- [~vgumashta] - should we just revert HIVE-12458 so that the test passes again, and make the relevant fix there itself. Jenkins was not run before committing HIVE-12458. > Fix reloadJar.q > > > Key: HIVE-14955 > URL: https://issues.apache.org/jira/browse/HIVE-14955 > Project: Hive > Issue Type: Sub-task >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > > Looks like removing identity_udf.jar in HIVE-12458 caused this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14957) HiveSortLimitPullUpConstantsRule misses branches when parent operator is Union
[
https://issues.apache.org/jira/browse/HIVE-14957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengcheng Xiong updated HIVE-14957:
---
Status: Patch Available (was: Open)
> HiveSortLimitPullUpConstantsRule misses branches when parent operator is Union
> --
>
> Key: HIVE-14957
> URL: https://issues.apache.org/jira/browse/HIVE-14957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-14957.01.patch
>
>
> {code}
> call.transformTo(parent.copy(parent.getTraitSet(),
> ImmutableList.of(relBuilder.build(;
> {code}
> When parent is an union operator which has 2 inputs, the parent.copy will
> only copy the one that has SortLimit and ignore the other branches.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14957) HiveSortLimitPullUpConstantsRule misses branches when parent operator is Union
[
https://issues.apache.org/jira/browse/HIVE-14957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengcheng Xiong updated HIVE-14957:
---
Attachment: HIVE-14957.01.patch
> HiveSortLimitPullUpConstantsRule misses branches when parent operator is Union
> --
>
> Key: HIVE-14957
> URL: https://issues.apache.org/jira/browse/HIVE-14957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-14957.01.patch
>
>
> {code}
> call.transformTo(parent.copy(parent.getTraitSet(),
> ImmutableList.of(relBuilder.build(;
> {code}
> When parent is an union operator which has 2 inputs, the parent.copy will
> only copy the one that has SortLimit and ignore the other branches.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14643) handle ctas
[
https://issues.apache.org/jira/browse/HIVE-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15574151#comment-15574151
]
Hive QA commented on HIVE-14643:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833236/HIVE-14643.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1555/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1555/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1555/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-14 04:27:54.810
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-1555/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 04:27:54.812
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 04:27:55.663
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java:304
error: ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java: patch
does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java:3766
error: ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java: patch does
not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:1514
error: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java: patch does not
apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:1840
error: ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:
patch does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcFactory.java:223
error:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcFactory.java:
patch does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java:307
error: ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java: patch does
not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:6544
error: ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java: patch
does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java:205
error: ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java: patch
does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/plan/FileSinkDesc.java:180
error: ql/src/java/org/apache/hadoop/hive/ql/plan/FileSinkDesc.java: patch does
not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java:60
error: ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java: patch does
not apply
error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/plan/MoveWork.java:70
error: ql/src/java/org/apache/hadoop/hive/ql/plan/MoveWork.java: patch does not
apply
error: ql/src/test/queries/clientpositive/mm_all.q: No such file or directory
error:
[jira] [Commented] (HIVE-14935) Add tests for beeline force option
[
https://issues.apache.org/jira/browse/HIVE-14935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15574145#comment-15574145
]
Hive QA commented on HIVE-14935:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833262/HIVE-14935.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10535 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_globallimit]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[order_null]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[reloadJar]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_fast_stats]
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJarWithoutAddDriverClazz[0]
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[0]
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[1]
org.apache.hive.jdbc.authorization.TestJdbcWithSQLAuthorization.testBlackListedUdfUsage
org.apache.hive.spark.client.TestSparkClient.testJobSubmission
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1554/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1554/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1554/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833262 - PreCommit-HIVE-Build
> Add tests for beeline force option
> --
>
> Key: HIVE-14935
> URL: https://issues.apache.org/jira/browse/HIVE-14935
> Project: Hive
> Issue Type: Test
>Reporter: Kavan Suresh
>Assignee: Kavan Suresh
> Fix For: 2.2.0
>
> Attachments: HIVE-14935.1.patch
>
>
> Add unit test for beeline with force option to ensure continuation of running
> script even after errors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14891) Parallelize TestHCatStorer
[ https://issues.apache.org/jira/browse/HIVE-14891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-14891: Attachment: HIVE-14891.1.patch > Parallelize TestHCatStorer > -- > > Key: HIVE-14891 > URL: https://issues.apache.org/jira/browse/HIVE-14891 > Project: Hive > Issue Type: Sub-task > Components: HCatalog, Testing Infrastructure >Affects Versions: 2.1.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-14891.1.patch, HIVE-14891.1.patch > > > Currently TestHCatStorer runs as a parameterized test, where it runs the same > tests for each storage format but within the same junit test case. This > prevents it from being parallelized using ptest where parallelism granularity > is at a test case level. Instead of using parameterized tests, it makes sense > to create a new test case for each storage format. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (HIVE-14926) Keep Schema in consistent state where schemaTool fails or succeeds.
[
https://issues.apache.org/jira/browse/HIVE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-14926:
---
Comment: was deleted
(was:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833201/HIVE-14926.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1547/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1547/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1547/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:02:14.908
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-1547/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:02:14.910
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
b9e8157..eacf9f9 hive-14535 -> origin/hive-14535
+ git reset --hard HEAD
HEAD is now at 10b866a HIVE-13798: Fix the unit test failure
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload (Aihua Xu,
reviewed by Ashutosh Chauhan) (addendum)
+ git clean -f -d
+ git checkout master
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:02:17.719
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 44: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 82: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 84: patch: command
not found
error: a/beeline/src/java/org/apache/hive/beeline/HiveSchemaHelper.java: No
such file or directory
error: a/beeline/src/java/org/apache/hive/beeline/HiveSchemaTool.java: No such
file or directory
error:
a/itests/hive-unit/src/test/java/org/apache/hive/beeline/TestSchemaTool.java:
No such file or directory
error:
a/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreSchemaInfo.java:
No such file or directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833201 - PreCommit-HIVE-Build)
> Keep Schema in consistent state where schemaTool fails or succeeds.
> -
>
> Key: HIVE-14926
> URL: https://issues.apache.org/jira/browse/HIVE-14926
> Project: Hive
> Issue Type: Improvement
> Components: Database/Schema
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-14926.1.patch, HIVE-14926.2.patch
>
>
> SchemaTool uses autocommit right now when executing the upgrade or init
> scripts. Seems we should use database transaction to commit or roll back to
> keep schema consistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Issue Comment Deleted] (HIVE-14921) Move slow CliDriver tests to MiniLlap - part 2
[
https://issues.apache.org/jira/browse/HIVE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-14921:
---
Comment: was deleted
(was:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833204/HIVE-14921.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1548/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1548/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1548/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:02:45.971
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-1548/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:02:45.973
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:02:46.801
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 44: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 82: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 84: patch: command
not found
error: a/itests/src/test/resources/testconfiguration.properties: No such file
or directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833204 - PreCommit-HIVE-Build)
> Move slow CliDriver tests to MiniLlap - part 2
> --
>
> Key: HIVE-14921
> URL: https://issues.apache.org/jira/browse/HIVE-14921
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14921.1.patch, HIVE-14921.1.patch,
> HIVE-14921.2.patch, HIVE-14921.2.patch, HIVE-14921.3.patch
>
>
> Continuation to HIVE-14877
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Issue Comment Deleted] (HIVE-14799) Query operation are not thread safe during its cancellation
[
https://issues.apache.org/jira/browse/HIVE-14799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-14799:
---
Comment: was deleted
(was:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833239/HIVE-14799.7.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1549/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1549/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1549/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:03:18.432
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-1549/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:03:18.434
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:03:19.285
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 44: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 82: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 84: patch: command
not found
error: a/ql/src/java/org/apache/hadoop/hive/ql/Driver.java: No such file or
directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833239 - PreCommit-HIVE-Build)
> Query operation are not thread safe during its cancellation
> ---
>
> Key: HIVE-14799
> URL: https://issues.apache.org/jira/browse/HIVE-14799
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Chaoyu Tang
>Assignee: Chaoyu Tang
> Attachments: HIVE-14799.1.patch, HIVE-14799.2.patch,
> HIVE-14799.3.patch, HIVE-14799.4.patch, HIVE-14799.5.patch,
> HIVE-14799.5.patch, HIVE-14799.6.patch, HIVE-14799.6.patch,
> HIVE-14799.7.patch, HIVE-14799.patch
>
>
> When a query is cancelled either via Beeline (Ctrl-C) or API call
> TCLIService.Client.CancelOperation, SQLOperation.cancel is invoked in a
> different thread from that running the query to close/destroy its
> encapsulated Driver object. Both SQLOperation and Driver are not thread-safe
> which could sometimes result in Runtime exceptions like NPE. The errors from
> the running query are not handled properly therefore probably causing some
> stuffs (files, locks etc) not being cleaned after the query termination.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Issue Comment Deleted] (HIVE-14643) handle ctas
[
https://issues.apache.org/jira/browse/HIVE-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-14643:
---
Comment: was deleted
(was:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833236/HIVE-14643.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1550/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1550/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1550/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:03:51.514
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-1550/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:03:51.517
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:03:52.378
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 44: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 82: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 84: patch: command
not found
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java:304
error: ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java: patch
does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java:3766
error: ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java: patch does
not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:1514
error: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java: patch does not
apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:1840
error: ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:
patch does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcFactory.java:223
error:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcFactory.java:
patch does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java:307
error: ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java: patch does
not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:6544
error: ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java: patch
does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java:205
error: ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java: patch
does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/plan/FileSinkDesc.java:180
error: ql/src/java/org/apache/hadoop/hive/ql/plan/FileSinkDesc.java: patch does
not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java:60
error: ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java: patch does
not apply
err
[jira] [Issue Comment Deleted] (HIVE-14935) Add tests for beeline force option
[
https://issues.apache.org/jira/browse/HIVE-14935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-14935:
---
Comment: was deleted
(was:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833262/HIVE-14935.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1551/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1551/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1551/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:04:24.199
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-1551/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:04:24.202
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:04:25.052
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 44: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 82: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 84: patch: command
not found
error:
a/itests/hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java:
No such file or directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833262 - PreCommit-HIVE-Build)
> Add tests for beeline force option
> --
>
> Key: HIVE-14935
> URL: https://issues.apache.org/jira/browse/HIVE-14935
> Project: Hive
> Issue Type: Test
>Reporter: Kavan Suresh
>Assignee: Kavan Suresh
> Fix For: 2.2.0
>
> Attachments: HIVE-14935.1.patch
>
>
> Add unit test for beeline with force option to ensure continuation of running
> script even after errors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14891) Parallelize TestHCatStorer
[
https://issues.apache.org/jira/browse/HIVE-14891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15574000#comment-15574000
]
Hive QA commented on HIVE-14891:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833268/HIVE-14891.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1553/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1553/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1553/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:11:13.999
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-1553/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:11:14.001
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:11:14.835
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with: patch -p1
patching file
hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hive/hcatalog/pig/AbstractHCatStorerTest.java
patching file
hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hive/hcatalog/pig/TestAvroHCatStorer.java
patching file
hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hive/hcatalog/pig/TestHCatStorer.java
patching file
hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hive/hcatalog/pig/TestOrcHCatStorer.java
patching file
hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hive/hcatalog/pig/TestParquetHCatStorer.java
patching file
hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hive/hcatalog/pig/TestRCFileHCatStorer.java
patching file
hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hive/hcatalog/pig/TestSequenceFileHCatStorer.java
patching file
hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hive/hcatalog/pig/TestTextFileHCatStorer.java
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
[ERROR] Failed to execute goal on project hive-shims-scheduler: Could not
resolve dependencies for project
org.apache.hive.shims:hive-shims-scheduler:jar:2.2.0-SNAPSHOT: The following
artifacts could not be resolved:
org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.7.2,
org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:2.7.2: Failure to find
org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.7.2 in
http://www.datanucleus.org/downloads/maven2 was cached in the local repository,
resolution will not be reattempted until the update interval of datanucleus has
elapsed or updates are forced -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence
[jira] [Issue Comment Deleted] (HIVE-14891) Parallelize TestHCatStorer
[
https://issues.apache.org/jira/browse/HIVE-14891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-14891:
---
Comment: was deleted
(was:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833268/HIVE-14891.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1552/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1552/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1552/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:04:58.615
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-1552/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:04:58.618
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:04:59.479
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 44: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 82: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 84: patch: command
not found
error:
a/hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hive/hcatalog/pig/TestHCatStorer.java:
No such file or directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833268 - PreCommit-HIVE-Build)
> Parallelize TestHCatStorer
> --
>
> Key: HIVE-14891
> URL: https://issues.apache.org/jira/browse/HIVE-14891
> Project: Hive
> Issue Type: Sub-task
> Components: HCatalog, Testing Infrastructure
>Affects Versions: 2.1.0
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Attachments: HIVE-14891.1.patch
>
>
> Currently TestHCatStorer runs as a parameterized test, where it runs the same
> tests for each storage format but within the same junit test case. This
> prevents it from being parallelized using ptest where parallelism granularity
> is at a test case level. Instead of using parameterized tests, it makes sense
> to create a new test case for each storage format.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14891) Parallelize TestHCatStorer
[
https://issues.apache.org/jira/browse/HIVE-14891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573989#comment-15573989
]
Hive QA commented on HIVE-14891:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833268/HIVE-14891.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1552/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1552/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1552/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:04:58.615
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-1552/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:04:58.618
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:04:59.479
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 44: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 82: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 84: patch: command
not found
error:
a/hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hive/hcatalog/pig/TestHCatStorer.java:
No such file or directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833268 - PreCommit-HIVE-Build
> Parallelize TestHCatStorer
> --
>
> Key: HIVE-14891
> URL: https://issues.apache.org/jira/browse/HIVE-14891
> Project: Hive
> Issue Type: Sub-task
> Components: HCatalog, Testing Infrastructure
>Affects Versions: 2.1.0
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Attachments: HIVE-14891.1.patch
>
>
> Currently TestHCatStorer runs as a parameterized test, where it runs the same
> tests for each storage format but within the same junit test case. This
> prevents it from being parallelized using ptest where parallelism granularity
> is at a test case level. Instead of using parameterized tests, it makes sense
> to create a new test case for each storage format.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14935) Add tests for beeline force option
[
https://issues.apache.org/jira/browse/HIVE-14935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573987#comment-15573987
]
Hive QA commented on HIVE-14935:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833262/HIVE-14935.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1551/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1551/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1551/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:04:24.199
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-1551/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:04:24.202
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:04:25.052
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 44: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 82: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 84: patch: command
not found
error:
a/itests/hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java:
No such file or directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833262 - PreCommit-HIVE-Build
> Add tests for beeline force option
> --
>
> Key: HIVE-14935
> URL: https://issues.apache.org/jira/browse/HIVE-14935
> Project: Hive
> Issue Type: Test
>Reporter: Kavan Suresh
>Assignee: Kavan Suresh
> Fix For: 2.2.0
>
> Attachments: HIVE-14935.1.patch
>
>
> Add unit test for beeline with force option to ensure continuation of running
> script even after errors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14643) handle ctas
[
https://issues.apache.org/jira/browse/HIVE-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573985#comment-15573985
]
Hive QA commented on HIVE-14643:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833236/HIVE-14643.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1550/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1550/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1550/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:03:51.514
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-1550/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:03:51.517
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:03:52.378
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 44: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 82: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 84: patch: command
not found
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java:304
error: ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java: patch
does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java:3766
error: ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java: patch does
not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:1514
error: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java: patch does not
apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:1840
error: ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:
patch does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcFactory.java:223
error:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcFactory.java:
patch does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java:307
error: ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java: patch does
not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:6544
error: ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java: patch
does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java:205
error: ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java: patch
does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/plan/FileSinkDesc.java:180
error: ql/src/java/org/apache/hadoop/hive/ql/plan/FileSinkDesc.java: patch does
not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java:60
error: ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java: patch d
[jira] [Commented] (HIVE-14921) Move slow CliDriver tests to MiniLlap - part 2
[
https://issues.apache.org/jira/browse/HIVE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573982#comment-15573982
]
Hive QA commented on HIVE-14921:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833204/HIVE-14921.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1548/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1548/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1548/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:02:45.971
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-1548/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:02:45.973
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:02:46.801
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 44: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 82: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 84: patch: command
not found
error: a/itests/src/test/resources/testconfiguration.properties: No such file
or directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833204 - PreCommit-HIVE-Build
> Move slow CliDriver tests to MiniLlap - part 2
> --
>
> Key: HIVE-14921
> URL: https://issues.apache.org/jira/browse/HIVE-14921
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14921.1.patch, HIVE-14921.1.patch,
> HIVE-14921.2.patch, HIVE-14921.2.patch, HIVE-14921.3.patch
>
>
> Continuation to HIVE-14877
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14799) Query operation are not thread safe during its cancellation
[
https://issues.apache.org/jira/browse/HIVE-14799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573983#comment-15573983
]
Hive QA commented on HIVE-14799:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833239/HIVE-14799.7.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1549/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1549/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1549/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:03:18.432
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-1549/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:03:18.434
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:03:19.285
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 44: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 82: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 84: patch: command
not found
error: a/ql/src/java/org/apache/hadoop/hive/ql/Driver.java: No such file or
directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833239 - PreCommit-HIVE-Build
> Query operation are not thread safe during its cancellation
> ---
>
> Key: HIVE-14799
> URL: https://issues.apache.org/jira/browse/HIVE-14799
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Chaoyu Tang
>Assignee: Chaoyu Tang
> Attachments: HIVE-14799.1.patch, HIVE-14799.2.patch,
> HIVE-14799.3.patch, HIVE-14799.4.patch, HIVE-14799.5.patch,
> HIVE-14799.5.patch, HIVE-14799.6.patch, HIVE-14799.6.patch,
> HIVE-14799.7.patch, HIVE-14799.patch
>
>
> When a query is cancelled either via Beeline (Ctrl-C) or API call
> TCLIService.Client.CancelOperation, SQLOperation.cancel is invoked in a
> different thread from that running the query to close/destroy its
> encapsulated Driver object. Both SQLOperation and Driver are not thread-safe
> which could sometimes result in Runtime exceptions like NPE. The errors from
> the running query are not handled properly therefore probably causing some
> stuffs (files, locks etc) not being cleaned after the query termination.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14926) Keep Schema in consistent state where schemaTool fails or succeeds.
[
https://issues.apache.org/jira/browse/HIVE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573980#comment-15573980
]
Hive QA commented on HIVE-14926:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833201/HIVE-14926.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1547/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1547/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1547/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:02:14.908
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-1547/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:02:14.910
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
b9e8157..eacf9f9 hive-14535 -> origin/hive-14535
+ git reset --hard HEAD
HEAD is now at 10b866a HIVE-13798: Fix the unit test failure
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload (Aihua Xu,
reviewed by Ashutosh Chauhan) (addendum)
+ git clean -f -d
+ git checkout master
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:02:17.719
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 44: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 82: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 84: patch: command
not found
error: a/beeline/src/java/org/apache/hive/beeline/HiveSchemaHelper.java: No
such file or directory
error: a/beeline/src/java/org/apache/hive/beeline/HiveSchemaTool.java: No such
file or directory
error:
a/itests/hive-unit/src/test/java/org/apache/hive/beeline/TestSchemaTool.java:
No such file or directory
error:
a/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreSchemaInfo.java:
No such file or directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833201 - PreCommit-HIVE-Build
> Keep Schema in consistent state where schemaTool fails or succeeds.
> -
>
> Key: HIVE-14926
> URL: https://issues.apache.org/jira/browse/HIVE-14926
> Project: Hive
> Issue Type: Improvement
> Components: Database/Schema
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-14926.1.patch, HIVE-14926.2.patch
>
>
> SchemaTool uses autocommit right now when executing the upgrade or init
> scripts. Seems we should use database transaction to commit or roll back to
> keep schema consistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14864) Distcp is not called from MoveTask when src is a directory
[
https://issues.apache.org/jira/browse/HIVE-14864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573955#comment-15573955
]
Hive QA commented on HIVE-14864:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833234/HIVE-14864.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 89 failed/errored test(s), 10534 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_7]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[crtseltbl_serdeprops]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_3]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_4]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_mat_4]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_mat_5]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_00_nonpart_empty]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_01_nonpart]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_02_00_part_empty]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_02_part]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_03_nonpart_over_compat]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_04_all_part]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_04_evolved_parts]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_05_some_part]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_06_one_part]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_07_all_part_over_nonoverlap]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_08_nonpart_rename]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_09_part_spec_nonoverlap]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_10_external_managed]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_11_managed_external]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_12_external_location]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_13_managed_location]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_14_managed_location_over_existing]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_15_external_part]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_16_part_external]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_17_part_managed]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_18_part_external]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_19_00_part_external_location]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_19_part_external_location]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_20_part_managed_location]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_21_export_authsuccess]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_22_import_exist_authsuccess]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_23_import_part_authsuccess]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_24_import_nonexist_authsuccess]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_25_export_parentpath_has_inaccessible_children]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_hidden_files]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[foldts]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_9]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[materialized_view_describe]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[reloadJar]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[repl_2_exim_basic]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[repl_3_exim_metadata]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_partial_size]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_gb1]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_join1]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_subquery1]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_tablesample_rows]
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_load_data_to_encrypted_tables]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[import_exported_table]
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[import_exported_table]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_import]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[ctas_noemptyfolder]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[exim_01_nonpart_over_loaded]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[exim_02_all_part_over_overl
[jira] [Updated] (HIVE-14876) make the number of rows to fetch from various HS2 clients/servers configurable
[ https://issues.apache.org/jira/browse/HIVE-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-14876: Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Committed to master some time ago. > make the number of rows to fetch from various HS2 clients/servers configurable > -- > > Key: HIVE-14876 > URL: https://issues.apache.org/jira/browse/HIVE-14876 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 2.2.0 > > Attachments: HIVE-14876.01.patch, HIVE-14876.patch > > > Right now, it's hardcoded to a variety of values -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14940) MiniTezCliDriver - switch back to SQL metastore as default
[
https://issues.apache.org/jira/browse/HIVE-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573800#comment-15573800
]
Hive QA commented on HIVE-14940:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833235/HIVE-14940.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 10532 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[reloadJar]
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1545/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1545/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-Build-1545/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833235 - PreCommit-HIVE-Build
> MiniTezCliDriver - switch back to SQL metastore as default
> --
>
> Key: HIVE-14940
> URL: https://issues.apache.org/jira/browse/HIVE-14940
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14940.1.patch, HIVE-14940.1.patch,
> HIVE-14940.2.patch, HIVE-14940.2.patch, HIVE-14940.3.patch,
> HIVE-14940.3.patch, HIVE-14940.4.patch
>
>
> HBase setup for metastore in MiniTez is taking around 3 mins for setup. The
> actual runtime of the queries is typically much lower. To avoid the high
> overhead we should be switch back to SQL metastore as default and if required
> we can have dedicated set of tests that run against hbase metastore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14891) Parallelize TestHCatStorer
[ https://issues.apache.org/jira/browse/HIVE-14891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-14891: Status: Patch Available (was: Open) > Parallelize TestHCatStorer > -- > > Key: HIVE-14891 > URL: https://issues.apache.org/jira/browse/HIVE-14891 > Project: Hive > Issue Type: Sub-task > Components: HCatalog, Testing Infrastructure >Affects Versions: 2.1.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-14891.1.patch > > > Currently TestHCatStorer runs as a parameterized test, where it runs the same > tests for each storage format but within the same junit test case. This > prevents it from being parallelized using ptest where parallelism granularity > is at a test case level. Instead of using parameterized tests, it makes sense > to create a new test case for each storage format. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14891) Parallelize TestHCatStorer
[ https://issues.apache.org/jira/browse/HIVE-14891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573764#comment-15573764 ] Vaibhav Gumashta commented on HIVE-14891: - cc [~sseth] [~sushanth] > Parallelize TestHCatStorer > -- > > Key: HIVE-14891 > URL: https://issues.apache.org/jira/browse/HIVE-14891 > Project: Hive > Issue Type: Sub-task > Components: HCatalog, Testing Infrastructure >Affects Versions: 2.1.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-14891.1.patch > > > Currently TestHCatStorer runs as a parameterized test, where it runs the same > tests for each storage format but within the same junit test case. This > prevents it from being parallelized using ptest where parallelism granularity > is at a test case level. Instead of using parameterized tests, it makes sense > to create a new test case for each storage format. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14891) Parallelize TestHCatStorer
[ https://issues.apache.org/jira/browse/HIVE-14891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-14891: Affects Version/s: (was: 2.2.0) 2.1.0 > Parallelize TestHCatStorer > -- > > Key: HIVE-14891 > URL: https://issues.apache.org/jira/browse/HIVE-14891 > Project: Hive > Issue Type: Sub-task > Components: HCatalog, Testing Infrastructure >Affects Versions: 2.1.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-14891.1.patch > > > Currently TestHCatStorer runs as a parameterized test, where it runs the same > tests for each storage format but within the same junit test case. This > prevents it from being parallelized using ptest where parallelism granularity > is at a test case level. Instead of using parameterized tests, it makes sense > to create a new test case for each storage format. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14891) Parallelize TestHCatStorer
[ https://issues.apache.org/jira/browse/HIVE-14891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-14891: Attachment: HIVE-14891.1.patch > Parallelize TestHCatStorer > -- > > Key: HIVE-14891 > URL: https://issues.apache.org/jira/browse/HIVE-14891 > Project: Hive > Issue Type: Sub-task > Components: HCatalog, Testing Infrastructure >Affects Versions: 2.1.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-14891.1.patch > > > Currently TestHCatStorer runs as a parameterized test, where it runs the same > tests for each storage format but within the same junit test case. This > prevents it from being parallelized using ptest where parallelism granularity > is at a test case level. Instead of using parameterized tests, it makes sense > to create a new test case for each storage format. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14935) Add tests for beeline force option
[ https://issues.apache.org/jira/browse/HIVE-14935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kavan Suresh updated HIVE-14935: Attachment: HIVE-14935.1.patch > Add tests for beeline force option > -- > > Key: HIVE-14935 > URL: https://issues.apache.org/jira/browse/HIVE-14935 > Project: Hive > Issue Type: Test >Reporter: Kavan Suresh >Assignee: Kavan Suresh > Fix For: 2.2.0 > > Attachments: HIVE-14935.1.patch > > > Add unit test for beeline with force option to ensure continuation of running > script even after errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14935) Add tests for beeline force option
[ https://issues.apache.org/jira/browse/HIVE-14935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kavan Suresh updated HIVE-14935: Fix Version/s: 2.2.0 Status: Patch Available (was: In Progress) > Add tests for beeline force option > -- > > Key: HIVE-14935 > URL: https://issues.apache.org/jira/browse/HIVE-14935 > Project: Hive > Issue Type: Test >Reporter: Kavan Suresh >Assignee: Kavan Suresh > Fix For: 2.2.0 > > Attachments: HIVE-14935.1.patch > > > Add unit test for beeline with force option to ensure continuation of running > script even after errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HIVE-14935) Add tests for beeline force option
[ https://issues.apache.org/jira/browse/HIVE-14935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-14935 started by Kavan Suresh. --- > Add tests for beeline force option > -- > > Key: HIVE-14935 > URL: https://issues.apache.org/jira/browse/HIVE-14935 > Project: Hive > Issue Type: Test >Reporter: Kavan Suresh >Assignee: Kavan Suresh > > Add unit test for beeline with force option to ensure continuation of running > script even after errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11957) SHOW TRANSACTIONS should show queryID/agent id of the creator
[
https://issues.apache.org/jira/browse/HIVE-11957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573672#comment-15573672
]
Eugene Koifman commented on HIVE-11957:
---
are you sure adding new fields in the middle of "struct TxnInfo {" won't break
compatibility with old clients?
Seems like adding to the end is safer, though I don't know for sure
Have you considered timezone issues when formatting a timestamp in human
readable?
Is it using the same TZ as when it was generated.
SHOW LOCKS uses timestamps - so this makes it impossible to correlate the data
between the 2 outputs.
I think raw timestamp is easier/more reliable.
> SHOW TRANSACTIONS should show queryID/agent id of the creator
> -
>
> Key: HIVE-11957
> URL: https://issues.apache.org/jira/browse/HIVE-11957
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 1.0.0
>Reporter: Eugene Koifman
>Assignee: Wei Zheng
> Attachments: HIVE-11957.1.patch, HIVE-11957.2.patch,
> HIVE-11957.3.patch, HIVE-11957.4.patch, HIVE-11957.5.patch
>
>
> this would be very useful for debugging
> should also include heartbeat/create timestamps
> would be nice to support some filtering/sorting options, like sort by create
> time, agent id. filter by table, database, etc
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14950) Support integer data type
[
https://issues.apache.org/jira/browse/HIVE-14950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573651#comment-15573651
]
Hive QA commented on HIVE-14950:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833195/HIVE-14950.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 10534 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[reloadJar]
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1544/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1544/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-Build-1544/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833195 - PreCommit-HIVE-Build
> Support integer data type
> -
>
> Key: HIVE-14950
> URL: https://issues.apache.org/jira/browse/HIVE-14950
> Project: Hive
> Issue Type: Sub-task
> Components: SQL
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-14950.1.patch
>
>
> maybe its just me bumping into this difference again and again...
> but it's in the sql2011 standard...
> adding an alias for int would be easy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HIVE-14932) handle bucketing for MM tables
[ https://issues.apache.org/jira/browse/HIVE-14932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-14932: --- Assignee: Sergey Shelukhin > handle bucketing for MM tables > -- > > Key: HIVE-14932 > URL: https://issues.apache.org/jira/browse/HIVE-14932 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14640) handle hive.merge.*files in select queries
[ https://issues.apache.org/jira/browse/HIVE-14640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-14640: Resolution: Fixed Status: Resolved (was: Patch Available) Committed to branch > handle hive.merge.*files in select queries > -- > > Key: HIVE-14640 > URL: https://issues.apache.org/jira/browse/HIVE-14640 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: hive-14535 > > Attachments: HIVE-14640.WIP.patch, HIVE-14640.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14643) handle ctas
[ https://issues.apache.org/jira/browse/HIVE-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-14643: Fix Version/s: hive-14535 Status: Patch Available (was: Open) > handle ctas > --- > > Key: HIVE-14643 > URL: https://issues.apache.org/jira/browse/HIVE-14643 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: hive-14535 > > Attachments: HIVE-14643.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14926) Keep Schema in consistent state where schemaTool fails or succeeds.
[ https://issues.apache.org/jira/browse/HIVE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-14926: Status: In Progress (was: Patch Available) > Keep Schema in consistent state where schemaTool fails or succeeds. > - > > Key: HIVE-14926 > URL: https://issues.apache.org/jira/browse/HIVE-14926 > Project: Hive > Issue Type: Improvement > Components: Database/Schema >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14926.1.patch, HIVE-14926.2.patch > > > SchemaTool uses autocommit right now when executing the upgrade or init > scripts. Seems we should use database transaction to commit or roll back to > keep schema consistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14926) Keep Schema in consistent state where schemaTool fails or succeeds.
[ https://issues.apache.org/jira/browse/HIVE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-14926: Status: Patch Available (was: In Progress) > Keep Schema in consistent state where schemaTool fails or succeeds. > - > > Key: HIVE-14926 > URL: https://issues.apache.org/jira/browse/HIVE-14926 > Project: Hive > Issue Type: Improvement > Components: Database/Schema >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14926.1.patch, HIVE-14926.2.patch > > > SchemaTool uses autocommit right now when executing the upgrade or init > scripts. Seems we should use database transaction to commit or roll back to > keep schema consistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14799) Query operation are not thread safe during its cancellation
[ https://issues.apache.org/jira/browse/HIVE-14799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang updated HIVE-14799: --- Attachment: HIVE-14799.7.patch revised the patch to address an issue raised by [~ychena]. [~sershe] [~ychena], could you review it. > Query operation are not thread safe during its cancellation > --- > > Key: HIVE-14799 > URL: https://issues.apache.org/jira/browse/HIVE-14799 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Chaoyu Tang >Assignee: Chaoyu Tang > Attachments: HIVE-14799.1.patch, HIVE-14799.2.patch, > HIVE-14799.3.patch, HIVE-14799.4.patch, HIVE-14799.5.patch, > HIVE-14799.5.patch, HIVE-14799.6.patch, HIVE-14799.6.patch, > HIVE-14799.7.patch, HIVE-14799.patch > > > When a query is cancelled either via Beeline (Ctrl-C) or API call > TCLIService.Client.CancelOperation, SQLOperation.cancel is invoked in a > different thread from that running the query to close/destroy its > encapsulated Driver object. Both SQLOperation and Driver are not thread-safe > which could sometimes result in Runtime exceptions like NPE. The errors from > the running query are not handled properly therefore probably causing some > stuffs (files, locks etc) not being cleaned after the query termination. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14476) Fix logging issue for branch-1
[ https://issues.apache.org/jira/browse/HIVE-14476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573529#comment-15573529 ] Tao Li commented on HIVE-14476: --- [~spena] What issues are you seeing locally? > Fix logging issue for branch-1 > -- > > Key: HIVE-14476 > URL: https://issues.apache.org/jira/browse/HIVE-14476 > Project: Hive > Issue Type: Bug >Reporter: Tao Li >Assignee: Tao Li > Attachments: HIVE-14476.1-branch-1.patch > > > This issue is from branch-1 code when we decide if a log entry is an > operational log or not (the operational logs are visible to the client). The > problem is that the code is checking the logging mode at the beginning of the > decide() method, while the logging mode is updated after that check. Due to > this issue, we ran into an issue that an operational log could be filtered > out if it's the very first log being checked from the this method. As a > result, that particular log is not showing up for the end user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14643) handle ctas
[ https://issues.apache.org/jira/browse/HIVE-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573516#comment-15573516 ] Sergey Shelukhin commented on HIVE-14643: - The patch. This also reworks the union logic which I noticed used the weird path structure (table/1/mm_..., creating multiple directories per write) instead of proper (table/mm_.../1) > handle ctas > --- > > Key: HIVE-14643 > URL: https://issues.apache.org/jira/browse/HIVE-14643 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-14643.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14643) handle ctas
[ https://issues.apache.org/jira/browse/HIVE-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-14643: Attachment: HIVE-14643.patch > handle ctas > --- > > Key: HIVE-14643 > URL: https://issues.apache.org/jira/browse/HIVE-14643 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-14643.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14940) MiniTezCliDriver - switch back to SQL metastore as default
[ https://issues.apache.org/jira/browse/HIVE-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14940: - Attachment: HIVE-14940.4.patch > MiniTezCliDriver - switch back to SQL metastore as default > -- > > Key: HIVE-14940 > URL: https://issues.apache.org/jira/browse/HIVE-14940 > Project: Hive > Issue Type: Sub-task > Components: Tests >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-14940.1.patch, HIVE-14940.1.patch, > HIVE-14940.2.patch, HIVE-14940.2.patch, HIVE-14940.3.patch, > HIVE-14940.3.patch, HIVE-14940.4.patch > > > HBase setup for metastore in MiniTez is taking around 3 mins for setup. The > actual runtime of the queries is typically much lower. To avoid the high > overhead we should be switch back to SQL metastore as default and if required > we can have dedicated set of tests that run against hbase metastore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14864) Distcp is not called from MoveTask when src is a directory
[
https://issues.apache.org/jira/browse/HIVE-14864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-14864:
Status: Patch Available (was: Open)
> Distcp is not called from MoveTask when src is a directory
> --
>
> Key: HIVE-14864
> URL: https://issues.apache.org/jira/browse/HIVE-14864
> Project: Hive
> Issue Type: Bug
>Reporter: Vihang Karajgaonkar
>Assignee: Sahil Takiar
> Attachments: HIVE-14864.patch
>
>
> In FileUtils.java the following code does not get executed even when src
> directory size is greater than HIVE_EXEC_COPYFILE_MAXSIZE because
> srcFS.getFileStatus(src).getLen() returns 0 when src is a directory. We
> should use srcFS.getContentSummary(src).getLength() instead.
> {noformat}
> /* Run distcp if source file/dir is too big */
> if (srcFS.getUri().getScheme().equals("hdfs") &&
> srcFS.getFileStatus(src).getLen() >
> conf.getLongVar(HiveConf.ConfVars.HIVE_EXEC_COPYFILE_MAXSIZE)) {
> LOG.info("Source is " + srcFS.getFileStatus(src).getLen() + " bytes.
> (MAX: " + conf.getLongVar(HiveConf.ConfVars.HIVE_EXEC_COPYFILE_MAXSIZE) +
> ")");
> LOG.info("Launch distributed copy (distcp) job.");
> HiveConfUtil.updateJobCredentialProviders(conf);
> copied = shims.runDistCp(src, dst, conf);
> if (copied && deleteSource) {
> srcFS.delete(src, true);
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14864) Distcp is not called from MoveTask when src is a directory
[
https://issues.apache.org/jira/browse/HIVE-14864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-14864:
Attachment: HIVE-14864.patch
Attaching a pre-lim patch just for reference. The patch is still a WIP.
Changes are pretty simple, {{FileUtils.copy}} uses {{getContentSummary}} to get
the number of files under the folder. It triggers a Distcp job based on the
size of files under the folder + the number of files under the folder.
If only a single file needs to be copied, the {{ContentSummary}} length will be
the size of that file, and the number of files under it will be 1.
For now the logic is pretty simple, if the number of files exceeds a threshold
set by {{hive.exec.copyfile.maxnumfiles}} (which defaults to 1) and the size of
the files exceeds a threshold set by hive.exec.copyfile.maxsize (which defaults
to 32 MB), the Distcp job will be triggered.
So basically any folder that contains more than 1 file and whose total contents
is greater than 32 MB.
> Distcp is not called from MoveTask when src is a directory
> --
>
> Key: HIVE-14864
> URL: https://issues.apache.org/jira/browse/HIVE-14864
> Project: Hive
> Issue Type: Bug
>Reporter: Vihang Karajgaonkar
>Assignee: Sahil Takiar
> Attachments: HIVE-14864.patch
>
>
> In FileUtils.java the following code does not get executed even when src
> directory size is greater than HIVE_EXEC_COPYFILE_MAXSIZE because
> srcFS.getFileStatus(src).getLen() returns 0 when src is a directory. We
> should use srcFS.getContentSummary(src).getLength() instead.
> {noformat}
> /* Run distcp if source file/dir is too big */
> if (srcFS.getUri().getScheme().equals("hdfs") &&
> srcFS.getFileStatus(src).getLen() >
> conf.getLongVar(HiveConf.ConfVars.HIVE_EXEC_COPYFILE_MAXSIZE)) {
> LOG.info("Source is " + srcFS.getFileStatus(src).getLen() + " bytes.
> (MAX: " + conf.getLongVar(HiveConf.ConfVars.HIVE_EXEC_COPYFILE_MAXSIZE) +
> ")");
> LOG.info("Launch distributed copy (distcp) job.");
> HiveConfUtil.updateJobCredentialProviders(conf);
> copied = shims.runDistCp(src, dst, conf);
> if (copied && deleteSource) {
> srcFS.delete(src, true);
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14940) MiniTezCliDriver - switch back to SQL metastore as default
[
https://issues.apache.org/jira/browse/HIVE-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573501#comment-15573501
]
Hive QA commented on HIVE-14940:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833233/HIVE-14940.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1543/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1543/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-Build-1543/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-13 23:13:53.170
+ [[ -n /usr/java/jdk1.8.0_25 ]]
+ export JAVA_HOME=/usr/java/jdk1.8.0_25
+ JAVA_HOME=/usr/java/jdk1.8.0_25
+ export
PATH=/usr/java/jdk1.8.0_25/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+
PATH=/usr/java/jdk1.8.0_25/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-1543/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-13 23:13:53.176
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-13 23:13:54.051
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
error: a/itests/src/test/resources/testconfiguration.properties: No such file
or directory
error:
a/itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CliConfigs.java:
No such file or directory
error:
a/ql/src/test/queries/clientpositive/partition_column_names_with_leading_and_trailing_spaces.q:
No such file or directory
error: a/ql/src/test/results/clientpositive/tez/explainanalyze_5.q.out: No such
file or directory
error:
a/ql/src/test/results/clientpositive/tez/partition_column_names_with_leading_and_trailing_spaces.q.out:
No such file or directory
error: a/ql/src/test/results/clientpositive/tez/unionDistinct_2.q.out: No such
file or directory
error:
a/ql/src/test/results/clientpositive/tez/vector_join_part_col_char.q.out: No
such file or directory
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833233 - PreCommit-HIVE-Build
> MiniTezCliDriver - switch back to SQL metastore as default
> --
>
> Key: HIVE-14940
> URL: https://issues.apache.org/jira/browse/HIVE-14940
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14940.1.patch, HIVE-14940.1.patch,
> HIVE-14940.2.patch, HIVE-14940.2.patch, HIVE-14940.3.patch, HIVE-14940.3.patch
>
>
> HBase setup for metastore in MiniTez is taking around 3 mins for setup. The
> actual runtime of the queries is typically much lower. To avoid the high
> overhead we should be switch back to SQL metastore as default and if required
> we can have dedicated set of tests that run against hbase metastore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14940) MiniTezCliDriver - switch back to SQL metastore as default
[ https://issues.apache.org/jira/browse/HIVE-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14940: - Attachment: HIVE-14940.3.patch > MiniTezCliDriver - switch back to SQL metastore as default > -- > > Key: HIVE-14940 > URL: https://issues.apache.org/jira/browse/HIVE-14940 > Project: Hive > Issue Type: Sub-task > Components: Tests >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-14940.1.patch, HIVE-14940.1.patch, > HIVE-14940.2.patch, HIVE-14940.2.patch, HIVE-14940.3.patch, HIVE-14940.3.patch > > > HBase setup for metastore in MiniTez is taking around 3 mins for setup. The > actual runtime of the queries is typically much lower. To avoid the high > overhead we should be switch back to SQL metastore as default and if required > we can have dedicated set of tests that run against hbase metastore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-14864) Distcp is not called from MoveTask when src is a directory
[
https://issues.apache.org/jira/browse/HIVE-14864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar reassigned HIVE-14864:
---
Assignee: Sahil Takiar (was: Vihang Karajgaonkar)
> Distcp is not called from MoveTask when src is a directory
> --
>
> Key: HIVE-14864
> URL: https://issues.apache.org/jira/browse/HIVE-14864
> Project: Hive
> Issue Type: Bug
>Reporter: Vihang Karajgaonkar
>Assignee: Sahil Takiar
>
> In FileUtils.java the following code does not get executed even when src
> directory size is greater than HIVE_EXEC_COPYFILE_MAXSIZE because
> srcFS.getFileStatus(src).getLen() returns 0 when src is a directory. We
> should use srcFS.getContentSummary(src).getLength() instead.
> {noformat}
> /* Run distcp if source file/dir is too big */
> if (srcFS.getUri().getScheme().equals("hdfs") &&
> srcFS.getFileStatus(src).getLen() >
> conf.getLongVar(HiveConf.ConfVars.HIVE_EXEC_COPYFILE_MAXSIZE)) {
> LOG.info("Source is " + srcFS.getFileStatus(src).getLen() + " bytes.
> (MAX: " + conf.getLongVar(HiveConf.ConfVars.HIVE_EXEC_COPYFILE_MAXSIZE) +
> ")");
> LOG.info("Launch distributed copy (distcp) job.");
> HiveConfUtil.updateJobCredentialProviders(conf);
> copied = shims.runDistCp(src, dst, conf);
> if (copied && deleteSource) {
> srcFS.delete(src, true);
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14940) MiniTezCliDriver - switch back to SQL metastore as default
[
https://issues.apache.org/jira/browse/HIVE-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573477#comment-15573477
]
Hive QA commented on HIVE-14940:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833227/HIVE-14940.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1542/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1542/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-Build-1542/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-13 23:01:59.740
+ [[ -n /usr/java/jdk1.8.0_25 ]]
+ export JAVA_HOME=/usr/java/jdk1.8.0_25
+ JAVA_HOME=/usr/java/jdk1.8.0_25
+ export
PATH=/usr/java/jdk1.8.0_25/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+
PATH=/usr/java/jdk1.8.0_25/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-1542/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-13 23:01:59.743
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-13 23:02:00.627
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
error: a/itests/src/test/resources/testconfiguration.properties: No such file
or directory
error:
a/itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CliConfigs.java:
No such file or directory
error:
a/ql/src/test/queries/clientpositive/partition_column_names_with_leading_and_trailing_spaces.q:
No such file or directory
error: a/ql/src/test/results/clientpositive/tez/explainanalyze_5.q.out: No such
file or directory
error:
a/ql/src/test/results/clientpositive/tez/partition_column_names_with_leading_and_trailing_spaces.q.out:
No such file or directory
error: a/ql/src/test/results/clientpositive/tez/unionDistinct_2.q.out: No such
file or directory
error:
a/ql/src/test/results/clientpositive/tez/vector_join_part_col_char.q.out: No
such file or directory
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833227 - PreCommit-HIVE-Build
> MiniTezCliDriver - switch back to SQL metastore as default
> --
>
> Key: HIVE-14940
> URL: https://issues.apache.org/jira/browse/HIVE-14940
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14940.1.patch, HIVE-14940.1.patch,
> HIVE-14940.2.patch, HIVE-14940.2.patch, HIVE-14940.3.patch
>
>
> HBase setup for metastore in MiniTez is taking around 3 mins for setup. The
> actual runtime of the queries is typically much lower. To avoid the high
> overhead we should be switch back to SQL metastore as default and if required
> we can have dedicated set of tests that run against hbase metastore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14940) MiniTezCliDriver - switch back to SQL metastore as default
[
https://issues.apache.org/jira/browse/HIVE-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573460#comment-15573460
]
Hive QA commented on HIVE-14940:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833227/HIVE-14940.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1541/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1541/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-Build-1541/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-13 22:51:37.564
+ [[ -n /usr/java/jdk1.8.0_25 ]]
+ export JAVA_HOME=/usr/java/jdk1.8.0_25
+ JAVA_HOME=/usr/java/jdk1.8.0_25
+ export
PATH=/usr/java/jdk1.8.0_25/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+
PATH=/usr/java/jdk1.8.0_25/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-1541/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-13 22:51:37.569
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-13 22:51:38.500
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
error: a/itests/src/test/resources/testconfiguration.properties: No such file
or directory
error:
a/itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CliConfigs.java:
No such file or directory
error:
a/ql/src/test/queries/clientpositive/partition_column_names_with_leading_and_trailing_spaces.q:
No such file or directory
error: a/ql/src/test/results/clientpositive/tez/explainanalyze_5.q.out: No such
file or directory
error:
a/ql/src/test/results/clientpositive/tez/partition_column_names_with_leading_and_trailing_spaces.q.out:
No such file or directory
error: a/ql/src/test/results/clientpositive/tez/unionDistinct_2.q.out: No such
file or directory
error:
a/ql/src/test/results/clientpositive/tez/vector_join_part_col_char.q.out: No
such file or directory
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833227 - PreCommit-HIVE-Build
> MiniTezCliDriver - switch back to SQL metastore as default
> --
>
> Key: HIVE-14940
> URL: https://issues.apache.org/jira/browse/HIVE-14940
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14940.1.patch, HIVE-14940.1.patch,
> HIVE-14940.2.patch, HIVE-14940.2.patch, HIVE-14940.3.patch
>
>
> HBase setup for metastore in MiniTez is taking around 3 mins for setup. The
> actual runtime of the queries is typically much lower. To avoid the high
> overhead we should be switch back to SQL metastore as default and if required
> we can have dedicated set of tests that run against hbase metastore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14940) MiniTezCliDriver - switch back to SQL metastore as default
[ https://issues.apache.org/jira/browse/HIVE-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14940: - Attachment: HIVE-14940.3.patch > MiniTezCliDriver - switch back to SQL metastore as default > -- > > Key: HIVE-14940 > URL: https://issues.apache.org/jira/browse/HIVE-14940 > Project: Hive > Issue Type: Sub-task > Components: Tests >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-14940.1.patch, HIVE-14940.1.patch, > HIVE-14940.2.patch, HIVE-14940.2.patch, HIVE-14940.3.patch > > > HBase setup for metastore in MiniTez is taking around 3 mins for setup. The > actual runtime of the queries is typically much lower. To avoid the high > overhead we should be switch back to SQL metastore as default and if required > we can have dedicated set of tests that run against hbase metastore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14476) Fix logging issue for branch-1
[ https://issues.apache.org/jira/browse/HIVE-14476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573427#comment-15573427 ] Tao Li commented on HIVE-14476: --- Sorry, I forgot "-Phadoop-2". Now it builds fine for me. > Fix logging issue for branch-1 > -- > > Key: HIVE-14476 > URL: https://issues.apache.org/jira/browse/HIVE-14476 > Project: Hive > Issue Type: Bug >Reporter: Tao Li >Assignee: Tao Li > Attachments: HIVE-14476.1-branch-1.patch > > > This issue is from branch-1 code when we decide if a log entry is an > operational log or not (the operational logs are visible to the client). The > problem is that the code is checking the logging mode at the beginning of the > decide() method, while the logging mode is updated after that check. Due to > this issue, we ran into an issue that an operational log could be filtered > out if it's the very first log being checked from the this method. As a > result, that particular log is not showing up for the end user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14940) MiniTezCliDriver - switch back to SQL metastore as default
[ https://issues.apache.org/jira/browse/HIVE-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14940: - Status: Open (was: Patch Available) > MiniTezCliDriver - switch back to SQL metastore as default > -- > > Key: HIVE-14940 > URL: https://issues.apache.org/jira/browse/HIVE-14940 > Project: Hive > Issue Type: Sub-task > Components: Tests >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-14940.1.patch, HIVE-14940.1.patch, > HIVE-14940.2.patch, HIVE-14940.2.patch > > > HBase setup for metastore in MiniTez is taking around 3 mins for setup. The > actual runtime of the queries is typically much lower. To avoid the high > overhead we should be switch back to SQL metastore as default and if required > we can have dedicated set of tests that run against hbase metastore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14940) MiniTezCliDriver - switch back to SQL metastore as default
[ https://issues.apache.org/jira/browse/HIVE-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14940: - Status: Patch Available (was: Open) > MiniTezCliDriver - switch back to SQL metastore as default > -- > > Key: HIVE-14940 > URL: https://issues.apache.org/jira/browse/HIVE-14940 > Project: Hive > Issue Type: Sub-task > Components: Tests >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-14940.1.patch, HIVE-14940.1.patch, > HIVE-14940.2.patch, HIVE-14940.2.patch > > > HBase setup for metastore in MiniTez is taking around 3 mins for setup. The > actual runtime of the queries is typically much lower. To avoid the high > overhead we should be switch back to SQL metastore as default and if required > we can have dedicated set of tests that run against hbase metastore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14940) MiniTezCliDriver - switch back to SQL metastore as default
[ https://issues.apache.org/jira/browse/HIVE-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14940: - Attachment: HIVE-14940.2.patch > MiniTezCliDriver - switch back to SQL metastore as default > -- > > Key: HIVE-14940 > URL: https://issues.apache.org/jira/browse/HIVE-14940 > Project: Hive > Issue Type: Sub-task > Components: Tests >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-14940.1.patch, HIVE-14940.1.patch, > HIVE-14940.2.patch, HIVE-14940.2.patch > > > HBase setup for metastore in MiniTez is taking around 3 mins for setup. The > actual runtime of the queries is typically much lower. To avoid the high > overhead we should be switch back to SQL metastore as default and if required > we can have dedicated set of tests that run against hbase metastore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14940) MiniTezCliDriver - switch back to SQL metastore as default
[
https://issues.apache.org/jira/browse/HIVE-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573397#comment-15573397
]
Hive QA commented on HIVE-14940:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833205/HIVE-14940.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1540/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1540/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-Build-1540/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-13 22:26:34.739
+ [[ -n /usr/java/jdk1.8.0_25 ]]
+ export JAVA_HOME=/usr/java/jdk1.8.0_25
+ JAVA_HOME=/usr/java/jdk1.8.0_25
+ export
PATH=/usr/java/jdk1.8.0_25/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+
PATH=/usr/java/jdk1.8.0_25/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-1540/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-13 22:26:34.743
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git clean -f -d
Removing ql/src/test/queries/clientpositive/primitive_types.q
Removing ql/src/test/results/clientpositive/primitive_types.q.out
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-13 22:26:35.778
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
error: a/itests/src/test/resources/testconfiguration.properties: No such file
or directory
error:
a/itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CliConfigs.java:
No such file or directory
error:
a/ql/src/test/queries/clientpositive/partition_column_names_with_leading_and_trailing_spaces.q:
No such file or directory
error: a/ql/src/test/results/clientpositive/tez/explainanalyze_5.q.out: No such
file or directory
error:
a/ql/src/test/results/clientpositive/tez/partition_column_names_with_leading_and_trailing_spaces.q.out:
No such file or directory
error: a/ql/src/test/results/clientpositive/tez/unionDistinct_2.q.out: No such
file or directory
error:
a/ql/src/test/results/clientpositive/tez/vector_join_part_col_char.q.out: No
such file or directory
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833205 - PreCommit-HIVE-Build
> MiniTezCliDriver - switch back to SQL metastore as default
> --
>
> Key: HIVE-14940
> URL: https://issues.apache.org/jira/browse/HIVE-14940
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 2.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14940.1.patch, HIVE-14940.1.patch,
> HIVE-14940.2.patch
>
>
> HBase setup for metastore in MiniTez is taking around 3 mins for setup. The
> actual runtime of the queries is typically much lower. To avoid the high
> overhead we should be switch back to SQL metastore as default and if required
> we can have dedicated set of tests that run against hbase metastore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14476) Fix logging issue for branch-1
[
https://issues.apache.org/jira/browse/HIVE-14476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573387#comment-15573387
]
Sergio Peña commented on HIVE-14476:
Nop, I got the same ones from ptest.
Use this {{mvn clean install -DskipTests -Phadoop-2}}
> Fix logging issue for branch-1
> --
>
> Key: HIVE-14476
> URL: https://issues.apache.org/jira/browse/HIVE-14476
> Project: Hive
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-14476.1-branch-1.patch
>
>
> This issue is from branch-1 code when we decide if a log entry is an
> operational log or not (the operational logs are visible to the client). The
> problem is that the code is checking the logging mode at the beginning of the
> decide() method, while the logging mode is updated after that check. Due to
> this issue, we ran into an issue that an operational log could be filtered
> out if it's the very first log being checked from the this method. As a
> result, that particular log is not showing up for the end user.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14476) Fix logging issue for branch-1
[ https://issues.apache.org/jira/browse/HIVE-14476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573383#comment-15573383 ] Tao Li commented on HIVE-14476: --- The build was against branch-1.2. I saw the same errors even without my change. > Fix logging issue for branch-1 > -- > > Key: HIVE-14476 > URL: https://issues.apache.org/jira/browse/HIVE-14476 > Project: Hive > Issue Type: Bug >Reporter: Tao Li >Assignee: Tao Li > Attachments: HIVE-14476.1-branch-1.patch > > > This issue is from branch-1 code when we decide if a log entry is an > operational log or not (the operational logs are visible to the client). The > problem is that the code is checking the logging mode at the beginning of the > decide() method, while the logging mode is updated after that check. Due to > this issue, we ran into an issue that an operational log could be filtered > out if it's the very first log being checked from the this method. As a > result, that particular log is not showing up for the end user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14476) Fix logging issue for branch-1
[ https://issues.apache.org/jira/browse/HIVE-14476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573377#comment-15573377 ] Tao Li commented on HIVE-14476: --- I ran into the build errors below. Are you seeing the same errors? [INFO] BUILD FAILURE [INFO] [INFO] Total time: 9.873 s [INFO] Finished at: 2016-10-13T15:15:01-07:00 [INFO] Final Memory: 58M/626M [INFO] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project hive-common: Compilation failure: Compilation failure: [ERROR] /Users/tli/dev2/hive/common/src/java/org/apache/hadoop/hive/common/ServerUtils.java:[23,28] cannot find symbol [ERROR] symbol: class FileSystem [ERROR] location: package org.apache.hadoop.fs [ERROR] /Users/tli/dev2/hive/common/src/java/org/apache/hadoop/hive/common/ServerUtils.java:[24,28] cannot find symbol [ERROR] symbol: class Path [ERROR] location: package org.apache.hadoop.fs [ERROR] /Users/tli/dev2/hive/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:[45,30] package org.apache.hadoop.conf does not exist [ERROR] /Users/tli/dev2/hive/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:[54,32] package org.apache.hadoop.mapred does not exist [ERROR] /Users/tli/dev2/hive/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:[55,34] package org.apache.hadoop.security does not exist [ERROR] /Users/tli/dev2/hive/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:[56,30] package org.apache.hadoop.util does not exist [ERROR] /Users/tli/dev2/hive/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:[64,31] cannot find symbol [ERROR] symbol: class Configuration [ERROR] /Users/tli/dev2/hive/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:[2453,31] cannot find symbol [ERROR] symbol: class Configuration [ERROR] location: class org.apache.hadoop.hive.conf.HiveConf [ERROR] /Users/tli/dev2/hive/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:[2458,32] cannot find symbol [ERROR] symbol: class Configuration [ERROR] location: class org.apache.hadoop.hive.conf.HiveConf [ERROR] /Users/tli/dev2/hive/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:[2471,33] cannot find symbol [ERROR] symbol: class Configuration [ERROR] location: class org.apache.hadoop.hive.conf.HiveConf [ERROR] /Users/tli/dev2/hive/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:[2475,33] cannot find symbol [ERROR] symbol: class Configuration [ERROR] location: class org.apache.hadoop.hive.conf.HiveConf [ERROR] /Users/tli/dev2/hive/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:[2547,33] cannot find symbol [ERROR] symbol: class Configuration [ERROR] location: class org.apache.hadoop.hive.conf.HiveConf > Fix logging issue for branch-1 > -- > > Key: HIVE-14476 > URL: https://issues.apache.org/jira/browse/HIVE-14476 > Project: Hive > Issue Type: Bug >Reporter: Tao Li >Assignee: Tao Li > Attachments: HIVE-14476.1-branch-1.patch > > > This issue is from branch-1 code when we decide if a log entry is an > operational log or not (the operational logs are visible to the client). The > problem is that the code is checking the logging mode at the beginning of the > decide() method, while the logging mode is updated after that check. Due to > this issue, we ran into an issue that an operational log could be filtered > out if it's the very first log being checked from the this method. As a > result, that particular log is not showing up for the end user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14476) Fix logging issue for branch-1
[ https://issues.apache.org/jira/browse/HIVE-14476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573327#comment-15573327 ] Sergio Peña commented on HIVE-14476: [~taoli-hwx] I tried compiling the branch-1 with JDK7, and it does not compile. It might be broken by another commit, and that's why ptest fails too. Does it compile for you? > Fix logging issue for branch-1 > -- > > Key: HIVE-14476 > URL: https://issues.apache.org/jira/browse/HIVE-14476 > Project: Hive > Issue Type: Bug >Reporter: Tao Li >Assignee: Tao Li > Attachments: HIVE-14476.1-branch-1.patch > > > This issue is from branch-1 code when we decide if a log entry is an > operational log or not (the operational logs are visible to the client). The > problem is that the code is checking the logging mode at the beginning of the > decide() method, while the logging mode is updated after that check. Due to > this issue, we ran into an issue that an operational log could be filtered > out if it's the very first log being checked from the this method. As a > result, that particular log is not showing up for the end user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14921) Move slow CliDriver tests to MiniLlap - part 2
[ https://issues.apache.org/jira/browse/HIVE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573315#comment-15573315 ] Siddharth Seth commented on HIVE-14921: --- +1, looks good to me. Lets wait another run to see if the test failures are related. I think testCliDriver[reloadJar] failure was introduced recently. > Move slow CliDriver tests to MiniLlap - part 2 > -- > > Key: HIVE-14921 > URL: https://issues.apache.org/jira/browse/HIVE-14921 > Project: Hive > Issue Type: Sub-task > Components: Tests >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-14921.1.patch, HIVE-14921.1.patch, > HIVE-14921.2.patch, HIVE-14921.2.patch, HIVE-14921.3.patch > > > Continuation to HIVE-14877 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14924) MSCK REPAIR table with single threaded is throwing null pointer exception
[ https://issues.apache.org/jira/browse/HIVE-14924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573268#comment-15573268 ] Ashutosh Chauhan commented on HIVE-14924: - +1 > MSCK REPAIR table with single threaded is throwing null pointer exception > - > > Key: HIVE-14924 > URL: https://issues.apache.org/jira/browse/HIVE-14924 > Project: Hive > Issue Type: Bug > Components: CLI >Affects Versions: 2.2.0 >Reporter: Ratheesh Kamoor >Assignee: Pengcheng Xiong > Attachments: HIVE-14924.01.patch > > > MSCK REPAIR TABLE is throwing Null Pointer Exception while running on single > threaded mode (hive.mv.files.thread=0) > Error: > 2016-10-10T22:27:13,564 ERROR [e9ce04a8-2a84-426d-8e79-a2d15b8cee09 > main([])]: exec.DDLTask (DDLTask.java:failed(581)) - > java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkPartitionDirs(HiveMetaStoreChecker.java:423) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.findUnknownPartitions(HiveMetaStoreChecker.java:315) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkTable(HiveMetaStoreChecker.java:291) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkTable(HiveMetaStoreChecker.java:236) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkMetastore(HiveMetaStoreChecker.java:113) > at org.apache.hadoop.hive.ql.exec.DDLTask.msck(DDLTask.java:1834) > In order to reproduce: > set hive.mv.files.thread=0 and run MSCK REPAIR TABLE command -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14580) Introduce || operator
[
https://issues.apache.org/jira/browse/HIVE-14580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573258#comment-15573258
]
Hive QA commented on HIVE-14580:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833184/HIVE-14580.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 10531 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver-orc_llap.q-delete_where_non_partitioned.q-vector_groupby_mapjoin.q-and-27-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[reloadJar]
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1535/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1535/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-Build-1535/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833184 - PreCommit-HIVE-Build
> Introduce || operator
> -
>
> Key: HIVE-14580
> URL: https://issues.apache.org/jira/browse/HIVE-14580
> Project: Hive
> Issue Type: Sub-task
> Components: SQL
>Reporter: Ashutosh Chauhan
>Assignee: Zoltan Haindrich
> Attachments: HIVE-14580.1.patch, HIVE-14580.2.patch,
> HIVE-14580.3.patch
>
>
> Functionally equivalent to concat() udf. But standard allows usage of || for
> string concatenations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14929) Adding JDBC test for query cancellation scenario
[ https://issues.apache.org/jira/browse/HIVE-14929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-14929: -- Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Committed to master. Thanks Deepak. > Adding JDBC test for query cancellation scenario > > > Key: HIVE-14929 > URL: https://issues.apache.org/jira/browse/HIVE-14929 > Project: Hive > Issue Type: Test >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal > Fix For: 2.2.0 > > Attachments: HIVE-14929.1.patch, HIVE-14929.2.patch > > > There is some functional testing for query cancellation using JDBC which is > missing in unit tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14950) Support integer data type
[
https://issues.apache.org/jira/browse/HIVE-14950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573209#comment-15573209
]
Alan Gates commented on HIVE-14950:
---
If you're going to create KW_INTEGER as a new keyword there are several places
that KW_INT is referenced in IdentifiersParser.g that should also be updated.
I'm wondering if instead it makes more sense to change the definition of KW_INT
in HiveLexer.g to be:
{code}
KW_INT: 'INT' | 'INTEGER'
{code}
then you're guaranteed that they're interchangeable now and in the future.
> Support integer data type
> -
>
> Key: HIVE-14950
> URL: https://issues.apache.org/jira/browse/HIVE-14950
> Project: Hive
> Issue Type: Sub-task
> Components: SQL
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-14950.1.patch
>
>
> maybe its just me bumping into this difference again and again...
> but it's in the sql2011 standard...
> adding an alias for int would be easy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14940) MiniTezCliDriver - switch back to SQL metastore as default
[ https://issues.apache.org/jira/browse/HIVE-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14940: - Attachment: HIVE-14940.2.patch > MiniTezCliDriver - switch back to SQL metastore as default > -- > > Key: HIVE-14940 > URL: https://issues.apache.org/jira/browse/HIVE-14940 > Project: Hive > Issue Type: Sub-task > Components: Tests >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-14940.1.patch, HIVE-14940.1.patch, > HIVE-14940.2.patch > > > HBase setup for metastore in MiniTez is taking around 3 mins for setup. The > actual runtime of the queries is typically much lower. To avoid the high > overhead we should be switch back to SQL metastore as default and if required > we can have dedicated set of tests that run against hbase metastore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14921) Move slow CliDriver tests to MiniLlap - part 2
[ https://issues.apache.org/jira/browse/HIVE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14921: - Attachment: HIVE-14921.3.patch > Move slow CliDriver tests to MiniLlap - part 2 > -- > > Key: HIVE-14921 > URL: https://issues.apache.org/jira/browse/HIVE-14921 > Project: Hive > Issue Type: Sub-task > Components: Tests >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-14921.1.patch, HIVE-14921.1.patch, > HIVE-14921.2.patch, HIVE-14921.2.patch, HIVE-14921.3.patch > > > Continuation to HIVE-14877 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13554) [Umbrella] SQL:2011 compliance
[ https://issues.apache.org/jira/browse/HIVE-13554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573178#comment-15573178 ] Damien Carol commented on HIVE-13554: - [~ashutoshc] do you have any specification document? What is your reference? > [Umbrella] SQL:2011 compliance > -- > > Key: HIVE-13554 > URL: https://issues.apache.org/jira/browse/HIVE-13554 > Project: Hive > Issue Type: Improvement > Components: SQL >Affects Versions: 2.1.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > > There are various gaps in language which needs to be addressed to bring Hive > under SQL:2011 compliance -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14942) HS2 UI: Canceled queries show up in "Open Queries"
[ https://issues.apache.org/jira/browse/HIVE-14942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573164#comment-15573164 ] Tao Li commented on HIVE-14942: --- [~mohitsabharwal] Thanks for the review. Can you please commit it? > HS2 UI: Canceled queries show up in "Open Queries" > -- > > Key: HIVE-14942 > URL: https://issues.apache.org/jira/browse/HIVE-14942 > Project: Hive > Issue Type: Bug >Reporter: Tao Li >Assignee: Tao Li > Attachments: HIVE-14942.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14926) Keep Schema in consistent state where schemaTool fails or succeeds.
[ https://issues.apache.org/jira/browse/HIVE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-14926: Attachment: (was: HIVE-14926.2.patch) > Keep Schema in consistent state where schemaTool fails or succeeds. > - > > Key: HIVE-14926 > URL: https://issues.apache.org/jira/browse/HIVE-14926 > Project: Hive > Issue Type: Improvement > Components: Database/Schema >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14926.1.patch, HIVE-14926.2.patch > > > SchemaTool uses autocommit right now when executing the upgrade or init > scripts. Seems we should use database transaction to commit or roll back to > keep schema consistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14926) Keep Schema in consistent state where schemaTool fails or succeeds.
[ https://issues.apache.org/jira/browse/HIVE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-14926: Attachment: HIVE-14926.2.patch > Keep Schema in consistent state where schemaTool fails or succeeds. > - > > Key: HIVE-14926 > URL: https://issues.apache.org/jira/browse/HIVE-14926 > Project: Hive > Issue Type: Improvement > Components: Database/Schema >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14926.1.patch, HIVE-14926.2.patch > > > SchemaTool uses autocommit right now when executing the upgrade or init > scripts. Seems we should use database transaction to commit or roll back to > keep schema consistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14951) ArrayIndexOutOfBoundsException in GroupByOperator (Hive on Tez)
[ https://issues.apache.org/jira/browse/HIVE-14951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhiyuan Yang updated HIVE-14951: Description: Engine: Tez Query: select * from (select distinct a from f16) as f16, (select distinct a from f1) as fprime where f16.a = fprime.a; Table: create table f1 (a int, b string); create table f16 (a int, b string); Config: set hive.auto.convert.sortmerge.join=true; set hive.auto.convert.join=false; was: Query: select * from (select distinct a from f16) as f16, (select distinct a from f1) as fprime where f16.a = fprime.a; Table: create table f1 (a int, b string); create table f16 (a int, b string); Config: set hive.auto.convert.sortmerge.join=true; set hive.auto.convert.join=false; > ArrayIndexOutOfBoundsException in GroupByOperator (Hive on Tez) > --- > > Key: HIVE-14951 > URL: https://issues.apache.org/jira/browse/HIVE-14951 > Project: Hive > Issue Type: Bug >Reporter: Zhiyuan Yang > > Engine: > Tez > Query: > select * from (select distinct a from f16) as f16, (select distinct a from > f1) as fprime where f16.a = fprime.a; > Table: > create table f1 (a int, b string); > create table f16 (a int, b string); > Config: > set hive.auto.convert.sortmerge.join=true; > set hive.auto.convert.join=false; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14951) ArrayIndexOutOfBoundsException in GroupByOperator
[ https://issues.apache.org/jira/browse/HIVE-14951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhiyuan Yang updated HIVE-14951: Summary: ArrayIndexOutOfBoundsException in GroupByOperator (was: ArrayIndexOutOfBoundsException in GroupByOperator (Hive on Tez)) > ArrayIndexOutOfBoundsException in GroupByOperator > - > > Key: HIVE-14951 > URL: https://issues.apache.org/jira/browse/HIVE-14951 > Project: Hive > Issue Type: Bug >Reporter: Zhiyuan Yang > > Engine: > Tez > Query: > select * from (select distinct a from f16) as f16, (select distinct a from > f1) as fprime where f16.a = fprime.a; > Table: > create table f1 (a int, b string); > create table f16 (a int, b string); > Config: > set hive.auto.convert.sortmerge.join=true; > set hive.auto.convert.join=false; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14951) ArrayIndexOutOfBoundsException in GroupByOperator (Hive on Tez)
[ https://issues.apache.org/jira/browse/HIVE-14951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhiyuan Yang updated HIVE-14951: Summary: ArrayIndexOutOfBoundsException in GroupByOperator (Hive on Tez) (was: ArrayIndexOutOfBoundsException in GroupByOperator) > ArrayIndexOutOfBoundsException in GroupByOperator (Hive on Tez) > --- > > Key: HIVE-14951 > URL: https://issues.apache.org/jira/browse/HIVE-14951 > Project: Hive > Issue Type: Bug >Reporter: Zhiyuan Yang > > Query: > select * from (select distinct a from f16) as f16, (select distinct a from > f1) as fprime where f16.a = fprime.a; > Table: > create table f1 (a int, b string); > create table f16 (a int, b string); > Config: > set hive.auto.convert.sortmerge.join=true; > set hive.auto.convert.join=false; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14951) ArrayIndexOutOfBoundsException in GroupByOperator
[
https://issues.apache.org/jira/browse/HIVE-14951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573083#comment-15573083
]
Zhiyuan Yang commented on HIVE-14951:
-
Here's the error message:
{code:java}
Error: Error while running task ( failure ) :
attempt_1475017598908_0282_2_02_00_3:java.lang.RuntimeException:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row (tag=0) {"key":{"_col0":2},"value":null}
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row (tag=0) {"key":{"_col0":2},"value":null}
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:284)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:252)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150)
... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
while processing row (tag=0) {"key":{"_col0":2},"value":null}
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:352)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:274)
... 16 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row (tag=1) {"key":{"_col0":1},"value":null}
at
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.fetchOneRow(CommonMergeJoinOperator.java:416)
at
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.fetchNextGroup(CommonMergeJoinOperator.java:379)
at
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.doFirstFetchIfNeeded(CommonMergeJoinOperator.java:485)
at
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.process(CommonMergeJoinOperator.java:207)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:838)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(GroupByOperator.java:1016)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processAggr(GroupByOperator.java:821)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:695)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:761)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:343)
... 17 more
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row (tag=1) {"key":{"_col0":1},"value":null}
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:284)
at
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.fetchOneRow(CommonMergeJoinOperator.java:404)
... 26 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
while processing row (tag=1) {"key":{"_col0":1},"value":null}
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:352)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:274)
... 27 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:708)
at
o
[jira] [Updated] (HIVE-14926) Keep Schema in consistent state where schemaTool fails or succeeds.
[ https://issues.apache.org/jira/browse/HIVE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-14926: Attachment: (was: HIVE-14926.2.patch) > Keep Schema in consistent state where schemaTool fails or succeeds. > - > > Key: HIVE-14926 > URL: https://issues.apache.org/jira/browse/HIVE-14926 > Project: Hive > Issue Type: Improvement > Components: Database/Schema >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14926.1.patch, HIVE-14926.2.patch > > > SchemaTool uses autocommit right now when executing the upgrade or init > scripts. Seems we should use database transaction to commit or roll back to > keep schema consistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14926) Keep Schema in consistent state where schemaTool fails or succeeds.
[ https://issues.apache.org/jira/browse/HIVE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-14926: Attachment: HIVE-14926.2.patch > Keep Schema in consistent state where schemaTool fails or succeeds. > - > > Key: HIVE-14926 > URL: https://issues.apache.org/jira/browse/HIVE-14926 > Project: Hive > Issue Type: Improvement > Components: Database/Schema >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14926.1.patch, HIVE-14926.2.patch > > > SchemaTool uses autocommit right now when executing the upgrade or init > scripts. Seems we should use database transaction to commit or roll back to > keep schema consistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14926) Keep Schema in consistent state where schemaTool fails or succeeds.
[ https://issues.apache.org/jira/browse/HIVE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-14926: Attachment: HIVE-14926.2.patch PATCH-2: update the test cases. > Keep Schema in consistent state where schemaTool fails or succeeds. > - > > Key: HIVE-14926 > URL: https://issues.apache.org/jira/browse/HIVE-14926 > Project: Hive > Issue Type: Improvement > Components: Database/Schema >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14926.1.patch, HIVE-14926.2.patch > > > SchemaTool uses autocommit right now when executing the upgrade or init > scripts. Seems we should use database transaction to commit or roll back to > keep schema consistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14942) HS2 UI: Canceled queries show up in "Open Queries"
[ https://issues.apache.org/jira/browse/HIVE-14942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573060#comment-15573060 ] Mohit Sabharwal commented on HIVE-14942: LGTM, +1. Looks like I missed this case in HIVE-13099 > HS2 UI: Canceled queries show up in "Open Queries" > -- > > Key: HIVE-14942 > URL: https://issues.apache.org/jira/browse/HIVE-14942 > Project: Hive > Issue Type: Bug >Reporter: Tao Li >Assignee: Tao Li > Attachments: HIVE-14942.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13316) Upgrade to Calcite 1.10
[
https://issues.apache.org/jira/browse/HIVE-13316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573059#comment-15573059
]
Hive QA commented on HIVE-13316:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833169/HIVE-13316.08.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 10530 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver-orc_llap.q-delete_where_non_partitioned.q-vector_groupby_mapjoin.q-and-27-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[druid_basic2]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[druid_intervals]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[druid_timeseries]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[druid_topn]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explain_logical]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_sort_1_23]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_sort_skew_1_23]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[limit_pushdown3]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[limit_pushdown]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[offset_limit_ppd_optimizer]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[reloadJar]
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_4]
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[limit_pushdown]
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[lineage3]
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[table_access_keys_stats]
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_dynpart_hashjoin_1]
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_vector_dynpart_hashjoin_1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_4]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[partition_column_names_with_leading_and_trailing_spaces]
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query75]
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby_sort_1_23]
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby_sort_skew_1_23]
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[limit_pushdown]
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[table_access_keys_stats]
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1534/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1534/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-Build-1534/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 25 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833169 - PreCommit-HIVE-Build
> Upgrade to Calcite 1.10
> ---
>
> Key: HIVE-13316
> URL: https://issues.apache.org/jira/browse/HIVE-13316
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-13316.01.patch, HIVE-13316.02.patch,
> HIVE-13316.05.patch, HIVE-13316.07.patch, HIVE-13316.08.patch,
> HIVE-13316.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14927) Remove code duplication from tests in TestLdapAtnProviderWithMiniDS
[ https://issues.apache.org/jira/browse/HIVE-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Illya Yalovyy updated HIVE-14927: - Status: Patch Available (was: Open) > Remove code duplication from tests in TestLdapAtnProviderWithMiniDS > --- > > Key: HIVE-14927 > URL: https://issues.apache.org/jira/browse/HIVE-14927 > Project: Hive > Issue Type: Improvement > Components: Test >Reporter: Illya Yalovyy >Assignee: Illya Yalovyy > Attachments: HIVE-14927.1.patch > > > * Extract inner class User and implement a proper builder for it. > * Extract all common code to LdapAuthenticationTestCase class > ** setting up the test case >** executing test case >** result validation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14927) Remove code duplication from tests in TestLdapAtnProviderWithMiniDS
[ https://issues.apache.org/jira/browse/HIVE-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Illya Yalovyy updated HIVE-14927: - Status: Open (was: Patch Available) > Remove code duplication from tests in TestLdapAtnProviderWithMiniDS > --- > > Key: HIVE-14927 > URL: https://issues.apache.org/jira/browse/HIVE-14927 > Project: Hive > Issue Type: Improvement > Components: Test >Reporter: Illya Yalovyy >Assignee: Illya Yalovyy > Attachments: HIVE-14927.1.patch > > > * Extract inner class User and implement a proper builder for it. > * Extract all common code to LdapAuthenticationTestCase class > ** setting up the test case >** executing test case >** result validation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14942) HS2 UI: Canceled queries show up in "Open Queries"
[ https://issues.apache.org/jira/browse/HIVE-14942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573018#comment-15573018 ] Thejas M Nair commented on HIVE-14942: -- [~vgumashta] Can you please review this ? > HS2 UI: Canceled queries show up in "Open Queries" > -- > > Key: HIVE-14942 > URL: https://issues.apache.org/jira/browse/HIVE-14942 > Project: Hive > Issue Type: Bug >Reporter: Tao Li >Assignee: Tao Li > Attachments: HIVE-14942.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14950) Support integer data type
[ https://issues.apache.org/jira/browse/HIVE-14950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-14950: Attachment: HIVE-14950.1.patch > Support integer data type > - > > Key: HIVE-14950 > URL: https://issues.apache.org/jira/browse/HIVE-14950 > Project: Hive > Issue Type: Sub-task > Components: SQL >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Minor > Attachments: HIVE-14950.1.patch > > > maybe its just me bumping into this difference again and again... > but it's in the sql2011 standard... > adding an alias for int would be easy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14950) Support integer data type
[ https://issues.apache.org/jira/browse/HIVE-14950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-14950: Status: Patch Available (was: Open) > Support integer data type > - > > Key: HIVE-14950 > URL: https://issues.apache.org/jira/browse/HIVE-14950 > Project: Hive > Issue Type: Sub-task > Components: SQL >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Minor > Attachments: HIVE-14950.1.patch > > > maybe its just me bumping into this difference again and again... > but it's in the sql2011 standard... > adding an alias for int would be easy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
