[jira] [Commented] (HIVE-14069) update curator version to 2.10.0
[ https://issues.apache.org/jira/browse/HIVE-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244895#comment-16244895 ] Sergey Shelukhin commented on HIVE-14069: - Sounds good to me... > update curator version to 2.10.0 > - > > Key: HIVE-14069 > URL: https://issues.apache.org/jira/browse/HIVE-14069 > Project: Hive > Issue Type: Improvement > Components: HiveServer2, Metastore >Reporter: Thejas M Nair >Assignee: Jason Dere > Attachments: HIVE-14069.1.patch, HIVE-14069.2.patch, > HIVE-14069.3.patch > > > curator-2.10.0 has several bug fixes over current version (2.6.0), updating > would help improve stability. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14069) update curator version to 2.10.0
[ https://issues.apache.org/jira/browse/HIVE-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244885#comment-16244885 ] Jason Dere commented on HIVE-14069: --- Looks like HADOOP-14187 (Hadoop-3.0) updated Curator to 2.12.0 .. should we go with that version as well for Hive? > update curator version to 2.10.0 > - > > Key: HIVE-14069 > URL: https://issues.apache.org/jira/browse/HIVE-14069 > Project: Hive > Issue Type: Improvement > Components: HiveServer2, Metastore >Reporter: Thejas M Nair >Assignee: Jason Dere > Attachments: HIVE-14069.1.patch, HIVE-14069.2.patch, > HIVE-14069.3.patch > > > curator-2.10.0 has several bug fixes over current version (2.6.0), updating > would help improve stability. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14069) update curator version to 2.10.0
[ https://issues.apache.org/jira/browse/HIVE-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244708#comment-16244708 ] Sergey Shelukhin commented on HIVE-14069: - That makes sense for now... is there a JIRA to also upgrade the dependency in Hadoop? > update curator version to 2.10.0 > - > > Key: HIVE-14069 > URL: https://issues.apache.org/jira/browse/HIVE-14069 > Project: Hive > Issue Type: Improvement > Components: HiveServer2, Metastore >Reporter: Thejas M Nair >Assignee: Jason Dere > Attachments: HIVE-14069.1.patch, HIVE-14069.2.patch, > HIVE-14069.3.patch > > > curator-2.10.0 has several bug fixes over current version (2.6.0), updating > would help improve stability. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14069) update curator version to 2.10.0
[ https://issues.apache.org/jira/browse/HIVE-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244689#comment-16244689 ] Jason Dere commented on HIVE-14069: --- Shading Curator-2.10.0 within Hive to avoid Curator dependency conflicts with Hadoop. Have seen a few cases of CURATOR-209 where HiveServer2 will periodically get stuck in an infinite loop with ZK connection errors. Upgrading to Curator 2.10.0 is supposed to help with this issue. > update curator version to 2.10.0 > - > > Key: HIVE-14069 > URL: https://issues.apache.org/jira/browse/HIVE-14069 > Project: Hive > Issue Type: Improvement > Components: HiveServer2, Metastore >Reporter: Thejas M Nair >Assignee: Jason Dere > Attachments: HIVE-14069.1.patch, HIVE-14069.2.patch, > HIVE-14069.3.patch > > > curator-2.10.0 has several bug fixes over current version (2.6.0), updating > would help improve stability. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14069) update curator version to 2.10.0
[ https://issues.apache.org/jira/browse/HIVE-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244677#comment-16244677 ] Sergey Shelukhin commented on HIVE-14069: - What approach to what? Removing the dependency makes sense to me. > update curator version to 2.10.0 > - > > Key: HIVE-14069 > URL: https://issues.apache.org/jira/browse/HIVE-14069 > Project: Hive > Issue Type: Improvement > Components: HiveServer2, Metastore >Reporter: Thejas M Nair >Assignee: Jason Dere > Attachments: HIVE-14069.1.patch, HIVE-14069.2.patch, > HIVE-14069.3.patch > > > curator-2.10.0 has several bug fixes over current version (2.6.0), updating > would help improve stability. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14069) update curator version to 2.10.0
[ https://issues.apache.org/jira/browse/HIVE-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244551#comment-16244551 ] Jason Dere commented on HIVE-14069: --- [~thejas] [~sershe] any comments about this approach? > update curator version to 2.10.0 > - > > Key: HIVE-14069 > URL: https://issues.apache.org/jira/browse/HIVE-14069 > Project: Hive > Issue Type: Improvement > Components: HiveServer2, Metastore >Reporter: Thejas M Nair >Assignee: Jason Dere > Attachments: HIVE-14069.1.patch, HIVE-14069.2.patch, > HIVE-14069.3.patch > > > curator-2.10.0 has several bug fixes over current version (2.6.0), updating > would help improve stability. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14069) update curator version to 2.10.0
[
https://issues.apache.org/jira/browse/HIVE-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16243427#comment-16243427
]
Hive QA commented on HIVE-14069:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12896497/HIVE-14069.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 11372 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=62)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketmapjoin7]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[ct_noperm_loc]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucketmapjoin7]
(batchId=116)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14]
(batchId=243)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query23]
(batchId=243)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testApplyPlanQpChanges
(batchId=281)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7696/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7696/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7696/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12896497 - PreCommit-HIVE-Build
> update curator version to 2.10.0
> -
>
> Key: HIVE-14069
> URL: https://issues.apache.org/jira/browse/HIVE-14069
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2, Metastore
>Reporter: Thejas M Nair
>Assignee: Jason Dere
> Attachments: HIVE-14069.1.patch, HIVE-14069.2.patch,
> HIVE-14069.3.patch
>
>
> curator-2.10.0 has several bug fixes over current version (2.6.0), updating
> would help improve stability.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-14069) update curator version to 2.10.0
[
https://issues.apache.org/jira/browse/HIVE-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16237459#comment-16237459
]
Hive QA commented on HIVE-14069:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12895454/HIVE-14069.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 19 failed/errored test(s), 11353 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=62)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[ct_noperm_loc]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testAmPoolInteractions
(batchId=281)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testApplyPlanQpChanges
(batchId=281)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testApplyPlanUserMapping
(batchId=281)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testAsyncSessionInitFailures
(batchId=281)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testClusterFractions
(batchId=281)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testDestroyAndReturn
(batchId=281)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testQueueing
(batchId=281)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testReopen (batchId=281)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testReuse (batchId=281)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testReuseWithDifferentPool
(batchId=281)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testReuseWithQueueing
(batchId=281)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7606/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7606/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7606/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 19 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12895454 - PreCommit-HIVE-Build
> update curator version to 2.10.0
> -
>
> Key: HIVE-14069
> URL: https://issues.apache.org/jira/browse/HIVE-14069
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2, Metastore
>Reporter: Thejas M Nair
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-14069.1.patch, HIVE-14069.2.patch
>
>
> curator-2.10.0 has several bug fixes over current version (2.6.0), updating
> would help improve stability.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-14069) update curator version to 2.10.0
[ https://issues.apache.org/jira/browse/HIVE-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234632#comment-16234632 ] Jason Dere commented on HIVE-14069: --- Trying to go with the shading approach - shading within each Hive module that uses Curator. One issue I have been running into is MSHADE-148. The maven-shade plugin normally tries to generate a new POM file which does not include any of the shaded dependencies, which should be a good thing because (1) we don't want the conflicting Curator versions from Hadoop and Hive being brought into the classpath together, and (2) since the Curator version used by Hive would be shaded, there is no need to bring it into the classpath. Unfortunately MSHADE-148 is causing some kind of infinite loop when generating this new dependency-reduced POM file. We can disable this POM generation step in the shade options, but then this means the original POM file for the Hive module is being used which specifies the new Curator version as a dependency. Thinking we can set the Curator dependencies as optional dependencies in maven so they do not get brought into the classpath as transitive dependencies. > update curator version to 2.10.0 > - > > Key: HIVE-14069 > URL: https://issues.apache.org/jira/browse/HIVE-14069 > Project: Hive > Issue Type: Improvement > Components: HiveServer2, Metastore >Reporter: Thejas M Nair >Assignee: Jason Dere >Priority: Major > Attachments: HIVE-14069.1.patch > > > curator-2.10.0 has several bug fixes over current version (2.6.0), updating > would help improve stability. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14069) update curator version to 2.10.0
[ https://issues.apache.org/jira/browse/HIVE-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16227180#comment-16227180 ] Jason Dere commented on HIVE-14069: --- Looks like folks are hesitant to bump the curator version due to it conflicting with the version used by the rest of the Hadoop dependencies. One option is to shade Curator within Hive to avoid the version conflicts. > update curator version to 2.10.0 > - > > Key: HIVE-14069 > URL: https://issues.apache.org/jira/browse/HIVE-14069 > Project: Hive > Issue Type: Improvement > Components: HiveServer2, Metastore >Reporter: Thejas M Nair >Assignee: Thejas M Nair > Attachments: HIVE-14069.1.patch > > > curator-2.10.0 has several bug fixes over current version (2.6.0), updating > would help improve stability. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14069) update curator version to 2.10.0
[ https://issues.apache.org/jira/browse/HIVE-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993591#comment-15993591 ] Ying Chen commented on HIVE-14069: -- Has there been any additional progress on this JIRA? and can it be included in 2.2, 2.3 or 2.4 release? (I think 2.2 and 2.3 are in progress). 3.0 seems pretty far away Thanks. > update curator version to 2.10.0 > - > > Key: HIVE-14069 > URL: https://issues.apache.org/jira/browse/HIVE-14069 > Project: Hive > Issue Type: Improvement > Components: HiveServer2, Metastore >Reporter: Thejas M Nair >Assignee: Thejas M Nair > Attachments: HIVE-14069.1.patch > > > curator-2.10.0 has several bug fixes over current version (2.6.0), updating > would help improve stability. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-14069) update curator version to 2.10.0
[ https://issues.apache.org/jira/browse/HIVE-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15941994#comment-15941994 ] Pengcheng Xiong commented on HIVE-14069: Hello, I am deferring this to Hive 3.0 as we are going to cut the first RC and it is not marked as blocker. Please feel free to commit to the branch if this can be resolved before the release. > update curator version to 2.10.0 > - > > Key: HIVE-14069 > URL: https://issues.apache.org/jira/browse/HIVE-14069 > Project: Hive > Issue Type: Improvement > Components: HiveServer2, Metastore >Reporter: Thejas M Nair >Assignee: Thejas M Nair > Attachments: HIVE-14069.1.patch > > > curator-2.10.0 has several bug fixes over current version (2.6.0), updating > would help improve stability. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-14069) update curator version to 2.10.0
[
https://issues.apache.org/jira/browse/HIVE-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15342494#comment-15342494
]
Sergey Shelukhin commented on HIVE-14069:
-
https://issues.apache.org/jira/browse/HIVE-13930 is the Hadoop JIRA.
As for lack of compat, I am not sure if that's backward compat issue in
binaries in Curator, or just a consequence of having multiple jar versions
(because of the difference between Hive and YARN); we've seen an error like
this: {noformat}Caused by: java.lang.NoSuchMethodError:
org.apache.curator.framework.recipes.shared.SharedCount.getVersionedValue()Lorg/apache/curator/framework/recipes/shared/VersionedValue;
{noformat}
> update curator version to 2.10.0
> -
>
> Key: HIVE-14069
> URL: https://issues.apache.org/jira/browse/HIVE-14069
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2, Metastore
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Attachments: HIVE-14069.1.patch
>
>
> curator-2.10.0 has several bug fixes over current version (2.6.0), updating
> would help improve stability.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14069) update curator version to 2.10.0
[ https://issues.apache.org/jira/browse/HIVE-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15342481#comment-15342481 ] Thejas M Nair commented on HIVE-14069: -- [~sershe] Thanks for the inputs. I wasn't aware of any incompatibilities, my quick review of the [curator release notes|https://cwiki.apache.org/confluence/display/CURATOR/Releases] didn't suggest any incompatible changes. Do you have more information about the 2.6 vs 2.7 incompatibility ? Also, I couldn't find the jira you refer to, do you have that ? Searching hive jira curator returned too many issues with curator in the build warnings. > update curator version to 2.10.0 > - > > Key: HIVE-14069 > URL: https://issues.apache.org/jira/browse/HIVE-14069 > Project: Hive > Issue Type: Improvement > Components: HiveServer2, Metastore >Reporter: Thejas M Nair >Assignee: Thejas M Nair > Attachments: HIVE-14069.1.patch > > > curator-2.10.0 has several bug fixes over current version (2.6.0), updating > would help improve stability. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14069) update curator version to 2.10.0
[ https://issues.apache.org/jira/browse/HIVE-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15342272#comment-15342272 ] Sergey Shelukhin commented on HIVE-14069: - [~ashutoshc] previously suggested we update Hadoop version for that. There's a JIRA for that... Note that YARN uses 2.7, I am not sure they would be compatible, we've seen issues with 2.6 vs 2.7 > update curator version to 2.10.0 > - > > Key: HIVE-14069 > URL: https://issues.apache.org/jira/browse/HIVE-14069 > Project: Hive > Issue Type: Improvement > Components: HiveServer2, Metastore >Reporter: Thejas M Nair >Assignee: Thejas M Nair > Attachments: HIVE-14069.1.patch > > > curator-2.10.0 has several bug fixes over current version (2.6.0), updating > would help improve stability. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
