[jira] [Commented] (HIVE-12375) ensure hive.compactor.check.interval cannot be set too low
[ https://issues.apache.org/jira/browse/HIVE-12375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941946#comment-15941946 ] Pengcheng Xiong commented on HIVE-12375: I am deferring this to Hive 3.0 as we are going to cut the first RC and it is not marked as blocker. Please feel free to commit to the branch if this can be resolved before the release. > ensure hive.compactor.check.interval cannot be set too low > -- > > Key: HIVE-12375 > URL: https://issues.apache.org/jira/browse/HIVE-12375 > Project: Hive > Issue Type: Bug > Components: Metastore, Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-12375.2.patch, HIVE-12375.3.patch, > HIVE-12375.4.patch, HIVE-12375.patch > > > hive.compactor.check.interval can currently be set to as low as 0, which > makes Initiator spin needlessly feeling up logs, etc -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-12375) ensure hive.compactor.check.interval cannot be set too low
[ https://issues.apache.org/jira/browse/HIVE-12375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15300373#comment-15300373 ] Jesus Camacho Rodriguez commented on HIVE-12375: Removing 2.1.0 target. Please feel free to commit to branch-2.1 anyway and fix for 2.1.0 if this happens before the release. > ensure hive.compactor.check.interval cannot be set too low > -- > > Key: HIVE-12375 > URL: https://issues.apache.org/jira/browse/HIVE-12375 > Project: Hive > Issue Type: Bug > Components: Metastore, Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-12375.2.patch, HIVE-12375.3.patch, > HIVE-12375.4.patch, HIVE-12375.patch > > > hive.compactor.check.interval can currently be set to as low as 0, which > makes Initiator spin needlessly feeling up logs, etc -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12375) ensure hive.compactor.check.interval cannot be set too low
[ https://issues.apache.org/jira/browse/HIVE-12375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15208328#comment-15208328 ] Ashutosh Chauhan commented on HIVE-12375: - NegativeCliDriver failed consistently for this one. Probably, related. > ensure hive.compactor.check.interval cannot be set too low > -- > > Key: HIVE-12375 > URL: https://issues.apache.org/jira/browse/HIVE-12375 > Project: Hive > Issue Type: Bug > Components: Metastore, Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-12375.2.patch, HIVE-12375.3.patch, > HIVE-12375.4.patch, HIVE-12375.patch > > > hive.compactor.check.interval can currently be set to as low as 0, which > makes Initiator spin needlessly feeling up logs, etc -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12375) ensure hive.compactor.check.interval cannot be set too low
[
https://issues.apache.org/jira/browse/HIVE-12375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15130172#comment-15130172
]
Hive QA commented on HIVE-12375:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12785681/HIVE-12375.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 10049 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_drop_with_concurrency
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_insert_into1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_insert_into2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_insert_into3
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_insert_into4
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testNewConnectionConfiguration
org.apache.hive.jdbc.TestSSL.testSSLVersion
org.apache.hive.service.cli.TestEmbeddedThriftBinaryCLIService.testExecuteStatementAsync
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6849/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6849/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6849/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12785681 - PreCommit-HIVE-TRUNK-Build
> ensure hive.compactor.check.interval cannot be set too low
> --
>
> Key: HIVE-12375
> URL: https://issues.apache.org/jira/browse/HIVE-12375
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Transactions
>Affects Versions: 1.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-12375.2.patch, HIVE-12375.3.patch,
> HIVE-12375.4.patch, HIVE-12375.patch
>
>
> hive.compactor.check.interval can currently be set to as low as 0, which
> makes Initiator spin needlessly feeling up logs, etc
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12375) ensure hive.compactor.check.interval cannot be set too low
[ https://issues.apache.org/jira/browse/HIVE-12375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15125538#comment-15125538 ] Ashutosh Chauhan commented on HIVE-12375: - Are reported test failures relevant? > ensure hive.compactor.check.interval cannot be set too low > -- > > Key: HIVE-12375 > URL: https://issues.apache.org/jira/browse/HIVE-12375 > Project: Hive > Issue Type: Bug > Components: Metastore, Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-12375.2.patch, HIVE-12375.3.patch, HIVE-12375.patch > > > hive.compactor.check.interval can currently be set to as low as 0, which > makes Initiator spin needlessly feeling up logs, etc -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12375) ensure hive.compactor.check.interval cannot be set too low
[
https://issues.apache.org/jira/browse/HIVE-12375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057794#comment-15057794
]
Hive QA commented on HIVE-12375:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12777604/HIVE-12375.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 24 failed/errored test(s), 9871 tests
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestMiniTezCliDriver-tez_smb_empty.q-transform_ppr2.q-vector_outer_join5.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-vector_decimal_round.q-cbo_windowing.q-tez_schema_evolution.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_drop_with_concurrency
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_order2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union9
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_dynamic
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_insert_into1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_insert_into2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_insert_into3
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_insert_into4
org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testMultiSessionMultipleUse
org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testSingleSessionMultipleUse
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testNewConnectionConfiguration
org.apache.hive.jdbc.TestSSL.testSSLVersion
org.apache.hive.spark.client.TestSparkClient.testAddJarsAndFiles
org.apache.hive.spark.client.TestSparkClient.testCounters
org.apache.hive.spark.client.TestSparkClient.testErrorJob
org.apache.hive.spark.client.TestSparkClient.testJobSubmission
org.apache.hive.spark.client.TestSparkClient.testMetricsCollection
org.apache.hive.spark.client.TestSparkClient.testRemoteClient
org.apache.hive.spark.client.TestSparkClient.testSimpleSparkJob
org.apache.hive.spark.client.TestSparkClient.testSyncRpc
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6356/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6356/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6356/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 24 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12777604 - PreCommit-HIVE-TRUNK-Build
> ensure hive.compactor.check.interval cannot be set too low
> --
>
> Key: HIVE-12375
> URL: https://issues.apache.org/jira/browse/HIVE-12375
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Transactions
>Affects Versions: 1.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-12375.2.patch, HIVE-12375.3.patch, HIVE-12375.patch
>
>
> hive.compactor.check.interval can currently be set to as low as 0, which
> makes Initiator spin needlessly feeling up logs, etc
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12375) ensure hive.compactor.check.interval cannot be set too low
[ https://issues.apache.org/jira/browse/HIVE-12375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056885#comment-15056885 ] Alan Gates commented on HIVE-12375: --- +1 > ensure hive.compactor.check.interval cannot be set too low > -- > > Key: HIVE-12375 > URL: https://issues.apache.org/jira/browse/HIVE-12375 > Project: Hive > Issue Type: Bug > Components: Metastore, Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-12375.2.patch, HIVE-12375.3.patch, HIVE-12375.patch > > > hive.compactor.check.interval can currently be set to as low as 0, which > makes Initiator spin needlessly feeling up logs, etc -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12375) ensure hive.compactor.check.interval cannot be set too low
[ https://issues.apache.org/jira/browse/HIVE-12375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15022851#comment-15022851 ] Eugene Koifman commented on HIVE-12375: --- [~alangates] could you review please > ensure hive.compactor.check.interval cannot be set too low > -- > > Key: HIVE-12375 > URL: https://issues.apache.org/jira/browse/HIVE-12375 > Project: Hive > Issue Type: Bug > Components: Metastore, Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-12375.patch > > > hive.compactor.check.interval can currently be set to as low as 0, which > makes Initiator spin needlessly feeling up logs, etc -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12375) ensure hive.compactor.check.interval cannot be set too low
[ https://issues.apache.org/jira/browse/HIVE-12375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15023092#comment-15023092 ] Alan Gates commented on HIVE-12375: --- In general patch looks good. I'm wondering if even 5s is too often. I see what you're saying about wanting to be able to set it low for testing. I wonder if in your getMin function you allow the min to be set lower if HIVE_IN_TEST is set. Then you could give a low of 30s (which seems more reasonable) in HiveConf and allow users to manually set it lower in tests. > ensure hive.compactor.check.interval cannot be set too low > -- > > Key: HIVE-12375 > URL: https://issues.apache.org/jira/browse/HIVE-12375 > Project: Hive > Issue Type: Bug > Components: Metastore, Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-12375.patch > > > hive.compactor.check.interval can currently be set to as low as 0, which > makes Initiator spin needlessly feeling up logs, etc -- This message was sent by Atlassian JIRA (v6.3.4#6332)
