[ https://issues.apache.org/jira/browse/HIVE-12389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015396#comment-15015396 ]
Hive QA commented on HIVE-12389: -------------------------------- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12772911/HIVE-12389.2.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 38 failed/errored test(s), 9827 tests executed *Failed tests:* {noformat} TestHWISessionManager - did not produce a TEST-*.xml file TestMiniLlapCliDriver - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testAlterPartition org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testAlterTable org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testAlterViewParititon org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testColumnStatistics org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testComplexTable org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testComplexTypeApi org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testConcurrentMetastores org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testDBOwner org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testDBOwnerChange org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testDatabase org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testDatabaseLocation org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testDatabaseLocationWithPermissionProblems org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testDropTable org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testFilterLastPartition org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testFilterSinglePartition org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testFunctionWithResources org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testGetConfigValue org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testGetTableObjects org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testListPartitionNames org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testListPartitions org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testNameMethods org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testPartition org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testPartitionFilter org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testRenamePartition org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testRetriableClientWithConnLifetime org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testSimpleFunction org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testSimpleTable org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testSimpleTypeApi org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testStatsFastTrivial org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testSynchronized org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testTableDatabase org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testTableFilter org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testValidateTableCols org.apache.hive.jdbc.TestSSL.testSSLVersion org.apache.hive.jdbc.miniHS2.TestHs2Metrics.testMetrics {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6079/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6079/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6079/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 38 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12772911 - PreCommit-HIVE-TRUNK-Build > CompactionTxnHandler.cleanEmptyAbortedTxns() should safeguard against huge IN > clauses > ------------------------------------------------------------------------------------- > > Key: HIVE-12389 > URL: https://issues.apache.org/jira/browse/HIVE-12389 > Project: Hive > Issue Type: Bug > Components: Metastore, Transactions > Affects Versions: 1.0.0 > Reporter: Eugene Koifman > Assignee: Eugene Koifman > Attachments: HIVE-12389.2.patch, HIVE-12389.patch > > > in extreme situations, due to misconfigurations, it may be possible to have > 100Ks or even 1Ms of aborted txns. > This causes delete from TXNS where txn_id in (...) to have a huge IN clause > and DB chokes. > Should use something like TxnHandler.TIMED_OUT_TXN_ABORT_BATCH_SIZE to break > up delete into multiple queries. (Incidentally the batch size should likely > be 1000, not 100, maybe even configurable). > On MySQL for example, it can cause query to fail with > bq. Packet for query is too large (9288598 > 1048576). You can change this > value on the server by setting the max_allowed_packet' variable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)