[jira] [Commented] (HIVE-17340) TxnHandler.checkLock() - reduce number of SQL statements
[ https://issues.apache.org/jira/browse/HIVE-17340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16142101#comment-16142101 ] Eugene Koifman commented on HIVE-17340: --- I cleaned up the imports. I think keeping commit() in the "main" methods makes the flow clearer. committed to master thanks Wei for the review cc [~gopalv] > TxnHandler.checkLock() - reduce number of SQL statements > > > Key: HIVE-17340 > URL: https://issues.apache.org/jira/browse/HIVE-17340 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Fix For: 3.0.0 > > Attachments: HIVE-17340.03.patch > > > This calls acquire(Connection dbConn, Statement stmt, long extLockId, > LockInfo lockInfo) > for each lock in the same DB transaction - 1 Update stmt per acquire(). > There is no reason all of them cannot be sent in 1 statement if all the locks > are granted > With a lot of partitions this can be a perf issue -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17340) TxnHandler.checkLock() - reduce number of SQL statements
[
https://issues.apache.org/jira/browse/HIVE-17340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16142023#comment-16142023
]
Wei Zheng commented on HIVE-17340:
--
Maybe it's better to move {code}dbConn.commit(){code} into the acquire() method
from checkLock().
nit: there's unused imports.
+1 otherwise.
> TxnHandler.checkLock() - reduce number of SQL statements
>
>
> Key: HIVE-17340
> URL: https://issues.apache.org/jira/browse/HIVE-17340
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-17340.03.patch
>
>
> This calls acquire(Connection dbConn, Statement stmt, long extLockId,
> LockInfo lockInfo)
> for each lock in the same DB transaction - 1 Update stmt per acquire().
> There is no reason all of them cannot be sent in 1 statement if all the locks
> are granted
> With a lot of partitions this can be a perf issue
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17340) TxnHandler.checkLock() - reduce number of SQL statements
[ https://issues.apache.org/jira/browse/HIVE-17340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141852#comment-16141852 ] Eugene Koifman commented on HIVE-17340: --- no related failures > TxnHandler.checkLock() - reduce number of SQL statements > > > Key: HIVE-17340 > URL: https://issues.apache.org/jira/browse/HIVE-17340 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-17340.03.patch > > > This calls acquire(Connection dbConn, Statement stmt, long extLockId, > LockInfo lockInfo) > for each lock in the same DB transaction - 1 Update stmt per acquire(). > There is no reason all of them cannot be sent in 1 statement if all the locks > are granted > With a lot of partitions this can be a perf issue -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17340) TxnHandler.checkLock() - reduce number of SQL statements
[
https://issues.apache.org/jira/browse/HIVE-17340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141321#comment-16141321
]
Hive QA commented on HIVE-17340:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12883624/HIVE-17340.03.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 10988 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=235)
org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver
(batchId=242)
org.apache.hive.hcatalog.api.repl.commands.TestCommands.org.apache.hive.hcatalog.api.repl.commands.TestCommands
(batchId=180)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testConnection (batchId=241)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testIsValid (batchId=241)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testIsValidNeg (batchId=241)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testNegativeProxyAuth
(batchId=241)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testNegativeTokenAuth
(batchId=241)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testProxyAuth (batchId=241)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testTokenAuth (batchId=241)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6533/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6533/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6533/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12883624 - PreCommit-HIVE-Build
> TxnHandler.checkLock() - reduce number of SQL statements
>
>
> Key: HIVE-17340
> URL: https://issues.apache.org/jira/browse/HIVE-17340
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-17340.03.patch
>
>
> This calls acquire(Connection dbConn, Statement stmt, long extLockId,
> LockInfo lockInfo)
> for each lock in the same DB transaction - 1 Update stmt per acquire().
> There is no reason all of them cannot be sent in 1 statement if all the locks
> are granted
> With a lot of partitions this can be a perf issue
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17340) TxnHandler.checkLock() - reduce number of SQL statements
[ https://issues.apache.org/jira/browse/HIVE-17340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140913#comment-16140913 ] Eugene Koifman commented on HIVE-17340: --- @Wei Zheng could you review please > TxnHandler.checkLock() - reduce number of SQL statements > > > Key: HIVE-17340 > URL: https://issues.apache.org/jira/browse/HIVE-17340 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-17340.03.patch > > > This calls acquire(Connection dbConn, Statement stmt, long extLockId, > LockInfo lockInfo) > for each lock in the same DB transaction - 1 Update stmt per acquire(). > There is no reason all of them cannot be sent in 1 statement if all the locks > are granted > With a lot of partitions this can be a perf issue -- This message was sent by Atlassian JIRA (v6.4.14#64029)
