[jira] [Commented] (HIVE-11833) TxnHandler heartbeat txn doesn't need to serializable DB txn level
[
https://issues.apache.org/jira/browse/HIVE-11833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14790809#comment-14790809
]
Hive QA commented on HIVE-11833:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12756101/HIVE-11833.patch
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 9443 tests executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler
org.apache.hive.hcatalog.streaming.TestStreaming.testRemainingTransactions
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5297/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5297/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5297/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12756101 - PreCommit-HIVE-TRUNK-Build
> TxnHandler heartbeat txn doesn't need to serializable DB txn level
> --
>
> Key: HIVE-11833
> URL: https://issues.apache.org/jira/browse/HIVE-11833
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-11833.patch
>
>
> What it does is:
> 1) Update lock heartbeat time, fails if not found.
> 2) Get txn state.
> 3) If not found, look for txn in completed, fails regardless of result.
> 4) Update txn heartbeat time if not (3) and not aborted.
> All this can run the same under repeatable-reads.
> Now if it runs under read-committed, someone could
> 1) update txn state after we read it
> 2) delete txn state (moving to completed) after we read it
> 3) same for completed state
> In case of 1 we will update heartbeat for e.g. aborted txn without detecting
> it. UPD: We can change queries to detect it
> In case of 2 the update will produce 0 rows so we will detect that and can
> check completed as we already do.
> The 3 case seems like it doesn't matter.
> I don't know if (1) matters. These heartbeats happen often and can cause
> contention on the db
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11833) TxnHandler heartbeat txn doesn't need to serializable DB txn level
[ https://issues.apache.org/jira/browse/HIVE-11833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746496#comment-14746496 ] Alan Gates commented on HIVE-11833: --- Just to be clear, is the main motivation here to lower contention on the DB? +1, patch looks good. > TxnHandler heartbeat txn doesn't need to serializable DB txn level > -- > > Key: HIVE-11833 > URL: https://issues.apache.org/jira/browse/HIVE-11833 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-11833.patch > > > What it does is: > 1) Update lock heartbeat time, fails if not found. > 2) Get txn state. > 3) If not found, look for txn in completed, fails regardless of result. > 4) Update txn heartbeat time if not (3) and not aborted. > All this can run the same under repeatable-reads. > Now if it runs under read-committed, someone could > 1) update txn state after we read it > 2) delete txn state (moving to completed) after we read it > 3) same for completed state > In case of 1 we will update heartbeat for e.g. aborted txn without detecting > it. UPD: We can change queries to detect it > In case of 2 the update will produce 0 rows so we will detect that and can > check completed as we already do. > The 3 case seems like it doesn't matter. > I don't know if (1) matters. These heartbeats happen often and can cause > contention on the db -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11833) TxnHandler heartbeat txn doesn't need to serializable DB txn level
[ https://issues.apache.org/jira/browse/HIVE-11833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746359#comment-14746359 ] Sergey Shelukhin commented on HIVE-11833: - [~alangates] [~ekoifman] can you take a look > TxnHandler heartbeat txn doesn't need to serializable DB txn level > -- > > Key: HIVE-11833 > URL: https://issues.apache.org/jira/browse/HIVE-11833 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-11833.patch > > > What it does is: > 1) Update lock heartbeat time, fails if not found. > 2) Get txn state. > 3) If not found, look for txn in completed, fails regardless of result. > 4) Update txn heartbeat time if not (3) and not aborted. > All this can run the same under repeatable-reads. > Now if it runs under read-committed, someone could > 1) update txn state after we read it > 2) delete txn state (moving to completed) after we read it > 3) same for completed state > In case of 1 we will update heartbeat for e.g. aborted txn without detecting > it. UPD: We can change queries to detect it > In case of 2 the update will produce 0 rows so we will detect that and can > check completed as we already do. > The 3 case seems like it doesn't matter. > I don't know if (1) matters. These heartbeats happen often and can cause > contention on the db -- This message was sent by Atlassian JIRA (v6.3.4#6332)
