[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17132608#comment-17132608 ] Vipin Vishvkarma commented on HIVE-21052: - [~dkuzmenko] Sorry for a bit late repl

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152785#comment-17152785 ] Amogh Margoor commented on HIVE-21052: -- We have created Spark Datasource that can re

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17218183#comment-17218183 ] Denys Kuzmenko commented on HIVE-21052: --- Pushed to master. Thanks [~jmarhuen], [~vp

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17218186#comment-17218186 ] Denys Kuzmenko commented on HIVE-21052: --- [~vpnvishv], do you plan to cherry-pick th

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105622#comment-17105622 ] Abhishek Somani commented on HIVE-21052:  This seems like a critical issue. Is th

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105787#comment-17105787 ] Hive QA commented on HIVE-21052: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17107128#comment-17107128 ] Peter Vary commented on HIVE-21052: --- [~asomani]: You are right, that this could still b

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17107480#comment-17107480 ] Abhishek Somani commented on HIVE-21052: Yes [~pvary] we are taking a look at thi

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126593#comment-17126593 ] Denys Kuzmenko commented on HIVE-21052: --- Hi [~asomani], please let us know if you a

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17424366#comment-17424366 ] Denys Kuzmenko commented on HIVE-21052: --- *Warning*: could lead to data-loss. {code}

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16739799#comment-16739799 ] Eugene Koifman commented on HIVE-21052: --- I don't think testing the manually is suff

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16739823#comment-16739823 ] Eugene Koifman commented on HIVE-21052: --- it would be useful to add some tests of pa

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16740339#comment-16740339 ] Jaume M commented on HIVE-21052: The only problem I see with doing it at {{TxnHandler.enq

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16740528#comment-16740528 ] Eugene Koifman commented on HIVE-21052: --- we should never allocate > 1 writeId per (

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16740538#comment-16740538 ] Jaume M commented on HIVE-21052: Got it, I've opened HIVE-21118 for this. > Make sure tr

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743169#comment-16743169 ] Hive QA commented on HIVE-21052: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743201#comment-16743201 ] Hive QA commented on HIVE-21052: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743242#comment-16743242 ] Hive QA commented on HIVE-21052: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743257#comment-16743257 ] Hive QA commented on HIVE-21052: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743371#comment-16743371 ] Hive QA commented on HIVE-21052: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743384#comment-16743384 ] Hive QA commented on HIVE-21052: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743444#comment-16743444 ] Hive QA commented on HIVE-21052: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743453#comment-16743453 ] Hive QA commented on HIVE-21052: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744315#comment-16744315 ] Jaume M commented on HIVE-21052: The flow through the database would be the following: *

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16745593#comment-16745593 ] Eugene Koifman commented on HIVE-21052: --- [~jmarhuen], I'm not sure I understand you

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16745618#comment-16745618 ] Jaume M commented on HIVE-21052: I was assuming it's possible there are different writeId

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16745627#comment-16745627 ] Eugene Koifman commented on HIVE-21052: --- there is 1 \{{writeId}} per (table, txnid)

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16745713#comment-16745713 ] Eugene Koifman commented on HIVE-21052: --- I left some comments on RB. I think the w

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16745818#comment-16745818 ] Hive QA commented on HIVE-21052: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16745833#comment-16745833 ] Hive QA commented on HIVE-21052: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16745871#comment-16745871 ] Hive QA commented on HIVE-21052: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16746727#comment-16746727 ] Jaume M commented on HIVE-21052: [~ekoifman] I agree with doing the db/tables/partitions

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16746854#comment-16746854 ] Eugene Koifman commented on HIVE-21052: --- Compaction (any part of it) never acquires

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16746897#comment-16746897 ] Eugene Koifman commented on HIVE-21052: --- I misunderstood which locks you were talki

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16746903#comment-16746903 ] Eugene Koifman commented on HIVE-21052: --- We'd like to prevent 2 concurrent p-type c

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16748962#comment-16748962 ] Jaume M commented on HIVE-21052: bq. We'd like to prevent 2 concurrent p-type cleans of

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16749022#comment-16749022 ] Eugene Koifman commented on HIVE-21052: --- You are right. I forgot that {{findReadyT

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16749145#comment-16749145 ] Jaume M commented on HIVE-21052: Maybe a solution for this would be to create a new state

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16749161#comment-16749161 ] Eugene Koifman commented on HIVE-21052: --- not sure that this is enough. Suppose you

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16749523#comment-16749523 ] Hive QA commented on HIVE-21052: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16749541#comment-16749541 ] Hive QA commented on HIVE-21052: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16750429#comment-16750429 ] Jaume M commented on HIVE-21052: Can you take a new look [~ekoifman], I've updated the rb

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16750653#comment-16750653 ] Eugene Koifman commented on HIVE-21052: --- Suppose you have p-type clean on table T t

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16751888#comment-16751888 ] Hive QA commented on HIVE-21052: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16751895#comment-16751895 ] Hive QA commented on HIVE-21052: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752664#comment-16752664 ] Hive QA commented on HIVE-21052: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752703#comment-16752703 ] Hive QA commented on HIVE-21052: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752926#comment-16752926 ] Hive QA commented on HIVE-21052: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752929#comment-16752929 ] Hive QA commented on HIVE-21052: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16754416#comment-16754416 ] Jaume M commented on HIVE-21052: [~ekoifman] I've added a CleanerExecutorService to come

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16754555#comment-16754555 ] Hive QA commented on HIVE-21052: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called

[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16754574#comment-16754574 ] Hive QA commented on HIVE-21052: Here are the results of testing the latest attachment: