[jira] [Updated] (HIVE-11228) Mutation API should use semi-shared locks.
[ https://issues.apache.org/jira/browse/HIVE-11228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-11228: Should this issue be backported to branch-1? It looks like a bug. > Mutation API should use semi-shared locks. > -- > > Key: HIVE-11228 > URL: https://issues.apache.org/jira/browse/HIVE-11228 > Project: Hive > Issue Type: Bug > Components: HCatalog >Affects Versions: 2.0.0 >Reporter: Elliot West >Assignee: Elliot West > Labels: streaming_api, transaction, transactions > Fix For: 2.0.0 > > Attachments: HIVE-11228.1.patch > > > An [issue was > identified|https://issues.apache.org/jira/browse/HIVE-10165?focusedCommentId=14619640&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14619640] > by [~ekoifman] in the streaming mutation API (HIVE-10165) where an > insufficiently restrictive lock was being used when issuing updates and > deletes to ACID tables and partitions. A shared lock was being used where in > fact a semi-shared lock is required. Additionally, the current lock scope > targets the entire table, whereas in theory if the table is partitioned, then > only the affected partitions are required to participate in the semi-shared > lock. However, there are a couple of technical challenges that prevent the > locks currently being applied on a per-partition basis: > * It is expected that the affected partitions are not known in advance so > individual partition locks would need to be acquired as needed. > * The API is expected to execute in a clustered environment and so acquiring > these locks as on an ‘as needed’ basis presents a risk that the meta store > may become overwhelmed. This is expected to be less of an problem when an > HBase based meta store is introduced (HIVE-9452). > * My understanding is that multiple fine grained lock acquisitions for a > single transaction are not possible at present. When they are available > they’ll introduce the possibility that deadlocks can occur. This should be > better handled when HIVE-9675 is complete. > Therefore, as advised, at this time the system will obtain a semi-shared lock > on participating tables. Although this will prevent other concurrent writes, > it will preserve snapshot isolation when reading. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11228) Mutation API should use semi-shared locks.
[ https://issues.apache.org/jira/browse/HIVE-11228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-11228: Target Version/s: 1.3.0 > Mutation API should use semi-shared locks. > -- > > Key: HIVE-11228 > URL: https://issues.apache.org/jira/browse/HIVE-11228 > Project: Hive > Issue Type: Bug > Components: HCatalog >Affects Versions: 2.0.0 >Reporter: Elliot West >Assignee: Elliot West > Labels: streaming_api, transaction, transactions > Fix For: 2.0.0 > > Attachments: HIVE-11228.1.patch > > > An [issue was > identified|https://issues.apache.org/jira/browse/HIVE-10165?focusedCommentId=14619640&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14619640] > by [~ekoifman] in the streaming mutation API (HIVE-10165) where an > insufficiently restrictive lock was being used when issuing updates and > deletes to ACID tables and partitions. A shared lock was being used where in > fact a semi-shared lock is required. Additionally, the current lock scope > targets the entire table, whereas in theory if the table is partitioned, then > only the affected partitions are required to participate in the semi-shared > lock. However, there are a couple of technical challenges that prevent the > locks currently being applied on a per-partition basis: > * It is expected that the affected partitions are not known in advance so > individual partition locks would need to be acquired as needed. > * The API is expected to execute in a clustered environment and so acquiring > these locks as on an ‘as needed’ basis presents a risk that the meta store > may become overwhelmed. This is expected to be less of an problem when an > HBase based meta store is introduced (HIVE-9452). > * My understanding is that multiple fine grained lock acquisitions for a > single transaction are not possible at present. When they are available > they’ll introduce the possibility that deadlocks can occur. This should be > better handled when HIVE-9675 is complete. > Therefore, as advised, at this time the system will obtain a semi-shared lock > on participating tables. Although this will prevent other concurrent writes, > it will preserve snapshot isolation when reading. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11228) Mutation API should use semi-shared locks.
[ https://issues.apache.org/jira/browse/HIVE-11228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliot West updated HIVE-11228: --- Summary: Mutation API should use semi-shared locks. (was: Mutation API should uses semi-shared locks.) > Mutation API should use semi-shared locks. > -- > > Key: HIVE-11228 > URL: https://issues.apache.org/jira/browse/HIVE-11228 > Project: Hive > Issue Type: Bug > Components: HCatalog >Affects Versions: 2.0.0 >Reporter: Elliot West >Assignee: Elliot West > Labels: streaming_api, transaction, transactions > > An [issue was > identified|https://issues.apache.org/jira/browse/HIVE-10165?focusedCommentId=14619640&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14619640] > by [~ekoifman] in the streaming mutation API (HIVE-10165) where an > insufficiently restrictive lock was being used when issuing updates and > deletes to ACID tables and partitions. A shared lock was being used where in > fact a semi-shared lock is required. Additionally, the current lock scope > targets the entire table, whereas in theory if the table is partitioned, then > only the affected partitions are required to participate in the semi-shared > lock. However, there are a couple of technical challenges that prevent the > locks currently being applied on a per-partition basis: > * It is expected that the affected partitions are not known in advance so > individual partition locks would need to be acquired as needed. > * The API is expected to execute in a clustered environment and so acquiring > these locks as on an ‘as needed’ basis presents a risk that the meta store > may become overwhelmed. This is expected to be less of an problem when an > HBase based meta store is introduced (HIVE-9452). > * My understanding is that multiple fine grained lock acquisitions for a > single transaction are not possible at present. When they are available > they’ll introduce the possibility that deadlocks can occur. This should be > better handled when HIVE-9675 is complete. > Therefore, as advised, at this time the system will obtain a semi-shared lock > on participating tables. Although this will prevent other concurrent writes, > it will preserve snapshot isolation when reading. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11228) Mutation API should use semi-shared locks.
[ https://issues.apache.org/jira/browse/HIVE-11228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliot West updated HIVE-11228: --- Attachment: HIVE-11228.1.patch > Mutation API should use semi-shared locks. > -- > > Key: HIVE-11228 > URL: https://issues.apache.org/jira/browse/HIVE-11228 > Project: Hive > Issue Type: Bug > Components: HCatalog >Affects Versions: 2.0.0 >Reporter: Elliot West >Assignee: Elliot West > Labels: streaming_api, transaction, transactions > Attachments: HIVE-11228.1.patch > > > An [issue was > identified|https://issues.apache.org/jira/browse/HIVE-10165?focusedCommentId=14619640&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14619640] > by [~ekoifman] in the streaming mutation API (HIVE-10165) where an > insufficiently restrictive lock was being used when issuing updates and > deletes to ACID tables and partitions. A shared lock was being used where in > fact a semi-shared lock is required. Additionally, the current lock scope > targets the entire table, whereas in theory if the table is partitioned, then > only the affected partitions are required to participate in the semi-shared > lock. However, there are a couple of technical challenges that prevent the > locks currently being applied on a per-partition basis: > * It is expected that the affected partitions are not known in advance so > individual partition locks would need to be acquired as needed. > * The API is expected to execute in a clustered environment and so acquiring > these locks as on an ‘as needed’ basis presents a risk that the meta store > may become overwhelmed. This is expected to be less of an problem when an > HBase based meta store is introduced (HIVE-9452). > * My understanding is that multiple fine grained lock acquisitions for a > single transaction are not possible at present. When they are available > they’ll introduce the possibility that deadlocks can occur. This should be > better handled when HIVE-9675 is complete. > Therefore, as advised, at this time the system will obtain a semi-shared lock > on participating tables. Although this will prevent other concurrent writes, > it will preserve snapshot isolation when reading. -- This message was sent by Atlassian JIRA (v6.3.4#6332)