[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093373#comment-17093373 ] Toshihiro Suzuki commented on HBASE-8458: - Ping [~zhangduo] [~stack]. Could you please review the PR when you get a chance? https://github.com/apache/hbase/pull/1320 The review points of the change are as follows: - The change has Protocol Buffers level changes. I moved MultiRequest#condition to RegionAction and MultiResponse#processed to RegionActionResult as mentioend in the follwoing comment. https://issues.apache.org/jira/browse/HBASE-8458?focusedCommentId=17064155&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17064155 - Introduce CheckAndMutate class that is instantiated by builder classes. - Add new 2 checkAndMutate methods with a single CheckAndMutate object and multiple CheckAndMutate objects. - Deprecated the old checkAndMutate methods. Thanks. > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: New Feature > Components: Client, regionserver >Reporter: Hari Mankude >Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0, 2.4.0 > > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17080103#comment-17080103 ] Toshihiro Suzuki commented on HBASE-8458: - I fixed the conflicts in the PR. [~zhangduo] [~stack] Could you please review the PR when you get a chance? https://github.com/apache/hbase/pull/1320 > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: New Feature > Components: Client, regionserver >Reporter: Hari Mankude >Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0, 2.4.0 > > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072324#comment-17072324 ] Toshihiro Suzuki commented on HBASE-8458: - [~stack] [~ndimiduk] Thank you for the confirmation. I made its target versions to 3.0 and 2.4. Thanks. > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude >Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0, 2.4.0 > > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072215#comment-17072215 ] Nick Dimiduk commented on HBASE-8458: - At this point, I prefer we not land any new features into 2.3. Looks like it's a very popular feature though! > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude >Assignee: Toshihiro Suzuki >Priority: Major > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071331#comment-17071331 ] Michael Stack commented on HBASE-8458: -- I'd say no to branch-2.3 since first RC is imminent. 2.4? [~ndimiduk] Pinging 2.3 RM for your opining. > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude >Assignee: Toshihiro Suzuki >Priority: Major > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17069125#comment-17069125 ] Toshihiro Suzuki commented on HBASE-8458: - [~stack] Thank you taking a look at this! Please find my inline answers below: > Old clients will work against new servers with this patch across all of their > API? checkAndMutate with RM too? Or will checkAndMutate break after this > patch goes in? Yes, old clients will work against new servers with this patch across all of their API. And checkAndMutate with RM too. > New clients will break against servers w/o this patch? For checkAndMutate > only? New clients will break against servers w/o this patch for checkAndMutate with RM and mutateRow. > For rolling upgrade, we'd do servers first? Old clients wouldn't be able to > make use of this new facility. We'd then roll out the new clients and they > could then use the new batching? Yes, that's right. For rolling upgrade, we will need to upgrade servers first, and then roll out the new clients. Thanks. > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude >Assignee: Toshihiro Suzuki >Priority: Major > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17066383#comment-17066383 ] Michael Stack commented on HBASE-8458: -- [~brfrn169] bq. After this change, old clients (before applying this patch) will be able to access to new RegionServers... Old clients will work against new servers with this patch across all of their API? checkAndMutate with RM too? Or will checkAndMutate break after this patch goes in? New clients will break against servers w/o this patch? For checkAndMutate only? For rolling upgrade, we'd do servers first? Old clients wouldn't be able to make use of this new facility. We'd then roll out the new clients and they could then use the new batching? Thanks. > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude >Assignee: Toshihiro Suzuki >Priority: Major > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17064155#comment-17064155 ] Toshihiro Suzuki commented on HBASE-8458: - I created a new PR for this: https://github.com/apache/hbase/pull/1320 This change has Protocol Buffers level changes. I moved MultiRequest#condition to RegionAction and MultiResponse#processed to RegionActionResult. However, I still keep them for backward compatibility. This change affects checkAndMutate() with RowMutations and mutateRow(). After this change, old clients (before applying this patch) will be able to access to new RegionServers, but new clients (after applying this patch) will fail to access to old RegionServers when performing checkAndMutate() with RowMutations and mutateRow(). Could you please review this when you get a chance? [~zhangduo] [~stack] [~ndimiduk] One more thing I would like to discuss here is whether we can commit this to branch-2 and branch-2.3 or not. Even after this change, old clients can access to the RegionServer as expected, but not vice versa. What do you think? > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude >Assignee: Toshihiro Suzuki >Priority: Major > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059587#comment-17059587 ] Toshihiro Suzuki commented on HBASE-8458: - I'll take it up. [~gxcheng] Please ping me if you still want to work on it. > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude >Assignee: Guangxu Cheng >Priority: Major > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054584#comment-17054584 ] Toshihiro Suzuki commented on HBASE-8458: - [~gxcheng] This is a very useful feature. and we need it for our use case. Are you still work on it? Or Can I take it up? > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude >Assignee: Guangxu Cheng >Priority: Major > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924331#comment-16924331 ] Guangxu Cheng commented on HBASE-8458: -- At Tencent, we implemented batch checkAndxxx using existing batch method on Table. Batch checkAndxxx has been online for more than a year, and it has achieved good results. Through testing, the performance of batch checkAndPut has doubled, and in our business, it has also improved by 50%. > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude >Assignee: Guangxu Cheng >Priority: Major > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16857595#comment-16857595 ] David Carballo commented on HBASE-8458: --- This would be also very useful for us. Any new on this? > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude >Priority: Major > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16259470#comment-16259470 ] Jean-Marc Spaggiari commented on HBASE-8458: Ping? > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15969584#comment-15969584 ] Andrew Purtell commented on HBASE-8458: --- No complaints, except for a small nit. We use "mutation" instead of "mutate" as a verb here and there in the codebase. It's not grammatically correct, at least in American and British English. In an API this matters. Call it CheckAndRowMutate. > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[
https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15969567#comment-15969567
]
Nick Dimiduk commented on HBASE-8458:
-
I have a strong interest in this functionality as well.
{quote}
Alright... It should not be that hard to add (and in a backward compatible):
* Introduce a CheckAndRowMutation class that implements Row.
* Add handling to the multi at the region sever. Can just handle it like multi
get at the RS, and just call HRegion.get(...) in a loop. Later we can optimize
this similar to miniBatchMutation.
* Use existing batch method on HTable.
{quote}
Any issues with this proposed design? Seems right to me.
> Support for batch version of checkAndPut() and checkAndDelete()
> ---
>
> Key: HBASE-8458
> URL: https://issues.apache.org/jira/browse/HBASE-8458
> Project: HBase
> Issue Type: Improvement
> Components: Client, regionserver
>Affects Versions: 0.95.0
>Reporter: Hari Mankude
>
> The use case is that the user has multiple threads loading hundreds of keys
> into a hbase table. Occasionally there are collisions in the keys being
> uploaded by different threads. So for correctness, it is required to do
> checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for
> every key update is non optimal. It would be good to have a batch version of
> checkAndPut() similar to batch put(). The client can partition the keys on
> region boundaries.
> The jira is NOT looking for any type of cross-row locking or multi-row
> atomicity with checkAndPut()
> Batch version of checkAndDelete() is a similar requirement.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15350491#comment-15350491 ] Chao Chu commented on HBASE-8458: - I am also interested in this issue. Having gone through all the comments above, it looks like we don't have a solution to achieve an equivalent of batch checkAndPut? > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271683#comment-15271683 ] Babar Tareen commented on HBASE-8458: - I implemented compare and set functionality for apache phoenix based on the exclusive locking and coprocessors (https://github.com/babartareen/phoenix/blob/master/phoenix-core/src/main/java/org/apache/phoenix/hbase/index/Indexer.java#L295). With recent change to using ReadWriteLock, a coprocessor will no longer have exclusive control over a row and thus won't be the correct place to intercept a row update. CheckAndPut can not be called from the coprocessor as locks are already acquired as @Esther noted. Where might be a better place to intercept an update and be able to allow/skip it? > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200126#comment-15200126 ] Esther Kundin commented on HBASE-8458: -- I was testing with HDP HBase 0.98. > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15198531#comment-15198531 ] Lars Hofhansl commented on HBASE-8458: -- [~ekundin] which version of HBase are you using. I do agree with previous comments that no bulk api for CheckAndXXX is a bug. > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15197849#comment-15197849 ] Esther Kundin commented on HBASE-8458: -- I actually have the same problem and attempted to fix with a coprocessor. An observer coprocessor is unable to do the appropriate locking if writes are happening from different clients. You cannot run a checkAndPut from within the coprocessor as by the time you are in the preBatchMutate, the row is locked for writing and a checkAndPut will be unable to run on the same row. If you implement it as an endpoint observer, then it should work, though I haven't tried it, but it will do a read/write lock on the row, which is a stricter locking than checkAndPut would do, so it would hurt performance more than a batch checkAndPut which will not block reads for as long. So it would seem that a batch checkAndPut would be the best solution. > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108315#comment-15108315 ] Aidar Bariev commented on HBASE-8458: - [~apurtell] any updates on this issue? I also need batched version of checkAndPut and haven't found it in current API. Could it be done with coprocessor or somehow differently? > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13859060#comment-13859060 ] Andrew Purtell commented on HBASE-8458: --- bq. Keep the old API in 0.94 and 0.96 (or only do this in 0.98+ and remove the old API) Sure, if something is available in the next week or so we can pull it in to 0.98 _and_ remove the old API, presuming the new CheckAndRowMutation can go into 0.96 and the old APIs are marked deprecated there. Otherwise a removal will have to wait for the next. > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13859058#comment-13859058 ] Lars Hofhansl commented on HBASE-8458: -- Alright... It should not be that hard to add (and in a backward compatible): * Introduce a CheckAndRowMutation class that implements Row. * Add handling to the multi at the region sever. Can just handle it like multi get at the RS, and just call HRegion.get(...) in a loop. Later we can optimize this similar to miniBatchMutation. * Use existing batch method on HTable. * Keep the old API in 0.94 and 0.96 (or only do this in 0.98+ and remove the old API) > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13859050#comment-13859050 ] Andrew Purtell commented on HBASE-8458: --- I agree with N that having a class of operations excluded from multi is an API design issue. We went down this road when the checkAndXXX operations were first added as special code paths, but they were needed at the time so the changes were accepted and have remained useful since. However we really should have all APIs derived from common ancestors that can be combined into a single RPC. bq. I did look at the other multis when I worked on the client in the 0.96, but doing a proper work would have cost more than a week of work, so I excluded it.. This is the crux of the issue. If we are going to do this we should do it before a 1.0 release adds expectations on client API compatibility we can sidestep currently. > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13858666#comment-13858666 ] Nicolas Liochon commented on HBASE-8458: To me, the fact that multi is not available for checkAndXX (or increment) is more or less a bug in the client api. In 0.96+, the code for multiget and multiput is the same on the client side (iirc multiexist is a subcase on get on 0.96). I did look at the other multis when I worked on the client in the 0.96, but doing a proper work would have cost more than a week of work, so I excluded it... > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13858665#comment-13858665 ] James Taylor commented on HBASE-8458: - Most anything can be done with a coprocessor. It'd be convenient not to have to write it - it should be part of the client API in my opinion. > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13858632#comment-13858632 ] Lars Hofhansl commented on HBASE-8458: -- This can actually be done with a coprocessor now, no? All the pieces are in place (locks, leases, etc). If there's something missing, we should add/expose that. If we wanted to add this to the HTable API, we'd have to discuss exact return value (array of booleans, probably) and behavior under partial failure (maybe just behave as multi put - checkAndPut are still idempotent so that should work) > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13858412#comment-13858412 ] James Taylor commented on HBASE-8458: - [~lhofhansl] - this would be useful. Any cycles for implementing it? > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
