[jira] [Commented] (HBASE-15046) Perf test doing all mutation steps under row lock
[ https://issues.apache.org/jira/browse/HBASE-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15073328#comment-15073328 ] Lars Hofhansl commented on HBASE-15046: --- It's this right now, though: # take rowlock # start mvcc # append to WAL # add to memstore # _let go of rowlock_ # _sync WAL_ # in case of error: rollback memstore # finish up mvcc Right? That was the whole point of rolling back the memstore, so that we can sync the wal to the DNs without holding the row lock, I doubt we want to undo that part. > Perf test doing all mutation steps under row lock > - > > Key: HBASE-15046 > URL: https://issues.apache.org/jira/browse/HBASE-15046 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack > > This issue is about perf testing a redo of the write pipeline so that rather > than: > * take rowlock > * start mvcc > * append to WAL > * add to memstore > * sync WAL > * let go of rowlock > * finish up mvcc > instead try... > * take rowlock > * start mvcc > * append to WAL > * sync WAL > * add to memstore > * finish up mvcc > * let go of rowlock > The latter is more straight-forward undoing need of rolling back memstore if > all does not succeed. > It might be slower though. This issue is a look-see/try it. > The redo will also help address the parent issue in a more general way so we > can do without the special-casing done for branch-1.0 and branch-1.1 done in > a sibling subtask. > Other benefits are that the current write pipeline is copy/pasted in a few > places -- in append, increment and checkand* -- and a refactor will allow us > to fix this duplication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15046) Perf test doing all mutation steps under row lock
[ https://issues.apache.org/jira/browse/HBASE-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15073403#comment-15073403 ] Heng Chen commented on HBASE-15046: --- Currently, sync WAL is not really 'sync' (it use hflush not hsync). So with 'rollback memstore', it may cause inconsistency between WAL and memstore. I am not sure the tradeoff between performance and correctness? > Perf test doing all mutation steps under row lock > - > > Key: HBASE-15046 > URL: https://issues.apache.org/jira/browse/HBASE-15046 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack > > This issue is about perf testing a redo of the write pipeline so that rather > than: > * take rowlock > * start mvcc > * append to WAL > * add to memstore > * sync WAL > * let go of rowlock > * finish up mvcc > instead try... > * take rowlock > * start mvcc > * append to WAL > * sync WAL > * add to memstore > * finish up mvcc > * let go of rowlock > The latter is more straight-forward undoing need of rolling back memstore if > all does not succeed. > It might be slower though. This issue is a look-see/try it. > The redo will also help address the parent issue in a more general way so we > can do without the special-casing done for branch-1.0 and branch-1.1 done in > a sibling subtask. > Other benefits are that the current write pipeline is copy/pasted in a few > places -- in append, increment and checkand* -- and a refactor will allow us > to fix this duplication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15046) Perf test doing all mutation steps under row lock
[ https://issues.apache.org/jira/browse/HBASE-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15073419#comment-15073419 ] Elliott Clark commented on HBASE-15046: --- bq.That was the whole point of rolling back the memstore, so that we can sync the wal to the DNs without holding the row lock, I doubt we want to undo that part. There can be any number of updates in flight for a single row. So holding the row lock longer doesn't have the down side that it used to. > Perf test doing all mutation steps under row lock > - > > Key: HBASE-15046 > URL: https://issues.apache.org/jira/browse/HBASE-15046 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack > > This issue is about perf testing a redo of the write pipeline so that rather > than: > * take rowlock > * start mvcc > * append to WAL > * add to memstore > * sync WAL > * let go of rowlock > * finish up mvcc > instead try... > * take rowlock > * start mvcc > * append to WAL > * sync WAL > * add to memstore > * finish up mvcc > * let go of rowlock > The latter is more straight-forward undoing need of rolling back memstore if > all does not succeed. > It might be slower though. This issue is a look-see/try it. > The redo will also help address the parent issue in a more general way so we > can do without the special-casing done for branch-1.0 and branch-1.1 done in > a sibling subtask. > Other benefits are that the current write pipeline is copy/pasted in a few > places -- in append, increment and checkand* -- and a refactor will allow us > to fix this duplication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15046) Perf test doing all mutation steps under row lock
[
https://issues.apache.org/jira/browse/HBASE-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15073451#comment-15073451
]
Heng Chen commented on HBASE-15046:
---
{quote}
So holding the row lock longer doesn't have the down side that it used to.
Since every row lock is reader/writer the only time that holding the lock
longer has any performance issue is when there are puts and check/mutates in
flight to the same row.
{quote}
Oh, the row lock in doMiniBatchMutation is shared lock. So there is no
performance regression if we move sync WAL into row lock scope. Thanks
[~eclark] for your explanation.
> Perf test doing all mutation steps under row lock
> -
>
> Key: HBASE-15046
> URL: https://issues.apache.org/jira/browse/HBASE-15046
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
>Reporter: stack
>
> This issue is about perf testing a redo of the write pipeline so that rather
> than:
> * take rowlock
> * start mvcc
> * append to WAL
> * add to memstore
> * sync WAL
> * let go of rowlock
> * finish up mvcc
> instead try...
> * take rowlock
> * start mvcc
> * append to WAL
> * sync WAL
> * add to memstore
> * finish up mvcc
> * let go of rowlock
> The latter is more straight-forward undoing need of rolling back memstore if
> all does not succeed.
> It might be slower though. This issue is a look-see/try it.
> The redo will also help address the parent issue in a more general way so we
> can do without the special-casing done for branch-1.0 and branch-1.1 done in
> a sibling subtask.
> Other benefits are that the current write pipeline is copy/pasted in a few
> places -- in append, increment and checkand* -- and a refactor will allow us
> to fix this duplication.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15046) Perf test doing all mutation steps under row lock
[ https://issues.apache.org/jira/browse/HBASE-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15073563#comment-15073563 ] Lars Hofhansl commented on HBASE-15046: --- bq. Currently, sync WAL is not really 'sync' (it use hflush not hsync). So with 'rollback memstore', it may cause inconsistency between WAL and memstore. I am not sure the tradeoff between performance and correctness? Not so. hflush might not persist to disk (just forces changes to the typically 3 involved data nodes), but it is the point at the we report back to the client that the write was successful, no amount of reordering mvcc and hflush can guard against (say) a power outage taking the wrong three machines down. [~eclark] and [~stack], so this is possible after HBASE-12751 (which is in master, only)? That would be cool simplification indeed. > Perf test doing all mutation steps under row lock > - > > Key: HBASE-15046 > URL: https://issues.apache.org/jira/browse/HBASE-15046 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack > > This issue is about perf testing a redo of the write pipeline so that rather > than: > * take rowlock > * start mvcc > * append to WAL > * add to memstore > * sync WAL > * let go of rowlock > * finish up mvcc > instead try... > * take rowlock > * start mvcc > * append to WAL > * sync WAL > * add to memstore > * finish up mvcc > * let go of rowlock > The latter is more straight-forward undoing need of rolling back memstore if > all does not succeed. > It might be slower though. This issue is a look-see/try it. > The redo will also help address the parent issue in a more general way so we > can do without the special-casing done for branch-1.0 and branch-1.1 done in > a sibling subtask. > Other benefits are that the current write pipeline is copy/pasted in a few > places -- in append, increment and checkand* -- and a refactor will allow us > to fix this duplication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15046) Perf test doing all mutation steps under row lock
[ https://issues.apache.org/jira/browse/HBASE-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15073616#comment-15073616 ] Elliott Clark commented on HBASE-15046: --- HBASE-12751 went into branch-1 as well via HBASE-14465. But yeah the optimization is possible because of that jira. > Perf test doing all mutation steps under row lock > - > > Key: HBASE-15046 > URL: https://issues.apache.org/jira/browse/HBASE-15046 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack > > This issue is about perf testing a redo of the write pipeline so that rather > than: > * take rowlock > * start mvcc > * append to WAL > * add to memstore > * sync WAL > * let go of rowlock > * finish up mvcc > instead try... > * take rowlock > * start mvcc > * append to WAL > * sync WAL > * add to memstore > * finish up mvcc > * let go of rowlock > The latter is more straight-forward undoing need of rolling back memstore if > all does not succeed. > It might be slower though. This issue is a look-see/try it. > The redo will also help address the parent issue in a more general way so we > can do without the special-casing done for branch-1.0 and branch-1.1 done in > a sibling subtask. > Other benefits are that the current write pipeline is copy/pasted in a few > places -- in append, increment and checkand* -- and a refactor will allow us > to fix this duplication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15046) Perf test doing all mutation steps under row lock
[ https://issues.apache.org/jira/browse/HBASE-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074710#comment-15074710 ] Lars Hofhansl commented on HBASE-15046: --- Hey [~stack]. I'm very interested in this. Wanna huddle first thing next year? I've also written a simple load tester for a kind of workload we have, which essentially overrides (i.e. writes new versions of) the same keys (row key + CF + CQ) over and over again in batches; it's likely multiple threads/clients will touch the same key in a batch. > Perf test doing all mutation steps under row lock > - > > Key: HBASE-15046 > URL: https://issues.apache.org/jira/browse/HBASE-15046 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack > Attachments: 1.2.svg, 1.2.v2.svg > > > This issue is about perf testing a redo of the write pipeline so that rather > than: > * take rowlock > * start mvcc > * append to WAL > * add to memstore > * sync WAL > * let go of rowlock > * finish up mvcc > instead try... > * take rowlock > * start mvcc > * append to WAL > * sync WAL > * add to memstore > * finish up mvcc > * let go of rowlock > The latter is more straight-forward undoing need of rolling back memstore if > all does not succeed. > It might be slower though. This issue is a look-see/try it. > The redo will also help address the parent issue in a more general way so we > can do without the special-casing done for branch-1.0 and branch-1.1 done in > a sibling subtask. > Other benefits are that the current write pipeline is copy/pasted in a few > places -- in append, increment and checkand* -- and a refactor will allow us > to fix this duplication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15046) Perf test doing all mutation steps under row lock
[ https://issues.apache.org/jira/browse/HBASE-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15075274#comment-15075274 ] stack commented on HBASE-15046: --- [~lhofhansl] I'm game to huddle, for sure. This doMiniBatch thingy needs refactor at a minimum. Let me post a few graphs for simple loading that shows a benefit doing all under row lock; I think there is more to be had here. I'd be interested in your load tester to see if benefit is still there when doing your loading type. > Perf test doing all mutation steps under row lock > - > > Key: HBASE-15046 > URL: https://issues.apache.org/jira/browse/HBASE-15046 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack > Attachments: 1.2.svg, 1.2.v2.svg > > > This issue is about perf testing a redo of the write pipeline so that rather > than: > * take rowlock > * start mvcc > * append to WAL > * add to memstore > * sync WAL > * let go of rowlock > * finish up mvcc > instead try... > * take rowlock > * start mvcc > * append to WAL > * sync WAL > * add to memstore > * finish up mvcc > * let go of rowlock > The latter is more straight-forward undoing need of rolling back memstore if > all does not succeed. > It might be slower though. This issue is a look-see/try it. > The redo will also help address the parent issue in a more general way so we > can do without the special-casing done for branch-1.0 and branch-1.1 done in > a sibling subtask. > Other benefits are that the current write pipeline is copy/pasted in a few > places -- in append, increment and checkand* -- and a refactor will allow us > to fix this duplication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15046) Perf test doing all mutation steps under row lock
[ https://issues.apache.org/jira/browse/HBASE-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15075308#comment-15075308 ] Elliott Clark commented on HBASE-15046: --- Thanks for this [~stack] everything is looking great. > Perf test doing all mutation steps under row lock > - > > Key: HBASE-15046 > URL: https://issues.apache.org/jira/browse/HBASE-15046 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack > Attachments: 1.2.svg, 1.2.v2.svg, all_under_lock.patch, > all_under_lock.svg, latencies.png, writes.png > > > This issue is about perf testing a redo of the write pipeline so that rather > than: > * take rowlock > * start mvcc > * append to WAL > * add to memstore > * sync WAL > * let go of rowlock > * finish up mvcc > instead try... > * take rowlock > * start mvcc > * append to WAL > * sync WAL > * add to memstore > * finish up mvcc > * let go of rowlock > The latter is more straight-forward undoing need of rolling back memstore if > all does not succeed. > It might be slower though. This issue is a look-see/try it. > The redo will also help address the parent issue in a more general way so we > can do without the special-casing done for branch-1.0 and branch-1.1 done in > a sibling subtask. > Other benefits are that the current write pipeline is copy/pasted in a few > places -- in append, increment and checkand* -- and a refactor will allow us > to fix this duplication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15046) Perf test doing all mutation steps under row lock
[ https://issues.apache.org/jira/browse/HBASE-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082710#comment-15082710 ] Heng Chen commented on HBASE-15046: --- Looks great. All operations happens on one single row or random rows? > Perf test doing all mutation steps under row lock > - > > Key: HBASE-15046 > URL: https://issues.apache.org/jira/browse/HBASE-15046 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack > Attachments: 1.2.svg, 1.2.v2.svg, 50threads_ycsb.png, > all_under_lock.patch, all_under_lock.svg, call_times_0-1_and_1-3_ycsb.png, > latencies.png, writes.png > > > This issue is about perf testing a redo of the write pipeline so that rather > than: > * take rowlock > * start mvcc > * append to WAL > * add to memstore > * sync WAL > * let go of rowlock > * finish up mvcc > instead try... > * take rowlock > * start mvcc > * append to WAL > * sync WAL > * add to memstore > * finish up mvcc > * let go of rowlock > The latter is more straight-forward undoing need of rolling back memstore if > all does not succeed. > It might be slower though. This issue is a look-see/try it. > The redo will also help address the parent issue in a more general way so we > can do without the special-casing done for branch-1.0 and branch-1.1 done in > a sibling subtask. > Other benefits are that the current write pipeline is copy/pasted in a few > places -- in append, increment and checkand* -- and a refactor will allow us > to fix this duplication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15046) Perf test doing all mutation steps under row lock
[
https://issues.apache.org/jira/browse/HBASE-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082733#comment-15082733
]
Heng Chen commented on HBASE-15046:
---
{quote}
not sure why reads would be faster but 95/5 seems a tad slower (in middle is an
aborted run – the patch seems to bring on mvcc lockup... need to look into
that).
{quote}
Is all read requests happens on latest inserted rows? what's the
requestdistribution param, uniform, zipfian or latest ?
> Perf test doing all mutation steps under row lock
> -
>
> Key: HBASE-15046
> URL: https://issues.apache.org/jira/browse/HBASE-15046
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
>Reporter: stack
>Assignee: stack
> Attachments: 1.2.svg, 1.2.v2.svg, 50threads_ycsb.png,
> all_under_lock.patch, all_under_lock.svg, call_times_0-1_and_1-3_ycsb.png,
> latencies.png, writes.png
>
>
> This issue is about perf testing a redo of the write pipeline so that rather
> than:
> * take rowlock
> * start mvcc
> * append to WAL
> * add to memstore
> * sync WAL
> * let go of rowlock
> * finish up mvcc
> instead try...
> * take rowlock
> * start mvcc
> * append to WAL
> * sync WAL
> * add to memstore
> * finish up mvcc
> * let go of rowlock
> The latter is more straight-forward undoing need of rolling back memstore if
> all does not succeed.
> It might be slower though. This issue is a look-see/try it.
> The redo will also help address the parent issue in a more general way so we
> can do without the special-casing done for branch-1.0 and branch-1.1 done in
> a sibling subtask.
> Other benefits are that the current write pipeline is copy/pasted in a few
> places -- in append, increment and checkand* -- and a refactor will allow us
> to fix this duplication.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15046) Perf test doing all mutation steps under row lock
[ https://issues.apache.org/jira/browse/HBASE-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083472#comment-15083472 ] stack commented on HBASE-15046: --- Its ycsb -- we run each workload for an hour -- so random rows [~chenheng]. > Perf test doing all mutation steps under row lock > - > > Key: HBASE-15046 > URL: https://issues.apache.org/jira/browse/HBASE-15046 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack > Attachments: 1.2.svg, 1.2.v2.svg, 50threads_ycsb.png, > all_under_lock.patch, all_under_lock.svg, call_times_0-1_and_1-3_ycsb.png, > latencies.png, writes.png > > > This issue is about perf testing a redo of the write pipeline so that rather > than: > * take rowlock > * start mvcc > * append to WAL > * add to memstore > * sync WAL > * let go of rowlock > * finish up mvcc > instead try... > * take rowlock > * start mvcc > * append to WAL > * sync WAL > * add to memstore > * finish up mvcc > * let go of rowlock > The latter is more straight-forward undoing need of rolling back memstore if > all does not succeed. > It might be slower though. This issue is a look-see/try it. > The redo will also help address the parent issue in a more general way so we > can do without the special-casing done for branch-1.0 and branch-1.1 done in > a sibling subtask. > Other benefits are that the current write pipeline is copy/pasted in a few > places -- in append, increment and checkand* -- and a refactor will allow us > to fix this duplication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15046) Perf test doing all mutation steps under row lock
[ https://issues.apache.org/jira/browse/HBASE-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15084961#comment-15084961 ] Heng Chen commented on HBASE-15046: --- yeah, so i want to know what is the request distribution param in your workload (It seems that default value is uniform). > Perf test doing all mutation steps under row lock > - > > Key: HBASE-15046 > URL: https://issues.apache.org/jira/browse/HBASE-15046 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack > Attachments: 1.2.svg, 1.2.v2.svg, 50threads_ycsb.png, > all_under_lock.patch, all_under_lock.svg, call_times_0-1_and_1-3_ycsb.png, > latencies.png, writes.png > > > This issue is about perf testing a redo of the write pipeline so that rather > than: > * take rowlock > * start mvcc > * append to WAL > * add to memstore > * sync WAL > * let go of rowlock > * finish up mvcc > instead try... > * take rowlock > * start mvcc > * append to WAL > * sync WAL > * add to memstore > * finish up mvcc > * let go of rowlock > The latter is more straight-forward undoing need of rolling back memstore if > all does not succeed. > It might be slower though. This issue is a look-see/try it. > The redo will also help address the parent issue in a more general way so we > can do without the special-casing done for branch-1.0 and branch-1.1 done in > a sibling subtask. > Other benefits are that the current write pipeline is copy/pasted in a few > places -- in append, increment and checkand* -- and a refactor will allow us > to fix this duplication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15046) Perf test doing all mutation steps under row lock
[ https://issues.apache.org/jira/browse/HBASE-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15084962#comment-15084962 ] Heng Chen commented on HBASE-15046: --- yeah, so i want to know what is the request distribution param in your workload (It seems that default value is uniform). > Perf test doing all mutation steps under row lock > - > > Key: HBASE-15046 > URL: https://issues.apache.org/jira/browse/HBASE-15046 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack > Attachments: 1.2.svg, 1.2.v2.svg, 50threads_ycsb.png, > all_under_lock.patch, all_under_lock.svg, call_times_0-1_and_1-3_ycsb.png, > latencies.png, writes.png > > > This issue is about perf testing a redo of the write pipeline so that rather > than: > * take rowlock > * start mvcc > * append to WAL > * add to memstore > * sync WAL > * let go of rowlock > * finish up mvcc > instead try... > * take rowlock > * start mvcc > * append to WAL > * sync WAL > * add to memstore > * finish up mvcc > * let go of rowlock > The latter is more straight-forward undoing need of rolling back memstore if > all does not succeed. > It might be slower though. This issue is a look-see/try it. > The redo will also help address the parent issue in a more general way so we > can do without the special-casing done for branch-1.0 and branch-1.1 done in > a sibling subtask. > Other benefits are that the current write pipeline is copy/pasted in a few > places -- in append, increment and checkand* -- and a refactor will allow us > to fix this duplication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15046) Perf test doing all mutation steps under row lock
[ https://issues.apache.org/jira/browse/HBASE-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15096995#comment-15096995 ] Elliott Clark commented on HBASE-15046: --- We might be able to get that 5% back by going with the complexity of rollbacks on the default doMiniBatchMutation code path. > Perf test doing all mutation steps under row lock > - > > Key: HBASE-15046 > URL: https://issues.apache.org/jira/browse/HBASE-15046 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack > Attachments: 1.2.svg, 1.2.v2.svg, 50threads_ycsb.png, > all_under_lock.patch, all_under_lock.svg, call_times_0-1_and_1-3_ycsb.png, > gc.png, latencies.png, ops.png, writes.png > > > This issue is about perf testing a redo of the write pipeline so that rather > than: > * take rowlock > * start mvcc > * append to WAL > * add to memstore > * sync WAL > * let go of rowlock > * finish up mvcc > instead try... > * take rowlock > * start mvcc > * append to WAL > * sync WAL > * add to memstore > * finish up mvcc > * let go of rowlock > The latter is more straight-forward undoing need of rolling back memstore if > all does not succeed. > It might be slower though. This issue is a look-see/try it. > The redo will also help address the parent issue in a more general way so we > can do without the special-casing done for branch-1.0 and branch-1.1 done in > a sibling subtask. > Other benefits are that the current write pipeline is copy/pasted in a few > places -- in append, increment and checkand* -- and a refactor will allow us > to fix this duplication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15046) Perf test doing all mutation steps under row lock
[ https://issues.apache.org/jira/browse/HBASE-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137929#comment-15137929 ] Elliott Clark commented on HBASE-15046: --- If we spend the time between append and sync putting the kv's in the concurrent skip list map, that's time that would otherwise be just wasted. > Perf test doing all mutation steps under row lock > - > > Key: HBASE-15046 > URL: https://issues.apache.org/jira/browse/HBASE-15046 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: 1.2.svg, 1.2.v2.svg, 50threads_ycsb.png, > all_under_lock.patch, all_under_lock.svg, call_times_0-1_and_1-3_ycsb.png, > compare.png, gc.png, latencies.png, nopatch.png, > notpatched.workloadw.80percentwrites.json, ops.png, patched.png, > patched.workloadw.80percentwrites.json, writes.png > > > This issue is about perf testing a redo of the write pipeline so that rather > than: > * take rowlock > * start mvcc > * append to WAL > * add to memstore > * sync WAL > * let go of rowlock > * finish up mvcc > instead try... > * take rowlock > * start mvcc > * append to WAL > * sync WAL > * add to memstore > * finish up mvcc > * let go of rowlock > The latter is more straight-forward undoing need of rolling back memstore if > all does not succeed. > It might be slower though. This issue is a look-see/try it. > The redo will also help address the parent issue in a more general way so we > can do without the special-casing done for branch-1.0 and branch-1.1 done in > a sibling subtask. > Other benefits are that the current write pipeline is copy/pasted in a few > places -- in append, increment and checkand* -- and a refactor will allow us > to fix this duplication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
