[ https://issues.apache.org/jira/browse/HBASE-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16306286#comment-16306286 ]
Hadoop QA commented on HBASE-19486: ----------------------------------- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s{color} | {color:blue} Docker mode activated. {color} | | {color:red}-1{color} | {color:red} docker {color} | {color:red} 13m 26s{color} | {color:red} Docker failed to build yetus/hbase:36a7029. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | HBASE-19486 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12904011/HBASE-19486-branch-1.v1.patch | | Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/10790/console | | Powered by | Apache Yetus 0.6.0 http://yetus.apache.org | This message was automatically generated. > Periodically ensure records are not buffered too long by BufferedMutator > ------------------------------------------------------------------------- > > Key: HBASE-19486 > URL: https://issues.apache.org/jira/browse/HBASE-19486 > Project: HBase > Issue Type: Improvement > Components: Client > Reporter: Niels Basjes > Assignee: Niels Basjes > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19486-20171212-2117.patch, > HBASE-19486-20171218-1229.patch, HBASE-19486-20171218-1300.patch, > HBASE-19486-20171219-0933.patch, HBASE-19486-20171219-1026.patch, > HBASE-19486-20171219-1122-trigger-qa-run.patch, > HBASE-19486-20171220-1612-trigger-qa-run.patch, > HBASE-19486-20171220-2228-trigger-qa-run.patch, > HBASE-19486-20171223-1438-trigger-qa-run.patch, > HBASE-19486-20171223-1728-trigger-qa-run.patch, > HBASE-19486-20171223-2222-trigger-qa-run.patch, > HBASE-19486-20171224-1101-trigger-qa-run.patch, > HBASE-19486-20171224-1602.patch, HBASE-19486-branch-1.v0.patch, > HBASE-19486-branch-1.v1.patch, HBASE-19486.v0.patch > > > I'm working on several projects where we are doing stream / event type > processing instead of batch type processing. We mostly use Apache Flink and > Apache Beam for these projects. > When we ingest a continuous stream of events and feed that into HBase via a > BufferedMutator this all works fine. The buffer fills up at a predictable > rate and we can make sure it flushes several times per second into HBase by > tuning the buffer size. > We also have situations where the event rate is unpredictable. Some times > because the source is in reality a batch job that puts records into Kafka, > sometimes because it is the "predictable in production" application in our > testing environment (where only the dev triggers a handful of events). > For these kinds of use cases we need a way to 'force' the BufferedMutator to > automatically flush any records in the buffer even if the buffer is not full. > I'll put up a pull request with a proposed implementation for review against > the master (i.e. 3.0.0). > When approved I would like to backport this to the 1.x and 2.x versions of > the client in the same (as close as possible) way. -- This message was sent by Atlassian JIRA (v6.4.14#64029)