[ https://issues.apache.org/jira/browse/HBASE-12728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14277318#comment-14277318 ]
Hadoop QA commented on HBASE-12728: ----------------------------------- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12692239/HBASE-12728-2.patch against master branch at commit 1a21c1684c5d68cb2d1da8ed33500993b0965f8a. ATTACHMENT ID: 12692239 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 96 new or modified tests. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 14 warning messages. {color:red}-1 checkstyle{color}. The applied patch generated 2081 checkstyle errors (more than the master's current 2074 errors). {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:red}-1 release audit{color}. The applied patch generated 2 release audit warnings (more than the master's current 0 warnings). {color:red}-1 lineLengths{color}. The patch introduces the following lines longer than 100: + * An implementation of {@link Lock} that doesn't actually lock anything. {@link BufferedMutatorImpl} uses + public static void loadData(final HBaseTestingUtility util, final BufferedMutator mutator, int rows, {color:red}-1 site{color}. The patch appears to cause mvn site goal to fail. {color:red}-1 core tests{color}. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/12460//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12460//artifact/patchprocess/patchReleaseAuditWarnings.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12460//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12460//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12460//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12460//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12460//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12460//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12460//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12460//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12460//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12460//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12460//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/12460//artifact/patchprocess/checkstyle-aggregate.html Javadoc warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12460//artifact/patchprocess/patchJavadocWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/12460//console This message is automatically generated. > buffered writes substantially less useful after removal of HTablePool > --------------------------------------------------------------------- > > Key: HBASE-12728 > URL: https://issues.apache.org/jira/browse/HBASE-12728 > Project: HBase > Issue Type: Bug > Components: hbase > Affects Versions: 0.98.0 > Reporter: Aaron Beppu > Assignee: Solomon Duskis > Priority: Blocker > Fix For: 1.0.0, 2.0.0, 1.1.0 > > Attachments: 12728.connection-owns-buffers.example.branch-1.0.patch, > HBASE-12728-2.patch, HBASE-12728.patch, bulk-mutator.patch > > > In previous versions of HBase, when use of HTablePool was encouraged, HTable > instances were long-lived in that pool, and for that reason, if autoFlush was > set to false, the table instance could accumulate a full buffer of writes > before a flush was triggered. Writes from the client to the cluster could > then be substantially larger and less frequent than without buffering. > However, when HTablePool was deprecated, the primary justification seems to > have been that creating HTable instances is cheap, so long as the connection > and executor service being passed to it are pre-provided. A use pattern was > encouraged where users should create a new HTable instance for every > operation, using an existing connection and executor service, and then close > the table. In this pattern, buffered writes are substantially less useful; > writes are as small and as frequent as they would have been with > autoflush=true, except the synchronous write is moved from the operation > itself to the table close call which immediately follows. > More concretely : > ``` > // Given these two helpers ... > private HTableInterface getAutoFlushTable(String tableName) throws > IOException { > // (autoflush is true by default) > return storedConnection.getTable(tableName, executorService); > } > private HTableInterface getBufferedTable(String tableName) throws IOException > { > HTableInterface table = getAutoFlushTable(tableName); > table.setAutoFlush(false); > return table; > } > // it's my contention that these two methods would behave almost identically, > // except the first will hit a synchronous flush during the put call, > and the second will > // flush during the (hidden) close call on table. > private void writeAutoFlushed(Put somePut) throws IOException { > try (HTableInterface table = getAutoFlushTable(tableName)) { > table.put(somePut); // will do synchronous flush > } > } > private void writeBuffered(Put somePut) throws IOException { > try (HTableInterface table = getBufferedTable(tableName)) { > table.put(somePut); > } // auto-close will trigger synchronous flush > } > ``` > For buffered writes to actually provide a performance benefit to users, one > of two things must happen: > - The writeBuffer itself shouldn't live, flush and die with the lifecycle of > it's HTableInstance. If the writeBuffer were managed elsewhere and had a long > lifespan, this could cease to be an issue. However, if the same writeBuffer > is appended to by multiple tables, then some additional concurrency control > will be needed around it. > - Alternatively, there should be some pattern for having long-lived HTable > instances. However, since HTable is not thread-safe, we'd need multiple > instances, and a mechanism for leasing them out safely -- which sure sounds a > lot like the old HTablePool to me. > See discussion on mailing list here : > http://mail-archives.apache.org/mod_mbox/hbase-user/201412.mbox/%3CCAPdJLkEzmUQZ_kvD%3D8mrxi4V%3DhCmUp3g9MUZsddD%2Bmon%2BAvNtg%40mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332)