[
https://issues.apache.org/jira/browse/ACCUMULO-2842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14007642#comment-14007642
]
Josh Elser commented on ACCUMULO-2842:
--------------------------------------
Aforementioned "contrived benchmarks"
{noformat}
1.5.2-SNAPSHOT (74acc30), 5 process, each writing 1M entries, with 500B
values. TServer running with 256M max WAL size, 4M tserver.mutation.queue.max,
on Apache Hadoop 2.4.0
With HSync
-------------
Run 1:
1,000,000 records written | 9,543 records/sec | 529,000,000 bytes written |
5,048,673 bytes/sec | 104.780 secs
1,000,000 records written | 9,523 records/sec | 529,000,000 bytes written |
5,038,095 bytes/sec | 105.000 secs
1,000,000 records written | 9,377 records/sec | 529,000,000 bytes written |
4,960,754 bytes/sec | 106.637 secs
1,000,000 records written | 9,355 records/sec | 529,000,000 bytes written |
4,949,244 bytes/sec | 106.885 secs
1,000,000 records written | 9,037 records/sec | 529,000,000 bytes written |
4,781,056 bytes/sec | 110.645 secs
Run 2:
1,000,000 records written | 8,981 records/sec | 529,000,000 bytes written |
4,750,956 bytes/sec | 111.346 secs
1,000,000 records written | 8,870 records/sec | 529,000,000 bytes written |
4,692,461 bytes/sec | 112.734 secs
1,000,000 records written | 8,873 records/sec | 529,000,000 bytes written |
4,694,210 bytes/sec | 112.692 secs
1,000,000 records written | 8,807 records/sec | 529,000,000 bytes written |
4,659,109 bytes/sec | 113.541 secs
1,000,000 records written | 8,666 records/sec | 529,000,000 bytes written |
4,584,373 bytes/sec | 115.392 secs
Run 3:
1,000,000 records written | 8,524 records/sec | 529,000,000 bytes written |
4,509,611 bytes/sec | 117.305 secs
1,000,000 records written | 8,389 records/sec | 529,000,000 bytes written |
4,438,031 bytes/sec | 119.197 secs
1,000,000 records written | 8,337 records/sec | 529,000,000 bytes written |
4,410,317 bytes/sec | 119.946 secs
1,000,000 records written | 8,312 records/sec | 529,000,000 bytes written |
4,397,559 bytes/sec | 120.294 secs
1,000,000 records written | 8,276 records/sec | 529,000,000 bytes written |
4,378,088 bytes/sec | 120.829 secs
Run 4:
1,000,000 records written | 9,318 records/sec | 529,000,000 bytes written |
4,929,321 bytes/sec | 107.317 secs
1,000,000 records written | 8,872 records/sec | 529,000,000 bytes written |
4,693,544 bytes/sec | 112.708 secs
1,000,000 records written | 8,860 records/sec | 529,000,000 bytes written |
4,687,430 bytes/sec | 112.855 secs
1,000,000 records written | 8,666 records/sec | 529,000,000 bytes written |
4,584,810 bytes/sec | 115.381 secs
1,000,000 records written | 8,247 records/sec | 529,000,000 bytes written |
4,363,138 bytes/sec | 121.243 secs
With HFlush:
===============
Run 1:
1,000,000 records written | 12,828 records/sec | 529,000,000 bytes written
| 6,786,140 bytes/sec | 77.953 secs
1,000,000 records written | 12,801 records/sec | 529,000,000 bytes written
| 6,771,893 bytes/sec | 78.117 secs
1,000,000 records written | 12,759 records/sec | 529,000,000 bytes written
| 6,749,687 bytes/sec | 78.374 secs
1,000,000 records written | 12,730 records/sec | 529,000,000 bytes written
| 6,734,392 bytes/sec | 78.552 secs
1,000,000 records written | 12,725 records/sec | 529,000,000 bytes written
| 6,731,564 bytes/sec | 78.585 secs
Run 2:
1,000,000 records written | 12,046 records/sec | 529,000,000 bytes written
| 6,372,572 bytes/sec | 83.012 secs
1,000,000 records written | 12,016 records/sec | 529,000,000 bytes written
| 6,356,492 bytes/sec | 83.222 secs
1,000,000 records written | 12,012 records/sec | 529,000,000 bytes written
| 6,354,507 bytes/sec | 83.248 secs
1,000,000 records written | 12,002 records/sec | 529,000,000 bytes written
| 6,349,092 bytes/sec | 83.319 secs
1,000,000 records written | 11,985 records/sec | 529,000,000 bytes written
| 6,340,340 bytes/sec | 83.434 secs
Run 3:
1,000,000 records written | 13,313 records/sec | 529,000,000 bytes written
| 7,042,909 bytes/sec | 75.111 secs
1,000,000 records written | 13,304 records/sec | 529,000,000 bytes written
| 7,038,318 bytes/sec | 75.160 secs
1,000,000 records written | 13,210 records/sec | 529,000,000 bytes written
| 6,988,203 bytes/sec | 75.699 secs
1,000,000 records written | 13,215 records/sec | 529,000,000 bytes written
| 6,991,066 bytes/sec | 75.668 secs
1,000,000 records written | 13,204 records/sec | 529,000,000 bytes written
| 6,985,065 bytes/sec | 75.733 secs
Run 4:
1,000,000 records written | 13,252 records/sec | 529,000,000 bytes written
| 7,010,801 bytes/sec | 75.455 secs
1,000,000 records written | 13,133 records/sec | 529,000,000 bytes written
| 6,947,454 bytes/sec | 76.143 secs
1,000,000 records written | 13,083 records/sec | 529,000,000 bytes written
| 6,921,184 bytes/sec | 76.432 secs
1,000,000 records written | 13,095 records/sec | 529,000,000 bytes written
| 6,927,529 bytes/sec | 76.362 secs
1,000,000 records written | 13,037 records/sec | 529,000,000 bytes written
| 6,896,641 bytes/sec | 76.704 secs
{noformat}
> Allow for use of hflush instead of hsync
> ----------------------------------------
>
> Key: ACCUMULO-2842
> URL: https://issues.apache.org/jira/browse/ACCUMULO-2842
> Project: Accumulo
> Issue Type: Improvement
> Components: logger
> Reporter: Josh Elser
> Assignee: Josh Elser
> Fix For: 1.5.2, 1.6.1, 1.7.0
>
>
> We keep getting onto the discussion on hsync versus hflush performance;
> however, we lack a way to actually test this.
> Make the invocation of hflush or hsync configurable. It should not change the
> default, so users on existing major releases should not be affected unless
> they choose to change their configuration.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
