I did a quick check with Accumulo 1.8 and I get the expected single node performance scalability. So between Accumulo 1.8 and 1.10.1 something changed that significantly slowed the performance.
> On Jun 5, 2021, at 8:48 PM, Kepner, Jeremy - LLSC - MITLL <[email protected]> > wrote: > > Hi Dave, > I am looking into the Accumulo/Hadoop configuration. Hopefully it is as > simple as getting the settings the same. The hardware configurations is: Dual > Xeon Platinum 8260 2.4 GHz 48 cores, DDR4 2.93 GHz 192 GB RAM. I am looking > into the disk specs, but that shouldn't matter since the writes are only a > few megabytes. I also just tested on some older hardware that is closer to > what was used in the 2014 paper, and the single process ingest rate is ~8x > slower. > > Has anyone done any recent benchmarking of Accumulo 1.10+? > > Regards. -Jeremy > > >> On Jun 5, 2021, at 7:08 PM, Dave Marion <[email protected]> wrote: >> >> Jeremy, >> >> Are you able to share any details about the hardware and the Accumulo >> configuration? Is the Accumulo/Hadoop configuration the same as the prior >> test (no replication, WAL turned off, batch writer configuration, etc.) >> >> Dave >> >> On Sat, Jun 5, 2021 at 6:12 PM Kepner, Jeremy - LLSC - MITLL < >> [email protected]> wrote: >> >>> Has anyone benchmarked Accumulo 1.10.1? I have been looking into repeating >>> the measurements we did in 2014 with Accumulo 1.5 ( >>> https://arxiv.org/abs/1406.4923) using Accumulo 1.10.1 on a bigger system >>> with more modern hardware. Unfortunately, when I repeat the single node >>> measurements, there is no performance improvement from having multiple >>> ingestors inserting into different presplits of a table. I get 120K >>> inserts/sec with one ingestor and 2x60K inserts/sec with two ingestors. In >>> 2014 we got linear speedup to ~6 ingestors, providing ~600K inserts/sec on >>> a single node. >>> >>> Regards. -Jeremy >
