Hi Dave, I am looking into the Accumulo/Hadoop configuration. Hopefully it is as simple as getting the settings the same. The hardware configurations is: Dual Xeon Platinum 8260 2.4 GHz 48 cores, DDR4 2.93 GHz 192 GB RAM. I am looking into the disk specs, but that shouldn't matter since the writes are only a few megabytes. I also just tested on some older hardware that is closer to what was used in the 2014 paper, and the single process ingest rate is ~8x slower.
Has anyone done any recent benchmarking of Accumulo 1.10+? Regards. -Jeremy > On Jun 5, 2021, at 7:08 PM, Dave Marion <[email protected]> wrote: > > Jeremy, > > Are you able to share any details about the hardware and the Accumulo > configuration? Is the Accumulo/Hadoop configuration the same as the prior > test (no replication, WAL turned off, batch writer configuration, etc.) > > Dave > > On Sat, Jun 5, 2021 at 6:12 PM Kepner, Jeremy - LLSC - MITLL < > [email protected]> wrote: > >> Has anyone benchmarked Accumulo 1.10.1? I have been looking into repeating >> the measurements we did in 2014 with Accumulo 1.5 ( >> https://arxiv.org/abs/1406.4923) using Accumulo 1.10.1 on a bigger system >> with more modern hardware. Unfortunately, when I repeat the single node >> measurements, there is no performance improvement from having multiple >> ingestors inserting into different presplits of a table. I get 120K >> inserts/sec with one ingestor and 2x60K inserts/sec with two ingestors. In >> 2014 we got linear speedup to ~6 ingestors, providing ~600K inserts/sec on >> a single node. >> >> Regards. -Jeremy
