Hi Jeremy - Which specific version of 1.8 (1.8.0 or 1.8.1) did you test that is still performing?
And I don't know of any benchmarking that was done for 1.10 or 1.9. On Sat, Jun 5, 2021 at 10:04 PM Kepner, Jeremy - LLSC - MITLL < [email protected]> wrote: > I did a quick check with Accumulo 1.8 and I get the expected single node > performance scalability. > So between Accumulo 1.8 and 1.10.1 something changed that significantly > slowed the performance. > > > On Jun 5, 2021, at 8:48 PM, Kepner, Jeremy - LLSC - MITLL < > [email protected]> wrote: > > > > Hi Dave, > > I am looking into the Accumulo/Hadoop configuration. Hopefully it is > as simple as getting the settings the same. The hardware configurations is: > Dual Xeon Platinum 8260 2.4 GHz 48 cores, DDR4 2.93 GHz 192 GB RAM. I am > looking into the disk specs, but that shouldn't matter since the writes are > only a few megabytes. I also just tested on some older hardware that is > closer to what was used in the 2014 paper, and the single process ingest > rate is ~8x slower. > > > > Has anyone done any recent benchmarking of Accumulo 1.10+? > > > > Regards. -Jeremy > > > > > >> On Jun 5, 2021, at 7:08 PM, Dave Marion <[email protected]> wrote: > >> > >> Jeremy, > >> > >> Are you able to share any details about the hardware and the Accumulo > >> configuration? Is the Accumulo/Hadoop configuration the same as the > prior > >> test (no replication, WAL turned off, batch writer configuration, etc.) > >> > >> Dave > >> > >> On Sat, Jun 5, 2021 at 6:12 PM Kepner, Jeremy - LLSC - MITLL < > >> [email protected]> wrote: > >> > >>> Has anyone benchmarked Accumulo 1.10.1? I have been looking into > repeating > >>> the measurements we did in 2014 with Accumulo 1.5 ( > >>> https://arxiv.org/abs/1406.4923) using Accumulo 1.10.1 on a bigger > system > >>> with more modern hardware. Unfortunately, when I repeat the single > node > >>> measurements, there is no performance improvement from having multiple > >>> ingestors inserting into different presplits of a table. I get 120K > >>> inserts/sec with one ingestor and 2x60K inserts/sec with two > ingestors. In > >>> 2014 we got linear speedup to ~6 ingestors, providing ~600K > inserts/sec on > >>> a single node. > >>> > >>> Regards. -Jeremy > > > >
