Jeremy, It seems that you have the ability to quickly run your test to determine if a release is "good". Testing 1.8 ruled out a lot of commits for us to look at. Would it be possible for you to test a few others so that we can try and narrow it down even more. The following releases are after 1.8.0:
1.8.1 1.9.0 1.9.1 1.9.2 1.9.3 1.10.0 1.10.1 Could you test with the 1.9.2 release? > On 06/05/2021 10:04 PM Kepner, Jeremy - LLSC - MITLL <[email protected]> > wrote: > > > I did a quick check with Accumulo 1.8 and I get the expected single node > performance scalability. > So between Accumulo 1.8 and 1.10.1 something changed that significantly > slowed the performance. > > > On Jun 5, 2021, at 8:48 PM, Kepner, Jeremy - LLSC - MITLL > > <[email protected]> wrote: > > > > Hi Dave, > > I am looking into the Accumulo/Hadoop configuration. Hopefully it is as > > simple as getting the settings the same. The hardware configurations is: > > Dual Xeon Platinum 8260 2.4 GHz 48 cores, DDR4 2.93 GHz 192 GB RAM. I am > > looking into the disk specs, but that shouldn't matter since the writes are > > only a few megabytes. I also just tested on some older hardware that is > > closer to what was used in the 2014 paper, and the single process ingest > > rate is ~8x slower. > > > > Has anyone done any recent benchmarking of Accumulo 1.10+? > > > > Regards. -Jeremy > > > > > >> On Jun 5, 2021, at 7:08 PM, Dave Marion <[email protected]> wrote: > >> > >> Jeremy, > >> > >> Are you able to share any details about the hardware and the Accumulo > >> configuration? Is the Accumulo/Hadoop configuration the same as the prior > >> test (no replication, WAL turned off, batch writer configuration, etc.) > >> > >> Dave > >> > >> On Sat, Jun 5, 2021 at 6:12 PM Kepner, Jeremy - LLSC - MITLL < > >> [email protected]> wrote: > >> > >>> Has anyone benchmarked Accumulo 1.10.1? I have been looking into repeating > >>> the measurements we did in 2014 with Accumulo 1.5 ( > >>> https://arxiv.org/abs/1406.4923) using Accumulo 1.10.1 on a bigger system > >>> with more modern hardware. Unfortunately, when I repeat the single node > >>> measurements, there is no performance improvement from having multiple > >>> ingestors inserting into different presplits of a table. I get 120K > >>> inserts/sec with one ingestor and 2x60K inserts/sec with two ingestors. > >>> In > >>> 2014 we got linear speedup to ~6 ingestors, providing ~600K inserts/sec on > >>> a single node. > >>> > >>> Regards. -Jeremy > >
