It's going to take some time to review the changes[1], but I don't see changes in the default JVM sizes. I was wondering if maybe the issue is that it's running faster. You are loading the same amount of data, but is it going faster by chance? If so, you could be creating more garbage per unit time putting more pressure on the GC. Just a thought.
[1] https://github.com/apache/accumulo/compare/rel/1.9.3..rel/1.10.2 On Fri, Nov 4, 2022 at 4:02 PM Logan Jones <lo...@codescratch.com> wrote: > Yeah, our memory usage is drastically different since the upgrade. > > We are seeing spikes in heap utilization on tablet servers that weren't > happening before the upgrade despite our ingest load being roughly the > same. This increase in heap utilization seems to be causing long GC times. > Those GC times are long enough that the tablet servers lose their locks and > then die. > > Looking into the JVM options, we don't see anything obvious that changed > around the garbage collector, and looking at the Accumulo release notes > didn't leave us any indication that something like this should have > changed, but nevertheless we are seeing crashes of tservers. I'm mostly > trying to identify whether or not rollback is even an option. > > - Logan > > On Fri, Nov 4, 2022 at 3:49 PM Dave Marion <dmario...@gmail.com> wrote: > > > Are you running into an error or some other issue that is making you > > think that you have to rollback? I don't know that rolling back has been > > tested. > > > > On Fri, Nov 4, 2022 at 3:40 PM Logan Jones <lo...@codescratch.com> > wrote: > > > > > Hello: > > > > > > We recently upgraded from Accumulo 1.9.3 to 1.10.2. Is it safe to roll > > back > > > to Accumulo 1.9.3? > > > > > > Thanks in advance, > > > > > > - Logan > > > > > >
