I am not using the HDFS class loader so I need to look in some other place to find out what the problem is with log4j2.
I increased the size of table.scan.max.memory up to 1MB and it worked. I do not have big Key-Values but I am using a "kind of document-partitioned table" in which my iterator is pre-loading a SortedSet of Keys from the index section of the row and then using them to go over the data section one key-value pair at time. In order to achieve good performance I need that my iterator re-calculates the set of Keys from the index as few times as possible. Thanks. Regards, Max From: Josh Elser <[email protected]> To: [email protected] Date: 22/11/2016 20:39 Subject: Re: how to reduce re-seeking rate? There isn't any funny classloading happening in the normal case, so having the log4j2.xml file in your jar should be sufficient. Caveat is if you're using the HDFS classloading stuff, but that's something you would have enabled by hand if you're using it. I think the scan max memory that Dave pointed out is the only knob for this one. I don't think we have any other sort of policy that governs lifecycle of iterators. It's not intended from a framework that re-instantiation of a batch of results is costly. Making a guess: are you returning very large Key-Values? [email protected] wrote: > In one case, the tserver will send data back to the client when it fills > its buffer. When this happens, it?s possible that the iterator could be > torn down and re-seeked to the last key returned. You could increase the > size of this buffer to see if that helps > ( http://accumulo.apache.org/1.8/accumulo_user_manual.html#_table_scan_max_memory ) > > *From:*Massimilian Mattetti [mailto:[email protected]] > *Sent:* Tuesday, November 22, 2016 10:37 AM > *To:* [email protected] > *Subject:* how to reduce re-seeking rate? > > Hi all, > > I developed an iterator that pre-loads a set of results every time it > jumps to a new row. Checking the logs (by the way I am using log4j2 as > logging library, but it is not able to locate the log4j2.xml that is > inside the iterator jar, should I put it in a different place?) I > noticed that Accumulo is re-seeking after each key returned by my > iterator. This is killing the performance of my system, is there a way > to reduce the rate in which Accumulo kill and re-seek the iterators? > Thanks > > Regards, > > *Massimiliano Mattetti* > > > > >
