Hi Uwe, I’m currently dealing with a customer trying to use Lucene/Solr/Elasticsearch and I expected that would be a perfect candidate for the G1 but.... I think that other “off-heap” solutions might also suffer. And, as you now know, it takes a serious amount of digging to sort these problems out. I am certain there are very few dev teams with the talent on board to work through the diagnostic process.
Regards, Kirk PS, yes it was indeed nice to meet you @ JAX. On Apr 30, 2015, at 4:29 PM, Uwe Schindler <uschind...@apache.org> wrote: > Hi Kirk, hi Mark, > > the Lucene/Solr/Elasticsearch people still recommend to their users to not > use G1GC, although for this type of application (full text search with the > requirement for very low response times and no pauses) is a good candidate > for G1GC. On the other hand, heap sizes for typical Lucene applications > should not be too high, because most of the processing is done on memory > mapped files off-heap. So heaps should be at most 1/4 of the physical RAM > available, because Lucene relies on the fact that the index files reside in > file system cache (too large heaps are contra-productive here). > > See also our recommendations for Apache Solr and Elasticsearch: > http://wiki.apache.org/solr/ShawnHeisey#GC_Tuning_for_Solr > http://www.elastic.co/guide/en/elasticsearch/guide/current/_don_8217_t_touch_these_settings.html > > Currently Lucene's indexing sometimes caused serious data corruption with > G1GC - leading to data loss, which was mainly caused by some bugs around G1GC > and its use of additional memory barriers and the very close interaction with > Hotspot, that seemed to broke some optimizations. We had (only in combination > with G1GC during our test suites) simple "assert" statements *sometimes* > failing that should never fail unless there is a bug in the JVM. > > We are aware that Java 8u40 declared G1GC as "production ready", so we are > still looking at failures in our extensive testing infrastructure. Indeed, I > have no seen any G1GC related problems recently, but that is not necessarily > a sign for correctness. > > Uwe > > P.S.: It was nice to meet you last week on JAX! > > ----- > Uwe Schindler > uschind...@apache.org > ASF Member, Apache Lucene PMC / Committer > Bremen, Germany > http://lucene.apache.org/ > >> -----Original Message----- >> From: hotspot-dev [mailto:hotspot-dev-boun...@openjdk.java.net] On >> Behalf Of Kirk Pepperdine >> Sent: Wednesday, April 29, 2015 9:11 AM >> To: hotspot-...@openjdk.java.net Source Developers >> Subject: Re: JEP 248: Make G1 the Default Garbage Collector >> >> Hi all, >> >> Is the G1 ready for this? I see many people moving to G1 but also I’m not >> sure that we’ve got the tunable correct. I’ve been sorting through a number >> of recent tuning engagements and my conclusion is that I would like the >> collector to be aggressive about collecting tenured regions at the beginning >> of a JVM’s life time but then become less aggressive over time. The reason is >> the residual waste that I see left behind because certain regions never hit >> the threshold needed to be included in the CSET. But, on aggregate, the >> number of regions in this state does start to retain a significant about of >> dead >> data. The only way to see the effects is to run regular Full GCs.. which of >> course you don’t really want to do. However, the problem seems to settle >> down a wee bit over time which is why I was thinking that being aggressive >> about what is collected in the early stages of a JVMs life should lead to >> better >> packing and hence less waste. >> >> Note, I don’t really care about the memory waste, only it’s effect on cycle >> frequencies and pause times. >> >> Sorry but I don’t have anything formal about this as I (and I believe many >> others) are still sorting out what to make of the G1 in prod. Generally the >> overall results are good but sometimes it’s not that way up front and how to >> improve things is sometimes challenging. >> >> On a side note, the move to Tiered in 8 has also caused a bit of grief. >> Metaspace has caused a bit of grief and even parallelStream, which works, >> has come with some interesting side effect. Everyone has been so enamored >> with Lambdas (rightfully so) that the other stuff has been completely >> forgotten and some of it has surprised people. I guess I’ll be submitting a >> talk >> for J1 on some of the field experience I’ve had with the other stuff. >> >> Regards, >> Kirk >> >> >> On Apr 28, 2015, at 11:02 PM, mark.reinh...@oracle.com wrote: >> >>> New JEP Candidate: http://openjdk.java.net/jeps/248 >>> >>> - Mark >
signature.asc
Description: Message signed with OpenPGP using GPGMail
