I wouldn't use jvm reuse at this point. It's had a number of issues over time and I've consistently switched it off for a long while now.
Arun On Dec 26, 2011, at 2:50 PM, Markus Jelsma wrote: > Hi, > >> Markus, >> >> Good to know you fixed it now :) >> >> Also consider raising reduce slowstart to a more suitable value, like >> 80-85% -- this is pretty beneficial in both heap consumption and more >> performant Reduce slots. The property name is >> "mapred.reduce.slowstart.completed.maps" and carries a float value >> (percentage). > > Is that going to help if one has a non-zero value for reusing JVM instances? > We have little concurrent jobs running because the process flow is linear and > we prefer the benefit of copying map output while it's running. > For this particular job, however, it is not the case since all mappers finish > almost simultaneously so changing that value would not make a difference > right? 80% finishes within a matter of seconds. > > For the record, it's an Apache Nutch fetcher job configured to terminate > after > 1 hour. This downloads URL's, hence the disabling of speculative execution.
