bq. and Mark is representing "just keep working, ok?". But I'm not :) Like I said, I don't view the purpose of a soft value cache as avoiding OOM's. Size your caches correctly for that.
For those that don't understand the how and why of soft value caches, they probably should not choose to use it. Lance Norskog wrote: > On-topic: Will the Google implementations + soft references behave > well with 8+ processors? > > Semi-on-topic: If you want to really know multiprocessor algorithms, > this is the bible: "The Art Of Multiprocessor Programming". Hundreds > of parallel algorithms for many different jobs, all coded in Java, and > cross-referenced with the java.util.concurrent package. Just amazing. > > http://www.elsevier.com/wps/find/bookdescription.cws_home/714091/description#description > > Off-topic: I was representing a system troubleshooting philosophy: > "Fail Early, Fail Loud". Meaning, if there is a problem like OOMs, > tell me and I'll fix it permanently. But different situations call for > different answers, and Mark is representing "just keep working, ok?". > Brittle v.s. Supple is one way to think of it. > > On Tue, Oct 20, 2009 at 11:27 AM, Shalin Shekhar Mangar > <shalinman...@gmail.com> wrote: > >> On Tue, Oct 20, 2009 at 3:56 PM, Mark Miller <markrmil...@gmail.com> wrote: >> >> >>> On Oct 20, 2009, at 12:12 AM, Shalin Shekhar Mangar < >>> shalinman...@gmail.com> wrote: >>> >>> I don't think the debate is about weak reference vs. soft references. >>> >>> There appears to be confusion between the two here no matter what the >>> debate - soft references are for cachinh, weak references are not so much. >>> Getting it right is important. >>> >>> I >>> >>>> guess the point that Lance is making is that using such a technique will >>>> make application performance less predictable. There's also a good chance >>>> that a soft reference based cache will cause cache thrashing and will hide >>>> OOMs caused by inadequate cache sizes. So basically we trade an OOM for >>>> more >>>> CPU usage (due to re-computation of results). >>>> >>>> >>> That's the whole point. Your not hiding anything. I don't follow you. >>> >>> >> Using a soft reference based cache can hide the fact that one has inadequate >> memory for the cache size one has configured. Don't get me wrong. I'm not >> against the feature. I was merely trying to explain Lance's concerns as I >> understood them. >> >> >> >>> >>> >>>> Personally, I think giving an option is fine. What if the user does not >>>> have >>>> enough RAM and he is willing to pay the price? Right now, there is no way >>>> he >>>> can do that at all. However, the most frequent reason behind OOMs is not >>>> having enough RAM to create the field caches and not Solr caches, so I'm >>>> not >>>> sure how important this is. >>>> >>>> >>> How important is any feature? You don't have a use for it, so it's not >>> important to you - someone else does so it is important to them. Soft value >>> caches can be useful. >>> >> Don't jump to conclusions :) >> >> The reason behind this feature request is to have Solr caches which resize >> themselves when enough memory is not available. I agree that soft value >> caches are useful for this. All I'm saying is that most OOMs that get >> reported on the list are due to inadequate free memory for allocating field >> caches. Finding a way around that will be the key to make a Lucene/Solr >> application practical in a limited memory environment. >> >> Just for the record, I'm +1 for adding this feature but keeping the current >> behavior as the default. >> >> -- >> Regards, >> Shalin Shekhar Mangar. >> >> > > > > -- - Mark http://www.lucidimagination.com
