John: If you'd like to add your experience to the Wiki, create an ID and let us know what it is and we'll add you to the contributors list. Unfortunately we had problems with spam pages to we added this step.
Make sure you include your logon in the request. Thanks, Erick On Fri, Jun 14, 2013 at 8:55 AM, John Nielsen <j...@mcb.dk> wrote: > Sorry for not getting back to the list sooner. It seems like I finally > solved the memory problems by following Toke's instruction of splitting the > cores up into smaller chunks. > > After some major refactoring, our 15 cores have now turned into ~500 cores > and our memory consumption has dropped dramaticly. Running 200 webshops now > actually uses less memory as our 24 test shops did before. > > Thank you to everyone who helped, and especially to Toke. > > I looked at the wiki, but could not find any reference to this unintuitive > way of using memory. Did I miss it somewhere? > > > > On Fri, Apr 19, 2013 at 1:30 PM, Erick Erickson > <erickerick...@gmail.com>wrote: > >> Hmmm. There has been quite a bit of work lately to support a couple of >> things that might be of interest (4.3, which Simon cut today, probably >> available to all mid next week at the latest). Basically, you can >> choose to pre-define all the cores in solr.xml (so-called "old style") >> _or_ use the new-style solr.xml which uses "auto-discover" mode to >> walk the indicated directory and find all the cores (indicated by the >> presence of a 'core.properties' file). Don't know if this would make >> your particular case easier, and I should warn you that this is >> relatively new code (although there are some reasonable unit tests). >> >> You also have the option to only load the cores when they are >> referenced, and only keep N cores open at a time (loadOnStartup and >> transient properties). >> >> See: http://wiki.apache.org/solr/CoreAdmin#Configuration and >> http://wiki.apache.org/solr/Solr.xml%204.3%20and%20beyond >> >> Note, the docs are somewhat sketchy, so if you try to go down this >> route let us know anything that should be improved (or you can be >> added to the list of wiki page contributors and help out!) >> >> Best >> Erick >> >> On Thu, Apr 18, 2013 at 8:31 AM, John Nielsen <j...@mcb.dk> wrote: >> >> You are missing an essential part: Both the facet and the sort >> >> structures needs to hold one reference for each document >> >> _in_the_full_index_, even when the document does not have any values in >> >> the fields. >> >> >> > >> > Wow, thank you for this awesome explanation! This is where the penny >> > dropped for me. >> > >> > I will definetely move to a multi-core setup. It will take some time and >> a >> > lot of re-coding. As soon as I know the result, I will let you know! >> > >> > >> > >> > >> > >> > >> > -- >> > Med venlig hilsen / Best regards >> > >> > *John Nielsen* >> > Programmer >> > >> > >> > >> > *MCB A/S* >> > Enghaven 15 >> > DK-7500 Holstebro >> > >> > Kundeservice: +45 9610 2824 >> > p...@mcb.dk >> > www.mcb.dk >> > > > > -- > Med venlig hilsen / Best regards > > *John Nielsen* > Programmer > > > > *MCB A/S* > Enghaven 15 > DK-7500 Holstebro > > Kundeservice: +45 9610 2824 > p...@mcb.dk > www.mcb.dk
