[
https://issues.apache.org/jira/browse/SOLR-5316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
olivier soyez updated SOLR-5316:
--------------------------------
Attachment: SOLR-5316.patch
> Solr 4.2.1 LotsOfCores new options
> ----------------------------------
>
> Key: SOLR-5316
> URL: https://issues.apache.org/jira/browse/SOLR-5316
> Project: Solr
> Issue Type: Improvement
> Components: multicore
> Affects Versions: 4.2.1, 4.7
> Reporter: olivier soyez
> Assignee: Erick Erickson
> Priority: Minor
> Labels: patch
> Fix For: 4.2.1
>
> Attachments: SOLR-5316.patch, solr-4.2.1.patchLOTSOFCORES
>
>
> The SOLR-5316.patch is for the lotsofcores use case: a large number of
> homogeneous cores with frequent loading/unloading. The new Cores options are:
> - defaultLoadOnStartup : true by default
> - defaultTransient : false by default
> In this way, you can change the default attributes value for all cores.
> The lotsofcores use case requires to have defaultLoadOnStartup = false and
> defaultTransient = true
> - "numBuckets" to create a subdirectory based on a hash on the corename %
> numBuckets in the core Datadir, because all cores cannot live in the same
> directory
> - "auto" with 2 differents values :
> 1) createLoad: create, if not exist, and load the core on the fly on
> the first incoming request (update, select).
> 2) onlyLoad: load the core on the fly on the first incoming request
> (update, select), if exist on disk
> The auto option uses an additional cores option named baseDataDir to
> automatically generate the dataDir of each core and uses the "numBucket"
> option if exists. All the cores have the same solr config defined in the
> solr_home/conf.
> - discoverOnDemand : false by default. If enabled, the discovery and loading
> of cores at the solr startup are disabled and a new option named
> coreDiscovery in the STATUS action is available to discover on demand all the
> cores in the coreRootDirectory (only add the cores descriptions)
> - noCorePropertiesFile : false by default. If enabled, don't use anymore the
> core.properties file, a core will be detected on disk based on the file
> segments.gen
> The unload command was also modified in order to handle the non loaded
> transient cores, to be able to apply for example the deleteIndex,
> deleteDataDir options.
> I added some Junit tests.
> An example of solr.xml:
> <solr>
> <str name="shareSchema">schema.xml</str>
> <int name="transientCacheSize">5000</int>
> <str name="coreRootDirectory">solr_home/data</str>
> <str name="baseDataDir">solr_home/data</str>
> <int name="numBuckets">100</int>
> <str name="auto">createLoad</str>
> <str name="defaultLoadOnStartup">false</str>
> <str name="defaultTransient">true</str>
> <bool name="discoverOnDemand">true</bool>
> <bool name="noCorePropertiesFile">true</bool>
> </solr>
> The patch SOLR-5316.patch in attachment is for svn solr branch_4X (revision
> number 1556554)
> ---------------
> The patch solr-4.2.1.patchLOTSOFCORES is for the lotsofcores use case,
> including some modification :
> - by default, all cores have loadOnStartup="false" and transient="true"
> attributes
> - the create admin command can register a lazy core (to take into account the
> transientCacheSize option)
> - add transient cores persistency
> - handle unload admin command for never launched transient cores (non active
> cores)
> To improve performance, we use this Solr patched version with the persistence
> disabled.
> In this way, Solr is working with a solr.xml file without any core entries,
> because it's useless in our use case (with the new Auto option for the cores)
> The new Cores options :
> - "numBuckets" to create a subdirectory based on a hash on the corename
> % numBuckets in the core Datadir, because all cores cannot live in the
> same directory
> - "Auto" with 3 differents values :
> 1) false : default behaviour
> 2) createLoad : create, if not exist, and load the core on the fly on
> the first incoming request (update, select).
> 3) onlyLoad : load the core on the fly on the first incoming request
> (update, select), if exist on disk
> The Auto option uses an additional cores option named baseDataDir to
> automatically generate the dataDir of each core and uses the "numBucket"
> option if exists.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
