[jira] [Updated] (SOLR-5316) Solr 4.2.1 LotsOfCores new options
[ https://issues.apache.org/jira/browse/SOLR-5316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] olivier soyez updated SOLR-5316: Description: The SOLR-5316.patch is for the lotsofcores use case: a large number of homogeneous cores with frequent loading/unloading. The new Cores options are: - defaultLoadOnStartup : true by default - defaultTransient : false by default In this way, you can change the default attributes value for all cores. The lotsofcores use case requires to have defaultLoadOnStartup = false and defaultTransient = true - numBuckets to create a subdirectory based on a hash on the corename % numBuckets in the core Datadir, because all cores cannot live in the same directory - auto with 2 differents values : 1) createLoad: create, if not exist, and load the core on the fly on the first incoming request (update, select). 2) onlyLoad: load the core on the fly on the first incoming request (update, select), if exist on disk The auto option uses an additional cores option named baseDataDir to automatically generate the dataDir of each core and uses the numBucket option if exists. All the cores have the same solr config defined in the solr_home/conf. - discoverOnDemand : false by default. If enabled, the discovery and loading of cores at the solr startup are disabled and a new option named coreDiscovery in the STATUS action is available to discover on demand all the cores in the coreRootDirectory (only add the cores descriptions) - noCorePropertiesFile : false by default. If enabled, don't use anymore the core.properties file, a core will be detected on disk based on the file segments.gen The unload command was also modified in order to handle the non loaded transient cores, to be able to apply for example the deleteIndex, deleteDataDir options. I added some Junit tests. An example of solr.xml: solr str name=shareSchemaschema.xml/str int name=transientCacheSize5000/int str name=coreRootDirectorysolr_home/data/str str name=baseDataDirsolr_home/data/str int name=numBuckets100/int str name=autocreateLoad/str str name=defaultLoadOnStartupfalse/str str name=defaultTransienttrue/str bool name=discoverOnDemandtrue/bool bool name=noCorePropertiesFiletrue/bool /solr The patch SOLR-5316.patch in attachment is for svn solr branch_4X (revision number 1556554) --- This patch is for the lotsofcores use case, including some modification : - by default, all cores have loadOnStartup=false and transient=true attributes - the create admin command can register a lazy core (to take into account the transientCacheSize option) - add transient cores persistency - handle unload admin command for never launched transient cores (non active cores) To improve performance, we use this Solr patched version with the persistence disabled. In this way, Solr is working with a solr.xml file without any core entries, because it's useless in our use case (with the new Auto option for the cores) The new Cores options : - numBuckets to create a subdirectory based on a hash on the corename % numBuckets in the core Datadir, because all cores cannot live in the same directory - Auto with 3 differents values : 1) false : default behaviour 2) createLoad : create, if not exist, and load the core on the fly on the first incoming request (update, select). 3) onlyLoad : load the core on the fly on the first incoming request (update, select), if exist on disk The Auto option uses an additional cores option named baseDataDir to automatically generate the dataDir of each core and uses the numBucket option if exists. was: This patch is for the lotsofcores use case, including some modification : - by default, all cores have loadOnStartup=false and transient=true attributes - the create admin command can register a lazy core (to take into account the transientCacheSize option) - add transient cores persistency - handle unload admin command for never launched transient cores (non active cores) To improve performance, we use this Solr patched version with the persistence disabled. In this way, Solr is working with a solr.xml file without any core entries, because it's useless in our use case (with the new Auto option for the cores) The new Cores options : - numBuckets to create a subdirectory based on a hash on the corename % numBuckets in the core Datadir, because all cores cannot live in the same directory - Auto with 3 differents values : 1) false : default behaviour 2) createLoad : create, if not exist, and load the core on the fly on the first incoming request (update, select). 3) onlyLoad : load the core on the fly on the first incoming request (update, select), if exist on disk The Auto option uses an additional cores option named baseDataDir to automatically generate the dataDir of each core and uses the numBucket option if exists. Solr 4.2.1 LotsOfCores new
[jira] [Updated] (SOLR-5316) Solr 4.2.1 LotsOfCores new options
[ https://issues.apache.org/jira/browse/SOLR-5316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] olivier soyez updated SOLR-5316: Affects Version/s: 4.7 Solr 4.2.1 LotsOfCores new options -- Key: SOLR-5316 URL: https://issues.apache.org/jira/browse/SOLR-5316 Project: Solr Issue Type: Improvement Components: multicore Affects Versions: 4.2.1, 4.7 Reporter: olivier soyez Assignee: Erick Erickson Priority: Minor Labels: patch Fix For: 4.2.1 Attachments: solr-4.2.1.patchLOTSOFCORES The SOLR-5316.patch is for the lotsofcores use case: a large number of homogeneous cores with frequent loading/unloading. The new Cores options are: - defaultLoadOnStartup : true by default - defaultTransient : false by default In this way, you can change the default attributes value for all cores. The lotsofcores use case requires to have defaultLoadOnStartup = false and defaultTransient = true - numBuckets to create a subdirectory based on a hash on the corename % numBuckets in the core Datadir, because all cores cannot live in the same directory - auto with 2 differents values : 1) createLoad: create, if not exist, and load the core on the fly on the first incoming request (update, select). 2) onlyLoad: load the core on the fly on the first incoming request (update, select), if exist on disk The auto option uses an additional cores option named baseDataDir to automatically generate the dataDir of each core and uses the numBucket option if exists. All the cores have the same solr config defined in the solr_home/conf. - discoverOnDemand : false by default. If enabled, the discovery and loading of cores at the solr startup are disabled and a new option named coreDiscovery in the STATUS action is available to discover on demand all the cores in the coreRootDirectory (only add the cores descriptions) - noCorePropertiesFile : false by default. If enabled, don't use anymore the core.properties file, a core will be detected on disk based on the file segments.gen The unload command was also modified in order to handle the non loaded transient cores, to be able to apply for example the deleteIndex, deleteDataDir options. I added some Junit tests. An example of solr.xml: solr str name=shareSchemaschema.xml/str int name=transientCacheSize5000/int str name=coreRootDirectorysolr_home/data/str str name=baseDataDirsolr_home/data/str int name=numBuckets100/int str name=autocreateLoad/str str name=defaultLoadOnStartupfalse/str str name=defaultTransienttrue/str bool name=discoverOnDemandtrue/bool bool name=noCorePropertiesFiletrue/bool /solr The patch SOLR-5316.patch in attachment is for svn solr branch_4X (revision number 1556554) --- The patch solr-4.2.1.patchLOTSOFCORES is for the lotsofcores use case, including some modification : - by default, all cores have loadOnStartup=false and transient=true attributes - the create admin command can register a lazy core (to take into account the transientCacheSize option) - add transient cores persistency - handle unload admin command for never launched transient cores (non active cores) To improve performance, we use this Solr patched version with the persistence disabled. In this way, Solr is working with a solr.xml file without any core entries, because it's useless in our use case (with the new Auto option for the cores) The new Cores options : - numBuckets to create a subdirectory based on a hash on the corename % numBuckets in the core Datadir, because all cores cannot live in the same directory - Auto with 3 differents values : 1) false : default behaviour 2) createLoad : create, if not exist, and load the core on the fly on the first incoming request (update, select). 3) onlyLoad : load the core on the fly on the first incoming request (update, select), if exist on disk The Auto option uses an additional cores option named baseDataDir to automatically generate the dataDir of each core and uses the numBucket option if exists. -- This message was sent by Atlassian JIRA (v6.1.5#6160) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5316) Solr 4.2.1 LotsOfCores new options
[ https://issues.apache.org/jira/browse/SOLR-5316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] olivier soyez updated SOLR-5316: Description: The SOLR-5316.patch is for the lotsofcores use case: a large number of homogeneous cores with frequent loading/unloading. The new Cores options are: - defaultLoadOnStartup : true by default - defaultTransient : false by default In this way, you can change the default attributes value for all cores. The lotsofcores use case requires to have defaultLoadOnStartup = false and defaultTransient = true - numBuckets to create a subdirectory based on a hash on the corename % numBuckets in the core Datadir, because all cores cannot live in the same directory - auto with 2 differents values : 1) createLoad: create, if not exist, and load the core on the fly on the first incoming request (update, select). 2) onlyLoad: load the core on the fly on the first incoming request (update, select), if exist on disk The auto option uses an additional cores option named baseDataDir to automatically generate the dataDir of each core and uses the numBucket option if exists. All the cores have the same solr config defined in the solr_home/conf. - discoverOnDemand : false by default. If enabled, the discovery and loading of cores at the solr startup are disabled and a new option named coreDiscovery in the STATUS action is available to discover on demand all the cores in the coreRootDirectory (only add the cores descriptions) - noCorePropertiesFile : false by default. If enabled, don't use anymore the core.properties file, a core will be detected on disk based on the file segments.gen The unload command was also modified in order to handle the non loaded transient cores, to be able to apply for example the deleteIndex, deleteDataDir options. I added some Junit tests. An example of solr.xml: solr str name=shareSchemaschema.xml/str int name=transientCacheSize5000/int str name=coreRootDirectorysolr_home/data/str str name=baseDataDirsolr_home/data/str int name=numBuckets100/int str name=autocreateLoad/str str name=defaultLoadOnStartupfalse/str str name=defaultTransienttrue/str bool name=discoverOnDemandtrue/bool bool name=noCorePropertiesFiletrue/bool /solr The patch SOLR-5316.patch in attachment is for svn solr branch_4X (revision number 1556554) --- The patch solr-4.2.1.patchLOTSOFCORES is for the lotsofcores use case, including some modification : - by default, all cores have loadOnStartup=false and transient=true attributes - the create admin command can register a lazy core (to take into account the transientCacheSize option) - add transient cores persistency - handle unload admin command for never launched transient cores (non active cores) To improve performance, we use this Solr patched version with the persistence disabled. In this way, Solr is working with a solr.xml file without any core entries, because it's useless in our use case (with the new Auto option for the cores) The new Cores options : - numBuckets to create a subdirectory based on a hash on the corename % numBuckets in the core Datadir, because all cores cannot live in the same directory - Auto with 3 differents values : 1) false : default behaviour 2) createLoad : create, if not exist, and load the core on the fly on the first incoming request (update, select). 3) onlyLoad : load the core on the fly on the first incoming request (update, select), if exist on disk The Auto option uses an additional cores option named baseDataDir to automatically generate the dataDir of each core and uses the numBucket option if exists. was: The SOLR-5316.patch is for the lotsofcores use case: a large number of homogeneous cores with frequent loading/unloading. The new Cores options are: - defaultLoadOnStartup : true by default - defaultTransient : false by default In this way, you can change the default attributes value for all cores. The lotsofcores use case requires to have defaultLoadOnStartup = false and defaultTransient = true - numBuckets to create a subdirectory based on a hash on the corename % numBuckets in the core Datadir, because all cores cannot live in the same directory - auto with 2 differents values : 1) createLoad: create, if not exist, and load the core on the fly on the first incoming request (update, select). 2) onlyLoad: load the core on the fly on the first incoming request (update, select), if exist on disk The auto option uses an additional cores option named baseDataDir to automatically generate the dataDir of each core and uses the numBucket option if exists. All the cores have the same solr config defined in the solr_home/conf. - discoverOnDemand : false by default. If enabled, the discovery and loading of cores at the solr startup are disabled and a new option named coreDiscovery in the STATUS action is available to
[jira] [Updated] (SOLR-5316) Solr 4.2.1 LotsOfCores new options
[ https://issues.apache.org/jira/browse/SOLR-5316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] olivier soyez updated SOLR-5316: Attachment: SOLR-5316.patch Solr 4.2.1 LotsOfCores new options -- Key: SOLR-5316 URL: https://issues.apache.org/jira/browse/SOLR-5316 Project: Solr Issue Type: Improvement Components: multicore Affects Versions: 4.2.1, 4.7 Reporter: olivier soyez Assignee: Erick Erickson Priority: Minor Labels: patch Fix For: 4.2.1 Attachments: SOLR-5316.patch, solr-4.2.1.patchLOTSOFCORES The SOLR-5316.patch is for the lotsofcores use case: a large number of homogeneous cores with frequent loading/unloading. The new Cores options are: - defaultLoadOnStartup : true by default - defaultTransient : false by default In this way, you can change the default attributes value for all cores. The lotsofcores use case requires to have defaultLoadOnStartup = false and defaultTransient = true - numBuckets to create a subdirectory based on a hash on the corename % numBuckets in the core Datadir, because all cores cannot live in the same directory - auto with 2 differents values : 1) createLoad: create, if not exist, and load the core on the fly on the first incoming request (update, select). 2) onlyLoad: load the core on the fly on the first incoming request (update, select), if exist on disk The auto option uses an additional cores option named baseDataDir to automatically generate the dataDir of each core and uses the numBucket option if exists. All the cores have the same solr config defined in the solr_home/conf. - discoverOnDemand : false by default. If enabled, the discovery and loading of cores at the solr startup are disabled and a new option named coreDiscovery in the STATUS action is available to discover on demand all the cores in the coreRootDirectory (only add the cores descriptions) - noCorePropertiesFile : false by default. If enabled, don't use anymore the core.properties file, a core will be detected on disk based on the file segments.gen The unload command was also modified in order to handle the non loaded transient cores, to be able to apply for example the deleteIndex, deleteDataDir options. I added some Junit tests. An example of solr.xml: solr str name=shareSchemaschema.xml/str int name=transientCacheSize5000/int str name=coreRootDirectorysolr_home/data/str str name=baseDataDirsolr_home/data/str int name=numBuckets100/int str name=autocreateLoad/str str name=defaultLoadOnStartupfalse/str str name=defaultTransienttrue/str bool name=discoverOnDemandtrue/bool bool name=noCorePropertiesFiletrue/bool /solr The patch SOLR-5316.patch in attachment is for svn solr branch_4X (revision number 1556554) --- The patch solr-4.2.1.patchLOTSOFCORES is for the lotsofcores use case, including some modification : - by default, all cores have loadOnStartup=false and transient=true attributes - the create admin command can register a lazy core (to take into account the transientCacheSize option) - add transient cores persistency - handle unload admin command for never launched transient cores (non active cores) To improve performance, we use this Solr patched version with the persistence disabled. In this way, Solr is working with a solr.xml file without any core entries, because it's useless in our use case (with the new Auto option for the cores) The new Cores options : - numBuckets to create a subdirectory based on a hash on the corename % numBuckets in the core Datadir, because all cores cannot live in the same directory - Auto with 3 differents values : 1) false : default behaviour 2) createLoad : create, if not exist, and load the core on the fly on the first incoming request (update, select). 3) onlyLoad : load the core on the fly on the first incoming request (update, select), if exist on disk The Auto option uses an additional cores option named baseDataDir to automatically generate the dataDir of each core and uses the numBucket option if exists. -- This message was sent by Atlassian JIRA (v6.1.5#6160) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5316) Solr 4.2.1 LotsOfCores new options
[ https://issues.apache.org/jira/browse/SOLR-5316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] olivier soyez updated SOLR-5316: Attachment: solr-4.2.1.patchLOTSOFCORES Solr 4.2.1 LotsOfCores new options -- Key: SOLR-5316 URL: https://issues.apache.org/jira/browse/SOLR-5316 Project: Solr Issue Type: Improvement Components: multicore Affects Versions: 4.2.1 Reporter: olivier soyez Priority: Minor Labels: patch Fix For: 4.2.1 Attachments: solr-4.2.1.patchLOTSOFCORES This patch is for the lotsofcores use case, including some modification : - by default, all cores have loadOnStartup=false and transient=true attributes - the create admin command can register a lazy core (to take into account the transientCacheSize option) - add transient cores persistency - handle unload admin command for never launched transient cores (non active cores) The new Cores options : - numBuckets to create a subdirectory based on a hash on the corename % numBuckets in the core Datadir, because all cores cannot live in the same directory - Auto with 3 differents values : 1) false : default behaviour 2) createLoad : create, if not exist, and load the core on the fly on the first incoming request (update, select). 3) onlyLoad : load the core on the fly on the first incoming request (update, select), if exist on disk The Auto option uses an additional cores option named baseDataDir to automatically generate the dataDir of each core and uses the numBucket option if exists. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5316) Solr 4.2.1 LotsOfCores new options
[ https://issues.apache.org/jira/browse/SOLR-5316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] olivier soyez updated SOLR-5316: Description: This patch is for the lotsofcores use case, including some modification : - by default, all cores have loadOnStartup=false and transient=true attributes - the create admin command can register a lazy core (to take into account the transientCacheSize option) - add transient cores persistency - handle unload admin command for never launched transient cores (non active cores) To improve performance, we use this Solr patched version with the persistence disabled. In this way, Solr is working with a solr.xml file without any core entries, because it's useless in our use case (with the new Auto option for the cores) The new Cores options : - numBuckets to create a subdirectory based on a hash on the corename % numBuckets in the core Datadir, because all cores cannot live in the same directory - Auto with 3 differents values : 1) false : default behaviour 2) createLoad : create, if not exist, and load the core on the fly on the first incoming request (update, select). 3) onlyLoad : load the core on the fly on the first incoming request (update, select), if exist on disk The Auto option uses an additional cores option named baseDataDir to automatically generate the dataDir of each core and uses the numBucket option if exists. was: This patch is for the lotsofcores use case, including some modification : - by default, all cores have loadOnStartup=false and transient=true attributes - the create admin command can register a lazy core (to take into account the transientCacheSize option) - add transient cores persistency - handle unload admin command for never launched transient cores (non active cores) The new Cores options : - numBuckets to create a subdirectory based on a hash on the corename % numBuckets in the core Datadir, because all cores cannot live in the same directory - Auto with 3 differents values : 1) false : default behaviour 2) createLoad : create, if not exist, and load the core on the fly on the first incoming request (update, select). 3) onlyLoad : load the core on the fly on the first incoming request (update, select), if exist on disk The Auto option uses an additional cores option named baseDataDir to automatically generate the dataDir of each core and uses the numBucket option if exists. Solr 4.2.1 LotsOfCores new options -- Key: SOLR-5316 URL: https://issues.apache.org/jira/browse/SOLR-5316 Project: Solr Issue Type: Improvement Components: multicore Affects Versions: 4.2.1 Reporter: olivier soyez Priority: Minor Labels: patch Fix For: 4.2.1 Attachments: solr-4.2.1.patchLOTSOFCORES This patch is for the lotsofcores use case, including some modification : - by default, all cores have loadOnStartup=false and transient=true attributes - the create admin command can register a lazy core (to take into account the transientCacheSize option) - add transient cores persistency - handle unload admin command for never launched transient cores (non active cores) To improve performance, we use this Solr patched version with the persistence disabled. In this way, Solr is working with a solr.xml file without any core entries, because it's useless in our use case (with the new Auto option for the cores) The new Cores options : - numBuckets to create a subdirectory based on a hash on the corename % numBuckets in the core Datadir, because all cores cannot live in the same directory - Auto with 3 differents values : 1) false : default behaviour 2) createLoad : create, if not exist, and load the core on the fly on the first incoming request (update, select). 3) onlyLoad : load the core on the fly on the first incoming request (update, select), if exist on disk The Auto option uses an additional cores option named baseDataDir to automatically generate the dataDir of each core and uses the numBucket option if exists. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
