[jira] Commented: (SOLR-920) Cache and reuse IndexSchema
[ https://issues.apache.org/jira/browse/SOLR-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833858#action_12833858 ] Noble Paul commented on SOLR-920: - bq., then all the older versions hang around in the cache forever, right? nope. The cache key is the FQN of the file . in replication , the same file is overwritten. So the cache entry is overwritten > Cache and reuse IndexSchema > --- > > Key: SOLR-920 > URL: https://issues.apache.org/jira/browse/SOLR-920 > Project: Solr > Issue Type: Improvement >Reporter: Noble Paul >Assignee: Noble Paul > Attachments: SOLR-920.patch, SOLR-920.patch, SOLR-920.patch > > > if there are 1000's of cores then the cost of loading unloading schema.xml > can be prohibitive > similar to SOLR-919 we can also cache the DOM object of schema.xml if the > location on disk is same. All the dynamic properties can be replaced lazily > when they are read. > We can go one step ahead in this case. Th IndexSchema object is immutable . > So if there are no core properties then the same IndexSchema object can be > used across all the cores -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-920) Cache and reuse IndexSchema
[ https://issues.apache.org/jira/browse/SOLR-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833857#action_12833857 ] Noble Paul commented on SOLR-920: - yes . it is possible > Cache and reuse IndexSchema > --- > > Key: SOLR-920 > URL: https://issues.apache.org/jira/browse/SOLR-920 > Project: Solr > Issue Type: Improvement >Reporter: Noble Paul >Assignee: Noble Paul > Attachments: SOLR-920.patch, SOLR-920.patch, SOLR-920.patch > > > if there are 1000's of cores then the cost of loading unloading schema.xml > can be prohibitive > similar to SOLR-919 we can also cache the DOM object of schema.xml if the > location on disk is same. All the dynamic properties can be replaced lazily > when they are read. > We can go one step ahead in this case. Th IndexSchema object is immutable . > So if there are no core properties then the same IndexSchema object can be > used across all the cores -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-920) Cache and reuse IndexSchema
[ https://issues.apache.org/jira/browse/SOLR-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833814#action_12833814 ] Drew Morris commented on SOLR-920: -- Just reviewing the syntax of the XML... if you have 2 different schemas that should be shared with multiple cores... will this still work? So if I had 1000 cores sharing 1 schema and 1000 cores sharing a different schema will that be possible with this implementation? > Cache and reuse IndexSchema > --- > > Key: SOLR-920 > URL: https://issues.apache.org/jira/browse/SOLR-920 > Project: Solr > Issue Type: Improvement >Reporter: Noble Paul >Assignee: Noble Paul > Attachments: SOLR-920.patch, SOLR-920.patch, SOLR-920.patch > > > if there are 1000's of cores then the cost of loading unloading schema.xml > can be prohibitive > similar to SOLR-919 we can also cache the DOM object of schema.xml if the > location on disk is same. All the dynamic properties can be replaced lazily > when they are read. > We can go one step ahead in this case. Th IndexSchema object is immutable . > So if there are no core properties then the same IndexSchema object can be > used across all the cores -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-920) Cache and reuse IndexSchema
[ https://issues.apache.org/jira/browse/SOLR-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12764556#action_12764556 ] Noble Paul commented on SOLR-920: - bq.seems like it's an unbounded cache? yes. It is unbounded. Let us make it bounded > Cache and reuse IndexSchema > --- > > Key: SOLR-920 > URL: https://issues.apache.org/jira/browse/SOLR-920 > Project: Solr > Issue Type: Improvement >Reporter: Noble Paul >Assignee: Noble Paul > Attachments: SOLR-920.patch, SOLR-920.patch, SOLR-920.patch > > > if there are 1000's of cores then the cost of loading unloading schema.xml > can be prohibitive > similar to SOLR-919 we can also cache the DOM object of schema.xml if the > location on disk is same. All the dynamic properties can be replaced lazily > when they are read. > We can go one step ahead in this case. Th IndexSchema object is immutable . > So if there are no core properties then the same IndexSchema object can be > used across all the cores -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-920) Cache and reuse IndexSchema
[ https://issues.apache.org/jira/browse/SOLR-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12764350#action_12764350 ] Yonik Seeley commented on SOLR-920: --- Just got around to reviewing this seems like it's an unbounded cache? If someone kept modifying the schema and reloading the core (as can happen with replication too), then all the older versions hang around in the cache forever, right? > Cache and reuse IndexSchema > --- > > Key: SOLR-920 > URL: https://issues.apache.org/jira/browse/SOLR-920 > Project: Solr > Issue Type: Improvement >Reporter: Noble Paul >Assignee: Noble Paul > Attachments: SOLR-920.patch, SOLR-920.patch, SOLR-920.patch > > > if there are 1000's of cores then the cost of loading unloading schema.xml > can be prohibitive > similar to SOLR-919 we can also cache the DOM object of schema.xml if the > location on disk is same. All the dynamic properties can be replaced lazily > when they are read. > We can go one step ahead in this case. Th IndexSchema object is immutable . > So if there are no core properties then the same IndexSchema object can be > used across all the cores -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-920) Cache and reuse IndexSchema
[ https://issues.apache.org/jira/browse/SOLR-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729955#action_12729955 ] Shalin Shekhar Mangar commented on SOLR-920: I think this change is not mentioned in the CHANGES.txt? Also, the fix version should be marked as 1.4. > Cache and reuse IndexSchema > --- > > Key: SOLR-920 > URL: https://issues.apache.org/jira/browse/SOLR-920 > Project: Solr > Issue Type: Improvement >Reporter: Noble Paul >Assignee: Noble Paul > Attachments: SOLR-920.patch, SOLR-920.patch, SOLR-920.patch > > > if there are 1000's of cores then the cost of loading unloading schema.xml > can be prohibitive > similar to SOLR-919 we can also cache the DOM object of schema.xml if the > location on disk is same. All the dynamic properties can be replaced lazily > when they are read. > We can go one step ahead in this case. Th IndexSchema object is immutable . > So if there are no core properties then the same IndexSchema object can be > used across all the cores -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-920) Cache and reuse IndexSchema
[ https://issues.apache.org/jira/browse/SOLR-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718299#action_12718299 ] Noble Paul commented on SOLR-920: - committed :r783631 > Cache and reuse IndexSchema > --- > > Key: SOLR-920 > URL: https://issues.apache.org/jira/browse/SOLR-920 > Project: Solr > Issue Type: Improvement >Reporter: Noble Paul >Assignee: Noble Paul > Attachments: SOLR-920.patch, SOLR-920.patch, SOLR-920.patch > > > if there are 1000's of cores then the cost of loading unloading schema.xml > can be prohibitive > similar to SOLR-919 we can also cache the DOM object of schema.xml if the > location on disk is same. All the dynamic properties can be replaced lazily > when they are read. > We can go one step ahead in this case. Th IndexSchema object is immutable . > So if there are no core properties then the same IndexSchema object can be > used across all the cores -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-920) Cache and reuse IndexSchema
[ https://issues.apache.org/jira/browse/SOLR-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712652#action_12712652 ] Noble Paul commented on SOLR-920: - I plan to commit this in a day or two > Cache and reuse IndexSchema > --- > > Key: SOLR-920 > URL: https://issues.apache.org/jira/browse/SOLR-920 > Project: Solr > Issue Type: Improvement >Reporter: Noble Paul >Assignee: Noble Paul > Attachments: SOLR-920.patch > > > if there are 1000's of cores then the cost of loading unloading schema.xml > can be prohibitive > similar to SOLR-919 we can also cache the DOM object of schema.xml if the > location on disk is same. All the dynamic properties can be replaced lazily > when they are read. > We can go one step ahead in this case. Th IndexSchema object is immutable . > So if there are no core properties then the same IndexSchema object can be > used across all the cores -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-920) Cache and reuse IndexSchema
[ https://issues.apache.org/jira/browse/SOLR-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712349#action_12712349 ] Noble Paul commented on SOLR-920: - bq.So if my core has its own schema.xml in the right place (in conf/schema.xml), that schema will be used, not the shard one? right. one instance per (file+version). > Cache and reuse IndexSchema > --- > > Key: SOLR-920 > URL: https://issues.apache.org/jira/browse/SOLR-920 > Project: Solr > Issue Type: Improvement >Reporter: Noble Paul >Assignee: Noble Paul > Attachments: SOLR-920.patch > > > if there are 1000's of cores then the cost of loading unloading schema.xml > can be prohibitive > similar to SOLR-919 we can also cache the DOM object of schema.xml if the > location on disk is same. All the dynamic properties can be replaced lazily > when they are read. > We can go one step ahead in this case. Th IndexSchema object is immutable . > So if there are no core properties then the same IndexSchema object can be > used across all the cores -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-920) Cache and reuse IndexSchema
[ https://issues.apache.org/jira/browse/SOLR-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712113#action_12712113 ] Otis Gospodnetic commented on SOLR-920: --- So if my core has its own schema.xml in the right place (in conf/schema.xml), that schema will be used, not the shard one? > Cache and reuse IndexSchema > --- > > Key: SOLR-920 > URL: https://issues.apache.org/jira/browse/SOLR-920 > Project: Solr > Issue Type: Improvement >Reporter: Noble Paul >Assignee: Noble Paul > Attachments: SOLR-920.patch > > > if there are 1000's of cores then the cost of loading unloading schema.xml > can be prohibitive > similar to SOLR-919 we can also cache the DOM object of schema.xml if the > location on disk is same. All the dynamic properties can be replaced lazily > when they are read. > We can go one step ahead in this case. Th IndexSchema object is immutable . > So if there are no core properties then the same IndexSchema object can be > used across all the cores -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-920) Cache and reuse IndexSchema
[
https://issues.apache.org/jira/browse/SOLR-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711941#action_12711941
]
Noble Paul commented on SOLR-920:
-
bq.What happens when a core has a copy of schema.xml in its conf/ dir and that
schema.xml is potentially different from the shared one?
for a given path on disk one instance of IndexSchema is maintained in memory.
The cache looks as follows .
{code:java}
// in CoreContainer.java.
//The key is the absolute path to the schema.xml. The value is the IndexSchema
instance.
private Map schemaCache;
{code}
It is also possible to maintain a timestamp of the file also in the key (say
/data/solr/confschema.xml:MMddhhmmss)so that if the file is modified a new
instance of IndexScema can be created
> Cache and reuse IndexSchema
> ---
>
> Key: SOLR-920
> URL: https://issues.apache.org/jira/browse/SOLR-920
> Project: Solr
> Issue Type: Improvement
>Reporter: Noble Paul
>Assignee: Noble Paul
>
> if there are 1000's of cores then the cost of loading unloading schema.xml
> can be prohibitive
> similar to SOLR-919 we can also cache the DOM object of schema.xml if the
> location on disk is same. All the dynamic properties can be replaced lazily
> when they are read.
> We can go one step ahead in this case. Th IndexSchema object is immutable .
> So if there are no core properties then the same IndexSchema object can be
> used across all the cores
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-920) Cache and reuse IndexSchema
[ https://issues.apache.org/jira/browse/SOLR-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711856#action_12711856 ] Otis Gospodnetic commented on SOLR-920: --- Looks good to me. What happens when a core has a copy of schema.xml in its conf/ dir and that schema.xml is potentially different from the shared one? > Cache and reuse IndexSchema > --- > > Key: SOLR-920 > URL: https://issues.apache.org/jira/browse/SOLR-920 > Project: Solr > Issue Type: Improvement >Reporter: Noble Paul >Assignee: Noble Paul > > if there are 1000's of cores then the cost of loading unloading schema.xml > can be prohibitive > similar to SOLR-919 we can also cache the DOM object of schema.xml if the > location on disk is same. All the dynamic properties can be replaced lazily > when they are read. > We can go one step ahead in this case. Th IndexSchema object is immutable . > So if there are no core properties then the same IndexSchema object can be > used across all the cores -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-920) Cache and reuse IndexSchema
[
https://issues.apache.org/jira/browse/SOLR-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710625#action_12710625
]
Noble Paul commented on SOLR-920:
-
how about an extra attribute in the cores tag to enable sharing as follows.
{code:xml}
{code}
if shareSchema is set to true then for a given path only one instance of
IndexSchema is created and it will be shared . It is the responsibilty of the
user to ensure that he does not use any core specific properties in the
schema.xml when this attribute is set to true
> Cache and reuse IndexSchema
> ---
>
> Key: SOLR-920
> URL: https://issues.apache.org/jira/browse/SOLR-920
> Project: Solr
> Issue Type: Improvement
>Reporter: Noble Paul
>
> if there are 1000's of cores then the cost of loading unloading schema.xml
> can be prohibitive
> similar to SOLR-919 we can also cache the DOM object of schema.xml if the
> location on disk is same. All the dynamic properties can be replaced lazily
> when they are read.
> We can go one step ahead in this case. Th IndexSchema object is immutable .
> So if there are no core properties then the same IndexSchema object can be
> used across all the cores
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
