Sharing index amongst multiple nodes
Hi. Wat are the thoughts on having multiple SOLR instances i.e. multiple SOLR war files, sharing the same index (i.e. sharing the same solr_home) where only one SOLR instance is used for writing and the others for reading? Is this possible? Is it beneficial - is it more performant than having just one solr instance? How does it affect auto-commits i.e. how would the read nodes know the index has been changed and re-populate cache etc.? Sole 3.6.1 Thanks.
Re: Sharing index amongst multiple nodes
I don't understand why this would be more performant.. seems like it'd be more memory and resource intensive as you'd have multiple class-loaders and multiple cache spaces for no good reason. Just have a single core with sufficiently large caches to handle your response needs. If you want to load balance reads consider having multiple physical nodes with a master/slaves or SolrCloud. On Sat, Apr 6, 2013 at 9:21 AM, Daire Mac Mathúna daire...@gmail.comwrote: Hi. Wat are the thoughts on having multiple SOLR instances i.e. multiple SOLR war files, sharing the same index (i.e. sharing the same solr_home) where only one SOLR instance is used for writing and the others for reading? Is this possible? Is it beneficial - is it more performant than having just one solr instance? How does it affect auto-commits i.e. how would the read nodes know the index has been changed and re-populate cache etc.? Sole 3.6.1 Thanks.
Re: Sharing index amongst multiple nodes
Hi Daire Mac Mathúna; If there is a way copying one Solr's indexes into another Solr instance, this may also solve the problem. Somebody generates indexes and some of other instances could get a copy of them. At synchronizing process you may eliminate some of indexes at reader instance. So you can filter something to become unsearchable. *This may not be efficient and good thing and maybe solved with built-in functionality somehow.* However I think somebody may need that mechanism. 2013/4/6 Amit Nithian anith...@gmail.com I don't understand why this would be more performant.. seems like it'd be more memory and resource intensive as you'd have multiple class-loaders and multiple cache spaces for no good reason. Just have a single core with sufficiently large caches to handle your response needs. If you want to load balance reads consider having multiple physical nodes with a master/slaves or SolrCloud. On Sat, Apr 6, 2013 at 9:21 AM, Daire Mac Mathúna daire...@gmail.com wrote: Hi. Wat are the thoughts on having multiple SOLR instances i.e. multiple SOLR war files, sharing the same index (i.e. sharing the same solr_home) where only one SOLR instance is used for writing and the others for reading? Is this possible? Is it beneficial - is it more performant than having just one solr instance? How does it affect auto-commits i.e. how would the read nodes know the index has been changed and re-populate cache etc.? Sole 3.6.1 Thanks.
Re: Sharing index amongst multiple nodes
This is precisely how Solr replication works. It copies the indexes then does a commit. wunder On Apr 6, 2013, at 2:40 PM, Furkan KAMACI wrote: Hi Daire Mac Mathúna; If there is a way copying one Solr's indexes into another Solr instance, this may also solve the problem. Somebody generates indexes and some of other instances could get a copy of them. At synchronizing process you may eliminate some of indexes at reader instance. So you can filter something to become unsearchable. *This may not be efficient and good thing and maybe solved with built-in functionality somehow.* However I think somebody may need that mechanism. 2013/4/6 Amit Nithian anith...@gmail.com I don't understand why this would be more performant.. seems like it'd be more memory and resource intensive as you'd have multiple class-loaders and multiple cache spaces for no good reason. Just have a single core with sufficiently large caches to handle your response needs. If you want to load balance reads consider having multiple physical nodes with a master/slaves or SolrCloud. On Sat, Apr 6, 2013 at 9:21 AM, Daire Mac Mathúna daire...@gmail.com wrote: Hi. Wat are the thoughts on having multiple SOLR instances i.e. multiple SOLR war files, sharing the same index (i.e. sharing the same solr_home) where only one SOLR instance is used for writing and the others for reading? Is this possible? Is it beneficial - is it more performant than having just one solr instance? How does it affect auto-commits i.e. how would the read nodes know the index has been changed and re-populate cache etc.? Sole 3.6.1 Thanks. -- Walter Underwood wun...@wunderwood.org
Re: Sharing index amongst multiple nodes
Hi Walter; I am new to Solr and digging into code to understand it. I think that when indexer copies indexes, before the commit it is unsearchable. Where exactly that commit occurs at code and can I say that: rollback something because I don't want that indexes (reason maybe anything else, maybe I will decline some indexes(index filtering) because of the documents they points. Is it possible? 2013/4/7 Walter Underwood wun...@wunderwood.org This is precisely how Solr replication works. It copies the indexes then does a commit. wunder On Apr 6, 2013, at 2:40 PM, Furkan KAMACI wrote: Hi Daire Mac Mathúna; If there is a way copying one Solr's indexes into another Solr instance, this may also solve the problem. Somebody generates indexes and some of other instances could get a copy of them. At synchronizing process you may eliminate some of indexes at reader instance. So you can filter something to become unsearchable. *This may not be efficient and good thing and maybe solved with built-in functionality somehow.* However I think somebody may need that mechanism. 2013/4/6 Amit Nithian anith...@gmail.com I don't understand why this would be more performant.. seems like it'd be more memory and resource intensive as you'd have multiple class-loaders and multiple cache spaces for no good reason. Just have a single core with sufficiently large caches to handle your response needs. If you want to load balance reads consider having multiple physical nodes with a master/slaves or SolrCloud. On Sat, Apr 6, 2013 at 9:21 AM, Daire Mac Mathúna daire...@gmail.com wrote: Hi. Wat are the thoughts on having multiple SOLR instances i.e. multiple SOLR war files, sharing the same index (i.e. sharing the same solr_home) where only one SOLR instance is used for writing and the others for reading? Is this possible? Is it beneficial - is it more performant than having just one solr instance? How does it affect auto-commits i.e. how would the read nodes know the index has been changed and re-populate cache etc.? Sole 3.6.1 Thanks. -- Walter Underwood wun...@wunderwood.org
Re: Sharing index amongst multiple nodes
Indexing happens on one Solr server. After a commit, the documents are searchable. In Solr 4, there is a soft commit, which makes the documents searchable, but does not create on-disk indexes. Solr replication copies the committed indexes to another Solr server. Solr Cloud uses a transaction log to make documents available before a hard commit. Solr does not have rollback. A commit succeeds or fails. After it succeeds, there is no going back. wunder On Apr 6, 2013, at 3:08 PM, Furkan KAMACI wrote: Hi Walter; I am new to Solr and digging into code to understand it. I think that when indexer copies indexes, before the commit it is unsearchable. Where exactly that commit occurs at code and can I say that: rollback something because I don't want that indexes (reason maybe anything else, maybe I will decline some indexes(index filtering) because of the documents they points. Is it possible? 2013/4/7 Walter Underwood wun...@wunderwood.org This is precisely how Solr replication works. It copies the indexes then does a commit. wunder On Apr 6, 2013, at 2:40 PM, Furkan KAMACI wrote: Hi Daire Mac Mathúna; If there is a way copying one Solr's indexes into another Solr instance, this may also solve the problem. Somebody generates indexes and some of other instances could get a copy of them. At synchronizing process you may eliminate some of indexes at reader instance. So you can filter something to become unsearchable. *This may not be efficient and good thing and maybe solved with built-in functionality somehow.* However I think somebody may need that mechanism. 2013/4/6 Amit Nithian anith...@gmail.com I don't understand why this would be more performant.. seems like it'd be more memory and resource intensive as you'd have multiple class-loaders and multiple cache spaces for no good reason. Just have a single core with sufficiently large caches to handle your response needs. If you want to load balance reads consider having multiple physical nodes with a master/slaves or SolrCloud. On Sat, Apr 6, 2013 at 9:21 AM, Daire Mac Mathúna daire...@gmail.com wrote: Hi. Wat are the thoughts on having multiple SOLR instances i.e. multiple SOLR war files, sharing the same index (i.e. sharing the same solr_home) where only one SOLR instance is used for writing and the others for reading? Is this possible? Is it beneficial - is it more performant than having just one solr instance? How does it affect auto-commits i.e. how would the read nodes know the index has been changed and re-populate cache etc.? Sole 3.6.1 Thanks. -- Walter Underwood wun...@wunderwood.org -- Walter Underwood wun...@wunderwood.org
Re: Sharing index amongst multiple nodes
Hi Walter; Thanks for your explanation. You said Indexing happens on one Solr server. Is it true even for SolrCloud? 2013/4/7 Walter Underwood wun...@wunderwood.org Indexing happens on one Solr server. After a commit, the documents are searchable. In Solr 4, there is a soft commit, which makes the documents searchable, but does not create on-disk indexes. Solr replication copies the committed indexes to another Solr server. Solr Cloud uses a transaction log to make documents available before a hard commit. Solr does not have rollback. A commit succeeds or fails. After it succeeds, there is no going back. wunder On Apr 6, 2013, at 3:08 PM, Furkan KAMACI wrote: Hi Walter; I am new to Solr and digging into code to understand it. I think that when indexer copies indexes, before the commit it is unsearchable. Where exactly that commit occurs at code and can I say that: rollback something because I don't want that indexes (reason maybe anything else, maybe I will decline some indexes(index filtering) because of the documents they points. Is it possible? 2013/4/7 Walter Underwood wun...@wunderwood.org This is precisely how Solr replication works. It copies the indexes then does a commit. wunder On Apr 6, 2013, at 2:40 PM, Furkan KAMACI wrote: Hi Daire Mac Mathúna; If there is a way copying one Solr's indexes into another Solr instance, this may also solve the problem. Somebody generates indexes and some of other instances could get a copy of them. At synchronizing process you may eliminate some of indexes at reader instance. So you can filter something to become unsearchable. *This may not be efficient and good thing and maybe solved with built-in functionality somehow.* However I think somebody may need that mechanism. 2013/4/6 Amit Nithian anith...@gmail.com I don't understand why this would be more performant.. seems like it'd be more memory and resource intensive as you'd have multiple class-loaders and multiple cache spaces for no good reason. Just have a single core with sufficiently large caches to handle your response needs. If you want to load balance reads consider having multiple physical nodes with a master/slaves or SolrCloud. On Sat, Apr 6, 2013 at 9:21 AM, Daire Mac Mathúna daire...@gmail.com wrote: Hi. Wat are the thoughts on having multiple SOLR instances i.e. multiple SOLR war files, sharing the same index (i.e. sharing the same solr_home) where only one SOLR instance is used for writing and the others for reading? Is this possible? Is it beneficial - is it more performant than having just one solr instance? How does it affect auto-commits i.e. how would the read nodes know the index has been changed and re-populate cache etc.? Sole 3.6.1 Thanks. -- Walter Underwood wun...@wunderwood.org -- Walter Underwood wun...@wunderwood.org
Re: Sharing index amongst multiple nodes
In Solr Cloud, a document is indexed on the shard leader. The replicas in that shard get the document and add it to their indexes. There is some indexing that happens on the replicas, but that is managed by Solr. wunder On Apr 6, 2013, at 3:58 PM, Furkan KAMACI wrote: Hi Walter; Thanks for your explanation. You said Indexing happens on one Solr server. Is it true even for SolrCloud? 2013/4/7 Walter Underwood wun...@wunderwood.org Indexing happens on one Solr server. After a commit, the documents are searchable. In Solr 4, there is a soft commit, which makes the documents searchable, but does not create on-disk indexes. Solr replication copies the committed indexes to another Solr server. Solr Cloud uses a transaction log to make documents available before a hard commit. Solr does not have rollback. A commit succeeds or fails. After it succeeds, there is no going back. wunder On Apr 6, 2013, at 3:08 PM, Furkan KAMACI wrote: Hi Walter; I am new to Solr and digging into code to understand it. I think that when indexer copies indexes, before the commit it is unsearchable. Where exactly that commit occurs at code and can I say that: rollback something because I don't want that indexes (reason maybe anything else, maybe I will decline some indexes(index filtering) because of the documents they points. Is it possible? 2013/4/7 Walter Underwood wun...@wunderwood.org This is precisely how Solr replication works. It copies the indexes then does a commit. wunder On Apr 6, 2013, at 2:40 PM, Furkan KAMACI wrote: Hi Daire Mac Mathúna; If there is a way copying one Solr's indexes into another Solr instance, this may also solve the problem. Somebody generates indexes and some of other instances could get a copy of them. At synchronizing process you may eliminate some of indexes at reader instance. So you can filter something to become unsearchable. *This may not be efficient and good thing and maybe solved with built-in functionality somehow.* However I think somebody may need that mechanism. 2013/4/6 Amit Nithian anith...@gmail.com I don't understand why this would be more performant.. seems like it'd be more memory and resource intensive as you'd have multiple class-loaders and multiple cache spaces for no good reason. Just have a single core with sufficiently large caches to handle your response needs. If you want to load balance reads consider having multiple physical nodes with a master/slaves or SolrCloud. On Sat, Apr 6, 2013 at 9:21 AM, Daire Mac Mathúna daire...@gmail.com wrote: Hi. Wat are the thoughts on having multiple SOLR instances i.e. multiple SOLR war files, sharing the same index (i.e. sharing the same solr_home) where only one SOLR instance is used for writing and the others for reading? Is this possible? Is it beneficial - is it more performant than having just one solr instance? How does it affect auto-commits i.e. how would the read nodes know the index has been changed and re-populate cache etc.? Sole 3.6.1 Thanks. -- Walter Underwood wun...@wunderwood.org -- Walter Underwood wun...@wunderwood.org -- Walter Underwood wun...@wunderwood.org
Re: Sharing index amongst multiple nodes
My last questions. 1) If I sent document to a replica does it pass document to shard leader and do you mean that even if I send document to shard leader does it can pass that document one of replicas to be indexed. 2) Does it possible to copy a shard into another shard, or merge them? By the way thanks for your explanations. 2013/4/7 Walter Underwood wun...@wunderwood.org In Solr Cloud, a document is indexed on the shard leader. The replicas in that shard get the document and add it to their indexes. There is some indexing that happens on the replicas, but that is managed by Solr. wunder On Apr 6, 2013, at 3:58 PM, Furkan KAMACI wrote: Hi Walter; Thanks for your explanation. You said Indexing happens on one Solr server. Is it true even for SolrCloud? 2013/4/7 Walter Underwood wun...@wunderwood.org Indexing happens on one Solr server. After a commit, the documents are searchable. In Solr 4, there is a soft commit, which makes the documents searchable, but does not create on-disk indexes. Solr replication copies the committed indexes to another Solr server. Solr Cloud uses a transaction log to make documents available before a hard commit. Solr does not have rollback. A commit succeeds or fails. After it succeeds, there is no going back. wunder On Apr 6, 2013, at 3:08 PM, Furkan KAMACI wrote: Hi Walter; I am new to Solr and digging into code to understand it. I think that when indexer copies indexes, before the commit it is unsearchable. Where exactly that commit occurs at code and can I say that: rollback something because I don't want that indexes (reason maybe anything else, maybe I will decline some indexes(index filtering) because of the documents they points. Is it possible? 2013/4/7 Walter Underwood wun...@wunderwood.org This is precisely how Solr replication works. It copies the indexes then does a commit. wunder On Apr 6, 2013, at 2:40 PM, Furkan KAMACI wrote: Hi Daire Mac Mathúna; If there is a way copying one Solr's indexes into another Solr instance, this may also solve the problem. Somebody generates indexes and some of other instances could get a copy of them. At synchronizing process you may eliminate some of indexes at reader instance. So you can filter something to become unsearchable. *This may not be efficient and good thing and maybe solved with built-in functionality somehow.* However I think somebody may need that mechanism. 2013/4/6 Amit Nithian anith...@gmail.com I don't understand why this would be more performant.. seems like it'd be more memory and resource intensive as you'd have multiple class-loaders and multiple cache spaces for no good reason. Just have a single core with sufficiently large caches to handle your response needs. If you want to load balance reads consider having multiple physical nodes with a master/slaves or SolrCloud. On Sat, Apr 6, 2013 at 9:21 AM, Daire Mac Mathúna daire...@gmail.com wrote: Hi. Wat are the thoughts on having multiple SOLR instances i.e. multiple SOLR war files, sharing the same index (i.e. sharing the same solr_home) where only one SOLR instance is used for writing and the others for reading? Is this possible? Is it beneficial - is it more performant than having just one solr instance? How does it affect auto-commits i.e. how would the read nodes know the index has been changed and re-populate cache etc.? Sole 3.6.1 Thanks. -- Walter Underwood wun...@wunderwood.org -- Walter Underwood wun...@wunderwood.org -- Walter Underwood wun...@wunderwood.org
Re: Sharing index amongst multiple nodes
A document sent to any Solr Cloud node will be sent to the right place. Shard merging and splitting is not supported now. There is work on shard splitting: https://issues.apache.org/jira/browse/SOLR-3755 wunder On Apr 6, 2013, at 4:15 PM, Furkan KAMACI wrote: My last questions. 1) If I sent document to a replica does it pass document to shard leader and do you mean that even if I send document to shard leader does it can pass that document one of replicas to be indexed. 2) Does it possible to copy a shard into another shard, or merge them? By the way thanks for your explanations. 2013/4/7 Walter Underwood wun...@wunderwood.org In Solr Cloud, a document is indexed on the shard leader. The replicas in that shard get the document and add it to their indexes. There is some indexing that happens on the replicas, but that is managed by Solr. wunder On Apr 6, 2013, at 3:58 PM, Furkan KAMACI wrote: Hi Walter; Thanks for your explanation. You said Indexing happens on one Solr server. Is it true even for SolrCloud? 2013/4/7 Walter Underwood wun...@wunderwood.org Indexing happens on one Solr server. After a commit, the documents are searchable. In Solr 4, there is a soft commit, which makes the documents searchable, but does not create on-disk indexes. Solr replication copies the committed indexes to another Solr server. Solr Cloud uses a transaction log to make documents available before a hard commit. Solr does not have rollback. A commit succeeds or fails. After it succeeds, there is no going back. wunder On Apr 6, 2013, at 3:08 PM, Furkan KAMACI wrote: Hi Walter; I am new to Solr and digging into code to understand it. I think that when indexer copies indexes, before the commit it is unsearchable. Where exactly that commit occurs at code and can I say that: rollback something because I don't want that indexes (reason maybe anything else, maybe I will decline some indexes(index filtering) because of the documents they points. Is it possible? 2013/4/7 Walter Underwood wun...@wunderwood.org This is precisely how Solr replication works. It copies the indexes then does a commit. wunder On Apr 6, 2013, at 2:40 PM, Furkan KAMACI wrote: Hi Daire Mac Mathúna; If there is a way copying one Solr's indexes into another Solr instance, this may also solve the problem. Somebody generates indexes and some of other instances could get a copy of them. At synchronizing process you may eliminate some of indexes at reader instance. So you can filter something to become unsearchable. *This may not be efficient and good thing and maybe solved with built-in functionality somehow.* However I think somebody may need that mechanism. 2013/4/6 Amit Nithian anith...@gmail.com I don't understand why this would be more performant.. seems like it'd be more memory and resource intensive as you'd have multiple class-loaders and multiple cache spaces for no good reason. Just have a single core with sufficiently large caches to handle your response needs. If you want to load balance reads consider having multiple physical nodes with a master/slaves or SolrCloud. On Sat, Apr 6, 2013 at 9:21 AM, Daire Mac Mathúna daire...@gmail.com wrote: Hi. Wat are the thoughts on having multiple SOLR instances i.e. multiple SOLR war files, sharing the same index (i.e. sharing the same solr_home) where only one SOLR instance is used for writing and the others for reading? Is this possible? Is it beneficial - is it more performant than having just one solr instance? How does it affect auto-commits i.e. how would the read nodes know the index has been changed and re-populate cache etc.? Sole 3.6.1 Thanks. -- Walter Underwood wun...@wunderwood.org -- Walter Underwood wun...@wunderwood.org -- Walter Underwood wun...@wunderwood.org -- Walter Underwood wun...@wunderwood.org
