Re: Multiple indexes/cores (aka solr-215) functional value?
This is precisely what I want to do. Yes, I can add JNDI entries to various Jetty XML files, but this is good only if you have a fixed set of indices known ahead of time (before starting the servlet container). I want the ability to add and remove indices on the fly, while the servlet container with Solr is running. This is where SOLR-215 comes in. Henri, hang in there. :) Otis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Simpy -- http://www.simpy.com/ - Tag - Search - Share - Original Message From: Yonik Seeley <[EMAIL PROTECTED]> To: solr-dev@lucene.apache.org Sent: Wednesday, June 27, 2007 5:00:08 PM Subject: Re: Multiple indexes/cores (aka solr-215) functional value? On 6/27/07, Henrib <[EMAIL PROTECTED]> wrote: > This http://www.nabble.com/multiple-indices-tf3982573.html thread triggers > the question again. > Solr-215 makes it easier to deploy multiple indexes than using multiple web > applications; but is "easier" enough for not being just a superfluous > feature? With a fixed handful of indicies, IMO, no. Though if one needs to programmatically add new indicies/schemas, SOLR-215 becomes interesting. I don't know how common of a case that is though. There are probably other use cases I've not considered. SOLR-215 does seem unrelated to distributed search though. -Yonik
Re: Multiple indexes/cores (aka solr-215) functional value?
: At this point, Solr-215 just looks like a solution to my problem... : My apologies for the clutter & waste of community resources. Please, please, PLEASE don't feel that way ... a functioning patch and/or a productive discussion is *NEVER* a waste of resources. I think Doug Cutting is the first person that told me the mantra all open source projects should live by "a patch in the bug queue that never gets commited is still a useful patch because it's available to people that want it." ... but commiting a patch (especially a big one) requires taking on a lot of responsibility to stand behind the APIs/functionality it includes and commit to a "best effort" to support them in a backwards compatible way for as long as feasible. On a personal note: Please keep in mind that just because i make suggestions to people on how to achieve multiple indexes / schemas / doctypes that doesn't require your patch doesn't mean i think there's anything wrong with your patch ... i'm just trying to make sure people are aware of what's posisble "out of the box".I think the appraoch you've advoated in teh jira issue description makes a lot of sense, and i kind of wish Solr had been implemented that way in the begining -- it just wasn't anticipated. the real question in my mind is: can we get from here to there cleanly, and is your patch the way to do it? i don't know the answer to that question yet. but please don't be discouraged. -Hoss
Re: Multiple indexes/cores (aka solr-215) functional value?
On 6/27/07, Henrib <[EMAIL PROTECTED]> wrote: At this point, Solr-215 just looks like a solution to my problem... My apologies for the clutter & waste of community resources. Not at all... I don't mean to discourage you. I'm still on the fence, and it's a large patch to swallow and figure out all the implications of. Of course, the nature of a volunteer community means that a good patch can still sit unused for quite until it happens to intersect with the priorities of a committer. -Yonik
Re: Multiple indexes/cores (aka solr-215) functional value?
The starting point for Solr-215 was this http://www.nabble.com/Embedding-Solr-vs-Lucene%2C-multiple-Solr-cores--tf3572324.html thread in April. I'll still say that some of our customers IT are making deployment of multiple webapps something more complex than it should...but they do! IT is in fact managing an internal hosting service and do not want to deal with the functional part of the applications. In these environments, a webapp that starts a thread & uses the file system is already a hard sell... In the same thread, I got the impression that removing the singletons would be useful anyway: hossman_lucene wrote: > >>That said: If anyone is interested in tackling a patch to eliminate all of >>the static Singletons i (and many others i suspect) would be >>extremely gratefull .. both for how much it would improve hte reusability >>of Solr in embedded situatiosn like this, but also for how it would >>(hopefully) make hte code eaier to follow for future developers. > More to your point, programmatic creation of indexes comes handy if you want to extend Solr with collection management (rather than dealing with them "externally" through scripts). You can easily add collections and/or languages and manage the collections definition/refresh/reindexing within the application; it seemed like a natural extension to the project. The main rationale being that since the Solr document/field format is most likely not the authoring format of the original "resource" (the authored source file), there is value in declaring how to transform a "resource" into an Solr-XML document from its collection/lang/suffix/etc. But this is beyond Solr-215. The same could be said for distributed search (aka solr-255) versus a 'Solr cluster management' application... At this point, Solr-215 just looks like a solution to my problem... My apologies for the clutter & waste of community resources. Yonik Seeley wrote: > > On 6/27/07, Henrib <[EMAIL PROTECTED]> wrote: >> This http://www.nabble.com/multiple-indices-tf3982573.html thread >> triggers >> the question again. >> Solr-215 makes it easier to deploy multiple indexes than using multiple >> web >> applications; but is "easier" enough for not being just a superfluous >> feature? > > With a fixed handful of indicies, IMO, no. > Though if one needs to programmatically add new indicies/schemas, > SOLR-215 becomes interesting. I don't know how common of a case that > is though. There are probably other use cases I've not considered. > > SOLR-215 does seem unrelated to distributed search though. > > -Yonik > > -- View this message in context: http://www.nabble.com/Multiple-indexes-cores-%28aka-solr-215%29-functional-value--tf3927076.html#a11327863 Sent from the Solr - Dev mailing list archive at Nabble.com.
Re: Multiple indexes/cores (aka solr-215) functional value?
On 6/27/07, Henrib <[EMAIL PROTECTED]> wrote: This http://www.nabble.com/multiple-indices-tf3982573.html thread triggers the question again. Solr-215 makes it easier to deploy multiple indexes than using multiple web applications; but is "easier" enough for not being just a superfluous feature? With a fixed handful of indicies, IMO, no. Though if one needs to programmatically add new indicies/schemas, SOLR-215 becomes interesting. I don't know how common of a case that is though. There are probably other use cases I've not considered. SOLR-215 does seem unrelated to distributed search though. -Yonik
Re: Multiple indexes/cores (aka solr-215) functional value?
This http://www.nabble.com/multiple-indices-tf3982573.html thread triggers the question again. Solr-215 makes it easier to deploy multiple indexes than using multiple web applications; but is "easier" enough for not being just a superfluous feature? Henrib wrote: > > > > The idea of the multiple core/indexes feature has been discussed in many > threads and it seems/seemed it has/had some functional value; how do we > ensure this value is generic enough for the issue (& patch) to ever be > solved? More importantly, is there an issue in the first place? > > For all use cases where it might make sense to use multiple indexes, the > common (as in community and commiters) wisdom is that one index is enough > and its schema will accommodate any functional need; from storing > documents that have no field in common to documents in different languages > or documents with varying publication workflows or lifetime, etc..., one > index (schema/config/core) - fits the functional bill. In very rare hard > cases, you might have to deploy multiple web applications. > > If having only one index is never a problem, the project has no need for a > patch that introduces multiple indexes. Thus, if the solr-215 issue is a > non-issue, what is the process to close it and mark it as 'invalid'? > It would be of great community value to state that thinking about using > multiple indexes is a mistake within the Solr project scope; use Lucene or > else if this is what you need. > > Is/should the single index best practice the sole one the Solr community > needs ? > I should have asked this earlier, still learning... > > -- View this message in context: http://www.nabble.com/Multiple-indexes-cores-%28aka-solr-215%29-functional-value--tf3927076.html#a11325717 Sent from the Solr - Dev mailing list archive at Nabble.com.
Re: Multiple indexes/cores (aka solr-215) functional value?
Here is a big +1 for the functionality that Henri added via SOLR-215! The last patch I looked was big-ish, but didn't look too difficult. The best way to avoid having to keep the patch up to date is to. commit it :) Otis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Simpy -- http://www.simpy.com/ - Tag - Search - Share - Original Message From: Yonik Seeley <[EMAIL PROTECTED]> To: solr-dev@lucene.apache.org Sent: Friday, June 15, 2007 5:11:18 PM Subject: Re: Multiple indexes/cores (aka solr-215) functional value? On 6/15/07, Henrib <[EMAIL PROTECTED]> wrote: > The idea of the multiple core/indexes feature has been discussed in many > threads and it seems/seemed it has/had some functional value; how do we > ensure this value is generic enough for the issue (& patch) to ever be > solved? More importantly, is there an issue in the first place? > > For all use cases where it might make sense to use multiple indexes, the > common (as in community and commiters) wisdom is that one index is enough I think that's the case mainly because it reflected the capabilities of solr at the time... so if someone asks how they create more than one index, the natural reaction has been to ask "do you really need to?" in the spirit of helping them quickly solve their problem with what was available. Of course, now there is your very ambitious SOLR-215, which I personally haven't had the time to look at, but I commend your great writeup on it - it really helps to have that type of detailed overview when attempting to evaluate such a large patch. -Yonik
Re: Multiple indexes/cores (aka solr-215) functional value?
That's pretty much how do it; it try an update and if it seems "very" broken, I just get a fresh copy of the trunk and apply the patch. Less than ideal since it's trial & error though but can't think of anything better. I'm might backport to 1.2 instead and continue work on 1.3 with less constraints, this would give everyone more time; if I were to do so, would it be better to base on 1.2.dev or 1.2.1 branch? Henrib hossman_lucene wrote: > > > : Do you have any recommendation to ease keeping the patch 'up to date' > with > : the trunk? > > One trick i've seen people talk about but haven't had much experience with > myself is that if a patch made by "svn diff" doesn't apply cleanly with > the truck because of subsequent changes, grep the patch for the svn > reviewion that the patch is based on, run 'svn update -r " to revert > your working copy to that verion, apply the patch, then run "svn update" > to get back to the head ... "they" say that sometimes hte incrimental > nature of "svn udpate" can rectify the changes better then patch (since > patch only knows what the current version looks like) > > i tried it once with a patch that didn't apply cleanly and it worked well, > (but one test is not a solid foundation to judge as a general rule) > > > -Hoss > > > -- View this message in context: http://www.nabble.com/Multiple-indexes-cores-%28aka-solr-215%29-functional-value--tf3927076.html#a11202451 Sent from the Solr - Dev mailing list archive at Nabble.com.
Re: Multiple indexes/cores (aka solr-215) functional value?
: Do you have any recommendation to ease keeping the patch 'up to date' with : the trunk? One trick i've seen people talk about but haven't had much experience with myself is that if a patch made by "svn diff" doesn't apply cleanly with the truck because of subsequent changes, grep the patch for the svn reviewion that the patch is based on, run 'svn update -r " to revert your working copy to that verion, apply the patch, then run "svn update" to get back to the head ... "they" say that sometimes hte incrimental nature of "svn udpate" can rectify the changes better then patch (since patch only knows what the current version looks like) i tried it once with a patch that didn't apply cleanly and it worked well, (but one test is not a solid foundation to judge as a general rule) -Hoss
Re: Multiple indexes/cores (aka solr-215) functional value?
It's a relief to know this is not out of the project scope. Do you have any recommendation to ease keeping the patch 'up to date' with the trunk? What else can/should I do to help the review? Yonik Seeley wrote: > > On 6/15/07, Henrib <[EMAIL PROTECTED]> wrote: >> The idea of the multiple core/indexes feature has been discussed in many >> threads and it seems/seemed it has/had some functional value; how do we >> ensure this value is generic enough for the issue (& patch) to ever be >> solved? More importantly, is there an issue in the first place? >> >> For all use cases where it might make sense to use multiple indexes, the >> common (as in community and commiters) wisdom is that one index is enough > > I think that's the case mainly because it reflected the capabilities > of solr at the time... so if someone asks how they create more than > one index, the natural reaction has been to ask "do you really need > to?" in the spirit of helping them quickly solve their problem with > what was available. > > Of course, now there is your very ambitious SOLR-215, which I > personally haven't had the time to look at, but I commend your great > writeup on it - it really helps to have that type of detailed overview > when attempting to evaluate such a large patch. > > -Yonik > > -- View this message in context: http://www.nabble.com/Multiple-indexes-cores-%28aka-solr-215%29-functional-value--tf3927076.html#a11178197 Sent from the Solr - Dev mailing list archive at Nabble.com.
Re: Multiple indexes/cores (aka solr-215) functional value?
On 6/15/07, Henrib <[EMAIL PROTECTED]> wrote: The idea of the multiple core/indexes feature has been discussed in many threads and it seems/seemed it has/had some functional value; how do we ensure this value is generic enough for the issue (& patch) to ever be solved? More importantly, is there an issue in the first place? For all use cases where it might make sense to use multiple indexes, the common (as in community and commiters) wisdom is that one index is enough I think that's the case mainly because it reflected the capabilities of solr at the time... so if someone asks how they create more than one index, the natural reaction has been to ask "do you really need to?" in the spirit of helping them quickly solve their problem with what was available. Of course, now there is your very ambitious SOLR-215, which I personally haven't had the time to look at, but I commend your great writeup on it - it really helps to have that type of detailed overview when attempting to evaluate such a large patch. -Yonik
Multiple indexes/cores (aka solr-215) functional value?
The idea of the multiple core/indexes feature has been discussed in many threads and it seems/seemed it has/had some functional value; how do we ensure this value is generic enough for the issue (& patch) to ever be solved? More importantly, is there an issue in the first place? For all use cases where it might make sense to use multiple indexes, the common (as in community and commiters) wisdom is that one index is enough and its schema will accommodate any functional need; from storing documents that have no field in common to documents in different languages or documents with varying publication workflows or lifetime, etc..., one index (schema/config/core) - fits the functional bill. In very rare hard cases, you might have to deploy multiple web applications. If having only one index is never a problem, the project has no need for a patch that introduces multiple indexes. Thus, if the solr-215 issue is a non-issue, what is the process to close it and mark it as 'invalid'? It would be of great community value to state that thinking about using multiple indexes is a mistake within the Solr project scope; use Lucene or else if this is what you need. Is/should the single index best practice the sole one the Solr community needs ? I should have asked this earlier, still learning... -- View this message in context: http://www.nabble.com/Multiple-indexes-cores-%28aka-solr-215%29-functional-value--tf3927076.html#a11137029 Sent from the Solr - Dev mailing list archive at Nabble.com.
