[jira] [Commented] (SOLR-2698) Enhance CoreAdmin STATUS command to return index size
[ https://issues.apache.org/jira/browse/SOLR-2698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240498#comment-13240498 ] Jason Rutherglen commented on SOLR-2698: +1 This'd be useful. > Enhance CoreAdmin STATUS command to return index size > - > > Key: SOLR-2698 > URL: https://issues.apache.org/jira/browse/SOLR-2698 > Project: Solr > Issue Type: Improvement > Components: multicore >Affects Versions: 4.0 >Reporter: Yury Kats >Assignee: Mark Miller > Fix For: 4.0 > > Attachments: SOLR-2698.patch, SOLR-2698.patch > > > CoreAdmin STATUS command returns all kinds of index info for all cores on the > server, except for the index size. > However, indexSize can be retrieved for an individual core via a > /replication&command=details request. > I have N Solrs servers, running M cores each. My application is monitoring > the status of all cores, including their index size. > As it stands today, I need to issue N status requests plus N*M replication > requests to get all the information I need. > If STATUS command returned indexSize, number of requests would be just N. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3221) Make Shard handler threadpool configurable
[ https://issues.apache.org/jira/browse/SOLR-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225788#comment-13225788 ] Jason Rutherglen commented on SOLR-3221: +1 Long overdue. > Make Shard handler threadpool configurable > -- > > Key: SOLR-3221 > URL: https://issues.apache.org/jira/browse/SOLR-3221 > Project: Solr > Issue Type: Improvement >Affects Versions: 3.6 >Reporter: Greg Bowyer >Assignee: Erick Erickson > Labels: distributed, http, shard > Attachments: SOLR-3221-3x_branch.patch > > > From profiling of monitor contention, as well as observations of the > 95th and 99th response times for nodes that perform distributed search > (or ‟aggregator‟ nodes) it would appear that the HttpShardHandler code > currently does a suboptimal job of managing outgoing shard level > requests. > Presently the code contained within lucene 3.5's SearchHandler and > Lucene trunk / 3x's ShardHandlerFactory create arbitrary threads in > order to service distributed search requests. This is done presently to > limit the size of the threadpool such that it does not consume resources > in deployment configurations that do not use distributed search. > This unfortunately has two impacts on the response time if the node > coordinating the distribution is under high load. > The usage of the MaxConnectionsPerHost configuration option results in > aggressive activity on semaphores within HttpCommons, it has been > observed that the aggregator can have a response time far greater than > that of the searchers. The above monitor contention would appear to > suggest that in some cases its possible for liveness issues to occur and > for simple queries to be starved of resources simply due to a lack of > attention from the viewpoint of context switching. > With, as mentioned above the http commons connection being hotly > contended > The fair, queue based configuration eliminates this, at the cost of > throughput. > This patch aims to make the threadpool largely configurable allowing for > those using solr to choose the throughput vs latency balance they > desire. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3141) Deprecate OPTIMIZE command in Solr
[ https://issues.apache.org/jira/browse/SOLR-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211467#comment-13211467 ] Jason Rutherglen commented on SOLR-3141: -1 Serious over/under-engineering. > Deprecate OPTIMIZE command in Solr > -- > > Key: SOLR-3141 > URL: https://issues.apache.org/jira/browse/SOLR-3141 > Project: Solr > Issue Type: Improvement > Components: update >Affects Versions: 3.5 >Reporter: Jan Høydahl > Labels: force, optimize > Fix For: 3.6 > > > Background: LUCENE-3454 renames optimize() as forceMerge(). Please read that > issue first. > Now that optimize() is rarely necessary anymore, and renamed in Lucene APIs, > what should be done with Solr's ancient optimize command? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2593) A new core admin action 'split' for splitting index
[ https://issues.apache.org/jira/browse/SOLR-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207407#comment-13207407 ] Jason Rutherglen commented on SOLR-2593: Is there a patch for this issue available? If not it's fine. > A new core admin action 'split' for splitting index > --- > > Key: SOLR-2593 > URL: https://issues.apache.org/jira/browse/SOLR-2593 > Project: Solr > Issue Type: New Feature >Reporter: Noble Paul > Fix For: 4.0 > > > If an index is too large/hot it would be desirable to split it out to another > core . > This core may eventually be replicated out to another host. > There can be to be multiple strategies > * random split of x or x% > * fq="user:johndoe" > example : > action=split&split=20percent&newcore=my_new_index > or > action=split&fq=user:johndoe&newcore=john_doe_index -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3756) Don't allow IndexWriterConfig setters to chain
[
https://issues.apache.org/jira/browse/LUCENE-3756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204160#comment-13204160
]
Jason Rutherglen commented on LUCENE-3756:
--
+1 I agree with Mike.
> Don't allow IndexWriterConfig setters to chain
> --
>
> Key: LUCENE-3756
> URL: https://issues.apache.org/jira/browse/LUCENE-3756
> Project: Lucene - Java
> Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
>
> Spinoff from LUCENE-3736.
> I don't like that IndexWriterConfig's setters are chainable; it
> results in code in our tests like this:
> {noformat}
> IndexWriter writer = new IndexWriter(dir, newIndexWriterConfig(
> TEST_VERSION_CURRENT, new
> MockAnalyzer(random)).setMaxBufferedDocs(2).setMergePolicy(newLogMergePolicy()));
> {noformat}
> I think in general we should avoid chaining since it encourages hard
> to read code (code is already hard enough to read!).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3759) Support joining a distributed environment.
[ https://issues.apache.org/jira/browse/LUCENE-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203118#comment-13203118 ] Jason Rutherglen commented on LUCENE-3759: -- +1 Nice, distributed join will be super useful. > Support joining a distributed environment. > -- > > Key: LUCENE-3759 > URL: https://issues.apache.org/jira/browse/LUCENE-3759 > Project: Lucene - Java > Issue Type: Improvement > Components: modules/join >Reporter: Martijn van Groningen > > Add two more methods in JoinUtil to support joining in a distributed manner. > * Method to retrieve all from values. > * Method to create a TermsQuery based on a set of from terms. > With these two methods distributed joining can be supported following these > steps: > # Retrieve from values from each shard > # Merge the retrieved from values. > # Create a TermsQuery based on the merged from terms and send this query to > all shards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3734) Allow customizing/subclassing of DirectoryReader
[ https://issues.apache.org/jira/browse/LUCENE-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13197074#comment-13197074 ] Jason Rutherglen commented on LUCENE-3734: -- The issues mentioned were brought up in LUCENE-3498 and LUCENE-3497 thus yielding a +1 from me. > Allow customizing/subclassing of DirectoryReader > > > Key: LUCENE-3734 > URL: https://issues.apache.org/jira/browse/LUCENE-3734 > Project: Lucene - Java > Issue Type: Sub-task > Components: core/index >Reporter: Uwe Schindler >Assignee: Uwe Schindler > Fix For: 4.0 > > Attachments: LUCENE-3734.patch > > > DirectoryReader is final and has only static factory methods. It is not > possible to subclass it in any way. > The problem is mainly Solr, as Solr accesses directory(), IndexCommits,... > and therefore cannot work on abstract IndexReader anymore. This should be > changed, by e.g. handling reopening in the IRFactory, also versions, > commits,... Currently its not possible to implement any other IRFactory that > returns something else. > On the other hand, it should be possible to "wrap" a DirectoryReader / > CompositeReader to handle filtering of collection based information > (subreaders, reopening hooks,...). This can be done by making DirectoryReader > abstract and let DirectoryReader.open return a internal hidden class > "StandardDirectoryReader". This is similar to the relatinship between > IndexReader and hidden DirectoryReader in the past. > DirectoryReader will have final implementations of most methods like getting > document stored fields, global docFreq and other statistics, but allows > hooking into doOpenIfChanged. Also it should not be limited to SegmentReaders > as childs - any AtomicReader is fine. This allows users to create e.g. a > Directory-based ParallelReader (see LUCENE-3736) that supports reopen and > (partially commits). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3602) Add join query to Lucene
[ https://issues.apache.org/jira/browse/LUCENE-3602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189509#comment-13189509 ] Jason Rutherglen commented on LUCENE-3602: -- Just following up on the per-segment terms collection. Join is going to be used as a filter in most cases (?). Filters can be applied per-segment (unlike scoring queries). So it seems possible to avoid the BRH creation by using the DTI? > Add join query to Lucene > > > Key: LUCENE-3602 > URL: https://issues.apache.org/jira/browse/LUCENE-3602 > Project: Lucene - Java > Issue Type: New Feature > Components: modules/join >Reporter: Martijn van Groningen > Fix For: 3.6, 4.0 > > Attachments: LUCENE-3602-3x.patch, LUCENE-3602.patch, > LUCENE-3602.patch, LUCENE-3602.patch, LUCENE-3602.patch, LUCENE-3602.patch, > LUCENE-3602.patch, LUCENE-3602.patch, LUCENE-3602.patch, LUCENE-3602.patch, > LUCENE-3602.patch > > > Solr has (psuedo) join query for a while now. I think this should also be > available in Lucene. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3602) Add join query to Lucene
[
https://issues.apache.org/jira/browse/LUCENE-3602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189501#comment-13189501
]
Jason Rutherglen commented on LUCENE-3602:
--
{quote}The terms collected in the first phase are from many segments{quote}
Why is that necessary?
{quote}Caching can be improved in the second phase as you described, by saving
a bitset per fromTerm?{quote}
Possibly, only for terms with a high number of documents. Or we can use a
faster to decode (less compressed) posting codec.
bq. The JoinUtil is between 2 till 3 times faster than Solr's JoinQuery with
this data set on my dev machine
Interesting, thanks for sharing.
> Add join query to Lucene
>
>
> Key: LUCENE-3602
> URL: https://issues.apache.org/jira/browse/LUCENE-3602
> Project: Lucene - Java
> Issue Type: New Feature
> Components: modules/join
>Reporter: Martijn van Groningen
> Fix For: 3.6, 4.0
>
> Attachments: LUCENE-3602-3x.patch, LUCENE-3602.patch,
> LUCENE-3602.patch, LUCENE-3602.patch, LUCENE-3602.patch, LUCENE-3602.patch,
> LUCENE-3602.patch, LUCENE-3602.patch, LUCENE-3602.patch, LUCENE-3602.patch,
> LUCENE-3602.patch
>
>
> Solr has (psuedo) join query for a while now. I think this should also be
> available in Lucene.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3602) Add join query to Lucene
[ https://issues.apache.org/jira/browse/LUCENE-3602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189375#comment-13189375 ] Jason Rutherglen commented on LUCENE-3602: -- I was reviewing this issue to use where Solr's join implementation may not be the right choice. In this Lucene Join implementation, a new BytesRefHash is built per query (and cannot be reused). This could generate a fair amount of garbage quickly. Also the sort compare using BRH is per byte (not as cheap as an ord compare). We can probably use DocTermsIndex to replace the use of BytesRefHash by comparing DTI's ords. Then we are saving off the bytes into BRH per query, and the comparison would be faster. Additionally, for a join with many terms, the number of postings could become a factor in performance. Because we are not caching bitsets like Solr does, it seems like an excellent occasion for a faster less-compressed codec. Further, to save on term seeking, if the term state was cached (eg, the file pointers into the posting list), the iteration on the terms dict would be removed. Granted all this requires more RAM, however in many cases (eg, mine), that would be acceptable. > Add join query to Lucene > > > Key: LUCENE-3602 > URL: https://issues.apache.org/jira/browse/LUCENE-3602 > Project: Lucene - Java > Issue Type: New Feature > Components: modules/join >Reporter: Martijn van Groningen > Fix For: 3.6, 4.0 > > Attachments: LUCENE-3602.patch, LUCENE-3602.patch, LUCENE-3602.patch, > LUCENE-3602.patch, LUCENE-3602.patch, LUCENE-3602.patch, LUCENE-3602.patch, > LUCENE-3602.patch, LUCENE-3602.patch, LUCENE-3602.patch > > > Solr has (psuedo) join query for a while now. I think this should also be > available in Lucene. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3044) Incrementally deprecate NamedList & replace with typesafe API
[ https://issues.apache.org/jira/browse/SOLR-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13188549#comment-13188549 ] Jason Rutherglen commented on SOLR-3044: +1 NamedList is an oldie, not not a goodie. > Incrementally deprecate NamedList & replace with typesafe API > - > > Key: SOLR-3044 > URL: https://issues.apache.org/jira/browse/SOLR-3044 > Project: Solr > Issue Type: Improvement >Affects Versions: 3.6, 4.0 >Reporter: Simon Willnauer >Priority: Critical > > The first thing I can think of when I see how and where NamedList is used in > solr is "if you have a hammer in your hands, every problem looks like a > nail". IMO and I know others think the same way the use of NamedList is way > over the top for a long time. However the biggest issues here is the massive > use of this class all over the place which has several problem, here is a > list just to name a some: > * no type safety > * produces lots of garbage > * makes things hard to refactor > * binds everything strongly to Solr and is contra modularization > * code is hardly readable - one example is all the distributed request / > response processing > * requires autoboxing of primitives all over the place > * some processing is N^2 where N is possible > * requires tons of instanceof conditions > * ... > Yet this task is not simple nor is it possible to do this in a single patch. > I think the target of this issue and all its subtasks will be 5.0 but we need > to start doing it to eventually clean up the API enough to get rid of all the > issues I named above. > One way of starting would be to create a couple of subtasks like: > * Refactor ResponseWriters to pass in a StreamWriter similar to what XML or > JSON apis (Jackson / STAX) and let the ResponseObject write itself based on > the StreamWriter impl. > * Refactor configuration and resourceloading to use some libraries that are > specialized to do that. > * Deprecate SearchComponent methods that accept named list in favor of a > typesafe API > I think we should start doing this its time to move on here! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3602) Add join query to Lucene
[ https://issues.apache.org/jira/browse/LUCENE-3602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187370#comment-13187370 ] Jason Rutherglen commented on LUCENE-3602: -- Sweet! How join would work in distributed mode, that would be very useful for BigData projects. > Add join query to Lucene > > > Key: LUCENE-3602 > URL: https://issues.apache.org/jira/browse/LUCENE-3602 > Project: Lucene - Java > Issue Type: New Feature > Components: modules/join >Reporter: Martijn van Groningen > Fix For: 3.6, 4.0 > > Attachments: LUCENE-3602.patch, LUCENE-3602.patch, LUCENE-3602.patch, > LUCENE-3602.patch, LUCENE-3602.patch, LUCENE-3602.patch, LUCENE-3602.patch, > LUCENE-3602.patch, LUCENE-3602.patch, LUCENE-3602.patch > > > Solr has (psuedo) join query for a while now. I think this should also be > available in Lucene. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2702) Add support for NRTCachingDirectory
[ https://issues.apache.org/jira/browse/SOLR-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177885#comment-13177885 ] Jason Rutherglen commented on SOLR-2702: This issue [only] needs to add configuration options given NRTCachingDirectoryFactory is in trunk. > Add support for NRTCachingDirectory > --- > > Key: SOLR-2702 > URL: https://issues.apache.org/jira/browse/SOLR-2702 > Project: Solr > Issue Type: Improvement >Reporter: Mark Miller >Assignee: Mark Miller > Fix For: 4.0 > > > would be nice to have this option for the new NRT support -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3654) Optimize BytesRef comparator to use Unsafe long based comparison (when possible)
[ https://issues.apache.org/jira/browse/LUCENE-3654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173853#comment-13173853 ] Jason Rutherglen commented on LUCENE-3654: -- +1 There are 3 other MAJOR Apache projects that have already integrated this efficiency. It's completely silly not to use it. > Optimize BytesRef comparator to use Unsafe long based comparison (when > possible) > > > Key: LUCENE-3654 > URL: https://issues.apache.org/jira/browse/LUCENE-3654 > Project: Lucene - Java > Issue Type: Improvement > Components: core/index, core/search >Reporter: Shay Banon > Attachments: LUCENE-3654.patch > > > Inspire by Google Guava UnsignedBytes lexi comparator, that uses unsafe to do > long based comparisons over the bytes instead of one by one (which yields > 2-4x better perf), use similar logic in BytesRef comparator. The code was > adapted to support offset/length. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3654) Optimize BytesRef comparator to use Unsafe long based comparison (when possible)
[ https://issues.apache.org/jira/browse/LUCENE-3654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171716#comment-13171716 ] Jason Rutherglen commented on LUCENE-3654: -- Nice, I mentioned this on the dev list over a month ago (originally it was mentioned on the Hadoop list), nice to see it get into Lucene, though am curious where the speed improvement will be for Lucene. > Optimize BytesRef comparator to use Unsafe long based comparison (when > possible) > > > Key: LUCENE-3654 > URL: https://issues.apache.org/jira/browse/LUCENE-3654 > Project: Lucene - Java > Issue Type: Improvement > Components: core/index, core/search >Reporter: Shay Banon > Attachments: LUCENE-3654.patch > > > Inspire by Google Guava UnsignedBytes lexi comparator, that uses unsafe to do > long based comparisons over the bytes instead of one by one (which yields > 2-4x better perf), use similar logic in BytesRef comparator. The code was > adapted to support offset/length. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3602) Add join query to Lucene
[ https://issues.apache.org/jira/browse/LUCENE-3602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167959#comment-13167959 ] Jason Rutherglen commented on LUCENE-3602: -- Maybe we can (in another issue) move bit set filter caching into SearchManager, for use by Lucene Join (here), and others. At the same time making bitset filtering per-segment, a fundamental improvement from the existing (old) Solr code. > Add join query to Lucene > > > Key: LUCENE-3602 > URL: https://issues.apache.org/jira/browse/LUCENE-3602 > Project: Lucene - Java > Issue Type: New Feature > Components: modules/join >Reporter: Martijn van Groningen > Attachments: LUCENE-3602.patch, LUCENE-3602.patch > > > Solr has (psuedo) join query for a while now. I think this should also be > available in Lucene. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3622) separate IndexDocValues interface from implementation
[ https://issues.apache.org/jira/browse/LUCENE-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166305#comment-13166305 ] Jason Rutherglen commented on LUCENE-3622: -- +! To the function naming being completely off. That's the naming that should change. > separate IndexDocValues interface from implementation > - > > Key: LUCENE-3622 > URL: https://issues.apache.org/jira/browse/LUCENE-3622 > Project: Lucene - Java > Issue Type: Task >Reporter: Robert Muir > Attachments: LUCENE-3622.patch > > > Currently the o.a.l.index.values contains both the abstract apis and > Lucene40's current implementation. > I think we should move the implementation underneath Lucene40Codec, leaving > only the abstract apis. > For example, simpletext might have a different implementation, and we might > make a int8 implementation > underneath preflexcodec to support norms. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3602) Add join query to Lucene
[ https://issues.apache.org/jira/browse/LUCENE-3602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159335#comment-13159335 ] Jason Rutherglen commented on LUCENE-3602: -- Great to see this moving out of Solr and getting new eyes on it (with added improvements)! > Add join query to Lucene > > > Key: LUCENE-3602 > URL: https://issues.apache.org/jira/browse/LUCENE-3602 > Project: Lucene - Java > Issue Type: New Feature > Components: modules/join >Reporter: Martijn van Groningen > Attachments: LUCENE-3602.patch > > > Solr has (psuedo) join query for a while now. I think this should also be > available in Lucene. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3587) Attempting to link to Java SE JavaDocs is competely unreliable
[ https://issues.apache.org/jira/browse/LUCENE-3587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156300#comment-13156300 ] Jason Rutherglen commented on LUCENE-3587: -- +1 on the patch. Javadoc external links should not destroy a build. > Attempting to link to Java SE JavaDocs is competely unreliable > -- > > Key: LUCENE-3587 > URL: https://issues.apache.org/jira/browse/LUCENE-3587 > Project: Lucene - Java > Issue Type: Bug >Reporter: Hoss Man > Fix For: 3.6, 4.0 > > Attachments: LUCENE-3587.3x.patch, > LUCENE-3587.keep-javadoc-link.3x.patch, > LUCENE-3587.keep-javadoc-link.trunk.patch, LUCENE-3587.trunk.patch > > > As noted several times since Oracle bought Sun, the canonical links to the > Java SE JavaDocs have been unreliable and frequently cause warnings. > Since we choose to fail the build on javadoc warnings, this is a serious > problem for anyone trying to build from source if/when the package-list we > reference in our common-build.xml is not available. > We should eliminate this dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2889) Implement Adaptive Replacement Cache
[ https://issues.apache.org/jira/browse/SOLR-2889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148269#comment-13148269 ] Jason Rutherglen commented on SOLR-2889: Simon and Yonik, re-read what you wrote, have fun. > Implement Adaptive Replacement Cache > > > Key: SOLR-2889 > URL: https://issues.apache.org/jira/browse/SOLR-2889 > Project: Solr > Issue Type: New Feature > Components: search >Affects Versions: 3.4 >Reporter: Shawn Heisey >Priority: Minor > > Currently Solr's caches are LRU, which doesn't look at hitcount to decide > which entries are most important. There is a method that takes both > frequency and time of cache hits into account: > http://en.wikipedia.org/wiki/Adaptive_Replacement_Cache > If it's feasible, this could be a good addition to Solr/Lucene. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2889) Implement Adaptive Replacement Cache
[ https://issues.apache.org/jira/browse/SOLR-2889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148065#comment-13148065 ] Jason Rutherglen commented on SOLR-2889: Yonik, Take a step back. No analyzers are in Solr, and the caching and other 'parts' will be moved out. It's reasonable to expect that process to happen on new additions to what is a singular project. > Implement Adaptive Replacement Cache > > > Key: SOLR-2889 > URL: https://issues.apache.org/jira/browse/SOLR-2889 > Project: Solr > Issue Type: New Feature > Components: search >Affects Versions: 3.4 >Reporter: Shawn Heisey >Priority: Minor > > Currently Solr's caches are LRU, which doesn't look at hitcount to decide > which entries are most important. There is a method that takes both > frequency and time of cache hits into account: > http://en.wikipedia.org/wiki/Adaptive_Replacement_Cache > If it's feasible, this could be a good addition to Solr/Lucene. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2889) Implement Adaptive Replacement Cache
[ https://issues.apache.org/jira/browse/SOLR-2889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148003#comment-13148003 ] Jason Rutherglen commented on SOLR-2889: +1 - Put it in Lucene and NOT Solr. thanks. When this is implemented, using Google collections should be developed as well (which appropriately jettisons the cache values before OOM), ala the previously created though not committed SOLR-1513. > Implement Adaptive Replacement Cache > > > Key: SOLR-2889 > URL: https://issues.apache.org/jira/browse/SOLR-2889 > Project: Solr > Issue Type: New Feature > Components: search >Affects Versions: 3.4 >Reporter: Shawn Heisey >Priority: Minor > > Currently Solr's caches are LRU, which doesn't look at hitcount to decide > which entries are most important. There is a method that takes both > frequency and time of cache hits into account: > http://en.wikipedia.org/wiki/Adaptive_Replacement_Cache > If it's feasible, this could be a good addition to Solr/Lucene. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2849) Solr maven dependencies: logging
[
https://issues.apache.org/jira/browse/SOLR-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13134357#comment-13134357
]
Jason Rutherglen commented on SOLR-2849:
{quote}As an aside, it's unfortunate to see all those velocity dependencies.
It even depends on struts -- seriously?! I hope solritas gets put back into a
contrib sometime: SOLR-2588{quote}
+1, move it out!
> Solr maven dependencies: logging
>
>
> Key: SOLR-2849
> URL: https://issues.apache.org/jira/browse/SOLR-2849
> Project: Solr
> Issue Type: Improvement
> Components: Build
>Affects Versions: 4.0
>Reporter: David Smiley
>Priority: Trivial
>
> I was looking at my maven based project's Solr-core dependencies (trunk), and
> observed some issues that I think should be fixed in Solr's maven poms. I ran
> {{mvn dependency:tree}} -- the output is further below. There are two
> changes I see needed, related to logging:
> * slf4j-jdk14 should be runtime scope, and optional.
> * httpclient depends on commons-logging. Exclude this dependency from the
> httpclient dependency, and add a dependency on jcl-over-slf4j with compile
> scope.
> * Zookeeper depends on Log4j, unfortunately. There is an issue to change this
> to SLF4J: ZOOKEEPER-850. In the mean time we should exclude it and use
> log4j-over-slf4j with compile scope, at the solrj pom.
> As an aside, it's unfortunate to see all those velocity dependencies. It
> even depends on struts -- seriously?! I hope solritas gets put back into a
> contrib sometime: SOLR-2588
> Steve, if you'd like to me to create the patch, I will.
> {code}
> [INFO] +- org.apache.solr:solr-core:jar:4.0-SNAPSHOT:compile
> [INFO] | +- org.apache.solr:solr-solrj:jar:4.0-SNAPSHOT:compile
> [INFO] | | \- org.apache.zookeeper:zookeeper:jar:3.3.3:compile
> [INFO] | | +- log4j:log4j:jar:1.2.15:compile
> [INFO] | | | \- javax.mail:mail:jar:1.4:compile
> [INFO] | | | \- javax.activation:activation:jar:1.1:compile
> [INFO] | | \- jline:jline:jar:0.9.94:compile
> [INFO] | +- org.apache.solr:solr-noggit:jar:4.0-SNAPSHOT:compile
> [INFO] | +-
> org.apache.lucene:lucene-analyzers-phonetic:jar:4.0-SNAPSHOT:compile
> [INFO] | +- org.apache.lucene:lucene-highlighter:jar:4.0-SNAPSHOT:compile
> [INFO] | +- org.apache.lucene:lucene-memory:jar:4.0-SNAPSHOT:compile
> [INFO] | +- org.apache.lucene:lucene-misc:jar:4.0-SNAPSHOT:compile
> [INFO] | +- org.apache.lucene:lucene-queryparser:jar:4.0-SNAPSHOT:compile
> [INFO] | | \- org.apache.lucene:lucene-sandbox:jar:4.0-SNAPSHOT:compile
> [INFO] | | \- jakarta-regexp:jakarta-regexp:jar:1.4:compile
> [INFO] | +- org.apache.lucene:lucene-spatial:jar:4.0-SNAPSHOT:compile
> [INFO] | +- org.apache.lucene:lucene-suggest:jar:4.0-SNAPSHOT:compile
> [INFO] | +- org.apache.lucene:lucene-grouping:jar:4.0-SNAPSHOT:compile
> [INFO] | +- org.apache.solr:solr-commons-csv:jar:4.0-SNAPSHOT:compile
> [INFO] | +- commons-codec:commons-codec:jar:1.4:compile
> [INFO] | +- commons-fileupload:commons-fileupload:jar:1.2.1:compile
> [INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:compile
> [INFO] | | \- commons-logging:commons-logging:jar:1.0.4:compile
> [INFO] | +- commons-io:commons-io:jar:1.4:compile
> [INFO] | +- org.apache.velocity:velocity:jar:1.6.4:compile
> [INFO] | | +- commons-collections:commons-collections:jar:3.2.1:compile
> [INFO] | | \- oro:oro:jar:2.0.8:compile
> [INFO] | +- org.apache.velocity:velocity-tools:jar:2.0:compile
> [INFO] | | +- commons-beanutils:commons-beanutils:jar:1.7.0:compile
> [INFO] | | +- commons-digester:commons-digester:jar:1.8:compile
> [INFO] | | +- commons-chain:commons-chain:jar:1.1:compile
> [INFO] | | +- commons-validator:commons-validator:jar:1.3.1:compile
> [INFO] | | +- dom4j:dom4j:jar:1.1:compile
> [INFO] | | +- sslext:sslext:jar:1.2-0:compile
> [INFO] | | +- org.apache.struts:struts-core:jar:1.3.8:compile
> [INFO] | | | \- antlr:antlr:jar:2.7.2:compile
> [INFO] | | +- org.apache.struts:struts-taglib:jar:1.3.8:compile
> [INFO] | | \- org.apache.struts:struts-tiles:jar:1.3.8:compile
> [INFO] | +- org.slf4j:slf4j-jdk14:jar:1.6.1:compile
> [INFO] | \- org.codehaus.woodstox:wstx-asl:jar:3.2.7:runtime
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3498) IndexReaderFactory for Lucene
[ https://issues.apache.org/jira/browse/LUCENE-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123697#comment-13123697 ] Jason Rutherglen commented on LUCENE-3498: -- The current Solr IndexReaderFactory can be deprecated and / or replaced with one that accepts a Solr core. > IndexReaderFactory for Lucene > - > > Key: LUCENE-3498 > URL: https://issues.apache.org/jira/browse/LUCENE-3498 > Project: Lucene - Java > Issue Type: New Feature > Components: core/index >Affects Versions: 3.4, 4.0 >Reporter: Jason Rutherglen >Priority: Minor > Attachments: LUCENE-3498.patch, LUCENE-3498.patch > > > An IndexReaderFactory can be used by IndexWriter and DirectoryReader to > enable subclasses of DR to be instantiated by Lucene, automatically. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3498) IndexReaderFactory for Lucene
[ https://issues.apache.org/jira/browse/LUCENE-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123695#comment-13123695 ] Jason Rutherglen commented on LUCENE-3498: -- Also, the patch is against 3.x. > IndexReaderFactory for Lucene > - > > Key: LUCENE-3498 > URL: https://issues.apache.org/jira/browse/LUCENE-3498 > Project: Lucene - Java > Issue Type: New Feature > Components: core/index >Affects Versions: 3.4, 4.0 >Reporter: Jason Rutherglen >Priority: Minor > Attachments: LUCENE-3498.patch, LUCENE-3498.patch > > > An IndexReaderFactory can be used by IndexWriter and DirectoryReader to > enable subclasses of DR to be instantiated by Lucene, automatically. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3498) IndexReaderFactory for Lucene
[ https://issues.apache.org/jira/browse/LUCENE-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123250#comment-13123250 ] Jason Rutherglen commented on LUCENE-3498: -- Simon, I think you'd be surprised at how many of the current [uber-complex] features of Lucene, few people use (readerTermsIndexDivisor, termIndexInterval, mergedSegmentWarmer are great examples, that are not-so-complex in IWC alone). For people who use this, the factory system is a lot more user friendly than subclassing. A protected method in IW doesn't take into account opening a DR from a DR, to do that please commit LUCENE-3497. If that gets in, we can open an issue to add a protected method to IW. > IndexReaderFactory for Lucene > - > > Key: LUCENE-3498 > URL: https://issues.apache.org/jira/browse/LUCENE-3498 > Project: Lucene - Java > Issue Type: New Feature > Components: core/index >Affects Versions: 3.4, 4.0 >Reporter: Jason Rutherglen >Priority: Minor > Attachments: LUCENE-3498.patch > > > An IndexReaderFactory can be used by IndexWriter and DirectoryReader to > enable subclasses of DR to be instantiated by Lucene, automatically. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3498) IndexReaderFactory for Lucene
[ https://issues.apache.org/jira/browse/LUCENE-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123232#comment-13123232 ] Jason Rutherglen commented on LUCENE-3498: -- I think the Solr one could be made better, by passing in the core that's creating it. In that regard, Solr's can be improved rather than nuked. > IndexReaderFactory for Lucene > - > > Key: LUCENE-3498 > URL: https://issues.apache.org/jira/browse/LUCENE-3498 > Project: Lucene - Java > Issue Type: New Feature > Components: core/index >Affects Versions: 3.4, 4.0 >Reporter: Jason Rutherglen >Priority: Minor > Attachments: LUCENE-3498.patch > > > An IndexReaderFactory can be used by IndexWriter and DirectoryReader to > enable subclasses of DR to be instantiated by Lucene, automatically. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3497) Make DirectoryReader protected methods non-final
[ https://issues.apache.org/jira/browse/LUCENE-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123102#comment-13123102 ] Jason Rutherglen commented on LUCENE-3497: -- I'll open another issue for the factory method of enabling Lucene to open custom DirectoryReader's. > Make DirectoryReader protected methods non-final > > > Key: LUCENE-3497 > URL: https://issues.apache.org/jira/browse/LUCENE-3497 > Project: Lucene - Java > Issue Type: Improvement > Components: core/index >Affects Versions: 3.4, 4.0 >Reporter: Jason Rutherglen >Priority: Minor > Attachments: LUCENE-3497.patch > > > DirectoryReader has protected methods that are overridden and made final. > This is silly because it prevents other classes from overriding > DirectoryReader. The methods are doOpenIfChanged(*) and a handful of related > variables that are private. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-1536) if a filter can support random access API, we should use it
[ https://issues.apache.org/jira/browse/LUCENE-1536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122858#comment-13122858 ] Jason Rutherglen commented on LUCENE-1536: -- +1 > if a filter can support random access API, we should use it > --- > > Key: LUCENE-1536 > URL: https://issues.apache.org/jira/browse/LUCENE-1536 > Project: Lucene - Java > Issue Type: Improvement > Components: core/search >Affects Versions: 2.4 >Reporter: Michael McCandless >Assignee: Michael McCandless >Priority: Minor > Labels: gsoc2011, lucene-gsoc-11, mentor > Fix For: 4.0 > > Attachments: CachedFilterIndexReader.java, LUCENE-1536-rewrite.patch, > LUCENE-1536-rewrite.patch, LUCENE-1536-rewrite.patch, > LUCENE-1536-rewrite.patch, LUCENE-1536-rewrite.patch, > LUCENE-1536-rewrite.patch, LUCENE-1536.patch, LUCENE-1536.patch, > LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, > LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, > LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, > LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, > LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch > > > I ran some performance tests, comparing applying a filter via > random-access API instead of current trunk's iterator API. > This was inspired by LUCENE-1476, where we realized deletions should > really be implemented just like a filter, but then in testing found > that switching deletions to iterator was a very sizable performance > hit. > Some notes on the test: > * Index is first 2M docs of Wikipedia. Test machine is Mac OS X > 10.5.6, quad core Intel CPU, 6 GB RAM, java 1.6.0_07-b06-153. > * I test across multiple queries. 1-X means an OR query, eg 1-4 > means 1 OR 2 OR 3 OR 4, whereas +1-4 is an AND query, ie 1 AND 2 > AND 3 AND 4. "u s" means "united states" (phrase search). > * I test with multiple filter densities (0, 1, 2, 5, 10, 25, 75, 90, > 95, 98, 99, 99.9 (filter is non-null but all bits are set), > 100 (filter=null, control)). > * Method high means I use random-access filter API in > IndexSearcher's main loop. Method low means I use random-access > filter API down in SegmentTermDocs (just like deleted docs > today). > * Baseline (QPS) is current trunk, where filter is applied as iterator up > "high" (ie in IndexSearcher's search loop). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3493) Solr reopen on a custom reader doesn't work
[
https://issues.apache.org/jira/browse/LUCENE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122506#comment-13122506
]
Jason Rutherglen commented on LUCENE-3493:
--
Uwe, check this out on the 3.3 version, what do you think? :)
{code}
@Override
public final IndexReader reopen() throws CorruptIndexException, IOException {
// Preserve current readOnly
return doReopen(readOnly, null);
}
@Override
public final IndexReader reopen(boolean openReadOnly) throws
CorruptIndexException, IOException {
return doReopen(openReadOnly, null);
}
@Override
public final IndexReader reopen(final IndexCommit commit) throws
CorruptIndexException, IOException {
return doReopen(true, commit);
}
{code}
> Solr reopen on a custom reader doesn't work
> ---
>
> Key: LUCENE-3493
> URL: https://issues.apache.org/jira/browse/LUCENE-3493
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/index
>Affects Versions: 3.4
>Reporter: Jason Rutherglen
>Priority: Blocker
> Attachments: LUCENE-3493.patch
>
>
> When a custom index reader is used with Solr and reopen, the custom reader
> vanishes after the reopen. It's a bug.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3493) Solr reopen on a custom reader doesn't work
[ https://issues.apache.org/jira/browse/LUCENE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122095#comment-13122095 ] Jason Rutherglen commented on LUCENE-3493: -- One way to solve all of this without subclassing, is to move the IndexReaderFactory to Lucene, integrate it into IW and DR. That will be much cleaner than forcing users to subclass, which is a monstrous pain, and will generate excessive unnecessary code in the end. > Solr reopen on a custom reader doesn't work > --- > > Key: LUCENE-3493 > URL: https://issues.apache.org/jira/browse/LUCENE-3493 > Project: Lucene - Java > Issue Type: Bug > Components: core/index >Affects Versions: 3.4 >Reporter: Jason Rutherglen >Priority: Blocker > Attachments: LUCENE-3493.patch > > > When a custom index reader is used with Solr and reopen, the custom reader > vanishes after the reopen. It's a bug. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3493) Solr reopen on a custom reader doesn't work
[ https://issues.apache.org/jira/browse/LUCENE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122089#comment-13122089 ] Jason Rutherglen commented on LUCENE-3493: -- Uwe, I tried your idea. It doesn't work! Here's why: DR.writeLock and DR.segmentInfos are private. Meaning the re-duplicated code because the useful methods aren't protected, cannot access these private variables. Of course one can use reflection but that's just 'atrocious'. :) > Solr reopen on a custom reader doesn't work > --- > > Key: LUCENE-3493 > URL: https://issues.apache.org/jira/browse/LUCENE-3493 > Project: Lucene - Java > Issue Type: Bug > Components: core/index >Affects Versions: 3.4 >Reporter: Jason Rutherglen >Priority: Blocker > Attachments: LUCENE-3493.patch > > > When a custom index reader is used with Solr and reopen, the custom reader > vanishes after the reopen. It's a bug. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3493) Solr reopen on a custom reader doesn't work
[
https://issues.apache.org/jira/browse/LUCENE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122050#comment-13122050
]
Jason Rutherglen commented on LUCENE-3493:
--
bq. Solr's NRT does not rely on a custom IndexReader
Yikes, logically the custom reader functionality should!
{quote}properly override doOpenIfChanged, else it would be a bug{quote}
It's a bug because there's no way to implement that today. The DirectoryReader
is created deep inside of IW.getReader (there's no way to re-implement it's
functionality either because of private variable access).
I think we need a protected method for creating reader in IW. I think though
this becomes almost endless because I don't think there's a way to implement a
custom IW in Solr.
> Solr reopen on a custom reader doesn't work
> ---
>
> Key: LUCENE-3493
> URL: https://issues.apache.org/jira/browse/LUCENE-3493
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/index
>Affects Versions: 3.4
>Reporter: Jason Rutherglen
>Priority: Blocker
> Attachments: LUCENE-3493.patch
>
>
> When a custom index reader is used with Solr and reopen, the custom reader
> vanishes after the reopen. It's a bug.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3493) Solr reopen on a custom reader doesn't work
[ https://issues.apache.org/jira/browse/LUCENE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122033#comment-13122033 ] Jason Rutherglen commented on LUCENE-3493: -- Uwe, I'd like to agree with you, however I cannot (because then I wouldn't have had to create an issue!). Look at DR.doOpen* methods. They're private. There's no reason for them to be. They need to be protected, that's in the next patch. Fairly simple. The follow on to this is overriding IW to return custom readers. I had an issue and patch for that a while back. It's best to implement both here, as Lucene 4.x Solr's NRT will show the same problem! I think you're right, looks like this *could* be done be overriding doOpenIfChanged* however, it doesn't make sense to duplicate code! > Solr reopen on a custom reader doesn't work > --- > > Key: LUCENE-3493 > URL: https://issues.apache.org/jira/browse/LUCENE-3493 > Project: Lucene - Java > Issue Type: Bug > Components: core/index >Affects Versions: 3.4 >Reporter: Jason Rutherglen >Priority: Blocker > Attachments: LUCENE-3493.patch > > > When a custom index reader is used with Solr and reopen, the custom reader > vanishes after the reopen. It's a bug. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3493) Solr reopen on a custom reader doesn't work
[ https://issues.apache.org/jira/browse/LUCENE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122019#comment-13122019 ] Jason Rutherglen commented on LUCENE-3493: -- The patch shows the bug only. Which needs a test in Solr. The next patch will show the fix etc. A Lucene test makes sense as well. > Solr reopen on a custom reader doesn't work > --- > > Key: LUCENE-3493 > URL: https://issues.apache.org/jira/browse/LUCENE-3493 > Project: Lucene - Java > Issue Type: Bug > Components: core/index >Affects Versions: 3.4 >Reporter: Jason Rutherglen >Priority: Blocker > Attachments: LUCENE-3493.patch > > > When a custom index reader is used with Solr and reopen, the custom reader > vanishes after the reopen. It's a bug. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3488) Factor out SearcherManager from NRTManager
[ https://issues.apache.org/jira/browse/LUCENE-3488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121556#comment-13121556 ] Jason Rutherglen commented on LUCENE-3488: -- bq. Arn't you used to pushback on your code / ideas by now? Mark, I missed this, it's particularly funny given this issue isn't mine! Please stay on topic! (Sorry Simon, nice work!) > Factor out SearcherManager from NRTManager > -- > > Key: LUCENE-3488 > URL: https://issues.apache.org/jira/browse/LUCENE-3488 > Project: Lucene - Java > Issue Type: Improvement >Affects Versions: 3.5, 4.0 >Reporter: Simon Willnauer > Fix For: 3.5, 4.0 > > Attachments: LUCENE-3488.patch, LUCENE-3488.patch > > > Currently we have NRTManager and SearcherManager while NRTManager contains a > big piece of the code that is already in SearcherManager. Users are kind of > forced to use NRTManager if they want to have SearcherManager goodness with > NRT. The integration into NRTManager also forces you to maintain two > instances even if you know you always want deletes. To me NRTManager tries to > do more than necessary and mixes lots of responsibilities ie. handling > searchers and handling indexing generations. NRTManager should use a > SearcherManager by aggregation rather than duplicate a lot of logic. > SearcherManager should have a NRT and Directory based implementation users > can simply choose from. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3488) Factor out SearcherManager from NRTManager
[ https://issues.apache.org/jira/browse/LUCENE-3488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121381#comment-13121381 ] Jason Rutherglen commented on LUCENE-3488: -- bq. Ok. lets got back to this code. non of the comments have been related to this! +1 to the code! > Factor out SearcherManager from NRTManager > -- > > Key: LUCENE-3488 > URL: https://issues.apache.org/jira/browse/LUCENE-3488 > Project: Lucene - Java > Issue Type: Improvement >Affects Versions: 3.5, 4.0 >Reporter: Simon Willnauer > Fix For: 3.5, 4.0 > > Attachments: LUCENE-3488.patch > > > Currently we have NRTManager and SearcherManager while NRTManager contains a > big piece of the code that is already in SearcherManager. Users are kind of > forced to use NRTManager if they want to have SearcherManager goodness with > NRT. The integration into NRTManager also forces you to maintain two > instances even if you know you always want deletes. To me NRTManager tries to > do more than necessary and mixes lots of responsibilities ie. handling > searchers and handling indexing generations. NRTManager should use a > SearcherManager by aggregation rather than duplicate a lot of logic. > SearcherManager should have a NRT and Directory based implementation users > can simply choose from. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3488) Factor out SearcherManager from NRTManager
[ https://issues.apache.org/jira/browse/LUCENE-3488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121369#comment-13121369 ] Jason Rutherglen commented on LUCENE-3488: -- bq. manager classes leaner Leaner, better, modularized, pluggable, etc ... etc. SolrCore is *final*. I remember having that debate a while back with Chris Hostetter. Why Solr needs to be monolithic, I don't know. Attempts to fix that have met with, and continue to be met with push back. That is quite evidently clear! > Factor out SearcherManager from NRTManager > -- > > Key: LUCENE-3488 > URL: https://issues.apache.org/jira/browse/LUCENE-3488 > Project: Lucene - Java > Issue Type: Improvement >Affects Versions: 3.5, 4.0 >Reporter: Simon Willnauer > Fix For: 3.5, 4.0 > > Attachments: LUCENE-3488.patch > > > Currently we have NRTManager and SearcherManager while NRTManager contains a > big piece of the code that is already in SearcherManager. Users are kind of > forced to use NRTManager if they want to have SearcherManager goodness with > NRT. The integration into NRTManager also forces you to maintain two > instances even if you know you always want deletes. To me NRTManager tries to > do more than necessary and mixes lots of responsibilities ie. handling > searchers and handling indexing generations. NRTManager should use a > SearcherManager by aggregation rather than duplicate a lot of logic. > SearcherManager should have a NRT and Directory based implementation users > can simply choose from. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3488) Factor out SearcherManager from NRTManager
[ https://issues.apache.org/jira/browse/LUCENE-3488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121350#comment-13121350 ] Jason Rutherglen commented on LUCENE-3488: -- Atrocious or perhaps horrible is: Lines 1041 - 1345 of [1]. Saying patches are welcome when these issues were brought up in SOLR-2193 when that gave way to SOLR-2565 which ended up being 155k plus several additional patches is ludicrous. :) A redesign could / should have yielded much better results. I didn't. 1. http://svn.apache.org/viewvc/lucene/dev/trunk/solr/core/src/java/org/apache/solr/core/SolrCore.java?view=markup > Factor out SearcherManager from NRTManager > -- > > Key: LUCENE-3488 > URL: https://issues.apache.org/jira/browse/LUCENE-3488 > Project: Lucene - Java > Issue Type: Improvement >Affects Versions: 3.5, 4.0 >Reporter: Simon Willnauer > Fix For: 3.5, 4.0 > > Attachments: LUCENE-3488.patch > > > Currently we have NRTManager and SearcherManager while NRTManager contains a > big piece of the code that is already in SearcherManager. Users are kind of > forced to use NRTManager if they want to have SearcherManager goodness with > NRT. The integration into NRTManager also forces you to maintain two > instances even if you know you always want deletes. To me NRTManager tries to > do more than necessary and mixes lots of responsibilities ie. handling > searchers and handling indexing generations. NRTManager should use a > SearcherManager by aggregation rather than duplicate a lot of logic. > SearcherManager should have a NRT and Directory based implementation users > can simply choose from. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3488) Factor out SearcherManager from NRTManager
[ https://issues.apache.org/jira/browse/LUCENE-3488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120984#comment-13120984 ] Jason Rutherglen commented on LUCENE-3488: -- bq. has been around for 7 years That's far too long. Hence the push for modules. > Factor out SearcherManager from NRTManager > -- > > Key: LUCENE-3488 > URL: https://issues.apache.org/jira/browse/LUCENE-3488 > Project: Lucene - Java > Issue Type: Improvement >Affects Versions: 3.5, 4.0 >Reporter: Simon Willnauer > Fix For: 3.5, 4.0 > > Attachments: LUCENE-3488.patch > > > Currently we have NRTManager and SearcherManager while NRTManager contains a > big piece of the code that is already in SearcherManager. Users are kind of > forced to use NRTManager if they want to have SearcherManager goodness with > NRT. The integration into NRTManager also forces you to maintain two > instances even if you know you always want deletes. To me NRTManager tries to > do more than necessary and mixes lots of responsibilities ie. handling > searchers and handling indexing generations. NRTManager should use a > SearcherManager by aggregation rather than duplicate a lot of logic. > SearcherManager should have a NRT and Directory based implementation users > can simply choose from. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3486) Add SearcherLifetimeManager, so you can retrieve the same searcher you previously used
[ https://issues.apache.org/jira/browse/LUCENE-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120960#comment-13120960 ] Jason Rutherglen commented on LUCENE-3486: -- bq. Looks good. I noticed you marked close() with @Override. Are we on Java 6 in 3.x? @Override is all over the place in Solr!? > Add SearcherLifetimeManager, so you can retrieve the same searcher you > previously used > -- > > Key: LUCENE-3486 > URL: https://issues.apache.org/jira/browse/LUCENE-3486 > Project: Lucene - Java > Issue Type: New Feature > Components: core/search >Reporter: Michael McCandless >Assignee: Michael McCandless > Fix For: 3.5, 4.0 > > Attachments: LUCENE-3486.patch, LUCENE-3486.patch > > > The idea is similar to SOLR-2809 (adding searcher leases to Solr). > This utility class sits above whatever your source is for "the > current" searcher (eg NRTManager, SearcherManager, etc.), and records > (holds a reference to) each searcher in recent history. > The idea is to ensure that when a user does a follow-on action (clicks > next page, drills down/up), or when two or more searcher invocations > within a single user search need to happen against the same searcher > (eg in distributed search), you can retrieve the same searcher you > used "last time". > I think with the new searchAfter API (LUCENE-2215), doing follow-on > searches on the same searcher is more important, since the "bottom" > (score/docID) held for that API can easily shift when a new searcher > is opened. > When you do a "new" search, you record the searcher you used with the > manager, and it returns to you a long token (currently just the > IR.getVersion()), which you can later use to retrieve the same > searcher. > Separately you must periodically call prune(), to prune the old > searchers, ideally from the same thread / at the same time that > you open a new searcher. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3488) Factor out SearcherManager from NRTManager
[ https://issues.apache.org/jira/browse/LUCENE-3488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120941#comment-13120941 ] Jason Rutherglen commented on LUCENE-3488: -- Great! Next step is integrating it into Solr and nuking the current atrocious Solr code. :) > Factor out SearcherManager from NRTManager > -- > > Key: LUCENE-3488 > URL: https://issues.apache.org/jira/browse/LUCENE-3488 > Project: Lucene - Java > Issue Type: Improvement >Affects Versions: 3.5, 4.0 >Reporter: Simon Willnauer > Fix For: 3.5, 4.0 > > Attachments: LUCENE-3488.patch > > > Currently we have NRTManager and SearcherManager while NRTManager contains a > big piece of the code that is already in SearcherManager. Users are kind of > forced to use NRTManager if they want to have SearcherManager goodness with > NRT. The integration into NRTManager also forces you to maintain two > instances even if you know you always want deletes. To me NRTManager tries to > do more than necessary and mixes lots of responsibilities ie. handling > searchers and handling indexing generations. NRTManager should use a > SearcherManager by aggregation rather than duplicate a lot of logic. > SearcherManager should have a NRT and Directory based implementation users > can simply choose from. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2809) searcher leases
[ https://issues.apache.org/jira/browse/SOLR-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120223#comment-13120223 ] Jason Rutherglen commented on SOLR-2809: SOLR-2778 is the issue that seeks to clean up and modularize the distributed search code. > searcher leases > --- > > Key: SOLR-2809 > URL: https://issues.apache.org/jira/browse/SOLR-2809 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley > > Leases/reservations on searcher instances would give us the ability to use > the same searcher across phases of a distributed search, or for clients to > send multiple requests and have them hit a consistent/unchanging view of the > index. The latter requires something extra to ensure that the load balancer > contacts the same replicas as before. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2809) searcher leases
[
https://issues.apache.org/jira/browse/SOLR-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120217#comment-13120217
]
Jason Rutherglen commented on SOLR-2809:
{quote}no need to modify any of the guts of the complex
SolrCore.getSearcher(){quote}
That code should be removed entirely, better now in 4.x.
The unique id per searcher idea would work, however it needs to also implement
a retry when a given id no longer exists.
Still, this would be best implemented in the context of rewriting distributed
search and the getSearcher code. Otherwise is layering hacked up code on top
of further hacked up code. It's a mess to debug and change later.
> searcher leases
> ---
>
> Key: SOLR-2809
> URL: https://issues.apache.org/jira/browse/SOLR-2809
> Project: Solr
> Issue Type: New Feature
>Reporter: Yonik Seeley
>
> Leases/reservations on searcher instances would give us the ability to use
> the same searcher across phases of a distributed search, or for clients to
> send multiple requests and have them hit a consistent/unchanging view of the
> index. The latter requires something extra to ensure that the load balancer
> contacts the same replicas as before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2809) searcher leases
[
https://issues.apache.org/jira/browse/SOLR-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120211#comment-13120211
]
Jason Rutherglen commented on SOLR-2809:
{quote}
if we support distributed stats (idf etc), then this assumption could break
down in a lot of cases, because the stats from the first phase don't make sense
with regards to the documents being scored in the second phrase.
{quote}
They would make sense. Though it's like discussing the wind since RT isn't
completed. Sounds like you're thinking of a general retry? That's a good
idea, it would need to retry the entire distributed query in all phases.
That functionality should be modular and should not be 'pre-baked / canned'
into Solr. [Again] a simple policy class would suffice here.
> searcher leases
> ---
>
> Key: SOLR-2809
> URL: https://issues.apache.org/jira/browse/SOLR-2809
> Project: Solr
> Issue Type: New Feature
>Reporter: Yonik Seeley
>
> Leases/reservations on searcher instances would give us the ability to use
> the same searcher across phases of a distributed search, or for clients to
> send multiple requests and have them hit a consistent/unchanging view of the
> index. The latter requires something extra to ensure that the load balancer
> contacts the same replicas as before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2595) Split and migrate indexes
[ https://issues.apache.org/jira/browse/SOLR-2595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120189#comment-13120189 ] Jason Rutherglen commented on SOLR-2595: How will splitting occur on an index that is actively being updated? > Split and migrate indexes > - > > Key: SOLR-2595 > URL: https://issues.apache.org/jira/browse/SOLR-2595 > Project: Solr > Issue Type: New Feature > Components: multicore, replication (java), SolrCloud >Reporter: Shalin Shekhar Mangar > Fix For: 4.0 > > > When an shard's index grows too large or a shard becomes too loaded, it > should be possible to split parts of a shard's index and migrate/merge to a > less loaded node. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2809) searcher leases
[ https://issues.apache.org/jira/browse/SOLR-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120187#comment-13120187 ] Jason Rutherglen commented on SOLR-2809: In RT the searchers are cheap. The easiest approach would be to record the segments and max doc ids used to satisfy phase 1 of a given distributed query, then send that signature back in subsequent phases. The retry would only be necessary in the infrequent case of a merge have occurred. With NRT it's probably best to implement a searcher policy similar to index deletion policy. Then any timeout / searcher removal system can be implemented by the user, rather than dictated by Solr. The described searcher management system belongs in a module in Lucene rather than Solr, probably in one of Mike's new classes. > searcher leases > --- > > Key: SOLR-2809 > URL: https://issues.apache.org/jira/browse/SOLR-2809 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley > > Leases/reservations on searcher instances would give us the ability to use > the same searcher across phases of a distributed search, or for clients to > send multiple requests and have them hit a consistent/unchanging view of the > index. The latter requires something extra to ensure that the load balancer > contacts the same replicas as before. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
