Tracking not teaching... Auto complete is fun... On Tue, Sep 1, 2015, 6:34 AM Jamie Johnson <jej2...@gmail.com> wrote:
> No worries, thanks again I'll begin teaching this > > On Mon, Aug 31, 2015, 5:16 PM Tomás Fernández Löbbe <tomasflo...@gmail.com> > wrote: > >> Sorry Jamie, I totally missed this email. There was no Jira that I could >> find. I created SOLR-7996 >> >> On Sat, Aug 29, 2015 at 5:26 AM, Jamie Johnson <jej2...@gmail.com> wrote: >> >> > This sounds like a good idea, I'm assuming I'd need to make my own >> > UnInvertingReader (or subclass) to do this right? Is there a way to do >> > this on the 5.x codebase or would I still need the solrindexer factory >> work >> > that Tomás mentioned previously? >> > >> > Tomás, is there a ticket for the SolrIndexer factory? I'd like to >> follow >> > it's work to know what version of 5.x (or later) I should be looking for >> > this in. >> > >> > On Thu, Aug 27, 2015 at 1:06 PM, Yonik Seeley <ysee...@gmail.com> >> wrote: >> > >> > > UnInvertingReader makes indexed fields look like docvalues fields. >> > > The caching itself is still done in FieldCache/FieldCacheImpl >> > > but you could perhaps wrap what is cached there to either screen out >> > > stuff or construct a new entry based on the user. >> > > >> > > -Yonik >> > > >> > > >> > > On Thu, Aug 27, 2015 at 12:55 PM, Jamie Johnson <jej2...@gmail.com> >> > wrote: >> > > > I think a custom UnInvertingReader would work as I could skip the >> > process >> > > > of putting things in the cache. Right now in Solr 4.x though I am >> > > caching >> > > > based but including the users authorities in the key of the cache so >> > > we're >> > > > not rebuilding the UnivertedField on every request. Where in 5.x is >> > the >> > > > object actually cached? Will this be possible in 5.x? >> > > > >> > > > On Thu, Aug 27, 2015 at 12:32 PM, Yonik Seeley <ysee...@gmail.com> >> > > wrote: >> > > > >> > > >> The FieldCache has become implementation rather than interface, so >> I >> > > >> don't think you're going to see plugins at that level (it's all >> > > >> package protected now). >> > > >> >> > > >> One could either subclass or re-implement UnInvertingReader though. >> > > >> >> > > >> -Yonik >> > > >> >> > > >> >> > > >> On Thu, Aug 27, 2015 at 12:09 PM, Jamie Johnson <jej2...@gmail.com >> > >> > > wrote: >> > > >> > Also in this vein I think that Lucene should support factories >> for >> > the >> > > >> > cache creation as described @ >> > > >> > https://issues.apache.org/jira/browse/LUCENE-2394. I'm not >> > endorsing >> > > >> the >> > > >> > patch that is provided (I haven't even looked at it) just the >> > concept >> > > in >> > > >> > general. >> > > >> > >> > > >> > On Thu, Aug 27, 2015 at 12:01 PM, Jamie Johnson < >> jej2...@gmail.com> >> > > >> wrote: >> > > >> > >> > > >> >> That makes sense, then I could extend the SolrIndexSearcher by >> > > creating >> > > >> a >> > > >> >> different factory class that did whatever magic I needed. If >> you >> > > >> create a >> > > >> >> Jira ticket for this please link it here so I can track it! >> Again >> > > >> thanks >> > > >> >> >> > > >> >> On Thu, Aug 27, 2015 at 11:59 AM, Tomás Fernández Löbbe < >> > > >> >> tomasflo...@gmail.com> wrote: >> > > >> >> >> > > >> >>> I don't think there is a way to do this now. Maybe we should >> > > separate >> > > >> the >> > > >> >>> logic of creating the SolrIndexSearcher to a factory. Moving >> this >> > > logic >> > > >> >>> away from SolrCore is already a win, plus it will make it >> easier >> > to >> > > >> unit >> > > >> >>> test and extend for advanced use cases. >> > > >> >>> >> > > >> >>> Tomás >> > > >> >>> >> > > >> >>> On Wed, Aug 26, 2015 at 8:10 PM, Jamie Johnson < >> jej2...@gmail.com >> > > >> > > >> wrote: >> > > >> >>> >> > > >> >>> > Sorry to poke this again but I'm not following the last >> comment >> > of >> > > >> how I >> > > >> >>> > could go about extending the solr index searcher and have the >> > > >> extension >> > > >> >>> > used. Is there an example of this? Again thanks >> > > >> >>> > >> > > >> >>> > Jamie >> > > >> >>> > On Aug 25, 2015 7:18 AM, "Jamie Johnson" <jej2...@gmail.com> >> > > wrote: >> > > >> >>> > >> > > >> >>> > > I had seen this as well, if I over wrote this by extending >> > > >> >>> > > SolrIndexSearcher how do I have my extension used? I >> didn't >> > > see a >> > > >> way >> > > >> >>> > that >> > > >> >>> > > could be plugged in. >> > > >> >>> > > On Aug 25, 2015 7:15 AM, "Mikhail Khludnev" < >> > > >> >>> mkhlud...@griddynamics.com> >> > > >> >>> > > wrote: >> > > >> >>> > > >> > > >> >>> > >> On Tue, Aug 25, 2015 at 2:03 PM, Jamie Johnson < >> > > jej2...@gmail.com >> > > >> > >> > > >> >>> > wrote: >> > > >> >>> > >> >> > > >> >>> > >> > Thanks Mikhail. If I'm reading the SimpleFacets class >> > > >> correctly, >> > > >> >>> out >> > > >> >>> > >> > delegates to DocValuesFacets when facet method is FC, >> what >> > > used >> > > >> to >> > > >> >>> be >> > > >> >>> > >> > FieldCache I believe. DocValuesFacets either uses >> > DocValues >> > > or >> > > >> >>> builds >> > > >> >>> > >> then >> > > >> >>> > >> > using the UninvertingReader. >> > > >> >>> > >> > >> > > >> >>> > >> >> > > >> >>> > >> Ah.. got it. Thanks for reminding this details.It seems >> like >> > > even >> > > >> >>> > >> docValues=true doesn't help with your custom >> implementation. >> > > >> >>> > >> >> > > >> >>> > >> >> > > >> >>> > >> > >> > > >> >>> > >> > I am not seeing a clean extension point to add a custom >> > > >> >>> > >> UninvertingReader >> > > >> >>> > >> > to Solr, would the only way be to copy the >> FacetComponent >> > and >> > > >> >>> > >> SimpleFacets >> > > >> >>> > >> > and modify as needed? >> > > >> >>> > >> > >> > > >> >>> > >> Sadly, yes. There is no proper extension point. Also, >> > consider >> > > >> >>> > overriding >> > > >> >>> > >> SolrIndexSearcher.wrapReader(SolrCore, DirectoryReader) >> where >> > > the >> > > >> >>> > >> particular UninvertingReader is created, there you can >> pass >> > the >> > > >> own >> > > >> >>> one, >> > > >> >>> > >> which refers to custom FieldCache. >> > > >> >>> > >> >> > > >> >>> > >> >> > > >> >>> > >> > On Aug 25, 2015 12:42 AM, "Mikhail Khludnev" < >> > > >> >>> > >> mkhlud...@griddynamics.com> >> > > >> >>> > >> > wrote: >> > > >> >>> > >> > >> > > >> >>> > >> > > Hello Jamie, >> > > >> >>> > >> > > I don't understand how it could choose DocValuesFacets >> > (it >> > > >> >>> occurs on >> > > >> >>> > >> > > docValues=true) field, but then switches to >> > > >> >>> > >> UninvertingReader/FieldCache >> > > >> >>> > >> > > which means docValues=false. If you can provide more >> > > details >> > > >> it >> > > >> >>> > would >> > > >> >>> > >> be >> > > >> >>> > >> > > great. >> > > >> >>> > >> > > Beside of that, I suppose you can only implement and >> > inject >> > > >> your >> > > >> >>> own >> > > >> >>> > >> > > UninvertingReader, I don't think there is an extension >> > > point >> > > >> for >> > > >> >>> > this. >> > > >> >>> > >> > It's >> > > >> >>> > >> > > too specific requirement. >> > > >> >>> > >> > > >> > > >> >>> > >> > > On Tue, Aug 25, 2015 at 3:50 AM, Jamie Johnson < >> > > >> >>> jej2...@gmail.com> >> > > >> >>> > >> > wrote: >> > > >> >>> > >> > > >> > > >> >>> > >> > > > as mentioned in a previous email I have a need to >> > provide >> > > >> >>> security >> > > >> >>> > >> > > controls >> > > >> >>> > >> > > > at the term level. I know that Lucene/Solr doesn't >> > > support >> > > >> >>> this >> > > >> >>> > so >> > > >> >>> > >> I >> > > >> >>> > >> > had >> > > >> >>> > >> > > > baked something onto a 4.x baseline that was >> sufficient >> > > for >> > > >> my >> > > >> >>> use >> > > >> >>> > >> > cases. >> > > >> >>> > >> > > > I am now looking to move that implementation to 5.x >> and >> > > am >> > > >> >>> running >> > > >> >>> > >> into >> > > >> >>> > >> > > an >> > > >> >>> > >> > > > issue around faceting. Previously we were able to >> > > provide a >> > > >> >>> > custom >> > > >> >>> > >> > cache >> > > >> >>> > >> > > > implementation that would create separate cache >> entries >> > > >> given a >> > > >> >>> > >> > > particular >> > > >> >>> > >> > > > set of security controls, but in Solr 5 some >> faceting >> > is >> > > >> >>> delegated >> > > >> >>> > >> to >> > > >> >>> > >> > > > DocValuesFacets which delegates to >> UninvertingReader in >> > > my >> > > >> case >> > > >> >>> > (we >> > > >> >>> > >> are >> > > >> >>> > >> > > not >> > > >> >>> > >> > > > storing DocValues). The issue I am running into is >> > that >> > > >> before >> > > >> >>> > 5.x >> > > >> >>> > >> I >> > > >> >>> > >> > had >> > > >> >>> > >> > > > the ability to influence the FieldCache that was >> used >> > at >> > > the >> > > >> >>> Solr >> > > >> >>> > >> level >> > > >> >>> > >> > > to >> > > >> >>> > >> > > > also include a security token into the key so each >> > cache >> > > >> entry >> > > >> >>> was >> > > >> >>> > >> > scoped >> > > >> >>> > >> > > > to a particular level. With the current >> implementation >> > > the >> > > >> >>> > >> FieldCache >> > > >> >>> > >> > > > seems to be an internal detail that I can't >> influence >> > in >> > > >> >>> anyway. >> > > >> >>> > Is >> > > >> >>> > >> > this >> > > >> >>> > >> > > > correct? I had noticed this Jira ticket >> > > >> >>> > >> > > > https://issues.apache.org/jira/browse/LUCENE-5427, >> is >> > > there >> > > >> >>> any >> > > >> >>> > >> > movement >> > > >> >>> > >> > > > on >> > > >> >>> > >> > > > this? Is there another way to influence the >> > information >> > > >> that >> > > >> >>> is >> > > >> >>> > put >> > > >> >>> > >> > into >> > > >> >>> > >> > > > these caches? As always thanks in advance for any >> > > >> suggestions. >> > > >> >>> > >> > > > >> > > >> >>> > >> > > > -Jamie >> > > >> >>> > >> > > > >> > > >> >>> > >> > > >> > > >> >>> > >> > > >> > > >> >>> > >> > > >> > > >> >>> > >> > > -- >> > > >> >>> > >> > > Sincerely yours >> > > >> >>> > >> > > Mikhail Khludnev >> > > >> >>> > >> > > Principal Engineer, >> > > >> >>> > >> > > Grid Dynamics >> > > >> >>> > >> > > >> > > >> >>> > >> > > <http://www.griddynamics.com> >> > > >> >>> > >> > > <mkhlud...@griddynamics.com> >> > > >> >>> > >> > > >> > > >> >>> > >> > >> > > >> >>> > >> >> > > >> >>> > >> >> > > >> >>> > >> >> > > >> >>> > >> -- >> > > >> >>> > >> Sincerely yours >> > > >> >>> > >> Mikhail Khludnev >> > > >> >>> > >> Principal Engineer, >> > > >> >>> > >> Grid Dynamics >> > > >> >>> > >> >> > > >> >>> > >> <http://www.griddynamics.com> >> > > >> >>> > >> <mkhlud...@griddynamics.com> >> > > >> >>> > >> >> > > >> >>> > > >> > > >> >>> > >> > > >> >>> >> > > >> >> >> > > >> >> >> > > >> >> > > >> > >> >
