@Override
public void collect(int doc) throws IOException {
docNumsByIds.put(context.reader().doc(doc).getField("id"), doc);
// don't call delegate
}
@Override
protected void doSetNextReader(LeafReaderContext context) throws
IOException {
flush(); // strictly before the next segment
super.doSetNextReader(context)
}
void flush(){
Set<String> allowedIDs = callRedisToRetainAllowed(docNumsByIds.keys())
docNumsByIds.keys().retainAll(allowedIDs )
for(int docNum:docNumsByIds.values()) {
delegate.collect(docNum);
}
}
public void finish() throws IOException {
flush();
super.finish()
}
On Thu, Nov 29, 2018 at 2:36 PM John Thorhauer <jthorha...@yakabod.com>
wrote:
> So my understanding is that the DelegatingCollector.collect() method has
> access to a single doc. At that point I must choose to either call
> super.collect() or not. So this is the point at which I have to check
> redis for security data for a single doc and determine if this doc should
> be allowed as part of the result set or not. So it seems that I have to
> check my redis cache one doc at a time since I am only provided one doc in
> the collect() method and I must determine at this point if I should call
> the super.collect() or not.
>
> I would like to find an option where I can get all the docs in the
> postfilter and run a single query to redis with all of the docs at once to
> get a single answer back from redis and then determine, based on the redis
> response, which of the docs should be allowed to pass thru my postfilter.
>
>
>
>
> On Fri, Nov 16, 2018 at 4:30 PM Mikhail Khludnev <m...@apache.org> wrote:
>
> > On Tue, Nov 13, 2018 at 6:36 AM John Thorhauer <jthorha...@yakabod.com>
> > wrote:
> >
> > > Mikhail,
> > >
> > > Where do I implement the buffering? I can not do it in then collect()
> > > method.
> >
> > Please clarify why exactly? Notice my statement about one segment only.
> >
> >
> > > I can not see how I can get access to what I need in the finish()
> > > method.
> > >
> > > Thanks,
> > > John
> > >
> > > On Tue, Nov 6, 2018 at 12:44 PM Mikhail Khludnev <m...@apache.org>
> > wrote:
> > >
> > > > Not really. It expect to work segment by segment. So it can buffer
> all
> > > doc
> > > > from one segment, hit redis and push all results into delegating
> > > collector.
> > > >
> > > > On Tue, Nov 6, 2018 at 8:29 PM John Thorhauer <
> jthorha...@yakabod.com>
> > > > wrote:
> > > >
> > > > > Mikhail,
> > > > >
> > > > > Thanks for the suggestion. After looking over the PostFilter
> > interface
> > > > and
> > > > > the DelegatingCollector, it appears that this would require me to
> > query
> > > > my
> > > > > outside datastore (redis) for security information once for each
> > > > document.
> > > > > This would be a big performance issue. I would like to be able to
> > > > iterate
> > > > > through the documents, gathering all the critical ID's and then
> send
> > a
> > > > > single query to redis, getting back my security related data, and
> > then
> > > > > iterate through the documents, pulling out the ones that the user
> > > should
> > > > > not see.
> > > > >
> > > > > Is this possible?
> > > > >
> > > > > Thanks again for your help!
> > > > > John
> > > > >
> > > > >
> > > > > On Tue, Nov 6, 2018 at 6:24 AM John Thorhauer <
> > jthorha...@yakabod.com>
> > > > > wrote:
> > > > >
> > > > > > We have a need to check the results of a search against a set of
> > > > security
> > > > > > lists that are maintained in a redis cache. I need to be able to
> > > take
> > > > > each
> > > > > > document that is returned for a search and check the redis cache
> to
> > > see
> > > > > if
> > > > > > the document should be displayed or not.
> > > > > >
> > > > > > I am attempting to do this by creating a SearchComponent. I am
> > able
> > > to
> > > > > > iterate thru the results and identify the items I want to remove
> > from
> > > > the
> > > > > > results but I am not sure how to proceed in removing them.
> > > > > >
> > > > > > Is SearchComponent the best way to do this? If so, any thoughts
> on
> > > how
> > > > > to
> > > > > > proceed?
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > > John Thorhauer
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > John Thorhauer
> > > > > Vice President, Software Development
> > > > > Yakabod, Inc.
> > > > > Cell: 240-818-9050
> > > > > Office: 301-662-4554 x2105
> > > > >
> > > >
> > > >
> > > > --
> > > > Sincerely yours
> > > > Mikhail Khludnev
> > > >
> > >
> > >
> > > --
> > > John Thorhauer
> > > Vice President, Software Development
> > > Yakabod, Inc.
> > > Cell: 240-818-9050
> > > Office: 301-662-4554 x2105
> > >
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev
> >
>
--
Sincerely yours
Mikhail Khludnev
