On Thu, Dec 13, 2012 at 2:31 PM, Emmanuel Lécharny <elecha...@gmail.com>wrote:
> Le 12/11/12 2:09 PM, Kiran Ayyagari a écrit : > > On Tue, Dec 11, 2012 at 4:11 PM, Emmanuel Lécharny <elecha...@gmail.com > >wrote: > > > >> Le 12/11/12 11:02 AM, Kiran Ayyagari a écrit : > >>> On Tue, Dec 11, 2012 at 3:23 PM, Emmanuel Lécharny < > elecha...@gmail.com > >>> wrote: > >>> > >>>> But, and there is a big but, when processing a search request, we do a > >>>> direct lookup from the backend, and we are not going through this > chain. > >>>> What happens is that we are building a list of filters, and we apply > >>>> this list of filters on every entry fetched from the backend doing a > >>>> direct lookup. So the filter added by the SchemaInterceptor should > also > >>>> do this filtering. > >>>> > >>>> as the filtering is only applicable for search(and lookup) why not > apply > >>> this logic in the cursor > >>> after evaluating an entry, I know that this filtering logic was > scattered > >>> before but with all that > >>> moved to a utility class I think using that inside the cursor is the > >> right > >>> thing rather than doing it in an > >>> interceptor > >> Because Lookup does not use a cursor at all :/ OTOH, cursors are based > >> on lookup (but not the same as for the lookup() operation. > > but unlike in lookup(LookupOperationContext) we don't need to filter > anything in lookup(id) method > > Yep, true. > > Btw, I'm done with the Lookup Cleanup (well, I hope). > > We don't filter the attributes in the backend anymore, it's all done in > the SchemaInterceptor. > > In the process, I discovered that a lookup going throug the ACI > interceptor was not going through the chain, as we were doing a direct > fetch from the backend, bypassing *all* the interceptors :/ > That says two things : > o we don't have a decent test coverage when it comes to ACI > o it's not because some code is 7 years old that it's bug free ! :) > > As a direct consequence of the cleanup, a lookup operation is now 10% > faster. Some numbers : > - a getRootDSE, no filtering : 242 000 lookup/s > - a lookup on rootDSE, all user attrs (*) : 148 544 lookup/s > - a lookup on rootDSE, all attributes (*, +) : 145 243 lookup/s > - a lookup on an entry, all user attributes (*) : 144 906 lookup/s > - a lookup on an entry, all attributes (*, +) : 140 154 lookup/s > - a lookup on an entry, all atributes, with ACI (*, +) : 147 362 lookup/s > > 73% of the time is used to create a clone of the entry we will return, > 10% to fetch the entry ID from the backend using the lookup DN, and 15% > to filter the attributes. It's pretty optimal at this point with 98% of > the processing done in areas we barelly can optimize :) > > great, thanks for the heads up > One potential optimization at this point would be to add a parallel data > structure to the entry we fetch, containing flags set to true for > Attributes we want to return and false for those that are filtered > outout. That would spare us the cost of cloning the whole entry, a very > costly operation. It will impact the way we write the response in the > encoder, but I'm quite sure that the gain could be substancial... > > I still have to check that lookups on subschemaSubentry is ok, and check > the lookup through the ACI subsystem. > > The next step will be to cleanup the search. > > -- > Regards, > Cordialement, > Emmanuel Lécharny > www.iktek.com > > -- Kiran Ayyagari http://keydap.com
