Re: question about ObjectCache
Hi,Andrzej Thank you for your detail answers.I have understood it uses.[?] 2012/4/10 Andrzej Bialecki > On 10/04/2012 05:00, Xiaolong Yang wrote: > >> Hi，all >> >> I'm reading source code of nutch and I have some puzzled about the >> ObjectCache.java in package org.apache.nutch.util.I just find it may be >> little benefit to use it in urlnormalizers and urlfiters.I also have >> read some discuss about cache in Nutch-169 and Nutch-501.But I can't >> understand it. >> >> Can anyone tell me where ObjectCache be used and get a good benefit in >> nutch ? >> > > ObjectCache is designed to cache ready-to-use instances of Nutch plugins. > The process of finding, instantiating and initializing plugins is > inefficient, because it involves parsing plugin descriptors, initializing > plugins, collecting the ones that implement correct extension points, etc. > > It would kill performance if this process were invoked each time you want > to run all plugins of a given type (e.g. URLNormalizer-s). The facade > URLNormalizers/URLFilters and others make sure that plugin instances of a > given type are initialized once per lifetime of a JVM, and then they are > cached in ObjectCache, so that next time you want to use them they can be > retrieved from a cache, instead of going again through the process of > parsing/instantiating/**initializing. > > -- > Best regards, > Andrzej Bialecki <>< > ___. ___ ___ ___ _ _ __** > [__ || __|__/|__||\/| Information Retrieval, Semantic Web > ___|||__|| \| || | Embedded Unix, System Integration > http://www.sigram.com Contact: info at sigram dot com > > <<349.gif>>
Re: question about ObjectCache
On 10/04/2012 05:00, Xiaolong Yang wrote: Hi，all I'm reading source code of nutch and I have some puzzled about the ObjectCache.java in package org.apache.nutch.util.I just find it may be little benefit to use it in urlnormalizers and urlfiters.I also have read some discuss about cache in Nutch-169 and Nutch-501.But I can't understand it. Can anyone tell me where ObjectCache be used and get a good benefit in nutch ? ObjectCache is designed to cache ready-to-use instances of Nutch plugins. The process of finding, instantiating and initializing plugins is inefficient, because it involves parsing plugin descriptors, initializing plugins, collecting the ones that implement correct extension points, etc. It would kill performance if this process were invoked each time you want to run all plugins of a given type (e.g. URLNormalizer-s). The facade URLNormalizers/URLFilters and others make sure that plugin instances of a given type are initialized once per lifetime of a JVM, and then they are cached in ObjectCache, so that next time you want to use them they can be retrieved from a cache, instead of going again through the process of parsing/instantiating/initializing. -- Best regards, Andrzej Bialecki <>< ___. ___ ___ ___ _ _ __ [__ || __|__/|__||\/| Information Retrieval, Semantic Web ___|||__|| \| || | Embedded Unix, System Integration http://www.sigram.com Contact: info at sigram dot com
question about ObjectCache
Hi，all I'm reading source code of nutch and I have some puzzled about the ObjectCache.java in package org.apache.nutch.util.I just find it may be little benefit to use it in urlnormalizers and urlfiters.I also have read some discuss about cache in Nutch-169 and Nutch-501.But I can't understand it. Can anyone tell me where ObjectCache be used and get a good benefit in nutch ? Appreciate your time and help.
