Hi, Stefan, On Sat, Jul 17, 2004 at 11:52:47PM +0200, Stefan Groschupf wrote: > Do you use Extension.getExtensionInstance() or / and > PluginRepository.getPluginInstance(PluginDescriptor pDescriptor) to > access the plugin? > Or you may be create own instances of the HttpPlugin.java? > As you can see in PluginRepository.getPluginInstance(.. ) the plugin > instance is cached.
I have tried all those things and the problem persists. > May you discover a bug and we need to synchronized access the plugin > instance cache in the PluginRepository. > I would say there is the problem... > Your code is ok. Can you commit it to the cvs and provide a unit test, > i will try to fix the problem immediately. The easiest may be (1) check out current cvs (2) add HttpPlugin.java to ./nutch/src/plugin/protocol-http/src/java/net/nutch/protocol/http/ (3) overwrite ./nutch/src/plugin/protocol-http/plugin.xml using my plugin.xml Both HttpPlugin.java and plugin.xml are in my previous email. It may not be a good idea to commit them for trouble-shooting purpose. Or for the best, I suggest adding a multi-threaded test in ./src/test/net/nutch/plugin/TestPluginSystem.java It will make the problem evident and provide further assurance. Multithreading is the primary usage mode of plugins in nutch. Thanks, John > > > Am 17.07.2004 um 23:54 schrieb [EMAIL PROTECTED]: > > >On Tue, Jul 13, 2004 at 11:56:16PM +0200, Stefan Groschupf wrote: > >> > >>>(1) in protocol-ftp, I would like to keep per-host connection > >>>PERSISTENT as long as the fetcher is hitting the same host. > >>>This can be extremely useful for ftp sites that are expensive for > >>>initial logins. In pre-plugin nutch, it was archived by passing > >>>handler > >>>ftp.client from Ftp.java to FtpResponse.java. However, with new > >>>plugin > >>>system, > >>>this object seemly gets garbage collected each time a url is visited, > >>>i.e., > >>>connection has to be reinitiated for next url. A huge headache. > >> > >>That is why there is a Plugin class you normally need to implement > >>within a plugin. Doug later remove this requirement. > >>Until runtime it exist only one instance of the object so it is a kind > >>of singleton. > >>Your cache should be hosted or accessed over your plugin Plugin.java > >>implementation. > >>Take a look in the snmp plugin i had submitted it iluustrate what you > >>need to do. Dont miss to setup the plugin class in the plugin.xml > >>manifest file. > >> > > > >Hi, Stefan, > > > >Using current cvs tree, I added in an HttpPlugin.java and modified > >protocol-http's plugin.xml accordingly. Both are attached at the end > >of this message. Nothing else in source tree was changed. > >I then tried to run the fetcher a few times, using multiple threads, > >on a small segment with a dozen of urls. Its behavior is puzzling: > > > >(1) There is no consistent number of start up of HttpPlugin.java, > >as indicated by logging messages such as: > >040717 135633 11 start up Plugin: 11 protocol-http > >040717 135633 12 start up Plugin: 12 protocol-http > >...... > > > >For runs using 4 threads, I observed HttpPlugin.java > >started any number btween 1-4 times. > >The same is true for runs using 6, 8, 10 threads. > > > >(2) Number of "shutdown" of HttpPlugin.java could be none or > >any number up to num_threads minus 1. > > > >Assuming HttpPlugin.java (Plugin.java) is of singleton pattern, > >wouldn't we > >expect its startup or shutdown only once? > > > >Or am I missing something? > > > >John > > ------------------------------------------------------- This SF.Net email is sponsored by BEA Weblogic Workshop FREE Java Enterprise J2EE developer tools! Get your free copy of BEA WebLogic Workshop 8.1 today. http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click _______________________________________________ Nutch-developers mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/nutch-developers
