----- Original Message -----
From: "Doğacan Güney" <[EMAIL PROTECTED]>
Sent: Friday, June 08, 2007 3:49 PM
[...]
>> Any idea?
>
> This will certainly help a lot. If it is not too much trouble, can you
> add debug outputs for hashCodes of conf objects (both for the one in
> the cache and for the parameter, because it seems Configuration object
> is created more than once so their hashCode may be different, which in
> turn causes the change in CACHE's hashCode(*)) and a stack trace?
> A stack trace of depth 2-3 will probably suffice, I am just wondering
> what is calling PluginRepository.get(conf).
OK, I changed my debug code as follows:
public static synchronized PluginRepository get(Configuration conf) {
PluginRepository result = CACHE.get(conf);
/* --- start debug code */
String tr = "";
StackTraceElement[] tes = Thread.currentThread().getStackTrace();
for(int j=2; j<tes.length; j++)
tr = tr+"\n "+tes[j].toString();
LOG.info("In thread "+Thread.currentThread()+
" a static method of the class "+
(new CurrentClassGetter()).getCurrentClass()+
" called CACHE.get("+conf+
"), where CACHE is "+CACHE+
" and CACHE.hashCode() = "+CACHE.hashCode()+
" - got result = "+result+
" conf.hashCode() was: "+conf.hashCode()+
" hashCode was: "+conf.hashCode()+
" Stack Trace:"+tr);
/* end debug code --- */
if (result == null) {
result = new PluginRepository(conf);
CACHE.put(conf, result);
}
return result;
}
/* --- start debug code */
public static class CurrentClassGetter extends SecurityManager {
public String getCurrentClass() {
Class cl = super.getClassContext()[1];
return cl.toString() + "@" + cl.hashCode();
}
}
/* end debug code --- */
(With full stack trace: bytes are cheap ;-) )
I did not bother to print the hashCode of the keys in CACHE because it's
become evident why CACHE.get(conf) returns null: the hashCode of conf
changes! This is strange, because, as you can see below, the strings that
make keys and values of conf appears unchanged. Perhaps we should override
the equals() method in org.apache.hadoop.conf.Configuration (invoked by
CACHE.get(), according to the specs of the java.util.Map interface), so that
the hashCode()s of the keys get ignored, and conf1.equals(conf2) return true
if and only if:
1. conf1.size() == conf2.size(),
2. for each key k1 of conf1 there is a key k2 in conf2 such as:
2.1 k1.equals(k2)
2.2 conf1.get(k1).equals(conf2.get(k2))
Anyway, I'm attaching the log below.
> Thanks for the detailed analysis!
Glad to be of help!
Enzo
2007-06-08 17:24:39,211 INFO plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 23315571 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
27058272 Stack Trace:
org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
org.apache.nutch.plugin.PluginDescriptor.collectLibs(PluginDescriptor.java:309)
org.apache.nutch.plugin.PluginDescriptor.getDependencyLibs(PluginDescriptor.java:298)
org.apache.nutch.plugin.PluginDescriptor.getClassLoader(PluginDescriptor.java:277)
org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:155)
org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)
2007-06-08 17:24:39,231 INFO plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 23315571 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
27058272 Stack Trace:
org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
org.apache.nutch.urlfilter.geoip.GeoIpURLFilter.setConf(GeoIpURLFilter.java:252)
org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:162)
org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)
2007-06-08 17:24:39,802 INFO plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 23315571 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
27058272 Stack Trace:
org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
org.apache.nutch.plugin.PluginDescriptor.collectLibs(PluginDescriptor.java:309)
org.apache.nutch.plugin.PluginDescriptor.getDependencyLibs(PluginDescriptor.java:298)
org.apache.nutch.plugin.PluginDescriptor.getClassLoader(PluginDescriptor.java:277)
org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:155)
org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)
2007-06-08 17:24:39,802 INFO plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 23315571 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
27058272 Stack Trace:
org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
org.apache.nutch.plugin.PluginDescriptor.collectLibs(PluginDescriptor.java:309)
org.apache.nutch.plugin.PluginDescriptor.getDependencyLibs(PluginDescriptor.java:298)
org.apache.nutch.plugin.PluginDescriptor.getClassLoader(PluginDescriptor.java:277)
org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:155)
org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)
2007-06-08 17:24:43,618 INFO plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 23315571 - got result = null conf.hashCode() was: 7461949
Stack Trace:
org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
org.apache.nutch.net.URLFilters.<init>(URLFilters.java:46)
org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)
2007-06-08 17:24:43,848 INFO plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 54703198 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
7461949 Stack Trace:
org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
org.apache.nutch.plugin.PluginDescriptor.collectLibs(PluginDescriptor.java:309)
org.apache.nutch.plugin.PluginDescriptor.getDependencyLibs(PluginDescriptor.java:298)
org.apache.nutch.plugin.PluginDescriptor.getClassLoader(PluginDescriptor.java:277)
org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:155)
org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)
2007-06-08 17:24:43,858 INFO plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 54703198 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
7461949 Stack Trace:
org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
org.apache.nutch.urlfilter.geoip.GeoIpURLFilter.setConf(GeoIpURLFilter.java:252)
org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:162)
org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)
2007-06-08 17:24:43,918 INFO plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 54703198 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
7461949 Stack Trace:
org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
org.apache.nutch.plugin.PluginDescriptor.collectLibs(PluginDescriptor.java:309)
org.apache.nutch.plugin.PluginDescriptor.getDependencyLibs(PluginDescriptor.java:298)
org.apache.nutch.plugin.PluginDescriptor.getClassLoader(PluginDescriptor.java:277)
org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:155)
org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)
2007-06-08 17:24:43,918 INFO plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 54703198 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
7461949 Stack Trace:
org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
org.apache.nutch.plugin.PluginDescriptor.collectLibs(PluginDescriptor.java:309)
org.apache.nutch.plugin.PluginDescriptor.getDependencyLibs(PluginDescriptor.java:298)
org.apache.nutch.plugin.PluginDescriptor.getClassLoader(PluginDescriptor.java:277)
org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:155)
org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)
2007-06-08 17:24:44,299 INFO plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 54703198 - got result = null conf.hashCode() was:
19647819 Stack Trace:
org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
org.apache.nutch.net.URLFilters.<init>(URLFilters.java:46)
org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)
2007-06-08 17:24:44,499 INFO plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 64604955 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
19647819 Stack Trace:
org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
org.apache.nutch.plugin.PluginDescriptor.collectLibs(PluginDescriptor.java:309)
org.apache.nutch.plugin.PluginDescriptor.getDependencyLibs(PluginDescriptor.java:298)
org.apache.nutch.plugin.PluginDescriptor.getClassLoader(PluginDescriptor.java:277)
org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:155)
org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)
2007-06-08 17:24:44,509 INFO plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 64604955 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
19647819 Stack Trace:
org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
org.apache.nutch.urlfilter.geoip.GeoIpURLFilter.setConf(GeoIpURLFilter.java:252)
org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:162)
org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)
2007-06-08 17:24:44,599 INFO plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 64604955 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
19647819 Stack Trace:
org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
org.apache.nutch.plugin.PluginDescriptor.collectLibs(PluginDescriptor.java:309)
org.apache.nutch.plugin.PluginDescriptor.getDependencyLibs(PluginDescriptor.java:298)
org.apache.nutch.plugin.PluginDescriptor.getClassLoader(PluginDescriptor.java:277)
org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:155)
org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)
2007-06-08 17:24:44,599 INFO plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 64604955 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
19647819 Stack Trace:
org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
org.apache.nutch.plugin.PluginDescriptor.collectLibs(PluginDescriptor.java:309)
org.apache.nutch.plugin.PluginDescriptor.getDependencyLibs(PluginDescriptor.java:298)
org.apache.nutch.plugin.PluginDescriptor.getClassLoader(PluginDescriptor.java:277)
org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:155)
org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)
2007-06-08 17:25:22,694 INFO plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 64604955 - got result = null conf.hashCode() was:
27692793 Stack Trace:
org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
org.apache.nutch.scoring.ScoringFilters.<init>(ScoringFilters.java:59)
org.apache.nutch.crawl.CrawlDbReducer.configure(CrawlDbReducer.java:46)
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:217)
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:155)
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
