Hi all, @Nick: could you check out the latest code on the master branch, re-generate the settings as Dimitris suggested and try again?
We should also re-generate the settings and commit them on github, but there was a bit more to do. The problem was caused by the new 'Topic' namespace that was added on enwiki in late July 2014 [1]. It has namespace number 2600 [2], but our namespace map only went up to 1999 [3]. That's why Nick got the error message "key not found: 2600". Solution: extend the namespace map in Namespace.scala [3] to 2998. I just commited that change [4]. We could also improve the error message in WikipediaDumpParser.java [5] by using get() instead of apply(), but I don't think that's important, and maybe dealing with Options in Java would be awkward. I don't know. By the way, the Topic namespace doesn't have a talk namespace, so it might seem that we also have to change the automatic adding of talk namespaces in Namespace.scala [6], but it doesn't matter - we don't really care about the namespaces above 400 anyway. We only use dummy namespaces like "Namespace 2600". Cheers, JC [1] https://www.mediawiki.org/wiki/Flow/Rollout#1.24wmf14.2C_deployed_to_enwiki_on_2014-07-24 [2] https://en.wikipedia.org/wiki/Wikipedia:Namespace [3] https://github.com/dbpedia/extraction-framework/blob/master/core/src/main/scala/org/dbpedia/extraction/wikiparser/Namespace.scala#L70 [4] https://github.com/dbpedia/extraction-framework/commit/fdad42edb0ab3e854e410129e226b4e2bf627517 [5] https://github.com/dbpedia/extraction-framework/blob/master/core/src/main/java/org/dbpedia/extraction/sources/WikipediaDumpParser.java#L219 [6] https://github.com/dbpedia/extraction-framework/blob/master/core/src/main/scala/org/dbpedia/extraction/wikiparser/Namespace.scala#L47 On Sep 23, 2014 7:58 AM, "Dimitris Kontokostas" <jimk...@gmail.com> wrote: > > Hi Nick, > > Can you try to re-generate the settings [1] and give it another go? > This should be the problem but let's wait for your feedback. > > Cheers, > Dimitris > > [1] > https://github.com/dbpedia/extraction-framework/wiki/Extraction-Instructions#generate-settings > > On Mon, Sep 22, 2014 at 7:54 PM, Blotto <blo...@gmail.com> wrote: >> >> Hi Alexandru. >> >> This happens on any of the extractors I use for example.. >> >> ../run extraction extraction.default.properties >> ../run extraction extraction.topical.properties >> ../run extraction extraction.abstracts.properties >> >> I've tried with the -X options as well, but get mostly verbose [INFO] log >> entries. >> >> Cheers, >> Nick >> >> On Sep 22, 2014, at 9:50 AM, Alexandru Todor <to...@inf.fu-berlin.de> wrote: >> >> Hi Nick, >> >> Your error log is to short to deduce the necessary info out of it. >> Are you trying to run the abstract extraction, which extractors are you >> using ? >> >> Cheers, >> Alexandru >> >> >> On 09/22/2014 06:40 PM, Blotto wrote: >> >> Hi all, >> >> I've exhausted my own troubleshooting on this issue below, and hope someone >> can point me in the right direction for next steps. This is the first time >> I am trying to do the extractions locally and have followed all directions I >> can find. I have tried the latest code from the master branch on the >> extraction-framework as well as the DBpedia Release 3.9 to no avail. My >> next try is an older version of the wikipedia dumps ( I have used the August >> and September 2014 dumps to date). >> >> After about 12 -13 hours of running an extraction I always get a >> "NoSuchElementException" failure at "Queen Isabel II Statue". Querying my >> local wikipedia db "select * from page where page_title like >> "Queen_Isabel_II_Statue_(Manila)" and page_namespace =0" does return a >> record. >> >> >> Thanks in advance, >> Nick >> >> >> found by dbpedia: [title=Queen Isabel II Statue, >> Manila;ns=0/Main/;language:wiki=en,locale=en]. >> >> found by wikipedia: [title=Queen Isabel II Statue >> (Manila);ns=0/Main/;language:wiki=en,locale=en] >> >> java.lang.reflect.InvocationTargetException >> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> >> at java.lang.reflect.Method.invoke(Method.java:606) >> >> at >> org_scala_tools_maven_executions.MainHelper.runMain(MainHelper.java:161) >> >> at >> org_scala_tools_maven_executions.MainWithArgsInFile.main(MainWithArgsInFile.java:26) >> >> Caused by: java.util.NoSuchElementException: key not found: 2600 >> >> at scala.collection.MapLike$class.default(MapLike.scala:228) >> >> at scala.collection.AbstractMap.default(Map.scala:58) >> >> at scala.collection.MapLike$class.apply(MapLike.scala:141) >> >> at scala.collection.AbstractMap.apply(Map.scala:58) >> >> at >> org.dbpedia.extraction.sources.WikipediaDumpParser.readPage(WikipediaDumpParser.java:219) >> >> at >> org.dbpedia.extraction.sources.WikipediaDumpParser.readPages(WikipediaDumpParser.java:187) >> >> at >> org.dbpedia.extraction.sources.WikipediaDumpParser.readDump(WikipediaDumpParser.java:145) >> >> at >> org.dbpedia.extraction.sources.WikipediaDumpParser.run(WikipediaDumpParser.java:116) >> >> at >> org.dbpedia.extraction.sources.XMLReaderSource.foreach(XMLSource.scala:65) >> >> at >> scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251) >> >> at >> org.dbpedia.extraction.sources.XMLReaderSource.flatMap(XMLSource.scala:61) >> >> at >> org.dbpedia.extraction.mappings.Redirects$.loadFromSource(Redirects.scala:165) >> >> at >> org.dbpedia.extraction.mappings.Redirects$.load(Redirects.scala:116) >> >> at >> org.dbpedia.extraction.dump.extract.ConfigLoader$$anon$1.<init>(ConfigLoader.scala:97) >> >> at >> org.dbpedia.extraction.dump.extract.ConfigLoader.org$dbpedia$extraction$dump$extract$ConfigLoader$$createExtractionJob(ConfigLoader.scala:52) >> >> at >> org.dbpedia.extraction.dump.extract.ConfigLoader$$anonfun$getExtractionJobs$1.apply(ConfigLoader.scala:37) >> >> at >> org.dbpedia.extraction.dump.extract.ConfigLoader$$anonfun$getExtractionJobs$1.apply(ConfigLoader.scala:37) >> >> at >> scala.collection.GenTraversableViewLike$Mapped$$anonfun$foreach$2.apply(GenTraversableViewLike.scala:81) >> >> at scala.collection.Iterator$class.foreach(Iterator.scala:727) >> >> at >> scala.collection.immutable.RedBlackTree$TreeIterator.foreach(RedBlackTree.scala:422) >> >> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) >> >> at >> scala.collection.IterableLike$$anon$1.foreach(IterableLike.scala:294) >> >> at >> scala.collection.GenTraversableViewLike$Mapped$class.foreach(GenTraversableViewLike.scala:80) >> >> at >> scala.collection.IterableViewLike$$anon$3.foreach(IterableViewLike.scala:84) >> >> at >> org.dbpedia.extraction.dump.extract.Extraction$.main(Extraction.scala:30) >> >> at >> org.dbpedia.extraction.dump.extract.Extraction.main(Extraction.scala) >> >> ... 6 more >> >> >> >> ------------------------------------------------------------------------------ >> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer >> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports >> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper >> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer >> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk >> >> >> >> _______________________________________________ >> Dbpedia-discussion mailing list >> Dbpedia-discussion@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion >> >> >> >> >> ------------------------------------------------------------------------------ >> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer >> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports >> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper >> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer >> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk >> _______________________________________________ >> Dbpedia-discussion mailing list >> Dbpedia-discussion@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion >> > > > > -- > Kontokostas Dimitris > > ------------------------------------------------------------------------------ > Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer > Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports > Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper > Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer > http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk > _______________________________________________ > Dbpedia-discussion mailing list > Dbpedia-discussion@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion > ------------------------------------------------------------------------------ Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk _______________________________________________ Dbpedia-discussion mailing list Dbpedia-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
