Final update, extractions work now with the latest pull and a clean
install. Thanks for your help!
N
On Wed, Sep 24, 2014 at 12:29 PM, Blotto <blo...@gmail.com> wrote:
> Running extractions now with clean install. My mistake, incorrect command
> before didn't actually kick off the extractions. :)
> N
>
> On Wed, Sep 24, 2014 at 12:24 PM, Blotto <blo...@gmail.com> wrote:
>
>> Ah, yea my oversight. I now reran with the clean-install-run script,
>> and it now shows success, but it doesn't seem to do any extraction it
>> finishes in only a few minutes. The only thing that stands out to me is
>> "[WARNING] Not mainClass or valid launcher found/define" near the end of
>> the summary.
>>
>>
>> I do see many of warnings as such:
>>
>> [WARNING]
>>
>> [WARNING]
>> /mnt/frameworks/extraction-framework/core/src/main/scala/org/dbpedia/extraction/mappings/wikidata/WikidataDescriptionExtractor.scala:40:
>> warning: reflective access of structural type member method ontology should
>> be enabled
>>
>> [WARNING] by making the implicit value scala.language.reflectiveCalls
>> visible.
>>
>> [WARNING] case Some(dbpedia_lang) => quads += new
>> Quad(dbpedia_lang, DBpediaDatasets.WikidataDescription, subjectUri,
>> aliasProperty,description, page.wikiPage.sourceUri,
>> context.ontology.datatypes("rdf:langString"))
>>
>> [WARNING]
>>
>>
>>
>> Heres the summary of the process :
>>
>>
>> [INFO]
>> ------------------------------------------------------------------------
>>
>> [INFO] Reactor Summary:
>>
>> [INFO]
>>
>> [INFO] Parent POM of the DBpedia framework ............... SUCCESS
>> [1.215s]
>>
>> [INFO] DBpedia Core Libraries ............................ SUCCESS
>> [1:33.725s]
>>
>> [INFO] DBpedia Server .................................... SUCCESS
>> [23.359s]
>>
>> [INFO] DBpedia Dump Extraction ........................... SUCCESS
>> [12.584s]
>>
>> [INFO] DBpedia Scripts ................................... SUCCESS
>> [17.912s]
>>
>> [INFO] Live extraction ................................... SUCCESS
>> [20.547s]
>>
>> [INFO] Wiktionary Dump ................................... SUCCESS
>> [22.118s]
>>
>> [INFO]
>> ------------------------------------------------------------------------
>>
>> [INFO] BUILD SUCCESS
>>
>> [INFO]
>> ------------------------------------------------------------------------
>>
>> [INFO] Total time: 3:11.751s
>>
>> [INFO] Finished at: Wed Sep 24 19:14:49 UTC 2014
>>
>> [INFO] Final Memory: 47M/930M
>>
>> [INFO]
>> ------------------------------------------------------------------------
>>
>> [INFO] Scanning for projects...
>>
>> [INFO]
>>
>>
>> [INFO]
>> ------------------------------------------------------------------------
>>
>> [INFO] Building DBpedia Dump Extraction 4.0-SNAPSHOT
>>
>> [INFO]
>> ------------------------------------------------------------------------
>>
>> [INFO]
>>
>> [INFO] >>> scala-maven-plugin:3.1.6:run (default-cli) @ dump >>>
>>
>> [INFO]
>>
>> [INFO] --- maven-enforcer-plugin:1.3.1:enforce (default) @ dump ---
>>
>> 2.565: [GC 122880K->12475K(470528K), 0.0155540 secs]
>>
>> [INFO]
>>
>> [INFO] --- maven-resources-plugin:2.3:resources (default-resources) @
>> dump ---
>>
>> [INFO] Using 'UTF-8' encoding to copy filtered resources.
>>
>> [INFO] Copying 0 resource
>>
>> [INFO]
>>
>> [INFO] --- scala-maven-plugin:3.1.6:compile (process-resources) @ dump ---
>>
>> [INFO] Nothing to compile - all classes are up to date
>>
>> [INFO]
>>
>> [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ dump ---
>>
>> [INFO] No sources to compile
>>
>> [INFO]
>>
>> [INFO] --- scala-maven-plugin:3.1.6:compile (compile) @ dump ---
>>
>> [INFO] Nothing to compile - all classes are up to date
>>
>> [INFO]
>>
>> [INFO] --- maven-resources-plugin:2.3:testResources
>> (default-testResources) @ dump ---
>>
>> [INFO] Using 'UTF-8' encoding to copy filtered resources.
>>
>> [INFO] Copying 0 resource
>>
>> [INFO]
>>
>> [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @
>> dump ---
>>
>> [INFO] No sources to compile
>>
>> [INFO]
>>
>> [INFO] --- scala-maven-plugin:3.1.6:testCompile (test-compile) @ dump ---
>>
>> [INFO] No sources to compile
>>
>> [INFO]
>>
>> [INFO] <<< scala-maven-plugin:3.1.6:run (default-cli) @ dump <<<
>>
>> [INFO]
>>
>> [INFO] --- scala-maven-plugin:3.1.6:run (default-cli) @ dump ---
>>
>> [WARNING] Not mainClass or valid launcher found/define
>>
>> [INFO]
>> ------------------------------------------------------------------------
>>
>> [INFO] BUILD SUCCESS
>>
>> [INFO]
>> ------------------------------------------------------------------------
>>
>> [INFO] Total time: 2.330s
>>
>> [INFO] Finished at: Wed Sep 24 19:14:53 UTC 2014
>>
>> 3.893: [GC 135355K->11062K(470528K), 0.0149900 secs]
>>
>> 3.908: [Full GC 11062K->10860K(470528K), 0.1144960 secs]
>>
>> [INFO] Final Memory: 12M/459M
>>
>> [INFO]
>> ------------------------------------------------------------------------
>>
>>
>>
>>
>>
>> On Wed, Sep 24, 2014 at 12:12 AM, Dimitris Kontokostas <jimk...@gmail.com
>> > wrote:
>>
>>> just to make sure in order to investigate further
>>>
>>> did you run
>>> $../clean-install-run extraction ...
>>> before you try again
>>>
>>> or did you just run
>>> $../run extraction ...
>>>
>>> If it;s the latter, changes in core did not compile when you run from
>>> the dump module
>>>
>>> On Wed, Sep 24, 2014 at 7:53 AM, Blotto <blo...@gmail.com> wrote:
>>>
>>>> Thanks Dimitris, Jona! I’ve pulled latest and re-generated settings.
>>>> Extraction still fails in the same place.
>>>>
>>>> "Caused by: java.util.NoSuchElementException: key not found: 2600"
>>>>
>>>> After generating settings , is there any additional step to do prior to
>>>> running extractions?
>>>>
>>>> Cheers,
>>>>
>>>> Nick
>>>>
>>>> On Tue, Sep 23, 2014 at 6:31 AM, Jona Christopher Sahnwaldt <
>>>> j...@sahnwaldt.de> wrote:
>>>>
>>>>> P.S.: By the way, the "Queen Isabel II Statue" has nothing to do with
>>>>> this error. That's just a warning that is printed because a redirect
>>>>> page cannot be parsed correctly. It just happens to be printed just
>>>>> before the error occurs. ;-)
>>>>>
>>>>> On 23 September 2014 15:25, Jona Christopher Sahnwaldt <
>>>>> j...@sahnwaldt.de> wrote:
>>>>> > Hi all,
>>>>> >
>>>>> > @Nick: could you check out the latest code on the master branch,
>>>>> > re-generate the settings as Dimitris suggested and try again?
>>>>> >
>>>>> > We should also re-generate the settings and commit them on github,
>>>>> but
>>>>> > there was a bit more to do. The problem was caused by the new 'Topic'
>>>>> > namespace that was added on enwiki in late July 2014 [1]. It has
>>>>> > namespace number 2600 [2], but our namespace map only went up to 1999
>>>>> > [3]. That's why Nick got the error message "key not found: 2600".
>>>>> >
>>>>> > Solution: extend the namespace map in Namespace.scala [3] to 2998. I
>>>>> > just commited that change [4].
>>>>> >
>>>>> > We could also improve the error message in WikipediaDumpParser.java
>>>>> > [5] by using get() instead of apply(), but I don't think that's
>>>>> > important, and maybe dealing with Options in Java would be awkward. I
>>>>> > don't know.
>>>>> >
>>>>> > By the way, the Topic namespace doesn't have a talk namespace, so it
>>>>> > might seem that we also have to change the automatic adding of talk
>>>>> > namespaces in Namespace.scala [6], but it doesn't matter - we don't
>>>>> > really care about the namespaces above 400 anyway. We only use dummy
>>>>> > namespaces like "Namespace 2600".
>>>>> >
>>>>> > Cheers,
>>>>> > JC
>>>>> >
>>>>> > [1]
>>>>> https://www.mediawiki.org/wiki/Flow/Rollout#1.24wmf14.2C_deployed_to_enwiki_on_2014-07-24
>>>>> > [2] https://en.wikipedia.org/wiki/Wikipedia:Namespace
>>>>> > [3]
>>>>> https://github.com/dbpedia/extraction-framework/blob/master/core/src/main/scala/org/dbpedia/extraction/wikiparser/Namespace.scala#L70
>>>>> > [4]
>>>>> https://github.com/dbpedia/extraction-framework/commit/fdad42edb0ab3e854e410129e226b4e2bf627517
>>>>> > [5]
>>>>> https://github.com/dbpedia/extraction-framework/blob/master/core/src/main/java/org/dbpedia/extraction/sources/WikipediaDumpParser.java#L219
>>>>> > [6]
>>>>> https://github.com/dbpedia/extraction-framework/blob/master/core/src/main/scala/org/dbpedia/extraction/wikiparser/Namespace.scala#L47
>>>>> >
>>>>> > On Sep 23, 2014 7:58 AM, "Dimitris Kontokostas" <jimk...@gmail.com>
>>>>> wrote:
>>>>> >>
>>>>> >> Hi Nick,
>>>>> >>
>>>>> >> Can you try to re-generate the settings [1] and give it another go?
>>>>> >> This should be the problem but let's wait for your feedback.
>>>>> >>
>>>>> >> Cheers,
>>>>> >> Dimitris
>>>>> >>
>>>>> >> [1]
>>>>> https://github.com/dbpedia/extraction-framework/wiki/Extraction-Instructions#generate-settings
>>>>> >>
>>>>> >> On Mon, Sep 22, 2014 at 7:54 PM, Blotto <blo...@gmail.com> wrote:
>>>>> >>>
>>>>> >>> Hi Alexandru.
>>>>> >>>
>>>>> >>> This happens on any of the extractors I use for example..
>>>>> >>>
>>>>> >>> ../run extraction extraction.default.properties
>>>>> >>> ../run extraction extraction.topical.properties
>>>>> >>> ../run extraction extraction.abstracts.properties
>>>>> >>>
>>>>> >>> I've tried with the -X options as well, but get mostly verbose
>>>>> [INFO] log entries.
>>>>> >>>
>>>>> >>> Cheers,
>>>>> >>> Nick
>>>>> >>>
>>>>> >>> On Sep 22, 2014, at 9:50 AM, Alexandru Todor <
>>>>> to...@inf.fu-berlin.de> wrote:
>>>>> >>>
>>>>> >>> Hi Nick,
>>>>> >>>
>>>>> >>> Your error log is to short to deduce the necessary info out of it.
>>>>> >>> Are you trying to run the abstract extraction, which extractors
>>>>> are you using ?
>>>>> >>>
>>>>> >>> Cheers,
>>>>> >>> Alexandru
>>>>> >>>
>>>>> >>>
>>>>> >>> On 09/22/2014 06:40 PM, Blotto wrote:
>>>>> >>>
>>>>> >>> Hi all,
>>>>> >>>
>>>>> >>> I've exhausted my own troubleshooting on this issue below, and
>>>>> hope someone can point me in the right direction for next steps. This is
>>>>> the first time I am trying to do the extractions locally and have followed
>>>>> all directions I can find. I have tried the latest code from the master
>>>>> branch on the extraction-framework as well as the DBpedia Release 3.9 to
>>>>> no
>>>>> avail. My next try is an older version of the wikipedia dumps ( I have
>>>>> used the August and September 2014 dumps to date).
>>>>> >>>
>>>>> >>> After about 12 -13 hours of running an extraction I always get a
>>>>> "NoSuchElementException" failure at "Queen Isabel II Statue". Querying
>>>>> my
>>>>> local wikipedia db "select * from page where page_title like
>>>>> "Queen_Isabel_II_Statue_(Manila)" and page_namespace =0" does return a
>>>>> record.
>>>>> >>>
>>>>> >>>
>>>>> >>> Thanks in advance,
>>>>> >>> Nick
>>>>> >>>
>>>>> >>>
>>>>> >>> found by dbpedia: [title=Queen Isabel II Statue,
>>>>> Manila;ns=0/Main/;language:wiki=en,locale=en].
>>>>> >>>
>>>>> >>> found by wikipedia: [title=Queen Isabel II Statue
>>>>> (Manila);ns=0/Main/;language:wiki=en,locale=en]
>>>>> >>>
>>>>> >>> java.lang.reflect.InvocationTargetException
>>>>> >>>
>>>>> >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>>> Method)
>>>>> >>>
>>>>> >>> at
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>> >>>
>>>>> >>> at
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>> >>>
>>>>> >>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>> >>>
>>>>> >>> at
>>>>> org_scala_tools_maven_executions.MainHelper.runMain(MainHelper.java:161)
>>>>> >>>
>>>>> >>> at
>>>>> org_scala_tools_maven_executions.MainWithArgsInFile.main(MainWithArgsInFile.java:26)
>>>>> >>>
>>>>> >>> Caused by: java.util.NoSuchElementException: key not found: 2600
>>>>> >>>
>>>>> >>> at
>>>>> scala.collection.MapLike$class.default(MapLike.scala:228)
>>>>> >>>
>>>>> >>> at scala.collection.AbstractMap.default(Map.scala:58)
>>>>> >>>
>>>>> >>> at scala.collection.MapLike$class.apply(MapLike.scala:141)
>>>>> >>>
>>>>> >>> at scala.collection.AbstractMap.apply(Map.scala:58)
>>>>> >>>
>>>>> >>> at
>>>>> org.dbpedia.extraction.sources.WikipediaDumpParser.readPage(WikipediaDumpParser.java:219)
>>>>> >>>
>>>>> >>> at
>>>>> org.dbpedia.extraction.sources.WikipediaDumpParser.readPages(WikipediaDumpParser.java:187)
>>>>> >>>
>>>>> >>> at
>>>>> org.dbpedia.extraction.sources.WikipediaDumpParser.readDump(WikipediaDumpParser.java:145)
>>>>> >>>
>>>>> >>> at
>>>>> org.dbpedia.extraction.sources.WikipediaDumpParser.run(WikipediaDumpParser.java:116)
>>>>> >>>
>>>>> >>> at
>>>>> org.dbpedia.extraction.sources.XMLReaderSource.foreach(XMLSource.scala:65)
>>>>> >>>
>>>>> >>> at
>>>>> scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251)
>>>>> >>>
>>>>> >>> at
>>>>> org.dbpedia.extraction.sources.XMLReaderSource.flatMap(XMLSource.scala:61)
>>>>> >>>
>>>>> >>> at
>>>>> org.dbpedia.extraction.mappings.Redirects$.loadFromSource(Redirects.scala:165)
>>>>> >>>
>>>>> >>> at
>>>>> org.dbpedia.extraction.mappings.Redirects$.load(Redirects.scala:116)
>>>>> >>>
>>>>> >>> at
>>>>> org.dbpedia.extraction.dump.extract.ConfigLoader$$anon$1.<init>(ConfigLoader.scala:97)
>>>>> >>>
>>>>> >>> at org.dbpedia.extraction.dump.extract.ConfigLoader.org
>>>>> $dbpedia$extraction$dump$extract$ConfigLoader$$createExtractionJob(ConfigLoader.scala:52)
>>>>> >>>
>>>>> >>> at
>>>>> org.dbpedia.extraction.dump.extract.ConfigLoader$$anonfun$getExtractionJobs$1.apply(ConfigLoader.scala:37)
>>>>> >>>
>>>>> >>> at
>>>>> org.dbpedia.extraction.dump.extract.ConfigLoader$$anonfun$getExtractionJobs$1.apply(ConfigLoader.scala:37)
>>>>> >>>
>>>>> >>> at
>>>>> scala.collection.GenTraversableViewLike$Mapped$$anonfun$foreach$2.apply(GenTraversableViewLike.scala:81)
>>>>> >>>
>>>>> >>> at
>>>>> scala.collection.Iterator$class.foreach(Iterator.scala:727)
>>>>> >>>
>>>>> >>> at
>>>>> scala.collection.immutable.RedBlackTree$TreeIterator.foreach(RedBlackTree.scala:422)
>>>>> >>>
>>>>> >>> at
>>>>> scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
>>>>> >>>
>>>>> >>> at
>>>>> scala.collection.IterableLike$$anon$1.foreach(IterableLike.scala:294)
>>>>> >>>
>>>>> >>> at
>>>>> scala.collection.GenTraversableViewLike$Mapped$class.foreach(GenTraversableViewLike.scala:80)
>>>>> >>>
>>>>> >>> at
>>>>> scala.collection.IterableViewLike$$anon$3.foreach(IterableViewLike.scala:84)
>>>>> >>>
>>>>> >>> at
>>>>> org.dbpedia.extraction.dump.extract.Extraction$.main(Extraction.scala:30)
>>>>> >>>
>>>>> >>> at
>>>>> org.dbpedia.extraction.dump.extract.Extraction.main(Extraction.scala)
>>>>> >>>
>>>>> >>> ... 6 more
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> ------------------------------------------------------------------------------
>>>>> >>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
>>>>> >>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS
>>>>> Reports
>>>>> >>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White
>>>>> paper
>>>>> >>> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog
>>>>> Analyzer
>>>>> >>>
>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>> _______________________________________________
>>>>> >>> Dbpedia-discussion mailing list
>>>>> >>> Dbpedia-discussion@lists.sourceforge.net
>>>>> >>> https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> ------------------------------------------------------------------------------
>>>>> >>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
>>>>> >>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS
>>>>> Reports
>>>>> >>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White
>>>>> paper
>>>>> >>> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog
>>>>> Analyzer
>>>>> >>>
>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
>>>>> >>> _______________________________________________
>>>>> >>> Dbpedia-discussion mailing list
>>>>> >>> Dbpedia-discussion@lists.sourceforge.net
>>>>> >>> https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
>>>>> >>>
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> --
>>>>> >> Kontokostas Dimitris
>>>>> >>
>>>>> >>
>>>>> ------------------------------------------------------------------------------
>>>>> >> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
>>>>> >> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS
>>>>> Reports
>>>>> >> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
>>>>> >> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
>>>>> >>
>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
>>>>> >> _______________________________________________
>>>>> >> Dbpedia-discussion mailing list
>>>>> >> Dbpedia-discussion@lists.sourceforge.net
>>>>> >> https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
>>>>> >>
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Kontokostas Dimitris
>>>
>>
>>
>
------------------------------------------------------------------------------
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
_______________________________________________
Dbpedia-discussion mailing list
Dbpedia-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
