Re: Why can i not serialize a Dictionary ?
yeah, i did https://gist.github.com/anonymous/542275c2fdb817c02c6275eb227c467d#file-test-java-L38-L41 The constructor will be called one time The InSpanFeatureGenerator does a similar thing. 2016-10-31 14:11 GMT+01:00 William Colen : > OK, you need to find a way to avoid executing POS Tagger for each call. > Maybe you can store the output to a cache. > > 2016-10-31 10:49 GMT-02:00 Damiano Porta : > > > hmm ok i load the postagger in the constructor of my custom > > FeatureGenerator, this is an example: > > > > https://gist.github.com/anonymous/542275c2fdb817c02c6275eb227c467d > > > > what do you think? > > > > 2016-10-31 13:16 GMT+01:00 William Colen : > > > > > Unfortunately I don't think the API supports this. You will need a > hack. > > > > > > 2016-10-30 12:59 GMT-02:00 Damiano Porta : > > > > > > > Jorn > > > > what suffix should i use if i need a postagger model in a > > > FeatureGenerator? > > > > > > > > For dictionary i use mydictionary.dictionary as you told me. What > about > > > > postagger .bin? > > > > > > > > Thanks > > > > Damiano > > > > > > > > Il 29/Ott/2016 14:27, "Damiano Porta" ha > > > scritto: > > > > > > > > > ok! thank you Jorn! > > > > > > > > > > 2016-10-29 13:54 GMT+02:00 Joern Kottmann : > > > > > > > > > >> The class has to be on your classpath otherwise it can't be > loaded. > > > > >> > > > > >> Jörn > > > > >> > > > > >> On Fri, 2016-10-28 at 22:59 +0200, Damiano Porta wrote: > > > > >> > Jorn, > > > > >> > as I wrote i have created the ner model, but when i try to use > it > > > via > > > > >> > "opennlp TokenNameFinder" tool it can't locate my custom > > > > >> > FeatureGenerator. > > > > >> > This is the output: > > > > >> > > > > > >> > bash opennlp TokenNameFinder /home/damiano/custom-ner-model.bin > > > > >> > Loading Token Name Finder model ... done (5,948s) > > > > >> > Exception in thread "main" > > > > >> > opennlp.tools.util.ext.ExtensionNotLoadedException: Unable to > > find > > > > >> > implementation for > > > > >> > opennlp.tools.util.featuregen.AdaptiveFeatureGenerator, > > > > >> > the class or service > > > > >> > com.damiano.parser.generator.SpanWindowFeatureGenerator could > not > > > be > > > > >> > located! > > > > >> > at > > > > >> > opennlp.tools.util.ext.ExtensionLoader. > > > instantiateExtension(Extension > > > > >> > Loader.java:122) > > > > >> > at > > > > >> > opennlp.tools.util.featuregen.GeneratorFactory$ > > > CustomFeatureGenerator > > > > >> > Factory.create(GeneratorFactory.java:582) > > > > >> > at > > > > >> > opennlp.tools.util.featuregen.GeneratorFactory. > > > createGenerator(Genera > > > > >> > torFactory.java:661) > > > > >> > at > > > > >> > opennlp.tools.util.featuregen.GeneratorFactory$ > > > AggregatedFeatureGener > > > > >> > atorFactory.create(GeneratorFactory.java:129) > > > > >> > at > > > > >> > opennlp.tools.util.featuregen.GeneratorFactory. > > > createGenerator(Genera > > > > >> > torFactory.java:661) > > > > >> > at > > > > >> > opennlp.tools.util.featuregen.GeneratorFactory$ > > > CachedFeatureGenerator > > > > >> > Factory.create(GeneratorFactory.java:171) > > > > >> > at > > > > >> > opennlp.tools.util.featuregen.GeneratorFactory. > > > createGenerator(Genera > > > > >> > torFactory.java:661) > > > > >> > at > > > > >> > opennlp.tools.util.featuregen.GeneratorFactory$ > > > AggregatedFeatureGener > > > > >> > atorFactory.create(GeneratorFactory.java:129) > > > > >> > at > > > > >> > opennlp.tools.util.featuregen.GeneratorFactory. > > > createGenerator(Genera > > > > >> > torFactory.java:661) > > > > >> > at > > > > >> > opennlp.tools.util.featuregen.GeneratorFactory.create( > > > GeneratorFactor > > > > >> > y.java:711) > > > > >> > at > > > > >> > opennlp.tools.namefind.TokenNameFinderFactory. > > > createFeatureGenerators > > > > >> > (TokenNameFinderFactory.java:153) > > > > >> > at > > > > >> > opennlp.tools.namefind.TokenNameFinderFactory. > > > createContextGenerator( > > > > >> > TokenNameFinderFactory.java:118) > > > > >> > at opennlp.tools.namefind.NameFinderME.( > > NameFinderME.java:90) > > > > >> > at > > > > >> > opennlp.tools.cmdline.namefind.TokenNameFinderTool. > > > run(TokenNameFinde > > > > >> > rTool.java:59) > > > > >> > at opennlp.tools.cmdline.CLI.main(CLI.java:227) > > > > >> > > > > > >> > Why does the custom generator is not included on the model? > > > > >> > This is the my xml descriptor: > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> >> > > >> > class="com.damiano.parser.generator.SpanWindowFeatureGenerator" > > > > >> > leftTokens="6" rightTokens="3"/> > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > Damiano > > > > >> > > > > > >> > 2016-10-28 14:00 GMT+02:00 Damiano Porta < > damianopo.
Re: Why can i not serialize a Dictionary ?
OK, you need to find a way to avoid executing POS Tagger for each call. Maybe you can store the output to a cache. 2016-10-31 10:49 GMT-02:00 Damiano Porta : > hmm ok i load the postagger in the constructor of my custom > FeatureGenerator, this is an example: > > https://gist.github.com/anonymous/542275c2fdb817c02c6275eb227c467d > > what do you think? > > 2016-10-31 13:16 GMT+01:00 William Colen : > > > Unfortunately I don't think the API supports this. You will need a hack. > > > > 2016-10-30 12:59 GMT-02:00 Damiano Porta : > > > > > Jorn > > > what suffix should i use if i need a postagger model in a > > FeatureGenerator? > > > > > > For dictionary i use mydictionary.dictionary as you told me. What about > > > postagger .bin? > > > > > > Thanks > > > Damiano > > > > > > Il 29/Ott/2016 14:27, "Damiano Porta" ha > > scritto: > > > > > > > ok! thank you Jorn! > > > > > > > > 2016-10-29 13:54 GMT+02:00 Joern Kottmann : > > > > > > > >> The class has to be on your classpath otherwise it can't be loaded. > > > >> > > > >> Jörn > > > >> > > > >> On Fri, 2016-10-28 at 22:59 +0200, Damiano Porta wrote: > > > >> > Jorn, > > > >> > as I wrote i have created the ner model, but when i try to use it > > via > > > >> > "opennlp TokenNameFinder" tool it can't locate my custom > > > >> > FeatureGenerator. > > > >> > This is the output: > > > >> > > > > >> > bash opennlp TokenNameFinder /home/damiano/custom-ner-model.bin > > > >> > Loading Token Name Finder model ... done (5,948s) > > > >> > Exception in thread "main" > > > >> > opennlp.tools.util.ext.ExtensionNotLoadedException: Unable to > find > > > >> > implementation for > > > >> > opennlp.tools.util.featuregen.AdaptiveFeatureGenerator, > > > >> > the class or service > > > >> > com.damiano.parser.generator.SpanWindowFeatureGenerator could not > > be > > > >> > located! > > > >> > at > > > >> > opennlp.tools.util.ext.ExtensionLoader. > > instantiateExtension(Extension > > > >> > Loader.java:122) > > > >> > at > > > >> > opennlp.tools.util.featuregen.GeneratorFactory$ > > CustomFeatureGenerator > > > >> > Factory.create(GeneratorFactory.java:582) > > > >> > at > > > >> > opennlp.tools.util.featuregen.GeneratorFactory. > > createGenerator(Genera > > > >> > torFactory.java:661) > > > >> > at > > > >> > opennlp.tools.util.featuregen.GeneratorFactory$ > > AggregatedFeatureGener > > > >> > atorFactory.create(GeneratorFactory.java:129) > > > >> > at > > > >> > opennlp.tools.util.featuregen.GeneratorFactory. > > createGenerator(Genera > > > >> > torFactory.java:661) > > > >> > at > > > >> > opennlp.tools.util.featuregen.GeneratorFactory$ > > CachedFeatureGenerator > > > >> > Factory.create(GeneratorFactory.java:171) > > > >> > at > > > >> > opennlp.tools.util.featuregen.GeneratorFactory. > > createGenerator(Genera > > > >> > torFactory.java:661) > > > >> > at > > > >> > opennlp.tools.util.featuregen.GeneratorFactory$ > > AggregatedFeatureGener > > > >> > atorFactory.create(GeneratorFactory.java:129) > > > >> > at > > > >> > opennlp.tools.util.featuregen.GeneratorFactory. > > createGenerator(Genera > > > >> > torFactory.java:661) > > > >> > at > > > >> > opennlp.tools.util.featuregen.GeneratorFactory.create( > > GeneratorFactor > > > >> > y.java:711) > > > >> > at > > > >> > opennlp.tools.namefind.TokenNameFinderFactory. > > createFeatureGenerators > > > >> > (TokenNameFinderFactory.java:153) > > > >> > at > > > >> > opennlp.tools.namefind.TokenNameFinderFactory. > > createContextGenerator( > > > >> > TokenNameFinderFactory.java:118) > > > >> > at opennlp.tools.namefind.NameFinderME.( > NameFinderME.java:90) > > > >> > at > > > >> > opennlp.tools.cmdline.namefind.TokenNameFinderTool. > > run(TokenNameFinde > > > >> > rTool.java:59) > > > >> > at opennlp.tools.cmdline.CLI.main(CLI.java:227) > > > >> > > > > >> > Why does the custom generator is not included on the model? > > > >> > This is the my xml descriptor: > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> >> > >> > class="com.damiano.parser.generator.SpanWindowFeatureGenerator" > > > >> > leftTokens="6" rightTokens="3"/> > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > Damiano > > > >> > > > > >> > 2016-10-28 14:00 GMT+02:00 Damiano Porta >: > > > >> > > > > >> > > > > > >> > > Pardon, my wrong, i forgot to change > > >> > > dict="damiano"/> into > > >> > > dict="damiano.dictionary"/>in my train.xml > > > >> > > > > > >> > > now it is working well! and the .bin has my dictionary too > > > >> > > > > > >> > > 2016-10-28 13:51 GMT+02:00 Damiano Porta < > damianopo...@gmail.com > > >: > > > >> > > > > > >> > > > > > > >> > > > Jorn > > > >> > > > i change the code as you told me, this exactly: > > > >> > > > https://gist.github.com/anonymous/8877b09d441d2e64c181fa9b5a >
Re: Why can i not serialize a Dictionary ?
hmm ok i load the postagger in the constructor of my custom FeatureGenerator, this is an example: https://gist.github.com/anonymous/542275c2fdb817c02c6275eb227c467d what do you think? 2016-10-31 13:16 GMT+01:00 William Colen : > Unfortunately I don't think the API supports this. You will need a hack. > > 2016-10-30 12:59 GMT-02:00 Damiano Porta : > > > Jorn > > what suffix should i use if i need a postagger model in a > FeatureGenerator? > > > > For dictionary i use mydictionary.dictionary as you told me. What about > > postagger .bin? > > > > Thanks > > Damiano > > > > Il 29/Ott/2016 14:27, "Damiano Porta" ha > scritto: > > > > > ok! thank you Jorn! > > > > > > 2016-10-29 13:54 GMT+02:00 Joern Kottmann : > > > > > >> The class has to be on your classpath otherwise it can't be loaded. > > >> > > >> Jörn > > >> > > >> On Fri, 2016-10-28 at 22:59 +0200, Damiano Porta wrote: > > >> > Jorn, > > >> > as I wrote i have created the ner model, but when i try to use it > via > > >> > "opennlp TokenNameFinder" tool it can't locate my custom > > >> > FeatureGenerator. > > >> > This is the output: > > >> > > > >> > bash opennlp TokenNameFinder /home/damiano/custom-ner-model.bin > > >> > Loading Token Name Finder model ... done (5,948s) > > >> > Exception in thread "main" > > >> > opennlp.tools.util.ext.ExtensionNotLoadedException: Unable to find > > >> > implementation for > > >> > opennlp.tools.util.featuregen.AdaptiveFeatureGenerator, > > >> > the class or service > > >> > com.damiano.parser.generator.SpanWindowFeatureGenerator could not > be > > >> > located! > > >> > at > > >> > opennlp.tools.util.ext.ExtensionLoader. > instantiateExtension(Extension > > >> > Loader.java:122) > > >> > at > > >> > opennlp.tools.util.featuregen.GeneratorFactory$ > CustomFeatureGenerator > > >> > Factory.create(GeneratorFactory.java:582) > > >> > at > > >> > opennlp.tools.util.featuregen.GeneratorFactory. > createGenerator(Genera > > >> > torFactory.java:661) > > >> > at > > >> > opennlp.tools.util.featuregen.GeneratorFactory$ > AggregatedFeatureGener > > >> > atorFactory.create(GeneratorFactory.java:129) > > >> > at > > >> > opennlp.tools.util.featuregen.GeneratorFactory. > createGenerator(Genera > > >> > torFactory.java:661) > > >> > at > > >> > opennlp.tools.util.featuregen.GeneratorFactory$ > CachedFeatureGenerator > > >> > Factory.create(GeneratorFactory.java:171) > > >> > at > > >> > opennlp.tools.util.featuregen.GeneratorFactory. > createGenerator(Genera > > >> > torFactory.java:661) > > >> > at > > >> > opennlp.tools.util.featuregen.GeneratorFactory$ > AggregatedFeatureGener > > >> > atorFactory.create(GeneratorFactory.java:129) > > >> > at > > >> > opennlp.tools.util.featuregen.GeneratorFactory. > createGenerator(Genera > > >> > torFactory.java:661) > > >> > at > > >> > opennlp.tools.util.featuregen.GeneratorFactory.create( > GeneratorFactor > > >> > y.java:711) > > >> > at > > >> > opennlp.tools.namefind.TokenNameFinderFactory. > createFeatureGenerators > > >> > (TokenNameFinderFactory.java:153) > > >> > at > > >> > opennlp.tools.namefind.TokenNameFinderFactory. > createContextGenerator( > > >> > TokenNameFinderFactory.java:118) > > >> > at opennlp.tools.namefind.NameFinderME.(NameFinderME.java:90) > > >> > at > > >> > opennlp.tools.cmdline.namefind.TokenNameFinderTool. > run(TokenNameFinde > > >> > rTool.java:59) > > >> > at opennlp.tools.cmdline.CLI.main(CLI.java:227) > > >> > > > >> > Why does the custom generator is not included on the model? > > >> > This is the my xml descriptor: > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> >> >> > class="com.damiano.parser.generator.SpanWindowFeatureGenerator" > > >> > leftTokens="6" rightTokens="3"/> > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > Damiano > > >> > > > >> > 2016-10-28 14:00 GMT+02:00 Damiano Porta : > > >> > > > >> > > > > >> > > Pardon, my wrong, i forgot to change > >> > > dict="damiano"/> into > >> > > dict="damiano.dictionary"/>in my train.xml > > >> > > > > >> > > now it is working well! and the .bin has my dictionary too > > >> > > > > >> > > 2016-10-28 13:51 GMT+02:00 Damiano Porta >: > > >> > > > > >> > > > > > >> > > > Jorn > > >> > > > i change the code as you told me, this exactly: > > >> > > > https://gist.github.com/anonymous/8877b09d441d2e64c181fa9b5a > > >> > > > de4550#file-test-java-L15 > > >> > > > > > >> > > > but i get this error: > > >> > > > > > >> > > > opennlp.tools.namefind.TokenNameFinderModel$ > FeatureGeneratorCreat > > >> > > > ionError: > > >> > > > opennlp.tools.util.InvalidFormatException: No dictionary > resource > > >> > > > for > > >> > > > key: damiano > > >> > > > at opennlp.tools.namefind.TokenNameFinderFactory.createFeatureG > > >> > > > enerators(TokenNameFinderFactory.java:176) > > >> > > > at opennlp.tools.
Re: Why can i not serialize a Dictionary ?
Unfortunately I don't think the API supports this. You will need a hack. 2016-10-30 12:59 GMT-02:00 Damiano Porta : > Jorn > what suffix should i use if i need a postagger model in a FeatureGenerator? > > For dictionary i use mydictionary.dictionary as you told me. What about > postagger .bin? > > Thanks > Damiano > > Il 29/Ott/2016 14:27, "Damiano Porta" ha scritto: > > > ok! thank you Jorn! > > > > 2016-10-29 13:54 GMT+02:00 Joern Kottmann : > > > >> The class has to be on your classpath otherwise it can't be loaded. > >> > >> Jörn > >> > >> On Fri, 2016-10-28 at 22:59 +0200, Damiano Porta wrote: > >> > Jorn, > >> > as I wrote i have created the ner model, but when i try to use it via > >> > "opennlp TokenNameFinder" tool it can't locate my custom > >> > FeatureGenerator. > >> > This is the output: > >> > > >> > bash opennlp TokenNameFinder /home/damiano/custom-ner-model.bin > >> > Loading Token Name Finder model ... done (5,948s) > >> > Exception in thread "main" > >> > opennlp.tools.util.ext.ExtensionNotLoadedException: Unable to find > >> > implementation for > >> > opennlp.tools.util.featuregen.AdaptiveFeatureGenerator, > >> > the class or service > >> > com.damiano.parser.generator.SpanWindowFeatureGenerator could not be > >> > located! > >> > at > >> > opennlp.tools.util.ext.ExtensionLoader.instantiateExtension(Extension > >> > Loader.java:122) > >> > at > >> > opennlp.tools.util.featuregen.GeneratorFactory$CustomFeatureGenerator > >> > Factory.create(GeneratorFactory.java:582) > >> > at > >> > opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(Genera > >> > torFactory.java:661) > >> > at > >> > opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFeatureGener > >> > atorFactory.create(GeneratorFactory.java:129) > >> > at > >> > opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(Genera > >> > torFactory.java:661) > >> > at > >> > opennlp.tools.util.featuregen.GeneratorFactory$CachedFeatureGenerator > >> > Factory.create(GeneratorFactory.java:171) > >> > at > >> > opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(Genera > >> > torFactory.java:661) > >> > at > >> > opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFeatureGener > >> > atorFactory.create(GeneratorFactory.java:129) > >> > at > >> > opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(Genera > >> > torFactory.java:661) > >> > at > >> > opennlp.tools.util.featuregen.GeneratorFactory.create(GeneratorFactor > >> > y.java:711) > >> > at > >> > opennlp.tools.namefind.TokenNameFinderFactory.createFeatureGenerators > >> > (TokenNameFinderFactory.java:153) > >> > at > >> > opennlp.tools.namefind.TokenNameFinderFactory.createContextGenerator( > >> > TokenNameFinderFactory.java:118) > >> > at opennlp.tools.namefind.NameFinderME.(NameFinderME.java:90) > >> > at > >> > opennlp.tools.cmdline.namefind.TokenNameFinderTool.run(TokenNameFinde > >> > rTool.java:59) > >> > at opennlp.tools.cmdline.CLI.main(CLI.java:227) > >> > > >> > Why does the custom generator is not included on the model? > >> > This is the my xml descriptor: > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> >>> > class="com.damiano.parser.generator.SpanWindowFeatureGenerator" > >> > leftTokens="6" rightTokens="3"/> > >> > > >> > > >> > > >> > > >> > > >> > Damiano > >> > > >> > 2016-10-28 14:00 GMT+02:00 Damiano Porta : > >> > > >> > > > >> > > Pardon, my wrong, i forgot to change >> > > dict="damiano"/> into >> > > dict="damiano.dictionary"/>in my train.xml > >> > > > >> > > now it is working well! and the .bin has my dictionary too > >> > > > >> > > 2016-10-28 13:51 GMT+02:00 Damiano Porta : > >> > > > >> > > > > >> > > > Jorn > >> > > > i change the code as you told me, this exactly: > >> > > > https://gist.github.com/anonymous/8877b09d441d2e64c181fa9b5a > >> > > > de4550#file-test-java-L15 > >> > > > > >> > > > but i get this error: > >> > > > > >> > > > opennlp.tools.namefind.TokenNameFinderModel$FeatureGeneratorCreat > >> > > > ionError: > >> > > > opennlp.tools.util.InvalidFormatException: No dictionary resource > >> > > > for > >> > > > key: damiano > >> > > > at opennlp.tools.namefind.TokenNameFinderFactory.createFeatureG > >> > > > enerators(TokenNameFinderFactory.java:176) > >> > > > at opennlp.tools.namefind.TokenNameFinderFactory.createContextG > >> > > > enerator(TokenNameFinderFactory.java:118) > >> > > > at > >> > > > opennlp.tools.namefind.NameFinderME.train(NameFinderME.java:333) > >> > > > at com.damiano.parser.trainer.NER.compileNER(NER.java:185) > >> > > > at com.damiano.parser.trainer.NER.main(NER.java:155) > >> > > > > >> > > > Caused by: opennlp.tools.util.InvalidFormatException: No > >> > > > dictionary > >> > > > resource for key: damiano > >> > > > at opennlp.tools.util.featuregen.GeneratorFactory$DictionaryFea > >> > > > tureG
Re: Why can i not serialize a Dictionary ?
Jorn what suffix should i use if i need a postagger model in a FeatureGenerator? For dictionary i use mydictionary.dictionary as you told me. What about postagger .bin? Thanks Damiano Il 29/Ott/2016 14:27, "Damiano Porta" ha scritto: > ok! thank you Jorn! > > 2016-10-29 13:54 GMT+02:00 Joern Kottmann : > >> The class has to be on your classpath otherwise it can't be loaded. >> >> Jörn >> >> On Fri, 2016-10-28 at 22:59 +0200, Damiano Porta wrote: >> > Jorn, >> > as I wrote i have created the ner model, but when i try to use it via >> > "opennlp TokenNameFinder" tool it can't locate my custom >> > FeatureGenerator. >> > This is the output: >> > >> > bash opennlp TokenNameFinder /home/damiano/custom-ner-model.bin >> > Loading Token Name Finder model ... done (5,948s) >> > Exception in thread "main" >> > opennlp.tools.util.ext.ExtensionNotLoadedException: Unable to find >> > implementation for >> > opennlp.tools.util.featuregen.AdaptiveFeatureGenerator, >> > the class or service >> > com.damiano.parser.generator.SpanWindowFeatureGenerator could not be >> > located! >> > at >> > opennlp.tools.util.ext.ExtensionLoader.instantiateExtension(Extension >> > Loader.java:122) >> > at >> > opennlp.tools.util.featuregen.GeneratorFactory$CustomFeatureGenerator >> > Factory.create(GeneratorFactory.java:582) >> > at >> > opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(Genera >> > torFactory.java:661) >> > at >> > opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFeatureGener >> > atorFactory.create(GeneratorFactory.java:129) >> > at >> > opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(Genera >> > torFactory.java:661) >> > at >> > opennlp.tools.util.featuregen.GeneratorFactory$CachedFeatureGenerator >> > Factory.create(GeneratorFactory.java:171) >> > at >> > opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(Genera >> > torFactory.java:661) >> > at >> > opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFeatureGener >> > atorFactory.create(GeneratorFactory.java:129) >> > at >> > opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(Genera >> > torFactory.java:661) >> > at >> > opennlp.tools.util.featuregen.GeneratorFactory.create(GeneratorFactor >> > y.java:711) >> > at >> > opennlp.tools.namefind.TokenNameFinderFactory.createFeatureGenerators >> > (TokenNameFinderFactory.java:153) >> > at >> > opennlp.tools.namefind.TokenNameFinderFactory.createContextGenerator( >> > TokenNameFinderFactory.java:118) >> > at opennlp.tools.namefind.NameFinderME.(NameFinderME.java:90) >> > at >> > opennlp.tools.cmdline.namefind.TokenNameFinderTool.run(TokenNameFinde >> > rTool.java:59) >> > at opennlp.tools.cmdline.CLI.main(CLI.java:227) >> > >> > Why does the custom generator is not included on the model? >> > This is the my xml descriptor: >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > > > class="com.damiano.parser.generator.SpanWindowFeatureGenerator" >> > leftTokens="6" rightTokens="3"/> >> > >> > >> > >> > >> > >> > Damiano >> > >> > 2016-10-28 14:00 GMT+02:00 Damiano Porta : >> > >> > > >> > > Pardon, my wrong, i forgot to change > > > dict="damiano"/> into > > > dict="damiano.dictionary"/>in my train.xml >> > > >> > > now it is working well! and the .bin has my dictionary too >> > > >> > > 2016-10-28 13:51 GMT+02:00 Damiano Porta : >> > > >> > > > >> > > > Jorn >> > > > i change the code as you told me, this exactly: >> > > > https://gist.github.com/anonymous/8877b09d441d2e64c181fa9b5a >> > > > de4550#file-test-java-L15 >> > > > >> > > > but i get this error: >> > > > >> > > > opennlp.tools.namefind.TokenNameFinderModel$FeatureGeneratorCreat >> > > > ionError: >> > > > opennlp.tools.util.InvalidFormatException: No dictionary resource >> > > > for >> > > > key: damiano >> > > > at opennlp.tools.namefind.TokenNameFinderFactory.createFeatureG >> > > > enerators(TokenNameFinderFactory.java:176) >> > > > at opennlp.tools.namefind.TokenNameFinderFactory.createContextG >> > > > enerator(TokenNameFinderFactory.java:118) >> > > > at >> > > > opennlp.tools.namefind.NameFinderME.train(NameFinderME.java:333) >> > > > at com.damiano.parser.trainer.NER.compileNER(NER.java:185) >> > > > at com.damiano.parser.trainer.NER.main(NER.java:155) >> > > > >> > > > Caused by: opennlp.tools.util.InvalidFormatException: No >> > > > dictionary >> > > > resource for key: damiano >> > > > at opennlp.tools.util.featuregen.GeneratorFactory$DictionaryFea >> > > > tureGeneratorFactory.create(GeneratorFactory.java:251) >> > > > at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat >> > > > or(GeneratorFactory.java:661) >> > > > at opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFea >> > > > tureGeneratorFactory.create(GeneratorFactory.java:129) >> > > > at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat >> > > > or(GeneratorFacto
Re: Why can i not serialize a Dictionary ?
ok! thank you Jorn! 2016-10-29 13:54 GMT+02:00 Joern Kottmann : > The class has to be on your classpath otherwise it can't be loaded. > > Jörn > > On Fri, 2016-10-28 at 22:59 +0200, Damiano Porta wrote: > > Jorn, > > as I wrote i have created the ner model, but when i try to use it via > > "opennlp TokenNameFinder" tool it can't locate my custom > > FeatureGenerator. > > This is the output: > > > > bash opennlp TokenNameFinder /home/damiano/custom-ner-model.bin > > Loading Token Name Finder model ... done (5,948s) > > Exception in thread "main" > > opennlp.tools.util.ext.ExtensionNotLoadedException: Unable to find > > implementation for > > opennlp.tools.util.featuregen.AdaptiveFeatureGenerator, > > the class or service > > com.damiano.parser.generator.SpanWindowFeatureGenerator could not be > > located! > > at > > opennlp.tools.util.ext.ExtensionLoader.instantiateExtension(Extension > > Loader.java:122) > > at > > opennlp.tools.util.featuregen.GeneratorFactory$CustomFeatureGenerator > > Factory.create(GeneratorFactory.java:582) > > at > > opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(Genera > > torFactory.java:661) > > at > > opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFeatureGener > > atorFactory.create(GeneratorFactory.java:129) > > at > > opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(Genera > > torFactory.java:661) > > at > > opennlp.tools.util.featuregen.GeneratorFactory$CachedFeatureGenerator > > Factory.create(GeneratorFactory.java:171) > > at > > opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(Genera > > torFactory.java:661) > > at > > opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFeatureGener > > atorFactory.create(GeneratorFactory.java:129) > > at > > opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(Genera > > torFactory.java:661) > > at > > opennlp.tools.util.featuregen.GeneratorFactory.create(GeneratorFactor > > y.java:711) > > at > > opennlp.tools.namefind.TokenNameFinderFactory.createFeatureGenerators > > (TokenNameFinderFactory.java:153) > > at > > opennlp.tools.namefind.TokenNameFinderFactory.createContextGenerator( > > TokenNameFinderFactory.java:118) > > at opennlp.tools.namefind.NameFinderME.(NameFinderME.java:90) > > at > > opennlp.tools.cmdline.namefind.TokenNameFinderTool.run(TokenNameFinde > > rTool.java:59) > > at opennlp.tools.cmdline.CLI.main(CLI.java:227) > > > > Why does the custom generator is not included on the model? > > This is the my xml descriptor: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> class="com.damiano.parser.generator.SpanWindowFeatureGenerator" > > leftTokens="6" rightTokens="3"/> > > > > > > > > > > > > Damiano > > > > 2016-10-28 14:00 GMT+02:00 Damiano Porta : > > > > > > > > Pardon, my wrong, i forgot to change > > dict="damiano"/> into > > dict="damiano.dictionary"/>in my train.xml > > > > > > now it is working well! and the .bin has my dictionary too > > > > > > 2016-10-28 13:51 GMT+02:00 Damiano Porta : > > > > > > > > > > > Jorn > > > > i change the code as you told me, this exactly: > > > > https://gist.github.com/anonymous/8877b09d441d2e64c181fa9b5a > > > > de4550#file-test-java-L15 > > > > > > > > but i get this error: > > > > > > > > opennlp.tools.namefind.TokenNameFinderModel$FeatureGeneratorCreat > > > > ionError: > > > > opennlp.tools.util.InvalidFormatException: No dictionary resource > > > > for > > > > key: damiano > > > > at opennlp.tools.namefind.TokenNameFinderFactory.createFeatureG > > > > enerators(TokenNameFinderFactory.java:176) > > > > at opennlp.tools.namefind.TokenNameFinderFactory.createContextG > > > > enerator(TokenNameFinderFactory.java:118) > > > > at > > > > opennlp.tools.namefind.NameFinderME.train(NameFinderME.java:333) > > > > at com.damiano.parser.trainer.NER.compileNER(NER.java:185) > > > > at com.damiano.parser.trainer.NER.main(NER.java:155) > > > > > > > > Caused by: opennlp.tools.util.InvalidFormatException: No > > > > dictionary > > > > resource for key: damiano > > > > at opennlp.tools.util.featuregen.GeneratorFactory$DictionaryFea > > > > tureGeneratorFactory.create(GeneratorFactory.java:251) > > > > at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat > > > > or(GeneratorFactory.java:661) > > > > at opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFea > > > > tureGeneratorFactory.create(GeneratorFactory.java:129) > > > > at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat > > > > or(GeneratorFactory.java:661) > > > > at opennlp.tools.util.featuregen.GeneratorFactory$CachedFeature > > > > GeneratorFactory.create(GeneratorFactory.java:171) > > > > at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat > > > > or(GeneratorFactory.java:661) > > > > at opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFea > > > > tureGeneratorFactory.create(GeneratorFactory
Re: Why can i not serialize a Dictionary ?
The class has to be on your classpath otherwise it can't be loaded. Jörn On Fri, 2016-10-28 at 22:59 +0200, Damiano Porta wrote: > Jorn, > as I wrote i have created the ner model, but when i try to use it via > "opennlp TokenNameFinder" tool it can't locate my custom > FeatureGenerator. > This is the output: > > bash opennlp TokenNameFinder /home/damiano/custom-ner-model.bin > Loading Token Name Finder model ... done (5,948s) > Exception in thread "main" > opennlp.tools.util.ext.ExtensionNotLoadedException: Unable to find > implementation for > opennlp.tools.util.featuregen.AdaptiveFeatureGenerator, > the class or service > com.damiano.parser.generator.SpanWindowFeatureGenerator could not be > located! > at > opennlp.tools.util.ext.ExtensionLoader.instantiateExtension(Extension > Loader.java:122) > at > opennlp.tools.util.featuregen.GeneratorFactory$CustomFeatureGenerator > Factory.create(GeneratorFactory.java:582) > at > opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(Genera > torFactory.java:661) > at > opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFeatureGener > atorFactory.create(GeneratorFactory.java:129) > at > opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(Genera > torFactory.java:661) > at > opennlp.tools.util.featuregen.GeneratorFactory$CachedFeatureGenerator > Factory.create(GeneratorFactory.java:171) > at > opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(Genera > torFactory.java:661) > at > opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFeatureGener > atorFactory.create(GeneratorFactory.java:129) > at > opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(Genera > torFactory.java:661) > at > opennlp.tools.util.featuregen.GeneratorFactory.create(GeneratorFactor > y.java:711) > at > opennlp.tools.namefind.TokenNameFinderFactory.createFeatureGenerators > (TokenNameFinderFactory.java:153) > at > opennlp.tools.namefind.TokenNameFinderFactory.createContextGenerator( > TokenNameFinderFactory.java:118) > at opennlp.tools.namefind.NameFinderME.(NameFinderME.java:90) > at > opennlp.tools.cmdline.namefind.TokenNameFinderTool.run(TokenNameFinde > rTool.java:59) > at opennlp.tools.cmdline.CLI.main(CLI.java:227) > > Why does the custom generator is not included on the model? > This is the my xml descriptor: > > > > > > > > > > > > > > > > class="com.damiano.parser.generator.SpanWindowFeatureGenerator" > leftTokens="6" rightTokens="3"/> > > > > > > Damiano > > 2016-10-28 14:00 GMT+02:00 Damiano Porta : > > > > > Pardon, my wrong, i forgot to change > dict="damiano"/> into > dict="damiano.dictionary"/>in my train.xml > > > > now it is working well! and the .bin has my dictionary too > > > > 2016-10-28 13:51 GMT+02:00 Damiano Porta : > > > > > > > > Jorn > > > i change the code as you told me, this exactly: > > > https://gist.github.com/anonymous/8877b09d441d2e64c181fa9b5a > > > de4550#file-test-java-L15 > > > > > > but i get this error: > > > > > > opennlp.tools.namefind.TokenNameFinderModel$FeatureGeneratorCreat > > > ionError: > > > opennlp.tools.util.InvalidFormatException: No dictionary resource > > > for > > > key: damiano > > > at opennlp.tools.namefind.TokenNameFinderFactory.createFeatureG > > > enerators(TokenNameFinderFactory.java:176) > > > at opennlp.tools.namefind.TokenNameFinderFactory.createContextG > > > enerator(TokenNameFinderFactory.java:118) > > > at > > > opennlp.tools.namefind.NameFinderME.train(NameFinderME.java:333) > > > at com.damiano.parser.trainer.NER.compileNER(NER.java:185) > > > at com.damiano.parser.trainer.NER.main(NER.java:155) > > > > > > Caused by: opennlp.tools.util.InvalidFormatException: No > > > dictionary > > > resource for key: damiano > > > at opennlp.tools.util.featuregen.GeneratorFactory$DictionaryFea > > > tureGeneratorFactory.create(GeneratorFactory.java:251) > > > at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat > > > or(GeneratorFactory.java:661) > > > at opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFea > > > tureGeneratorFactory.create(GeneratorFactory.java:129) > > > at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat > > > or(GeneratorFactory.java:661) > > > at opennlp.tools.util.featuregen.GeneratorFactory$CachedFeature > > > GeneratorFactory.create(GeneratorFactory.java:171) > > > at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat > > > or(GeneratorFactory.java:661) > > > at opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFea > > > tureGeneratorFactory.create(GeneratorFactory.java:129) > > > at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat > > > or(GeneratorFactory.java:661) > > > at opennlp.tools.util.featuregen.GeneratorFactory.create(Genera > > > torFactory.java:711) > > > at opennlp.tools.namefind.TokenNameFinderFactory.createFeatureG > > > enerators(TokenNameFinderFactory.java
Re: Why can i not serialize a Dictionary ?
Jorn, as I wrote i have created the ner model, but when i try to use it via "opennlp TokenNameFinder" tool it can't locate my custom FeatureGenerator. This is the output: bash opennlp TokenNameFinder /home/damiano/custom-ner-model.bin Loading Token Name Finder model ... done (5,948s) Exception in thread "main" opennlp.tools.util.ext.ExtensionNotLoadedException: Unable to find implementation for opennlp.tools.util.featuregen.AdaptiveFeatureGenerator, the class or service com.damiano.parser.generator.SpanWindowFeatureGenerator could not be located! at opennlp.tools.util.ext.ExtensionLoader.instantiateExtension(ExtensionLoader.java:122) at opennlp.tools.util.featuregen.GeneratorFactory$CustomFeatureGeneratorFactory.create(GeneratorFactory.java:582) at opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(GeneratorFactory.java:661) at opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFeatureGeneratorFactory.create(GeneratorFactory.java:129) at opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(GeneratorFactory.java:661) at opennlp.tools.util.featuregen.GeneratorFactory$CachedFeatureGeneratorFactory.create(GeneratorFactory.java:171) at opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(GeneratorFactory.java:661) at opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFeatureGeneratorFactory.create(GeneratorFactory.java:129) at opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(GeneratorFactory.java:661) at opennlp.tools.util.featuregen.GeneratorFactory.create(GeneratorFactory.java:711) at opennlp.tools.namefind.TokenNameFinderFactory.createFeatureGenerators(TokenNameFinderFactory.java:153) at opennlp.tools.namefind.TokenNameFinderFactory.createContextGenerator(TokenNameFinderFactory.java:118) at opennlp.tools.namefind.NameFinderME.(NameFinderME.java:90) at opennlp.tools.cmdline.namefind.TokenNameFinderTool.run(TokenNameFinderTool.java:59) at opennlp.tools.cmdline.CLI.main(CLI.java:227) Why does the custom generator is not included on the model? This is the my xml descriptor: Damiano 2016-10-28 14:00 GMT+02:00 Damiano Porta : > Pardon, my wrong, i forgot to change dict="damiano"/> into dict="damiano.dictionary"/>in my train.xml > > now it is working well! and the .bin has my dictionary too > > 2016-10-28 13:51 GMT+02:00 Damiano Porta : > >> Jorn >> i change the code as you told me, this exactly: >> https://gist.github.com/anonymous/8877b09d441d2e64c181fa9b5a >> de4550#file-test-java-L15 >> >> but i get this error: >> >> opennlp.tools.namefind.TokenNameFinderModel$FeatureGeneratorCreationError: >> opennlp.tools.util.InvalidFormatException: No dictionary resource for >> key: damiano >> at opennlp.tools.namefind.TokenNameFinderFactory.createFeatureG >> enerators(TokenNameFinderFactory.java:176) >> at opennlp.tools.namefind.TokenNameFinderFactory.createContextG >> enerator(TokenNameFinderFactory.java:118) >> at opennlp.tools.namefind.NameFinderME.train(NameFinderME.java:333) >> at com.damiano.parser.trainer.NER.compileNER(NER.java:185) >> at com.damiano.parser.trainer.NER.main(NER.java:155) >> >> Caused by: opennlp.tools.util.InvalidFormatException: No dictionary >> resource for key: damiano >> at opennlp.tools.util.featuregen.GeneratorFactory$DictionaryFea >> tureGeneratorFactory.create(GeneratorFactory.java:251) >> at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat >> or(GeneratorFactory.java:661) >> at opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFea >> tureGeneratorFactory.create(GeneratorFactory.java:129) >> at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat >> or(GeneratorFactory.java:661) >> at opennlp.tools.util.featuregen.GeneratorFactory$CachedFeature >> GeneratorFactory.create(GeneratorFactory.java:171) >> at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat >> or(GeneratorFactory.java:661) >> at opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFea >> tureGeneratorFactory.create(GeneratorFactory.java:129) >> at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat >> or(GeneratorFactory.java:661) >> at opennlp.tools.util.featuregen.GeneratorFactory.create(Genera >> torFactory.java:711) >> at opennlp.tools.namefind.TokenNameFinderFactory.createFeatureG >> enerators(TokenNameFinderFactory.java:153) >> ... 4 more >> >> 2016-10-28 12:55 GMT+02:00 Joern Kottmann : >> >>> Try to rename the dictionary key to xyz.dictionary then the serializer >>> will >>> be mapped correctly. >>> >>> Jörn >>> >>> On Thu, Oct 27, 2016 at 11:14 PM, Damiano Porta >>> wrote: >>> >>> > Jorn i add the Dictionary here: >>> > https://gist.github.com/anonymous/bc822fb0520c4c42b75748bf4147da >>> > 34#file-train-java-L15 >>> > >>> > And unfortunately i only see this error: >>> > >>> > java.lang.IllegalStateException: Missing serializer for damiano >>> > at opennlp.tools.util.model.BaseModel.serialize(Ba
Re: Why can i not serialize a Dictionary ?
Pardon, my wrong, i forgot to change into in my train.xml now it is working well! and the .bin has my dictionary too 2016-10-28 13:51 GMT+02:00 Damiano Porta : > Jorn > i change the code as you told me, this exactly: https://gist.github.com/ > anonymous/8877b09d441d2e64c181fa9b5ade4550#file-test-java-L15 > > but i get this error: > > opennlp.tools.namefind.TokenNameFinderModel$FeatureGeneratorCreationError: > opennlp.tools.util.InvalidFormatException: No dictionary resource for > key: damiano > at opennlp.tools.namefind.TokenNameFinderFactory.createFeatureGenerators( > TokenNameFinderFactory.java:176) > at opennlp.tools.namefind.TokenNameFinderFactory.createContextGenerator( > TokenNameFinderFactory.java:118) > at opennlp.tools.namefind.NameFinderME.train(NameFinderME.java:333) > at com.damiano.parser.trainer.NER.compileNER(NER.java:185) > at com.damiano.parser.trainer.NER.main(NER.java:155) > > Caused by: opennlp.tools.util.InvalidFormatException: No dictionary > resource for key: damiano > at opennlp.tools.util.featuregen.GeneratorFactory$ > DictionaryFeatureGeneratorFactory.create(GeneratorFactory.java:251) > at opennlp.tools.util.featuregen.GeneratorFactory.createGenerator( > GeneratorFactory.java:661) > at opennlp.tools.util.featuregen.GeneratorFactory$ > AggregatedFeatureGeneratorFactory.create(GeneratorFactory.java:129) > at opennlp.tools.util.featuregen.GeneratorFactory.createGenerator( > GeneratorFactory.java:661) > at opennlp.tools.util.featuregen.GeneratorFactory$ > CachedFeatureGeneratorFactory.create(GeneratorFactory.java:171) > at opennlp.tools.util.featuregen.GeneratorFactory.createGenerator( > GeneratorFactory.java:661) > at opennlp.tools.util.featuregen.GeneratorFactory$ > AggregatedFeatureGeneratorFactory.create(GeneratorFactory.java:129) > at opennlp.tools.util.featuregen.GeneratorFactory.createGenerator( > GeneratorFactory.java:661) > at opennlp.tools.util.featuregen.GeneratorFactory.create( > GeneratorFactory.java:711) > at opennlp.tools.namefind.TokenNameFinderFactory.createFeatureGenerators( > TokenNameFinderFactory.java:153) > ... 4 more > > 2016-10-28 12:55 GMT+02:00 Joern Kottmann : > >> Try to rename the dictionary key to xyz.dictionary then the serializer >> will >> be mapped correctly. >> >> Jörn >> >> On Thu, Oct 27, 2016 at 11:14 PM, Damiano Porta >> wrote: >> >> > Jorn i add the Dictionary here: >> > https://gist.github.com/anonymous/bc822fb0520c4c42b75748bf4147da >> > 34#file-train-java-L15 >> > >> > And unfortunately i only see this error: >> > >> > java.lang.IllegalStateException: Missing serializer for damiano >> > at opennlp.tools.util.model.BaseModel.serialize(BaseModel.java:610) >> > >> > I do not have other info. >> > Do i have to create a custom Serializer too? >> > >> > >> > >> > >> > 2016-10-27 22:04 GMT+02:00 Joern Kottmann : >> > >> > > On Thu, 2016-10-27 at 21:18 +0200, Joern Kottmann wrote: >> > > > On Tue, 2016-10-25 at 18:49 +0200, Damiano Porta wrote: >> > > > > >> > > > > i am getting a strange error during the compiling of a NER model. >> > > > > Basically, the end of the build output is: >> > > > > >> > > > > 98: ... loglikelihood=-13340.018762351776 0.999005934601099 >> > > > > 99: ... loglikelihood=-13258.358751926637 0.9990120681028991 >> > > > > 100: ... loglikelihood=-13178.039964721707 0.9990177634974279 >> > > > > Exception in thread "main" java.lang.IllegalStateException: >> Missing >> > > > > serializer for *mydictionary* >> > > > > at opennlp.tools.util.model.BaseModel.serialize(BaseModel.java: >> 610) >> > > > >> > > > >> > > > Can you please post the full exception stack trace? >> > > > >> > > >> > > >> > > And what is the name of they key you used for the dictionary? >> > > The dictionary serializers are only mapped by extension. >> > > >> > > Jörn >> > > >> > >> > >
Re: Why can i not serialize a Dictionary ?
Jorn i change the code as you told me, this exactly: https://gist.github.com/anonymous/8877b09d441d2e64c181fa9b5ade4550#file-test-java-L15 but i get this error: opennlp.tools.namefind.TokenNameFinderModel$FeatureGeneratorCreationError: opennlp.tools.util.InvalidFormatException: No dictionary resource for key: damiano at opennlp.tools.namefind.TokenNameFinderFactory.createFeatureGenerators(TokenNameFinderFactory.java:176) at opennlp.tools.namefind.TokenNameFinderFactory.createContextGenerator(TokenNameFinderFactory.java:118) at opennlp.tools.namefind.NameFinderME.train(NameFinderME.java:333) at com.damiano.parser.trainer.NER.compileNER(NER.java:185) at com.damiano.parser.trainer.NER.main(NER.java:155) Caused by: opennlp.tools.util.InvalidFormatException: No dictionary resource for key: damiano at opennlp.tools.util.featuregen.GeneratorFactory$DictionaryFeatureGeneratorFactory.create(GeneratorFactory.java:251) at opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(GeneratorFactory.java:661) at opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFeatureGeneratorFactory.create(GeneratorFactory.java:129) at opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(GeneratorFactory.java:661) at opennlp.tools.util.featuregen.GeneratorFactory$CachedFeatureGeneratorFactory.create(GeneratorFactory.java:171) at opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(GeneratorFactory.java:661) at opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFeatureGeneratorFactory.create(GeneratorFactory.java:129) at opennlp.tools.util.featuregen.GeneratorFactory.createGenerator(GeneratorFactory.java:661) at opennlp.tools.util.featuregen.GeneratorFactory.create(GeneratorFactory.java:711) at opennlp.tools.namefind.TokenNameFinderFactory.createFeatureGenerators(TokenNameFinderFactory.java:153) ... 4 more 2016-10-28 12:55 GMT+02:00 Joern Kottmann : > Try to rename the dictionary key to xyz.dictionary then the serializer will > be mapped correctly. > > Jörn > > On Thu, Oct 27, 2016 at 11:14 PM, Damiano Porta > wrote: > > > Jorn i add the Dictionary here: > > https://gist.github.com/anonymous/bc822fb0520c4c42b75748bf4147da > > 34#file-train-java-L15 > > > > And unfortunately i only see this error: > > > > java.lang.IllegalStateException: Missing serializer for damiano > > at opennlp.tools.util.model.BaseModel.serialize(BaseModel.java:610) > > > > I do not have other info. > > Do i have to create a custom Serializer too? > > > > > > > > > > 2016-10-27 22:04 GMT+02:00 Joern Kottmann : > > > > > On Thu, 2016-10-27 at 21:18 +0200, Joern Kottmann wrote: > > > > On Tue, 2016-10-25 at 18:49 +0200, Damiano Porta wrote: > > > > > > > > > > i am getting a strange error during the compiling of a NER model. > > > > > Basically, the end of the build output is: > > > > > > > > > > 98: ... loglikelihood=-13340.018762351776 0.999005934601099 > > > > > 99: ... loglikelihood=-13258.358751926637 0.9990120681028991 > > > > > 100: ... loglikelihood=-13178.039964721707 0.9990177634974279 > > > > > Exception in thread "main" java.lang.IllegalStateException: > Missing > > > > > serializer for *mydictionary* > > > > > at opennlp.tools.util.model.BaseModel.serialize(BaseModel. > java:610) > > > > > > > > > > > > Can you please post the full exception stack trace? > > > > > > > > > > > > > And what is the name of they key you used for the dictionary? > > > The dictionary serializers are only mapped by extension. > > > > > > Jörn > > > > > >
Re: Why can i not serialize a Dictionary ?
Try to rename the dictionary key to xyz.dictionary then the serializer will be mapped correctly. Jörn On Thu, Oct 27, 2016 at 11:14 PM, Damiano Porta wrote: > Jorn i add the Dictionary here: > https://gist.github.com/anonymous/bc822fb0520c4c42b75748bf4147da > 34#file-train-java-L15 > > And unfortunately i only see this error: > > java.lang.IllegalStateException: Missing serializer for damiano > at opennlp.tools.util.model.BaseModel.serialize(BaseModel.java:610) > > I do not have other info. > Do i have to create a custom Serializer too? > > > > > 2016-10-27 22:04 GMT+02:00 Joern Kottmann : > > > On Thu, 2016-10-27 at 21:18 +0200, Joern Kottmann wrote: > > > On Tue, 2016-10-25 at 18:49 +0200, Damiano Porta wrote: > > > > > > > > i am getting a strange error during the compiling of a NER model. > > > > Basically, the end of the build output is: > > > > > > > > 98: ... loglikelihood=-13340.018762351776 0.999005934601099 > > > > 99: ... loglikelihood=-13258.358751926637 0.9990120681028991 > > > > 100: ... loglikelihood=-13178.039964721707 0.9990177634974279 > > > > Exception in thread "main" java.lang.IllegalStateException: Missing > > > > serializer for *mydictionary* > > > > at opennlp.tools.util.model.BaseModel.serialize(BaseModel.java:610) > > > > > > > > > Can you please post the full exception stack trace? > > > > > > > > > And what is the name of they key you used for the dictionary? > > The dictionary serializers are only mapped by extension. > > > > Jörn > > >
Re: Why can i not serialize a Dictionary ?
Jorn i add the Dictionary here: https://gist.github.com/anonymous/bc822fb0520c4c42b75748bf4147da34#file-train-java-L15 And unfortunately i only see this error: java.lang.IllegalStateException: Missing serializer for damiano at opennlp.tools.util.model.BaseModel.serialize(BaseModel.java:610) I do not have other info. Do i have to create a custom Serializer too? 2016-10-27 22:04 GMT+02:00 Joern Kottmann : > On Thu, 2016-10-27 at 21:18 +0200, Joern Kottmann wrote: > > On Tue, 2016-10-25 at 18:49 +0200, Damiano Porta wrote: > > > > > > i am getting a strange error during the compiling of a NER model. > > > Basically, the end of the build output is: > > > > > > 98: ... loglikelihood=-13340.018762351776 0.999005934601099 > > > 99: ... loglikelihood=-13258.358751926637 0.9990120681028991 > > > 100: ... loglikelihood=-13178.039964721707 0.9990177634974279 > > > Exception in thread "main" java.lang.IllegalStateException: Missing > > > serializer for *mydictionary* > > > at opennlp.tools.util.model.BaseModel.serialize(BaseModel.java:610) > > > > > > Can you please post the full exception stack trace? > > > > > And what is the name of they key you used for the dictionary? > The dictionary serializers are only mapped by extension. > > Jörn >
Re: Why can i not serialize a Dictionary ?
On Thu, 2016-10-27 at 21:18 +0200, Joern Kottmann wrote: > On Tue, 2016-10-25 at 18:49 +0200, Damiano Porta wrote: > > > > i am getting a strange error during the compiling of a NER model. > > Basically, the end of the build output is: > > > > 98: ... loglikelihood=-13340.018762351776 0.999005934601099 > > 99: ... loglikelihood=-13258.358751926637 0.9990120681028991 > > 100: ... loglikelihood=-13178.039964721707 0.9990177634974279 > > Exception in thread "main" java.lang.IllegalStateException: Missing > > serializer for *mydictionary* > > at opennlp.tools.util.model.BaseModel.serialize(BaseModel.java:610) > > > Can you please post the full exception stack trace? > And what is the name of they key you used for the dictionary? The dictionary serializers are only mapped by extension. Jörn
Re: Why can i not serialize a Dictionary ?
On Tue, 2016-10-25 at 18:49 +0200, Damiano Porta wrote: > i am getting a strange error during the compiling of a NER model. > Basically, the end of the build output is: > > 98: ... loglikelihood=-13340.018762351776 0.999005934601099 > 99: ... loglikelihood=-13258.358751926637 0.9990120681028991 > 100: ... loglikelihood=-13178.039964721707 0.9990177634974279 > Exception in thread "main" java.lang.IllegalStateException: Missing > serializer for *mydictionary* > at opennlp.tools.util.model.BaseModel.serialize(BaseModel.java:610) Can you please post the full exception stack trace? Jörn
Why can i not serialize a Dictionary ?
Hello,
i am getting a strange error during the compiling of a NER model.
Basically, the end of the build output is:
98: ... loglikelihood=-13340.018762351776 0.999005934601099
99: ... loglikelihood=-13258.358751926637 0.9990120681028991
100: ... loglikelihood=-13178.039964721707 0.9990177634974279
Exception in thread "main" java.lang.IllegalStateException: Missing
serializer for *mydictionary*
at opennlp.tools.util.model.BaseModel.serialize(BaseModel.java:610)
Where "mydictionary" is the key of my custom dictionary (that i add on the
resources)
I load the dictionary with this code:
HashMap map = new HashMap<>();
try (InputStream modelIn = new
FileInputStream("/home/damiano/mydictionary.xml")) {
Dictionary dictionary = new Dictionary(modelIn);
map.put("mydictionary", dictionary);
}
catch(Exception e) {
System.out.println(e.getMessage());
}
Then i create the TokenNameFinderFactory with:
TokenNameFinderFactory factory = new TokenNameFinderFactory(
IOUtils.toByteArray(in),
map,
new BioCodec()
);
where "*in"* is the InputStream of my featuregenerators.xml
and map is the Map with my resources.
then train the model with:
TokenNameFinderModel model = NameFinderME.train("it", "person",
sampleStream, TrainingParameters.defaultParams(), factory);
at the end i serialize the model doing:
try (BufferedOutputStream modelOut = new BufferedOutputStream(new
FileOutputStream(outputFile.toFile( {
model.serialize(modelOut);
}
As i wrote at the beginning it cant serialize my dictionary so the build of
the model does not finish properly.
I read that a Dictionary can be serialized, so why this error?
Thank you!
Damiano
