The feature function initialization message is just a general purpose exception handler. I’ve seen this quite often when language models fail to load. The most interesting part of the log to me is:
> Caused by: java.lang.RuntimeException: Something wrong with I/O. > > at edu.berkeley.nlp.lm.io.ArpaLmReader.parseHeader(ArpaLmReader.java:114) > > at edu.berkeley.nlp.lm.io.ArpaLmReader.parse(ArpaLmReader.java:76) To me it looks like it could only be caused by the lack of the text "\\1-grams:" in the file you’re opening. Reference this function: https://github.com/smilli/berkeleylm/blob/master/src/edu/berkeley/nlp/lm/io/ArpaLmReader.java#L105 Are you trying to load a binary lm with an Arpa reader by any chance? Do you have the quoted text in your text based LM? -Kellen From: Tommaso Teofili Sent: Monday, October 16, 2017 4:09 PM To: dev@joshua.incubator.apache.org Subject: Re: problems with LM loading p.s.: I've tried with other LPs (e.g. sd-en) and I get the same ... Il giorno lun 16 ott 2017 alle ore 15:06 Tommaso Teofili < tommaso.teof...@gmail.com> ha scritto: > Hi all, > > I am trying to use the ES-EN language pack from our "Language Packs" page > with Joshua 6.1, but when I get to load the two language models I get an IO > execption. > The config looks like: > > feature-function = LanguageModel -lm_type berkeleylm -lm_order 4 -lm_file > model/lm.berkeleylm > feature-function = Distortion > feature-function = LanguageModel -lm_type berkeleylm -lm_order 4 -lm_file > model/en.giga.twopercent.4.lm.berkeleylm > feature-function = PhrasePenalty > > and I get the following: > > java.lang.RuntimeException: java.lang.RuntimeException: Unable to > instantiate feature function 'LanguageModel -lm_type berkeleylm -lm_order 4 > -lm_file model/lm.berkeleylm'! > > ... > > Caused by: java.lang.RuntimeException: Unable to instantiate feature > function 'LanguageModel -lm_type berkeleylm -lm_order 4 -lm_file > model/lm.berkeleylm'! > > at > org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:642) > > at org.apache.joshua.decoder.Decoder.initialize(Decoder.java:394) > > at org.apache.joshua.decoder.Decoder.<init>(Decoder.java:128) > > Caused by: java.lang.reflect.InvocationTargetException: null > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > at java.lang.reflect.Constructor.newInstance(Constructor.java:422) > > at > org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:638) > > ... 58 common frames omitted > > Caused by: java.lang.RuntimeException: Something wrong with I/O. > > at edu.berkeley.nlp.lm.io.ArpaLmReader.parseHeader(ArpaLmReader.java:114) > > at edu.berkeley.nlp.lm.io.ArpaLmReader.parse(ArpaLmReader.java:76) > > at edu.berkeley.nlp.lm.io.ArpaLmReader.parse(ArpaLmReader.java:18) > > at edu.berkeley.nlp.lm.io.LmReaders.firstPassCommon(LmReaders.java:549) > > at edu.berkeley.nlp.lm.io.LmReaders.firstPassArpa(LmReaders.java:526) > > at > edu.berkeley.nlp.lm.io.LmReaders.readArrayEncodedLmFromArpa(LmReaders.java:171) > > at > edu.berkeley.nlp.lm.io.LmReaders.readArrayEncodedLmFromArpa(LmReaders.java:151) > > at > org.apache.joshua.decoder.ff.lm.berkeley_lm.LMGrammarBerkeley.<init>(LMGrammarBerkeley.java:94) > > at > org.apache.joshua.decoder.ff.lm.LanguageModelFF.initializeLM(LanguageModelFF.java:158) > > at > org.apache.joshua.decoder.ff.lm.LanguageModelFF.<init>(LanguageModelFF.java:132) > > Any hints on what I could be doing wrong ? Encoding ? > Did anyone else experience such issue ? > > BTW I am running this from within a Java application, Decoder is > initialized as follows: > > JoshuaConfiguration configuration = new JoshuaConfiguration(); > configuration.readConfigFile(pathToJoshuaConfig); > configuration.use_structured_output = true; > Decoder decoder = new Decoder(configuration, pathToJoshuaConfig); > > Regards, > Tommaso >
