Re: [Moses-support] moses installation problem

There is not dist directory anymore, only a bin/ directory Hieu Sent from my flying horse On 7 Aug 2012, at 06:39 AM, ygra...@computing.dcu.ie wrote: Hi Kenneth, Thanks for that. I now have no errors but also no dist/bin folder. I found a thread with that problem in the archive but can't

Re: [Moses-support] Using mosesserver

Hi Duygu The only documentation on moses server is here: http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc23 There are several Moses tutorials on the Moses website, and you should also be able to find something on using xml-rpc from python. To run the moses server, yes you just need to

Re: [Moses-support] how does Moses handle with the apostrophes?

Hi Jun Is the apostrophe in your source data an ascii apostrophe, or a unicode variant (use xxd to check this)? As Tom said, recent versions of the Moses tokeniser escape apostrophes, so either you're using an old version, or it does not recognise it as an apostrophe. Make sure you are using

Re: [Moses-support] how does Moses handle with the apostrophes?

Hi Barry, How to check the Moses version? I'm sure that the tokeniser for training is same with testing. I'm using Standford Word Segmenter for Chinese language. -Original Message- From: Barry Haddow [mailto:bhad...@staffmail.ed.ac.uk] Sent: Tuesday, August 07, 2012 4:43 PM To: Tan,

Re: [Moses-support] how does Moses handle with the apostrophes?

Hi Moses, I'm not sure if your problem has any relation with whether the apostrophe is escaped or not. If you open the script mosesdecoder/scripts/tokenizer/escape-special-chars.perl line 19 : s/\'/\apos;/g; # xml This means that the apostrophe is escaped with respect to XML. Here's my

Re: [Moses-support] how does Moses handle with the apostrophes?

Hi Jun Recent versions of the tokeniser have a line like $text =~ s/\'/\apos;/g; # xml to escape apostrophes. cheers - Barry On 07/08/12 09:51, Tan, Jun wrote: Hi Barry, How to check the Moses version? I'm sure that the tokeniser for training is same with testing. I'm using Standford

Re: [Moses-support] how does Moses handle with the apostrophes?

Hi Barry, I think the version is new, below is output from the file tokenizer.perl #escape special chars $text =~ s/\/\amp;/g; # escape escape $text =~ s/\|/\#124;/g; # factor separator $text =~ s/\/\lt;/g;# xml $text =~ s/\/\gt;/g;# xml $text =~ s/\'/\apos;/g; # xml

Re: [Moses-support] how does Moses handle with the apostrophes?

Hi Jun If you're using this version of the tokeniser on your source sentence, then I would expect it to convert the apostrophe to apos; The fact that there is no apos; in your output suggests that either the decoder is translating it to ' (unlikely) or the apostrophe in your source is not a

Re: [Moses-support] how does Moses handle with the apostrophes?

Marwen, yes. Tokenizer.perl escapes the five reserved XML characters to avoid conflicts with XML-RPC support. It also escapes other reserved characters used internally in Moses such as the vertical bar | character and square brackets [ and ]. On Tue, 07 Aug 2012 11:42:57 +0200, Marwen AZOUZI

Re: [Moses-support] how does Moses handle with the apostrophes?

Tan, It's difficult to tell through email if the character is ASCII or Unicode. On my system it appears to be ASCII. Can you save those two sentences in a UTF-8 text file, tarball them and forward? Tom On Tue, 07 Aug 2012 12:12:52 +0700, Tom Hoar wrote: If you're using Moses'

[Moses-support] Positive decoder scores

Hi all, Is it possible to have a positive number for weighted overall decoder score? I previously thought that those are log probabilities thus negative. However, in a few n-best list, I found positive overall scores (the last score in each line) for some translations. This happened with

[Moses-support] How to use queryLexicalTable

Hi all, I'd like to look up some entries in my reordering models. Does anyone know how to use queryLexicalTable for that? Calling ./queryLexicalTable -table ~/path/to/reordering model -f foreign phrase -e English phrase -c context of phrase does not work for me. Any ideas? Regards, Daniel

[Moses-support] BitmapContainer.cpp:418: check failed

Hi Everyone, as I am new with Moses, I would very much appreciate help with the following issue. The Moses installation I use fails during the decoding with this error message: Collecting options took 0.130 seconds Check item-GetHypothesis()-GetTotalScore() =

Re: [Moses-support] Experiment error: mkcls

Dear all, I've not solved the problem, could someone help me? Thank you Mauro On Fri, Aug 3, 2012 at 12:07 PM, Mauro Zanotti mau.zano...@gmail.comwrote: Dear all, I launched an experiment in the new moses installation (latest version of irstlm and randlm, yesterday git version of moses)

Re: [Moses-support] Experiment error: mkcls

Hi Mauro It looks like train-model.perl is not reading your external-bin-dir parameter, but this should be picked up by checks earlier in the script. Could you post TRAINING_prepare-data.4 and your ems config file? cheers - Barry On 07/08/12 15:00, Mauro Zanotti wrote: Dear all, I've not

Re: [Moses-support] Positive decoder scores

Hi Rasul Positive scores are possible, but unusual, depending on the feature weights. The scores can be interpreted as log probabilities, but they have to be nornalised first. The standard Moses feature functions are explained here http://www.statmt.org/moses/?n=Moses.FeatureFunctions cheers

Re: [Moses-support] Experiment error: mkcls

Hi Mauro It looks as though you're using an old version of experiment.perl. Can you try updating? cheers - Barry On 07/08/12 17:31, Mauro Zanotti wrote: Hi Barry, attached you will find the files. Thank you Mauro On Tue, Aug 7, 2012 at 4:29 PM, Barry

Re: [Moses-support] how does Moses handle with the apostrophes?

Hi Barry, I have checked the source data for training. I found that some of the apostrophe already got converted to 'apos;', but there are still some apostrophe like ’and #91; . With my understanding, the tool you mentioned will convert the apostrophe from Unicode to ASCIII, so the tool can