Dears,
I have a BLEU result with 77.67
I used files with 5000 lines to test with, could this result be a fake one?
as the quality does not fit this result as I think
Best Regards
Ihab Ramadan| Senior Developer| http://www.saudisoft.com/ Saudisoft -
Egypt | Tel +2 02 330 320 37 Ext- 0 |
5,000 lines is way too many. You probably only need 2000-3000.
Here are my questions:
1. How many duplicate segments (source target sentence pairs) are
replicated/repeated in your training corpus?
2. Are your test segments similar to/same/different from your tuning set?
3. How did you
Hi please help me,
I have been can run Moses and get a score of BLUE is: BLEU= 74.25,
86.2/77.1/70.7/64.8 (BP=1.000, ratio= 1.000, hyp_len= 59134, ref_len= 59144)
but I dont understand these values. Can you explain the meaning of each of
these values?Therefore, I have to explain my current
Hi Prashant
I tried to answer your questions inline:
On 12/11/14 20:27, Prashant Mathur wrote:
Hi All,
I have a question about implementing sparse feature function.
I went through the details on its implementation, still somethings are
not clear.
FYI, I am using an old version of moses
Dear moses,
I have the following error in training my translation model with ems on SGE.
/home/users/moses/v-2014-08-15/mosesdecoder/bin/build_binary: error while
loading shared libraries: libboost_thread-mt.so.1.56.0: cannot open shared
object file: No such file or directory
In the offline
re-iterating what Barry said, you should use the github moses if you want
to create your own feature functions, especially with sparse features. The
reasons:
1. Adding new feature functions is a pain in v 0.91. It's trivial now.
You can watch my talk to find out why
personally, I would try to ensure that moses was statically linked to the
boost libraries so it won't go looking for the libboost*.so at runtime.
I do this by building my own boost library and linking to it, rather than
the system boost. The instructions are here
Thanks Hieu,
I did link boost (and sirlm, irstlm) when i built moses.
./bjam --with-boost=/export/sw/std/boost/v1.56.0/x86_64
--with-srilm=/export/sw/spl/srilm/v1.6.0/srilm
--with-irstlm=/export/sw/spl/irstlm/v5.80.03/x86_64 -j8
But I think at run time I still need to load the path.
setenv
you don't need to set LD_LIBRARY_PATH if the boost lib statically linked.
This is how I always use it, I never set this variable.
You need to make sure there's no .so file in the folder
/export/sw/std/boost/v1.56.0/x86_64/lib
/export/sw/std/boost/v1.56.0/x86_64/lib64
On 13 November 2014
Alternatively, if you must link dynamically, configure your shell to
always run
setenv LD_LIBRARY_PATH /export/sw/std/boost/v1.56.0/x86_64/lib
by adding it to whatever file csh sources when it runs on the cluster.
It would help if you posted --debug-configuration from bjam (gzip the
stdout).
Dear Moses,
Do the -e and -f arguments to train-model.perl and clean-corpus-n.perl
actually get interpreted by anything? Or are they just there as file
name extensions that could just as easily be src and tgt? I think
it doesn't matter.
Kenneth
Hi Kenneth,
In train-model.perl, the -e and -f arguments are used to determine
filenames and extensions so they could easily be changed to src and tgt
within the script. But Tom has a handle on how this could be painful to
change in the wrapper code. clean-corpus-n.perl doesn't have a -e and -f
Perl being the worlds best programming language allows for options like
f|src :)
I have to admit I always have to pause for half a second, mumbling f
stands for 'from' ...
W dniu 2014-11-13 16:21, Alexandra Birch napisał(a):
Hi Kenneth,
In train-model.perl, the -e and -f arguments
I'm not proposing to change the script or the arguments. Just want to
make sure that somebody didn't write an if statement three levels deep
in perl that deletes words longer than 100 words unless the -f
language string is de.
On 11/13/14 10:17, Tom Hoar wrote:
Are you kidding? I thought
Thanks a lot Barry for your answers.
I have another question.
When I print these sparse features at the end of decoding, all sparse
features are assigned a weight of 0 because all of them were initialized
during decoding.
How can I set these weights for sparse features before they are evaluated?
Hi Prashant
You add something like this to your moses.ini:
[weight-file]
/path/to/sparse/weights/file
The sparse weights file has the form:
name1 weight1
name2 weight2
name3 weight3
.
.
.
At least that's how it works in Moses v2.
cheers - Barry
On 13/11/14 15:42, Prashant Mathur wrote:
what if I don't know the feature names before hand?
In that case, can I set the weights directly during decoding?
On Thu, Nov 13, 2014 at 4:59 PM, Barry Haddow bhad...@staffmail.ed.ac.uk
wrote:
Hi Prashant
You add something like this to your moses.ini:
[weight-file]
Hi Prashant
I don't think that is possible without code changes. There are methods
in StaticData to update the feature weights, which are used by the
online mira implementation, so you could look to see how this is done,
cheers - Barry
On 13/11/14 16:03, Prashant Mathur wrote:
what if I
Hi,
Typically you want to learn these feature weights when tuning. The current
setup supports and produces a sparse feature file.
-phi
On Nov 13, 2014 11:18 AM, Prashant Mathur prash...@fbk.eu wrote:
what if I don't know the feature names before hand?
In that case, can I set the weights
Hi Hieu,
I just recompiled moses to link boost statically.
When I did this last time months ago, I remember I failed to link boost
statically. I didn't trace the error and just had a dynamic build.
A minor trick may be necessary for others to link boost statically when
compiling moses
In my
But in CAT scenario we do like this:
translate: sentence 1
tune: sentence 1 , post-edit 1
translate: sentence 2
tune: sentence 2 , post-edit 2
...
In this case, I don't have any features generated or tuned before I start
translating the first sentence.
Old version is complicated, I am coding on
this was due to a change a couple of months ago.
https://www.mail-archive.com/moses-support%40mit.edu/msg10653.html
The documentation to create the boost libraries has been updated
http://www.statmt.org/moses/?n=Development.GetStarted
On 13 November 2014 16:59, Raymond W. M. Ng
Dear Moses,
The PYTHONIOENCODING environment variable sets Python's default
encoding because Python is too lame to default to utf-8. When is not
set, and a particular flavor of python is installed, merge_alignment.py
from mgiza fails. That triggers a snippet of train-model.perl output
Hello,
I'am trying to build the baseline SMT for english to german language. I've
got a strange problem because i work for the 1st time on german language
and i've got a score Bleu on Dev corpus = 16 and on the test corpus = 9 !
i prepared the corpus before training and mert as usual like this:
Hi,
if the tuning and test set are sampled from the same data, you should expect
similar scores. You could swap the two sets and see what happens - maybe
it is just that they are of different difficulty.
Phrase-based models have some deficiencies when it comes to German, such as
translation of
25 matches
Mail list logo
