Re: [Moses-support] Moses compile fails with irstlm and g++4.3
I just tried RANDLM and it compiles with g++4.3. So, Moses --with-irstlm is the only configuration with g++4.3 problems. 4) Moses configured --with-randm=/usr/local/src/randlm compiles successfully Tom On Tue, Jul 28, 2009 at 8:26 AM, Tom Hoar wrote: > On my clean system build with g++4.3 (i.e. g++4.1 never installed): > > 1) GIZA++, SRILM and IRSTLM compile successfully > 2) Moses configured --with-srilm=/usr/local/src/srilm compiles successfully > 3) Moses configured --with-irstlm=/usr/local/src/irstlm fails to compile > with the errors below. > > What else do we need to include? > > Tom > > * * * * * * > In file included from LanguageModelIRST.cpp:28: > /usr/local/src/irstlm/include/lmtable.h:227: warning: unused parameter > 'lmfile' > /usr/local/src/irstlm/include/lmtable.h: In member function 'void > lmtable::filter2(const char*, int)': > /usr/local/src/irstlm/include/lmtable.h:230: error: 'exit' was not > declared in this scope > /usr/local/src/irstlm/include/lmtable.h: At global scope: > /usr/local/src/irstlm/include/lmtable.h:228: warning: unused parameter > 'lmfile' > /usr/local/src/irstlm/include/lmtable.h:228: warning: unused parameter > 'buffMb' > . > . > . > PhraseDictionaryTree.cpp:591: instantiated from here > PrefixTree.h:141: warning: ignoring return value of 'size_t fread(void*, > size_t, size_t, FILE*)', declared with attribute warn_unused_result > make[3]: *** [LanguageModelIRST.o] Error 1 > make[3]: *** Waiting for unfinished jobs > LanguageModelInternal.cpp: In member function 'virtual bool > Moses::LanguageModelInternal::Load(const std::string&, Moses::FactorType, > float, size_t)': > LanguageModelInternal.cpp:71: warning: 'nGram' may be used uninitialized in > this function > LanguageModelInternal.cpp:72: warning: 'factor' may be used uninitialized > in this function > make[3]: Leaving directory `/usr/local/src/moses-irstlm/moses/src' > make[2]: *** [all] Error 2 > make[2]: Leaving directory `/usr/local/src/moses-irstlm/moses/src' > make[1]: *** [all-recursive] Error 1 > make[1]: Leaving directory `/usr/local/src/moses-irstlm' > make: *** [all] Error 2 > * * * * * * > > ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] Moses compile fails with irstlm and g++4.3
On my clean system build with g++4.3 (i.e. g++4.1 never installed): 1) GIZA++, SRILM and IRSTLM compile successfully 2) Moses configured --with-srilm=/usr/local/src/srilm compiles successfully 3) Moses configured --with-irstlm=/usr/local/src/irstlm fails to compile with the errors below. What else do we need to include? Tom * * * * * * In file included from LanguageModelIRST.cpp:28: /usr/local/src/irstlm/include/lmtable.h:227: warning: unused parameter 'lmfile' /usr/local/src/irstlm/include/lmtable.h: In member function 'void lmtable::filter2(const char*, int)': /usr/local/src/irstlm/include/lmtable.h:230: error: 'exit' was not declared in this scope /usr/local/src/irstlm/include/lmtable.h: At global scope: /usr/local/src/irstlm/include/lmtable.h:228: warning: unused parameter 'lmfile' /usr/local/src/irstlm/include/lmtable.h:228: warning: unused parameter 'buffMb' . . . PhraseDictionaryTree.cpp:591: instantiated from here PrefixTree.h:141: warning: ignoring return value of 'size_t fread(void*, size_t, size_t, FILE*)', declared with attribute warn_unused_result make[3]: *** [LanguageModelIRST.o] Error 1 make[3]: *** Waiting for unfinished jobs LanguageModelInternal.cpp: In member function 'virtual bool Moses::LanguageModelInternal::Load(const std::string&, Moses::FactorType, float, size_t)': LanguageModelInternal.cpp:71: warning: 'nGram' may be used uninitialized in this function LanguageModelInternal.cpp:72: warning: 'factor' may be used uninitialized in this function make[3]: Leaving directory `/usr/local/src/moses-irstlm/moses/src' make[2]: *** [all] Error 2 make[2]: Leaving directory `/usr/local/src/moses-irstlm/moses/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/local/src/moses-irstlm' make: *** [all] Error 2 * * * * * * ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] Fast EM Model 1
BTW, just wanted to pass this on to you guys. As you may have gathered from recent questions I've been implementing EM Model 1 in a simple code. I've got different versions of the code using different data structures and I have a few observations which I think you may be interested in. Obviously, the central and most performance crucial decision is the data structures you use to code your counts, totals and probabilities. For the counts and probabilities I've experimented with a number of strategies including using simple 2D arrays, linked lists, hash tables and hybrid data structures. From a performance point of view the 2D array is the fastest to lookup an e|f entry and update as you can address directly. From a memory efficient point of view you can fit much more into memory using a linked list or a hash table. The reason for this is simply that the 2D array is extremely sparse and many of the e|f pairs a complete array would have simply do not exist in any corpus i.e. this is a sparse matrix. However, using a hash table is much slower than using a 2D array. So this is the trick I did. I first find all the words in a corpus, count their frequencies and then sort them by their frequencies giving the most frequent words the lowest ids. The laws of statistics thus make the sparse matrix have a dense quadrant in the upper left corner. I then grow the 2D array in memory to a maximum but making sure that it does not overflow into swap space. I then filter the corpus such that only sentences which consist of words in the 2D array bounds. The results were surprisingly positive. By making such a dense matrix you can run EM on a large proportion of sentences with a much smaller 2D array (I use a 2D array of structs with a count and a probability for each e|f). The resulting code is *fast*. It can finish a 5 iteration EM Model 1 on the en-fr europarl corpus in a matter of a few minutes. Using a 1GB table it can process 890477 of 1555073 sentences Using a 2GB table it can process 1043215 of 1555073 sentences Using a 3GB table it can process 1132865 of 1555073 sentences ... Using a 7GB table it can process 1291927 of 1555073 sentences My hash table version is not as fast but it can fit the entire set of e|f pairs in 3.1GB of memory. A hybrid code of 2D array and hash table has shown that you get the best of both worlds. You can do fast updates to counts and probabilities using a 2D table for the most commonly occuring e|f and use the hash table for the rest. My 2D table uses pointers to entries in the hash table and so really everything is in the hash table. The 2D array just gives faster lookups to the structs containing the counts and probabilities. Just thought this information might be interesting to some of you as GIZA++ seems to be taking the lion's share of the processing time in training Moses. James -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] EM Model 1 question
Yeah. I kind of figured that this would be the case. The inherent
problems of using floating point numbers and rounding errors and all
that kind of thing.
Quoting Miles Osborne :
> the good thing about probabilities is that they should sum to one
>
> (but you can get numerical errors giving you slightly more / less ...)
> Miles
>
> 2009/7/27 James Read
>
>> Ok. Thanks. I think I understand this now. I also think I have found
>> the bug in the code which was causing the dodgy output.
>>
>> So, in conclusion, would you say that a good automated check to see if
>> the code is working correctly would be to add up the probabilities at
>> the end of the EM iterations and check that probabilities add up to 1
>> (or slightly less)?
>>
>> James
>>
>> Quoting Philipp Koehn :
>>
>> > Hi,
>> >
>> > because the final loop in each iteration is:
>> >
>> > // estimate probabilities
>> > for all foreign words f
>> > for all English words e
>> > t(e|f) = count}(e|f) / total(f)
>> >
>> > As I said, there are two normalizations: one on the
>> > sentence level, the other on the corpus level.
>> >
>> > -phi
>> >
>> > On Mon, Jul 27, 2009 at 10:30 PM, James Read
>> wrote:
>> >> In that case I really don't see how the code is guaranteed to give
>> results
>> >> which add up to 1.
>> >>
>> >> Quoting Philipp Koehn :
>> >>
>> >>> Hi,
>> >>>
>> >>> this is LaTex {algorithmic} code.
>> >>>
>> >>> count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
>> >>>
>> >>> means
>> >>>
>> >>> count(e|f) += t(e|f) / s-total(e)
>> >>>
>> >>> So, you got that right.
>> >>>
>> >>> -phi
>> >>>
>> >>> On Mon, Jul 27, 2009 at 10:18 PM, James Read
>> wrote:
>>
>> Hi,
>>
>> this seems to be pretty much what I implemented. What exactly do you
>> mean
>> by
>> these three lines?:
>>
>> \STATE count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
>> \STATE total($f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
>> \STATE $t(e|f)$ = $\frac{\text{count}(e|f)}{\text{total}(f)}$
>>
>> What do you mean by $\frac? The pseudocode I was using shows these
>> lines
>> as
>> a simple division and this is what my code does. i.e
>>
>> t(e|f) = count(e|f) / total(f)
>>
>> In C code something like:
>>
>> for ( f = 0; f < size_source; f++ )
>> {
>> for ( e = 0; e < size_target; e++ )
>> {
>> t[f][e] = count[f][e] / total[f];
>> }
>> }
>>
>>
>> Is this the kind of thing you mean?
>>
>> Thanks
>> James
>>
>> Quoting Philipp Koehn :
>>
>> > Hi,
>> >
>> > I think there was a flaw in some versions of the pseudo code.
>> > The probabilities certainly need to add up to one. There are
>> > two normalizations going on in the algorithm: one on the sentence
>> > level (so the probability of all alignments add up to one) and
>> > one on the word level.
>> >
>> > Here the most recent version:
>> >
>> > \REQUIRE set of sentence pairs $(\text{\bf e},\text{\bf f})$
>> > \ENSURE translation prob. $t(e|f)$
>> > \STATE initialize $t(e|f)$ uniformly
>> > \WHILE{not converged}
>> > \STATE \COMMENT{initialize}
>> > \STATE count($e|f$) = 0 {\bf for all} $e,f$
>> > \STATE total($f$) = 0 {\bf for all} $f$
>> > \FORALL{sentence pairs ({\bf e},{\bf f})}
>> > \STATE \COMMENT{compute normalization}
>> > \FORALL{words $e$ in {\bf e}}
>> > \STATE s-total($e$) = 0
>> > \FORALL{words $f$ in {\bf f}}
>> > \STATE s-total($e$) += $t(e|f)$
>> > \ENDFOR
>> > \ENDFOR
>> > \STATE \COMMENT{collect counts}
>> > \FORALL{words $e$ in {\bf e}}
>> > \FORALL{words $f$ in {\bf f}}
>> > \STATE count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
>> > \STATE total($f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
>> > \ENDFOR
>> > \ENDFOR
>> > \ENDFOR
>> > \STATE \COMMENT{estimate probabilities}
>> > \FORALL{foreign words $f$}
>> > \FORALL{English words $e$}
>> > \STATE $t(e|f)$ = $\frac{\text{count}(e|f)}{\text{total}(f)}$
>> > \ENDFOR
>> > \ENDFOR
>> > \ENDWHILE
>> >
>> > -phi
>> >
>> >
>> >
>> > On Sun, Jul 26, 2009 at 5:24 PM, James Read
>> > wrote:
>> >
>> >> Hi,
>> >>
>> >> I have implemented the EM Model 1 algorithm as outlined in Koehn's
>> >> lecture notes. I was surprised to find the raw output of the
>> algorithm
>> >> gives a translation table that given any particular source word the
>> >> sum of the probabilities of each possible target word is far greater
>> >> than 1.
>> >>
>> >> Is this normal?
>> >>
>> >> Thanks
>> >> James
>> >>
>> >> --
>> >> The University of Edinburgh is a charitable body, registered in
>> >> Scotland, with registration number SC005336.
>> >>
>> >>
>> >> ___
Re: [Moses-support] GIZA++
I also got buffer overflow, rebuilt with an older g++ and then got stack smashing. As per the google code page for GIZA++, under "Issue 11", comment 3, I changed the size of time_stmp in file file_spec.h (in GIZA++) from 17 char's to 37 (pseudo-randomly selected (larger) number), rebuilt things, and had stuff start to work (without using an older g++). Note that there's a newer comment on the issue, comment 4, which may be a less brute-force fix; I haven't tried it. Girard - Original Message - From: maria sol ferrer To: Tom Hoar Cc: moses-support@mit.edu Sent: Sunday, July 26, 2009 8:47 PM Subject: Re: [Moses-support] GIZA++ hey i posted a message about a month ago with the exact same problem, and I still haven´t found a solution here or on the web.. I don´t know if anyone can help us... One of the suggestions was to compile with g++ 4.1 because otherwise it wouldn't run, but that still didn't work for me... If you find anything else please let me know! 2009/7/26 Tom Hoar I recently rebuilt my Moses machine with the latest SVN updates of Moses, SRILM, GIZA++ and compiled with g++4.3. Everything seemed to go fine. In my first training, I ran clean-corpus-n.perl, then train-factored-phrase-model.perl. It terminated with the message below. Can you tell if this problem is corrupted data problem or a result of the compile? Thanks, Tom line 1776000 line 1777000 line 1778000 END. (2.1b) running giza f-e @ Mon Jul 27 08:49:08 ICT 2009 GIZA++ -CoocurrenceFile /usr/local/share/model/20090717-0815.5/giza.f-e/f-e.cooc -c /usr/local/share/model/20090717-0815.5/corpus/f-e-int-train.snt -m1 5 -m2 0 -m3 3 -m4 3 -model1dumpfrequency 1 -model4smoothfactor 0.4 -nodumps 1 -nsmooth 4 -o /usr/local/share/model/20090717-0815.5/giza.f-e/f-e -onlyaldumps 1 -p0 0.999 -s /usr/local/share/model/20090717-0815.5/corpus/e.vcb -t /usr/local/share/model/20090717-0815.5/corpus/f.vcb Executing: GIZA++ -CoocurrenceFile /usr/local/share/model/20090717-0815.5/giza.f-e/f-e.cooc -c /usr/local/share/model/20090717-0815.5/corpus/f-e-int-train.snt -m1 5 -m2 0 -m3 3 -m4 3 -model1dumpfrequency 1 -model4smoothfactor 0.4 -nodumps 1 -nsmooth 4 -o /usr/local/share/model/20090717-0815.5/giza.f-e/f-e -onlyaldumps 1 -p0 0.999 -s /usr/local/share/model/20090717-0815.5/corpus/e.vcb -t /usr/local/share/model/20090717-0815.5/corpus/f.vcb GIZA++ -CoocurrenceFile /usr/local/share/model/20090717-0815.5/giza.f-e/f-e.cooc -c /usr/local/share/model/20090717-0815.5/corpus/f-e-int-train.snt -m1 5 -m2 0 -m3 3 -m4 3 -model1dumpfrequency 1 -model4smoothfactor 0.4 -nodumps 1 -nsmooth 4 -o /usr/local/share/model/20090717-0815.5/giza.f-e/f-e -onlyaldumps 1 -p0 0.999 -s /usr/local/share/model/20090717-0815.5/corpus/e.vcb -t /usr/local/share/model/20090717-0815.5/corpus/f.vcb *** buffer overflow detected ***: GIZA++ terminated === Backtrace: = [0x817971e] [0x81796b2] [0x81794b8] [0x8162488] [0x818576c] [0x8179554] [0x81794ad] [0x8069d7d] [0x806a387] [0x8070d88] [0x8149f4a] [0x8048171] === Memory map: 08048000-081f1000 r-xp 08:11 477229 GIZA++ 081f1000-081f3000 rw-p 001a9000 08:11 477229 GIZA++ 081f3000-081fb000 rw-p 081f3000 00:00 0 08e8d000-08f01000 rw-p 08e8d000 00:00 0 [heap] b7f13000-b7f14000 r-xp b7f13000 00:00 0 [vdso] bfefe000-bff13000 rw-p bffeb000 00:00 0 [stack] ERROR: Execution of: GIZA++ -CoocurrenceFile /usr/local/share/model/20090717-0815.5/giza.f-e/f-e.cooc -c /usr/local/share/model/20090717-0815.5/corpus/f-e-int-train.snt -m1 5 -m2 0 -m3 3 -m4 3 -model1dumpfrequency 1 -model4smoothfactor 0.4 -nodumps 1 -nsmooth 4 -o /usr/local/share/model/20090717-0815.5/giza.f-e/f-e -onlyaldumps 1 -p0 0.999 -s /usr/local/share/model/20090717-0815.5/corpus/e.vcb -t /usr/local/share/model/20090717-0815.5/corpus/f.vcb died with signal 6, without coredump ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] EM Model 1 question
the good thing about probabilities is that they should sum to one
(but you can get numerical errors giving you slightly more / less ...)
Miles
2009/7/27 James Read
> Ok. Thanks. I think I understand this now. I also think I have found
> the bug in the code which was causing the dodgy output.
>
> So, in conclusion, would you say that a good automated check to see if
> the code is working correctly would be to add up the probabilities at
> the end of the EM iterations and check that probabilities add up to 1
> (or slightly less)?
>
> James
>
> Quoting Philipp Koehn :
>
> > Hi,
> >
> > because the final loop in each iteration is:
> >
> > // estimate probabilities
> > for all foreign words f
> > for all English words e
> > t(e|f) = count}(e|f) / total(f)
> >
> > As I said, there are two normalizations: one on the
> > sentence level, the other on the corpus level.
> >
> > -phi
> >
> > On Mon, Jul 27, 2009 at 10:30 PM, James Read
> wrote:
> >> In that case I really don't see how the code is guaranteed to give
> results
> >> which add up to 1.
> >>
> >> Quoting Philipp Koehn :
> >>
> >>> Hi,
> >>>
> >>> this is LaTex {algorithmic} code.
> >>>
> >>> count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
> >>>
> >>> means
> >>>
> >>> count(e|f) += t(e|f) / s-total(e)
> >>>
> >>> So, you got that right.
> >>>
> >>> -phi
> >>>
> >>> On Mon, Jul 27, 2009 at 10:18 PM, James Read
> wrote:
>
> Hi,
>
> this seems to be pretty much what I implemented. What exactly do you
> mean
> by
> these three lines?:
>
> \STATE count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
> \STATE total($f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
> \STATE $t(e|f)$ = $\frac{\text{count}(e|f)}{\text{total}(f)}$
>
> What do you mean by $\frac? The pseudocode I was using shows these
> lines
> as
> a simple division and this is what my code does. i.e
>
> t(e|f) = count(e|f) / total(f)
>
> In C code something like:
>
> for ( f = 0; f < size_source; f++ )
> {
> for ( e = 0; e < size_target; e++ )
> {
> t[f][e] = count[f][e] / total[f];
> }
> }
>
>
> Is this the kind of thing you mean?
>
> Thanks
> James
>
> Quoting Philipp Koehn :
>
> > Hi,
> >
> > I think there was a flaw in some versions of the pseudo code.
> > The probabilities certainly need to add up to one. There are
> > two normalizations going on in the algorithm: one on the sentence
> > level (so the probability of all alignments add up to one) and
> > one on the word level.
> >
> > Here the most recent version:
> >
> > \REQUIRE set of sentence pairs $(\text{\bf e},\text{\bf f})$
> > \ENSURE translation prob. $t(e|f)$
> > \STATE initialize $t(e|f)$ uniformly
> > \WHILE{not converged}
> > \STATE \COMMENT{initialize}
> > \STATE count($e|f$) = 0 {\bf for all} $e,f$
> > \STATE total($f$) = 0 {\bf for all} $f$
> > \FORALL{sentence pairs ({\bf e},{\bf f})}
> > \STATE \COMMENT{compute normalization}
> > \FORALL{words $e$ in {\bf e}}
> > \STATE s-total($e$) = 0
> > \FORALL{words $f$ in {\bf f}}
> > \STATE s-total($e$) += $t(e|f)$
> > \ENDFOR
> > \ENDFOR
> > \STATE \COMMENT{collect counts}
> > \FORALL{words $e$ in {\bf e}}
> > \FORALL{words $f$ in {\bf f}}
> > \STATE count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
> > \STATE total($f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
> > \ENDFOR
> > \ENDFOR
> > \ENDFOR
> > \STATE \COMMENT{estimate probabilities}
> > \FORALL{foreign words $f$}
> > \FORALL{English words $e$}
> > \STATE $t(e|f)$ = $\frac{\text{count}(e|f)}{\text{total}(f)}$
> > \ENDFOR
> > \ENDFOR
> > \ENDWHILE
> >
> > -phi
> >
> >
> >
> > On Sun, Jul 26, 2009 at 5:24 PM, James Read
> > wrote:
> >
> >> Hi,
> >>
> >> I have implemented the EM Model 1 algorithm as outlined in Koehn's
> >> lecture notes. I was surprised to find the raw output of the
> algorithm
> >> gives a translation table that given any particular source word the
> >> sum of the probabilities of each possible target word is far greater
> >> than 1.
> >>
> >> Is this normal?
> >>
> >> Thanks
> >> James
> >>
> >> --
> >> The University of Edinburgh is a charitable body, registered in
> >> Scotland, with registration number SC005336.
> >>
> >>
> >> ___
> >> Moses-support mailing list
> >> Moses-support@mit.edu
> >> http://mailman.mit.edu/mailman/listinfo/moses-support
> >>
> >
>
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
>
> >>>
>
Re: [Moses-support] EM Model 1 question
Ok. Thanks. I think I understand this now. I also think I have found
the bug in the code which was causing the dodgy output.
So, in conclusion, would you say that a good automated check to see if
the code is working correctly would be to add up the probabilities at
the end of the EM iterations and check that probabilities add up to 1
(or slightly less)?
James
Quoting Philipp Koehn :
> Hi,
>
> because the final loop in each iteration is:
>
> // estimate probabilities
> for all foreign words f
> for all English words e
> t(e|f) = count}(e|f) / total(f)
>
> As I said, there are two normalizations: one on the
> sentence level, the other on the corpus level.
>
> -phi
>
> On Mon, Jul 27, 2009 at 10:30 PM, James Read wrote:
>> In that case I really don't see how the code is guaranteed to give results
>> which add up to 1.
>>
>> Quoting Philipp Koehn :
>>
>>> Hi,
>>>
>>> this is LaTex {algorithmic} code.
>>>
>>> count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
>>>
>>> means
>>>
>>> count(e|f) += t(e|f) / s-total(e)
>>>
>>> So, you got that right.
>>>
>>> -phi
>>>
>>> On Mon, Jul 27, 2009 at 10:18 PM, James Read wrote:
Hi,
this seems to be pretty much what I implemented. What exactly do you mean
by
these three lines?:
\STATE count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
\STATE total($f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
\STATE $t(e|f)$ = $\frac{\text{count}(e|f)}{\text{total}(f)}$
What do you mean by $\frac? The pseudocode I was using shows these lines
as
a simple division and this is what my code does. i.e
t(e|f) = count(e|f) / total(f)
In C code something like:
for ( f = 0; f < size_source; f++ )
{
for ( e = 0; e < size_target; e++ )
{
t[f][e] = count[f][e] / total[f];
}
}
Is this the kind of thing you mean?
Thanks
James
Quoting Philipp Koehn :
> Hi,
>
> I think there was a flaw in some versions of the pseudo code.
> The probabilities certainly need to add up to one. There are
> two normalizations going on in the algorithm: one on the sentence
> level (so the probability of all alignments add up to one) and
> one on the word level.
>
> Here the most recent version:
>
> \REQUIRE set of sentence pairs $(\text{\bf e},\text{\bf f})$
> \ENSURE translation prob. $t(e|f)$
> \STATE initialize $t(e|f)$ uniformly
> \WHILE{not converged}
> \STATE \COMMENT{initialize}
> \STATE count($e|f$) = 0 {\bf for all} $e,f$
> \STATE total($f$) = 0 {\bf for all} $f$
> \FORALL{sentence pairs ({\bf e},{\bf f})}
> \STATE \COMMENT{compute normalization}
> \FORALL{words $e$ in {\bf e}}
> \STATE s-total($e$) = 0
> \FORALL{words $f$ in {\bf f}}
> \STATE s-total($e$) += $t(e|f)$
> \ENDFOR
> \ENDFOR
> \STATE \COMMENT{collect counts}
> \FORALL{words $e$ in {\bf e}}
> \FORALL{words $f$ in {\bf f}}
> \STATE count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
> \STATE total($f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
> \ENDFOR
> \ENDFOR
> \ENDFOR
> \STATE \COMMENT{estimate probabilities}
> \FORALL{foreign words $f$}
> \FORALL{English words $e$}
> \STATE $t(e|f)$ = $\frac{\text{count}(e|f)}{\text{total}(f)}$
> \ENDFOR
> \ENDFOR
> \ENDWHILE
>
> -phi
>
>
>
> On Sun, Jul 26, 2009 at 5:24 PM, James Read
> wrote:
>
>> Hi,
>>
>> I have implemented the EM Model 1 algorithm as outlined in Koehn's
>> lecture notes. I was surprised to find the raw output of the algorithm
>> gives a translation table that given any particular source word the
>> sum of the probabilities of each possible target word is far greater
>> than 1.
>>
>> Is this normal?
>>
>> Thanks
>> James
>>
>> --
>> The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
>>
>>
>> ___
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
>>>
>>>
>>
>>
>>
>> --
>> The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
>>
>>
>>
>
>
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] EM Model 1 question
Hi,
because the final loop in each iteration is:
// estimate probabilities
for all foreign words f
for all English words e
t(e|f) = count}(e|f) / total(f)
As I said, there are two normalizations: one on the
sentence level, the other on the corpus level.
-phi
On Mon, Jul 27, 2009 at 10:30 PM, James Read wrote:
> In that case I really don't see how the code is guaranteed to give results
> which add up to 1.
>
> Quoting Philipp Koehn :
>
>> Hi,
>>
>> this is LaTex {algorithmic} code.
>>
>> count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
>>
>> means
>>
>> count(e|f) += t(e|f) / s-total(e)
>>
>> So, you got that right.
>>
>> -phi
>>
>> On Mon, Jul 27, 2009 at 10:18 PM, James Read wrote:
>>>
>>> Hi,
>>>
>>> this seems to be pretty much what I implemented. What exactly do you mean
>>> by
>>> these three lines?:
>>>
>>> \STATE count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
>>> \STATE total($f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
>>> \STATE $t(e|f)$ = $\frac{\text{count}(e|f)}{\text{total}(f)}$
>>>
>>> What do you mean by $\frac? The pseudocode I was using shows these lines
>>> as
>>> a simple division and this is what my code does. i.e
>>>
>>> t(e|f) = count(e|f) / total(f)
>>>
>>> In C code something like:
>>>
>>> for ( f = 0; f < size_source; f++ )
>>> {
>>> for ( e = 0; e < size_target; e++ )
>>> {
>>> t[f][e] = count[f][e] / total[f];
>>> }
>>> }
>>>
>>>
>>> Is this the kind of thing you mean?
>>>
>>> Thanks
>>> James
>>>
>>> Quoting Philipp Koehn :
>>>
Hi,
I think there was a flaw in some versions of the pseudo code.
The probabilities certainly need to add up to one. There are
two normalizations going on in the algorithm: one on the sentence
level (so the probability of all alignments add up to one) and
one on the word level.
Here the most recent version:
\REQUIRE set of sentence pairs $(\text{\bf e},\text{\bf f})$
\ENSURE translation prob. $t(e|f)$
\STATE initialize $t(e|f)$ uniformly
\WHILE{not converged}
\STATE \COMMENT{initialize}
\STATE count($e|f$) = 0 {\bf for all} $e,f$
\STATE total($f$) = 0 {\bf for all} $f$
\FORALL{sentence pairs ({\bf e},{\bf f})}
\STATE \COMMENT{compute normalization}
\FORALL{words $e$ in {\bf e}}
\STATE s-total($e$) = 0
\FORALL{words $f$ in {\bf f}}
\STATE s-total($e$) += $t(e|f)$
\ENDFOR
\ENDFOR
\STATE \COMMENT{collect counts}
\FORALL{words $e$ in {\bf e}}
\FORALL{words $f$ in {\bf f}}
\STATE count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
\STATE total($f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
\ENDFOR
\ENDFOR
\ENDFOR
\STATE \COMMENT{estimate probabilities}
\FORALL{foreign words $f$}
\FORALL{English words $e$}
\STATE $t(e|f)$ = $\frac{\text{count}(e|f)}{\text{total}(f)}$
\ENDFOR
\ENDFOR
\ENDWHILE
-phi
On Sun, Jul 26, 2009 at 5:24 PM, James Read
wrote:
> Hi,
>
> I have implemented the EM Model 1 algorithm as outlined in Koehn's
> lecture notes. I was surprised to find the raw output of the algorithm
> gives a translation table that given any particular source word the
> sum of the probabilities of each possible target word is far greater
> than 1.
>
> Is this normal?
>
> Thanks
> James
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>>>
>>>
>>>
>>> --
>>> The University of Edinburgh is a charitable body, registered in
>>> Scotland, with registration number SC005336.
>>>
>>>
>>>
>>
>>
>
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] EM Model 1 question
In that case I really don't see how the code is guaranteed to give
results which add up to 1.
Quoting Philipp Koehn :
> Hi,
>
> this is LaTex {algorithmic} code.
>
> count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
>
> means
>
> count(e|f) += t(e|f) / s-total(e)
>
> So, you got that right.
>
> -phi
>
> On Mon, Jul 27, 2009 at 10:18 PM, James Read wrote:
>> Hi,
>>
>> this seems to be pretty much what I implemented. What exactly do you mean by
>> these three lines?:
>>
>> \STATE count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
>> \STATE total($f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
>> \STATE $t(e|f)$ = $\frac{\text{count}(e|f)}{\text{total}(f)}$
>>
>> What do you mean by $\frac? The pseudocode I was using shows these lines as
>> a simple division and this is what my code does. i.e
>>
>> t(e|f) = count(e|f) / total(f)
>>
>> In C code something like:
>>
>> for ( f = 0; f < size_source; f++ )
>> {
>> for ( e = 0; e < size_target; e++ )
>> {
>>t[f][e] = count[f][e] / total[f];
>> }
>> }
>>
>>
>> Is this the kind of thing you mean?
>>
>> Thanks
>> James
>>
>> Quoting Philipp Koehn :
>>
>>> Hi,
>>>
>>> I think there was a flaw in some versions of the pseudo code.
>>> The probabilities certainly need to add up to one. There are
>>> two normalizations going on in the algorithm: one on the sentence
>>> level (so the probability of all alignments add up to one) and
>>> one on the word level.
>>>
>>> Here the most recent version:
>>>
>>> \REQUIRE set of sentence pairs $(\text{\bf e},\text{\bf f})$
>>> \ENSURE translation prob. $t(e|f)$
>>> \STATE initialize $t(e|f)$ uniformly
>>> \WHILE{not converged}
>>> \STATE \COMMENT{initialize}
>>> \STATE count($e|f$) = 0 {\bf for all} $e,f$
>>> \STATE total($f$) = 0 {\bf for all} $f$
>>> \FORALL{sentence pairs ({\bf e},{\bf f})}
>>>\STATE \COMMENT{compute normalization}
>>>\FORALL{words $e$ in {\bf e}}
>>> \STATE s-total($e$) = 0
>>> \FORALL{words $f$ in {\bf f}}
>>>\STATE s-total($e$) += $t(e|f)$
>>> \ENDFOR
>>>\ENDFOR
>>>\STATE \COMMENT{collect counts}
>>>\FORALL{words $e$ in {\bf e}}
>>> \FORALL{words $f$ in {\bf f}}
>>>\STATE count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
>>>\STATE total($f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
>>> \ENDFOR
>>>\ENDFOR
>>> \ENDFOR
>>> \STATE \COMMENT{estimate probabilities}
>>> \FORALL{foreign words $f$}
>>>\FORALL{English words $e$}
>>> \STATE $t(e|f)$ = $\frac{\text{count}(e|f)}{\text{total}(f)}$
>>>\ENDFOR
>>> \ENDFOR
>>> \ENDWHILE
>>>
>>> -phi
>>>
>>>
>>>
>>> On Sun, Jul 26, 2009 at 5:24 PM, James Read wrote:
>>>
Hi,
I have implemented the EM Model 1 algorithm as outlined in Koehn's
lecture notes. I was surprised to find the raw output of the algorithm
gives a translation table that given any particular source word the
sum of the probabilities of each possible target word is far greater
than 1.
Is this normal?
Thanks
James
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>
>>
>>
>> --
>> The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
>>
>>
>>
>
>
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] EM Model 1 question
Hi,
this is LaTex {algorithmic} code.
count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
means
count(e|f) += t(e|f) / s-total(e)
So, you got that right.
-phi
On Mon, Jul 27, 2009 at 10:18 PM, James Read wrote:
> Hi,
>
> this seems to be pretty much what I implemented. What exactly do you mean by
> these three lines?:
>
> \STATE count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
> \STATE total($f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
> \STATE $t(e|f)$ = $\frac{\text{count}(e|f)}{\text{total}(f)}$
>
> What do you mean by $\frac? The pseudocode I was using shows these lines as
> a simple division and this is what my code does. i.e
>
> t(e|f) = count(e|f) / total(f)
>
> In C code something like:
>
> for ( f = 0; f < size_source; f++ )
> {
> for ( e = 0; e < size_target; e++ )
> {
>t[f][e] = count[f][e] / total[f];
> }
> }
>
>
> Is this the kind of thing you mean?
>
> Thanks
> James
>
> Quoting Philipp Koehn :
>
>> Hi,
>>
>> I think there was a flaw in some versions of the pseudo code.
>> The probabilities certainly need to add up to one. There are
>> two normalizations going on in the algorithm: one on the sentence
>> level (so the probability of all alignments add up to one) and
>> one on the word level.
>>
>> Here the most recent version:
>>
>> \REQUIRE set of sentence pairs $(\text{\bf e},\text{\bf f})$
>> \ENSURE translation prob. $t(e|f)$
>> \STATE initialize $t(e|f)$ uniformly
>> \WHILE{not converged}
>> \STATE \COMMENT{initialize}
>> \STATE count($e|f$) = 0 {\bf for all} $e,f$
>> \STATE total($f$) = 0 {\bf for all} $f$
>> \FORALL{sentence pairs ({\bf e},{\bf f})}
>>\STATE \COMMENT{compute normalization}
>>\FORALL{words $e$ in {\bf e}}
>> \STATE s-total($e$) = 0
>> \FORALL{words $f$ in {\bf f}}
>>\STATE s-total($e$) += $t(e|f)$
>> \ENDFOR
>>\ENDFOR
>>\STATE \COMMENT{collect counts}
>>\FORALL{words $e$ in {\bf e}}
>> \FORALL{words $f$ in {\bf f}}
>>\STATE count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
>>\STATE total($f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
>> \ENDFOR
>>\ENDFOR
>> \ENDFOR
>> \STATE \COMMENT{estimate probabilities}
>> \FORALL{foreign words $f$}
>>\FORALL{English words $e$}
>> \STATE $t(e|f)$ = $\frac{\text{count}(e|f)}{\text{total}(f)}$
>>\ENDFOR
>> \ENDFOR
>> \ENDWHILE
>>
>> -phi
>>
>>
>>
>> On Sun, Jul 26, 2009 at 5:24 PM, James Read wrote:
>>
>>> Hi,
>>>
>>> I have implemented the EM Model 1 algorithm as outlined in Koehn's
>>> lecture notes. I was surprised to find the raw output of the algorithm
>>> gives a translation table that given any particular source word the
>>> sum of the probabilities of each possible target word is far greater
>>> than 1.
>>>
>>> Is this normal?
>>>
>>> Thanks
>>> James
>>>
>>> --
>>> The University of Edinburgh is a charitable body, registered in
>>> Scotland, with registration number SC005336.
>>>
>>>
>>> ___
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>
>
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] EM Model 1 question
Hi,
this seems to be pretty much what I implemented. What exactly do you
mean by these three lines?:
\STATE count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
\STATE total($f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
\STATE $t(e|f)$ = $\frac{\text{count}(e|f)}{\text{total}(f)}$
What do you mean by $\frac? The pseudocode I was using shows these
lines as a simple division and this is what my code does. i.e
t(e|f) = count(e|f) / total(f)
In C code something like:
for ( f = 0; f < size_source; f++ )
{
for ( e = 0; e < size_target; e++ )
{
t[f][e] = count[f][e] / total[f];
}
}
Is this the kind of thing you mean?
Thanks
James
Quoting Philipp Koehn :
> Hi,
>
> I think there was a flaw in some versions of the pseudo code.
> The probabilities certainly need to add up to one. There are
> two normalizations going on in the algorithm: one on the sentence
> level (so the probability of all alignments add up to one) and
> one on the word level.
>
> Here the most recent version:
>
> \REQUIRE set of sentence pairs $(\text{\bf e},\text{\bf f})$
> \ENSURE translation prob. $t(e|f)$
> \STATE initialize $t(e|f)$ uniformly
> \WHILE{not converged}
> \STATE \COMMENT{initialize}
> \STATE count($e|f$) = 0 {\bf for all} $e,f$
> \STATE total($f$) = 0 {\bf for all} $f$
> \FORALL{sentence pairs ({\bf e},{\bf f})}
> \STATE \COMMENT{compute normalization}
> \FORALL{words $e$ in {\bf e}}
> \STATE s-total($e$) = 0
> \FORALL{words $f$ in {\bf f}}
> \STATE s-total($e$) += $t(e|f)$
> \ENDFOR
> \ENDFOR
> \STATE \COMMENT{collect counts}
> \FORALL{words $e$ in {\bf e}}
> \FORALL{words $f$ in {\bf f}}
> \STATE count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
> \STATE total($f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
> \ENDFOR
> \ENDFOR
> \ENDFOR
> \STATE \COMMENT{estimate probabilities}
> \FORALL{foreign words $f$}
> \FORALL{English words $e$}
> \STATE $t(e|f)$ = $\frac{\text{count}(e|f)}{\text{total}(f)}$
> \ENDFOR
> \ENDFOR
> \ENDWHILE
>
> -phi
>
>
>
> On Sun, Jul 26, 2009 at 5:24 PM, James Read wrote:
>
>> Hi,
>>
>> I have implemented the EM Model 1 algorithm as outlined in Koehn's
>> lecture notes. I was surprised to find the raw output of the algorithm
>> gives a translation table that given any particular source word the
>> sum of the probabilities of each possible target word is far greater
>> than 1.
>>
>> Is this normal?
>>
>> Thanks
>> James
>>
>> --
>> The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
>>
>>
>> ___
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] EM Model 1 question
Hi,
I think there was a flaw in some versions of the pseudo code.
The probabilities certainly need to add up to one. There are
two normalizations going on in the algorithm: one on the sentence
level (so the probability of all alignments add up to one) and
one on the word level.
Here the most recent version:
\REQUIRE set of sentence pairs $(\text{\bf e},\text{\bf f})$
\ENSURE translation prob. $t(e|f)$
\STATE initialize $t(e|f)$ uniformly
\WHILE{not converged}
\STATE \COMMENT{initialize}
\STATE count($e|f$) = 0 {\bf for all} $e,f$
\STATE total($f$) = 0 {\bf for all} $f$
\FORALL{sentence pairs ({\bf e},{\bf f})}
\STATE \COMMENT{compute normalization}
\FORALL{words $e$ in {\bf e}}
\STATE s-total($e$) = 0
\FORALL{words $f$ in {\bf f}}
\STATE s-total($e$) += $t(e|f)$
\ENDFOR
\ENDFOR
\STATE \COMMENT{collect counts}
\FORALL{words $e$ in {\bf e}}
\FORALL{words $f$ in {\bf f}}
\STATE count($e|f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
\STATE total($f$) += $\frac{t(e|f)}{\text{s-total}(e)}$
\ENDFOR
\ENDFOR
\ENDFOR
\STATE \COMMENT{estimate probabilities}
\FORALL{foreign words $f$}
\FORALL{English words $e$}
\STATE $t(e|f)$ = $\frac{\text{count}(e|f)}{\text{total}(f)}$
\ENDFOR
\ENDFOR
\ENDWHILE
-phi
On Sun, Jul 26, 2009 at 5:24 PM, James Read wrote:
> Hi,
>
> I have implemented the EM Model 1 algorithm as outlined in Koehn's
> lecture notes. I was surprised to find the raw output of the algorithm
> gives a translation table that given any particular source word the
> sum of the probabilities of each possible target word is far greater
> than 1.
>
> Is this normal?
>
> Thanks
> James
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] urgent query..pls help
Hi, does the phrase table file exist? Is it listed in moses.ini? -phi On Mon, Jul 27, 2009 at 6:39 AM, nikita joshi wrote: > can anyone pls help me out. > > > after training the model , in the end i get > (9) create moses.ini > > > but when i test for a sample data, i get the following error > Defined parameters (per moses.ini or switch): > config: moses. ini > ERROR:No phrase translation table (ttable-file) > > pls guide me that why is this error coming..i will be very thankful > to u. > waiting for a positive and early reply from ur side > > with kind regards > nikitaa > > ___ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support > > ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] urgent query..pls help
can anyone pls help me out. after training the model , in the end i get (9) create moses.ini but when i test for a sample data, i get the following error Defined parameters (per moses.ini or switch): config: moses. ini ERROR:No phrase translation table (ttable-file) pls guide me that why is this error coming..i will be very thankful to u. waiting for a positive and early reply from ur side with kind regards nikitaa ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
