I'll update that- the inputtype should be "2" for lattices...
Chris
On Wed, Feb 27, 2008 at 4:39 AM, ThuyLinh Nguyen <[EMAIL PROTECTED]> wrote:
>
>
> Hi Chris,
> Thanks for clarification, so the lattice format is different with confusion
> network format
> but in moses binary, there are only two options for inputtype: -inputtype:
> text (0) or confusion network (1)
>
> It does n't recognize the lattice format input.
> This is an example of lattice translation error:
>
> echo "((('A',1.0,1),),(('B',1.0,1),),)" | moses -f moses.ini -inputtype 1
> Defined parameters (per moses.ini or switch):
> config: moses.ini
> distortion-limit: 6
> input-factors: 0
> inputtype: 1
> lmodel-file: 0 0 3 /SMT/Workplace/Linh/IWSLT_0802/train.en.srilm
> mapping: 0 T 0
> ttable-file: 0 0 5
> /SMT/Workplace/Linh/IWSLT_0802/gold_morph_space/model/phrase-table.bin
> ttable-limit: 20 0
> weight-d: 0.6
> weight-l: 0.5000
> weight-t: 0.2 0.2 0.2 0.2 0.2
> weight-w: -1
> Loading lexical distortion models...
> have 0 models
> Start loading LanguageModel /SMT/Workplace/Linh/IWSLT_0802/train.en.srilm :
> [0.000] seconds
> Finished loading LanguageModels : [0.000] seconds
> Start loading PhraseTable
> /SMT/Workplace/Linh/IWSLT_0802/gold_morph_space/model/phrase-table.bin :
> [0.000] seconds
> using binary phrase tables for idx 0
> reading bin ttable
> size of OFF_T 8
> binary phrasefile loaded, default OFF_T: -1
> Finished loading phrase tables : [0.000] seconds
> IO from STDOUT/STDIN
> Created input-output object : [0.000] seconds
> read confusion net with format 0
> End. : [0.000] seconds
> confusion net statistics:
> created: 1
> destroyed: 1
> succ. read: 0
> columns: 0
> words: 0
> avg. word/column: nan
> avg. cols/sent: nan
>
>
> Let me know if I made mistake somewhere.
>
> Thanks
> Linh
>
>
>
>
>
> Chris Dyer wrote:
>
> I am still confused about the lattice format,
> In your examples:
>
> 1 ((('A',1.0,1),),(('B',1.0,1),),)
> 2 ((('A',1.0,1),('Z',1.0,2),),(('B',1.0,1),),(('C',1.0,1),),)
>
> Can I interpret it as:
> from node 0 to node 1 there are 2 lattices: (('A',1.0,1),) and
> (('B',1.0,1),)
>
> Each entire lattice is encoded on a single line. In line 1, there are
> two arcs from node 0 to node 1, 'A' and 'B'. The 1.0 is the cost of
> the arc and the "1" is the length of the arc (measured in nodes). In
> line two, node 0 has two arcs, arc 'A' that goes to node 1 and arc 'Z'
> that goes to node 2. Node 1 has a single arc, 'B', that goes to node
> 2. Node 2 has a single arc 'C' that goes to 3.
>
>
>
>
> And also what are the meaning of number 1.0 and 1, 2 there? where can I put
> the lattice probabilities?
> Is it possible to add an empty lattice (so that the decoder skip a word)?
>
> Currently, moses only lets you specify a single cost for an arc, and
> it is actually treated as a probability (the decoder sees it as
> -log(p) -- you can change this in WordLattice.cpp if you want to deal
> with more conventional costs, but the rest of the inputs to the
> decoder are given as probabilities so I wanted to be consistent). If
> you want a null transition, set the arc label to '*eps*' and the
> decoder will treat this as a null.
>
> --Chris
>
>
>
>
> Linh
>
>
>
>
> Chris Dyer wrote:
>
> >Also, if you are using general lattices (as opposed to regular
> >confusion networks) as input, you should update to the latest version
> >of the decoder from Subversion, since I checked in a fairly crucial
> >bug fix yesterday.
> >
> >Chris
> >
> >On Wed, Feb 20, 2008 at 4:37 PM, Chris Dyer <[EMAIL PROTECTED]> wrote:
> >
> >
> >>The lattice format isn't documented yet on the webpage, but you can
> >> see some examples of it in the lattice-distortion test directory Hieu
> >> mentions. It should be fairly straightforward to decipher. Since
> >> this format encodes a single lattice/CN per line of text, it can be
> >> used easily with MER training.
> >>
> >> Chris
> >>
> >>
> >>
> >> On Wed, Feb 20, 2008 at 4:30 PM, Hieu Hoang <[EMAIL PROTECTED]>
> wrote:
> >> > chao anh/chi linh
> >> >
> >> > i'm not sure if anyone answered your question and i'm prob not the
> best
> >> > person to answer question on lattice/confusion net input. to my
> knowledge,
> >> > mert should run fine with these input types.
> >> >
> >> > perhaps you can find an example of the lattice input format from the
> >> > regression test :
> >> >
> >> >
> http://mosesdecoder.svn.sourceforge.net/viewvc/mosesdecoder/trunk/regression-testing/tests/
> >> >
> >> >
> >> >
> >> > ThuyLinh Nguyen <[EMAIL PROTECTED]> wrote:
> >> >
> >> >
> >> > -------- Original Message --------
> >> > Subject: Run mert-moses.pl with confusion network
> >> > Date: Sat, 16 Feb 2008 21:33:44 -0500
> >> > From: ThuyLinh Nguyen
> >> > To: moses-support@mit.edu
> >> >
> >> >
> >> >
> >> > Hello,
> >> > I want to run mer for a development set which is the output of other
> >> > translation job.
> >> > therefore the development input is a set of lattices. Are there anyway
> >> > to run MER with lattice input and if so how can i represent the
> lattice
> >> > of multiple sentences?
> >> > Thank you
> >> > Linh
> >> >
> >> >
> >> > _______________________________________________
> >> > Moses-support mailing list
> >> > Moses-support@mit.edu
> >> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >> >
> >> >
> >> >
> >> >
> >> > Hieu Hoang
> >> > http//www.hoang.co.uk/hieu
> >> >
> >> > ________________________________
> >> >
> >> > Sent from Yahoo! - a smarter inbox.
> >> > _______________________________________________
> >> > Moses-support mailing list
> >> > Moses-support@mit.edu
> >> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >> >
> >> >
> >>
> >>
> >>
> >
> >
> >
>
>
>
>
>
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
