Hello!
It doesn't work for me:
><px3sp><nom>+да<cnjcoo>$ ^бит/бит<mod_ass>$
_
^ул/бул<v><tv><imp><p2><sg>$^,/,<cm>$ ^театраль/театраль<adj>$ ^жест/*жест$
^ясап/яса<v><tv><gna_perf>$
^-/-<guio>$ ^Синнән/Син<prn><pers><p2><sg><abl>$
^сорап/сора<v><tv><prc_perf>$ ^торырмын/тор<vaux><fut><p1><sg>$
^Барлык/Барлык<det><qnt>$ ^иптәшләрдән/иптәш<n><pl><abl>$
^кул/кул<n><sg><nom>+и<cop><aor><p3><sg>$ ^куйды/куй<v><tv><ifi><p3><sg>$
^рам да/рам<n><sg><nom>+да<cnjcoo>$ ^тикшерү/тикшерү<n><sg><attr>$
^органнарына/орган<n><pl><px3sp><dat>$
_
_
_
Problems are where we see _ symbol. In the end 3 new lines. And almost each
line loses last character or even words (it should be "рам да тикшерү
органнарына тапшыра").
By the way, rules:
tr '\n' '\0' |
apertium-destxt -n |
lt-proc -z -w 'apertium-tat/tat.automorf.bin' |
cg-proc -z 'apertium-tat/tat.rlx.bin' |
cg-proc -z -w -1 'apertium-tat/dev/mansur.bin' |
tr '\0' '\n' |
apertium-retxt |
Replacing these 'tr' commands with previous recommendations from Fran gives
correct output.
Am Di., 6. Nov. 2018 um 22:45 Uhr schrieb Francis Tyers <fty...@prompsit.com
>:
> El 2018-11-06 20:36, Kevin Brubeck Unhammer escribió:
> > Francis Tyers <fty...@prompsit.com> čálii:
> >
> >> Yes it does. It will put a sentence boundary after every word, meaning
> >> that you won't get reliable tagger output. Apertium as far as I know
> >> has no way to treat sentences as a sequence of lines. This is because
> >> of how the format handling works.
> >>
> >> I think it would really be an excellent feature though. Perhaps a
> >> GitHub issue? I do however think it would involve messing with quite a
> >> bit of the pipeline.
> >
> > However, we *should* treat NUL as hard separators – if we don't,
> > apertium-apy (and thus www.apertium.org) will risk sending output meant
> > for person1 to person2. (I have an inkling there might still be bugs in
> > apertium-transfer related to this.)
> >
> > Anyway, if we at least handle NUL's correctly in lt-proc and cg-proc,
> > you could turn linebreak's into NUL's (first deleting any existing
> > NUL's
> > in the corpus) and tag with the -z option to lt-/cg-proc:
> >
> > cat corpus.txt \
> > | tr -d '\0' \
> > | tr '\n' '\0' \
> > | apertium-deshtml -n \
> > | lt-proc -z -w 'apertium-tat/tat.automorf.bin' \
> > | cg-proc -z 'apertium-tat/tat.rlx.bin' \
> > | cg-proc -z -w -1 'apertium-tat/dev/mansur.bin' \
> > | tr '\0' '\n' \
> > | apertium-rehtml-noent
> >
> > … finally turning NUL's back into newlines.
> >
> > With apertium-nob, this doesn't seem to run slower than without -z, and
> > doesn't merge lines in my test corpus.
> >
>
> Ooh, this is great, we should probably put this on the wiki!
>
> F.
>
>
> _______________________________________________
> Apertium-stuff mailing list
> Apertium-stuff@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/apertium-stuff
>
_______________________________________________
Apertium-stuff mailing list
Apertium-stuff@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/apertium-stuff
