Hi Tomasz
The moseserver is just the decoder, so it doesn't do any of the pre- and
post-processing steps that you also need. In particular it does not do
tokenisation. You need to send it tokenised text, and then de-tokenise
the output,
cheers - Barry
On 12/11/15 13:40, Tomasz Gawryl wrote:
> Hi Ulrich,
>
> I have a question about Moses server too. I'm testing it as a wrapper for
> Across server to check pre-translation possibilities. It generally works but
> there is one problem. Input segments are translated without tokenization, so
> every word close to special character (for example `this is small house.`)
> remains untranslated ('to jest mały house.'). I was searching list archive
> and I found similar question here:
> http://comments.gmane.org/gmane.comp.nlp.moses.user/14020 but for me it's
> not yet answered. I would appreciate any information on this subject.
>
> Best regards,
> Tomek
>
> -----Original Message-----
> From: moses-support-boun...@mit.edu [mailto:moses-support-boun...@mit.edu]
> On Behalf Of moses-support-requ...@mit.edu
> Sent: Wednesday, November 11, 2015 10:31 AM
> To: moses-support@mit.edu
> Subject: Moses-support Digest, Vol 109, Issue 19
>
> Send Moses-support mailing list submissions to
> moses-support@mit.edu
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mailman.mit.edu/mailman/listinfo/moses-support
> or, via email, send a message with subject or body 'help' to
> moses-support-requ...@mit.edu
>
> You can reach the person managing the list at
> moses-support-ow...@mit.edu
>
> When replying, please edit your Subject line so it is more specific than
> "Re: Contents of Moses-support digest..."
>
>
> Today's Topics:
>
> 1. Re: use placeholder with mosesserver (Ulrich Germann)
> 2. Re: use placeholder with mosesserver (Evgeny Matusov)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 11 Nov 2015 01:58:40 +0000
> From: Ulrich Germann <ulrich.germ...@gmail.com>
> Subject: Re: [Moses-support] use placeholder with mosesserver
> To: Evgeny Matusov <ematu...@apptek.com>
> Cc: "moses-support@mit.edu" <moses-support@mit.edu>
> Message-ID:
> <cahqsruojzbwqie_+1-si7i7e1cmyt3tq4gcfl-2ge2bggd6...@mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi all,
>
> I've just pushed what I believe might address a few of the issues in this
> thread:
>
> - the more fine-grained configuration options for request handling and
> queuing, server timeouts etc. (added in August due to threading issue) have
> been transferred to the main moses executable.
>
> - the server now pays attention to the xml-input option specified via json;
> the range of accepted values is the same as when specified on the command
> line. I have not written the xml-input handling and do not actively use it,
> so it may or may not work. I don't think there are any regression tests that
> test this right now. Reports from the trenches are welcome.
>
> - mosesserver.cpp is deprecated. It is now merely a shell around the regular
> moses call with --server. I did not remove it from the code base entirely,
> as I assume that there's a plethora of setups out there that rely on the
> existence of mosesserver. What the wrapper does is add --server to the
> options and then call run regular moses.
>
> - anyone adding stuff to mosesserver.cpp from now on owes me a lifetime
> supply of the finest Laphroaig. Just send me a quarter cask every year for
> Burns Nicht for the rest of my life if you do. If I haven't pushed anything
> for two years, you may assume I'm dead.
>
>
> - Uli
>
> On Tue, Nov 10, 2015 at 2:58 PM, Ulrich Germann <ulrich.germ...@gmail.com>
> wrote:
>
>> Hi all,
>>
>> mosesserver is deprecated and should not be used any more. I'll
>> transfer the threading-related changes to the server implementation in
>> the regular moses executable and let you know once I'm done so that
>> other things (like
>> passthrough) can be added. By the looks of it, the changes are fairly
>> straightforward, so it shouldn't take long. However, I can't guarantee
>> that the new server will do everything the old server did, (or do it
>> the same way).
>>
>> It would be fantastic if a few people could design and contribute test
>> cases so that we can do some regression testing for the server.
>> Ideally a test case should provide:
>>
>> - tiny models to work with (or we may be able to recycle some that
>> already
>> exist)
>> - sample input (json)
>> - expected output (json)
>>
>> Cheers - Uli
>>
>> On Tue, Nov 10, 2015 at 11:37 AM, Evgeny Matusov <ematu...@apptek.com>
>> wrote:
>>
>>> Hi,
>>>
>>> can any of the more active recent developers advise what is the
>>> latest stable mosesserver implementation?
>>>
>>> It seems to be the one in moses/server, but the one in in
>>> contrib/server/mosesserver.cpp has been updated in August of this
>>> year with an important fix related to multiple threads:
>>>
>>>
>>> https://github.com/moses-smt/mosesdecoder/commit/3c682fa8b05af6bff1a0
>>> 9f420141795875cf9685
>>> https://www.mail-archive.com/moses-support%40mit.edu/msg12875.html
>>>
>>> As Gregor mentioned, we would like to share our fix so that
>>> Mosesserver correctly supports placeholders. I want to make sure that
>>> this is a fix for something that many people use without problems.
>>>
>>> Thanks,
>>> Evgeny.
>>>
>>>
>>> ________________________________________
>>> From: moses-support-boun...@mit.edu <moses-support-boun...@mit.edu>
>>> on behalf of moses-support-requ...@mit.edu
>>> <moses-support-requ...@mit.edu>
>>> Sent: Monday, November 9, 2015 6:02 PM
>>> To: moses-support@mit.edu
>>> Subject: Moses-support Digest, Vol 109, Issue 16
>>>
>>> Send Moses-support mailing list submissions to
>>> moses-support@mit.edu
>>>
>>> To subscribe or unsubscribe via the World Wide Web, visit
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>> or, via email, send a message with subject or body 'help' to
>>> moses-support-requ...@mit.edu
>>>
>>> You can reach the person managing the list at
>>> moses-support-ow...@mit.edu
>>>
>>> When replying, please edit your Subject line so it is more specific
>>> than "Re: Contents of Moses-support digest..."
>>>
>>>
>>> Today's Topics:
>>>
>>> 1. Question about output alignment info (Marta Ruiz)
>>> 2. Re: use placeholder with mosesserver (Leusch, Gregor)
>>>
>>>
>>> ---------------------------------------------------------------------
>>> -
>>>
>>> Message: 1
>>> Date: Mon, 9 Nov 2015 16:10:26 +0100
>>> From: Marta Ruiz <martaruizcostaju...@gmail.com>
>>> Subject: [Moses-support] Question about output alignment info
>>> To: moses-support@mit.edu
>>> Message-ID:
>>> <CABEBqHLL=
>>> vrajg5btd3amfvmhpcynda3nj-0ynyqwf3xuzd...@mail.gmail.com>
>>> Content-Type: text/plain; charset="utf-8"
>>>
>>> Hi all,
>>>
>>> When I use the option "-alignment-output-file [file]", I get just a
>>> few alignments. Most sentences are in blank, except some that have
>>> one alignment...
>>>
>>> best,
>>> Marta
>>>
>>>
>>>
>>>
>>> --
>>> Marta Ruiz Costa-juss?
>>> martaruizcostaju...@gmail.com
>>> http://www.costa-jussa.com
>>> -------------- next part -------------- An HTML attachment was
>>> scrubbed...
>>> URL:
>>> http://mailman.mit.edu/mailman/private/moses-support/attachments/2015
>>> 1109/4da5ec6d/attachment-0001.html
>>>
>>> ------------------------------
>>>
>>> Message: 2
>>> Date: Mon, 9 Nov 2015 15:37:32 +0000
>>> From: "Leusch, Gregor" <gleu...@ebay.com>
>>> Subject: Re: [Moses-support] use placeholder with mosesserver
>>> To: Vito Mandorino <vito.mandor...@linguacustodia.com>, moses-support
>>> <moses-support@mit.edu>
>>> Message-ID: <d2667743.9f24%gleu...@ebay.com>
>>> Content-Type: text/plain; charset="utf-8"
>>>
>>> Hi,
>>>
>>> we saw the same issue a while ago in an older version of Moses.
>>> Mosesserver and moses use different routines to parse the input
>>> string; in particular the code in mosesserver did not parse
>>> placeholders input correctly. It seems to me that this is fixed in
>>> the most recent version of mosesserver (though I have not tested
>>> this; I just looked at the code); in addition, our team is currently
>>> discussing whether it makes sense to make available our patches to
>>> the mosesserver code either on the version we are using, or on a more
> recent version, available end of this week.
>>> Best,
>>>
>>> Gregor
>>>
>>>
>>>
>>>
>>> From:
>>> <moses-support-boun...@mit.edu<mailto:moses-support-boun...@mit.edu>>
>>> on behalf of Vito Mandorino <vito.mandor...@linguacustodia.com<mailto:
>>> vito.mandor...@linguacustodia.com>>
>>> Date: Friday 6 November 2015 16:22
>>> To: moses-support
>>> <moses-support@mit.edu<mailto:moses-support@mit.edu>>
>>> Subject: [Moses-support] use placeholder with mosesserver
>>>
>>> Dear all,
>>>
>>> I have been unsuccessful so far in using the placeholder approach
>>> with mosesserver. The translated segments contain the placeholder
>>> token @num@ instead of numbers.
>>> Do you know how to get the numbers in the output?
>>>
>>> Many thanks,
>>>
>>> Vito Mandorino
>>>
>>>
>>> --
>>> M. Vito MANDORINO -- Chief Scientist
>>>
>>>
>>> [Description?: Description?: lingua_custodia_final full logo]
>>>
>>> The Translation Trustee
>>>
>>> 1, Place Charles de Gaulle, 78180 Montigny-le-Bretonneux
>>>
>>> Tel : +33 1 30 44 04 23 Mobile : +33 6 84 65 68 89
>>>
>>> Email : vito.mandor...@linguacustodia.com<mailto:
>>> massinissa.ah...@linguacustodia.com>
>>>
>>> Website : www.linguacustodia.com<http://www.linguacustodia.com/> -
>>> www.thetranslationtrustee.com <http://www.thetranslationtrustee.com/>
>>>
>>>
>>> -------------- next part -------------- An HTML attachment was
>>> scrubbed...
>>> URL:
>>> http://mailman.mit.edu/mailman/private/moses-support/attachments/2015
>>> 1109/c046c74c/attachment-0001.html
>>> -------------- next part -------------- A non-text attachment was
>>> scrubbed...
>>> Name: image001.jpg
>>> Type: image/jpeg
>>> Size: 4421 bytes
>>> Desc: image001.jpg
>>> Url :
>>> http://mailman.mit.edu/mailman/private/moses-support/attachments/2015
>>> 1109/c046c74c/attachment-0001.jpg
>>>
>>> ------------------------------
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>> End of Moses-support Digest, Vol 109, Issue 16
>>> **********************************************
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>> --
>> Ulrich Germann
>> Senior Researcher
>> School of Informatics
>> University of Edinburgh
>>
>
>
> --
> Ulrich Germann
> Senior Researcher
> School of Informatics
> University of Edinburgh
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mailman.mit.edu/mailman/private/moses-support/attachments/20151110/67
> 5c857b/attachment-0001.html
>
> ------------------------------
>
> Message: 2
> Date: Wed, 11 Nov 2015 09:30:21 +0000
> From: Evgeny Matusov <ematu...@apptek.com>
> Subject: Re: [Moses-support] use placeholder with mosesserver
> To: "ugerm...@inf.ed.ac.uk" <ugerm...@inf.ed.ac.uk>
> Cc: "moses-support@mit.edu" <moses-support@mit.edu>
> Message-ID:
>
> <bn3pr08mb19850f1df1b9303939007bc7ac...@bn3pr08mb1985.namprd08.prod.outlook.
> com>
>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi Uli,
>
>
> thanks a lot! We will try to add some test cases for Mosesserver, including
> XML input with/without placeholders.
>
>
> Best,
>
> Evgeny.
>
>
> ________________________________
> From: Ulrich Germann <ulrich.germ...@gmail.com>
> Sent: Wednesday, November 11, 2015 2:58 AM
> To: Evgeny Matusov
> Cc: moses-support@mit.edu
> Subject: Re: [Moses-support] use placeholder with mosesserver
>
> Hi all,
>
> I've just pushed what I believe might address a few of the issues in this
> thread:
>
> - the more fine-grained configuration options for request handling and
> queuing, server timeouts etc. (added in August due to threading issue) have
> been transferred to the main moses executable.
>
> - the server now pays attention to the xml-input option specified via json;
> the range of accepted values is the same as when specified on the command
> line. I have not written the xml-input handling and do not actively use it,
> so it may or may not work. I don't think there are any regression tests that
> test this right now. Reports from the trenches are welcome.
>
> - mosesserver.cpp is deprecated. It is now merely a shell around the regular
> moses call with --server. I did not remove it from the code base entirely,
> as I assume that there's a plethora of setups out there that rely on the
> existence of mosesserver. What the wrapper does is add --server to the
> options and then call run regular moses.
>
> - anyone adding stuff to mosesserver.cpp from now on owes me a lifetime
> supply of the finest Laphroaig. Just send me a quarter cask every year for
> Burns Nicht for the rest of my life if you do. If I haven't pushed anything
> for two years, you may assume I'm dead.
>
>
> - Uli
>
> On Tue, Nov 10, 2015 at 2:58 PM, Ulrich Germann
> <ulrich.germ...@gmail.com<mailto:ulrich.germ...@gmail.com>> wrote:
> Hi all,
>
> mosesserver is deprecated and should not be used any more. I'll transfer the
> threading-related changes to the server implementation in the regular moses
> executable and let you know once I'm done so that other things (like
> passthrough) can be added. By the looks of it, the changes are fairly
> straightforward, so it shouldn't take long. However, I can't guarantee that
> the new server will do everything the old server did, (or do it the same
> way).
>
> It would be fantastic if a few people could design and contribute test cases
> so that we can do some regression testing for the server. Ideally a test
> case should provide:
>
> - tiny models to work with (or we may be able to recycle some that already
> exist)
> - sample input (json)
> - expected output (json)
>
> Cheers - Uli
>
> On Tue, Nov 10, 2015 at 11:37 AM, Evgeny Matusov
> <ematu...@apptek.com<mailto:ematu...@apptek.com>> wrote:
>
> Hi,
>
> can any of the more active recent developers advise what is the latest
> stable mosesserver implementation?
>
> It seems to be the one in moses/server, but the one in in
> contrib/server/mosesserver.cpp has been updated in August of this year with
> an important fix related to multiple threads:
>
> https://github.com/moses-smt/mosesdecoder/commit/3c682fa8b05af6bff1a09f42014
> 1795875cf9685
> https://www.mail-archive.com/moses-support%40mit.edu/msg12875.html
>
> As Gregor mentioned, we would like to share our fix so that Mosesserver
> correctly supports placeholders. I want to make sure that this is a fix for
> something that many people use without problems.
>
> Thanks,
> Evgeny.
>
>
> ________________________________________
> From: moses-support-boun...@mit.edu<mailto:moses-support-boun...@mit.edu>
> <moses-support-boun...@mit.edu<mailto:moses-support-boun...@mit.edu>> on
> behalf of
> moses-support-requ...@mit.edu<mailto:moses-support-requ...@mit.edu>
> <moses-support-requ...@mit.edu<mailto:moses-support-requ...@mit.edu>>
> Sent: Monday, November 9, 2015 6:02 PM
> To: moses-support@mit.edu<mailto:moses-support@mit.edu>
> Subject: Moses-support Digest, Vol 109, Issue 16
>
> Send Moses-support mailing list submissions to
> moses-support@mit.edu<mailto:moses-support@mit.edu>
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mailman.mit.edu/mailman/listinfo/moses-support
> or, via email, send a message with subject or body 'help' to
> moses-support-requ...@mit.edu<mailto:moses-support-requ...@mit.edu>
>
> You can reach the person managing the list at
> moses-support-ow...@mit.edu<mailto:moses-support-ow...@mit.edu>
>
> When replying, please edit your Subject line so it is more specific than
> "Re: Contents of Moses-support digest..."
>
>
> Today's Topics:
>
> 1. Question about output alignment info (Marta Ruiz)
> 2. Re: use placeholder with mosesserver (Leusch, Gregor)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 9 Nov 2015 16:10:26 +0100
> From: Marta Ruiz
> <martaruizcostaju...@gmail.com<mailto:martaruizcostaju...@gmail.com>>
> Subject: [Moses-support] Question about output alignment info
> To: moses-support@mit.edu<mailto:moses-support@mit.edu>
> Message-ID:
>
> <CABEBqHLL=vrajg5btd3amfvmhpcynda3nj-0ynyqwf3xuzd...@mail.gmail.com<mailto:V
> rajg5btd3amfvmhpcynda3nj-0ynyqwf3xuzd...@mail.gmail.com>>
> Content-Type: text/plain; charset="utf-8"
>
> Hi all,
>
> When I use the option "-alignment-output-file [file]", I get just a few
> alignments. Most sentences are in blank, except some that have one
> alignment...
>
> best,
> Marta
>
>
>
>
> --
> Marta Ruiz Costa-juss?
> martaruizcostaju...@gmail.com<mailto:martaruizcostaju...@gmail.com>
> http://www.costa-jussa.com
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mailman.mit.edu/mailman/private/moses-support/attachments/20151109/4d
> a5ec6d/attachment-0001.html
>
> ------------------------------
>
> Message: 2
> Date: Mon, 9 Nov 2015 15:37:32 +0000
> From: "Leusch, Gregor" <gleu...@ebay.com<mailto:gleu...@ebay.com>>
> Subject: Re: [Moses-support] use placeholder with mosesserver
> To: Vito Mandorino
> <vito.mandor...@linguacustodia.com<mailto:vito.mandor...@linguacustodia.com>
>> , moses-support
> <moses-support@mit.edu<mailto:moses-support@mit.edu>>
> Message-ID:
> <d2667743.9f24%gleu...@ebay.com<mailto:d2667743.9f24%25gleu...@ebay.com>>
> Content-Type: text/plain; charset="utf-8"
>
> Hi,
>
> we saw the same issue a while ago in an older version of Moses. Mosesserver
> and moses use different routines to parse the input string; in particular
> the code in mosesserver did not parse placeholders input correctly. It seems
> to me that this is fixed in the most recent version of mosesserver (though I
> have not tested this; I just looked at the code); in addition, our team is
> currently discussing whether it makes sense to make available our patches to
> the mosesserver code either on the version we are using, or on a more recent
> version, available end of this week.
>
> Best,
>
> Gregor
>
>
>
>
> From:
> <moses-support-boun...@mit.edu<mailto:moses-support-boun...@mit.edu><mailto:
> moses-support-boun...@mit.edu<mailto:moses-support-boun...@mit.edu>>> on
> behalf of Vito Mandorino
> <vito.mandor...@linguacustodia.com<mailto:vito.mandor...@linguacustodia.com>
> <mailto:vito.mandor...@linguacustodia.com<mailto:vito.mandorino@linguacustod
> ia.com>>>
> Date: Friday 6 November 2015 16:22
> To: moses-support
> <moses-support@mit.edu<mailto:moses-support@mit.edu><mailto:moses-support@mi
> t.edu<mailto:moses-support@mit.edu>>>
> Subject: [Moses-support] use placeholder with mosesserver
>
> Dear all,
>
> I have been unsuccessful so far in using the placeholder approach with
> mosesserver. The translated segments contain the placeholder token @num@
> instead of numbers.
> Do you know how to get the numbers in the output?
>
> Many thanks,
>
> Vito Mandorino
>
>
> --
> M. Vito MANDORINO -- Chief Scientist
>
>
> [Description?: Description?: lingua_custodia_final full logo]
>
> The Translation Trustee
>
> 1, Place Charles de Gaulle, 78180 Montigny-le-Bretonneux
>
> Tel : +33 1 30 44 04 23<tel:%2B33%201%2030%2044%2004%2023> Mobile : +33 6
> 84 65 68 89<tel:%2B33%206%2084%2065%2068%2089>
>
> Email :
> vito.mandor...@linguacustodia.com<mailto:vito.mandor...@linguacustodia.com><
> mailto:massinissa.ah...@linguacustodia.com<mailto:massinissa.ahmim@linguacus
> todia.com>>
>
> Website :
> www.linguacustodia.com<http://www.linguacustodia.com><http://www.linguacusto
> dia.com/> -
> www.thetranslationtrustee.com<http://www.thetranslationtrustee.com>
> <http://www.thetranslationtrustee.com/>
>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mailman.mit.edu/mailman/private/moses-support/attachments/20151109/c0
> 46c74c/attachment-0001.html
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: image001.jpg
> Type: image/jpeg
> Size: 4421 bytes
> Desc: image001.jpg
> Url :
> http://mailman.mit.edu/mailman/private/moses-support/attachments/20151109/c0
> 46c74c/attachment-0001.jpg
>
> ------------------------------
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu<mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
> End of Moses-support Digest, Vol 109, Issue 16
> **********************************************
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu<mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
> --
> Ulrich Germann
> Senior Researcher
> School of Informatics
> University of Edinburgh
>
>
>
> --
> Ulrich Germann
> Senior Researcher
> School of Informatics
> University of Edinburgh
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mailman.mit.edu/mailman/private/moses-support/attachments/20151111/7b
> 5649fc/attachment.html
>
> ------------------------------
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
> End of Moses-support Digest, Vol 109, Issue 19
> **********************************************
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
