[Moses-support] How to print out intermediate confusion networks / lattices?

2017-03-23 Thread Angli Liu 
Hi Moses community,

In decoding, is it possible to have Moses output a confusion network (CN)
or a word lattice (WL), instead of the decoded text for each sentence? I'm
aware that one parameter of the decoder is "-inputtype", so the question is
what parameter of the decoder should be used to determine the output type
(among CN, WL and plain texts)?

Also, where can I exactly find the decoder code (responsible for what the
binary "moses" does) inside https://github.com/moses-smt/mosesdecoder?

Thanks,
Angli
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

[Moses-support] Fwd: Moses-support post from isabella...@gmail.com requires approval

2017-03-23 Thread Hieu Hoang 
I've replicated the problem in the current master, in release-3, and
release-2.1.1. It doesn't appear to affect moses2.

I guess it's a bug in the code that has existed for a while but no-one has
seen it 'cos it's an edge case.

ps. you example model is too big to include in an email. I've put it into
dropbox and sharing the link so others can replicate it too
   https://www.dropbox.com/s/pn5mu04l1za70y1/example_mosesserver.zip?dl=0

* Looking for MT/NLP opportunities *
Hieu Hoang
http://moses-smt.org/


-- Forwarded message --
From: 
Date: 23 March 2017 at 08:39
Subject: Moses-support post from isabella...@gmail.com requires approval
To: moses-support-ow...@mit.edu


As list administrator, your authorization is requested for the
following mailing list posting:

List:Moses-support@mit.edu
From:isabella...@gmail.com
Subject: Dear Friend.
Reason:  Post by non-member to a members-only list

At your convenience, visit:

http://mailman.mit.edu/mailman/admindb/moses-support

to approve or deny the request.


-- Forwarded message --
From: "Mr. Hassan Mohamed Ali" 
To: undisclosed-recipients:;
Cc:
Bcc:
Date: Thu, 23 Mar 2017 09:39:02 +0100
Subject: Dear Friend.
Dear Friend,

I know that this mail will come to you as a surprise since we have not
known or met before now, but please, I would like you to treat it like
blood brother affair and with the urgency and secrecy it requires. I
am Mr. Hassan Mohammed Ali, an Audit staff of (C.B.N) Central Bank of
Nigeria.


An investor (name with-held) died without naming any next of kin to
his fund he deposited in my bank. The amount is $10.5M (Ten Million
Five hundred Thousand Dollars) and banking regulation/legislation in
Nigeria demands that I should notify the fiscal authorities after
three years.

The above set of facts underscores my reason to seek your permission
to have you stand in as the next of kin to the deceased.

This fund will be approved and released in your favour as the next of
kin if only you will adhere to my instruction and cooperate with me in
one accord.

I have all the legal and banking details of the deceased client that
will facilitate our putting you forward as the claimant/beneficiary of
the funds and ultimately transfer of the $10.5M plus interest to any
bank account nominated by you.

I am prepared to compensate you with a 35% share of the total funds
for your efforts. The final details will be given upon receipt an
affirmation of your desire to participate.

Please contact me immediately on (hassan_mohame...@yahoo.com) if you
are interest in my proposal to enable me not to start scouting for
another foreign partner.

Thanks,
Mr. Hassan Mohammed Ali.


-- Forwarded message --
From: moses-support-requ...@mit.edu
To:
Cc:
Bcc:
Date:
Subject: confirm e30b5966b8815fc7e6a47258b4eaa3069178d914
If you reply to this message, keeping the Subject: header intact,
Mailman will discard the held message.  Do this if the message is
spam.  If you reply to this message and include an Approved: header
with the list password in it, the message will be approved for posting
to the list.  The Approved: header can also appear in the first line
of the body of the reply.
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

[Moses-support] Training vs Incremental Training

2017-03-23 Thread Adel Khalifa 
Dear all,

I just need to Know what is the difference between Training & Incremental
Training rather than time.

Regards,
Adel Khalifa .. Software developer
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] memory problems running moses server

2017-03-23 Thread Mathias Müller 
Hi Sarah

What version of the Moses decoder (including the code for Moses server) do
you use exactly? Do you get an error message from the Moses server, i.e.
does it crash? Is the process killed by the OS because of memory problems?

If you can also supply a toy model and ini file, other people can try and
reproduce your issue.

Regards!
Mathias

—

Mathias Müller
AND-2-20
Institute of Computational Linguistics
University of Zurich
Switzerland
+41 44 635 75 81
mathias.muel...@uzh.ch

On Thu, Mar 23, 2017 at 11:17 AM, Sarah Schulz <
sarah.sch...@ims.uni-stuttgart.de> wrote:

> Hi everyone,
>
> It seems like I run into memory problems when using moses server even
> though I have plenty of memory available.
>
> I isolated the problem in a little script (attached). I start the server
> and send strings for translation iteratively. At a certain point it will
> send the request and wait for a reply forever. The number of iterations
> decreases, when I increase the length of the string.
> It seems as if the already sent requests are "piling up" somewhere in
> the server's or client's memory and at a certain point it doesn't have
> the capacity to process the new request.
>
> Has anyone gone through anything similar?
>
> Cheers
> Sarah
> --
> Sarah Schulz
> University of Stuttgart
> Institute for Natural Language Processing (IMS)
> Pfaffenwaldring 5B, 70569 Stuttgart
> Germany
>
> http://www.ims.uni-stuttgart.de/institut/mitarbeiter/schulzsh/
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

[Moses-support] Deadline Extension and Final Call - EAMT 2017 Workshop on Social Media and User Generated Content Machine Translation (Social MT 2017)

2017-03-23 Thread Haithem AFLI 
Dear all, due to several requests, we extend the submission deadline to *April
3, 2017*.

Regards, Workshop Organizers

=
LAST CALL FOR PAPERS
Apologies for multiple postings.

The first Workshop on Social Media and User Generated Content Machine
Translation (Social MT 2017) Co-located with EAMT 2017, Prague, Czech
Republic
For more information please visit:
https://sites.google.com/view/socialmt

CALL FOR PAPERS

With the widespread adoption of social media and online forums, individual
users have been able to actively participate in the generation of online
content in different languages and dialects. As a result, user-generated
content (UGC) has seen an enormous growth in the recent years. The nature
of UGC means that it can be generated at any time and in non-standard
language or formats. Compared to professionally edited text, it is often
more noisy, and likely to take some liberty with commonly established
grammar, punctuation and spelling norms. All this can make it difficult to
translate but UGC can also be incredibly valuable. This workshop will
explore the multifarious aspects of effective MT of data extracted from
social media.
The workshop aims to provide a research platform dedicated to new method
and techniques on translating user-generated content and exploring the use
of such transition on social media analytics. The workshop will solicit
original research contributions related to the theme, which includes (but
is not limited to):

- Models and Tools Development for Social MT
- Machine translation on Microblogs
- Multi-lingual social analytics
- Neural MT for UGC translation
- Multilingual crowdsourcing
- Building resources for UGC translation
- Sentiment translation of UGC
- Analyzing the diffusion of multilingual information
- Using MT for monitoring emergency responses among social crowds
- Multilingual Social-based web platform for disaster management
- Multilingual and language-specific Information Retrieval on Social Web
- Crosslingual document alignment using UGC data
- Named entity transliteration on social media content
- Code-mixed UGC translation
- MT for Big social data analysis

Submissions may include work in progress as well as finished work.
Submissions must have a clear focus on specific issues pertaining to UGC
and its translation. Descriptions of commercial systems are welcome, but
authors should be willing to discuss the details of their work.

IMPORTANT DATES

January 30, 2017: First Call for Workshop Papers
March 8, 2017: Second Call for Workshop Papers
*April 3, 2017*: Workshop Paper Due Date
April 14, 2017: Notification of Acceptance
May 12, 2017: Camera-ready papers due
May 31, 2017: Workshop Date (half-day workshop)

SUBMISSION FORMAT

Submissions must conform to the official style guidelines for EAMT 2017 (
https://ufal.mff.cuni.cz/pbml/instructions-authors).
Overleaf Project to Clone: https://www.overleaf.com/read/jgvrjmrqnwct
MS Word Template: https://ufal.mff.cuni.cz/eamt2017/files/
templates/eamt17.doc

Contributions can be short or long papers. Short paper submission must
describe original and unpublished work without exceeding eight (8) pages plus
any number of pages for references. Characteristics of short papers
include: a small, focused contribution; work in progress; a negative
result; an opinion piece; an interesting application nugget. Long paper
submissions must describe substantial, original, completed and unpublished
work without exceeding twelve (12) pages plus any number of pages for
references.
Reviewing will be double-blind, so the papers should not reveal the
authors’ identity. Accepted papers will be published in the workshop
proceedings.
Double submission policy: Parallel submission to other meetings or
publications is possible but must be immediately notified to the workshop
organizers.
Submission Website: https://easychair.org/conferences/?conf=socialmt2017

Extended versions of the best papers will be published into an upcoming
special issue of “Translating User Generated Content” on Machine
Translation Journal

INVITED SPEAKER:

Houda Bouamor  (Carnegie Mellon University, Qatar)

WORKSHOP ORGANIZERS

General Chair: Andy Way (ADAPT Centre, Dublin City University)

Program Chair :Haithem Afli (ADAPT Centre, Dublin City University)

Program Committee
Loïc Barrault (LIUM, Le Mans University)
Laurent Besacier (LIG, Grenoble University)
Philipp Koehn (University of Edinburgh / Johns Hopkins University)
Abdelkarim Mars (Grenoble University)
Matteo Negri (FBK)
Houda Bouamor (CMU Qatar)
Yvette Graham (ADAPT Centre, Dublin City University)
Dimitar Shterionov (KantanMT)
Marco Turchi (FBK)
Antonio Toral (University of Groningen)
Lucia Specia (University of Sheffield)
Kashif Shah (eBay)
Rejwanul Haque (Lingo24)
Barry Haddow (University of Edinburgh)
Jinhua Du (ADAPT Centre, Dublin City University)
Daniel Stein (eBay)
Mohammed Hasanuzzaman (ADAPT Centre, Dublin City University)

[Moses-support] memory problems running moses server

2017-03-23 Thread Sarah Schulz 
Hi everyone,

It seems like I run into memory problems when using moses server even
though I have plenty of memory available.

I isolated the problem in a little script (attached). I start the server
and send strings for translation iteratively. At a certain point it will
send the request and wait for a reply forever. The number of iterations
decreases, when I increase the length of the string.
It seems as if the already sent requests are "piling up" somewhere in
the server's or client's memory and at a certain point it doesn't have
the capacity to process the new request.

Has anyone gone through anything similar?

Cheers
Sarah
-- 
Sarah Schulz
University of Stuttgart
Institute for Natural Language Processing (IMS)
Pfaffenwaldring 5B, 70569 Stuttgart
Germany

http://www.ims.uni-stuttgart.de/institut/mitarbeiter/schulzsh/
import random
import xmlrpclib as x
import time
import sys
import subprocess

#initialize random port and start moses server as subprocess
port_uni= str(random.randint(4, 5))
p1 = subprocess.Popen("/mount/arbeitsdaten13/users/schulzsh/tools/Moses/mosesdecoder/bin/mosesserver -f ../static/Moses/decoder_files/SMT_Werther_uni.ini --server-port "+ port_uni,stderr=subprocess.PIPE,bufsize=8000,shell=True,env={"LD_LIBRARY_PATH":"/mount/projekte/sfb-732/inf/users/sarah/tools/xmlrpc-c/lib"})

#wait for it to start sucessfully
elapsed_time=0
start_time = time.time()
   
err=''
out=''
for line in iter(p1.stderr.readline, ''):

if "Listening on port" in line or elapsed_time > 150:
print "loaded uni1"
break
   
elapsed_time = time.time() - start_time

if elapsed_time >= 150:
print "SMT Unigram Server Mode doesn't start."
sys.exit()

#connect to ServerProxy
s_unigram = x.ServerProxy("http://localhost:"+port_uni+"/RPC2",allow_none=True,verbose=True)


#send a message to moses server for translation
#note: when message length increases, the bug appears after fewer iterations
for i in range(1000):

trans= s_unigram.translate({"text":"# I c h j k k k # k k k k  k k k k k k k #","align":"true"})
print trans
print i
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support