[Moses-support] How to print out intermediate confusion networks / lattices?
Hi Moses community, In decoding, is it possible to have Moses output a confusion network (CN) or a word lattice (WL), instead of the decoded text for each sentence? I'm aware that one parameter of the decoder is "-inputtype", so the question is what parameter of the decoder should be used to determine the output type (among CN, WL and plain texts)? Also, where can I exactly find the decoder code (responsible for what the binary "moses" does) inside https://github.com/moses-smt/mosesdecoder? Thanks, Angli ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] Fwd: Moses-support post from isabella...@gmail.com requires approval
I've replicated the problem in the current master, in release-3, and release-2.1.1. It doesn't appear to affect moses2. I guess it's a bug in the code that has existed for a while but no-one has seen it 'cos it's an edge case. ps. you example model is too big to include in an email. I've put it into dropbox and sharing the link so others can replicate it too https://www.dropbox.com/s/pn5mu04l1za70y1/example_mosesserver.zip?dl=0 * Looking for MT/NLP opportunities * Hieu Hoang http://moses-smt.org/ -- Forwarded message -- From:Date: 23 March 2017 at 08:39 Subject: Moses-support post from isabella...@gmail.com requires approval To: moses-support-ow...@mit.edu As list administrator, your authorization is requested for the following mailing list posting: List:Moses-support@mit.edu From:isabella...@gmail.com Subject: Dear Friend. Reason: Post by non-member to a members-only list At your convenience, visit: http://mailman.mit.edu/mailman/admindb/moses-support to approve or deny the request. -- Forwarded message -- From: "Mr. Hassan Mohamed Ali" To: undisclosed-recipients:; Cc: Bcc: Date: Thu, 23 Mar 2017 09:39:02 +0100 Subject: Dear Friend. Dear Friend, I know that this mail will come to you as a surprise since we have not known or met before now, but please, I would like you to treat it like blood brother affair and with the urgency and secrecy it requires. I am Mr. Hassan Mohammed Ali, an Audit staff of (C.B.N) Central Bank of Nigeria. An investor (name with-held) died without naming any next of kin to his fund he deposited in my bank. The amount is $10.5M (Ten Million Five hundred Thousand Dollars) and banking regulation/legislation in Nigeria demands that I should notify the fiscal authorities after three years. The above set of facts underscores my reason to seek your permission to have you stand in as the next of kin to the deceased. This fund will be approved and released in your favour as the next of kin if only you will adhere to my instruction and cooperate with me in one accord. I have all the legal and banking details of the deceased client that will facilitate our putting you forward as the claimant/beneficiary of the funds and ultimately transfer of the $10.5M plus interest to any bank account nominated by you. I am prepared to compensate you with a 35% share of the total funds for your efforts. The final details will be given upon receipt an affirmation of your desire to participate. Please contact me immediately on (hassan_mohame...@yahoo.com) if you are interest in my proposal to enable me not to start scouting for another foreign partner. Thanks, Mr. Hassan Mohammed Ali. -- Forwarded message -- From: moses-support-requ...@mit.edu To: Cc: Bcc: Date: Subject: confirm e30b5966b8815fc7e6a47258b4eaa3069178d914 If you reply to this message, keeping the Subject: header intact, Mailman will discard the held message. Do this if the message is spam. If you reply to this message and include an Approved: header with the list password in it, the message will be approved for posting to the list. The Approved: header can also appear in the first line of the body of the reply. ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] Training vs Incremental Training
Dear all, I just need to Know what is the difference between Training & Incremental Training rather than time. Regards, Adel Khalifa .. Software developer ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] memory problems running moses server
Hi Sarah What version of the Moses decoder (including the code for Moses server) do you use exactly? Do you get an error message from the Moses server, i.e. does it crash? Is the process killed by the OS because of memory problems? If you can also supply a toy model and ini file, other people can try and reproduce your issue. Regards! Mathias — Mathias Müller AND-2-20 Institute of Computational Linguistics University of Zurich Switzerland +41 44 635 75 81 mathias.muel...@uzh.ch On Thu, Mar 23, 2017 at 11:17 AM, Sarah Schulz < sarah.sch...@ims.uni-stuttgart.de> wrote: > Hi everyone, > > It seems like I run into memory problems when using moses server even > though I have plenty of memory available. > > I isolated the problem in a little script (attached). I start the server > and send strings for translation iteratively. At a certain point it will > send the request and wait for a reply forever. The number of iterations > decreases, when I increase the length of the string. > It seems as if the already sent requests are "piling up" somewhere in > the server's or client's memory and at a certain point it doesn't have > the capacity to process the new request. > > Has anyone gone through anything similar? > > Cheers > Sarah > -- > Sarah Schulz > University of Stuttgart > Institute for Natural Language Processing (IMS) > Pfaffenwaldring 5B, 70569 Stuttgart > Germany > > http://www.ims.uni-stuttgart.de/institut/mitarbeiter/schulzsh/ > > ___ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support > > ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] Deadline Extension and Final Call - EAMT 2017 Workshop on Social Media and User Generated Content Machine Translation (Social MT 2017)
Dear all, due to several requests, we extend the submission deadline to *April 3, 2017*. Regards, Workshop Organizers = LAST CALL FOR PAPERS Apologies for multiple postings. The first Workshop on Social Media and User Generated Content Machine Translation (Social MT 2017) Co-located with EAMT 2017, Prague, Czech Republic For more information please visit: https://sites.google.com/view/socialmt CALL FOR PAPERS With the widespread adoption of social media and online forums, individual users have been able to actively participate in the generation of online content in different languages and dialects. As a result, user-generated content (UGC) has seen an enormous growth in the recent years. The nature of UGC means that it can be generated at any time and in non-standard language or formats. Compared to professionally edited text, it is often more noisy, and likely to take some liberty with commonly established grammar, punctuation and spelling norms. All this can make it difficult to translate but UGC can also be incredibly valuable. This workshop will explore the multifarious aspects of effective MT of data extracted from social media. The workshop aims to provide a research platform dedicated to new method and techniques on translating user-generated content and exploring the use of such transition on social media analytics. The workshop will solicit original research contributions related to the theme, which includes (but is not limited to): - Models and Tools Development for Social MT - Machine translation on Microblogs - Multi-lingual social analytics - Neural MT for UGC translation - Multilingual crowdsourcing - Building resources for UGC translation - Sentiment translation of UGC - Analyzing the diffusion of multilingual information - Using MT for monitoring emergency responses among social crowds - Multilingual Social-based web platform for disaster management - Multilingual and language-specific Information Retrieval on Social Web - Crosslingual document alignment using UGC data - Named entity transliteration on social media content - Code-mixed UGC translation - MT for Big social data analysis Submissions may include work in progress as well as finished work. Submissions must have a clear focus on specific issues pertaining to UGC and its translation. Descriptions of commercial systems are welcome, but authors should be willing to discuss the details of their work. IMPORTANT DATES January 30, 2017: First Call for Workshop Papers March 8, 2017: Second Call for Workshop Papers *April 3, 2017*: Workshop Paper Due Date April 14, 2017: Notification of Acceptance May 12, 2017: Camera-ready papers due May 31, 2017: Workshop Date (half-day workshop) SUBMISSION FORMAT Submissions must conform to the official style guidelines for EAMT 2017 ( https://ufal.mff.cuni.cz/pbml/instructions-authors). Overleaf Project to Clone: https://www.overleaf.com/read/jgvrjmrqnwct MS Word Template: https://ufal.mff.cuni.cz/eamt2017/files/ templates/eamt17.doc Contributions can be short or long papers. Short paper submission must describe original and unpublished work without exceeding eight (8) pages plus any number of pages for references. Characteristics of short papers include: a small, focused contribution; work in progress; a negative result; an opinion piece; an interesting application nugget. Long paper submissions must describe substantial, original, completed and unpublished work without exceeding twelve (12) pages plus any number of pages for references. Reviewing will be double-blind, so the papers should not reveal the authors’ identity. Accepted papers will be published in the workshop proceedings. Double submission policy: Parallel submission to other meetings or publications is possible but must be immediately notified to the workshop organizers. Submission Website: https://easychair.org/conferences/?conf=socialmt2017 Extended versions of the best papers will be published into an upcoming special issue of “Translating User Generated Content” on Machine Translation Journal INVITED SPEAKER: Houda Bouamor (Carnegie Mellon University, Qatar) WORKSHOP ORGANIZERS General Chair: Andy Way (ADAPT Centre, Dublin City University) Program Chair :Haithem Afli (ADAPT Centre, Dublin City University) Program Committee Loïc Barrault (LIUM, Le Mans University) Laurent Besacier (LIG, Grenoble University) Philipp Koehn (University of Edinburgh / Johns Hopkins University) Abdelkarim Mars (Grenoble University) Matteo Negri (FBK) Houda Bouamor (CMU Qatar) Yvette Graham (ADAPT Centre, Dublin City University) Dimitar Shterionov (KantanMT) Marco Turchi (FBK) Antonio Toral (University of Groningen) Lucia Specia (University of Sheffield) Kashif Shah (eBay) Rejwanul Haque (Lingo24) Barry Haddow (University of Edinburgh) Jinhua Du (ADAPT Centre, Dublin City University) Daniel Stein (eBay) Mohammed Hasanuzzaman (ADAPT Centre, Dublin City University)
[Moses-support] memory problems running moses server
Hi everyone, It seems like I run into memory problems when using moses server even though I have plenty of memory available. I isolated the problem in a little script (attached). I start the server and send strings for translation iteratively. At a certain point it will send the request and wait for a reply forever. The number of iterations decreases, when I increase the length of the string. It seems as if the already sent requests are "piling up" somewhere in the server's or client's memory and at a certain point it doesn't have the capacity to process the new request. Has anyone gone through anything similar? Cheers Sarah -- Sarah Schulz University of Stuttgart Institute for Natural Language Processing (IMS) Pfaffenwaldring 5B, 70569 Stuttgart Germany http://www.ims.uni-stuttgart.de/institut/mitarbeiter/schulzsh/ import random import xmlrpclib as x import time import sys import subprocess #initialize random port and start moses server as subprocess port_uni= str(random.randint(4, 5)) p1 = subprocess.Popen("/mount/arbeitsdaten13/users/schulzsh/tools/Moses/mosesdecoder/bin/mosesserver -f ../static/Moses/decoder_files/SMT_Werther_uni.ini --server-port "+ port_uni,stderr=subprocess.PIPE,bufsize=8000,shell=True,env={"LD_LIBRARY_PATH":"/mount/projekte/sfb-732/inf/users/sarah/tools/xmlrpc-c/lib"}) #wait for it to start sucessfully elapsed_time=0 start_time = time.time() err='' out='' for line in iter(p1.stderr.readline, ''): if "Listening on port" in line or elapsed_time > 150: print "loaded uni1" break elapsed_time = time.time() - start_time if elapsed_time >= 150: print "SMT Unigram Server Mode doesn't start." sys.exit() #connect to ServerProxy s_unigram = x.ServerProxy("http://localhost:"+port_uni+"/RPC2",allow_none=True,verbose=True) #send a message to moses server for translation #note: when message length increases, the bug appears after fewer iterations for i in range(1000): trans= s_unigram.translate({"text":"# I c h j k k k # k k k k k k k k k k k #","align":"true"}) print trans print i ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support