Re: [Django] #11869: makemessages command hangs when there're many errors in the .po file

[Django]

#11869: makemessages command hangs when there're many errors in the .po file
---+
  Reporter:  jmv   | Owner:  nobody 

Status:  new   | Milestone:  1.2

 Component:  Internationalization  |   Version:  SVN

Resolution:|  Keywords:  gettext, msgmerge, 
popen3, i18n
 Stage:  Accepted  | Has_patch:  1  

Needs_docs:  0 |   Needs_tests:  0  

Needs_better_patch:  0 |  
---+
Changes (by ramiro):

  * has_patch:  0 => 1
  * version:  1.1 => SVN

Comment:

 I've implemented (and tested under Linux and Windows with .po files that
 generate 250+ msgmerge warnings) the change suggested by soulburner and
 the `subprocess` module documentation (particularly with the use of the
 `close_fds` argument argument in the `Popen` object).

 This patch also fixes #12783 (using `universal_newlines=True` argument in
 the `Popen` object construction).

 Feel free to change the `_popen3` name I chose for the helper function.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-upda...@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #11869: makemessages command hangs when there're many errors in the .po file

[Django]

#11869: makemessages command hangs when there're many errors in the .po file
---+
  Reporter:  jmv   | Owner:  nobody 

Status:  new   | Milestone:  1.2

 Component:  Internationalization  |   Version:  1.1

Resolution:|  Keywords:  gettext, msgmerge, 
popen3, i18n
 Stage:  Accepted  | Has_patch:  0  

Needs_docs:  0 |   Needs_tests:  0  

Needs_better_patch:  0 |  
---+
Changes (by jacob):

  * stage:  Unreviewed => Accepted
  * milestone:  => 1.2

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-upda...@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #11869: makemessages command hangs when there're many errors in the .po file

[Django]

#11869: makemessages command hangs when there're many errors in the .po file
---+
  Reporter:  jmv   | Owner:  nobody 

Status:  new   | Milestone: 

 Component:  Internationalization  |   Version:  1.1

Resolution:|  Keywords:  gettext, msgmerge, 
popen3, i18n
 Stage:  Unreviewed| Has_patch:  0  

Needs_docs:  0 |   Needs_tests:  0  

Needs_better_patch:  0 |  
---+
Changes (by soulburner):

  * needs_better_patch:  => 0
  * needs_tests:  => 0
  * needs_docs:  => 0

Comment:

 ran into the same issue (repeatable, just had a file with a lot of
 error(warning because i had a lot of strings with unnamed arguments)),
 from python sub-process docs
 "
 Warning

 This will deadlock if the child process generates enough output to a
 stdout or stderr pipe such that it blocks waiting for the OS pipe buffer
 to accept more data. Use communicate() to avoid that.
 "
 so my fix was this

 from subprocess import Popen, PIPE
 p = Popen(cmd, shell=True,stdin=PIPE, stdout=PIPE,
 stderr=PIPE)
 (msgs, errors) = p.communicate()

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

[Django] #11869: makemessages command hangs when there're many errors in the .po file

[Django]

#11869: makemessages command hangs when there're many errors in the .po file
-+--
 Reporter:  jmv  |   Owner:  nobody
   Status:  new  |   Milestone:
Component:  Internationalization | Version:  1.1   
 Keywords:  gettext, msgmerge, popen3, i18n  |   Stage:  Unreviewed
Has_patch:  0|  
-+--
 I have found a bug in the `makemessages` command. It deadlocks when
 spawned `msgmerge` utility spits too many errors or warnings (for example,
 about wrong escape sequences in the po file). In the ideal world these
 messages wouldn't appear, but we've used custom written po editor for too
 long and it didn't validate translations as strictly as `msgmerge` does,
 so we get quite a lot of warnings from the `msgmerge`.

 makemessages.py uses the following code for spawning msgmerge.py:
 {{{
 (stdin, stdout, stderr) = os.popen3('msgmerge -q "%s"
 "%s"' % (pofile, potfile), 't')
 msgs = stdout.read()
 errors = stderr.read()
 if errors:
 raise CommandError("errors happened while running
 msgmerge\n%s" % errors)
 }}}
 which blocks when there're warnings or errors. I've googled only one
 report of the `makemessages` deadlock, so this doesn't happen often, but
 when it happens, it's a real PITA.

 It's also very hard to repeat - sometimes it doesn't hang even if there're
 many errors, sometimes there're fewer errors but it hangs anyway. I
 believe I have one .po file which can be used to reproduce the bug, which
 I can share via private communication channel (for example, email, since I
 cannot share here where anyone would be able to see it).

 In the meantime, I'll clone `makemessages` command in the code of my
 application and change the relevant code with the non-blocking version.
 Eugene

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---