Re: [Tutor] Python 2 & 3 and unittest
Steven D'Aprano wrote: > On Thu, Sep 05, 2013 at 09:11:50AM +1000, Steven D'Aprano wrote: > >> I don't believe there is a way to make >> string literals unicode, you just have to get used to writing u"" and >> b"" strings by hand. > > Sorry, that is unclear. I meant to say, there is no way to force > unprefixed strings "" to be Unicode in 2.x. For 2.6 and above there is from __future__ import unicode_literals ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] myown.getfilesystemencoding()
On Wed, Sep 04, 2013 at 05:39:10AM -0700, Albert-Jan Roskam wrote: > Wow, thanks for looking all this up. Thanks also to other people who > replied. It's not really desirable that a IDE adds confusion to an > area that's already confusing to begin with. Well, naturally it isn't desirable to add confusion, but I think that when dealing with IDEs it is unavoidable. The whole point of an IDE is that it is an *integrated* environment, which implies that the environment that Python runs in is not the same as unintegrated Python would be running in. > But given that chcp > returns cp850 on my windows system (commandline), wouldn't it be more > descriptive if sys.getfilesystemencoding() returned 'cp850'? I cannot comment on the gory details of Windows file system encodings, except to say the sooner Windows moves to UTF-8 everywhere like the rest of the civilized world, the better. -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python 2 & 3 and unittest
On Thu, Sep 05, 2013 at 09:11:50AM +1000, Steven D'Aprano wrote: > I don't believe there is a way to make > string literals unicode, you just have to get used to writing u"" and > b"" strings by hand. Sorry, that is unclear. I meant to say, there is no way to force unprefixed strings "" to be Unicode in 2.x. -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] resolution
On Wed, Sep 04, 2013 at 10:50:54AM +0200, musa ezeri wrote: > im building an image processing software, but the python image processing > modules are in 32 bit and require 32bit python 2.7 where can i find a 32 > bit python 2.7? http://www.python.org/download/ -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python 2 & 3 and unittest
On Wed, Sep 04, 2013 at 06:30:12AM -0700, Albert-Jan Roskam wrote:
> Hi,
>
> I am trying to make my app work in Python 2.7 and Python 3.3 (one
> codebase) and I might later also try to make it work on Python 2.6 and
> Python 3.2 (if I am not too fed up with it ;-). I was very happy to
> notice that the 'b' prefix for bytes objects is also supported for
> byte strings in Python 2.7. Likewise, I just found out that the "u"
> has been re-introduced in Python 3.3 (I believe this is the first
> Python 3 version where this re-appears).
>
> I am now cursing myself for having used doctest for my tests.
Well that's just silly :-)
Doc tests and unit tests have completely different purposes, and
besides, in general doc tests are easier to make version independent.
Many of your doc tests will still continue to work, and those that don't
can almost always be adapted to be cross-platform.
> So I am planning to rewrite everything in unittest.
> Is the try-except block below the best way to make this test work in
> Python 2.6 through 3.3?
>
> import unitttest
> import blah # my package
>
>
> class test_blah(unittest.TestCase):
> def test_someTest(self):
> try:
> expected = [u"lalala", 1] # Python 2.6>= & Python 3.3>=
> except SyntaxError:
> expected = ["lalala", 1] # Python 3.0, 3.1, 3.2
That cannot work. try...except catches *run time* exceptions.
SyntaxError occurs at *compile time*, before the try...except gets a
chance to run.
Unfortunately, there is no good way to write version-independent code
involving strings across Python 2.x and 3.x. If you just support 3.3 and
better, it is simple, but otherwise you're stuck with various nasty work
arounds, none of which are ideal.
Probably the least worst for your purposes is to create a helper
function in your unit test:
if version < '3':
def u(astr):
return unicode(astr)
else:
def u(astr):
return astr
Then, everywhere you want a Unicode string, use:
u("something")
The two problems with this are:
1) It is slightly slower, but for testing purposes that doesn't really
matter; and
2) You cannot write non-ASCII literals in your strings. Or at least not
safely.
> Another, though related question. We have Python 2.7 in the office and
> eventually we will move to some Python3 version. The code does not
> generally need to remain Python2 compatible. What is the best
> strategy: [a] write forward compatible code when using Python 2.7.
> (using a 'b' prefix for byte strings, parentheses for the print
> *statement*, sys.exc_info()[1] for error messages, etc.). [b] totally
> rely on 2to3 script and don't make the Python2 code less reabable and
> less idiomatic before the upgrade.
Option a, but not the way you say it. Start by putting
from __future__ import division, print_function
at the top of your 2.x code. I don't believe there is a way to make
string literals unicode, you just have to get used to writing u"" and
b"" strings by hand.
You can also do:
from future_builtins import *
range = xrange
which will replace a bunch of builtins with Python3 compatible versions.
Be prepared for a torrid time getting used to Unicode strings. Not
because Unicode is hard, it isn't, but because you'll have to unlearn a
lot of things that you thought you knew. The first thing to unlearn is
this: there is no such thing as "plain text".
Unfortunately Python 2 tries to be helpful when dealing with text versus
bytes, and that actually teaches bad habits. This is the sort of thing
I'm talking about:
[steve@ando ~]$ python2.7 -c "print 'a' + u'b'"
ab
That sort of implicit conversion of bytes and strings is actually a bad
idea, and Python 3 prohibits it:
[steve@ando ~]$ python3.3 -c "print(b'a' + u'b')"
Traceback (most recent call last):
File "", line 1, in
TypeError: can't concat bytes to str
The other bad thing about Unicode is that, unless you are lucky enough
to be generating all your own textual data, you'll eventually have to
deal with cross-platform text issues, and text generated by people who
didn't understand Unicode and therefore produce rubbish data containing
mojibake and worse.
But the good thing is, the Unicode model actually isn't hard to
understand, and once you learn the language of "encodings", "code
points" etc. it makes great sense.
Unless you're working with binary data, you are much better off learning
how to use Unicode u"" strings now. Just be aware that unless you are
careful, Python 2 will try to be helpful, and you don't want that.
--
Steven
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] myown.getfilesystemencoding()
On Wed, Sep 4, 2013 at 8:39 AM, Albert-Jan Roskam wrote:
> But given that chcp returns cp850 on my windows system (commandline),
> wouldn't it be more descriptive if sys.getfilesystemencoding()
> returned 'cp850'?
The common file systems (NTFS, FAT32, UDF, exFAT) support Unicode
filenames. The console also uses Unicode, but proper display depends
on the current font.
The cmd shell encodes to the current codepage when redirecting output
from an internal command, unless it was started with /U to force
Unicode (e.g. cmd /U /c dir > files.txt). For subprocess, run cmd.exe
explicitly with /U (i.e. don't use shell=True), and decode the output
as UTF-16. Also, some utilities, such as tree.com, display Unicode
fine but always use the OEM code page when output is redirected to a
file or pipe (i.e. changing the console code page won't help).
> In other words: In the code below, isn't line [1] an obfuscated version of
> line [2]? Both versions return only question marks on my system.
>
> # Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)]
> on win32
> import ctypes
>
> ords = [3629, 3633, 3585, 3625, 3619, 3652, 3607, 3618]
> u = "".join([unichr(i) for i in ords])
> print u.encode("mbcs") # [1]
>
> #cp850 is what chcp returns on my Windows system
> print u.encode("cp850", "replace") # [2]
>
> thai_latin_cp = "cp874"
> cp_ = int(thai_latin_cp[2:])
> ctypes.windll.kernel32.SetConsoleCP(cp_)
> ctypes.windll.kernel32.SetConsoleOutputCP(cp_)
> print u.encode("cp874", "replace")
"mbcs" is the ANSI codepage (1252), not the OEM codepage (850) nor the
current codepage. Neither supports Thai characters. It would be better
to compare an OEM box drawing character:
>>> from unicodedata import name
>>> u = u'\u2500'
>>> name(u)
'BOX DRAWINGS LIGHT HORIZONTAL'
>>> name(u.encode('850', 'replace').decode('850'))
'BOX DRAWINGS LIGHT HORIZONTAL'
>>> name(u.encode('mbcs', 'replace').decode('mbcs'))
'HYPHEN-MINUS'
> ctypes.windll.kernel32.SetConsoleCP() and SetConsoleOutputCP seem useful.
> Can these functions be used to correctly display the Thai characters on
> my western European Windows version? (last block of code is an attempt)
> Or is that not possible altogether?
If stdout is a console, a write eventually ends up at WriteConsoleA(),
which decodes to the console's native Unicode based on the current
output codepage. If you're using codepage 847 and the current font
supports Thai characters, it should display fine. It's also possible
to write a Unicode string directly by calling WriteConsoleW with
ctypes.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Map Element Automation
Please use ReplyAll to include the list in your responses. I'm using python to automate this process. The map elements can be automatically >updated using python, I just can't figure out how to access an attribute in >the layer's >attribute table. I don't think there is any other way to automate a map >without python.You are going to have to give us an awful lot more information. What OS and Python version? What technology is your map using? (database, language etc?) Is is a commercial package? Open source? Which one? What architecture does your solution have? (eg client/server, web server? desktop?) Also what is your experience level in programming in general and Python in particular? Its probable that there is a more appropriate forum to ask your questions but we can't even direct you there with the little information you have given so far. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] postgresql was: Re: Tutor Digest, Vol 115, Issue 6
On 04-Sep-2013, at 14:28, Alan Gauld wrote:
> On 03/09/13 08:25, Ismar Sehic wrote:
>> help with postgres and csv:
>> i solved my problem by playing with the sql line a little.
>> it looked like this : sql = " UPDATE hotel SET path_picture =
>> "+"';"+hotel_url+"'
>> WHERE code LIKE '"+"%"+hotel_code+"'"
>> now it's like this : " UPDATE hotel SET path_picture = '" + hot_url +
>> "' WHERE code LIKE '%" + hot_code + "';"
>>
>> i guess the problem was in building the sql string, but i don't yet
>> quite understand what i did.can someone point me to some online resorces
>> about postgres and python integration?
>
> https://wiki.python.org/moin/PostgreSQL
>
While you're looking at all the information Alan pointed you to, consider one
other general bit of advice when programming with SQL queries. It is generally
a very convenient trick to use string formatting or string catenation to build
the bits of your query from pieces, like you did above ("UPDATE … SET
path_picture='" + hot_url + …).
Convenient, but a very, very bad idea in practice. This makes your program
vulnerable to SQL injection, which in many cases can have devastating effects
when someone exploits it. Assuming that the variables come from sources beyond
your control (and even if they are--at the moment--generated by you), use
parameterized queries (look for those in your API libraries). They usually
look something like the following (although specifics can vary), where you
leave a placeholder character like ? in the SQL string, and supply the data
values separately. Unlike using string-maniputation features of Python, the
database API knows exactly how to properly include those data values into the
SQL command for you:
some_api_function_to_do_sql("UPDATE hotel SET path_picture = ? WHERE code LIKE
?",
hot_url, '%' + hot_code + '%')
--steve
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] postgresql was: Re: Tutor Digest, Vol 115, Issue 6
On 03/09/13 08:25, Ismar Sehic wrote: help with postgres and csv: i solved my problem by playing with the sql line a little. it looked like this : sql = " UPDATE hotel SET path_picture = "+"';"+hotel_url+"' WHERE code LIKE '"+"%"+hotel_code+"'" now it's like this : " UPDATE hotel SET path_picture = '" + hot_url + "' WHERE code LIKE '%" + hot_code + "';" i guess the problem was in building the sql string, but i don't yet quite understand what i did.can someone point me to some online resorces about postgres and python integration? Googling "python postgres" seems to throw up quite a few links. Did you try any of them? If so do you have a specific area of interest? Also there are several APIs to use, each has links to their own sites. You can find a list here: https://wiki.python.org/moin/PostgreSQL Finally, please don't quote the entire digest when replying and please use a sensible subject line. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Map Element Automation
On 04/09/13 17:42, Abbott, Jennifer ENV wrote: I am trying to automate a map so that every time I make a new map with the same template the sub titles and other text changes according to the specific map properties. For instance, a field from the attribute table has the PROPERTY # and with each map I need that property number to be shown on each map. And where does Python fit in? This is a list for those learning Python and its standard library. How does your question relate to that? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] resolution
On 04/09/13 09:50, musa ezeri wrote: im building an image processing software, but the python image processing modules are in 32 bit and require 32bit python 2.7 where can i find a 32 bit python 2.7? On the Python.org download page I would say. Which OS? If it's Linux and you are on 64bit it might be more tricky... If so which distro? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Tutor Digest, Vol 115, Issue 6
help with postgres and csv:
i solved my problem by playing with the sql line a little.
it looked like this : sql = " UPDATE hotel SET path_picture =
"+"';"+hotel_url+"'
WHERE code LIKE '"+"%"+hotel_code+"'"
now it's like this : " UPDATE hotel SET path_picture = '" + hot_url + "'
WHERE code LIKE '%" + hot_code + "';"
i guess the problem was in building the sql string, but i don't yet quite
understand what i did.can someone point me to some online resorces about
postgres and python integration?as i see it, python and postgresql are a
quite powerful combination, when done properly.i would like to learn more
about this, and would appreciate some pointing in the right direction.
On Tue, Sep 3, 2013 at 4:35 AM, wrote:
> Send Tutor mailing list submissions to
> tutor@python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mail.python.org/mailman/listinfo/tutor
> or, via email, send a message with subject or body 'help' to
> tutor-requ...@python.org
>
> You can reach the person managing the list at
> tutor-ow...@python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Tutor digest..."
>
>
> Today's Topics:
>
>1. os.system vs subprocess.Popen args problems (learner404)
>2. Re: os.system vs subprocess.Popen args problems (Oscar Benjamin)
>3. help with postgreSQL and .csv (Ismar Sehic)
>4. Using tkinter for cross platform application (Max Bedacht)
>5. Re: help with postgreSQL and .csv (R. Alan Monroe)
>
>
> --
>
> Message: 1
> Date: Mon, 2 Sep 2013 16:19:17 +0200
> From: learner404
> To: Tutor Python
> Subject: [Tutor] os.system vs subprocess.Popen args problems
> Message-ID:
> oaozuvrgth1ru-...@mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hello,
>
> I can't understand why the command below works with os.system but not with
> subprocess.Popen (on windows, recording video with FFMPEG.exe)
>
> cmd=('ffmpeg -f dshow -i video="%s" -f dshow -i audio="%s" -q 5
> "%s"')%(videoinputName, audioinputName, videoFileOutput)
> os.system(cmd)
> *works*
>
>
> subprocess.Popen(["ffmpeg","-f","dshow","-i",'video="%s"'%videoinputName,"-f","dshow","-i",'audio="%s"'%audioinputName,"-q","5","%s"%videoFileOutput])
> *don't work*
> >> [dshow @ 025984a0] Could not find video device.
> >> video="QuickCam Orbit/Sphere AF": Input/output error
>
> The reason I'm going with subprocess is to avoid to have the DOS window
> coming in the forefront.
> Any idea of what I'm doing wrong? Or any other way to hide or minimize the
> DOS window?
>
> Thanks.
> -- next part --
> An HTML attachment was scrubbed...
> URL: <
> http://mail.python.org/pipermail/tutor/attachments/20130902/8e44abe0/attachment-0001.html
> >
>
> --
>
> Message: 2
> Date: Mon, 2 Sep 2013 15:44:25 +0100
> From: Oscar Benjamin
> To: learner404
> Cc: Tutor Python
> Subject: Re: [Tutor] os.system vs subprocess.Popen args problems
> Message-ID:
> ky5zt7db+p5ofb4erlpcmbmg...@mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> On 2 September 2013 15:19, learner404 wrote:
> > Hello,
> >
> > I can't understand why the command below works with os.system but not
> with
> > subprocess.Popen (on windows, recording video with FFMPEG.exe)
> >
> > cmd=('ffmpeg -f dshow -i video="%s" -f dshow -i audio="%s" -q 5
> > "%s"')%(videoinputName, audioinputName, videoFileOutput)
> > os.system(cmd)
> > *works*
> >
> >
> subprocess.Popen(["ffmpeg","-f","dshow","-i",'video="%s"'%videoinputName,"-f","dshow","-i",'audio="%s"'%audioinputName,"-q","5","%s"%videoFileOutput])
>
> I think you need additional quotes for the videoFileOutput part e.g.
>
> '"%s"' % videoFileOutput
>
> > *don't work*
> >>> [dshow @ 025984a0] Could not find video device.
> >>> video="QuickCam Orbit/Sphere AF": Input/output error
>
> I see that video has spaces in it. You'd think it was unnecessary to
> have those quotes since you've already split the command string into a
> list of strings. Unfortunately Windows has a fairly arcane way of
> handling arguments to subprocesses and Python tries to emulate that:
>
> http://docs.python.org/2/library/subprocess.html#converting-an-argument-sequence-to-a-string-on-windows
>
> > The reason I'm going with subprocess is to avoid to have the DOS window
> > coming in the forefront.
> > Any idea of what I'm doing wrong? Or any other way to hide or minimize
> the
> > DOS window?
>
> Another thing is that you shouldn't use Popen directly unless you're
> doing something fancy (which you're not if os.system does what you
> want). Use subprocess.check_call instead:
>
>
> subprocess.check_call(["ffmpeg","-f","dshow","-i",'video="%s"'%videoinputName,"-f","dshow","-i",'audio="%s"'%audioinputName,"-q","5",'"%s"'%videoFileOutput])
>
>
> Oscar
>
>
> --
>
> Message: 3
> Date:
[Tutor] Map Element Automation
HI, I am trying to automate a map so that every time I make a new map with the same template the sub titles and other text changes according to the specific map properties. For instance, a field from the attribute table has the PROPERTY # and with each map I need that property number to be shown on each map. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] resolution
im building an image processing software, but the python image processing modules are in 32 bit and require 32bit python 2.7 where can i find a 32 bit python 2.7? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python 2 & 3 and unittest
On Sep 4, 2013, at 9:30 AM, Albert-Jan Roskam wrote: > Hi, > > I am trying to make my app work in Python 2.7 and Python 3.3 (one codebase) > and I might later also try to make it work on Python 2.6 and Python 3.2 (if I > am not too fed up with it ;-). You might like to read Armin Ronacher's (Flask, Jinja2) take on this topic [1]. Take care, Don [1] http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Python 2 & 3 and unittest
Hi, I am trying to make my app work in Python 2.7 and Python 3.3 (one codebase) and I might later also try to make it work on Python 2.6 and Python 3.2 (if I am not too fed up with it ;-). I was very happy to notice that the 'b' prefix for bytes objects is also supported for byte strings in Python 2.7. Likewise, I just found out that the "u" has been re-introduced in Python 3.3 (I believe this is the first Python 3 version where this re-appears). I am now cursing myself for having used doctest for my tests. So I am planning to rewrite everything in unittest. Is the try-except block below the best way to make this test work in Python 2.6 through 3.3? import unitttest import blah # my package class test_blah(unittest.TestCase): def test_someTest(self): try: expected = [u"lalala", 1] # Python 2.6>= & Python 3.3>= except SyntaxError: expected = ["lalala", 1] # Python 3.0, 3.1, 3.2 got = blah.yadiyadiyadi() self.assertEqual(expected, got) if __name__ == "__main__": unittest.main() Another, though related question. We have Python 2.7 in the office and eventually we will move to some Python3 version. The code does not generally need to remain Python2 compatible. What is the best strategy: [a] write forward compatible code when using Python 2.7. (using a 'b' prefix for byte strings, parentheses for the print *statement*, sys.exc_info()[1] for error messages, etc.). [b] totally rely on 2to3 script and don't make the Python2 code less reabable and less idiomatic before the upgrade. Regards, Albert-Jan ~~ All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health, what have the Romans ever done for us? ~~ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] myown.getfilesystemencoding()
- Original Message -
> From: eryksun
> To: Oscar Benjamin ; Albert-Jan Roskam
>
> Cc: Python Mailing List
> Sent: Sunday, September 1, 2013 7:30 AM
> Subject: Re: [Tutor] myown.getfilesystemencoding()
>
> On Sat, Aug 31, 2013 at 9:16 AM, Oscar Benjamin
> wrote:
>> Spyder has both an internal interpreter and an external interpreter.
>> One is the same interpreter process that runs the Spyder GUI. The
>> other is run in a subprocess which keeps the GUI safe but reduces your
>> ability to inspect the workspace data via the GUI. So presumable
>> Albert means the "external" interpreter here.
>
> I installed Spyder on Windows to look into this. It's using Qt
> QProcess to run the external interpreter in a child process.
> sys.stdin.isatty() confirms it's not a tty, and Process Explorer
> confirms that all 3 standard I/O handles (from msvcrt.get_osfhandle())
> are pipes.
>
> The file encoding is None for piped standard I/O, so printing unicode
> falls back to the default encoding. Normally this is ASCII in 2.x, but
> Spyder uses sitecustomize to set the default encoding based on the
> default locale. It also sets the hidden console's codepage:
>
> if os.name == 'nt': # Windows platforms
>
> # Setting console encoding (otherwise Python does not
> # recognize encoding)
> try:
> import locale, ctypes
> _t, _cp = locale.getdefaultlocale('LANG')
> try:
> _cp = int(_cp[2:])
> ctypes.windll.kernel32.SetConsoleCP(_cp)
> ctypes.windll.kernel32.SetConsoleOutputCP(_cp)
> except (ValueError, TypeError):
> # Code page number in locale is not valid
> pass
> except ImportError:
> pass
>
> http://code.google.com/p/spyderlib/source/browse/spyderlib/
> widgets/externalshell/sitecustomize.py?name=v2.2.0#74
>
> Probably this was added for a good reason, but I don't grok the point.
> Python isn't interested in the hidden console window at this stage,
> and the standard handles are all pipes. I didn't notice any difference
> with these lines commented out, running with Python 2.7.5. YMMV
>
> There's a design flaw here since sys.stdin.encoding is used by the
> parser in single-input mode. With it set to None, Unicode literals
> entered in the REPL will be incorrectly parsed if they use non-ASCII
> byte values. For example, given the input is Windows 1252, then u'€'
> will be parsed as u'\x80' (i.e. PAD, a C1 Control code).
>
> Here's an alternative to messing with the default encoding -- at least
> for the new version of Spyder that doesn't have to support 2.5. Python
> 2.6+ checks for the PYTHONIOENCODING environment variable. This
> overrides the encoding/errors values in Py_InitializeEx():
>
> http://hg.python.org/cpython/file/70274d53c1dd/Python/pythonrun.c#l265
>
> You can test setting PYTHONIOENCODING without restarting Spyder. Just
> bring up Spyder's "Internal Console" and set
> os.environ['PYTHONIOENCODING']. The change applies to new interpreters
> started from the "Interpreters" menu. Spyder could set this itself in
> the environment that gets passed to the QProcess object.
Wow, thanks for looking all this up. Thanks also to other people who replied.
It's not really desirable that a IDE adds confusion to an area that's already
confusing to begin with. But given that chcp returns cp850 on my windows system
(commandline), wouldn't it be more descriptive if sys.getfilesystemencoding()
returned 'cp850'?
In other words: In the code below, isn't line [1] an obfuscated version of line
[2]? Both versions return only question marks on my system.
# Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on
win32
import ctypes
ords = [3629, 3633, 3585, 3625, 3619, 3652, 3607, 3618]
u = "".join([unichr(i) for i in ords])
print u.encode("mbcs") # [1]
#cp850 is what chcp returns on my Windows system
print u.encode("cp850", "replace") # [2]
thai_latin_cp = "cp874"
cp_ = int(thai_latin_cp[2:])
ctypes.windll.kernel32.SetConsoleCP(cp_)
ctypes.windll.kernel32.SetConsoleOutputCP(cp_)
print u.encode("cp874", "replace")
ctypes.windll.kernel32.SetConsoleCP() and SetConsoleOutputCP seem useful. Can
these functions be used to correctly display the Thai characters on my western
European Windows version? (last block of code is an attempt) Or is that not
possible altogether?
Best wishes,
Albert-Jan
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] os.system vs subprocess.Popen args problems
On Wed, Sep 4, 2013 at 7:11 AM, Oscar Benjamin wrote: > On 4 September 2013 11:42, eryksun wrote: si = subprocess.STARTUPINFO() si.dwFlags |= subprocess.STARTF_USESHOWWINDOW si.wShowWindow = subprocess.SW_HIDE >>> >>> Do these statements modify module-level state? Or do you need to pass >>> startupinfo=si when calling Popen? >> >> It's a documented option for instances of Popen, to configure a few >> fields in the STARTUPINFO struct that gets passed to CreateProcess. > > It's not clear from the docs (or your explanation) whether the lines > above modify a global STARTUPINFO struct that gives default behaviour > to subprocesses or if it is just the way to initialise one that can be > passed to Popen or the convenience APIs. I guess the latter in which > case it's probably better to be explicit and say: > > si = subprocess.STARTUPINFO() > si.dwFlags |= subprocess.STARTF_USESHOWWINDOW > si.wShowWindow = subprocess.SW_HIDE > subprocess.check_call(cmdargs, startupinfo=si) When I said, "[s]et startupinfo=si, where si is defined", I assumed familiarity with the Popen constructor: subprocess.Popen( args, bufsize=0, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=False, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0) ... If given, startupinfo will be a STARTUPINFO object, which is passed to the underlying CreateProcess function. creationflags, if given, can be CREATE_NEW_CONSOLE or CREATE_NEW_PROCESS_GROUP. http://docs.python.org/2/library/subprocess#popen-constructor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] os.system vs subprocess.Popen args problems
On 4 September 2013 11:42, eryksun wrote: >>> si = subprocess.STARTUPINFO() >>> si.dwFlags |= subprocess.STARTF_USESHOWWINDOW >>> si.wShowWindow = subprocess.SW_HIDE >> >> Do these statements modify module-level state? Or do you need to pass >> startupinfo=si when calling Popen? > > It's a documented option for instances of Popen, to configure a few > fields in the STARTUPINFO struct that gets passed to CreateProcess. It's not clear from the docs (or your explanation) whether the lines above modify a global STARTUPINFO struct that gives default behaviour to subprocesses or if it is just the way to initialise one that can be passed to Popen or the convenience APIs. I guess the latter in which case it's probably better to be explicit and say: si = subprocess.STARTUPINFO() si.dwFlags |= subprocess.STARTF_USESHOWWINDOW si.wShowWindow = subprocess.SW_HIDE subprocess.check_call(cmdargs, startupinfo=si) Oscar ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] os.system vs subprocess.Popen args problems
On Wed, Sep 4, 2013 at 6:22 AM, Oscar Benjamin wrote: > On 4 September 2013 11:11, eryksun wrote: > >> Using shell=True also sets startupinfo to hide the window. If that's >> the only reason you're using the shell, you may as well cut out the >> middleman (and potential security hole). Set startupinfo=si, where si >> is defined like this: > > I think it should just be possible to use shell=False here. > >> si = subprocess.STARTUPINFO() >> si.dwFlags |= subprocess.STARTF_USESHOWWINDOW >> si.wShowWindow = subprocess.SW_HIDE > > Do these statements modify module-level state? Or do you need to pass > startupinfo=si when calling Popen? It's a documented option for instances of Popen, to configure a few fields in the STARTUPINFO struct that gets passed to CreateProcess. http://docs.python.org/2/library/subprocess#subprocess.STARTUPINFO STARTUPINFO struct: http://msdn.microsoft.com/en-us/library/ms686331 PC/_subprocess.c: /* note: we only support a small subset of all SI attributes */ si.dwFlags = getint(startup_info, "dwFlags"); si.wShowWindow = getint(startup_info, "wShowWindow"); si.hStdInput = gethandle(startup_info, "hStdInput"); si.hStdOutput = gethandle(startup_info, "hStdOutput"); si.hStdError = gethandle(startup_info, "hStdError"); http://hg.python.org/cpython/file/ab05e7dd2788/PC/_subprocess.c#l444 ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] os.system vs subprocess.Popen args problems
On 4 September 2013 11:11, eryksun wrote: > On Wed, Sep 4, 2013 at 5:14 AM, learner404 wrote: >> >> Yes, this worked :) >> >> subprocess.Popen(["ffmpeg","-f","dshow","-i","video="+videoinputName,"-f", >> "dshow","-i","audio="+audioinputName,"-q","5","%s"%videoFileOutput], >> shell=True) You should use subprocess.check_call() rather than subprocess.Popen. subprocess.check_call() waits for the command to complete and detects when an error has occurred in the child process and turns it into a Python error which is normally what you want to happen before any subsequent code runs. >From the subprocess docs: "The recommended approach to invoking subprocesses is to use the following convenience functions for all use cases they can handle. For more advanced use cases, the underlying Popen interface can be used directly." http://docs.python.org/2/library/subprocess.html#using-the-subprocess-module I'm pretty sure that your problem does not come under the "more advanced" use cases of subprocess. > Using shell=True also sets startupinfo to hide the window. If that's > the only reason you're using the shell, you may as well cut out the > middleman (and potential security hole). Set startupinfo=si, where si > is defined like this: I think it should just be possible to use shell=False here. > si = subprocess.STARTUPINFO() > si.dwFlags |= subprocess.STARTF_USESHOWWINDOW > si.wShowWindow = subprocess.SW_HIDE Do these statements modify module-level state? Or do you need to pass startupinfo=si when calling Popen? Oscar ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] os.system vs subprocess.Popen args problems
On Wed, Sep 4, 2013 at 5:14 AM, learner404 wrote: > > Yes, this worked :) > > subprocess.Popen(["ffmpeg","-f","dshow","-i","video="+videoinputName,"-f", > "dshow","-i","audio="+audioinputName,"-q","5","%s"%videoFileOutput], > shell=True) Using shell=True also sets startupinfo to hide the window. If that's the only reason you're using the shell, you may as well cut out the middleman (and potential security hole). Set startupinfo=si, where si is defined like this: si = subprocess.STARTUPINFO() si.dwFlags |= subprocess.STARTF_USESHOWWINDOW si.wShowWindow = subprocess.SW_HIDE ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] os.system vs subprocess.Popen args problems
Thanks a lot Oscar and Eryksun for all the explanations and answers, I really appreciate. "So the extra quotes used for the video and audio arguments do actually get passed through to ffmpeg causing confusion." Yes, this worked :) subprocess.Popen(["ffmpeg","-f","dshow","-i","video="+videoinputName,"-f","dshow","-i","audio="+audioinputName,"-q","5","%s"%videoFileOutput],shell=True) On Tue, Sep 3, 2013 at 5:42 PM, Oscar Benjamin wrote: > On 3 September 2013 14:48, eryksun wrote: > >> It occurs to me that another possibility is if ffmpeg isn't really an > >> .exe on PATH but rather a .bat file or something. In that case > >> os.system or subprocess shell=True would pick it up but subprocess > >> shell=False might not. I say "might" because at least on some version > >> of Windows CreateProcess can run .bat files directly but I've never > >> seen this documented anywhere. > > > > cmd tries each PATHEXT extension, but CreateProcess only tries .exe, > > and finds ffmpeg.exe. > > > > As to batch files, "Windows Internals" (Microsoft Press) documents > > that CreateProcess starts cmd.exe to run .bat and .cmd files. > > Okay, I see. So it can run a .bat if you give the extension but not > implicitly via PATHEXT. In which case anything without an extension > would have to be an .exe file. And of course this wouldn't explain the > OP's problem anyway since they're getting output from ffmpeg. > > Testing the OP's actual commands out with ffmpeg I see that the > problem is with the quotes. But, as you pointed out in your first > post, it is the unnecessary additional quotes that are the problem > rather than any missing ones. i.e. when I test it I get: > > # os.system output > [dshow @ 02548460] Could not enumerate video devices. > video=video: Input/output error > > # subprocess.Popen output > [dshow @ 02548460] Could not enumerate video devices. > video="video": Input/output error > > So the extra quotes used for the video and audio arguments do actually > get passed through to ffmpeg causing confusion. (I don't have any > video devices here so it's an error either way on this machine). > > > Oscar > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
