Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython
On Fri, Jan 22, 2010 at 11:07 AM, Collin Winter wrote: > Hey Jake, > > On Thu, Jan 21, 2010 at 10:48 AM, Jake McGuire wrote: >> On Thu, Jan 21, 2010 at 10:19 AM, Reid Kleckner wrote: >>> On Thu, Jan 21, 2010 at 12:27 PM, Jake McGuire wrote: On Wed, Jan 20, 2010 at 2:27 PM, Collin Winter wrote: > Profiling > - > > Unladen Swallow integrates with oProfile 0.9.4 and newer [#oprofile]_ to > support > assembly-level profiling on Linux systems. This means that oProfile will > correctly symbolize JIT-compiled functions in its reports. Do the current python profiling tools (profile/cProfile/pstats) still work with Unladen Swallow? >>> >>> Sort of. They disable the use of JITed code, so they don't quite work >>> the way you would want them to. Checking tstate->c_tracefunc every >>> line generated too much code. They still give you a rough idea of >>> where your application hotspots are, though, which I think is >>> acceptable. >> >> Hmm. So cProfile doesn't break, but it causes code to run under a >> completely different execution model so the numbers it produces are >> not connected to reality? >> >> We've found the call graph and associated execution time information >> from cProfile to be extremely useful for understanding performance >> issues and tracking down regressions. Giving that up would be a huge >> blow. > > FWIW, cProfile's call graph information is still perfectly accurate, > but you're right: turning on cProfile does trigger execution under a > different codepath. That's regrettable, but instrumentation-based > profiling is always going to introduce skew into your numbers. That's > why we opted to improve oProfile, since we believe sampling-based > profiling to be a better model. Sampling-based may be theoretically better, but we've gotten a lot of mileage out of profile, hotshot and especially cProfile. I know that other people at Google have also used cProfile (backported to 2.4) with great success. The couple of times I tried to use oProfile it was less illuminating than I'd hoped, but that could just be inexperience. > Profiling was problematic to support in machine code because in > Python, you can turn profiling on from user code at arbitrary points. > To correctly support that, we would need to add lots of hooks to the > generated code to check whether profiling is enabled, and if so, call > out to the profiler. Those "is profiling enabled now?" checks are > (almost) always going to be false, which means we spend cycles for no > real benefit. Well, we put the ability to profile on demand to good use - in particular by restricting profiling to one particular servlet (or a subset of servlets) and by skipping the first few executions of that servlet in a process to avoid startup noise. All of this gets kicked off by talking to the management process of our app server via http. > Can YouTube use oProfile for profiling, or is instrumented profiling > critical? [snip] I don't know that instrumented profiling is critical, but the level of insight we have now is very important for keeping the our site happy. It seems like it'd be a fair bit of work to get oProfile to give us the same level of insight, and it's not clear who would be motivated to do that work. > - Add the necessary profiling hooks to JITted code to better support > cProfile, but add a command-line flag (something explicit like -O3) > that removes the hooks and activates the current behaviour (or > something even more restrictive, possibly). This would be workable albeit suboptimal; as I said we start and stop profiling on the fly, and while we currently fork a new process to do this, that's only because we don't have a good arbitrary RPC mechanism from parent to child. Having to start up a new python process from scratch would be a big step back. > - Initially compile Python code without the hooks, but have a > trip-wire set to detect the installation of profiling hooks. When > profiling hooks are installed, purge all machine code from the system > and recompile all hot functions to include the profiling hooks. This would be the closest to the way we are doing things now. If Unladen Swallow is sufficiently faster, we would probably make oProfile work. But if it's a marginal improvement, we'd be more inclined to try for more incremental improvements (e.g. your excellent cPickle work). -jake ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed downstream change to site.py in Fedora (sys.defaultencoding)
On 1/23/2010 2:53 AM, "Martin v. Löwis" wrote: So for the limited case of text IO, Python 3.x now makes a guess. However, this guess is not in the face of ambiguity: it is the locale that the user (or his administrator) has selected, That is a mistaken assumption for many. I have never, that I know of, selected a locale on any of about 10 dos and windows machnes I have used and administered. It is whatever Microsoft or the computer manufacturer selected for machines sold in the U.S. Maybe things are different in Europe. > which identifies the language that they speak and the character encoding they use for text. I have in the past changed the encoding for outgoing email from whatever limited extended-ascii encoding MS or whoever set, to utf-8, but I have never, I am really sure, selected a default text file encoding. I have no idea if I can even change it, or even what it is. If the current guess is based on a mistaken assumption -- that it is giving the user what the user asked for -- it might be reconsidered. I personally would prefer that the default file encoding for Python 3 be utf-8 on any machine my code runs on unless *I* or the *user*, rather than some anonymous third party, ask otherwise. That would make files guaranteed portable unless asked to not be. It would seem better to me than some unknown value set by some unknown person. This seems counter to Python's general policy of operating consistently across machines and operating systems. > So if Python also uses that encoding, it's not really an ambiguous guess. It is unknown to me as the user. How do I find out what the default text file encoding is on my machine, in case it is not utf-8? Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed downstream change to site.py in Fedora (sys.defaultencoding)
>> So for the limited case of text IO, Python 3.x now makes a guess. >> However, this guess is not in the face of ambiguity: it is the >> locale that the user (or his administrator) has selected, > > That is a mistaken assumption for many. I have never, that I know of, > selected a locale on any of about 10 dos and windows machnes I have used > and administered. It is whatever Microsoft or the computer manufacturer > selected for machines sold in the U.S. Maybe things are different in > Europe. You have, most definitely, made a selection when choosing a system that came with "English" Windows preinstalled. If you hadn't wanted to use "English" Windows, you would either have asked for a different preinstallation, or installed some other language version. That you are unaware of actually having made that decision is likely because you live in an environment where there is a single predominant language, so vendors actually know the preference of their customers very well. Indeed, at my work place, the administrators always ask "German or English windows?". They would ask the same question when buying keyboards, except that people so often respond so often asking for a German keyboard that they stopped asking, making the German keyboard the default - even when used with an English windows installation. > I have in the past changed the encoding for outgoing email from whatever > limited extended-ascii encoding MS or whoever set, to utf-8, but I have > never, I am really sure, selected a default text file encoding. I have > no idea if I can even change it, or even what it is. That's even more reason to have Python programs use the choice somebody else made for you. If you open a text file with a text editor, the text editor will also make a choice of text encoding. It would be good if Python made the same choice - so that you can actually read the text that a Python script put into the file. If you, as an end user, would have to make a selection, you wouldn't be able to make a good choice, as you don't even know what encoding is typically used on your system. > It is unknown to me as the user. How do I find out what the default text > file encoding is on my machine, in case it is not utf-8? That depends on the operating system. For U.S./Western Windows, it's code page 1252. Microsoft doesn't support changing it to UTF-8 (although they do support changing it to other values). To verify that you are still using the vendor default, check out the regional settings in control panel. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed downstream change to site.py in Fedora (sys.defaultencoding)
Terry Reedy udel.edu> writes: > > If the current guess is based on a mistaken assumption -- that it is > giving the user what the user asked for -- it might be reconsidered. I > personally would prefer that the default file encoding for Python 3 be > utf-8 on any machine my code runs on unless *I* or the *user*, rather > than some anonymous third party, ask otherwise. That would make files > guaranteed portable unless asked to not be. Why do you think utf-8 is "guaranteed portable" and other encodings are not? All encodings are portable, the issue at hand is to select the one which will be expected by other tools. If this was only about Python then, yes, we could happily settle on utf-8 as a default for all systems. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Automatic sys.prefix change
Hi list, It looks like the python interpreter environment is easily movable between two directories because the sys.prefix variable is changed automatically when I run the interpreter so the interpreter will find his libraries... however in the Python documentation I found that (http://docs.python.org/library/sys.html?highlight=sys#sys.prefix) the sys.prefix is defined at the compile time with ./configure --prefix=dir command... Can I use this in production without any problem of the paths? Please consider the following test, after I use ./configure --prefix=/opt/python2.6.4 && make && make install: [r...@centos-5-4-test1 ~]# /opt/python2.6.4/bin/python Python 2.6.4 (r264:75706, Jan 20 2010, 20:26:15) [GCC 4.1.2 20080704 (Red Hat 4.1.2-46)] on linux2 Type "help", "copyright", "credits" or "license" for more information. ...import sys ...sys.prefix '/opt/python2.6.4' ...sys.path ['', '/opt/python2.6.4/lib/python26.zip', '/opt/python2.6.4/lib/python2.6', '/opt/python2.6.4/lib/python2.6/plat-linux2', '/opt/python2.6.4/lib/python2.6/lib-tk', '/opt/python2.6.4/lib/python2.6/lib-old', '/opt/python2.6.4/lib/python2.6/lib-dynload', '/opt/python2.6.4/lib/python2.6/site-packages'] [r...@centos-5-4-test1 ~]# mv /opt/python2.6.4/ /opt/python2.6.4_newplace [r...@centos-5-4-test1 ~]# /opt/python2.6.4_newplace/bin/python Python 2.6.4 (r264:75706, Jan 20 2010, 20:26:15) [GCC 4.1.2 20080704 (Red Hat 4.1.2-46)] on linux2 Type "help", "copyright", "credits" or "license" for more information. ...import sys ...sys.prefix '/opt/python2.6.4_newplace' ...sys.path ['', '/opt/python2.6.4_newplace/lib/python26.zip', '/opt/python2.6.4_newplace/lib/python2.6', '/opt/python2.6.4_newplace/lib/python2.6/plat-linux2', '/opt/python2.6.4_newplace/lib/python2.6/lib-tk', '/opt/python2.6.4_newplace/lib/python2.6/lib-old', '/opt/python2.6.4_newplace/lib/python2.6/lib-dynload', '/opt/python2.6.4_newplace/lib/python2.6/site-packages'] Thanks for the help, Balazs ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed downstream change to site.py in Fedora (sys.defaultencoding)
"Martin v. Löwis" wrote:
>> This all begs the question: why is there a default? and why is the
>> default a guess?
>>
>> I have to admit that I was completely oblivious to this potential
>> pitfall, and mostly that's because in the most common case, I am working
>> with ASCII files.
>
> You answered your own question: it is this reason why there is a
> default for the IO encoding.
>
>> It's just serendipity that most systems specify (if
>> not require) the locale encoding be an ASCII superset.
>
> No, it's not. It is deliberate that the locale's encoding is
> an ASCII superset. On systems where it isn't, users are typically
> well aware that they are not using ASCII. On the systems where
> it is, users get completely oblivious of the entire issue.
>
>> I already know that this suggestion will not get any following because,
>> for most people, it just works. However: "In the face of ambiguity,
>> refuse the temptation to guess." Would it really be that unfortunate to
>> force everyone to reconsider what they are doing when they open() files?
>
> Yes, definitely. It is this very reasoning that caused Python 2.x to
> use ASCII as the default encoding (when mixing strings and unicode),
> and, for the entire lifetime of 2.x, has caused endless pain for
> developers, which simply fail to understand the notion of encodings
> in the first place. The majority of developers is unable to get it
> right, in particular if their native language is English. These
> developers just hate Unicode. They google for solutions, and come
> up with all kinds of proposals which are all wrong (such as reloading
> the sys module to get back sys.setdefaultencoding, to then set it
> to UTF-8).
>
> So for the limited case of text IO, Python 3.x now makes a guess.
> However, this guess is not in the face of ambiguity: it is the
> locale that the user (or his administrator) has selected, which
> identifies the language that they speak and the character encoding
> they use for text. So if Python also uses that encoding, it's not
> really an ambiguous guess.
No, but it's most likely a wrong guess, since text files don't
really have anything to do with what the user wants to see in
a user interface.
It may be a good guess to stdin/out/err, since these provide part
of the user interface (if they are connected to a TTY), but not
for arbitrary text files which are normally meant for data exchange.
With the current guessing, you will get different encodings for
the text files depending on the locale setting of the user running
the application, so if you create a file on a Linux machine
you'll write a UTF-8 file, then try to open it on Windows and process
garbage, since the Windows Python installation will assume CP1252:
>>> print u'äöü'.encode('utf-8').decode('cp1252')
äöü
If you're lucky, you'll notice, if not, you'll work with corrupted
data.
I think that's what the Zen sentence is all about: "In the face of
ambiguity, refuse the temptation to guess." - it saves you from
situations that are difficult to detect and recover from.
Esp. when processing data, it's usually better to fail and provide
an opportunity to fix the data, rather than proceeding based
on some guessed assumption.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Jan 23 2010)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free !
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed downstream change to site.py in Fedora (sys.defaultencoding)
> No, but it's most likely a wrong guess, since text files don't > really have anything to do with what the user wants to see in > a user interface. That also depends on the operating system. On Windows, there is a long tradition of encoding all text in the system code page. All text editors on Windows know that. On Unix, text editors also assume that the text is in the locale's encoding unless directed otherwise. > It may be a good guess to stdin/out/err, since these provide part > of the user interface (if they are connected to a TTY), but not > for arbitrary text files which are normally meant for data exchange. Hmm - what do you mean by "normally"? Normally, text files are meant for human readers, not for exchange between programs. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed downstream change to site.py in Fedora (sys.defaultencoding)
"Martin v. Löwis" wrote: >> No, but it's most likely a wrong guess, since text files don't >> really have anything to do with what the user wants to see in >> a user interface. > > That also depends on the operating system. On Windows, there is > a long tradition of encoding all text in the system code page. > All text editors on Windows know that. On Unix, text editors also > assume that the text is in the locale's encoding unless directed > otherwise. > >> It may be a good guess to stdin/out/err, since these provide part >> of the user interface (if they are connected to a TTY), but not >> for arbitrary text files which are normally meant for data exchange. > > Hmm - what do you mean by "normally"? Normally, text files are meant > for human readers, not for exchange between programs. It's rather common to exchange text files between users... and in form of XML and CSV files, these also tend to get used as input for programs every now and then ;-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 23 2010) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed downstream change to site.py in Fedora (sys.defaultencoding)
M.-A. Lemburg wrote: > "Martin v. Löwis" wrote: >> Hmm - what do you mean by "normally"? Normally, text files are meant >> for human readers, not for exchange between programs. > > It's rather common to exchange text files between users... and > in form of XML and CSV files, these also tend to get used as > input for programs every now and then ;-) >From a non-Unicode-maven point of view, this seems like a case where practicality beats purity. The pure solution would be to refuse to guess and force application developers to deal with this (in full knowledge that they usually won't, since too many applications are still written by English speakers that assume everyone uses ASCII or latin-1). The practical solution is to guess an encoding that should work for files that are restricted to a single machine, or a network of machines configured to all use the same default text file encoding. At least the latter approach only runs into trouble when there is a genuine encoding mismatch problem. A Python that refused to guess a plausible default encoding for text files would cause problems *all* the time. Use the encoding recommended by the locale eliminates a lot of false alarms at the potential cost of making the true faults more difficult to analyse when they do arise. That sounds like a reasonable trade-off from where I'm standing. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed downstream change to site.py in Fedora (sys.defaultencoding)
M.-A. Lemburg egenix.com> writes: > > It's rather common to exchange text files between users... and > in form of XML and CSV files, these also tend to get used as > input for programs every now and then For XML files, encoding should be taken care of by the XML layer, not the IO layer. That is, they will be read and written as binary files. For CSV files, the situation is in my experience hopeless. Even on an UTF-8 system some software will interpret them as latin-1 by default. The "solution" I've come with is to first decode them as UTF-8 and fall back on latin-1 if it fails. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed downstream change to site.py in Fedora (sys.defaultencoding)
Antoine Pitrou wrote: > M.-A. Lemburg egenix.com> writes: >> >> It's rather common to exchange text files between users... and >> in form of XML and CSV files, these also tend to get used as >> input for programs every now and then > > For XML files, encoding should be taken care of by the XML layer, not the IO > layer. That is, they will be read and written as binary files. > > For CSV files, the situation is in my experience hopeless. Even on an UTF-8 > system some software will interpret them as latin-1 by default. The "solution" > I've come with is to first decode them as UTF-8 and fall back on latin-1 if it > fails. ... and then there are the UTF-16-LE CSV files that Excel exports. In any case, the encodings of these files don't have anything to do with the user's locale setting and that's why guessing based on this setting is less than ideal. Now, we cannot easily remove this guessing since we're in stable mode again with 3.1. Perhaps we should add a way to at least be able to switch off this guessing, so that applications can be tested in a predictable way, rather than depending on the test runner's locale settings ?! -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 23 2010) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed downstream change to site.py in Fedora (sys.defaultencoding)
Nick Coghlan wrote: > M.-A. Lemburg wrote: >> "Martin v. Löwis" wrote: >>> Hmm - what do you mean by "normally"? Normally, text files are meant >>> for human readers, not for exchange between programs. >> >> It's rather common to exchange text files between users... and >> in form of XML and CSV files, these also tend to get used as >> input for programs every now and then ;-) > >>From a non-Unicode-maven point of view, this seems like a case where > practicality beats purity. The pure solution would be to refuse to guess > and force application developers to deal with this (in full knowledge > that they usually won't, since too many applications are still written > by English speakers that assume everyone uses ASCII or latin-1). The > practical solution is to guess an encoding that should work for files > that are restricted to a single machine, or a network of machines > configured to all use the same default text file encoding. > > At least the latter approach only runs into trouble when there is a > genuine encoding mismatch problem. A Python that refused to guess a > plausible default encoding for text files would cause problems *all* the > time. Use the encoding recommended by the locale eliminates a lot of > false alarms at the potential cost of making the true faults more > difficult to analyse when they do arise. That sounds like a reasonable > trade-off from where I'm standing. Simply ignoring the problems is not going to educate developers and since the world is moving towards Unicode and Python 3.x uses Unicode throughout, punting on such problems is a poor way to move forward, at least IMHO. What's worse: it makes running Python applications unpredictable, since it's likely that some user is going to run the application using a different locale setting than you've used in your test cases. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 23 2010) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython
Hi Collin IMO it'll be better to make Unladen Swallow project a module, to be installed and used if needed, so demanding to users the choice of having it or not. The same way psyco does, indeed. Nowadays it requires too much memory, longer loading time, and fat binaries for not-so-great performances. I know that some issues have being worked on, but I don't think that they'll show something comparable to the current CPython status. Introducing C++ is a big step, also. Aside the problems it can bring on some platforms, it means that C++ can now be used by CPython developers. It doesn't make sense to force people use C for everything but the JIT part. In the end, CPython could become a mix of C and C++ code, so a bit more difficult to understand and manage. What I see is that LLVM is a too big project for the goal of having "just" a JIT-ed Python VM. It can be surely easier to use and integrate into CPython, but requires too much resources (on the contrary, Psyco demands little resources, give very good performances, but seems to be like a mess to manage and extend). I know that a new, custom JIT code is an hard project to work on, requiring long time, but the harry to have something faster to the current CPython can bring to a mammoth that runs just a bit bitter. Anyway, it seems that performance is a sensible argument for the Python community. I think that a lot can be made to squeeze out more speed, working both on CPython internals and on the JIT side. Best regards, Cesare 2010/1/20 Collin Winter > > Hello python-dev, > > I've just committed the initial draft of PEP 3146, proposing to merge > Unladen Swallow into CPython's source tree and roadmap. The initial > draft is included below. I've also uploaded the PEP to Rietveld at > http://codereview.appspot.com/186247, where individual fine-grained > updates will be tracked. Feel free to comment either in this thread or > on the Rietveld issue. I'll post periodic summaries of the > discussion-to-date. > > We're looking forward to discussing this with everyone. > > Thanks, > Collin Winter > [snip...] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed downstream change to site.py in Fedora (sys.defaultencoding)
On 1/23/2010 7:53 AM, Antoine Pitrou wrote: Terry Reedy udel.edu> writes: If the current guess is based on a mistaken assumption -- that it is giving the user what the user asked for -- it might be reconsidered. I personally would prefer that the default file encoding for Python 3 be utf-8 on any machine my code runs on unless *I* or the *user*, rather than some anonymous third party, ask otherwise. That would make files guaranteed portable unless asked to not be. Why do you think utf-8 is "guaranteed portable" and other encodings are not? All encodings are portable, Dreadfully rong. In general, the default encoding on my Windows *does not work* on Python3 strings but causes UnicodeEncodeError: If the text is not written to the file, it is completely non-portable. > the issue at hand is to select the one which will be expected by other tools. If the text is not written to a file, it does not matter what the other tool expects. Anyway, any tool that does not accept a complete unicode encoding is not really compatible with Python3 strings. If this was only about Python then, yes, we could happily settle on utf-8 as a default for all systems. OpenOffice on Windows prompts me to select an encoding to use for decoding. utf-8 is one of the choices, so it can read any Python3 string written out that way. I disagree that the default behavior for a basic function should be to not work. Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed downstream change to site.py in Fedora (sys.defaultencoding)
Terry Reedy udel.edu> writes: > > Dreadfully rong. In general, the default encoding on my Windows *does > not work* on Python3 strings but causes > UnicodeEncodeError: > If the text is not written to the file, it is completely non-portable. You must mistake "portable" for something else. "Portable" doesn't mean it is able to represent all possible unicode texts. It just means that it can be carried from one computer to the other. > If the text is not written to a file, it does not matter what the other > tool expects. We are talking about file I/O, so *yes* the text is written to a file. > Anyway, any tool that does not accept a complete unicode > encoding is not really compatible with Python3 strings. Well, we are talking about the *default* encoding choices here. Whether the tools can be configured to any other encoding is rather irrelevant. > I disagree that the default behavior for a basic function should be to > not work. Fine, but "not working" must be defined and it's quite clear that we don't really agree about what it means. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed downstream change to site.py in Fedora (sys.defaultencoding)
Le samedi 23 janvier 2010 à 20:43 +0100, M.-A. Lemburg a écrit : > > Now, we cannot easily remove this guessing since we're in stable > mode again with 3.1. Perhaps we should add a way to at least be > able to switch off this guessing, so that applications can be > tested in a predictable way, rather than depending on the test > runner's locale settings ?! The simple way to switch off the guessing is to specify an encoding in open(). I don't know what other means of switching it off could be added. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
