Re: [Python-Dev] PyUnicodeObject / PyASCIIObject questions
Le mardi 13 décembre 2011 02:09:02 Jim Jewett a écrit : > (3) I would feel much less nervous if the remaining 4 values of > PyUnicode_Kind were explicitly reserved, and the macros raised an > error when they showed up. (Better still would be to allow other > values, and to have the macros delegate to some attribute on the (sub) > type object.) A macro is not supposed to raise an error. In debug mode, _PyUnicode_CheckConsistency() ensures that the kind is valid and PyUnicode_KIND() fails with an assertion error if kind is PyUnicode_WCHAR_KIND. Python cannot create a string with a kind different than PyUnicode_1BYTE_KIND, PyUnicode_2BYTE_KIND or PyUnicode_4BYTE_KIND (the legacy API creates strings with a temporary PyUnicode_WCHAR_KIND kind, kind quickly replaces by PyUnicode_READY). If you write your own extension generating an invalid string, I don't think that Python can help you. Python cannot check all data, it would be too slow. If we change something, I would suggest to remove PyUnicode_WCHAR_KIND from the PyUnicode_Kind, so you can be sure that PyUnicode_KIND() result is an enum with 3 possible values (PyUnicode_1BYTE_KIND, PyUnicode_2BYTE_KIND or PyUnicode_4BYTE_KIND). It would help to make quiet the compiler on switch/case ;-) Victor ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
> > In effect, 2to3 is a "purity" solution, but > > six is more like a "practicality" solution. > > This sounds like your personal interpretation. I see nothing "pure" in > 2to3. > > > It's "pure" in being optimized for a world where you just stop using > Python 2 one day, and start using 3 the next, without any crossover support. That's not true. 2to3 is well suited for supporting both 2 and 3 from the same code base, and reduces the number of compromises you have to make compared to an identical-source approach (more dramatically so if you also want to support 2.5 or 2.4). > Anyway, if you're supporting both 2 and 3, a common code base offers > many attractions, so if it can be done, it will. And 2to3 is a good approach to maintaining a common code base. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyUnicodeObject / PyASCIIObject questions
On Tue, 13 Dec 2011 23:51:00 -0500 Terry Reedy wrote: > So by analogy, reserved type value would be ignored, neither corrupting > data or raising errors, until put in use. That simply doesn't make sense. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/14/2011 04:15 AM, "Martin v. Löwis" wrote: >> It's "pure" in being optimized for a world where you just stop >> using Python 2 one day, and start using 3 the next, without any >> crossover support. > > That's not true. 2to3 is well suited for supporting both 2 and 3 from > the same code base, and reduces the number of compromises you have to > make compared to an identical-source approach (more dramatically so if > you also want to support 2.5 or 2.4). > >> Anyway, if you're supporting both 2 and 3, a common code base >> offers many attractions, so if it can be done, it will. > > And 2to3 is a good approach to maintaining a common code base. Not in the experience of the folks who are actually doing that task: the overhead of running 2to3 every time 'setup.py develop' etc. runs dooms the effort. For instance, we have a report that the 2to3 step takes more than half an hour (on at least one user's development machine) when installing / refreshing zope.interface in a Python 3.2 virtualenv. (Note that I'm in the process of getting that package's unit test coverage up to snuff before ripping out the 2to3 support in favor of a subset). Using 2to3 during ongoing development makes Python feel like Java/C++, where "get a cup of coffee while we rebuild the world" is a frequent occurence. Tres. - -- === Tres Seaver +1 540-429-0999 [email protected] Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk7oz9wACgkQ+gerLs4ltQ7i4wCgh+9GliqukApx1skTs/0AnjKU CUMAoLzzkctR0gcSBR3qBxZmsAg1kvvt =FVtj -END PGP SIGNATURE- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
>> And 2to3 is a good approach to maintaining a common code base. > > > Not in the experience of the folks who are actually doing that task: Well, I personally actually *did* the task, so that experience certainly isn't universal. > the > overhead of running 2to3 every time 'setup.py develop' etc. runs dooms > the effort. How so? Running 2to3 after every change is very fast. I never use setup.py develop, though. > Using 2to3 during ongoing development makes Python feel like Java/C++, > where "get a cup of coffee while we rebuild the world" is a frequent > occurence. Unfortunately, these issues never get reported. I worked on porting zope.interface, and it never took 30 minutes for me, not even remotely. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
"Martin v. Löwis", 14.12.2011 18:23: overhead of running 2to3 every time 'setup.py develop' etc. runs dooms the effort. How so? Running 2to3 after every change is very fast. I never use setup.py develop, though. I think the problem starts with the fact that it needs to be run in the first place. It's not enough any more to just fire up the interpreter and run a test, you first have to build your code before you can get back to work, and it gets moved away into a separate directory and runs from there. So your workspace looks different depending on the environment you are currently testing with, and all your development tools have to support that as well. Even if the build step does not take half an hour, it's an otherwise unnecessary step that makes working and testing with Python 3 substantially less comfortable, and thus less likely to happen. And we all know where a reluctance against testing leads us. And, just for the record, we use 2to3 for Cython's code base, and I'm not convinced that this was a good decision. Testing the code in Py3 is actually something that I avoid if not strictly necessary, and that I leave to our CI server in most cases. I'm much more happy with lxml which was ported before there even was a 2to3, so it works on 2 and 3 out of the box. That alone makes it much nicer to develop on, and I think that it was clearly worth the additional porting work at the time. Stefan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing the XML batteries
Am 12.12.2011 10:04, schrieb Stefan Behnel: > "Martin v. Löwis", 11.12.2011 23:39: >>> I can't recall anyone working on any substantial improvements during the >>> last six years or so, and the reason for that seems obvious to me. >> >> What do you think is the reason? It's not at all obvious to me. > > Just to repeat myself for the third time here: lack of interest. Ah, that's certainly wrong. I am interested in these libraries. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing the XML batteries
> Just look through the xml-sig page, basically all requests regarding > PyXML during the last five years deal with problems in installing it, > i.e. *before* even starting to use it. So you can't use this to claim > that people really *are* still using it. I'm not so sure. In many of these cases, it turned out that they were trying to run some software that uses PyXML, and that they tried installing PyXML to satisfy the prerequisite. So while they may not be software developers, they are indirectly "users" of PyXML. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Compiling the source without stat
Hi. I just started to port latest python 2.7.2 to another platform (don't think you're eager to know... well it's CASIO ClassPad). And I faced a "minor" problem... It hasn't got stat or fstat or anything. (It supports a very limited set of c std lib). As pyport.c suggested, i defined both DONT_HAVE_STAT and DONT_HAVE_FSTAT, but problems only began. It failed to compile most of import.c, particularly because it fell into the wrong `#if !defined(PYOS_Something)' blocks. Sometimes it just fell into an #else part which assumed stat are available. So although HAVE_STAT is meant to control file operations, most of the source code aren't implement to use it. You see how "minor" the problem was? So now I need advice from developers. Is there a fix for it? What a question... definitely no replacement to stat. It's 99% definite that I can't compile further without touching the source code. I have to #define my own PYOS_whatever and handle files in my own way. In that case where should my special file handling cases go? I saw some marshal.c code which seemed it wants to abstract away platform's file handling from source code; but from what I understood it can't be made to use alternate file handling methods. If there is anything I should do (maybe show you my handmade pyconfig.h?) tell me. [My first post in a mailing list... Should I say] Best Regards, Hossein [in here?] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Compiling the source without stat
Hossein wrote: > Hi. I just started to port latest python 2.7.2 to another platform > (don't think you're eager to know... well it's CASIO ClassPad). > And I faced a "minor" problem... It hasn't got stat or fstat or > anything. (It supports a very limited set of c std lib). > As pyport.c suggested, i defined both DONT_HAVE_STAT and > DONT_HAVE_FSTAT, but problems only began. > It failed to compile most of import.c, particularly because it fell > into the wrong `#if !defined(PYOS_Something)' blocks. Sometimes it > just fell into an #else part which assumed stat are available. So > although HAVE_STAT is meant to control file operations, most of the > source code aren't implement to use it. You see how "minor" the > problem was? > So now I need advice from developers. > Is there a fix for it? What a question... definitely no replacement to stat. > It's 99% definite that I can't compile further without touching the > source code. I have to #define my own PYOS_whatever and handle files > in my own way. In that case where should my special file handling > cases go? I saw some marshal.c code which seemed it wants to > abstract away platform's file handling from source code; but from > what I understood it can't be made to use alternate file handling > methods. > If there is anything I should do (maybe show you my handmade > pyconfig.h?) tell me. See http://bugs.python.org/issue12082. Currently neither Python 2.x nor 3.x can be compiled without stat() or fstat(). Python 2.7 almost compiles, but Python 3 depends heavily on them. The problem boils down to the fact that you cannot really check whether a filesystem entry is a directory without calling stat() or fstat(). My personal opinion is that support for DONT_HAVE_STAT and DONT_HAVE_FSTAT defines should be dropped because they don't work, and would only be useful in a very limited set of cases. > [My first post in a mailing list... Should I say] Best Regards, n> Hossein [in here?] Yeah, why not? :) Regards, Petri ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing the XML batteries
"Martin v. Löwis", 14.12.2011 19:14: Am 12.12.2011 10:04, schrieb Stefan Behnel: "Martin v. Löwis", 11.12.2011 23:39: I can't recall anyone working on any substantial improvements during the last six years or so, and the reason for that seems obvious to me. What do you think is the reason? It's not at all obvious to me. Just to repeat myself for the third time here: lack of interest. Ah, that's certainly wrong. I am interested in these libraries. I meant: "lack of interest in improving them". It's clear from the discussion that there are still users and that new code is still being written that uses MiniDOM. However, I would argue that this cannot possibly be performance critical code and that it only deals with somewhat small documents. I say that because MiniDOM is evidently not suitable for large documents or performance critical applications, so this is the only explanation I have why the performance problems would not be obvious in the cases where it is still being used. And if they do show, it appears to be much more likely that users rewrite their code using ElementTree or lxml than that they try to fix MiniDOM's performance issues. Now, read my first quote above again (and preferably also its context, which I already emphasized in a previous post), it should be clearer now. Stefan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Compiling the source without stat
> It's 99% definite that I can't compile further without touching the > source code. I have to #define my own PYOS_whatever and handle files in > my own way. In that case where should my special file handling cases go? It's difficult to say how to proceed. On one hand, I don't see an overwhelming need to support systems without stat, and am tempted to say that you are on your own. On the other hand, it appears that people keep asking for it, from time to time. So if it was possible to support such systems without making the code too convoluted, it may be worth supporting it. One thing seems clear: without stat(), we cannot possibly support .pyc files, at least not in __pycache__. So one consequence of a lacking stat should be that all the code dealing with caching of byte code files needs to be disabled. Supporting .pyc as modules might still be possible. It's questionable how to deal with path searching in the absence of stat. Testing for the presence of a file is possible in principle by trying to open the file, and closing it when it was found to be present. So in the places where we only check for the presence of a file, an alternative implementation could be provided. In any case, it needs someone to champion such a project, preferably in an ongoing manner (i.e. several years). So if you are interested, you should - volunteer to maintain stat-less systems for some time - create a port of Python 3 that works stat-less - come back to python-dev for review to determine whether it's worth to support such systems. Alternatively, you can just make your own fork of Python, which you may or may not publish. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing the XML batteries
On 2011-12-14, at 20:41 , Stefan Behnel wrote: > I meant: "lack of interest in improving them". It's clear from the discussion > that there are still users and that new code is still being written that uses > MiniDOM. However, I would argue that this cannot possibly be performance > critical code and that it only deals with somewhat small documents. I say > that because MiniDOM is evidently not suitable for large documents or > performance critical applications, so this is the only explanation I have why > the performance problems would not be obvious in the cases where it is still > being used. And if they do show, it appears to be much more likely that users > rewrite their code using ElementTree or lxml than that they try to fix > MiniDOM's performance issues. Could also be because "XML is slow (and sucks)" is part of the global consciousness at this point, and that minidom is slow and verbose doesn't surprise much. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing the XML batteries
Xavier Morel, 14.12.2011 20:54: On 2011-12-14, at 20:41 , Stefan Behnel wrote: I meant: "lack of interest in improving them". It's clear from the discussion that there are still users and that new code is still being written that uses MiniDOM. However, I would argue that this cannot possibly be performance critical code and that it only deals with somewhat small documents. I say that because MiniDOM is evidently not suitable for large documents or performance critical applications, so this is the only explanation I have why the performance problems would not be obvious in the cases where it is still being used. And if they do show, it appears to be much more likely that users rewrite their code using ElementTree or lxml than that they try to fix MiniDOM's performance issues. Could also be because "XML is slow (and sucks)" is part of the global consciousness at this point, and that minidom is slow and verbose doesn't surprise much. Possibly, yes. Or that "Python is slow and sucks". But I think there are good counter arguments against both. Stefan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Compiling the source without stat
On 12/14/2011 2:26 PM, Petri Lehtinen wrote: The problem boils down to the fact that you cannot really check whether a filesystem entry is a directory without calling stat() or fstat(). My personal opinion is that support for DONT_HAVE_STAT and DONT_HAVE_FSTAT defines should be dropped because they don't work, and would only be useful in a very limited set of cases. At present, it seems to be an attractive nuisance, tempting people like Hossein to try something that does not work. -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing the XML batteries
Am 14.12.2011 20:41, schrieb Stefan Behnel: > "Martin v. Löwis", 14.12.2011 19:14: >> Am 12.12.2011 10:04, schrieb Stefan Behnel: >>> "Martin v. Löwis", 11.12.2011 23:39: > I can't recall anyone working on any substantial improvements > during the > last six years or so, and the reason for that seems obvious to me. What do you think is the reason? It's not at all obvious to me. >>> >>> Just to repeat myself for the third time here: lack of interest. >> >> Ah, that's certainly wrong. I am interested in these libraries. > > I meant: "lack of interest in improving them". That's also what I meant. I'm interested in improving them. > Now, read my first quote above again (and preferably also its context, > which I already emphasized in a previous post), it should be clearer now. I (now) know what you mean - but you are incorrect. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing the XML batteries
"Martin v. Löwis", 14.12.2011 22:20: Am 14.12.2011 20:41, schrieb Stefan Behnel: "Martin v. Löwis", 14.12.2011 19:14: Am 12.12.2011 10:04, schrieb Stefan Behnel: "Martin v. Löwis", 11.12.2011 23:39: I can't recall anyone working on any substantial improvements during the last six years or so, and the reason for that seems obvious to me. What do you think is the reason? It's not at all obvious to me. Just to repeat myself for the third time here: lack of interest. Ah, that's certainly wrong. I am interested in these libraries. I meant: "lack of interest in improving them". That's also what I meant. I'm interested in improving them. Then please do. I posted the numbers, so you know what the baseline is, both in terms of speed and memory usage. If you need further benchmarks of other areas of the API (e.g. tag search or whatever), just ask. Note, however, that even an improvement by an order of magnitude wouldn't solve the API issue for new users, so I'd still suggest to add an appropriate link towards ET to the MiniDOM documentation. Stefan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On Wed, Dec 14, 2011 at 17:33, Tres Seaver wrote: > Not in the experience of the folks who are actually doing that task: the > overhead of running 2to3 every time 'setup.py develop' etc. runs dooms > the effort. For instance, we have a report that the 2to3 step takes more > than half an hour (on at least one user's development machine) when > installing / refreshing zope.interface in a Python 3.2 virtualenv. If that is true, then there has to be a bug somewhere... I might not have tried on 3.2 with virtualenv, but it doesn't take anywhere near that time normally, and this is not a normal runtime at all. When we are talking about 2to3 being slow here we are talking about it taking 10 seconds to install a software that would have taken under a second to install on Python 2. (Yes, I'm thinking of Distribute, I just checked. ;-) ). //Lennart ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
