Re: [Python-Dev] cffi in stdlib
On Wed, Feb 27, 2013 at 3:39 AM, Terry Reedy wrote: > On 2/26/2013 10:13 AM, Maciej Fijalkowski wrote: > >> I would like to discuss on the language summit a potential inclusion >> of cffi[1] into stdlib. > > > How does it compare in terms of speed. One reason ctypes has not replaces > hand-tuned swig is that it apparently is much slower. I know that someone, > for instance, once did a pygame fork using ctypes to wrap SDL, and got it to > run, but gave it up for comparative slowness reasons. I never did the comparison with hand-tuned swig. It's faster than ctypes for calls and manipulation, C extension beat it to some degree. It's uber fast on PyPy (a bit over the native call, but we'll get it to the native call level). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On Wed, Feb 27, 2013 at 6:53 AM, Eli Bendersky wrote:
>
> * Work either at the level of the ABI (Application Binary Interface)
>>
>> or the API (Application Programming Interface). Usually, C libraries
>> have a specified C API but often not an ABI (e.g. they may document a
>> “struct” as having at least these fields, but maybe more). (ctypes
>> works at the ABI level, whereas Cython and native C extensions work at
>> the API level.)
>
>
>
> I read the cffi docs once again and went through some of the examples. I
> want to divide this to two topics.
>
> One is what you call the "ABI" level. IMHO, it's hands down superior to
> ctypes. Your readdir demo demonstrates this very nicely. I would definitely
> want to see this in the stdlib as an alternative way to interface to C
> shared objects & DLLs.
>
> Two is what you call the "API" level, which is where my opinion becomes
> mixed. Some things just don't feel right to me:
>
> 1. Tying in a C compiler into the flow of a program. I'm not sure whether we
> have precedents for it in the stdlib. Does this work on Windows where
> libraries and DLLs are usually built with MSVC?
>
Yes. Precedent in the stdlib is really the C API. All the same rules
apply (including build and ship a dll).
> 2. Using a function called "verify" to create stuff. This may sound like a
> naming bikeshed, but it's not. It ties in to the question - why is this
> needed?
We welcome a better opinion of name (indeed verify is not that great).
This elevates ABI to API so either invokes the C compiler or reads
stuff from the cache.
>
> 3. The partial type specifications in C with ellipsis. What is the point?
> You have the C declarations somewhere anyhow, so why introduce this? The
> "ABI level" boasts having just C and Python to write, but those partial
> ellipsis-ridden declarations are hardly C.
No, you don't. Some libraries contain macros for example (like
OpenSSL) where you just can't use ABI because it makes no sense. It's
less common on windows where binary compatibility is important,
however looking on linux, multiple stdlib declaration would use
ellipsis in the man page. I can't seem to find one right now, but it's
something like:
struct X {
int public_field;
...
}
which is impossible to do correctly with ctypes without exposing some
sort of platform dependency that might change without warning.
Another usages are #define SQLITE_OK ... which you don't know at the
time of writing (people assume those won't change and the do change).
>
> Please see this as constructive criticism, and I fully admit that I may not
> have went deep enough to see the big benefits of the "API" level that would
> justify all that complexity.
Sure. It might sound strange, but on linux (or posix in general)
binary compatibility is not a thing, so you need to do stuff to
account for that.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On Wed, Feb 27, 2013 at 9:29 AM, Ronald Oussoren wrote:
>
> On 26 Feb, 2013, at 16:13, Maciej Fijalkowski wrote:
>
>> Hello.
>>
>> I would like to discuss on the language summit a potential inclusion
>> of cffi[1] into stdlib.
>
> The API in general looks nice, but I do have some concens w.r.t. including
> cffi in the stdlib.
>
> 1. Why is cffi completely separate from ctypes, instead of layered on top of
> it? That is, add a utility module to ctypes that can parse C declarations and
> generate the right ctypes definitions.
Because ctypes API is a mess and magic. We needed a cleaner (and much
smaller) model.
>
> 2. Cffi has a dependencies on pycparser and that module and its dependencies
> would therefore also be added to the stdlib (even if they'd be hidden in the
> cffi package)
Yes. pycparser and ply.
>
> 3. Cffi basicly contains a (limited) C parser, and those are notoriously hard
> to get exactly right. Luckily cffi only needs to interpret declarations and
> not the full language, but even so this can be a risk of subtle bugs.
It seems to work.
>
> 4. And finally a technical concern: how well does cffi work with fat binaries
> on OSX? In particular, will the distutils support generate cached data for
> all architectures supported by a fat binary?
no idea.
>
> Also, after playing around with it for 5 minutes I don't quite understand how
> to use it. Let's say I want to wrap a function "CGPoint CGPointMake(CGFloat
> x, CGFloat y)". Is is possible to avoid mentioning the exact typedef for
> CGFloat somewhere? I tried using:
>
>ffi.cdef("typedef ... CGFloat; typedef struct { CGFloat x; CGFloat y; }
> CGPoint; CGPoint CGPointMake(CGFloat x, CGFloat y);")
>
> But that results in an error when calling verify:
>
>TypeError: field 'struct $CGPoint.x' has ctype 'struct $CGFloat' of
> unknown size
>
> From a first glance this doesn't seem to buy me that much w.r.t. ctypes, I
> still have to declare the actual type of CGFloat which is documented as "some
> floating point type".
>
> Ronald
typedef ... is assumed to be a struct. Copy pasting from
documentation, we might want to have a better support for that, but
what you do right now is to do:
ffi.cdef("const int mysize;")
lib = ffi.verify("const int mysize = sizeof(THE_TYPE);")
print lib.mysize
which will tell you the size (and then you can pick up which float you
want). I agree that some generic support for that would be cool.
Cheers,
fijal
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On 27 Feb, 2013, at 10:06, Maciej Fijalkowski wrote: > On Wed, Feb 27, 2013 at 9:29 AM, Ronald Oussoren > wrote: >> >> On 26 Feb, 2013, at 16:13, Maciej Fijalkowski wrote: >> >>> Hello. >>> >>> I would like to discuss on the language summit a potential inclusion >>> of cffi[1] into stdlib. >> >> The API in general looks nice, but I do have some concens w.r.t. including >> cffi in the stdlib. >> >> 1. Why is cffi completely separate from ctypes, instead of layered on top of >> it? That is, add a utility module to ctypes that can parse C declarations >> and generate the right ctypes definitions. > > Because ctypes API is a mess and magic. We needed a cleaner (and much > smaller) model. The major advantages of starting over is probably that you can hide the complexity and that opens opportunities for optimizations. That said, I'm not convinced that ctypes is unnecessarily complex. > >> >> 2. Cffi has a dependencies on pycparser and that module and its dependencies >> would therefore also be added to the stdlib (even if they'd be hidden in the >> cffi package) > > Yes. pycparser and ply. Which aren't part of the stdlib right now. > >> >> 3. Cffi basicly contains a (limited) C parser, and those are notoriously >> hard to get exactly right. Luckily cffi only needs to interpret declarations >> and not the full language, but even so this can be a risk of subtle bugs. > > It seems to work. That's not a confidency inspiring comment :-). That said, I use a hacked up fork of pycparser to parse Apple's Cocoa headers for PyObjC and it appears to work fine for that. > >> >> 4. And finally a technical concern: how well does cffi work with fat >> binaries on OSX? In particular, will the distutils support generate cached >> data for all architectures supported by a fat binary? > > no idea. That's somehting that will have to be resolved before cffi can be included in the stdlib, fat binaries are supported by CPython and are used the binary installers. Ronald ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On Wed, Feb 27, 2013 at 11:18 AM, Ronald Oussoren wrote: > > On 27 Feb, 2013, at 10:06, Maciej Fijalkowski wrote: > >> On Wed, Feb 27, 2013 at 9:29 AM, Ronald Oussoren >> wrote: >>> >>> On 26 Feb, 2013, at 16:13, Maciej Fijalkowski wrote: >>> Hello. I would like to discuss on the language summit a potential inclusion of cffi[1] into stdlib. >>> >>> The API in general looks nice, but I do have some concens w.r.t. including >>> cffi in the stdlib. >>> >>> 1. Why is cffi completely separate from ctypes, instead of layered on top >>> of it? That is, add a utility module to ctypes that can parse C >>> declarations and generate the right ctypes definitions. >> >> Because ctypes API is a mess and magic. We needed a cleaner (and much >> smaller) model. > > The major advantages of starting over is probably that you can hide the > complexity and that opens opportunities for optimizations. That said, I'm not > convinced that ctypes is unnecessarily complex. I implemented ctypes. It is. >>> 4. And finally a technical concern: how well does cffi work with fat >>> binaries on OSX? In particular, will the distutils support generate cached >>> data for all architectures supported by a fat binary? >> >> no idea. > > That's somehting that will have to be resolved before cffi can be included in > the stdlib, fat binaries are supported by CPython and are used the binary > installers. > > Ronald if cpython supports it and you can load it using dlopen, it does work then (it really is just building a C extension on the API level). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PLY in stdlib (was cffi in stdlib)
> > From: Eli Bendersky > > I'll be the first one to admit that pycparser is almost certainly not > generally useful enough to be exposed in the stdlib. So just using it as an > implementation detail is absolutely fine. PLY is a more interesting > question, however, since PLY is somewhat more generally useful. That said, > I see all this as implementation details that shouldn't distract us from > the main point of whether cffi should be added. > Regarding the inclusion of PLY or some subcomponent of it in the standard library, it's not an entirely crazy idea in my opinion. LALR(1) parsers have been around for a long time, are generally known to anyone who's used yacc/bison, and would be useful outside the context of cffi or pycparser. PLY has also been around for about 12 years and is what I would call stable. It gets an update about every year or two, but that's about it. PLY is also relatively small--just two files and about 4300 lines of code (much of which could probably be scaled down a bit). The only downside to including PLY might be the fact that there are very few people walking around who've actually had to *implement* an LALR(1) parser generator. Some of the code for that is extremely hairy and mathematical. At this time, I don't think there are any bugs in it, but it's not the sort of thing that one wants to wander into casually.Also, there are some horrible hacks in PLY that I'd really like to get rid of, but am currently stuck with due to backwards compatibility issues. Alex Gaynor has been working on a PLY variant (RPLY) geared at RPython and which has a slightly different programming interface.I'd say if we were to go down this route, he and I should work together to put together some kind of more general "parsing.lalr" package (or similar) that cleans it up and makes it more suitable as a library for building different kinds of parsing tools on top of. Cheers, Dave ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PLY in stdlib (was cffi in stdlib)
On 27 Feb 2013, at 11:00, David Beazley wrote: >> >> From: Eli Bendersky >> >> I'll be the first one to admit that pycparser is almost certainly not >> generally useful enough to be exposed in the stdlib. So just using it as an >> implementation detail is absolutely fine. PLY is a more interesting >> question, however, since PLY is somewhat more generally useful. That said, >> I see all this as implementation details that shouldn't distract us from >> the main point of whether cffi should be added. >> > > Regarding the inclusion of PLY or some subcomponent of it in the standard > library, it's not an entirely crazy idea in my opinion. +1 PLY is capable and well tried-and-tested. We used it in Resolver One to implement a pretty large grammar and it is (in my opinion) best of breed in the Python parser generator world. Being stable and widely used, with an "available maintainer", makes it an ideal candidate for standard library inclusion. Michael > LALR(1) parsers have been around for a long time, are generally known to > anyone who's used yacc/bison, and would be useful outside the context of cffi > or pycparser. PLY has also been around for about 12 years and is what I > would call stable. It gets an update about every year or two, but that's > about it. PLY is also relatively small--just two files and about 4300 lines > of code (much of which could probably be scaled down a bit). > > The only downside to including PLY might be the fact that there are very few > people walking around who've actually had to *implement* an LALR(1) parser > generator. Some of the code for that is extremely hairy and mathematical. > At this time, I don't think there are any bugs in it, but it's not the sort > of thing that one wants to wander into casually.Also, there are some > horrible hacks in PLY that I'd really like to get rid of, but am currently > stuck with due to backwards compatibility issues. > > Alex Gaynor has been working on a PLY variant (RPLY) geared at RPython and > which has a slightly different programming interface.I'd say if we were > to go down this route, he and I should work together to put together some > kind of more general "parsing.lalr" package (or similar) that cleans it up > and makes it more suitable as a library for building different kinds of > parsing tools on top of. > > Cheers, > Dave > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On 27/02/2013 9:21am, Maciej Fijalkowski wrote: >That's somehting that will have to be resolved before cffi can be included in the stdlib, fat binaries are supported by CPython and are used the binary installers. > >Ronald if cpython supports it and you can load it using dlopen, it does work then (it really is just building a C extension on the API level). On Windows ctypes has CDLL for libraries using the cdecl calling convention and WINDLL for libraries with the stdcall calling convention. I can't see any similar distinction in cffi's documentation. Can cffi magically work out which calling convention to use, or are you restricted to the cdecl calling convention? -- Richard ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On Wed, Feb 27, 2013 at 1:45 PM, Richard Oudkerk wrote: > On 27/02/2013 9:21am, Maciej Fijalkowski wrote: >>> >>> >That's somehting that will have to be resolved before cffi can be >>> > included in the stdlib, fat binaries are supported by CPython and are used >>> > the binary installers. >>> > >>> >Ronald >> >> if cpython supports it and you can load it using dlopen, it does work >> then (it really is just building a C extension on the API level). > > > On Windows ctypes has CDLL for libraries using the cdecl calling convention > and WINDLL for libraries with the stdcall calling convention. > > I can't see any similar distinction in cffi's documentation. Can cffi > magically work out which calling convention to use, or are you restricted to > the cdecl calling convention? copy pasting from docs: Windows: you can’t yet specify the calling convention of callbacks. (For regular calls, the correct calling convention should be automatically inferred by the C backend.) Use an indirection, like in the example just above. I think it means you can't use the ABI version and specify the calling convention. It's a reasonable bug report (the calling convention on API version works though) > > -- > Richard > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On Wed, Feb 27, 2013 at 7:06 PM, Maciej Fijalkowski wrote: >> 3. Cffi basicly contains a (limited) C parser, and those are notoriously >> hard to get exactly right. Luckily cffi only needs to interpret declarations >> and not the full language, but even so this can be a risk of subtle bugs. > > It seems to work. C itself shouldn't be *too* bad - it's trying to parse C++ that is truly insane (which is why Dave Beazley explicitly warned Maciej and Armin away from that based on his experience with SWIG). Looking at pycparsing more closely though, there may be a problem with the C preprocessing step on Windows. While on Linux it uses the system "cpp", for Windows it currently bundles a copy of the LCC preprocessor, which may pose licensing challenges (https://github.com/drh/lcc/blob/master/CPYRIGHT). However, this concern may not be applicable to cffi: given cffi's requirement for a full external C compiler to actually *create* the C extension modules during development, and the ability to retrieve everything necessary directly from the created C extension when no C compiler is available, it seems there would be no reason to ever need the bundled C preprocessor. So we simply don't ship it, and the LCC license becomes irrelevant. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On Wed, Feb 27, 2013 at 2:39 PM, Nick Coghlan wrote: > On Wed, Feb 27, 2013 at 7:06 PM, Maciej Fijalkowski wrote: >>> 3. Cffi basicly contains a (limited) C parser, and those are notoriously >>> hard to get exactly right. Luckily cffi only needs to interpret >>> declarations and not the full language, but even so this can be a risk of >>> subtle bugs. >> >> It seems to work. > > C itself shouldn't be *too* bad - it's trying to parse C++ that is > truly insane (which is why Dave Beazley explicitly warned Maciej and > Armin away from that based on his experience with SWIG). > > Looking at pycparsing more closely though, there may be a problem with > the C preprocessing step on Windows. While on Linux it uses the system > "cpp", for Windows it currently bundles a copy of the LCC > preprocessor, which may pose licensing challenges > (https://github.com/drh/lcc/blob/master/CPYRIGHT). However, this > concern may not be applicable to cffi: given cffi's requirement for a > full external C compiler to actually *create* the C extension modules > during development, and the ability to retrieve everything necessary > directly from the created C extension when no C compiler is available, > it seems there would be no reason to ever need the bundled C > preprocessor. So we simply don't ship it, and the LCC license becomes > irrelevant. > > Cheers, > Nick. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia cffi does not use preprocessor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On Wed, Feb 27, 2013 at 7:00 PM, Maciej Fijalkowski wrote: > On Wed, Feb 27, 2013 at 6:53 AM, Eli Bendersky wrote: >> 2. Using a function called "verify" to create stuff. This may sound like a >> naming bikeshed, but it's not. It ties in to the question - why is this >> needed? > > We welcome a better opinion of name (indeed verify is not that great). > This elevates ABI to API so either invokes the C compiler or reads > stuff from the cache. Have you considered the name "create_api"? After all, you're essentially creating a Python API given a C header declaration and a specified extension module to cache the result. The details on extension module caching, and the fact that it won't even look for a C compiler if the extension module is already available should also be in the main section of the docs on the verification step rather than only being down in the internal documentation for cffi.verifier.Verifier Cheers, Nick. P.S. I created both of those as tracker issues: - https://bitbucket.org/cffi/cffi/issue/59/missing-content-in-cffiverify - https://bitbucket.org/cffi/cffi/issue/60/proposed-rename-for-verify-and-verifier -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
Le Wed, 27 Feb 2013 12:15:05 +1300, Greg Ewing a écrit : > Antoine Pitrou wrote: > > Or we'll go straight to 5. > > (or switch to date-based numbering :-)) > > We could go the Apple route and start naming them after > species of snake. We have to find sufficiently silly species of snakes, though. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] peps: Pre-alpha draft for PEP 435 (enum). The name is not important at the moment, as
Le Wed, 27 Feb 2013 12:33:35 +0900, "Stephen J. Turnbull" a écrit : > > As far as I can see, what you (Antoine) want is an identifier with a > constant value, no more and no less. Grouping into an enum is merely > a lexical convention, since you are happy to compare enums of > different enum classes and enums with ints. > > I think such a thing is pretty generally desirable, but most of the > people who have created Enum classes implement some type-checking that > this value came from that Enum. To me that is what differentiates an > enumerated constant from a generic named constant. I don't see why > you keep deprecating that feature. Do you really think it's useless, > or is it just that you wouldn't use it yourself? I just wouldn't care for it myself. I don't think extra-strong typing of constants is really useful in practice; it smells a bit like private methods to me. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On 2013-02-27, at 14:31 , Antoine Pitrou wrote: > Le Wed, 27 Feb 2013 12:15:05 +1300, > Greg Ewing a écrit : >> Antoine Pitrou wrote: >>> Or we'll go straight to 5. >>> (or switch to date-based numbering :-)) >> >> We could go the Apple route and start naming them after >> species of snake. > > We have to find sufficiently silly species of snakes, though. With about 3000 extant snake species, that should be manageable. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On 27 February 2013 10:31, Antoine Pitrou wrote: > Le Wed, 27 Feb 2013 12:15:05 +1300, > Greg Ewing a écrit : > > Antoine Pitrou wrote: > > > Or we'll go straight to 5. > > > (or switch to date-based numbering :-)) > > > > We could go the Apple route and start naming them after > > species of snake. > > We have to find sufficiently silly species of snakes, though. > Monty Python feature movies. There are less than snake species, but imagine porting Django to be able to run on Life of Brian! > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/jsbueno%40python.org.br > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
Hi Guido, On Tue, Feb 26, 2013 at 8:24 PM, Guido van Rossum wrote: > From a software engineering perspective, 10 years is indistinguishable > from infinity, so I don't care what happens 10 years from now -- as > long as you don't blame me. :-) I can't resist: around today it is the 10th anniversary of PyPy. :-) A bientôt, Armin. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] peps: Pre-alpha draft for PEP 435 (enum). The name is not important at the moment, as
On Wed, Feb 27, 2013 at 8:30 AM, Antoine Pitrou wrote: > I don't think extra-strong typing of constants is really useful in > practice; it smells a bit like private methods to me. I think checking that a value comes from a particular enum *is* a degree of hand-holding. For libraries or frameworks whose users aren't expected to know them exhaustively, making reasonable checks of parameters can substantially reduce the number of ways it can be used incorrectly. Outside performance critical code, this is a win. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On Tue, Feb 26, 2013 at 11:29 PM, Ronald Oussoren wrote: > > On 26 Feb, 2013, at 16:13, Maciej Fijalkowski wrote: > > > Hello. > > > > I would like to discuss on the language summit a potential inclusion > > of cffi[1] into stdlib. > > The API in general looks nice, but I do have some concens w.r.t. including > cffi in the stdlib. > > 1. Why is cffi completely separate from ctypes, instead of layered on top > of it? That is, add a utility module to ctypes that can parse C > declarations and generate the right ctypes definitions. > > 2. Cffi has a dependencies on pycparser and that module and its > dependencies would therefore also be added to the stdlib (even if they'd be > hidden in the cffi package) > > 3. Cffi basicly contains a (limited) C parser, and those are notoriously > hard to get exactly right. Luckily cffi only needs to interpret > declarations and not the full language, but even so this can be a risk of > subtle bugs. > pycparser has been around for more than 4 years and is pretty stable. I know that it's been used to parse Windows headers, Linux system headers and other "scary" things for a C parser to handle. It's only known problem has to do with over-creative abuses of C's context sensitivity: http://eli.thegreenplace.net/2011/05/02/the-context-sensitivity-of-c%E2%80%99s-grammar-revisited/ In practice it doesn't come up often, and almost never in declarations, which is the subset cffi uses pycparser for. Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
> > I read the cffi docs once again and went through some of the examples. I
> > want to divide this to two topics.
> >
> > One is what you call the "ABI" level. IMHO, it's hands down superior to
> > ctypes. Your readdir demo demonstrates this very nicely. I would
> definitely
> > want to see this in the stdlib as an alternative way to interface to C
> > shared objects & DLLs.
> >
> > Two is what you call the "API" level, which is where my opinion becomes
> > mixed. Some things just don't feel right to me:
> >
> > 1. Tying in a C compiler into the flow of a program. I'm not sure
> whether we
> > have precedents for it in the stdlib. Does this work on Windows where
> > libraries and DLLs are usually built with MSVC?
> >
>
> Yes. Precedent in the stdlib is really the C API. All the same rules
> apply (including build and ship a dll).
>
So would you say that the main use of the API level is provide an
alternative for writing C API code to interface to C libraries. IOW, it's
in competition with Swig?
>
> > 2. Using a function called "verify" to create stuff. This may sound like
> a
> > naming bikeshed, but it's not. It ties in to the question - why is this
> > needed?
>
> We welcome a better opinion of name (indeed verify is not that great).
> This elevates ABI to API so either invokes the C compiler or reads
> stuff from the cache.
>
Can you elaborate on what "elevates ABI to API" means here?
> > 3. The partial type specifications in C with ellipsis. What is the point?
> > You have the C declarations somewhere anyhow, so why introduce this? The
> > "ABI level" boasts having just C and Python to write, but those partial
> > ellipsis-ridden declarations are hardly C.
>
> No, you don't. Some libraries contain macros for example (like
> OpenSSL) where you just can't use ABI because it makes no sense. It's
> less common on windows where binary compatibility is important,
> however looking on linux, multiple stdlib declaration would use
> ellipsis in the man page.
It would be useful to find an actual example and discuss it concretely.
> I can't seem to find one right now, but it's
> something like:
>
> struct X {
>int public_field;
>...
> }
>
> which is impossible to do correctly with ctypes without exposing some
> sort of platform dependency that might change without warning.
>
> Another usages are #define SQLITE_OK ... which you don't know at the
> time of writing (people assume those won't change and the do change).
>
Do you mean that the value of SQLITE_OK changed over time (now it's 0, but
used to be different?)
If so, then in a realistic use case, how would the API level help solve
this?
Eli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On Wed, Feb 27, 2013 at 5:57 PM, Eli Bendersky wrote:
>
>> > I read the cffi docs once again and went through some of the examples. I
>> > want to divide this to two topics.
>> >
>> > One is what you call the "ABI" level. IMHO, it's hands down superior to
>> > ctypes. Your readdir demo demonstrates this very nicely. I would
>> > definitely
>> > want to see this in the stdlib as an alternative way to interface to C
>> > shared objects & DLLs.
>> >
>> > Two is what you call the "API" level, which is where my opinion becomes
>> > mixed. Some things just don't feel right to me:
>> >
>> > 1. Tying in a C compiler into the flow of a program. I'm not sure
>> > whether we
>> > have precedents for it in the stdlib. Does this work on Windows where
>> > libraries and DLLs are usually built with MSVC?
>> >
>>
>> Yes. Precedent in the stdlib is really the C API. All the same rules
>> apply (including build and ship a dll).
>
>
> So would you say that the main use of the API level is provide an
> alternative for writing C API code to interface to C libraries. IOW, it's in
> competition with Swig?
the general goal is to provide alternative for writing C API by hand
(so SWIG, ctypes, cython, whatever).
>
>>
>>
>> > 2. Using a function called "verify" to create stuff. This may sound like
>> > a
>> > naming bikeshed, but it's not. It ties in to the question - why is this
>> > needed?
>>
>> We welcome a better opinion of name (indeed verify is not that great).
>> This elevates ABI to API so either invokes the C compiler or reads
>> stuff from the cache.
>
>
> Can you elaborate on what "elevates ABI to API" means here?
>
>>
>> > 3. The partial type specifications in C with ellipsis. What is the
>> > point?
>> > You have the C declarations somewhere anyhow, so why introduce this? The
>> > "ABI level" boasts having just C and Python to write, but those partial
>> > ellipsis-ridden declarations are hardly C.
>>
>> No, you don't. Some libraries contain macros for example (like
>> OpenSSL) where you just can't use ABI because it makes no sense. It's
>> less common on windows where binary compatibility is important,
>> however looking on linux, multiple stdlib declaration would use
>> ellipsis in the man page.
>
>
> It would be useful to find an actual example and discuss it concretely.
#define Py_INCREF(x) (x -> refcnt++)
the API here is Py_INCREF, which is a macro and does not exist in the
DLL. copy-pasting the implementation is not using the API, it's hoping
that stuff won't change in the future (and say makes it not working
when PyPy implements Py_INCREF as a function). Some POSIX things are
macros in one version of glibc and functions in other and there is no
change in API so your code will silently stop working.
>
>>
>> I can't seem to find one right now, but it's
>> something like:
>>
>> struct X {
>>int public_field;
>>...
>> }
>>
>> which is impossible to do correctly with ctypes without exposing some
>> sort of platform dependency that might change without warning.
>>
>> Another usages are #define SQLITE_OK ... which you don't know at the
>> time of writing (people assume those won't change and the do change).
>
>
> Do you mean that the value of SQLITE_OK changed over time (now it's 0, but
> used to be different?)
>
> If so, then in a realistic use case, how would the API level help solve
> this?
>
> Eli
no the API is SQLITE_OK. The actual numeric value of 0 is *not* part
of the API. it might be 0 it might be not, you never know. In cffi you
write:
#define SQLITE_OK ...
and let the compiler figure out. in ctypes you just hardcode 0. I
ended up once with a scary mess of ifs because BSD Linux and OS X were
declaring the same numeric constants with different values.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] peps: Pre-alpha draft for PEP 435 (enum). The name is not important at the moment, as
Hi, offering my DKK 0.50 on the subject. I've used an in-house enum type for the better part of a decade - put up at http://unicollect.dk/download/oss/dc_enum.zip for inspiration. I'm partial to an int subclass, or at least something very int-like, because wrapping C libraries is such a major use case. Enums should support all the operations that C enum's do: indexing, ordering, bitwise math and unnamed values. There's a whole bunch of these wrapped C enum's in the standard library, e.g. in the os, stat and urllib modules. Wouldn't it be good to see them converted to enumeration values, to make them sort-of self-documenting and sort-of typechecked? - About unnamed values: Suppose for example you gather up some stat constants in an enum like this: class S_I(Enum): XOTH = 1 WOTH = 2 ROTH = 4 RWXO = 7 XGRP = 8 WGRP = 16 RGRP = 32 RWXG = 56 XUSR = 64 EXEC = 64 WRITE = 128 WUSR = 128 RUSR = 256 READ = 256 RWXU = 448 SVTX = 512 SGID = 1024 SUID = 2048 FIFO = 4096 FCHR = 8192 FDIR = 16384 FBLK = 24576 FREG = 32768 FLNK = 40960 FSOCK = 49152 Now what happens if you do >>> S_I(5) ? This should not raise an exception: 5 is the perfectly valid combination of XOTH and ROTH. Supporting unnamed values is also useful when reading integer values from an external source - file, network, serial port - where typically you'd first convert the received value to the enum type before doing any processing. If converting to enum type may raise an exception, that would force you to print raw integer values in diagnostics, where you'd rather have printed the userfriendly symbolic names. - The repr of an enum value should be short and sweet. Because sometimes you'll be looking at long lists of information with 2-3 of these on every line, and then is just going to be annoyingly verbose. This is how my library eventually printed enums: >>> S_I.ROTH S_I.ROTH >>> S_I(4) S_I.ROTH >>> S_I(5) S_I(5) >>> print(S_I(4)) S_I.ROTH(4) >>> print(S_I(5)) S_I.?(5) - A final thing, attempting to compare or convert values from different enum classes should be a TypeError. E.g. in wxPython, it's easy to mistake wx.OK and wx.ID_OK, and a friendly TypeError message is a good way of detecting this early. regards, Anders ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python Language Summit at PyCon: Agenda
Hello all, PyCon, and the Python Language Summit, is nearly upon us. We have a good number of people confirmed to attend. If you are intending to come to the language summit but haven't let me know please do so. The agenda of topics for discussion so far includes the following: * A report on pypy status - Maciej and Armin * Jython and IronPython status reports - Dino / Frank * Packaging (Doug Hellmann and Monty Taylor at least) * Cleaning up interpreter initialisation (both in hopes of finding areas to rationalise and hence speed things up, as well as making things more embedding friendly). Nick Coghlan * Adding new async capabilities to the standard library (Guido) * cffi and the standard library - Maciej * flufl.enum and the standard library - Barry Warsaw * The argument clinic - Larry Hastings If you have other items you'd like to discuss please let me know and I can add them to the agenda. All the best, Michael Foord -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Wed, Feb 27, 2013 at 10:51 AM, Michael Foord wrote: > Hello all, > > PyCon, and the Python Language Summit, is nearly upon us. We have a good > number of people confirmed to attend. If you are intending to come to the > language summit but haven't let me know please do so. > > The agenda of topics for discussion so far includes the following: > > * A report on pypy status - Maciej and Armin > * Jython and IronPython status reports - Dino / Frank > * Packaging (Doug Hellmann and Monty Taylor at least) > * Cleaning up interpreter initialisation (both in hopes of finding areas > to rationalise and hence speed things up, as well as making things > more embedding friendly). Nick Coghlan > * Adding new async capabilities to the standard library (Guido) > * cffi and the standard library - Maciej > * flufl.enum and the standard library - Barry Warsaw > * The argument clinic - Larry Hastings > > If you have other items you'd like to discuss please let me know and I can > add them to the agenda. I'll take detailed notes again this year. Within a few days of the end of the conference I'll post a write-up to blog.python.org and this list to keep everyone informed. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On 27 February 2013 11:53, Maciej Fijalkowski wrote: > I think it means you can't use the ABI version and specify the calling > convention. It's a reasonable bug report (the calling convention on > API version works though) That would be a deal-breaker for my use case of quick scripts working with the Windows API on a machine with no C compiler. I'd have to use ctypes in that case until cffi had this feature. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
27 February 2013 18:24, Paul Moore wrote: > On 27 February 2013 11:53, Maciej Fijalkowski wrote: >> I think it means you can't use the ABI version and specify the calling >> convention. It's a reasonable bug report (the calling convention on >> API version works though) > > That would be a deal-breaker for my use case of quick scripts working > with the Windows API on a machine with no C compiler. I'd have to use > ctypes in that case until cffi had this feature. One other use case for the ABI level over the API level - the ABI level (no C extension) can be used across Python versions, where the API level needs a separate compiled extension per Python version. This can be a big deal on Windows at least, where users (not developers) with no compilers on their systems are common. I'm not trying to argue against cffi, just pointing out some cases that should be considered. Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On Wed, Feb 27, 2013 at 1:28 PM, Paul Moore wrote: > 27 February 2013 18:24, Paul Moore wrote: > > On 27 February 2013 11:53, Maciej Fijalkowski wrote: > >> I think it means you can't use the ABI version and specify the calling > >> convention. It's a reasonable bug report (the calling convention on > >> API version works though) > > > > That would be a deal-breaker for my use case of quick scripts working > > with the Windows API on a machine with no C compiler. I'd have to use > > ctypes in that case until cffi had this feature. > > One other use case for the ABI level over the API level - the ABI > level (no C extension) can be used across Python versions, where the > API level needs a separate compiled extension per Python version. This > can be a big deal on Windows at least, where users (not developers) > with no compilers on their systems are common. > > Is that still true for Windows even with the stable ABI and the dropping of narrow/wide Unicode builds? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Wed, 27 Feb 2013 16:51:16 + Michael Foord wrote: > Hello all, > > PyCon, and the Python Language Summit, is nearly upon us. We have a good > number of people confirmed to attend. If you are intending to come to the > language summit but haven't let me know please do so. > > The agenda of topics for discussion so far includes the following: > > * A report on pypy status - Maciej and Armin > * Jython and IronPython status reports - Dino / Frank > * Packaging (Doug Hellmann and Monty Taylor at least) > * Cleaning up interpreter initialisation (both in hopes of finding areas > to rationalise and hence speed things up, as well as making things > more embedding friendly). Nick Coghlan > * Adding new async capabilities to the standard library (Guido) > * cffi and the standard library - Maciej > * flufl.enum and the standard library - Barry Warsaw > * The argument clinic - Larry Hastings > > If you have other items you'd like to discuss please let me know and I can > add them to the agenda. Perhaps someone wants to discuss http://www.python.org/dev/peps/pep-0428/, but I won't be there and the PEP isn't terribly up-to-date either :-) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On 27 February 2013 18:50, Brett Cannon wrote: >> One other use case for the ABI level over the API level - the ABI >> level (no C extension) can be used across Python versions, where the >> API level needs a separate compiled extension per Python version. This >> can be a big deal on Windows at least, where users (not developers) >> with no compilers on their systems are common. >> > > Is that still true for Windows even with the stable ABI and the dropping of > narrow/wide Unicode builds? Probably not, but I've never actually seen the stable ABI used in practice (and I don't know if cffi restricts itself to the stable ABI). I'm not sure that any toolchain (such as bdist_wininst or wheel) really supports it (in the sense that they tend to make the assumption that if there is a C extension, the code is version-specific). None of these are insurmountable problems, though. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Wed, Feb 27, 2013 at 8:51 AM, Michael Foord wrote: > If you have other items you'd like to discuss please let me know and I can > add them to the agenda. I'd like to discuss merging Jython's standard Lib (the *.py files). We have in the past had agreement that this would be a good idea - I just want to bring it up since I think this is probably the year that I'm actually going to do it. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
Hi Paul, On Wed, Feb 27, 2013 at 7:24 PM, Paul Moore wrote: > On 27 February 2013 11:53, Maciej Fijalkowski wrote: >> I think it means you can't use the ABI version and specify the calling >> convention. It's a reasonable bug report (the calling convention on >> API version works though) > > That would be a deal-breaker for my use case of quick scripts working > with the Windows API on a machine with no C compiler. I'd have to use > ctypes in that case until cffi had this feature. That's not correct: you can't indeed give the calling convention, but it is not needed for the common case. What is not supported is only Python-defined callbacks using the Windows-specific convention --- as documented, there is a workaround for that case. And, in case you wonder, this automatic detection comes from ctypes. I copied the hacked-up version of libffi for Windows from ctypes to cffi, and the logic to detect the actual calling convention and support both is there. The difference is only that in ctypes, after it did the call (always successfully), it checks the convention that was found to be used by the C function, and if it differs from the one specified by the user, then it complains. I basically just removed the "complaining" part. A bientôt, Armin. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Wed, Feb 27, 2013 at 2:01 PM, fwierzbi...@gmail.com < fwierzbi...@gmail.com> wrote: > On Wed, Feb 27, 2013 at 8:51 AM, Michael Foord > wrote: > > If you have other items you'd like to discuss please let me know and I > can add them to the agenda. > > I'd like to discuss merging Jython's standard Lib (the *.py files). We > have in the past had agreement that this would be a good idea - I just > want to bring it up since I think this is probably the year that I'm > actually going to do it. > Do you mean more generally getting more pure Python implementations of modules in the stdlib? If so then as a reference there is http://bugs.python.org/issue16651 which lists the modules in the stdlib w/ only extension module implementations (although operator and random have patches; the latter is waiting for Raymond to approve). And if I am right about what you are suggesting, Frank, this should probably be expanded to a more concerted effort with IronPython and PyPy. IOW it warrants a discussion. =) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On 27 February 2013 19:08, Armin Rigo wrote: > That's not correct: you can't indeed give the calling convention, but > it is not needed for the common case. What is not supported is only > Python-defined callbacks using the Windows-specific convention --- as > documented, there is a workaround for that case. OK, that's cool. As I said, I really need to actually experiment with cffi - this thread has certainly made me want to do so. Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Wed, Feb 27, 2013 at 11:21 AM, Brett Cannon wrote: > Do you mean more generally getting more pure Python implementations of > modules in the stdlib? If so then as a reference there is > http://bugs.python.org/issue16651 which lists the modules in the stdlib w/ > only extension module implementations (although operator and random have > patches; the latter is waiting for Raymond to approve). That is part of it, although my bigger goal is slightly more ambitious. I'm hoping to eventually delete the entire Lib/ directory from Jython and just pull in CPython's. > And if I am right > about what you are suggesting, Frank, this should probably be expanded to a > more concerted effort with IronPython and PyPy. IOW it warrants a > discussion. =) Oh sure sorry - I have some tunnel vision lately :) I am suggesting that we push forward on the "shared library" approach to the files in the Lib/* directory, so that would certainly include IronPython and PyPy as well I hope. The easy part for Jython is pushing some of our "if is_jython:" stuff into the appropriate spots in CPython's Lib/. I'd also like to do something at the top of CPython specific .py files that would fail the import in case it is called from Jython. I suspect that OS packagers would like it if the Lib directory for a particular Python version could be entirely shared between implementations. There are a couple of spots that might be more controversial. For example, Jython has a file Lib/zlib.py that implements zlib in terms of the existing Java support for zlib. I do wonder if such a file is acceptable in CPython's Lib since its 195 lines of code would be entirely skipped by CPython. Anyway I think I might be rambling - it seems like a good thing to discuss :) -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On Wed, Feb 27, 2013 at 4:18 AM, Ronald Oussoren wrote: > > On 27 Feb, 2013, at 10:06, Maciej Fijalkowski wrote: > >> On Wed, Feb 27, 2013 at 9:29 AM, Ronald Oussoren >> wrote: >>> >>> On 26 Feb, 2013, at 16:13, Maciej Fijalkowski wrote: >>> Hello. I would like to discuss on the language summit a potential inclusion of cffi[1] into stdlib. >>> >>> The API in general looks nice, but I do have some concens w.r.t. including >>> cffi in the stdlib. >>> >>> 1. Why is cffi completely separate from ctypes, instead of layered on top >>> of it? That is, add a utility module to ctypes that can parse C >>> declarations and generate the right ctypes definitions. >> >> Because ctypes API is a mess and magic. We needed a cleaner (and much >> smaller) model. > > The major advantages of starting over is probably that you can hide the > complexity and that opens opportunities for optimizations. That said, I'm not > convinced that ctypes is unnecessarily complex. cffi actually does have a ctypes backend in addition to the ffi and "compile a CPython extension" backends. But the ctypes backend is guaranteed to be slow and messy because it is ctypes. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Wed, Feb 27, 2013 at 7:51 PM, Michael Foord wrote: > Hello all, > > PyCon, and the Python Language Summit, is nearly upon us. We have a good > number of people confirmed to attend. If you are intending to come to the > language summit but haven't let me know please do so. > > The agenda of topics for discussion so far includes the following: > > * A report on pypy status - Maciej and Armin > * Jython and IronPython status reports - Dino / Frank > * Packaging (Doug Hellmann and Monty Taylor at least) > * Cleaning up interpreter initialisation (both in hopes of finding areas > to rationalise and hence speed things up, as well as making things > more embedding friendly). Nick Coghlan > * Adding new async capabilities to the standard library (Guido) > * cffi and the standard library - Maciej > * flufl.enum and the standard library - Barry Warsaw > * The argument clinic - Larry Hastings > > If you have other items you'd like to discuss please let me know and I can > add them to the agenda. > I won't be able to visit, so this PyCon promises to be all good nice and relaxing for everybody. =) But here is some things that can spice up the meeting in case it becomes boring. * poor introspection capabilities * if you pause the code - what kind of data you'd like to be available? * magical locals() dict that breaks the rules * execution frames that lose information present in original source (such as function vs method and probably others) * as an exercise - try to build a scroller for a running Python script * it is impossible for Python 2 and probably for Python 3 as well * visibility issues with language development * physically split the information flow about work being done on interpreter and stdlib * split the information about stdlib development by modules * describe modules composing in stdlib in formal way https://bitbucket.org/techtonik/python-stdlib * build a roadmap by module (join personal wishlist from involved people) * external people can not join teams dedicated only to several modules people are interested in * IPython and PyPy as a distraction for people who could improve core language and stdlib here * security by obscurity in legal position of PSF towards contributors https://code.google.com/legal/individual-cla-v1.0.html vs http://www.python.org/psf/contrib/contrib-form/ + http://www.samurajdata.se/opensource/mirror/licenses/afl-2.1.php or http://www.python.org/psf/contrib/contrib-form/ + http://opensource.org/licenses/apache2.0.php and why PSF doesn't comply the 4. Redistribution clause from Apache 2.0 license * how to get more pictures and less text for reference, especially for internals * user story approach for writing PEPs Can only wish to have a constructive PyCon with great results. Bye. -- anatoly t. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On 27 February 2013 19:26, Paul Moore wrote: > On 27 February 2013 19:08, Armin Rigo wrote: >> That's not correct: you can't indeed give the calling convention, but >> it is not needed for the common case. What is not supported is only >> Python-defined callbacks using the Windows-specific convention --- as >> documented, there is a workaround for that case. > > OK, that's cool. As I said, I really need to actually experiment with > cffi - this thread has certainly made me want to do so. Actually, looking at the cffi documentation, I can't see how I'd translate the following very simple example of something I do quite a lot in ctypes: from ctypes import windll MessageBox = windll.User32.MessageBoxW MessageBox(0, "Hello, world!", "Title", 0) Note - I wrote this from memory, I honestly don't know without looking it up the precise argument types for MessageBoxW - and even if I did, I suspect they would all be macros from windows.h. I don't want to invoke the C compiler by using verify, and I don't see in the docs how I'd get the macro definitions any other way. If anyone could show me a cffi equivalent, I'd be very grateful. Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
Antoine Pitrou wrote: We have to find sufficiently silly species of snakes, though. Glancing through http://en.wikipedia.org/wiki/List_of_snakes, we have: Wart snakes Java wart snakes Pipe snakes Stiletto snakes Rubber boas Dog-faced water snakes Shovel-nosed snakes Hook-nosed snakes Leaf-nosed snakes Lyre snakes Cat-eyed snakes Worm snakes North American hog-nosed snakes Moccasins Humpnose vipers Puff adders Carpet vipers False horned vipers Sunbeam snakes And then there's this: http://www.nairaland.com/1009227/new-species-snake-discovered-brazil -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
On 2/27/2013 3:31 PM, Paul Moore wrote: from ctypes import windll MessageBox = windll.User32.MessageBoxW MessageBox(0, "Hello, world!", "Title", 0) > Note - I wrote this from memory, Gee, that is pretty easy. Worked perfectly -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
Hi Paul,
On Wed, Feb 27, 2013 at 9:31 PM, Paul Moore wrote:
> from ctypes import windll
> MessageBox = windll.User32.MessageBoxW
> MessageBox(0, "Hello, world!", "Title", 0)
You are right that it's a bit cumbersome in cffi up to and including
0.5, but in the cffi trunk all standard Windows types are included.
So the general answer to your question is: we google MessageBox and
copy that line from the microsoft site, and manually remove the
unnecessary WINAPI and _In_opt declarations:
from cffi import FFI
ffi = FFI()
ffi.cdef("""
int MessageBox(HWND hWnd, LPCTSTR lpText, LPCTSTR lpCaption, UINT uType);
""")
lib = ffi.dlopen("USER32.DLL")
lib.MessageBox(ffi.NULL, "Hello, world!", "Title", 0)
That's a slightly unfair example, because in this case it happens to
work with ctypes without specifying the argtypes and the restype. I
would argue that this feature of ctypes is not a good thing: it's
mostly the same as saying "you only need to declare argtypes and
restype if you get nonsense results or segfaults".
Note for completeness that the version with verify() simply replaces
"lib = ffi.dlopen("USER32.DLL")" with "lib = ffi.verify("")" (no
#include necessary here). Then you cannot misdeclare anything without
getting clear compile-time errors at the verify(). The compilation
occurs only once (it's cached) and by writing two lines in your
setup.py you can distribute binary installations, just like you do
with hand-written C extension modules; so the extra burden of
accessing the API level is in my opinion very small compared to its
segfault-avoiding gain. But I know that either level of access can
make sense in different situations. Typically, on Windows, the
ABI-level works fine; as argued elsewhere, on other platforms and/or
with some libraries, the API-level is definitely more suited.
A bientôt,
Armin.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cffi in stdlib
Armin Rigo: > So the general answer to your question is: we google MessageBox and > copy that line from the microsoft site, and manually remove the > unnecessary WINAPI and _In_opt declarations: Wouldn't it be better to understand the SAL annotations like _In_opt so that spurious NULLs (for example) produce a good exception from cffi instead of failing inside the system call? Neil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Wed, Feb 27, 2013 at 12:33 PM, fwierzbi...@gmail.com wrote: > There are a couple of spots that might be more controversial. For > example, Jython has a file Lib/zlib.py that implements zlib in terms > of the existing Java support for zlib. I do wonder if such a file is > acceptable in CPython's Lib since its 195 lines of code would be > entirely skipped by CPython. Even if it's pure Python, that sounds like an accelerator module (a la PEP 399) to me. -eric ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Feb 27, 2013, at 11:33 AM, fwierzbi...@gmail.com wrote: >I am suggesting that we push forward on the "shared library" approach to the >files in the Lib/* directory, so that would certainly include IronPython and >PyPy as well I hope. +1 >The easy part for Jython is pushing some of our "if is_jython:" stuff >into the appropriate spots in CPython's Lib/. I wonder if there isn't a better way to do this than sprinkling is_jython, is_pypy, is_ironpython, is_thenextbigthing all over the code base. I have no bright ideas here, but it seems like a feature matrix would be a better way to go than something that assumes a particular Python implementation has a particular feature set (which may change in the future). -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
On Feb 27, 2013, at 04:51 PM, Michael Foord wrote: >If you have other items you'd like to discuss please let me know and I can >add them to the agenda. I'd like to have some discussions around promotion of Python 3, how we can accelerate its adoption, availability of supporting packages, what critical projects are still missing, etc. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Language Summit at PyCon: Agenda
Am 27.02.2013 17:51, schrieb Michael Foord: > Hello all, > > PyCon, and the Python Language Summit, is nearly upon us. We have a good > number of people confirmed to attend. If you are intending to come to the > language summit but haven't let me know please do so. > > The agenda of topics for discussion so far includes the following: > > * A report on pypy status - Maciej and Armin > * Jython and IronPython status reports - Dino / Frank > * Packaging (Doug Hellmann and Monty Taylor at least) > * Cleaning up interpreter initialisation (both in hopes of finding areas > to rationalise and hence speed things up, as well as making things > more embedding friendly). Nick Coghlan > * Adding new async capabilities to the standard library (Guido) > * cffi and the standard library - Maciej > * flufl.enum and the standard library - Barry Warsaw > * The argument clinic - Larry Hastings > > If you have other items you'd like to discuss please let me know and I can > add them to the agenda. May I in absentia propose at least a short discussion of the XML fixes and accompanying security releases? FWIW, for 3.2 and 3.3 I have no objections to secure-by-default. Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
