[Python-Dev] Working on the Python code-base with a VIM-based setup

[Eli Bendersky]

Hello pydev,

This is a meta-question which I hope is appropriate in this list (**).
Recently I've switched to to VIM as my main development platform (in terms
of code editing and navigation). Working on the Python code-base is both a
concrete requirement and a yardstick for me - I want to be as effective as
possible at it. Therefore I would like to ask those of you working on
Python's code with VIM about your setups - the special tweaks to VIM &
plugins you use to make working with the code as simple and effective as
possible.

Myself, since I'm still a VIM newbie, my setup is quite spartan. I created
tags with:

ctags -R Grammar Include Modules/ Objects/ Parser/ Python/

And now happily browse around the source code with Ctrl-], Ctrl-I/O, Ctrl-T
and so on. I've also installed the Taglist plugin to show all
functions/macros in open buffers - it's sometimes helpful. Other plugins
I've found useful ar NERD-commenter (for commenting out chunks of code) and
highlight_current_line (for a visual cue of where I am in a file).
Besides, I've taken the suggested settings from the .vimrc file in Misc/ to
help enforcing PEP-8.

I heard that there are more sophisticated tags plugins that also allow one
to check where a function is called for, and other intellisens-y stuff,
though I'm not sure whether anyone uses it for Python's code.

Thanks in advance,
Eli


(**) Note that it deals with the source code *of Python* (the stuff you
download from Python's official SVN), not Python source code.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 384 status

[Adal Chiriliuc]

On Wed, Sep 1, 2010 at 1:42 AM, Antoine Pitrou  wrote:
> What I think would be a mistake would be to define the API in terms of
> Windows-specific quirks. All this discussion seems bent on satisfying
> the needs of Windows developers at the expense of non-Windows
> developers. "FILE*" is a perfectly standard C feature (and a
> widely-used one at that). If Windows doesn't handle it correctly then
> it's a Windows issue and we shouldn't annoy other people by refusing
> access to that feature.

As an alternative the stable ABI could use the native system file
handle instead of FILE*. HANDLE (returned by CreateFile) on Windows
and int (returned by open) on Linux (or even FILE*).

The extension writer would have to write it's own abstraction over
this. Python could provide a default wrapper (as a sample).
Unfortunately you would also lose the buffering provided by fopen
(unless the sample also does this).

As a side note, I wrote a plugin for a big Windows application
(Mastercam) once. There were at least 5 different CRTs running in that
process - 3 different release versions and 2 different debug versions.
Besides a few other statically linked into other plugin DLLs. This is
quite common, especially since you can have foreign CRTs injected into
your process by various shell extensions like TortoiseSVN.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 384 status

[Greg Ewing]

M.-A. Lemburg wrote:


But isn't exactly that a major problem for Python ?

An extension written for a different MS CRT version would
not be able to safely free a buffer allocated by the Python
DLL.


Python provides its own set of memory alloc/free functions
to deal with this exact problem. For every Python function
that allocates something, there's a corresponding function
for freeing it, and you do it any other way at your peril.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 384 status

[Antoine Pitrou]

On Wed, 01 Sep 2010 10:23:42 +1200
Greg Ewing  wrote:
> Daniel Stutzbach wrote:
> 
> > Likewise, a FILE * isn't safe to pass around, unless I can guarantee 
> > that the application really is one big happy family compiled against the 
> > same version of the C library.
> 
> Given that, it seems to me that it's a mistake for Python
> to provide any APIs that take a FILE* directly in the
> first place.

What I think would be a mistake would be to define the API in terms of
Windows-specific quirks. All this discussion seems bent on satisfying
the needs of Windows developers at the expense of non-Windows
developers. "FILE*" is a perfectly standard C feature (and a
widely-used one at that). If Windows doesn't handle it correctly then
it's a Windows issue and we shouldn't annoy other people by refusing
access to that feature.

Again, putting a warning in the documentation should be enough for
those people who want to have perfect Windows support. And if
the Microsoft compiler/linker doesn't bother warning the user when
passing FILE* between various CRTs, then it's not Python's job to
supplement it.

After all, we don't usually try to workaround platform-specific
bugs (not as a low level such as the C API level); at worse, we mention
their existence in the docs.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 384 status

[Nick Coghlan]

On Tue, Aug 31, 2010 at 2:15 PM, Mark Hammond  wrote:
> It would be interesting to know how, in practice, these FILE pointers come
> to life.  In my experience they are generally obtained via fopen. If that is
> broadly true, then a middle-ground may be for Python to expose something
> like Py_fopen, Py_fclose and a PyFILE opaque "handle".  API elements which
> currently take a FILE * could be exposed using a PyFILE * in the ABI.
>  People who didn't care about this level of portability could continue to
> use the non-ABI FILE * functions, but people who do could use
> Py_fopen/Py_fclose in place of fopen/fclose but otherwise work as now.

No need for that - the Python-specific equivalent is our own I/O stack
(and things like os.fdopen).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 384 status

[Nick Coghlan]

On Wed, Sep 1, 2010 at 7:49 AM, M.-A. Lemburg  wrote:
>> Yes, and it's a pretty common situation.   The fopen() that I call within a
>> DLL may not be the same fopen() called by another DLL.  When writing a DLL
>> for Windows, the API must be designed with the assumption that anything
>> returned by the C library cannot be passed a different C library.  For
>> example, suppose I a expose a function in my DLL that allocates some memory,
>> populates it with useful information, and returns a pointer.  I must also
>> supply a function to free the memory.  I cannot ask the caller to simply
>> call free(), because their free() may not be using the same heap as my
>> malloc().
>
> But isn't exactly that a major problem for Python ?
>
> An extension written for a different MS CRT version would
> not be able to safely free a buffer allocated by the Python
> DLL.

Correct, but we generally don't allow raw pointers across the API
boundary like that. Instead, users are told to release memory with
PyMem_Free, PyObject_Delete, reference decrementing, etc. That means
the memory release happens in the runtime linked with the Python DLL
and everything is fine.

>
>> Likewise, a FILE * isn't safe to pass around, unless I can guarantee that
>> the application really is one big happy family compiled against the same
>> version of the C library.
>
> According to the MS document I found this also applies to the OS
> environment and handles.
>
> Taking all these things together makes it sound like there's
> a rather small chance of actually getting PEP 384 working
> across Windows compiler upgrades.

Not really - we just need to keep an eye out for the places where
those things can leak through and ensure they're instead abstracted so
the extension module only talks to Python rather than being given
direct access to the underlying C runtime details. FILE* is one, errno
is another, locale we may just have to live with (alternatively, we
could add a callback API to allow extension modules to be notified
when the locale changes, which may be useful for other reasons... see
issue9727). Antoine pointed out file descriptors as another possible
problem, but I'm not sure we can prevent that at the API level (there
are too many ways for Python code to get at file descriptors - it is
likely better to just say that if you get a file descriptor from
Python, only use it with Python APIs, not directly with the C
runtime).

> Then again, I haven't heard of many people actually running into
> those possible issues.

Even today you can use a different C runtime in a Python extension so
long as you avoid the APIs that would cause problems (e.g. I have
legacy SWIG-wrapped extensions built with VC6 that run quite happily
even when linked against Python 2.4).

The idea with the stable ABI is to explicitly exclude those functions
so that you *know* you aren't going to run into those problems.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 384 status

[Greg Ewing]

Daniel Stutzbach wrote:

Likewise, a FILE * isn't safe to pass around, unless I can guarantee 
that the application really is one big happy family compiled against the 
same version of the C library.


Given that, it seems to me that it's a mistake for Python
to provide any APIs that take a FILE* directly in the
first place.

Maybe PyObject_Print should be deprecated in favour of
something that takes a Python I/O object instead.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 384 status

[M.-A. Lemburg]

Neil Hodgson wrote:
> M.-A. Lemburg:
> 
>> Is it possible to have multiple versions of the lib C loaded
>> on Windows ?
> 
>Yes. It is possible not only to mix C runtimes from different
> vendors but different variants from a single vendor.
> 
>Historically, each vendor has shipped their own C runtime
> libraries. This was also the case with CP/M and OS/2.
> 
>Many applications can be extended with DLLs and if it were not
> possible to load DLLs which use different runtimes then that would
> limit which compilers can be used to extend particular applications.
> If Microsoft were to stop DLLs compiled with Borland or Intel from
> working inside Internet Explorer or Excel then there would be
> considerable controversy and likely anti-trust actions.

Thanks for the feedback.

This sounds like the issues such a mix can cause are mostly
theoretical and don't really bother much in practice, so
PEP 384 on Windows does have a chance :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 27 2010)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2010-08-19: Released mxODBC 3.1.0  http://python.egenix.com/
2010-09-15: DZUG Tagung, Dresden, Germany  18 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 384 status

[M.-A. Lemburg]

Daniel Stutzbach wrote:
> On Tue, Aug 31, 2010 at 4:54 AM, M.-A. Lemburg  wrote:
> 
>> Is it possible to have multiple versions of the lib C loaded
>> on Windows ?
>>
> 
> Yes, and it's a pretty common situation.   The fopen() that I call within a
> DLL may not be the same fopen() called by another DLL.  When writing a DLL
> for Windows, the API must be designed with the assumption that anything
> returned by the C library cannot be passed a different C library.  For
> example, suppose I a expose a function in my DLL that allocates some memory,
> populates it with useful information, and returns a pointer.  I must also
> supply a function to free the memory.  I cannot ask the caller to simply
> call free(), because their free() may not be using the same heap as my
> malloc().

But isn't exactly that a major problem for Python ?

An extension written for a different MS CRT version would
not be able to safely free a buffer allocated by the Python
DLL.

> Likewise, a FILE * isn't safe to pass around, unless I can guarantee that
> the application really is one big happy family compiled against the same
> version of the C library.

According to the MS document I found this also applies to the OS
environment and handles.

Taking all these things together makes it sound like there's
a rather small chance of actually getting PEP 384 working
across Windows compiler upgrades.

Then again, I haven't heard of many people actually running into
those possible issues.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 27 2010)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2010-08-19: Released mxODBC 3.1.0  http://python.egenix.com/
2010-09-15: DZUG Tagung, Dresden, Germany  18 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 384 status

[Daniel Stutzbach]

On Tue, Aug 31, 2010 at 4:54 AM, M.-A. Lemburg  wrote:

> Is it possible to have multiple versions of the lib C loaded
> on Windows ?
>

Yes, and it's a pretty common situation.   The fopen() that I call within a
DLL may not be the same fopen() called by another DLL.  When writing a DLL
for Windows, the API must be designed with the assumption that anything
returned by the C library cannot be passed a different C library.  For
example, suppose I a expose a function in my DLL that allocates some memory,
populates it with useful information, and returns a pointer.  I must also
supply a function to free the memory.  I cannot ask the caller to simply
call free(), because their free() may not be using the same heap as my
malloc().

Likewise, a FILE * isn't safe to pass around, unless I can guarantee that
the application really is one big happy family compiled against the same
version of the C library.
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Internal counter to debug leaking file descriptors

[David Malcolm]

On Tue, 2010-08-31 at 17:40 +, exar...@twistedmatrix.com wrote:
> On 05:22 pm, gl...@twistedmatrix.com wrote:
> >
> >On Aug 31, 2010, at 10:03 AM, Guido van Rossum wrote:
> >>On Linux you can look somewhere in /proc, but I don't know that it
> >>would help you find where a file was opened.
> >
> >"/dev/fd" is actually a somewhat portable way of getting this 
> >information.  I don't think it's part of a standard, but on Linux it's 
> >usually a symlink to "/proc/self/fd", and it's available on MacOS and 
> >most BSDs (based on a hasty and completely-not-comprehensive 
> >investigation).  But it won't help you find out when the FDs were 
> >originally opened, no.
> >___
> 
> On OS X and Solaris, dtrace and ustack will tell you exactly when and 
> where the FDs were originally opened, though.  On Linux, SystemTap might 
> give you the same information (but I know much less about SystemTap). 
> If http://bugs.python.org/issue4111 is resolved, then this may even be 
> possible without using a patched version of Python.

I believe you can do something like this:
$ cat /tmp/trace-all-syscalls.stp 
/*
  Watch all syscalls in a specified process, dumping a user-space
  backtrace 
*/
probe syscall.* {
  if (pid() == target()) {
  printf("%s(%s)\n", probefunc(), argstr)
  print_ubacktrace();
  }
}

$ sudo stap --ldd -d /usr/bin/python /tmp/trace-all-syscalls.stp -c "python -c 
'print 42'"

This generates a torrent of debug data like this:
sys_mmap_pgoff(0x0, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 
0)
 0x38f44e17aa : mmap64+0xa/0x30 [libc-2.11.90.so]
 0x38f44673fc : _IO_file_doallocate+0x7c/0x110 [libc-2.11.90.so]
 0x38f447498c : _IO_doallocbuf+0x2c/0x50 [libc-2.11.90.so]
 0x38f4472ef4 : _IO_file_underflow@@GLIBC_2.2.5+0x1b4/0x230 [libc-2.11.90.so]
 0x38f44749ce : _IO_default_uflow+0xe/0x30 [libc-2.11.90.so]
 0x38f446fdcb : getc+0xab/0xf0 [libc-2.11.90.so]
 0x39054f3e13 : r_long+0x23/0x120 [libpython2.6.so.1.0]
 0x39054f3f3b : PyMarshal_ReadLongFromFile+0x2b/0x30 [libpython2.6.so.1.0]
 0x39054f0661 : load_source_module+0x271/0x640 [libpython2.6.so.1.0]
 0x39054f1cc5 : import_submodule+0x155/0x300 [libpython2.6.so.1.0]
 0x39054f1f85 : load_next+0x115/0x2a0 [libpython2.6.so.1.0]
 0x39054f2592 : import_module_level+0x212/0x730 [libpython2.6.so.1.0]
 0x39054f3314 : PyImport_ImportModuleLevel+0x44/0xb0 [libpython2.6.so.1.0]
 0x39054d843f : builtin___import__+0x8f/0xa0 [libpython2.6.so.1.0]
 0x3905443f43 : PyObject_Call+0x53/0x100 [libpython2.6.so.1.0]
 0x39054d89b3 : PyEval_CallObjectWithKeywords+0x43/0xf0 [libpython2.6.so.1.0]
 0x39054db674 : PyEval_EvalFrameEx+0x21b4/0x65b0 [libpython2.6.so.1.0]
 0x39054e03a8 : PyEval_EvalCodeEx+0x938/0x9e0 [libpython2.6.so.1.0]
 0x39054e0482 : PyEval_EvalCode+0x32/0x40 [libpython2.6.so.1.0]
 0x39054f02c2 : PyImport_ExecCodeModuleEx+0xc2/0x1f0 [libpython2.6.so.1.0]
 0x39054f07a6 : load_source_module+0x3b6/0x640 [libpython2.6.so.1.0]


You may want to specify specific syscalls in the above to narrow the
scope.

Issue 4111 patches cpython to statically mark Python frame entry/exit so
that systemtap can directly instrument that; in Fedora 13 onwards I've
built Python with systemtap hooks so that you can add:

probe python.function.entry {
printf("%s:%s:%d\n", filename, funcname, lineno);
}


(Arguably this is wrong, it's frame entry/exit, rather than function
entry/exit).

Potentially systemtap could be taught how to decipher/prettyprint Python
backtraces in a similar way to how gdb does it (by hooking into
PyEval_EvalFrameEx)


Hope this is helpful
Dave

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Internal counter to debug leaking file descriptors

[exarkun]

On 05:22 pm, gl...@twistedmatrix.com wrote:


On Aug 31, 2010, at 10:03 AM, Guido van Rossum wrote:

On Linux you can look somewhere in /proc, but I don't know that it
would help you find where a file was opened.


"/dev/fd" is actually a somewhat portable way of getting this 
information.  I don't think it's part of a standard, but on Linux it's 
usually a symlink to "/proc/self/fd", and it's available on MacOS and 
most BSDs (based on a hasty and completely-not-comprehensive 
investigation).  But it won't help you find out when the FDs were 
originally opened, no.

___


On OS X and Solaris, dtrace and ustack will tell you exactly when and 
where the FDs were originally opened, though.  On Linux, SystemTap might 
give you the same information (but I know much less about SystemTap). 
If http://bugs.python.org/issue4111 is resolved, then this may even be 
possible without using a patched version of Python.


Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Internal counter to debug leaking file descriptors

[Glyph Lefkowitz]

On Aug 31, 2010, at 10:03 AM, Guido van Rossum wrote:

> On Linux you can look somewhere in /proc, but I don't know that it
> would help you find where a file was opened.

"/dev/fd" is actually a somewhat portable way of getting this information.  I 
don't think it's part of a standard, but on Linux it's usually a symlink to 
"/proc/self/fd", and it's available on MacOS and most BSDs (based on a hasty 
and completely-not-comprehensive investigation).  But it won't help you find 
out when the FDs were originally opened, no.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Internal counter to debug leaking file descriptors

[Guido van Rossum]

If you wanted to do something like this in the Python stdlib, you'd
have to monkey-patch (with a proxy/wrapper) all places that can open
or close a filedescriptor -- os.open, os.popen, os.close, file
open/close, socket open/close, and probably a bunch more that I've
forgotten. Also some extension modules may open file descriptors
directly through the C interfaces.

I don't know if the Windows libc has some kind of tracking feature for
file descriptors; of course it complicates things by using separate
(numeric) namespaces for sockets and files.

On Linux you can look somewhere in /proc, but I don't know that it
would help you find where a file was opened.

--Guido

On Mon, Aug 30, 2010 at 11:49 PM, anatoly techtonik  wrote:
> Hi,
>
> Is there any kind of internal file descriptor counter that can be
> queried to debug issues with leaking resources?
> It can be used in tests to check that all tests are finish with 0
> opened descriptors.
> It will be very useful while porting Python applications from Unix to
> Windows. Unix is more tolerant to open files and can overwrite them
> and do other nasty things. See the thread from comment #17 -
> https://bugs.edge.launchpad.net/dulwich/+bug/557585/ - there is an
> example of mmap that starts holding file descriptor somewhere long
> before an error occurs. How could one debug this?
>
> Right now I have to use FileMon. It includes information about
> operated filenames, but no info about source code where this happens.
> It will be nice to have some kind of counter with filename information
> inside Python, so that it can be possible to get the full log of
> events without manually messing with external system-specific tools
> like FileMon.
>
> --
> anatoly t.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] versioned .so files for Python 3.2

[Barry Warsaw]

On Aug 31, 2010, at 11:22 AM, M.-A. Lemburg wrote:

>BTW: Wasn't one of the main reasons for having versioned .so files
>the idea to be able to have UCS2 and UCS4 versions installed
>side-by-side ?

Yes.  This isn't an issue for PEP 3149 because it adds a flag to the shared
library file name for wide unicodes.  It's an issue for PEP 384.

-Barry


signature.asc
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 384 status

[Neil Hodgson]

M.-A. Lemburg:

> Is it possible to have multiple versions of the lib C loaded
> on Windows ?

   Yes. It is possible not only to mix C runtimes from different
vendors but different variants from a single vendor.

   Historically, each vendor has shipped their own C runtime
libraries. This was also the case with CP/M and OS/2.

   Many applications can be extended with DLLs and if it were not
possible to load DLLs which use different runtimes then that would
limit which compilers can be used to extend particular applications.
If Microsoft were to stop DLLs compiled with Borland or Intel from
working inside Internet Explorer or Excel then there would be
considerable controversy and likely anti-trust actions.

   Neil
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 384 status

[Antoine Pitrou]

> > So it means that, for example, a FileIO object couldn't be shared
> > between runtimes either? How about a socket object?
> > Do you want to forbid FileIO and socket objects as part of the API?
> 
> Python objects don't have this concern: all methods of FileIO are implemented
> in a single file (fileio.c), linked to a single C runtime.

Ah, right. But you still can call the fileno() method.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 384 status

[Amaury Forgeot d'Arc]

Hi,

2010/8/31 Antoine Pitrou :
> David Cournapeau  wrote:
>> As far as IO is concerned, FILE* is just a special case of a more
>> generic issue, though, so maybe this could be a bit reworded. For
>> example, file descriptor cannot be shared between runtimes either.
>
> Er, really?

Yes, on Windows, file descriptors returned by open() or dup() cannot be shared
between runtimes. What can be safely shared is the underlying "handle",
you can get it with the _get_osfhandle() function.

> So it means that, for example, a FileIO object couldn't be shared
> between runtimes either? How about a socket object?
> Do you want to forbid FileIO and socket objects as part of the API?

Python objects don't have this concern: all methods of FileIO are implemented
in a single file (fileio.c), linked to a single C runtime.

> Again, I propose that FILE* functions are allowed in the API, but their
> use discouraged in the docs (with proper explanations from those who
> know how to word them).

IMO the warnings you'd write there would be similar to the motivations of
PEP 384.

-- 
Amaury Forgeot d'Arc
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 384 status

[M.-A. Lemburg]

Michael Foord wrote:
>  On 30/08/2010 17:35, Barry Warsaw wrote:
>> On Aug 30, 2010, at 10:18 PM, Nick Coghlan wrote:
>>
>>> Since the Linkage section of PEP 384 specifically states the
>>> availability of a generic "python3.dll" that dynamically redirects to
>>> the relevant "python3y.dll" to allow an extension module to run
>>> against any 3.2 or later Python version as a goal of the PEP, I would
>>> say that allowing mixing of C runtimes is definitely one of the PEP's
>>> goals.
>> It should be explicit about that then, and provide detail about why the
>> runtime is relevant to Windows programmers (and probably not relevant in
>> practice for *nix programmers).
> 
> An extension compiled for one version of Python will be linked against
> the version of the C runtime used by that version of Python (if it is
> compiled with the same version of Visual Studio of course).
> 
> If the extension binary is to remain compatible with a later version of
> Python, compiled against a different version of the C runtime, then it
> *must* be possible for multiple C runtimes to be loaded. 

Is it possible to have multiple versions of the lib C loaded
on Windows ?

I know that it is not possible on Linux. Instead, the glibc takes great
care to version all sorts of APIs in the lib to make it possible
to run applications using different versions  within the same runtime
(they include different ABI versions in the same lib).

AFAIK, on Windows, the lib C is using a different approach: up until
the invention of the manifests, they took great care not to break
the APIs in incompatible ways.

With manifests, things are more complicated, since the DLLs now
explicitly reference a particular version of a DLL (down to the
patch level) and if the versions are not installed, the application
won't run. Not sure what effect that has on the way they engineered
the new lib C versions...

This document suggests that it is possible to have an application
use multiple CRTs, but care has to be taken with respect to
things that are initialized by the CRTs (env vars, locales,
handles, etc.):

http://msdn.microsoft.com/en-us/library/abx4dbyh(v=VS.90).aspx

> If the stable
> ABI doesn't allow this then binaries will *still* have to be recompiled
> when we update the version of Visual Studio used to compile Python -
> defeating the purpose of the PEP. Right?

If we keep changing the main compiler version used for creating
Python binaries on Windows: yes. Fortunately, we don't :-)

On Unix this shouldn't be too much of a problem, though, so the
PEP is still valid for those platforms.

> If this is the case then I agree that it should be explicit in the PEP.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 27 2010)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2010-08-19: Released mxODBC 3.1.0  http://python.egenix.com/
2010-09-15: DZUG Tagung, Dresden, Germany  18 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] versioned .so files for Python 3.2

[M.-A. Lemburg]

Ronald Oussoren wrote:
> 
> On 28 Aug, 2010, at 12:29, Martin v. Löwis wrote:
>>
>> - wide-unicode: this is a tricky one. I'm tempted to say that the
>>  stable ABI should always use a Py_UNICODE that matches the platform's
>>  wchar_t. Alternative proposals are welcome.
> 
> Sizeof(wchar_t) is 4 on OSX, but the Apple frameworks use a 16-bit type to 
> represent unicode codepoints (UniChar).  Current builds on OSX use a 16-bit 
> unicode type which makes it pretty cheap to convert strings from Python to a 
> C array of UniChar.
> 
> I'm therefore -1 on switching to a wide unicode build on OSX. 

-1 on always using wchar_t as well. Python's default is UCS2
and the stable ABI should not change that.

I also think that this information is not relevant for the
stable ABI: Extensions that want to stick to the stable ABI
should really not have to know whether Py_UNICODE maps to
wchar_t or not. If they want to interface to a local whcar_t
type they can use the conversion APIs we have for that in the
Unicode API: PyUnicode_FromWideChar() and PyUnicode_AsWideChar().

BTW: Wasn't one of the main reasons for having versioned .so files
the idea to be able to have UCS2 and UCS4 versions installed
side-by-side ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 27 2010)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2010-08-19: Released mxODBC 3.1.0  http://python.egenix.com/
2010-09-15: DZUG Tagung, Dresden, Germany  18 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 384 status

[Antoine Pitrou]

On Tue, 31 Aug 2010 11:12:17 +0900
David Cournapeau  wrote:
> 
> > Hmm... that last point is a bit of any issue actually, since it also
> > flows the other way (changes made via the locale module won't be
> > visible to any extension modules using a different C runtime). So I
> > suspect mixing C runtimes is still going to come with the caveat of
> > potential locale related glitches.
> 
> As far as IO is concerned, FILE* is just a special case of a more
> generic issue, though, so maybe this could be a bit reworded. For
> example, file descriptor cannot be shared between runtimes either.

Er, really?
So it means that, for example, a FileIO object couldn't be shared
between runtimes either? How about a socket object?
Do you want to forbid FileIO and socket objects as part of the API?

Again, I propose that FILE* functions are allowed in the API, but their
use discouraged in the docs (with proper explanations from those who
know how to word them).

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] versioned .so files for Python 3.2

[Ronald Oussoren]

On 28 Aug, 2010, at 12:29, Martin v. Löwis wrote:
> 
> - wide-unicode: this is a tricky one. I'm tempted to say that the
>  stable ABI should always use a Py_UNICODE that matches the platform's
>  wchar_t. Alternative proposals are welcome.

Sizeof(wchar_t) is 4 on OSX, but the Apple frameworks use a 16-bit type to 
represent unicode codepoints (UniChar).  Current builds on OSX use a 16-bit 
unicode type which makes it pretty cheap to convert strings from Python to a C 
array of UniChar.

I'm therefore -1 on switching to a wide unicode build on OSX. 

Ronald



smime.p7s
Description: S/MIME cryptographic signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com