Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-20 Thread Nick Coghlan
On 20 June 2013 15:37, Victor Stinner wrote: > Le jeudi 20 juin 2013, Nick Coghlan a écrit : >> >> > Is PyMemMappingAllocator complete enough for your usage at CCP Games? >> >> Can we go back to calling this the "Arena" allocator? Or at least >> "Mapped"? When I see "Mapping" in the context of Pyt

Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-20 Thread Kristján Valur Jónsson
[mailto:victor.stin...@gmail.com] > Sent: 19. júní 2013 15:59 > To: Kristján Valur Jónsson > Cc: Python Dev > Subject: Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python > memory allocators > > Is PyMemMappingAllocator complete enough for your usage at CCP Games?

Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-20 Thread Victor Stinner
> * Add new GIL-free (no need to hold the GIL) memory allocator functions: > > - ``void* PyMem_RawMalloc(size_t size)`` > - ``void* PyMem_RawRealloc(void *ptr, size_t new_size)`` > - ``void PyMem_RawFree(void *ptr)`` > - the behaviour of requesting zero bytes is not defined: return *NULL* >

Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-19 Thread Victor Stinner
Le jeudi 20 juin 2013, Nick Coghlan a écrit : > > > Is PyMemMappingAllocator complete enough for your usage at CCP Games? > > Can we go back to calling this the "Arena" allocator? Or at least > "Mapped"? When I see "Mapping" in the context of Python I think of the > container API, not a memory allo

Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-19 Thread Nick Coghlan
On 20 Jun 2013 02:03, "Victor Stinner" wrote: > > 2013/6/19 Kristján Valur Jónsson : > > Oh, it should be public, in my opinion. > > Ok. And do you think that the PyMemMappingAllocator structure is > complete, or that we should add something to be future-proof? At > least, PyMemMappingAllocator is

Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-19 Thread Victor Stinner
PyMem_RawAlloc()/Realloc/Free should be part of the stable ABI. I agree that all other new fumctions ans structures should not. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http

Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-19 Thread Antoine Pitrou
On Wed, 19 Jun 2013 17:49:02 +0200 Victor Stinner wrote: > 2013/6/19 Antoine Pitrou : > > On Wed, 19 Jun 2013 17:24:21 +0200 > >> >> Drawback: the caller has to check if the result is 0, or handle the > >> >> error. > >> > > >> > Or you can just call Py_FatalError() if the domain is invalid. > >>

Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-19 Thread Terry Reedy
On 6/19/2013 11:24 AM, Victor Stinner wrote: 2013/6/19 Antoine Pitrou : Le Tue, 18 Jun 2013 22:40:49 +0200, Victor Stinner a écrit : Only one get/set function for block allocators -- Replace the 6 functions: * ``void PyMem_GetRawAllocator(PyMemBl

Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-19 Thread Victor Stinner
2013/6/19 Kristján Valur Jónsson : > Oh, it should be public, in my opinion. Ok. And do you think that the PyMemMappingAllocator structure is complete, or that we should add something to be future-proof? At least, PyMemMappingAllocator is enough for pymalloc usage :-) Is PyMemMappingAllocator com

Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-19 Thread Kristján Valur Jónsson
hon Dev > Subject: Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python > memory allocators > > typedef struct { > /* user context passed as the first argument >to the 2 functions */ > void *ctx; > > /* allocate a memo

Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-19 Thread Victor Stinner
2013/6/19 Antoine Pitrou : > On Wed, 19 Jun 2013 17:24:21 +0200 >> >> Drawback: the caller has to check if the result is 0, or handle the >> >> error. >> > >> > Or you can just call Py_FatalError() if the domain is invalid. >> >> I don't like Py_FatalError(), especially when Python is embedded. It'

Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-19 Thread Antoine Pitrou
On Wed, 19 Jun 2013 17:24:21 +0200 Victor Stinner wrote: > > For the "track memory usage" use case, it is important to track memory > allocated in external libraries to have accurate reports, because > these allocations may be huge. [...] > Not in main(). The Python expat and zlib modules call di

Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-19 Thread Kristján Valur Jónsson
mailto:python-dev- > bounces+kristjan=ccpgames@python.org] On Behalf Of Scott Dial > Sent: 19. júní 2013 04:34 > To: ncogh...@gmail.com > Cc: Python-Dev@python.org > Subject: Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python > memory allocators > > On 6/18/2013 11:32

Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-19 Thread Victor Stinner
2013/6/19 Antoine Pitrou : > Le Tue, 18 Jun 2013 22:40:49 +0200, > Victor Stinner a écrit : >> >> Other changes >> - >> > [...] >> >> * Configure external libraries like zlib or OpenSSL to allocate memory >> using ``PyMem_RawMalloc()`` > > Why so, and is it done by default? (Oh, I r

Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-19 Thread Antoine Pitrou
Le Tue, 18 Jun 2013 22:40:49 +0200, Victor Stinner a écrit : > > Other changes > - > [...] > > * Configure external libraries like zlib or OpenSSL to allocate memory > using ``PyMem_RawMalloc()`` Why so, and is it done by default? > Only one get/set function for block allocators

Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-18 Thread Victor Stinner
Le mercredi 19 juin 2013, Scott Dial a écrit : > On 6/18/2013 4:40 PM, Victor Stinner wrote: > > No context argument > > I think there is a lack of justification for the extra argument, and the > extra argument is not free. The typical use-case for doing this > continuation-passing style is when t

Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-18 Thread Scott Dial
On 6/18/2013 11:32 PM, Nick Coghlan wrote: > Agreed more of that rationale needs to be moved from the issue tracker > into the PEP, though. Thanks for the clarification. I hadn't read the issue tracker at all. On it's face value, I didn't see what purpose it served, but having read Kristján's comm

Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-18 Thread Nick Coghlan
On 19 June 2013 09:23, Scott Dial wrote: > On 6/18/2013 4:40 PM, Victor Stinner wrote: >> No context argument >> --- >> >> Simplify the signature of allocator functions, remove the context >> argument: >> >> * ``void* malloc(size_t size)`` >> * ``void* realloc(void *ptr, size_t new

Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-18 Thread Scott Dial
On 6/18/2013 4:40 PM, Victor Stinner wrote: > No context argument > --- > > Simplify the signature of allocator functions, remove the context > argument: > > * ``void* malloc(size_t size)`` > * ``void* realloc(void *ptr, size_t new_size)`` > * ``void free(void *ptr)`` > > It is l

Re: [Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-18 Thread Victor Stinner
typedef struct { /* user context passed as the first argument to the 2 functions */ void *ctx; /* allocate a memory mapping */ void* (*alloc) (void *ctx, size_t size); /* release a memory mapping */ void (*free) (void *ctx, void *ptr,

[Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

2013-06-18 Thread Victor Stinner
If you prefer the HTML version: http://www.python.org/dev/peps/pep-0445/ PEP: 445 Title: Add new APIs to customize Python memory allocators Version: $Revision$ Last-Modified: $Date$ Author: Victor Stinner Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 15-june-2013 Python-Ve