On 20 June 2013 15:37, Victor Stinner <victor.stin...@gmail.com> wrote: > Le jeudi 20 juin 2013, Nick Coghlan a écrit : >> >> > Is PyMemMappingAllocator complete enough for your usage at CCP Games? >> >> Can we go back to calling this the "Arena" allocator? Or at least >> "Mapped"? When I see "Mapping" in the context of Python I think of the >> container API, not a memory allocation API. > > This function is written to be able to use mmap() and VirtualAlloc(). There > is no Python function to use directly this allocator yet, but I chose > "memory mapping" name because it is very different than the heap and it may > be useful for other functions than pymalloc. > > If I change the name, it would be called PyObject_SetArenaAllocator() with a > PyObjectArenaAllocator structure. I'm not sure that PyMemMappingAllocator > API is future-proof, so I'm fine to call it "arena" again.
Yeah, I think making that API specifically about pymalloc is a good idea. It also makes it clearer that if you're bypassing pymalloc entirely (by replacing the object allocators), then you shouldn't need to worry about those. >> > I hope that the PEP 445 is flexible enough to allow you to decide >> > which functions are hooked and replaced, and which functions will be >> > leaved unchanged. That's why I'm not in favor of the "Make >> > PyMem_Malloc() reuse PyMem_RawMalloc() by default" alternative. >> >> It's also why I'm in favour of the "domain" API rather than separate >> functions. >> >> 1. In the initial iteration, just have the three basic domains (raw, >> interpreter, objects). Replacing allocators for third party libraries is the >> responsibility of embedding applications. >> >> 2. In a later iteration, add "PyMem_AddDomain" and "PyMem_GetDomains" APIs >> so that extension modules can register new domains for wrapped libraries. >> Replacing allocators is still the responsibility of embedding applications, >> but there's a consistent API to do it. >> >> (Alternatively, we could do both now) > > How would you use an allocator of a new domain? PyMemBlockAllocator > structure is not convinient, and if Py_GetAllocator() only once, you may > loose a hook installed later. Say that, for the current PEP, we assume we configure standard library extension modules to use the PyMem or PyMem_Raw APIs (depending on GIL usage), thus allowing those to be redirected automatically to an externally configured allocator when the new PEP 445 APIs are used. The notion I had an mind as a possible future change is that extension modules could register a "set_allocator" callback with the interpreter so they will be automatically notified if the allocator they're interested in changes. However, I also realised that this would actually be independent of the APIs in the current PEP. You could do something like: typedef void (*PyMem_AllocatorSetter)(PyMemAllocator *allocator); void PyMem_AddExternalAllocator( PyMemAllocatorDomain domain, PyMemAllocatorSetter set_allocator ); Then, whenever the allocator for the specified domain was changed, the "set_allocator" callback would be invoked to set the allocator in the extension module as well. The setter would also be called immediately on registration, using the currently defined allocator. We don't have to do this right away (and we should give the basic API a chance to establish itself first). I just like it as something that the "memory domain" model may allow us to pursue in the future. (That said, we may end up wanting something like this internally anyway, even just for the standard library extension modules that do memory allocations) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com