[Python-ideas] Re: How to prevent shared memory from being corrupted ?

I understand that I won’t need locks with immutable objects at some level, but I don’t understand how they can be used to synchronise shared memory segments. For every change in an immutable object, a copy is created which will have a different address. Now, for processes to use this updated obj

[Python-ideas] Re: Default behavior for random.sample when no k

I submitted a patch now, but Serhiy showed me that it's already been proposed before, and rejected by Raymond Hettinger and Terry Reedy in issues 26393 and 27964. On Sun, Aug 2, 2020 at 8:05 AM Steven D'Aprano wrote: > On Sat, Aug 01, 2020 at 08:54:16PM +0300, Ram Rachum wrote: > > > When writin

[Python-ideas] Re: Access (ordered) dict by index; insert slice

On Sat, 1 Aug 2020 at 19:34, Christopher Barker wrote: > I would think the goal here would be to re-order once in a while to remove > the holes. But that would take time, of course, so you wouldn't want to do > it on every deletion. But when? > You should have a separate process that does this,

[Python-ideas] Filtered wildcard imports?

Hi all, yet another (possibly bad?) idea from day-to-day work ... I occasionally need to import a lot of "stuff" from certain modules. The "stuff" is usually following a pattern. E.g. I have modules that (mostly) collect special exception classes and I usually need them all in one push - but noth

[Python-ideas] Re: Access (ordered) dict by index; insert slice

On Sat, 1 Aug 2020 at 20:25, Stestagg wrote: > I wrote some (better than the previously shared) benchmarks for this > change a while ago. > I think that you could speed up the algorithm if you check if dict->ma_keys->dk_lookup == lookdict_unicode_nodummy. If so, the dict is a combined dict with

[Python-ideas] Re: Filtered wildcard imports?

Why not just import the module they're defined in and then just do attribute access syntax? from mod.modb.dtx import somemod somemod.SomeError ___ Python-ideas mailing list -- [email protected] To unsubscribe send an email to python-ideas-le...@py

[Python-ideas] Re: How to prevent shared memory from being corrupted ?

It's best to avoid those synchronization barriers if possible. If you have all of the data in SHM (RAM) on one node, and you need to notify processes / wait for other workers to be available to perform a task that requires that data, you need a method for IPC: a queue, channel subscriptions, a sou

[Python-ideas] Re: Access (ordered) dict by index; insert slice

https://github.com/python/cpython/blob/master/Objects/odictobject.c : """ In order to preserve O(1) performance for node removal (finding nodes), we must do better than just looping through the linked-list. Here are options we've considered: 1. use a second dict to map keys to nodes (a la the pur

[Python-ideas] Re: How to prevent shared memory from being corrupted ?

There's also the possibility to use shared ctypes: https://docs.python.org/3/library/multiprocessing.html#shared-ctypes-objects Operations like += which involve a read and write are not atomic. So if, > for instance, you want to atomically increment a shared value it is > insufficient to just do >

[Python-ideas] Re: How to prevent shared memory from being corrupted ?

sharedctypes can only be used with related processes. There is no way that you can pass a sharedctype to an unrelated process. multiprocessing.shared_memory was created to handle this i.e. allow usage of shared memory IPC across unrelated processes. > On 02-Aug-2020, at 9:42 PM, Marco Sulla wr

[Python-ideas] Re: How to prevent shared memory from being corrupted ?

On Sat, 1 Aug 2020 at 22:42, Eric V. Smith > wrote: While they're immutable at the Python level, strings (and all other objects) are mutated at the C level, due to reference count updates. You need to consider this if you're sharing objects without loc

[Python-ideas] Re: How to prevent shared memory from being corrupted ?

On 8/2/2020 12:20 PM, Eric V. Smith wrote: On Sat, 1 Aug 2020 at 22:42, Eric V. Smith > wrote: While they're immutable at the Python level, strings (and all other objects) are mutated at the C level, due to reference count updates. You need to consid

[Python-ideas] Re: How to prevent shared memory from being corrupted ?

How is this a different problem than the cache coherency problem? https://en.wikipedia.org/wiki/Cache_coherence Perhaps that's an unhelpful abstraction? This hasn't gone anywhere: https://en.wikipedia.org/wiki/Distributed_shared_memory#Directory_memory_coherence Here's a great comparison chart f

[Python-ideas] Re: Filtered wildcard imports?

On Sun, Aug 02, 2020 at 03:36:44PM +0200, Sebastian M. Ernst wrote: > Hi all, > > yet another (possibly bad?) idea from day-to-day work ... > > I occasionally need to import a lot of "stuff" from certain modules. The > "stuff" is usually following a pattern. E.g. I have modules that > (mostly) co

[Python-ideas] Re: Default behavior for random.sample when no k

Steven D'Aprano wrote: > > This is easily solved with a three-line helper: > def shuffled(iterable): ... > I have implemented this probably a half a dozen times, and I expect > others have too. FWIW, we've already documented a clean way to do it, https://docs.python.org/3/library/random.html#ra