[Antoine] > Well, sending oceans of tiny integers will also incur many system calls > and additional synchronization costs, since sending data on a > multiprocessing Queue has to acquire a semaphore. So it generally > sounds like a bad idea, IMHO. > > That said, I agree with: >> Since pickle intends to be "the" Python serialization format, it >> really should try to be friendly for all plausible uses. > > I simply don't think adding a fixed 8-byte overhead is actually > annoying. It's less than the PyObject overhead in 64-bit mode...
A long-running process can legitimately put billions of items on work queues, far more than could ever fit in RAM simultaneously. Comparing this to PyObject overhead makes no sense to me. Neither does the line of argument "there are several kinds of overheads, so making this overhead worse too doesn't matter". When possible, we should strive not to add overheads that don't repay their costs. For small pickles, an 8-byte size field doesn't appear to buy anything. But I appreciate that it costs implementation effort to avoid producing it in these cases. _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
