[issue26827] PyObject *PyInit_myextention -> PyMODINIT_FUNC PyInit_myextention

[Herbert]

Herbert added the comment:

The exact error was:

ImportError: dynamic module does not define init function (PyInit_myextension)

--

___
Python tracker 
<http://bugs.python.org/issue26827>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26827] PyObject *PyInit_myextention -> PyMODINIT_FUNC PyInit_myextention

[Herbert]

New submission from Herbert:

I think 

PyObject *PyInit_myextention(void)

should be

PyMODINIT_FUNC PyInit_myextention(void)

on https://docs.python.org/3/howto/cporting.html#module-initialization-and-state

It didn't work for me until I replaced this with a message in the about 
'undefined PyInit_myextention'. However, when I used nm to inspect the .so 
object file, I fond the PyInit_myextention (but probably with the wrong return 
type). Moreover, whenever I would remove the same .so importing resulted in a 
different error complaining that the module does not exist (strongly suggesting 
that I did not mix up .so files).

Good luck!

--
assignee: docs@python
components: Documentation
messages: 264005
nosy: docs@python, prinsherbert
priority: normal
severity: normal
status: open
title: PyObject *PyInit_myextention -> PyMODINIT_FUNC PyInit_myextention
versions: Python 3.2

___
Python tracker 
<http://bugs.python.org/issue26827>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25465] Pickle uses O(n) memory overhead

[Herbert]

Herbert added the comment:

It may be fair to note that I have no swap installed on one of the machines, 
just 16GB of RAM, on which the 'crash' happens. Hence I'm not sure how this 
affects paging, I would think there is no paging if there is no swap.

I can verify that the machine is 'stuck' for more than just several minutes (at 
least 30 minutes), nevertheless cannot confirm if this is due to the desktop 
environment or actually the kernel. I would agree to verify this first when I 
have access to the specific machines again.

Thank you for your input so far!

--

___
Python tracker 
<http://bugs.python.org/issue25465>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25465] Pickle uses O(n) memory overhead

[Herbert]

Herbert added the comment:

Hi Eric,

I would assume that for the right range-parameter (in my case 30 * 1000 ** 2), 
which just fits in memory, your system would also crash after a pickle.dump. 
That is, I had this behavior on two of my machine both running a Ubuntu setup 
though.

Nevertheless, if you give me some time I'm happy to check my dmesg and any log 
you wish. I find it strange that sometimes I get a MemoryError when I run out 
of memory (in particular when using numpy), and sometimes the system crashes 
(in particular when using other python-stuff). Therefore I don't think this is 
pickle-specific, or even if this is a bug instead of a 'feature'.

--

___
Python tracker 
<http://bugs.python.org/issue25465>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25465] Pickle uses O(n) memory overhead

[Herbert]

Herbert added the comment:

That sound reasonable regarding why O(n), but it does not explain why linux 
crashes (I've seen this on two ubuntu systems)if pickle runs out of memory.

--

___
Python tracker 
<http://bugs.python.org/issue25465>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25465] Pickle uses O(n) memory overhead

[Herbert]

Changes by Herbert :


--
type:  -> performance
versions: +Python 3.4

___
Python tracker 
<http://bugs.python.org/issue25465>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25465] Pickle uses O(n) memory overhead

[Herbert]

New submission from Herbert:

I very often want to use pickle to store huge objects, such that I do not need 
to recalculate them again.

However, I noticed that pickle uses O(n) (for n the object size in memory) 
amount of memory. That is, using python 3:

data = {'%06d' % i: i for i in range(30 * 1000 ** 2)}
# data consumes a lot of my 8GB ram
import pickle
with open('dict-database.p3', 'wb') as f: pickle.dump(data, f)
# I have to kill the process, in order to not overflow in memory. If I 
don't, the OS crashes. IMHO the OS should never crash due to python.

I don't think pickle should require a O(n) memory overhead.

--
messages: 253371
nosy: prinsherbert
priority: normal
severity: normal
status: open
title: Pickle uses O(n) memory overhead

___
Python tracker 
<http://bugs.python.org/issue25465>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com