Parent module not loaded error
I have a pretty strange error that I can't figure out the cause off. This is in a Django app. I am using berkelydb, with secondary databases for indexing. The secondary databases are associated with a callback that uses cPickle to serialize index values. The problem is that cPickle.dumps(value) fails when I run it through mod_wsgi or mod_python on the deployment server (Ubuntu Linux 8.10), but works fine when when I use Djangos runserver on the same computer. Also, on my own computer (Ubuntu Linux 8.10), it works fine through mod_python. Each time I call on cPickle.dumps(value) I get: SystemError(Parent module 'd4' not loaded,) Using sys.exc_info() I get no type or traceback, just that exception. The callstack as far as I can figure is: django view bsddb.put indexer callback cPickle.dumps cPickle.loads works fine within the callback, I can also use cPickle.dumps() outside the indexer callback. But inside the callback, I need to use normal pickle instead. I just want to know why. I am assuming it has something to do with the fact that I go from python to C (bsddb) back to python (indexer callback) and back to C (cPickle). It is still strange then that cPickle.loads() work, but not cPickle.dumps(). I am also only storing integer values when I get this error, so no fancy list, dictionaries or objects. -- http://mail.python.org/mailman/listinfo/python-list
Re: Parent module not loaded error
On Jan 14, 9:20 pm, Ståle Undheim staa...@gmail.com wrote: I have a pretty strange error that I can't figure out the cause off. This is in a Django app. I am using berkelydb, with secondary databases for indexing. The secondary databases are associated with a callback that uses cPickle to serialize index values. The problem is that cPickle.dumps(value) fails when I run it throughmod_wsgior mod_python on the deployment server (Ubuntu Linux 8.10), but works fine when when I use Djangos runserver on the same computer. Also, on my own computer (Ubuntu Linux 8.10), it works fine through mod_python. Each time I call on cPickle.dumps(value) I get: SystemError(Parent module 'd4' not loaded,) Using sys.exc_info() I get no type or traceback, just that exception. The callstack as far as I can figure is: django view bsddb.put indexer callback cPickle.dumps cPickle.loads works fine within the callback, I can also use cPickle.dumps() outside the indexer callback. But inside the callback, I need to use normal pickle instead. I just want to know why. I am assuming it has something to do with the fact that I go from python to C (bsddb) back to python (indexer callback) and back to C (cPickle). It is still strange then that cPickle.loads() work, but not cPickle.dumps(). I am also only storing integer values when I get this error, so no fancy list, dictionaries or objects. Where is module 'd4'? Is that one of yours defined inside of your Django project? One of the problems with Django runserver is that the parent of the site directory and the site directory are effectively starting points for searching for modules, even though neither is explicitly listed in sys.path. The parent is used because runserver does momentarily add parent to sys.path to import site package root, after which it removes directory from sys.path. The site directory itself is used because for runserver that is the current working directory and imports look there automatically. End result is that modules inside of site can be imported either via site package root, or direct. For example, if d4.py was at same level as settings.py file and site was called mysite, you could import it as mysite.d4 or just d4. This is bad because means you could end up with two copies of a module imported under the different name references. If they way the pickle was done in application was such that reference was via d4, and then when you deployed to mod_python or mod_wsgi you only added parent directory of site to Python path, then it will not be able to find d4, since at that point would only be found as mysite.d4. If this is the case, add the site directory to Python path by setting PythonPath for mod_python or sys.path in WSGI script file for mod_wsgi. Better still, fix up your code so module references in code, ie., urls.py, or anywhere else, are always done by site package root reference and not direct. Ie., such that it uses mysite.d4 and not just d4. Would that seem about right? Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: Parent module not loaded error
On Jan 14, 11:31 am, Graham Dumpleton graham.dumple...@gmail.com wrote: On Jan 14, 9:20 pm, Ståle Undheim staa...@gmail.com wrote: I have a pretty strange error that I can't figure out the cause off. This is in a Django app. I am using berkelydb, with secondary databases for indexing. The secondary databases are associated with a callback that uses cPickle to serialize index values. The problem is that cPickle.dumps(value) fails when I run it throughmod_wsgior mod_python on the deployment server (Ubuntu Linux 8.10), but works fine when when I use Djangos runserver on the same computer. Also, on my own computer (Ubuntu Linux 8.10), it works fine through mod_python. Each time I call on cPickle.dumps(value) I get: SystemError(Parent module 'd4' not loaded,) Using sys.exc_info() I get no type or traceback, just that exception. The callstack as far as I can figure is: django view bsddb.put indexer callback cPickle.dumps cPickle.loads works fine within the callback, I can also use cPickle.dumps() outside the indexer callback. But inside the callback, I need to use normal pickle instead. I just want to know why. I am assuming it has something to do with the fact that I go from python to C (bsddb) back to python (indexer callback) and back to C (cPickle). It is still strange then that cPickle.loads() work, but not cPickle.dumps(). I am also only storing integer values when I get this error, so no fancy list, dictionaries or objects. Where is module 'd4'? Is that one of yours defined inside of your Django project? d4 is the root project name. This message occurs in d4/views.py. Here is how I set up my wsgi.py - which is in the d4/ folder: sys.path.append(path.join(path.dirname(__file__), '..')) os.environ['DJANGO_SETTINGS_MODULE'] = 'd4.settings' application = django.core.handlers.wsgi.WSGIHandler() One of the problems with Django runserver is that the parent of the site directory and the site directory are effectively starting points for searching for modules, even though neither is explicitly listed in sys.path. The parent is used because runserver does momentarily add parent to sys.path to import site package root, after which it removes directory from sys.path. The site directory itself is used because for runserver that is the current working directory and imports look there automatically. End result is that modules inside of site can be imported either via site package root, or direct. For example, if d4.py was at same level as settings.py file and site was called mysite, you could import it as mysite.d4 or just d4. This is bad because means you could end up with two copies of a module imported under the different name references. Everywhere in the code, the imports refer to d4.something.something. All urls.py files refer to the full package names. The test code where I am running this is actually quit minimal. If they way the pickle was done in application was such that reference was via d4, and then when you deployed to mod_python or mod_wsgi you only added parent directory of site to Python path, then it will not be able to find d4, since at that point would only be found as mysite.d4. I use import cPickle as pickle at the top off my views file. So I don't reference imports from other files. If this is the case, add the site directory to Python path by setting PythonPath for mod_python or sys.path in WSGI script file for mod_wsgi. Better still, fix up your code so module references in code, ie., urls.py, or anywhere else, are always done by site package root reference and not direct. Ie., such that it uses mysite.d4 and not just d4. Would that seem about right? Unfortunatly, at least to me, it seems that all code is allready correctly using packagenames. The views.py file where I am testing this is a simple file that doesn't depend on anything else in the project. It only imports a set of system modules for testing and info. The only dependency is really the urls.py file, which does refer to all the views as d4.views.somemethod. -- http://mail.python.org/mailman/listinfo/python-list
Re: Parent module not loaded error
On Jan 14, 9:41 pm, Ståle Undheim staa...@gmail.com wrote: On Jan 14, 11:31 am, Graham Dumpleton graham.dumple...@gmail.com wrote: On Jan 14, 9:20 pm, Ståle Undheim staa...@gmail.com wrote: I have a pretty strange error that I can't figure out the cause off. This is in a Django app. I am using berkelydb, with secondary databases for indexing. The secondary databases are associated with a callback that uses cPickle to serialize index values. The problem is that cPickle.dumps(value) fails when I run it throughmod_wsgior mod_python on the deployment server (Ubuntu Linux 8.10), but works fine when when I use Djangos runserver on the same computer. Also, on my own computer (Ubuntu Linux 8.10), it works fine through mod_python. Each time I call on cPickle.dumps(value) I get: SystemError(Parent module 'd4' not loaded,) Using sys.exc_info() I get no type or traceback, just that exception. The callstack as far as I can figure is: django view bsddb.put indexer callback cPickle.dumps cPickle.loads works fine within the callback, I can also use cPickle.dumps() outside the indexer callback. But inside the callback, I need to use normal pickle instead. I just want to know why. I am assuming it has something to do with the fact that I go from python to C (bsddb) back to python (indexer callback) and back to C (cPickle). It is still strange then that cPickle.loads() work, but not cPickle.dumps(). I am also only storing integer values when I get this error, so no fancy list, dictionaries or objects. Where is module 'd4'? Is that one of yours defined inside of your Django project? d4 is the root project name. This message occurs in d4/views.py. Here is how I set up my wsgi.py - which is in the d4/ folder: sys.path.append(path.join(path.dirname(__file__), '..')) os.environ['DJANGO_SETTINGS_MODULE'] = 'd4.settings' application = django.core.handlers.wsgi.WSGIHandler() One of the problems with Django runserver is that the parent of the site directory and the site directory are effectively starting points for searching for modules, even though neither is explicitly listed in sys.path. The parent is used because runserver does momentarily add parent to sys.path to import site package root, after which it removes directory from sys.path. The site directory itself is used because for runserver that is the current working directory and imports look there automatically. End result is that modules inside of site can be imported either via site package root, or direct. For example, if d4.py was at same level as settings.py file and site was called mysite, you could import it as mysite.d4 or just d4. This is bad because means you could end up with two copies of a module imported under the different name references. Everywhere in the code, the imports refer to d4.something.something. All urls.py files refer to the full package names. The test code where I am running this is actually quit minimal. If they way the pickle was done in application was such that reference was via d4, and then when you deployed to mod_python or mod_wsgi you only added parent directory of site to Python path, then it will not be able to find d4, since at that point would only be found as mysite.d4. I use import cPickle as pickle at the top off my views file. So I don't reference imports from other files. If this is the case, add the site directory to Python path by setting PythonPath for mod_python or sys.path in WSGI script file for mod_wsgi. Better still, fix up your code so module references in code, ie., urls.py, or anywhere else, are always done by site package root reference and not direct. Ie., such that it uses mysite.d4 and not just d4. Would that seem about right? Unfortunatly, at least to me, it seems that all code is allready correctly using packagenames. The views.py file where I am testing this is a simple file that doesn't depend on anything else in the project. It only imports a set of system modules for testing and info. The only dependency is really the urls.py file, which does refer to all the views as d4.views.somemethod. As a test, just prior to where unpickle is done, do: import sys import d4 print sys.stderr, d4.__name__, dr.__name__ print sys.stderr, d4.__file__, dr.__file__ This will just confirm that manual import works and what values of those attributes are. The messages will end up in Apache error log for that virtual host or main server error log as appropriate. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: Parent module not loaded error
On Jan 14, 11:48 am, Graham Dumpleton graham.dumple...@gmail.com wrote: As a test, just prior to where unpickle is done, do: import sys import d4 print sys.stderr, d4.__name__, dr.__name__ print sys.stderr, d4.__file__, dr.__file__ This will just confirm that manual import works and what values of those attributes are. The messages will end up in Apache error log for that virtual host or main server error log as appropriate. d4.__name__ d4 d4.__file__ /home/petromarker/django-apps/d4/src/d4/../d4/__init__.pyc It's the same both before and after the exception. -- http://mail.python.org/mailman/listinfo/python-list
