On 18/05/07, Daniel J. Popowich <[EMAIL PROTECTED]> wrote:
Graham Dumpleton writes: > And that is what the documentation I pointed you at states: > > """Note that with the new module importer, as directories associated > with Python*Handler directives are no longer being added automatically > to sys.path and they are instead used directly by the module importer > only when required, some existing code which expected to be able to > import modules in the handler root directory from a module in a > subdirectory may no longer work. In these situations it will be > necessary to set the mod_python module importer path to include '~' or > list '~' in the __mp_path__ attribute of the module performing the > import.""" OK, I think it's slowly sinking in...when I put the ~ in, it didn't affect sys.path, so my import still wasn't going to work. Right?
Correct, you were only changing the path related to modules managed by the new module importer and not those handled by the standard Python importer. Adding "~" like that would also only have been needed if in a subdirectory of the handler root directory an import was trying to be done with either 'import' or 'apache.import_module()' for a module in the handler root directory. I had assumed in suggesting that, that you had already converted mpservlets to use the new module importer and were triggering that specific case.
So, how the heck do I import the modules?
In just about all cases exactly how you do it now. Although, where you were using your own module importing and caching system for working under mod_python, you would need to adapt it to use the new importer.
Do I have to use apache.import_module() now?
No. In fact there are now cases where before you had to use apache.import_module() because of bugs but you now can use just 'import' and it will also work, thus allowing one to create sets of modules that can also be used safely outside of mod_python in a test framework. That you have an existing module importing and caching system of your own is a special case but only the module importer code in moservlets should need to change, not anything else.
But I don't want to!
As far as using it in mpservlets itself, that would be a pity really as one of the main reasons that support for importing files by full path with an arbitrary extension was added was so that mpservlets could make use of the new importer rather than use its own, and thus benefit from some of the features the new module importer provided such as root modules being imported automatically when a child was changed thereby avoiding the need to restart Apache after anything but simple changes. Thus, in the tutorial example, I could adjust _SitePage and thereby notionally change the appearance of the whole site without a restart, something that couldn't be avoided with how mpservlets worked with previous versions of mod_python.
> If one wants to be precise, the addition of the handler directory to > sys.path was actually done by the top level mod_python dispatch code > rather than the module importer itself. And this I don't understand at all. If this "feature" was in the old system, why would reimplementing an api function change this behaviour in the dispatch code? I would consider this a bug.
That it did add the handler directory to sys.path previously is what caused a lot of the bugs documented in the page I referred to in the first place. There was no choice but to change it else you couldn't fix the existing bugs. You may have avoided the bugs by using your own module importer, but the bugs still affected everyone else. The whole point of the changes which were made was to draw a well defined line between the code modules used in the web application and which reside in the document tree, or other specially specified areas by way of mod_python module importer path, and the standard Python sys.path locations. That there was no distinction before is what caused all the issues with same named modules, loading of incorrect modules, reloading of modules which shouldn't be reloaded etc etc.
I would also say this is WAY too much functionality change for a minor release update.
I did suggest that the release be called 4.0, but since there was a backwards compatibility switch so people could reenable old behaviour for module importer while they update their code the consensus was that 3.3 was okay. Also, the whole thing was intended to be as compatible as possible. In the end there was only a few cases where people would actually have to make changes, the main one being where people were relying on handler root direct being in sys.path so imports from non mod_python managed modules could find a module in that directory.
> The other important bit is: > > """As a new feature in mod_python 3.3, when using the standard Python > 'import' statement to import a module, if the import is being done > from a module which was previously imported by the mod_python module > importer, it is equivalent to having called apache.import_module() > directly.""" > > It is because of this that the directory doesn't need to be in > sys.path, as under the covers, when 'import' is being used by modules > imported by the mod_python module importer directly in some way, then > 'import' is equivalent to having used 'apache.import_module()' and > therefore it looks in places related to mod_python including stuff in > the separate mod_python importer path. And I would call this a bug, too. Why should using mod_python change the behaviour of the import builtin? What if I'm including a package that includes a package that includes a package? I'd have no idea what code is doing the work.
Once a module is imported which isn't being managed by mod_python, then the mapping of 'import' to 'apache.import_module()' doesn't get triggered. Thus, it applies only to stuff in the web application and not to anything on sys.path. FWIW, overriding 'import' in ways like this is not uncommon in Python as this is how importing stuff from Zip files and unexpanded Python eggs works. The stuff works and it steps out of the way when it shouldn't be applied. I remember when I first brought up this idea of 'import' working in magic ways so things would just work as one expected, that there were at least one person who thought it was a silly idea. In the end I had to give up on even trying to explain it because it would have taken me months just to get people to understand what I had in mind. Instead, I just wrote the code and I believe most would have to say the result in certainly better than what was there before and that code now just works as how one would expect it to work when one uses 'import' whereas it didn't always before.
This is getting way too obfuscated for my simple ways.
All I can suggest is you try it by starting with the changes to mpservlets as I described and then simply see if your existing applications which use mpservlets still work. Since the release we have only had a few people even be slightly affected by the differences, plus all the complaints we used to get about problems with the old importer have pretty well vanished. The only known issues which currently exist are case sensitivity on Windows and a race condition on imports causing a redundant reload. Both issues are something that will be able to be fixed though. Graham
