[mod_python] Threading and apache.import_module

michael bayer wrote ..
> I just searched around for this, and hey youre right, it does.     
> Myghty imports "in-memory" modules in a similar way (i.e. via  
> imp.new_module + exec)....but I hadnt thought of just bypassing  
> sys.modules altogether.  A "request-level" module really doesnt need  
> to be generally importable so that now makes sense.

Unfortunately, if you want an alternate module import system to support
packages in a useful way, you don't have a choice but to still place loaded
modules into sys.modules. The key though is knowing that it isn't actually
necessary to name the module with its true module name. It can instead
be an arbitrary name based on the path name to the ".py" file. The name
can't be too long though as you run up against name length restrictions
in Python. Something like a md5 hexdigest of the full pathname generally
works okay. That said, the only problem with trying to support packages
is that dependency management can become hellish. It isn't enough to
simply reload the single file in a package which has changed, you must
reload the whole package because of the interdependencies that can exist
between the modules in the package. Also, if only one file in the package
is changed, anything dependent on any part of the module will need to
be reloaded at the next opportunity.

> I would most like to find a way to do this while also taking  
> advantage of .pyc files (seems like, create .pyc file if it doesnt  
> exist, else if it exists, read four bytes + one long value, then use  
> marshal.load() to create a code object, then exec that).

I would advise against trying to accomodate ".pyc" files. If you were to
do this you will in practice have to monitor the timestamps on both the
".py" and ".pyc" files and contend with issues such as what to do when
the ".pyc" file is removed but the ".py" still exists but you were originally
loaded from the ".pyc" file. Similarly, what do you do if you are loaded
from the ".pyc" file and the ".py" file is removed. Does this imply that
the user is intending the page to now no longer be available. Another
cases is what do you do if you are loaded from the ".pyc" file and the
".py" file modification time is changed to be older than what it was before.
Do you take this as meaning that an older file has been restored and
that should be used instead. As you can see you have to make various
assumptions and in some respects hope for the best. Although it can
be made to work in a reasonably sensible way, I found it complicates
the code unnecessarily and not worth the trouble, especially when
Apache generally runs as a user where ".pyc" files can't be created in
the first place.

Graham