[mod_python] Threading and apache.import_module

2005/6/20, Graham Dumpleton <grahamd at dscpl.com.au>:
> michael bayer wrote ..
> > I just searched around for this, and hey youre right, it does.
> > Myghty imports "in-memory" modules in a similar way (i.e. via
> > imp.new_module + exec)....but I hadnt thought of just bypassing
> > sys.modules altogether.  A "request-level" module really doesnt need
> > to be generally importable so that now makes sense.
> 
> Unfortunately, if you want an alternate module import system to support
> packages in a useful way, you don't have a choice but to still place loaded
> modules into sys.modules. The key though is knowing that it isn't actually
> necessary to name the module with its true module name. It can instead
> be an arbitrary name based on the path name to the ".py" file. The name
> can't be too long though as you run up against name length restrictions
> in Python. Something like a md5 hexdigest of the full pathname generally
> works okay. That said, the only problem with trying to support packages
> is that dependency management can become hellish. It isn't enough to
> simply reload the single file in a package which has changed, you must
> reload the whole package because of the interdependencies that can exist
> between the modules in the package. Also, if only one file in the package
> is changed, anything dependent on any part of the module will need to
> be reloaded at the next opportunity.

Regarding the naming of the package, don't give a module the same name
as its filename, that is to say with slashes and dots. It seems that
this breaks some behaviour of the Python import system, that is to say
that some imports that are performed in the dynamically loaded module
misteriously fail. I use a re.sub(r'\W+','_',filename) and everything
is OK. That being said, I didn't know about name length restrictions
so I have to correct my code to cope with this issue.

> > I would most like to find a way to do this while also taking
> > advantage of .pyc files (seems like, create .pyc file if it doesnt
> > exist, else if it exists, read four bytes + one long value, then use
> > marshal.load() to create a code object, then exec that).
> 
> I would advise against trying to accomodate ".pyc" files. If you were to
> do this you will in practice have to monitor the timestamps on both the
> ".py" and ".pyc" files and contend with issues such as what to do when
> the ".pyc" file is removed but the ".py" still exists but you were originally
> loaded from the ".pyc" file. Similarly, what do you do if you are loaded
> from the ".pyc" file and the ".py" file is removed. Does this imply that
> the user is intending the page to now no longer be available. Another
> cases is what do you do if you are loaded from the ".pyc" file and the
> ".py" file modification time is changed to be older than what it was before.
> Do you take this as meaning that an older file has been restored and
> that should be used instead. As you can see you have to make various
> assumptions and in some respects hope for the best. Although it can
> be made to work in a reasonably sensible way, I found it complicates
> the code unnecessarily and not worth the trouble, especially when
> Apache generally runs as a user where ".pyc" files can't be created in
> the first place.
> 
> Graham
> 
>