[mod_python] Pickle can't save classes instances in mod_python ?

2008/7/31 Eric Baumer <ebaumer at ics.uci.edu>:
> I was having trouble loading a pickled class object using mod_python when I
> came across this post in the mod_python archives from June 2007. even though
> the link below implies that loading a pickled object in this manner cannot
> be done, I believe I may have found a work around. I wanted to bounce this
> off the mod_python list for two reasons: (a) to make sure I'm not breaking
> anything, and (b) to share what may be a viable solution to this problem.
>
> the link mentioned here seems to imply that one cannot load a pickled
> instance of a class in a script running through mod_python. the problem
> seems to be related to mod_python's method of importing. since there doesn't
> seem to be a good way to guarantee where on the file system the script will
> run when apache receives a request that invokes mod_python. thus, rather
> than referring to a module by name alone (with the standard "import
> module_name" approach), modules are referred to by their path on the file
> system using apache.import_module, for example "my_module =
> apache.import_module('/home/username/my_proj/my_module.py')".
>
> the problem becomes that this method of importing doesn't register the
> module name with sys.modules. therefore, if I have a pickled instance of
> some class my_module.MyClass, when I try to load the pickled instance,
> python would complain that there is "No module named my_module" (as
> described by the OP), even if I had loaded it using apache.import_module.
>
> the workaround that I found looks something like the following.
>
> import sys
> def index(req):
>     my_module = apache.import_module('/home/username/my_proj/my_module.py')
>     sys.modules['my_module'] = my_module
>
> this code ensures that my_module is registered with the list of modules
> python knows about in sys.modules, so that when I load that pickled instance
> of my_module.MyClass, python knows about a module named "my_module" and thus
> can properly load the class.
>
> I should note that, in practice, this has actually worked for me. whether or
> not it's a good idea, though, is another question.
>
> what do you all think? have I gone horribly wrong, or is this a viable work
> around to loading a pickled instance of a class?

Why don't you do what the document suggests you do?

That is, put common code into a proper Python module/package in a
directory outside of the document tree served/managed by Apache and
use PythonPath directive to indicate where that directory is. Then use
standard 'import' statement to import that module. Thus code is in
standard module/package and not a special mod_python code module.

It is generally regarded as being bad practice to put too much code
within directories managed by Apache/mod_python anyway, because a
misconfiguration could expose your code to download, especially if the
code isn't even in a handler code file.

As to your code change. This could technically break at some point
because a thread race condition exists in modifying sys.modules. It is
also not really a good idea to have pickleable code coming from a
module which could be reloaded as multiple versions of class types and
thus multiple variations of instances could exist. This can cause
problems when comparing object types etc, if object instances are
passed/cached outside of that specific code file. You also have a
potential ordering issue here as well. What if something tries to
unpickle data requiring my_module before you have managed to load it.

So, I certainly would just recommend you factor out your common class
definitions etc into standard Python modules/packages installed on
normal Python module search path and don't store them in reloadable
mod_python managed code module files.

Graham