|
Graham Dumpleton
grahamd at dscpl.com.au
Tue Mar 15 23:19:25 EST 2005
John Belmonte wrote .. > Several months ago an installation of my application which uses > mod_python/Quixote/ZODB/ZEO suddenly stopped working. It appears to > have been caused by a mod_python upgrade. I'm on Debian and now using > 3.1.3, although I don't know what version I was using previously. > > I traced the error to a "ImportError: No module named copy_reg" > resulting from a cPickle.Pickler() call. This is caused by Python > thinking that it's in restricted execution mode. The error is happening > within ZEO's client thread code. > > The following mod_python list message "Threads running in restricted > mode?" seems to detail similar problems, but no one responded to the issue. > > http://www.modpython.org/pipermail/mod_python/2005-January/017129.html > > Tim Peters speculates that mod_python might be starting a new > interpreter from time to time using the Python C API. The new > interpreter will have distinct copies of the sys module, the sys.modules > dict, and the sys.path list, triggering Python's restricted execution mode. > > Anyone have some insight to this problem? Or at least, would someone > explain how and when mod_python creates new interpreters? For an explaination of interpreters and what gets created when, read: http://www.modpython.org/live/mod_python-3.1.2b/doc-html/pyapi-interps.html The closest I have ever got to problems with restricted mode is the runtime exception "classes are read-only in restricted mode". In my circumstance, which was probably pretty unique in terms of what I was actually doing, I was trying to communicate between subinterpreters within the one Apache process using a messaging system (http://ose.sourceforge.net). In order to do this the event system which runs the messaging system is started up within its own thread from Python. The messaging system then actually executes in its own right as C++ code, not Python code. So that messages can come back into the Python world, callbacks are registered in the C++ world for the Python functions to be called. At some later point when the messaging system has something it will execute the Python callback. What happened though was that when the callback was executed I got the Python run time error "classes are read-only in restricted mode". The cause of this turned out to be because I had created the initial thread for the messaging system in a dedicated interpreter created for that purpose by using PythonImport directive with a special interpreter name. A programming problem in OSE however meant that it saved away the thread state corresponding to that special interpreter and it tried to use that thread state when trying to callback into other interpreters. Because the thread state wasn't associated with these other interpreters, it would only execute code in restricted mode. Correspondingly, the messaging system worked great for where you wanted to communicate within the one interpreter and to outside processes, but not between multiple interpreters created by mod_python in the same Apache process. I have yet to fix OSE to make it work but might be motivated now to do something about it. How is this tale relevant. Well, it may if any of those packages you are using have a C/C++ based component which operates a distinct thread which at some point needs to callback into Python code. Now you may think, "but I don't create any extra interpreters", but that may not matter. This is because there are some bugs in the mod_python code which can cause multiple sub interpreters of the same name to be errornously created if a web server is under load immediately from the point it was started and you are using a multithread MPM. What the bug means is that an interpreter gets created with a specific name but because of inadequate thread locking on the creation of interpreters, a second interpreter of the same name can get created at the same time. One of these will get thrown away, with possibly leaking of resources and after that you wouldn't normally notice anything. Theorectically however, if the first of those interpreters initialised some C/C++ based module which created an internal thread it may have saved away the thread state for that interpreter. Now if the second interpreter registered some callback with C/C++ code, when that code executes the callback, the saved thread state could be wrong for that interpreter and thus it may execute in restricted mode. Now all this may be totally irrelevant, but thread state problems and callbacks is the only way I have seen that restricted mode can be triggered, so thus simple mentioning it in case it might be helpful. As to what to do about it, first thing would be to patch the thread problems in mod_python using patches at: http://www.dscpl.com.au/projects/vampire/PATCHES If however you aren't using a multithread MPM, this may make no difference at all. The only other thing I can add is to ask whether you are using a multithreaded MPM? Do any of the third party packages create internal threads? Do any of the third party packages have a C/C++ component to them? Hope this helps. Graham
|