|
Greg Stein
gstein at lyra.org
Mon Jun 5 17:16:39 EST 2000
On Mon, 5 Jun 2000, [iso-8859-1] Stéphane Bidoul wrote: >... > Now, in mod_python, threads are created by Apache, so they should > somehow be bootstrapped for python. Correct. > In my understanding, mod_python only initializes one thread per > interpreter: the Apache thread that is happening to create the interpreter > (using Py_NewInterpreter(), which in turns calls PyThreadState_New() > for the interpreter's "main" thread). > All other apache threads using that same interpreter will not call > PyThreadState_New()... hence the potential conflict. ThreadState structures are not specific to a particular (OS) thread. You can actually use them (not at the same time!) across different threads. Given the protection of the global lock, it might be okay to even attempt to use them at the same time. [ haven't thought through this particular scenario fully yet ] > To summarize, my understanding of the python doc is that there should > be one PyThreadState_New() for each (apache thread, python > interpreter) couple. Theoretically: yes. Python's central lock "should" arbitrate the use of the per-interpreter thread state so that it can be used by multiple Apache threads. Probably isn't the best idea, though. Eek. Definitely wrong for the Apache scenario. The scoop: if python_handler() calls into Python, then returns, then tstate->frame should be NULL. In this scenario, you could do the share-tstate thing. However: somewhere down inside of python_handler's call into Python, it is quite possible for Python to use Py_BEGIN_ALLOW_THREADS (unlock the global lock). This will allow another python_handler to grab the tstate and attempt execution. Let's say the thread execution then flips back to normal: the original thread's tstate->frame is now *totally* wrong. [ tstate->frame has a direct correspondence to the C stack; these must stay in sync or Bad Things happen ] > AFAIK, threads are used only in the Windows version of Apache, > so the problem should not occur on unix (athough that will change with Apache 2). > Also, if ThreadsPerChild is set to 1, the problem disappears, > which seems to confirm my diagnostic. > > Again, bear with me if this is totally wrong... > If this is correct, however, I've no idea (yet) on how to fix it... Your underlying hypothesis is quite correct. Above is a more detailed breakdown of "why". > What do you think? > Can a python threading expert confirm or infirm this diagnostic? That would be me :-) One other thing: if Apache is going to run threads, then Python *MUST* be compiled with thread support. Otherwise, things will die very rapidly. Presuming that you are using the standard Windows distribution, then the thread support is automatically enabled. I would recommend that mod_python barf with #error if it detects Apache/Win32 and WITH_THREAD is not enabled. Heh. All of this just supports my belief in "free threading" Python even more... :-) What is the solution? Each entry to python-handler should construct a tstate on the fly, call into Python, then destroy the tstate. I used this design for the Python/COM support in the Win32 distribution. That had to deal with reentrancy issues, but (luckily) that isn't a problem here. Presumably, you could also cache/pool tstates and yank them from your pool on entry to python-handler, returning them on exit. If two threads came into python-handler (against the same interp!), then additional tstates would be constructed. Cheers, -g -- Greg Stein, http://www.lyra.org/
|