[mod_python] Re: child process XXXX still did not exit, sending a SIGTERM

On 1/3/06, Graham Dumpleton <grahamd at dscpl.com.au> wrote:
> Graham Dumpleton wrote ..
> > Martin Blais wrote ..
> > > After much hacking (all day today), I added the explicit
> > > Py_EndInterpreter calls and it still hangs on destruction.
> > > Specifically, it hangs on PyImport_Cleanup() in Py_EndInterpreter().
> > > I'm having trouble pinpointing exactly which symbol makes it hang.  It
> > > seems to happen when cleaning the logging module.  I saw that logging
> > > register an atexit hook, tried to remove that, does not fix the
> > > problem.
> >
> > Hmmm, you have gotten ahead of me. I was going to work out some code to
> > end each interpreter explicitly this morning.
>
> FWIW, I would have stuck the following just prior to get_interpreter()
> call in python_finalize() to try and debug which intepreter it may have
> been getting stuck in. It is a bit tricky as it makes use of the thread
> state object that make_interpreter() left inaccessible from when the
> interepreter was first created. In general, this code probably will not
> work for more complicated systems where additional C code had
> created additional thread states associated with an intepreter as the
> Py_EndInterpreter() call expects there to be only one remaining. Note
> this assumes that multiple interpreter fix I reference is applied.

Hehe it seems we have been walking on the same paths, indeed.

On my side, I changed the code to remove that original thread-state
right after a new interpreter is created.  I traced all the tstate
creations and made sure to not have any extra thread-states lying
around.  Will submit patch when I clean up the stuff.   However, I
found that trying to do remove the original thread-state from the
first interpreter results in some other error (PyDict_??? ... related
to modules, lost my log file).  In the end I just decided to leave it
for now, and not to explicitly end the main_interpreter (because at
that point it has two thread-states).  I thought of doing like you did
for the main_interpreter, with the original tstate.

Anyway, in any case, it hangs in my own custom interpreter, and not in
the main_interpreter.

My main problem is getting more info at that point, because even if I
attach a gdb to the cihld process, apache insists on sending it
SIGKILL 10 seconds later.  I just had an idea on the way to dinner,
that I could "simulate" finalisation by calling python_finalize from
the handler by returning a special signal number, just for this
purpose.

One other thing I did is reproduce Py_EndInterpreter within
mod_python, and it hangs on PyImport_Cleanup().  I turned up
Py_VerboseFlag and it seems to stop at the logging module.  logging
has some atexit calls, so I tried clearing that but it did not stop
the problem.  Need to look further, this is a tough bugger of a bug.

About the thread-state creation: it might be faster to eventually use
the pooling solution, rather than creating new tstates all the time,
the thread-state creation is light, but there is a small lock in
there.

cheers,