|
Graham Dumpleton
graham.dumpleton at gmail.com
Wed Apr 11 18:16:42 EDT 2007
Can you point me to the relevant files and sections of files where the callback from C code where no GIL is held, is made into Python code. If I can see how that is being done and understand it, may be able to suggest something. Graham On 12/04/07, George Feinberg <george.feinberg at oracle.com> wrote: > > Graham, > > This is George Feinberg, and I am responsible for BDB XML development, > and I happened to make the change to use -threads in SWIG generation. > > Perhaps you can help -- you seem to know more about it than I do. > Here's > the history: > > David's using release 2.3.10, where -threads was added. > In release 2.2.13 (the previous one), it was not used, but there was > a problem. > Multi-threaded Python programs, they would deadlock themselves on > a combination of BDB XML locking and the GIL. One of the threads's > stacks > looked a bit like this: > > #6 0x080dd045 in PyThread_acquire_lock () > #7 0x080b1141 in PyEval_EvalFrame () > #8 0x080b726d in PyEval_EvalCodeEx () > #9 0x080b5719 in PyEval_EvalFrame () > #10 0x080b580c in PyEval_EvalFrame () > #11 0x080b726d in PyEval_EvalCodeEx () > #12 0x080fb6fd in PyFunction_SetClosure () > #13 0x0805950c in PyObject_Call () > #14 0x0805eac5 in PyMethod_New () > #15 0x0805950c in PyObject_Call () > #16 0x0808a965 in _PyType_Lookup () > #17 0x0808ca88 in _PyObject_SlotCompare () > #18 0x0805950c in PyObject_Call () > #19 0x080afed7 in PyEval_CallObjectWithKeywords () > #20 0x080594e3 in PyObject_CallObject () > #21 0xb7b0b360 in SWIG_Python_NewPointerObj (ptr=Variable "ptr" is not > available. > ) at dbxml_python_wrap.cpp:1266 > #22 0xb7b32df6 in _wrap_XmlManager_query__SWIG_3 (args=Variable > "args" is not > available. > ) at dbxml_python_wrap.cpp:6675 > #23 0xb7b3332a in _wrap_XmlManager_query (self=0x0, args=0xb6baeb1c) at > > There was another thread involved, and it was blocked inside BDB XML > waiting > for a page lock held by the thread above. Presumably it also held > the lock > (GIL) that caused this thread to block. > > So... I tried a bunch of things to fix this, but the fact that python > 2.3 and 2.4 have > different lock-related APIs and you crash in 2.3 if you use the wrong > one didn't help. > Turning on -threads, plus some other custom code did the trick. > > I'm perfectly happy with another solution if there is one. > Eliminating Python 2.3 > support, or at least making it conditional it might help. > Perhaps you have a suggestion. I'm not a Python expert by any > stretch of the imagination. > I ended up here via hacking. > > There are a couple of things that I'm *trying* to accomplish in the > module: > 1. release the GIL while in BDB XML to allow other threads to run in > Python. > 2. the module uses SWIG Directors, so calls back into those (from C+ > +) need to > reacquire the lock, or things go badly. > > What is the best way to do this? Just eliminating -threads will > cause things > to appear to be working until they deadlock in the non-mod_python > environment. I'm guessing that using your > active state threads may deadlock too, but I'm not certain. > > Thanks for any advice, > > George > > On Apr 10, 2007, at 11:06 PM, Graham Dumpleton wrote: > > > FWIW, am not sure why the -threads option is used as I can't see that > > it would be required when the package is used exclusively from Python > > code. The only time one would potentially need the GIL wrappers that > > SWIG puts in there is when some C code which doesn't currently have > > the Python GIL held or an active thread state makes a call to one of > > the generated SWIG wrapper functions. > > > > Unless the dbxml package has been specially designed to be used in > > embedded systems where C code is calling out of C code into > > dbxml/Python code and the calling C code hasn't performed any prior > > Python initialisation steps, this is unlikely to occur. Only other > > possibility is that dbxml specifically has sections where it releases > > the GIL but then wants to be able to perform a callback into Python > > code, but I can't see any instances of Py_BEGIN_ALLOW_THREADS and > > Py_END_ALLOW_THREADS being used in the code. > > > > The reason the GIL wrappers cause a problem for mod_python is that > > mod_python has already correctly acquired the GIL and created an > > active thread state, but such thread states are not registered with > > Python's simplified GIL API system because one is not meant to use the > > Python simplified GIL API system when multiple Python sub interpreters > > are used for various reasons. It says as much in the Python code and > > also I think the documentation. > > > > Not sure how much joy you will get with getting dbxml changed. Really > > need to find out exactly why dbxml authors think the -threads option > > is required. > > > > Graham > > > >> Graham Dumpleton wrote: > >> > Looking a bit further, the GIL macros are in my own stuff, but I > >> > suspect it doesn't get enabled because I don't use the -threads > >> option > >> > to SWIG. I don't understand the consequences of the option, but > >> > changing: > >> > > >> > dbxml/dist/s_swig > >> > > >> > so that '-threads' isn't in swig_args for Python would probably get > >> > rid of the stuff. > >> > > >> > Would need to work out why they use the option in the first > >> place as > >> > turning it off could break things if dbxml is dependent on it in > >> some > >> > way. > >> > > >> > Graham > >> > > >> > On 11/04/07, Graham Dumpleton <graham.dumpleton at gmail.com> wrote: > >> >> Try rebuilding dbxml but ensure that compiler option: > >> >> > >> >> -DSWIG_PYTHON_NO_USE_GIL > >> >> > >> >> is used. > >> >> > >> >> I don't know when SWIG changed, it certainly didn't generate these > >> >> PyGILState calls the last time I played with it, which wasn't that > >> >> long ago, but it isn't going to work under mod_python the way > >> they now > >> >> do it as the default. > >> >> > >> >> I'll have to look into this further when I get a chance as this > >> new > >> >> behaviour of SWIG is going to cause all my Apache API SWIG > >> bindings to > >> >> break. :-( > >> >> > >> >> Graham > >> >> > >> >> On 11/04/07, David Schachter <ds at bittorrent.com> wrote: > >> >> > > >> >> > Folks-- > >> >> > > >> >> > I'm having a problem embedding Berkeley DB XML into Apache with > >> >> mod_python. > >> >> > The first attempt to create an XmlManager causes a lockup in > >> >> Python--the > >> >> > Global Interpreter Lock is deadlocked, I think. (Stack trace > >> >> below.) Any > >> >> > ideas of how to debug? Is there a BDB XML Python expert? I > >> checked > >> >> Google > >> >> > without success. > >> >> > > >> >> > The code to demonstrate the problem is simple and shown below. > >> >> When the > >> >> > line "xmlManager = dbxml.XmlManager()" is commented out, the > >> code > >> >> runs ok. > >> >> > Commented in, Apache locks up. > >> >> > > >> >> > The most recent mod_python has this comment at > >> >> > > >> >> http://www.modpython.org/live/mod_python-3.3.1/doc-html/app- > >> changes.html: > >> >> > >> >> > > >> >> > > >> >> > Third party C modules that use the simplified API for the Global > >> >> Interpreter > >> >> > Lock (GIL), as described in PEP 311, can now be used. The only > >> >> requirement > >> >> > is that such modules can only be used in the context of the > >> >> > "main_interpreter". (More info at > >> >> > http://issues.apache.org/jira/browse/MODPYTHON-77.) > >> >> > The documentation for mod_python at > >> >> > http://www.modpython.org/live/current/doc-html/pyapi- > >> interps.html > >> >> > says: > >> >> > > >> >> > > >> >> > Note that if any third party module is being used which has a > >> C code > >> >> > component that uses the simplified API for access to the Global > >> >> Interpreter > >> >> > Lock (GIL) for Python extension modules, then the interpreter > >> name > >> >> must be > >> >> > forcibly set to be "main_interpreter". This is necessary as > >> such a > >> >> module > >> >> > will only work correctly if run within the context of the > >> first Python > >> >> > interpreter created by the process. > >> >> > Conforming to this, I set the interpreter name in the > >> httpd.conf > >> >> file using > >> >> > the directive "PythonInterpreter main_interpreter" and I > >> verified > >> >> this by > >> >> > printing the interpreter name from my Python code. > >> >> > > >> >> > In dbxml-2.3.10/dbxml/src/python, there's some code relating to > >> >> the GIL in > >> >> > dbxml_python_wrap.cpp. If someone else has already beaten this > >> >> problem into > >> >> > submission, I'd rather not duplicate the effort. > >> >> > > >> >> > -- David Schachter > >> >> > > >> >> > #! /usr/bin/python > >> >> > > >> >> > import dbxml > >> >> > from mod_python import apache > >> >> > import os > >> >> > > >> >> > > >> >> > def handler(req): > >> >> > id = "pid %d, interpreter '%s'" % (os.getpid(), > >> req.interpreter) > >> >> > > >> >> > apache.log_error("About to start BDB XML from %s." % id) > >> >> > xmlManager = dbxml.XmlManager() > >> >> > apache.log_error("Started BDB XML.") > >> >> > > >> >> > req.write("Hi from %s." % id) > >> >> > return apache.OK > >> >> > > >> >> > > >> >> > STACK TRACE IS: > >> >> > > >> >> > (gdb) bt > >> >> > #0 0xffffe410 in __kernel_vsyscall () > >> >> > #1 0xb7e97af0 in sem_wait at GLIBC_2.0 () from > >> >> > /lib/tls/i686/cmov/libpthread.so.0 > >> >> > #2 0x00000004 in ?? () > >> >> > #3 0xb7ce7a00 in PyThread_acquire_lock (lock=0x0, > >> waitflag=1) at > >> >> > thread_pthread.h:298 > >> >> > #4 0xb7cb5689 in PyEval_RestoreThread (tstate=0x810f888) at > >> >> > Python/ceval.c:305 > >> >> > #5 0xb7ce16e7 in PyGILState_Ensure () at Python/pystate.c:512 > >> >> > #6 0xb78282f9 in _wrap_new_XmlManager (self=0x0, > >> args=0xb7b9102c) at > >> >> > dbxml_python_wrap.cpp:898 > >> >> > (The reference to line 898 is correct but surprising, the > >> >> result of a > >> >> > macro expansion at line 4195. --D.S.) > >> >> > #7 0xb7c65967 in PyObject_Call (func=0x0, arg=0xfffffffc, > >> >> kw=0xfffffffc) > >> >> > at Objects/abstract.c:1795 > >> >> > #8 0xb7cb9e1d in PyEval_EvalFrame (f=0x822cdbc) at > >> >> Python/ceval.c:3845 > >> >> > #9 0xb7cbd719 in PyEval_EvalCodeEx (co=0xb6b12e20, > >> >> globals=0xfffffffc, > >> >> > locals=0x0, args=0x0, argcount=1, > >> >> > kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at > >> >> > Python/ceval.c:2741 > >> >> > #10 0xb7d0d6be in function_call (func=0xb6b0902c, > >> arg=0xb6b892ac, > >> >> kw=0x0) > >> >> > at Objects/funcobject.c:548 > >> >> > #11 0xb7c65967 in PyObject_Call (func=0x0, arg=0xfffffffc, > >> >> kw=0xfffffffc) > >> >> > at Objects/abstract.c:1795 > >> >> > #12 0xb7c6e389 in instancemethod_call (func=0xfffffffc, > >> >> arg=0xb6b892ac, > >> >> > kw=0xfffffffc) > >> >> > at Objects/classobject.c:2532 > >> >> > #13 0xb7c65967 in PyObject_Call (func=0x0, arg=0xfffffffc, > >> >> kw=0xfffffffc) > >> >> > at Objects/abstract.c:1795 > >> >> > #14 0xb7c99728 in slot_tp_init (self=0xb6af1fac, > >> args=0xfffffffc, > >> >> > kwds=0xfffffffc) > >> >> > at Objects/typeobject.c:4774 > >> >> > #15 0xb7c91ad4 in type_call (type=0xb7c99675, args=0xb7b9102c, > >> >> kwds=0x0) at > >> >> > Objects/typeobject.c:435 > >> >> > #16 0xb7c65967 in PyObject_Call (func=0x0, arg=0xfffffffc, > >> >> kw=0xfffffffc) > >> >> > at Objects/abstract.c:1795 > >> >> > #17 0xb7cba239 in PyEval_EvalFrame (f=0x820f224) at > >> >> Python/ceval.c:3776 > >> >> > #18 0xb7cbd0eb in PyEval_EvalFrame (f=0x8195854) at > >> >> Python/ceval.c:3651 > >> >> > #19 0xb7cbd0eb in PyEval_EvalFrame (f=0x81c9ac4) at > >> >> Python/ceval.c:3651 > >> >> > #20 0xb7cbd719 in PyEval_EvalCodeEx (co=0xb6b87e20, > >> >> globals=0xfffffffc, > >> >> > locals=0x0, args=0x7, argcount=0, > >> >> > kws=0x8282440, kwcount=7, defs=0x0, defcount=0, > >> closure=0x0) at > >> >> > Python/ceval.c:2741 > >> >> > #21 0xb7cbb46a in PyEval_EvalFrame (f=0x82822a4) at > >> >> Python/ceval.c:3660 > >> >> > #22 0xb7cbd719 in PyEval_EvalCodeEx (co=0xb6b87ee0, > >> >> globals=0xfffffffc, > >> >> > locals=0x0, args=0x0, argcount=2, > >> >> > kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at > >> >> > Python/ceval.c:2741 > >> >> > #23 0xb7d0d6be in function_call (func=0xb6af3a74, > >> arg=0xb6b7c78c, > >> >> kw=0x0) > >> >> > at Objects/funcobject.c:548 > >> >> > #24 0xb7c65967 in PyObject_Call (func=0x0, arg=0xfffffffc, > >> >> kw=0xfffffffc) > >> >> > at Objects/abstract.c:1795 > >> >> > #25 0xb7c6e389 in instancemethod_call (func=0xfffffffc, > >> >> arg=0xb6b7c78c, > >> >> > kw=0xfffffffc) > >> >> > at Objects/classobject.c:2532 > >> >> > #26 0xb7c65bf6 in PyObject_CallMethod (o=0x0, > >> name=0xb6b891ec "\001", > >> >> > format=0xb6b891ec "\001") > >> >> > at Objects/abstract.c:1795 > >> >> > #27 0xb7c5f505 in python_handler (req=0x828a7b8, > >> phase=0xb7d16e59 > >> >> > "PythonHandler") at mod_python.c:1652 > >> >> > #28 0x08073daf in ap_run_handler (r=0x828a7b8) at config.c:157 > >> >> > #29 0x080741b1 in ap_invoke_handler (r=0x828a7b8) at > >> config.c:372 > >> >> > ---Type <return> to continue, or q <return> to quit--- > >> >> > #30 0x0808ba77 in ap_process_request (r=0x828a7b8) at > >> >> http_request.c:258 > >> >> > #31 0x0808929b in ap_process_http_connection (c=0x8185638) at > >> >> > http_core.c:184 > >> >> > #32 0x0807a599 in ap_run_process_connection (c=0x8185638) at > >> >> > connection.c:43 > >> >> > #33 0x0809e55a in child_main (child_num_arg=-4) at prefork.c: > >> 640 > >> >> > #34 0x0809e7a8 in make_child (s=0x80c9c80, slot=0) at > >> prefork.c:680 > >> >> > #35 0x0809ee53 in ap_mpm_run (_pconf=0xbf86d6f0, > >> plog=0x81031a0, > >> >> > s=0xbf86d6f4) at prefork.c:956 > >> >> > #36 0x080622c8 in main (argc=2, argv=0xbf86d8b4) at main.c:717 > >> >> > (gdb) > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > _______________________________________________ > >> >> > Mod_python mailing list > >> >> > Mod_python at modpython.org > >> >> > http://mailman.modpython.org/mailman/listinfo/mod_python > >> >> > > >> >> > > >> >> > >> > > >> > >> > >
|