|
Sean Davis
sdavis2 at mail.nih.gov
Thu Sep 28 08:40:27 EDT 2006
Graham Dumpleton wrote: > Sean Davis wrote .. > >> Richard Lewis wrote: >> >>> On Wednesday 27 September 2006 14:55, Sean Davis wrote: >>> >>> >>>> On Wednesday 27 September 2006 09:16, Fredrik Sandin wrote: >>>> >>>> >>>>> I asked the same question some time ago: >>>>> http://modpython.org/pipermail/mod_python/2006-August/021924.html >>>>> Graham Dumpleton suggested to use XML-RPC. >>>>> >>>>> As far as I have understood mod_python, or actually Apache, was >>>>> not designed to allow for object persistency. >>>>> >>>>> >>>> Is this actually true (at least the Apache part)? Under mod_perl (I'm >>>> >> a >> >>>> mod_python total newbie), object persistence can be accomplished in >>>> >> several >> >>>> ways including global variables that remain in shared memory (but this >>>> >> is >> >>>> only within a single child), a database, in an in-memory or on-disk >>>> >> cache, >> >>>> or in a session (just an interface to a cache or database). Depends >>>> >> on >> >>>> your definition, but Apache, it seems to me, is absolutely suited to >>>> >> allow >> >>>> object persistence. >>>> >>>> >>>> >>> Persistance is all right: you can use global variables as you suggested. >>> >>> The problem comes at tidying up time. Apache provides no method of performing >>> operations when the server shuts down. This is especially relevant with >>> things like open database connections >>> >> Richard, >> >> Thanks for the clarification. >> >> Under mod_perl, there is Apache::DBI that overrides the connect() method >> of the DBI module to allow reuse of connections (maintaining a >> persistent connection). When the server process dies, the DESTROY() >> method is called and the connections are cleaned up. Would this be >> treated differently under mod_python for some reason, if a database >> connection object had a destroy() method that included a disconnect? I >> have to admit that I don't understand many of the intricacies, but it >> seems that if it can be done in perl under mod_perl, there is likely a >> way to do it under python and mod_python. >> > > Whether one can reliably cause actions to be run on server shutdown in > mod_python is a contentious issue as the moment. I have been a bit lax in going > back and doing some further testing on the issue to settle it one way or the > other and no one else has stepped up to the plate to do the required analysis. > > What I can say is the following. > > 1. Even where a mechanism is available for triggering actions to be called on > shutdown, there is no guarantee that the actions will be called purely because > the Apache parent process can make a decision that a child process isn't > shutting down properly by itself and it will send it a KILL signal to cause it > to immediately quit. Processes can always crash for no good reason as well. > > Thus, one shouldn't write code that absolutely requires such clean shutdowns to > occur. Any startup code should always be written to deal with a prior unclean > shutdown and do any special cleanup first if required. > > Also, sometimes a persons belief that cleanup code is required has been > based on a mistaken belief that if not done then resources would be leaked. > Certainly in the case of database connections, this shouldn't be the case as > the server side will detect that client connections have gone away and will > automatically cleanup on the server side even if a clean close of the connection > hadn't been done. > > 2. In mod_python 3.3, any Python interpreter instances will not be destroyed on > child process shutdown. The interpreter instances are being destroyed in > mod_python prior to 3.3, but doing so can cause the Apache child process to > hang, as this can get triggered from within a signal handler in certain > circumstances and if there is an active request handler for mod_python running > at the time. the signal handler will block due to not being able to acquire the > Python GIL. Thus, calling Py_Finalize() for interpreters is no longer being > done at this point. > > The implications of this are that Python objects are not being destroyed on > child process shutdown, thus you cannot rely on the destructor of Python > objects doing anything to cleanup resources. > > 3. In mod_python, there are the functions apache.register_cleanup() and > req.server.register_cleanup() for registering actions to be done on server > cleanup. However, the trigger for these being called is the destruction of > the childs Apache memory pool. As with 2, this seems in some cases to be > triggered from a signal handler and thus can similarly result in the Apache > child process hanging on shutdown. > > See: > > http://issues.apache.org/jira/browse/MODPYTHON-109 > > for more details. > > Thus, the question has been whether if no guarantees can be given as to > whether cleanup actions will even be called and if an attempt is made to > call them, it still may result in them not being run because of the Apache > child process hanging due to locking problems, should the feature even > be provided. > > Now it is possible that prior analysis of the Apache source code and how > these cleanup actions are triggered could be wrong, or based on old Apache > code where it worked a bit differently. As I said though, no one else has > stepped up to do a proper analysis of the Apache code and do testing in > respect of the issue and I haven't had a chance lately to look at it again. > The other core developers have also been quite busy of late. Without this > further analysis, we will not know if 2 and 3 are really problems or not. > > Graham, Thanks for the awesome analysis! I'm not much of a programmer, so I can't say I follow all of it, but I think I get the take-home messages. Thanks again for the explanation and for continuing to actively make improvements. Sean
|