[mod_python] sys.path shared?

I'm using SQL Relay to share database connections on my pre-forked 
apache. This way, when load increases, every process doesn't need their 
own db connection, they only need their own SQL Relay connection. It's 
a pretty neat project. David Muse did a good job. Python bindings are 
included :)

http://sqlrelay.sf.net

-- 
Todd

p.s. email me if any of you are on OS X and try SQL Relay 0.35, there 
are a few gotchas (termio lines in a rudiments header file needs to be 
commented out; CFLAGS needs "-framework Python"; and finally shared 
memory needs to be increased in /etc/rc).

On Sep 22, 2004, at 1:01 PM, Justin Ryan wrote:

> You certainly can't share a database connection across apache children 
> using a forking model, and I haven't decided if I think you can using 
> a threaded mpm either.  You probably don't want the interpreter shared 
> at all, b/c of the good ol' GIL.  Also, if you use psycopg (afaik, the 
> only threadsafety level 2 adapter), you're really opening a new 
> connection each time you grab a cursor - it just handles the magic for 
> you.  In fact, you've gotta be pretty careful using psycopg if you 
> dont want it to hold a gangload of connections open for no reason.
>
> If you are globally setting sys.path in apache conf, you know that 
> each instance of mod_python is _beginning_ with the same sys.path (or 
> maybe each req - i forget things alot).  If you have some code which 
> modifies it, it won't propogate.  You could take a look at the code 
> for application variables in PSE[1] and copy the general idea (think 
> of app vars as session vars that span sessions the way session vars 
> span requests).
>
> We also hacked up a way of holding onto a db connection, but only 
> per-apache process.  This isn't too bad, because you can handle 
> several hundred requests before tearing down the child (and, in a 
> shared env, the connection is only opened after the first mod_python 
> req comes through).
>
> Cheers,
>
> -=JR
>
> [1] http://nick.borko.org/pse - it's totally fly
>
> On Sep 22, 2004, at 8:58 AM, Brian Bird wrote:
>
>> Not sure that's actually evidence against threads. In my case I do 
>> actually
>> want to share variables (I want only one database connection shared 
>> across
>> all requests and I also have a global variable counting the number of
>> requests which each reuqest updates)
>>
>> I've got my sys.path set up in the apache config so it only has to be
>> initialised once. The (potential) problem with forking is the amount 
>> of
>> extra overhead required for each request. Mind you, apache seems to 
>> do a
>> pretty good job of reducing the overhead so it's not really a 
>> problem. But
>> to share something like a database connection using a forking 
>> model... I'm
>> not sure that's even possible is it?
>>
>> Brian
>>
>> -----Original Message-----
>> From: mod_python-bounces at modpython.org
>> [mailto:mod_python-bounces at modpython.org] On Behalf Of Terry MacDonald
>> Sent: 22 September 2004 14:38
>> To: mod_python user mailing list
>> Subject: RE: [mod_python] sys.path shared?
>>
>>
>> ah-ah! More evidence against threads in the threads versus processes 
>> debate.
>> ;3)
>>
>> On Wed, 2004-09-22 at 14:10, Brian Bird wrote:
>>> Sorry, I should have given this a bit more thought. If you're using a
>>> forked process environment then I would expect the sys.path variables
>>> to be separate for each request. If you're using a threaded mpm 
>>> (which
>>> is what I
>>> use) then I assume sys.path would work like global variables which 
>>> are
>>> shared across requests.
>>>
>>> Brian
>>>
>>> -----Original Message-----
>>> From: mod_python-bounces at modpython.org
>>> [mailto:mod_python-bounces at modpython.org] On Behalf Of Terry 
>>> MacDonald
>>> Sent: 22 September 2004 13:54
>>> To: mod_python user mailing list
>>> Subject: RE: [mod_python] sys.path shared?
>>>
>>>
>>> Can anyone put me right on my understanding of the way mod_python
>>> works in an apache forked process environment.  Each process has its
>>> own embedded interpreter and therefore its own sys.path variable.
>>>
>>> How is it possible for an apache instance/process with its own
>>> embedded interpreter to 'reach' across and use/inherit another
>>> processes/interpreters sys.path?
>>>
>>>
>>>
>>> On Wed, 2004-09-22 at 09:58, Brian Bird wrote:
>>>> Might be best to try it: Write a cgi script which prints the
>>>> sys.path
>>>> at the start and at the end. Put a long sleep in the middle then you
>>>> can visit your script twice with a browser (start the second visit
>>>> while the first one is still running) and see what the sys.path is 
>>>> of
>>>> your second visit.
>>>>
>>>> My gut feeling would be that it would be possible but I'm pretty new
>>>> to mod_python so I wouldn't rely on that! :-)
>>>>
>>>> Brian
>>>>
>>>> -----Original Message-----
>>>> From: mod_python-bounces at modpython.org
>>>> [mailto:mod_python-bounces at modpython.org] On Behalf Of Terry 
>>>> MacDonald
>>>> Sent: 21 September 2004 23:44
>>>> To: mod_python user mailing list
>>>> Subject: Re: [mod_python] sys.path shared?
>>>>
>>>>
>>>> Each apache instance/process has its own interpreter therefore its
>>>> own
>>>> sys.path.  So I would say it is not possible.
>>>>
>>>> On Tue, 2004-09-21 at 23:24, Adrian Holovaty wrote:
>>>>> I have mod_python code that appends a directory to sys.path and
>>>>> deletes that directory from sys.path at the end of the handler. Is
>>>>> it possible that, between that push and pop, another Apache
>>>>> instance running the same code
>>>
>>>>> might get the non-popped sys.path, making its sys.path out of sync
>>>>> with
>>>> the
>>>>> other Apaches?
>>>>>
>>>>> Adrian
>>>>> _______________________________________________
>>>>> Mod_python mailing list
>>>>> Mod_python at modpython.org
>>>>> http://mailman.modpython.org/mailman/listinfo/mod_python
>> -- 
>> Terry
>> Registered Linux User # 311806
>> www.taumu.com
>>
>>
>> _______________________________________________
>> Mod_python mailing list
>> Mod_python at modpython.org
>> http://mailman.modpython.org/mailman/listinfo/mod_python
>>
>>
>> _______________________________________________
>> Mod_python mailing list
>> Mod_python at modpython.org
>> http://mailman.modpython.org/mailman/listinfo/mod_python
>
> _______________________________________________
> Mod_python mailing list
> Mod_python at modpython.org
> http://mailman.modpython.org/mailman/listinfo/mod_python
>