[mod_python] High traffic recommendations

Marc D. Murray marcdm at phronein.com
Wed Feb 15 12:13:53 EST 2006


Thank you all for Your tips and responses. 

> Is this server live on the internet or in a stand alone lab?
> Is it possible that there maybe a new virus attaching your machine
> when it connected to the net
It's live on the 'net. But I hope it's not a virus on Debian. Cuz I have
no anti-virus on any of my Debian Boxes :)


> And what do you get in error logs? Does it crash while processing your
> code or on directly in Apache?
> 
> Where did you see DBNoMemory and who reported it? What drivers do you
> use and how do you handle db connections?
The  DB Errors were from the bsddb.DBEnv runninng out of connections.
Not sure why this would happen as it was supposedly designed for many
connections. 
But I'm currently using a global bsddb.DBEnv connection to a folder that
holds many *.bdb files. These are opened stuffed into a queue. The are
closed if older than 2 hrs.
I don't get these errors  anymore. 


> How does Apache behave WRT memory ? There are known memory leaks in
> mod_python version prior to 3.2.7, so 
Each apache process hovers between 3.2 and 4.7 % (readin from top). I
ran some tests on my local server using /usr/sbin/ab, but it never went
down. Not even when requests set to 50000. 

> I would advise you to upgrade to 3.2.7 if you can
Now this is something I've been thinking of. I'll look into it later
tonite. 
 

> >       MaxRequestsPerChild  500
> 
> Make that 10000 or so. This low a number is only required if your
> system is leaking lots of resources (which mod_python does not do).
Now I'm confused. Does it leak or not? How do I test for that?


I've changed the config in the following way :
  - AddHandler python-program py
  + SetHandler mod_python
  - PythonOption SessionDbm=blahblah
  + lines to SetHandler = Default for *.(jpg|gif|png|html|txt)

  -MultiViews 
  KeepAlive On
  KeepAliveTimout 300
  MaxKeepAlive 100
  
  MaxRequestsPerChild 1000

This seems to solve the problem with errors I saw in the logs stating
[No Available Variant] 

Occationally, I see a 404 error in the logs pointing to the wrong
web-root. Actually it points to the only site on the server running
under mod_jk. Is there some sort of contention between the two?

Actually, while writing this email I was checking the server and it
seems that I've solved the problem.

I had a custom session object, that inherited from DBMSession. This was
done to implement some other actions at cleanup time. I commented out
the lines relating to the custom session, and replaced it with the
regular mod_python.Session. Seems to be working ok so far.

I suspect it was either this or the MultiViews that was causing my
probs. 

Let's watch it and see what happens.

Regards,

Marc DM.

On Wed, 2006-02-15 at 09:06 -0500, Marc D. Murray wrote:
> I just 'completed' an application (website) using mod_python. 
> - I'm using the mod_python.publisher handler 
> - Each page is a function in a module. 
> - Apache config has MultiViews enabled
> - Apache/2.0.54 (Debian GNU/Linux) mod_python/3.1.3 Python/2.3.5
> PHP/4.3.10-16 mod_ssl/2.0.54 OpenSSL/0r
> 
> However, I'm having a problem where the server (Apache) goes down every
> 3 - 4 hours. 
> 
> At first, I was getting a DBNoMemory error every half hour. Then I
> started caching database look-ups using memcached. 
> 
> Site was still going down. Now, every hour or two, instead of 40 min.
> 
> So now, I'm caching entire pages but it still going down. Every few
> hours. 
> 
> This is how I'm outputting pages : 
> --> request comes in 
> --> mod_python.publisher decides which module.function to execute 
> --> function checks session variables and loads config from cache or
> *.conf file.
> --> generate cache_key as sitename:req.unparsed_uri:lang  
> --> If item in cache, return cached paged
> --> generate page, insert into cache for 4 hrs --> return output.
> 
> But I'm still being told by Apache that it has reached MaxClients every
> few hours.
> 
> in /etc/apache2/apache.conf I have the following :
> 
> 	KeepAlive Off
> 	Timeout 10
> 	<IfModule prefork.c>
> 	StartServers         5
> 	MinSpareServers      5
> 	MaxSpareServers     10
> 	MaxClients          80
> 	MaxRequestsPerChild  500
> 	</IfModule>
> 	
> Anyone see anything wrong with what's going on here? It's driving me
> crazy with this crashing, and I can't seem to find a solution.
> 
> Thanks.
> 
> Marc DM.
> 
> PS. The server is the same one that used to run the same site in
> mod_mono. Except with mod_mono the server would climb to 32%cpu and stay
> there or climb higher. With mod_python, it never passes 4% Apache Dies.
> Why?
> 
> 
> _______________________________________________
> Mod_python mailing list
> Mod_python at modpython.org
> http://mailman.modpython.org/mailman/listinfo/mod_python
> 



More information about the Mod_python mailing list