[mod_python] High traffic recommendations

Nicolas Lehuen nicolas at lehuen.com
Wed Feb 15 13:27:46 EST 2006


Marc,

What did your custom Session code do ? Can you post it, so that we try
and see what could cause the problem ?

Regards,
Nicolas

2006/2/15, Marc D. Murray <marcdm at phronein.com>:
> Thank you all for Your tips and responses.
>
> > Is this server live on the internet or in a stand alone lab?
> > Is it possible that there maybe a new virus attaching your machine
> > when it connected to the net
> It's live on the 'net. But I hope it's not a virus on Debian. Cuz I have
> no anti-virus on any of my Debian Boxes :)
>
>
> > And what do you get in error logs? Does it crash while processing your
> > code or on directly in Apache?
> >
> > Where did you see DBNoMemory and who reported it? What drivers do you
> > use and how do you handle db connections?
> The  DB Errors were from the bsddb.DBEnv runninng out of connections.
> Not sure why this would happen as it was supposedly designed for many
> connections.
> But I'm currently using a global bsddb.DBEnv connection to a folder that
> holds many *.bdb files. These are opened stuffed into a queue. The are
> closed if older than 2 hrs.
> I don't get these errors  anymore.
>
>
> > How does Apache behave WRT memory ? There are known memory leaks in
> > mod_python version prior to 3.2.7, so
> Each apache process hovers between 3.2 and 4.7 % (readin from top). I
> ran some tests on my local server using /usr/sbin/ab, but it never went
> down. Not even when requests set to 50000.
>
> > I would advise you to upgrade to 3.2.7 if you can
> Now this is something I've been thinking of. I'll look into it later
> tonite.
>
>
> > >       MaxRequestsPerChild  500
> >
> > Make that 10000 or so. This low a number is only required if your
> > system is leaking lots of resources (which mod_python does not do).
> Now I'm confused. Does it leak or not? How do I test for that?
>
>
> I've changed the config in the following way :
>   - AddHandler python-program py
>   + SetHandler mod_python
>   - PythonOption SessionDbm=blahblah
>   + lines to SetHandler = Default for *.(jpg|gif|png|html|txt)
>
>   -MultiViews
>   KeepAlive On
>   KeepAliveTimout 300
>   MaxKeepAlive 100
>
>   MaxRequestsPerChild 1000
>
> This seems to solve the problem with errors I saw in the logs stating
> [No Available Variant]
>
> Occationally, I see a 404 error in the logs pointing to the wrong
> web-root. Actually it points to the only site on the server running
> under mod_jk. Is there some sort of contention between the two?
>
> Actually, while writing this email I was checking the server and it
> seems that I've solved the problem.
>
> I had a custom session object, that inherited from DBMSession. This was
> done to implement some other actions at cleanup time. I commented out
> the lines relating to the custom session, and replaced it with the
> regular mod_python.Session. Seems to be working ok so far.
>
> I suspect it was either this or the MultiViews that was causing my
> probs.
>
> Let's watch it and see what happens.
>
> Regards,
>
> Marc DM.
>
> On Wed, 2006-02-15 at 09:06 -0500, Marc D. Murray wrote:
> > I just 'completed' an application (website) using mod_python.
> > - I'm using the mod_python.publisher handler
> > - Each page is a function in a module.
> > - Apache config has MultiViews enabled
> > - Apache/2.0.54 (Debian GNU/Linux) mod_python/3.1.3 Python/2.3.5
> > PHP/4.3.10-16 mod_ssl/2.0.54 OpenSSL/0r
> >
> > However, I'm having a problem where the server (Apache) goes down every
> > 3 - 4 hours.
> >
> > At first, I was getting a DBNoMemory error every half hour. Then I
> > started caching database look-ups using memcached.
> >
> > Site was still going down. Now, every hour or two, instead of 40 min.
> >
> > So now, I'm caching entire pages but it still going down. Every few
> > hours.
> >
> > This is how I'm outputting pages :
> > --> request comes in
> > --> mod_python.publisher decides which module.function to execute
> > --> function checks session variables and loads config from cache or
> > *.conf file.
> > --> generate cache_key as sitename:req.unparsed_uri:lang
> > --> If item in cache, return cached paged
> > --> generate page, insert into cache for 4 hrs --> return output.
> >
> > But I'm still being told by Apache that it has reached MaxClients every
> > few hours.
> >
> > in /etc/apache2/apache.conf I have the following :
> >
> >       KeepAlive Off
> >       Timeout 10
> >       <IfModule prefork.c>
> >       StartServers         5
> >       MinSpareServers      5
> >       MaxSpareServers     10
> >       MaxClients          80
> >       MaxRequestsPerChild  500
> >       </IfModule>
> >
> > Anyone see anything wrong with what's going on here? It's driving me
> > crazy with this crashing, and I can't seem to find a solution.
> >
> > Thanks.
> >
> > Marc DM.
> >
> > PS. The server is the same one that used to run the same site in
> > mod_mono. Except with mod_mono the server would climb to 32%cpu and stay
> > there or climb higher. With mod_python, it never passes 4% Apache Dies.
> > Why?
> >
> >
> > _______________________________________________
> > Mod_python mailing list
> > Mod_python at modpython.org
> > http://mailman.modpython.org/mailman/listinfo/mod_python
> >
>
> _______________________________________________
> Mod_python mailing list
> Mod_python at modpython.org
> http://mailman.modpython.org/mailman/listinfo/mod_python
>



More information about the Mod_python mailing list