|
Suresh Lakshmanan
suresh.lakshmanan at gmail.com
Tue Jan 23 23:09:14 EST 2007
Hi Graham,
I'm not very familiar with the apache MPM model. Thanks for your references,
im able to get an idea of how the subinterpreters work in mod_python.
My python application is a django based application. It uses a few python
modules like
PIL (python imaging library), dnspython, django, libxx (our c++ extension
written in python)
Multiple requests to my django application (running inside apache with
mod_python) are serialized and they execute one by one.
Lot of them get 503 responses.
Here's my apache conf.
# worker MPM
# StartServers: initial number of server processes to start
# MaxClients: maximum number of simultaneous client connections
# MinSpareThreads: minimum number of worker threads which are kept spare
# MaxSpareThreads: maximum number of worker threads which are kept spare
# ThreadsPerChild: constant number of worker threads in each server process
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule worker.c>
StartServers 1
MaxClients 64
MinSpareThreads 64
MaxSpareThreads 64
ThreadsPerChild 64
MaxRequestsPerChild 0
ServerLimit 1
</IfModule>
<Location "/reg/">
SetEnv PYTHONHOME "/usr/local/bin/python2.4"
SetHandler python-program
PythonPath "['/usr/local/pi/applications/',
'/usr/local/pi/nodes/data/regist
ration', '/usr/local/pi/pimgr/lib', '/usr/local/pi/pimgr/lib/ca' ,
'/usr/local/p
i/pimgr/lib/MySQLdb'] + sys.path"
PythonHandler django.core.handlers.modpython
SetEnv PINET_HOME
"/usr/local/pi/nodes/data/registration/pimgr/reg/ca/config
"
SetEnv PIMGR_PATH "/usr/local/pi/nodes/data/registration/pimgr"
SetEnv DJANGO_SETTINGS_MODULE pimgr.settings
PythonDebug On
</Location>
On 1/24/07, Graham Dumpleton <grahamd at dscpl.com.au> wrote:
>
> Suresh Lakshmanan wrote ..
> > Hi,
> > Is it possible for mod_python to create new subinterpreter when requests
> > arrive? I was thinking that mod_python has a configurable pool of python
> > interpreters
> > and it will delegate a request to an interpreter from the pool.
> >
> > But, it looks like this is not the case.
>
> Correct, that is not the case. You are misunderstand what the
> subinterpreters
> are there for.
>
> > Only one subinterpreter is only
> > created for a virtual host or directory config according to
> documentation.
>
> More or less, yes. Each sub interpreter is named and is tied to some part
> of
> the URL namespace. The default is that there will be a sub interpreter for
> each virtual host, but this can be overridden by various directives.
>
> > This is
> > creating problems because the other requests have to wait for a long
> time
> > and i get 503 from apache.
>
> The way Python sub interpreters are used, it is highly unlikely that a 503
> response is going to be because of them. It may be how you have Apache
> configured, but to know you will have to state what platform you are using
> and if on UNIX whether you have compiled Apache with the prefork MPM
> or worker MPM. You also should post what you Apache configuration is
> that controls the number of Apache sub processes and/or threads within
> a process.
>
> > It will be useful if i have multiple
> > subinterpreters
> > to which apache can assign requests arbitrarily. I do not maintain state
> > in
> > any of the interpreters.
>
> This will not make any difference, what matters is how many Apache child
> process are allowed to be created when using prefork MPM, or how many
> child processes and/or threads with a process that Apache is allowed to
> create when using a multithreaded MPM.
>
> If using prefork MPM and you have restricted the number of Apache child
> processes to a small fixed number, then if you get more requests than
> that arrive, there will be no processes available to handle the request.
> If
> you are using a multithreaded MPM, then a small number of threads across
> all processes will limit how many requests can be handled concurrently.
>
> Creating extra Python subinterpreters only gives you separation of data,
> it doesn't provide any additional concurrency abilities. You seem not to
> understand this point.
>
> > iow, how do i scale this to a large number of users. how do i introduce
> > parallel processing of requests for the same virtual host/directory. at
> > the
> > moment it looks
> > like its single threaded.
>
> If you use prefork, then only one request in each child process can be
> handled at a time, but then you should be running with adequate child
> processes so that isn't a problem. If using a multithreaded MPM, then
> there
> already can be multiple requests handled at the same time and these
> can be handled even within the same Python subinterpreter.
>
> > definitely there should be a way as this is a common problem... can
> someone
> > please point me to the solution.
>
> You probably need to research more about the different MPM models for
> Apache. For starters, read:
>
>
> http://www.dscpl.com.au/wiki/ModPython/Articles/TheProcessInterpreterModel
>
> You might then see what you can find in documentation about Apache
> referenced on:
>
> http://wiki.apache.org/mod_python/
>
> On first guess I would say how you have configured Apache is limiting
> things.
>
> If you are using a multithreaded MPM, then you might have a third party
> module
> for Python you are using which doesn't release the Python GIL around a
> long
> running operation being performed in C code. Thus, what extra Python
> modules
> are you using?
>
>
> Graham
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mm_cfg_has_not_been_edited_to_set_host_domains/pipermail/mod_python/attachments/20070124/f5b92723/attachment.html
|