|
Graham Dumpleton
grahamd at dscpl.com.au
Tue Jan 23 17:18:14 EST 2007
Suresh Lakshmanan wrote .. > Hi, > Is it possible for mod_python to create new subinterpreter when requests > arrive? I was thinking that mod_python has a configurable pool of python > interpreters > and it will delegate a request to an interpreter from the pool. > > But, it looks like this is not the case. Correct, that is not the case. You are misunderstand what the subinterpreters are there for. > Only one subinterpreter is only > created for a virtual host or directory config according to documentation. More or less, yes. Each sub interpreter is named and is tied to some part of the URL namespace. The default is that there will be a sub interpreter for each virtual host, but this can be overridden by various directives. > This is > creating problems because the other requests have to wait for a long time > and i get 503 from apache. The way Python sub interpreters are used, it is highly unlikely that a 503 response is going to be because of them. It may be how you have Apache configured, but to know you will have to state what platform you are using and if on UNIX whether you have compiled Apache with the prefork MPM or worker MPM. You also should post what you Apache configuration is that controls the number of Apache sub processes and/or threads within a process. > It will be useful if i have multiple > subinterpreters > to which apache can assign requests arbitrarily. I do not maintain state > in > any of the interpreters. This will not make any difference, what matters is how many Apache child process are allowed to be created when using prefork MPM, or how many child processes and/or threads with a process that Apache is allowed to create when using a multithreaded MPM. If using prefork MPM and you have restricted the number of Apache child processes to a small fixed number, then if you get more requests than that arrive, there will be no processes available to handle the request. If you are using a multithreaded MPM, then a small number of threads across all processes will limit how many requests can be handled concurrently. Creating extra Python subinterpreters only gives you separation of data, it doesn't provide any additional concurrency abilities. You seem not to understand this point. > iow, how do i scale this to a large number of users. how do i introduce > parallel processing of requests for the same virtual host/directory. at > the > moment it looks > like its single threaded. If you use prefork, then only one request in each child process can be handled at a time, but then you should be running with adequate child processes so that isn't a problem. If using a multithreaded MPM, then there already can be multiple requests handled at the same time and these can be handled even within the same Python subinterpreter. > definitely there should be a way as this is a common problem... can someone > please point me to the solution. You probably need to research more about the different MPM models for Apache. For starters, read: http://www.dscpl.com.au/wiki/ModPython/Articles/TheProcessInterpreterModel You might then see what you can find in documentation about Apache referenced on: http://wiki.apache.org/mod_python/ On first guess I would say how you have configured Apache is limiting things. If you are using a multithreaded MPM, then you might have a third party module for Python you are using which doesn't release the Python GIL around a long running operation being performed in C code. Thus, what extra Python modules are you using? Graham
|