[mod_python] Concurrence and virtual host interpreter python

Pau Freixes wrote:
> Hi to all, Im new in a list.
> 
> Well, this last days I was work for testing and learn how work with multi
> thread and multi interpreter environment in embedding python in C
> applications, and of course mod_python it's a excellent place for
> understand this.

You'll need to throw multi-process into the mix of things to understand.
  I'm not actually sure if mod_python *is* an excellent environment for
your learning endeavours as there are a number of complicating factors 
introduced by apache and your browser.

The behaviour that you'll see will depend on which apache mpm you are
using: prefork, worker or winnt. If you haven't already done so you
should take a look at http://httpd.apache.org/docs/2.0/mpm.html

In order to effectively answer your questions you'll need to let us know
  us know which mpm you are using. Also, what are you using as your
client?

> Ok, with my lectures and test with mod_python code, i have a some
> questions, somebody can help me for understand this concepts ?
> 
> 1. I was try multi concurrence in a virtual host python script with this
> easy code
> 
> def handler(req):
> 
>     # Request Content Type
>     req.content_type = "text/plain"
> 
>     VALUE2 = 1
>     for i in range(1,10):
>         req.write("Value is %d\r\n" % VALUE2)
>         time.sleep(5)
> 
>         # change global variable
>         VALUE2 = VALUE2 + 1;
> 
> 
>     req.write("Hellow World")
>     return apache.OK
> 
> And my surprise it's that apache only process the next request when the
> previous request has been served. !!
> I don't understand this strange behavior, because mod_python it's made
> for accept concurrent threads in same interpreter, and i test a similar
> situation  in a c code and it's possible do run some threads at same
> time in same interpreter.
> 
> time.sleep(5) it's a pretty situation for python core for change the
> context and release GIL and assign to this to other thread. yes ?

Are you sure you're not seeing an artifact of the browser keeping the 
connection open? If this is the case then your request is being 
serialized by apache and the GIL doesn't enter into it.

You could play with your apache config (KeepAlive off and so on), but 
it's likely easier to use a different test client such as wget. Also, 
modify your req.write() calls to immediately flush the data, which will 
make it easier to see what's going on. eg. req.write("something",1)

Open multiple terminals and run the following in each simultaneously (or 
as near as you can make it).

wget -S -O - http://..../index.py

I think you'll see that mod_python is behaving exactly as you'd expect.

> 
> 2. And de second question it's in similar situation, when you reuse a
> interpreter for all request to same virtual host, with this code :
> 
> import time
> 
> VALUE = 1
> 
> def handler(req):
>     global VALUE
> 
>     # Request Content Type
>     req.content_type = "text/plain"
> 
>     for i in range(1,10):
>         req.write("Value is %d\r\n" % VALUE)
>         time.sleep(5)
> 
>         # change global variable
>         VALUE = VALUE + 1;
> 
> 
>     req.write("Hellow World")
>     return apache.OK
> 
> VALUE it's a global variable for de index.py, and subsequent request for
> the same url (http://...../index.py) show one incremental value for
> VALUE. For example, this two request produced this two results
> 
> http://...../index.py
> 
> Value is 1
> Value is 2
> Value is 3
> Value is 4
> Value is 5
> Value is 6
> Value is 7
> Value is 8
> Value is 9
> Hellow World
> 
> http://...../index.py
> 
> Value is 10
> Value is 11
> Value is 12
> Value is 13
> Value is 14
> Value is 15
> Value is 16
> Value is 17
> Value is 18
> Hellow World
> 
> 
> Ok, this situation it's really and i think its very normal because only
> callback to handler it's made with the same interpreter and VALUE is
> not re assigned to value 1, may be ?

The statement "VALUE = 1" is only evaluated when the module (index.py)
is first loaded. It is not re-evaluated for each request. Note however
that VALUE should *not* be considered a global variable. Depending on
which apache mpm is being used you could have multiple, independent
copies of VALUE. Using the mpm-prefork and the default MaxClients value
of 256, you could potentially have 256 copies of VALUE if your server is
under a heavy load. A similar situation exists with mpm-worker where you
could still have a number of independent multi-threaded child processes.

> 3. And the most important question, why mod_python not use a separate
> interpreters for all request ?

I'm not 100% sure what you are asking. Do you mean a separate
interpreter for each request? If so then the resource usage would be
unacceptably high. The default is to have a separate subinterpreter for
each virtual host.

Jim