[mod_python] Interpreters

On Thu, 22 May 2003, Stefan C. Kremer wrote:

> When I start Apapche mod_python not yet initialized.
>   It only gets initialized when someone accesses a web-page
>   associated with a directory directive that requires mod_python.
> Is that right?

yes

> When mod_python is initialized it creates an interpreter called
> "main_interpreter".  This interpreter doesn't actually run your python
> code it just generates sub interpreters which it keeps in a dictionay.

correct

> Where is the main interpreter named "main_interpreter"?  Can I access that
> somewhere?

It is named inside the C code for mod_python. If force your interpreter
name by using "PythonInterpreter main_interpreter", then mod_python won't
create subinterpreters, it will just select "main_interpreter" to run
things.

> If apache is running as multiple processes and maybe multiple threads how
> many main_interpreters are there?  Just one?

As many as there are processes.

> Asside: req.interpreter doesn't work for me (under 3.0.0 (was it
> added in 3.0.1 or 3.0.2) or am I confused?

3.0.0 is a beta version, you should stay away from it anyway.

> When do the sub-interpreters get created?

on first use

> The sub-interpreters are, by default, associated associated with a virtual
> server name (but there can be more than one sub-interpreter per virtual
> server name).
>
> Is that right?

yes, if you use InterpPerDirective or InterpPerDirectory

> If there are mutliple processes of apache with multiple threads, are the
> sub-interpreters shared betweenn processes and/or threads?

between threads within a process - yes, between processes - no.

> When a http request comes in that uses a mod_python handler, it is
> assigned to a sub-interpreter.  It might be assigned to a brand new
> sub-interpreter (usually if traffic on the server is reaching a new high
> level), or a previously used sub-interpreter (if traffic is less than it
> was at peak).  (Assuming we keep hitting the same virtual
> server/directory/etc.)
>
> Is that right?

sort-of... Apache will assign a request to one of it's children/threads to
be serviced. Mod_python will determine subinterpreter name based on
config, if this this subinterpreter already exists within this child, it's
reused, otherwise a new one is created. Server load is only one of a few
reasons why there may be a new child process.

> Sub-interpreters are never destoryed, so the number of sub-interpreters
> increases monotonically from when apache is started until it dies and
> your chances of getting a used sub-interpreter goes up the longer apache
> has been running.

true (note though that there is a finite number of subinterpeters, e.g. if
you're using the default interp per vrtual host, then there will only be
as many subinterpreters per process as there are virtual hosts - thus it
increases up to a point when all possible subinterpeters are created, then
it stops)

> Also the longer apache is running the more memory mod_python would
> proabely use.

this depends on how your code is written. mod_python itself has no known
memory leaks - but if you fnd any, let us know!

> If apache reduces (kills) the processes it is running, are any
> sub-interpreters affected?

well - everything in the process (inclduing subinterpreters) is affected.

> If you happen to get a re-used sub-interpreter, you will also get saddled
> with stuff that was defined in it before.

yes

> This can be a good thing or a bad thing.  If you are accessing a database,
> the sub-interpreter may already have an open DB connection which can save
> you time.  It may also have already loaded some of the modules that your
> code calls, which can save time.  Unfortunately this can be bad as well.
>
> One case occurs when you modify python code that is included as a module.
> If you get a sub-interpreter where the original version of the module
> has already been included, it will use that instead of the new one.

But how is this bad? This is one of the main reasons mod_python is so fast
- module code is only interpreted once (as opposed to every request with
CGI).

> Another example of when getting a used sub-interpreter is bad might be:
> User 1 writes a script that changes the sys.path variable and loads a
> module that they have written called MyModule.  User 2 writes a script
> that changes the sys.path variable and loads a different module that thay
> have written called MyModule also!

this is precisely why you have subinterpreters - you need to make sure
that the users are running in separate subinterpreters.

> Obviously the solution to the above problem is to use a different
> sub-interpreter naming scheme with "PythonInterpPerDirectory or
> PythonInterpPerDirective".

exactly!

Grisha