[mod_python] mod_python executing old versions of my code

Graham Dumpleton wrote:
> On 26/01/2006, at 5:40 PM, Mike Looijmans wrote:
> 
>> When faced with the reload problem a few years ago, I created a 
>> special "reload" request. The request would list all the .py[oc] files 
>> in the script folder, and tries to unload or reload() each of them.
> 
> Will not work for mod_python.publisher in mod_python 3.2 and possibly
> not at all in future versions of mod_python, but for mod_python <= 3.1.4,
> it is better to iterate over modules in sys.modules and if they contain
> the attribute "__mtime__" set it to 0. By doing this, it will cause the
> mod_python importer to reload it automatically the next time it is 
> accessed.
> By doing it this way, you don't expose yourself to issues that might arise
> from mixing "apache.import_module()" and "reload()".

I was not mixing - I was using "import" exclusively. I also wasn't using 
publisher, but I wrote my own. (this was about 4 years ago)

"My" publisher could detect changes in the handlers, but not in the 
sub-modules they imported. 90% of code changes only affected the UI 
parts which were in the headers.

The fix was there to get changes in the library modules to become 
effective, without having to log in on the apache machine.


>> As a result, the "main" modules as well as dependent modules will have 
>> been reloaded or removed (At least, the ones that came from my project 
>> - it would not make sense to unload modules like 'os').
> 
> Other projects should really be run in the context of a different 
> interpreter
> by specifying PythonInterpreter. On that basis, you would have to be 
> selective
> if above method used.

There were no other projects on that server...

>> So just sending "http://myhost/pu/reload" to the server would cause 
>> the next request to get all new scripts.
> 
> This is only guaranteed to work if you are using mod_python on Win32 
> platform.
> It most definitely will not work with "prefork" on UNIX and usually not on
> "worker" on UNIX. The reason the latter will not work reliably is that 
> there
> can be multiple Apache child processes and your request will only hit 
> one of
> those child processes. You would have to hit reload many times and even 
> then
> you wouldn't know for sure it had been done in all of the children.

True.
I was using Apache 2.0.something and Solaris 2.6 at the time (had to 
compile everything, no binary distributions), but I already forced 
apache to use a single-process multi-threading worker. Benchmarks showed 
no real difference with the prefork mpm. The machine was seriously 
limited by memory, and the childs would take up >12 MB of memory each, 
most of which could be shared. Putting it into threading mode solved the 
memory hogging, and also had advantages in caching stuff in simple 
dictionaries in memory. I also did DB connection pooling, but after 
implementing and doing some benchmarks, there was no real performance 
advantage in doing it on the MySQL database, which is amazingly fast in 
setting up connections.

Getting this to work on a prefork system is indeed quite a challenge...

But at the time, it was a much better option than restarting an Apache 
server process when hundreds of machines are using it...

--
Mike Looijmans
Philips Natlab / Topic Automation