[mod_python] Python, mod_python, Apache on WIN32 - threading - solutions found

Hi all,

This mail is basically intended for developpers of  NT or W2K -based 
web applications who are willing to test win+Python as a web platform
or have difficulties in running their mod_python apps under windows.

Just wanted to post a few words to say how great mod_python is and
share a bit of code...

As far we are concerned, this works great and fast.  
Our current development combination is:
- WIN2K server,
- apache 1.3
- mod_python,
- python
- mxodbc for db connectivity.

This combo works great for database-app applications requiring a lot of buffered 
databased-informations intensive sql querying.

It took us less than 2 months to do switch a whole demanding app from 
ms-tool based development on winK2 to apache+mod_python on winK2  .... 
including learning python, a short and enjoyable experience :)

The result is great and fast as all demanding queries are buffered in
global dictionnaries. The only problems we have encoutered are :
- threading,
- threading,
- threading.

Python is threaded, apache-win is threaded .. Be aware that
your mod_python code should also be. 

It took us A COUPLE OF WEEKS to grasp pbs that occured in 
demanding race-conditions and then solve them. 

This is the reason for this mail. Intended for those with limited
skills with thread. Programming data-intensive python+mod_python apps 
on win32 is an easy game including for programmers with limited IT background. 

BUT it is imperative to code threadsafe when coding wK2+apache+mod_python.

For those with no experience with building threadsafe code
applications (possibly the case if you work with interpreted languages),
a few recommendations for mod_python:
1) use as few globals as possible,
pass over your "state-variables" as local python objects and keep
shared variables to the absolute mimimum.

2) make sure to avoid race-conditions on those globals

if you need global (you will if you want high perfs - to store buffered database output), 
make sure those global variables are updated in a tight block of code with a lock
to prevent concurrent threads to access "shared info" whilst this
information is beeing updated

The kind of python code that helped us build threadsafe mod_python code:

##################################
# in mod_python init code
global_data_lock = threading.RLock()

##################################
# in your mod_python code
# run when needed
global global_data_lock 
global_data_lock.acquire()
try:
   # load or update your shared data .. dictionnaries and al
   load_shared_data()
finally:
   global_data_lock.release()

 
3 ) odbc connectivy (thru mxodbc) may or not be threadsafe

Encapsulate your odbc connections in a "pool" of connections
with round-robin access to them through a shared queue
(a "webwarish" bit of code, my thanks to webware, 
a great tool we'd like to investigate soon)

The kind of python code that helped us build threadsafe odbc python :

#################################
# in mod_python init code
# build a series of sleeping odbc connections
NB_CONNECTIONS = 20
connection_queue = Queue()
for each_value in range(NB_CONNECTIONS):
  database_handle = ODBC.Windows.Connect('my_dsn','my_user','my_password',0)
  connection_queue.put(database_handle) 

##################################
# in your database code
# use them in round-robin
# every time
global connection_queue
database_handle = connection_queue.get()
cursor_handle = database_handle.cursor()
cursor_handle.execute("SELECT * FROM DUMMY")
connection_queue.put(database_handle)

mod_python on win32 rocks!

François
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.modpython.org/pipermail/mod_python/attachments/20011107/e8538e9c/attachment-0003.htm