[mod_python] Problem with PSP and unicode

Well, in fact, I'm not using PSP, but my own templating system. The
whole code is around 500 lines of pure Python. It uses a regular
expression based tokenizer, a recursive hand-written parser (the
grammar is simple enough), and the result of a template compilation is
an optimized (simplified) AST that is visited each time the template
needs to be evaluated (it is not Python code that gets compiled
afterwards, I don't feel the need to do so).

The template markup are close to those used in the Velocity Java
template engine, that is to say you cannot insert any Python code in
the template, you have to play by some rules as far as flow control is
concerned.

However, the evaluation tags can contain full Python evaluation, so
they are not as limited as the one you can find in Zope's TAL, Kid or
Django. I know they are limited on purpose, but I can't bear to be
forced to jump through hoops to please a development philosophy that
mostly gets in the way IMHO. There is no need to prevent your
developers to write full blown Python expression in their templates ;
if they have a little discipline, it is a real time saver.

The template are used by modules published by the
mod_python.publisher. Like it was mentioned in a thread a few weeks
ago, I use a Python decorator to specify that the dictionary returned
by a published function or method must be used with a given template -
the decorator handle all the fuss about computing the relative path of
the template, and calling the template cache.

All this is not perfect yet but I've been happily using it for nearly
a year now on various projects which are currently in production
(though under a quite moderate load so I cannot boast about its
performance apart that saying it is not a problem at all).

All this to say that yes, writing an efficient pure Python template is
possible, there already exists a zillion and if none please you, then
with about 500 lines of code you have something good enough. Now, why
should we have to maintain some fairly obscure C code for the PSP
module, that eludes me  :).

Regards,
Nicolas

2006/2/16, Nick <nick at dd.revealed.net>:
> Nicolas Lehuen wrote:
> > If it were up to me, I would reimplement it in pure Python, and I
> > don't think the performance loss would be so big. Granted, compilation
> > would be a little slower, but once the PSP is compiled, performance
> > should be exactly the same - and implementing a compiler cache is
> > extremely easy. Plus, the code being more easy to maintain, we could
> > easily optimize it. Call it "doing it the PyPy way" if you like ;).
>
> The original PSE parser I did worked very similar to PSP.  I used HTMLParser
> in the standard lib to do it.  However, it was different in that it was not
> PHP/ASP-ish where code could just be stuck anywhere; it had to be parsable
> HTML.  But I suspect you could just use a regular expression to do what the
> PSP parser does.
>
> Nick
>