My problem is that linotype is too slow for the load that my blog generates and runs at 98% of server CPU. The reason for this is that very little caching is performed. I could pregenerate the pages and let them served static, but that would prevent me from playing with run-time data like the referring request headers.
Now, the core of linotype is the following pipeline:
1) scans the directories that hold the news 2) includes them 3) filters them
as you can easily see, the approach is totally wrong: filtering should happen *before* the inclusion... the problem is that since the file system is such a stupid repository, metadata is included in the files, so I have to include them before I can filter them.
The use of a serious repository and a serious query language on top of the metadata will solve that issue and allow to scale a few orders of magnitue more in terms of requests, which will keep me happy for a while.
But still, thinking a little bigger than my own blog, the ideal solution would be to avoid querying entirely if there is no need to.
Now, the way the event cache works is like this:
a) a cache validity is generated b) pipeline is executed c) result is stored in the cache
then the pipeline is never called, until an event is triggered externally (from an avalon component) that invalidates that particular cache entity.
Now, the problem is: if I am not the one who generates the validity (in linotype, it's the directory generator), how can I invalidate it? how can I have access to it?
I'm starting to think that the cocoon cache needs a serious redesign to deal with these issues... and this scares me :-( but it's *waay* too important to let go.
Thoughts?
-- Stefano.
