> Sometimes, if the customer wants to do an audit, we have to process > over a month's worth, so we are bound by the actual execution > time of the Perl script and the access time of a Sleepycat (Berkeley DB) > database. The Perl programs must be able to scale > with call/message volume, and right now the only thing we can do to > improve it is put faster processors on it. (We have 8-way boxes with > 64GB RAM, so 4GB hashes work just fine, but 1.5 hrs per day is still very > finite > when you have to baby sit scripts for a week to give customers the answers > they want to know). Granted, I could rewrite this stuff in C, but we > typically > modify these things on very short notice and Perl gives us the flexibility > to react quickly.
well you could opt for generated C (which is what I do). I generate function calls based on the content of the sql that is to be executed (ie: a md5 hash-like thingy) and bound back to flat files and/or perl data structures. Its a pain in the ****, but it works, and we are IO bound rather than processor bound on a system with *very* high IO throughput. We also generate the C from perl, of course. Anyways, I'd much rather be able to have perl be able to handle everything, for the reasons you mentioned. Its just too bad that its too slow for the task at hand, given that the system that we were using. Hence the post. > So, when we are discussing dates, I am one very interested party. > > -Melvin > > PS: Sorry I'm so vague about the numbers. The customers are very > sensitive about those numbers and I could get in trouble, but lets say > they are in the billions for a rather small time sample. (oops. didn't think of that one. fortunately, I was thinking about it some more, and my estimates I gave in my last email weren't completely accurate.. ;-)) Ed
