It occurs to me that the unix os is basically a database in and of itself and perl interacts directly with the os, therefore, using it to store and retrieve data may not be that inefficient.
I agree with this - you can get good results with a well-planned directory structure.
Now, if you have one server dedicated to serving only 2500 users and in 2-3 years you have 5000 users and upgrade that server to one twice as fast and big, and so on....
This is true to a point, but disk drives haven't progressed at nearly the rate of CPU/RAM, so you could definitely start running into problems like this.
The main disadvantage of using a database engine like MySQL is that users cannot define data fields. If other applications are going to access the data in question than you must reformat it to provide the access. And again, I'm lazy (actually, I have other things I like to do) and really don't want to learn more. I'd rather use what I already know and leverage what I already have.
Since I don't know exactly what you're building here, it's hard to comment, but I agree, that is one area that hasn't been solved very well with relational databases, at least not in MySQL. If some of your users want columns different than others, you either need to split the tables somehow, or have all the columns (and maybe some extras) you think you may ever need available and only expose the ones particular users ask for. If anybody knows a cleaner way to do this, I'd love to hear it.
If Rick's "Dream" comes true I can just port the data at that time. There are a lot of programmers out there working on faster, easier to use, database engines that have more features. Chris, you may be right, XML may be a fad, but the next big thing in data storage/retrieval could be right around the corner too.
The above are some of the excuses I've come up with to avoid spending more time learning stuff. If I'm deluded, it's because I have boxes upon boxes of software that doesn't work anymore and time invested in each of them. It's not that I don't believe that MySQL and other database engines have a place, I'm just trying to avoid learning how to use them if I don't really need too.
Personally I think it's worth it in the case of MySQL (or other relational databases). The basics are pretty easily learned in an afternoon or two, and as your application and needs change, you'll definitely save yourself days worth of work by being able to leverage a good DB when your solution really calls for one.
Ian
