> A database would not work because I wouldn't be able to > #1 make this easy tree structure to scan
If it 'works' as a directory, it is therefore hierarchical in nature and can be structured in a database. > #2 it will be smaller storage wise in files and in a DB > with all that overhead It has the appearances of that, but it is untrue. You're comparing just the size of the data files against the resulting database size. Of course these values are significantly different. The difference is the result of not including the size of the same indexing information pushed as inodes et. Al. on the disk. Including those in your calculation will result in similar sizes. Yes, with a database there will be some overhead. But that overhead will result in improved runtime performance; the average time to return a record from the database will be shorter than the average time on the filesystem. > #3 digging through the tree is faster > than an index scan in mysql, postgresq or DB2 with as many records as > I'll have. I don't believe you have the a) empirical evidence to back this up and more likely b) the experience in data modeling. If such a statement were true, everyone would be dumping databases in favor of building directory structures which mapped their data. > The projected database structure would be >=3Tb, but by using > directories as my index I can eleminate much of that overhead and shring > my storage requirement to a maintainable 750G. You don't learn that in > programming class, you learn that building financial software for the > last 10 years. Again, that's just the apparent disk consumption. Try creating your filesystem with that many nested directories and compare the usage on the disk itself. I think you'll be surprised to find the similar usage requirements. > You > #1 didn't have enough information to tell me how to arch. my software, I gave you the benefit of the doubt that you were not the architect of the software. With more than your 10 years I have enough experience to know that such filesystem usage is doomed to failure. > #2 assumed the wrong info and made the wrong suggestion And that would be the same suggestion pointed out by numerous other responders. How many responses saying 'yes your idea is good' have you received? If, in your 10 years of financial software development, you had ever worked with software that did try to use the filesystem in the manner that you plan, you wouldn't have had to ask the question in the first place. You'd already have the experience to know that, while it might work, it would suffer from numerous performance and stability issues that you won't be able to track down and resolve. > #3 didn't provide any helpful information I certainly did. I pointed you towards a much more workable system (a database-based one) than your unworkable one. > #4 attempted an insult on a list that is supposed to assist. Hmm, I didn't see any insult to you in my reply. It did suggest that you use a database for the implementation, and not the filesystem. That suggestion was meant to assist; you took it as an insult because you were offended that someone would point out flaws in your architecture. > Why even send your post? (don't bother with answer) Oh, but I love getting the last word in. -- gentoo-user@gentoo.org mailing list
