I need some help with a database design question:
To summarize our design (I'll spare you a lot of the details.) We collect text files from 20,000 different sources every day. The actual file is stored as a text file. Each source has its own table which contains a list of the text files, date, title, url, etc. We thought about having one massive file for all this data, BUT, we archive something like 100,000 articles a day. We store articles for 6 months, so this table would very quickly have 18,000,000 rows. Now, we need to select a group of articles based on source and date. We also need to select based on url. I can only imagine how long it would take to search 18,000,000 rows for a specific url. Our CURRENT concept is to have one table for each of the 20,000 sources. This way we can just query the matching table for what we need. The problem is that 20,000 tables can be a bit unwieldy for mysql to handle. If you have any ideas, I would love to hear them. Thanks, -N --------------------------------------------------------------------- Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail <[EMAIL PROTECTED]> To unsubscribe, e-mail <[EMAIL PROTECTED]> Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
