> In further digging, I have found Hadoop, which I assume some of you have > heard about (I was kind of surprised I had never come across it before). > > http://hadoop.apache.org > > Does anyone have experience with this platform? Positive or negative?
Hadoop is used for managing and processing massive amounts of data and is very efficient in doing parallel processing on that data. Think in terms of data work flows and data warehousing. It might be overkill for what you want. We looked at using it with the following items: hive, hdfs, hbase, mapreduce and pig. After we looked into in great detail we realized that we did not have the amount of data that would warrant such and enterprise system. Even though we have a ton of data into the TB on many of our databases, we came to the realization that hadoop is to much. Most relational dbs are very sufficient for very large data sets. I would recommend using one of them at the bigging and then if you need something bigger you can move to it. If you design your system with growth in mind then the migration to something bigger will not be hard. thanks, -- thebigdog _______________________________________________ UPHPU mailing list [email protected] http://uphpu.org/mailman/listinfo/uphpu IRC: #uphpu on irc.freenode.net
