Hello all, Is anyone using Hadoop as more of a near/almost real-time processing of log data for their systems to aggregate stats, etc? I know that Hadoop has generally been good at off-line processing of large amounts of data, but I've wondered if anyone has tried using it for processing of near real-time log data as it is appears in your systems with any success? My gut feeling is that Hadoop isn't suitable for this yet given redundancy issues around the JobTracker/NameNode, as well as the overhead of moving blocks around in HDFS. Thoughts?
Thanks, Ryan
