> > I have been mulling over just that sort of thing. Some sort of > HadoopAppender that outputs files in the SequenceFile format and > periodically submits to a DFS node. Hey, this is something I _can_ > contribute! I might start another thread. Thanks. > > Paul
Look at scribe it was decided to handle messages and add them to hadoop in chunks. Also there is a hive jira that talkes about a similar feature "Super Highway" I think that is dealing with near real time <5 minute processing. Edward
