I would recommend to skip Chukwa 0.4 and go to the trunk. In addition, use HBaseWriter to stream data into HBase in parallel, hence, the data can be processed in near real time for demux.
Regards, Eric On 5/26/11 8:30 PM, "Bill Graham" <[email protected]> wrote: This seems possible, but one thing that would need to be changed is the directories that demux uses. For example: demuxProcessing/mrInput demuxProcessing/mrOutput These would need to dynamic directories with the timestamp or something else in them to keep two jobs from interfering with each other. On Thu, May 26, 2011 at 8:23 PM, Corbin Hoenes <[email protected]> wrote: Finding demux to be a bit too slow for our needs. It seems like only 1 runs at a time; is there some technical reason why we couldn't run a couple in parallel? If so any hints on how difficult it would be to run multiple demuxers at a time?
