Folks, i have two use-cases and both seem to be landing under this requirement
1. Support to publish files in HDFS in /flume-data/YYYY/MM/DD/HH/MN. Timestamp is the arrival time on this agent. >> Can be addressed by passing timestamp" in HEADERS of event. Caveat is i want to pass this header at the final agent in pipeline. 2. Have multiple flume agents configured behind a VIP writing to the same HDFS sink path. >> One of the way's is to have the path like - /flume-data/<flume-agent-hostname>/YYYY/MM/DD/HH/MN Again can be addressed by passing a header "hostname" at flume agent and configuring the sink path appropriately. Would appreciate any help on how to address this in a generic way in FLUME. Seems to be a generic use-case for anyone planning to take FLUME to production. -- Thanks, - Inder Tech Platforms @Inmobi Linkedin - http://goo.gl/eR4Ub
