If I understand it right you want to have multiple instances of flume-ng (flume agents) to run on the source - such as webserver for example? but do not want all of them active at once? Is your goal is to achieve HA over flume agents or something else?
On Thu, Mar 31, 2016 at 1:21 AM, أنس الليثي <[email protected]> wrote: > Dears > > We are planning to use flume in consuming data from Gnip. We already > configured a flume agent that uses a customized source to read from gnip > and write the streamed data to HDSF. > > This agent now is considered a single point of failure in the > architecture. > If this agent goes down for any reason, no data will be read and the > system will not be updated. > > Is there a way to configure multiple agents doing the same job but only > one is working and the others in something like standby mode. If the > working agent is down for any reason, one of the other standby agents works > automatically. > > We can achieve this using a monitoring tool like nagios that monitor the > status of the agent and take an action based on that status. If the agent > is down for 3 subsequent checks, start another agent or restart the same > agent. > > Is there anything like this from flume side? something like a master flume > agent monitoring the other agents and so on ? > > Best regards >
