[
https://issues.apache.org/jira/browse/FLUME-1191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Andrews updated FLUME-1191:
----------------------------------
Description:
We have an agent running on each of our servers, and our servers are writing
audit data to these agents. The agents are configured to use autoE2EChain for
their sinks.
The problem is, out of 60+ servers, five of them generate the vast majority of
the traffic, and, with this many agents configured to write to our two
collectors, this means that there is a high probability (about one in three)
ofautoE2EChain assigning four or five of those servers to write to one
collector.
Ideally, we'd have a way to set the sink as: autoE2EChain("high-traffic") so
that some 'class' of agents would be assigned to collectors separately.
Alternatively, I suppose, autoE2EChain could pass a weight in: autoE2EChain(20).
I realize most work on the NG Flume, so perhaps the terminology will be
different there, but the idea is likely still applicable.
was:
We have an agent running on each of our servers, and our servers are writing
audit data to these agents. The agents are configured to use autoE2EChain for
their sinks.
The problem is, out of 60+ servers, five of them generate the vast majority of
the traffic, and, with this many agents configured to write to our two
collectors, this means that there is a high probability (about one in three) of
the autoE2EChain assigning four or five of those servers to write to one
collector.
Ideally, we'd have a way to set the sink as: autoE2EChain("high-traffic") so
that some 'class' of agents would be assigned to collectors separately.
Alternatively, I suppose, autoE2EChain could pass a weight in: autoE2EChain(20).
I realize most work on the NG Flume, so perhaps the terminology will be
different there, but the idea is likely still applicable.
> Would like autoE2EChain to have the ability to handle different classes of
> agents
> ---------------------------------------------------------------------------------
>
> Key: FLUME-1191
> URL: https://issues.apache.org/jira/browse/FLUME-1191
> Project: Flume
> Issue Type: New Feature
> Components: Sinks+Sources
> Affects Versions: v0.9.4
> Environment: CentOS
> Flume build:
> Flume 0.9.4-cdh3u3
> Git repository https://github.com/cloudera/flume/flume-core
> rev unknown
> Compiled by jenkins on 20120126-1114
> Reporter: Thomas Andrews
>
> We have an agent running on each of our servers, and our servers are writing
> audit data to these agents. The agents are configured to use autoE2EChain
> for their sinks.
> The problem is, out of 60+ servers, five of them generate the vast majority
> of the traffic, and, with this many agents configured to write to our two
> collectors, this means that there is a high probability (about one in three)
> ofautoE2EChain assigning four or five of those servers to write to one
> collector.
> Ideally, we'd have a way to set the sink as: autoE2EChain("high-traffic") so
> that some 'class' of agents would be assigned to collectors separately.
> Alternatively, I suppose, autoE2EChain could pass a weight in:
> autoE2EChain(20).
> I realize most work on the NG Flume, so perhaps the terminology will be
> different there, but the idea is likely still applicable.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
