[
https://issues.apache.org/jira/browse/FLUME-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhipcui updated FLUME-3323:
---------------------------
Description:
As the doc said:
{noformat}
The Kafka channel can be used for multiple scenarios:
1. With Flume source and sink - it provides a reliable and highly available
channel for events
2. With Flume source and interceptor but no sink - it allows writing Flume
events into a Kafka topic, for use by other apps
3. With Flume sink, but no source - it is a low-latency, fault tolerant way to
send events from Kafka to Flume sinks such as HDFS, HBase or Solr{noformat}
Kafka channel will generates producer and consumer both now.
When using Kafka channel as scenario 2 (3), the producer(consumer) is just
useless, and if our kafka cluster enable SASL the situation is getting worse.
Assume we have a kafka account which only has read permissions, and using kafka
channel without source, producer shouldn't be generated.
I propose that add two properties to disable producer/consumer.
{noformat}
agent.channels.c.withSource = false
agent.channels.c.withSink= false{noformat}
was:
As the doc said:
{noformat}
The Kafka channel can be used for multiple scenarios:
1. With Flume source and sink - it provides a reliable and highly available
channel for events
2. With Flume source and interceptor but no sink - it allows writing Flume
events into a Kafka topic, for use by other apps
3. With Flume sink, but no source - it is a low-latency, fault tolerant way to
send events from Kafka to Flume sinks such as HDFS, HBase or Solr{noformat}
Kafka channel will generates producer and consumer both now.
When using Kafka channel as scenario 2 (3), the producer(consumer) is just
useless, and if ours kafka cluster is enable SASL situation is getting worse.
Assume we have a kafka account which only has read permissions, and using kafka
channel without source, producer shouldn't be generated.
I propose that add two properties to disable producer/consumer.
{noformat}
agent.channels.c.withSource = false
agent.channels.c.withSink= false{noformat}
> Kafka channel should supports disable producer/consumer according to the need
> -----------------------------------------------------------------------------
>
> Key: FLUME-3323
> URL: https://issues.apache.org/jira/browse/FLUME-3323
> Project: Flume
> Issue Type: Improvement
> Components: Kafka Channel
> Affects Versions: 1.9.0
> Reporter: zhipcui
> Priority: Minor
>
> As the doc said:
> {noformat}
> The Kafka channel can be used for multiple scenarios:
> 1. With Flume source and sink - it provides a reliable and highly available
> channel for events
> 2. With Flume source and interceptor but no sink - it allows writing Flume
> events into a Kafka topic, for use by other apps
> 3. With Flume sink, but no source - it is a low-latency, fault tolerant way
> to send events from Kafka to Flume sinks such as HDFS, HBase or Solr{noformat}
>
> Kafka channel will generates producer and consumer both now.
> When using Kafka channel as scenario 2 (3), the producer(consumer) is just
> useless, and if our kafka cluster enable SASL the situation is getting worse.
> Assume we have a kafka account which only has read permissions, and using
> kafka channel without source, producer shouldn't be generated.
> I propose that add two properties to disable producer/consumer.
>
> {noformat}
> agent.channels.c.withSource = false
> agent.channels.c.withSink= false{noformat}
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
