[
https://issues.apache.org/jira/browse/EAGLE-670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15601509#comment-15601509
]
ASF GitHub Bot commented on EAGLE-670:
--------------------------------------
Github user garrettlish commented on the issue:
https://github.com/apache/incubator-eagle/pull/556
Thx @haoch @RalphSu. I thought we set async for kafka producer and what we
need to change is using callback rather than future wait, it is wrong, thanks
for pointing out.
I have updated the code to add kafka_client_config (list of name/value map)
to specify kafka producer configurations.
By default, I only set producer.type=async.
For batch.num.messages, queue.buffering.max.ms and
queue.buffering.max.messages, I think we can use kafka producer default
values.
The only difference for default value is batch.num.messages, it is 200 if
not specified. Could u please share with us what is your reason to set it to
3000?
the kafka producer properties could be defined in publish properties as
follows:
{
"name": "***",
"properties" : {
"kafka_broker": "***",
"topics": "***",
"kafka_client_config" : [
{
"name" : "request.requrie.acks",
"value": 1
},
{
"name" : "producer.type",
"value": "async"
},
...
]
}
}
> AlertEngine: Make Kafka Publisher configurable to async, for throughput tuning
> ------------------------------------------------------------------------------
>
> Key: EAGLE-670
> URL: https://issues.apache.org/jira/browse/EAGLE-670
> Project: Eagle
> Issue Type: Task
> Affects Versions: v0.5.0
> Reporter: Su Ralph
> Assignee: Garrett Li
> Fix For: v0.5.0
>
>
> Kafka send alert in sync would limit the throughput. Make this configurable,
> and use async by default.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
