Github user garrettlish commented on the issue:
https://github.com/apache/incubator-eagle/pull/556
Thx @haoch @RalphSu. I thought we set async for kafka producer and what we
need to change is using callback rather than future wait, it is wrong, thanks
for pointing out.
I have updated the code to add kafka_client_config (list of name/value map)
to specify kafka producer configurations.
By default, I only set producer.type=async.
For batch.num.messages, queue.buffering.max.ms and
queue.buffering.max.messages, I think we can use kafka producer default
values.
The only difference for default value is batch.num.messages, it is 200 if
not specified. Could u please share with us what is your reason to set it to
3000?
the kafka producer properties could be defined in publish properties as
follows:
{
"name": "***",
"properties" : {
"kafka_broker": "***",
"topics": "***",
"kafka_client_config" : [
{
"name" : "request.requrie.acks",
"value": 1
},
{
"name" : "producer.type",
"value": "async"
},
...
]
}
}
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
