[
https://issues.apache.org/jira/browse/CHUKWA-707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shreyas subramanya updated CHUKWA-707:
--------------------------------------
Attachment: CHUKWA-707.patch2
I have modified the patch to create topics as CHUKWA-datatype-host. Each chukwa
agent consumer thread will only subscribe to topics matching its own host
according to the regex CHUKWA-(.*)-currentHost.
The messages will have timestamp as their key. This design will also ensure
that any streaming software can perform computation on a datatype stream by
subscribing to topics CHUKWA-datatype-.*
Currently the topic name and key format is defined by KafkaKeyFormatter class.
This can be made configurable in future when there is a need for it.
> Replace Chukwa collector with Apache Kafka
> ------------------------------------------
>
> Key: CHUKWA-707
> URL: https://issues.apache.org/jira/browse/CHUKWA-707
> Project: Chukwa
> Issue Type: New Feature
> Reporter: Eric Yang
> Assignee: shreyas subramanya
> Attachments: CHUKWA-707.patch1, CHUKWA-707.patch2
>
>
> Chukwa collector has stopped evolving since 2010. Newer framework has offer
> better features of message queues, and Apache Kafka looks like a good
> replacement for Chukwa collector.
> Chukwa agent can implement a connector to Apache Kafka to replace Chukwa
> collector, and HBase consumer to write data to HBase. HICC REST API change
> to new HBase storage format.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
