[jira] [Commented] (KAFKA-2077) Add ability to specify a TopicPicker class for KafkaLog4jApender
[ https://issues.apache.org/jira/browse/KAFKA-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992363#comment-14992363 ] Ashish K Singh commented on KAFKA-2077: --- Hey guys, this seems to be interesting. Is anyone, specifically [~benoyantony], working on this. If not, I would like to pick it up. Thanks! > Add ability to specify a TopicPicker class for KafkaLog4jApender > > > Key: KAFKA-2077 > URL: https://issues.apache.org/jira/browse/KAFKA-2077 > Project: Kafka > Issue Type: Improvement > Components: producer >Reporter: Benoy Antony >Assignee: Jun Rao > Attachments: KAFKA-2077.patch, kafka-2077-001.patch > > > KafkaLog4jAppender is the Log4j Appender to publish messages to Kafka. > Currently , a topic name has to be passed as a parameter. In some use cases, > it may be required to use a different topics for the same appender instance. > So it may be beneficial to enable KafkaLog4jAppender to accept TopicClass > which will provide a topic for a given message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2077) Add ability to specify a TopicPicker class for KafkaLog4jApender
[ https://issues.apache.org/jira/browse/KAFKA-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14624895#comment-14624895 ] Jason Gustafson commented on KAFKA-2077: [~benoyantony], I think KafkaLog4jAppender was moved to a separate module in KAFKA-2132. Can you update your patch? Add ability to specify a TopicPicker class for KafkaLog4jApender Key: KAFKA-2077 URL: https://issues.apache.org/jira/browse/KAFKA-2077 Project: Kafka Issue Type: Improvement Components: producer Reporter: Benoy Antony Assignee: Jun Rao Attachments: KAFKA-2077.patch, kafka-2077-001.patch KafkaLog4jAppender is the Log4j Appender to publish messages to Kafka. Currently , a topic name has to be passed as a parameter. In some use cases, it may be required to use a different topics for the same appender instance. So it may be beneficial to enable KafkaLog4jAppender to accept TopicClass which will provide a topic for a given message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2077) Add ability to specify a TopicPicker class for KafkaLog4jApender
[ https://issues.apache.org/jira/browse/KAFKA-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601987#comment-14601987 ] Benoy Antony commented on KAFKA-2077: - [~junrao], could I please get a review of this enhancement ? Add ability to specify a TopicPicker class for KafkaLog4jApender Key: KAFKA-2077 URL: https://issues.apache.org/jira/browse/KAFKA-2077 Project: Kafka Issue Type: Improvement Components: producer Reporter: Benoy Antony Assignee: Jun Rao Attachments: KAFKA-2077.patch, kafka-2077-001.patch KafkaLog4jAppender is the Log4j Appender to publish messages to Kafka. Currently , a topic name has to be passed as a parameter. In some use cases, it may be required to use a different topics for the same appender instance. So it may be beneficial to enable KafkaLog4jAppender to accept TopicClass which will provide a topic for a given message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2077) Add ability to specify a TopicPicker class for KafkaLog4jApender
[ https://issues.apache.org/jira/browse/KAFKA-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14509468#comment-14509468 ] Benoy Antony commented on KAFKA-2077: - Could I please get a review of this enhancement ? Add ability to specify a TopicPicker class for KafkaLog4jApender Key: KAFKA-2077 URL: https://issues.apache.org/jira/browse/KAFKA-2077 Project: Kafka Issue Type: Improvement Components: producer Reporter: Benoy Antony Assignee: Jun Rao Attachments: KAFKA-2077.patch, kafka-2077-001.patch KafkaLog4jAppender is the Log4j Appender to publish messages to Kafka. Currently , a topic name has to be passed as a parameter. In some use cases, it may be required to use a different topics for the same appender instance. So it may be beneficial to enable KafkaLog4jAppender to accept TopicClass which will provide a topic for a given message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2077) Add ability to specify a TopicPicker class for KafkaLog4jApender
[ https://issues.apache.org/jira/browse/KAFKA-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487557#comment-14487557 ] Benoy Antony commented on KAFKA-2077: - Yes. Add ability to specify a TopicPicker class for KafkaLog4jApender Key: KAFKA-2077 URL: https://issues.apache.org/jira/browse/KAFKA-2077 Project: Kafka Issue Type: Improvement Components: producer Reporter: Benoy Antony Assignee: Jun Rao Attachments: KAFKA-2077.patch, kafka-2077-001.patch KafkaLog4jAppender is the Log4j Appender to publish messages to Kafka. Currently , a topic name has to be passed as a parameter. In some use cases, it may be required to use a different topics for the same appender instance. So it may be beneficial to enable KafkaLog4jAppender to accept TopicClass which will provide a topic for a given message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2077) Add ability to specify a TopicPicker class for KafkaLog4jApender
[ https://issues.apache.org/jira/browse/KAFKA-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487555#comment-14487555 ] Benoy Antony commented on KAFKA-2077: - Use case is as below : We are using a system and we do not control its source code. The system has a Log4j logger which logs messages. Some of the messages has username as key and some of the messages has ip address as key. We like to publish these two types of messages to different topics so that they can be partitioned on different keys. An alternate approach is to associate 2 Log4j appenders. Then attach a Log4j filter to each appender. The filter drops one of the message types. Each appender publishes to a single topic. I found it to be slightly more complex and slower. Add ability to specify a TopicPicker class for KafkaLog4jApender Key: KAFKA-2077 URL: https://issues.apache.org/jira/browse/KAFKA-2077 Project: Kafka Issue Type: Improvement Components: producer Reporter: Benoy Antony Assignee: Jun Rao Attachments: KAFKA-2077.patch, kafka-2077-001.patch KafkaLog4jAppender is the Log4j Appender to publish messages to Kafka. Currently , a topic name has to be passed as a parameter. In some use cases, it may be required to use a different topics for the same appender instance. So it may be beneficial to enable KafkaLog4jAppender to accept TopicClass which will provide a topic for a given message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2077) Add ability to specify a TopicPicker class for KafkaLog4jApender
[ https://issues.apache.org/jira/browse/KAFKA-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14486562#comment-14486562 ] Jun Rao commented on KAFKA-2077: Thanks for the patch. The customizable key class seems useful. Could you explain a bit more on the use case to customize the topic for every message? Add ability to specify a TopicPicker class for KafkaLog4jApender Key: KAFKA-2077 URL: https://issues.apache.org/jira/browse/KAFKA-2077 Project: Kafka Issue Type: Improvement Components: producer Reporter: Benoy Antony Assignee: Jun Rao Attachments: KAFKA-2077.patch, kafka-2077-001.patch KafkaLog4jAppender is the Log4j Appender to publish messages to Kafka. Currently , a topic name has to be passed as a parameter. In some use cases, it may be required to use a different topics for the same appender instance. So it may be beneficial to enable KafkaLog4jAppender to accept TopicClass which will provide a topic for a given message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2077) Add ability to specify a TopicPicker class for KafkaLog4jApender
[ https://issues.apache.org/jira/browse/KAFKA-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14486575#comment-14486575 ] Jun Rao commented on KAFKA-2077: Also, does this patch cover KAFKA-2041 as well? Add ability to specify a TopicPicker class for KafkaLog4jApender Key: KAFKA-2077 URL: https://issues.apache.org/jira/browse/KAFKA-2077 Project: Kafka Issue Type: Improvement Components: producer Reporter: Benoy Antony Assignee: Jun Rao Attachments: KAFKA-2077.patch, kafka-2077-001.patch KafkaLog4jAppender is the Log4j Appender to publish messages to Kafka. Currently , a topic name has to be passed as a parameter. In some use cases, it may be required to use a different topics for the same appender instance. So it may be beneficial to enable KafkaLog4jAppender to accept TopicClass which will provide a topic for a given message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2077) Add ability to specify a TopicPicker class for KafkaLog4jApender
[ https://issues.apache.org/jira/browse/KAFKA-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14395257#comment-14395257 ] Benoy Antony commented on KAFKA-2077: - Thanks , I posted to reviewBoard - https://reviews.apache.org/r/32842/ Add ability to specify a TopicPicker class for KafkaLog4jApender Key: KAFKA-2077 URL: https://issues.apache.org/jira/browse/KAFKA-2077 Project: Kafka Issue Type: Improvement Components: producer Reporter: Benoy Antony Assignee: Jun Rao Attachments: KAFKA-2077.patch, kafka-2077-001.patch KafkaLog4jAppender is the Log4j Appender to publish messages to Kafka. Currently , a topic name has to be passed as a parameter. In some use cases, it may be required to use a different topics for the same appender instance. So it may be beneficial to enable KafkaLog4jAppender to accept TopicClass which will provide a topic for a given message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2077) Add ability to specify a TopicPicker class for KafkaLog4jApender
[ https://issues.apache.org/jira/browse/KAFKA-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14395256#comment-14395256 ] Benoy Antony commented on KAFKA-2077: - Created reviewboard https://reviews.apache.org/r/32842/diff/ against branch origin/trunk Add ability to specify a TopicPicker class for KafkaLog4jApender Key: KAFKA-2077 URL: https://issues.apache.org/jira/browse/KAFKA-2077 Project: Kafka Issue Type: Improvement Components: producer Reporter: Benoy Antony Assignee: Jun Rao Attachments: KAFKA-2077.patch, kafka-2077-001.patch KafkaLog4jAppender is the Log4j Appender to publish messages to Kafka. Currently , a topic name has to be passed as a parameter. In some use cases, it may be required to use a different topics for the same appender instance. So it may be beneficial to enable KafkaLog4jAppender to accept TopicClass which will provide a topic for a given message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2077) Add ability to specify a TopicPicker class for KafkaLog4jApender
[ https://issues.apache.org/jira/browse/KAFKA-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387690#comment-14387690 ] Sriharsha Chintalapani commented on KAFKA-2077: --- [~benoyantony] It will be helpful if you can send the patch using kafka-patch-review tool here are the instructions https://cwiki.apache.org/confluence/display/KAFKA/Kafka+patch+review+tool Add ability to specify a TopicPicker class for KafkaLog4jApender Key: KAFKA-2077 URL: https://issues.apache.org/jira/browse/KAFKA-2077 Project: Kafka Issue Type: Improvement Components: producer Reporter: Benoy Antony Assignee: Jun Rao Attachments: kafka-2077-001.patch KafkaLog4jAppender is the Log4j Appender to publish messages to Kafka. Currently , a topic name has to be passed as a parameter. In some use cases, it may be required to use a different topics for the same appender instance. So it may be beneficial to enable KafkaLog4jAppender to accept TopicClass which will provide a topic for a given message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
