[
https://issues.apache.org/jira/browse/KAFKA-3472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214859#comment-15214859
]
Jiangjie Qin commented on KAFKA-3472:
-------------------------------------
[~granthenke] I think we can either use consumer interceptor or a plugin mirror
maker message handler.
However, both of them are consume-then-filter approach. If the mirror maker
consumer itself is not powerful enough to consume from the source cluster,
consumer interceptor or message handler approaches would not work.
Another hacky way is to simply use the mirror maker in the production
environment to do the sampling. e.g. let the message handler to instantiate an
internal producer and send the sampled messages using that producer. It should
have very little impact on the performance. This would work but is hacky. Also
if you need to change the sampling topic, you might have to stop mirror maker
unless the implementation of the message handler takes some dynamic
configuration.
> Allow MirrorMaker to copy selected partitions and choose target topic name
> --------------------------------------------------------------------------
>
> Key: KAFKA-3472
> URL: https://issues.apache.org/jira/browse/KAFKA-3472
> Project: Kafka
> Issue Type: Improvement
> Components: tools
> Affects Versions: 0.9.0.1
> Reporter: Hang Sun
> Priority: Minor
> Labels: mirror-maker
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> It would be nice if MirrorMaker can be used to copy only a few partitions
> instead of all to a different topic. My use case is to sample a small
> portion of production traffic in the pre-production environment for testing.
> The pre-production environment is usually smaller and cannot handle the full
> load from production.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
