Dave DeMaagd created KAFKA-395:
----------------------------------
Summary: kafka.tools.MirrorMaker black/white list improvement
Key: KAFKA-395
URL: https://issues.apache.org/jira/browse/KAFKA-395
Project: Kafka
Issue Type: Improvement
Components: config
Reporter: Dave DeMaagd
Priority: Minor
Current black/white list topics are specified directly on the command line,
while functional, this has two drawbacks:
1) Changes become unwieldy if there are a large number of running instances -
potentially many instances to restart, which can have implications for data
stream lag
2) Maintaining the list itself can become increasingly complex if there are a
large number of elements in the list (particularly if they are complex
expressions)
Suggest extending the way that black/white lists can be fed to the mirror maker
application, in particular, being able to specify the black/white list as a
file (or possibly a generic URI). Thinking that this could be accomplished
either by adding '--whitelistfile' and '--blacklistfile' command line
parameters, or modifying the existing '--blacklist' and '--whitelist'
parameters to include a 'is this a valid file?' test and decide how to handle
it based on that (if it is a file, read it, if not, use current behavior).
Follow up suggestion would be to have the mirror maker process check for
updates to the list file, and on change, validate and reload it, and run from
that point with the new list information.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
