something like dynamic filtering that can be updated at runtime or deny all
but allow a certain set of topics that cannot be specified easily by regex

On Thu, Mar 12, 2015 at 9:06 PM, Guozhang Wang <wangg...@gmail.com> wrote:

> Hmm, what kind of customized filtering do you have in mind? I thought with
> "--whitelist" you could already specify regex to do filtering.
>
> On Thu, Mar 12, 2015 at 5:56 AM, tao xiao <xiaotao...@gmail.com> wrote:
>
> > Hi Guozhang,
> >
> > I was meant to be topicfilter not topic-count. sorry for the confusion.
> > What I want to achieve is to pass my own customized topicfilter to MM so
> > that I can filter out topics what ever I like. I know MM doesn't support
> > this now. I am just thinking if this is a good feature to add in
> >
> > On Thu, Mar 12, 2015 at 8:24 PM, Guozhang Wang <wangg...@gmail.com>
> wrote:
> >
> > > Hi Tao,
> > >
> > > Sorry I was mistaken before, yes in MM you can only directly specify
> > > "--whitelist", "--blacklist", and  the number of streams you want to
> > create
> > > via "--num.streams", but cannot set specific topic-count. This is
> because
> > > MM is mainly used for cross DC replication, and hence usually will pipe
> > all
> > > topics or a majority of the topics from the source cluster to the
> > > destination, hence usually you do not care about "some topics should
> get
> > X
> > > streams, while some other topics should get Y streams".
> > >
> > > Guozhang
> > >
> > > On Wed, Mar 11, 2015 at 11:59 PM, tao xiao <xiaotao...@gmail.com>
> wrote:
> > >
> > > > The topic list is not specified in consumer.properties and I don't
> > think
> > > > there is any property in consumer config that allows us to specify
> what
> > > > topics we want to consume. Can you point me to the property if there
> is
> > > > any?
> > > >
> > > > On Thu, Mar 12, 2015 at 12:14 AM, Guozhang Wang <wangg...@gmail.com>
> > > > wrote:
> > > >
> > > > > Tao,
> > > > >
> > > > > In MM people can pass in consumer configs, in which people can
> > specify
> > > > > consumption topics, either in regular topic list format or
> whitelist
> > /
> > > > > blacklist. So I think it already does what you need?
> > > > >
> > > > > Guozhang
> > > > >
> > > > > On Tue, Mar 10, 2015 at 10:09 PM, tao xiao <xiaotao...@gmail.com>
> > > wrote:
> > > > >
> > > > > > Thank you guys for answering. I think it will be good that we can
> > > pass
> > > > > in a
> > > > > > customised topicCount ( I think this is the interface whitelist
> and
> > > > > > backlist implement if I am not mistaken) to MM to achieve similar
> > > thing
> > > > > >
> > > > > > On Wednesday, March 11, 2015, Guozhang Wang <wangg...@gmail.com>
> > > > wrote:
> > > > > >
> > > > > > > Hi Tao,
> > > > > > >
> > > > > > > Unfortunately MM does not support whitelist / blacklist at the
> > same
> > > > > time,
> > > > > > > and you have to choose either one upon initialization. As for
> > your
> > > > > case,
> > > > > > I
> > > > > > > think it can be captured by some reg-ex to exclude nothing else
> > but
> > > > > "10",
> > > > > > > but I do not know the exact expression.
> > > > > > >
> > > > > > > Guozhang
> > > > > > >
> > > > > > > On Tue, Mar 10, 2015 at 7:58 AM, tao xiao <
> xiaotao...@gmail.com
> > > > > > > <javascript:;>> wrote:
> > > > > > >
> > > > > > > > I actually mean if we can achieve this in mirror maker.
> > > > > > > >
> > > > > > > > On Tue, Mar 10, 2015 at 10:52 PM, tao xiao <
> > xiaotao...@gmail.com
> > > > > > > <javascript:;>> wrote:
> > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > I have an user case where I need to consume a list topics
> > with
> > > > name
> > > > > > > that
> > > > > > > > > matches pattern topic.* except for one that is topic.10. Is
> > > > there a
> > > > > > way
> > > > > > > > > that I can combine the use of whitelist and blacklist so
> > that I
> > > > can
> > > > > > > > achieve
> > > > > > > > > something like accept all topics with regex topic.* but
> > exclude
> > > > > > > topic.10?
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Regards,
> > > > > > > > > Tao
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Regards,
> > > > > > > > Tao
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > -- Guozhang
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Regards,
> > > > > > Tao
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > -- Guozhang
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > > Tao
> > > >
> > >
> > >
> > >
> > > --
> > > -- Guozhang
> > >
> >
> >
> >
> > --
> > Regards,
> > Tao
> >
>
>
>
> --
> -- Guozhang
>



-- 
Regards,
Tao

Reply via email to