Hello Boyang,
Thanks for brining this up. Currently since the StreamsPartitionAssingor
can only be initiated within the Consumer instance, I think letting users
to pass in the config values with prefix is the preferrable way, i.e. we
can improve our docs to educate users about that. BTW I'm curious what
configs you want to pass into the StreamsPartitionAssignor? Currently since
there is no user configurable part inside that class, I do not know what
configs can take effects.
Guozhang
On Mon, Jul 9, 2018 at 2:06 PM, Boyang Chen <bche...@outlook.com> wrote:
> Hey there,
>
>
> over the weekend I was debugging the streams configuration not passed
> within threads. I noticed that one of the code path from KafkaConsumer
> (L743) was to initialize the StreamPartitionAssignor:
>
> this.assignors = config.getConfiguredInstances(
> ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY_CONFIG,
> PartitionAssignor.class);
>
> However, it was using the ConsumerConfig instance (that config is passed
> in), so if I want to make some configuration change in the assignor, I need
> to put consumer prefix. To make the debugging even harder, there was an
> logAll() function in AbstractConfig which will print "StreamsConfig values"
> at the beginning, since it is indeed a stream config:
>
> @Override
> public void configure(final Map<String, ?> configs) {
> final StreamsConfig streamsConfig = new StreamsConfig(configs);
>
> (L190 in StreamPartitionAssignor)
>
>
> This would further confuse developer as they see two different sets of
> StreamsConfig: one from top level, one from this derived level per thread.
>
>
> My point is that we could either: 1. let developer be aware that they need
> to add consumer prefix to pass in configs to StreamPartitionAssignor 2. we
> found a way to pass in original StreamsConfig.
>
> I know this is a little bit lengthy description, let me know if you feel
> unclear about my proposal, or this is not a concern since most people
> already know the trick here, thank you!
>
>
--
-- Guozhang
