[
https://issues.apache.org/jira/browse/KAFKA-18053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
A. Sophie Blee-Goldman updated KAFKA-18053:
-------------------------------------------
Description:
As mentioned briefly in KIP-1112, the current API around configs which need to
be applied prior to building the topology can be cumbersome and easy to
implement incorrectly. This leads to awkward code and confused users at best,
and applications that silently drop some configs at worst.
There are three main problems we want to solve:
# The most serious issue is that the KafkaStreams constructor requires the
application configs whereas the topology can be built without them, so most
people will only pass their configs into KafkaStreams. However certain configs
have to be passed into the topology as well, which is easy for users to miss
and leads to a silent misconfiguration that can easily go unnoticed
# Over time we've added several new APIs for configs that need to be applied
at different/earlier stages of the topology building process. So now users are
faced with multiple ways of passing these configs into the topology, and
configs that may work with any of them or may need to be used with a specific
API
# At some point we also made the TopologyConfig class public, which was
originally intended only for internal use. Furthermore it's only purpose was to
separate configs between different topologies, a feature that has now been
deprecated and will soon be removed, at which point TopologyConfig serves no
purpose and should be removed as well. Unfortunately, the newest APIs for
passing configs into a topology are a Topology and StreamsBuilder constructor
that have a TopologyConfig parameter, rather than the StreamsConfig parameter
used in the previous API (ie StreamsBuilder#build). This is unnecessary since
StreamsConfig would work just as well, and in fact a TopologyConfig has to be
constructed from a StreamsConfig so users are just adding a wrapper layer
As of filing this ticket, the list of topology-specific configs is:
* topology.optimization
* processor.wrapper.class
* default.dsl.store
* dsl.store.suppliers.class
was:
As mentioned briefly in KIP-1112, the current API around configs which need to
be applied prior to building the topology can be cumbersome and easy to
implement incorrectly. This leads to awkward code and confused users at best,
and applications that silently drop some configs at worst.
There are three main problems we want to solve:
# The most serious issue is that the KafkaStreams constructor requires the
application configs whereas the topology can be built without them, so most
people will only pass their configs into KafkaStreams. However certain configs
have to be passed into the topology as well, which is easy for users to miss
and leads to a silent misconfiguration that can easily go unnoticed
# Over time we've added several new APIs for configs that need to be applied
at different/earlier stages of the topology building process. So now users are
faced with multiple ways of passing these configs into the topology, and
configs that may work with any of them or may need to be used with a specific
API
# At some point we also made the TopologyConfig class public, which was
originally intended only for internal use. Furthermore it's only purpose was to
separate configs between different topologies, a feature that has now been
deprecated and will soon be removed, at which point TopologyConfig serves no
purpose and should be removed as well. Unfortunately, the newest APIs for
passing configs into a topology are a Topology and StreamsBuilder constructor
that have a TopologyConfig parameter, rather than the StreamsConfig parameter
used in the previous API (ie StreamsBuilder#build). This is unnecessary since
StreamsConfig would work just as well, and in fact a TopologyConfig has to be
constructed from a StreamsConfig so users are just adding a wrapper layer
As of filing this ticket, the list of topology-specific configs is:
*
topology.optimization
* processor.wrapper.class
*
default.dsl.store
*
dsl.store.suppliers.class
> Clean up TopologyConfig and API for supplying configs needed by the topology
> ----------------------------------------------------------------------------
>
> Key: KAFKA-18053
> URL: https://issues.apache.org/jira/browse/KAFKA-18053
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: A. Sophie Blee-Goldman
> Priority: Major
> Labels: needs-kip
>
> As mentioned briefly in KIP-1112, the current API around configs which need
> to be applied prior to building the topology can be cumbersome and easy to
> implement incorrectly. This leads to awkward code and confused users at best,
> and applications that silently drop some configs at worst.
> There are three main problems we want to solve:
> # The most serious issue is that the KafkaStreams constructor requires the
> application configs whereas the topology can be built without them, so most
> people will only pass their configs into KafkaStreams. However certain
> configs have to be passed into the topology as well, which is easy for users
> to miss and leads to a silent misconfiguration that can easily go unnoticed
> # Over time we've added several new APIs for configs that need to be applied
> at different/earlier stages of the topology building process. So now users
> are faced with multiple ways of passing these configs into the topology, and
> configs that may work with any of them or may need to be used with a specific
> API
> # At some point we also made the TopologyConfig class public, which was
> originally intended only for internal use. Furthermore it's only purpose was
> to separate configs between different topologies, a feature that has now been
> deprecated and will soon be removed, at which point TopologyConfig serves no
> purpose and should be removed as well. Unfortunately, the newest APIs for
> passing configs into a topology are a Topology and StreamsBuilder constructor
> that have a TopologyConfig parameter, rather than the StreamsConfig
> parameter used in the previous API (ie StreamsBuilder#build). This is
> unnecessary since StreamsConfig would work just as well, and in fact a
> TopologyConfig has to be constructed from a StreamsConfig so users are just
> adding a wrapper layer
> As of filing this ticket, the list of topology-specific configs is:
> * topology.optimization
> * processor.wrapper.class
> * default.dsl.store
> * dsl.store.suppliers.class
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
