[jira] [Updated] (FLINK-27872) Allow KafkaBuilder to set arbitrary subscribers
[
https://issues.apache.org/jira/browse/FLINK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Salva updated FLINK-27872:
--
Description:
Currently,
[KafkaSourceBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has two setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding (concrete) subscribers. This
covers the most common needs, I agree, but it might fall short in some cases.
Why not add a more generic setter:
* {{setSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
was:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has two setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding (concrete) subscribers. This
covers the most common needs, I agree, but it might fall short in some cases.
Why not add a more generic setter:
* {{setSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
> Allow KafkaBuilder to set arbitrary subscribers
> ---
>
> Key: FLINK-27872
> URL: https://issues.apache.org/jira/browse/FLINK-27872
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Kafka
>Reporter: Salva
>Priority: Major
>
> Currently,
> [KafkaSourceBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
> has two setters:
> *
> [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
> *
> [setTopicPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
> which under the hood instantiate the corresponding (concrete) subscribers.
> This covers the most common needs, I agree, but it might fall short in some
> cases. Why not add a more generic setter:
> * {{setSubscriber (???)}}
> Otherwise, how can users read from kafka in combination with custom
> subscribing logic? Without looking much into it, it seems that they basically
> cannot, at least without having to replicate some parts of the connector,
> which seems rather inconvenient.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (FLINK-27872) Allow KafkaBuilder to set arbitrary subscribers
[
https://issues.apache.org/jira/browse/FLINK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Salva updated FLINK-27872:
--
Description:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has two setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding (concrete) subscribers. This
covers the most common needs, I agree, but it might fall short in some cases.
Why not add a more generic setter:
* {{setSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
was:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has two setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding (concrete) subscribers. This
covers the most common needs, I agree, but it might fall short in some cases.
Why not add a more generic setter:
* {{setKafkaSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
> Allow KafkaBuilder to set arbitrary subscribers
> ---
>
> Key: FLINK-27872
> URL: https://issues.apache.org/jira/browse/FLINK-27872
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Kafka
>Reporter: Salva
>Priority: Major
>
> Currently,
> [KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
> has two setters:
> *
> [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
> *
> [setTopicPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
> which under the hood instantiate the corresponding (concrete) subscribers.
> This covers the most common needs, I agree, but it might fall short in some
> cases. Why not add a more generic setter:
> * {{setSubscriber (???)}}
> Otherwise, how can users read from kafka in combination with custom
> subscribing logic? Without looking much into it, it seems that they basically
> cannot, at least without having to replicate some parts of the connector,
> which seems rather inconvenient.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (FLINK-27872) Allow KafkaBuilder to set arbitrary subscribers
[
https://issues.apache.org/jira/browse/FLINK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Salva updated FLINK-27872:
--
Description:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has two setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding (concrete) subscribers. This
covers the most common needs, I agree, but it might fall short in some cases.
Why not add a more generic setter:
* {{setKafkaSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
was:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has two setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding (concrete) subscribers. This
covers the most common needs, I agree, but it might fall short in some cases.
Why not add a more generic setter:
* {{setKafkaSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
> Allow KafkaBuilder to set arbitrary subscribers
> ---
>
> Key: FLINK-27872
> URL: https://issues.apache.org/jira/browse/FLINK-27872
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Kafka
>Reporter: Salva
>Priority: Major
>
> Currently,
> [KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
> has two setters:
> *
> [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
> *
> [setTopicPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
> which under the hood instantiate the corresponding (concrete) subscribers.
> This covers the most common needs, I agree, but it might fall short in some
> cases. Why not add a more generic setter:
> * {{setKafkaSubscriber (???)}}
> Otherwise, how can users read from kafka in combination with custom
> subscribing logic? Without looking much into it, it seems that they basically
> cannot, at least without having to replicate some parts of the connector,
> which seems rather inconvenient.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (FLINK-27872) Allow KafkaBuilder to set arbitrary subscribers
[
https://issues.apache.org/jira/browse/FLINK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Salva updated FLINK-27872:
--
Description:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has two setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding (concrete) subscribers. This
covers the most common needs, I agree, but it might fall short in some cases.
Why not add a more generic setter:
* {{setKafkaSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
was:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has two setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding (concrete) subscribers. This
should cover the most common needs, I agree, but it might fall short in some
cases. Why not add a more generic setter:
* {{setKafkaSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
> Allow KafkaBuilder to set arbitrary subscribers
> ---
>
> Key: FLINK-27872
> URL: https://issues.apache.org/jira/browse/FLINK-27872
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Kafka
>Reporter: Salva
>Priority: Major
>
> Currently,
> [KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
> has two setters:
> *
> [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
> *
> [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
> which under the hood instantiate the corresponding (concrete) subscribers.
> This covers the most common needs, I agree, but it might fall short in some
> cases. Why not add a more generic setter:
> * {{setKafkaSubscriber (???)}}
> Otherwise, how can users read from kafka in combination with custom
> subscribing logic? Without looking much into it, it seems that they basically
> cannot, at least without having to replicate some parts of the connector,
> which seems rather inconvenient.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (FLINK-27872) Allow KafkaBuilder to set arbitrary subscribers
[
https://issues.apache.org/jira/browse/FLINK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Salva updated FLINK-27872:
--
Description:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has two setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding (concrete) subscribers. This
should cover the most common needs, I agree, but it might fall short in some
cases. Why not add a more generic setter:
* {{setKafkaSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
was:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has two setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding (concrete) subscribers. They
should cover most common needs, I agree, but they might fall short in some
cases. Why not add a more generic setter:
* {{setKafkaSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
> Allow KafkaBuilder to set arbitrary subscribers
> ---
>
> Key: FLINK-27872
> URL: https://issues.apache.org/jira/browse/FLINK-27872
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Kafka
>Reporter: Salva
>Priority: Major
>
> Currently,
> [KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
> has two setters:
> *
> [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
> *
> [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
> which under the hood instantiate the corresponding (concrete) subscribers.
> This should cover the most common needs, I agree, but it might fall short in
> some cases. Why not add a more generic setter:
> * {{setKafkaSubscriber (???)}}
> Otherwise, how can users read from kafka in combination with custom
> subscribing logic? Without looking much into it, it seems that they basically
> cannot, at least without having to replicate some parts of the connector,
> which seems rather inconvenient.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (FLINK-27872) Allow KafkaBuilder to set arbitrary subscribers
[
https://issues.apache.org/jira/browse/FLINK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Salva updated FLINK-27872:
--
Description:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has two setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding (concrete) subscribers. They
should cover most common needs, I agree, but they might fall short in some
cases. Why not adding a more generic setter:
* {{setKafkaSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
was:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has two setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding (concrete) subscribers. They
should cover the most common needs, I agree, but they might fall short in some
cases. Why not adding a more generic setter:
* {{setKafkaSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
> Allow KafkaBuilder to set arbitrary subscribers
> ---
>
> Key: FLINK-27872
> URL: https://issues.apache.org/jira/browse/FLINK-27872
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Kafka
>Reporter: Salva
>Priority: Major
>
> Currently,
> [KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
> has two setters:
> *
> [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
> *
> [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
> which under the hood instantiate the corresponding (concrete) subscribers.
> They should cover most common needs, I agree, but they might fall short in
> some cases. Why not adding a more generic setter:
> * {{setKafkaSubscriber (???)}}
> Otherwise, how can users read from kafka in combination with custom
> subscribing logic? Without looking much into it, it seems that they basically
> cannot, at least without having to replicate some parts of the connector,
> which seems rather inconvenient.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (FLINK-27872) Allow KafkaBuilder to set arbitrary subscribers
[
https://issues.apache.org/jira/browse/FLINK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Salva updated FLINK-27872:
--
Description:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has two setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding (concrete) subscribers. They
should cover most common needs, I agree, but they might fall short in some
cases. Why not add a more generic setter:
* {{setKafkaSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
was:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has two setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding (concrete) subscribers. They
should cover most common needs, I agree, but they might fall short in some
cases. Why not adding a more generic setter:
* {{setKafkaSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
> Allow KafkaBuilder to set arbitrary subscribers
> ---
>
> Key: FLINK-27872
> URL: https://issues.apache.org/jira/browse/FLINK-27872
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Kafka
>Reporter: Salva
>Priority: Major
>
> Currently,
> [KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
> has two setters:
> *
> [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
> *
> [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
> which under the hood instantiate the corresponding (concrete) subscribers.
> They should cover most common needs, I agree, but they might fall short in
> some cases. Why not add a more generic setter:
> * {{setKafkaSubscriber (???)}}
> Otherwise, how can users read from kafka in combination with custom
> subscribing logic? Without looking much into it, it seems that they basically
> cannot, at least without having to replicate some parts of the connector,
> which seems rather inconvenient.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (FLINK-27872) Allow KafkaBuilder to set arbitrary subscribers
[
https://issues.apache.org/jira/browse/FLINK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Salva updated FLINK-27872:
--
Description:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has two setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding (concrete) subscribers. They
should cover the most common needs, I agree, but they might fall short in some
cases. Why not adding a more generic setter:
* {{setKafkaSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
was:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has two setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding (concrete) subscribers.
Those two subscribers should cover most of the cases, I agree, but in some
cases they might fall short. Why not adding a more generic setter:
* {{setKafkaSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
> Allow KafkaBuilder to set arbitrary subscribers
> ---
>
> Key: FLINK-27872
> URL: https://issues.apache.org/jira/browse/FLINK-27872
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Kafka
>Reporter: Salva
>Priority: Major
>
> Currently,
> [KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
> has two setters:
> *
> [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
> *
> [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
> which under the hood instantiate the corresponding (concrete) subscribers.
> They should cover the most common needs, I agree, but they might fall short
> in some cases. Why not adding a more generic setter:
> * {{setKafkaSubscriber (???)}}
> Otherwise, how can users read from kafka in combination with custom
> subscribing logic? Without looking much into it, it seems that they basically
> cannot, at least without having to replicate some parts of the connector,
> which seems rather inconvenient.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (FLINK-27872) Allow KafkaBuilder to set arbitrary subscribers
[
https://issues.apache.org/jira/browse/FLINK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Salva updated FLINK-27872:
--
Description:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has two setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding (concrete) subscribers.
Those two subscribers should cover most of the cases, I agree, but in some
cases they might fall short. Why not adding a more generic setter:
* {{setKafkaSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
was:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has two concrete setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding subscribers. Those two
subscribers should cover most of the cases, I agree, but in some cases they
might fall short. Why not adding a more generic setter:
* {{setKafkaSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
> Allow KafkaBuilder to set arbitrary subscribers
> ---
>
> Key: FLINK-27872
> URL: https://issues.apache.org/jira/browse/FLINK-27872
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Kafka
>Reporter: Salva
>Priority: Major
>
> Currently,
> [KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
> has two setters:
> *
> [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
> *
> [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
> which under the hood instantiate the corresponding (concrete) subscribers.
> Those two subscribers should cover most of the cases, I agree, but in some
> cases they might fall short. Why not adding a more generic setter:
> * {{setKafkaSubscriber (???)}}
> Otherwise, how can users read from kafka in combination with custom
> subscribing logic? Without looking much into it, it seems that they basically
> cannot, at least without having to replicate some parts of the connector,
> which seems rather inconvenient.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (FLINK-27872) Allow KafkaBuilder to set arbitrary subscribers
[
https://issues.apache.org/jira/browse/FLINK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Salva updated FLINK-27872:
--
Description:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has two concrete setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding subscribers. Those two
subscribers should cover most of the cases, I agree, but in some cases they
might fall short. Why not adding a more generic setter:
* {{setKafkaSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
was:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has this two concrete setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding subscribers. Those two
subscribers should cover most of the cases, I agree, but in some cases they
might fall short. Why not adding a more generic setter:
* {{setKafkaSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
> Allow KafkaBuilder to set arbitrary subscribers
> ---
>
> Key: FLINK-27872
> URL: https://issues.apache.org/jira/browse/FLINK-27872
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Kafka
>Reporter: Salva
>Priority: Major
>
> Currently,
> [KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
> has two concrete setters:
> *
> [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
> *
> [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
> which under the hood instantiate the corresponding subscribers. Those two
> subscribers should cover most of the cases, I agree, but in some cases they
> might fall short. Why not adding a more generic setter:
> * {{setKafkaSubscriber (???)}}
> Otherwise, how can users read from kafka in combination with custom
> subscribing logic? Without looking much into it, it seems that they basically
> cannot, at least without having to replicate some parts of the connector,
> which seems rather inconvenient.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (FLINK-27872) Allow KafkaBuilder to set arbitrary subscribers
[
https://issues.apache.org/jira/browse/FLINK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Salva updated FLINK-27872:
--
Description:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has this two concrete setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding subscribers. Those two
subscribers should cover most of the cases, I agree, but in some cases they
might fall short. Why not adding a more generic setter:
* {{setKafkaSubscriber (???)}}
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
was:
Currently,
[KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
has this two concrete setters:
*
[setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
*
[setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
which under the hood instantiate the corresponding subscribers. Those two
subscribers should cover most of the cases, I agree, but in some cases they
might fall short. Why not adding a more generic setter:
* `setKafkaSubscriber` (???)
Otherwise, how can users read from kafka in combination with custom subscribing
logic? Without looking much into it, it seems that they basically cannot, at
least without having to replicate some parts of the connector, which seems
rather inconvenient.
> Allow KafkaBuilder to set arbitrary subscribers
> ---
>
> Key: FLINK-27872
> URL: https://issues.apache.org/jira/browse/FLINK-27872
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Kafka
>Reporter: Salva
>Priority: Major
>
> Currently,
> [KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
> has this two concrete setters:
> *
> [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
> *
> [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
> which under the hood instantiate the corresponding subscribers. Those two
> subscribers should cover most of the cases, I agree, but in some cases they
> might fall short. Why not adding a more generic setter:
> * {{setKafkaSubscriber (???)}}
> Otherwise, how can users read from kafka in combination with custom
> subscribing logic? Without looking much into it, it seems that they basically
> cannot, at least without having to replicate some parts of the connector,
> which seems rather inconvenient.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (FLINK-27872) Allow KafkaBuilder to set arbitrary subscribers
[ https://issues.apache.org/jira/browse/FLINK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Salva updated FLINK-27872: -- Description: Currently, [KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java] has this two concrete setters * [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157] * [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168] which under the hood instantiate the corresponding subscribers. Those two subscribers should cover most of the cases, I agree, but in some cases they might fall short. Why not adding a more generic setter: * `setKafkaSubscriber` Otherwise, how can users read from kafka in combination with custom subscribing logic? Without looking much into it, it seems that they basically cannot, at least without having to replicate some parts of the connector, which seems rather inconvenient. was: Currently, [KafkaBuilder|[https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]] has this two concrete setters * [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157] * [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168] which under the hood instantiate the corresponding subscribers. Those two subscribers should cover most of the cases, I agree, but in some cases they might fall short. Why not adding a more generic setter: * `setKafkaSubscriber` Otherwise, how can users read from kafka in combination with custom subscribing logic? Without looking much into it, it seems that they basically cannot, at least without having to replicate some parts of the connector, which seems rather inconvenient. > Allow KafkaBuilder to set arbitrary subscribers > --- > > Key: FLINK-27872 > URL: https://issues.apache.org/jira/browse/FLINK-27872 > Project: Flink > Issue Type: New Feature > Components: Connectors / Kafka >Reporter: Salva >Priority: Major > > Currently, > [KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java] > has this two concrete setters > * > [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157] > * > [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168] > which under the hood instantiate the corresponding subscribers. Those two > subscribers should cover most of the cases, I agree, but in some cases they > might fall short. Why not adding a more generic setter: > * `setKafkaSubscriber` > Otherwise, how can users read from kafka in combination with custom > subscribing logic? Without looking much into it, it seems that they basically > cannot, at least without having to replicate some parts of the connector, > which seems rather inconvenient. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (FLINK-27872) Allow KafkaBuilder to set arbitrary subscribers
[ https://issues.apache.org/jira/browse/FLINK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Salva updated FLINK-27872: -- Description: Currently, [KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java] has this two concrete setters: * [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157] * [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168] which under the hood instantiate the corresponding subscribers. Those two subscribers should cover most of the cases, I agree, but in some cases they might fall short. Why not adding a more generic setter: * `setKafkaSubscriber` (???) Otherwise, how can users read from kafka in combination with custom subscribing logic? Without looking much into it, it seems that they basically cannot, at least without having to replicate some parts of the connector, which seems rather inconvenient. was: Currently, [KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java] has this two concrete setters * [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157] * [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168] which under the hood instantiate the corresponding subscribers. Those two subscribers should cover most of the cases, I agree, but in some cases they might fall short. Why not adding a more generic setter: * `setKafkaSubscriber` Otherwise, how can users read from kafka in combination with custom subscribing logic? Without looking much into it, it seems that they basically cannot, at least without having to replicate some parts of the connector, which seems rather inconvenient. > Allow KafkaBuilder to set arbitrary subscribers > --- > > Key: FLINK-27872 > URL: https://issues.apache.org/jira/browse/FLINK-27872 > Project: Flink > Issue Type: New Feature > Components: Connectors / Kafka >Reporter: Salva >Priority: Major > > Currently, > [KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java] > has this two concrete setters: > * > [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157] > * > [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168] > which under the hood instantiate the corresponding subscribers. Those two > subscribers should cover most of the cases, I agree, but in some cases they > might fall short. Why not adding a more generic setter: > * `setKafkaSubscriber` (???) > Otherwise, how can users read from kafka in combination with custom > subscribing logic? Without looking much into it, it seems that they basically > cannot, at least without having to replicate some parts of the connector, > which seems rather inconvenient. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (FLINK-27872) Allow KafkaBuilder to set arbitrary subscribers
[ https://issues.apache.org/jira/browse/FLINK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Salva updated FLINK-27872: -- Description: Currently, [KafkaBuilder|[https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]] has this two concrete setters * [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157] * [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168] which under the hood instantiate the corresponding subscribers. Those two subscribers should cover most of the cases, I agree, but in some cases they might fall short. Why not adding a more generic setter: * `setKafkaSubscriber` Otherwise, how can users read from kafka in combination with custom subscribing logic? Without looking much into it, it seems that they basically cannot, at least without having to replicate some parts of the connector, which seems rather inconvenient. was: [KafkaBuilder|[https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]] has the following methods: * [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157] * [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168] which under the hood instantiate the corresponding subscribers. Those two subscribers should cover most of the cases, I agree, but in some cases they might fall short. Why not adding a more generic setter: * `setKafkaSubscriber` Otherwise, how can users read from kafka in combination with custom subscribing logic? Without looking much into it, it seems that they basically cannot, at least without having to replicate some parts of the connector, which seems rather inconvenient. > Allow KafkaBuilder to set arbitrary subscribers > --- > > Key: FLINK-27872 > URL: https://issues.apache.org/jira/browse/FLINK-27872 > Project: Flink > Issue Type: New Feature > Components: Connectors / Kafka >Reporter: Salva >Priority: Major > > Currently, > [KafkaBuilder|[https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]] > has this two concrete setters > * > [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157] > * > [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168] > which under the hood instantiate the corresponding subscribers. Those two > subscribers should cover most of the cases, I agree, but in some cases they > might fall short. Why not adding a more generic setter: > * `setKafkaSubscriber` > Otherwise, how can users read from kafka in combination with custom > subscribing logic? Without looking much into it, it seems that they basically > cannot, at least without having to replicate some parts of the connector, > which seems rather inconvenient. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (FLINK-27872) Allow KafkaBuilder to set arbitrary subscribers
[ https://issues.apache.org/jira/browse/FLINK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Salva updated FLINK-27872: -- Description: [KafkaBuilder|[https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]] has the following methods: * [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157] * [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168] which under the hood instantiate the corresponding subscribers. Those two subscribers should cover most of the cases, I agree, but in some cases they might fall short. Why not adding a more generic setter: * `setKafkaSubscriber` Otherwise, how can users read from kafka in combination with custom subscribing logic? Without looking much into it, it seems that they basically cannot, at least without having to replicate some parts of the connector, which seems rather inconvenient. was: The [KafkaBuilder|[https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]) has the following methods: * [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157] * [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168] which under the hood instantiate the corresponding subscribers. Those two subscribers should cover most of the cases, I agree, but in some cases they might fall short. Why not adding a more generic setter: * `setKafkaSubscriber` Otherwise, how can users read from kafka in combination with custom subscribing logic? Without looking much into it, it seems that they basically cannot, at least without having to replicate some parts of the connector, which seems rather inconvenient. > Allow KafkaBuilder to set arbitrary subscribers > --- > > Key: FLINK-27872 > URL: https://issues.apache.org/jira/browse/FLINK-27872 > Project: Flink > Issue Type: New Feature > Components: Connectors / Kafka >Reporter: Salva >Priority: Major > > [KafkaBuilder|[https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]] > has the following methods: > * > [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157] > * > [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168] > which under the hood instantiate the corresponding subscribers. Those two > subscribers should cover most of the cases, I agree, but in some cases they > might fall short. Why not adding a more generic setter: > * `setKafkaSubscriber` > Otherwise, how can users read from kafka in combination with custom > subscribing logic? Without looking much into it, it seems that they basically > cannot, at least without having to replicate some parts of the connector, > which seems rather inconvenient. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (FLINK-27872) Allow KafkaBuilder to set arbitrary subscribers
[ https://issues.apache.org/jira/browse/FLINK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Salva updated FLINK-27872: -- Description: The [KafkaBuilder|[https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]) has the following methods: * [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157] * [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168] which under the hood instantiate the corresponding subscribers. Those two subscribers should cover most of the cases, I agree, but in some cases they might fall short. Why not adding a more generic setter: * `setKafkaSubscriber` Otherwise, how can users read from kafka in combination with custom subscribing logic? Without looking much into it, it seems that they basically cannot, at least without having to replicate some parts of the connector, which seems rather inconvenient. was: The [KafkaBuilder](https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java) has the following methods: * [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157] * [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168] which under the hood instantiate the corresponding subscribers. Those two subscribers should cover most of the cases, I agree, but in some cases they might fall short. Why not adding a more generic setter: * `setKafkaSubscriber` Otherwise, how can users read from kafka in combination with custom subscribing logic? Without looking much into it, it seems that they basically cannot, at least without having to replicate some parts of the connector, which seems rather inconvenient. > Allow KafkaBuilder to set arbitrary subscribers > --- > > Key: FLINK-27872 > URL: https://issues.apache.org/jira/browse/FLINK-27872 > Project: Flink > Issue Type: New Feature > Components: Connectors / Kafka >Reporter: Salva >Priority: Major > > The > [KafkaBuilder|[https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]) > has the following methods: > * > [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157] > * > [setTopicsPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168] > which under the hood instantiate the corresponding subscribers. Those two > subscribers should cover most of the cases, I agree, but in some cases they > might fall short. Why not adding a more generic setter: > * `setKafkaSubscriber` > Otherwise, how can users read from kafka in combination with custom > subscribing logic? Without looking much into it, it seems that they basically > cannot, at least without having to replicate some parts of the connector, > which seems rather inconvenient. -- This message was sent by Atlassian Jira (v8.20.7#820007)
