[jira] [Commented] (FLINK-34193) Remove usage of Flink-Shaded Jackson and Snakeyaml in flink-connector-kafka
[
https://issues.apache.org/jira/browse/FLINK-34193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17811307#comment-17811307
]
Jinsui Chen commented on FLINK-34193:
-
Hi, [~martijnvisser]. Thank you for you response. I have one more question.
JSONKeyValueDeserializationSchema which depends on a specific version of
Jackson is annotated by '@PublicEvolving'. Is it required to remove this class
for this refactoring?
> Remove usage of Flink-Shaded Jackson and Snakeyaml in flink-connector-kafka
> ---
>
> Key: FLINK-34193
> URL: https://issues.apache.org/jira/browse/FLINK-34193
> Project: Flink
> Issue Type: Technical Debt
> Components: Connectors / Kafka
>Reporter: Martijn Visser
>Priority: Blocker
>
> The Flink Kafka connector doesn't have a direct dependency in the POM on
> flink-shaded, but it still uses the shaded versions of Jackson and SnakeYAML
> in {{YamlFileMetaDataService.java}} and
> {{KafkaRecordDeserializationSchemaTest}}
> Those cause problems when trying to compile the Flink Kafka connector for
> Flink 1.19, since these dependencies have been updated in there. Since
> connectors shouldn't rely on Flink-Shaded, we should refactor these
> implementations
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (FLINK-34193) Remove usage of Flink-Shaded Jackson and Snakeyaml in flink-connector-kafka
[
https://issues.apache.org/jira/browse/FLINK-34193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17810385#comment-17810385
]
Jinsui Chen edited comment on FLINK-34193 at 1/24/24 3:00 PM:
--
I think it's useful to specify Jackson and SnakeYAML versions directly in the
pom.xml file to address conflicts. Specifically, look for and replace
references to org.apache.flink.shaded.* with standard Jackson and SnakeYAML
libraries. If you think this solution suitable, feel free to assign the issue
to me.
was (Author: JIRAUSER303504):
I think it's useful to specify Jackson and SnakeYAML versions directly in the
pom.xml file to address conflicts. Specifically, look for and replace
references to org.apache.flink.shaded.* with standard Jackson and SnakeYAML
libraries. If you think this solution suitable, feel free to assign the issue
to me.
> Remove usage of Flink-Shaded Jackson and Snakeyaml in flink-connector-kafka
> ---
>
> Key: FLINK-34193
> URL: https://issues.apache.org/jira/browse/FLINK-34193
> Project: Flink
> Issue Type: Technical Debt
> Components: Connectors / Kafka
>Reporter: Martijn Visser
>Priority: Critical
>
> The Flink Kafka connector doesn't have a direct dependency in the POM on
> flink-shaded, but it still uses the shaded versions of Jackson and SnakeYAML
> in {{YamlFileMetaDataService.java}} and
> {{KafkaRecordDeserializationSchemaTest}}
> Those cause problems when trying to compile the Flink Kafka connector for
> Flink 1.19, since these dependencies have been updated in there. Since
> connectors shouldn't rely on Flink-Shaded, we should refactor these
> implementations
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-34193) Remove usage of Flink-Shaded Jackson and Snakeyaml in flink-connector-kafka
[
https://issues.apache.org/jira/browse/FLINK-34193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17810385#comment-17810385
]
Jinsui Chen commented on FLINK-34193:
-
I think it's useful to specify Jackson and SnakeYAML versions directly in the
pom.xml file to address conflicts. Specifically, look for and replace
references to org.apache.flink.shaded.* with standard Jackson and SnakeYAML
libraries. If you think this solution suitable, feel free to assign the issue
to me.
> Remove usage of Flink-Shaded Jackson and Snakeyaml in flink-connector-kafka
> ---
>
> Key: FLINK-34193
> URL: https://issues.apache.org/jira/browse/FLINK-34193
> Project: Flink
> Issue Type: Technical Debt
> Components: Connectors / Kafka
>Reporter: Martijn Visser
>Priority: Critical
>
> The Flink Kafka connector doesn't have a direct dependency in the POM on
> flink-shaded, but it still uses the shaded versions of Jackson and SnakeYAML
> in {{YamlFileMetaDataService.java}} and
> {{KafkaRecordDeserializationSchemaTest}}
> Those cause problems when trying to compile the Flink Kafka connector for
> Flink 1.19, since these dependencies have been updated in there. Since
> connectors shouldn't rely on Flink-Shaded, we should refactor these
> implementations
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-33873) Create a Redis HyperLogLog Connector for Flink
[ https://issues.apache.org/jira/browse/FLINK-33873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798204#comment-17798204 ] Jinsui Chen commented on FLINK-33873: - [~martijnvisser] I'm willing to prepare for it. But I don't have permission to create new FLIP pages. According to the documentation, an email has been sent to the dev mailing list for help. > Create a Redis HyperLogLog Connector for Flink > -- > > Key: FLINK-33873 > URL: https://issues.apache.org/jira/browse/FLINK-33873 > Project: Flink > Issue Type: New Feature > Components: Connectors / Redis Streams >Reporter: Jinsui Chen >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > Redis HyperLogLog is a probabilistic data structure used for estimating the > cardinality of a dataset, which is the number of unique elements in a set. I > think it is possible to create a sink connector for HyperLogLog. > FLINK-15571 is about Redis stream connector. > Since there is no component for the Redis connector as a whole, the issue is > created under this component. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-33873) Create a Redis HyperLogLog Connector for Flink
[ https://issues.apache.org/jira/browse/FLINK-33873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798153#comment-17798153 ] Jinsui Chen commented on FLINK-33873: - hi, [~MartijnVisser] . This is the issue about Redis HyperLogLog Connector. Would you assign it to me? > Create a Redis HyperLogLog Connector for Flink > -- > > Key: FLINK-33873 > URL: https://issues.apache.org/jira/browse/FLINK-33873 > Project: Flink > Issue Type: New Feature > Components: Connectors / Redis Streams >Reporter: Jinsui Chen >Priority: Minor > Labels: features > Original Estimate: 168h > Remaining Estimate: 168h > > Redis HyperLogLog is a probabilistic data structure used for estimating the > cardinality of a dataset, which is the number of unique elements in a set. I > think it is possible to create a sink connector for HyperLogLog. > FLINK-15571 is about Redis stream connector. > Since there is no component for the Redis connector as a whole, the issue is > created under this component. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33873) Create a Redis HyperLogLog Connector for Flink
Jinsui Chen created FLINK-33873: --- Summary: Create a Redis HyperLogLog Connector for Flink Key: FLINK-33873 URL: https://issues.apache.org/jira/browse/FLINK-33873 Project: Flink Issue Type: New Feature Components: Connectors / Redis Streams Reporter: Jinsui Chen Redis HyperLogLog is a probabilistic data structure used for estimating the cardinality of a dataset, which is the number of unique elements in a set. I think it is possible to create a sink connector for HyperLogLog. FLINK-15571 is about Redis stream connector. Since there is no component for the Redis connector as a whole, the issue is created under this component. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-15571) Create a Redis Streams Connector for Flink
[ https://issues.apache.org/jira/browse/FLINK-15571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797097#comment-17797097 ] Jinsui Chen commented on FLINK-15571: - In addition to Redis stream, is it possible for other Redis data structures to work with Flink, such as Hyperloglog? > Create a Redis Streams Connector for Flink > -- > > Key: FLINK-15571 > URL: https://issues.apache.org/jira/browse/FLINK-15571 > Project: Flink > Issue Type: New Feature > Components: Connectors / Redis Streams >Reporter: Tugdual Grall >Assignee: ZhuoYu Chen >Priority: Minor > Labels: pull-request-available, stale-assigned > > Redis has a "log data structure" called Redis Streams, it would be nice to > integrate Redis Streams and Apache Flink as: > * Source > * Sink > See Redis Streams introduction: [https://redis.io/topics/streams-intro] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-32573) Translate "Custom Serialization for Managed State" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-32573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796980#comment-17796980 ] Jinsui Chen commented on FLINK-32573: - hi, [~Wencong Liu] . Would you please assign this to me？ > Translate "Custom Serialization for Managed State" page into Chinese > > > Key: FLINK-32573 > URL: https://issues.apache.org/jira/browse/FLINK-32573 > Project: Flink > Issue Type: New Feature > Components: chinese-translation, Documentation >Affects Versions: 1.18.0 >Reporter: Yongping Li >Priority: Minor > Labels: pull-request-available > Fix For: 1.19.0 > > > The page url is > https://nightlies.apache.org/flink/flink-docs-master/zh/docs/dev/datastream/fault-tolerance/serialization/custom_serialization/ > The markdown file is located in > docs/content.zh/docs/dev/datastream/fault-tolerance/serialization/custom_serialization.md -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-32573) Translate "Custom Serialization for Managed State" page into Chinese
[ https://issues.apache.org/jira/browse/FLINK-32573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796700#comment-17796700 ] Jinsui Chen commented on FLINK-32573: - hi, [~jark] . Would you please assign this to me？ > Translate "Custom Serialization for Managed State" page into Chinese > > > Key: FLINK-32573 > URL: https://issues.apache.org/jira/browse/FLINK-32573 > Project: Flink > Issue Type: New Feature > Components: chinese-translation, Documentation >Affects Versions: 1.18.0 >Reporter: Yongping Li >Priority: Minor > Labels: pull-request-available > Fix For: 1.19.0 > > > The page url is > https://nightlies.apache.org/flink/flink-docs-master/zh/docs/dev/datastream/fault-tolerance/serialization/custom_serialization/ > The markdown file is located in > docs/content.zh/docs/dev/datastream/fault-tolerance/serialization/custom_serialization.md -- This message was sent by Atlassian Jira (v8.20.10#820010)
