[DISCUSS] Connector releases for Flink 1.18
Hey all. Now Flink 1.18 is released we need to do some connector releases for integration parity. We can use this thread to start the discussions for each connector release and spawn separate vote threads. Kafka is done [1] (thanks Gordon) and AWS connectors are in process [2], I appreciate help with votes on that one. Opensearch: Flink 1.18 nightly build passing [3]. I volunteer to be release manager for this one. I will consolidate 1.0.2 [4] and 1.1.0 [5] into a single release as 1.1.0. MongoDB: Flink 1.18 nightly build passing [6]. I volunteer to be release manager for this one. I will work with Jiabao to get FLINK-33257 merged into 1.1.0 and release that [7]. GCP Pub Sub: Flink 1.18 nightly build passing [8]. I volunteer to be release manager for this one. Looks like 3.0.2 is ready to go [9], we should proceed with this. ElasticSearch: Flink 1.18 nightly build passing [10]. There are a good stack of changes ready for 3.1.0 [11], suggest we release that. JDBC: Flink 1.18 nightly build passing [12]. There are a good stack of changes ready for 3.2.0 [13], suggest we release that. RabbitMQ: Flink 1.18 nightly build passing [14]. There are no changes ready for 3.0.2 [15], recommend we do a minimal 3.0.1-1.18. Pulsar: The nightly CI is failing [16], needs a deeper look Cassandra: The nightly CI is failing [17], needs a deeper look Once I have completed Opensearch/MongoDB/GCP I will pick up others, but hope others can help out. Thanks, Danny [1] https://lists.apache.org/thread/0lvrm9hl3hnn1fpr74k68lsm22my8xh7 [2] https://lists.apache.org/thread/ko6nrtfsykkz9c9k9392jfj4l9f7qg11 [3] https://github.com/apache/flink-connector-opensearch/actions [4] https://issues.apache.org/jira/projects/FLINK/versions/12353142 [5] https://issues.apache.org/jira/projects/FLINK/versions/12353141 [6] https://github.com/apache/flink-connector-mongodb/actions [7] https://issues.apache.org/jira/projects/FLINK/versions/12353483 [8] https://github.com/apache/flink-connector-gcp-pubsub/actions [9] https://issues.apache.org/jira/projects/FLINK/versions/12353144 [10] https://github.com/apache/flink-connector-elasticsearch/actions [11] https://issues.apache.org/jira/projects/FLINK/versions/12352520 [12] https://github.com/apache/flink-connector-jdbc/actions [13] https://issues.apache.org/jira/projects/FLINK/versions/12353143 [14] https://github.com/apache/flink-connector-rabbitmq/actions [15] https://issues.apache.org/jira/projects/FLINK/versions/12353145 [16] https://github.com/apache/flink-connector-pulsar/actions [17] https://github.com/apache/flink-connector-cassandra/actions
Re: [DISCUSS] Connector releases for Flink 1.18
Hi Danny, Thanks a lot for starting the discussion on this topic! I know that Pulsar is failing because of Archunit, which I expect the same issue to be for Cassandra (I know that Etienne was working on this). Happy to help. Best regards, Martijn On Thu, Nov 2, 2023 at 9:08 PM Danny Cranmer wrote: > > Hey all. > > Now Flink 1.18 is released we need to do some connector releases for > integration parity. We can use this thread to start the discussions for > each connector release and spawn separate vote threads. Kafka is done [1] > (thanks Gordon) and AWS connectors are in process [2], I appreciate help > with votes on that one. > > Opensearch: Flink 1.18 nightly build passing [3]. I volunteer to be release > manager for this one. I will consolidate 1.0.2 [4] and 1.1.0 [5] into a > single release as 1.1.0. > MongoDB: Flink 1.18 nightly build passing [6]. I volunteer to be release > manager for this one. I will work with Jiabao to get FLINK-33257 merged > into 1.1.0 and release that [7]. > GCP Pub Sub: Flink 1.18 nightly build passing [8]. I volunteer to be > release manager for this one. Looks like 3.0.2 is ready to go [9], we > should proceed with this. > > ElasticSearch: Flink 1.18 nightly build passing [10]. There are a good > stack of changes ready for 3.1.0 [11], suggest we release that. > JDBC: Flink 1.18 nightly build passing [12]. There are a good stack of > changes ready for 3.2.0 [13], suggest we release that. > RabbitMQ: Flink 1.18 nightly build passing [14]. There are no changes ready > for 3.0.2 [15], recommend we do a minimal 3.0.1-1.18. > > Pulsar: The nightly CI is failing [16], needs a deeper look > Cassandra: The nightly CI is failing [17], needs a deeper look > > Once I have completed Opensearch/MongoDB/GCP I will pick up others, but > hope others can help out. > > Thanks, > Danny > > [1] https://lists.apache.org/thread/0lvrm9hl3hnn1fpr74k68lsm22my8xh7 > [2] https://lists.apache.org/thread/ko6nrtfsykkz9c9k9392jfj4l9f7qg11 > [3] https://github.com/apache/flink-connector-opensearch/actions > [4] https://issues.apache.org/jira/projects/FLINK/versions/12353142 > [5] https://issues.apache.org/jira/projects/FLINK/versions/12353141 > [6] https://github.com/apache/flink-connector-mongodb/actions > [7] https://issues.apache.org/jira/projects/FLINK/versions/12353483 > [8] https://github.com/apache/flink-connector-gcp-pubsub/actions > [9] https://issues.apache.org/jira/projects/FLINK/versions/12353144 > [10] https://github.com/apache/flink-connector-elasticsearch/actions > [11] https://issues.apache.org/jira/projects/FLINK/versions/12352520 > [12] https://github.com/apache/flink-connector-jdbc/actions > [13] https://issues.apache.org/jira/projects/FLINK/versions/12353143 > [14] https://github.com/apache/flink-connector-rabbitmq/actions > [15] https://issues.apache.org/jira/projects/FLINK/versions/12353145 > [16] https://github.com/apache/flink-connector-pulsar/actions > [17] https://github.com/apache/flink-connector-cassandra/actions
Re: [DISCUSS] Connector releases for Flink 1.18
Hi, I can look into pulsar and Cassandra fixes. I agree it's archunit issue based on the build logs. I don't have much experience with it, but I was able to reproduce it locally. I can send out PR's over the weekend. Do we have JIRA's for this? If not I can create to track. Thanks @sh On Fri, 3 Nov 2023 at 02:20, Martijn Visser wrote: > Hi Danny, > > Thanks a lot for starting the discussion on this topic! I know that > Pulsar is failing because of Archunit, which I expect the same issue > to be for Cassandra (I know that Etienne was working on this). Happy > to help. > > Best regards, > > Martijn > > On Thu, Nov 2, 2023 at 9:08 PM Danny Cranmer > wrote: > > > > Hey all. > > > > Now Flink 1.18 is released we need to do some connector releases for > > integration parity. We can use this thread to start the discussions for > > each connector release and spawn separate vote threads. Kafka is done [1] > > (thanks Gordon) and AWS connectors are in process [2], I appreciate help > > with votes on that one. > > > > Opensearch: Flink 1.18 nightly build passing [3]. I volunteer to be > release > > manager for this one. I will consolidate 1.0.2 [4] and 1.1.0 [5] into a > > single release as 1.1.0. > > MongoDB: Flink 1.18 nightly build passing [6]. I volunteer to be release > > manager for this one. I will work with Jiabao to get FLINK-33257 merged > > into 1.1.0 and release that [7]. > > GCP Pub Sub: Flink 1.18 nightly build passing [8]. I volunteer to be > > release manager for this one. Looks like 3.0.2 is ready to go [9], we > > should proceed with this. > > > > ElasticSearch: Flink 1.18 nightly build passing [10]. There are a good > > stack of changes ready for 3.1.0 [11], suggest we release that. > > JDBC: Flink 1.18 nightly build passing [12]. There are a good stack of > > changes ready for 3.2.0 [13], suggest we release that. > > RabbitMQ: Flink 1.18 nightly build passing [14]. There are no changes > ready > > for 3.0.2 [15], recommend we do a minimal 3.0.1-1.18. > > > > Pulsar: The nightly CI is failing [16], needs a deeper look > > Cassandra: The nightly CI is failing [17], needs a deeper look > > > > Once I have completed Opensearch/MongoDB/GCP I will pick up others, but > > hope others can help out. > > > > Thanks, > > Danny > > > > [1] https://lists.apache.org/thread/0lvrm9hl3hnn1fpr74k68lsm22my8xh7 > > [2] https://lists.apache.org/thread/ko6nrtfsykkz9c9k9392jfj4l9f7qg11 > > [3] https://github.com/apache/flink-connector-opensearch/actions > > [4] https://issues.apache.org/jira/projects/FLINK/versions/12353142 > > [5] https://issues.apache.org/jira/projects/FLINK/versions/12353141 > > [6] https://github.com/apache/flink-connector-mongodb/actions > > [7] https://issues.apache.org/jira/projects/FLINK/versions/12353483 > > [8] https://github.com/apache/flink-connector-gcp-pubsub/actions > > [9] https://issues.apache.org/jira/projects/FLINK/versions/12353144 > > [10] https://github.com/apache/flink-connector-elasticsearch/actions > > [11] https://issues.apache.org/jira/projects/FLINK/versions/12352520 > > [12] https://github.com/apache/flink-connector-jdbc/actions > > [13] https://issues.apache.org/jira/projects/FLINK/versions/12353143 > > [14] https://github.com/apache/flink-connector-rabbitmq/actions > > [15] https://issues.apache.org/jira/projects/FLINK/versions/12353145 > > [16] https://github.com/apache/flink-connector-pulsar/actions > > [17] https://github.com/apache/flink-connector-cassandra/actions >
Re: [DISCUSS] Connector releases for Flink 1.18
Hi, I looked into both connectors, they both are failing at archunit. I tried a few things but with my limited experience couldn't make much progress. Don't want to block the release. If anyone knows what needs to be done, please move forward with the fixes. Hopefully, shall be able to help more in the future. Thanks Ash On Fri, 3 Nov 2023 at 08:35, mystic lama wrote: > Hi, > > I can look into pulsar and Cassandra fixes. I agree it's archunit issue > based on the build logs. > I don't have much experience with it, but I was able to reproduce it > locally. > > I can send out PR's over the weekend. Do we have JIRA's for this? If not I > can create to track. > > Thanks > @sh > > On Fri, 3 Nov 2023 at 02:20, Martijn Visser > wrote: > >> Hi Danny, >> >> Thanks a lot for starting the discussion on this topic! I know that >> Pulsar is failing because of Archunit, which I expect the same issue >> to be for Cassandra (I know that Etienne was working on this). Happy >> to help. >> >> Best regards, >> >> Martijn >> >> On Thu, Nov 2, 2023 at 9:08 PM Danny Cranmer >> wrote: >> > >> > Hey all. >> > >> > Now Flink 1.18 is released we need to do some connector releases for >> > integration parity. We can use this thread to start the discussions for >> > each connector release and spawn separate vote threads. Kafka is done >> [1] >> > (thanks Gordon) and AWS connectors are in process [2], I appreciate help >> > with votes on that one. >> > >> > Opensearch: Flink 1.18 nightly build passing [3]. I volunteer to be >> release >> > manager for this one. I will consolidate 1.0.2 [4] and 1.1.0 [5] into a >> > single release as 1.1.0. >> > MongoDB: Flink 1.18 nightly build passing [6]. I volunteer to be release >> > manager for this one. I will work with Jiabao to get FLINK-33257 merged >> > into 1.1.0 and release that [7]. >> > GCP Pub Sub: Flink 1.18 nightly build passing [8]. I volunteer to be >> > release manager for this one. Looks like 3.0.2 is ready to go [9], we >> > should proceed with this. >> > >> > ElasticSearch: Flink 1.18 nightly build passing [10]. There are a good >> > stack of changes ready for 3.1.0 [11], suggest we release that. >> > JDBC: Flink 1.18 nightly build passing [12]. There are a good stack of >> > changes ready for 3.2.0 [13], suggest we release that. >> > RabbitMQ: Flink 1.18 nightly build passing [14]. There are no changes >> ready >> > for 3.0.2 [15], recommend we do a minimal 3.0.1-1.18. >> > >> > Pulsar: The nightly CI is failing [16], needs a deeper look >> > Cassandra: The nightly CI is failing [17], needs a deeper look >> > >> > Once I have completed Opensearch/MongoDB/GCP I will pick up others, but >> > hope others can help out. >> > >> > Thanks, >> > Danny >> > >> > [1] https://lists.apache.org/thread/0lvrm9hl3hnn1fpr74k68lsm22my8xh7 >> > [2] https://lists.apache.org/thread/ko6nrtfsykkz9c9k9392jfj4l9f7qg11 >> > [3] https://github.com/apache/flink-connector-opensearch/actions >> > [4] https://issues.apache.org/jira/projects/FLINK/versions/12353142 >> > [5] https://issues.apache.org/jira/projects/FLINK/versions/12353141 >> > [6] https://github.com/apache/flink-connector-mongodb/actions >> > [7] https://issues.apache.org/jira/projects/FLINK/versions/12353483 >> > [8] https://github.com/apache/flink-connector-gcp-pubsub/actions >> > [9] https://issues.apache.org/jira/projects/FLINK/versions/12353144 >> > [10] https://github.com/apache/flink-connector-elasticsearch/actions >> > [11] https://issues.apache.org/jira/projects/FLINK/versions/12352520 >> > [12] https://github.com/apache/flink-connector-jdbc/actions >> > [13] https://issues.apache.org/jira/projects/FLINK/versions/12353143 >> > [14] https://github.com/apache/flink-connector-rabbitmq/actions >> > [15] https://issues.apache.org/jira/projects/FLINK/versions/12353145 >> > [16] https://github.com/apache/flink-connector-pulsar/actions >> > [17] https://github.com/apache/flink-connector-cassandra/actions >> >
Re: [DISCUSS] Connector releases for Flink 1.18
Hi, Thanks for starting the discussion, I already started testing Cassandra connector on Flink 1.18. The error in the nightly build linked here is a known issue tracked by FLINK-32353. To fix it, it needs a change in connector-parent and in the ci utils that are tracked by FLINK-32563. Best Etienne Le 07/11/2023 à 02:20, mystic lama a écrit : Hi, I looked into both connectors, they both are failing at archunit. I tried a few things but with my limited experience couldn't make much progress. Don't want to block the release. If anyone knows what needs to be done, please move forward with the fixes. Hopefully, shall be able to help more in the future. Thanks Ash On Fri, 3 Nov 2023 at 08:35, mystic lama wrote: Hi, I can look into pulsar and Cassandra fixes. I agree it's archunit issue based on the build logs. I don't have much experience with it, but I was able to reproduce it locally. I can send out PR's over the weekend. Do we have JIRA's for this? If not I can create to track. Thanks @sh On Fri, 3 Nov 2023 at 02:20, Martijn Visser wrote: Hi Danny, Thanks a lot for starting the discussion on this topic! I know that Pulsar is failing because of Archunit, which I expect the same issue to be for Cassandra (I know that Etienne was working on this). Happy to help. Best regards, Martijn On Thu, Nov 2, 2023 at 9:08 PM Danny Cranmer wrote: Hey all. Now Flink 1.18 is released we need to do some connector releases for integration parity. We can use this thread to start the discussions for each connector release and spawn separate vote threads. Kafka is done [1] (thanks Gordon) and AWS connectors are in process [2], I appreciate help with votes on that one. Opensearch: Flink 1.18 nightly build passing [3]. I volunteer to be release manager for this one. I will consolidate 1.0.2 [4] and 1.1.0 [5] into a single release as 1.1.0. MongoDB: Flink 1.18 nightly build passing [6]. I volunteer to be release manager for this one. I will work with Jiabao to get FLINK-33257 merged into 1.1.0 and release that [7]. GCP Pub Sub: Flink 1.18 nightly build passing [8]. I volunteer to be release manager for this one. Looks like 3.0.2 is ready to go [9], we should proceed with this. ElasticSearch: Flink 1.18 nightly build passing [10]. There are a good stack of changes ready for 3.1.0 [11], suggest we release that. JDBC: Flink 1.18 nightly build passing [12]. There are a good stack of changes ready for 3.2.0 [13], suggest we release that. RabbitMQ: Flink 1.18 nightly build passing [14]. There are no changes ready for 3.0.2 [15], recommend we do a minimal 3.0.1-1.18. Pulsar: The nightly CI is failing [16], needs a deeper look Cassandra: The nightly CI is failing [17], needs a deeper look Once I have completed Opensearch/MongoDB/GCP I will pick up others, but hope others can help out. Thanks, Danny [1]https://lists.apache.org/thread/0lvrm9hl3hnn1fpr74k68lsm22my8xh7 [2]https://lists.apache.org/thread/ko6nrtfsykkz9c9k9392jfj4l9f7qg11 [3]https://github.com/apache/flink-connector-opensearch/actions [4]https://issues.apache.org/jira/projects/FLINK/versions/12353142 [5]https://issues.apache.org/jira/projects/FLINK/versions/12353141 [6]https://github.com/apache/flink-connector-mongodb/actions [7]https://issues.apache.org/jira/projects/FLINK/versions/12353483 [8]https://github.com/apache/flink-connector-gcp-pubsub/actions [9]https://issues.apache.org/jira/projects/FLINK/versions/12353144 [10]https://github.com/apache/flink-connector-elasticsearch/actions [11]https://issues.apache.org/jira/projects/FLINK/versions/12352520 [12]https://github.com/apache/flink-connector-jdbc/actions [13]https://issues.apache.org/jira/projects/FLINK/versions/12353143 [14]https://github.com/apache/flink-connector-rabbitmq/actions [15]https://issues.apache.org/jira/projects/FLINK/versions/12353145 [16]https://github.com/apache/flink-connector-pulsar/actions [17]https://github.com/apache/flink-connector-cassandra/actions
