Re: [DISCUSSION] flink-connector-shared-utils release process
Thanks Leonard for the review. Best, Etienne Le 29/11/2023 à 08:25, Leonard Xu a écrit : Thanks Etienne for update the wiki, I just checked the change by comparing with history version, LGTM Best, Leonard 2023年11月28日 下午10:43，Etienne Chauchot 写道： Hi all, I just updated the doc:https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development Best Etienne Le 27/11/2023 à 12:43, Etienne Chauchot a écrit : Sure! Le 23/11/2023 à 02:57, Leonard Xu a écrit : Thanks Etienne for driving this. - a flink-connector-shared-utils-*test* clone repo and a *io.github.user.flink*:flink-connector-parent custom artifact to be able to directly commit and install the artifact in the CI - a custom ci script that does the cloning and mvn install in the ci.yml github action script for testing with the new flink-connector-parent artifact If people agree on the process and location +1 for the process and location, could we also add a short paragraph and the location link as well in [1] to remind connector developers? Best, Leonard [1]https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development
Re: [DISCUSSION] flink-connector-shared-utils release process
Thanks Etienne for update the wiki, I just checked the change by comparing with history version, LGTM Best, Leonard > 2023年11月28日 下午10:43，Etienne Chauchot 写道： > > Hi all, > > I just updated the doc: > https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development > > Best > > Etienne > > Le 27/11/2023 à 12:43, Etienne Chauchot a écrit : >> >> Sure! >> >> Le 23/11/2023 à 02:57, Leonard Xu a écrit : >>> Thanks Etienne for driving this. >>> - a flink-connector-shared-utils-*test* clone repo and a *io.github.user.flink*:flink-connector-parent custom artifact to be able to directly commit and install the artifact in the CI - a custom ci script that does the cloning and mvn install in the ci.yml github action script for testing with the new flink-connector-parent artifact If people agree on the process and location >>> +1 for the process and location, could we also add a short paragraph and >>> the location link as well in [1] to remind connector developers? >>> >>> Best, >>> Leonard >>> >>> [1]https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development >>> >>>
Re: [DISCUSSION] flink-connector-shared-utils release process
Hi all, I just updated the doc: https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development Best Etienne Le 27/11/2023 à 12:43, Etienne Chauchot a écrit : Sure! Le 23/11/2023 à 02:57, Leonard Xu a écrit : Thanks Etienne for driving this. - a flink-connector-shared-utils-*test* clone repo and a *io.github.user.flink*:flink-connector-parent custom artifact to be able to directly commit and install the artifact in the CI - a custom ci script that does the cloning and mvn install in the ci.yml github action script for testing with the new flink-connector-parent artifact If people agree on the process and location +1 for the process and location, could we also add a short paragraph and the location link as well in [1] to remind connector developers? Best, Leonard [1]https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development
Re: [DISCUSSION] flink-connector-shared-utils release process
Sure! Le 23/11/2023 à 02:57, Leonard Xu a écrit : Thanks Etienne for driving this. - a flink-connector-shared-utils-*test* clone repo and a *io.github.user.flink*:flink-connector-parent custom artifact to be able to directly commit and install the artifact in the CI - a custom ci script that does the cloning and mvn install in the ci.yml github action script for testing with the new flink-connector-parent artifact If people agree on the process and location +1 for the process and location, could we also add a short paragraph and the location link as well in [1] to remind connector developers? Best, Leonard [1]https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development
Re: [DISCUSSION] flink-connector-shared-utils release process
Thanks Etienne for driving this. > - a flink-connector-shared-utils-*test* clone repo and a > *io.github.user.flink*:flink-connector-parent custom artifact to be able to > directly commit and install the artifact in the CI > - a custom ci script that does the cloning and mvn install in the ci.yml > github action script for testing with the new flink-connector-parent artifact > If people agree on the process and location +1 for the process and location, could we also add a short paragraph and the location link as well in [1] to remind connector developers? Best, Leonard [1]https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development
Re: [DISCUSSION] flink-connector-shared-utils release process
I agree, it would make sense to add this process to https://cwiki.apache.org/confluence/display/FLINK/Continuous+Integration If people agree on the process and location, I can do the addition to the doc. Best Etienne Le 22/11/2023 à 14:29, Sergey Nuyanzin a écrit : yes, that's the way how I tested my changes thanks for confirming that it's ok I wonder whether it should be documented somewhere, since it is not very obvious? On Wed, Nov 22, 2023 at 10:35 AM Etienne Chauchot wrote: Hi Sergey, The other alternative that allows for autonomous testing of ci changes (thanks @Chesnay for the suggestion) is to create you own repo: - a flink-connector-shared-utils-*test* clone repo and a *io.github.user.flink*:flink-connector-parent custom artifact to be able to directly commit and install the artifact in the CI - a custom ci script that does the cloning and mvn install in the ci.yml github action script for testing with the new flink-connector-parent artifact seehttps://github.com/echauchot/flink-connector-shared-utils-test FLINK-* branches for details. I hope it helps, Best Etienne Le 22/11/2023 à 00:52, Sergey Nuyanzin a écrit : Hi Etienne thanks for starting this discussion +1 from my side for snapshots since I also faced same issue On Thu, Nov 9, 2023 at 11:21 AM Etienne Chauchot wrote: Hi all, flink-connector-shared-utils contains utilities for connectors (parent pom, ci scripts, template test connector project etc...). It is divided into 2 main branches: - parent_pom (1) containing just a pom.xml - ci_utils (2) containing the test project using the parent pom and the ci scripts. The problem is when we want to test changes to the parent pom in the test project, we need to release org.apache.flink:flink-connector-parent so that the test project in the other branch can use this updated parent. This seems bad to trigger a release for testing. So I would like to propose setting up a snapshot for org.apache.flink:flink-connector-parent with regular deployment to https://repository.apache.org/content/repositories/snapshots like we do for flink. An alternative could be to keep using io.github.zentol.flink:flink-connector-parent for testing and release this special artifact when needed by ci_utils test project. WDYT ? [1] https://github.com/apache/flink-connector-shared-utils/tree/parent_pom [2]https://github.com/apache/flink-connector-shared-utils/tree/ci_utils Best Etienne
Re: [DISCUSSION] flink-connector-shared-utils release process
yes, that's the way how I tested my changes thanks for confirming that it's ok I wonder whether it should be documented somewhere, since it is not very obvious? On Wed, Nov 22, 2023 at 10:35 AM Etienne Chauchot wrote: > Hi Sergey, > > The other alternative that allows for autonomous testing of ci changes > (thanks @Chesnay for the suggestion) is to create you own repo: > > - a flink-connector-shared-utils-*test* clone repo and a > *io.github.user.flink*:flink-connector-parent custom artifact to be able > to directly commit and install the artifact in the CI > - a custom ci script that does the cloning and mvn install in the ci.yml > github action script for testing with the new flink-connector-parent > artifact > > see https://github.com/echauchot/flink-connector-shared-utils-test > FLINK-* branches for details. > > I hope it helps, > > Best > > Etienne > > > Le 22/11/2023 à 00:52, Sergey Nuyanzin a écrit : > > Hi Etienne > > > > thanks for starting this discussion > > +1 from my side for snapshots since I also faced same issue > > > > > > > > On Thu, Nov 9, 2023 at 11:21 AM Etienne Chauchot > > wrote: > > > >> Hi all, > >> > >> flink-connector-shared-utils contains utilities for connectors (parent > >> pom, ci scripts, template test connector project etc...). It is divided > >> into 2 main branches: > >> > >> - parent_pom (1) containing just a pom.xml > >> > >> - ci_utils (2) containing the test project using the parent pom and the > >> ci scripts. > >> > >> > >> The problem is when we want to test changes to the parent pom in the > >> test project, we need to release org.apache.flink:flink-connector-parent > >> so that the test project in the other branch can use this updated > >> parent. This seems bad to trigger a release for testing. > >> > >> So I would like to propose setting up a snapshot for > >> org.apache.flink:flink-connector-parent with regular deployment to > >> https://repository.apache.org/content/repositories/snapshots like we > do > >> for flink. > >> > >> An alternative could be to keep using > >> io.github.zentol.flink:flink-connector-parent for testing and release > >> this special artifact when needed by ci_utils test project. > >> > >> WDYT ? > >> > >> > >> [1] > https://github.com/apache/flink-connector-shared-utils/tree/parent_pom > >> > >> [2]https://github.com/apache/flink-connector-shared-utils/tree/ci_utils > >> > >> Best > >> > >> Etienne > >> > > -- Best regards, Sergey
Re: [DISCUSSION] flink-connector-shared-utils release process
Hi Sergey, The other alternative that allows for autonomous testing of ci changes (thanks @Chesnay for the suggestion) is to create you own repo: - a flink-connector-shared-utils-*test* clone repo and a *io.github.user.flink*:flink-connector-parent custom artifact to be able to directly commit and install the artifact in the CI - a custom ci script that does the cloning and mvn install in the ci.yml github action script for testing with the new flink-connector-parent artifact see https://github.com/echauchot/flink-connector-shared-utils-test FLINK-* branches for details. I hope it helps, Best Etienne Le 22/11/2023 à 00:52, Sergey Nuyanzin a écrit : Hi Etienne thanks for starting this discussion +1 from my side for snapshots since I also faced same issue On Thu, Nov 9, 2023 at 11:21 AM Etienne Chauchot wrote: Hi all, flink-connector-shared-utils contains utilities for connectors (parent pom, ci scripts, template test connector project etc...). It is divided into 2 main branches: - parent_pom (1) containing just a pom.xml - ci_utils (2) containing the test project using the parent pom and the ci scripts. The problem is when we want to test changes to the parent pom in the test project, we need to release org.apache.flink:flink-connector-parent so that the test project in the other branch can use this updated parent. This seems bad to trigger a release for testing. So I would like to propose setting up a snapshot for org.apache.flink:flink-connector-parent with regular deployment to https://repository.apache.org/content/repositories/snapshots like we do for flink. An alternative could be to keep using io.github.zentol.flink:flink-connector-parent for testing and release this special artifact when needed by ci_utils test project. WDYT ? [1]https://github.com/apache/flink-connector-shared-utils/tree/parent_pom [2]https://github.com/apache/flink-connector-shared-utils/tree/ci_utils Best Etienne
Re: [DISCUSSION] flink-connector-shared-utils release process
Hi Etienne thanks for starting this discussion +1 from my side for snapshots since I also faced same issue On Thu, Nov 9, 2023 at 11:21 AM Etienne Chauchot wrote: > Hi all, > > flink-connector-shared-utils contains utilities for connectors (parent > pom, ci scripts, template test connector project etc...). It is divided > into 2 main branches: > > - parent_pom (1) containing just a pom.xml > > - ci_utils (2) containing the test project using the parent pom and the > ci scripts. > > > The problem is when we want to test changes to the parent pom in the > test project, we need to release org.apache.flink:flink-connector-parent > so that the test project in the other branch can use this updated > parent. This seems bad to trigger a release for testing. > > So I would like to propose setting up a snapshot for > org.apache.flink:flink-connector-parent with regular deployment to > https://repository.apache.org/content/repositories/snapshots like we do > for flink. > > An alternative could be to keep using > io.github.zentol.flink:flink-connector-parent for testing and release > this special artifact when needed by ci_utils test project. > > WDYT ? > > > [1] https://github.com/apache/flink-connector-shared-utils/tree/parent_pom > > [2] https://github.com/apache/flink-connector-shared-utils/tree/ci_utils > > Best > > Etienne > -- Best regards, Sergey
