Thanks for investigating Danny. It looks like the best direction to go to :)
On Mon, Jan 8, 2024 at 5:56 PM Péter Váry <peter.vary.apa...@gmail.com> wrote: > > Thanks Danny for working on this! > > It would be good to do this in a way that the different connectors could > reuse as much code as possible, so if possible put most of the code to the > flink connector shared utils repo [1] > > +1 from for the general direction (non-binding) > > Thanks, > Peter > > [1] https://github.com/apache/flink-connector-shared-utils > > > Danny Cranmer <dannycran...@apache.org> ezt írta (időpont: 2024. jan. 8., > H, 17:31): > > > Hello all, > > > > I have been working with Péter and Marton on externalizing python > > connectors [1] from the main repo to the connector repositories. We have > > the code moved and the CI running tests for Kafka and AWS Connectors. I am > > now looking into the release process. > > > > When we undertake a Flink release we perform the following steps [2], > > regarding Python: 1/ run python build on CI, 2/ download Wheels artifacts, > > 3/ upload artifacts to the dist and 4/ deploy to pypi. The plan is to > > follow the same steps for connectors, using Github actions instead of Azure > > pipeline. > > > > Today we have a single pypi project for pyflink that contains all the Flink > > libs, apache-flink [3]. I propose we create a new pypi project per > > connector using the existing connector version, and following naming > > convention: apache-<connector-name>, for example: > > apache-flink-connector-aws, apache-flink-connector-kafka. Therefore to use > > a DataStream API connector in python, users would need to first install the > > lib, for example "python -m pip install apache-flink-connector-aws". > > > > Once we have consensus I will update the release process and perform a > > release of the flink-connector-aws project to test it end-to-end. I look > > forward to any feedback. > > > > Thanks, > > Danny > > > > [1] https://issues.apache.org/jira/browse/FLINK-33528 > > [2] > > https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release > > [3] https://pypi.org/project/apache-flink/ > >
