+1 Thanks, Danny - I really appreciate you taking the time for the in-depth investigation. Please proceed, looking forward to your experience.
On Mon, Jan 8, 2024 at 6:04 PM Martijn Visser <martijnvis...@apache.org> wrote: > Thanks for investigating Danny. It looks like the best direction to go to > :) > > On Mon, Jan 8, 2024 at 5:56 PM Péter Váry <peter.vary.apa...@gmail.com> > wrote: > > > > Thanks Danny for working on this! > > > > It would be good to do this in a way that the different connectors could > > reuse as much code as possible, so if possible put most of the code to > the > > flink connector shared utils repo [1] > > > > +1 from for the general direction (non-binding) > > > > Thanks, > > Peter > > > > [1] https://github.com/apache/flink-connector-shared-utils > > > > > > Danny Cranmer <dannycran...@apache.org> ezt írta (időpont: 2024. jan. > 8., > > H, 17:31): > > > > > Hello all, > > > > > > I have been working with Péter and Marton on externalizing python > > > connectors [1] from the main repo to the connector repositories. We > have > > > the code moved and the CI running tests for Kafka and AWS Connectors. > I am > > > now looking into the release process. > > > > > > When we undertake a Flink release we perform the following steps [2], > > > regarding Python: 1/ run python build on CI, 2/ download Wheels > artifacts, > > > 3/ upload artifacts to the dist and 4/ deploy to pypi. The plan is to > > > follow the same steps for connectors, using Github actions instead of > Azure > > > pipeline. > > > > > > Today we have a single pypi project for pyflink that contains all the > Flink > > > libs, apache-flink [3]. I propose we create a new pypi project per > > > connector using the existing connector version, and following naming > > > convention: apache-<connector-name>, for example: > > > apache-flink-connector-aws, apache-flink-connector-kafka. Therefore to > use > > > a DataStream API connector in python, users would need to first > install the > > > lib, for example "python -m pip install apache-flink-connector-aws". > > > > > > Once we have consensus I will update the release process and perform a > > > release of the flink-connector-aws project to test it end-to-end. I > look > > > forward to any feedback. > > > > > > Thanks, > > > Danny > > > > > > [1] https://issues.apache.org/jira/browse/FLINK-33528 > > > [2] > > > > https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release > > > [3] https://pypi.org/project/apache-flink/ > > > >
