Hello Zhiqiang, Although it is strange to receive such an email on the dev mailing list, I read your email carefully. The Pluggable mechanism for Connectors, Formats is indeed necessary, and I believe that developers in the community also have this consensus. Regarding the relationship between 2.0 and this feature, I would like to use two questions to elaborate:
* Can we only develop this feature in 2.0? No, we can also develop in versions 1.19 and 2.x. This is just a new improvement or mechanism, nothing special. * Does deprecate API only happen in 2.0? No, we can also deprecate API in versions 1.19 and 2.1 as well as other minor versions. So, I don't see any reason that we have to link this feature to 2.0 version. Regarding the feature discussion in the dev mail group, the correct way is to prepare a FLIP or at least a FLIP draft, and then open a [DISCUSS] xxx thread for discussion. Btw, some community contributors are no longer active due to personal interest or career development direction, usually we should not directly ping the corresponding developers, we should just open a [DISCUSS] thread to dev mailing-list and discuss the feature. Best, Leonard > On Jul 13, 2023, at 4:22 PM, zhiqiang li <lizhiqiang....@gmail.com> wrote: > > Flink CDC Maintainer > xbjt...@gmail.com <mailto:xbjt...@gmail.com> > > Factory Interface > Timo Walther <twal...@apache.org <mailto:twal...@apache.org>> > > Plugin Interface > Aleksey Pak <alek...@ververica.com <mailto:alek...@ververica.com>> > > Connector Base > Stephan Ewen <se...@apache.org <mailto:se...@apache.org>> > > Flink File System > Stephan Ewen <se...@apache.org <mailto:se...@apache.org>> > > I have observed that in [1], connectors and formats are pluggable, allowing > user code to be easily integrated. The advantages of having pluggable > connectors are evident, as it helps avoid conflicts between different > versions of jar packages. If classloader isolation is not used, shading > becomes necessary to resolve conflicts, resulting in a significant waste of > development time. I understand that implementing this change may require > numerous API modifications, so I would like to discuss in this email whether > it is feasible to make these changes in Flink 2.0. > > > Plugins cannot access classes from other plugins or from Flink that have > > not been specifically whitelisted. > > This strict isolation allows plugins to contain conflicting versions of the > > same library without the need to relocate classes or to converge to common > > versions. > > Currently, file systems and metric reporters are pluggable but, in the > > future, connectors, formats, and even user code should also be pluggable. > > I hope to be able to integrate into a set of APIs, which would require > significant changes to the integration API. Is it possible to address this in > Flink 2.0? However, due to [2] FLIP-321, Flink 1.18 should deprecate some > APIs, and the community is uncertain about the existence of Flink 1.20. This > may make it impossible to integrate a set of APIs. @Xintong Song > <tonysong...@gmail.com <mailto:tonysong...@gmail.com>> suggests continuing to > use two APIs. > > I am aware of some API modifications, but considering the extensive changes > involved, I am concerned that my understanding may not be comprehensive. I am > willing to do the work, but I have not yet found a committer. > > > [1] > https://nightlies.apache.org/Flink/Flink-docs-master/docs/deployment/filesystems/overview/ > > <https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/filesystems/overview/> > [2] FLIP-321 > https://cwiki.apache.org/confluence/display/FLINK/FLIP-321%3A+Introduce+an+API+deprecation+process > > <https://cwiki.apache.org/confluence/display/FLINK/FLIP-321%3A+Introduce+an+API+deprecation+process> > > > Best, > Zhiqiang
