Hi devs, I have observed that in [1], connectors and formats are pluggable, allowing user code to be easily integrated. The advantages of having pluggable connectors are evident, as it helps avoid conflicts between different versions of jar packages. If classloader isolation is not used, shading becomes necessary to resolve conflicts, resulting in a significant waste of development time. I understand that implementing this change may require numerous API modifications, so I would like to discuss in this email.
> Plugins cannot access classes from other plugins or from Flink that have not > been specifically whitelisted. > This strict isolation allows plugins to contain conflicting versions of the > same library without the need to relocate classes or to converge to common > versions. > Currently, file systems and metric reporters are pluggable but, in the > future, connectors, formats, and even user code should also be pluggable. [2] It is my preliminary idea. [1] https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/filesystems/overview/ [2] https://docs.google.com/document/d/1XP2fBpcntK0YIdQ_Ax7JV2MhNdebvkFxSiNJRp6WQ24/edit?usp=sharing Best, Zhiqiang
