Hi Pan,
Users who wish to utilize only one database and prefer not to use
flink-connector-jdbc-${version}.jar + ${database-connector-driver}.jar should
opt for option 1: flink-connector-jdbc-core-${version}.jar +
flink-connector-jdbc-mysql-${version}.jar + ${database-connector-driver}.jar.
We could introduce a flink-connector-jdbc-mysql-${version}-fat.jar that
includes flink-connector-jdbc-core-${version}.jar, but this could create
potential challenges. This approach could lead to duplicate classes if a user
intends to read from MySQL and write to PostgreSQL while utilizing both fat
JARs simultaneously.
To maintain clarity and minimize conflicts, we're currently leaning towards
maintaining the existing structure, where flink-connector-jdbc-${version}.jar
remains shaded for simplicity, encompassing the core functionality and all
database-related features within the same JAR.
Please let me know if you require further clarification on any aspect.
Best regards,
Joao Boto
On 2024/04/25 11:41:00 Yuepeng Pan wrote:
> Hi, Boto.
>
> Thanks for your driving it !
> +1 from me on the proposal.
>
>
>
>
> Maybe we need to ensure that a simple usage method is provided to users after
> the refactoring.
>
> In the current situation, which supported database does the user intend to
> use,
>
> Users only need to add the flink-connector-jdbc-${version}.jar +
> ${database-connector-driver}.jar
>
> into the dependencies, which could be used out of the box.
>
> I noticed in FLIP that we will perform shadow related operations to ensure
>
> the same usage and semantics as before.
>
> So, if users only want to use one type of database (eg. MySQL),
>
> what forms would we plan to provide jars in?
>
> For example:
>
> 1. flink-connector-jdbc-core-${version}.jar +
> flink-connector-jdbc-mysql-${version}.jar + ${database-connector-driver}.jar.
>
> 2. Or flink-connector-jdbc-mysql-${version}.jar +
> ${database-connector-driver}.jar.
>
> 3. Or a another different concise way?
>
>
>
>
> Thank you.
>
> Best,
> Yuepeng Pan
>
>
>
>
> At 2024-04-25 16:54:13, "Joao Boto" <b...@boto.pro> wrote:
> >Hi all,
> >
> >I'd like to start a discussion on FLIP-449: Reorganization of
> >flink-connector-jdbc [1].
> >As Flink continues to evolve, we've noticed an increasing level of
> >complexity within the JDBC connector.
> >The proposed solution is to address this complexity by separating the core
> >functionality from individual database components, thereby streamlining the
> >structure into distinct modules.
> >
> >Looking forward to your feedback and suggestions, thanks.
> >Best regards,
> >Joao Boto
> >
> >[1]
> >https://cwiki.apache.org/confluence/display/FLINK/FLIP-449%3A+Reorganization+of+flink-connector-jdbc
>
