[DISCUSS] Cross-version compatibility guarantees of Flink Modules/Jars

2020-06-23 Thread Aljoscha Krettek
Hi, this has come up a few times now and I think we need to discuss the guarantees that we want to officially give for this. What I mean by cross-version compatibility is using, say, a Flink 1.10 Kafka connector dependency/jar with Flink 1.11, or a Flink 1.10.0 connector with Flink 1.10.1. In

Re: [DISCUSS] Cross-version compatibility guarantees of Flink Modules/Jars

2020-06-23 Thread Konstantin Knauf
Hi Aljoscha, Thank you for bringing this up. IMHO the situation is different for minor & patch version upgrades. 1) I don't think we need to provide any guarantees across Flink minor versions (e.g. 1.10.1 -> 1.11.0). It seems reasonable to expect users to recompile their user JARs when upgrading

Re: [DISCUSS] Cross-version compatibility guarantees of Flink Modules/Jars

2020-06-24 Thread Aljoscha Krettek
Yes, I would say the situation is different for minor vs. patch. Side note: in a version like 1.x.y most people in the Flink community see x as the major version and y as the minor version. I know that this is not proper semver and I could be wrong about how people see it. It doesn't change an

Re: [DISCUSS] Cross-version compatibility guarantees of Flink Modules/Jars

2020-06-24 Thread Chesnay Schepler
Given that there are no compatibility guarantees forĀ  @PublicEvolving classes I'm not sure how much value there is in basing this discussion on their current state. Here's the thing; if we want connectors to be compatible across versions, then this should also work for user-defined connectors

Re: [DISCUSS] Cross-version compatibility guarantees of Flink Modules/Jars

2020-06-24 Thread Aljoscha Krettek
I recently did change the attitude there, see for example the new WatermarkStrategy and WatermarkGenerator and code around that. [1] [2] I think this is up to the judgement of folks that add new APIs, sometimes we don't know very well how things will shake out but I agree with Chesnay that we

Re: [DISCUSS] Cross-version compatibility guarantees of Flink Modules/Jars

2020-06-24 Thread Chesnay Schepler
> /I didn't check manually but I would say this probably affects all connectors and we shouldn't recommend people use them across different (minor, that is 1.x) versions. For patch versions it should be fine with our recent discussion of not breaking @PublicEvolving for patch (1.x.y) releases.