Re: [DISCUSS] FLIP-401: REST API JSON response deserialization unknown field tolerance
Thanks Gabor! +1 from my side, this sounds like a reasonable change that will improve integration and backward compatibility. Let's keep this open for at least another day before starting the vote. Cheers, Gyula On Thu, Dec 7, 2023 at 10:48 AM Gabor Somogyi wrote: > Hi All, > > I'd like to start a discussion of FLIP-401: REST API JSON response > deserialization unknown field tolerance[1]. > > *Problem statement:* > > At the moment Flink is not ignoring unknown fields when parsing REST > responses. An example for such a class is *JobDetailsInfo* but this applies > to all others. It would be good to add this support to increase > compatibility. > > The real life use-case is when the Flink k8s operator wants to handle 2 > jobs with 2 different Flink versions where the newer version has added a > new field to any REST response. Such case the operator has basically 2 > options: > * Use the old Flink version -> Such case exception comes because new field > comes but it's not expected > * Use the new Flink version -> Such case exception comes because new field > is not coming but expected > > To hack around this issue it requires quite some ugly code parts in the > operator. > > *Proposed solution:* > > Ignore all unknown fields in case of REST response JSON deserialization. > Important to know that strict serialization would stay the same as-is. > > Actual object mapper configuration which is intended to be changed can be > found here[2]. > > Please share your opinion on this topic. > > BR, > G > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-401%3A+REST+API+JSON+response+deserialization+unknown+field+tolerance > [2] > > https://github.com/apache/flink/blob/3060ccd49cc8d19634b431dbf0f09ac875d0d422/flink-runtime/src/main/java/org/apache/flink/runtime/rest/util/RestMapperUtils.java#L31-L38 >
Re: [DISCUSS] FLIP-401: REST API JSON response deserialization unknown field tolerance
+1 This greatly improves interfacing with multiple Flink versions, e.g. upgrades from the Kubernetes Operator. On Mon, Dec 11, 2023 at 12:36 PM Gyula Fóra wrote: > Thanks Gabor! > > +1 from my side, this sounds like a reasonable change that will > improve integration and backward compatibility. > > Let's keep this open for at least another day before starting the vote. > > Cheers, > Gyula > > On Thu, Dec 7, 2023 at 10:48 AM Gabor Somogyi > wrote: > > > Hi All, > > > > I'd like to start a discussion of FLIP-401: REST API JSON response > > deserialization unknown field tolerance[1]. > > > > *Problem statement:* > > > > At the moment Flink is not ignoring unknown fields when parsing REST > > responses. An example for such a class is *JobDetailsInfo* but this > applies > > to all others. It would be good to add this support to increase > > compatibility. > > > > The real life use-case is when the Flink k8s operator wants to handle 2 > > jobs with 2 different Flink versions where the newer version has added a > > new field to any REST response. Such case the operator has basically 2 > > options: > > * Use the old Flink version -> Such case exception comes because new > field > > comes but it's not expected > > * Use the new Flink version -> Such case exception comes because new > field > > is not coming but expected > > > > To hack around this issue it requires quite some ugly code parts in the > > operator. > > > > *Proposed solution:* > > > > Ignore all unknown fields in case of REST response JSON deserialization. > > Important to know that strict serialization would stay the same as-is. > > > > Actual object mapper configuration which is intended to be changed can be > > found here[2]. > > > > Please share your opinion on this topic. > > > > BR, > > G > > > > [1] > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-401%3A+REST+API+JSON+response+deserialization+unknown+field+tolerance > > [2] > > > > > https://github.com/apache/flink/blob/3060ccd49cc8d19634b431dbf0f09ac875d0d422/flink-runtime/src/main/java/org/apache/flink/runtime/rest/util/RestMapperUtils.java#L31-L38 > > >
Re: [DISCUSS] FLIP-401: REST API JSON response deserialization unknown field tolerance
Thanks for the responses! I'm going to start a vote based on that... On Mon, Dec 11, 2023 at 1:13 PM Márton Balassi wrote: > +1 This greatly improves interfacing with multiple Flink versions, e.g. > upgrades from the Kubernetes Operator. > > On Mon, Dec 11, 2023 at 12:36 PM Gyula Fóra wrote: > > > Thanks Gabor! > > > > +1 from my side, this sounds like a reasonable change that will > > improve integration and backward compatibility. > > > > Let's keep this open for at least another day before starting the vote. > > > > Cheers, > > Gyula > > > > On Thu, Dec 7, 2023 at 10:48 AM Gabor Somogyi > > > wrote: > > > > > Hi All, > > > > > > I'd like to start a discussion of FLIP-401: REST API JSON response > > > deserialization unknown field tolerance[1]. > > > > > > *Problem statement:* > > > > > > At the moment Flink is not ignoring unknown fields when parsing REST > > > responses. An example for such a class is *JobDetailsInfo* but this > > applies > > > to all others. It would be good to add this support to increase > > > compatibility. > > > > > > The real life use-case is when the Flink k8s operator wants to handle 2 > > > jobs with 2 different Flink versions where the newer version has added > a > > > new field to any REST response. Such case the operator has basically 2 > > > options: > > > * Use the old Flink version -> Such case exception comes because new > > field > > > comes but it's not expected > > > * Use the new Flink version -> Such case exception comes because new > > field > > > is not coming but expected > > > > > > To hack around this issue it requires quite some ugly code parts in the > > > operator. > > > > > > *Proposed solution:* > > > > > > Ignore all unknown fields in case of REST response JSON > deserialization. > > > Important to know that strict serialization would stay the same as-is. > > > > > > Actual object mapper configuration which is intended to be changed can > be > > > found here[2]. > > > > > > Please share your opinion on this topic. > > > > > > BR, > > > G > > > > > > [1] > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-401%3A+REST+API+JSON+response+deserialization+unknown+field+tolerance > > > [2] > > > > > > > > > https://github.com/apache/flink/blob/3060ccd49cc8d19634b431dbf0f09ac875d0d422/flink-runtime/src/main/java/org/apache/flink/runtime/rest/util/RestMapperUtils.java#L31-L38 > > > > > >
