I agree that it would be nice to offer a backport for all 1.4 users. Thomas would it be possible to contribute the backport back to the community?
Cheers, Till On Wed, Jun 6, 2018 at 5:25 PM Thomas Weise <[email protected]> wrote: > --> > > On Wed, Jun 6, 2018 at 1:06 AM, Bowen Li <[email protected]> wrote: > > > Hi, > > > > I think the following email thread might have gone lost. > > > > Dyana brought up the attention that AWS has informed users that KPL > > versions > > < 0.12.6 will *stop working* starting from the 12th of June. Flink 1.4 is > > using KPL 0.12.5 and Flink 1.5 uses 0.12.6, *so Flink 1.4 and below (1.3, > > etc) will be impacted.* > > > > I think we probably should try our best to communicate this out to our > > users in user email alias and all other possible channels, in order not > to > > disrupt their production pipeline. > > > > A quick solution we can suggest to users is to package and use > > flink-connector-kinesis in Flink 1.5. Given that the public APIs don't > > change, flink-connector-kinesis in Flink 1.5 should work with Flink 1.4 > and > > below, but that needs verification. > > > > The connector might work, but it is going to require a little extra > juggling for the users to exclude the 1.5 transitive dependencies. > > How about just backporting the changes? We internally use the newer SDK > with a number of other consumer patches on top of 1.4.2 and it works fine. > > Normally such changes don't qualify for a patch version, but since it is > going to stop working anyways, there is probably no harm in this instance. > > Thomas > > > > > What do you think? > > > > Thanks, Bowen > > > > > > ---------- Forwarded message ---------- > > From: Bowen Li <[email protected]> > > Date: Fri, May 11, 2018 at 10:28 AM > > Subject: Re: KPL in current stable 1.4.2 and below, upcoming problem > > To: [email protected], "Tzu-Li (Gordon) Tai" <[email protected]> > > > > > > Thanks, this is a great heads-up! Flink 1.4 is using KPL 0.12.5, *so > Flink > > version 1.4 or below will be effected.* > > > > Kinesis sink is in flink-kinesis-connector. Good news is that whoever is > > using flink-kinesis-connector right now is building it themself, because > > Flink doesn't publish that jar into maven due to licensing issue. So > these > > users, like you Dyana, already have build experience and may try to bump > > KPL themselves. > > > > I think it'll be great if Flink can bump KPL in Flink 1.2/1.3/1.4 and > > release minor versions for them, as an official support. It also requires > > checking backward compatibility. This can be done after releasing 1.5. > > @Gordon may take the final call of how to eventually do it. > > > > Thanks, > > Bowen > > > > > > On Thu, May 10, 2018 at 1:35 AM, Dyana Rose <[email protected]> > > wrote: > > > > > Hello, > > > > > > We've received notification from AWS that the Kinesis Producer Library > > > versions < 0.12.6 will stop working after the 12th of June (assuming > the > > > date in the email is in US format...) > > > > > > Flink v1.5.0 has the KPL version at 0.12.6 so it will be fine when it's > > > released. However using the kinesis connector in any previous version > > looks > > > like they'll have an issue. > > > > > > I'm not sure how/if you want to communicate this. We build Flink > > ourselves, > > > so I plan on having a look at any changes done to the Kinesis Sink in > > > v1.5.0 and then bumpimg the KPL version in our fork and rebuilding. > > > > > > Thanks, > > > Dyana > > > > > > below is the email we received (note: we're in eu-west-1): > > > -------- > > > > > > Hello, > > > > > > > > > > > > Your action is required: please update clients running Kinesis Producer > > > Library 0.12.5 or older or you will experience a breaking change to > your > > > application. > > > > > > > > > > > > We've discovered you have one or more clients writing data to Amazon > > > Kinesis Data Streams running an outdated version of the Kinesis > Producer > > > Library. On 6/12 these clients will be impacted if they are not updated > > to > > > Kinesis Producer Library version 0.12.6 or newer. On 06/12 Kinesis Data > > > Streams will install ATS certificates which will prevent these outdated > > > clients from writing to a Kinesis Data Stream. The result of this > change > > > will break any producer using KPL 0.12.5 or older. > > > > > > > > > * How do I update clients and applications to use the latest version of > > the > > > Kinesis Producer Library? > > > > > > You will need to ensure producers leveraging the Kinesis Producer > Library > > > have upgraded to version 0.12.6 or newer. If you operate older versions > > > your application will break due untrusted SSL certification. > > > > > > Via Maven install Kinesis Producer Library version 0.12.6 or higher [2] > > > > > > After you've configured your clients to use the new version, you're > done. > > > > > > * What if I have questions or issues? > > > > > > If you have questions or issues, please contact your AWS Technical > > Account > > > Manager or AWS support and file a support ticket [3]. > > > > > > [1] https://docs.aws.amazon.com/streams/latest/dev/kinesis-kpl-u > > > pgrades.html > > > > > > [2] http://search.maven.org/#artifactdetails|com.amazonaws|amazo > > > n-kinesis-produ... > > > <http://search.maven.org/#artifactdetails%7Ccom.amazonaws%7C > > > amazon-kinesis-producer%7C0.12.6%7Cjar> > > > > > > [3] https://aws.amazon.com/support > > > > > > > > > > > > - Amazon Kinesis Data Streams Team > > > ----- > > > > > >
