You'll want to reach out to the users@ thread for getting feedback on deprecation.
On Mon, May 11, 2020 at 9:53 AM Alexey Romanenko <aromanenko....@gmail.com> wrote: > > Thanks Ismaël for recalling of this thread, I think we should start to > take some efforts to deprecate the AWS SDK V1 IOs that are already > implemeneted using V2 (if there are no other objections). In this case, it > would make sense to abstract some common code ONLY if ,for some reasons, we > would like to keep the both versions of IO (which is not a case for now, > afaik). > > > On 8 May 2020, at 00:03, Ismaël Mejía <ieme...@gmail.com> wrote: > > > > Achieving good abstractions will prove elusive since the APIs differ > > and we will end up with a ton of extra maintenance work that should be > > not Beam's responsibility. I know that similar code (almost copy > > pasteable) is not nice to have but we should consider this as a > > temporary measure and probably try to make deprecation (and possible > > removal) of older versions as soon as possible. > > Please remember that this was already discussed in the past [1] and > > the soft consensus was around rapid deprecation AWS SDK v1 IOs already > > available in their v2 version and only do improvements on v1 IOs for > > security related issues and dependency updates. This has not happened > > yet but the only missing thing is someone to take action. > > > > +cammac...@gmail.com would you have to me to start rolling this plan? > > otherwise someone else can jump and do it too. > > > > [1] > https://lists.apache.org/thread.html/130cb60e6bcdd58c5afdd0c375663eaf05e705aab9ee0196535cd17f%40%3Cdev.beam.apache.org%3E > > > > On Thu, May 7, 2020 at 11:05 PM Luke Cwik <lc...@google.com> wrote: > >> > >> I think you should try and share as much as is reasonable. Using what > is shared between AWS V1 and V2 SDKs would be a good signal as to what > should be shared in Beam. There might be some places where a trivial > wrapper could help but I wouldn't try to create a bunch of grand > abstractions that fit both versions of the AWS SDKs. > >> > >> On Wed, May 6, 2020 at 9:59 AM Alexey Romanenko < > aromanenko....@gmail.com> wrote: > >>> > >>> I’d like to get back to this question, fairly raised by Jonothan a > while ago, since actually it affects not only KinesisIO but also all other > AWS IOs in general, that uses AWS SDK of two different versions. > >>> > >>> My personal opinion - I’d strongly like to avoid a copy-pasted code > for two different versions of the same IO (and double support of this), > that is SDK-independent. In this case, if we wish too provide IO for two > AWS SDK versions, we have to extract this core logic into one common and > SDK-independent framework that could used by any of SDK versions then. > >>> > >>> In the same time, if it will require a lot of efforts to achieve that, > then probably we need to decide to leave only one IO based on single AWS > SDK version (I guess it will be V2 as more modern) and deprecate old > version (and then remove it to avoid a confusion for users). > >>> > >>> What others think about that? > >>> > >>> On 18 Apr 2020, at 01:48, Jonothan Farr <jonothan.f...@gmail.com> > wrote: > >>> > >>> Hi, I wanted to try out KinesisIO with enhanced fanout for a PoC I was > working on, so I ended up submitting that as > https://github.com/apache/beam/pull/9899. > >>> > >>> But since that still needs work to be fully functional (right now it > does not handle resharding), I figured I could at least contribute the > updates to make KinesisIO compatible with the AWS SDK v2, so I split that > out and submitted it as: https://github.com/apache/beam/pull/11318. > >>> > >>> Now some totally reasonable concerns have been raised about > maintaining two separate codebases, and having to dual-commit every bugfix > or feature enhancement. That seems to already be the case for all of the > other AWS IOs in amazon-web-services2 (dynamodb, sns, sqs), but do we want > to continue that trend? What do members of the community think? > >>> > >>> One solution is obviously to rewrite the AWS IOs with an added > abstraction layer so that the SDK-specific code can be factored out. > However, the KinesisIO class itself already has dependencies on the v1 AWS > SDK so I'm not aware of a way that this could be done so that it's > backwards-compatible. This means that everyone currently using KinesisIO > would most likely have to update their code to use the new interface > whether they wanted to use the new v2 SDK or continue to use v1. I think a > rewrite would also be a non-trivial task with more than a few details to be > worked out first. > >>> > >>> From my perspective, the options are: > >>> > >>> - Merge #11318 now and the community can use the v2 AWS SDK with > KinesisIO and I can continue working on enhanced fanout while a rewrite is > being worked on, but the community now has to dual-commit any changes to > KinesisIO. > >>> - Close #11318 and continue working on #9899 and I alone have to > dual-commit the changes until it can be merged, at which point everyone can > use enhanced fanout while the rewrite is being worked on. Then the > community will also have to dual-commit but only after enhanced fanout is > available. > >>> - Close both PRs and just wait for a rewrite, and no one can use the > v2 AWS SDK or enhanced fanout in KinesisIO until after it's done. (Not to > imply that I wouldn't be interested in contributing to that effort, just > for the sake of these 2 PRs.) > >>> - Don't do the rewrite at all and just live with dual-committing until > the KinesisIO based on the v1 SDK reaches its end of life. No one has to > update their code to continue using the v1 SDK, only if they want to move > to v2. > >>> > >>> There may be other options or considerations I'm missing here. What do > others think? > >>> > >>> Thanks, > >>> Jonothan > >>> > >>> > >
