Hi Alexey, With regards to @Experimental there are a couple of discussions around its usage ( or rather over usage! ) on dev@. It is something that we need to clean up ( some of those IO are now being used on production env for years!).
Cheers Reza On Wed, 27 Nov 2019 at 04:50, Luke Cwik <lc...@google.com> wrote: > I suggested the wrapper because sometimes the intent of the APIs can be > translated easily but this is not always the case. > > Good to know that it is all marked @Experimental. > > On Tue, Nov 26, 2019 at 12:30 PM Cam Mach <cammac...@gmail.com> wrote: > >> Thank you, Alex for sharing the information, and Luke for the questions. >> I like the idea that just depreciate the V1 IOs, and just maintain V2 >> IOs, so we can support whoever want continue with V1. >> Just as Alex said, a lot of users, including my teams :-) , use the V1 >> IOs in production for real workload. So it'll be hard to remove V1 IOs or >> force them migrate to V2. But let hear if there are any other ideas? >> >> Btw, making V1 a wrapper around V2 is not very positive, code will get >> more complicated since V2 API is very different from V1's. >> >> Thanks, >> >> >> >> On Tue, Nov 26, 2019 at 8:21 AM Alexey Romanenko < >> aromanenko....@gmail.com> wrote: >> >>> AFAICT, all AWS SDK V1 IOs (SnsIO, SqsIO, DynamoDBIO, KinesisIO) are >>> marked as "Experimental". So, it should not be a problem to gracefully >>> deprecate and finally remove them. We already did the similar procedure for >>> “HadoopInputFormatIO”, which was renamed to just “HadoopFormatIO” (since it >>> started to support HadoopOutputFormatI as well). Old “HadoopInputFormatIO” >>> was deprecated and removed after *3 consecutive* Beam releases (as we >>> agreed on mailing list). >>> >>> In the same time, some users for some reasons would not be able or to >>> want to move on AWS SDK V2. So, I’d prefer to just deprecate AWS SDK V1 IOs >>> and accept new features/fixes *only* for V2 IOs. >>> >>> Talking about “Experimental” annotation. Sorry in advance If I missed >>> that and switch a subject a bit, but do we have clear rules or an agreement >>> when IO becomes stable and should not be marked as experimental anymore? >>> *Most* of our Java IOs are marked as Experimental but many of them were >>> using in production by real users under real load. Does it mean that they >>> are ready to be stable in terms of API? Perhaps, this topic deserves a new >>> discussion if there are several opinions on that. >>> >>> On 26 Nov 2019, at 00:39, Luke Cwik <lc...@google.com> wrote: >>> >>> Phase I sounds fine. >>> >>> Apache Beam follows semantic versioning and I believe removing the IOs >>> will be a backwards incompatible change unless they were marked >>> experimental which will be a problem for Phase 2. >>> >>> What is the feasibility of making the V1 transforms wrappers around V2? >>> >>> On Mon, Nov 25, 2019 at 1:46 PM Cam Mach <cammac...@gmail.com> wrote: >>> >>>> Hello Beam Devs, >>>> >>>> I have been working on the migration of Amazon Web Services IO >>>> connectors into the new AWS SDK for Java V2. The goal is to have an updated >>>> implementation aligned with the most recent AWS improvements. So far we >>>> have already migrated the connectors for AWS SNS, SQS and DynamoDB. >>>> >>>> In the meantime some contributions are still going on V1 IOs. So far we >>>> have dealt with those by porting (or asking contributors) to port the >>>> changes into V2 IOs too because we don’t want features of both versions to >>>> be unaligned but this may quickly become a maintenance issue, so we want to >>>> discuss a plan to stop supporting (deprecate) V1 IOs and encourage users to >>>> move to V2. >>>> >>>> Phase I (ASAP): >>>> >>>> - Mark migrated AWS V1 IOs as deprecated >>>> - Document migration path to V2 >>>> >>>> Phase II (end of 2020): >>>> >>>> - Decide a date or Beam release to remove the V1 IOs >>>> - Send a notification to the community 3 months before we remove >>>> them >>>> - Completely get rid of V1 IOs >>>> >>>> >>>> Please let me know what you think or if you see any potential issues? >>>> >>>> Thanks, >>>> Cam Mach >>>> >>>> >>> -- This email may be confidential and privileged. If you received this communication by mistake, please don't forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person. The above terms reflect a potential business arrangement, are provided solely as a basis for further discussion, and are not intended to be and do not constitute a legally binding obligation. No legally binding obligations will be created, implied, or inferred until an agreement in final form is executed in writing by all parties involved.
