AFAICT, all AWS SDK V1 IOs (SnsIO, SqsIO, DynamoDBIO, KinesisIO) are marked as "Experimental". So, it should not be a problem to gracefully deprecate and finally remove them. We already did the similar procedure for “HadoopInputFormatIO”, which was renamed to just “HadoopFormatIO” (since it started to support HadoopOutputFormatI as well). Old “HadoopInputFormatIO” was deprecated and removed after 3 consecutive Beam releases (as we agreed on mailing list).
In the same time, some users for some reasons would not be able or to want to move on AWS SDK V2. So, I’d prefer to just deprecate AWS SDK V1 IOs and accept new features/fixes only for V2 IOs. Talking about “Experimental” annotation. Sorry in advance If I missed that and switch a subject a bit, but do we have clear rules or an agreement when IO becomes stable and should not be marked as experimental anymore? Most of our Java IOs are marked as Experimental but many of them were using in production by real users under real load. Does it mean that they are ready to be stable in terms of API? Perhaps, this topic deserves a new discussion if there are several opinions on that. > On 26 Nov 2019, at 00:39, Luke Cwik <lc...@google.com> wrote: > > Phase I sounds fine. > > Apache Beam follows semantic versioning and I believe removing the IOs will > be a backwards incompatible change unless they were marked experimental which > will be a problem for Phase 2. > > What is the feasibility of making the V1 transforms wrappers around V2? > > On Mon, Nov 25, 2019 at 1:46 PM Cam Mach <cammac...@gmail.com > <mailto:cammac...@gmail.com>> wrote: > Hello Beam Devs, > > I have been working on the migration of Amazon Web Services IO connectors > into the new AWS SDK for Java V2. The goal is to have an updated > implementation aligned with the most recent AWS improvements. So far we have > already migrated the connectors for AWS SNS, SQS and DynamoDB. > > In the meantime some contributions are still going on V1 IOs. So far we have > dealt with those by porting (or asking contributors) to port the changes into > V2 IOs too because we don’t want features of both versions to be unaligned > but this may quickly become a maintenance issue, so we want to discuss a plan > to stop supporting (deprecate) V1 IOs and encourage users to move to V2. > > Phase I (ASAP): > Mark migrated AWS V1 IOs as deprecated > Document migration path to V2 > Phase II (end of 2020): > Decide a date or Beam release to remove the V1 IOs > Send a notification to the community 3 months before we remove them > Completely get rid of V1 IOs > > Please let me know what you think or if you see any potential issues? > > Thanks, > Cam Mach >
