As for 2, maybe it's time to remove @Experimental from SchemaCoder? 1 is tricky though. Changes like this have caused a lot of trouble for users in the past, and I think some users still have unpleasant memories of being told "you just have to change some package names and imports."
On Thu, Sep 3, 2020 at 6:18 PM Brian Hulette <bhule...@google.com> wrote: > Hi everyone, > The fact that core Beam has a dependency on Avro has led to a lot of > headaches when users (or runners) are using a different version. zeidoo [1] > was generous enough to put up a WIP PR [2] that moves everything that > depends on Avro (primarily AvroCoder and the Avro SchemaProvider I believe) > out of core Beam and into a separate extensions module. This way we could > have multiple extensions for different versions of Avro in the future. > > As I understand it, the downsides to making this change are: > 1) It's a breaking change, users with AvroCoder in their pipeline will > need to change their build dependencies and import statements. > 2) AvroCoder is the only (non-experimental) coder in core Beam that can > encode complex user types. So new users will need to dabble with the > Experimental SchemaCoder or add a second dependency to build a pipeline > with their own types. > > I think these costs are outweighed by the benefit of removing the > dependency in core Beam, but I wanted to reach out to the community to see > if there are any objections. > > Brian > > [1] github.com/zeidoo > [2] https://github.com/apache/beam/pull/12748 >
