We're trying to do some semi-advanced custom logic (custom writers and
schemas per destination) with AvroIO, and want to use
DynamicAvroDestinations to accomplish this.

However, AvroIO.to(DynamicAvroDestinations) is deprecated, but there
doesn't seem to be any other way to accomplish what we want here.
AvroIO.sink is much less sophisticated than the non-sink options, missing
much of the configurability that the non-sink version has.  For example,
there's no way to project from the UserT -> OutputT with the sink version,
only from UserT -> GenericRecord, which isn't what we want.

It seems like most things would be trivial to fix or add on the AvroIO.sink
implementation, is that the intended way that people would be consuming
AvroIO?  I'm a little confused with FileIO.write/writeDynamic vs WriteFiles
vs AvroIO.write, some seem deprecated, and some seem
not-deprecated-but-not-recommended.  To add to the confusion AvroIO.write
uses WriteFiles, but the documentation for the deprecated
AvroIO.to(DynamicAvroDestinations) points to FileIO.write.  Which is the
"right" one to use?

Reply via email to