There is an alternative approach with getting PCollection<Row> using AvroIO and then converting Row to SpecificRecord using new Schemas APIs.
On Fri, Jun 14, 2019 at 1:58 AM Chamikara Jayalath <[email protected]> wrote: > I see. I don't think this is supported by AvroIO currently so your best > bet will probably be to do a GenericRecord to SpecificRecord conversion in > a ParDo that follows the read transform. > > On Thu, Jun 13, 2019 at 4:32 PM Neville Li <[email protected]> wrote: > >> That gives me a GenericRecord which is not type safe. >> In my case I have the compiled SpecificRecord class i.e. MyRecord >> available, but would like to pass in a schema other than >> MyRecord.getClassSchema() to say populate a subset of the fields. >> >> On Thu, Jun 13, 2019 at 6:18 PM Chamikara Jayalath <[email protected]> >> wrote: >> >>> Does AvroIO.readGenericRecords() work ? >>> >>> https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/io/AvroIO.java#L333 >>> >>> Thanks, >>> Cham >>> >>> On Thu, Jun 13, 2019 at 1:46 PM Neville Li <[email protected]> >>> wrote: >>> >>>> Hi, >>>> >>>> Is it just me or is there no way for AvroIO to read SpecificRecords >>>> with a custom reader schema? >>>> >>>> AvroIO.read(Class<T> recordClass) will use the schema of T and there's >>>> no way to override it. >>>> >>>> Cheers, >>>> Neville >>>> >>> -- Cheers, Gleb
