Re: Why is Avro Date field using InstantCoder?

2021-10-17 Thread Cristian Constantinescu
If I had to change things, I would: 1. When deriving the SCHEMA add a few new types (JAVA_TIME, JAVA_DATE or something along those lines). 2. RowCoderGenerator around line 159 calls "SchemaCoder.coderForFieldType(schema.getField(rowIndex).getType().withNullable(false));" which eventually gets to

Define custom trigger

2021-10-17 Thread Eleanore Jin
Hi community, Here is my usecase: My pipeline uses another kafka topic as a SideInputs, that contains the filter criteria. Then when processing the mainstream, the pipeline is trying to see if each message from mainstream matches *any *existing filter criteria. The sideInputs logic is: whenever

Re: coder in ReadFromBigQuery doesn't do "anything"

2021-10-17 Thread Mark Striebeck
It's the same if I do a beam.Map(print) or write a test against the result. On Sun, Oct 17, 2021 at 4:42 AM Evan Galpin wrote: > Is “the result” being printed or viewed via debugger? Is there a chance > that the __repr__ or similar method for proto produces a dict strictly for >

Re: coder in ReadFromBigQuery doesn't do "anything"

2021-10-17 Thread Evan Galpin
Is “the result” being printed or viewed via debugger? Is there a chance that the __repr__ or similar method for proto produces a dict strictly for printing/serialization? Thanks, Evan On Sun, Oct 17, 2021 at 01:50 Mark Striebeck wrote: > Hi, > > I have the following BigQuery table: > > Name: