Convert flink table with field of type RAW to datastream

2020-06-17 Thread YI
Hi all, I am using flink to process external data. The source format is json, and the underlying data types are defined in a external library. I generated table schema with `TableSchema.fromTypeInfo` and `TypeInformation.of[_]`. From what I read, this method is deprecated. But I didn't find any

Re: Convert flink table with field of type RAW to datastream

2020-06-17 Thread Jark Wu
Hi YI, Flink doesn't have a TypeInformation for `java.util.Date`, but only SqlTimeTypeInfo.DATE for `java.sql.Date`. That's why the TypeInformation.of(java.util.Date) is being recognized as a RAW type. To resolve your problem, I think in `TypeInformation.of(..)` you should use a concrete type for

Re: Convert flink table with field of type RAW to datastream

2020-06-17 Thread YI
Hi Jark, Thank you for your suggestion. My current problem is that there are quite a few data types. All these data types are defined upstream which I have no control. I don't think I can easily change the type information of a specific field. Can I? Things become nasty when there are so many `j

Re: Convert flink table with field of type RAW to datastream

2020-06-17 Thread Jark Wu
Flink SQL/Table requires to know the field data types explicitly. Maybe you can apply a MapFunction before `toTable` to convert/normalize the data and type. Best, Jark On Thu, 18 Jun 2020 at 14:12, YI wrote: > Hi Jark, > > Thank you for your suggestion. My current problem is that there are quit