Hi tanu, This seems like a Spark-Parquet type conversion issue. I use timestamp type and don’t have any issue with it.
Would you try the following and provide more context? - save your dataframe as parquet instead of Hudi to see if the issue still persists - try timestamp type. - are you querying mor table using Sparksql? Thanks, Gary On Tue, Jul 21, 2020 at 1:23 AM tanu dua <[email protected]> wrote: > Thanks and even I am struggling with all data types except String with same > decode exception. For eg for both double and int and I got the exception > and when I convert to string all works fine in spark sql. > > On Tue, 21 Jul 2020 at 1:38 PM, Balaji Varadarajan > <[email protected]> wrote: > > > > > Gary/Udit, > > As you are familiar with this part of it, Can you please answer this > > question ? > > Thanks,Balaji.V On Monday, July 20, 2020, 08:18:16 AM PDT, tanu dua < > > [email protected]> wrote: > > > > Hi Guys, > > May I know how do you guys handle date and time stamp in Hudi. > > When I set DataTypes as Date in StructType it’s getting ingested as int > but > > when I query using spark sql I get the following > > > > https://issues.apache.org/jira/plugins/servlet/mobile#issue/SPARK-17557 > > > > So not sure if it’s only me who face this. Do I need to change to String > > ? >
