20200714 Weekly Sync Minutes

2020-07-21 Thread vbal...@apache.org
It was a very short meeting. The major highlight was : AWS Athena officially supporting Apache Hudi as a queryable source. https://cwiki.apache.org/confluence/display/HUDI/20200721+Weekly+Sync+Minutes Thanks,Balaji.V

Re: Date handling in HUDI

2020-07-21 Thread Mehrotra, Udit
Hi Tanu/Balaji, I have not really faced the issue mentioned here. AFAIK, the Date and Timestamp types should work fine. The Logical Date type is represented as INT in Avro, that is why you see the integer ingested there https://avro.apache.org/docs/current/spec.html#Date . But it should not

Re: the contributor permission

2020-07-21 Thread vbal...@apache.org
Welcome to Hudi. I have added your jira id.  Balaji.VOn Tuesday, July 21, 2020, 10:19:21 AM PDT, zjing...@sina.com wrote: Hi,I want to contribute to Apache Hudi. Would you please give me the contributor permission? My JIRA ID is AndyZhang0419

the contributor permission

2020-07-21 Thread zjing627
Hi,I want to contribute to Apache Hudi. Would you please give me the contributor permission? My JIRA ID is AndyZhang0419

Re: Kafka Hudi pipeline design

2020-07-21 Thread Balaji Varadarajan
Please see answers inline... On Sunday, July 19, 2020, 10:08:09 PM PDT, Lian Jiang wrote: Hi, I have a kafka topic using a kafka s3 connector to dump data into s3 hourly in parquet format. These parquet files are partitioned in ingestion time and each record has fields which are

Re: Date handling in HUDI

2020-07-21 Thread tanu dua
Thanks and even I am struggling with all data types except String with same decode exception. For eg for both double and int and I got the exception and when I convert to string all works fine in spark sql. On Tue, 21 Jul 2020 at 1:38 PM, Balaji Varadarajan wrote: > > Gary/Udit, > As you are

Re: Date handling in HUDI

2020-07-21 Thread Balaji Varadarajan
Gary/Udit, As you are familiar with this part of it, Can you please answer this question ? Thanks,Balaji.VOn Monday, July 20, 2020, 08:18:16 AM PDT, tanu dua wrote: Hi Guys, May I know how do you guys handle date and time stamp in Hudi. When I set DataTypes as Date in StructType it’s