Re: Update / Delete records in Parquet

2019-05-03 Thread Chetan Khatri
quot; > *Date: *Tuesday, 23 April 2019 at 11:35 > *To: *Chetan Khatri , Jason Nerothin < > jasonnerot...@gmail.com> > *Cc: *user > *Subject: *Re: Update / Delete records in Parquet > > > > Hi Chetan, > > > > I also agree that for this usecase parquet would

Re: Update / Delete records in Parquet

2019-05-01 Thread Vitaliy Pisarev
Ankit, you should take a look at delta.io<https://delta.io/> that was recently open sourced by databricks. Full DML support is on the way. From: "Khare, Ankit" Date: Tuesday, 23 April 2019 at 11:35 To: Chetan Khatri , Jason Nerothin Cc: user Subject: Re: Update / Delete re

Re: Update / Delete records in Parquet

2019-04-23 Thread Khare, Ankit
to connect to MSSQL and then get CDC data to Apache KUDU Total records. : 3 B Thanks Ankit From: Chetan Khatri Date: Tuesday, 23. April 2019 at 05:58 To: Jason Nerothin Cc: user Subject: Re: Update / Delete records in Parquet Hello Jason, Thank you for reply. My use case is that, first time I

Re: Update / Delete records in Parquet

2019-04-22 Thread Chetan Khatri
s like it might be the wrong sink for a high-frequency change > scenario. > > What are you trying to accomplish? > > Thanks, > Jason > > On Mon, Apr 22, 2019 at 2:09 PM Chetan Khatri > wrote: > >> Hello All, >> >> If I am doing incremental load / delta

Re: Update / Delete records in Parquet

2019-04-22 Thread Jason Nerothin
and would like to update / delete > the records in parquet, I understands that parquet is immutable and can't > be deleted / updated theoretically only append / overwrite can be done. But > I can see utility tools which claims to add value for that. > > https://github.com/Factual/par

Update / Delete records in Parquet

2019-04-22 Thread Chetan Khatri
Hello All, If I am doing incremental load / delta and would like to update / delete the records in parquet, I understands that parquet is immutable and can't be deleted / updated theoretically only append / overwrite can be done. But I can see utility tools which claims to add value