Re: Updates/Deletes/Upserts in Iceberg

2019-05-07 Thread Jacques Nadeau
Awesome. This was on my list for some time. Glad you got it started. On Wed, May 8, 2019, 3:42 AM Anton Okolnychyi wrote: > Hi folks, > > Miguel (cc) and I have spent some time thinking about how to perform > updates/deletes/upserts on top of Iceberg tables. This functionality is > essential for

Updates/Deletes/Upserts in Iceberg

2019-05-07 Thread Anton Okolnychyi
Hi folks, Miguel (cc) and I have spent some time thinking about how to perform updates/deletes/upserts on top of Iceberg tables. This functionality is essential for many modern use cases. We've summarized our ideas in a doc [1], which, hopefully, will trigger a discussion in the community. The

Re: Micro-Batch Streaming

2019-05-07 Thread Anton Okolnychyi
> I'm reluctant to do this without an explicit call from the user or in a > service. The problem is when to expire snapshots. Iceberg is called regularly > to read and write tables. That might seem like a good time to expire > snapshots, but it doesn't make sense for either one to have a side ef