Re: Consuming delta from Hive tables

2019-05-20 Thread Alan Gates
On Sun, May 19, 2019 at 11:21 PM Bhargav Bipinchandra Naik (Seller Platform-BLR) wrote: > Hi Alan, > > > Are write_ids monotonically increasing? > They are assigned monotonically, but the transactions they are a part of may commit at different times, so you can't use it as a low water mark. That

Re: Consuming delta from Hive tables

2019-05-20 Thread Bhargav Bipinchandra Naik (Seller Platform-BLR)
Hi Alan, Are write_ids monotonically increasing? Are write_ids accessible in the hive query? For e.g.: select * from table_name where write_id > N; Basically I am trying to understand if I can use write_id to consume only updated rows. Store the maximum write_id(X) seen in the result and next