Re: Question on ordering on partitions when read

2021-03-25 Thread Ryan Blue
Yeah, I'd use IcebergGenerics to read a table. That's the simplest way. On Thu, Mar 25, 2021 at 11:49 AM Chen Song wrote: > Thanks Ryan. Reading one partition at a time sounds a logical thing to me > in my case. > > I cannot use a query engine for now. In that case, if IcebergGenerics > still th

Re: Question on ordering on partitions when read

2021-03-25 Thread Chen Song
Thanks Ryan. Reading one partition at a time sounds a logical thing to me in my case. I cannot use a query engine for now. In that case, if IcebergGenerics still the best way to read via core API? On Thu, Mar 25, 2021 at 2:16 PM Ryan Blue wrote: > Hi Chen, > > Iceberg doesn't guarantee any orde

Re: Question on ordering on partitions when read

2021-03-25 Thread Ryan Blue
Hi Chen, Iceberg doesn't guarantee any order for records returned by `IcebergGenerics`. If you want a specific order, I'd recommend using a query engine to sort or to read a partition at a time and then sort within that partition. Iceberg can't really guarantee order across files. The sort order

Re: Question on ordering on partitions when read

2021-03-25 Thread Chen Song
Popping up the question. On Wed, Mar 24, 2021 at 2:01 PM Chen Song wrote: > I want to clarify the ordering semantics (if deterministic) on partitions > returned when using iceberg core data API to read. > > Say I define a table with a *time* column and partition by *day(time)*, and > do the foll

Question on ordering on partitions when read

2021-03-24 Thread Chen Song
I want to clarify the ordering semantics (if deterministic) on partitions returned when using iceberg core data API to read. Say I define a table with a *time* column and partition by *day(time)*, and do the following writes. partition (day)time other data fields