Question on partitioning using Java API

2020-07-02 Thread Chen Song
I have a question on how hidden partitioning works in Iceberg using Java API. The code is something like the following. ``` // records is the list of records with a time column // table is created using partition spec hour(time) // records have different rows with different hours Table table = lo

Re: Iceberg V2 Spec

2020-07-02 Thread Ryan Blue
I'd like to get 0.9.0 out as soon as possible. I expect to get an early RC out next week, once we have more tests committed. That way, people can start trying it out and reporting back where it doesn't work. I'd rather not block 0.9.0 to wait on Flink connector components. There's still a lot of w

Re: The relationship between issues and pull requests

2020-07-02 Thread Ryan Blue
Thanks for the suggestion, Jingsong. I think it sounds like a good idea to link related pull requests and issues. We should all try to do that, and thank you for the link describing how to do it in GitHub. On Wed, Jul 1, 2020 at 11:51 PM Jingsong Li wrote: > Hi, > > When I look at the issues, it

Re: Question on partitioning using Java API

2020-07-02 Thread Ryan Blue
Hi Chen, Iceberg's API requires that the caller divides data correctly into files according to the partition spec. Most of the time, users interact with Iceberg using a processing engine like Spark or Presto that will do it for you. If you're using the API directly, then you'll need to ensure you

Re: Iceberg V2 Spec

2020-07-02 Thread OpenInx
Sounds good to me. Thanks. On Fri, Jul 3, 2020 at 12:58 AM Ryan Blue wrote: > I'd like to get 0.9.0 out as soon as possible. I expect to get an early RC > out next week, once we have more tests committed. That way, people can > start trying it out and reporting back where it doesn't work. > > I