Re: Iceberg in Spark 3.0.0

2019-11-25 Thread Ryan Blue
orized reads. >>>> >>>> - Anton >>>> >>>> On 22 Nov 2019, at 02:26, Saisai Shao wrote: >>>> >>>> Hi Ryan and team, >>>> >>>> Thanks a lot for your response. I was wondering how do we share our >>

Re: Iceberg in Spark 3.0.0

2019-11-24 Thread Saisai Shao
;>> - Anton >>> >>> On 22 Nov 2019, at 02:26, Saisai Shao wrote: >>> >>> Hi Ryan and team, >>> >>> Thanks a lot for your response. I was wondering how do we share our >>> branch, one possible way s that we maintain a forked Icebe

Re: Iceberg in Spark 3.0.0

2019-11-22 Thread Ryan Blue
orized reads. >> >> - Anton >> >> On 22 Nov 2019, at 02:26, Saisai Shao wrote: >> >> Hi Ryan and team, >> >> Thanks a lot for your response. I was wondering how do we share our >> branch, one possible way s that we maintain a forked Iceberg re

Re: Iceberg in Spark 3.0.0

2019-11-22 Thread Anton Okolnychyi
orked Iceberg repo with Spark > 3.0.0-preview branch, another possible way is to create a branch in upstream > Iceberg repo. I'm inclined to choose the second way, so that they community > could review and contribute on it. > > I would like to hear your suggestions. > > Best regards

Re: Iceberg in Spark 3.0.0

2019-11-21 Thread Saisai Shao
Hi Ryan and team, Thanks a lot for your response. I was wondering how do we share our branch, one possible way s that we maintain a forked Iceberg repo with Spark 3.0.0-preview branch, another possible way is to create a branch in upstream Iceberg repo. I'm inclined to choose the second way, so

Re: Iceberg in Spark 3.0.0

2019-11-19 Thread Ryan Blue
Sounds great, thanks Saisai! On Mon, Nov 18, 2019 at 3:29 AM Saisai Shao wrote: > Thanks Anton, I will share our branch soon. > > Best regards, > Saisai > > Anton Okolnychyi 于2019年11月18日周一 下午6:54写道: > >> I think it would be great if you can share what you have, Saisai. That >> way, we can all

Re: Iceberg in Spark 3.0.0

2019-11-18 Thread Saisai Shao
Thanks Anton, I will share our branch soon. Best regards, Saisai Anton Okolnychyi 于2019年11月18日周一 下午6:54写道: > I think it would be great if you can share what you have, Saisai. That > way, we can all collaborate and ensure we build a full 3.0 integration as > soon as possible. > > - Anton > > >

Re: Iceberg in Spark 3.0.0

2019-11-18 Thread Anton Okolnychyi
I think it would be great if you can share what you have, Saisai. That way, we can all collaborate and ensure we build a full 3.0 integration as soon as possible. - Anton > On 18 Nov 2019, at 02:08, Saisai Shao wrote: > > Hi Anton, > > Thanks to bring this out. We already have a branch

Re: Iceberg in Spark 3.0.0

2019-11-17 Thread Saisai Shao
Hi Anton, Thanks to bring this out. We already have a branch building against Spark 3.0 (Master branch actually) internally, and we're actively working on it. I think it is a good idea to create an upstream Spark 3.0 branch, we could share it if the community would like to do so. Best regards,

Re: Iceberg in Spark 3.0.0

2019-11-17 Thread Anton Okolnychyi
I think it is a good time to create a branch to build our 3.0 integration as the 3.0 preview was released. What does everyone think? Has anybody started already? - Anton > On 8 Aug 2019, at 23:47, Edgar Rodriguez wrote: > > > > On Thu, Aug 8, 2019 at 3:37 PM Ryan Blue

Re: Iceberg in Spark 3.0.0

2019-08-08 Thread Edgar Rodriguez
On Thu, Aug 8, 2019 at 3:37 PM Ryan Blue wrote: > I think it's a great idea to branch and get ready for Spark 3.0.0. Right > now, I'm focused on getting a release out, but I can review patches for > Spark 3.0. > > Anyone know if there are nightly builds of Spark 3.0 to test with? > Seems like

Re: Iceberg in Spark 3.0.0

2019-08-08 Thread Ryan Blue
> Edgar Rodriguez 于2019年8月8日周四 上午4:58写道: > >> Hi everyone, >> >> I was wondering if there's a branch tracking the changes happening in >> Spark 3.0.0 for Iceberg. The DataSource V2 API has substantially changed >> from the one implemented in Iceberg master bra

Re: Iceberg in Spark 3.0.0

2019-08-07 Thread Saisai Shao
月8日周四 上午4:58写道: > Hi everyone, > > I was wondering if there's a branch tracking the changes happening in > Spark 3.0.0 for Iceberg. The DataSource V2 API has substantially changed > from the one implemented in Iceberg master branch and since Spark 3.0.0 > would allow us to i

Iceberg in Spark 3.0.0

2019-08-07 Thread Edgar Rodriguez
Hi everyone, I was wondering if there's a branch tracking the changes happening in Spark 3.0.0 for Iceberg. The DataSource V2 API has substantially changed from the one implemented in Iceberg master branch and since Spark 3.0.0 would allow us to introduce Spark SQL support then it seems