I'm all for a release. The only thing still required for basic Hive read support (other than documentation of course!) is producing a *single* jar that can be added to Hive's classpath, the PR for that is at https://github.com/apache/iceberg/pull/1267.
Thanks, Adrian On Thu, 27 Aug 2020 at 01:26, Anton Okolnychyi <aokolnyc...@apple.com.invalid> wrote: > +1 on releasing structured streaming source. I should be able to do one > more review round tomorrow. > > - Anton > > On 26 Aug 2020, at 17:12, Jungtaek Lim <kabhwan.opensou...@gmail.com> > wrote: > > I hope we include Spark structured streaming read as well in the next > release; that was proposed in Feb this year and still around. Quoting my > comment on benefit of the streaming read on Spark; > > This would be the major feature to cover the gap on use case for >> structured streaming between Delta Lake and Iceberg. There's a technical >> limitation on Spark structured streaming itself (global watermark), which >> requires workaround via splitting query into multiple queries & >> intermediate storage supporting end-to-end exactly once. Delta Lake covers >> the case, and I really would like to see the case also covered by Iceberg. >> I see there're lots of works in progress on the milestone (and these are >> great features which should be done), but after this we cover both batch >> and streaming workloads being done with Spark, which is a huge step forward >> on Spark users. > > > Thanks, > Jungtaek Lim (HeartSaVioR) > > On Thu, Aug 27, 2020 at 1:13 AM Ryan Blue <rb...@netflix.com.invalid> > wrote: > >> Hi Marton, >> >> 0.9.0 was released about 6 weeks ago, so I don't think we've planned when >> the next release will be yet. I think it's a good idea to release soon, >> though. The Flink sink is close to being ready as well and I'd like to get >> both of those released so that the contributors can start using them. >> >> Seems like a good question for the broader community: how about a release >> in the next month or so for Hive reads and the Flink sink? >> >> rb >> >> On Wed, Aug 26, 2020 at 8:58 AM Marton Bod <marton....@gmail.com> wrote: >> >>> Hi Team, >>> >>> I was wondering whether there is a release cadence already in place for >>> Iceberg, e.g. how often releases will take place approximately? Which >>> commits/features as release candidates in the near term? >>> >>> We're looking to integrate Iceberg into Hive, however, the current 0.9.1 >>> release does not yet contain the StorageHandler code in iceberg-mr. Knowing >>> the approximate release timelines would help greatly with our integration >>> planning. >>> >>> Of course, happy to get involved with ongoing dev/stability efforts to >>> help achieve a new release of this module. >>> >>> Thanks a lot, >>> Marton >>> >> >> >> -- >> Ryan Blue >> Software Engineer >> Netflix >> > >
