Thanks Siddharth, great work! I'd like to run through it a final time and possibly try out with Spark before merging to master to make sure there won't be any big issues. Could we set tomorrow EOD as the time as the time to merge, unless anything comes up?
Thanks, Bryan On Mon, Nov 13, 2017 at 2:39 PM, Li Jin <ice.xell...@gmail.com> wrote: > Thanks Siddharth. > > I agree we should start merging refactor branch to master - this will > allows us to make further Java changes easier. There are still left items > on https://issues.apache.org/jira/browse/ARROW-1463, we can work on those > after merging to master. > > > On Mon, Nov 13, 2017 at 5:31 PM, Siddharth Teotia <siddha...@dremio.com> > wrote: > > > Functional and Performance testing has been completed with Dremio. We > have > > seen overall improvement in TPCH numbers. We had about 8000 regression > > tests and 12000 unit tests. > > > > I would like to start the process of merging java-vector-refactor branch > > into master. The branch has 2 patches with 95% of code changes and a > third > > patch will be available in couple of hours -- minor bug fixes needed as > > part of testing in Dremio. > > > > I would also like to request to not merge any orthogonal set of changes > to > > refactor branch at this point. It will require grabbing changes again, > and > > testing downstream and thus increase the timeline of merging this into > > master. It is become slightly difficult to maintain local branches with > > such volume of changes and work across patches. > > > > Follow-up items in the order of priority: > > > > - buffer consolidation (sooner but probably not this Arrow release) -- > > possibly the highest priority item in the effort to reduce heap usage. > > This > > was touched upon in design spec but it wasn't feasible to squeeze it > > into > > the work already done on refactor branch. > > > > > > - fate of non-nullable fixed and var width vectors > > > > Thanks, > > Siddharth > > >
