I agree that we should hold off on the Arrow upgrade if it requires major changes to our testing. I did have another thought that maybe we could just add another job to test against Python 3.5 and pyarrow 0.10.0 and keep all current testing the same? I'm not sure how doable that is right now and don't want to make a ton of extra work, so no objections from me to hold off on things for now.
On Fri, Aug 10, 2018 at 9:48 AM, shane knapp <skn...@berkeley.edu> wrote: > On Fri, Aug 10, 2018 at 9:47 AM, Wenchen Fan <cloud0...@gmail.com> wrote: > >> It seems safer to skip the arrow 0.10.0 upgrade for Spark 2.4 and leave >> it to Spark 3.0, so that we have more time to test. Any objections? >> > > none here. > > -- > Shane Knapp > UC Berkeley EECS Research / RISELab Staff Technical Lead > https://rise.cs.berkeley.edu >
