see: https://github.com/apache/spark/pull/21939#issuecomment-412154343
yes, i can set up a build. have some Qs in the PR about building the spark package before running the python tests. On Fri, Aug 10, 2018 at 10:41 AM, Bryan Cutler <cutl...@gmail.com> wrote: > I agree that we should hold off on the Arrow upgrade if it requires major > changes to our testing. I did have another thought that maybe we could just > add another job to test against Python 3.5 and pyarrow 0.10.0 and keep all > current testing the same? I'm not sure how doable that is right now and > don't want to make a ton of extra work, so no objections from me to hold > off on things for now. > > On Fri, Aug 10, 2018 at 9:48 AM, shane knapp <skn...@berkeley.edu> wrote: > >> On Fri, Aug 10, 2018 at 9:47 AM, Wenchen Fan <cloud0...@gmail.com> wrote: >> >>> It seems safer to skip the arrow 0.10.0 upgrade for Spark 2.4 and leave >>> it to Spark 3.0, so that we have more time to test. Any objections? >>> >> >> none here. >> >> -- >> Shane Knapp >> UC Berkeley EECS Research / RISELab Staff Technical Lead >> https://rise.cs.berkeley.edu >> > > -- Shane Knapp UC Berkeley EECS Research / RISELab Staff Technical Lead https://rise.cs.berkeley.edu
