python 3.5/pyarrow 0.10.0 build: https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test/job/spark-master-test-sbt-hadoop-2.6-python-3.5-arrow-0.10.0-ubuntu-testing/
On Fri, Aug 10, 2018 at 10:44 AM, shane knapp <skn...@berkeley.edu> wrote: > see: https://github.com/apache/spark/pull/21939#issuecomment-412154343 > > yes, i can set up a build. have some Qs in the PR about building the > spark package before running the python tests. > > On Fri, Aug 10, 2018 at 10:41 AM, Bryan Cutler <cutl...@gmail.com> wrote: > >> I agree that we should hold off on the Arrow upgrade if it requires major >> changes to our testing. I did have another thought that maybe we could just >> add another job to test against Python 3.5 and pyarrow 0.10.0 and keep all >> current testing the same? I'm not sure how doable that is right now and >> don't want to make a ton of extra work, so no objections from me to hold >> off on things for now. >> >> On Fri, Aug 10, 2018 at 9:48 AM, shane knapp <skn...@berkeley.edu> wrote: >> >>> On Fri, Aug 10, 2018 at 9:47 AM, Wenchen Fan <cloud0...@gmail.com> >>> wrote: >>> >>>> It seems safer to skip the arrow 0.10.0 upgrade for Spark 2.4 and leave >>>> it to Spark 3.0, so that we have more time to test. Any objections? >>>> >>> >>> none here. >>> >>> -- >>> Shane Knapp >>> UC Berkeley EECS Research / RISELab Staff Technical Lead >>> https://rise.cs.berkeley.edu >>> >> >> > > > -- > Shane Knapp > UC Berkeley EECS Research / RISELab Staff Technical Lead > https://rise.cs.berkeley.edu > -- Shane Knapp UC Berkeley EECS Research / RISELab Staff Technical Lead https://rise.cs.berkeley.edu
