[
https://issues.apache.org/jira/browse/SPARK-25079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shane knapp reopened SPARK-25079:
---------------------------------
going to reopen this until i'm done w/the branch-2.3 and -2.4 python deployment.
> [PYTHON] upgrade python 3.4 -> 3.6
> ----------------------------------
>
> Key: SPARK-25079
> URL: https://issues.apache.org/jira/browse/SPARK-25079
> Project: Spark
> Issue Type: Improvement
> Components: Build, PySpark
> Affects Versions: 2.3.1
> Reporter: shane knapp
> Assignee: shane knapp
> Priority: Major
> Fix For: 3.0.0
>
>
> for the impending arrow upgrade
> (https://issues.apache.org/jira/browse/SPARK-23874) we need to bump python
> 3.4 -> 3.5.
> i have been testing this here:
> [https://amplab.cs.berkeley.edu/jenkins/view/RISELab%20Infra/job/ubuntuSparkPRB/|https://amplab.cs.berkeley.edu/jenkins/view/RISELab%20Infra/job/ubuntuSparkPRB/69]
> my methodology:
> 1) upgrade python + arrow to 3.5 and 0.10.0
> 2) run python tests
> 3) when i'm happy that Things Won't Explode Spectacularly, pause jenkins and
> upgrade centos workers to python3.5
> 4) simultaneously do the following:
> - create a symlink in /home/anaconda/envs/py3k/bin for python3.4 that
> points to python3.5 (this is currently being tested here:
> [https://amplab.cs.berkeley.edu/jenkins/view/RISELab%20Infra/job/ubuntuSparkPRB/69)]
> - push a change to python/run-tests.py replacing 3.4 with 3.5
> 5) once the python3.5 change to run-tests.py is merged, we will need to
> back-port this to all existing branches
> 6) then and only then can i remove the python3.4 -> python3.5 symlink
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
