Re: [discuss] dropping Python 2.6 support

FWIW, RHEL 6 still uses Python 2.6, although 2.7.8 and 3.3.2 are available through Red Hat Software Collections. See: https://www.softwarecollections.org/en/ I run an academic compute cluster on RHEL 6. We do, however, provide Python 2.7.x and 3.5.x via modulefiles. On Tue, Jan 5, 2016 at 8:45

Re: [discuss] dropping Python 2.6 support

Sasha, it is more complicated than that: many RHEL 6 OS utilities rely on Python 2.6. Upgrading it to 2.7 breaks the system. For large enterprises migrating to another server OS means re-certifying (re-testing) hundreds of applications, so yes, they do prefer to stay where they are until the

Re: [discuss] dropping Python 2.6 support

+1 Companies that use stock python in redhat 2.6 will need to upgrade or install fresh version wich is total of 3.5 minutes so no issues ... On Tue, Jan 5, 2016 at 2:17 AM, Reynold Xin wrote: > Does anybody here care about us dropping support for Python 2.6 in Spark > 2.0?

Re: [discuss] dropping Python 2.6 support

Chiming in late, but my take on this line of argument is: these companies are welcome to keep using Spark 1.x. If anything the argument here is about how long to maintain 1.x, and indeed, it's going to go dormant quite soon. But using RHEL 6 (or any old-er version of any platform) and not wanting

Re: [discuss] dropping Python 2.6 support

> On 7 Jan 2016, at 19:55, Juliet Hougland wrote: > > @ Reynold Xin @Josh Rosen: What is current maintenance burden of supporting > Python 2.6? What libraries are no longer supporting Python 2.6 and where does > Spark use them? > generally the cost comes in the

Re: [discuss] dropping Python 2.6 support

On Sat, Jan 9, 2016 at 1:48 PM, Sean Owen wrote: > (For similar reasons I personally don't favor supporting Java 7 or > Scala 2.10 in Spark 2.x.) That reflects my sentiments as well. Thanks Sean for bringing that up! Jacek

Re: [discuss] dropping Python 2.6 support

+1 On Tue, Jan 5, 2016 at 1:57 PM, Jian Feng Zhang wrote: > +1 > > We use Python 2.7+ and 3.4+ to call PySpark. > > 2016-01-05 15:58 GMT+08:00 Kushal Datta : > >> +1 >> >> >> Dr. Kushal Datta >> Senior Research Scientist >> Big Data Research &

Re: [discuss] dropping Python 2.6 support

+1 Red Hat supports Python 2.6 on REHL 5 until 2020 , but otherwise yes, Python 2.6 is ancient history and the core Python developers stopped supporting it in 2013. REHL 5 is not a good enough reason to continue support for Python

Re: [discuss] dropping Python 2.6 support

+1 > On Jan 5, 2016, at 10:49 AM, Davies Liu wrote: > > +1 > > On Tue, Jan 5, 2016 at 5:45 AM, Nicholas Chammas > wrote: >> +1 >> >> Red Hat supports Python 2.6 on REHL 5 until 2020, but otherwise yes, Python >> 2.6 is ancient history and

Re: [discuss] dropping Python 2.6 support

Unfortunately, Koert is right. I've been in a couple of projects using Spark (banking industry) where CentOS + Python 2.6 is the toolbox available. That said, I believe it should not be a concern for Spark. Python 2.6 is old and busted, which is totally opposite to the Spark philosophy IMO.

Re: [discuss] dropping Python 2.6 support

I don't see a reason Spark 2.0 would need to support Python 2.6. At this point, Python 3 should be the default that is encouraged. Most organizations acknowledge the 2.7 is common, but lagging behind the version they should theoretically use. Dropping python 2.6 support sounds very reasonable to

Re: [discuss] dropping Python 2.6 support

+1 On Tue, Jan 5, 2016 at 5:45 AM, Nicholas Chammas wrote: > +1 > > Red Hat supports Python 2.6 on REHL 5 until 2020, but otherwise yes, Python > 2.6 is ancient history and the core Python developers stopped supporting it > in 2013. REHL 5 is not a good enough reason

Re: [discuss] dropping Python 2.6 support

rhel/centos 6 ships with python 2.6, doesnt it? if so, i still know plenty of large companies where python 2.6 is the only option. asking them for python 2.7 is not going to work so i think its a bad idea On Tue, Jan 5, 2016 at 1:52 PM, Juliet Hougland wrote: > I

Re: [discuss] dropping Python 2.6 support

As I pointed out in my earlier email, RHEL will support Python 2.6 until 2020. So I'm assuming these large companies will have the option of riding out Python 2.6 until then. Are we seriously saying that Spark should likewise support Python 2.6 for the next several years? Even though the core

Re: [discuss] dropping Python 2.6 support

If users are able to install Spark 2.0 on their RHEL clusters, then I imagine that they're also capable of installing a standalone Python alongside that Spark version (without changing Python systemwide). For instance, Anaconda/Miniconda make it really easy to install Python 2.7.x/3.x without

Re: [discuss] dropping Python 2.6 support

i do not think so. does the python 2.7 need to be installed on all slaves? if so, we do not have direct access to those. also, spark is easy for us to ship with our software since its apache 2 licensed, and it only needs to be present on the machine that launches the app (thanks to yarn). even

Re: [discuss] dropping Python 2.6 support

yeah, the practical concern is that we have no control over java or python version on large company clusters. our current reality for the vast majority of them is java 7 and python 2.6, no matter how outdated that is. i dont like it either, but i cannot change it. we currently don't use pyspark

Re: [discuss] dropping Python 2.6 support

plus 1, we are currently using python 2.7.2 in production environment. 在 2016-01-05 18:11:45，"Meethu Mathew" 写道： +1 We use Python 2.7 Regards, Meethu Mathew On Tue, Jan 5, 2016 at 12:47 PM, Reynold Xin wrote: Does anybody here care

Re: [discuss] dropping Python 2.6 support

+1 On Wed, Jan 6, 2016 at 9:18 AM, Juliet Hougland wrote: > Most admins I talk to about python and spark are already actively (or on > their way to) managing their cluster python installations. Even if people > begin using the system python with pyspark, there is

Re: [discuss] dropping Python 2.6 support

hey evil admin:) i think the bit about java was from me? if so, i meant to indicate that the reality for us is java is 1.7 on most (all?) clusters. i do not believe spark prefers java 1.8. my point was that even although java 1.7 is getting old as well it would be a major issue for me if spark

Re: [discuss] dropping Python 2.6 support

interesting i didnt know that! On Tue, Jan 5, 2016 at 5:57 PM, Nicholas Chammas wrote: > even if python 2.7 was needed only on this one machine that launches the > app we can not ship it with our software because its gpl licensed > > Not to nitpick, but maybe this

Re: [discuss] dropping Python 2.6 support

I think all the slaves need the same (or a compatible) version of Python installed since they run Python code in PySpark jobs natively. On Tue, Jan 5, 2016 at 6:02 PM Koert Kuipers wrote: > interesting i didnt know that! > > On Tue, Jan 5, 2016 at 5:57 PM, Nicholas Chammas <

Re: [discuss] dropping Python 2.6 support

even if python 2.7 was needed only on this one machine that launches the app we can not ship it with our software because its gpl licensed Not to nitpick, but maybe this is important. The Python license is GPL-compatible but not GPL : Note GPL-compatible

Re: [discuss] dropping Python 2.6 support

Created JIRA: https://issues.apache.org/jira/browse/SPARK-12661 On Tue, Jan 5, 2016 at 2:49 PM, Koert Kuipers wrote: > i do not think so. > > does the python 2.7 need to be installed on all slaves? if so, we do not > have direct access to those. > > also, spark is easy for us

Re: [discuss] dropping Python 2.6 support

if python 2.7 only has to be present on the node that launches the app (does it?) than that could be important indeed. On Tue, Jan 5, 2016 at 6:02 PM, Koert Kuipers wrote: > interesting i didnt know that! > > On Tue, Jan 5, 2016 at 5:57 PM, Nicholas Chammas < >

Re: [discuss] dropping Python 2.6 support

+1 Dr. Kushal Datta Senior Research Scientist Big Data Research & Pathfinding Intel Corporation, USA. On Mon, Jan 4, 2016 at 11:52 PM, Jean-Baptiste Onofré wrote: > +1 > > no problem for me to remove Python 2.6 in 2.0. > > Thanks > Regards > JB > > > On 01/05/2016 08:17