interesting i didnt know that! On Tue, Jan 5, 2016 at 5:57 PM, Nicholas Chammas <nicholas.cham...@gmail.com > wrote:
> even if python 2.7 was needed only on this one machine that launches the > app we can not ship it with our software because its gpl licensed > > Not to nitpick, but maybe this is important. The Python license is > GPL-compatible > but not GPL <https://docs.python.org/3/license.html>: > > Note GPL-compatible doesn’t mean that we’re distributing Python under the > GPL. All Python licenses, unlike the GPL, let you distribute a modified > version without making your changes open source. The GPL-compatible > licenses make it possible to combine Python with other software that is > released under the GPL; the others don’t. > > Nick > > > On Tue, Jan 5, 2016 at 5:49 PM Koert Kuipers <ko...@tresata.com> wrote: > >> i do not think so. >> >> does the python 2.7 need to be installed on all slaves? if so, we do not >> have direct access to those. >> >> also, spark is easy for us to ship with our software since its apache 2 >> licensed, and it only needs to be present on the machine that launches the >> app (thanks to yarn). >> even if python 2.7 was needed only on this one machine that launches the >> app we can not ship it with our software because its gpl licensed, so the >> client would have to download it and install it themselves, and this would >> mean its an independent install which has to be audited and approved and >> now you are in for a lot of fun. basically it will never happen. >> >> >> On Tue, Jan 5, 2016 at 5:35 PM, Josh Rosen <joshro...@databricks.com> >> wrote: >> >>> If users are able to install Spark 2.0 on their RHEL clusters, then I >>> imagine that they're also capable of installing a standalone Python >>> alongside that Spark version (without changing Python systemwide). For >>> instance, Anaconda/Miniconda make it really easy to install Python >>> 2.7.x/3.x without impacting / changing the system Python and doesn't >>> require any special permissions to install (you don't need root / sudo >>> access). Does this address the Python versioning concerns for RHEL users? >>> >>> On Tue, Jan 5, 2016 at 2:33 PM, Koert Kuipers <ko...@tresata.com> wrote: >>> >>>> yeah, the practical concern is that we have no control over java or >>>> python version on large company clusters. our current reality for the vast >>>> majority of them is java 7 and python 2.6, no matter how outdated that is. >>>> >>>> i dont like it either, but i cannot change it. >>>> >>>> we currently don't use pyspark so i have no stake in this, but if we >>>> did i can assure you we would not upgrade to spark 2.x if python 2.6 was >>>> dropped. no point in developing something that doesnt run for majority of >>>> customers. >>>> >>>> On Tue, Jan 5, 2016 at 5:19 PM, Nicholas Chammas < >>>> nicholas.cham...@gmail.com> wrote: >>>> >>>>> As I pointed out in my earlier email, RHEL will support Python 2.6 >>>>> until 2020. So I'm assuming these large companies will have the option of >>>>> riding out Python 2.6 until then. >>>>> >>>>> Are we seriously saying that Spark should likewise support Python 2.6 >>>>> for the next several years? Even though the core Python devs stopped >>>>> supporting it in 2013? >>>>> >>>>> If that's not what we're suggesting, then when, roughly, can we drop >>>>> support? What are the criteria? >>>>> >>>>> I understand the practical concern here. If companies are stuck using >>>>> 2.6, it doesn't matter to them that it is deprecated. But balancing that >>>>> concern against the maintenance burden on this project, I would say that >>>>> "upgrade to Python 2.7 or stay on Spark 1.6.x" is a reasonable position to >>>>> take. There are many tiny annoyances one has to put up with to support >>>>> 2.6. >>>>> >>>>> I suppose if our main PySpark contributors are fine putting up with >>>>> those annoyances, then maybe we don't need to drop support just yet... >>>>> >>>>> Nick >>>>> 2016년 1월 5일 (화) 오후 2:27, Julio Antonio Soto de Vicente <ju...@esbet.es>님이 >>>>> 작성: >>>>> >>>>>> Unfortunately, Koert is right. >>>>>> >>>>>> I've been in a couple of projects using Spark (banking industry) >>>>>> where CentOS + Python 2.6 is the toolbox available. >>>>>> >>>>>> That said, I believe it should not be a concern for Spark. Python 2.6 >>>>>> is old and busted, which is totally opposite to the Spark philosophy IMO. >>>>>> >>>>>> >>>>>> El 5 ene 2016, a las 20:07, Koert Kuipers <ko...@tresata.com> >>>>>> escribió: >>>>>> >>>>>> rhel/centos 6 ships with python 2.6, doesnt it? >>>>>> >>>>>> if so, i still know plenty of large companies where python 2.6 is the >>>>>> only option. asking them for python 2.7 is not going to work >>>>>> >>>>>> so i think its a bad idea >>>>>> >>>>>> On Tue, Jan 5, 2016 at 1:52 PM, Juliet Hougland < >>>>>> juliet.hougl...@gmail.com> wrote: >>>>>> >>>>>>> I don't see a reason Spark 2.0 would need to support Python 2.6. At >>>>>>> this point, Python 3 should be the default that is encouraged. >>>>>>> Most organizations acknowledge the 2.7 is common, but lagging behind >>>>>>> the version they should theoretically use. Dropping python 2.6 >>>>>>> support sounds very reasonable to me. >>>>>>> >>>>>>> On Tue, Jan 5, 2016 at 5:45 AM, Nicholas Chammas < >>>>>>> nicholas.cham...@gmail.com> wrote: >>>>>>> >>>>>>>> +1 >>>>>>>> >>>>>>>> Red Hat supports Python 2.6 on REHL 5 until 2020 >>>>>>>> <https://alexgaynor.net/2015/mar/30/red-hat-open-source-community/>, >>>>>>>> but otherwise yes, Python 2.6 is ancient history and the core Python >>>>>>>> developers stopped supporting it in 2013. REHL 5 is not a good enough >>>>>>>> reason to continue support for Python 2.6 IMO. >>>>>>>> >>>>>>>> We should aim to support Python 2.7 and Python 3.3+ (which I >>>>>>>> believe we currently do). >>>>>>>> >>>>>>>> Nick >>>>>>>> >>>>>>>> On Tue, Jan 5, 2016 at 8:01 AM Allen Zhang <allenzhang...@126.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> plus 1, >>>>>>>>> >>>>>>>>> we are currently using python 2.7.2 in production environment. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> 在 2016-01-05 18:11:45，"Meethu Mathew" <meethu.mat...@flytxt.com> >>>>>>>>> 写道： >>>>>>>>> >>>>>>>>> +1 >>>>>>>>> We use Python 2.7 >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Meethu Mathew >>>>>>>>> >>>>>>>>> On Tue, Jan 5, 2016 at 12:47 PM, Reynold Xin <r...@databricks.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Does anybody here care about us dropping support for Python 2.6 >>>>>>>>>> in Spark 2.0? >>>>>>>>>> >>>>>>>>>> Python 2.6 is ancient, and is pretty slow in many aspects (e.g. >>>>>>>>>> json parsing) when compared with Python 2.7. Some libraries that >>>>>>>>>> Spark >>>>>>>>>> depend on stopped supporting 2.6. We can still convince the library >>>>>>>>>> maintainers to support 2.6, but it will be extra work. I'm curious if >>>>>>>>>> anybody still uses Python 2.6 to run Spark. >>>>>>>>>> >>>>>>>>>> Thanks. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> >>>> >>> >>
