+1
I think phasing out EOL of any feature or supported language is a better
strategy if possible than a quick drop. With enough admonition, it can
gradually be dropped in 3.x— of course, there are exceptions.
Cheers
Jules
Sent from my iPhone
Pardon the dumb thumb typos :)
> On Sep 15, 2018, at 10:49 AM, Reynold Xin <r...@databricks.com> wrote:
>
> we can also declare python 2 as deprecated and drop it in 3.x, not
> necessarily 3.0.
>
> --
> excuse the brevity and lower case due to wrist injury
>
>
>> On Sat, Sep 15, 2018 at 10:33 AM Erik Erlandson <eerla...@redhat.com> wrote:
>> I am probably splitting hairs to finely, but I was considering the
>> difference between improvements to the jvm-side (py4j and the scala/java
>> code) that would make it easier to write the python layer ("python-friendly
>> api"), and actual improvements to the python layers ("friendly python api").
>>
>> They're not mutually exclusive of course, and both worth working on. But
>> it's *possible* to improve either without the other.
>>
>> Stub files look like a great solution for type annotations, maybe even if
>> only python 3 is supported.
>>
>> I definitely agree that any decision to drop python 2 should not be taken
>> lightly. Anecdotally, I'm seeing an increase in python developers announcing
>> that they are dropping support for python 2 (and loving it). As people have
>> already pointed out, if we don't drop python 2 for spark 3.0, we're stuck
>> with it until 4.0, which would place spark in a possibly-awkward position of
>> supporting python 2 for some time after it goes EOL.
>>
>> Under the current release cadence, spark 3.0 will land some time in early
>> 2019, which at that point will be mere months until EOL for py2.
>>
>>> On Fri, Sep 14, 2018 at 5:01 PM, Holden Karau <hol...@pigscanfly.ca> wrote:
>>>
>>>
>>>> On Fri, Sep 14, 2018, 3:26 PM Erik Erlandson <eerla...@redhat.com> wrote:
>>>> To be clear, is this about "python-friendly API" or "friendly python API" ?
>>>
>>> Well what would you consider to be different between those two statements?
>>> I think it would be good to be a bit more explicit, but I don't think we
>>> should necessarily limit ourselves.
>>>>
>>>> On the python side, it might be nice to take advantage of static typing.
>>>> Requires python 3.6 but with python 2 going EOL, a spark-3.0 might be a
>>>> good opportunity to jump the python-3-only train.
>>>
>>> I think we can make types sort of work without ditching 2 (the types only
>>> would work in 3 but it would still function in 2). Ditching 2 entirely
>>> would be a big thing to consider, I honestly hadn't been considering that
>>> but it could be from just spending so much time maintaining a 2/3 code
>>> base. I'd suggest reaching out to to user@ before making that kind of
>>> change.
>>>>
>>>>> On Fri, Sep 14, 2018 at 12:15 PM, Holden Karau <hol...@pigscanfly.ca>
>>>>> wrote:
>>>>> Since we're talking about Spark 3.0 in the near future (and since some
>>>>> recent conversation on a proposed change reminded me) I wanted to open up
>>>>> the floor and see if folks have any ideas on how we could make a more
>>>>> Python friendly API for 3.0? I'm planning on taking some time to look at
>>>>> other systems in the solution space and see what we might want to learn
>>>>> from them but I'd love to hear what other folks are thinking too.
>>>>>
>>>>> --
>>>>> Twitter: https://twitter.com/holdenkarau
>>>>> Books (Learning Spark, High Performance Spark, etc.):
>>>>> https://amzn.to/2MaRAG9
>>>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>>
>>
