Re: Should python-2 be supported in Spark 3.0?

> > I don't have a good sense of the overhead of continuing to support > Python 2; is it large enough to consider dropping it in Spark 3.0? > > from the build/test side, it will actually be pretty easy to continue support for python2.7 for spark 2.x as the feature sets won't be expanding. that

Re: Should python-2 be supported in Spark 3.0?

Here’s the tweet from the horse’s mouth: https://twitter.com/gvanrossum/status/1133496146700058626?s=21 Cheers Jules — Sent from my iPhone Pardon the dumb thumb typos :) > On May 29, 2019, at 10:12 PM, Sean Owen wrote: > > Deprecated -- certainly and sooner than later. > I don't have a

Re: Should python-2 be supported in Spark 3.0?

Deprecated -- certainly and sooner than later. I don't have a good sense of the overhead of continuing to support Python 2; is it large enough to consider dropping it in Spark 3.0? On Wed, May 29, 2019 at 11:47 PM Xiangrui Meng wrote: > > Hi all, > > I want to revive this old thread since no

Re: Should python-2 be supported in Spark 3.0?

Hi all, I want to revive this old thread since no action was taken so far. If we plan to mark Python 2 as deprecated in Spark 3.0, we should do it as early as possible and let users know ahead. PySpark depends on Python, numpy, pandas, and pyarrow, all of which are sunsetting Python 2 support by

Re: Interesting implications of supporting Scala 2.13

I think the particular issue here isn't resolved by scala-collection-compat: TraversableOnce goes away. However I hear that maybe Scala 2.13 retains it as a deprecated alias, which might help. On Wed, May 29, 2019 at 4:59 PM antonkulaga wrote: > > There is

Re: [VOTE][SPARK-27396] SPIP: Public APIs for extended Columnar Processing Support

Add a +1 from me as well. Just managed to finish going over it. Thanks Bobby for leading this effort ! Regards, Mridul On Wed, May 29, 2019 at 2:51 PM Tom Graves wrote: > > Ok, I'm going to call this vote and send the result email. We had 9 +1's (4 > binding) and 1 +0 and no -1's. > > Tom > >

DataSourceV2 sync notes - 15 May 2019

Sorry these notes are so late, I didn’t get to the write up until now. As usual, if anyone has corrections or comments, please reply. *Attendees*: John Zhuge Ryan Blue Andrew Long Wenchen Fan Gengliang Wang Russell Spitzer Yuanjian Li Yifei Huang Matt Cheah Amardeep Singh Dhilon Zhilmil Dhion

Re: [RESULT][VOTE] SPIP: Public APIs for extended Columnar Processing Support

Thanks Tom. I finally had time to look at the updated SPIP 10 mins ago. I support the high level idea and +1 on the SPIP. That said, I think the proposed API is too complicated and invasive change to the existing internals. A much simpler API would be to expose a columnar batch iterator

Re: Interesting implications of supporting Scala 2.13

There is https://github.com/scala/scala-collection-compat to enable 2.13 collections in Scala 2.12, so probably you can use it to avoid having separate source trees for 2.12 and 2.13 -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

Re: [VOTE][SPARK-27396] SPIP: Public APIs for extended Columnar Processing Support

Ok, I'm going to call this vote and send the result email. We had 9 +1's (4 binding) and 1 +0 and no -1's. Tom On Monday, May 27, 2019, 3:25:14 PM CDT, Felix Cheung wrote: #yiv8731614492 html {background-color:transparent;}#yiv8731614492 body

[RESULT][VOTE] SPIP: Public APIs for extended Columnar Processing Support

Hi all, The vote passed with 9 +1's (4 binding) and 1 +0 and no -1's. +1s (* = binding) : Bobby Evans* Thomas Graves* DB Tsai* Felix Cheung* Bryan Cutler Kazuaki Ishizaki Tyson Condie Dongjoon Hyun Jason Lowe +0s: Xiangrui Meng Thanks, Tom Graves

subscribe