Re: [VOTE] Spark 2.3.1 (RC1)

+1, checked new py4j related changes. Marcelo Vanzin 于2018年5月17日周四 上午5:41写道： > This is actually in 2.3, jira is just missing the version. > > https://github.com/apache/spark/pull/20765 > > On Wed, May 16, 2018 at 2:14 PM, kant kodali wrote: > > I am not

Re: [VOTE] Spark 2.3.1 (RC1)

This is actually in 2.3, jira is just missing the version. https://github.com/apache/spark/pull/20765 On Wed, May 16, 2018 at 2:14 PM, kant kodali wrote: > I am not sure how SPARK-23406 is a new feature. since streaming joins are > already part of SPARK 2.3.0. The self joins

Re: [VOTE] Spark 2.3.1 (RC1)

I am not sure how SPARK-23406 is a new feature. since streaming joins are already part of SPARK 2.3.0. The self joins didn't work because of a bug and it is fixed but I can understand if it touches some other code paths. On Wed, May 16, 2018 at

[DISCUSS] PySpark Window UDF

Hi All, I have been looking into leverage the Arrow and Pandas UDF work we have done so far for Window UDF in PySpark. I have done some investigation and believe there is a way to do PySpark window UDF efficiently. The basic idea is instead of passing each window to Python separately, we can

Re: [VOTE] Spark 2.3.1 (RC1)

RC1 pyspark install works into a venv for what its worth. On Wed, May 16, 2018 at 5:26 AM, Sean Owen wrote: > +1 the release (otherwise) looks fine to me. Sigs and licenses are OK. > Builds and passes tests on Debian with -Pyarn -Phadoop-2.7 -Phive > -Phive-thriftserver

Re: [VOTE] Spark 2.3.1 (RC1)

+1 the release (otherwise) looks fine to me. Sigs and licenses are OK. Builds and passes tests on Debian with -Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pkubernetes On Tue, May 15, 2018 at 4:00 PM Marcelo Vanzin wrote: > Please vote on releasing the following candidate

Re: [VOTE] Spark 2.3.1 (RC1)

I'd be against having a new feature in a minor maintenance release. I think such a release should contain only bugfixes. 2018-05-16 12:11 GMT+02:00 kant kodali : > Can this https://issues.apache.org/jira/browse/SPARK-23406 be part of > 2.3.1? > > On Tue, May 15, 2018 at 2:07

Re: [VOTE] Spark 2.3.1 (RC1)

Can this https://issues.apache.org/jira/browse/SPARK-23406 be part of 2.3.1? On Tue, May 15, 2018 at 2:07 PM, Marcelo Vanzin wrote: > Bummer. People should still feel welcome to test the existing RC so we > can rule out other issues. > > On Tue, May 15, 2018 at 2:04 PM,