[Spark-Core]port opened by the SparkDriver is vulnerable to flooding attacks

SparkSubmit will open the port to communicate with the APP Master and executors. This port is not closing the IDLE connections,so it is vulnerable for DOS attack,I did telnet IP port and this connection is not closed. In order to fix this I tried to Handle in the *userEventTriggered * of

Re: [VOTE] Spark 2.3.0 (RC4)

+1 On Tue, Feb 20, 2018 at 2:14 PM, Xingbo Jiang wrote: > +1 > > > Wenchen Fan 于2018年2月20日 周二下午1:09写道： > >> +1 >> >> On Tue, Feb 20, 2018 at 12:53 PM, Reynold Xin >> wrote: >> >>> +1 >>> >>> On Feb 20, 2018, 5:51 PM +1300,

Re: [VOTE] Spark 2.3.0 (RC4)

+1 Wenchen Fan 于2018年2月20日 周二下午1:09写道： > +1 > > On Tue, Feb 20, 2018 at 12:53 PM, Reynold Xin wrote: > >> +1 >> >> On Feb 20, 2018, 5:51 PM +1300, Sameer Agarwal , >> wrote: >> >> this file shouldn't be included? >>>

Re: [VOTE] Spark 2.3.0 (RC4)

+1 On Tue, Feb 20, 2018 at 12:53 PM, Reynold Xin wrote: > +1 > > On Feb 20, 2018, 5:51 PM +1300, Sameer Agarwal , > wrote: > > this file shouldn't be included? https://dist.apache.org/repos/ >> dist/dev/spark/v2.3.0-rc4-bin/spark-parent_2.11.iml >> >

Re: [VOTE] Spark 2.3.0 (RC4)

+1 On Feb 20, 2018, 5:51 PM +1300, Sameer Agarwal , wrote: > > > this file shouldn't be included? > > > https://dist.apache.org/repos/dist/dev/spark/v2.3.0-rc4-bin/spark-parent_2.11.iml > > > > I've now deleted this file > > > > > From: Sameer Agarwal

Re: [VOTE] Spark 2.3.0 (RC4)

> > this file shouldn't be included? https://dist.apache.org/repos/ > dist/dev/spark/v2.3.0-rc4-bin/spark-parent_2.11.iml > I've now deleted this file *From:* Sameer Agarwal > *Sent:* Saturday, February 17, 2018 1:43:39 PM > *To:* Sameer Agarwal > *Cc:* dev > *Subject:*

Re: [VOTE] Spark 2.3.0 (RC4)

In addition to Hyukjin's `github.io` result, `jekyll` also forwards the search result links correctly. SKIP_SCALADOC=1 SKIP_PYTHONDOC=1 SKIP_RDOC=1 jekyll serve --watch And, connect `http://127.0.0.1:4000`. This will be the same in Apache Spark websites. Bests, Dongjoon. On Mon, Feb 19,

Re: [VOTE] Spark 2.3.0 (RC4)

+1 Regards, Vaquar khan On Mon, Feb 19, 2018 at 10:29 PM, Xiao Li wrote: > +1. > > So far, no function/performance regression in Spark SQL, Core and PySpark. > > Thanks! > > Xiao > > 2018-02-19 19:47 GMT-08:00 Hyukjin Kwon : > >> Ah, I see. For 1), I

Re: [VOTE] Spark 2.3.0 (RC4)

+1. So far, no function/performance regression in Spark SQL, Core and PySpark. Thanks! Xiao 2018-02-19 19:47 GMT-08:00 Hyukjin Kwon : > Ah, I see. For 1), I overlooked Felix's input here. I couldn't foresee > this when I added this documentation because it worked in my

Re: [VOTE] Spark 2.3.0 (RC4)

Ah, I see. For 1), I overlooked Felix's input here. I couldn't foresee this when I added this documentation because it worked in my simple demo: https://spark-test.github.io/sparksqldoc/search.html?q=approx https://spark-test.github.io/sparksqldoc/#approx_percentile Will try to investigate this

Re: [VOTE] Spark 2.3.0 (RC4)

For (1) I think it has something to do with https://dist.apache.org/repos/dist/dev/spark/v2.3.0-rc4-docs/_site/api/sql/ not automatically going to https://dist.apache.org/repos/dist/dev/spark/v2.3.0-rc4-docs/_site/api/sql/index.html -- So if you see the link to approx_percentile the link we

Re: [VOTE] Spark 2.3.0 (RC4)

Ah sorry I realize my wordings were unclear (not enough zzz or coffee) So to clarify, 1) when searching for a word in the Sql function doc, it does return that search result page correctly, however, none of the link in result opens to the actual doc page, so to take the search I included as an

Re: [VOTE] Spark 2.3.0 (RC4)

FWIW The search result link works for me Shivaram On Mon, Feb 19, 2018 at 6:21 PM, Felix Cheung wrote: > These are two separate things: > > Does the search result links work for you? > > The second is the dist location we are voting on has a .iml file. > >

Re: [VOTE] Spark 2.3.0 (RC4)

These are two separate things: Does the search result links work for you? The second is the dist location we are voting on has a .iml file. _ From: Sean Owen Sent: Tuesday, February 20, 2018 2:19 AM Subject: Re: [VOTE] Spark 2.3.0 (RC4) To: Felix

Re: [VOTE] Spark 2.3.0 (RC4)

Maybe I misunderstand, but I don't see any .iml file in the 4 results on that page? it looks reasonable. On Mon, Feb 19, 2018 at 8:02 PM Felix Cheung wrote: > Any idea with sql func docs search result returning broken links as below? > > *From:* Felix Cheung

Re: [VOTE] Spark 2.3.0 (RC4)

Any idea with sql func docs search result returning broken links as below? From: Felix Cheung Sent: Sunday, February 18, 2018 10:05:22 AM To: Sameer Agarwal; Sameer Agarwal Cc: dev Subject: Re: [VOTE] Spark 2.3.0 (RC4) Quick questions:

Re: [VOTE] Spark 2.3.0 (RC4)

+1. I tested RC4 on CentOS 7.4 / OpenJDK 1.8.0_161 with `-Pyarn -Phadoop-2.7 -Pkinesis-asl -Phive -Phive-thriftserver -Psparkr`. Bests, Dongjoon. On Sun, Feb 18, 2018 at 3:22 PM, Denny Lee wrote: > +1 (non-binding) > > Built and tested on macOS and Ubuntu. > > > On