Re: Incorrect Scala version for Spark 2.4.x releases in the docs?

2020-09-17 Thread Jacek Laskowski
Thanks Sean for such a quick response! Let me propose a fix for the docs. Pozdrawiam, Jacek Laskowski https://about.me/JacekLaskowski "The Internals Of" Online Books Follow me on https://twitter.com/jaceklaskowski On Thu, Sep

Re: Incorrect Scala version for Spark 2.4.x releases in the docs?

2020-09-17 Thread Sean Owen
The pre-built binary distros should use 2.11 in 2.4.x. Artifacts for both Scala versions are available, yes. Yeah I think it should really say you can use 2.11 or 2.12. On Thu, Sep 17, 2020 at 9:12 AM Jacek Laskowski wrote: > > Hi, > > Just found this paragraph in >

Incorrect Scala version for Spark 2.4.x releases in the docs?

2020-09-17 Thread Jacek Laskowski
Hi, Just found this paragraph in http://spark.apache.org/docs/2.4.6/index.html#downloading: "Spark runs on Java 8, Python 2.7+/3.4+ and R 3.1+. For the Scala API, Spark 2.4.6 uses Scala 2.12. You will need to use a compatible Scala version (2.12.x)." That seems to contradict the version of

unsubscribe

2020-09-17 Thread Mark Daniels
>

Re: Spark-Locality: Hinting Spark location of the executor does not take effect.

2020-09-17 Thread nakhanha
Even though i provide hint of wn4's ip address Spark schedules on wn1 2020-09-17 01:21:50,038 DEBUG [dag-scheduler-event-loop] scheduler.TaskSetManager: Valid locality levels for TaskSet 11.0: NODE_LOCAL, RACK_LOCAL, ANY 2020-09-17 01:21:50,039 DEBUG [dispatcher-event-loop-0]

Re: Spark-Locality: Hinting Spark location of the executor does not take effect.

2020-09-17 Thread nakhanha
Even though i provide hint of wn4's ip address Spark schedules on wn1 2020-09-17 01:21:50,038 DEBUG [dag-scheduler-event-loop] scheduler.TaskSetManager: Valid locality levels for TaskSet 11.0: NODE_LOCAL, RACK_LOCAL, ANY 2020-09-17 01:21:50,039 DEBUG [dispatcher-event-loop-0]

Re: Spark-Locality: Hinting Spark location of the executor does not take effect.

2020-09-17 Thread nakhanha
Even though i provide hint of wn4's ip addressSpark schedules on wn12020-09-17 01:21:50,038 DEBUG [dag-scheduler-event-loop] scheduler.TaskSetManager: Valid locality levels for TaskSet 11.0: NODE_LOCAL, RACK_LOCAL, ANY2020-09-17 01:21:50,039 DEBUG [dispatcher-event-loop-0] cluster.YarnScheduler: