Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/21495#discussion_r193935371
--- Diff:
repl/scala-2.11/src/main/scala/org/apache/spark/repl/SparkILoopInterpreter.scala
---
@@ -21,8 +21,22 @@ import scala.collection.mutable
import scala.tools.nsc.Settings
import scala.tools.nsc.interpreter._
-class SparkILoopInterpreter(settings: Settings, out: JPrintWriter) extends
IMain(settings, out) {
- self =>
+class SparkILoopInterpreter(settings: Settings, out: JPrintWriter,
initializeSpark: () => Unit)
+ extends IMain(settings, out) { self =>
+
+ /**
+ * We override `initializeSynchronous` to initialize Spark *after*
`intp` is properly initialized
+ * and *before* the REPL sees any files in the private `loadInitFiles`
functions, so that
+ * the Spark context is visible in those files.
+ *
+ * This is a bit of a hack, but there isn't another hook available to us
at this point.
+ *
+ * See the discussion in Scala community
https://github.com/scala/bug/issues/10913 for detail.
+ */
+ override def initializeSynchronous(): Unit = {
+ super.initializeSynchronous()
+ initializeSpark()
--- End diff --
Yep. Of course, it's a clean clone and build of this PR. Accoring to the
error message and the followings. We need `jline-2.14.3.jar` because `scala`
uses the API of higher version of `jline`. Could you confirm this, @som-snytt ?
```
$ java -version
openjdk version "1.8.0_171"
OpenJDK Runtime Environment (build 1.8.0_171-8u171-b11-0ubuntu0.18.04.1-b11)
OpenJDK 64-Bit Server VM (build 25.171-b11, mixed mode)
$ javap -cp jline-2.12.1.jar
jline.console.completer.CandidateListCompletionHandler
Compiled from "CandidateListCompletionHandler.java"
public class jline.console.completer.CandidateListCompletionHandler
implements jline.console.completer.CompletionHandler {
public jline.console.completer.CandidateListCompletionHandler();
public boolean complete(jline.console.ConsoleReader,
java.util.List<java.lang.CharSequence>, int) throws java.io.IOException;
public static void setBuffer(jline.console.ConsoleReader,
java.lang.CharSequence, int) throws java.io.IOException;
public static void printCandidates(jline.console.ConsoleReader,
java.util.Collection<java.lang.CharSequence>) throws java.io.IOException;
}
$ javap -cp jline-2.14.3.jar
jline.console.completer.CandidateListCompletionHandler
Compiled from "CandidateListCompletionHandler.java"
public class jline.console.completer.CandidateListCompletionHandler
implements jline.console.completer.CompletionHandler {
public jline.console.completer.CandidateListCompletionHandler();
public boolean getPrintSpaceAfterFullCompletion();
public void setPrintSpaceAfterFullCompletion(boolean);
public boolean isStripAnsi();
public void setStripAnsi(boolean);
public boolean complete(jline.console.ConsoleReader,
java.util.List<java.lang.CharSequence>, int) throws java.io.IOException;
public static void setBuffer(jline.console.ConsoleReader,
java.lang.CharSequence, int) throws java.io.IOException;
public static void printCandidates(jline.console.ConsoleReader,
java.util.Collection<java.lang.CharSequence>) throws java.io.IOException;
}
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
