Boris Alexeev created SPARK-10806:
-------------------------------------
Summary: Following val redefinition, sometimes the old value is
still visible
Key: SPARK-10806
URL: https://issues.apache.org/jira/browse/SPARK-10806
Project: Spark
Issue Type: Bug
Components: Spark Shell
Affects Versions: 1.5.0
Environment: on EC2, uname -a gives:
Linux ip-172-31-19-173 3.13.0-61-generic #100-Ubuntu SMP Wed Jul 29 11:21:34
UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
spark-shell itself prints:
Using Scala version 2.11.7 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_60)
Reporter: Boris Alexeev
I am seeing odd behavior when I redefine a val in the REPL of the spark-shell
of 1.5.0. Here is my minimal test case:
val a = 1
def id(a:Int) = {a}
val a = 2
a
id(a)
Specifically, if I run "~/spark/bin/spark-shell --master local" and
enter each of these five lines one-by-one (not in :paste mode, because
of the redefinition), I get the output at the end of my message below.
Expected behavior: both of the last two expressions evaluate to 2.
Observed behavior: "a" returns 2, but "id(a)" still returns 1.
Reproducible: always (for me) on Spark 1.5.0 but not 1.4.1.
I believe that the example is sensitive to the variable name use! I
can also reproduce the problem with more complicated "dependencies" in
the variable name use, e.g. if I define id using b, but val b was
defined using a:
val a = 1
val b = a // this line is necessary for the problem!
def id(b:Int) = {b}
val a = 2
a
id(a)
I cannot reproduce this behavior in the Scala REPL directly for the
few versions and configurations that I've tried, but I may have not
been able to find the appropriate version (I have tried the obvious candidate).
That is, my Scala interactions have all had the expected behavior: they
returned 2 for both of the last two expressions. Similarly, I cannot reproduce
this in Spark 1.4.1.
I believe this is a bug, but is this the desired behavior for some
reason? Why does it happen in either case?
===
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 1.5.0
/_/
Using Scala version 2.11.7 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_60)
Type in expressions to have them evaluated.
Type :help for more information.
scala> val a = 1
a: Int = 1
scala> def id(a:Int) = {a}
id: (a: Int)Int
scala> val a = 2
a: Int = 2
scala> a
res0: Int = 2
scala> id(a)
res1: Int = 1
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
