[jira] [Created] (LIVY-505) sparkR.session failed with "invalid jobj 1" error in Spark 2.3

[shanyu zhao (JIRA)]

shanyu zhao created LIVY-505:
--------------------------------

             Summary: sparkR.session failed with "invalid jobj 1" error in 
Spark 2.3
                 Key: LIVY-505
                 URL: https://issues.apache.org/jira/browse/LIVY-505
             Project: Livy
          Issue Type: Bug
          Components: Interpreter
    Affects Versions: 0.5.0, 0.5.1
            Reporter: shanyu zhao


In Spark 2.3 cluster, use Zeppelin with livy2 interpreter, and type:
{code:java}
%sparkr
sparkR.session(){code}
You will see error:

[1] "Error in writeJobj(con, object): invalid jobj 1"

In a successful case with older livy and spark versions, we see something like 
this:

Java ref type org.apache.spark.sql.SparkSession id 1

This indicates isValidJobj() function in Spark code returned false for 
SparkSession obj. This is isValidJobj() function in Spark 2.3 code FYI:
{code:java}
isValidJobj <- function(jobj) {
  if (exists(".scStartTime", envir = .sparkREnv)) {
    jobj$appId == get(".scStartTime", envir = .sparkREnv)
  } else {
    FALSE
  }
}{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)