wypoon commented on a change in pull request #296:
URL: https://github.com/apache/incubator-livy/pull/296#discussion_r451965507
##########
File path:
thriftserver/session/src/main/java/org/apache/livy/thriftserver/session/ResultSet.java
##########
@@ -88,7 +88,7 @@ private String toHiveString(Object value, boolean
quoteStrings) {
} else if (quoteStrings && value instanceof String) {
return "\"" + value + "\"";
} else if (value instanceof BigDecimal) {
- return ((BigDecimal) value).stripTrailingZeros().toString();
+ return ((BigDecimal) value).toString();
Review comment:
My understanding of the implementation of the Livy Thrift server (from
ThriftSessionState.java) is that each Thrift session creates a new SparkSession
(with the same underlying SparkContext). So when you connect to the Thrift
server using the session id of an existing interactive scala session, the
Thrift session is using a new SparkSession, not the SparkSession in the
existing scala session.
```
scala> import org.apache.spark.sql.types.DecimalType
import org.apache.spark.sql.types.DecimalType
scala> val df = spark.range(10).select($"id".cast(DecimalType(10,-1)))
df: org.apache.spark.sql.DataFrame = [id: decimal(10,-1)]
scala> df.show()
+----+
| id|
+----+
|0E+1|
|0E+1|
|0E+1|
|0E+1|
|0E+1|
|1E+1|
|1E+1|
|1E+1|
|1E+1|
|1E+1|
+----+
scala> df.createTempView("aa")
scala> val spark2 = spark.newSession()
spark2: org.apache.spark.sql.SparkSession =
org.apache.spark.sql.SparkSession@7abdfd36
scala> spark2.sql("select * from aa").show()
Hive Session ID = c73613e6-cb1e-4156-a6dc-31165d02dec0
org.apache.spark.sql.AnalysisException: Table or view not found: aa; line 1
pos 14
at
org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:47)
...
Caused by: org.apache.spark.sql.catalyst.analysis.NoSuchTableException:
Table or view 'aa' not found in database 'default';
at
org.apache.spark.sql.hive.client.HiveClient$$anonfun$getTable$1.apply(HiveClient.scala:81)
...
```
I don't see how you can query the temp view from another SparkSession.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
