Github user onursatici commented on a diff in the pull request:
https://github.com/apache/spark/pull/21805#discussion_r204001527
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DatasetCacheSuite.scala ---
@@ -206,4 +206,19 @@ class DatasetCacheSuite extends QueryTest with
SharedSQLContext with TimeLimits
// first time use, load cache
checkDataset(df5, Row(10))
}
+
+ test("SPARK-24850 InMemoryRelation string representation does not
include cached plan") {
+ val dummyQueryExecution = spark.range(0, 1).toDF().queryExecution
+ val inMemoryRelation = InMemoryRelation(
+ true,
+ 1000,
+ StorageLevel.MEMORY_ONLY,
+ dummyQueryExecution.sparkPlan,
+ Some("test-relation"),
+ dummyQueryExecution.logical)
+
+
assert(!inMemoryRelation.simpleString.contains(dummyQueryExecution.sparkPlan.toString))
+ assert(inMemoryRelation.simpleString.contains(
+ "CachedRDDBuilder(true, 1000, StorageLevel(memory, deserialized, 1
replicas))"))
--- End diff --
@gatorsmile tried to keep this close to its default value, maybe we can do
something like `CachedRDDBuilder(useCompression = true, batchSize = 1000,
...)`? But that will break the consistency across logging case classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
