[ https://issues.apache.org/jira/browse/SPARK-9733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Reynold Xin resolved SPARK-9733. -------------------------------- Resolution: Fixed Fix Version/s: 1.5.0 > Improve explain message for data source scan node > ------------------------------------------------- > > Key: SPARK-9733 > URL: https://issues.apache.org/jira/browse/SPARK-9733 > Project: Spark > Issue Type: Improvement > Components: SQL > Reporter: Reynold Xin > Assignee: Reynold Xin > Fix For: 1.5.0 > > > All data sources show up as "PhysicalRDD" in physical plan explain. It'd be > better if we can show the name of the data source. > Existing: > {code} > == Physical Plan == > NewAggregate with UnsafeHybridAggregationIterator ArrayBuffer(date#0, cat#1) > ArrayBuffer((sum(CAST((CAST(count#2, IntegerType) + 1), > LongType))2,mode=Final,isDistinct=false)) > Exchange hashpartitioning(date#0,cat#1) > NewAggregate with UnsafeHybridAggregationIterator ArrayBuffer(date#0, > cat#1) ArrayBuffer((sum(CAST((CAST(count#2, IntegerType) + 1), > LongType))2,mode=Partial,isDistinct=false)) > PhysicalRDD [date#0,cat#1,count#2], MapPartitionsRDD[3] at > {code} > Better version: > {code} > == Physical Plan == > TungstenAggregate(key=[date#0,cat#1], value=[(sum(CAST((CAST(count#2, > IntegerType) + 1), LongType)),mode=Final,isDistinct=false)] > Exchange hashpartitioning(date#0,cat#1) > TungstenAggregate(key=[date#0,cat#1], value=[(sum(CAST((CAST(count#2, > IntegerType) + 1), LongType)),mode=Partial,isDistinct=false)] > ConvertToUnsafe > Scan > ParquetRelation[file:/scratch/rxin/spark/sales4][date#0,cat#1,count#2] > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org