[ https://issues.apache.org/jira/browse/SPARK-30781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17036683#comment-17036683 ]
Hyukjin Kwon commented on SPARK-30781: -------------------------------------- Currently we don't have the plan to add this given that we now have many higher order functions to workaround this problem, and the dev overhead by adding a new type - we should consider how it works in Scala, Java, R, Python and SQL. > Missing SortedMap type in pyspark > --------------------------------- > > Key: SPARK-30781 > URL: https://issues.apache.org/jira/browse/SPARK-30781 > Project: Spark > Issue Type: Improvement > Components: PySpark > Affects Versions: 2.4.5 > Reporter: Mateusz User > Priority: Major > Labels: features > > Currently there is only MapType in pyspark API, which does not keep order of > key-value map. > > *SortedMapType* would fill this gap: a map with sorted key-value pairs (like > TreeMap in Java). > > For example: > *SortedMapType* would be very useful when user wants to persist row from > DataFrame into Mongo-db. > row which consist a column with values of an ordered map type: > col : [1 -> 22, 2 -> 16, 3 -> 25] > will be persisted as following json: > { "1": 22, "2": 16, "3": 25 } > > instead of MapType which currently results in: > { "2": 16, "1": 22, "3": 25 } > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org