[ https://issues.apache.org/jira/browse/SPARK-16892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646443#comment-15646443 ]
Jayadevan M commented on SPARK-16892: ------------------------------------- I hope you can use flatMap for this scenario. For example var array=Array(Array(1, 2, 3), Array(4, 5), Array(-1, -2, 0)); var rdd = sc.parallelize(array); rdd.flatMap(x=>x).collect(); > flatten function to get flat array (or map) column from array of array (or > array of map) column > ----------------------------------------------------------------------------------------------- > > Key: SPARK-16892 > URL: https://issues.apache.org/jira/browse/SPARK-16892 > Project: Spark > Issue Type: New Feature > Components: SQL > Reporter: Kapil Singh > > flatten(input) > Converts input of array of array type into flat array type by inserting > elements of all element arrays into single array. Example: > input: [[1, 2, 3], [4, 5], [-1, -2, 0]] > output: [1, 2, 3, 4, 5, -1, -2, 0] > Converts input of array of map type into flat map type by inserting key-value > pairs of all element maps into single map. Example: > input: [(1 -> "one", 2 -> "two"), (0 -> "zero"), (4 -> "four")] > output: (1 -> "one", 2 -> "two", 0 -> "zero", 4 -> "four") -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org