[ https://issues.apache.org/jira/browse/SPARK-916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Wendell resolved SPARK-916. ----------------------------------- Resolution: Fixed Turned out [~marmbrus] did all of this and more in SparkSQL (which btw also works for nested types). So I'm gonna close this very old issue. > Better Support for Flat/Tabular RDD's > ------------------------------------- > > Key: SPARK-916 > URL: https://issues.apache.org/jira/browse/SPARK-916 > Project: Spark > Issue Type: Improvement > Reporter: Patrick Cogan > > Many people use Spark to run analysis on flat datasets, where the RDD is > composed records with a single set of non-nested fields. We could have better > support for this use case in a variety of areas. Two of which are: > 1. Allowing people to name individual fields and access them by name, rather > than using tuple indices (see Scalding[1]). This avoids the mess that is {x > => (x._3(), x._4())} > 2. Support columnar in-memory storage. > This is just an umbrella/brainstorming JIRA to see if other people have > thoughts about this. Curious to hear feedback. > [1] https://dev.twitter.com/blog/scalding -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org