[ https://issues.apache.org/jira/browse/SPARK-9999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14706244#comment-14706244 ]
Reynold Xin commented on SPARK-9999: ------------------------------------ This needs to be designed first. I'm not sure if static code analysis is a great idea since they fail often. I'm open to ideas though. > RDD-like API on top of Catalyst/DataFrame > ----------------------------------------- > > Key: SPARK-9999 > URL: https://issues.apache.org/jira/browse/SPARK-9999 > Project: Spark > Issue Type: Story > Components: SQL > Reporter: Reynold Xin > > The RDD API is very flexible, and as a result harder to optimize its > execution in some cases. The DataFrame API, on the other hand, is much easier > to optimize, but lacks some of the nice perks of the RDD API (e.g. harder to > use UDFs, lack of strong types in Scala/Java). > As a Spark user, I want an API that sits somewhere in the middle of the > spectrum so I can write most of my applications with that API, and yet it can > be optimized well by Spark to achieve performance and stability. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org