[ https://issues.apache.org/jira/browse/SPARK-560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nicholas Chammas updated SPARK-560: ----------------------------------- Component/s: Spark Core > Specialize RDDs / iterators > --------------------------- > > Key: SPARK-560 > URL: https://issues.apache.org/jira/browse/SPARK-560 > Project: Spark > Issue Type: Bug > Components: Spark Core > Reporter: Matei Zaharia > > When you're working on in-memory data, the overhead of boxing / unboxing > starts to matter, and it looks like specializing would give a 2-4x speedup. > We can't just throw in @specialized though because Scala's Iterator is not > specialized. We probably need to make our own and also ensure that the right > methods get called remotely when you have a chain of RDDs (i.e. it doesn't > "lose" its specialization). -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org