Kazuaki Ishizaki created SPARK-15985:
----------------------------------------
Summary: Reduce runtime overhead of a program that reads an
primitive array in Dataset
Key: SPARK-15985
URL: https://issues.apache.org/jira/browse/SPARK-15985
Project: Spark
Issue Type: Improvement
Components: SQL
Reporter: Kazuaki Ishizaki
When a program read an array in Dataset, the code generator create some copy
operations. If an array is for primitive type, there are some opportunities for
optimizations in generated code to reduce runtime overhead.
{code}
val ds = Seq(Array(1.0, 2.0, 3.0), Array(4.0, 5.0, 6.0)).toDS()
ds.map(p => {
var s = 0.0
for (i <- 0 to 2) { s += p(i) }
s
}).show
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
