[ https://issues.apache.org/jira/browse/SPARK-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xiao Li updated SPARK-12374: ---------------------------- Description: Creating an actual logical/physical operator for range for matching the performance of RDD Range APIs. Compared with the old Range API, the new version is 3 times faster than the old version. {code} scala> val startTime = System.currentTimeMillis; sqlContext.oldRange(0, 1000000000, 1, 15).count(); val endTime = System.currentTimeMillis; val start = new Timestamp(startTime); val end = new Timestamp(endTime); val elapsed = (endTime - startTime)/ 1000.0 startTime: Long = 1450416394240 endTime: Long = 1450416421199 start: java.sql.Timestamp = 2015-12-17 21:26:34.24 end: java.sql.Timestamp = 2015-12-17 21:27:01.199 elapsed: Double = 26.959 {code} {code} scala> val startTime = System.currentTimeMillis; sqlContext.range(0, 1000000000, 1, 15).count(); val endTime = System.currentTimeMillis; val start = new Timestamp(startTime); val end = new Timestamp(endTime); val elapsed = (endTime - startTime)/ 1000.0 startTime: Long = 1450416360107 endTime: Long = 1450416368590 start: java.sql.Timestamp = 2015-12-17 21:26:00.107 end: java.sql.Timestamp = 2015-12-17 21:26:08.59 elapsed: Double = 8.483 {code} was:Creating an actual logical/physical operator for range for matching the performance of RDD Range APIs. > Improve performance of Range APIs via adding logical/physical operators > ----------------------------------------------------------------------- > > Key: SPARK-12374 > URL: https://issues.apache.org/jira/browse/SPARK-12374 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 1.6.0 > Reporter: Xiao Li > Assignee: Apache Spark > Priority: Critical > > Creating an actual logical/physical operator for range for matching the > performance of RDD Range APIs. > Compared with the old Range API, the new version is 3 times faster than the > old version. > {code} > scala> val startTime = System.currentTimeMillis; sqlContext.oldRange(0, > 1000000000, 1, 15).count(); val endTime = System.currentTimeMillis; val start > = new Timestamp(startTime); val end = new Timestamp(endTime); val elapsed = > (endTime - startTime)/ 1000.0 > startTime: Long = 1450416394240 > > endTime: Long = 1450416421199 > start: java.sql.Timestamp = 2015-12-17 21:26:34.24 > end: java.sql.Timestamp = 2015-12-17 21:27:01.199 > elapsed: Double = 26.959 > {code} > {code} > scala> val startTime = System.currentTimeMillis; sqlContext.range(0, > 1000000000, 1, 15).count(); val endTime = System.currentTimeMillis; val start > = new Timestamp(startTime); val end = new Timestamp(endTime); val elapsed = > (endTime - startTime)/ 1000.0 > startTime: Long = 1450416360107 > > endTime: Long = 1450416368590 > start: java.sql.Timestamp = 2015-12-17 21:26:00.107 > end: java.sql.Timestamp = 2015-12-17 21:26:08.59 > elapsed: Double = 8.483 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org