[jira] [Commented] (SPARK-1241) Support sliding in RDD

2014-09-25 Thread Frens Jan Rumph (JIRA)

[ 
https://issues.apache.org/jira/browse/SPARK-1241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14148281#comment-14148281
 ] 

Frens Jan Rumph commented on SPARK-1241:


Hi,

I'm investigating use of Spark for matching patterns in symbolically 
represented time series where the sliding functionality as available from scala 
iterators would make life a lot easier.

From the ticket I'd say this functionality is implemented (status resolved, 
fix version 1.0.0, ...), but I can't find it in the docs and the PR indicates 
that this functionality hasn't made it into Spark (just yet ...). Is this 
functionality available?

Cheers,
Frens

Background: I want to compare strings of segments to other (larger) strings of 
segments. As segment strings may be split up over partitions, the more straight 
forward aproaches I could come up with don't work.

 Support sliding in RDD
 --

 Key: SPARK-1241
 URL: https://issues.apache.org/jira/browse/SPARK-1241
 Project: Spark
  Issue Type: New Feature
  Components: Spark Core
Reporter: Xiangrui Meng
Assignee: Xiangrui Meng
 Fix For: 1.0.0


 Sliding is useful for operations like creating n-grams, calculating total 
 variation, numerical integration, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Commented] (SPARK-1241) Support sliding in RDD

2014-09-25 Thread Xiangrui Meng (JIRA)

[ 
https://issues.apache.org/jira/browse/SPARK-1241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14148549#comment-14148549
 ] 

Xiangrui Meng commented on SPARK-1241:
--

This is implemented MLlib: 
https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/mllib/rdd/SlidingRDD.scala
 . You can check the discussion about where to put it here: 
https://github.com/apache/spark/pull/136 .

 Support sliding in RDD
 --

 Key: SPARK-1241
 URL: https://issues.apache.org/jira/browse/SPARK-1241
 Project: Spark
  Issue Type: New Feature
  Components: Spark Core
Reporter: Xiangrui Meng
Assignee: Xiangrui Meng
 Fix For: 1.0.0


 Sliding is useful for operations like creating n-grams, calculating total 
 variation, numerical integration, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org