[jira] [Commented] (SPARK-2315) drop, dropRight and dropWhile which take RDD input and return RDD
[ https://issues.apache.org/jira/browse/SPARK-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567478#comment-14567478 ] Erik Erlandson commented on SPARK-2315: --- The 'drop' RDD methods have been made available on the 'silex' project (beginning with release 0.0.6): https://github.com/willb/silex Documentation: http://silex.freevariable.com/latest/api/#com.redhat.et.silex.rdd.drop.DropRDDFunctions drop, dropRight and dropWhile which take RDD input and return RDD - Key: SPARK-2315 URL: https://issues.apache.org/jira/browse/SPARK-2315 Project: Spark Issue Type: Sub-task Components: Spark Core Reporter: Erik Erlandson Assignee: Erik Erlandson Labels: features Last time I loaded in a text file, I found myself wanting to just skip the first element as it was a header. I wrote candidate methods drop, dropRight and dropWhile to satisfy this kind of need: val txt = sc.textFile(text_with_header.txt) val data = txt.drop(1) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-2315) drop, dropRight and dropWhile which take RDD input and return RDD
[ https://issues.apache.org/jira/browse/SPARK-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14089411#comment-14089411 ] Apache Spark commented on SPARK-2315: - User 'erikerlandson' has created a pull request for this issue: https://github.com/apache/spark/pull/1839 drop, dropRight and dropWhile which take RDD input and return RDD - Key: SPARK-2315 URL: https://issues.apache.org/jira/browse/SPARK-2315 Project: Spark Issue Type: New Feature Components: Spark Core Reporter: Erik Erlandson Assignee: Erik Erlandson Labels: features Last time I loaded in a text file, I found myself wanting to just skip the first element as it was a header. I wrote candidate methods drop, dropRight and dropWhile to satisfy this kind of need: val txt = sc.textFile(text_with_header.txt) val data = txt.drop(1) -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-2315) drop, dropRight and dropWhile which take RDD input and return RDD
[ https://issues.apache.org/jira/browse/SPARK-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14079300#comment-14079300 ] Erik Erlandson commented on SPARK-2315: --- Updated the PR with a proper lazy-transform implementation: http://erikerlandson.github.io/blog/2014/07/29/deferring-spark-actions-to-lazy-transforms-with-the-promise-rdd/ drop, dropRight and dropWhile which take RDD input and return RDD - Key: SPARK-2315 URL: https://issues.apache.org/jira/browse/SPARK-2315 Project: Spark Issue Type: New Feature Components: Spark Core Reporter: Erik Erlandson Labels: features Last time I loaded in a text file, I found myself wanting to just skip the first element as it was a header. I wrote candidate methods drop, dropRight and dropWhile to satisfy this kind of need: val txt = sc.textFile(text_with_header.txt) val data = txt.drop(1) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (SPARK-2315) drop, dropRight and dropWhile which take RDD input and return RDD
[ https://issues.apache.org/jira/browse/SPARK-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046725#comment-14046725 ] Erik Erlandson commented on SPARK-2315: --- PR: https://github.com/apache/spark/pull/1254 drop, dropRight and dropWhile which take RDD input and return RDD - Key: SPARK-2315 URL: https://issues.apache.org/jira/browse/SPARK-2315 Project: Spark Issue Type: New Feature Components: Spark Core Reporter: Erik Erlandson Labels: features Last time I loaded in a text file, I found myself wanting to just skip the first element as it was a header. I wrote candidate methods drop, dropRight and dropWhile to satisfy this kind of need: val txt = sc.textFile(text_with_header.txt) val data = txt.drop(1) -- This message was sent by Atlassian JIRA (v6.2#6252)
