[
https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14679849#comment-14679849
]
Jerry Z edited comment on SPARK-9744 at 8/10/15 9:34 AM:
---------------------------------------------------------
With regards to cogroup, I phrased that badly, but meant to say iterable as in:
<Iterable<V>,Iterable<W1>,Iterable<W2>>>.
Anyway, how do I do a PR? Not seeing that as one of the options when you click
create and get that dialog.
Also... what started this all from the beginning is I'd get exceptions any time
I'd try to use lag and lead in data fames stating lag wasn't a valid name. I
have a feeling the spark environment wasn't setup properly as things got worse
when we moved to 1.4.1 from 1.3 (which, yes I'm aware didn't support those
commands) and I couldn't even run the provided example using a bean class any
more and had to use a json file. On a side note, mind testing lag and lead on a
data frame so I know it's just a setup issue on my end and not a bug?
was (Author: mrunowen):
With regards to cogroup, I phrased that badly, but meant to say iterable as in:
<Iterable<V>,Iterable<W1>,Iterable<W2>>>.
Anyway, how do I do a PR? Not seeing that as one of the options when you click
create and get that dialog.
Also... what started this all from the beginning is I'd get exceptions any time
I'd try to use lag and lead in data fames. I have a feeling the spark
environment wasn't setup properly as things got worse when we moved to 1.4.1
from 1.3 (which, yes I'm aware didn't support those commands) and I couldn't
even run the provided example using a bean class any more. On a side note, mind
testing lag and lead on a data frame so I know it's just a setup issue on my
end and not a bug?
> Add RDD method to map with lag and lead
> ---------------------------------------
>
> Key: SPARK-9744
> URL: https://issues.apache.org/jira/browse/SPARK-9744
> Project: Spark
> Issue Type: Wish
> Components: Spark Core
> Reporter: Jerry Z
> Priority: Minor
>
> To avoid zipping with index and doing numerous mapping and joins, having a
> single method call to map with an additional two parameters (1: list of
> offsets [(-) for lag, 0 for current and (+) for lead])) and (2:default
> value). The other difference to the map function takes an argument of List<T>
> and not just T.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
