Hi,
On Fri, Dec 19, 2014 at 6:53 PM, Ashic Mahtab as...@live.com wrote:
val doSomething(entry:SomeEntry, session:Session) : SomeOutput = {
val result = session.SomeOp(entry)
SomeOutput(entry.Key, result.SomeProp)
}
I could use a transformation for rdd.map, but in case of failures,
Hi,Say we have an operation that writes something to an external resource and
gets some output. For example:
val doSomething(entry:SomeEntry, session:Session) : SomeOutput = {val
result = session.SomeOp(entry)SomeOutput(entry.Key, result.SomeProp)}
I could use a transformation for
Hi,Say we have an operation that writes something to an external resource and
gets some output. For example:
val doSomething(entry:SomeEntry, session:Session) : SomeOutput = {val
result = session.SomeOp(entry)SomeOutput(entry.Key, result.SomeProp)}
I
could use a transformation for
To really be correct, I think you may have to use the foreach action
to persist your data, since this isn't idempotent, and then read it
again in a new RDD. You might get away with map as long as you can
ensure that your write process is idempotent.
On Fri, Dec 19, 2014 at 10:57 AM, ashic
Thanks Sean. That's kind of what I figured. Luckily, for my use case writes are
idempotent, so map works.
From: so...@cloudera.com
Date: Fri, 19 Dec 2014 11:06:51 +
Subject: Re: How to run an action and get output?
To: as...@live.com
CC: user@spark.apache.org
To really be correct, I
