Hi Ami - I suspect that your code is completing because you have nothing to actually force resolution of your job. Spark executes lazily, so for example, if you have a bunch of maps in sequence but nothing else, Spark will not actually execute anything.
Try adding an RDD.count() on the last RDD that you generate with spark to ensure that something is forcing execution at the end. On 10/28/14, 5:32 PM, "akhandeshi" <ami.khande...@gmail.com> wrote: >I recently starting seeing this new problem where spark-submt is >terminated >by "Killed" message but no error message indicating what happened. I have >enable logging on in spark configuration. has anyone seen this or know >how >to troubleshoot? > > > > >-- >View this message in context: >http://apache-spark-user-list.1001560.n3.nabble.com/Spark-submt-job-Killed >-tp17560.html >Sent from the Apache Spark User List mailing list archive at Nabble.com. > >--------------------------------------------------------------------- >To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >For additional commands, e-mail: user-h...@spark.apache.org > ________________________________________________________ The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
