Hi From personal experience....and I might be asking u obvious question 1. Does it work in standalone (no cluster) 2. Can u break down app in pieces and try to see at which step the code gets killed? 3. Have u had a look at spark gui to see if we executors go oom?
I might be oversimplifying what spark does. But if ur logic works standalone and does not work in cluster....the cluster might b ur problem..(apart from modules not being serializable) If it breaks in no cluster mode then it's easier to debug I am no way an expert, just talking from my little personal experience. I m sure someone here can give more hints on how to debug properly a spark app Hth On Jan 26, 2018 1:18 PM, "Chandu" <chandra.vedant...@gmail.com> wrote: > @Esa Thanks for posting this as I was thinking the same way when trying to > get some help about Spark (I am just a beginner) > > @Jack > I posted a question @ here ( > http://apache-spark-user-list.1001560.n3.nabble.com/Spark- > Standalone-Mode-application-runs-but-executor-is-killed-tc30739.html > <http://apache-spark-user-list.1001560.n3.nabble.com/ > Spark-Standalone-Mode-application-runs-but-executor-is-killed-tc30739.html > > > ) and stackoverflow ( > https://stackoverflow.com/questions/48445145/spark- > standalone-mode-application-runs-but-executor-is-killed-with-exitstatus > <https://stackoverflow.com/questions/48445145/spark- > standalone-mode-application-runs-but-executor-is-killed-with-exitstatus> > ) and haven't received much of views or even a comment. > > I am new to Spark and may be my question is framed badly. > Would you be able to take a look? > > > > -- > Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >
