Hi Zhiliang, You should be able to see them in the executor logs, which you can view via the Spark UI, in the Executors page (stderr log).
HTH, Deng On Tue, Nov 10, 2015 at 11:33 AM, Zhiliang Zhu <zchl.j...@yahoo.com.invalid> wrote: > Hi All, > > I need debug spark job, my general way is to print out the log, however, > some bug is in spark functions as mapPartitions etc, and not any log > printed from those functions > could be found... > Would you help point what is way to the log in the spark own function as > mapPartitions? Or, what is general way to debug spark job. > > Thank you in advance! > > Zhiliang >
