Hello Gourav, On 30 Dec 2017, at 20:20, Gourav Sengupta <gourav.sengu...@gmail.com> wrote: > Please try to use the SPARK UI from the way that AWS EMR recommends, it > should be available from the resource manager. I never ever had any problem > working with it. THAT HAS ALWAYS BEEN MY PRIMARY AND SOLE SOURCE OF DEBUGGING.
For some reason sometimes there is absolutely nothing showing up in the Spark UI or the UI is not refreshed, e.g. for the current stage is #x while the logs shows stage #y (with y > x) is currently under way. It may very well be that the source of this problem lies between the keyboard and the chair, but if this is the case, I do not know how to solve this. > Also, I ALWAYS prefer the maximize Resource Allocation setting in EMR to be > set to true. Thanks for the tip -- will try this setting in my next batch of experiments! JM --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
