Re: Spark UI crashes on Large Workloads

2017-07-18 Thread Saatvik Shah
input >>>> throws the error explained here sometimes: >>>> https://stackoverflow.com/questions/32340639/unable-to-under >>>> stand-error-sparklistenerbus-has-already-stopped-dropping-event. >>>> All my code is running inside the main function, and t

Re: Spark UI crashes on Large Workloads

2017-07-18 Thread Riccardo Ferrari
stopped-dropping-event. >>> All my code is running inside the main function, and the only slightly >>> peculiar thing I am doing in this app is using a custom PySpark ML >>> Transformer(Modified from >>> https://stacko

Re: Spark UI crashes on Large Workloads

2017-07-18 Thread Saatvik Shah
htly >> peculiar thing I am doing in this app is using a custom PySpark ML >> Transformer(Modified from >> https://stackoverflow.com/questions/32331848/create-a-custom >> -transformer-in-pyspark-ml). >> Could this be the issue? How can I debug why this is happeni

Re: Spark UI crashes on Large Workloads

2017-07-18 Thread Riccardo Ferrari
custom PySpark ML > Transformer(Modified from > https://stackoverflow.com/questions/32331848/create-a- > custom-transformer-in-pyspark-ml). > Could this be the issue? How can I debug why this is happening? > > > > -- > View this message in context: http://apache-spark-use

Spark UI crashes on Large Workloads

2017-07-17 Thread saatvikshah1994
: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-UI-crashes-on-Large-Workloads-tp28873.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe e-mail: user-unsubscr