How to set Description in UI SQL tab

2020-06-04 Thread gpatcham
Is there a way can we set description to display in UI SQL TAB ?. Like we can set sc.setJobDescription for Jobs and stages Thanks Giri -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail

Re: [PySpark] Tagging descriptions

2020-06-04 Thread Rishi Shah
Thanks everyone. While working on Tagging I stumbled upon another setback.. There are about 5000 regex I am dealing with, out of with couple of hundreds have variable length lookbehind (originally these worked in a JVM). In order to use this with Python/Pyspark udf - we need to either modify these