I feel this quite lacks information. Full stack traces from driver/executors are essential at least to determine what was happening.
On Tue, Mar 2, 2021 at 5:26 PM Sachit Murarka <connectsac...@gmail.com> wrote: > Hi All, > > My spark job is crashing (Structured stream) . Can anyone help please. I > am using spark 3.0.1 with kubernetes. > > [ERROR] - StreamingQueryException Exception in query.awaitTermination() > File "/opt/spark/python/lib/pyspark.zip/pyspark/sql/streaming.py", line > 103, in awaitTermination > return self._jsq.awaitTermination() > File "/opt/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py", > line 1304, in __call__ > return_value = get_return_value( > File "/opt/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 134, > in deco > raise_from(converted) > > Kind Regards, > Sachit Murarka >
