Re: [Spark SQL][Beginner] Spark throw Catalyst error while writing the dataframe in ORC format

2020-05-11 Thread Deepak Garg
Hi Jeff, I increased the broadcast timeout. Now facing the new error. Caused by: java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039) at

Re: [Spark SQL][Beginner] Spark throw Catalyst error while writing the dataframe in ORC format

2020-05-07 Thread Jeff Evans
You appear to be hitting the broadcast timeout. See: https://stackoverflow.com/a/41126034/375670 On Thu, May 7, 2020 at 8:56 AM Deepak Garg wrote: > Hi, > > I am getting following error while running a spark job. Error > occurred when Spark is trying to write the dataframe in ORC format . I am

[Spark SQL][Beginner] Spark throw Catalyst error while writing the dataframe in ORC format

2020-05-07 Thread Deepak Garg
Hi, I am getting following error while running a spark job. Error occurred when Spark is trying to write the dataframe in ORC format . I am pasting the error trace. Any help in resolving this would be appreciated. Caused by: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: