Hi, On Thu, Mar 31, 2022 at 4:18 PM Pralabh Kumar <pralabhku...@gmail.com> wrote:
> Hi Spark Team > > Some of my spark applications on K8s ended with the below error . These > applications though completed successfully (as per the event log > SparkListenerApplicationEnd event at the end) > stil have even files with .inprogress. This causes the application to be > shown as inprogress in SHS. > > Spark v : 3.0.1 > I'd suggest you to try with newer version, e.g. 3.2.1 or even one built from branch-3.3. > > > > 22/03/31 08:33:34 WARN ShutdownHookManager: ShutdownHook '$anon$2' > timeout, java.util.concurrent.TimeoutException > > java.util.concurrent.TimeoutException > > at java.util.concurrent.FutureTask.get(FutureTask.java:205) > > at > org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:68) > > 22/03/31 08:33:34 WARN SparkContext: Ignoring Exception while stopping > SparkContext from shutdown hook > > java.lang.InterruptedException > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) > > at > java.util.concurrent.ThreadPoolExecutor.awaitTermination(ThreadPoolExecutor.java:1475) > > at > org.apache.spark.util.ThreadUtils$.shutdown(ThreadUtils.scala:348) > > > > > > Please let me know if there is a solution for it .. > > Regards > > Pralabh Kumar >