You don't need to care about this sleep. It runs in a separate thread and usually won't affect the performance of your application.
Best Regards, Shixiong Zhu 2015-10-09 6:03 GMT+08:00 yael aharon <yael.aharo...@gmail.com>: > Hello, > I am working on improving the performance of our Spark on Yarn > applications. > Scanning through the logs I found the following lines: > > > [2015-10-07T16:25:17.245-04:00] [DataProcessing] [INFO] [] > [org.apache.spark.Logging$class] [tid:main] [userID:yarn] Started progress > reporter thread - sleep time : 5000 > [2015-10-07T16:25:22.262-04:00] [DataProcessing] [INFO] [] > [org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl] [tid:Reporter] > [userID:yarn] Received new token for : hostname:8041 > > > As the log says, the main thread sleeps for 5 seconds. Is there a way to > configure/eliminate this sleep? > thanks, Yael >
