Re: Future timeout

Just a suggestion, Looks like its timing out when you are broadcasting big object. Generally its not advisable to do so, if you can get rid of that, program may behave consistent. On Tue, Jul 21, 2020 at 3:17 AM Piyush Acharya wrote: > spark.conf.set("spark.sql.broadcastTimeout", ##) > >

Re: Future timeout

spark.conf.set("spark.sql.broadcastTimeout", ##) On Mon, Jul 20, 2020 at 11:51 PM Amit Sharma wrote: > Please help on this. > > > Thanks > Amit > > On Fri, Jul 17, 2020 at 9:10 AM Amit Sharma wrote: > >> Hi, sometimes my spark streaming job throw this exception Futures timed >> out after

Re: Future timeout

"spark.sql.broadcastTimeout" is the config you can use: https://github.com/apache/spark/blob/fe07521c9efd9ce0913eee0d42b0ffd98b1225ec/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala#L863 Thanks, Terry On Mon, Jul 20, 2020 at 11:20 AM Amit Sharma wrote: > Please help on

Re: Future timeout

Please help on this. Thanks Amit On Fri, Jul 17, 2020 at 9:10 AM Amit Sharma wrote: > Hi, sometimes my spark streaming job throw this exception Futures timed > out after [300 seconds]. > I am not sure where is the default timeout configuration. Can i increase > it. Please help. > > > >

Future timeout

Hi, sometimes my spark streaming job throw this exception Futures timed out after [300 seconds]. I am not sure where is the default timeout configuration. Can i increase it. Please help. Thanks Amit Caused by: java.util.concurrent.TimeoutException: Futures timed out after [300 seconds]