Re: Spark Version 3.0.1 Gui Display Query

Okay, Please share console outputs and network logs of executors tab from browser On Tue, Mar 2, 2021 at 11:56 AM Ranju Jain wrote: > Hi Kapil, > > > > I am not able to see executor info throughout the application lifetime. > > Attaching screenshots. > >1. Jobs Tab during application start >

RE: Spark Version 3.0.1 Gui Display Query

Hi Kapil, I am not able to see executor info throughout the application lifetime. Attaching screenshots. 1. Jobs Tab during application start 2. Executors Tab during application lifetime I need to tune my application , this Executor Info would be a great help for tuning the parameters.

RE: Spark Version 3.0.1 Gui Display Query

Hi Kapil, I am not able to see executor info throughout the application lifetime. Attaching screenshots. 1. Jobs Tab during application start 2. Executors Tab during application lifetime I need to tune my application , this Executor Info would be a great help for tuning the parameters.

Re: Spark Version 3.0.1 Gui Display Query

Hi Ranju, Is it happening just after you submit the spark application ? or are you not able to see executors info throughout the application lifetime ? Because you won't be able to see any info there until executors have been added to the application and tasks have been submitted. On Tue, Mar 2,

Spark Version 3.0.1 Gui Display Query

Hi , I started using Spark 3.0.1 version recently and noticed the Executors Tab on Spark GUI appears as blank. Please suggest what could be the reason of this type of display? Regards Ranju

[Virtual meetup] March 11th 10am PST

Join Chris Crosbie, a Product Manager at Google Cloud, Feng Lu from Google Cloud and Ajay Singh from Databricks to talk about data lakes, Delta Lake, Apache Hive Metastore & everything in between. Join us by signing up with the link 👇🏽 www.crowdcast.io/e/data-lake-managemen

Re: s3a staging committer(directory committer )not writing data to s3 bucket (final output directory) in spark3

Hi, We also have an issue with data not being displayed in Google Cloud DataProc 2 that uses Spark 3.1.1. It works on 3.0.1 on Prem but not on 3.1.1 on Google Data Proc (offered as a service). It may be related to Spark version It is concerning. HTH LinkedIn * https://www.linkedin.com/pr

Re: s3a staging committer(directory committer )not writing data to s3 bucket (final output directory) in spark3

Hi Mich Talebzadeh, Thanks for your reply, the issue is seen in spark 3.0.0 and with spark 2.4.5 it works without any problem. Regards, Shiva -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe

Re: s3a staging committer(directory committer )not writing data to s3 bucket (final output directory) in spark3

Hi, What exact version of spark is it? HTH LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw * *Disclaimer:* Use it at your own risk. Any and all responsibility for

Re: s3a staging committer(directory committer )not writing data to s3 bucket (final output directory) in spark3

Any suggestions or help is greatly appreciated! -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: Spark structured streaming Stuck on Batch = 0 on spark 3.1.1, Dataproc cluster

BY the way as per streaming doc , one can monitor streaming status with result = streamingDataFrame.select( \ writeStream. \ outputMode('append'). \