Re: How to debug Spark job

master local[*] --driver-memory 10g --executor-memory 10g ... --files ... --driver-class-path ... ... How can I debug or check which part of my code might cause the problem (so I can improve it)? Thanks

Re: [External Sender] How to debug Spark job

ts: 137128 ms exceeds timeout 120000 ms >> >> Memory in top command looks like >> >> VIRT RES SHR%CPU %MEM >> 15.866g 8.001g 41.4m 740.3 25.6 >> >> The command used to submit spark job is >> >> spark-submit --cla

Re: [External Sender] How to debug Spark job

gt; Memory in top command looks like >> >> VIRT RES SHR%CPU %MEM >> 15.866g 8.001g 41.4m 740.3 25.6 >> >> The command used to submit spark job is >> >> spark-submit --class ... --master local[*] --driver-memory 10g >>

Re: [External Sender] How to debug Spark job

WARN HeartbeatReceiver:66 - Removing executor driver with no recent > heartbeats: 137128 ms exceeds timeout 12 ms > > Memory in top command looks like > > VIRT RES SHR%CPU %MEM > 15.866g 8.001g 41.4m 740.3 25.6 > > The command used to

How to debug Spark job

25.6 The command used to submit spark job is spark-submit --class ... --master local[*] --driver-memory 10g --executor-memory 10g ... --files ... --driver-class-path ... ... How can I debug or check which part of my code might cause the problem (so I can improve it)? Thanks

how to debug app with cluster mode please?

Hi all, I am learning spark 2.1code. I write app with "master[4]", I run and debug code.It works well. when I change code with "master[2,2??1024]" and debug it as before, I meet error as follow: java.lang.classnotfindexception: com.xxx.xxx$$anonfun$main$1 the class is my main class.

Re: how to debug spark app?

Related question: what are good profiling tools other than watching along the application master with the running code? Are there things that can be logged during the run? If I have say 2 ways of accomplishing the same thing, and I want to learn about the time/memory/general resource blocking

Re: how to debug spark app?

Am not really sure of the best practices on this , but I either consult the localhost:4040/jobs/ etc or better this : val customSparkListener: CustomSparkListener = new CustomSparkListener() sc.addSparkListener(customSparkListener) class CustomSparkListener extends SparkListener { override def

Re: how to debug spark app?

Have you looked at: https://spark.apache.org/docs/latest/running-on-yarn.html#debugging-your-application If you use Mesos: https://spark.apache.org/docs/latest/running-on-mesos.html#troubleshooting-and-debugging On Wed, Aug 3, 2016 at 6:13 PM, glen wrote: > Any tool like gdb?

how to debug spark app?

Any tool like gdb? Which support break point at some line or some function?

How to debug spark-core with function call stack?

hello everyone, I would like to draw call stack of Spark-core by analyzing source code. But, I'm not sure how to apply debugging tool like gdb which can support backtrace command. Please let me know if you have any suggestion. Best Regards, Daejin Jung

Re: How to debug ClassCastException: java.lang.String cannot be cast to java.lang.Long in SparkSQL

> the data type mapping has been taken care of in my code, could you share this? On Tue, Jan 26, 2016 at 8:30 PM, Anfernee Xu wrote: > Hi, > > I'm using Spark 1.5.0, I wrote a custom Hadoop InputFormat to load data from > 3rdparty datasource, the data type mapping has been

How to debug ClassCastException: java.lang.String cannot be cast to java.lang.Long in SparkSQL

Hi, I'm using Spark 1.5.0, I wrote a custom Hadoop InputFormat to load data from 3rdparty datasource, the data type mapping has been taken care of in my code, but when I issued below query, SELECT * FROM ( SELECT count(*) as failures from test WHERE state != 'success' ) as tmp WHERE (

How to debug

Hi, I'm using Spark 1.5.0, I wrote a custom Hadoop InputFormat to load data from 3rdparty datasource, the data type mapping has been taken care of in my code, but when I issued below query, SELECT * FROM ( SELECT count(*) as failures from test WHERE state != 'success' ) as tmp WHERE (

How to debug join operations on a cluster.

some pointers how to debug this? I'm assuming the RDD is partitioned and shuffled and whatever is happening behind the scenes, except it is not behaving correctly, there aren't any exceptions, errors or even warnings and I have no clue why the join would be empty. Again: identical code run

how to debug java.lang.IllegalArgumentException: object is not an instance of declaring class

Hi Any idea how I can debug this problem. I suspect the problem has to do with how I am converting a JavaRDD<Tuple2<Long, Double>> to a DataFrame. Is it boxing problem? I tried to use long and double instead of Long and Double when ever possible. Thanks in advance, Happy Hol

Re: how to debug java.lang.IllegalArgumentException: object is not an instance of declaring class

vidson <a...@santacruzintegration.com> Date: Thursday, December 24, 2015 at 9:55 AM To: "user @spark" <user@spark.apache.org> Subject: how to debug java.lang.IllegalArgumentException: object is not an instance of declaring class > Hi > > Any idea how I can de

How to debug Spark source using IntelliJ/ Eclipse

tch after deleting everything and re-installing again, but I always face some dependency issues. It would be great if someone from the Spark users group could point me to the steps for setting up Spark debug environment. -- View this message in context: http://apache-spark-user-list.1001560

Re: How to debug java.io.OptionalDataException issues

Did you try kryo? Wrap everything with kryo and see if you are still hitting the exception. (At least you could see a different exception stack). Thanks Best Regards On Tue, Jul 7, 2015 at 6:05 AM, Yana Kadiyska yana.kadiy...@gmail.com wrote: Hi folks, suffering from a pretty strange issue:

How to debug java.io.OptionalDataException issues

Hi folks, suffering from a pretty strange issue: Is there a way to tell what object is being successfully serialized/deserialized? I have a maven-installed jar that works well when fat jarred within another, but shows the following stack when marked as provided and copied to the runtime

How to debug spark in IntelliJ Idea

to remote actor(spark master), the breakpoint would be enabled. I don't know how to debug it in IntelliJ Idea. I need help. Thanks. Regards, Yi -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-debug-spark-in-IntelliJ-Idea-tp22932.html Sent from

Re: How to debug Spark on Yarn?

On 27 Apr 2015, at 07:51, ÐΞ€ρ@Ҝ (๏̯͡๏) deepuj...@gmail.commailto:deepuj...@gmail.com wrote: Spark 1.3 1. View stderr/stdout from executor from Web UI: when the job is running i figured out the executor that am suppose to see, and those two links show 4 special characters on browser. 2.

Re: How to debug Spark on Yarn?

Spark 1.3 1. View stderr/stdout from executor from Web UI: when the job is running i figured out the executor that am suppose to see, and those two links show 4 special characters on browser. 2. Tail on Yarn logs: /apache/hadoop/bin/yarn logs -applicationId application_1429087638744_151059 |

Re: How to debug Spark on Yarn?

1) Application container logs from Web RM UI never load on browser. I eventually have to kill the browser. 2) /apache/hadoop/bin/yarn logs -applicationId application_1429087638744_151059 | less emits logs only after the application has completed. Are there no better ways to see the logs as they

Re: How to debug Spark on Yarn?

You can check container logs from RM web UI or when log-aggregation is enabled with the yarn command. There are other, but less convenient options. On Mon, Apr 27, 2015 at 8:53 AM ÐΞ€ρ@Ҝ (๏̯͡๏) deepuj...@gmail.com wrote: Spark 1.3 1. View stderr/stdout from executor from Web UI: when the job

Re: How to debug Spark on Yarn?

On top of what's been said... On Wed, Apr 22, 2015 at 10:48 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) deepuj...@gmail.com wrote: 1) I can go to Spark UI and see the status of the APP but cannot see the logs as the job progresses. How can i see logs of executors as they progress ? Spark 1.3 should have links to the

Re: How to debug Spark on Yarn?

For #1, click on a worker node on the YARN dashboard. From there, Tools-Local logs-Userlogs has the logs for each application, and you can view them by executor even while an application is running. (This is for Hadoop 2.4, things may have changed in 2.6.) -Sven On Thu, Apr 23, 2015 at 6:27 AM,

Re: How to debug Spark on Yarn?

On Fri, Apr 24, 2015 at 11:31 AM, Marcelo Vanzin van...@cloudera.com wrote: Spark 1.3 should have links to the executor logs in the UI while the application is running. Not yet in the history server, though. You're absolutely correct -- didn't notice it until now. This is a great addition!

Re: How to debug Spark on Yarn?

For step 2, you can pipe application log to a file instead of copy-pasting. Cheers On Apr 22, 2015, at 10:48 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) deepuj...@gmail.com wrote: I submit a spark app to YARN and i get these messages 15/04/22 22:45:04 INFO yarn.Client: Application report for

How to debug Spark on Yarn?

I submit a spark app to YARN and i get these messages 15/04/22 22:45:04 INFO yarn.Client: Application report for application_1429087638744_101363 (state: RUNNING) 15/04/22 22:45:04 INFO yarn.Client: Application report for application_1429087638744_101363 (state: RUNNING). ... 1) I can go to

How to debug a hung spark application

this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-debug-a-hung-spark-application-tp21859.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: How to debug a Hung task

To: user@spark.apache.org user@spark.apache.org Sent: Friday, February 27, 2015 3:50 PM Subject: How to debug a Hung task Hi,  I have a spark application that hangs on doing just one task (Rest 200-300 task gets completed in reasonable time)I can see in the Thread dump which function gets stuck

How to debug a Hung task

Hi, I have a spark application that hangs on doing just one task (Rest 200-300 task gets completed in reasonable time) I can see in the Thread dump which function gets stuck how ever I don't have a clue as to what value is causing that behaviour. Also, logging the inputs before the function is

Re: how to debug this kind of error, e.g. lost executor?

...@gmail.com Date: Friday, 6 February 2015 3:53 pm To: Ankur Srivastava ankur.srivast...@gmail.commailto:ankur.srivast...@gmail.com Cc: user@spark.apache.orgmailto:user@spark.apache.org user@spark.apache.orgmailto:user@spark.apache.org Subject: Re: how to debug this kind of error, e.g. lost executor? Hi

Re: how to debug this kind of error, e.g. lost executor?

could you find the shuffle files? or the files were deleted by other processes? Yours, Xuefeng Wu 吴雪峰 敬上 On 2015年2月5日, at 下午11:14, Yifan LI iamyifa...@gmail.com wrote: Hi, I am running a heavy memory/cpu overhead graphx application, I think the memory is sufficient and set RDDs’

how to debug this kind of error, e.g. lost executor?

Hi, I am running a heavy memory/cpu overhead graphx application, I think the memory is sufficient and set RDDs’ StorageLevel using MEMORY_AND_DISK. But I found there were some tasks failed due to following errors: java.io.FileNotFoundException:

Re: how to debug this kind of error, e.g. lost executor?

Anyone has idea on where I can find the detailed log of that lost executor(why it was lost)? Thanks in advance! On 05 Feb 2015, at 16:14, Yifan LI iamyifa...@gmail.com wrote: Hi, I am running a heavy memory/cpu overhead graphx application, I think the memory is sufficient and set

Re: how to debug this kind of error, e.g. lost executor?

Li, I cannot tell you the reason for this exception but have seen these kind of errors when using HASH based shuffle manager (which is default) until v 1.2. Try the SORT shuffle manager. Hopefully that will help Thanks Ankur Anyone has idea on where I can find the detailed log of that lost

Re: how to debug ExecutorLostFailure

in context: http://apache-spark-user-list.1001560.n3.nabble.com/how-to-debug-ExecutorLostFailure-tp15646p15680.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr

how to debug ExecutorLostFailure

and completes the task. Just wondering what is the best way to understand why these tasks failed (couldn't seem to find anything in the logs), and how to avoid in the future? thanks, -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/how-to-debug-ExecutorLostFailure

Re: How to debug this error?

It's not allowed to use RDD in map function. RDD can only operated at driver of spark program. At your case, group RDD can't be found at every executor. I guess you want to implement subquery like operation, try to use RDD.intersection() or join() 2014-08-29 12:43 GMT+08:00 Gary Zhao

How to debug this error?

Hello I'm new to Spark and playing around, but saw the following error. Could anyone to help on it? Thanks Gary scala c res15: org.apache.spark.rdd.RDD[String] = FlatMappedRDD[7] at flatMap at console:23 scala group res16: org.apache.spark.rdd.RDD[(String, Iterable[String])] =

Re: How to debug: Runs locally but not on cluster

://geonames).sample(false,0.35) // 64MB, 2849439 Lines // fails val samples = sc.textFile(s3n://geonames).sample(false,0.4) // 64MB, 2849439 Lines Any ideas? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-debug-Runs-locally-but-not-on-cluster

How to debug: Runs locally but not on cluster

Hi all, I have an issue where I'm able to run my code in standalone mode but not on my cluster. I've isolated it to a few things but am at a lost at how to debug this. Below is the code. Any suggestions would be much appreciated Thanks! 1) RDD size is causing the problem. The code below