On 27 Apr 2015, at 07:51, ÐΞ€ρ@Ҝ (๏̯͡๏)
deepuj...@gmail.commailto:deepuj...@gmail.com wrote:
Spark 1.3
1. View stderr/stdout from executor from Web UI: when the job is running i
figured out the executor that am suppose to see, and those two links show 4
special characters on browser.
2.
Spark 1.3
1. View stderr/stdout from executor from Web UI: when the job is running i
figured out the executor that am suppose to see, and those two links show 4
special characters on browser.
2. Tail on Yarn logs:
/apache/hadoop/bin/yarn logs -applicationId
application_1429087638744_151059 |
1) Application container logs from Web RM UI never load on browser. I
eventually have to kill the browser.
2) /apache/hadoop/bin/yarn logs -applicationId
application_1429087638744_151059
| less emits logs only after the application has completed.
Are there no better ways to see the logs as they
You can check container logs from RM web UI or when log-aggregation is
enabled with the yarn command. There are other, but less convenient options.
On Mon, Apr 27, 2015 at 8:53 AM ÐΞ€ρ@Ҝ (๏̯͡๏) deepuj...@gmail.com wrote:
Spark 1.3
1. View stderr/stdout from executor from Web UI: when the job
On top of what's been said...
On Wed, Apr 22, 2015 at 10:48 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) deepuj...@gmail.com wrote:
1) I can go to Spark UI and see the status of the APP but cannot see the
logs as the job progresses. How can i see logs of executors as they progress
?
Spark 1.3 should have links to the
For #1, click on a worker node on the YARN dashboard. From there,
Tools-Local logs-Userlogs has the logs for each application, and you can
view them by executor even while an application is running. (This is for
Hadoop 2.4, things may have changed in 2.6.)
-Sven
On Thu, Apr 23, 2015 at 6:27 AM,
On Fri, Apr 24, 2015 at 11:31 AM, Marcelo Vanzin van...@cloudera.com
wrote:
Spark 1.3 should have links to the executor logs in the UI while the
application is running. Not yet in the history server, though.
You're absolutely correct -- didn't notice it until now. This is a great
addition!
For step 2, you can pipe application log to a file instead of copy-pasting.
Cheers
On Apr 22, 2015, at 10:48 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) deepuj...@gmail.com wrote:
I submit a spark app to YARN and i get these messages
15/04/22 22:45:04 INFO yarn.Client: Application report for
I submit a spark app to YARN and i get these messages
15/04/22 22:45:04 INFO yarn.Client: Application report for
application_1429087638744_101363 (state: RUNNING)
15/04/22 22:45:04 INFO yarn.Client: Application report for
application_1429087638744_101363 (state: RUNNING).
...
1) I can go to
