Hi,This time instead of manually starting worker node using ./bin/spark-class org.apache.spark.deploy.worker.Worker spark://IP:PORT I used start-slaves script on the master node. I also enabled -v (verbose flag) in ssh. Here is the o/p that I see. The log file for to the worker node was not created. I will switch back to the manual process for starting the cluster. bash-4.1$ ./start-slaves.sh172.16.48.44: OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010172.16.48.44: debug1: Reading configuration data /users/userid/.ssh/config172.16.48.44: debug1: Reading configuration data /etc/ssh/ssh_config172.16.48.44: debug1: Applying options for *172.16.48.44: debug1: Connecting to 172.16.48.44 [172.16.48.44] port 22.172.16.48.44: debug1: Connection established.172.16.48.44: debug1: identity file /users/p529444/.ssh/identity type -1172.16.48.44: debug1: identity file /users/p529444/.ssh/identity-cert type -1172.16.48.44: debug1: identity file /users/p529444/.ssh/id_rsa type 1172.16.48.44: debug1: identity file /users/p529444/.ssh/id_rsa-cert type -1172.16.48.44: debug1: identity file /users/p529444/.ssh/id_dsa type -1172.16.48.44: debug1: identity file /users/p529444/.ssh/id_dsa-cert type -1172.16.48.44: debug1: Remote protocol version 2.0, remote software version OpenSSH_5.2p1_q17.gM-hpn13v6172.16.48.44: debug1: match: OpenSSH_5.2p1_q17.gM-hpn13v6 pat OpenSSH*172.16.48.44: debug1: Enabling compatibility mode for protocol 2.0172.16.48.44: debug1: Local version string SSH-2.0-OpenSSH_5.3172.16.48.44: debug1: SSH2_MSG_KEXINIT sent172.16.48.44: debug1: SSH2_MSG_KEXINIT received172.16.48.44: debug1: kex: server->client aes128-ctr hmac-md5 none172.16.48.44: debug1: kex: client->server aes128-ctr hmac-md5 none172.16.48.44: debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent172.16.48.44: debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP172.16.48.44: debug1: SSH2_MSG_KEX_DH_GEX_INIT sent172.16.48.44: debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY172.16.48.44: debug1: Host '172.16.48.44' is known and matches the RSA host key.172.16.48.44: debug1: Found key in /users/p529444/.ssh/known_hosts:6172.16.48.44: debug1: ssh_rsa_verify: signature correct172.16.48.44: debug1: SSH2_MSG_NEWKEYS sent172.16.48.44: debug1: expecting SSH2_MSG_NEWKEYS172.16.48.44: debug1: SSH2_MSG_NEWKEYS received172.16.48.44: debug1: SSH2_MSG_SERVICE_REQUEST sent172.16.48.44: debug1: SSH2_MSG_SERVICE_ACCEPT received172.16.48.44: ================================================================172.16.48.44: This is a private computer system. Access to and use requires172.16.48.44: explicit current authorization and is limited to business use.172.16.48.44: All users express consent to monitoring by system personnel to172.16.48.44: detect improper use of or access to the system, system personnel172.16.48.44: may provide evidence of such conduct to law enforcement172.16.48.44: officials and/or company management.172.16.48.44: ================================================================172.16.48.44: UAM R2 account support: http://ussweb.crdc.kp.org/UAM/172.16.48.44: ================================================================172.16.48.44: For password resets, please call the Helpdesk 888-457-4872172.16.48.44: ================================================================172.16.48.44: debug1: Authentications that can continue: gssapi-keyex,gssapi-with-mic,publickey,password,keyboard-interactive172.16.48.44: debug1: Next authentication method: gssapi-keyex172.16.48.44: debug1: No valid Key exchange context172.16.48.44: debug1: Next authentication method: gssapi-with-mic172.16.48.44: debug1: Unspecified GSS failure. Minor code may provide more information172.16.48.44: Cannot find KDC for requested realm172.16.48.44:172.16.48.44: debug1: Unspecified GSS failure. Minor code may provide more information172.16.48.44: Cannot find KDC for requested realm172.16.48.44:172.16.48.44: debug1: Unspecified GSS failure. Minor code may provide more information172.16.48.44:172.16.48.44:172.16.48.44: debug1: Authentications that can continue: gssapi-keyex,gssapi-with-mic,publickey,password,keyboard-interactive172.16.48.44: debug1: Next authentication method: publickey172.16.48.44: debug1: Trying private key: /users/userid/.ssh/identity172.16.48.44: debug1: Offering public key: /users/userid/.ssh/id_rsa172.16.48.44: debug1: Server accepts key: pkalg ssh-rsa blen 277172.16.48.44: debug1: read PEM private key done: type RSA172.16.48.44: debug1: Authentication succeeded (publickey).172.16.48.44: debug1: channel 0: new [client-session]172.16.48.44: debug1: Requesting no-more-sessions@openssh.com172.16.48.44: debug1: Entering interactive session.172.16.48.44: debug1: Sending environment.172.16.48.44: debug1: Sending env LANG = en_US.UTF-8172.16.48.44: debug1: Sending command: cd /apps/software/spark-1.0.0-bin-hadoop1/sbin/.. ; /apps/software/spark-1.0.0-bin-hadoop1/sbin/start-slave.sh 1 spark://172.16.48.41:7077172.16.48.44: starting org.apache.spark.deploy.worker.Worker, logging to /apps/software/spark-1.0.0-bin-hadoop1/sbin/../logs/spark-p529444-org.apache.spark.deploy.worker.Worker-1-pzxnvm2022.out172.16.48.44: debug1: client_input_channel_req: channel 0 rtype exit-status reply 0172.16.48.44: debug1: client_input_channel_req: channel 0 rtype e...@openssh.com reply 0172.16.48.44: debug1: channel 0: free: client-session, nchannels 1172.16.48.44: debug1: fd 1 clearing O_NONBLOCK172.16.48.44: Transferred: sent 2664, received 3488 bytes, in 2.1 seconds172.16.48.44: Bytes per second: sent 1286.2, received 1684.0172.16.48.44: debug1: Exit status 0
