Spark submit hbase issue

[KhajaAsmath Mohammed]

Hi,

Spark submit is connecting to local host instead of zookeeper mentioned in 
hbase-site.xml. This same program works in ide which gets connected to 
hbase-site.xml. What am I missing in spark submit?
> 
> 
> spark-submit --driver-class-path  
> C:\Users\mdkha\bitbucket\clx-spark-scripts\src\test\resources\hbase-site.xml 
> --files 
> C:\Users\mdkha\bitbucket\clx-spark-scripts\src\test\resources\hbase-site.xml  
> --conf 
> "spark.driver.extraLibraryPath=C:\Users\mdkha\bitbucket\clx-spark-scripts\src\test\resources\hbase-site.xml"
>  --executor-memory 4g --class com.drivers.HBASEExportToS3 
> C:\Users\mdkha\bitbucket\clx-spark-scripts\target\clx-spark-scripts.jar -c 
> C:\Users\mdkha\bitbucket\clx-spark-scripts\src\test\resources\job.properties 
> 
> 
> 21/04/14 16:13:36 WARN ClientCnxn: Session 0x0 for server null, unexpected 
> error, closing socket connection and attempting reconnect
> java.net.ConnectException: Connection refused: no further information
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at 
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:715)
> at 
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
> at 
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
> 21/04/14 16:13:37 WARN ReadOnlyZKClient: 0x2d73767e to localhost:2181 failed 
> for get of /hbase/hbaseid, code = CONNECTIONLOSS, retries = 1

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: Spark submit hbase issue

[Mich Talebzadeh]

Try adding hbase-site.xml file to  %SPARK_HOME%\conf and see if it works


HTH




   view my Linkedin profile




*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Wed, 14 Apr 2021 at 22:40, KhajaAsmath Mohammed 
wrote:

> Hi,
>
> Spark submit is connecting to local host instead of zookeeper mentioned in
> hbase-site.xml. This same program works in ide which gets connected to
> hbase-site.xml. What am I missing in spark submit?
> >
> > 
> > spark-submit --driver-class-path
> C:\Users\mdkha\bitbucket\clx-spark-scripts\src\test\resources\hbase-site.xml
> --files
> C:\Users\mdkha\bitbucket\clx-spark-scripts\src\test\resources\hbase-site.xml
> --conf
> "spark.driver.extraLibraryPath=C:\Users\mdkha\bitbucket\clx-spark-scripts\src\test\resources\hbase-site.xml"
> --executor-memory 4g --class com.drivers.HBASEExportToS3
> C:\Users\mdkha\bitbucket\clx-spark-scripts\target\clx-spark-scripts.jar -c
> C:\Users\mdkha\bitbucket\clx-spark-scripts\src\test\resources\job.properties
>
> >
> >
> > 21/04/14 16:13:36 WARN ClientCnxn: Session 0x0 for server null,
> unexpected error, closing socket connection and attempting reconnect
> > java.net.ConnectException: Connection refused: no further information
> > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:715)
> > at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
> > at
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
> > 21/04/14 16:13:37 WARN ReadOnlyZKClient: 0x2d73767e to localhost:2181
> failed for get of /hbase/hbaseid, code = CONNECTIONLOSS, retries = 1
>
> -
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>
>