Re: Welcoming some new committers and PMC members
Hi, Congrats! Kind Regards, Furkan KAMACI On Tue, Sep 10, 2019 at 2:48 PM Stavros Kontopoulos < stavros.kontopou...@lightbend.com> wrote: > Congrats! Well deserved. > > On Tue, Sep 10, 2019 at 1:20 PM Driesprong, Fokko > wrote: > >> Congrats all, well deserved! >> >> >> Cheers, Fokko >> >> Op di 10 sep. 2019 om 10:21 schreef Gabor Somogyi < >> gabor.g.somo...@gmail.com>: >> >>> Congrats Guys! >>> >>> G >>> >>> >>> On Tue, Sep 10, 2019 at 2:32 AM Matei Zaharia >>> wrote: >>> >>>> Hi all, >>>> >>>> The Spark PMC recently voted to add several new committers and one PMC >>>> member. Join me in welcoming them to their new roles! >>>> >>>> New PMC member: Dongjoon Hyun >>>> >>>> New committers: Ryan Blue, Liang-Chi Hsieh, Gengliang Wang, Yuming >>>> Wang, Weichen Xu, Ruifeng Zheng >>>> >>>> The new committers cover lots of important areas including ML, SQL, and >>>> data sources, so it’s great to have them here. All the best, >>>> >>>> Matei and the Spark PMC >>>> >>>> >>>> - >>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >>>> >>>> > > -- >
Re: Spark Cannot Connect to HBaseClusterSingleton
Hi Ted, You can check full stack trace log from the attachment at Jira: https://issues.apache.org/jira/browse/GORA-386 Kind Regards, Furkan KAMACI On Wed, Aug 26, 2015 at 6:55 PM, Ted Yu yuzhih...@gmail.com wrote: My understanding is that people on this mailing list who are interested to help can log comments on the GORA JIRA. HBase integration with Spark is proven to work. So the intricacies should be on Gora side. On Wed, Aug 26, 2015 at 8:08 AM, Furkan KAMACI furkankam...@gmail.com wrote: Btw, here is the source code of GoraInputFormat.java : https://github.com/kamaci/gora/blob/master/gora-core/src/main/java/org/apache/gora/mapreduce/GoraInputFormat.java 26 Ağu 2015 18:05 tarihinde Furkan KAMACI furkankam...@gmail.com yazdı: I'll send an e-mail to Gora dev list too and also attach my patch into my GSoC Jira issue you mentioned and then we can continue at there. Before I do that stuff, I wanted to get Spark dev community's ideas to solve my problem due to you may have faced such kind of problems before. 26 Ağu 2015 17:13 tarihinde Ted Yu yuzhih...@gmail.com yazdı: I found GORA-386 Gora Spark Backend Support Should the discussion be continued there ? Cheers On Wed, Aug 26, 2015 at 7:02 AM, Ted Malaska ted.mala...@cloudera.com wrote: Where is the input format class. When every I use the search on your github it says We couldn’t find any issues matching 'GoraInputFormat' On Wed, Aug 26, 2015 at 9:48 AM, Furkan KAMACI furkankam...@gmail.com wrote: Hi, Here is the MapReduceTestUtils.testSparkWordCount() https://github.com/kamaci/gora/blob/master/gora-core/src/test/java/org/apache/gora/mapreduce/MapReduceTestUtils.java#L108 Here is SparkWordCount https://github.com/kamaci/gora/blob/8f1acc6d4ef6c192e8fc06287558b7bc7c39b040/gora-core/src/examples/java/org/apache/gora/examples/spark/SparkWordCount.java Lastly, here is GoraSparkEngine: https://github.com/kamaci/gora/blob/master/gora-core/src/main/java/org/apache/gora/spark/GoraSparkEngine.java Kind Regards, Furkan KAMACI On Wed, Aug 26, 2015 at 4:40 PM, Ted Malaska ted.mala...@cloudera.com wrote: Where can I find the code for MapReduceTestUtils.testSparkWordCount? On Wed, Aug 26, 2015 at 9:29 AM, Furkan KAMACI furkankam...@gmail.com wrote: Hi, Here is the test method I've ignored due to Connection Refused problem failure: https://github.com/kamaci/gora/blob/master/gora-hbase/src/test/java/org/apache/gora/hbase/mapreduce/TestHBaseStoreWordCount.java#L65 I've implemented a Spark backend for Apache Gora as GSoC project and this is the latest obstacle that I should solve. If you can help me, you are welcome. Kind Regards, Furkan KAMACI On Wed, Aug 26, 2015 at 3:45 PM, Ted Malaska ted.mala...@cloudera.com wrote: I've always used HBaseTestingUtility and never really had much trouble. I use that for all my unit testing between Spark and HBase. Here are some code examples if your interested --Main HBase-Spark Module https://github.com/apache/hbase/tree/master/hbase-spark --Unit test that cover all basic connections https://github.com/apache/hbase/blob/master/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/HBaseContextSuite.scala --If you want to look at the old stuff before it went into HBase https://github.com/cloudera-labs/SparkOnHBase Let me know if that helps On Wed, Aug 26, 2015 at 5:40 AM, Ted Yu yuzhih...@gmail.com wrote: Can you log the contents of the Configuration you pass from Spark ? The output would give you some clue. Cheers On Aug 26, 2015, at 2:30 AM, Furkan KAMACI furkankam...@gmail.com wrote: Hi Ted, I'll check Zookeeper connection but another test method which runs on hbase without Spark works without any error. Hbase version is 0.98.8-hadoop2 and I use Spark 1.3.1 Kind Regards, Furkan KAMACI 26 Ağu 2015 12:08 tarihinde Ted Yu yuzhih...@gmail.com yazdı: The connection failure was to zookeeper. Have you verified that localhost:2181 can serve requests ? What version of hbase was Gora built against ? Cheers On Aug 26, 2015, at 1:50 AM, Furkan KAMACI furkankam...@gmail.com wrote: Hi, I start an Hbase cluster for my test class. I use that helper class: https://github.com/apache/gora/blob/master/gora-hbase/src/test/java/org/apache/gora/hbase/util/HBaseClusterSingleton.java and use it as like that: private static final HBaseClusterSingleton cluster = HBaseClusterSingleton.build(1); I retrieve configuration object as follows: cluster.getConf() and I use it at Spark as follows: sparkContext.newAPIHadoopRDD(conf, MyInputFormat.class, clazzK, clazzV); When I run my test there is no need to startup an Hbase cluster because Spark will connect to my dummy cluster. However when I run my test method it throws an error: 2015-08-26 01:19:59,558 INFO [Executor task launch worker-0-SendThread(localhost:2181)] zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(966)) - Opening
Spark Cannot Connect to HBaseClusterSingleton
Hi, I start an Hbase cluster for my test class. I use that helper class: https://github.com/apache/gora/blob/master/gora-hbase/src/test/java/org/apache/gora/hbase/util/HBaseClusterSingleton.java and use it as like that: private static final HBaseClusterSingleton cluster = HBaseClusterSingleton.build(1); I retrieve configuration object as follows: cluster.getConf() and I use it at Spark as follows: sparkContext.newAPIHadoopRDD(conf, MyInputFormat.class, clazzK, clazzV); When I run my test there is no need to startup an Hbase cluster because Spark will connect to my dummy cluster. However when I run my test method it throws an error: 2015-08-26 01:19:59,558 INFO [Executor task launch worker-0-SendThread(localhost:2181)] zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(966)) - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2015-08-26 01:19:59,559 WARN [Executor task launch worker-0-SendThread(localhost:2181)] zookeeper.ClientCnxn (ClientCnxn.java:run(1089)) - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068) Hbase tests, which do not run on Spark, works well. When I check the logs I see that cluster and Spark is started up correctly: 2015-08-26 01:35:21,791 INFO [main] hdfs.MiniDFSCluster (MiniDFSCluster.java:waitActive(2055)) - Cluster is active 2015-08-26 01:35:40,334 INFO [main] util.Utils (Logging.scala:logInfo(59)) - Successfully started service 'sparkDriver' on port 56941. I realized that when I start up an hbase from command line my test method for Spark connects to it! So, does it means that it doesn't care about the conf I passed to it? Any ideas about how to solve it?
Re: Spark Cannot Connect to HBaseClusterSingleton
Hi, Here is the test method I've ignored due to Connection Refused problem failure: https://github.com/kamaci/gora/blob/master/gora-hbase/src/test/java/org/apache/gora/hbase/mapreduce/TestHBaseStoreWordCount.java#L65 I've implemented a Spark backend for Apache Gora as GSoC project and this is the latest obstacle that I should solve. If you can help me, you are welcome. Kind Regards, Furkan KAMACI On Wed, Aug 26, 2015 at 3:45 PM, Ted Malaska ted.mala...@cloudera.com wrote: I've always used HBaseTestingUtility and never really had much trouble. I use that for all my unit testing between Spark and HBase. Here are some code examples if your interested --Main HBase-Spark Module https://github.com/apache/hbase/tree/master/hbase-spark --Unit test that cover all basic connections https://github.com/apache/hbase/blob/master/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/HBaseContextSuite.scala --If you want to look at the old stuff before it went into HBase https://github.com/cloudera-labs/SparkOnHBase Let me know if that helps On Wed, Aug 26, 2015 at 5:40 AM, Ted Yu yuzhih...@gmail.com wrote: Can you log the contents of the Configuration you pass from Spark ? The output would give you some clue. Cheers On Aug 26, 2015, at 2:30 AM, Furkan KAMACI furkankam...@gmail.com wrote: Hi Ted, I'll check Zookeeper connection but another test method which runs on hbase without Spark works without any error. Hbase version is 0.98.8-hadoop2 and I use Spark 1.3.1 Kind Regards, Furkan KAMACI 26 Ağu 2015 12:08 tarihinde Ted Yu yuzhih...@gmail.com yazdı: The connection failure was to zookeeper. Have you verified that localhost:2181 can serve requests ? What version of hbase was Gora built against ? Cheers On Aug 26, 2015, at 1:50 AM, Furkan KAMACI furkankam...@gmail.com wrote: Hi, I start an Hbase cluster for my test class. I use that helper class: https://github.com/apache/gora/blob/master/gora-hbase/src/test/java/org/apache/gora/hbase/util/HBaseClusterSingleton.java and use it as like that: private static final HBaseClusterSingleton cluster = HBaseClusterSingleton.build(1); I retrieve configuration object as follows: cluster.getConf() and I use it at Spark as follows: sparkContext.newAPIHadoopRDD(conf, MyInputFormat.class, clazzK, clazzV); When I run my test there is no need to startup an Hbase cluster because Spark will connect to my dummy cluster. However when I run my test method it throws an error: 2015-08-26 01:19:59,558 INFO [Executor task launch worker-0-SendThread(localhost:2181)] zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(966)) - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2015-08-26 01:19:59,559 WARN [Executor task launch worker-0-SendThread(localhost:2181)] zookeeper.ClientCnxn (ClientCnxn.java:run(1089)) - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068) Hbase tests, which do not run on Spark, works well. When I check the logs I see that cluster and Spark is started up correctly: 2015-08-26 01:35:21,791 INFO [main] hdfs.MiniDFSCluster (MiniDFSCluster.java:waitActive(2055)) - Cluster is active 2015-08-26 01:35:40,334 INFO [main] util.Utils (Logging.scala:logInfo(59)) - Successfully started service 'sparkDriver' on port 56941. I realized that when I start up an hbase from command line my test method for Spark connects to it! So, does it means that it doesn't care about the conf I passed to it? Any ideas about how to solve it?
Re: Spark Cannot Connect to HBaseClusterSingleton
Hi Ted, I'll check Zookeeper connection but another test method which runs on hbase without Spark works without any error. Hbase version is 0.98.8-hadoop2 and I use Spark 1.3.1 Kind Regards, Furkan KAMACI 26 Ağu 2015 12:08 tarihinde Ted Yu yuzhih...@gmail.com yazdı: The connection failure was to zookeeper. Have you verified that localhost:2181 can serve requests ? What version of hbase was Gora built against ? Cheers On Aug 26, 2015, at 1:50 AM, Furkan KAMACI furkankam...@gmail.com wrote: Hi, I start an Hbase cluster for my test class. I use that helper class: https://github.com/apache/gora/blob/master/gora-hbase/src/test/java/org/apache/gora/hbase/util/HBaseClusterSingleton.java and use it as like that: private static final HBaseClusterSingleton cluster = HBaseClusterSingleton.build(1); I retrieve configuration object as follows: cluster.getConf() and I use it at Spark as follows: sparkContext.newAPIHadoopRDD(conf, MyInputFormat.class, clazzK, clazzV); When I run my test there is no need to startup an Hbase cluster because Spark will connect to my dummy cluster. However when I run my test method it throws an error: 2015-08-26 01:19:59,558 INFO [Executor task launch worker-0-SendThread(localhost:2181)] zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(966)) - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2015-08-26 01:19:59,559 WARN [Executor task launch worker-0-SendThread(localhost:2181)] zookeeper.ClientCnxn (ClientCnxn.java:run(1089)) - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068) Hbase tests, which do not run on Spark, works well. When I check the logs I see that cluster and Spark is started up correctly: 2015-08-26 01:35:21,791 INFO [main] hdfs.MiniDFSCluster (MiniDFSCluster.java:waitActive(2055)) - Cluster is active 2015-08-26 01:35:40,334 INFO [main] util.Utils (Logging.scala:logInfo(59)) - Successfully started service 'sparkDriver' on port 56941. I realized that when I start up an hbase from command line my test method for Spark connects to it! So, does it means that it doesn't care about the conf I passed to it? Any ideas about how to solve it?
Re: Spark Cannot Connect to HBaseClusterSingleton
Hi, Here is the MapReduceTestUtils.testSparkWordCount() https://github.com/kamaci/gora/blob/master/gora-core/src/test/java/org/apache/gora/mapreduce/MapReduceTestUtils.java#L108 Here is SparkWordCount https://github.com/kamaci/gora/blob/8f1acc6d4ef6c192e8fc06287558b7bc7c39b040/gora-core/src/examples/java/org/apache/gora/examples/spark/SparkWordCount.java Lastly, here is GoraSparkEngine: https://github.com/kamaci/gora/blob/master/gora-core/src/main/java/org/apache/gora/spark/GoraSparkEngine.java Kind Regards, Furkan KAMACI On Wed, Aug 26, 2015 at 4:40 PM, Ted Malaska ted.mala...@cloudera.com wrote: Where can I find the code for MapReduceTestUtils.testSparkWordCount? On Wed, Aug 26, 2015 at 9:29 AM, Furkan KAMACI furkankam...@gmail.com wrote: Hi, Here is the test method I've ignored due to Connection Refused problem failure: https://github.com/kamaci/gora/blob/master/gora-hbase/src/test/java/org/apache/gora/hbase/mapreduce/TestHBaseStoreWordCount.java#L65 I've implemented a Spark backend for Apache Gora as GSoC project and this is the latest obstacle that I should solve. If you can help me, you are welcome. Kind Regards, Furkan KAMACI On Wed, Aug 26, 2015 at 3:45 PM, Ted Malaska ted.mala...@cloudera.com wrote: I've always used HBaseTestingUtility and never really had much trouble. I use that for all my unit testing between Spark and HBase. Here are some code examples if your interested --Main HBase-Spark Module https://github.com/apache/hbase/tree/master/hbase-spark --Unit test that cover all basic connections https://github.com/apache/hbase/blob/master/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/HBaseContextSuite.scala --If you want to look at the old stuff before it went into HBase https://github.com/cloudera-labs/SparkOnHBase Let me know if that helps On Wed, Aug 26, 2015 at 5:40 AM, Ted Yu yuzhih...@gmail.com wrote: Can you log the contents of the Configuration you pass from Spark ? The output would give you some clue. Cheers On Aug 26, 2015, at 2:30 AM, Furkan KAMACI furkankam...@gmail.com wrote: Hi Ted, I'll check Zookeeper connection but another test method which runs on hbase without Spark works without any error. Hbase version is 0.98.8-hadoop2 and I use Spark 1.3.1 Kind Regards, Furkan KAMACI 26 Ağu 2015 12:08 tarihinde Ted Yu yuzhih...@gmail.com yazdı: The connection failure was to zookeeper. Have you verified that localhost:2181 can serve requests ? What version of hbase was Gora built against ? Cheers On Aug 26, 2015, at 1:50 AM, Furkan KAMACI furkankam...@gmail.com wrote: Hi, I start an Hbase cluster for my test class. I use that helper class: https://github.com/apache/gora/blob/master/gora-hbase/src/test/java/org/apache/gora/hbase/util/HBaseClusterSingleton.java and use it as like that: private static final HBaseClusterSingleton cluster = HBaseClusterSingleton.build(1); I retrieve configuration object as follows: cluster.getConf() and I use it at Spark as follows: sparkContext.newAPIHadoopRDD(conf, MyInputFormat.class, clazzK, clazzV); When I run my test there is no need to startup an Hbase cluster because Spark will connect to my dummy cluster. However when I run my test method it throws an error: 2015-08-26 01:19:59,558 INFO [Executor task launch worker-0-SendThread(localhost:2181)] zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(966)) - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2015-08-26 01:19:59,559 WARN [Executor task launch worker-0-SendThread(localhost:2181)] zookeeper.ClientCnxn (ClientCnxn.java:run(1089)) - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068) Hbase tests, which do not run on Spark, works well. When I check the logs I see that cluster and Spark is started up correctly: 2015-08-26 01:35:21,791 INFO [main] hdfs.MiniDFSCluster (MiniDFSCluster.java:waitActive(2055)) - Cluster is active 2015-08-26 01:35:40,334 INFO [main] util.Utils (Logging.scala:logInfo(59)) - Successfully started service 'sparkDriver' on port 56941. I realized that when I start up an hbase from command line my test method for Spark connects to it! So, does it means that it doesn't care about the conf I passed to it? Any ideas about how to solve it?
Re: Spark Cannot Connect to HBaseClusterSingleton
Btw, here is the source code of GoraInputFormat.java : https://github.com/kamaci/gora/blob/master/gora-core/src/main/java/org/apache/gora/mapreduce/GoraInputFormat.java 26 Ağu 2015 18:05 tarihinde Furkan KAMACI furkankam...@gmail.com yazdı: I'll send an e-mail to Gora dev list too and also attach my patch into my GSoC Jira issue you mentioned and then we can continue at there. Before I do that stuff, I wanted to get Spark dev community's ideas to solve my problem due to you may have faced such kind of problems before. 26 Ağu 2015 17:13 tarihinde Ted Yu yuzhih...@gmail.com yazdı: I found GORA-386 Gora Spark Backend Support Should the discussion be continued there ? Cheers On Wed, Aug 26, 2015 at 7:02 AM, Ted Malaska ted.mala...@cloudera.com wrote: Where is the input format class. When every I use the search on your github it says We couldn’t find any issues matching 'GoraInputFormat' On Wed, Aug 26, 2015 at 9:48 AM, Furkan KAMACI furkankam...@gmail.com wrote: Hi, Here is the MapReduceTestUtils.testSparkWordCount() https://github.com/kamaci/gora/blob/master/gora-core/src/test/java/org/apache/gora/mapreduce/MapReduceTestUtils.java#L108 Here is SparkWordCount https://github.com/kamaci/gora/blob/8f1acc6d4ef6c192e8fc06287558b7bc7c39b040/gora-core/src/examples/java/org/apache/gora/examples/spark/SparkWordCount.java Lastly, here is GoraSparkEngine: https://github.com/kamaci/gora/blob/master/gora-core/src/main/java/org/apache/gora/spark/GoraSparkEngine.java Kind Regards, Furkan KAMACI On Wed, Aug 26, 2015 at 4:40 PM, Ted Malaska ted.mala...@cloudera.com wrote: Where can I find the code for MapReduceTestUtils.testSparkWordCount? On Wed, Aug 26, 2015 at 9:29 AM, Furkan KAMACI furkankam...@gmail.com wrote: Hi, Here is the test method I've ignored due to Connection Refused problem failure: https://github.com/kamaci/gora/blob/master/gora-hbase/src/test/java/org/apache/gora/hbase/mapreduce/TestHBaseStoreWordCount.java#L65 I've implemented a Spark backend for Apache Gora as GSoC project and this is the latest obstacle that I should solve. If you can help me, you are welcome. Kind Regards, Furkan KAMACI On Wed, Aug 26, 2015 at 3:45 PM, Ted Malaska ted.mala...@cloudera.com wrote: I've always used HBaseTestingUtility and never really had much trouble. I use that for all my unit testing between Spark and HBase. Here are some code examples if your interested --Main HBase-Spark Module https://github.com/apache/hbase/tree/master/hbase-spark --Unit test that cover all basic connections https://github.com/apache/hbase/blob/master/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/HBaseContextSuite.scala --If you want to look at the old stuff before it went into HBase https://github.com/cloudera-labs/SparkOnHBase Let me know if that helps On Wed, Aug 26, 2015 at 5:40 AM, Ted Yu yuzhih...@gmail.com wrote: Can you log the contents of the Configuration you pass from Spark ? The output would give you some clue. Cheers On Aug 26, 2015, at 2:30 AM, Furkan KAMACI furkankam...@gmail.com wrote: Hi Ted, I'll check Zookeeper connection but another test method which runs on hbase without Spark works without any error. Hbase version is 0.98.8-hadoop2 and I use Spark 1.3.1 Kind Regards, Furkan KAMACI 26 Ağu 2015 12:08 tarihinde Ted Yu yuzhih...@gmail.com yazdı: The connection failure was to zookeeper. Have you verified that localhost:2181 can serve requests ? What version of hbase was Gora built against ? Cheers On Aug 26, 2015, at 1:50 AM, Furkan KAMACI furkankam...@gmail.com wrote: Hi, I start an Hbase cluster for my test class. I use that helper class: https://github.com/apache/gora/blob/master/gora-hbase/src/test/java/org/apache/gora/hbase/util/HBaseClusterSingleton.java and use it as like that: private static final HBaseClusterSingleton cluster = HBaseClusterSingleton.build(1); I retrieve configuration object as follows: cluster.getConf() and I use it at Spark as follows: sparkContext.newAPIHadoopRDD(conf, MyInputFormat.class, clazzK, clazzV); When I run my test there is no need to startup an Hbase cluster because Spark will connect to my dummy cluster. However when I run my test method it throws an error: 2015-08-26 01:19:59,558 INFO [Executor task launch worker-0-SendThread(localhost:2181)] zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(966)) - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2015-08-26 01:19:59,559 WARN [Executor task launch worker-0-SendThread(localhost:2181)] zookeeper.ClientCnxn (ClientCnxn.java:run(1089)) - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect
