Unable to Connect to Apache Phoenix From Spark

Simple Java Program to Connect to Phoenix DB: SparkConf sparkConf = new SparkConf(); sparkConf.setAppName("Using-spark-phoenix-df"); sparkConf.setMaster("local[*]"); JavaSparkContext sc = new JavaSparkContext(sparkConf);

Spark with HBase on Spark Runtime 2.2.1

I wrote a simple program to read data from HBase, the program works find in Cloudera backed by HDFS. The program works fine on SPARK RUNTIME 1.6 on Cloudera. But does NOT work on EMR with Spark Runtime 2.2.1. But getting an exception while testing data on EMR with S3. // Spark conf

help needed in perforance improvement of spark structured streaming

Hi Community, I have a use case where i need to call stored procedure through structured streaming. I am able to send kafka message and call stored procedure , but since foreach sink keeps on executing stored procedure per message i want to combine all the messages in single dtaframe and then

Advice on multiple streaming job

Hi All, Need advice on executing multiple streaming jobs. Problem:- We have 100's of streaming job. Every streaming job uses new port. Also, Spark automatically checks port from 4040 to 4056, post that it fails. One of the workaround, is to provide port explicitly. Is there a way to tackle this