Re: Spark program not receiving messages from Cloud Pubsub

2022-08-09 Thread Pramod Biligiri
I was able to get it working. It needed a SparkSession to be instantiated and wait for termination signal from the user. In my case I used a StreamingContext - https://spark.apache.org/docs/2.2.0/api/java/org/apache/spark/streaming/StreamingContext.html Pramod Biligiri On Sun, Aug 7, 2022 at

Spark program not receiving messages from Cloud Pubsub

2022-08-06 Thread Pramod Biligiri
Hi, I have a simple Java program that reads messages from a Google Cloud Pubsub topic and prints them. It works correctly when I run the program as a standalone, but it fails to receive messages when run using spark-submit. It connects to the subscription using my authentication credentials, but