I am trying to stream the data from Kafka to Spark.

JavaPairInputDStream<String, String> directKafkaStream =
KafkaUtils.createDirectStream(ssc,
                String.class,
                String.class,
                StringDecoder.class,
                StringDecoder.class,
                kafkaParams, topics);

Here i am iterating over the JavaPairInputDStream to process the RDD's.

directKafkaStream.foreachRDD(rdd ->{
            rdd.foreachPartition(items ->{
                while (items.hasNext()) {
                    String[] State = items.next()._2.split("\\,");
                    System.out.println(State[2]+","+State[3]+","+State[4]+"--");
                };
            });
        });


In this i can able to access the String Array but when i am trying to
access the String Array data globally i can't access the data. Here my
requirement is if i had access these data globally i had another lookup
table in Hive. So i am trying to perform an operation on these. Any
suggestions please, Thanks.

-- 
Best Regards,
Sreeharsha Eedupuganti

Reply via email to