hi team any help with this
I have a use case where i need to call stored procedure through structured streaming. I am able to send kafka message and call stored procedure , but since foreach sink keeps on executing stored procedure per message i want to combine all the messages in single dtaframe and then call stored procedure at once is it possible to do current code select('value cast "string",'topic) .select('topic,concat_ws(",", 'value cast "string") as 'value1) .groupBy('topic cast "string").count() .coalesce(1) .as[String] .writeStream .trigger(ProcessingTime("60 seconds")) .option("checkpointLocation", checkpointUrl) .foreach(new SimpleSqlServerSink(jdbcUrl, connectionProperties)) On Sat, May 5, 2018 at 12:20 PM, amit kumar singh <amitiem...@gmail.com> wrote: > Hi Community, > > I have a use case where i need to call stored procedure through structured > streaming. > > I am able to send kafka message and call stored procedure , > > but since foreach sink keeps on executing stored procedure per message > > i want to combine all the messages in single dtaframe and then call > stored procedure at once > > is it possible to do > > > current code > > select('value cast "string",'topic) > .select('topic,concat_ws(",", 'value cast "string") as 'value1) > .groupBy('topic cast "string").count() > .coalesce(1) > .as[String] > .writeStream > .trigger(ProcessingTime("60 seconds")) > .option("checkpointLocation", checkpointUrl) > .foreach(new SimpleSqlServerSink(jdbcUrl, connectionProperties)) > > > > > thanks > rohit >