Re: Spark Streaming + Kudu

Have you considered checking your session error count or pending errors in your while loop every so often? Can you identify where your code is hanging when the connection is lost (what line)? Mike On Mon, Mar 5, 2018 at 9:08 PM, Ravi Kanth wrote: > In addition to my

Re: Spark Streaming + Kudu

In addition to my previous comment, I raised a support ticket for this issue with Cloudera and one of the support person mentioned below, *"Thank you for clarifying, The exceptions are logged but not re-thrown to an upper layer, so that explains why the Spark application is not aware of the

Re: Spark Streaming + Kudu

Mike, Thanks for the information. But, once the connection to any of the Kudu servers is lost then there is no way I can have a control on the KuduSession object and so with getPendingErrors(). The KuduClient in this case is becoming a zombie and never returned back till the connection is

Re: Spark Streaming + Kudu

Hi Ravi, it would be helpful if you could attach what you are getting back from getPendingErrors() -- perhaps from dumping RowError.toString() from items in the returned array -- and indicate what you were hoping to get back. Note that a RowError can also return to you the Operation

Re: Spark Streaming + Kudu

Hi Mike, Thanks for the reply. Yes, I am using AUTO_FLUSH_BACKGROUND. So, I am trying to use Kudu Client API to perform UPSERT into Kudu and I integrated this with Spark. I am trying to test a case where in if any of Kudu server fails. So, in this case, if there is any problem in writing,

Re: Spark Streaming + Kudu

Hi Ravi, are you using AUTO_FLUSH_BACKGROUND ? You mention that you are trying to use getPendingErrors()

Re: Kudu as a Graphite backend

Hey Mark, Yea, I wrote the original Graphite integration in the samples repo several years ago (prior to Kudu 0.5 even), but it was more of a quick prototype in order to have a demo of the Python client rather than something meant to be used in a production scenario. Of course with some work it

Re: Problems connecting form Spark

Any chance you can try spark2-shell with Kudu 1.6 and then re-try your tests? spark-shell --packages org.apache.kudu:kudu-spark2_2.11:1.6.0 On Fri, Mar 2, 2018 at 5:02 AM, Saúl Nogueras wrote: > I cannot properly connect to Kudu from Spark, error says “Kudu master has > no

Kudu as a Graphite backend

Hi List, has anybody experiences running Kudu as a Graphite backend in production? I've been looking at the samples repository, but have been unsure, primarily because of the 'samples' tag associated with the code. Best, Mark -- Mark Meyer Systems Engineer mark.me...@smaato.com Smaato Inc. San