Hi Roshan,
How many analyzer nodes are there in the cluster? If the master count is
set to 1 and the master is down, spark cluster will not survive. It you set
the master count to 2, then if the master is down other node become the
master and it survive. However until spark context is initialize properly
in the other node(will take roughly about 5 - 30secs) you will see above
error.
On Sun, Jan 24, 2016 at 8:25 PM, Roshan Wijesena <ros...@wso2.com> wrote:
> Hi Niranda/DAS team,
>
> I have updated DAS server into 3.0.1. I am testing a minimum HA cluster
> when one server is in down situation. I am getting this exception
> periodically, and spark scripts are not running at all. it seems we can
> *not* survive when one server is in down situation?
>
> TID: [-1234] [] [2016-01-24 21:10:10,015] ERROR
> {org.wso2.carbon.analytics.spark.core.AnalyticsTask} - Error while
> executing the scheduled task for the script: is_log_analytics
> {org.wso2.carbon.analytics.spark.core.AnalyticsTask}
> org.wso2.carbon.analytics.spark.core.exception.AnalyticsExecutionException:
> Spark SQL Context is not available. Check if the cluster has instantiated
> properly.
> at
> org.wso2.carbon.analytics.spark.core.internal.SparkAnalyticsExecutor.executeQueryLocal(SparkAnalyticsExecutor.java:728)
> at
> org.wso2.carbon.analytics.spark.core.internal.SparkAnalyticsExecutor.executeQuery(SparkAnalyticsExecutor.java:709)
> at
> org.wso2.carbon.analytics.spark.core.CarbonAnalyticsProcessorService.executeQuery(CarbonAnalyticsProcessorService.java:201)
> at
> org.wso2.carbon.analytics.spark.core.CarbonAnalyticsProcessorService.executeScript(CarbonAnalyticsProcessorService.java:151)
> at
> org.wso2.carbon.analytics.spark.core.AnalyticsTask.execute(AnalyticsTask.java:59)
> at
> org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter.execute(TaskQuartzJobAdapter.java:67)
> at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
> -Roshan
>
>
>
>
>
>
>
> On Fri, Dec 11, 2015 at 1:08 AM, Roshan Wijesena <ros...@wso2.com> wrote:
>
>> Hi Niranda / Inosh,
>>
>> Thanks a lot for the quick call and reply. Yes issue seems to be fixed
>> now. Did not appear for a while.
>>
>> -Roshan
>>
>> On Fri, Dec 11, 2015 at 12:47 AM, Niranda Perera <nira...@wso2.com>
>> wrote:
>>
>>> Hi Roshan,
>>>
>>> This happens when you have a malformed HA cluster. When you put the
>>> master count as 2, the spark cluster would not get initiated until there
>>> are 2 members in the analytics cluster. when the count as 2 and there is a
>>> task scheduled already, you may come across this issue, until the 2nd node
>>> is up and running. You should see that after sometime, the exception gets
>>> resolved., and that is when the analytics cluster is at a workable state.
>>>
>>> But I agree, an NPE is not acceptable here and this has been already
>>> fixed in 3.0.1 [1]
>>>
>>> as per the query modification, yes, the query gets modified to handle
>>> multi tenancy in the spark runtime.
>>>
>>> hope this resolves your issues.
>>>
>>> rgds
>>>
>>> [1] https://wso2.org/jira/browse/DAS-329
>>>
>>> On Fri, Dec 11, 2015 at 11:40 AM, Roshan Wijesena <ros...@wso2.com>
>>> wrote:
>>>
>>>> I reproduced the error. If we set carbon.spark.master.count value to 2
>>>> this error will occur. Any solution available in this case?
>>>>
>>>>
>>>> On Thu, Dec 10, 2015 at 9:05 PM, Roshan Wijesena <ros...@wso2.com>
>>>> wrote:
>>>>
>>>>> After I enabled the debug, it looks like below
>>>>>
>>>>> [2015-12-10 22:03:00,001] INFO
>>>>> {org.wso2.carbon.analytics.spark.core.AnalyticsTask} - Executing the
>>>>> schedule task for: httpd_log_analytics for tenant id: -1234
>>>>> [2015-12-10 22:03:00,013] DEBUG
>>>>> {org.wso2.carbon.analytics.spark.core.internal.SparkAnalyticsExecutor} -
>>>>> Executing : CREATE TEMPORARY TABLE X1234_HttpLogTable USING
>>>>> org.wso2.carbon.analytics.spark.core.sources.AnalyticsRelationProvider
>>>>> OPTIONS (tableName "ORG_WSO2_SAMPLE_HTTPD_LOGS" , tenantId "-1234")
>>>>> [2015-12-10 22:03:00,013] ERROR
>>>>> {org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter} - Error in
>>>>> executing task: null
>>>>> java.lang.NullPointerException
>>>>> at
>>>>> org.wso2.carbon.analytics.spark.core.internal.SparkAnalyticsExecutor.executeQueryLocal(SparkAnalyticsExecutor.java:710)
>>>>> at
>>>>> org.wso2.carbon.analytics.spark.core.internal.SparkAnalyticsExecutor.executeQuery(SparkAnalyticsExecutor.java:692)
>>>>> at
>>>>> org.wso2.carbon.analytics.spark.core.CarbonAnalyticsProcessorService.executeQuery(CarbonAnalyticsProcessorService.java:199)
>>>>> at
>>>>> org.wso2.carbon.analytics.spark.core.CarbonAnalyticsProcessorService.executeScript(CarbonAnalyticsProcessorService.java:149)
>>>>> at
>>>>> org.wso2.carbon.analytics.spark.core.AnalyticsTask.execute(AnalyticsTask.java:57)
>>>>> at
>>>>> org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter.execute(TaskQuartzJobAdapter.java:67)
>>>>> at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
>>>>> at
>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>>> at
>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>>> at
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>
>>>>> does that query got modified?
>>>>>
>>>>> CREATE TEMPORARY TABLE X1234_HttpLogTable USING
>>>>> org.wso2.carbon.analytics.spark.core.sources.AnalyticsRelationProvider
>>>>> OPTIONS (tableName "ORG_WSO2_SAMPLE_HTTPD_LOGS" , tenantId "-1234")
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Dec 10, 2015 at 6:53 PM, Roshan Wijesena <ros...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> Hi DAS teaam,
>>>>>>
>>>>>> I am getting below null pointer exception while trying to execute a
>>>>>> scheduled task. What I simply did was created a two node HA cluster and
>>>>>> tried to run this example[1]. It has a scheduled task. However, this
>>>>>> error
>>>>>> can not be observed in a single node fresh pack.
>>>>>>
>>>>>> The error is,
>>>>>>
>>>>>> [2015-12-10 19:32:00,573] INFO
>>>>>> {org.wso2.carbon.analytics.spark.core.AnalyticsTask} - Executing the
>>>>>> schedule task for: httpd_log_analytics for tenant id: -1234
>>>>>> [2015-12-10 19:32:21,899] INFO
>>>>>> {org.wso2.carbon.event.processor.manager.core.internal.CarbonEventManagementService}
>>>>>> - Starting polling event receivers
>>>>>> [2015-12-10 19:32:32,096] ERROR
>>>>>> {org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter} - Error in
>>>>>> executing task: null
>>>>>> java.lang.NullPointerException
>>>>>> at
>>>>>> org.wso2.carbon.analytics.spark.core.internal.SparkAnalyticsExecutor.executeQueryLocal(SparkAnalyticsExecutor.java:710)
>>>>>> at
>>>>>> org.wso2.carbon.analytics.spark.core.internal.SparkAnalyticsExecutor.executeQuery(SparkAnalyticsExecutor.java:692)
>>>>>> at
>>>>>> org.wso2.carbon.analytics.spark.core.CarbonAnalyticsProcessorService.executeQuery(CarbonAnalyticsProcessorService.java:199)
>>>>>> at
>>>>>> org.wso2.carbon.analytics.spark.core.CarbonAnalyticsProcessorService.executeScript(CarbonAnalyticsProcessorService.java:149)
>>>>>> at
>>>>>> org.wso2.carbon.analytics.spark.core.AnalyticsTask.execute(AnalyticsTask.java:57)
>>>>>> at
>>>>>> org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter.execute(TaskQuartzJobAdapter.java:67)
>>>>>> at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
>>>>>> at
>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>>>> at
>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>>>> at
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>>
>>>>>>
>>>>>> [1] https://docs.wso2.com/display/DAS300/Analyzing+HTTPD+Logs
>>>>>>
>>>>>> --
>>>>>> Roshan Wijesena.
>>>>>> Senior Software Engineer-WSO2 Inc.
>>>>>> Mobile: *+94719154640 <%2B94719154640>*
>>>>>> Email: ros...@wso2.com
>>>>>> *WSO2, Inc. :** wso2.com <http://wso2.com/>*
>>>>>> lean.enterprise.middleware.
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Roshan Wijesena.
>>>>> Senior Software Engineer-WSO2 Inc.
>>>>> Mobile: *+94719154640 <%2B94719154640>*
>>>>> Email: ros...@wso2.com
>>>>> *WSO2, Inc. :** wso2.com <http://wso2.com/>*
>>>>> lean.enterprise.middleware.
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Roshan Wijesena.
>>>> Senior Software Engineer-WSO2 Inc.
>>>> Mobile: *+94719154640 <%2B94719154640>*
>>>> Email: ros...@wso2.com
>>>> *WSO2, Inc. :** wso2.com <http://wso2.com/>*
>>>> lean.enterprise.middleware.
>>>>
>>>
>>>
>>>
>>> --
>>> *Niranda Perera*
>>> Software Engineer, WSO2 Inc.
>>> Mobile: +94-71-554-8430
>>> Twitter: @n1r44 <https://twitter.com/N1R44>
>>> https://pythagoreanscript.wordpress.com/
>>>
>>
>>
>>
>> --
>> Roshan Wijesena.
>> Senior Software Engineer-WSO2 Inc.
>> Mobile: *+94719154640 <%2B94719154640>*
>> Email: ros...@wso2.com
>> *WSO2, Inc. :** wso2.com <http://wso2.com/>*
>> lean.enterprise.middleware.
>>
>
>
>
> --
> Roshan Wijesena.
> Senior Software Engineer-WSO2 Inc.
> Mobile: *+94719154640 <%2B94719154640>*
> Email: ros...@wso2.com
> *WSO2, Inc. :** wso2.com <http://wso2.com/>*
> lean.enterprise.middleware.
>
--
Thanks & Regards,
Inosh Goonewardena
Associate Technical Lead- WSO2 Inc.
Mobile: +94779966317
_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
