Re: [Dev] Web-Deployment order in an OSGI execution flow

[Thilini Cooray]

Hi Ayyoob,

[1] has explained the carbon architecture and its workflow. In the "How
server starts" section, it has mentioned that webapp deployment will take
place after loading OSGI bundles.

[1]
http://wso2.com/library/articles/2014/01/understanding-wso2-carbon-architecture/

Thanks.

On Sat, Jul 25, 2015 at 1:45 AM, Ruwan Abeykoon  wrote:

> Hi Ayoob,
>
> I am not sure this answers your question.
> However I observed following duplicate exports in your bundle. Please try
> to remove one of them manually and check.
>
> javax.xml.soap; version="0.0.0"
> javax.xml.soap; version="1.0.0"
>
> org.apache.ws.security.handler;
> version="1.5.11.wso2v9"
> org.apache.ws.security.handler;
> version="1.5.11.wso2v9"
>
>
> osgi>
> osgi> p javax.xml.soap
> javax.xml.soap; version="0.0.0"
>   org.eclipse.core.expressions_3.4.500.v20130515-1343 [78] imports
>   org.eclipse.core.runtime_3.9.0.v20130326-1255 [80] imports
>   org.wso2.carbon.p2.touchpoint_4.4.0 [290] imports
>   tomcat_7.0.59.wso2v1 [390] imports
> javax.xml.soap; version="1.0.0"
>   axis2_1.6.1.wso2v12 [14] imports
>   org.wso2.carbon.apimgt.core_1.4.0 [154] imports
>   spring.framework_3.2.9.wso2v1 [387] imports
>   wss4j_1.5.11.wso2v10 [401] imports
>   wss4j_1.5.11.wso2v9 [402] imports
> osgi>
> osgi>
> osgi>
> osgi> p org.apache.ws.security.handler
> org.apache.ws.security.handler;
> version="1.5.11.wso2v9"
>   org.wso2.carbon.security.mgt_4.4.0 [331] imports
>   rampart-core_1.6.1.wso2v14 [377] imports
>   rampart-trust_1.6.1.wso2v14 [379] imports
> org.apache.ws.security.handler;
> version="1.5.11.wso2v9"
> osgi>
>
> Cheers,
> Ruwan
>
> On Fri, Jul 24, 2015 at 3:22 PM, Ayyoob Hamza  wrote:
>
>> Hi All,
>> In Carbon, when the webdeployer features are installed, will the web apps
>> be deployed after all the osgi's are activated or will they run soon after
>> the tomcat is activated ?.
>> Is there a such a order ?
>>
>> Thanks,
>> *Ayyoob Hamza*
>> *Software Engineer*
>> WSO2 Inc.; http://wso2.com
>> email: ayy...@wso2.com cell: +94 77 1681010 <%2B94%2077%207779495>
>>
>> ___
>> Dev mailing list
>> Dev@wso2.org
>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>
>>
>
>
> --
>
> *Ruwan Abeykoon*
> *Architect,*
> *WSO2, Inc. http://wso2.com  *
> *lean.enterprise.middleware.*
>
> email: ruw...@wso2.com
> phone:(+94) 39736
>
> ___
> Dev mailing list
> Dev@wso2.org
> http://wso2.org/cgi-bin/mailman/listinfo/dev
>
>


-- 
Best Regards,

*Thilini Cooray*
Software Engineer
Mobile : +94 (0) 774 570 112 <%2B94%20%280%29%20773%20451194>
E-mail : thili...@wso2.com

WSO2 Inc. www.wso2.com
lean.enterprise.middleware
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Dev] [BPMN-Explorer] Improving Page redirect.

[Hasitha Aravinda]

Hi Amal,

Use case 1: Sometimes users will require to embedded task url ( Like [1] )
in emails. ( Eg: Task Reminder email etc). When user click on these links,
BPMN webapp redirect user back to webapp login page, if user didn't login
already.
But after user login, webapp doesn't redirect back to task page, instead it
will redirect to home page.

Use case 2: When a user tries to view invalid or non existing task id ( eg:
task?id=53016. ; task?id=12334566223 ), webapp shows 500 page. But 500 page
doesn't have any home page link. Btw I think 404 - not found status code is
correct status code to return.

[1] https://localhost:9443/bpmn-explorer/task?id=53016

Can we fix these, if time permits.

Thanks,
Hasitha.

-- 
Hasitha Aravinda,
Senior Software Engineer,
WSO2 Inc.
Email: hasi...@wso2.com
Mobile : +1 201 887 1971, +94 718 210 200
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [BPMN-Explorer] Improving Page redirect.

[Hasitha Aravinda]

​Created BPS-750 & BPS-751 for this.

Thanks,
Hasitha. ​

On Sun, Jul 26, 2015 at 11:04 AM, Hasitha Aravinda  wrote:

> Hi Amal,
>
> Use case 1: Sometimes users will require to embedded task url ( Like [1] )
> in emails. ( Eg: Task Reminder email etc). When user click on these links,
> BPMN webapp redirect user back to webapp login page, if user didn't login
> already.
> But after user login, webapp doesn't redirect back to task page, instead
> it will redirect to home page.
>
> Use case 2: When a user tries to view invalid or non existing task id (
> eg: task?id=53016. ; task?id=12334566223 ), webapp shows 500 page. But 500
> page doesn't have any home page link. Btw I think 404 - not found status
> code is correct status code to return.
>
> [1] https://localhost:9443/bpmn-explorer/task?id=53016
>
> Can we fix these, if time permits.
>
> Thanks,
> Hasitha.
>
> --
> Hasitha Aravinda,
> Senior Software Engineer,
> WSO2 Inc.
> Email: hasi...@wso2.com
> Mobile : +1 201 887 1971, +94 718 210 200
>



-- 
Hasitha Aravinda,
Senior Software Engineer,
WSO2 Inc.
Email: hasi...@wso2.com
Mobile : +1 201 887 1971, +94 718 210 200
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Dev] [Dev}[ESB][Carbon]

[Viraj Senevirathne]

Hi all,

Hi this is issue came up with https://wso2.org/jira/browse/ESBJAVA-3364.

When statistics are enabled and persistence is enabled every time ESB tried
to send statistics to Carbon Registry following error comes up. I came
across many Jiras with the similar issues. Is this a problem with carbon
configuration? An y help will be useful.

[2015-07-26 21:10:16,304] ERROR - MediationStatisticsObserver Error while
accessing the registry for statistics persistence
java.lang.NullPointerException: Tenant domain has not been set in
CarbonContext
at
org.wso2.carbon.caching.impl.CacheManagerFactoryImpl.getCacheManager(CacheManagerFactoryImpl.java:79)
at
org.wso2.carbon.registry.core.utils.RegistryUtils.getCacheManager(RegistryUtils.java:324)
at
org.wso2.carbon.registry.core.utils.RegistryUtils.getResourcePathCache(RegistryUtils.java:307)
at
org.wso2.carbon.registry.core.caching.PathCache.getCache(PathCache.java:35)
at
org.wso2.carbon.registry.core.jdbc.dao.JDBCPathCache.getPathID(JDBCPathCache.java:298)
at
org.wso2.carbon.registry.core.jdbc.dao.JDBCResourceDAO.getResourceID(JDBCResourceDAO.java:67)
at
org.wso2.carbon.registry.core.jdbc.dao.JDBCResourceDAO.resourceExists(JDBCResourceDAO.java:151)
at
org.wso2.carbon.registry.core.jdbc.Repository.resourceExists(Repository.java:134)
at
org.wso2.carbon.registry.core.jdbc.EmbeddedRegistry.put(EmbeddedRegistry.java:688)
at
org.wso2.carbon.registry.core.caching.CacheBackedRegistry.put(CacheBackedRegistry.java:508)
at
org.wso2.carbon.registry.core.session.UserRegistry.putInternal(UserRegistry.java:824)
at
org.wso2.carbon.registry.core.session.UserRegistry.access$1000(UserRegistry.java:60)
at
org.wso2.carbon.registry.core.session.UserRegistry$11.run(UserRegistry.java:800)
at
org.wso2.carbon.registry.core.session.UserRegistry$11.run(UserRegistry.java:797)
at java.security.AccessController.doPrivileged(Native Method)
at
org.wso2.carbon.registry.core.session.UserRegistry.put(UserRegistry.java:797)
at
org.wso2.carbon.mediation.statistics.persistence.PersistingStatisticsObserver.persistRecord(PersistingStatisticsObserver.java:113)
at
org.wso2.carbon.mediation.statistics.persistence.PersistingStatisticsObserver.access$400(PersistingStatisticsObserver.java:41)
at
org.wso2.carbon.mediation.statistics.persistence.PersistingStatisticsObserver$RegistryWriter.run(PersistingStatisticsObserver.java:158)

Thank you,

-- 
Viraj Senevirathne
Software Engineer; WSO2, Inc.

Mobile : +94 71 818 4742 <%2B94%20%280%29%20773%20451194>
Email : vir...@wso2.com 
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [BPMN-Explorer] Improving Page redirect.

[Milinda Perera]

Of course, this should add to humantask ui also. We can fix this simply
adding query parameter in login redirection and checking for request url in
refer header as below:


Thanks,
Milinda

On Sun, Jul 26, 2015 at 8:42 PM, Hasitha Aravinda  wrote:

> ​Created BPS-750 & BPS-751 for this.
>
> Thanks,
> Hasitha. ​
>
> On Sun, Jul 26, 2015 at 11:04 AM, Hasitha Aravinda 
> wrote:
>
>> Hi Amal,
>>
>> Use case 1: Sometimes users will require to embedded task url ( Like [1]
>> ) in emails. ( Eg: Task Reminder email etc). When user click on these
>> links, BPMN webapp redirect user back to webapp login page, if user didn't
>> login already.
>> But after user login, webapp doesn't redirect back to task page, instead
>> it will redirect to home page.
>>
>> Use case 2: When a user tries to view invalid or non existing task id (
>> eg: task?id=53016. ; task?id=12334566223 ), webapp shows 500 page. But 500
>> page doesn't have any home page link. Btw I think 404 - not found status
>> code is correct status code to return.
>>
>> [1] https://localhost:9443/bpmn-explorer/task?id=53016
>>
>> Can we fix these, if time permits.
>>
>> Thanks,
>> Hasitha.
>>
>> --
>> Hasitha Aravinda,
>> Senior Software Engineer,
>> WSO2 Inc.
>> Email: hasi...@wso2.com
>> Mobile : +1 201 887 1971, +94 718 210 200
>>
>
>
>
> --
> Hasitha Aravinda,
> Senior Software Engineer,
> WSO2 Inc.
> Email: hasi...@wso2.com
> Mobile : +1 201 887 1971, +94 718 210 200
>



-- 
Milinda Perera
Software Engineer;
WSO2 Inc. http://wso2.com ,
Mobile: (+94) 714 115 032
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [BPMN-Explorer] Improving Page redirect.

[Milinda Perera]

[resending missed content]
Of course, this should add to humantask ui also. We can fix this simply
adding query parameter in login redirection and checking for request url in
refer header as below:

   1. Referer:

   
https://192.168.1.2:9443/humantask-explorer/login?requestUrl=https://192.168.1.2:9443/humantask-explorer/task?id=2352

Thanks,
Milinda

On Sun, Jul 26, 2015 at 11:36 PM, Milinda Perera  wrote:

> Of course, this should add to humantask ui also. We can fix this simply
> adding query parameter in login redirection and checking for request url in
> refer header as below:
>
>
> Thanks,
> Milinda
>
> On Sun, Jul 26, 2015 at 8:42 PM, Hasitha Aravinda 
> wrote:
>
>> ​Created BPS-750 & BPS-751 for this.
>>
>> Thanks,
>> Hasitha. ​
>>
>> On Sun, Jul 26, 2015 at 11:04 AM, Hasitha Aravinda 
>> wrote:
>>
>>> Hi Amal,
>>>
>>> Use case 1: Sometimes users will require to embedded task url ( Like [1]
>>> ) in emails. ( Eg: Task Reminder email etc). When user click on these
>>> links, BPMN webapp redirect user back to webapp login page, if user didn't
>>> login already.
>>> But after user login, webapp doesn't redirect back to task page, instead
>>> it will redirect to home page.
>>>
>>> Use case 2: When a user tries to view invalid or non existing task id (
>>> eg: task?id=53016. ; task?id=12334566223 ), webapp shows 500 page. But 500
>>> page doesn't have any home page link. Btw I think 404 - not found status
>>> code is correct status code to return.
>>>
>>> [1] https://localhost:9443/bpmn-explorer/task?id=53016
>>>
>>> Can we fix these, if time permits.
>>>
>>> Thanks,
>>> Hasitha.
>>>
>>> --
>>> Hasitha Aravinda,
>>> Senior Software Engineer,
>>> WSO2 Inc.
>>> Email: hasi...@wso2.com
>>> Mobile : +1 201 887 1971, +94 718 210 200
>>>
>>
>>
>>
>> --
>> Hasitha Aravinda,
>> Senior Software Engineer,
>> WSO2 Inc.
>> Email: hasi...@wso2.com
>> Mobile : +1 201 887 1971, +94 718 210 200
>>
>
>
>
> --
> Milinda Perera
> Software Engineer;
> WSO2 Inc. http://wso2.com ,
> Mobile: (+94) 714 115 032
>
>


-- 
Milinda Perera
Software Engineer;
WSO2 Inc. http://wso2.com ,
Mobile: (+94) 714 115 032
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] Web-Deployment order in an OSGI execution flow

[Ayyoob Hamza]

Hi Ruwan,
Thanks for pointing out the duplications. I tried it and I am still getting
the No class def found on the referred bundle classes. And this behaviour
is random. This is why, it made me think to know whether is it always
guaranteed that the war deployment happens after the osgi bundle activation.

Thanks
*Ayyoob Hamza*
*Software Engineer*
WSO2 Inc.; http://wso2.com
email: ayy...@wso2.com cell: +94 77 1681010 <%2B94%2077%207779495>

On Sat, Jul 25, 2015 at 1:45 AM, Ruwan Abeykoon  wrote:

> Hi Ayoob,
>
> I am not sure this answers your question.
> However I observed following duplicate exports in your bundle. Please try
> to remove one of them manually and check.
>
> javax.xml.soap; version="0.0.0"
> javax.xml.soap; version="1.0.0"
>
> org.apache.ws.security.handler;
> version="1.5.11.wso2v9"
> org.apache.ws.security.handler;
> version="1.5.11.wso2v9"
>
>
> osgi>
> osgi> p javax.xml.soap
> javax.xml.soap; version="0.0.0"
>   org.eclipse.core.expressions_3.4.500.v20130515-1343 [78] imports
>   org.eclipse.core.runtime_3.9.0.v20130326-1255 [80] imports
>   org.wso2.carbon.p2.touchpoint_4.4.0 [290] imports
>   tomcat_7.0.59.wso2v1 [390] imports
> javax.xml.soap; version="1.0.0"
>   axis2_1.6.1.wso2v12 [14] imports
>   org.wso2.carbon.apimgt.core_1.4.0 [154] imports
>   spring.framework_3.2.9.wso2v1 [387] imports
>   wss4j_1.5.11.wso2v10 [401] imports
>   wss4j_1.5.11.wso2v9 [402] imports
> osgi>
> osgi>
> osgi>
> osgi> p org.apache.ws.security.handler
> org.apache.ws.security.handler;
> version="1.5.11.wso2v9"
>   org.wso2.carbon.security.mgt_4.4.0 [331] imports
>   rampart-core_1.6.1.wso2v14 [377] imports
>   rampart-trust_1.6.1.wso2v14 [379] imports
> org.apache.ws.security.handler;
> version="1.5.11.wso2v9"
> osgi>
>
> Cheers,
> Ruwan
>
> On Fri, Jul 24, 2015 at 3:22 PM, Ayyoob Hamza  wrote:
>
>> Hi All,
>> In Carbon, when the webdeployer features are installed, will the web apps
>> be deployed after all the osgi's are activated or will they run soon after
>> the tomcat is activated ?.
>> Is there a such a order ?
>>
>> Thanks,
>> *Ayyoob Hamza*
>> *Software Engineer*
>> WSO2 Inc.; http://wso2.com
>> email: ayy...@wso2.com cell: +94 77 1681010 <%2B94%2077%207779495>
>>
>> ___
>> Dev mailing list
>> Dev@wso2.org
>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>
>>
>
>
> --
>
> *Ruwan Abeykoon*
> *Architect,*
> *WSO2, Inc. http://wso2.com  *
> *lean.enterprise.middleware.*
>
> email: ruw...@wso2.com
> phone:(+94) 39736
>
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] Web-Deployment order in an OSGI execution flow

[Ayyoob Hamza]

Hi Thilini
As in this link (Under How tomcat works) it suggest that the war deployment
happens after the tomcat bundle get activated, not the end of all osgi
bundle activated.

If thats the case how can we guarantee that the web deployment can only be
happen after the osgi bundle activation ?.
[ 1 ]
http://wso2.com/library/articles/2014/01/understanding-wso2-carbon-architecture/
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [ML] Update - Deeplearning Integration to WSO2-ml

[Thushan Ganegedara]

Hi all,

I'm doing some tests with several datasets and most of them seemed to be
working fine. Somehow, I stumbled upon the leaf dataset (
https://archive.ics.uci.edu/ml/datasets/Leaf), which does not seem to be
working well for. However, the dataset works fine with other algorithms
(e.g. Logistic Regression L-BFGS) Therefore, I suspect this is due to some
sort of malformed data format. I'm right now looking into that.

Furthermore, I am thinking of starting with the D3 visualization on the
parameter setting stage. Should we be moving forward with that idea?

Finally, I would like to remind that, we haven't decided a date for code
review. Should we do that?

Thank you

On Wed, Jul 22, 2015 at 11:13 PM, Thushan Ganegedara 
wrote:

> Hi,
>
> Apologies about the late reply.
>
> Notes of the Demonstration
>
> Time duration: approximately 30 mins
>
> The demonstration was to demonstrate the implemented deeplearning feature
> of WSO2-ML. The demo started first explaining the dataset used (i.e.
> MNIST). The dataset is a CSV file with approximately 3 rows and 784
> features.
>
> Next the dataset was loaded to WSO2-ml. Here a concern was raised
> regarding selecting the type of data in the Preprocessing Phase (i.e.
> Categorical vs Numerical) The suggestion was that there should be a UI
> feature to change the data type for all the variables at once (very useful
> for large amounts of features).
>
> Next the deeplearning algorithm for MNIST dataset was demonstrated and was
> able to achieve an appx 95% accuracy. Regarding the deeplearning
> algorithms, H-2-O doesn't seem to have different deeplearning algorithms at
> the moment, but a general deep network + classifier (probably autoencoder).
> So the idea was to ask H-2-O team whether they are planning to implement
> different networks in the future.
>
> Also, it was suggested to add a visualization feature in parameter setting
> stage to provide a summarized visualization of the network to the user.
>
> Furthermore, another suggestion was to test the deep network on real world
> datasets and see how it performs. For this datasets from Kaggle will be
> used.
>
>
> About progress.
>
> I'm currently testing the algorithm against different datasets. and I'll
> provide a detailed report on that in the recent future.
>
> Thank you
>
>
>
> On Wed, Jul 22, 2015 at 2:00 PM, Nirmal Fernando  wrote:
>
>> @Thushan how are you progressing? Could you please send the notes of our
>> last review?
>>
>> On Thu, Jul 16, 2015 at 10:43 AM, CD Athuraliya 
>> wrote:
>>
>>>
>>>
>>> On Mon, Jul 13, 2015 at 11:12 AM, Thushan Ganegedara 
>>> wrote:
>>>
 Hello CD,

 Yes, it seems to be working fine now. But why does it show the axes in
 meters? Is this a d3 specific thing?

>>>
>>> I think *m* stands for *Milli* here.
>>>

 On Mon, Jul 13, 2015 at 3:17 PM, Thushan Ganegedara 
 wrote:

> Hi all,
>
> Thank you very much for pointing out. I'll get the latest update and
> see.
>
> On Mon, Jul 13, 2015 at 3:03 PM, CD Athuraliya 
> wrote:
>
>> Hi Thushan,
>>
>> That method has been updated. Please get the latest. You might have
>> to define your own case depending on predicted values.
>>
>> CD Athuraliya
>> Sent from my mobile device
>> On Jul 13, 2015 10:24 AM, "Nirmal Fernando"  wrote:
>>
>>> Great work Thushan! On the UI issues, @CD could help you. AFAIK
>>> actual keeps the pointer to the actual label and predicted is the
>>> probability and predictedLabel is after rounding it using a threshold.
>>>
>>> On Mon, Jul 13, 2015 at 7:14 AM, Thushan Ganegedara <
>>> thu...@gmail.com> wrote:
>>>
 Hi all,

 I have integrated H-2-O deeplearning to WSO2-ml successfully.
 Following are the stats on 2 tests conducted (screenshots attached).

 Iris dataset - 93.62% Accuracy
 MNIST (Small) dataset - 94.94% Accuracy

 However, there were few unusual issues that I had to spend lot of
 time to identify.

 *FrameSplitter does not work for any value other than 0.5. Any
 value other than 0.5, the following error is returned*
 (Frame splitter is used to split trainingData to train and valid
 sets)
 barrier onExCompletion for
 hex.deeplearning.DeepLearning$DeepLearningDriver@25e994ae
 ​java.lang.RuntimeException: java.lang.RuntimeException:
 java.lang.NullPointerException
 at
 hex.deeplearning.DeepLearning$DeepLearningDriver.trainModel(DeepLearning.java:382)​

 *​DeepLearningModel.score(double[] vec) method doesn't work. *
 The predictions obtained with ​score(Frame f) and score(double[] v)
 is shown below.

 *Actual, score(Frame f), score(double[] v)*
 ​0.0, 0.0, 1.0
 1.0, 1.0, 2.0
 2.0, 2.0, 2.0
 2.0, 1.0, 2.0
 1.0, 1.0, 2.0

Re: [Dev] [ML] Update - Deeplearning Integration to WSO2-ml

[Nirmal Fernando]

Thanks Thushan for the update.

On Mon, Jul 27, 2015 at 6:01 AM, Thushan Ganegedara 
wrote:

> Hi all,
>
> I'm doing some tests with several datasets and most of them seemed to be
> working fine. Somehow, I stumbled upon the leaf dataset (
> https://archive.ics.uci.edu/ml/datasets/Leaf), which does not seem to be
> working well for. However, the dataset works fine with other algorithms
> (e.g. Logistic Regression L-BFGS) Therefore, I suspect this is due to some
> sort of malformed data format. I'm right now looking into that.
>
> Furthermore, I am thinking of starting with the D3 visualization on the
> parameter setting stage. Should we be moving forward with that idea?
>
+1

>
> Finally, I would like to remind that, we haven't decided a date for code
> review. Should we do that?
>

Yes, let's have it this week, if you are ok.

>
> Thank you
>
> On Wed, Jul 22, 2015 at 11:13 PM, Thushan Ganegedara 
> wrote:
>
>> Hi,
>>
>> Apologies about the late reply.
>>
>> Notes of the Demonstration
>>
>> Time duration: approximately 30 mins
>>
>> The demonstration was to demonstrate the implemented deeplearning feature
>> of WSO2-ML. The demo started first explaining the dataset used (i.e.
>> MNIST). The dataset is a CSV file with approximately 3 rows and 784
>> features.
>>
>> Next the dataset was loaded to WSO2-ml. Here a concern was raised
>> regarding selecting the type of data in the Preprocessing Phase (i.e.
>> Categorical vs Numerical) The suggestion was that there should be a UI
>> feature to change the data type for all the variables at once (very useful
>> for large amounts of features).
>>
>> Next the deeplearning algorithm for MNIST dataset was demonstrated and
>> was able to achieve an appx 95% accuracy. Regarding the deeplearning
>> algorithms, H-2-O doesn't seem to have different deeplearning algorithms at
>> the moment, but a general deep network + classifier (probably autoencoder).
>> So the idea was to ask H-2-O team whether they are planning to implement
>> different networks in the future.
>>
>> Also, it was suggested to add a visualization feature in parameter
>> setting stage to provide a summarized visualization of the network to the
>> user.
>>
>> Furthermore, another suggestion was to test the deep network on real
>> world datasets and see how it performs. For this datasets from Kaggle will
>> be used.
>>
>>
>> About progress.
>>
>> I'm currently testing the algorithm against different datasets. and I'll
>> provide a detailed report on that in the recent future.
>>
>> Thank you
>>
>>
>>
>> On Wed, Jul 22, 2015 at 2:00 PM, Nirmal Fernando  wrote:
>>
>>> @Thushan how are you progressing? Could you please send the notes of our
>>> last review?
>>>
>>> On Thu, Jul 16, 2015 at 10:43 AM, CD Athuraliya 
>>> wrote:
>>>


 On Mon, Jul 13, 2015 at 11:12 AM, Thushan Ganegedara 
 wrote:

> Hello CD,
>
> Yes, it seems to be working fine now. But why does it show the axes in
> meters? Is this a d3 specific thing?
>

 I think *m* stands for *Milli* here.

>
> On Mon, Jul 13, 2015 at 3:17 PM, Thushan Ganegedara 
> wrote:
>
>> Hi all,
>>
>> Thank you very much for pointing out. I'll get the latest update and
>> see.
>>
>> On Mon, Jul 13, 2015 at 3:03 PM, CD Athuraliya 
>> wrote:
>>
>>> Hi Thushan,
>>>
>>> That method has been updated. Please get the latest. You might have
>>> to define your own case depending on predicted values.
>>>
>>> CD Athuraliya
>>> Sent from my mobile device
>>> On Jul 13, 2015 10:24 AM, "Nirmal Fernando"  wrote:
>>>
 Great work Thushan! On the UI issues, @CD could help you. AFAIK
 actual keeps the pointer to the actual label and predicted is the
 probability and predictedLabel is after rounding it using a threshold.

 On Mon, Jul 13, 2015 at 7:14 AM, Thushan Ganegedara <
 thu...@gmail.com> wrote:

> Hi all,
>
> I have integrated H-2-O deeplearning to WSO2-ml successfully.
> Following are the stats on 2 tests conducted (screenshots attached).
>
> Iris dataset - 93.62% Accuracy
> MNIST (Small) dataset - 94.94% Accuracy
>
> However, there were few unusual issues that I had to spend lot of
> time to identify.
>
> *FrameSplitter does not work for any value other than 0.5. Any
> value other than 0.5, the following error is returned*
> (Frame splitter is used to split trainingData to train and valid
> sets)
> barrier onExCompletion for
> hex.deeplearning.DeepLearning$DeepLearningDriver@25e994ae
> ​java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.NullPointerException
> at
> hex.deeplearning.DeepLearning$DeepLearningDriver.trainModel(DeepLearning.java:382)​
>
> *​DeepLearningModel.score(double[] vec) met

Re: [Dev] [ML] Update - Deeplearning Integration to WSO2-ml

[Thushan Ganegedara]

Hi,

Yes, I'm alright with the idea of having the review this week.



On Mon, Jul 27, 2015 at 11:27 AM, Nirmal Fernando  wrote:

> Thanks Thushan for the update.
>
> On Mon, Jul 27, 2015 at 6:01 AM, Thushan Ganegedara 
> wrote:
>
>> Hi all,
>>
>> I'm doing some tests with several datasets and most of them seemed to be
>> working fine. Somehow, I stumbled upon the leaf dataset (
>> https://archive.ics.uci.edu/ml/datasets/Leaf), which does not seem to be
>> working well for. However, the dataset works fine with other algorithms
>> (e.g. Logistic Regression L-BFGS) Therefore, I suspect this is due to some
>> sort of malformed data format. I'm right now looking into that.
>>
>> Furthermore, I am thinking of starting with the D3 visualization on the
>> parameter setting stage. Should we be moving forward with that idea?
>>
> +1
>
>>
>> Finally, I would like to remind that, we haven't decided a date for code
>> review. Should we do that?
>>
>
> Yes, let's have it this week, if you are ok.
>
>>
>> Thank you
>>
>> On Wed, Jul 22, 2015 at 11:13 PM, Thushan Ganegedara 
>> wrote:
>>
>>> Hi,
>>>
>>> Apologies about the late reply.
>>>
>>> Notes of the Demonstration
>>>
>>> Time duration: approximately 30 mins
>>>
>>> The demonstration was to demonstrate the implemented deeplearning
>>> feature of WSO2-ML. The demo started first explaining the dataset used
>>> (i.e. MNIST). The dataset is a CSV file with approximately 3 rows and
>>> 784 features.
>>>
>>> Next the dataset was loaded to WSO2-ml. Here a concern was raised
>>> regarding selecting the type of data in the Preprocessing Phase (i.e.
>>> Categorical vs Numerical) The suggestion was that there should be a UI
>>> feature to change the data type for all the variables at once (very useful
>>> for large amounts of features).
>>>
>>> Next the deeplearning algorithm for MNIST dataset was demonstrated and
>>> was able to achieve an appx 95% accuracy. Regarding the deeplearning
>>> algorithms, H-2-O doesn't seem to have different deeplearning algorithms at
>>> the moment, but a general deep network + classifier (probably autoencoder).
>>> So the idea was to ask H-2-O team whether they are planning to implement
>>> different networks in the future.
>>>
>>> Also, it was suggested to add a visualization feature in parameter
>>> setting stage to provide a summarized visualization of the network to the
>>> user.
>>>
>>> Furthermore, another suggestion was to test the deep network on real
>>> world datasets and see how it performs. For this datasets from Kaggle will
>>> be used.
>>>
>>>
>>> About progress.
>>>
>>> I'm currently testing the algorithm against different datasets. and I'll
>>> provide a detailed report on that in the recent future.
>>>
>>> Thank you
>>>
>>>
>>>
>>> On Wed, Jul 22, 2015 at 2:00 PM, Nirmal Fernando 
>>> wrote:
>>>
 @Thushan how are you progressing? Could you please send the notes of
 our last review?

 On Thu, Jul 16, 2015 at 10:43 AM, CD Athuraliya 
 wrote:

>
>
> On Mon, Jul 13, 2015 at 11:12 AM, Thushan Ganegedara  > wrote:
>
>> Hello CD,
>>
>> Yes, it seems to be working fine now. But why does it show the axes
>> in meters? Is this a d3 specific thing?
>>
>
> I think *m* stands for *Milli* here.
>
>>
>> On Mon, Jul 13, 2015 at 3:17 PM, Thushan Ganegedara > > wrote:
>>
>>> Hi all,
>>>
>>> Thank you very much for pointing out. I'll get the latest update and
>>> see.
>>>
>>> On Mon, Jul 13, 2015 at 3:03 PM, CD Athuraliya 
>>> wrote:
>>>
 Hi Thushan,

 That method has been updated. Please get the latest. You might have
 to define your own case depending on predicted values.

 CD Athuraliya
 Sent from my mobile device
 On Jul 13, 2015 10:24 AM, "Nirmal Fernando" 
 wrote:

> Great work Thushan! On the UI issues, @CD could help you. AFAIK
> actual keeps the pointer to the actual label and predicted is the
> probability and predictedLabel is after rounding it using a threshold.
>
> On Mon, Jul 13, 2015 at 7:14 AM, Thushan Ganegedara <
> thu...@gmail.com> wrote:
>
>> Hi all,
>>
>> I have integrated H-2-O deeplearning to WSO2-ml successfully.
>> Following are the stats on 2 tests conducted (screenshots attached).
>>
>> Iris dataset - 93.62% Accuracy
>> MNIST (Small) dataset - 94.94% Accuracy
>>
>> However, there were few unusual issues that I had to spend lot of
>> time to identify.
>>
>> *FrameSplitter does not work for any value other than 0.5. Any
>> value other than 0.5, the following error is returned*
>> (Frame splitter is used to split trainingData to train and valid
>> sets)
>> barrier onExCompletion for
>> hex.deeplearning.DeepLearning$DeepLearningDriv

Re: [Dev] Fix for ESBJAVA-2536

[Malaka Silva]

Hi Shakila,

There are some javascripts included. Did you test this with different
browsers?

Also added some comments.

On Fri, Jul 24, 2015 at 10:17 PM, Shakila Sivagnanarajah 
wrote:

> Hi Malaka,
>
> Please review and merge the PR [1] for $Subject.
>
> [1] https://github.com/wso2/carbon-mediation/pull/321
>
> 
> Thank you
>
> --
> Shakila Sivagnanarajah
> Associate Software Engineer
> Mobile :+94 (0) 770 760240
> shak...@wso2.com
>



-- 

Best Regards,

Malaka Silva
Senior Tech Lead
M: +94 777 219 791
Tel : 94 11 214 5345
Fax :94 11 2145300
Skype : malaka.sampath.silva
LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77
Blog : http://mrmalakasilva.blogspot.com/

WSO2, Inc.
lean . enterprise . middleware
http://www.wso2.com/
http://www.wso2.com/about/team/malaka-silva/


Save a tree -Conserve nature & Save the world for your future. Print this
email only if it is absolutely necessary.
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] Web-Deployment order in an OSGI execution flow

[Ruwan Abeykoon]

Hi Ayoob,
There might be some other duplicate package exports in the pack which
causes the CNF exception.

Yes, I also second your observation (in carbon 4.4.0), that the Webapp
Deployment occurs once the "carbon-deployer" bundle is activated. It will
then make sure tomcat bundle is activated. However none of the other user
bundles are not guaranteed to be activated at this time.

@Thilini
This is contrary to the document [1], which is for carbon 4.2.0 based.

Also I have seen this error when Tomcat sees the offending class(package)
in BundleA while other user bundle sees this class in BundleB. This is most
probably what you are experiencing. Thats why it is important that no too
bundles should export same package name.

Cheers,
Ruwan

[1]
http://wso2.com/library/articles/2014/01/understanding-wso2-carbon-architecture/


On Mon, Jul 27, 2015 at 12:50 AM, Ayyoob Hamza  wrote:

> Hi Thilini
> As in this link (Under How tomcat works) it suggest that the war
> deployment happens after the tomcat bundle get activated, not the end of
> all osgi bundle activated.
>
> If thats the case how can we guarantee that the web deployment can only be
> happen after the osgi bundle activation ?.
> [ 1 ]
> http://wso2.com/library/articles/2014/01/understanding-wso2-carbon-architecture/
>
> ___
> Dev mailing list
> Dev@wso2.org
> http://wso2.org/cgi-bin/mailman/listinfo/dev
>
>


-- 

*Ruwan Abeykoon*
*Architect,*
*WSO2, Inc. http://wso2.com  *
*lean.enterprise.middleware.*

email: ruw...@wso2.com
phone:(+94) 39736
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [DAS 3.0.0 Beta] No FileSystem for scheme: file

[Gokul Balakrishnan]

Hi Anuruddha,

Are you running HBase on a cluster (i.e. on top of HDFS)? If yes, can you
ensure that you have the following in the analytics-datasource.xml for the
HBase datasource?


fs.hdfs.impl
org.apache.hadoop.hdfs.DistributedFileSystem


fs.file.impl
org.apache.hadoop.fs.LocalFileSystem


Thanks,


On 27 July 2015 at 10:43, Anuruddha Liyanarachchi 
wrote:

> Hi DAS team,
>
> I have created DAS receiver and analytics cluster as in the diagram [1].
> In the setup DAS receivers are  connected to MySQL (FS_DB)  and HBase
> (EventStore).
>
> I am getting following errors when I start DAS receivers. What could be
> the reason for this error ?
> I have attached the carbon log as well.
>
> TID: [-1] [] [2015-07-27 04:37:40,838]  WARN
> {org.apache.hadoop.hbase.util.DynamicClassLoader} -  Failed to identify the
> fs of dir /opt/wso2das-3.0.0-SNAPSHOT/tmp/hbase-root/hbase/lib, ignored
> {org.apac
> he.hadoop.hbase.util.DynamicClassLoader}
> java.io.IOException: No FileSystem for scheme: file
> at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584)
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:354)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
> at
> org.apache.hadoop.hbase.util.DynamicClassLoader.(DynamicClassLoader.java:104)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.(ProtobufUtil.java:229)
> at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)
> at
> org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:86)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:833)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.(ConnectionManager.java:623)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
> at
> org.wso2.carbon.datasource.reader.hadoop.HadoopDataSourceReaderUtil.getHBaseConnection(HadoopDataSourceReaderUtil.java:79)
> at
> org.wso2.carbon.datasource.reader.hadoop.HBaseDataSourceReader.createDataSource(HBaseDataSourceReader.java:35)
> at
> org.wso2.carbon.ndatasource.core.DataSourceRepository.createDataSourceObject(DataSourceRepository.java:202)
> at
> org.wso2.carbon.ndatasource.core.DataSourceRepository.registerDataSource(DataSourceRepository.java:359)
> at
> org.wso2.carbon.ndatasource.core.DataSourceRepository.addDataSource(DataSourceRepository.java:473)
> at
> org.wso2.carbon.ndatasource.core.DataSourceManager.initSystemDataSource(DataSourceManager.java:185)
> at
> org.wso2.carbon.ndatasource.core.DataSourceManager.initSystemDataSources(DataSourceManager.java:164)
> at
> org.wso2.carbon.ndatasource.core.internal.DataSourceServiceComponent.initSystemDataSources(DataSourceServiceComponent.java:192)
> at
> org.wso2.carbon.ndatasource.core.internal.DataSourceServiceComponent.setSecretCallbackHandlerService(DataSourceServiceComponent.java:178)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.eclipse.equinox.internal.ds.model.ComponentReference.bind(ComponentReference.java:376)
> at
> org.eclipse.equinox.internal.ds.model.ServiceComponentProp.bindReference(ServiceComponentProp.java:430)
> at
> org.eclipse.equinox.internal.ds.model.ServiceComponentProp.bind(ServiceComponentProp.java:218)
> at
> org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:343)
> at
> org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
> at
> org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
> at org.eclipse.equinox.internal.ds.Resol

[Dev] [DEV][DAS] Spark cluster in DAS does not have worker nodes

[Maheshakya Wijewardena]

Hi,

I have tried to create a Spark cluster with DAS using Carbon clustering. 3
DAS nodes are configured and number of Spark masters is set to 2. In this
setting, one of the 3 nodes should have a Spark worker node, but all 3
nodes are starting as Spark masters. What can be the reason for this?

Configuration files (one of *axis2.xml* files of DAS clusters and
*spark-defaults.conf*) of DAS are attached herewith.

Best regards.
-- 
Pruthuvi Maheshakya Wijewardena
Software Engineer
WSO2 : http://wso2.com/
Email: mahesha...@wso2.com
Mobile: +94711228855












true
true
optional
true
work/mtom
4000

${childfirstCL}


true


true



false

inmemory






services


axis2services


axis2modules


WSO2 Data Analytics Server-3.0.0


WSO2 Data Analytics Server-3.0.0






false





false


true



./repository/deployment/server/synapse-configs



.


.


WSO2 Carbon Server






${jaxwsparam}
























http://www.w3.org/ns/wsdl/in-only";
 class="org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver"/>
http://www.w3.org/ns/wsdl/robust-in-only";
 class="org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver"/>
http://www.w3.org/ns/wsdl/in-out";
 class="org.apache.axis2.rpc.receivers.RPCMessageReceiver"/>






















































9763











9443































HTTP/1.1
chunked

true


HTTP/1.1
chunked

true



















































































































	









true


wka




wso2.carbon.domain





45564

100

60





127.0.0.1





4000






https://${hostName}:${httpsPort}/services/"/>
https://${hostName}:${httpsPort}/"/>









127.0.0.1
4000


127.0.0.1
4001


127.0.0.1
4002











spark-defaults.conf
Description: Binary data
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [DEV][DAS] Spark cluster in DAS does not have worker nodes

[Anjana Fernando]

Hi,

Actually, when the 3'rd sever has started up, all 3 servers should have
worker instances. This seems to be a bug. @Niranda, please check it out
ASAP.

Cheers,
Anjana.

On Mon, Jul 27, 2015 at 11:01 AM, Maheshakya Wijewardena <
mahesha...@wso2.com> wrote:

> Hi,
>
> I have tried to create a Spark cluster with DAS using Carbon clustering. 3
> DAS nodes are configured and number of Spark masters is set to 2. In this
> setting, one of the 3 nodes should have a Spark worker node, but all 3
> nodes are starting as Spark masters. What can be the reason for this?
>
> Configuration files (one of *axis2.xml* files of DAS clusters and
> *spark-defaults.conf*) of DAS are attached herewith.
>
> Best regards.
> --
> Pruthuvi Maheshakya Wijewardena
> Software Engineer
> WSO2 : http://wso2.com/
> Email: mahesha...@wso2.com
> Mobile: +94711228855
>
>
>


-- 
*Anjana Fernando*
Senior Technical Lead
WSO2 Inc. | http://wso2.com
lean . enterprise . middleware
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] Renaming carbon-store related features

[Udara Rathnayake]

Hi All,

We have done the above change so please update feature references
accordingly.

https://github.com/wso2/carbon-store/commit/961565403770d3b8a2790284ef5da9c25a25fc40

Regards,
UdaraR

On Mon, Jul 20, 2015 at 2:24 PM, Udara Rathnayake  wrote:

> Hi All,
>
> Right now we have following artifact ID's,
>
> es-feature : org.wso2.es.feature
> store-feature : org.wso2.store.feature
>
> We had a discussion during the previous carbon-store component release to
> rename these to the standard way and correct this mistake.
>
> eg:- org.wso2.es.feature > org.wso2.carbon.es.feature
>
> We need to update people, who depend on es/store features before moving to
> v 2.3.1.
> Let us know if you have any concerns about this change.
>
> [1]
> https://github.com/wso2/carbon-store/tree/master/features/org.wso2.store.feature
> [2]
> https://github.com/wso2/carbon-store/tree/master/features/org.wso2.store.feature
>
> Regards,
> UdaraR
>
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [BPMN-Explorer] Improving Page redirect.

[Amal Gunatilake]

+1 for the suggestions.
Let's do the improvements.

Regards,
Amal.

On Sun, Jul 26, 2015 at 11:37 PM, Milinda Perera  wrote:

> [resending missed content]
> Of course, this should add to humantask ui also. We can fix this simply
> adding query parameter in login redirection and checking for request url in
> refer header as below:
>
>1. Referer:
>
>
> https://192.168.1.2:9443/humantask-explorer/login?requestUrl=https://192.168.1.2:9443/humantask-explorer/task?id=2352
>
> Thanks,
> Milinda
>
> On Sun, Jul 26, 2015 at 11:36 PM, Milinda Perera 
> wrote:
>
>> Of course, this should add to humantask ui also. We can fix this simply
>> adding query parameter in login redirection and checking for request url in
>> refer header as below:
>>
>>
>> Thanks,
>> Milinda
>>
>> On Sun, Jul 26, 2015 at 8:42 PM, Hasitha Aravinda 
>> wrote:
>>
>>> ​Created BPS-750 & BPS-751 for this.
>>>
>>> Thanks,
>>> Hasitha. ​
>>>
>>> On Sun, Jul 26, 2015 at 11:04 AM, Hasitha Aravinda 
>>> wrote:
>>>
 Hi Amal,

 Use case 1: Sometimes users will require to embedded task url ( Like
 [1] ) in emails. ( Eg: Task Reminder email etc). When user click on these
 links, BPMN webapp redirect user back to webapp login page, if user didn't
 login already.
 But after user login, webapp doesn't redirect back to task page,
 instead it will redirect to home page.

 Use case 2: When a user tries to view invalid or non existing task id (
 eg: task?id=53016. ; task?id=12334566223 ), webapp shows 500 page. But 500
 page doesn't have any home page link. Btw I think 404 - not found status
 code is correct status code to return.

 [1] https://localhost:9443/bpmn-explorer/task?id=53016

 Can we fix these, if time permits.

 Thanks,
 Hasitha.

 --
 Hasitha Aravinda,
 Senior Software Engineer,
 WSO2 Inc.
 Email: hasi...@wso2.com
 Mobile : +1 201 887 1971, +94 718 210 200

>>>
>>>
>>>
>>> --
>>> Hasitha Aravinda,
>>> Senior Software Engineer,
>>> WSO2 Inc.
>>> Email: hasi...@wso2.com
>>> Mobile : +1 201 887 1971, +94 718 210 200
>>>
>>
>>
>>
>> --
>> Milinda Perera
>> Software Engineer;
>> WSO2 Inc. http://wso2.com ,
>> Mobile: (+94) 714 115 032
>>
>>
>
>
> --
> Milinda Perera
> Software Engineer;
> WSO2 Inc. http://wso2.com ,
> Mobile: (+94) 714 115 032
>
>


-- 
*Amal Gunatilake*
 Software Engineer
WSO2 Inc.; http://wso2.com
lean.enterprise.middleware
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [DAS 3.0.0 Beta] No FileSystem for scheme: file

[Anuruddha Liyanarachchi]

Hi Gokul,

Thanks for the reply.

Yes this runs on a HBase cluster. I have added the properties and it fixed
the No FileSystem for scheme: file Error. Now I am getting following error.

org.wso2.carbon.analytics.datasource.commons.exception.AnalyticsException:
Error checking existence of table __SHARD_INDEX_UPDATE_RECORDS__1 for
tenant -1000
at
org.wso2.carbon.analytics.datasource.hbase.HBaseAnalyticsRecordStore.tableExists(HBaseAnalyticsRecordStore.java:126)
at
org.wso2.carbon.analytics.datasource.hbase.HBaseAnalyticsRecordStore.get(HBaseAnalyticsRecordStore.java:274)
at
org.wso2.carbon.analytics.dataservice.indexing.AnalyticsDataIndexer.loadIndexOperationRecords(AnalyticsDataIndexer.java:554)
at
org.wso2.carbon.analytics.dataservice.indexing.AnalyticsDataIndexer.loadIndexOperationUpdateRecords(AnalyticsDataIndexer.java:519)
at
org.wso2.carbon.analytics.dataservice.indexing.AnalyticsDataIndexer.loadIndexOperationUpdateRecords(AnalyticsDataIndexer.java:515)
at
org.wso2.carbon.analytics.dataservice.indexing.AnalyticsDataIndexer.processIndexUpdateOperations(AnalyticsDataIndexer.java:403)
at
org.wso2.carbon.analytics.dataservice.indexing.AnalyticsDataIndexer.processIndexOperations(AnalyticsDataIndexer.java:491)
at
org.wso2.carbon.analytics.dataservice.indexing.AnalyticsDataIndexer.access$200(AnalyticsDataIndexer.java:118)
at
org.wso2.carbon.analytics.dataservice.indexing.AnalyticsDataIndexer$IndexWorker.run(AnalyticsDataIndexer.java:1744)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Can't
get the locations
at
org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:305)
at
org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:131)
at
org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:56)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:287)
at
org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:267)
at
org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:139)
at
org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:134)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:823)
at
org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:601)
at
org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:365)
at
org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:281)
at
org.wso2.carbon.analytics.datasource.hbase.HBaseAnalyticsRecordStore.tableExists(HBaseAnalyticsRecordStore.java:123)
... 11 more


On Mon, Jul 27, 2015 at 10:54 AM, Gokul Balakrishnan  wrote:

> Hi Anuruddha,
>
> Are you running HBase on a cluster (i.e. on top of HDFS)? If yes, can you
> ensure that you have the following in the analytics-datasource.xml for the
> HBase datasource?
>
> 
> fs.hdfs.impl
> org.apache.hadoop.hdfs.DistributedFileSystem
> 
> 
> fs.file.impl
> org.apache.hadoop.fs.LocalFileSystem
> 
>
> Thanks,
>
>
> On 27 July 2015 at 10:43, Anuruddha Liyanarachchi 
> wrote:
>
>> Hi DAS team,
>>
>> I have created DAS receiver and analytics cluster as in the diagram [1].
>> In the setup DAS receivers are  connected to MySQL (FS_DB)  and HBase
>> (EventStore).
>>
>> I am getting following errors when I start DAS receivers. What could be
>> the reason for this error ?
>> I have attached the carbon log as well.
>>
>> TID: [-1] [] [2015-07-27 04:37:40,838]  WARN
>> {org.apache.hadoop.hbase.util.DynamicClassLoader} -  Failed to identify the
>> fs of dir /opt/wso2das-3.0.0-SNAPSHOT/tmp/hbase-root/hbase/lib, ignored
>> {org.apac
>> he.hadoop.hbase.util.DynamicClassLoader}
>> java.io.IOException: No FileSystem for scheme: file
>> at
>> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584)
>> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
>> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
>> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
>> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169)
>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:354)
>> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
>> at
>> org.apache.hadoop.hbase.util.DynamicClassLoader.(DynamicClassLoader.java:104)
>> at
>> org.apache.hadoop.hbase.protobuf.ProtobufUtil.(ProtobufUtil.java:229)
>> at org.apache.hadoop.hbase.ClusterId.parseFrom(Cluster

Re: [Dev] [DAS 3.0.0 Beta] No FileSystem for scheme: file

[Gokul Balakrishnan]

Hi Anuruddha,

This seems to be an issue with the HBase cluster itself rather than an
integration issue. Could you check the HBase logs and see if anything is
reported there? In any case, please also specify the zookeeper quorum
property as well (hbase.zookeeper.quorum and specify the zk nodes)

Thanks,

On 27 July 2015 at 11:45, Anuruddha Liyanarachchi 
wrote:

> Hi Gokul,
>
> Thanks for the reply.
>
> Yes this runs on a HBase cluster. I have added the properties and it fixed
> the No FileSystem for scheme: file Error. Now I am getting following error.
>
> org.wso2.carbon.analytics.datasource.commons.exception.AnalyticsException:
> Error checking existence of table __SHARD_INDEX_UPDATE_RECORDS__1 for
> tenant -1000
> at
> org.wso2.carbon.analytics.datasource.hbase.HBaseAnalyticsRecordStore.tableExists(HBaseAnalyticsRecordStore.java:126)
> at
> org.wso2.carbon.analytics.datasource.hbase.HBaseAnalyticsRecordStore.get(HBaseAnalyticsRecordStore.java:274)
> at
> org.wso2.carbon.analytics.dataservice.indexing.AnalyticsDataIndexer.loadIndexOperationRecords(AnalyticsDataIndexer.java:554)
> at
> org.wso2.carbon.analytics.dataservice.indexing.AnalyticsDataIndexer.loadIndexOperationUpdateRecords(AnalyticsDataIndexer.java:519)
> at
> org.wso2.carbon.analytics.dataservice.indexing.AnalyticsDataIndexer.loadIndexOperationUpdateRecords(AnalyticsDataIndexer.java:515)
> at
> org.wso2.carbon.analytics.dataservice.indexing.AnalyticsDataIndexer.processIndexUpdateOperations(AnalyticsDataIndexer.java:403)
> at
> org.wso2.carbon.analytics.dataservice.indexing.AnalyticsDataIndexer.processIndexOperations(AnalyticsDataIndexer.java:491)
> at
> org.wso2.carbon.analytics.dataservice.indexing.AnalyticsDataIndexer.access$200(AnalyticsDataIndexer.java:118)
> at
> org.wso2.carbon.analytics.dataservice.indexing.AnalyticsDataIndexer$IndexWorker.run(AnalyticsDataIndexer.java:1744)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Can't
> get the locations
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:305)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:131)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:56)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
> at
> org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:287)
> at
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:267)
> at
> org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:139)
> at
> org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:134)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:823)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:601)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:365)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:281)
> at
> org.wso2.carbon.analytics.datasource.hbase.HBaseAnalyticsRecordStore.tableExists(HBaseAnalyticsRecordStore.java:123)
> ... 11 more
>
>
> On Mon, Jul 27, 2015 at 10:54 AM, Gokul Balakrishnan 
> wrote:
>
>> Hi Anuruddha,
>>
>> Are you running HBase on a cluster (i.e. on top of HDFS)? If yes, can you
>> ensure that you have the following in the analytics-datasource.xml for the
>> HBase datasource?
>>
>> 
>> fs.hdfs.impl
>> org.apache.hadoop.hdfs.DistributedFileSystem
>> 
>> 
>> fs.file.impl
>> org.apache.hadoop.fs.LocalFileSystem
>> 
>>
>> Thanks,
>>
>>
>> On 27 July 2015 at 10:43, Anuruddha Liyanarachchi 
>> wrote:
>>
>>> Hi DAS team,
>>>
>>> I have created DAS receiver and analytics cluster as in the diagram [1].
>>> In the setup DAS receivers are  connected to MySQL (FS_DB)  and HBase
>>> (EventStore).
>>>
>>> I am getting following errors when I start DAS receivers. What could be
>>> the reason for this error ?
>>> I have attached the carbon log as well.
>>>
>>> TID: [-1] [] [2015-07-27 04:37:40,838]  WARN
>>> {org.apache.hadoop.hbase.util.DynamicClassLoader} -  Failed to identify the
>>> fs of dir /opt/wso2das-3.0.0-SNAPSHOT/tmp/hbase-root/hbase/lib, ignored
>>> {org.apac
>>> he.hadoop.hbase.util.DynamicClassLoader}
>>> java.io.IOException: No FileSystem for scheme: file
>>> at
>>> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584)
>>> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
>>> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
>>> at
>>> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
>>> at

Re: [Dev] [DEV][DAS] Spark cluster in DAS does not have worker nodes

[Niranda Perera]

Hi Maheshakya,

I will look into this.

According to your setting, the ideal scenario is,
node1 ->  spark master (active) + worker
node2 -> spark master (standby) + worker
node3 -> worker

did you start the servers all at once or one by one?

rgds

On Mon, Jul 27, 2015 at 11:07 AM, Anjana Fernando  wrote:

> Hi,
>
> Actually, when the 3'rd sever has started up, all 3 servers should have
> worker instances. This seems to be a bug. @Niranda, please check it out
> ASAP.
>
> Cheers,
> Anjana.
>
> On Mon, Jul 27, 2015 at 11:01 AM, Maheshakya Wijewardena <
> mahesha...@wso2.com> wrote:
>
>> Hi,
>>
>> I have tried to create a Spark cluster with DAS using Carbon clustering.
>> 3 DAS nodes are configured and number of Spark masters is set to 2. In this
>> setting, one of the 3 nodes should have a Spark worker node, but all 3
>> nodes are starting as Spark masters. What can be the reason for this?
>>
>> Configuration files (one of *axis2.xml* files of DAS clusters and
>> *spark-defaults.conf*) of DAS are attached herewith.
>>
>> Best regards.
>> --
>> Pruthuvi Maheshakya Wijewardena
>> Software Engineer
>> WSO2 : http://wso2.com/
>> Email: mahesha...@wso2.com
>> Mobile: +94711228855
>>
>>
>>
>
>
> --
> *Anjana Fernando*
> Senior Technical Lead
> WSO2 Inc. | http://wso2.com
> lean . enterprise . middleware
>



-- 
*Niranda Perera*
Software Engineer, WSO2 Inc.
Mobile: +94-71-554-8430
Twitter: @n1r44 
https://pythagoreanscript.wordpress.com/
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [DEV][DAS] Spark cluster in DAS does not have worker nodes

[Maheshakya Wijewardena]

Hi Niranda,

I started those one by one.

On Mon, Jul 27, 2015 at 12:07 PM, Niranda Perera  wrote:

> Hi Maheshakya,
>
> I will look into this.
>
> According to your setting, the ideal scenario is,
> node1 ->  spark master (active) + worker
> node2 -> spark master (standby) + worker
> node3 -> worker
>
> did you start the servers all at once or one by one?
>
> rgds
>
> On Mon, Jul 27, 2015 at 11:07 AM, Anjana Fernando  wrote:
>
>> Hi,
>>
>> Actually, when the 3'rd sever has started up, all 3 servers should have
>> worker instances. This seems to be a bug. @Niranda, please check it out
>> ASAP.
>>
>> Cheers,
>> Anjana.
>>
>> On Mon, Jul 27, 2015 at 11:01 AM, Maheshakya Wijewardena <
>> mahesha...@wso2.com> wrote:
>>
>>> Hi,
>>>
>>> I have tried to create a Spark cluster with DAS using Carbon clustering.
>>> 3 DAS nodes are configured and number of Spark masters is set to 2. In this
>>> setting, one of the 3 nodes should have a Spark worker node, but all 3
>>> nodes are starting as Spark masters. What can be the reason for this?
>>>
>>> Configuration files (one of *axis2.xml* files of DAS clusters and
>>> *spark-defaults.conf*) of DAS are attached herewith.
>>>
>>> Best regards.
>>> --
>>> Pruthuvi Maheshakya Wijewardena
>>> Software Engineer
>>> WSO2 : http://wso2.com/
>>> Email: mahesha...@wso2.com
>>> Mobile: +94711228855
>>>
>>>
>>>
>>
>>
>> --
>> *Anjana Fernando*
>> Senior Technical Lead
>> WSO2 Inc. | http://wso2.com
>> lean . enterprise . middleware
>>
>
>
>
> --
> *Niranda Perera*
> Software Engineer, WSO2 Inc.
> Mobile: +94-71-554-8430
> Twitter: @n1r44 
> https://pythagoreanscript.wordpress.com/
>



-- 
Pruthuvi Maheshakya Wijewardena
Software Engineer
WSO2 : http://wso2.com/
Email: mahesha...@wso2.com
Mobile: +94711228855
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev