Re: [Dev] [Architecture] Configuration for SHARD_ID column length in siddhi
Hi Rukshan, As per the offline discussion with, @Suho, we have decided to ignore 'field.length' validation for SHARD_ID as it is much cleaner and this column is introduced by Siddhi Aggregation implementation. Please find the pull request here[1]. [1] https://github.com/siddhi-io/siddhi-store-rdbms/pull/214 Best Regards, Niveathika On Mon, Nov 25, 2019 at 12:05 PM Niveathika Rajendran wrote: > Hi Rukshan and Fazlan, > > Regarding aggregation, since this config is based on the store type > (RDBMS), we cannot introduce this support in Siddhi Aggregation > implementation. However, we can introduce a system property config for > the siddhi-store-rdbms extension to ignore validation of 'field.length' > property. This will ensure that 'SHARD_ID' column can be added to the > 'field.length' property by default. > > @Fazlan, using the collation cannot be enabled by default, since we have > to maintain backward compatibility. However, this can be now enabled by a > single property '*use.collation'*. > > Best Regards, > Niveathika > > > On Sat, Nov 23, 2019 at 5:13 PM Fazlan Nazeem wrote: > >> One related and good to have feature is to set a single character >> encoding by default for the tables when siddhi creates them on the >> database. The primary key length issue is mostly due to having collations >> which take more bytes to represent a single character. If we can default to >> a character encoding of a single byte per character for each database type, >> then we could avoid these issues. Just a thought. >> >> On Fri, Nov 22, 2019 at 5:31 PM Rukshan Premathunga >> wrote: >> >>> Hi All, >>> >>> When we configure partitionById in SP, siddhi automatically adds >>> a SHARD_ID column to all the aggregated tables in RDBMS. But we are having >>> a "too long key issue" in the database. As a solution, we need to properly >>> set the column length for each attribute in the aggregated event stream. >>> >>> Limit of columns in the aggregated streams can be defined when we >>> implement the siddhi app. But SHARD_ID is used only when partitionById is >>> configured. So we cannot provide and initial column length for that. >>> >>> So it will be an ideal solution for us to have a configuration for this. >>> Otherwise, users need to alter the database or siddhi applications to >>> define these values. So can you please check the possibility to support >>> this? >>> >>> ex: >>> siddhi: >>> properties: >>>partitionById: true >>>shardId: dc1 >>>shardId_size: 20 # or derive length from 'shardId' >>> >>> Thanks and Regards >>> >>> -- >>> Rukshan C. Premathunga | Associate Technical Lead | WSO2 Inc. >>> (m) +94711822074 | (w) +94112145345 | Email: ruks...@wso2.com >>> GET INTEGRATION AGILE >>> Integration Agility for Digitally Driven Business >>> ___ >>> Dev mailing list >>> Dev@wso2.org >>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >> >> >> -- >> Thanks & Regards, >> >> *Fazlan Nazeem | *Associate Technical Lead | WSO2 Inc >> Mobile : +94772338839 | fazl...@wso2.com >> >> >> ___ >> Architecture mailing list >> architect...@wso2.org >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> > > > -- > Niveathika Rajendran | Senior Software Engineer | WSO2 Inc. > (m) +94779037536 | (w) +9411743 5800 | (e) niveath...@wso2.com > <https://wso2.com/signature> > > -- Niveathika Rajendran | Senior Software Engineer | WSO2 Inc. (m) +94779037536 | (w) +9411743 5800 | (e) niveath...@wso2.com <https://wso2.com/signature> ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [Architecture] Configuration for SHARD_ID column length in siddhi
Hi Rukshan and Fazlan, Regarding aggregation, since this config is based on the store type (RDBMS), we cannot introduce this support in Siddhi Aggregation implementation. However, we can introduce a system property config for the siddhi-store-rdbms extension to ignore validation of 'field.length' property. This will ensure that 'SHARD_ID' column can be added to the 'field.length' property by default. @Fazlan, using the collation cannot be enabled by default, since we have to maintain backward compatibility. However, this can be now enabled by a single property '*use.collation'*. Best Regards, Niveathika On Sat, Nov 23, 2019 at 5:13 PM Fazlan Nazeem wrote: > One related and good to have feature is to set a single character encoding > by default for the tables when siddhi creates them on the database. The > primary key length issue is mostly due to having collations which take more > bytes to represent a single character. If we can default to a character > encoding of a single byte per character for each database type, then we > could avoid these issues. Just a thought. > > On Fri, Nov 22, 2019 at 5:31 PM Rukshan Premathunga > wrote: > >> Hi All, >> >> When we configure partitionById in SP, siddhi automatically adds >> a SHARD_ID column to all the aggregated tables in RDBMS. But we are having >> a "too long key issue" in the database. As a solution, we need to properly >> set the column length for each attribute in the aggregated event stream. >> >> Limit of columns in the aggregated streams can be defined when we >> implement the siddhi app. But SHARD_ID is used only when partitionById is >> configured. So we cannot provide and initial column length for that. >> >> So it will be an ideal solution for us to have a configuration for this. >> Otherwise, users need to alter the database or siddhi applications to >> define these values. So can you please check the possibility to support >> this? >> >> ex: >> siddhi: >> properties: >>partitionById: true >>shardId: dc1 >>shardId_size: 20 # or derive length from 'shardId' >> >> Thanks and Regards >> >> -- >> Rukshan C. Premathunga | Associate Technical Lead | WSO2 Inc. >> (m) +94711822074 | (w) +94112145345 | Email: ruks...@wso2.com >> GET INTEGRATION AGILE >> Integration Agility for Digitally Driven Business >> ___ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> > > > -- > Thanks & Regards, > > *Fazlan Nazeem | *Associate Technical Lead | WSO2 Inc > Mobile : +94772338839 | fazl...@wso2.com > > > ___ > Architecture mailing list > architect...@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > -- Niveathika Rajendran | Senior Software Engineer | WSO2 Inc. (m) +94779037536 | (w) +9411743 5800 | (e) niveath...@wso2.com <https://wso2.com/signature> ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Stream processor studio application to dashboard
Hi Henk, WSO2 dashboard portal displays data fetched from a Database. You can save the data in a store <https://siddhi.io/en/v5.1/docs/query-guide/#store>[1] using the Siddhi event table construct. Dashboard portal will fetch them through 'Data providers'. See Working with Data Providers [2] for more information on different types of data providers supported and Generating Widgets [2] to see how to configure them when you create widgets. [1] https://siddhi.io/en/v5.1/docs/query-guide/#store <https://siddhi.io/en/v5.1/docs/query-guide/#store> [2] https://docs.wso2.com/display/SP440/Working+with+Data+Providers [3] https://docs.wso2.com/display/SP440/Generating+Widgets Best Regards, Niveathika On Wed, Nov 20, 2019 at 3:53 PM W. H. wrote: > Hi, > > I've created an siddhi application in Stream processor studio and I want > to sent the data from a outputstream to the wso2 dashboard portal. How can > I do this? Can I directly sent the data from the application to the portal? > Or do I have to save the data first in a store? I've looked into the > documentation but I dont understand it. > > Best regards, > Henk > ___ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > -- Niveathika Rajendran | Senior Software Engineer | WSO2 Inc. (m) +94779037536 | (w) +9411743 5800 | (e) niveath...@wso2.com <https://wso2.com/signature> ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [VOTE] Release of WSO2 Stream Processor 4.4.0 RC6
Hi all, I have tested the IS, APIM and EI analytics solutions. No issues found. +1 Go ahead and release. Best Regards, *Niveathika Rajendran,* *Senior Software Engineer.* *Mobile : +94 077 903 7536* On Wed, May 15, 2019 at 6:37 PM Damith Wickramasinghe wrote: > Hi All, > > Tested editor forms, utf8 support, latest improvements > Tested HA deployment > > No issues found. > +1 Go ahead and release. > > Regards, > Damith > > On Wed, May 15, 2019 at 4:30 PM Eranga Liyanage wrote: > >> Hi, >> >> Tested following components. >> >>- Editor >>- Dashboard >>- Portal >>- Business rules manager >> >> No issues found. >> +1 Go ahead and release. >> >> Regards >> Eranga >> >> >> On Wed, May 15, 2019 at 2:36 PM Lasantha Samarakoon >> wrote: >> >>> Following components/functions were tested. >>> >>>- Dashboard portal >>>- Editor >>>- SSO feature >>> >>> No issues found. Hence +1. >>> >>> Thanks, >>> >>> On Wed, May 15, 2019 at 1:12 PM Minudika Malshan >>> wrote: >>> >>>> Hi, >>>> >>>> Tested functionalities of the following components. >>>> >>>>- Business rules manager >>>>- Siddhi-io-http >>>> >>>> No issues found. >>>> +1 Go ahead and release. >>>> >>>> Regards, >>>> Minudika >>>> >>>> On Wed, May 15, 2019 at 3:11 AM Ramindu De Silva >>>> wrote: >>>> >>>>> Hi all, >>>>> >>>>> WSO2 Stream Processor team is pleased to announce the sixth release >>>>> candidate of WSO2 Stream Processor 4.4.0. >>>>> >>>>> WSO2 Stream Processor is an open source embodiment of the WSO2 >>>>> Analytics platform, of which the real-time, incremental & intelligent data >>>>> processing capabilities let digital businesses create actionable business >>>>> insights and data products. >>>>> >>>>> Please find the improvements and fixes related to this release: >>>>> >>>>>- siddhi >>>>> >>>>> <https://github.com/siddhi-io/siddhi/issues?utf8=%E2%9C%93=is%3Aissue+closed%3A2018-10-12..2019-04-13> >>>>>- carbon-analytics-common >>>>> >>>>> <https://github.com/wso2/carbon-analytics-common/issues?utf8=%E2%9C%93=is%3Aissue+closed%3A2018-10-12..2019-04-13> >>>>>- carbon-analytics >>>>> >>>>> <https://github.com/wso2/carbon-analytics/issues?utf8=%E2%9C%93=is%3Aissue+closed%3A2018-10-12..2019-04-13> >>>>>- carbon-dashboards >>>>> >>>>> <https://github.com/wso2/carbon-dashboards/issues?utf8=%E2%9C%93=is%3Aissue+closed%3A2018-10-12..2019-04-13> >>>>>- analytics-solutions >>>>> >>>>> <https://github.com/wso2/analytics-solutions/issues?utf8=%E2%9C%93=is%3Aissue+closed%3A2018-10-12..2019-04-13> >>>>>- product-sp >>>>> >>>>> <https://github.com/wso2/product-sp/issues?utf8=%E2%9C%93=is%3Aissue+closed%3A2018-10-12..2019-04-13> >>>>> >>>>> You can download the product distribution from here >>>>> <https://github.com/wso2/product-sp/releases/download/v4.4.0-RC6/wso2sp-4.4.0.zip> >>>>> . >>>>> >>>>> The tag to be voted upon: >>>>> https://github.com/wso2/product-sp/releases/tag/v4.4.0-RC6 >>>>> >>>>> Please download, test the product and vote. >>>>> >>>>> [+] Stable - go ahead and release >>>>> [-] Broken - do not release (explain why) >>>>> >>>>> >>>>> You can find the official documentation in >>>>> https://docs.wso2.com/display/SP440 >>>>> >>>>> Best Regards, >>>>> WSO2 Stream Processor Team >>>>> -- >>>>> *Ramindu De Silva* >>>>> Associate Technical Lead >>>>> WSO2 Inc.: http://wso2.com >>>>> lean.enterprise.middleware >>>>> >>>>> email: ramin...@wso2.com >>>>> mob: +94 719678895 >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>&
Re: [Dev] [VOTE] Release of WSO2 Stream Processor 4.4.0 RC4
Hi all, I have tested APIM, IS and EI analytics solutions. +1 Go ahead and release Best Regards, *Niveathika Rajendran,* *Senior Software Engineer.* *Mobile : +94 077 903 7536* On Thu, May 9, 2019 at 10:30 PM Ramindu De Silva wrote: > Hi All, > > WSO2 Stream Processor team is pleased to announce the fourth release > candidate of WSO2 Stream Processor 4.4.0. > > WSO2 Stream Processor is an open source embodiment of the WSO2 Analytics > platform, of which the real-time, incremental & intelligent data processing > capabilities let digital businesses create actionable business insights and > data products. > > Please find the improvements and fixes related to this release: > >- siddhi > > <https://github.com/siddhi-io/siddhi/issues?utf8=%E2%9C%93=is%3Aissue+closed%3A2018-10-12..2019-04-13> >- carbon-analytics-common > > <https://github.com/wso2/carbon-analytics-common/issues?utf8=%E2%9C%93=is%3Aissue+closed%3A2018-10-12..2019-04-13> >- carbon-analytics > > <https://github.com/wso2/carbon-analytics/issues?utf8=%E2%9C%93=is%3Aissue+closed%3A2018-10-12..2019-04-13> >- carbon-dashboards > > <https://github.com/wso2/carbon-dashboards/issues?utf8=%E2%9C%93=is%3Aissue+closed%3A2018-10-12..2019-04-13> >- analytics-solutions > > <https://github.com/wso2/analytics-solutions/issues?utf8=%E2%9C%93=is%3Aissue+closed%3A2018-10-12..2019-04-13> >- product-sp > > <https://github.com/wso2/product-sp/issues?utf8=%E2%9C%93=is%3Aissue+closed%3A2018-10-12..2019-04-13> > > You can download the product distribution from here > <https://github.com/wso2/product-sp/releases/download/v4.4.0-RC4/wso2sp-4.4.0.zip> > > The tag to be voted upon: > https://github.com/wso2/product-sp/releases/tag/v4.4.0-RC4 > > Please download, test the product and vote. > > [+] Stable - go ahead and release > [-] Broken - do not release (explain why) > > You can find the official documentation in > https://docs.wso2.com/display/SP440 > > Best Regards, > WSO2 Stream Processor Team > > -- > *Ramindu De Silva* > Associate Technical Lead > WSO2 Inc.: http://wso2.com > lean.enterprise.middleware > > email: ramin...@wso2.com > mob: +94 719678895 > ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] init table ERRORs with EI_Analytics_StatApp.siddhi using mysql
Hi Bernard, RDBMS's have limit on the for the primary key and seems that is exceeding, thus the tables could not be created. >From your previous emails, I understand you are trying to test this in MySQL. If so, 1. What is the MySQL database storage engine type? (InnoDB..) 2. What is the character set the DB is configured with? Best Regards, *Niveathika Rajendran,* *Senior Software Engineer.* *Mobile : +94 077 903 7536* On Tue, Feb 19, 2019 at 3:01 PM Bernard Paris wrote: > Hello, > > (using wso2 EI-6.4.0) > > Starting analytics worker log lot of errors about several similar table > initialization -errors are repeated every 2 minutes - it talks about a key > which is too long. > > Here is an log example for MediatorStatAgg_DAYS table, same error logs > occur for > MediatorStatAgg_YEARS > MediatorStatAgg_HOURS > MediatorStatAgg_MINUTES > MediatorStatAgg_SECONDS > > note: I never had this problem when we tested with postgres dbs, strange... > > Here are: the only thing I have changed in EI_Analytics_StatApp.siddhi : > > @store(type = 'rdbms', datasource = 'EI_ANALYTICS', > field.length="configData:18000")[18000 instead of default 6000] > > and add the convert time function at the end of the file : > > fromTablePurgingTriggerStream > select convert(time:dateSub(triggered_time, 2, 'week'), 'long') as > purgingTime > delete ConfigEntryTable > on ConfigEntryTable.eventTimestamp < purgingTime; > > fromTablePurgingTriggerStream > select convert(time:dateSub(triggered_time, 2, 'week'), 'long') as > purgingTime > delete ESBEventTable > on ESBEventTable.eventTimestamp < purgingTime; > > > > > > ERROR {org.wso2.siddhi.core.table.Table} - Error on > 'EI_Analytics_StatApp'. Failed to initialize store for table name > 'MediatorStatAgg_DAYS': Unable to initialize table 'MediatorStatAgg_DAYS': > Specified key was too long; max key length is 3072 bytes Error while > connecting to Table 'MediatorStatAgg_DAYS', will retry in '5 sec'. > org.wso2.siddhi.core.exception.ConnectionUnavailableException: Failed to > initialize store for table name 'MediatorStatAgg_DAYS': Unable to > initialize table 'MediatorStatAgg_DAYS': Specified key was too long; max > key length is 3072 bytes > at > org.wso2.extension.siddhi.store.rdbms.RDBMSEventTable.connect(RDBMSEventTable.java:882) > at org.wso2.siddhi.core.table.Table.connectWithRetry(Table.java:364) > at > org.wso2.siddhi.core.SiddhiAppRuntime.startWithoutSources(SiddhiAppRuntime.java:400) > at org.wso2.siddhi.core.SiddhiAppRuntime.start(SiddhiAppRuntime.java:375) > at > org.wso2.carbon.stream.processor.core.internal.StreamProcessorService.deploySiddhiApp(StreamProcessorService.java:224) > at > org.wso2.carbon.stream.processor.core.internal.StreamProcessorDeployer.deploySiddhiQLFile(StreamProcessorDeployer.java:93) > at > org.wso2.carbon.stream.processor.core.internal.StreamProcessorDeployer.deploy(StreamProcessorDeployer.java:306) > at > org.wso2.carbon.deployment.engine.internal.DeploymentEngine.lambda$deployArtifacts$0(DeploymentEngine.java:291) > at java.util.ArrayList.forEach(ArrayList.java:1257) > at > org.wso2.carbon.deployment.engine.internal.DeploymentEngine.deployArtifacts(DeploymentEngine.java:282) > at > org.wso2.carbon.deployment.engine.internal.RepositoryScanner.sweep(RepositoryScanner.java:112) > at > org.wso2.carbon.deployment.engine.internal.RepositoryScanner.scan(RepositoryScanner.java:68) > at > org.wso2.carbon.deployment.engine.internal.DeploymentEngine.start(DeploymentEngine.java:121) > at > org.wso2.carbon.deployment.engine.internal.DeploymentEngineListenerComponent.onAllRequiredCapabilitiesAvailable(DeploymentEngineListenerComponent.java:216) > at > org.wso2.carbon.kernel.internal.startupresolver.StartupComponentManager.lambda$notifySatisfiableComponents$7(StartupComponentManager.java:266) > at java.util.ArrayList.forEach(ArrayList.java:1257) > at > org.wso2.carbon.kernel.internal.startupresolver.StartupComponentManager.notifySatisfiableComponents(StartupComponentManager.java:252) > at > org.wso2.carbon.kernel.internal.startupresolver.StartupOrderResolver$1.run(StartupOrderResolver.java:204) > at java.util.TimerThread.mainLoop(Timer.java:555) > at java.util.TimerThread.run(Timer.java:505) > Caused by: > org.wso2.extension.siddhi.store.rdbms.exception.RDBMSTableException: Unable > to initialize table 'MediatorStatAgg_DAYS': Specified key was too long; max > key length is 3072 bytes > at > org.wso2.extension.siddhi.store.rdbms.RDBMSEventTable.createTable(RDBMSEventTable.java:1125) > at > org.wso2.extension.siddhi.store.rdbms.RDBMSEventTable.connect(RDBMSEventTable.java:874) > ... 19 more > Caus
Re: [Dev] disabling SSL for the connexion pool in SP EI-6.4
Hi Bernard, You can add MySQL connection tag as follows, *jdbc:mysql://localhost:3306/Test?useSSL=false,* to disable SSL connection to the DB.[1] [1] https://stackoverflow.com/a/34449182/4027644 Best Regards, *Niveathika Rajendran,* *Senior Software Engineer.* *Mobile : +94 077 903 7536* On Mon, Feb 18, 2019 at 10:18 PM Bernard Paris wrote: > Hello, > > I follow the documentation at > https://docs.wso2.com/display/SP430/Configuring+Datasources > to configure mySql for our datasources with StreamProcessor (Analytics > worker in wso2 EI-6.4). > > I get this SSL error on SP startup. Is it possible to disable SSL for > the connexion pool ? how to do that ? > > Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: > unknown_ca > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) > at sun.security.ssl.Alerts.getSSLException(Alerts.java:154) > at > sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:2020) > at > sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1127) > at > sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761) > at > sun.security.ssl.HandshakeOutStream.flush(HandshakeOutStream.java:124) > at > sun.security.ssl.Handshaker.sendChangeCipherSpec(Handshaker.java:1152) > at > sun.security.ssl.ClientHandshaker.sendChangeCipherAndFinish(ClientHandshaker.java:1279) > at > sun.security.ssl.ClientHandshaker.serverHelloDone(ClientHandshaker.java:1189) > at > sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:369) > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037) > at sun.security.ssl.Handshaker.process_record(Handshaker.java:965) > at > sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064) > at > sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367) > at > sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395) > at > sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379) > at > com.mysql.jdbc.ExportControlled.transformSocketToSSLSocket(ExportControlled.java:188) > ... 34 more > > Thanks > Bernard > > ___ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] SP Analytics REST APIs
Hi Bernard, The analytics data can be queried using the exposed Store API in Stream Processor / EI Analytics worker node. Please see the doc[1] for the query formats that can be used. [1] https://docs.wso2.com/display/SP430/Managing+Stored+Data+via+REST+APIs#ManagingStoredDataviaRESTAPIs-Retrievingrecords Best Regards, *Niveathika Rajendran,* *Senior Software Engineer.* *Mobile : +94 077 903 7536* On Thu, Jan 3, 2019 at 9:07 PM Sajith Ariyarathna wrote: > Hi Bernard, > > Analytics profile of EI-6.4.0 is an instance of SP-4.3.0. Please refer doc > [1] on the REST APIs exposed by SP. > > [1] https://docs.wso2.com/display/SP430/Stream+Processor+REST+API+Guide > > Thanks. > > On Thu, Jan 3, 2019 at 8:51 PM Bernard Paris > wrote: > >> Hi devs, >> >> before EI640 and previous Analytics version software we were using the >> REST API to query Analytics for its datas then build our own graphical web >> pages with results in a web application >> >> >> >> >> This doesn't work anymore with new Analytics now embedded into a >> StreamProcessor worker. I read the doc >> "Analyzing Statistics for REST APIs" at >> https://docs.wso2.com/display/EI640/Analyzing+Statistics+for+REST+APIs >> but cannot find there any REST API URL we could use. >> >> Is it possible to query the dashboard worker as it is from external >> applications using REST APIs ? >> >> >> Thanks >> Bernard >> >> ___ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> > > > -- > Sajith Janaprasad Ariyarathna > WSO2, Inc.; http://wso2.com/ > <https://wso2.com/signature> > ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] [EI Analytics 6.4.0] Usecase of Message flow widget
Hi all, What are the publishers for the message flow widget in EI Analytics dashboard? Best Regards, *Niveathika Rajendran,* *Senior Software Engineer.* *Mobile : +94 077 903 7536* -- Forwarded message - From: Niveathika Rajendran Date: Wed, Dec 19, 2018 at 2:05 PM Subject: Re: Connect ESB to SP To: Reza Ameri Hi Reza, Glad to hear it! As message flow widget is a custom widget, this wiring is handled in the code [1]. That's why this wiring is not explicitly done in the MessageFlow widget case. Furthermore, as I am unaware of the exact use case of message flow widget, I am routing this email to the WSO2 Developers list. [1] https://github.com/wso2/analytics-solutions/blob/master/components/ei-analytics/org.wso2.analytics.solutions.ei.analytics/widgets/EIAnalyticsMessageFlow/src/EIAnalyticsMessageFlow.jsx#L1208 Best Regards, *Niveathika Rajendran,* *Senior Software Engineer.* *Mobile : +94 077 903 7536* On Mon, Dec 17, 2018 at 9:07 PM Reza Ameri wrote: > It worked. Thank you. > I am wondering how the original MessageFlow widget works. Its > configuration is not like what I have made; it does subscribe to other > widgets but it does not have parameters like mine. Also, the other > difference between my widget and the original one is that by clicking on > any message row, you can see the message flow graphically. > > Thank you Niveathika, > Reza > > *Reza Ameri* > Software Architect > https://www.linkedin.com/in/rezaameri/ > > > On Fri, Dec 14, 2018 at 1:47 PM Niveathika Rajendran > wrote: > >> Hi Reza, >> >> In the configuration pane, under the tick box, you will see the receiving >> properties followed by a drop down. This has to be connected properly. In >> your case, componentName should be connected to selectedComponent as the >> component name is published as selectedComponent[1] >> >> >> [1] >> https://github.com/wso2/analytics-solutions/blob/master/components/ei-analytics/org.wso2.analytics.solutions.ei.analytics/widgets/EIAnalyticsSearchBox/src/resources/widgetConf.json#L11 >> >> Best Regards, >> *Niveathika Rajendran,* >> *Senior Software Engineer.* >> *Mobile : +94 077 903 7536* >> >> >> >> >> >> On Thu, Dec 13, 2018 at 5:37 PM Reza Ameri wrote: >> >>> Yes, it is. >>> >>> *Reza Ameri* >>> Software Architect >>> https://www.linkedin.com/in/rezaameri/ >>> >>> >>> On Thu, Dec 13, 2018 at 3:32 PM Niveathika Rajendran < >>> niveath...@wso2.com> wrote: >>> >>>> Hi Reza, >>>> >>>> When you meant filter changes, is the component Name change? >>>> >>>> Best Regards, >>>> *Niveathika Rajendran,* >>>> *Senior Software Engineer.* >>>> *Mobile : +94 077 903 7536* >>>> >>>> >>>> >>>> >>>> >>>> On Thu, Dec 13, 2018 at 4:32 PM Reza Ameri wrote: >>>> >>>>> Hi, >>>>> I did what you said and unfortunately, it does not work for me! >>>>> >>>>> [image: image.png] >>>>> >>>>> >>>>> >>>>> *Reza Ameri* >>>>> Software Architect >>>>> https://www.linkedin.com/in/rezaameri/ >>>>> >>>>> >>>>> On Thu, Dec 13, 2018 at 1:27 PM Niveathika Rajendran < >>>>> niveath...@wso2.com> wrote: >>>>> >>>>>> Hi Reza, >>>>>> >>>>>> If you have removed and added a new widget. the wiring between the >>>>>> publishers and subscribers instances have to be done again. >>>>>> >>>>>> Please see the subsection, "Wring publishers and subscribers"[1] on >>>>>> how to do wire publishers and subscribers. >>>>>> >>>>>> [1] >>>>>> https://docs.wso2.com/display/SP4xx/Inter+Widget+Communication+via+the+PubSub+Model >>>>>> >>>>>> Best Regards, >>>>>> *Niveathika Rajendran,* >>>>>> *Senior Software Engineer.* >>>>>> *Mobile : +94 077 903 7536* >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Dec 13, 2018 at 3:11 PM Reza Ameri >>>>>> wrote: >>>>>> >>>>>>> Hello again, >>>>>>> >>>>>>> Thank you. your links worked. I am trying to change the Message >>>&g
Re: [Dev] Host header is missing when using siddhi-io-http
Hi Sujanan, As indicated in the issue, this bug was identified before and the fix is available from version 1.0.40 You can get this fix by removing the siddhi-io-http-*.jar in the {SP_HOME}/lib directory and add the latest version of the extension. Best Regards, *Niveathika Rajendran,* *Senior Software Engineer.* *Mobile : +94 077 903 7536* On Thu, Dec 13, 2018 at 12:50 PM Sujanan Bhathiya wrote: > HI, > > As a part of project "TeleSign two-factor authenticator for IAM" which I > was assigned, following strange behavior was identified while configuring > the Stream Processor. In this scenario my goal was to configure the > Identity Server for risk based adaptive authentication with the help of > Stream Processor using an external API which means the risk score is > fetched from a external API. In order to this to get succeed I wrote a > Siddhi App using siddhi-io-http extension. First I checked with a mock API > which is locally established for testing purposes and it worked correctly. > Later I changed the "publisher.url" parameter in "http-request" type in > @sink to remote API's URL and the response was a "400 Bad Request" from the > remote server. Based on a offline discussion with Tishan we Identified > that we should set a HOST header in our request to get a successful result. > So he opened this issue > <https://github.com/wso2-extensions/siddhi-io-http/issues/99>. > > For now as a workaround I set the host header manually and it worked. Any > suggestions on this? > > -- > *Sujanan Bhathiya* | Intern | WSO2 Inc. > (m) (+94) 710830823 | (e) suja...@wso2.com > GET INTEGRATION AGILE > Integration Agility for Digitally Driven Business > ___ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Remote login to analytics ok but SQL errors on EI_Analytics_StatApp
Hi Bernard, The ConfigEntryTable[1] is used to store the Tracing details of an EI flow, if the flow is really long this can easily exceed the "6000" character limit set in the coloumn definition. If the coloum length is not enough you can increase it by increasing the > field.length="configData:6000" in the siddhi app residing in /wso2/analytics/wso2/worker/deployment/siddhi-files directory. Please note for this to be valid, the table must be deleted and created again by deploying the siddhi app. Please note, tracing is not recommened to be enabled y default in production environment as it will lead to high amount of data being persisted which can lead to performance drop. Hope this solves the problem! [1] https://github.com/wso2/analytics-solutions/blob/master/features/ei-analytics-feature/org.wso2.analytics.solutions.ei.analytics.feature/src/main/resources/siddhi-files/EI_Analytics_StatApp.siddhi#L51 Best Regards, *Niveathika Rajendran,* *Senior Software Engineer.* *Mobile : +94 077 903 7536* On Wed, Nov 28, 2018 at 9:13 PM Bernard Paris wrote: > Hi Niveathika > > I checked my carbon.xml and found missing block related to > > I corrected it and now the client can connect to analytics, great ! > > Anyway, the message about any "CryptoService.Secret" i missing is still > logged at server startup. > The property "CryptoService.Secret" is never referenced in any of the > config files and the documentation I found do not tell a word about it. > > But ok, I can live with this. > The connexion between the agent and analytics server is now ok, I can see > datas report in the analytics dashboard console. > Anyway there are some errors in the worker logs related to data collects: > it seems some SQL request are wrong: > > [2018-11-28 16:30:07,921] ERROR > {org.wso2.siddhi.core.stream.input.source.InputEventHandler} - Error on > 'EI_Analytics_StatApp'. Cannot execute update/insert operation (update) on > table 'ConfigEntryTable' with SQL query INSERT INTO ConfigEntryTable > (metaTenantId , hashcode , entryName , configData , eventTimestamp ) > VALUES (?, ?, ?, ?, ?) . Error in applying transport property mapping for > 'wso2event' source at 'ConfigEntryEventStream' stream. > org.wso2.extension.siddhi.store.rdbms.exception.RDBMSTableException: Error > on 'EI_Analytics_StatApp'. Cannot execute update/insert operation (update) > on table 'ConfigEntryTable' with SQL query INSERT INTO ConfigEntryTable > (metaTenantId , hashcode , entryName , configData , eventTimestamp ) > VALUES (?, ?, ?, ?, ?) . > at > org.wso2.extension.siddhi.store.rdbms.RDBMSEventTable.batchProcessInsert(RDBMSEventTable.java:733) > at > org.wso2.extension.siddhi.store.rdbms.RDBMSEventTable.updateOrAdd(RDBMSEventTable.java:622) > at > org.wso2.siddhi.core.table.record.AbstractRecordTable.updateOrAdd(AbstractRecordTable.java:323) > at > org.wso2.siddhi.core.table.Table.updateOrAddEvents(Table.java:288) > at > org.wso2.siddhi.core.query.output.callback.UpdateOrInsertTableCallback.send(UpdateOrInsertTableCallback.java:78) > > > Thanks, > Bernard > > > > Le 27 nov. 2018 à 17:35, Niveathika Rajendran a > écrit : > > Hi Bernard, > > Yes, it could be. > > After analysing the workflow, I have concluded the following, > 1. Encryption failing - Leads to an empty password > 2. The login credentials send to SP will be as follows, "admin:" i.e, > username: admin and password empty > 3. SP is refusing connection as it is a NOT valid credential > > Since the encryption is failing, please check on the mentioned configs in > "Configuring Keystores[1]" > > [1] > https://docs.wso2.com/display/ADMIN44x/Configuring+Keystores+in+WSO2+Products > > Best Regards, > * Niveathika Rajendran,* > *Senior Software Engineer.* > *Mobile : +94 077 903 7536* > > > > > > On Tue, Nov 27, 2018 at 9:30 PM Bernard Paris > wrote: > >> >> Starting EI-640 server I see this >> >> [2018-11-27 16:56:33,612] [EI-Core] INFO - >> DefaultCryptoProviderComponent 'CryptoService.Secret' property has not been >> set. 'org.wso2.carbon.crypto.provider.SymmetricKeyInternalCryptoProvider' >> won't be registered as an internal crypto provider. Please set the secret >> if the provider needs to be registered. >> >> What is about ? Could it be related with the issue ? >> >> >> Bernard >> >> >> Le 27 nov. 2018 à 11:57, Bernard Paris a >> écrit : >> >> Hi, >> >> I have found a data-agent-config.xml file under >> {EI_HOME}/conf/data-bridge/ where lines about SSL and >> client-trustore.jks were commented by default. &g
[Dev] EI Analytics stats are not visible in WSO2 Stream Processor based dashboard
Hi Reza, This is due to a default configuration bug[1] we identified recently and have send a fix[2]. Please do the same in the deployment.yaml file found under /conf/worker folder and the stats should be visible. Moreover, please see "Configuring Datasource"[1] to configure different type of datasources for further testing, since H2 is not recommened on production environment. Please note the configurations have to bechanged in both worker and dashboard deplyment.yaml files. [1] https://github.com/wso2/product-sp/issues/887 [2] https://github.com/wso2/product-sp/pull/888 [3] https://docs.wso2.com/display/SP430/Configuring+Datasources Best Regards, *Niveathika Rajendran,* *Senior Software Engineer.* *Mobile : +94 077 903 7536* On Thu, Nov 29, 2018 at 2:27 PM Reza Ameri wrote: > Hi, > > I have opened an issue on github[1], I really need to solve the problem > would you help me with this? > > Thanks, > Reza > > [1] https://github.com/wso2/product-sp/issues/867 > > *Reza Ameri* > Software Architect > https://www.linkedin.com/in/rezaameri/ > ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] remote login to analytics worker issue
Hi Bernard, Yes, it could be. After analysing the workflow, I have concluded the following, 1. Encryption failing - Leads to an empty password 2. The login credentials send to SP will be as follows, "admin:" i.e, username: admin and password empty 3. SP is refusing connection as it is a NOT valid credential Since the encryption is failing, please check on the mentioned configs in "Configuring Keystores[1]" [1] https://docs.wso2.com/display/ADMIN44x/Configuring+Keystores+in+WSO2+Products Best Regards, *Niveathika Rajendran,* *Senior Software Engineer.* *Mobile : +94 077 903 7536* On Tue, Nov 27, 2018 at 9:30 PM Bernard Paris wrote: > > Starting EI-640 server I see this > > [2018-11-27 16:56:33,612] [EI-Core] INFO - DefaultCryptoProviderComponent > 'CryptoService.Secret' property has not been set. > 'org.wso2.carbon.crypto.provider.SymmetricKeyInternalCryptoProvider' won't > be registered as an internal crypto provider. Please set the secret if the > provider needs to be registered. > > What is about ? Could it be related with the issue ? > > > Bernard > > > Le 27 nov. 2018 à 11:57, Bernard Paris a > écrit : > > Hi, > > I have found a data-agent-config.xml file under > {EI_HOME}/conf/data-bridge/ where lines about SSL and > client-trustore.jks were commented by default. > > I uncommented these lines and restart the server from which datas are to > be sent to remote analytics but no way, that does absolutely no changes to > my issue: > Caused by: ThriftAuthenticationException(message:wrong userName or > password) > > Is it necessarily to define client-trustore in this file ? (what is the > one used if not) > > Can someone explain how the password in MessageFlowStatisticsPublisher.xml > and MessageFlowConfigurationPublisher.xml are being cyphered by the agent > then uncyphered on server side ? > > Thanks, > Bernard > > > Le 26 nov. 2018 à 09:46, Bernard Paris a > écrit : > > Hello Niveathika > > > Le 19 nov. 2018 à 14:53, Niveathika Rajendran a > écrit : > > Hi Bernard, > > In your earlier email, you mentioned that the config changes as below on > your remote machine. > > > > Is the issue still continuing? > > > > yes > > > > Could you check configurations under the tag in > /conf/carbon.xml? I was able to reproduce this *only* with a > faulty configuration[1]. > > > > my keystore is named wso2carbon.jks, alias and password same as originals; > I imported my own certificate in this keystore under alias wso2carbon and > set the passwd to wso2carbon > > > keytool -v -list -keystore wso2carbon.jks -alias wso2carbon > > Alias name: wso2carbon > Creation date: Nov 15, 2018 > Entry type: PrivateKeyEntry > Certificate chain length: 3 > Certificate[1]: > Owner: CN=wso2ei.sgsi.ucl.ac.be, OU=SGSI, O=Université catholique de > Louvain, L=Ottignies-Louvain-la-Neuve, C=BE > Issuer: CN=TERENA SSL CA 3, O=TERENA, L=Amsterdam, ST=Noord-Holland, C=NL > Serial number: 516e3c8586ebb5e0aab45fdb05ae071 > > > I'm sure the passwd has been changed correctly because: > > $ keytool -v -keypasswd -alias wso2carbon -keystore wso2carbon.jks > Enter keystore password: > New key password for : > Passwords must differ > New key password for : > > The same keystore jks file is used by all EI servers (EI servers are load > balanced); and the certificate have been imported in the > client-trustore.jks of analytics. > > > > What can explain that remote servers cannot login into Analytics as >> localhost does ? Are there any grants to set somewhere in Analytics for >> remote access ? > > Regarding the above question, In SP we do not have grants for remote > access. The certificate keys and the hostname will be verified. Please note > for hostname verification, it must be enabled explicitly by adding the > below configuration in the > /wso2/analytics/conf/worker/deployment.yaml file. > >> wso2.carbon: >> >> hostnameVerificationEnabled: true >>> >> > > > I did this but that not resolve the problem. > > > Regards from Belgium, > > Bernard > > > > > > > [1] > https://stackoverflow.com/questions/50642058/how-to-solve-thrift-authentication-exception-wrong-username-or-password-in-wso?answertab=votes#tab-top > > > Best Regards, > * Niveathika Rajendran,* > *Senior Software Engineer.* > *Mobile : +94 077 903 7536* > > > > > > On Mon, Nov 19, 2018 at 4:03 PM Bernard Paris > wrote: > >> Hello, >> >> after 3 weeks of investigations, no way about this issue: remote >> EI-6.4.0 servers definitively cannot send their
[Dev] [EI-Analytics] Top Invocation widget shows data only from the 'MINUTE' aggregation
Hi all, The above widget[1] fetches data from the invocation aggregation for a specific period. Even though the Date Time Range Filter consists of granularity selector, the fetch query is hardcoded to "minute" granularity [2] . [image: Screen Shot 2018-09-26 at 4.53.37 PM.png] For argument sake, let's say the API was not invoked for the last 3 days, here even if the user selects "*7 Days per Day*", we would not be showing any data, since the "Minute" table only keeps records for last 3 days due to purging. IMO, this is misleading. Shall we fix it such that the user selected granularity is passed to the query itself? Your thoughts on the matter are appreciated. [1] https://docs.wso2.com/display/EI640/Analyzing+the+ESB+Profile+Statistics+Overview#AnalyzingtheESBProfileStatisticsOverview-TopAPIsbyRequestCount [2] https://github.com/wso2/analytics-solutions/blob/master/components/ei-analytics/org.wso2.analytics.solutions.ei.analytics/widgets/EIAnalyticsHorizontalBarChart/src/resources/widgetConf.json#L17 Best Regards, *Niveathika Rajendran,* *Senior Software Engineer.* *Mobile : +94 077 903 7536* ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] upgrading to analytics 6.4.0
Hi Bernard, The new EI Analytics profile is based on Carbon5, which is designed to run with minimum configurations. As mentioned in the documentation, these are the default configurations applicable. You can override it by adding the 'auth.configs' element in the deployment.yaml file found under {EI_HOME}//wso2/analytics/conf/worker/ directory. The following configs will add another user to the system namely admin2/admin auth.configs: > type: 'local' > userManager: > adminRole: admin > userStore: > users: >- > user: >username: admin >password: YWRtaW4= >roles: 1 > - user: >username: admin2 >password: YWRtaW4= roles: 1 roles: >- > role: >id: 1 > displayName: admin > Best Regards, *Niveathika Rajendran,* *Senior Software Engineer.* */Mobile : +94 077 903 7536* On Wed, Nov 7, 2018 at 2:46 PM Bernard Paris wrote: > Hello Niveathika > > in the analytics distribution inside EI_640, I can't find any file-based > user store you are talking about. > Neither I can't find any auth.configs to configure in deployment.yaml as > described at https://docs.ws > o2.com/display/SP430/User+Management+via+the+IdP+Client+Interface > > To answer your questions here is my config just before deployment > tcp://10.1.3.12:7612 > admin > > > Thanks for nay help, regards from Belgium, > Bernard > > > Le 6 nov. 2018 à 14:04, Niveathika Rajendran a > écrit : > > Hi Bernard, > > Latest EI-Analytics profile is based on WSO2 Stream Processor for which > the configurations are different from the previous versions. > > EI-Analytics 6.4.0 by default uses a file-based user store(via Local IdP > Client). As you have mentioned, by default admin/admin is the user > credentials used. with admin/admin credentials EI instance should have been > able to publish to Analytics profile w/o any issue. > > Could you give follwing information to reproduce this issue? > 1. Is the TCP protocol used for publishing or SSL protocol? (i.e > tcp://localhost:7612 ) > 2. Is the password encrypted? (i.e name="password">admin) > > In the meantime, please refer to "User Management via IdP client > Interface" for more information on setting up the user store. > > Best Regards, > Niveathika Rajendran, > Senior Software Engineer. > Mobile : +94 077 903 7536 > > > > > > On Tue, Nov 6, 2018 at 3:59 PM Bernard Paris > wrote: > Hi Ramindu, > > thanks for this. > > Effectively I saw that everything has been changed in new analytics > distribution, and I can't cope with it ;-( > By default I don't see any users stuffs, like the use user-mgt.xml file. > So I suppose the "admin" user is just store in the DB, nothing else. > > I didn't change the default password for it, but remote ESB instances are > unable to send data to analytics because of > ERROR > {org.wso2.carbon.databridge.core.internal.authentication.CarbonAuthenticationHandler} > - Authentication failed for username 'admin'. Error : > 'Invalid_Credentials'. Error Description : 'The login credential used for > login are invalid, username : 'admin'.' > ERROR > {org.wso2.carbon.databridge.core.internal.authentication.Authenticator} - > wrong userName or password > > Local instance (I mean IE640 running on same localhost) can. > > I just edited this files to put admin/admin as user/passwd > /repository/deployment/server/eventpublishers/MessageFlowConfigurationPublisher.xml > & MessageFlowStatisticsPublisher.xml > > > > Thanks for your help, > Bernard > > > Le 1 nov. 2018 à 14:34, Ramindu De Silva a écrit : > > Hi Bernard, > > EI analytics 6.3.0 uses WSO2 DAS runtime. EI analytics 6.4.0 uses WSO2 > SP runtime. WSO2 SP is a total rewrite of the WSO2 DAS, which has a feature > enhancements as well. > > On Tue, Oct 23, 2018 at 1:50 PM Bernard Paris > wrote: > Hi devs, > > we were using these only 4 DBs to make analytics aggregating datas from > our ESB until version 6.3.0 ; > ANALYTICS_CARBON_DB > ANALYTICS_METRICS_DB > ANALYTICS_EVENT_STORE_DB > ANALYTICS_PROCESSED_DATA_STORE_DB > We have our from the scratch data aggregation mechanism instead of using > apache spark which was in DAS. And that the the explanation for not having > configurations for ANALYTICS_EVENT_STORE_DB and > ANALYTICS_PROCESSED_DATA_STORE_DB's . > We still have the ANALYTICS_CARBON_DB and ANALYTICS_METRICS_DB. > Please look into Monitoring Stream Processor in-order to configure the > metrics for WSO2 SP. > > These were postgres databases. >
Re: [Dev] upgrading to analytics 6.4.0
Hi Bernard, Latest EI-Analytics profile is based on WSO2 Stream Processor for which the configurations are different from the previous versions. EI-Analytics 6.4.0 by default uses a file-based user store(via Local IdP Client). As you have mentioned, by default admin/admin is the user credentials used. with admin/admin credentials EI instance should have been able to publish to Analytics profile w/o any issue. Could you give follwing information to reproduce this issue? 1. Is the TCP protocol used for publishing or SSL protocol? (i.e *tcp*://localhost:7612 ) 2. Is the password encrypted? (i.e admin) In the meantime, please refer to "User Management via IdP client Interface <https://docs.wso2.com/display/SP430/User+Management+via+the+IdP+Client+Interface>" for more information on setting up the user store. Best Regards, *Niveathika Rajendran,* *Senior Software Engineer.* *Mobile : +94 077 903 7536* On Tue, Nov 6, 2018 at 3:59 PM Bernard Paris wrote: > Hi Ramindu, > > thanks for this. > > Effectively I saw that everything has been changed in new analytics > distribution, and I can't cope with it ;-( > By default I don't see any users stuffs, like the use user-mgt.xml file. > So I suppose the "admin" user is just store in the DB, nothing else. > > I didn't change the default password for it, but remote ESB instances are > unable to send data to analytics because of > ERROR > {org.wso2.carbon.databridge.core.internal.authentication.CarbonAuthenticationHandler} > - Authentication failed for username 'admin'. Error : > 'Invalid_Credentials'. Error Description : 'The login credential used for > login are invalid, username : 'admin'.' > ERROR > {org.wso2.carbon.databridge.core.internal.authentication.Authenticator} - > wrong userName or password > > Local instance (I mean IE640 running on same localhost) can. > > I just edited this files to put admin/admin as user/passwd > /repository/deployment/server/eventpublishers/MessageFlowConfigurationPublisher.xml > & MessageFlowStatisticsPublisher.xml > > > > Thanks for your help, > Bernard > > > Le 1 nov. 2018 à 14:34, Ramindu De Silva a écrit : > > Hi Bernard, > > EI analytics 6.3.0 uses WSO2 DAS <https://docs.wso2.com/display/DAS320> > runtime. EI analytics 6.4.0 uses WSO2 SP > <https://docs.wso2.com/display/SP430> runtime. WSO2 SP is a total rewrite > of the WSO2 DAS, which has a feature enhancements as well. > > On Tue, Oct 23, 2018 at 1:50 PM Bernard Paris > wrote: > >> Hi devs, >> >> we were using these only 4 DBs to make analytics aggregating datas from >> our ESB until version 6.3.0 ; >> >> ANALYTICS_CARBON_DB >> ANALYTICS_METRICS_DB >> ANALYTICS_EVENT_STORE_DB >> ANALYTICS_PROCESSED_DATA_STORE_DB >> >> We have our from the scratch data aggregation mechanism > <https://wso2.github.io/siddhi/documentation/siddhi-4.0/#aggregate-function> > instead of using apache spark which was in DAS. And that the the > explanation for not having configurations for ANALYTICS_EVENT_STORE_DB and > ANALYTICS_PROCESSED_DATA_STORE_DB's . > We still have the ANALYTICS_CARBON_DB and ANALYTICS_METRICS_DB. > Please look into Monitoring Stream Processor > <https://docs.wso2.com/display/SP430/Monitoring+Stream+Processor> in-order > to configure the metrics for WSO2 SP. > >> >> >> These were postgres databases. >> Now I see there are more then 10 databases preconfigured in the default >> analytics 6.4.0 config (conf/dashboard/deployment.yaml >> and conf/worker/deployment.yaml). >> Well, … this is suggesting me lot of questions. >> >> First of all, is it still recommended (like it is for ESB databases) >> to do *not* use local H2 databases in production environment ? >> This question comes because the 6.4.0 analytics seems to me to be used as >> it is, out of the box, lot of DBs and no documentation for a any >> configuration as it was for previous versions. >> > Yes. We still recommend NOT to use the embedded H2 databases. > >> >> If we need to create external DBs for all the stuffs, what exactly are >> each DB for ? >> > Please refer Configuring Datasources > <https://docs.wso2.com/display/SP430/Configuring+Datasources> > >> >> Is there any migration tool and/or documentation about migrating from >> analytics 6.3.0 to 6.4.0 ? ( >> https://docs.wso2.com/display/EI640/Upgrading+from+WSO2+EI+6.3.0 does't >> talk about that) >> > > >> Any matching between former 4 DBs and the 6.4.0 new ones ? >> > ANALYTICS_CARBON_DB - In-order to use/ migrate this database the previous > analytics, please answer the followi
Re: [Dev] Allowing Siddhi Aggregations to run independently in different data centers
Hi Suho, In this case, how would the dashboards be working? Is it separate dashboards for different nodes or a combined one? Best Regards, *Niveathika Rajendran,* *Senior Software Engineer.* *Mobile : +94 077 903 7536* On Sun, Nov 4, 2018 at 8:39 PM Niveathika Rajendran wrote: > Yes, that would solve the problem. > > Best Regards, > *Niveathika Rajendran,* > *Senior Software Engineer.* > *Mobile : +94 077 903 7536* > > > > > > On Sun, Nov 4, 2018 at 8:39 PM Sriskandarajah Suhothayan > wrote: > >> >> >> On Sun, Nov 4, 2018 at 8:10 PM Niveathika Rajendran >> wrote: >> >>> Hi Suho, >>> >>> As per the offline discussion with @Shehan Panditharatne >>> , the aggregation was run independently, however >>> with the same data source. The issue we faced was that when in-memory >>> buckets were calculated from the database, there were overlaps. >>> For instance when "SP1" node has already stored the second data, then >>> only the SP2 node gets the first message, it had loaded "SP1" nodes >>> aggregation from the database, thus we faced duplicate key exceptions. >>> >>> AFAIK, this shouldn't occur if we use different data centres. >>> >> Even in that case if we are sharing the DB this will be a problem is it? >> How about using the node ID and filtering the data when loading the >> messages from DB. Will that solve the problem? >> >> Regards >> Suho >> >> >>> Best Regards, >>> *Niveathika Rajendran,* >>> *Senior Software Engineer.* >>> *Mobile : +94 077 903 7536* >>> >>> >>> >>> >>> >>> On Sun, Nov 4, 2018 at 5:01 PM Sriskandarajah Suhothayan >>> wrote: >>> >>>> Hi Nivedika >>>> >>>> This seems not to be possible, can you explain reason for failure? >>>> >>>> Regards >>>> Suho >>>> >>>> On Sat, Nov 3, 2018 at 5:56 PM Shehan Panditharatne >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> I have attached the modified siddhi app for two parallel worker nodes. >>>>> I have added the attribute node to PreProcessedESBStatStream and >>>>> PreProcessedMediatorStatStream. It will be "sp1" in the first instance and >>>>> "sp2" in thee second instance. >>>>> >>>>> Best regards >>>>> >>>>> -- >>>>> Shehan Panditharatne >>>>> Software Engineer Intern | WSO2, Inc >>>>> Mobile: +94771554990 >>>>> >>>> >>>> >>>> -- >>>> *S. Suhothayan* | Director | WSO2 Inc. <https://wso2.com/> >>>> (m) (+94) 779 756 757 | (e) s...@wso2.com | (t) @suhothayan >>>> <https://twitter.com/suhothayan> >>>> GET INTEGRATION AGILE >>>> Integration Agility for Digitally Driven Business >>>> >>> >> >> -- >> *S. Suhothayan* | Director | WSO2 Inc. <https://wso2.com/> >> (m) (+94) 779 756 757 | (e) s...@wso2.com | (t) @suhothayan >> <https://twitter.com/suhothayan> >> GET INTEGRATION AGILE >> Integration Agility for Digitally Driven Business >> > ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Allowing Siddhi Aggregations to run independently in different data centers
Yes, that would solve the problem. Best Regards, *Niveathika Rajendran,* *Senior Software Engineer.* *Mobile : +94 077 903 7536* On Sun, Nov 4, 2018 at 8:39 PM Sriskandarajah Suhothayan wrote: > > > On Sun, Nov 4, 2018 at 8:10 PM Niveathika Rajendran > wrote: > >> Hi Suho, >> >> As per the offline discussion with @Shehan Panditharatne >> , the aggregation was run independently, however with >> the same data source. The issue we faced was that when in-memory buckets >> were calculated from the database, there were overlaps. >> For instance when "SP1" node has already stored the second data, then >> only the SP2 node gets the first message, it had loaded "SP1" nodes >> aggregation from the database, thus we faced duplicate key exceptions. >> >> AFAIK, this shouldn't occur if we use different data centres. >> > Even in that case if we are sharing the DB this will be a problem is it? > How about using the node ID and filtering the data when loading the > messages from DB. Will that solve the problem? > > Regards > Suho > > >> Best Regards, >> *Niveathika Rajendran,* >> *Senior Software Engineer.* >> *Mobile : +94 077 903 7536* >> >> >> >> >> >> On Sun, Nov 4, 2018 at 5:01 PM Sriskandarajah Suhothayan >> wrote: >> >>> Hi Nivedika >>> >>> This seems not to be possible, can you explain reason for failure? >>> >>> Regards >>> Suho >>> >>> On Sat, Nov 3, 2018 at 5:56 PM Shehan Panditharatne >>> wrote: >>> >>>> Hi, >>>> >>>> I have attached the modified siddhi app for two parallel worker nodes. >>>> I have added the attribute node to PreProcessedESBStatStream and >>>> PreProcessedMediatorStatStream. It will be "sp1" in the first instance and >>>> "sp2" in thee second instance. >>>> >>>> Best regards >>>> >>>> -- >>>> Shehan Panditharatne >>>> Software Engineer Intern | WSO2, Inc >>>> Mobile: +94771554990 >>>> >>> >>> >>> -- >>> *S. Suhothayan* | Director | WSO2 Inc. <https://wso2.com/> >>> (m) (+94) 779 756 757 | (e) s...@wso2.com | (t) @suhothayan >>> <https://twitter.com/suhothayan> >>> GET INTEGRATION AGILE >>> Integration Agility for Digitally Driven Business >>> >> > > -- > *S. Suhothayan* | Director | WSO2 Inc. <https://wso2.com/> > (m) (+94) 779 756 757 | (e) s...@wso2.com | (t) @suhothayan > <https://twitter.com/suhothayan> > GET INTEGRATION AGILE > Integration Agility for Digitally Driven Business > ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Allowing Siddhi Aggregations to run independently in different data centers
Hi Suho, As per the offline discussion with @Shehan Panditharatne , the aggregation was run independently, however with the same data source. The issue we faced was that when in-memory buckets were calculated from the database, there were overlaps. For instance when "SP1" node has already stored the second data, then only the SP2 node gets the first message, it had loaded "SP1" nodes aggregation from the database, thus we faced duplicate key exceptions. AFAIK, this shouldn't occur if we use different data centres. Best Regards, *Niveathika Rajendran,* *Senior Software Engineer.* *Mobile : +94 077 903 7536* On Sun, Nov 4, 2018 at 5:01 PM Sriskandarajah Suhothayan wrote: > Hi Nivedika > > This seems not to be possible, can you explain reason for failure? > > Regards > Suho > > On Sat, Nov 3, 2018 at 5:56 PM Shehan Panditharatne > wrote: > >> Hi, >> >> I have attached the modified siddhi app for two parallel worker nodes. >> I have added the attribute node to PreProcessedESBStatStream and >> PreProcessedMediatorStatStream. It will be "sp1" in the first instance and >> "sp2" in thee second instance. >> >> Best regards >> >> -- >> Shehan Panditharatne >> Software Engineer Intern | WSO2, Inc >> Mobile: +94771554990 >> > > > -- > *S. Suhothayan* | Director | WSO2 Inc. <https://wso2.com/> > (m) (+94) 779 756 757 | (e) s...@wso2.com | (t) @suhothayan > <https://twitter.com/suhothayan> > GET INTEGRATION AGILE > Integration Agility for Digitally Driven Business > ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] [SP] All user input in a business rules template is option
Hi All, As of the current implementation, we have assumed that the user input of the templated field is optional, i.e, always a default value should be provided in the template. However, in some cases, it is not possible to give default values such, 1. Latitude, longitude field templated siddhi apps used for Geofencing 2. App Id in twitter source IMO, we should have another property added to the template field such as "isUserInputMandatory". In this case, we should validate that all properties should have non-empty values(Default values is not applicable). *Workaround Tried:* I tried to add a script to validate that the input is non-empty. However, then the default values will be validated at the template loading thus the template becomes broken. Your thoughts on the matter are much appreciated. Github Issue: https://github.com/wso2/carbon-analytics/issues/1465 Best Regards, *Niveathika Rajendran,* *Software Engineer.* *Mobile : +94 077 903 7536* ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [VOTE] Release of WSO2 Stream Processor 4.3.0 RC3
Hi all, I have tested the following, 1. Local IdPClient 2. Login/Logout Rest API 3. Authentication Interceptor 4. HTTP Analytics Solution [worker/dashboard] 5. Monitoring dashboard for a single node [Node view/App view] 6. Editor sample: - MongoDB - Aggregate Data Incrementally [+] Stable go ahead and release Best Regards, *Niveathika Rajendran,* *Software Engineer.* *Mobile : +94 077 903 7536* On Mon, Sep 17, 2018 at 1:55 AM Sajith Perera wrote: > Tested the following. > > Siddhi Stores Query API[editor/worker runtime] > Distributed Message Tracer[worker/dashboard runtime] > SP Distributed Deployment[2 Manger/ 5 Worker(including single receiver) > runtimes] > Editor Sample: > - Store RDBMS MySQL/Oracle[editor] > - Store Cassedra[editor] > > No blockers found. > > [+] Stable - go ahead and release > > Thanks, > SajithD > > On Sat, Sep 15, 2018 at 1:05 AM, Dilini Muthumala wrote: > >> Hi all, >> >> >> WSO2 Stream Processor team is pleased to announce the third >> release candidate of WSO2 Stream Processor 4.3.0. >> >> >> WSO2 Stream Processor is an open source embodiment of the WSO2 Analytics >> platform, of which the real-time, incremental & intelligent data processing >> capabilities let digital businesses create actionable business insights and >> data products. >> >> >> Please find the improvements and fixes related to this release: >> >> - siddhi >> <https://github.com/wso2/siddhi/issues?utf8=%E2%9C%93=is%3Aissue+closed%3A2018-06-20..2018-09-15> >> >> - carbon-analytics-common >> <https://github.com/wso2/carbon-analytics-common/issues?utf8=%E2%9C%93=is%3Aissue+closed%3A2018-06-20..2018-09-15> >> >> - carbon-analytics >> <https://github.com/wso2/carbon-analytics/issues?utf8=%E2%9C%93=is%3Aissue+closed%3A2018-06-20..2018-09-15> >> >> - carbon-dashboards >> <https://github.com/wso2/carbon-dashboards/issues?utf8=%E2%9C%93=is%3Aissue+closed%3A2018-06-20..2018-09-15> >> >> - analytics-solutions >> <https://github.com/wso2/analytics-solutions/issues?utf8=%E2%9C%93=is%3Aissue+closed%3A2018-06-20..2018-09-15> >> >> - product-sp >> <https://github.com/wso2/product-sp/issues?utf8=%E2%9C%93=is%3Aissue+closed%3A2018-06-20..2018-09-15> >> >> >> You can download the product distribution from: >> https://github.com/wso2/product-sp/releases/download/v4.3.0-rc3/wso2sp-4.3.0-rc3.zip >> >> >> The tag to be voted upon: >> *https://github.com/wso2/product-sp/releases/tag/v4.3.0-rc3 >> <https://github.com/wso2/product-sp/releases/tag/v4.3.0-rc3>* >> >> >> Please download, test the product and vote. >> >> >> [+] Stable - go ahead and release >> >> [-] Broken - do not release (explain why) >> >> >> Thanks, >> >> - WSO2 Stream Processor Team - >> > > > > -- > <http://wso2.com/signature> > Sajith Dimal > Seniror Software Engineer > Email : saji...@wso2.com > Mobile : +94783101496 > WSO2 Inc. | http://wso2.com > lean.enterprise.middleware > > > Disclaimer: This communication may contain privileged or other > confidential information and is intended exclusively for the addressee/s. > If you are not the intended recipient/s, or believe that you may have > received this communication in error, please reply to the sender indicating > that fact and delete the copy you received and in addition, you should not > print, copy, re-transmit, disseminate, or otherwise use the information > contained in this communication. Internet communications cannot be > guaranteed to be timely, secure, error or virus-free. The sender does not > accept liability for any errors or omissions. > ___ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Hour table aggregates data only when new event comes to the SP.
[+Adding Dev] Hi Chamila, In latest WSO2 SP RC1 pack, I ran the above scenario, the hour data aggregates on calendar hour end time. Please note that is in GMT time zone, ie. September 11, 2018 7:00:00 AM(Aggregation start time is 6AM in the record) and 12.30 pm IST. [image: Screenshot from 2018-09-11 12-31-16.png] Only one event was send to test this scenario at around 12.20pm IST. Best Regards, *Niveathika Rajendran,* *Software Engineer.* *Mobile : +94 077 903 7536* On Tue, Sep 11, 2018 at 12:13 PM Nuwan Dias wrote: > Shall we discuss these publicly please? I don't see a reason for us to > discuss internally only. > > On Tue, Sep 11, 2018 at 12:00 PM Chamila Adhikarinayake > wrote: > >> APIM has latency graphs which shows day related graphs and we use hour >> granularity to get the data within the day. following is the query >> >> >> {"appName":"APIM_ACCESS_SUMMARY","query":"from ApiExecutionTimeAgg >> on(apiName=='PizzaShackAPI' AND apiVersion=='1.0.0' AND >> apiCreatorTenantDomain=='carbon.super') within 153655954L, >> 153664594L per 'hours' select apiName, apiContext, apiCreator, >> apiVersion, AGG_TIMESTAMP, responseTime, securityLatency, >> throttlingLatency, requestMediationLatency, responseMediationLatency, >> backendLatency, otherLatency;"} >> >> >> While doing test, we noticed that with hours granularity, data is only >> getting aggregated only when an event is sent to the SP. We tried it for >> after keeping the system idle for more than one hour and didn't see the >> latest data. once we send a request, above query returns the aggregated >> data. >> >> isn't this aggregation suppose to work in the background? >> >> -- >> Regards, >> Chamila Adhikarinayake >> Associate Technical Lead >> WSO2, Inc. >> Mobile - +94712346437 >> Email - chami...@wso2.com >> Blog - http://helpfromadhi.blogspot.com/ >> > > > -- > Nuwan Dias > > Director - WSO2, Inc. http://wso2.com > email : nuw...@wso2.com > Phone : +94 777 775 729 > ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Using regex in Siddhi SQL queries
Hi Sajith, The use case is when retrieving data from aggregations, from StockStream as S join TradeAggregation as T on S.symbol == T.symbol within "2014-02-15 00:00:00 +05:30", "2014-03-16 00:00:00 +05:30" per "days" select S.symbol, T.total, T.avgPrice insert into AggregateStockStream; Here, On condition equals should support a regex match, for instance, S.symbol == "WSO2(.*)". As of now the equal operator only supports object match[1]. [1] https://github.com/wso2/siddhi/blob/master/modules/siddhi-core/src/main/java/org/wso2/siddhi/core/executor/condition/compare/equal/EqualCompareConditionExpressionExecutorStringString.java#L34 Best Regards, *Niveathika Rajendran,* *Software Engineer.* *Mobile : +94 077 903 7536* On Thu, Aug 9, 2018 at 11:04 AM Thisaru Guruge wrote: > Here's my Siddhi Code: > > @App:name("IsAnalytics_Authentication_Aggregations") > @App:description("Handle Event from IS and save data in a table for > Overall Authentication") > > @source(type='inMemory', > topic='IsAnalytics.Authentication.Common.OutputSinkFromProcessedAuthenticationDataStream', > @map(type='passThrough')) > define stream ProcessedAuthenticationDataStream ( > meta_tenantId int, > eventId string, > username string, > localUsername string, > userStoreDomain string, > tenantDomain string, > remoteIp string, > region string, > inboundAuthType string, > serviceProvider string, > rememberMeEnabled bool, > forceAuthEnabled bool, > passiveAuthEnabled bool, > rolesCommaSeparated string, > authenticationStep string, > identityProvider string, > authStepSuccess bool, > stepAuthenticator string, > isFirstLogin bool, > successValue int, > failureValue int, > stepSuccessValue int, > firstLoginValue int, > identityProviderType string, > timestamp long > ); > > define stream PerUserRolesDuplicateEventStream ( > meta_tenantId int, > eventId string, > username string, > localUsername string, > userStoreDomain string, > tenantDomain string, > remoteIp string, > region string, > inboundAuthType string, > serviceProvider string, > rememberMeEnabled bool, > forceAuthEnabled bool, > passiveAuthEnabled bool, > rolesCommaSeparated string, > authenticationStep string, > identityProvider string, > authStepSuccess bool, > stepAuthenticator string, > isFirstLogin bool, > successValue int, > failureValue int, > stepSuccessValue int, > firstLoginValue int, > identityProviderType string, > timestamp long, > token string > ); > > @store(type='rdbms', datasource='IS_ANALYTICS_DB') > define aggregation RoleAggregation > from PerUserRolesDuplicateEventStream > select > meta_tenantId, > username, > serviceProvider, > identityProvider, > token as role, > remoteIp, > region, > userStoreDomain, > sum(successValue) as authSuccessCount, > sum(failureValue) as authFailureCount, > sum(stepSuccessValue) as authStepSuccessCount, > identityProviderType, > timestamp > group by > meta_tenantId, > username, > serviceProvider, > token > aggregate by timestamp every second ... year; > > @store(type='rdbms', datasource='IS_ANALYTICS_DB') > define aggregation AuthenticationStatAggregation > from ProcessedAuthenticationDataStream > select > meta_tenantId, > username, > localUsername, > rolesCommaSeparated, > serviceProvider, > identityProvider, > remoteIp, > region, > userStoreDomain, > sum(successValue) as authSuccessCount, > sum(failureValue) as authFailureCount, > sum(stepSuccessValue) as authStepSuccessCount, > sum(firstLoginValue) as authFirstSuccessCount, > identityProviderType, > timestamp > group by > meta_tenantId, > username, > localUsername, > remoteIp, > rolesCommaSeparated > aggregate by timestamp every second ... year; > > from ProcessedAuthenticationDataStream#str:tokenize(rolesCommaSeparated, > ',') > insert into PerUserRolesDuplicateEventStream; > > I have two aggregations, AuthenticationStatAggregation and the > RoleAggregation. I need to retrieve aggregated data from the > AuthenticationStatAggregation, based on the role. > > Issue with the "IN" operator is that if we have user roles like "Admin", > "SystemAdmin", "LocalAdmin", etc., query will match "Admin" for all these > entries. One way to overco
Re: [Dev] Connect to a RDMBS datastore to retrieve data for a SP dashboard widget
@Rukshani Looks good. Thank you. Best Regards, *Niveathika Rajendran,* *Software Engineer.* *Mobile : +94 077 903 7536* On Mon, Jul 9, 2018 at 12:14 PM Rukshani Weerasinha wrote: > Hello Niveathika, > > I added a tip on this in the *Extending from base widget* section in [1], > [2], [3] and [4]. Can you please check? > > [1] https://docs.wso2.com/display/SP4xx/Creating+Custom+Widgets > [2] https://docs.wso2.com/display/SP400/Creating+Custom+Widgets > [3] https://docs.wso2.com/display/SP410/Creating+Custom+Widgets > [4] https://docs.wso2.com/display/SP420/Creating+Custom+Widgets > > Best Regards, > Rukshani. > > On Mon, Jul 9, 2018 at 12:07 PM, Nisala Nanayakkara > wrote: > >> Hi, >> >> @Nivethika: +1 for documenting it. >> @ Thisaru: If you are using a previous version of SP such as 4.0.0,4.1.0 >> or 4.2.0-rc1, Please upgrade it to SP 4.2.0-rc2 which is treated as the GA. >> So you can use the latest artefacts such as base-widget which will simplify >> the widget development. >> >> Thanks, >> Nisala >> >> On Mon, Jul 9, 2018 at 11:52 AM Niveathika Rajendran >> wrote: >> >>> Hi Nisala, >>> >>> Thank you for the explanation, as we have not documented the depended >>> base widget version, I thought they should be compatible with all versions >>> of SP. >>> >>> How about adding a note in SP creating custom widget documentation about >>> depended base widget version? >>> SP 4.2.0 -> @wso2-dashboards/widget: 1.2.2 >>> SP 4.1.0 -> @wso2-dashboards/widget: 1.1.0 >>> SP 4.0.0 -> @wso2-dashboards/widget: 1.1.0 >>> >>> Best Regards, >>> *Niveathika Rajendran,* >>> *Software Engineer.* >>> *Mobile : +94 077 903 7536* >>> >>> >>> >>> >>> >>> On Mon, Jul 9, 2018 at 12:48 AM Nisala Nanayakkara >>> wrote: >>> >>>> Hi Nivethika, >>>> >>>> @Damith What Thisaru means is that custom widgets(with a dependency of >>>> @wso2-dashboards/widgets:1.2.2) are not backward compatible. It only works >>>> in SP 4.2.0 >>>> >>>> AFAIK older widgets which are built for SP 4.0.0/4.1.0, work with SP >>>> 4.2.0 release. So the widgets' backward compatibility is preserved. Can you >>>> please clarify a bit more about this? As you mentioned earlier, >>>> wso2-dashboards/widgets only work with SP 4.2.0 release. Moreover, I think >>>> that although there is a requirement to support older widgets with 4.2.0 >>>> release version, there is no use-case to support widgets which are built >>>> for SP 4.2.0 to work with older versions such as SP 4.0.0/4.1.0. >>>> >>>> Thanks, >>>> Nisala >>>> >>>> On Fri, Jul 6, 2018 at 8:41 PM Niveathika Rajendran < >>>> niveath...@wso2.com> wrote: >>>> >>>>> @Damith What Thisaru means is that custom widgets(with a dependency of >>>>> @wso2-dashboards/widgets:1.2.2) are not backward compatible. It only works >>>>> in SP 4.2.0 (SP-4.2.0-rc2 is the successful release candidate, we still >>>>> waiting on quarterly release announcement to change the tags). This issue >>>>> is only for the getWidgetChannelManager API, other functionalities are >>>>> backward compatible. Users essentially need not use the >>>>> widgetChannelManager API, as they can create the WebSocket connection >>>>> within the custom widget. >>>>> >>>>> @Nisala & @Lasantha: This is due to a fix made in the latest version, >>>>> to reuse WidgetChannelManager passed from the portal app[1]. This makes >>>>> the >>>>> base widget lib dependant upon the portal app. How about a fix such that >>>>> base widget will reuse the WidgetChannelManager if passed from the portal >>>>> app or will create a new instance of WidgetChannelManger? WDYT? >>>>> >>>>> [1] >>>>> https://github.com/wso2/carbon-dashboards/blob/master/base-widget/src/Widget.jsx#L39 >>>>> >>>>> Best Regards, >>>>> *Niveathika Rajendran,* >>>>> *Software Engineer.* >>>>> *Mobile : +94 077 903 7536* >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Fri, Jul 6, 2018 at 6:46 PM Damith Wickramasinghe >>>>> wrote: >>>>> >>>
Re: [Dev] Connect to a RDMBS datastore to retrieve data for a SP dashboard widget
Hi Nisala, Thank you for the explanation, as we have not documented the depended base widget version, I thought they should be compatible with all versions of SP. How about adding a note in SP creating custom widget documentation about depended base widget version? SP 4.2.0 -> @wso2-dashboards/widget: 1.2.2 SP 4.1.0 -> @wso2-dashboards/widget: 1.1.0 SP 4.0.0 -> @wso2-dashboards/widget: 1.1.0 Best Regards, *Niveathika Rajendran,* *Software Engineer.* *Mobile : +94 077 903 7536* On Mon, Jul 9, 2018 at 12:48 AM Nisala Nanayakkara wrote: > Hi Nivethika, > > @Damith What Thisaru means is that custom widgets(with a dependency of > @wso2-dashboards/widgets:1.2.2) are not backward compatible. It only works > in SP 4.2.0 > > AFAIK older widgets which are built for SP 4.0.0/4.1.0, work with SP 4.2.0 > release. So the widgets' backward compatibility is preserved. Can you > please clarify a bit more about this? As you mentioned earlier, > wso2-dashboards/widgets only work with SP 4.2.0 release. Moreover, I think > that although there is a requirement to support older widgets with 4.2.0 > release version, there is no use-case to support widgets which are built > for SP 4.2.0 to work with older versions such as SP 4.0.0/4.1.0. > > Thanks, > Nisala > > On Fri, Jul 6, 2018 at 8:41 PM Niveathika Rajendran > wrote: > >> @Damith What Thisaru means is that custom widgets(with a dependency of >> @wso2-dashboards/widgets:1.2.2) are not backward compatible. It only works >> in SP 4.2.0 (SP-4.2.0-rc2 is the successful release candidate, we still >> waiting on quarterly release announcement to change the tags). This issue >> is only for the getWidgetChannelManager API, other functionalities are >> backward compatible. Users essentially need not use the >> widgetChannelManager API, as they can create the WebSocket connection >> within the custom widget. >> >> @Nisala & @Lasantha: This is due to a fix made in the latest version, to >> reuse WidgetChannelManager passed from the portal app[1]. This makes the >> base widget lib dependant upon the portal app. How about a fix such that >> base widget will reuse the WidgetChannelManager if passed from the portal >> app or will create a new instance of WidgetChannelManger? WDYT? >> >> [1] >> https://github.com/wso2/carbon-dashboards/blob/master/base-widget/src/Widget.jsx#L39 >> >> Best Regards, >> *Niveathika Rajendran,* >> *Software Engineer.* >> *Mobile : +94 077 903 7536* >> >> >> >> >> >> On Fri, Jul 6, 2018 at 6:46 PM Damith Wickramasinghe >> wrote: >> >>> Hi Thisaru, >>> >>> First thing is we need to use SP 4.2.0-rc2 version in order to use >>>> WidgetChannelManager. Otherwise the communication channel will fail >>>> >>> >>> Do u mean in 4.2.0 released version this does not work ? >>> >>> On Fri, Jul 6, 2018 at 12:47 PM, Thisaru Guruge >>> wrote: >>> >>>> Hi Nivethika, >>>> >>>> Thanks for the immense support you gave, Now I am able to create the >>>> widget. >>>> >>>> There were some issues I faced when creating the widget, and thought to >>>> share them, and how I resolved them. >>>> >>>> - First thing is we need to use SP 4.2.0-rc2 version in order to use >>>> WidgetChannelManager. Otherwise the communication channel will fail >>>> - Next thing is to define the datastore in the widgetConf.json file. In >>>> the existing solutions in the GitHub does not have such data. >>>> But after the discussion with Nivethika got to know that we should >>>> provide datastore name using "datasourceName": "" >>>> under the providerConfig -> config in the widgetConf.json file. >>>> >>>> Thank you again for the help given in resolving the issues I had. >>>> >>>> On Thu, Jul 5, 2018 at 4:05 PM Niveathika Rajendran < >>>> niveath...@wso2.com> wrote: >>>> >>>>> Hi Thirasu, >>>>> >>>>> The above information is still not added to the docs. >>>>> >>>>> The functionality behind this is to create a WebSocket connection to >>>>> the data-provider endpoint with a required configuration. The >>>>> documentation @Sajith pointed out has the details on various data provider >>>>> endpoints and the configurations. >>>>> >>>>> You have to do the following steps, >>>>> 1. Access data provider configurations in
Re: [Dev] Connect to a RDMBS datastore to retrieve data for a SP dashboard widget
@Damith What Thisaru means is that custom widgets(with a dependency of @wso2-dashboards/widgets:1.2.2) are not backward compatible. It only works in SP 4.2.0 (SP-4.2.0-rc2 is the successful release candidate, we still waiting on quarterly release announcement to change the tags). This issue is only for the getWidgetChannelManager API, other functionalities are backward compatible. Users essentially need not use the widgetChannelManager API, as they can create the WebSocket connection within the custom widget. @Nisala & @Lasantha: This is due to a fix made in the latest version, to reuse WidgetChannelManager passed from the portal app[1]. This makes the base widget lib dependant upon the portal app. How about a fix such that base widget will reuse the WidgetChannelManager if passed from the portal app or will create a new instance of WidgetChannelManger? WDYT? [1] https://github.com/wso2/carbon-dashboards/blob/master/base-widget/src/Widget.jsx#L39 Best Regards, *Niveathika Rajendran,* *Software Engineer.* *Mobile : +94 077 903 7536* On Fri, Jul 6, 2018 at 6:46 PM Damith Wickramasinghe wrote: > Hi Thisaru, > > First thing is we need to use SP 4.2.0-rc2 version in order to use >> WidgetChannelManager. Otherwise the communication channel will fail >> > > Do u mean in 4.2.0 released version this does not work ? > > On Fri, Jul 6, 2018 at 12:47 PM, Thisaru Guruge wrote: > >> Hi Nivethika, >> >> Thanks for the immense support you gave, Now I am able to create the >> widget. >> >> There were some issues I faced when creating the widget, and thought to >> share them, and how I resolved them. >> >> - First thing is we need to use SP 4.2.0-rc2 version in order to use >> WidgetChannelManager. Otherwise the communication channel will fail >> - Next thing is to define the datastore in the widgetConf.json file. In >> the existing solutions in the GitHub does not have such data. >> But after the discussion with Nivethika got to know that we should >> provide datastore name using "datasourceName": "" >> under the providerConfig -> config in the widgetConf.json file. >> >> Thank you again for the help given in resolving the issues I had. >> >> On Thu, Jul 5, 2018 at 4:05 PM Niveathika Rajendran >> wrote: >> >>> Hi Thirasu, >>> >>> The above information is still not added to the docs. >>> >>> The functionality behind this is to create a WebSocket connection to the >>> data-provider endpoint with a required configuration. The >>> documentation @Sajith pointed out has the details on various data provider >>> endpoints and the configurations. >>> >>> You have to do the following steps, >>> 1. Access data provider configurations in widgetConf.json file [1] >>> 2. Use the base widget API to create WebSocket connection. For this, you >>> can use the following APIs, >>> >>>- Subscribe to the endpoint: >>> - syntax : >>> super.getWidgetChannelManager().subscribeWidget(, >> back >>> method to handle data>, ) >>> - Example: >>> - >>> >>> super.getWidgetChannelManager() >>> .subscribeWidget(this.props.id, this.handleDataReceived, >>> dataProviderConfigs)); >>> >>> >>> >>>- Unsubscribe to the endpoint >>> - syntax: super.getWidgetChannelManager().unsubscribeWidget(>> ID>); >>> - Example: >>> - >>> >>> super.getWidgetChannelManager().unsubscribeWidget(this.props.id); >>> >>> >>> >>> @Nisala Nanayakkara Please add anything I have missed. >>> >>> @Rukshani Weerasinha Shall we add the above to the >>> documentation, as this will be a blocker to anyone writing custom widgets. >>> >>> [1] mail thread : [SP] [Solutions] Data provider configurations in >>> solution widgets >>> >>> Best Regards, >>> *Niveathika Rajendran,* >>> *Software Engineer.* >>> *Mobile : +94 077 903 7536* >>> >>> >>> >>> >>> >>> On Thu, Jul 5, 2018 at 3:01 PM Thisaru Guruge wrote: >>> >>>> Hi Sajith, >>>> >>>> Thank you for the reply, but I was trying to create a custom widget, >>>> not from the Wizard. >>>> >>>> On Thu, Jul 5, 2018 at 1:08 PM Sajith Perera wrote: >>>> >>>>> Hi, >>>>> >>>>> Please have a look on the sample doc: >>>>> http
Re: [Dev] Connect to a RDMBS datastore to retrieve data for a SP dashboard widget
Hi Thirasu, The above information is still not added to the docs. The functionality behind this is to create a WebSocket connection to the data-provider endpoint with a required configuration. The documentation @Sajith pointed out has the details on various data provider endpoints and the configurations. You have to do the following steps, 1. Access data provider configurations in widgetConf.json file [1] 2. Use the base widget API to create WebSocket connection. For this, you can use the following APIs, - Subscribe to the endpoint: - syntax : super.getWidgetChannelManager().subscribeWidget(, , ) - Example: - super.getWidgetChannelManager() .subscribeWidget(this.props.id, this.handleDataReceived, dataProviderConfigs)); - Unsubscribe to the endpoint - syntax: super.getWidgetChannelManager().unsubscribeWidget(); - Example: - super.getWidgetChannelManager().unsubscribeWidget(this.props.id); @Nisala Nanayakkara Please add anything I have missed. @Rukshani Weerasinha Shall we add the above to the documentation, as this will be a blocker to anyone writing custom widgets. [1] mail thread : [SP] [Solutions] Data provider configurations in solution widgets Best Regards, *Niveathika Rajendran,* *Software Engineer.* *Mobile : +94 077 903 7536* On Thu, Jul 5, 2018 at 3:01 PM Thisaru Guruge wrote: > Hi Sajith, > > Thank you for the reply, but I was trying to create a custom widget, not > from the Wizard. > > On Thu, Jul 5, 2018 at 1:08 PM Sajith Perera wrote: > >> Hi, >> >> Please have a look on the sample doc: >> https://docs.wso2.com/display/SP4xx/Generating+Widgets >> >> Regards, >> SajithD >> >> On Thu, Jul 5, 2018 at 10:56 AM, Thisaru Guruge wrote: >> >>> Hi, >>> >>> I am looking for a way to retrieve data from a RDBMS to show them in a >>> widget in SP dashboard. >>> >>> I found an example from this link >>> <https://github.com/wso2/product-sp/tree/master/modules/solutions/message-tracer/widgets/OpenTracingSearch> >>> thanks >>> to Nisala, and followed it. But widget is not showing the data. >>> >>> Is there any documentation where I can refer about this ? >>> >>> -- >>> Thanks and Best Regards, >>> Thisaru Guruge >>> Software Engineer >>> Mobile: +94 71 720 9720 >>> >>> >>> WSO2 Inc.: http://www.wso2.com >>> >>> >>> >> >> >> -- >> <http://wso2.com/signature> >> Sajith Dimal >> Software Engineer >> Email : saji...@wso2.com >> Mobile : +94783101496 >> WSO2 Inc. | http://wso2.com >> lean.enterprise.middleware >> > > > -- > Thanks and Best Regards, > Thisaru Guruge > Software Engineer > Mobile: +94 71 720 9720 > > > WSO2 Inc.: http://www.wso2.com > > > ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [Stream Processor] Issue in creating custom widgets in Stream Processor
@Damith Wickramasinghe It's not that we are missing any steps, the documentation is disjoint and unclear. @Rukshani Weerasinha Shall we revamp the documentation under "Creating custom widgets", especially under SP4.2.0 space, since the improvements[1] done for 4.2.0 release, will make the example redundant. [1] [SP] [Solutions] Data provider configurations in solution widgets Best Regards, *Niveathika Rajendran,* *Software Engineer.* *Mobile : +94 077 903 7536* On Tue, Jul 3, 2018 at 2:55 PM Damith Wickramasinghe wrote: > @Niveathika if we are missing any step in the doc , please create a doc > jira . > > Thanks, > Damith > > On Tue, Jul 3, 2018 at 9:12 AM, Niveathika Rajendran > wrote: > >> Hi Thisaru, >> >> From the logs, it seems you have updated HelloWorld.jsx file as in Step >> 4. Here, we are importing "ExtendedWiget.jsx" file created in Step 3, >> adding that to the src folder should resolve this issue. >> >> Best Regards, >> *Niveathika Rajendran,* >> *Software Engineer.* >> *Mobile : +94 077 903 7536* >> >> >> >> >> >> On Tue, Jul 3, 2018 at 2:30 PM Thisaru Guruge wrote: >> >>> Hi, >>> >>> I am trying to create a custom widget in SP dashboard and I followed the >>> example shown in the documentation on creating custom widgets >>> <https://docs.wso2.com/display/SP410/Creating+Custom+Widgets> >>> >>> I followed exactly same and getting following error when tried to build >>> the widget: >>> >>> ERROR in ./HelloWorld.jsx >>> >>> Module not found: Error: Can't resolve './ExtendedWidget' in >>> '/Users/Thisaru/Projects/IS_Analytics_Migation/carbon-dashboards/samples/widgets/HelloWorld/src' >>> >>> @ ./HelloWorld.jsx 9:23-50 >>> >>> npm ERR! code ELIFECYCLE >>> >>> npm ERR! errno 2 >>> >>> npm ERR! hello-world-widget@1.0.0 build: `webpack -p` >>> >>> npm ERR! Exit status 2 >>> >>> npm ERR! >>> >>> npm ERR! Failed at the hello-world-widget@1.0.0 build script. >>> >>> npm ERR! This is probably not a problem with npm. There is likely >>> additional logging output above. >>> >>> Can anyone help me on this ? >>> >>> -- >>> Thanks and Best Regards >>> Thisaru Guruge >>> Software Engineer >>> WSO2 Inc.: http://www.wso2.com >>> >>> Mobile: +94 71 720 9720 >>> ___ >>> Dev mailing list >>> Dev@wso2.org >>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >> >> ___ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > Senior Software Engineer > WSO2 Inc.; http://wso2.com > <http://www.google.com/url?q=http%3A%2F%2Fwso2.com=D=1=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg> > lean.enterprise.middleware > > mobile: *+94728671315* > > ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [Stream Processor] Issue in creating custom widgets in Stream Processor
Hi Thisaru, >From the logs, it seems you have updated HelloWorld.jsx file as in Step 4. Here, we are importing "ExtendedWiget.jsx" file created in Step 3, adding that to the src folder should resolve this issue. Best Regards, *Niveathika Rajendran,* *Software Engineer.* *Mobile : +94 077 903 7536* On Tue, Jul 3, 2018 at 2:30 PM Thisaru Guruge wrote: > Hi, > > I am trying to create a custom widget in SP dashboard and I followed the > example shown in the documentation on creating custom widgets > <https://docs.wso2.com/display/SP410/Creating+Custom+Widgets> > > I followed exactly same and getting following error when tried to build > the widget: > > ERROR in ./HelloWorld.jsx > > Module not found: Error: Can't resolve './ExtendedWidget' in > '/Users/Thisaru/Projects/IS_Analytics_Migation/carbon-dashboards/samples/widgets/HelloWorld/src' > > @ ./HelloWorld.jsx 9:23-50 > > npm ERR! code ELIFECYCLE > > npm ERR! errno 2 > > npm ERR! hello-world-widget@1.0.0 build: `webpack -p` > > npm ERR! Exit status 2 > > npm ERR! > > npm ERR! Failed at the hello-world-widget@1.0.0 build script. > > npm ERR! This is probably not a problem with npm. There is likely > additional logging output above. > > Can anyone help me on this ? > > -- > Thanks and Best Regards > Thisaru Guruge > Software Engineer > WSO2 Inc.: http://www.wso2.com > > Mobile: +94 71 720 9720 > ___ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [Architecture] [VOTE] Release of WSO2 Stream Processor 4.2.0 RC2
Hi, I have tested the following and found no issues, 1. HTTP Analytics Dashboard 2. Twitter Dashboard 3. Authentication & Authorization 4. Generated widget with dynamic query 5. Dashboard with pub-sub widgets [+] Stable - go ahead and release Best Regards, *Niveathika Rajendran,* *Software Engineer.* *Mobile : +94 077 903 7536* On Wed, Jun 20, 2018 at 9:57 PM Nisala Nanayakkara wrote: > Hi all, > > WSO2 Stream Processor team is pleased to announce the second release > candidate of WSO2 Stream Processor 4.2.0. > > WSO2 Stream Processor is an open source embodiment of the WSO2 Analytics > platform, of which the real-time, incremental & intelligent data processing > capabilities let digital businesses create actionable business insights and > data products. > > Please find the improvements and fixes related to this release here > <https://github.com/wso2/product-sp/issues?utf8=%E2%9C%93=is%3Aissue+closed%3A2018-03-16..2018-06-20> > . > > You can find the source and distribution here > <https://github.com/wso2/product-sp/releases/tag/v4.2.0-rc2> > > The tag to be voted upon: > https://github.com/wso2/product-sp/releases/tag/v4.2.0-rc2 > <https://github.com/wso2/analytics-apim/releases/tag/v2.5.0-rc1> > Please download, test the product and vote. > > [+] Stable - go ahead and release > [-] Broken - do not release (explain why) > > Thanks, > *- WSO2 Stream Processor Team -* > > > -- > *Nisala Niroshana Nanayakkara,* > Senior Software Engineer > Mobile | +94 717600022 > WSO2 Inc | http://wso2.com/ > ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Siddhi long value comparison
Hi Dinusha, The reason for the above issue is that the '259200' value, we get from the multiplication exceeds the maximum value of the int datatype( 2,147,483,647). As per the use case, since only the 30 in the multiplication needs to be configured, you can use the following query, *from democondition[ timediff > 30***8640l ]* *select timediff, status* *insert into finalStream;* Best Regards, Niveathika Rajendran On Tue, Mar 6, 2018 at 2:13 PM, Dinusha Dissanayake <dinus...@wso2.com> wrote: > Hi all, > > I came across with an issue regarding comparing a long value. > My intention was to check if the time difference is greater that 30 days, > and then insert the corresponding values to target stream. > > I tested this behavior with APIM Analytics 2.0.0. > > However I noticed an unusual behavior. > > I have mentioned an example stream below. > > *@Import('demoConditions:1.0.0')* > *define stream democondition (timediff long, status bool);* > > *@Export('finalStream:1.0.0')* > *define stream finalStream (timediff long, status bool);* > > > *from democondition[timediff > 30*24*60*60*1000]* > *select timediff, status* > *insert into finalStream;* > > Ideally, If I give a value greater than 30*24*60*60*1000 only, values > should be inserted into finalStream. > However I observed even if I give 1 as timeDiff, the values in timeDiff > and status get inserted into finalStream. > > When I check the validity of the execution plan through the admin console, > it gets success. > However I replaced the value 30*24*60*60*1000 with 259200 and it > produces and error. > > > > However when I append the "l" add the end (259200l), it got worked. > I tried to use the same with 30*24*60*60*1000 -> 30l*24l*60l*60l*1000l > and I couldn't get it worked since it produce an error while validating the > script. > > The reason I am using 30*24*60*60*1000 is because I want the value "30" to > be configured. It should be changed using the template manager. > > Please advice on how should I get it into working? > > Thanks, > DinushaD. > > > -- > Dinusha Dissanayake > Software Engineer > WSO2 Inc > Mobile: +94712939439 <+94%2071%20293%209439> > <https://wso2.com/signature> > > ___ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- Best Regards, *Niveathika Rajendran,* *Software Engineer.* *Mobile : +94 077 903 7536* ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [Vote] Release WSO2 Stream Processor (SP) 4.0.0-RC3
Hi all, Tested SP Monitoring dashboard, [+] Stable - Go ahead and release On Fri, Dec 22, 2017 at 9:54 PM, Sriskandarajah Suhothayan <s...@wso2.com> wrote: > Tested the editor, event simulation, dashboard creation, widget > generation. > > Suho > > > On Fri, Dec 22, 2017 at 9:24 PM, Gokul Balakrishnan <go...@wso2.com> > wrote: > >> Hi Devs, >> >> This is the third release candidate (RC) of WSO2 Stream Processor 4.0.0 >> release. >> >> New / Improvements In This Release after RC2 >> >>- Bug fixes in editor and the RDBMS store >>- UI Improvements >> >> Reported Issues >> >>- https://github.com/wso2/product-sp/issues >> >> Source and distribution packages: >> >>- https://github.com/wso2/product-sp/releases/tag/v4.0.0-RC3 >> >> Tag to be voted upon: >> >>- https://github.com/wso2/product-sp/releases/tag/v4.0.0-RC3 >> >> Please download, test, and vote. The README file under the distribution >> contains guide and instructions on how to try it out locally. >> >> [+] Stable - Go ahead and release >> [-] Broken - Do not release (explain why) >> >> This vote will be open for 72 hours or as needed. >> >> Regards, >> The WSO2 Analytics Team. >> >> -- >> Gokul Balakrishnan >> Senior Software Engineer, >> WSO2, Inc. http://wso2.com >> M +94 77 5935 789 | +44 7563 570502 <+44%207563%20570502> >> >> >> ___ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > > *S. Suhothayan* > Associate Director / Architect > *WSO2 Inc. *http://wso2.com > * <http://wso2.com/>* > lean . enterprise . middleware > > > *cell: (+94) 779 756 757 <+94%2077%20975%206757> | blog: > http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: > http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: > http://lk.linkedin.com/in/suhothayan <http://lk.linkedin.com/in/suhothayan>* > > ___ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- Best Regards, *Niveathika Rajendran,* *Software Engineer.* *Mobile : +94 077 903 7536* ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] [Architecture] WSO2 Stream Processor 4.0.0-M10 Released !
Hi All, The WSO2 Analytics team is pleased to announce the release of *WSO2 Stream Processor Version 4.0.0 Milestone 10*. WSO2 Smart Analytics let digital business creating real-time, intelligent, actionable business insights, and data products which are achieved by WSO2 Stream Processor's real-time, incremental & intelligent data processing capabilities. WSO2 Stream Processor can: - Receive events from various data sources - Process & correlate them in real-time with the sate of the art high-performance real-time Siddhi Complex Event Processing Engine that works with easy to learn the SQL-Like query language. - Process analysis that spans for longer time duration with its incremental processing capability by achieving high performance with low infrastructure cost. - Uses Machine Learning and other models to drive intelligent insights from the data - Notifications interesting event occurrences as alerts via multiple types of transport & let users visualize the results via customizable dashboards. - WSO2 SP is released under Apache Software License Version 2.0, one of the most business-friendly licenses available today. You can find the product at https://github.com/wso2/ <https://github.com/wso2/product-sp/releases/download/v4.0.0-M9/wso2sp-4.0.0-M9.zip> Documentation at https://docs.wso2.com/display/ <https://docs.wso2.com/display/SP400/> Source code at https://github.com/wso2/ <https://github.com/wso2/product-sp/releases/tag/v4.0.0-M9> *WSO2 SP 4.0.0-M10 includes the following* *New Features* - Incremental Processing in Siddhi - Minimum High Availability Deployment Support for SP - Carbon Data Sources Integration in SP *New Extensions* - Siddhi-io-email : Sink feature - Siddhi-io-rabbitmq - Siddhi-io-mqtt - Siddhi-execution-streamingml *Reporting Issues* Issues can be reported using the github issue tracker available at https://github.com/wso2/product-sp <https://github.com/wso2/product-sp/issues> * Contact us* WSO2 Stream Processor developers can be contacted via the mailing lists: Developer List : dev@wso2.org | Subscribe <dev-requ...@wso2.org?subject=subscribe> | M <http://wso2.org/mailarchive/dev/> Alternatively, questions can also be raised in the Stackoverflow: Forum http://stackoverflow.com/questwso2/ <http://stackoverflow.com/questions/tagged/wso2/> *Support * We are committed to ensuring that your enterprise middleware deployment is completely supported from evaluation to production. Our unique approach ensures that all support leverages our open development methodology and is provided by the very same engineers who build the technology. For more details and to take advantage of this unique opportunity please visit http://wso2.com/support/. <http://wso2.com/support/> For more information on WSO2 Smart Analytics and Smart Analytics Solutions, visit the WSO2 Smart Analytics Page <http://wso2.com/smart-analytics>. *~ The WSO2 Analytics Team ~* -- Best Regards, *Niveathika Rajendran,* *Software Engineer.* *Mobile : +94 077 903 7536 <+94%2077%20903%207536>* ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev