[Dev] [Architecture] [Announcement] WSO2 Enterprise Integrator 7.1.0 Released !
gration-Studio/ Streaming Integrator 1.1.0Key Features and Improvements Error Capturing and Event Replay - Capability to capture erroneous events and store them - Error store browser for detailed information on each error - Event replay support to replay the erroneous events after corrective action Data Flow Monitoring - Grafana and Prometheus based monitoring dashboard - Statistics on CDC, File processing and DB interactions - Stats and figures at different granularities with drill down capability New Connectors - Azure Data Lake - CDC for MongoDB - File system event listener - Improvements to S3 connector to support file operations Extension Installation - Command line tool assist installing extensions/connectors with ease - Automatically download all required dependencies from a single command - Notify users of required extensions by parsing siddhi apps DownloadWSO2 Streaming Integrator can be downloaded via the website <https://wso2.com/integration/streaming-integrator/>.DocumentationOnline documentation is available at https://ei.docs.wso2.com/en/7.1.0/streaming-integrator/quick-start-guide/quick-start-guide/ DockerDocker image can be downloaded from docker hub under the name wso2/streaming-integrator <https://hub.docker.com/r/wso2/streaming-integrator>Streaming Integrator Tooling 1.1.0Key Features and Improvements Streaming ETL Task wizard - Allows to build a simple ETL task with no code in matter of minutes - Configure the data source and sink - Process input data with using filters and aggregations - Graphical data mapping capability Improvements To Design View - Extensive support for CDC and File connectors with Cron expression generation UI Error Store Browser - Browse through the captured errors - Detailed view on each error with the log extracts, input payload - Allows user to alter the payload if required and replay the events DownloadWSO2 Streaming Integrator tooling can be downloaded via the website <https://wso2.com/integration/streaming-integrator/>.DocumentationOnline documentation is available at https://ei.docs.wso2.com/en/7.1.0/streaming-integrator/develop/developing-streaming-integration-solutions/ ~ The Enterprise Integrator Team ~ -- Sasikala Kottegoda *Associate Technical Lead* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928 [image: https://wso2.com/signature] <https://wso2.com/signature> ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [Architecture] [VOTE] Release WSO2 Micro Integrator 1.2.0-RC1
hanced experience for handling security (Ciphertool, >>>>>>>Securevault, Kubernetes Secret) >>>>>>>- >>>>>>> >>>>>>>Support for hot deployment >>>>>>>- >>>>>>> >>>>>>>Schema-based XML to JSON transformations with 'JSON Transform >>>>>>>mediator' >>>>>>>- >>>>>>> >>>>>>>Swagger generation for data services >>>>>>>- >>>>>>> >>>>>>>Transaction counter for services >>>>>>>- >>>>>>> >>>>>>>Support for SMB version 3.0 >>>>>>>- >>>>>>> >>>>>>>Environment / System variable support for K8s >>>>>>>- >>>>>>> >>>>>>>CLI with enhanced functionalities >>>>>>> >>>>>>> >>>>>>> List of tasks/bug fixes and improvements shipped with this release >>>>>>> can be found in Tasks/bug fixes and improvements >>>>>>> <https://github.com/wso2/micro-integrator/issues?q=is%3Aissue+closed%3A2019-10-08..2020-07-28> >>>>>>> >>>>>>> Downloads >>>>>>> >>>>>>> Source and binary distributions: >>>>>>> https://github.com/wso2/micro-integrator/releases/tag/v1.2.0-rc1 >>>>>>> >>>>>>> Docker: Please check wso2mi <https://hub.docker.com/r/wso2/wso2mi> >>>>>>> and wso2mi-dashboard >>>>>>> <https://hub.docker.com/r/wso2/wso2mi-dashboard> repositories in >>>>>>> Docker Hub. >>>>>>> >>>>>>> >>>>>>> >>>>>>> Known Issues: https://github.com/wso2/micro-integrator/issues >>>>>>> >>>>>>> The tag to be voted upon: >>>>>>> https://github.com/wso2/micro-integrator/releases/tag/v1.2.0-rc1 >>>>>>> >>>>>>> Please test and vote as follows: >>>>>>> >>>>>>> [+] Stable - go ahead and release >>>>>>> >>>>>>> [-] Broken - do not release (explain why) >>>>>>> >>>>>>> You may find the official documentation in >>>>>>> https://ei.docs.wso2.com/en/7.1.0/micro-integrator/overview/introduction/ >>>>>>> >>>>>>> ~ The WSO2 Enterprise Integration Team ~ >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *S.Srivathsan* >>>>>>> Software Engineer | WSO2 >>>>>>> (M) +94 774606335 | (E) srivath...@wso2.com >>>>>>> ___ >>>>>>> Dev mailing list >>>>>>> Dev@wso2.org >>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Prabushi Samarakoon* >>>>>> Mobile: +94715434580 >>>>>> Email: prabus...@wso2.com >>>>>> >>>>>> >>>>>> ___ >>>>>> Dev mailing list >>>>>> Dev@wso2.org >>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Dilan Tharaka* | Senior Software Engineer | WSO2 Inc. >>>>> (m) +94 (71) 3056011 | (w) +94 (11) 2145345 | (e) dila...@wso2.com >>>>> <https://wso2.com/signature> >>>>> ___ >>>>> Dev mailing list >>>>> Dev@wso2.org >>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>> >>>> >>>> >>>> -- >>>> >>>> *Rosen Silva* >>>> Senior Software Engineer - WSO2 >>>> >>>> Email: ros...@wso2.com >>>> Mobile: +94770677418 >>>> Web: https://wso2.com/ >>>> ___ >>>> Dev mailing list >>>> Dev@wso2.org >>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>> >>> >>> >>> -- >>> >>> *Danushka Madhuranga* >>> >>> Software Engineer | WSO2 Inc >>> >>> Mobile : +94 774665325 >>> >>> >>> <http://wso2.com/signature> >>> ___ >>> Architecture mailing list >>> architect...@wso2.org >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >> >> >> -- >> *S.Srivathsan* >> Software Engineer | WSO2 >> (M) +94 774606335 | (E) srivath...@wso2.com >> ___ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> > ___ > Architecture mailing list > architect...@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > -- Sasikala Kottegoda *Associate Technical Lead* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928 [image: https://wso2.com/signature] <https://wso2.com/signature> ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [Architecture] [VOTE] Release WSO2 Micro Integrator 1.2.0-RC1
Hi all, I have conducted basic smoke tests against Adopt Open jdk 11 in windows and also have tested plugging in an external WSO2 user store based on the database/LDAP, and found no blockers. [+] Stable go ahead and release. Thank you, Sasikala On Tue, Jul 28, 2020 at 1:04 PM Srivathsan Shanmuganathan < srivath...@wso2.com> wrote: > Hi all, > > We are pleased to announce the first release candidate of WSO2 Micro > Integrator 1.2.0. > > What's new in Micro Integrator 1.2.0 > > >- > >Task coordination support for distributed deployments >- > >Support to view and configure logs without accessing the file system >- > >User management with external user stores >- > >Readiness probe for health checking >- > >Enhanced experience for handling security (Ciphertool, Securevault, >Kubernetes Secret) >- > >Support for hot deployment >- > >Schema-based XML to JSON transformations with 'JSON Transform mediator' >- > >Swagger generation for data services >- > >Transaction counter for services >- > >Support for SMB version 3.0 >- > >Environment / System variable support for K8s >- > >CLI with enhanced functionalities > > > List of tasks/bug fixes and improvements shipped with this release can be > found in Tasks/bug fixes and improvements > <https://github.com/wso2/micro-integrator/issues?q=is%3Aissue+closed%3A2019-10-08..2020-07-28> > > Downloads > > Source and binary distributions: > https://github.com/wso2/micro-integrator/releases/tag/v1.2.0-rc1 > > Docker: Please check wso2mi <https://hub.docker.com/r/wso2/wso2mi> and > wso2mi-dashboard <https://hub.docker.com/r/wso2/wso2mi-dashboard> > repositories in Docker Hub. > > > > Known Issues: https://github.com/wso2/micro-integrator/issues > > The tag to be voted upon: > https://github.com/wso2/micro-integrator/releases/tag/v1.2.0-rc1 > > Please test and vote as follows: > > [+] Stable - go ahead and release > > [-] Broken - do not release (explain why) > > You may find the official documentation in > https://ei.docs.wso2.com/en/7.1.0/micro-integrator/overview/introduction/ > > ~ The WSO2 Enterprise Integration Team ~ > > > -- > *S.Srivathsan* > Software Engineer | WSO2 > (M) +94 774606335 | (E) srivath...@wso2.com > ___ > Architecture mailing list > architect...@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > -- Sasikala Kottegoda *Associate Technical Lead* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928 [image: https://wso2.com/signature] <https://wso2.com/signature> ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] [Architecture] [Announce] WSO2 Micro Integrator 1.2.0-M4 Released!
The WSO2 Enterprise Integrator Team team is pleased to announce the release of WSO2 Micro Integrator 1.2.0-m4. Download This release lets you download the following artifacts: - Micro Integrator Runtime <https://github.com/wso2/micro-integrator/releases/download/v1.2.0-m4/wso2mi-1.2.0-m4.zip> - Monitoring Dashboard <https://github.com/wso2/micro-integrator/releases/download/v1.2.0-m4/wso2mi-monitoring-dashboard-1.2.0-m4.zip> - CLI Distributions - Linux 64bit <https://github.com/wso2/micro-integrator/releases/download/v1.2.0-m4/wso2mi-cli-1.2.0-m4-linux-x64.tar.gz> - Linux 32bit <https://github.com/wso2/micro-integrator/releases/download/v1.2.0-m4/wso2mi-cli-1.2.0-m4-linux-i586.tar.gz> - Windows 64bit <https://github.com/wso2/micro-integrator/releases/download/v1.2.0-m4/wso2mi-cli-1.2.0-m4-windows-x64.zip> - Windows 32bit <https://github.com/wso2/micro-integrator/releases/download/v1.2.0-m4/wso2mi-cli-1.2.0-m4-windows-i586.zip> - Mac OS <https://github.com/wso2/micro-integrator/releases/download/v1.2.0-m4/wso2mi-cli-1.2.0-m4-macosx-x64.tar.gz> Docker Simply get started with Micro Integrator in Docker by browsing wso2mi <https://hub.docker.com/r/wso2/wso2mi> and wso2mi-dashboard <https://hub.docker.com/r/wso2/wso2mi-dashboard> repositories in Docker Hub. What's new in WSO2 Micro Integrator 1.2.0-M4 A list of tasks/bug fixes and improvements shipped with this release can be found below: - Tasks/bug fixes and improvements <https://github.com/wso2/micro-integrator/milestone/13?closed=1> DocumentationOnline documentation is available at https://ei.docs.wso2.com/en/next/micro-integrator/overview/introduction/Contribute to WSO2 Micro IntegratorMailing ListsJoin our mailing lists and correspond with the developers directly. We also encourage you to take part in discussions related to the product in the architecture mailing list. If you have any questions regarding the product you can use our StackOverflow forum and Slack Channel.Developer List: dev@wso2.orgArchitecture List: architecture@wso2.orgUser Forum: StackOverflow <https://stackoverflow.com/questions/tagged/wso2mi>Slack Channel: https://ei-slack.wso2.com/Reporting Issues We encourage you to report issues, improvements and feature requests regarding WSO2 Micro Integrator through our public WSO2 Micro Integrator GIT Issues <https://github.com/wso2/micro-integrator/issues> Important: Please be advised that security issues must be reported to secur...@wso2.com, not as GitHub issues, in order to reach the proper audience. We strongly advise following the WSO2 Security Vulnerability Reporting Guidelines <https://docs.wso2.com/display/Security/WSO2+Security+Vulnerability+Reporting+Guidelines> when reporting the security issues. ~ The WSO2 Enterprise Integration Team ~ -- Sasikala Kottegoda *Associate Technical Lead* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928 [image: https://wso2.com/signature] <https://wso2.com/signature> ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] WSO2 Committers += Srivathsan Shanmuganathan
Hi All, It's my pleasure to announce Srivathsan Shanmuganathan as a WSO2 Committer. He has been a valuable contributor to WSO2. Congratulations Srivathsan and keep up the good work..! Thank you, Sasikala -- Sasikala Kottegoda *Associate Technical Lead* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928 [image: https://wso2.com/signature] <https://wso2.com/signature> ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] WSO2 Committers += Arunan Sugunakumar
Hi All, It's my pleasure to announce Arunan Sugunakumar as a WSO2 Committer. He has been a valuable contributor to WSO2. Congratulations Arunan and keep up the good work..! Thank you, Sasikala -- Sasikala Kottegoda *Associate Technical Lead* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928 [image: https://wso2.com/signature] <https://wso2.com/signature> ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Requesting suggestions for micro integrator secure vault script name
+1 to *securevault* since it's already inside micro-integrator. Thank you, Sasikala On Tue, Aug 7, 2018 at 6:53 PM Godwin Shrimal wrote: > +1 to *securevault*. Since word micro is redundant here and its coming > with the profile name itself. > > Thanks > Godwin > > On Tue, Aug 7, 2018 at 2:09 PM Lahiru Madushanka > wrote: > >> +1 to *micro-*securevault*.sh* >> >> On Tue, Aug 7, 2018 at 5:19 PM Hasitha Hiranya wrote: >> >>> Hi Arshardh, >>> >>> I am +1 for the simple name "*securevault*" >>> >>> Thanks >>> >>> On Tue, Aug 7, 2018 at 10:36 AM Arshardh Ifthikar >>> wrote: >>> >>>> Hi all, >>>> The Micro Integrator profile of the Enterprise Integrator, did not >>>> support secure vault [1]. One reason for this was, the absence of an user >>>> interface to add new entries to the secure vault. >>>> >>>> Since the micro integrator does not consist of a management console, >>>> users will be given the opportunity to add entries to the micro integrator >>>> secure vault via executing a script. Once the user enters the alias and the >>>> password, the entry will be stored in the micro integrator registry as >>>> properties, and will be used subsequently for the vault-lookup function. >>>> >>>> [image: image.png] >>>> >>>> The above mentioned script located within the micro integrator bin >>>> folder, needs an appropriate name. This particular script is currently >>>> named, "*securevault*". Please mention any suggestions for a different >>>> name. >>>> >>>> [1] >>>> https://docs.wso2.com/display/EI630/Working+with+Passwords+in+the+ESB+profile >>>> >>>> Thanks, >>>> Arshardh >>>> >>>> -- >>>> *Arshardh Ifthikar* >>>> Software Engineer | WSO2 Inc. >>>> >>>> Email: arsha...@wso2.com >>>> Mobile: +94719806525 >>>> Web: http://wso2.com >>>> >>>> <http://wso2.com/signature> >>>> ___ >>>> Dev mailing list >>>> Dev@wso2.org >>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>> >>> >>> >>> -- >>> *Hasitha Abeykoon* >>> Associate Technical Lead; WSO2, Inc.; http://wso2.com >>> *cell:* *+94 719363063* >>> *blog: **abeykoon.blogspot.com* <http://abeykoon.blogspot.com> >>> >>> ___ >>> Dev mailing list >>> Dev@wso2.org >>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >> ___ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> > > > -- > *Godwin Amila Shrimal* > Associate Technical Lead > WSO2 Inc.; http://wso2.com > lean.enterprise.middleware > > mobile: *+94772264165* > linkedin: *https://www.linkedin.com/in/godwin-amila-2ba26844/ > <https://www.linkedin.com/in/godwin-amila-2ba26844/>* > twitter: https://twitter.com/godwinamila > <http://wso2.com/signature> > ___ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > -- Sasikala Kottegoda *Senior Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928 [image: https://wso2.com/signature] <https://wso2.com/signature> ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] WSO2 Enterprise Integrator 6.1.1-update6 Released!
Hi all, The WSO2 Integration team is pleased to announce the release of Enterprise Integrator 6.1.1-update6. It is available for download from here <https://github.com/wso2/product-ei/releases/download/v6.1.1-update6/wso2ei-6.1.1-update6.zip> . *Tasks/Bug Fixes and Improvements* https://wso2.org/jira/issues?filter=13993 https://github.com/wso2/product-ei/issues?q=is%3Aissue+milestone%3A6.1.1-update6+is%3Aclosed *List of Open Issues* https://wso2.org/jira/issues/?filter=13994 https://github.com/wso2/product-ei/issues?q=is%3Aopen+is%3Aissue *Mailing Lists* Join our mailing lists and correspond with the developers directly. Developer List : dev@wso2.org | Subscribe | Mail Archive WSO2 Architecture List: architect...@wso2.org User Forum : StackOverflow *How To Contribute* Your feedback is most welcome! *Reporting Issues* We encourage you to report issues, improvements and feature requests regarding WSO2 Integrator through WSO2 EI GIT Issues. ~ The WSO2 Integration Team ~ -- Sasikala Kottegoda *Senior Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928 [image: https://wso2.com/signature] <https://wso2.com/signature> ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [Vote] Release WSO2 Enterprise Integrator 6.1.1 RC1
Hi all, I have tested distributed transactions for the broker profile of WSO2 Enterprise Integrator 6.1.1 for the following scenarios: - dtx commit - Publish message to queue. Transactionally receive and publish to another queue - dtx rollback - Publish message to the same queue. Rollback the message continuously and and verify that it ends up in DLC. - dtx recover - Recover before and after preparing - Session joining [+] Stable - go ahead and release Thank you, Sasikala On Fri, Apr 28, 2017 at 5:50 PM, Milinda Perera <milin...@wso2.com> wrote: > Hi Devs, > > This is the first release candidate of WSO2 Enterprise Integrator 6.1.1. > > The vote will be open for 24 hours or as needed. Please download, test the > product and vote. > > Source & binary distribution files: > > *Product-EI:* > > https://svn.wso2.org/repos/wso2/scratch/EI/Votes/6.1.1- > RC1/wso2ei-6.1.1-rc1.zip > > *Tooling pack to be used for EI runtime testing:* > > https://svn.wso2.org/repos/wso2/people/erandac/ei_610_tooling/ > > > > *The tag to be voted upon:*https://github.com/wso2/ > product-ei/releases/tag/v6.1.1-rc1 > > [-] Broken - do not release (explain why) > [+] Stable - go ahead and release > > Thanks, > Integration Team > > -- > Milinda Perera > Senior Software Engineer; > WSO2 Inc. http://wso2.com , > Mobile: (+94) 714 115 032 <+94%2071%20411%205032> > > > ___ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- Sasikala Kottegoda *Senior Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928 [image: https://wso2.com/signature] <https://wso2.com/signature> ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] WSO2 Message Broker 3.2.0 RC2 Released !!
Hi all, Tested the following scenarios in distributed transactions. 1. publish message to queue. Transactionally receive and publish to another queue. 2. Publish errorneous message to the same queue. Keep recovering messages until the malformed message gets moved to DLC. [+] Stable - go ahead and release Thank you, Sasikala On Thu, Apr 20, 2017 at 6:47 PM, Pamod Sylvester <pa...@wso2.com> wrote: > Hi Devs, > > This is the 2nd release candidate of WSO2 Message Broker 3.2.0. > > *This release fixes the following issues:* > *https://wso2.org/jira/browse/MB-1910?filter=13675 > <https://wso2.org/jira/browse/MB-1910?filter=13675>* > > *Known Issues:* > *https://wso2.org/jira/browse/MB-1757?filter=13676 > <https://wso2.org/jira/browse/MB-1757?filter=13676>* > > <https://wso2.org/jira/issues/?filter=12422> > Please download MB 3.2.0 RC2 and test the functionality and vote. The > vote will be open for 72 hours or as needed. > > *Source & binary distribution files:* > https://github.com/wso2/product-mb/releases/tag/v3.2.0-RC2 > > *The tag to be voted upon:* > https://github.com/wso2/product-mb/tree/v3.2.0-RC2 > > > [+] Stable - go ahead and release > [-] Broken - do not release (explain why) > > Thanks and Regards, > > -- > *Pamod Sylvester * > > *WSO2 Inc.; http://wso2.com <http://wso2.com>* > cell: +94 77 7779495 <077%20777%209495> > > _______ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- Sasikala Kottegoda *Senior Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928 [image: https://wso2.com/signature] <https://wso2.com/signature> ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [Architecture] WSO2 Message Broker 3.2.0 RC1 Released !!
Following test cases fail intermittently: - org.wso2.mb.integration.tests.amqp.functional.AMQPSessionRecoverTestCase.performSessionRecoverWithAckTestCase <https://wso2.org/jenkins/job/products/job/product-mb/org.wso2.mb$org.wso2.carbon.mb.amqp.test/1174/testReport/junit/org.wso2.mb.integration.tests.amqp.functional/AMQPSessionRecoverTestCase/performSessionRecoverWithAckTestCase/> - org.wso2.mb.integration.tests.amqp.functional.AMQPSessionRecoverTestCase.performSessionRecoverWithoutAckTestCase <https://wso2.org/jenkins/job/products/job/product-mb/org.wso2.mb$org.wso2.carbon.mb.amqp.test/1174/testReport/junit/org.wso2.mb.integration.tests.amqp.functional/AMQPSessionRecoverTestCase/performSessionRecoverWithoutAckTestCase/> - org.wso2.mb.integration.tests.amqp.functional.dtx.DtxStartPositiveTestCase.xaStartJoinQueuePublishTestCase <https://wso2.org/jenkins/job/products/job/product-mb/org.wso2.mb$org.wso2.carbon.mb.amqp.test/1178/testReport/junit/org.wso2.mb.integration.tests.amqp.functional.dtx/DtxStartPositiveTestCase/xaStartJoinQueuePublishTestCase/> Hence, we will need to release another RC. Thank you, Sasikala On Sat, Apr 8, 2017 at 4:25 AM, Hasitha Hiranya <hasit...@wso2.com> wrote: > Hi Devs, > > This is the first release candidate of WSO2 Message Broker 3.2.0. > > This release fixes the following issues: > https://wso2.org/jira/issues/?filter=13656 > > Known Issues > https://wso2.org/jira/issues/?filter=13655 > > > <https://wso2.org/jira/issues/?filter=12422> > Please download MB 3.2.0 RC1 and test the functionality and vote. The > vote will be open for 72 hours or as needed. > > Source & binary distribution files: > https://github.com/wso2/product-mb/releases/tag/v3.2.0-RC1 > > > The tag to be voted upon: > https://github.com/wso2/product-mb/tree/release-3.2.0-RC1 > <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fwso2%2Fproduct-mb%2Ftree%2Frelease-3.2.0-RC1=D=1=AFQjCNGE9aUrfTCBg3Yw9afBVLipa6fC6A> > > > [+] Stable - go ahead and release > [-] Broken - do not release (explain why) > > Thanks and Regards > > -- > *Hasitha Abeykoon* > Senior Software Engineer; WSO2, Inc.; http://wso2.com > *cell:* *+94 719363063* > *blog: **abeykoon.blogspot.com* <http://abeykoon.blogspot.com> > > > _______ > Architecture mailing list > architect...@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928 [image: https://wso2.com/signature] <https://wso2.com/signature> ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] For reference - data changes between MB 3.1.0 and MB 3.2.0 (internal feature)
Hi Hasitha, Following are the changes in MB 3.2.0 compared to 3.1.0 in a summary as I can see. 1. Changes in subscriptions data in MB_DURABLE_SUBSCRIPTION 2. Simplified queue data in MB_QUEUE 3. Simplified binding data in MB_BINDINGS 4. All queue names being lower case We have already written a tool to cater the first 3 requirements. What is remaining is to only change the queue names to lower case which was intorduced recently. Regarding the concerns on the changes in subscription data, we actually have to do almost nothing since there won't be any active subscriptions when the system is migrated. We only need to remove all susbription entries from the table so that new subscriptions to MB 3.2.0 will be of the new format. Ragarding your suggestion on the approach for migration, we cannot wait until all the subscriptions are done consuming since there could be inactive subscribers, slow subscribers, a large number of pending messages, etc. In that case, what we can do is to provide a migration client which will consume from MB 3.1.0 and publish to MB 3.2.0 as in what we have suggested in [1]. If the system could have a down time, both approaches would be okay. Where as, if we need to migrate with zero down time, we'll need to go with the first approach which is to write a mgration tool. [1] https://docs.wso2.com/display/MB300/Upgrading+from+a+Previous+Release Thank you, Sasikala On Sun, Apr 2, 2017 at 8:41 PM, Ramith Jayasinghe <ram...@wso2.com> wrote: > whats the possibility of writing a tool which can migrate the data? > > > On Sun, Apr 2, 2017 at 7:53 PM, Hasitha Hiranya <hasit...@wso2.com> wrote: > >> Hi, >> >> Please see attached. >> >> 1. Queue names should be fixed in about 5 tables (make all simple) >> 2. Queue data string is updated in the model >> 3. Subscriber data string is updated in the model >> 4. Subscriber has a UUID as subscriber ID now. >> 5. Subscriber connection is base64 encoded. Code will decode and work. In >> 3.1.0 there was nothing like that. >> 6. Inactive durable topic subscribers are derived from the queues now. No >> entries in subscriber table. >> >> >> Since there is a lot of changes in schema and code, IMO migrating from >> 3.1.0 to 3.2.0 is error prone. Mainly this is because the model of working >> is different now. >> >> Can we migrate a production MB customer to EI using below approach? >> >> 1. Create an EI cluster. MB cluster is in production. >> 2. Switch publishers to EI cluster. Let subscribers be done with MB >> cluster >> 3. Switch subscribers also to EI cluster. >> >> For durable topic/queue subscribers above should work. >> >> WDYT? >> >> >> -- >> *Hasitha Abeykoon* >> Senior Software Engineer; WSO2, Inc.; http://wso2.com >> *cell:* *+94 719363063* >> *blog: **abeykoon.blogspot.com* <http://abeykoon.blogspot.com> >> >> > > > -- > Ramith Jayasinghe > Technical Lead > WSO2 Inc., http://wso2.com > lean.enterprise.middleware > > E: ram...@wso2.com > P: +94 777542851 <+94%2077%20754%202851> > -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928 [image: https://wso2.com/signature] <https://wso2.com/signature> ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Permission tree is not updated accroding to the DB
Copying Ramith and Dev On Wed, Oct 12, 2016 at 4:55 PM, Sasikala Kottegoda <sasik...@wso2.com> wrote: > Hi all, > > We get an issue in MB where permission is denied for a particular > subscription where permission is granted for another subscription which > requests permission for the same resource and action. This was reproducible > when subscriptions were added concurrently. > > The underlying cause for the issue was that the permission tree was > incorrectly updated. > > I have created a public jira at [1] for the issue with all the findings. > It would be great if we can get this fixed since it is an L1 for the MB > 3.2.0 release. > > [1] https://wso2.org/jira/browse/IDENTITY-5228 > > Thank you, > Sasikala > -- > Sasikala Kottegoda > *Software Engineer* > WSO2 Inc., http://wso2.com/ > lean. enterprise. middleware > Mobile: +94 774835928 > > [image: https://wso2.com/signature] <https://wso2.com/signature> > -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928 [image: https://wso2.com/signature] <https://wso2.com/signature> ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [MB] Message Expiration Task and Messages shown in UI
I think we can filter out messages when reading from the database itself. Filtering from the UI will be an overhead. WDYT? On Tue, Oct 4, 2016 at 12:01 PM, Asanka Abeyweera <asank...@wso2.com> wrote: > Hi Sirojan, > > Yes, In user's perspective it is better not to show the expired messages. > > How are we going to filter out the expired messages from UI? Since we have > the expiry time information in the database, I think we can use that > information to filter. > > On Tue, Oct 4, 2016 at 12:00 PM, Sirojan Tharmakulasingam < > siro...@wso2.com> wrote: > >> Hi All, >> >> Currently, we have two tasks to deal with expired messages. >> >> 1. Pre-delivery expiry message deletion task --> Will catch the expired >> messages at the Message flusher >> 2. Periodic message deletion task --> each 15 minutes it checks expired >> messages and deleted them for the messages which are not in our safety slot >> count region (we have this buffer assuming that those will be assigned to >> slot delivery worker in the near future) >> >> The problem: >> >> Even though the messages are expired, they are still residing in the >> database for some time. >> >> 1. Messages that are not in the safe-zone - 15 minutes (worst case) - >> until the periodic task runs >> 2. Messages that are in the safe-zone - until a subscription comes and >> messages tries to deliver and get caught at the flusher/ >> >> During the above time, those messages are in the DB even though they are >> expired. In the UI we are showing them as valid messages and valid message >> count during the aforementioned time. >> >> Solution: >> >> 1. We are not showing the messages in the UI even though they are in the >> DB (waiting for delete by the expiry tasks) >> >> WDYT? Suggestions, please? >> >> Regards, >> -- >> Sirojan Tharmakulasingam >> Software Engineer, WSO2 Inc. http://wso2.com >> Email: siro...@wso2.com >> Mobile: +94779797913 >> >> ___ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > Asanka Abeyweera > Senior Software Engineer > WSO2 Inc. > > Phone: +94 712228648 > Blog: a5anka.github.io > > <https://wso2.com/signature> > > ___ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928 [image: https://wso2.com/signature] <https://wso2.com/signature> ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] [Architecture] WSO2 Message Broker 3.2.0-ALPHA Released !
Hi All, The WSO2 Message Broker team is pleased to announce the alpha release of WSO2 Message Broker (MB) 3.2.0. Source & binary distribution files of WSO2 Message Broker can be found in: https://github.com/wso2/product-mb/releases/tag/v3.2.0-ALPHA All the known issues identified are listed in JIRA[1]. [1] https://wso2.org/jira/browse/MB Regards, ~MB Team~ -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928 [image: https://wso2.com/signature] <https://wso2.com/signature> ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [MB]Schedule time interval for deleting expired messages
Hi Sirojan, I think it's not a good idea to blindly put 10secs. Could we do some perf tests and decide a suitable range? On Mon, Sep 5, 2016 at 11:28 AM, Sajini De Silva <saj...@wso2.com> wrote: > Its hard to say what should be the exact correct time interval. IIt > depends on the messaging rate actually. I think one DB call per 10 seconds > seems oky. Are there any issues with this value? > > > On Mon, Sep 5, 2016 at 11:16 AM, Sirojan Tharmakulasingam < > siro...@wso2.com> wrote: > >> Hi all, >> >> At the MB message delivery path, we capture the expired messages and >> batch it for a batch delete from the database rather than issue a delete DB >> call for each expired messages. >> >> what is the suitable time interval for batching the expired messages for >> the DB batch delete? >> >> The current suggestion is 10sec. Your suggestions please... >> >> Regards, >> -- >> Sirojan Tharmakulasingam >> Software Engineer, WSO2 Inc. http://wso2.com >> Email: siro...@wso2.com >> Mobile: +94779797913 >> > > > > -- > Sajini De SIlva > Senior Software Engineer; WSO2 Inc.; http://wso2.com , > Email: saj...@wso2.com > Blog: http://sajinid.blogspot.com/ > Git hub profile: https://github.com/sajinidesilva > > Phone: +94 712797729 > > -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928 [image: https://wso2.com/signature] <https://wso2.com/signature> ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [VOTE] Release WSO2 Message Broker 3.1.0 RC 2
Tested the usual test scenarios in a clustered deployment by using the test suite. [+] Stable - go ahead and release Thank you, Sasikala On Fri, Feb 5, 2016 at 12:21 PM, Akalanka Pagoda Arachchi < darsha...@wso2.com> wrote: > Test UI scenarios by running UI Tests. > > [+] Stable - go ahead and release > > On Fri, Feb 5, 2016 at 12:00 PM, Hemika Kodikara <hem...@wso2.com> wrote: > >> Tested Failover for queues and durable topics. >> >> [+] Stable - go ahead and release >> >> Thanks, >> Hemika >> >> Hemika Kodikara >> Software Engineer >> WSO2 Inc. >> lean . enterprise . middleware >> http://wso2.com >> >> Mobile : +9477762 >> >> On Fri, Feb 5, 2016 at 11:55 AM, Pamod Sylvester <pa...@wso2.com> wrote: >> >>> Tested MQTT, by running the interop test suite. >>> >>> [+] Stable - go ahead and release >>> >>> Thanks, >>> Pamod >>> >>> On Fri, Feb 5, 2016 at 11:53 AM, Ramith Jayasinghe <ram...@wso2.com> >>> wrote: >>> >>>> I tested samples (with compression flag on) >>>> >>>> [+] Stable - go ahead and release >>>> >>>> On Thu, Feb 4, 2016 at 3:25 AM, Akalanka Pagoda Arachchi >>>> <darsha...@wso2.com> wrote: >>>> > Hi Devs, >>>> > >>>> > This is the second release candidate of WSO2 Message Broker 3.1.0. >>>> > >>>> > This release fixes the following issues: >>>> > https://wso2.org/jira/issues/?filter=12637 >>>> > >>>> > Please download MB 3.1.0 RC2 and test the functionality and vote. >>>> Vote will >>>> > be open for 72 hours or as needed. >>>> > >>>> > Source & binary distribution files: >>>> > https://github.com/wso2/product-mb/releases/tag/v3.1.0-RC2 >>>> > >>>> > Maven staging repo: >>>> > http://maven.wso2.org/nexus/content/repositories/orgwso2mb-323/ >>>> > >>>> > The tag to be voted upon: >>>> > https://github.com/wso2/product-mb/tree/v3.1.0-RC2 >>>> > >>>> > >>>> > [+] Stable - go ahead and release >>>> > [-] Broken - do not release (explain why) >>>> > >>>> > Thanks and Regards, >>>> > >>>> > ~ WSO2 MB Team ~ >>>> > >>>> > -- >>>> > Darshana Akalanka Pagoda Arachchi, >>>> > Software Engineer >>>> > 078-4721791 >>>> > >>>> > ___ >>>> > Dev mailing list >>>> > Dev@wso2.org >>>> > http://wso2.org/cgi-bin/mailman/listinfo/dev >>>> > >>>> >>>> >>>> >>>> -- >>>> Ramith Jayasinghe >>>> Technical Lead >>>> WSO2 Inc., http://wso2.com >>>> lean.enterprise.middleware >>>> >>>> E: ram...@wso2.com >>>> P: +94 777542851 >>>> ___ >>>> Dev mailing list >>>> Dev@wso2.org >>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>> >>> >>> >>> >>> -- >>> *Pamod Sylvester * >>> >>> *WSO2 Inc.; http://wso2.com <http://wso2.com>* >>> cell: +94 77 7779495 >>> >>> ___ >>> Dev mailing list >>> Dev@wso2.org >>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >> >> ___ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > *Darshana Akalanka Pagoda Arachchi,* > *Software Engineer* > *078-4721791* > > ___ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Issue when installing MB features in products that use carbon.event.core
I propose to change the name of 'master-datasources.xml' as well. When we tried to install MB features in the IoT server, we had to manually add mb-store configurations into their master-datasources.xml from the code level. Thank you On Wed, Jan 6, 2016 at 3:00 PM, Ramith Jayasinghe <ram...@wso2.com> wrote: > I propose changing the file name at MB side. so the changes are minimal. > we plan to further optimize this area in upcoming versions. > @Hemika, shall we do this? > > On Wed, Jan 6, 2016 at 2:52 PM, Ruwan Yatawara <ruw...@wso2.com> wrote: > >> Hi All, >> >> We ran to in to the above problem when trying to install MB features in >> the IOTServer. As we discovered when investigating, root cause of the >> problem was the use of event-broker.xml, which is used for loading >> configurations in both components. >> >> MB uses the andes.event.core component which has now come to replace the >> event.core from back in the day (event.core is still referred to from >> governance components and synapse). Since both components refer to the same >> file, each try to load contradictory (internal) classes that lead to class >> not found exceptions. >> >> As a workaround to overcome this problem, we will have to rename the >> event-broker.xml file from event.core feature to something like >> carbon-event-core.xml to ensure the co-existing of both components in >> harmony. In addition we will also have to remove specific checks for >> andes/qpid packages inside of the event.core. >> >> I have done the changes against the 4.4.8 tag in a separate branch on my >> personal repository [1]. Can we get a 4.4.9 version released with this >> change [2] for us to move forward with the milestone? >> >> [1] - https://github.com/ruwany/carbon-commons/tree/IoTS-1.0.0-M1 >> [2] - >> https://github.com/ruwany/carbon-commons/commit/4ea0f1f52b7fa9bc57b856e2271ffcad25eb965a >> >> Thanks and Regards, >> >> Ruwan Yatawara >> >> Senior Software Engineer, >> WSO2 Inc. >> >> email : ruw...@wso2.com >> mobile : +94 77 9110413 >> blog : http://ruwansrants.blogspot.com/ >> www: :http://wso2.com >> >> > > > -- > Ramith Jayasinghe > Technical Lead > WSO2 Inc., http://wso2.com > lean.enterprise.middleware > > E: ram...@wso2.com > P: +94 777542851 > > > ___ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [VOTE] Release WSO2 Message Broker 3.0.0 RC 2
Hi all, Sajini and I tested the following in a 3 node MB cluster where a mixture of topics, queues and shared topics were used: An MB node in the cluster was killed when the clients were in the middle of publishing and subscriber. All the publishers and subscribers are connected to the node that is going to get killed. Following are the scenarios that were verified in this test: - Kill a normal node. All the publishers and subscribers fail-over successfully. - The coordinator selection happens successfully. Also, the publishers and subscribers fail-over successfully. - First kill the coordinator node, then kill the newly appointed coordinator node. Fail-over happens successfully. Did not encounter issues: Thus, +1 Stable-go ahead and release Thank you On Sat, Nov 7, 2015 at 3:28 AM, Akalanka Pagoda Arachchi <darsha...@wso2.com > wrote: > Hi Devs, > > This is the second release candidate of WSO2 Message Broker 3.0.0. > > This release fixes the following issues: > https://wso2.org/jira/issues/?filter=12508 > > <https://wso2.org/jira/issues/?filter=12422> > Please download, test and vote. > > Source & binary distribution files: > https://github.com/wso2/product-mb/releases/tag/v3.0.0-RC2 > > Maven staging repo: > http://maven.wso2.org/nexus/content/repositories/orgwso2mb-118/ > > The tag to be voted upon: > https://github.com/wso2/product-mb/tree/v3.0.0-RC2 > > > [+] Stable - go ahead and release > [-] Broken - do not release (explain why) > > Thanks and Regards, > MB Team. > > -- > *Darshana Akalanka Pagoda Arachchi,* > *Software Engineer* > *078-4721791* > > _______ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] carbon-multitenancy-4.4.4 released !
Hi all, When I tried to upgrade to multitency 4.4.4, I get the following error 'Missing requirement: WSO2 Tenant Common Composite Feature 4.4.4 (org.wso2.carbon.tenant.common.feature.group 4.4.4) requires 'org.wso2.carbon.registry.core.feature.group [4.5.0,4.6.0)' but it could not be found' But there are no registry 4.5.x releases Thank you On Mon, Oct 26, 2015 at 3:30 PM, Kalpa Welivitigoda <kal...@wso2.com> wrote: > Hi devs, > > Please note $subject, > > org.wso2.carbon.multitenancy > carbon-multitenancy > 4.4.4 > > > -- > Best Regards, > > Kalpa Welivitigoda > Software Engineer, WSO2 Inc. http://wso2.com > Email: kal...@wso2.com > Mobile: +94776509215 > > ___ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [VOTE] Release WSO2 Carbon Kernel 4.4.2 RC2
WDYT ? Can we do this ?
>>>>>
>>>>> Regards,
>>>>> Johann.
>>>>>
>>>>>
>>>>> On Wed, Oct 14, 2015 at 6:30 PM, Kishanthan Thangarajah <
>>>>> kishant...@wso2.com> wrote:
>>>>>
>>>>>> Can we also have test case for this fix please?
>>>>>>
>>>>>> On Wed, Oct 14, 2015 at 6:13 PM, Isura Karunaratne <is...@wso2.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> This issue is fixed in [1].
>>>>>>>
>>>>>>>
>>>>>>> Thanks
>>>>>>> isura
>>>>>>>
>>>>>>>
>>>>>>> [1] https://wso2.org/jira/browse/CARBON-15517
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Oct 14, 2015 at 11:25 AM, Johann Nallathamby <
>>>>>>> joh...@wso2.com> wrote:
>>>>>>>
>>>>>>>> Hi Isura,
>>>>>>>>
>>>>>>>> Can you look into this issue urgently. I remember you fixing an
>>>>>>>> issue related to this.
>>>>>>>>
>>>>>>>> Thanks.
>>>>>>>>
>>>>>>>> On Wed, Oct 14, 2015 at 7:16 AM, Indika Sampath <indi...@wso2.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> I debug code of our and found issue. It seems implementation of
>>>>>>>>> some API changed in user-core. Let me explain the flow.
>>>>>>>>>
>>>>>>>>> Our queue/topic creation has two call.
>>>>>>>>>
>>>>>>>>> 1. We create internal role when adding queue and assign
>>>>>>>>> "changePermission", "publish", "consume" permissions to it. Which
>>>>>>>>> means
>>>>>>>>> that, user who created particular queue can update permission,
>>>>>>>>> publish or
>>>>>>>>> consume.
>>>>>>>>>
>>>>>>>>> - Below code line used to get internal role name:
>>>>>>>>>
>>>>>>>>> UserCoreUtil.addInternalDomainName(QUEUE_ROLE_PREFIX +
>>>>>>>>> queueName.replace(".","-").replace("/", "-"))
>>>>>>>>>
>>>>>>>>> result = {java.lang.String@10289}"*Internal/Q_userQueue*"
>>>>>>>>> value = {char[21]@10290}
>>>>>>>>> hash = 0
>>>>>>>>> hash32 = 0
>>>>>>>>>
>>>>>>>>> - assign permission as below:
>>>>>>>>>
>>>>>>>>> userStoreManager.addRole(roleName, user, null);
>>>>>>>>> userRealm.getAuthorizationManager().authorizeRole(roleName,
>>>>>>>>> queueId, PERMISSION_CHANGE_PERMISSION);
>>>>>>>>> userRealm.getAuthorizationManager().authorizeRole(roleName,
>>>>>>>>> queueId, TreeNode.Permission.CONSUME.toString().toLowerCase());
>>>>>>>>> userRealm.getAuthorizationManager().authorizeRole(roleName,
>>>>>>>>> queueId, TreeNode.Permission.PUBLISH.toString().toLowerCase());
>>>>>>>>>
>>>>>>>>> 2. User can select some other role listed in in queue add page. He
>>>>>>>>> can select these role when adding queue or later by updating queue.
>>>>>>>>> So in
>>>>>>>>> update permission we checked whether any of user's role has above
>>>>>>>>> assign
>>>>>>>>> change permission.
>>>>>>>>>
>>>>>>>>> - get role list of user:
>>>>>>>>>
>>>>>>>>> userRealm.getUserStoreManager().getRoleListOfUser(loggedInUser)
>>>>>>>>>
>>>>>>>>> result = {java.lang.String[3]@9689}
>>>>>>>>> [0] = {java.lang.String@9690}"*Internal/Q_userQueue*"
>>&
Re: [Dev] Hazelcast Issue - sequence is too small. the current head sequence...
Hi Hasitha, Yeah, the default TopicOverloadPolicy is BLOCK. Thank you On Wed, Oct 14, 2015 at 1:06 PM, Hasitha Amal De Silva <hasit...@wso2.com> wrote: > Hi Sasikala, > > 2. In case the producing node is attempting an over-write on the buffer, > hold it until the buffer is processed. (flow control for subscriptions..) > > This is provided by default configurations of Hazelcast 'RealiableTopic' > as per [1] > > So the default TopicOverloadPolicy is BLOCK in an overwrite scenario ? if > not that would be the ideal setting IMO. > > Thanks > > On Wed, Oct 14, 2015 at 9:53 AM, Sasikala Kottegoda <sasik...@wso2.com> > wrote: > >> Hi Hasitha, >> >> Please find the comments in line for the ways of handling the issue. >> >> 1. Come to a default balance between the buffer size (currently 1000) and >> the TTL value for each item in buffer (seconds) / make this configurable. >> >> I don't think there is an impact of the ring buffer size on the >> expiration of items. Increasing the TTL seems the only solution. But we did >> not increase this since we needed to avoid the possibility of getting old >> notifications in the case of a network failure or a crashed node. Making >> the TTL configurable seems like a good solution. >> >> 2. In case the producing node is attempting an over-write on the buffer, >> hold it until the buffer is processed. (flow control for subscriptions..) >> >> This is provided by default configurations of Hazelcast 'RealiableTopic' >> as per [1] >> >> [1] >> http://docs.hazelcast.org/docs/latest-dev/manual/html/reliabletopic.html#reliable-topic-configuration >> >> Thank you >> >> On Tue, Oct 13, 2015 at 8:07 PM, Hasitha Amal De Silva <hasit...@wso2.com >> > wrote: >> >>> increasing the Time-to-Live value [1] from 1 second to about 30 seconds >>> got rid of this issue for my scenario. This exception can only occur if the >>> item is expired or had been rewritten within the buffer as per [2]. >>> >>> 2 ways to properly handle this as I can think : >>> >>> 1. Come to a default balance between the buffer size (currently 1000) >>> and the TTL value for each item in buffer (seconds) / make this >>> configurable. >>> 2. In case the producing node is attempting an over-write on the buffer, >>> hold it until the buffer is processed. (flow control for subscriptions..) >>> >>> For me, the issue was reproduced by creating about 400 durable JMS topic >>> subscriptions with a ramp time of about 2 seconds from Jmeter. >>> >>> [1] : http://blog.hazelcast.com/ringbuffer-data-structure/ >>> [2] : >>> http://docs.hazelcast.org/docs/3.5/javadoc/com/hazelcast/ringbuffer/StaleSequenceException.html >>> >>> >>> >>> On Tue, Oct 6, 2015 at 4:47 PM, Hasitha Hiranya <hasit...@wso2.com> >>> wrote: >>> >>>> Hi Peter, >>>> >>>> We are using Hazelcast in our Message Broker project. We are getting >>>> following exception when using a iTopic for syncing data. >>>> >>>> Appreciate any idea why this happen? Are we doing something wrong in >>>> our client application? >>>> >>>> We found code [1] throwing the exception. >>>> >>>> Oct 06, 2015 4:26:44 PM >>>> com.hazelcast.ringbuffer.impl.operations.ReadManyOperation >>>> SEVERE: [192.168.1.50]:4000 [wso2.mb.domain] [3.5] sequence:8 is too >>>> small. The current headSequence is:9 >>>> com.hazelcast.ringbuffer.StaleSequenceException: sequence:8 is too >>>> small. The current headSequence is:9 >>>> at >>>> com.hazelcast.ringbuffer.impl.RingbufferContainer.checkBlockableReadSequence(RingbufferContainer.java:176) >>>> at >>>> com.hazelcast.ringbuffer.impl.operations.ReadManyOperation.beforeRun(ReadManyOperation.java:55) >>>> at >>>> com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:131) >>>> at >>>> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.processOperation(OperationThread.java:154) >>>> at >>>> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.process(OperationThread.java:110) >>>> at >>>> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.doRun(OperationThread.java:101) >>>> at >>>> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.run(OperationThread.java:76) >>>
Re: [Dev] Hazelcast Issue - sequence is too small. the current head sequence...
Hi Hasitha, Please find the comments in line for the ways of handling the issue. 1. Come to a default balance between the buffer size (currently 1000) and the TTL value for each item in buffer (seconds) / make this configurable. I don't think there is an impact of the ring buffer size on the expiration of items. Increasing the TTL seems the only solution. But we did not increase this since we needed to avoid the possibility of getting old notifications in the case of a network failure or a crashed node. Making the TTL configurable seems like a good solution. 2. In case the producing node is attempting an over-write on the buffer, hold it until the buffer is processed. (flow control for subscriptions..) This is provided by default configurations of Hazelcast 'RealiableTopic' as per [1] [1] http://docs.hazelcast.org/docs/latest-dev/manual/html/reliabletopic.html#reliable-topic-configuration Thank you On Tue, Oct 13, 2015 at 8:07 PM, Hasitha Amal De Silva <hasit...@wso2.com> wrote: > increasing the Time-to-Live value [1] from 1 second to about 30 seconds > got rid of this issue for my scenario. This exception can only occur if the > item is expired or had been rewritten within the buffer as per [2]. > > 2 ways to properly handle this as I can think : > > 1. Come to a default balance between the buffer size (currently 1000) and > the TTL value for each item in buffer (seconds) / make this configurable. > 2. In case the producing node is attempting an over-write on the buffer, > hold it until the buffer is processed. (flow control for subscriptions..) > > For me, the issue was reproduced by creating about 400 durable JMS topic > subscriptions with a ramp time of about 2 seconds from Jmeter. > > [1] : http://blog.hazelcast.com/ringbuffer-data-structure/ > [2] : > http://docs.hazelcast.org/docs/3.5/javadoc/com/hazelcast/ringbuffer/StaleSequenceException.html > > > > On Tue, Oct 6, 2015 at 4:47 PM, Hasitha Hiranya <hasit...@wso2.com> wrote: > >> Hi Peter, >> >> We are using Hazelcast in our Message Broker project. We are getting >> following exception when using a iTopic for syncing data. >> >> Appreciate any idea why this happen? Are we doing something wrong in our >> client application? >> >> We found code [1] throwing the exception. >> >> Oct 06, 2015 4:26:44 PM >> com.hazelcast.ringbuffer.impl.operations.ReadManyOperation >> SEVERE: [192.168.1.50]:4000 [wso2.mb.domain] [3.5] sequence:8 is too >> small. The current headSequence is:9 >> com.hazelcast.ringbuffer.StaleSequenceException: sequence:8 is too small. >> The current headSequence is:9 >> at >> com.hazelcast.ringbuffer.impl.RingbufferContainer.checkBlockableReadSequence(RingbufferContainer.java:176) >> at >> com.hazelcast.ringbuffer.impl.operations.ReadManyOperation.beforeRun(ReadManyOperation.java:55) >> at >> com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:131) >> at >> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.processOperation(OperationThread.java:154) >> at >> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.process(OperationThread.java:110) >> at >> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.doRun(OperationThread.java:101) >> at >> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.run(OperationThread.java:76) >> >> >> [1]. >> https://github.com/hazelcast/hazelcast/blob/master/hazelcast/src/main/java/com/hazelcast/ringbuffer/impl/RingbufferContainer.java >> >> Thanks >> >> -- >> *Hasitha Abeykoon* >> Senior Software Engineer; WSO2, Inc.; http://wso2.com >> *cell:* *+94 719363063* >> *blog: **abeykoon.blogspot.com* <http://abeykoon.blogspot.com> >> >> >> ___ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > Cheers, > > Hasitha Amal De Silva > Software Engineer > Mobile : 0772037426 > Blog: http://devnutshell.tumblr.com/ > WSO2 Inc.: http://wso2.com ( lean.enterprise.middleware. ) > -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [VOTE] Release WSO2 Carbon Kernel 4.4.2 RC2
Hi Kishanthan,
Yes, it is present in 4.4.2 RC1 as well. But not in 4.4.1.
Thank you.
On Tue, Oct 13, 2015 at 5:22 PM, Kishanthan Thangarajah <kishant...@wso2.com
> wrote:
> Was this issue found in 4.4.2 RC1 too?
>
> On Tue, Oct 13, 2015 at 4:58 PM, Sasikala Kottegoda <sasik...@wso2.com>
> wrote:
>
>> Hi Manuri,
>>
>> We tested MB 3.0.0 with this release and our scenario of queue creation
>> fails after giving a permission denied error. The scenario is as follows:
>>
>> 1. Create a user "user1" with a role assigned with permission to create
>> queues.
>> 2. Login from "user1" and try to create a queue, we get a permission
>> denied error.
>>
>> When creating a queue the following happens from our code.
>>
>> 1. We create an internal role for the queue and assign it to the current
>> user with permissions assigned.
>>
>> userRealm.getAuthorizationManager().authorizeRole(roleName, queueId,
>>
>> PERMISSION_CHANGE_PERMISSION);
>>
>> 2. Next, we create the queue and update permissions for the queue. In this
>> step, we check if the current user has permissions to change the queue.
>>
>> String[] userRoles =
>> userRealm.getUserStoreManager().getRoleListOfUser(loggedInUser);
>> for (String userRole : userRoles) {
>> if (userRealm.getAuthorizationManager().isRoleAuthorized(
>> userRole, queueID, PERMISSION_CHANGE_PERMISSION)) {
>> isUserHasChangePermission = true;
>> }
>> }
>>
>> At this stage, *'*(userRealm.getAuthorizationManager().isRoleAuthorized(
>> userRole, queueID, PERMISSION_CHANGE_PERMISSION))' false
>> implying that any of roles assigned to the user do not have permissions to
>> change the queue, thus not allowing the user to create the queue.
>>
>>
>> Thank you
>>
>>
>> On Mon, Oct 12, 2015 at 9:24 PM, Manuri Amaya Perera <manu...@wso2.com>
>> wrote:
>>
>>> Hi Devs,
>>>
>>> WSO2 Carbon Kernel 4.4.2 RC2 Release Vote.
>>>
>>> This release fixes the following issues:
>>> https://wso2.org/jira/issues/?filter=12396
>>>
>>> Please download and test your products with kernel 4.4.2 RC2 and vote.
>>> Vote will be open for 72 hours or longer as needed.
>>>
>>> *Source and binary distribution files:*
>>> https://svn.wso2.org/repos/wso2/people/aruna/v4.4.2-rc2
>>>
>>> *Maven staging repository:*
>>> http://maven.wso2.org/nexus/content/repositories/orgwso2carbon-019/
>>>
>>> *The tag to be voted upon:*
>>> https://github.com/wso2/carbon-kernel/tree/v4.4.2-rc2
>>>
>>>
>>> [ ] Broken - do not release (explain why)
>>> [ ] Stable - go ahead and release
>>>
>>>
>>> Thank you
>>> Carbon Team
>>>
>>> --
>>>
>>> *Manuri Amaya Perera*
>>>
>>> *Software Engineer*
>>>
>>> *WSO2 Inc.*
>>>
>>> *Blog: http://manuriamayaperera.blogspot.com
>>> <http://manuriamayaperera.blogspot.com>*
>>>
>>> ___
>>> Dev mailing list
>>> Dev@wso2.org
>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>
>>>
>>
>>
>> --
>> Sasikala Kottegoda
>> *Software Engineer*
>> WSO2 Inc., http://wso2.com/
>> lean. enterprise. middleware
>> Mobile: +94 774835928/712792401
>>
>> ___
>> Dev mailing list
>> Dev@wso2.org
>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>
>>
>
>
> --
> *Kishanthan Thangarajah*
> Associate Technical Lead,
> Platform Technologies Team,
> WSO2, Inc.
> lean.enterprise.middleware
>
> Mobile - +94773426635
> Blog - *http://kishanthan.wordpress.com <http://kishanthan.wordpress.com>*
> Twitter - *http://twitter.com/kishanthan <http://twitter.com/kishanthan>*
>
--
Sasikala Kottegoda
*Software Engineer*
WSO2 Inc., http://wso2.com/
lean. enterprise. middleware
Mobile: +94 774835928/712792401
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [VOTE] Release WSO2 Carbon Kernel 4.4.2 RC2
Hi Manuri,
We tested MB 3.0.0 with this release and our scenario of queue creation
fails after giving a permission denied error. The scenario is as follows:
1. Create a user "user1" with a role assigned with permission to create
queues.
2. Login from "user1" and try to create a queue, we get a permission denied
error.
When creating a queue the following happens from our code.
1. We create an internal role for the queue and assign it to the current
user with permissions assigned.
userRealm.getAuthorizationManager().authorizeRole(roleName, queueId,
PERMISSION_CHANGE_PERMISSION);
2. Next, we create the queue and update permissions for the queue. In
this step, we check if the current user has permissions to change the
queue.
String[] userRoles =
userRealm.getUserStoreManager().getRoleListOfUser(loggedInUser);
for (String userRole : userRoles) {
if (userRealm.getAuthorizationManager().isRoleAuthorized(
userRole, queueID, PERMISSION_CHANGE_PERMISSION)) {
isUserHasChangePermission = true;
}
}
At this stage, *'*(userRealm.getAuthorizationManager().isRoleAuthorized(
userRole, queueID, PERMISSION_CHANGE_PERMISSION))' false
implying that any of roles assigned to the user do not have
permissions to change the queue, thus not allowing the user to create
the queue.
Thank you
On Mon, Oct 12, 2015 at 9:24 PM, Manuri Amaya Perera <manu...@wso2.com>
wrote:
> Hi Devs,
>
> WSO2 Carbon Kernel 4.4.2 RC2 Release Vote.
>
> This release fixes the following issues:
> https://wso2.org/jira/issues/?filter=12396
>
> Please download and test your products with kernel 4.4.2 RC2 and vote.
> Vote will be open for 72 hours or longer as needed.
>
> *Source and binary distribution files:*
> https://svn.wso2.org/repos/wso2/people/aruna/v4.4.2-rc2
>
> *Maven staging repository:*
> http://maven.wso2.org/nexus/content/repositories/orgwso2carbon-019/
>
> *The tag to be voted upon:*
> https://github.com/wso2/carbon-kernel/tree/v4.4.2-rc2
>
>
> [ ] Broken - do not release (explain why)
> [ ] Stable - go ahead and release
>
>
> Thank you
> Carbon Team
>
> --
>
> *Manuri Amaya Perera*
>
> *Software Engineer*
>
> *WSO2 Inc.*
>
> *Blog: http://manuriamayaperera.blogspot.com
> <http://manuriamayaperera.blogspot.com>*
>
> ___
> Dev mailing list
> Dev@wso2.org
> http://wso2.org/cgi-bin/mailman/listinfo/dev
>
>
--
Sasikala Kottegoda
*Software Engineer*
WSO2 Inc., http://wso2.com/
lean. enterprise. middleware
Mobile: +94 774835928/712792401
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] WSO2 Committers += Sasikala Kottegoda
Thanks all... :) And special thanks to the MB team for the help that I've received so far... :) On Mon, Oct 12, 2015 at 10:59 AM, Ramith Jayasinghe <ram...@wso2.com> wrote: > Hi Sasikala, > Congratulations! > It is with great pleasure that we welcome you as a WSO2 committer. Keep up > the good work! > regards, > > -- > Ramith Jayasinghe > Technical Lead > WSO2 Inc., http://wso2.com > lean.enterprise.middleware > > > -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] [Architecture] WSO2 Message Broker 3.0.0 Alpha-4 Released !
rowse/MB-1388>] - Remove cassandra related source and libs The documentation for MB 3.0.0 Alpha4 can be found at [2]. Your feedback is most welcome. Any issue regarding the project can be reported at [3]. [1] https://github.com/wso2/product-mb/releases/download/v3.0.0-ALPHA4/wso2mb-3.0.0-ALPHA4.zip [2] https://docs.wso2.com/display/MB300/WSO2+Message+Broker+Documentation [3] https://wso2.org/jira/browse/MB Thanks, -WSO2 MB Team- -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] Allowing jms publishers to create queues when publishing
Hi all, According to the current implementation of WSO2 MB, publishers cannot create queues when publishing even though the subscribers can. Also, even though the queue is not internally getting created, messages published for a new queue are written into the database with the published queue name and therefore are stored. An issue regarding this situation can be found at [1]. My concern is whether we should allow the creation of queues by publishers or whether we should restrict it so that message publishing to a non-existing queue is not allowed. Any comment will be highly appreciated. [1] https://wso2.org/jira/browse/MB-1433 Thank you -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] [Architecture] WSO2 Message Broker 3.0.0 Alpha-3 Released !
<https://wso2.org/jira/browse/MB-1274>] - When update the
permission, duplicate role is getting created
- [MB-1277 <https://wso2.org/jira/browse/MB-1277>] -
DurableTopicsSubscribtion gets permanently active when shutdown the server
while database connection is lost
- [MB-1278 <https://wso2.org/jira/browse/MB-1278>] - Duplicate
'subscription_id' were created in the MB_DURABLE_SUBSCRIPTION table
- [MB-1281 <https://wso2.org/jira/browse/MB-1281>] - Info popup instead
of warning shown when subscribe is clicked when mandatory fields are not
filled under topic subscription
- [MB-1287 <https://wso2.org/jira/browse/MB-1287>] - MB hanged when
shutting down
- [MB-1288 <https://wso2.org/jira/browse/MB-1288>] - Tenant - Can not
delete a topic after subscriber is stopped
- [MB-1289 <https://wso2.org/jira/browse/MB-1289>] - Cannot update the
topic permission to a user that has all permissions(super admin/admin)
- [MB-1290 <https://wso2.org/jira/browse/MB-1290>] - Tenants can consume
messages from super tenants queues when the queue name is same
- [MB-1291 <https://wso2.org/jira/browse/MB-1291>] - Message duplicate
when subscriber create and close connection per message in queues
- [MB-1296 <https://wso2.org/jira/browse/MB-1296>] - Tenant - When
subscribers are stopped still the subsciber details are shown in topic
details section
- [MB-1297 <https://wso2.org/jira/browse/MB-1297>] - Number Of Messages
Delivery Pending is shown as “-1” when browsing active queue subscriptions
- [MB-1299 <https://wso2.org/jira/browse/MB-1299>] - User with queue and
topic permissions can see topics created by other users
- [MB-1300 <https://wso2.org/jira/browse/MB-1300>] -
NullPointerException occurred in message delivery flow
- [MB-1303 <https://wso2.org/jira/browse/MB-1303>] - Super user cannot
publish messages to a Queue created by a user in the same domain in MC
- [MB-1304 <https://wso2.org/jira/browse/MB-1304>] - Users gets "Queue
already created" popup when trying to create queue with a name that super
user has already used
- [MB-1305 <https://wso2.org/jira/browse/MB-1305>] - "Purge Topic"
permission under topic permissions section is unnecessary
- [MB-1308 <https://wso2.org/jira/browse/MB-1308>] - Observed WARN
{org.wso2.carbon.event.core.internal.topic.registry.RegistryTopicManager} -
while updating topic permissions
- [MB-1309 <https://wso2.org/jira/browse/MB-1309>] - Truststore
configurations are not used when creating the SSLContext in server
- [MB-1312 <https://wso2.org/jira/browse/MB-1312>] - Subscription to
Dead Letter Channel is shown under active subscriptions
- [MB-1314 <https://wso2.org/jira/browse/MB-1314>] -
NullPointerException when subscriber closes connection without consuming
messages
- [MB-1316 <https://wso2.org/jira/browse/MB-1316>] - A new user with
default Admin permission cannot publish messages to a queue created by the
primary admin user
- [MB-1336 <https://wso2.org/jira/browse/MB-1336>] - Cannot start MB
pack due to an exception thrown in Oracle 12c
- [MB-1341 <https://wso2.org/jira/browse/MB-1341>] - When a topic is
created by console and when a subscriber is subscribed two roles are
created for the topic
- [MB-1346 <https://wso2.org/jira/browse/MB-1346>] - Refactor code in
RDBMSConstants
- [MB-1350 <https://wso2.org/jira/browse/MB-1350>] - Unable to update
permissions for a topic when a role has special characters.
- [MB-1353 <https://wso2.org/jira/browse/MB-1353>] - The number of
delivery attempts has a mismatch with the 'maximumRedeliveryAttempts'
configuration
- [MB-1354 <https://wso2.org/jira/browse/MB-1354>] - Filtering messages
by a queue in DLC gives an excpetion
Improvements:
- [MB-1178 <https://wso2.org/jira/browse/MB-1178>] - Implement
StoreHealthListener to DatabaseSlotManager
- [MB-1198 <https://wso2.org/jira/browse/MB-1198>] - ability configure
message cache
- [MB-1236 <https://wso2.org/jira/browse/MB-1236>] - Implement store
health aware database slot agent
- [MB-1237 <https://wso2.org/jira/browse/MB-1237>] - Replace Hazelcast
topics from Hazelcast Reliable Topics
- [MB-1253 <https://wso2.org/jira/browse/MB-1253>] - Replace " SELECT *
" in RDBMS queries
- [MB-1259 <https://wso2.org/jira/browse/MB-1259>] - Add mqtt retain to
new delivery model
New Features:
- [MB-965 <https://wso2.org/jira/browse/MB-965>] - Implement/Test multi
tenancy for MQTT
Security Vulnerabilities:
- [MB-851 <https://wso2.org/jira/browse/MB-851>] - Users In tenant
domain can create, consume and publish to carbon.super domain.
Tasks:
- [MB-1343 <h
Re: [Dev] [Vote] Release WSO2 Carbon Kernel 4.4.2 RC1
Hi all,
We applied the patch given by Indunil which had cascade delete on the
tables. However, it gave a dead lock in 'DELETE FROM UM_PERMISSION WHERE
UM_RESOURCE_ID = ? AND UM_TENANT_ID=?' and 'DELETE FROM UM_HYBRID_ROLE
WHERE UM_ROLE_NAME = ? AND UM_TENANT_ID=?'(The stack trace is attached).
Following are the code lines which give the error:
dbConnection = DatabaseUtil.getDBConnection(this.dataSource);
DatabaseUtil.updateDatabase(dbConnection, "DELETE FROM UM_HYBRID_ROLE
WHERE UM_ROLE_NAME = ? AND UM_TENANT_ID=?", new Object[]{roleName,
Integer.valueOf(this.tenantId)});
dbConnection.commit();
and
dbConnection1 = this.getDBConnection();
DatabaseUtil.updateDatabase(dbConnection1, "DELETE FROM UM_PERMISSION
WHERE UM_RESOURCE_ID = ? AND UM_TENANT_ID=?", new Object[]{resourceId,
Integer.valueOf(this.tenantId)});
this.permissionTree.clearResourceAuthorizations(resourceId);
dbConnection1.commit();
Also when I looked at the script, I found out the following
information regarding foreign key relationships:
1. UM_ROLE_PERMISSION has 2 parent tables, UM_PERMISSION and UM_DOMAIN
2. UM_HYBRID_USER_ROLE has 2 parent tables, UM HYBRID_ROLE and UM_DOMAIN
One possible reason for the deadlock is if oracle locks the parent
table when deleting from a child table. A similar issue is reported
at[1]
[1]
http://stackoverflow.com/questions/9373843/meet-deadlock-when-deleting-different-records-from-same-table-in-different-trans
On Fri, Sep 25, 2015 at 1:06 PM, Ramith Jayasinghe <ram...@wso2.com> wrote:
> Hi,
> If you are hoping to handle/catch
> 'IntegrityConstraintViolationException' consider following:
>
> 'IntegrityConstraintViolationException' is only thrown in Oracle and
> mysql database drivers ( check their driver source code and you will
> see). from what I know MSSQL drivers doesn't throw these. I have seen
> similar approach has been adapted in Hibernate (ORM) framework code as
> well.
>
> therefore, if you go down this path you need to check SQLState [1].
> You might find[2] useful as reference.
>
> [1]
> http://docs.oracle.com/javase/7/docs/api/java/sql/SQLException.html#getSQLState()
> [2]
> https://github.com/wso2/andes/blob/master/modules/andes-core/broker/src/main/java/org/wso2/andes/store/rdbms/RDBMSStoreUtils.java#115
>
> On Fri, Sep 25, 2015 at 11:16 AM, Darshana Gunawardana
> <darsh...@wso2.com> wrote:
> > Hi Indunil,
> >
> > On Fri, Sep 25, 2015 at 10:15 AM, Indunil Upeksha Rathnayake
> > <indu...@wso2.com> wrote:
> >>
> >> Hi,
> >>
> >> The deadlock issue(https://wso2.org/jira/browse/MB-1326) come up in
> oracle
> >> and mssql when clearResourceAuthorizations where permissions are
> deleted in
> >> UM_ROLE_PERMISSIONS, UM_USER_PERMISSIONS and UM_PERMISSION tables
> through
> >> one transaction. The fix would be using cascade delete for above tables.
> >
> >
> > +1
> >
> >>
> >> But I think another issue(foreign key constraint violation) may come up
> >> when addAuthorizationForRole, as following example scenario. Ex: Node1
> may
> >> be retrieve the permission entry in UM_PERMISSION table (in
> >> addAuthorizationForRole()) and just after that node2 may delete the
> entry
> >> (in clearResourceAuthorizations()) and after that when node1 trying to
> >> delete & add permissions in UM_ROLE_PERMISSIONS (in
> >> addAuthorizationForRole()) will return the above exception since the
> entry
> >> in UM_PERMISSION is already deleted.
> >>
> >> I think the fix need to be, if UM_ROLE_PERMISSIONS and UM_PERMISSION
> >> tables are going to update, first have to lock both tables, until all
> the
> >> required operations are finished. Then meanwhile another node can't
> delete
> >> any entries.
> >
> >
> > Have you able to reproduce this behavior? I guess we cannot lock those
> > tables from the java code in db in dependent manner. Please check with
> our
> > db experts.
> >
> > In worst case we could handle IntegrityConstraintViolationException since
> > its ok to avoiding addAuthorization for a permission which got deleted
> > later.
> >
> > Thanks,
> >
> >>
> >> Please provide your opinions regarding this.
> >>
> >> Thanks and Regards
> >> --
> >> Indunil Upeksha Rathnayake
> >> Software Engineer | WSO2 Inc
> >> Emailindu...@wso2.com
> >
> >
> >
> >
> > --
> > Regards,
> >
> > Darshana Gunawardana
> > Senior Software Engineer
> > WSO2 Inc.; http://wso2.com
> > E-mail: darsh...@wso2.com
> > Mobile: +94718566859
> > Lean . Enterprise . Middleware
>
>
>
> --
> Ramith Jayasinghe
> Technical Lead
> WSO2 Inc., http://wso2.com
> lean.enterprise.middleware
>
> E: ram...@wso2.com
> P: +94 777542851
>
--
Sasikala Kottegoda
*Software Engineer*
WSO2 Inc., http://wso2.com/
lean. enterprise. middleware
Mobile: +94 774835928/712792401
ORA-00060_
Description: Binary data
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] [Architecture] WSO2 Message Broker 3.0.0 Alpha Released !
rs - [MB-1175 <https://wso2.org/jira/browse/MB-1175>] - Exception thrown when creating a consumer - [MB-1177 <https://wso2.org/jira/browse/MB-1177>] - When a cluster is started (1st node) MB_SLOT table must be cleared - [MB-1187 <https://wso2.org/jira/browse/MB-1187>] - purgeMessagesTestCase fails after deleted queue being observed - [MB-1189 <https://wso2.org/jira/browse/MB-1189>] - Ability to disable content cache used in RDBM message store. - [MB-1195 <https://wso2.org/jira/browse/MB-1195>] - Unexpected behavior in JMS Queue with in client-ack mode listener - [MB-1201 <https://wso2.org/jira/browse/MB-1201>] - Error in getting roles through when adding a topic through UI. - [MB-1202 <https://wso2.org/jira/browse/MB-1202>] - Messages are not getting moved into the dead letter channel - [MB-1208 <https://wso2.org/jira/browse/MB-1208>] - Restoring/re-routing messages from DLC causes an exception - [MB-1209 <https://wso2.org/jira/browse/MB-1209>] - Samples are broken. Dependency missing. - [MB-1213 <https://wso2.org/jira/browse/MB-1213>] - Subscription Recovery Does not happen properly for durable topic subscriptions - [MB-1215 <https://wso2.org/jira/browse/MB-1215>] - Update release docs and read me files with latest changes. - [MB-1216 <https://wso2.org/jira/browse/MB-1216>] - Increase minimum heap size to 2G - [MB-1217 <https://wso2.org/jira/browse/MB-1217>] - Messages that are moved to DLC are still shown in the queue when cache is enabled Improvements: - [MB-939 <https://wso2.org/jira/browse/MB-939>] - Make the maximum display length configurable for message content shown through management console. - [MB-1038 <https://wso2.org/jira/browse/MB-1038>] - Transaction Support for Andes - [MB-1057 <https://wso2.org/jira/browse/MB-1057>] - Message content instant removal - [MB-1062 <https://wso2.org/jira/browse/MB-1062>] - Add logs to trace message path - [MB-1109 <https://wso2.org/jira/browse/MB-1109>] - Message latency increase as message publishing rate reduced - [MB-1122 <https://wso2.org/jira/browse/MB-1122>] - Better to have generic jms client details in flow control message instead of channel number - [MB-1138 <https://wso2.org/jira/browse/MB-1138>] - Not enough details when flow control is enabled - [MB-1146 <https://wso2.org/jira/browse/MB-1146>] - Remove Performance Counter - [MB-1147 <https://wso2.org/jira/browse/MB-1147>] - Implement StoreHealthListener to AndesRecoveryTask - [MB-1152 <https://wso2.org/jira/browse/MB-1152>] - Add message tracer events - [MB-1161 <https://wso2.org/jira/browse/MB-1161>] - SecureVault Support for jndi.properties - [MB-1174 <https://wso2.org/jira/browse/MB-1174>] - Stop server startup with 0.0.0.0 as Thrift server host - [MB-1179 <https://wso2.org/jira/browse/MB-1179>] - Set default auto commit mode to false in message store and context store - [MB-1186 <https://wso2.org/jira/browse/MB-1186>] - Separate method deleteMessages in MessagingEngine into two New Features: - [MB-126 <https://wso2.org/jira/browse/MB-126>] - Support JMS Local Transactions - [MB-1022 <https://wso2.org/jira/browse/MB-1022>] - Publisher Acknowledgement for Andes Core (Used by MQTT Transport) - [MB-1149 <https://wso2.org/jira/browse/MB-1149>] - Implement a mechanism to save slot information in a RDBMS Security Vulnerabilities: - [MB-1160 <https://wso2.org/jira/browse/MB-1160>] - Move amqp SSL configuration and add MQTT ssl configuration into cipher-tool compliant broker.xml Tasks: - [MB-24 <https://wso2.org/jira/browse/MB-24>] - Run a long running Test for Queue Scenario - [MB-948 <https://wso2.org/jira/browse/MB-948>] - Enable Queue purge delivery rule after testing. - [MB-1049 <https://wso2.org/jira/browse/MB-1049>] - Write a sample for JMS transaction based message publishing - [MB-1190 <https://wso2.org/jira/browse/MB-1190>] - upgrade guava lib version - [MB-1212 <https://wso2.org/jira/browse/MB-1212>] - Make 'tracker rejected' warning to a debug The documentation for MB 3.0.0 Alpha can be found at [2]. Your feedback is most welcome, and any issue regarding the project can be reported at [3]. [1] https://github.com/wso2/product-mb/releases/download/v3.0.0-ALPHA/wso2mb-3.0.0-ALPHA.zip [2] https://docs.wso2.com/display/MB300/WSO2+Message+Broker+Documentation [3] https://wso2.org/jira/browse/MB Thanks, -WSO2 MB Team- -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [Architecture] WSO2 Message Broker 3.0.0 Alpha Released !
Hi all,
Correction in the download link for the product. Use the link at [1] to
download WSO2 Message Broker 3.0.0-Alpha2
[1]
https://github.com/wso2/product-mb/releases/download/v3.0.0-ALPHA2/wso2mb-3.0.0-ALPHA2.zip
Thank you
On Tue, Sep 1, 2015 at 11:21 PM, Sasikala Kottegoda <sasik...@wso2.com>
wrote:
>
> WSO2 MB Team is pleased to announce the release of WSO2 Message Broker
> 3.0.0-Alpha2. You can now download the distribution which is available at
> [1].
>
> Following are the major changes applied in this version apart from the
> bugs and fixes:
>
> 1. Removal of the support for Cassandra
> 2. Support for JMS local transactions
>
> Bugs:
>
>- [MB-326 <https://wso2.org/jira/browse/MB-326>] - OOM in MB 2.1.0
>Beta When having more that 60 subscriptions
>- [MB-725 <https://wso2.org/jira/browse/MB-725>] - When a node is
>killed, the queues which belog to the subscribers of the dead node are also
>removed
>- [MB-734 <https://wso2.org/jira/browse/MB-734>] - NPE when browsing
>queue/topic messages sent by a C# JMS client
>- [MB-826 <https://wso2.org/jira/browse/MB-826>] - Message can be
>resent when channel closed before acknowledgement is processed.
>- [MB-848 <https://wso2.org/jira/browse/MB-848>] - Subscriber with
>permission to consume but doesn't have queue add, browse or delete
>permission failed to consume from queue
>- [MB-878 <https://wso2.org/jira/browse/MB-878>] - Incompatibility
>with with AMQP Content header when using RabbitMQ publishers
>- [MB-889 <https://wso2.org/jira/browse/MB-889>] - Display topic
>sections to an user who have only the login permissions
>- [MB-905 <https://wso2.org/jira/browse/MB-905>] - created topic
>subscriptions not removed when running with jmeter
>- [MB-924 <https://wso2.org/jira/browse/MB-924>] - broker.xml has to
>support Secure vault
>- [MB-940 <https://wso2.org/jira/browse/MB-940>] - Observed "ERROR
>{org.wso2.andes.kernel.distruptor.inbound.InboundLogExceptionHandler} - [
>Sequence: 3884182 ] Exception occurred while processing inbound
>events.Event type: MESSAGE_EVENT", with a load test
>- [MB-972 <https://wso2.org/jira/browse/MB-972>] - MQTT : Supporting
>Message Sizes Which Exceeds the Frame Size
>- [MB-993 <https://wso2.org/jira/browse/MB-993>] - Subscriber Re
>Connection Causes Stability Issues
>- [MB-998 <https://wso2.org/jira/browse/MB-998>] - An error is printed
>for each message when subscriber is disconnected while messages are
>publishing for MQTT
>- [MB-1025 <https://wso2.org/jira/browse/MB-1025>] - MQTT : Fix
>Message Size Limitation
>- [MB-1034 <https://wso2.org/jira/browse/MB-1034>] -
>ArrayIndexOutOfBoundsException when publishing MQTT messages while adding
>subscribers to same topic
>- [MB-1039 <https://wso2.org/jira/browse/MB-1039>] - MB doesn't
>support large messages for AMQP with MySQL datasource
>- [MB-1041 <https://wso2.org/jira/browse/MB-1041>] - Cassandra
>InvalidQueryException when durable subscriber goes offline
>- [MB-1052 <https://wso2.org/jira/browse/MB-1052>] -
>NullPointerException occurred in shutting down server when non durable
>subscription active
>- [MB-1053 <https://wso2.org/jira/browse/MB-1053>] - Warning log :
>Invalid State transition from DELETED suggested
>- [MB-1054 <https://wso2.org/jira/browse/MB-1054>] - MB server cannot
>restart after subscribe/publish messages to DSE cassandra
>- [MB-1058 <https://wso2.org/jira/browse/MB-1058>] - Message content
>not shown in the summary view
>- [MB-1063 <https://wso2.org/jira/browse/MB-1063>] - Browsing a queue
>which has 5 million messages is not possible to view
>- [MB-1065 <https://wso2.org/jira/browse/MB-1065>] - User with
>permission to browse queue messages is unable to browse the message
>- [MB-1066 <https://wso2.org/jira/browse/MB-1066>] - Messages are
>stuck in DB when the only publishing node is killed while publishing
>- [MB-1067 <https://wso2.org/jira/browse/MB-1067>] - Disable amqp
>protocol and publish / browse queue messages from console leads to dead
> page
>- [MB-1068 <https://wso2.org/jira/browse/MB-1068>] - When browsing
>queue messages pagination is shown for more than restricted number of
>messages
>- [MB-1069 <https://wso2.org/jira/browse/MB-1069>] - Can not delete a
>queue which has subscribed clients
>- [MB-1070 <https://wso2.org/jira/browse/MB-1070>] - MB 3.0.0 build
>with test fail
[Dev] Permission denied exception when running integration tests in MB with carbon kernal 4.4.1
Hi all, I tried to run all the tests after upgrading the carbon kernel version of MB from 4.4.0 to 4.4.1 and all the authorization tests fail with the following exception. javax.jms.JMSException: Error registering consumer: org.wso2.andes.AMQSecurityException: Permission denied: queue-name 'authQueue1' [error code 403: access refused] at org.wso2.andes.client.AMQSession$5.execute(AMQSession.java:2114) at org.wso2.andes.client.AMQSession$5.execute(AMQSession.java:2057) at org.wso2.andes.client.AMQConnectionDelegate_8_0.executeRetrySupport(AMQConnectionDelegate_8_0.java:315) at org.wso2.andes.client.AMQConnection.executeRetrySupport(AMQConnection.java:620) at org.wso2.andes.client.failover.FailoverRetrySupport.execute(FailoverRetrySupport.java:102) at org.wso2.andes.client.AMQSession.createConsumerImpl(AMQSession.java:2055) at org.wso2.andes.client.AMQSession.createConsumer(AMQSession.java:1028) at org.wso2.andes.client.AMQSession.createReceiver(AMQSession.java:1423) at org.wso2.andes.client.AMQQueueSessionAdaptor.createReceiver(AMQQueueSessionAdaptor.java:75) at org.wso2.mb.integration.common.clients.AndesJMSConsumer.createQueueConnection(AndesJMSConsumer.java:208) at org.wso2.mb.integration.common.clients.AndesJMSConsumer.init(AndesJMSConsumer.java:119) at org.wso2.mb.integration.common.clients.AndesClient.init(AndesClient.java:108) at org.wso2.mb.integration.common.clients.AndesClient.init(AndesClient.java:76) at org.wso2.mb.integration.tests.amqp.functional.QueueUserAuthorizationTestCase.createPublishAndSubscribeFromUser(QueueUserAuthorizationTestCase.java:517) at org.wso2.mb.integration.tests.amqp.functional.QueueUserAuthorizationTestCase.performQueuePermissionTestCase(QueueUserAuthorizationTestCase.java:154) Caused by: org.wso2.andes.AMQSecurityException: Permission denied: queue-name 'authQueue1' [error code 403: access refused] at org.wso2.andes.client.handler.ConnectionCloseMethodHandler.methodReceived(ConnectionCloseMethodHandler.java:85) at org.wso2.andes.client.handler.ClientMethodDispatcherImpl.dispatchConnectionClose(ClientMethodDispatcherImpl.java:192) at org.wso2.andes.framing.amqp_0_91.ConnectionCloseBodyImpl.execute(ConnectionCloseBodyImpl.java:140) at org.wso2.andes.client.state.AMQStateManager.methodReceived(AMQStateManager.java:111) at org.wso2.andes.client.protocol.AMQProtocolHandler.methodBodyReceived(AMQProtocolHandler.java:515) at org.wso2.andes.client.protocol.AMQProtocolSession.methodFrameReceived(AMQProtocolSession.java:461) at org.wso2.andes.framing.AMQMethodBodyImpl.handle(AMQMethodBodyImpl.java:96) at org.wso2.andes.client.protocol.AMQProtocolHandler$2.run(AMQProtocolHandler.java:466) at org.wso2.andes.pool.Job.processAll(Job.java:109) at org.wso2.andes.pool.Job.run(Job.java:153) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) I tried to reproduce this locally by creating a jms consumer from jmeter but it worked fine. Also, Since the test method stops running after the exception, I caught the exception and included a Thread.sleep to view the status of the user. The role 'create_queue' that was at the beginning of the test case was no longer assigned after the exception occurred even though it was there right before it occurred. Any help on this matter would be highly appreciated. Thank you. -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] WSO2 MB build failure after upgrading kernel version
Hi all, I tried to build WSO2 MB with the following version upgrades - carbon.kernel.version - from 4.4.0 to 4.4.1 - carbon.commons.version - from 4.4.0 to 4.4.3 - carbon.multitenancy.version - from 4.4.0 to 4.4.1 - carbon.identity.version - from 4.4.0 to 4.5.3 The build fails with the following message: Missing requirement: org.wso2.carbon.tenant.usage.agent 4.4.1 (org.wso2.carbon.tenant.usage.agent 4.4.1) requires 'package org.wso2.carbon.stratos.common.util [4.4.0,5.0.0)' but it could not be found Cannot satisfy dependency: From: WSO2 Tenant Usage Agent Feature 4.4.1 (org.wso2.carbon.tenant.usage.agent.feature.group 4.4.1) To: org.wso2.carbon.tenant.usage.agent [4.4.1] Could somebody help me solve this issue? Thank you -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [Architecture] Enabling flow control in Message Broker based on channel size in Bytes
Hi Asitha, The problem here is that we cannot convert the bytes value into the chunk count. For example when the bytes value is configured to 1MB, - If the user publishes messages of size 10K, 100 chunks will be allowed - If the user publishes messages of size 1K, 1000 chunks will be allowed The smaller the message size gets, the larger the number of allowed chunks. Therefore, it's unpredictable at the time of configuration. Thank you On Tue, Jul 21, 2015 at 4:43 AM, Asitha Nanayakkara asi...@wso2.com wrote: Hi all, Can't we use the bytes value in configuration (broker.xml) and convert into chunk count when reading the configuration value and use it. That way internally nothing is changed plus the user has a better idea on sizes as well. A win win situation IMO. Anyway if the global chunk count is greater than 4/5 (or some reasonable fraction) of the Disruptor ring size we can throw an exception and stop the server at startup. Am I missing something here in that approach? Regards, Asitha On Thu, Jul 16, 2015 at 6:12 PM, Sasikala Kottegoda sasik...@wso2.com wrote: Hi all, Currently Message Broker enables flow controlling based on the content chunk count. The configuration as shown below. flowControl global lowLimit800/lowLimit highLimit8000/highLimit /global bufferBased lowLimit100/lowLimit highLimit1000/highLimit /bufferBased /flowControl Flow controlling is enabled when the chunk count that is read from the buffer is higher than the specified number. Since this is misleading for a user, we decided to set the above limits for buffer sizes in Bytes. With the new implementation, we face many problems since we cannot not relate between the number of messages and the total space occupied by messages at the configuration time (Messages with different sizes could be published). Following are the problems faced due to this: - Flow controlling buffer limits are set for sizes in bytes. - Mina buffer accepts all the messages until we enable flow controlling. (It is enabled when the total size of the messages read by *the broker* is higher than the specified limit.) - For the given limit, the number of messages that could be taken in is high for small messages. - More time is taken to read and process a large number of small messages. - Therefore, by the time the broker enables flow controlling, the Mina buffer would have overflowed. - The ring buffer in the disruptor is initialized with a specified buffer size with a given number of elements. - If the publisher sends messages with smaller sizes, a large number of messages will be taken in before flow controlling is enabled, and the disruptor will overflow. - The user cannot relate between the number of messages that are published before flow controlling is enabled (By having a count specified as in the existing approach, the number of messages that are taken in is constant whatever the message size is). Therefore, we decided to leave the configuration as it is. Any comments are highly appreciated. Thank you -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Architecture mailing list architect...@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Asitha Nanayakkara* Software Engineer WSO2, Inc. http://wso2.com/ Mob: + 94 77 85 30 682 -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] [Architecture]Enabling flow control in Message Broker based on channel size in Bytes
Hi all, Currently Message Broker enables flow controlling based on the content chunk count. The configuration as shown below. flowControl global lowLimit800/lowLimit highLimit8000/highLimit /global bufferBased lowLimit100/lowLimit highLimit1000/highLimit /bufferBased /flowControl Flow controlling is enabled when the chunk count that is read from the buffer is higher than the specified number. Since this is misleading for a user, we decided to set the above limits for buffer sizes in Bytes. With the new implementation, we face many problems since we cannot not relate between the number of messages and the total space occupied by messages at the configuration time (Messages with different sizes could be published). Following are the problems faced due to this: - Flow controlling buffer limits are set for sizes in bytes. - Mina buffer accepts all the messages until we enable flow controlling. (It is enabled when the total size of the messages read by *the broker* is higher than the specified limit.) - For the given limit, the number of messages that could be taken in is high for small messages. - More time is taken to read and process a large number of small messages. - Therefore, by the time the broker enables flow controlling, the Mina buffer would have overflowed. - The ring buffer in the disruptor is initialized with a specified buffer size with a given number of elements. - If the publisher sends messages with smaller sizes, a large number of messages will be taken in before flow controlling is enabled, and the disruptor will overflow. - The user cannot relate between the number of messages that are published before flow controlling is enabled (By having a count specified as in the existing approach, the number of messages that are taken in is constant whatever the message size is). Therefore, we decided to leave the configuration as it is. Any comments are highly appreciated. Thank you -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] Dead Lock encountered when deleting a hybrid role
Hi all,
The following error was reported when trying to delete a role while running
the MB server (The complete issue can be found at [1])
TID: [-1234] [] [2015-06-29 14:46:08,002] ERROR
{org.wso2.carbon.user.core.util.DatabaseUtil} - Error! Deadlock found when
trying to get lock; try restarting transaction
{org.wso2.carbon.user.core.util.DatabaseUtil}
com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: Deadlock
found when trying to get lock; try restarting transaction
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.Util.getInstance(Util.java:386)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1066)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4190)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4122)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2570)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2731)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2818)
at
com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2157)
at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2460)
at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2377)
at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2361)
at
org.wso2.carbon.user.core.util.DatabaseUtil.updateDatabase(DatabaseUtil.java:529)
at
org.wso2.carbon.user.core.hybrid.HybridRoleManager.deleteHybridRole(HybridRoleManager.java:479)
at
org.wso2.carbon.user.core.common.AbstractUserStoreManager.deleteRole(AbstractUserStoreManager.java:2294)
When the stack trace was analized, it was observed that the error was cased
by the following method.
public void deleteHybridRole(String roleName) throws UserStoreException {
if(UserCoreUtil.isEveryoneRole(roleName, this.realmConfig)) {
throw new UserStoreException(Invalid operation);
} else {
Connection dbConnection = null;
try {
dbConnection = this.getDBConnection();
DatabaseUtil.updateDatabase(dbConnection, DELETE FROM
UM_HYBRID_USER_ROLE WHERE UM_ROLE_ID=(SELECT UM_ID FROM UM_HYBRID_ROLE
WHERE UM_ROLE_NAME = ? AND UM_TENANT_ID=?) AND UM_TENANT_ID=?, new
Object[]{roleName, Integer.valueOf(this.tenantId),
Integer.valueOf(this.tenantId)});
DatabaseUtil.updateDatabase(dbConnection, DELETE FROM
UM_HYBRID_ROLE WHERE UM_ROLE_NAME = ? AND UM_TENANT_ID=?, new
Object[]{roleName, Integer.valueOf(this.tenantId)});
dbConnection.commit();
} catch (SQLException var7) {
log.error(var7.getMessage(), var7);
throw new UserStoreException(var7.getMessage(), var7);
} finally {
DatabaseUtil.closeAllConnections(dbConnection, new
PreparedStatement[0]);
}
this.userRealm.getAuthorizationManager().clearRoleAuthorization(roleName);
}
}
Could the problem be the fact that the first statement is locking
UM_HYBRID_ROLE which is written to, in the second statement? Any comment
would be highly appreciated.
[1] https://wso2.org/jira/browse/MB-1128
Thank you
--
Sasikala Kottegoda
*Software Engineer*
WSO2 Inc., http://wso2.com/
lean. enterprise. middleware
Mobile: +94 774835928/712792401
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [Architecture] Suggestion to add a separate DLC table for WSO2MB in order to enhance performance
Hi all, With the new database design with foreign keys specified, message content get deleted automatically when message metadata is deleted. Therefore, a separate message content table is needed for messages in the DLC. And that introduces a lot of problems: - Moving to and from DLC involves several inserts and deletions - For each message(or batch) that is read from the database, which table that message should be read from, has to be specified. Due to these reasons we decided to add just a single attribute to the metadata table indicating to which DLC queue the message was moved to. That way, we could prevent moving data from one table to another. Thank you. On Wed, Jun 17, 2015 at 2:52 PM, Sasikala Kottegoda sasik...@wso2.com wrote: Hi all, Currently WSO2MB stores message metadata in a single table with the following attributes: 1. MESSAGE_ID 2. QUEUE_ID 3. MESSAGE_METADATA Also, WSO2MB supports movement of messages to a DLC (dead letter channel) when a problem is identified in the receiver's end. When moving a message from a queue to a DLC, the DLCQueueName (which is tenant specific) is retrieved, the metadata row under the relevant storageQueue is removed and a new metadata row is added with the QUEUE_ID set to the DLC queue. Therefore, when we need to, - reroute messages for a specific queue - purge a particular queue we need to retrieve MESSAGE_METADATA of each row with the QUEUE_ID set to any DLCQueue and extract the actual storageQueue (The storageQueue is one of the information stored under the MESSAGE_METADATA attribute). This operation is highly computationally expensive since we need to read all rows for all DLC queues even though we might actually have a few relevant rows. What I'm suggesting is to have a seperate MB_DLC_METADATA table with the following attributes (same as the MB_METADATA table): 1. MESSAGE_ID 2. SOTRAGE_QUEUE_ID 3. MESSAGE_METADATA The respective cassandra column family (DLCMetaData) structure will be the same as the current cassandra MetaData column family. Each row will have storageQueue as the key, each message as a column with the message_id as the column name and metadata as the value as follows: Queue_ID m_id1 m_id2 m_id3 meta1 meta2 meta3 By having this we will be able to easily filter out messages by the storageQueue from which the messages were moved. Therefore, it will save a lot of time and memory in the previously mentioned scenarios and in any other DLC specific operations. Comments and suggestions are highly valued. Thank you -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Supporting referential integrity in the database of WSO2MB
Hi Hasitha, In a durable topic scenario, for each message received from the publisher, we generate a number of messages that is equal to the number of durable subscriptions (with different message IDs). Therefore, we have each content entry associated to a unique metadata entry. So I think cascade delete will not be a problem. Thank you On Tue, Jul 7, 2015 at 3:15 PM, Hasitha Amal De Silva hasit...@wso2.com wrote: Hi guys, In a durable topic scenario, say we have 2 subscribers A and B, do we maintain 2 entries for the same message content @ message_content table ? If we refer to the same message_content entry for both subscribers, be wary of cascade deletes. Thanks On Fri, Jun 26, 2015 at 3:06 PM, Anuja Herath an...@wso2.com wrote: Hi Sasikala, Since we dropped cassandra, this will enable us to use full power of RDBMS. But make sure that mentioned operations will work on MySQL, MS SQL and oracle in the same way. Thanks, Anuja On Fri, Jun 26, 2015 at 2:56 PM, Sasikala Kottegoda sasik...@wso2.com wrote: Hi all, Currently our database schema does not support referential integrity between the tables MB_CONTENT (which stores message contet) and MB_METADATA (which store message metadata). But message content with no related metadata is meaningless. To address this problem, what we have done is deleting message content along with metadata. But if we have have a foreign key relationship between the two tables, we will be able to ensure that no message content will be left without metadata associated. Also, it will reduce the number of database calls that are made thus speeding up the process. Any comments? Thank you. -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev -- Anuja Herath *Software Engineer* *WSO2, Inc.* Mobile : +94 (0)71 429 8861 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev -- Cheers, Hasitha Amal De Silva Software Engineer Mobile : 0772037426 Blog: http://devnutshell.tumblr.com/ WSO2 Inc.: http://wso2.com ( lean.enterprise.middleware. ) -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] Supporting referential integrity in the database of WSO2MB
Hi all, Currently our database schema does not support referential integrity between the tables MB_CONTENT (which stores message contet) and MB_METADATA (which store message metadata). But message content with no related metadata is meaningless. To address this problem, what we have done is deleting message content along with metadata. But if we have have a foreign key relationship between the two tables, we will be able to ensure that no message content will be left without metadata associated. Also, it will reduce the number of database calls that are made thus speeding up the process. Any comments? Thank you. -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] WSO2 Committers += Pumudu Ruhunage
Officially congratulations Pumudu!!! :) On Wed, Jun 24, 2015 at 10:59 AM, Ramith Jayasinghe ram...@wso2.com wrote: Hi Pumudu, Congratulations! It is with great pleasure that we welcome you as a WSO2 committer. Keep up the good work! regards, -- Ramith Jayasinghe Technical Lead WSO2 Inc., http://wso2.com lean.enterprise.middleware E: ram...@wso2.com ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] [Architecture] Suggestion to add a separate DLC table for WSO2MB in order to enhance performance
Hi all, Currently WSO2MB stores message metadata in a single table with the following attributes: 1. MESSAGE_ID 2. QUEUE_ID 3. MESSAGE_METADATA Also, WSO2MB supports movement of messages to a DLC (dead letter channel) when a problem is identified in the receiver's end. When moving a message from a queue to a DLC, the DLCQueueName (which is tenant specific) is retrieved, the metadata row under the relevant storageQueue is removed and a new metadata row is added with the QUEUE_ID set to the DLC queue. Therefore, when we need to, - reroute messages for a specific queue - purge a particular queue we need to retrieve MESSAGE_METADATA of each row with the QUEUE_ID set to any DLCQueue and extract the actual storageQueue (The storageQueue is one of the information stored under the MESSAGE_METADATA attribute). This operation is highly computationally expensive since we need to read all rows for all DLC queues even though we might actually have a few relevant rows. What I'm suggesting is to have a seperate MB_DLC_METADATA table with the following attributes (same as the MB_METADATA table): 1. MESSAGE_ID 2. SOTRAGE_QUEUE_ID 3. MESSAGE_METADATA The respective cassandra column family (DLCMetaData) structure will be the same as the current cassandra MetaData column family. Each row will have storageQueue as the key, each message as a column with the message_id as the column name and metadata as the value as follows: Queue_ID m_id1 m_id2 m_id3 meta1 meta2 meta3 By having this we will be able to easily filter out messages by the storageQueue from which the messages were moved. Therefore, it will save a lot of time and memory in the previously mentioned scenarios and in any other DLC specific operations. Comments and suggestions are highly valued. Thank you -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [Architecture] Suggestion to add a separate DLC table for WSO2MB in order to enhance performance
Hi Hasitha, Each row has the storageQueue as the key, each message as a column with the message_id as the column name and metadata as the value. Same as the suggested new column family. Thank you On Wed, Jun 17, 2015 at 3:12 PM, Hasitha Hiranya hasit...@wso2.com wrote: Hi Sasikala, +1 for the idea. The respective cassandra column family (DLCMetaData) structure will be the same as the current cassandra MetaData column family. Each row will have storageQueue as the key, each message as a column with the message_id as the column name and metadata as the value as follows: How does it store messages in Cassandra in current impl? Thanks On Wed, Jun 17, 2015 at 4:22 AM, Sasikala Kottegoda sasik...@wso2.com wrote: Hi all, Currently WSO2MB stores message metadata in a single table with the following attributes: 1. MESSAGE_ID 2. QUEUE_ID 3. MESSAGE_METADATA Also, WSO2MB supports movement of messages to a DLC (dead letter channel) when a problem is identified in the receiver's end. When moving a message from a queue to a DLC, the DLCQueueName (which is tenant specific) is retrieved, the metadata row under the relevant storageQueue is removed and a new metadata row is added with the QUEUE_ID set to the DLC queue. Therefore, when we need to, - reroute messages for a specific queue - purge a particular queue we need to retrieve MESSAGE_METADATA of each row with the QUEUE_ID set to any DLCQueue and extract the actual storageQueue (The storageQueue is one of the information stored under the MESSAGE_METADATA attribute). This operation is highly computationally expensive since we need to read all rows for all DLC queues even though we might actually have a few relevant rows. What I'm suggesting is to have a seperate MB_DLC_METADATA table with the following attributes (same as the MB_METADATA table): 1. MESSAGE_ID 2. SOTRAGE_QUEUE_ID 3. MESSAGE_METADATA The respective cassandra column family (DLCMetaData) structure will be the same as the current cassandra MetaData column family. Each row will have storageQueue as the key, each message as a column with the message_id as the column name and metadata as the value as follows: Queue_ID m_id1 m_id2 m_id3 meta1 meta2 meta3 By having this we will be able to easily filter out messages by the storageQueue from which the messages were moved. Therefore, it will save a lot of time and memory in the previously mentioned scenarios and in any other DLC specific operations. Comments and suggestions are highly valued. Thank you -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev -- *Hasitha Abeykoon* Senior Software Engineer; WSO2, Inc.; http://wso2.com *cell:* *+94 719363063* *blog: **abeykoon.blogspot.com* http://abeykoon.blogspot.com -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [Architecture] Suggestion to add a separate DLC table for WSO2MB in order to enhance performance
Hi Hasitha, I think, the problem will be solved in cassandra too. As for the current design, there will be a row for each of the DLC queues (with no information as to which storageQueue it was moved from). If we are to purge a specific queue, we need to iterate through each of these dlc queues and check message metadata to find that information. But from this, we can directly access the messages moved from a particular storageQueue. Please do correct me if I'm wrong. Thank you On Wed, Jun 17, 2015 at 3:29 PM, Hasitha Hiranya hasit...@wso2.com wrote: Hi Sasikala, OK. This means in RDBMS problem will be solved. Thanks for clarification. Thanks On Wed, Jun 17, 2015 at 4:53 AM, Sasikala Kottegoda sasik...@wso2.com wrote: Hi Hasitha, Each row has the storageQueue as the key, each message as a column with the message_id as the column name and metadata as the value. Same as the suggested new column family. Thank you On Wed, Jun 17, 2015 at 3:12 PM, Hasitha Hiranya hasit...@wso2.com wrote: Hi Sasikala, +1 for the idea. The respective cassandra column family (DLCMetaData) structure will be the same as the current cassandra MetaData column family. Each row will have storageQueue as the key, each message as a column with the message_id as the column name and metadata as the value as follows: How does it store messages in Cassandra in current impl? Thanks On Wed, Jun 17, 2015 at 4:22 AM, Sasikala Kottegoda sasik...@wso2.com wrote: Hi all, Currently WSO2MB stores message metadata in a single table with the following attributes: 1. MESSAGE_ID 2. QUEUE_ID 3. MESSAGE_METADATA Also, WSO2MB supports movement of messages to a DLC (dead letter channel) when a problem is identified in the receiver's end. When moving a message from a queue to a DLC, the DLCQueueName (which is tenant specific) is retrieved, the metadata row under the relevant storageQueue is removed and a new metadata row is added with the QUEUE_ID set to the DLC queue. Therefore, when we need to, - reroute messages for a specific queue - purge a particular queue we need to retrieve MESSAGE_METADATA of each row with the QUEUE_ID set to any DLCQueue and extract the actual storageQueue (The storageQueue is one of the information stored under the MESSAGE_METADATA attribute). This operation is highly computationally expensive since we need to read all rows for all DLC queues even though we might actually have a few relevant rows. What I'm suggesting is to have a seperate MB_DLC_METADATA table with the following attributes (same as the MB_METADATA table): 1. MESSAGE_ID 2. SOTRAGE_QUEUE_ID 3. MESSAGE_METADATA The respective cassandra column family (DLCMetaData) structure will be the same as the current cassandra MetaData column family. Each row will have storageQueue as the key, each message as a column with the message_id as the column name and metadata as the value as follows: Queue_ID m_id1 m_id2 m_id3 meta1 meta2 meta3 By having this we will be able to easily filter out messages by the storageQueue from which the messages were moved. Therefore, it will save a lot of time and memory in the previously mentioned scenarios and in any other DLC specific operations. Comments and suggestions are highly valued. Thank you -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev -- *Hasitha Abeykoon* Senior Software Engineer; WSO2, Inc.; http://wso2.com *cell:* *+94 719363063* *blog: **abeykoon.blogspot.com* http://abeykoon.blogspot.com -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 -- *Hasitha Abeykoon* Senior Software Engineer; WSO2, Inc.; http://wso2.com *cell:* *+94 719363063* *blog: **abeykoon.blogspot.com* http://abeykoon.blogspot.com -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [DEV] Problem regarding Transaction Handling in MySQL
Hi all,
The problem was solved by changing the database engine type to innoDB.
Previously it was set to MyISAM which does not provide transactional
support.
Thank you
On Wed, May 27, 2015 at 10:20 AM, Sasikala Kottegoda sasik...@wso2.com
wrote:
Hi all,
MB stores message metadata and content in two separate tables.
Currently, message metadata is being deleted once the messages are
delivered, but the content is not (deleted later by a scheduled task).
What I'm trying to do is to delete (transactional delete) both message
metadata and content once they get delivered. For this, I have created two
prepared statements. I add messages to these two statements and execute the
batches as shown below.
for (AndesRemovableMetadata md : messagesToRemove) {
//add parameters to delete metadata
metadataRemovalPreparedStatement.setInt(1, queueID);
metadataRemovalPreparedStatement.setLong(2, md.getMessageID());
metadataRemovalPreparedStatement.addBatch();
//add parameters to delete content
contentRemovalPreparedStatment.setLong(1, md.getMessageID());
contentRemovalPreparedStatment.addBatch();
}
metadataRemovalPreparedStatement.executeBatch();
contentRemovalPreparedStatment.executeBatch();
connection.commit();
This works fine in a normal scenario. But when I kill the MB node (which
acts as the SQL client) suddenly, it seems like the execution of the second
prepared statement is not completed. Meaning that, even though the metadata
is removed from the respective table, the content is not. When I
interchange the statement executions, metadata gets left in the metadata
table and content gets deleted.
Could I please know what I'm doing wrong?
Also, another solution to this would be to add a foreign key constraint to
the content table to enable cascade delete. Would that be a better approach?
Thank you
--
Sasikala Kottegoda
*Software Engineer*
WSO2 Inc., http://wso2.com/
lean. enterprise. middleware
Mobile: +94 774835928/712792401
--
Sasikala Kottegoda
*Software Engineer*
WSO2 Inc., http://wso2.com/
lean. enterprise. middleware
Mobile: +94 774835928/712792401
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] Suggested change in the implementation of moving messages to DLC in MB
Hi all, Currently when messages are moved to DLC in MB, message metadata is added to the DLC queue and then deleted from the previous. This could just be handled by an update method in an RDBMS database. Therefore, could we change it? Thank you -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [DEV] Problem regarding Transaction Handling in MySQL
Yes I had set the connection autoCommit value to false.
Below is the complete code snippet:
Connection connection = null;
PreparedStatement preparedStatement = null;
PreparedStatement contentRemovalPreparedStatment = null;
Context context =
DataAccessMatrixManager.addAndGetTimer(MatrixConstants.DELETE_MESSAGE_META_DATA_FROM_QUEUE,
this).
start();
try {
int queueID = getCachedQueueID(storageQueueName);
connection = getConnection();
connection.setAutoCommit(false);
preparedStatement = connection
.prepareStatement(RDBMSConstants.PS_DELETE_METADATA_FROM_QUEUE);
contentRemovalPreparedStatment =
connection.prepareStatement(RDBMSConstants.PS_DELETE_MESSAGE_PARTS);
for (AndesRemovableMetadata md : messagesToRemove) {
//add parameter to delete metadata
preparedStatement.setInt(1, queueID);
preparedStatement.setLong(2, md.getMessageID());
preparedStatement.addBatch();
//add parameters to delete content
contentRemovalPreparedStatment.setLong(1, md.getMessageID());
contentRemovalPreparedStatment.addBatch();
}
preparedStatement.executeBatch();
contentRemovalPreparedStatment.executeBatch();
connection.commit();
//TODO find a better phrase
if (log.isDebugEnabled()) {
log.debug(Metadata and content removed. + messagesToRemove.size() +
metadata from destination + storageQueueName);
}
} catch (SQLNonTransientConnectionException sqlConEx) {
rollback(connection,
RDBMSConstants.TASK_DELETING_METADATA_FROM_QUEUE + storageQueueName
+ and + RDBMSConstants.TASK_DELETING_MESSAGE_PARTS);
throw new AndesStoreUnavailableException(error occurred while
deleting message metadata +
and content for queue + storageQueueName,
sqlConEx.getSQLState(), sqlConEx);
} catch (SQLException e) {
rollback(connection,
RDBMSConstants.TASK_DELETING_METADATA_FROM_QUEUE + storageQueueName
+ and + RDBMSConstants.TASK_DELETING_MESSAGE_PARTS);
throw new AndesException(error occurred while deleting message
metadata and content for queue ,
e);
} finally {
context.stop();
String task = RDBMSConstants.TASK_DELETING_METADATA_FROM_QUEUE +
storageQueueName;
String messageContentRemovalTask =
RDBMSConstants.TASK_DELETING_MESSAGE_PARTS;
close(preparedStatement, task);
close(contentRemovalPreparedStatment, messageContentRemovalTask);
close(connection, task + and + messageContentRemovalTask);
}
Thanks
On Wed, May 27, 2015 at 1:12 PM, Harsha Kumara hars...@wso2.com wrote:
Hi Sasikala,
Can you share your full code snippet. I hope you have initially set
connection autoCommit value to false. As Lahiru mentioned, it would be best
to have foreign key constraint with cascade delete.
Thanks,
Harsha
On Wed, May 27, 2015 at 10:20 AM, Sasikala Kottegoda sasik...@wso2.com
wrote:
Hi all,
MB stores message metadata and content in two separate tables.
Currently, message metadata is being deleted once the messages are
delivered, but the content is not (deleted later by a scheduled task).
What I'm trying to do is to delete (transactional delete) both message
metadata and content once they get delivered. For this, I have created two
prepared statements. I add messages to these two statements and execute the
batches as shown below.
for (AndesRemovableMetadata md : messagesToRemove) {
//add parameters to delete metadata
metadataRemovalPreparedStatement.setInt(1, queueID);
metadataRemovalPreparedStatement.setLong(2, md.getMessageID());
metadataRemovalPreparedStatement.addBatch();
//add parameters to delete content
contentRemovalPreparedStatment.setLong(1, md.getMessageID());
contentRemovalPreparedStatment.addBatch();
}
metadataRemovalPreparedStatement.executeBatch();
contentRemovalPreparedStatment.executeBatch();
connection.commit();
This works fine in a normal scenario. But when I kill the MB node (which
acts as the SQL client) suddenly, it seems like the execution of the second
prepared statement is not completed. Meaning that, even though the metadata
is removed from the respective table, the content is not. When I
interchange the statement executions, metadata gets left in the metadata
table and content gets deleted.
Could I please know what I'm doing wrong?
Also, another solution to this would be to add a foreign key constraint
to the content table to enable cascade delete. Would that be a better
approach?
Thank you
--
Sasikala Kottegoda
*Software Engineer*
WSO2 Inc., http://wso2.com/
lean. enterprise. middleware
Mobile: +94 774835928/712792401
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
--
Harsha Kumara
Software Engineer, WSO2 Inc.
Mobile: +94775505618
Blog:harshcreationz.blogspot.com
--
Sasikala Kottegoda
*Software Engineer
[Dev] [DEV] Problem regarding Transaction Handling in MySQL
Hi all,
MB stores message metadata and content in two separate tables.
Currently, message metadata is being deleted once the messages are
delivered, but the content is not (deleted later by a scheduled task).
What I'm trying to do is to delete (transactional delete) both message
metadata and content once they get delivered. For this, I have created two
prepared statements. I add messages to these two statements and execute the
batches as shown below.
for (AndesRemovableMetadata md : messagesToRemove) {
//add parameters to delete metadata
metadataRemovalPreparedStatement.setInt(1, queueID);
metadataRemovalPreparedStatement.setLong(2, md.getMessageID());
metadataRemovalPreparedStatement.addBatch();
//add parameters to delete content
contentRemovalPreparedStatment.setLong(1, md.getMessageID());
contentRemovalPreparedStatment.addBatch();
}
metadataRemovalPreparedStatement.executeBatch();
contentRemovalPreparedStatment.executeBatch();
connection.commit();
This works fine in a normal scenario. But when I kill the MB node (which
acts as the SQL client) suddenly, it seems like the execution of the second
prepared statement is not completed. Meaning that, even though the metadata
is removed from the respective table, the content is not. When I
interchange the statement executions, metadata gets left in the metadata
table and content gets deleted.
Could I please know what I'm doing wrong?
Also, another solution to this would be to add a foreign key constraint to
the content table to enable cascade delete. Would that be a better approach?
Thank you
--
Sasikala Kottegoda
*Software Engineer*
WSO2 Inc., http://wso2.com/
lean. enterprise. middleware
Mobile: +94 774835928/712792401
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] Unable to build Carbon Kernal 4.1.0 with maven-3.2.1
Hi, I tried to build the checked out source code of Carbon Kernal using apache-maven-3.2.1 and it gives the following error. Error injecting: org.apache.maven.reporting.exec.DefaultMavenReportExecutor java.lang.NoClassDefFoundError: org/sonatype/aether/graph/DependencyFilter at java.lang.Class.getDeclaredConstructors0(Native Method) at java.lang.Class.privateGetDeclaredConstructors(Class.java:2389) at java.lang.Class.getDeclaredConstructors(Class.java:1836) at com.google.inject.spi.InjectionPoint.forConstructorOf(InjectionPoint.java:245) at com.google.inject.internal.ConstructorBindingImpl.create(ConstructorBindingImpl.java:99) at com.google.inject.internal.InjectorImpl.createUninitializedBinding(InjectorImpl.java:653) at com.google.inject.internal.InjectorImpl.createJustInTimeBinding(InjectorImpl.java:863) at com.google.inject.internal.InjectorImpl.createJustInTimeBindingRecursive(InjectorImpl.java:790) at com.google.inject.internal.InjectorImpl.getJustInTimeBinding(InjectorImpl.java:278) at com.google.inject.internal.InjectorImpl.getBindingOrThrow(InjectorImpl.java:210) at com.google.inject.internal.InjectorImpl.getProviderOrThrow(InjectorImpl.java:986) at com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:1019) As the article https://cwiki.apache.org/confluence/display/MAVEN/AetherClassNotFoundexplains, This error is caused by Maven 3.1-alpha-1 migration from Sonatype Aether to Eclipse Aether (MNG-5354https://jira.codehaus.org/browse/MNG-5354), which is an incompatible change for some plugins. I think changes need to be made to enable the Kernal to be built using newer versions of maven. I installed maven-3.0.5 and the Kernal was built successfully. Thank you, Sasikala ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
