Re: [Architecture] Implement File Connector for zip file for WSO2 ESB
Thx Dushan, We will consider this in our roadmap On Sun, Aug 2, 2015 at 9:03 PM, Dushan Abeyruwan dus...@wso2.com wrote: Hi Malaka, [Adding Gayan G] I guess those are nice operations to have but, first we should stabilize file-connector existing methods, there has been few issues reported which were really blockers, so I guess we should test these while covering Test the existing file connector (Please check mail I have sent to dev list related to this topic), some aren't visible to public Cheers, Dushan On Thu, Jul 30, 2015 at 12:16 PM, Vivekananthan Sivanayagam vivekanant...@wso2.com wrote: Hi Malaka, I will do that one . *Thank youVivekananthan Sivanayagam* *Associate Software Engineer | WSO2* *E:vivekanant...@wso2.com e%3avivekanant...@wso2.com* *M:+94752786138 %2B94752786138* On Thu, Jul 30, 2015 at 7:55 PM, Malaka Silva mal...@wso2.com wrote: Looks good. Also include a load testing in your plan. On Thu, Jul 30, 2015 at 6:49 PM, Vivekananthan Sivanayagam vivekanant...@wso2.com wrote: Introduction The File connector https://docs.wso2.org/display/ESB481/Working+with+Connectors allows you to connect to different file systems and perform various operations with the file systems. The file connector uses the apache commons VFS I/O functionalities to execute operations. File Connector for Zip folder 1. Reads a zip files. 2. Writes files and compress to zip file. 3. Unzip file. https://docs.google.com/spreadsheets/d/1w3ZgUhhJSpEFQXRYqRm7Zq_acdnpBMhY-u9AKQ5E7uM/edit#gid=0 *Thank youVivekananthan Sivanayagam* *Associate Software Engineer | WSO2* *E:vivekanant...@wso2.com e%3avivekanant...@wso2.com* *M:+94752786138 %2B94752786138* -- Best Regards, Malaka Silva Senior Tech Lead M: +94 777 219 791 Tel : 94 11 214 5345 Fax :94 11 2145300 Skype : malaka.sampath.silva LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 Blog : http://mrmalakasilva.blogspot.com/ WSO2, Inc. lean . enterprise . middleware http://www.wso2.com/ http://www.wso2.com/about/team/malaka-silva/ http://wso2.com/about/team/malaka-silva/ Save a tree -Conserve nature Save the world for your future. Print this email only if it is absolutely necessary. ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Dushan Abeyruwan | Technical Lead Technical Support-Bloomington US PMC Member Apache Synpase WSO2 Inc. http://wso2.com/ Blog:*http://www.dushantech.com/ http://www.dushantech.com/* Mobile:(001)812-391-7441 -- Best Regards, Malaka Silva Senior Tech Lead M: +94 777 219 791 Tel : 94 11 214 5345 Fax :94 11 2145300 Skype : malaka.sampath.silva LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 Blog : http://mrmalakasilva.blogspot.com/ WSO2, Inc. lean . enterprise . middleware http://www.wso2.com/ http://www.wso2.com/about/team/malaka-silva/ http://wso2.com/about/team/malaka-silva/ Save a tree -Conserve nature Save the world for your future. Print this email only if it is absolutely necessary. ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [BPS] BPMN data publisher for DAS
Hi Isuru, On Thu, Jul 30, 2015 at 11:57 AM, Isuru Wijesinghe isur...@wso2.com wrote: Hi all, I already developed back-end component of $subject as my fast track training project and now I have planned to extend this with the front-end part (console UI component). Below I briefly describe the project description and features available there. *Description* Execution data on process instances has to be captured for analyzing the behavior of processes. In order to facilitate this, execution data of BPMN process instances has to be published to WSO2 DAS. Once published, such data can be analyzed and presented from DAS. This data publishing component can run independent of the process runtime and access instance data using the activiti engine Java API. *Back-end component* Two streams are provided for publishing the the events of completed BPMN process and task Instances to the DAS.. process.instance.stream: process_instance_id, start_activiti_id, start_user_id, start_time, end_time, duration task.instance.stream: task_instance_id, process_instance_id, start_time, complete_time, assignee, duration. *Front-end component* In the UI, user can configure the thrift url, user name and password. In addition to that it provides the configuration to select set of attributes which belongs to the both process and task instances to create two steams for publishing the attributes as events to DAS. Here it has predefined set of check boxes with attribute names and user can able to select attributes. Once they selected some attributes to initialize the two streams those are automatically disabled from the UI Here why do we need to disable them? Shouldn't user be able to change them later? (On a related note, if you want to disable such things, disabling it from UI only is not enough because backend service can be invoked from other clients as well.) I saw in a different email to dev@ from you, saying your next plan is to support multiple versions of these stream definitions. So, with that I believe you can get rid of this attribute disabling thing too. Thanks, Bhathiya and stored in the configuration registry as properties. From the back-end side read those properties and create streams to publish data as events to DAS. for More Details about the back-end component click here https://docs.google.com/document/d/1uwRm5bhC3ShJ1fh-7Nqkq31UFQI-U4-x22bLK2XYRCM/edit# . -- Isuru Wijesinghe *Software Engineer* Mobile: 0710933706 isur...@wso2.com ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Bhathiya Jayasekara* *Senior Software Engineer,* *WSO2 inc., http://wso2.com http://wso2.com* *Phone: +94715478185* *LinkedIn: http://www.linkedin.com/in/bhathiyaj http://www.linkedin.com/in/bhathiyaj* *Twitter: https://twitter.com/bhathiyax https://twitter.com/bhathiyax* *Blog: http://movingaheadblog.blogspot.com http://movingaheadblog.blogspot.com/* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Implement File Connector for zip file for WSO2 ESB
Hi Malaka, [Adding Gayan G] I guess those are nice operations to have but, first we should stabilize file-connector existing methods, there has been few issues reported which were really blockers, so I guess we should test these while covering Test the existing file connector (Please check mail I have sent to dev list related to this topic), some aren't visible to public Cheers, Dushan On Thu, Jul 30, 2015 at 12:16 PM, Vivekananthan Sivanayagam vivekanant...@wso2.com wrote: Hi Malaka, I will do that one . *Thank youVivekananthan Sivanayagam* *Associate Software Engineer | WSO2* *E:vivekanant...@wso2.com e%3avivekanant...@wso2.com* *M:+94752786138 %2B94752786138* On Thu, Jul 30, 2015 at 7:55 PM, Malaka Silva mal...@wso2.com wrote: Looks good. Also include a load testing in your plan. On Thu, Jul 30, 2015 at 6:49 PM, Vivekananthan Sivanayagam vivekanant...@wso2.com wrote: Introduction The File connector https://docs.wso2.org/display/ESB481/Working+with+Connectors allows you to connect to different file systems and perform various operations with the file systems. The file connector uses the apache commons VFS I/O functionalities to execute operations. File Connector for Zip folder 1. Reads a zip files. 2. Writes files and compress to zip file. 3. Unzip file. https://docs.google.com/spreadsheets/d/1w3ZgUhhJSpEFQXRYqRm7Zq_acdnpBMhY-u9AKQ5E7uM/edit#gid=0 *Thank youVivekananthan Sivanayagam* *Associate Software Engineer | WSO2* *E:vivekanant...@wso2.com e%3avivekanant...@wso2.com* *M:+94752786138 %2B94752786138* -- Best Regards, Malaka Silva Senior Tech Lead M: +94 777 219 791 Tel : 94 11 214 5345 Fax :94 11 2145300 Skype : malaka.sampath.silva LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 Blog : http://mrmalakasilva.blogspot.com/ WSO2, Inc. lean . enterprise . middleware http://www.wso2.com/ http://www.wso2.com/about/team/malaka-silva/ http://wso2.com/about/team/malaka-silva/ Save a tree -Conserve nature Save the world for your future. Print this email only if it is absolutely necessary. ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Dushan Abeyruwan | Technical Lead Technical Support-Bloomington US PMC Member Apache Synpase WSO2 Inc. http://wso2.com/ Blog:*http://www.dushantech.com/ http://www.dushantech.com/* Mobile:(001)812-391-7441 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [BPS] BPMN data publisher for DAS
Isuru, I also think we should not disable configuration. Instead allow creating multiple versions of streams if config is changed. - Chathura On Sun, Aug 2, 2015 at 4:46 PM, Bhathiya Jayasekara bhath...@wso2.com wrote: Hi Isuru, On Thu, Jul 30, 2015 at 11:57 AM, Isuru Wijesinghe isur...@wso2.com wrote: Hi all, I already developed back-end component of $subject as my fast track training project and now I have planned to extend this with the front-end part (console UI component). Below I briefly describe the project description and features available there. *Description* Execution data on process instances has to be captured for analyzing the behavior of processes. In order to facilitate this, execution data of BPMN process instances has to be published to WSO2 DAS. Once published, such data can be analyzed and presented from DAS. This data publishing component can run independent of the process runtime and access instance data using the activiti engine Java API. *Back-end component* Two streams are provided for publishing the the events of completed BPMN process and task Instances to the DAS.. process.instance.stream: process_instance_id, start_activiti_id, start_user_id, start_time, end_time, duration task.instance.stream: task_instance_id, process_instance_id, start_time, complete_time, assignee, duration. *Front-end component* In the UI, user can configure the thrift url, user name and password. In addition to that it provides the configuration to select set of attributes which belongs to the both process and task instances to create two steams for publishing the attributes as events to DAS. Here it has predefined set of check boxes with attribute names and user can able to select attributes. Once they selected some attributes to initialize the two streams those are automatically disabled from the UI Here why do we need to disable them? Shouldn't user be able to change them later? (On a related note, if you want to disable such things, disabling it from UI only is not enough because backend service can be invoked from other clients as well.) I saw in a different email to dev@ from you, saying your next plan is to support multiple versions of these stream definitions. So, with that I believe you can get rid of this attribute disabling thing too. Thanks, Bhathiya and stored in the configuration registry as properties. From the back-end side read those properties and create streams to publish data as events to DAS. for More Details about the back-end component click here https://docs.google.com/document/d/1uwRm5bhC3ShJ1fh-7Nqkq31UFQI-U4-x22bLK2XYRCM/edit# . -- Isuru Wijesinghe *Software Engineer* Mobile: 0710933706 isur...@wso2.com ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Bhathiya Jayasekara* *Senior Software Engineer,* *WSO2 inc., http://wso2.com http://wso2.com* *Phone: +94715478185 %2B94715478185* *LinkedIn: http://www.linkedin.com/in/bhathiyaj http://www.linkedin.com/in/bhathiyaj* *Twitter: https://twitter.com/bhathiyax https://twitter.com/bhathiyax* *Blog: http://movingaheadblog.blogspot.com http://movingaheadblog.blogspot.com/* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Implement File Connector for zip file for WSO2 ESB
On Sun, Aug 2, 2015 at 9:03 PM, Dushan Abeyruwan dus...@wso2.com wrote: Hi Malaka, [Adding Gayan G] I guess those are nice operations to have but, first we should stabilize file-connector existing methods, there has been few issues reported which were really blockers, so I guess we should test these while covering Test the existing file connector (Please check mail I have sent to dev list related to this topic), some aren't visible to public Yeah we can add those existing issues identified during some support issues to our roadmap. +1 to have those zip file related operations which seems have lot of use cases. Thanks, Gayan Cheers, Dushan On Thu, Jul 30, 2015 at 12:16 PM, Vivekananthan Sivanayagam vivekanant...@wso2.com wrote: Hi Malaka, I will do that one . *Thank youVivekananthan Sivanayagam* *Associate Software Engineer | WSO2* *E:vivekanant...@wso2.com e%3avivekanant...@wso2.com* *M:+94752786138 %2B94752786138* On Thu, Jul 30, 2015 at 7:55 PM, Malaka Silva mal...@wso2.com wrote: Looks good. Also include a load testing in your plan. On Thu, Jul 30, 2015 at 6:49 PM, Vivekananthan Sivanayagam vivekanant...@wso2.com wrote: Introduction The File connector https://docs.wso2.org/display/ESB481/Working+with+Connectors allows you to connect to different file systems and perform various operations with the file systems. The file connector uses the apache commons VFS I/O functionalities to execute operations. File Connector for Zip folder 1. Reads a zip files. 2. Writes files and compress to zip file. 3. Unzip file. https://docs.google.com/spreadsheets/d/1w3ZgUhhJSpEFQXRYqRm7Zq_acdnpBMhY-u9AKQ5E7uM/edit#gid=0 *Thank youVivekananthan Sivanayagam* *Associate Software Engineer | WSO2* *E:vivekanant...@wso2.com e%3avivekanant...@wso2.com* *M:+94752786138 %2B94752786138* -- Best Regards, Malaka Silva Senior Tech Lead M: +94 777 219 791 Tel : 94 11 214 5345 Fax :94 11 2145300 Skype : malaka.sampath.silva LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 Blog : http://mrmalakasilva.blogspot.com/ WSO2, Inc. lean . enterprise . middleware http://www.wso2.com/ http://www.wso2.com/about/team/malaka-silva/ http://wso2.com/about/team/malaka-silva/ Save a tree -Conserve nature Save the world for your future. Print this email only if it is absolutely necessary. ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Dushan Abeyruwan | Technical Lead Technical Support-Bloomington US PMC Member Apache Synpase WSO2 Inc. http://wso2.com/ Blog:*http://www.dushantech.com/ http://www.dushantech.com/* Mobile:(001)812-391-7441 -- Gayan Gunarathne Technical Lead, WSO2 Inc. (http://wso2.com) Committer PMC Member, Apache Stratos email : gay...@wso2.com | mobile : +94 775030545 %2B94%20766819985 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Possible issue with the new GW implementation
They recommend not to block the EventLoop thread under any circumstances [1]. So, the application level logic (mediation/services) needs to be handled from a separate thread pool. [1] http://normanmaurer.me/presentations/2014-facebook-eng-netty/slides.html#23.0 On Thu, Jul 30, 2015 at 11:42 PM, Isuru Ranawaka isu...@wso2.com wrote: Hi all, I think there are two aspects, handling connections and handling requests coming through connections. Basically what happens is Threads in BossGroup accepts the incoming connections and after accepting it. It creates a channel pipelines for each connection and handover accepted connections to handle by netty worker threads. So here after IO operations are handle by worker threads. each worker is responsible for handling its pipeline. Requests are coming through that pipe line and through netty handlers we can access those requests . If we use same Netty worker IO thread for handle requests then if one requests takes long processing time then following requests also affected from that which applies back pressure on IO Threads. So I think we need to handle incoming requests from separate threads to run netty worker threads smoothly. For Netty 4 on wards they have mentioned that we can share same worker pool for boss and worker groups as well. thanks IsuruR On Thu, Jul 30, 2015 at 4:14 AM, Ravi Undupitiya r...@wso2.com wrote: Netty 5.x is still in alpha and 4.x is current stable. On Thu, Jul 30, 2015 at 2:32 PM, Afkham Azeez az...@wso2.com wrote: Shouldn't we be using Netty 5? On Thu, Jul 30, 2015 at 2:26 PM, Ravi Undupitiya r...@wso2.com wrote: Hi Azeez, The reason for using a worker for executing engine tasks is to avoid using an I/O thread to do the same. AFAIU the pipeline is executed from an I/O thread. However, we might be able to share the existing netty worker pool [1] and place our worker in it. We'll update the code to use this. [1] http://netty.io/4.0/api/io/netty/channel/ChannelHandlerContext.html#executor() Thanks, Ravi On Thu, Jul 30, 2015 at 10:02 AM, Afkham Azeez az...@wso2.com wrote: I was going through the code, and in addition to the Netty boss worker pools, we have our own worker pool. Why is that? Looks like a remnant from the old PT architecture but do we really need that? Regards Azeez -- *Afkham Azeez* Director of Architecture; WSO2, Inc.; http://wso2.com Member; Apache Software Foundation; http://www.apache.org/ * http://www.apache.org/* *email: **az...@wso2.com* az...@wso2.com * cell: +94 77 3320919 %2B94%2077%203320919blog: * *http://blog.afkham.org* http://blog.afkham.org *twitter: **http://twitter.com/afkham_azeez* http://twitter.com/afkham_azeez *linked-in: **http://lk.linkedin.com/in/afkhamazeez http://lk.linkedin.com/in/afkhamazeez* *Lean . Enterprise . Middleware* -- *Ravi Undupitiya* Senior Software Engineer; WSO2 http://wso2.com *E-mail: r...@wso2.com http://wso2.com**M: **+94 772 930 712 %2B94%C2%A0772%20930%20712* Lean . Enterprise . Middleware -- *Afkham Azeez* Director of Architecture; WSO2, Inc.; http://wso2.com Member; Apache Software Foundation; http://www.apache.org/ * http://www.apache.org/* *email: **az...@wso2.com* az...@wso2.com * cell: +94 77 3320919 %2B94%2077%203320919blog: * *http://blog.afkham.org* http://blog.afkham.org *twitter: **http://twitter.com/afkham_azeez* http://twitter.com/afkham_azeez *linked-in: **http://lk.linkedin.com/in/afkhamazeez http://lk.linkedin.com/in/afkhamazeez* *Lean . Enterprise . Middleware* -- *Ravi Undupitiya* Senior Software Engineer; WSO2 http://wso2.com *E-mail: r...@wso2.com http://wso2.com**M: **+94 772 930 712 %2B94%C2%A0772%20930%20712* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Best Regards Isuru Ranawaka M: +94714629880 Blog : http://isurur.blogspot.com/ ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Kasun Indrasiri Software Architect WSO2, Inc.; http://wso2.com lean.enterprise.middleware cell: +94 77 556 5206 Blog : http://kasunpanorama.blogspot.com/ ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
