Re: [DISCUSS] Proposal for Ignite Extensions as a separate Bahir module or Incubator project
Hello, As discussed I have created the following jira issues for tracking the migration work for ignite-extensions. Main issue : IEP-36: Modularization https://issues.apache.org/jira/browse/IGNITE-12355 Sub-Tasks Migrate flink module to ignite-extensions https://issues.apache.org/jira/browse/IGNITE-12356 Migrate Twitter module to ignite-extensions https://issues.apache.org/jira/browse/IGNITE-12357 Migrate ZeroMQ module to ignite-extensions https://issues.apache.org/jira/browse/IGNITE-12358 Migrate RocketMQ module to ignite-extensions https://issues.apache.org/jira/browse/IGNITE-12359 Migrate Storm module to ignite-extensions https://issues.apache.org/jira/browse/IGNITE-12360 Migrate Flume module to ignite-extensions https://issues.apache.org/jira/browse/IGNITE-12361 Migrate MQTT module to ignite-extensions https://issues.apache.org/jira/browse/IGNITE-12362 Migrate Camel module to ignite-extensions https://issues.apache.org/jira/browse/IGNITE-12363 Migrate JMS module to ignite-extensions https://issues.apache.org/jira/browse/IGNITE-12364 Regards, Saikat On Fri, Nov 1, 2019 at 9:23 PM Saikat Maitra wrote: > Ivan, Thank you for initialising the master branch, much appreciate it. > > Denis, Emmanouil, > > We use both local server and mockito to write the tests for flink. Incase > of sink it is easier to use a local server mode and publish the data into > Ignite Sink where as in case of Ignite used as data source we use mockito > to mock Ignite events and Ignite clusters. > > Regards, > Saikat > > > > On Fri, Nov 1, 2019 at 5:28 PM Emmanouil Gkatziouras > wrote: > >> Hi, >> >> I just took another look at the Kafka Implementation and I realised that >> it >> is a local implementation that is used on the tests (I thought it was a >> server already existing somewhere). >> It seems I might be able to spin up the Pub/Sub server locally (I found an >> official tool which might work), in the same way it is done for Kafka. >> I will try to do so and come back with a pull request. >> Kind regards >> >> *Emmanouil Gkatziouras* >> https://egkatzioura.com/ | >> https://www.linkedin.com/in/gkatziourasemmanouil/ >> https://github.com/gkatzioura >> >> >> On Fri, 1 Nov 2019 at 21:31, Denis Magda wrote: >> >> > Saikat, Roman, >> > >> > What's your thinking? I'm not sure we set any infrastructure to validate >> > our Kafka or Flink integrations. For instance, for Flink we use Mockito >> > just to ensure that basic tests pass: >> > https://en.wikipedia.org/wiki/Mockito >> > >> > Our IP finders for AWS, GCE or K8 don't start up any environment for >> > testing. Instead, we create some smoke tests ensuring that the APIs are >> not >> > broken. >> > >> > - >> > Denis >> > >> > >> > On Fri, Nov 1, 2019 at 1:32 PM Emmanouil Gkatziouras < >> gkatzio...@gmail.com >> > > >> > wrote: >> > >> > > Hi all, >> > > >> > > That's great news! Yes I am ready to do a pull request on the new >> repo. >> > > I believe it is ok to create a ticket for the Pub/Sub proposal [1]. >> > > My next question has to do with the CI and the integration tests. In >> > order >> > > for my tests to work, a local Pub/Sub server needs to be up and >> running. >> > > I tried to find any deployment scripts that set the infrastructure >> needed >> > > up and running (for example Kafka) but I was not successful on this >> one. >> > > How should I proceed on that? >> > > >> > > Kind regards >> > > >> > > [1] >> > > >> > > >> > >> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Pub-Sub-Streamer-Implementation-tc43944.html >> > > >> > > *Emmanouil Gkatziouras* >> > > https://egkatzioura.com/ | >> > > https://www.linkedin.com/in/gkatziourasemmanouil/ >> > > https://github.com/gkatzioura >> > > >> > > >> > > On Fri, 1 Nov 2019 at 06:52, Ivan Pavlukhin >> wrote: >> > > >> > > > Saikat, >> > > > >> > > > I initialized master branch with empty README file [1]. I suppose >> all >> > > > ignite committers have rights for push. >> > > > >> > > > [1] https://github.com/apache/ignite-extensions >> > > > >> > > > пт, 1 нояб. 2019 г. в 08:01, Saikat Maitra > >: >> > > > > >> > > > > Hello Dmitriy, Denis >> > > > > >> > > > > Thank you for help in creating the repo. I can see it has synced >> in >> > > > github >> > > > > as well. >> > > > > >> > > > > Can you please add a sample readme.md file as the repo is empty I >> am >> > > > unable >> > > > > to fork it? >> > > > > >> > > > > or let me check if I can directly commit a sample readme file from >> > > local >> > > > > then I should be able to fork this repo. >> > > > > >> > > > > I will start working on the migration for Flink, Camel, MQTT, >> etc. >> > > > > >> > > > > Emmanouil, please take a look and feel free to raise PR in this >> new >> > > repo. >> > > > > >> > > > > Regards, >> > > > > Saikat >> > > > > >> > > > > On Thu, Oct 31, 2019 at 3:14 PM Denis Magda >> > wrote: >> > > > > >> > > > > > Dmitriy, thanks a lot for a quick turnaround! Saikat, please >> let us >> > > > know if >> > > > > >
[jira] [Created] (IGNITE-12364) Migrate JMS module to ignite-extensions
Saikat Maitra created IGNITE-12364: -- Summary: Migrate JMS module to ignite-extensions Key: IGNITE-12364 URL: https://issues.apache.org/jira/browse/IGNITE-12364 Project: Ignite Issue Type: Sub-task Components: streaming Reporter: Saikat Maitra Migrate JMS module to ignite-extensions [https://github.com/apache/ignite-extensions] Details: [https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations] Discussion : [http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td44064.html#a44107] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12363) Migrate Camel module to ignite-extensions
Saikat Maitra created IGNITE-12363: -- Summary: Migrate Camel module to ignite-extensions Key: IGNITE-12363 URL: https://issues.apache.org/jira/browse/IGNITE-12363 Project: Ignite Issue Type: Sub-task Components: streaming Reporter: Saikat Maitra Migrate Camel module to ignite-extensions [https://github.com/apache/ignite-extensions] Details: [https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations] Discussion : [http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td44064.html#a44107] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12362) Migrate MQTT module to ignite-extensions
Saikat Maitra created IGNITE-12362: -- Summary: Migrate MQTT module to ignite-extensions Key: IGNITE-12362 URL: https://issues.apache.org/jira/browse/IGNITE-12362 Project: Ignite Issue Type: Sub-task Components: streaming Affects Versions: 2.8 Reporter: Saikat Maitra Migrate MQTT module to ignite-extensions [https://github.com/apache/ignite-extensions] Details: [https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations] Discussion : [http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td44064.html#a44107] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12361) Migrate Flume module to ignite-extensions
Saikat Maitra created IGNITE-12361: -- Summary: Migrate Flume module to ignite-extensions Key: IGNITE-12361 URL: https://issues.apache.org/jira/browse/IGNITE-12361 Project: Ignite Issue Type: Sub-task Components: streaming Reporter: Saikat Maitra Migrate Flume module to ignite-extensions [https://github.com/apache/ignite-extensions] Details: [https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations] Discussion : [http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td44064.html#a44107] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12360) Migrate Storm module to ignite-extensions
Saikat Maitra created IGNITE-12360: -- Summary: Migrate Storm module to ignite-extensions Key: IGNITE-12360 URL: https://issues.apache.org/jira/browse/IGNITE-12360 Project: Ignite Issue Type: Sub-task Components: streaming Affects Versions: 2.8 Reporter: Saikat Maitra Migrate Storm module to ignite-extensions [https://github.com/apache/ignite-extensions] Details: [https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations] Discussion : [http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td44064.html#a44107] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12359) Migrate RocketMQ module to ignite-extensions
Saikat Maitra created IGNITE-12359: -- Summary: Migrate RocketMQ module to ignite-extensions Key: IGNITE-12359 URL: https://issues.apache.org/jira/browse/IGNITE-12359 Project: Ignite Issue Type: Sub-task Components: streaming Affects Versions: 2.8 Reporter: Saikat Maitra Migrate RocketMQ module to ignite-extensions [https://github.com/apache/ignite-extensions] Details: [https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations] Discussion : [http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td44064.html#a44107] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12358) Migrate ZeroMQ module to ignite-extensions
Saikat Maitra created IGNITE-12358: -- Summary: Migrate ZeroMQ module to ignite-extensions Key: IGNITE-12358 URL: https://issues.apache.org/jira/browse/IGNITE-12358 Project: Ignite Issue Type: Improvement Components: streaming Affects Versions: 2.8 Reporter: Saikat Maitra Migrate ZeroMQ module to ignite-extensions [https://github.com/apache/ignite-extensions] Details: [https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations] Discussion : [http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td44064.html#a44107] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12357) Migrate Twitter module to ignite-extensions
Saikat Maitra created IGNITE-12357: -- Summary: Migrate Twitter module to ignite-extensions Key: IGNITE-12357 URL: https://issues.apache.org/jira/browse/IGNITE-12357 Project: Ignite Issue Type: Improvement Components: streaming Affects Versions: 2.8 Reporter: Saikat Maitra -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12356) Migrate flink module to ignite-extensions
Saikat Maitra created IGNITE-12356: -- Summary: Migrate flink module to ignite-extensions Key: IGNITE-12356 URL: https://issues.apache.org/jira/browse/IGNITE-12356 Project: Ignite Issue Type: Improvement Components: streaming Reporter: Saikat Maitra Migrate flink module to ignite-extensions -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12355) IEP-36: Modularization
Saikat Maitra created IGNITE-12355: -- Summary: IEP-36: Modularization Key: IGNITE-12355 URL: https://issues.apache.org/jira/browse/IGNITE-12355 Project: Ignite Issue Type: Improvement Components: streaming Affects Versions: 2.8 Reporter: Saikat Maitra Migrate streaming modules in ignite-extensions repo [https://github.com/apache/ignite-extensions] Details: [https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations] -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: SQL query timeout: in progress or abandoned
Replied in JIRA Thanks On Sat, Nov 9, 2019 at 3:13 AM Saikat Maitra wrote: > Pavel Tupitsyn, Igor Sapego > > I wanted to connect and confirm on changes for this PR. > > https://github.com/apache/ignite/pull/6490 > > Do you think keeping the change for PlatformConfigurationUtils.java will > cause regression on the .NET component or C++ components? > > Jira : https://issues.apache.org/jira/browse/IGNITE-7285 > > Regards, > Saikat > > > > On Mon, Oct 14, 2019 at 3:14 AM Ivan Pavlukhin > wrote: > > > Hi Saikat, > > > > Sorry for delay. I will do my best to check it in the beginning of this > > week. > > > > сб, 12 окт. 2019 г. в 08:15, Saikat Maitra : > > > > > > Hello Ivan, > > > > > > I have updated the PR as per our discussion. > > > > > > Please review and share your feedback. > > > > > > Regards, > > > Saikat > > > > > > On Sun, Sep 1, 2019 at 3:20 PM Saikat Maitra > > > wrote: > > > > > > > Hi Ivan, > > > > > > > > I have taken care of review comments and also have shared a question > > for > > > > the application of default Query timeout value. > > > > > > > > Can you please review and share feedback? > > > > > > > > Regards, > > > > Saikat > > > > > > > > On Sat, Aug 24, 2019 at 7:22 PM Saikat Maitra < > saikat.mai...@gmail.com > > > > > > > wrote: > > > > > > > >> Hi Ivan, > > > >> > > > >> Thank you, I have shared my comments and have few questions related > to > > > >> the issue. > > > >> > > > >> Please take a look and share your thoughts. > > > >> > > > >> Regards, > > > >> Saikat > > > >> > > > >> On Tue, Aug 20, 2019 at 4:03 PM Павлухин Иван > > > >> wrote: > > > >> > > > >>> Hi Saikat, > > > >>> > > > >>> I left a comment in JIRA ticket [1]. Also, I invited Andrey to help > > > >>> with a further review. > > > >>> > > > >>> Andrey, could you please step in and continue the review? > > > >>> Unfortunately, for a couple of weeks I have limited access to my > > > >>> computer and cannot do a review in a timely manner. > > > >>> > > > >>> [1] https://issues.apache.org/jira/browse/IGNITE-7285 > > > >>> > > > >>> 2019-08-19 7:24 GMT+11:00, Saikat Maitra >: > > > >>> > Hi Ivan, > > > >>> > > > > >>> > I have updated the PR and made changes in IgniteH2Indexing for > > query > > > >>> > timeout so that default query timeout get used during query > > execution. > > > >>> > > > > >>> > Please take a look and let me know if this change looks good. > > > >>> > > > > >>> > I will update tests if the approach looks good. > > > >>> > > > > >>> > PR https://github.com/apache/ignite/pull/6490 > > > >>> > > > > >>> > Regards, > > > >>> > > > > >>> > Saikat > > > >>> > > > > >>> > On Sat, Aug 17, 2019 at 8:30 PM Saikat Maitra < > > saikat.mai...@gmail.com > > > >>> > > > > >>> > wrote: > > > >>> > > > > >>> >> Hi Ivan, Denis > > > >>> >> > > > >>> >> Thank you for your feedback, I am looking into the changes > needed > > for > > > >>> >> this > > > >>> >> issue. > > > >>> >> > > > >>> >> I am also looking into these configurations parameters > > > >>> >> > https://apacheignite.readme.io/v2.2/docs/configuration-parameters > > to > > > >>> see > > > >>> >> if there are similar attributes being used in SqlFieldsQuery > and > > > >>> >> SqlQuery. > > > >>> >> > > > >>> >> > > > >>> >> Regards, > > > >>> >> > > > >>> >> Saikat > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> >> On Thu, Aug 15, 2019 at 6:13 AM Павлухин Иван < > > vololo...@gmail.com> > > > >>> >> wrote: > > > >>> >> > > > >>> >>> Saikat, Denis, > > > >>> >>> > > > >>> >>> I left comments in the ticket [1]. > > > >>> >>> > > > >>> >>> [1] https://issues.apache.org/jira/browse/IGNITE-7285 > > > >>> >>> > > > >>> >>> вт, 13 авг. 2019 г. в 21:53, Denis Magda : > > > >>> >>> > > > > >>> >>> > Hi Saikat, > > > >>> >>> > > > > >>> >>> > Thanks for a quick turnaround! Ivan, could you please step in > > and > > > >>> do a > > > >>> >>> > review? > > > >>> >>> > > > > >>> >>> > - > > > >>> >>> > Denis > > > >>> >>> > > > > >>> >>> > > > > >>> >>> > On Sun, Aug 11, 2019 at 6:26 AM Saikat Maitra > > > >>> >>> > > > > >>> >>> > wrote: > > > >>> >>> > > > > >>> >>> > > Hi Denis, Ivan > > > >>> >>> > > > > > >>> >>> > > As discussed I have updated the PR and incorporated review > > > >>> comments. > > > >>> >>> > > > > > >>> >>> > > https://github.com/apache/ignite/pull/6490/files > > > >>> >>> > > > > > >>> >>> > > Please take a look and share your feedback. > > > >>> >>> > > > > > >>> >>> > > Regard, > > > >>> >>> > > Saikat > > > >>> >>> > > > > > >>> >>> > > > > > >>> >>> > > > > > >>> >>> > > On Sat, Aug 10, 2019 at 5:51 PM Saikat Maitra < > > > >>> >>> saikat.mai...@gmail.com> > > > >>> >>> > > wrote: > > > >>> >>> > > > > > >>> >>> > > > Hello Denis, Ivan > > > >>> >>> > > > > > > >>> >>> > > > Yes, I can take up the changes for IGNITE-7825. > > > >>> >>> > > > > > > >>> >>> > > > I had a doubt on the usage of the Default Query Timeout. > > > >>> >>> > > > > > > >>> >>> > > > I had raised the PR in an assumption that Default Query > > Timeout > > >
