[jira] [Created] (IGNITE-10810) [ML] Import models from MLeap

2018-12-25 Thread Anton Dmitriev (JIRA)
Anton Dmitriev created IGNITE-10810:
---

 Summary: [ML] Import models from MLeap
 Key: IGNITE-10810
 URL: https://issues.apache.org/jira/browse/IGNITE-10810
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Affects Versions: 2.8
Reporter: Anton Dmitriev
Assignee: Anton Dmitriev
 Fix For: 2.8


We want to have an ability to import models saved using MLeap library.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-10811) Add new TC build-config to test Service Grid new and old implementations

2018-12-25 Thread Vyacheslav Daradur (JIRA)
Vyacheslav Daradur created IGNITE-10811:
---

 Summary: Add new TC build-config to test Service Grid new and old 
implementations
 Key: IGNITE-10811
 URL: https://issues.apache.org/jira/browse/IGNITE-10811
 Project: Ignite
  Issue Type: Task
  Components: managed services
Reporter: Vyacheslav Daradur
Assignee: Vyacheslav Daradur
 Fix For: 2.8


It's necessary to add new build configuration on TeamCity (after merge 
IGNITE-9607) to have ability testing new and old implementations of the service 
processor.

In general, new build plan we should contain 2 test suites:
1) To place Service Grid related tests which *do not depend on* service 
processor's implementation (like classic unit tests) and *should be executed 
once*;
2) To place Service Grid related tests which *depend* on service processor's 
implementation and *should be executed twice* in the new mode and old mode 
(-DIGNITE_EVENT_DRIVEN_SERVICE_PROCESSOR_ENABLED=false)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Apache Ignite TeamCity Bot - muted tests

2018-12-25 Thread Andrey Mashenkov
Dmitry,

As far as I understand, @IgniteIgnore and fail("") works in similar way and
requires test to be muted on TC manually.
And we have to unmute these tests on TC right after fix will be merged to
master, isn't it?

@Ignore and Assume.that works different way comparing to previous two.
Junit just exclude such tests from run.
So, there is no need to mute\unmute them on TC at all, but...
Does TC treat such tests as muted ones and we can see them on "muted tests"
page?
This may be useful to track tests state if someone forget to remove @ignore
while fixing related ticket.


On Mon, Dec 17, 2018 at 5:54 PM Anton Vinogradov  wrote:

> >>  In this case, we will not need TeamCity mutes anymore.
> Sounds great!
>
> On Mon, Dec 17, 2018 at 5:23 PM Dmitrii Ryabov 
> wrote:
>
> > Dmitriy,
> >
> > with ignore annotations we will need to check forgotten tests
> > manually. Thats bad, but we will save some machine time in common
> > runs.
> > пн, 17 дек. 2018 г. в 17:06, Dmitriy Pavlov :
> > >
> > > Investigation parameters: Can leave as empty
> > > Unmute: Manual (don't use auto because of flaky tests)
> > > Text: https://issues.apache.org/jira/browse/IGNITE-N
> > >
> > > But anyway, I hope in the nearest future best way to mute test will be
> to
> > > ignore it using:
> > > @IgniteIgnore
> > > or @Ignore
> > > or Assume.that()
> > >
> > > In this case, we will not need TeamCity mutes anymore.
> > >
> > > I'm not sure which way of ignoring will be best.
> > >
> > > пн, 17 дек. 2018 г. в 17:02, Anton Vinogradov :
> > >
> > > > Dmitrii,
> > > >
> > > > My question was mostly about how to mute test properly.
> > > >
> > > >
> > > > On Mon, Dec 17, 2018 at 4:27 PM Dmitrii Ryabov <
> somefire...@gmail.com>
> > > > wrote:
> > > >
> > > > > Anton,
> > > > >
> > > > > In the "Mute" column you can find link to the test on TeamCity. On
> > > > > this page you can see test details like run history and current
> > mutes.
> > > > > Here you should be able to unmute existing mutes.
> > > > > пн, 17 дек. 2018 г. в 16:21, Anton Vinogradov :
> > > > > >
> > > > > > Great feature!
> > > > > >
> > > > > > Could you please explain how to have a deal with it?
> > > > > > How should I mite/unmute tests for now?
> > > > > >
> > > > > > On Fri, Dec 14, 2018 at 6:50 PM Dmitriy Pavlov <
> dpav...@apache.org
> > >
> > > > > wrote:
> > > > > >
> > > > > > > We have IgniteIgnore, so we can use it. It seems behavior is
> the
> > > > same.
> > > > > We
> > > > > > > need just to understand in which case we can get issue ID from
> TC
> > > > REST
> > > > > (for
> > > > > > > the Bot).
> > > > > > >
> > > > > > > пт, 14 дек. 2018 г. в 17:51, Vyacheslav Daradur <
> > daradu...@gmail.com
> > > > >:
> > > > > > >
> > > > > > > > Dmitry Pavlov,
> > > > > > > >
> > > > > > > > As far as I know, we are able to use following construction
> to
> > > > ignore
> > > > > > > test:
> > > > > > > > Assume.assumeTrue("link-to-issue", false);
> > > > > > > >
> > > > > > > > On Fri, Dec 14, 2018 at 5:44 PM Dmitriy Pavlov <
> > dpav...@apache.org
> > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Thank you, Dmitrii,
> > > > > > > > >
> > > > > > > > >  I've started unmuting tests starting from yesterday.
> > > > > > > > >
> > > > > > > > > Now muted test count was decreased from 3540 to 3408
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&tab=mutedProblems&branch_IgniteTests24Java8=%3Cdefault%3E
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Probably we could automate this activity by auto-unmute,
> but
> > > > > hopefully,
> > > > > > > > > JUnit4 & @Ignore opportunity will come earlier.
> > > > > > > > >
> > > > > > > > > Sincerely,
> > > > > > > > > Dmitriy Pavlov
> > > > > > > > >
> > > > > > > > > пт, 14 дек. 2018 г. в 17:24, Dmitrii Ryabov <
> > > > somefire...@gmail.com
> > > > > >:
> > > > > > > > >
> > > > > > > > > > Hello, Igniters!
> > > > > > > > > >
> > > > > > > > > > Our Bot got new functionality - it shows tests muted on
> > > > TeamCity
> > > > > and
> > > > > > > > their
> > > > > > > > > > ticket status [1].
> > > > > > > > > >
> > > > > > > > > > So, we can see forgotten mutes for resolved tickets now.
> > > > > > > > > >
> > > > > > > > > > Igniters with rights to unmute tests on TC - please,
> check
> > this
> > > > > page
> > > > > > > > > > periodically for forgotten mutes.
> > > > > > > > > >
> > > > > > > > > > [1] https://mtcga.gridgain.com/mutes.html
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Best Regards, Vyacheslav D.
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> >
>


-- 
Best regards,
Andrey V. Mashenkov


Re: [MTCGA]: new failures in builds [2635316] needs to be handled

2018-12-25 Thread Vyacheslav Daradur
I prepared PR [1] to fix the issue, the test became green.

Anton, could you assist with the merge, please?

[1] https://github.com/apache/ignite/pull/5739
[2] https://ci.ignite.apache.org/viewQueued.html?itemId=2641732

On Tue, Dec 25, 2018 at 10:53 AM Vyacheslav Daradur  wrote:
>
> This is my fault, introduced within
> https://issues.apache.org/jira/browse/IGNITE-10715
>
> I have no idea why it had not been failed during testing on TC.
>
> I will prepare the fix quickly.
>
> On Tue, Dec 25, 2018 at 2:37 AM  wrote:
> >
> > Hi Igniters,
> >
> >  I've detected some new issue on TeamCity to be handled. You are more than 
> > welcomed to help.
> >
> >  If your changes can lead to this failure(s): We're grateful that you were 
> > a volunteer to make the contribution to this project, but things change and 
> > you may no longer be able to finalize your contribution.
> >  Could you respond to this email and indicate if you wish to continue and 
> > fix test failures or step down and some committer may revert you commit.
> >
> >  *New stable failure of a flaky test in master 
> > IgniteClientConnectTest.testClientConnectToBigTopology 
> > https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=9021926721143358489&branch=%3Cdefault%3E&tab=testDetails
> >  Changes may lead to failure were done by
> >  - ilya.kasnacheev 
> > https://ci.ignite.apache.org/viewModification.html?modId=850683
> >  - daradurvs 
> > https://ci.ignite.apache.org/viewModification.html?modId=850663
> >  - antonovsergey93 
> > https://ci.ignite.apache.org/viewModification.html?modId=850675
> >
> >  - Here's a reminder of what contributors were agreed to do 
> > https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
> >  - Should you have any questions please contact 
> > dev@ignite.apache.org
> >
> > Best Regards,
> > Apache Ignite TeamCity Bot
> > https://github.com/apache/ignite-teamcity-bot
> > Notification generated at 02:36:58 25-12-2018
>
>
>
> --
> Best Regards, Vyacheslav D.



-- 
Best Regards, Vyacheslav D.


Re: [MTCGA]: new failures in builds [2634856] needs to be handled

2018-12-25 Thread Vyacheslav Daradur
The test has been fixed within
https://issues.apache.org/jira/browse/IGNITE-10739

Thanks to Oleg Ignatenko!

On Tue, Dec 25, 2018 at 7:37 AM Vyacheslav Daradur  wrote:
>
> The failure has been introduced within task:
> https://issues.apache.org/jira/browse/IGNITE-10671 Double
> initialization of segmentAware and FileArchiver lead to race breaking
> file compression.
>
> The cause is a missed '@Test' annotation.
>
> Here is the patch to fix [1], TC test is green [2].
>
> [1] https://github.com/apache/ignite/pull/5737
> [2] https://ci.ignite.apache.org/viewLog.html?buildId=2638777
>
>
> On Mon, Dec 24, 2018 at 8:52 PM  wrote:
> >
> > Hi Igniters,
> >
> >  I've detected some new issue on TeamCity to be handled. You are more than 
> > welcomed to help.
> >
> >  If your changes can lead to this failure(s): We're grateful that you were 
> > a volunteer to make the contribution to this project, but things change and 
> > you may no longer be able to finalize your contribution.
> >  Could you respond to this email and indicate if you wish to continue and 
> > fix test failures or step down and some committer may revert you commit.
> >
> >  *Recently contributed test failed in master 
> > WalCompactionSwitchOnTest.initializationError 
> > https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=6257556032945731756&branch=%3Cdefault%3E&tab=testDetails
> >  Changes may lead to failure were done by
> >  - pvoronkin 
> > https://ci.ignite.apache.org/viewModification.html?modId=850576
> >
> >  - Here's a reminder of what contributors were agreed to do 
> > https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
> >  - Should you have any questions please contact 
> > dev@ignite.apache.org
> >
> > Best Regards,
> > Apache Ignite TeamCity Bot
> > https://github.com/apache/ignite-teamcity-bot
> > Notification generated at 20:51:57 24-12-2018
>
>
>
> --
> Best Regards, Vyacheslav D.



-- 
Best Regards, Vyacheslav D.


[jira] [Created] (IGNITE-10812) SQL: split classes responsible for distributed joins

2018-12-25 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-10812:


 Summary: SQL: split classes responsible for distributed joins
 Key: IGNITE-10812
 URL: https://issues.apache.org/jira/browse/IGNITE-10812
 Project: Ignite
  Issue Type: Task
  Components: sql
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov
 Fix For: 2.8


This is just a refactoring task to create more precise hierarchy of classes 
responsible for distributed joins.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Apache Ignite TeamCity Bot - muted tests

2018-12-25 Thread oignatenko
Hi Andrey,

As far as I know tests annotated by @Ignore can be conveniently tracked on
Teamcity but this requires such tests to be included into JUnit 4 test suite
(org.junit.runners.Suite).

Above requirement currently isn't met by many Ignite tests which are
included in JUnit 3 suites (junit.framework.TestSuite). This means that test
cases annotated @Ignore simply are hidden when present in such old-fashioned
suites.

We are currently working on migrating Junit 3 suites to JUnit 4. If you are
interested, work breakdown and progress can be found in JIRA:
- https://issues.apache.org/jira/browse/IGNITE-10762

-

With regards to Assume API, it seems more complicated. I have recently read
that tests that are conditionally skipped using this API are reported by
JUnit 4 test framework as passed. This is not very convenient.

I think I have seen somewhere mentioned that proper reporting of
conditionally skipped tests can be achieved only with JUnit 5. But I didn't
dig deeper there because we haven't yet decided whether it is worth it for
Ignite to move to JUnit 5
(https://issues.apache.org/jira/browse/IGNITE-10180).

Given above, I believe that the proper way to mute tests is currently with
@Ignore and Assume API. It's not 100% ideal but overall looks better than
the old one.

regards, Oleg


Andrew Mashenkov wrote
> Dmitry,
> 
> As far as I understand, @IgniteIgnore and fail("") works in similar way
> and
> requires test to be muted on TC manually.
> And we have to unmute these tests on TC right after fix will be merged to
> master, isn't it?
> 
> @Ignore and Assume.that works different way comparing to previous two.
> Junit just exclude such tests from run.
> So, there is no need to mute\unmute them on TC at all, but...
> Does TC treat such tests as muted ones and we can see them on "muted
> tests"
> page?
> This may be useful to track tests state if someone forget to remove
> @ignore
> while fixing related ticket.
> 
> 
> On Mon, Dec 17, 2018 at 5:54 PM Anton Vinogradov <

> av@

> > wrote:
> 
>> >>  In this case, we will not need TeamCity mutes anymore.
>> Sounds great!
>>
>> On Mon, Dec 17, 2018 at 5:23 PM Dmitrii Ryabov <

> somefireone@

> >
>> wrote:
>>
>> > Dmitriy,
>> >
>> > with ignore annotations we will need to check forgotten tests
>> > manually. Thats bad, but we will save some machine time in common
>> > runs.
>> > пн, 17 дек. 2018 г. в 17:06, Dmitriy Pavlov <

> dpavlov@

> >:
>> > >
>> > > Investigation parameters: Can leave as empty
>> > > Unmute: Manual (don't use auto because of flaky tests)
>> > > Text: https://issues.apache.org/jira/browse/IGNITE-N
>> > >
>> > > But anyway, I hope in the nearest future best way to mute test will
>> be
>> to
>> > > ignore it using:
>> > > @IgniteIgnore
>> > > or @Ignore
>> > > or Assume.that()
>> > >
>> > > In this case, we will not need TeamCity mutes anymore.
>> > >
>> > > I'm not sure which way of ignoring will be best.
>> > >
>> > > пн, 17 дек. 2018 г. в 17:02, Anton Vinogradov <

> av@

> >:
>> > >
>> > > > Dmitrii,
>> > > >
>> > > > My question was mostly about how to mute test properly.
>> > > >
>> > > >
>> > > > On Mon, Dec 17, 2018 at 4:27 PM Dmitrii Ryabov <
>> 

> somefireone@

>>
>> > > > wrote:
>> > > >
>> > > > > Anton,
>> > > > >
>> > > > > In the "Mute" column you can find link to the test on TeamCity.
>> On
>> > > > > this page you can see test details like run history and current
>> > mutes.
>> > > > > Here you should be able to unmute existing mutes.
>> > > > > пн, 17 дек. 2018 г. в 16:21, Anton Vinogradov <

> av@

> >:
>> > > > > >
>> > > > > > Great feature!
>> > > > > >
>> > > > > > Could you please explain how to have a deal with it?
>> > > > > > How should I mite/unmute tests for now?
>> > > > > >
>> > > > > > On Fri, Dec 14, 2018 at 6:50 PM Dmitriy Pavlov <
>> 

> dpavlov@

>> > >
>> > > > > wrote:
>> > > > > >
>> > > > > > > We have IgniteIgnore, so we can use it. It seems behavior is
>> the
>> > > > same.
>> > > > > We
>> > > > > > > need just to understand in which case we can get issue ID
>> from
>> TC
>> > > > REST
>> > > > > (for
>> > > > > > > the Bot).
>> > > > > > >
>> > > > > > > пт, 14 дек. 2018 г. в 17:51, Vyacheslav Daradur <
>> > 

> daradurvs@

>> > > > >:
>> > > > > > >
>> > > > > > > > Dmitry Pavlov,
>> > > > > > > >
>> > > > > > > > As far as I know, we are able to use following construction
>> to
>> > > > ignore
>> > > > > > > test:
>> > > > > > > > Assume.assumeTrue("link-to-issue", false);
>> > > > > > > >
>> > > > > > > > On Fri, Dec 14, 2018 at 5:44 PM Dmitriy Pavlov <
>> > 

> dpavlov@

>> > > > >
>> > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > Thank you, Dmitrii,
>> > > > > > > > >
>> > > > > > > > >  I've started unmuting tests starting from yesterday.
>> > > > > > > > >
>> > > > > > > > > Now muted test count was decreased from 3540 to 3408
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > >
>> > > >
>> >
>> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Jav

[jira] [Created] (IGNITE-10813) Run CheckpointReadLockFailureTest with JUnit4 runner

2018-12-25 Thread Andrey Kuznetsov (JIRA)
Andrey Kuznetsov created IGNITE-10813:
-

 Summary: Run CheckpointReadLockFailureTest with JUnit4 runner
 Key: IGNITE-10813
 URL: https://issues.apache.org/jira/browse/IGNITE-10813
 Project: Ignite
  Issue Type: Bug
Reporter: Andrey Kuznetsov
Assignee: Andrey Kuznetsov


The test fails on TeamCity. Should be run in JUnit4 manner.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Apache Ignite TeamCity Bot - muted tests

2018-12-25 Thread Andrey Mashenkov
Oleg,

Thanks for clarification.

On Tue, Dec 25, 2018 at 2:24 PM oignatenko  wrote:

> Hi Andrey,
>
> As far as I know tests annotated by @Ignore can be conveniently tracked on
> Teamcity but this requires such tests to be included into JUnit 4 test
> suite
> (org.junit.runners.Suite).
>
> Above requirement currently isn't met by many Ignite tests which are
> included in JUnit 3 suites (junit.framework.TestSuite). This means that
> test
> cases annotated @Ignore simply are hidden when present in such
> old-fashioned
> suites.
>
> We are currently working on migrating Junit 3 suites to JUnit 4. If you are
> interested, work breakdown and progress can be found in JIRA:
> - https://issues.apache.org/jira/browse/IGNITE-10762
>
> -
>
> With regards to Assume API, it seems more complicated. I have recently read
> that tests that are conditionally skipped using this API are reported by
> JUnit 4 test framework as passed. This is not very convenient.
>
> I think I have seen somewhere mentioned that proper reporting of
> conditionally skipped tests can be achieved only with JUnit 5. But I didn't
> dig deeper there because we haven't yet decided whether it is worth it for
> Ignite to move to JUnit 5
> (https://issues.apache.org/jira/browse/IGNITE-10180).
>
> Given above, I believe that the proper way to mute tests is currently with
> @Ignore and Assume API. It's not 100% ideal but overall looks better than
> the old one.
>
> regards, Oleg
>
>
> Andrew Mashenkov wrote
> > Dmitry,
> >
> > As far as I understand, @IgniteIgnore and fail("") works in similar way
> > and
> > requires test to be muted on TC manually.
> > And we have to unmute these tests on TC right after fix will be merged to
> > master, isn't it?
> >
> > @Ignore and Assume.that works different way comparing to previous two.
> > Junit just exclude such tests from run.
> > So, there is no need to mute\unmute them on TC at all, but...
> > Does TC treat such tests as muted ones and we can see them on "muted
> > tests"
> > page?
> > This may be useful to track tests state if someone forget to remove
> > @ignore
> > while fixing related ticket.
> >
> >
> > On Mon, Dec 17, 2018 at 5:54 PM Anton Vinogradov <
>
> > av@
>
> > > wrote:
> >
> >> >>  In this case, we will not need TeamCity mutes anymore.
> >> Sounds great!
> >>
> >> On Mon, Dec 17, 2018 at 5:23 PM Dmitrii Ryabov <
>
> > somefireone@
>
> > >
> >> wrote:
> >>
> >> > Dmitriy,
> >> >
> >> > with ignore annotations we will need to check forgotten tests
> >> > manually. Thats bad, but we will save some machine time in common
> >> > runs.
> >> > пн, 17 дек. 2018 г. в 17:06, Dmitriy Pavlov <
>
> > dpavlov@
>
> > >:
> >> > >
> >> > > Investigation parameters: Can leave as empty
> >> > > Unmute: Manual (don't use auto because of flaky tests)
> >> > > Text: https://issues.apache.org/jira/browse/IGNITE-N
> >> > >
> >> > > But anyway, I hope in the nearest future best way to mute test will
> >> be
> >> to
> >> > > ignore it using:
> >> > > @IgniteIgnore
> >> > > or @Ignore
> >> > > or Assume.that()
> >> > >
> >> > > In this case, we will not need TeamCity mutes anymore.
> >> > >
> >> > > I'm not sure which way of ignoring will be best.
> >> > >
> >> > > пн, 17 дек. 2018 г. в 17:02, Anton Vinogradov <
>
> > av@
>
> > >:
> >> > >
> >> > > > Dmitrii,
> >> > > >
> >> > > > My question was mostly about how to mute test properly.
> >> > > >
> >> > > >
> >> > > > On Mon, Dec 17, 2018 at 4:27 PM Dmitrii Ryabov <
> >>
>
> > somefireone@
>
> >>
> >> > > > wrote:
> >> > > >
> >> > > > > Anton,
> >> > > > >
> >> > > > > In the "Mute" column you can find link to the test on TeamCity.
> >> On
> >> > > > > this page you can see test details like run history and current
> >> > mutes.
> >> > > > > Here you should be able to unmute existing mutes.
> >> > > > > пн, 17 дек. 2018 г. в 16:21, Anton Vinogradov <
>
> > av@
>
> > >:
> >> > > > > >
> >> > > > > > Great feature!
> >> > > > > >
> >> > > > > > Could you please explain how to have a deal with it?
> >> > > > > > How should I mite/unmute tests for now?
> >> > > > > >
> >> > > > > > On Fri, Dec 14, 2018 at 6:50 PM Dmitriy Pavlov <
> >>
>
> > dpavlov@
>
> >> > >
> >> > > > > wrote:
> >> > > > > >
> >> > > > > > > We have IgniteIgnore, so we can use it. It seems behavior is
> >> the
> >> > > > same.
> >> > > > > We
> >> > > > > > > need just to understand in which case we can get issue ID
> >> from
> >> TC
> >> > > > REST
> >> > > > > (for
> >> > > > > > > the Bot).
> >> > > > > > >
> >> > > > > > > пт, 14 дек. 2018 г. в 17:51, Vyacheslav Daradur <
> >> >
>
> > daradurvs@
>
> >> > > > >:
> >> > > > > > >
> >> > > > > > > > Dmitry Pavlov,
> >> > > > > > > >
> >> > > > > > > > As far as I know, we are able to use following
> construction
> >> to
> >> > > > ignore
> >> > > > > > > test:
> >> > > > > > > > Assume.assumeTrue("link-to-issue", false);
> >> > > > > > > >
> >> > > > > > > > On Fri, Dec 14, 2018 at 5:44 PM Dmitriy Pavlov <
> >> >
>
> > dpavlov@
>
> >> > > > >
> >> 

Re: [MTCGA]: new failures in builds [2635316] needs to be handled

2018-12-25 Thread Anton Vinogradov
Slava,

please fill the issue with fix and tcbot visa.

On Tue, Dec 25, 2018 at 12:18 PM Vyacheslav Daradur 
wrote:

> I prepared PR [1] to fix the issue, the test became green.
>
> Anton, could you assist with the merge, please?
>
> [1] https://github.com/apache/ignite/pull/5739
> [2] https://ci.ignite.apache.org/viewQueued.html?itemId=2641732
>
> On Tue, Dec 25, 2018 at 10:53 AM Vyacheslav Daradur 
> wrote:
> >
> > This is my fault, introduced within
> > https://issues.apache.org/jira/browse/IGNITE-10715
> >
> > I have no idea why it had not been failed during testing on TC.
> >
> > I will prepare the fix quickly.
> >
> > On Tue, Dec 25, 2018 at 2:37 AM  wrote:
> > >
> > > Hi Igniters,
> > >
> > >  I've detected some new issue on TeamCity to be handled. You are more
> than welcomed to help.
> > >
> > >  If your changes can lead to this failure(s): We're grateful that you
> were a volunteer to make the contribution to this project, but things
> change and you may no longer be able to finalize your contribution.
> > >  Could you respond to this email and indicate if you wish to continue
> and fix test failures or step down and some committer may revert you commit.
> > >
> > >  *New stable failure of a flaky test in master
> IgniteClientConnectTest.testClientConnectToBigTopology
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=9021926721143358489&branch=%3Cdefault%3E&tab=testDetails
> > >  Changes may lead to failure were done by
> > >  - ilya.kasnacheev
> https://ci.ignite.apache.org/viewModification.html?modId=850683
> > >  - daradurvs
> https://ci.ignite.apache.org/viewModification.html?modId=850663
> > >  - antonovsergey93
> https://ci.ignite.apache.org/viewModification.html?modId=850675
> > >
> > >  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
> > >  - Should you have any questions please contact
> dev@ignite.apache.org
> > >
> > > Best Regards,
> > > Apache Ignite TeamCity Bot
> > > https://github.com/apache/ignite-teamcity-bot
> > > Notification generated at 02:36:58 25-12-2018
> >
> >
> >
> > --
> > Best Regards, Vyacheslav D.
>
>
>
> --
> Best Regards, Vyacheslav D.
>


[jira] [Created] (IGNITE-10814) Exceptions thrown in InitNewCoordinatorFuture.init() are ignored

2018-12-25 Thread Anton Kurbanov (JIRA)
Anton Kurbanov created IGNITE-10814:
---

 Summary: Exceptions thrown in InitNewCoordinatorFuture.init() are 
ignored
 Key: IGNITE-10814
 URL: https://issues.apache.org/jira/browse/IGNITE-10814
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Anton Kurbanov


Exceptions thrown by InitNewCoordinatorFuture.init() called from 
GridDhtPartitionsExchangeFuture.onNodeLeft is ignored silently and may result 
in cluster hang without cause seen in logs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-10815) NullPointerException during InitNewCoordinatorFuture.init() leads to cluster hang

2018-12-25 Thread Anton Kurbanov (JIRA)
Anton Kurbanov created IGNITE-10815:
---

 Summary: NullPointerException during 
InitNewCoordinatorFuture.init() leads to cluster hang
 Key: IGNITE-10815
 URL: https://issues.apache.org/jira/browse/IGNITE-10815
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Anton Kurbanov


Possible scenario to reproduce:

1. Force few consecutive exchange merges and finish.

2. Trigger exchange.

3. Shutdown coordinator node before sending/receiving full partitions message.

 

Stacktrace:

2018-12-24 15:54:02,664 [sys-#48%gg%] ERROR 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture
 - Failed to init new coordinator future: bd74f7ed-6984-4f78-9941-480df673ab77

java.lang.NullPointerException: null
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.events(GridDhtPartitionsExchangeFuture.java:534)
 ~[ignite-core-2.4.13.b4.jar:2.4.13.b4]
 at 
org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager$18.applyx(CacheAffinitySharedManager.java:1790)
 ~[ignite-core-2.4.13.b4.jar:2.4.13.b4]
 at 
org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager$18.applyx(CacheAffinitySharedManager.java:1738)
 ~[ignite-core-2.4.13.b4.jar:2.4.13.b4]
 at 
org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.forAllRegisteredCacheGroups(CacheAffinitySharedManager.java:1107)
 ~[ignite-core-2.4.13.b4.jar:2.4.13.b4]
 at 
org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.initCoordinatorCaches(CacheAffinitySharedManager.java:1738)
 ~[ignite-core-2.4.13.b4.jar:2.4.13.b4]
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.InitNewCoordinatorFuture.init(InitNewCoordinatorFuture.java:104)
 ~[ignite-core-2.4.13.b4.jar:2.4.13.b4]
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$8$1.call(GridDhtPartitionsExchangeFuture.java:3439)
 [ignite-core-2.4.13.b4.jar:2.4.13.b4]
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$8$1.call(GridDhtPartitionsExchangeFuture.java:3435)
 [ignite-core-2.4.13.b4.jar:2.4.13.b4]
 at 
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6720)
 [ignite-core-2.4.13.b4.jar:2.4.13.b4]
 at 
org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:967)
 [ignite-core-2.4.13.b4.jar:2.4.13.b4]
 at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) 
[ignite-core-2.4.13.b4.jar:2.4.13.b4]
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
[?:1.8.0_171]
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
[?:1.8.0_171]
 at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: [MTCGA]: new failures in builds [2636079] needs to be handled

2018-12-25 Thread Andrey Kuznetsov
Separate issue has been created to address this: IGNITE-10813.

Best regards,
Andrey Kuznetsov.

пн, 24 дек. 2018, 19:37 dpavlov.ta...@gmail.com:

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *Recently contributed test failed in master
> CheckpointReadLockFailureTest.initializationError
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-5099637610936041665&branch=%3Cdefault%3E&tab=testDetails
>  Changes may lead to failure were done by
>  - stkuzma
> https://ci.ignite.apache.org/viewModification.html?modId=850850
>
>  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact
> dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 19:36:56 24-12-2018
>


[jira] [Created] (IGNITE-10816) MVCC: create benchmarks for bulk update operations.

2018-12-25 Thread Andrew Mashenkov (JIRA)
Andrew Mashenkov created IGNITE-10816:
-

 Summary: MVCC: create benchmarks for bulk update operations.
 Key: IGNITE-10816
 URL: https://issues.apache.org/jira/browse/IGNITE-10816
 Project: Ignite
  Issue Type: Bug
  Components: mvcc, sql, yardstick
Reporter: Andrew Mashenkov


For now, we have no any benchmark for bulk update operations (putAll or 
multiple SQL inserts within single transaction)  that can be run in mvcc mode.

1. We should adapt existed PutAllTx benchmarks as they can failed due to write 
conflicts.
2. We should add SQL benchmarks for batched insert\update operations within 
same Tx similar to existed putAll benches.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Replace Cron4J with Quartz for ignite-schedule module.

2018-12-25 Thread Alexey Kuznetsov
Hi, Sergey!

I think we should keep compatibility as much as possible for Ignite 2.x.
And we can do breaking changes in Ignite 3.x

What do you think?


On Mon, Dec 24, 2018 at 11:58 PM Sergey  wrote:

> HI, Igniters!
>
> I've updated and rebased implementation to master branch and made some
> fixes.
> Also I have a question regarding current implementation.
>
> As I found Cron4J source code this implementation checks schedule every
> minute (seconds not supported) but spawns a thread for every task which
> scheduling pattern matches the current time. There no any limits to the
> number of tasks launched silmultaneously.
>
>  New implementation is based on
> org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler
> with its default parameters currently, i.e thread pool size is 1.
>
> Could you advise me, do we need to add some system property or introduce
> some attribute to IgniteConfiguration to configure Scheduler thread pool
> size? And what should be default value?
>
>
> Best regards,
> Sergey Kosarev.
>
>
> чт, 10 мая 2018 г. в 20:23, Dmitry Pavlov :
>
> > Hi Anton,
> >
> > Thank you for joining and review.
> > I hope all proposals will be applied.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пт, 4 мая 2018 г. в 16:04, Anton Vinogradov :
> >
> > > Folks,
> > >
> > > How can it be at PATCH AVAILABLE since *none* of my latest comments
> (made
> > > Feb 8) are resolved at Upsource?
> > > Changed state to IP.
> > >
> > > пн, 23 апр. 2018 г. в 20:00, Dmitry Pavlov :
> > >
> > > > Hi Andrey,
> > > >
> > > > Could you please pick up review?
> > > >
> > > > Sincerely,
> > > > Dmitriy Pavlov
> > > >
> > > > пн, 23 апр. 2018 г. в 17:39, Dmitriy Setrakyan <
> dsetrak...@apache.org
> > >:
> > > >
> > > > > Dmitriy, who is a good candidate within the community to review
> this
> > > > > ticket?
> > > > >
> > > > > On Mon, Apr 23, 2018 at 6:10 AM, Dmitry Pavlov <
> > dpavlov@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Igniters,
> > > > > >
> > > > > > it seems ticket
> https://issues.apache.org/jira/browse/IGNITE-5565
> > is
> > > > > still
> > > > > > in PA state. What are our next steps?
> > > > > >
> > > > > > Who did review of this patch?
> > > > > >
> > > > > > Sincerely,
> > > > > > Dmitriy Pavlov
> > > > > >
> > > > > > ср, 28 июн. 2017 г. в 1:40, Denis Magda :
> > > > > >
> > > > > > > Yakov,
> > > > > > >
> > > > > > > No, the mentioned discussion didn’t turn into a JIRA ticket.
> > > > > > >
> > > > > > > Alex K., please follow to some thoughts from there and wrap
> them
> > up
> > > > in
> > > > > a
> > > > > > > form of the ticket.
> > > > > > >
> > > > > > > —
> > > > > > > Denis
> > > > > > >
> > > > > > > > On Jun 26, 2017, at 2:58 AM, Yakov Zhdanov <
> > yzhda...@apache.org>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > Guys, I remember we discussed this some time ago.
> > > > > > > >
> > > > > > > >
> > > > > > > http://apache-ignite-developers.2346864.n4.nabble.
> > > > > > com/Tasks-Scheduling-and-Chaining-td14293.html
> > > > > > > >
> > > > > > > > Denis, do you have any ticket or SoW?
> > > > > > > >
> > > > > > > > --Yakov
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>


-- 
Alexey Kuznetsov


[jira] [Created] (IGNITE-10817) Service Grid: Introduce diagnostic tool to dump pending deployment tasks

2018-12-25 Thread Vyacheslav Daradur (JIRA)
Vyacheslav Daradur created IGNITE-10817:
---

 Summary: Service Grid: Introduce diagnostic tool to dump pending 
deployment tasks
 Key: IGNITE-10817
 URL: https://issues.apache.org/jira/browse/IGNITE-10817
 Project: Ignite
  Issue Type: Task
  Components: managed services
Reporter: Vyacheslav Daradur
Assignee: Vyacheslav Daradur
 Fix For: 2.8


It's necessary to introduce some kind of diagnostic tools to dump service 
deployments task which are pending in the queue in a log.

If possible, existing PME diagnostic mechanics should be reused.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Replace Cron4J with Quartz for ignite-schedule module.

2018-12-25 Thread Ilya Kasnacheev
Hello!

I have started reviewing your pull request.
I will expect that scheduled tasks are executed on Public pool. Is it
possible that tasks are launched on Public pool? If Spring Scheduler
insists on its own thread pool, we can have single-thread pool which will
execute put of tasks to public pool and immediately return. Is it possible
to do that?

Regards,
-- 
Ilya Kasnacheev


вт, 25 дек. 2018 г. в 17:46, Alexey Kuznetsov :

> Hi, Sergey!
>
> I think we should keep compatibility as much as possible for Ignite 2.x.
> And we can do breaking changes in Ignite 3.x
>
> What do you think?
>
>
> On Mon, Dec 24, 2018 at 11:58 PM Sergey  wrote:
>
> > HI, Igniters!
> >
> > I've updated and rebased implementation to master branch and made some
> > fixes.
> > Also I have a question regarding current implementation.
> >
> > As I found Cron4J source code this implementation checks schedule every
> > minute (seconds not supported) but spawns a thread for every task which
> > scheduling pattern matches the current time. There no any limits to the
> > number of tasks launched silmultaneously.
> >
> >  New implementation is based on
> > org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler
> > with its default parameters currently, i.e thread pool size is 1.
> >
> > Could you advise me, do we need to add some system property or introduce
> > some attribute to IgniteConfiguration to configure Scheduler thread pool
> > size? And what should be default value?
> >
> >
> > Best regards,
> > Sergey Kosarev.
> >
> >
> > чт, 10 мая 2018 г. в 20:23, Dmitry Pavlov :
> >
> > > Hi Anton,
> > >
> > > Thank you for joining and review.
> > > I hope all proposals will be applied.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > пт, 4 мая 2018 г. в 16:04, Anton Vinogradov :
> > >
> > > > Folks,
> > > >
> > > > How can it be at PATCH AVAILABLE since *none* of my latest comments
> > (made
> > > > Feb 8) are resolved at Upsource?
> > > > Changed state to IP.
> > > >
> > > > пн, 23 апр. 2018 г. в 20:00, Dmitry Pavlov :
> > > >
> > > > > Hi Andrey,
> > > > >
> > > > > Could you please pick up review?
> > > > >
> > > > > Sincerely,
> > > > > Dmitriy Pavlov
> > > > >
> > > > > пн, 23 апр. 2018 г. в 17:39, Dmitriy Setrakyan <
> > dsetrak...@apache.org
> > > >:
> > > > >
> > > > > > Dmitriy, who is a good candidate within the community to review
> > this
> > > > > > ticket?
> > > > > >
> > > > > > On Mon, Apr 23, 2018 at 6:10 AM, Dmitry Pavlov <
> > > dpavlov@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Igniters,
> > > > > > >
> > > > > > > it seems ticket
> > https://issues.apache.org/jira/browse/IGNITE-5565
> > > is
> > > > > > still
> > > > > > > in PA state. What are our next steps?
> > > > > > >
> > > > > > > Who did review of this patch?
> > > > > > >
> > > > > > > Sincerely,
> > > > > > > Dmitriy Pavlov
> > > > > > >
> > > > > > > ср, 28 июн. 2017 г. в 1:40, Denis Magda :
> > > > > > >
> > > > > > > > Yakov,
> > > > > > > >
> > > > > > > > No, the mentioned discussion didn’t turn into a JIRA ticket.
> > > > > > > >
> > > > > > > > Alex K., please follow to some thoughts from there and wrap
> > them
> > > up
> > > > > in
> > > > > > a
> > > > > > > > form of the ticket.
> > > > > > > >
> > > > > > > > —
> > > > > > > > Denis
> > > > > > > >
> > > > > > > > > On Jun 26, 2017, at 2:58 AM, Yakov Zhdanov <
> > > yzhda...@apache.org>
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Guys, I remember we discussed this some time ago.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > http://apache-ignite-developers.2346864.n4.nabble.
> > > > > > > com/Tasks-Scheduling-and-Chaining-td14293.html
> > > > > > > > >
> > > > > > > > > Denis, do you have any ticket or SoW?
> > > > > > > > >
> > > > > > > > > --Yakov
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
>
> --
> Alexey Kuznetsov
>


Re: Request for review : IGNITE-3303 Apache Flink Integration - Flink source

2018-12-25 Thread Dmitriy Pavlov
Hi Saikat,

I'm going to slightly change the code before commit.

First of all, I will add more code style and abbreviation rules related
changes.
Second of all, I add pom.xml variables usages where possible.
And last but not least, I'm going to change the implementation of
IgniteSource.

AFAIK drainTo() method is not blocking so IgniteSource will be continuously
spinning if there are no events coming from a cache. So I'm going to commit
this variant:

while (isRunning) {
// block here for some time if there is no events from source
CacheEvent firstEvt = evtBuf.poll(1, TimeUnit.SECONDS);

if (firstEvt != null)
evts.add(firstEvt);

if (evtBuf.drainTo(evts, evtBatchSize) > 0) {
synchronized (ctx.getCheckpointLock()) {
for (CacheEvent evt : evts)
ctx.collect(evt);

evts.clear();
}
}
}


I hope you don't mind. This option allows us both to stop and does not
spin in the while.


Sincerely,

Dmitriy Pavlov


сб, 13 окт. 2018 г. в 20:37, Saikat Maitra :

> Hi Alex ,
>
> Can you please review and let me know if the PR looks good to merge.
>
> I have completed the requested changes.
>
> Regards,
> Saikat
>
> On Mon, Oct 1, 2018 at 9:24 PM Saikat Maitra 
> wrote:
>
> > Thank you everyone for reviewing the changes. As discussed I have removed
> > the FlinkIgniteSourceSelfExample and also verified that
> FlinkIgniteSourceSelfTestSuite
> > is part of Ignite Streamers in Team City.
> >
> > Please let me know if any further changes are required.
> >
> > Alex , will you please review and let me know if the PR looks good?
> >
> > Regards,
> > Saikat
> >
> > On Mon, Oct 1, 2018 at 11:58 AM Nikolay Izhikov 
> > wrote:
> >
> >> Hello, Andrey.
> >>
> >> Yes, I know it.
> >> I've looked at the PR befor mailing :)
> >>
> >> Do you think we can include this improvement to the 2.7 release?
> >> Do you have time to assist Saikat with TC setup and other tasks?
> >>
> >>
> >> В Пн, 01/10/2018 в 19:54 +0300, Andrey Mashenkov пишет:
> >> > Dmitry, Nikolay,
> >> >
> >> > Ignite-3303 is a new Ignite module and there is no changes related to
> >> core or other existed modules.
> >> > So, PR should not affected existed functional ity and can be safely
> >> merged.
> >> >
> >> > Thanks.
> >> >
> >> >
> >> > пн, 1 окт. 2018 г., 16:04 Dmitriy Pavlov :
> >> > > Hi Saikat,
> >> > >
> >> > > I don't mind merging to master, but I have concern if it will go to
> >> 2.7. In
> >> > > the separate discussion, we agreed on code freeze should happen
> >> during last
> >> > > weekend, September, 30.
> >> > >
> >> > > So it is now up to community and release manager to decide if fix
> >> should go
> >> > > to the upcoming release. Usually, after the freeze, only bug/test
> >> fixes can
> >> > > be merged to release branch.
> >> > >
> >> > > Hi Nikolay,
> >> > >
> >> > > could you please announce that code freeze happened?
> >> > >
> >> > > Sincerely,
> >> > > Dmitriy Pavlov
> >> > >
> >> > > пн, 1 окт. 2018 г. в 3:58, Saikat Maitra :
> >> > >
> >> > > > Hi Alex, Nicolay
> >> > > >
> >> > > > As discussed with Andrew the changes looks good. Would it be ok to
> >> merge
> >> > > > this change to master considering the 2.7 release plan?
> >> > > >
> >> > > > Regards,
> >> > > > Saikat
> >> > > >
> >> > > > On Fri, Sep 28, 2018 at 7:15 PM Saikat Maitra <
> >> saikat.mai...@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Thank you Andrew
> >> > > > >
> >> > > > > Regards,
> >> > > > > Saikat
> >> > > > >
> >> > > > > On Fri, Sep 28, 2018 at 7:00 PM Andrey Mashenkov <
> >> > > > > andrey.mashen...@gmail.com> wrote:
> >> > > > >
> >> > > > >> Hi Saikat,
> >> > > > >>
> >> > > > >> Sorry for late answer. I've checked changes a day ago. Now,
> >> looks good.
> >> > > > >> Hope, it will be merged soon.
> >> > > > >>
> >> > > > >> Alex, would you please merge PR to master.
> >> > > > >>
> >> > > > >> сб, 29 сент. 2018 г., 2:29 Saikat Maitra <
> >> saikat.mai...@gmail.com>:
> >> > > > >>
> >> > > > >> > Hi Andrew,
> >> > > > >> >
> >> > > > >> > I have updated the changes.
> >> > > > >> >
> >> > > > >> > Can you please review and share feedback.
> >> > > > >> >
> >> > > > >> > Regards
> >> > > > >> > Saikat
> >> > > > >> >
> >> > > > >> > On Sat, Sep 22, 2018 at 2:23 PM Saikat Maitra <
> >> > > > saikat.mai...@gmail.com>
> >> > > > >> > wrote:
> >> > > > >> >
> >> > > > >> > > Hi Andrew
> >> > > > >> > >
> >> > > > >> > >
> >> > > > >> > > I have updated the changes.
> >> > > > >> > >
> >> > > > >> > >
> >> > > > >> > > Can you please review and share feedback.
> >> > > > >> > >
> >> > > > >> > >
> >> > > > >> > > Regards
> >> > > > >> > > Saikat
> >> > > > >> > >
> >> > > > >> > >
> >> > > > >> > > On Wed, Sep 19, 2018 at 8:11 PM, Saikat Maitra <
> >> > > > >> saikat.mai...@gmail.com>
> >> > > > >> > > wrote:
> >> > > > >> > >
> >> > > > >> > >> Hi Andrew,
> >> > > > >> > >>
> >> > > > >> > >> I have updated the tests and also added java docs.
> >> > > > >> > >>
> >> > > > >> >

Re: Request for review : IGNITE-3303 Apache Flink Integration - Flink source

2018-12-25 Thread Dmitriy Pavlov
Hi All,

Please check the commit
https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=commit;h=3b9c415ee2c4b6461c18f87f46f1950f324f5662

Thanks to everyone involved.

Sincerely,
Dmitriy Pavlov

вт, 25 дек. 2018 г. в 19:38, Dmitriy Pavlov :

> Hi Saikat,
>
> I'm going to slightly change the code before commit.
>
> First of all, I will add more code style and abbreviation rules related
> changes.
> Second of all, I add pom.xml variables usages where possible.
> And last but not least, I'm going to change the implementation of
> IgniteSource.
>
> AFAIK drainTo() method is not blocking so IgniteSource will be
> continuously spinning if there are no events coming from a cache. So I'm
> going to commit this variant:
>
> while (isRunning) {
> // block here for some time if there is no events from source
> CacheEvent firstEvt = evtBuf.poll(1, TimeUnit.SECONDS);
>
> if (firstEvt != null)
> evts.add(firstEvt);
>
> if (evtBuf.drainTo(evts, evtBatchSize) > 0) {
> synchronized (ctx.getCheckpointLock()) {
> for (CacheEvent evt : evts)
> ctx.collect(evt);
>
> evts.clear();
> }
> }
> }
>
>
> I hope you don't mind. This option allows us both to stop and does not spin 
> in the while.
>
>
> Sincerely,
>
> Dmitriy Pavlov
>
>
> сб, 13 окт. 2018 г. в 20:37, Saikat Maitra :
>
>> Hi Alex ,
>>
>> Can you please review and let me know if the PR looks good to merge.
>>
>> I have completed the requested changes.
>>
>> Regards,
>> Saikat
>>
>> On Mon, Oct 1, 2018 at 9:24 PM Saikat Maitra 
>> wrote:
>>
>> > Thank you everyone for reviewing the changes. As discussed I have
>> removed
>> > the FlinkIgniteSourceSelfExample and also verified that
>> FlinkIgniteSourceSelfTestSuite
>> > is part of Ignite Streamers in Team City.
>> >
>> > Please let me know if any further changes are required.
>> >
>> > Alex , will you please review and let me know if the PR looks good?
>> >
>> > Regards,
>> > Saikat
>> >
>> > On Mon, Oct 1, 2018 at 11:58 AM Nikolay Izhikov 
>> > wrote:
>> >
>> >> Hello, Andrey.
>> >>
>> >> Yes, I know it.
>> >> I've looked at the PR befor mailing :)
>> >>
>> >> Do you think we can include this improvement to the 2.7 release?
>> >> Do you have time to assist Saikat with TC setup and other tasks?
>> >>
>> >>
>> >> В Пн, 01/10/2018 в 19:54 +0300, Andrey Mashenkov пишет:
>> >> > Dmitry, Nikolay,
>> >> >
>> >> > Ignite-3303 is a new Ignite module and there is no changes related to
>> >> core or other existed modules.
>> >> > So, PR should not affected existed functional ity and can be safely
>> >> merged.
>> >> >
>> >> > Thanks.
>> >> >
>> >> >
>> >> > пн, 1 окт. 2018 г., 16:04 Dmitriy Pavlov :
>> >> > > Hi Saikat,
>> >> > >
>> >> > > I don't mind merging to master, but I have concern if it will go to
>> >> 2.7. In
>> >> > > the separate discussion, we agreed on code freeze should happen
>> >> during last
>> >> > > weekend, September, 30.
>> >> > >
>> >> > > So it is now up to community and release manager to decide if fix
>> >> should go
>> >> > > to the upcoming release. Usually, after the freeze, only bug/test
>> >> fixes can
>> >> > > be merged to release branch.
>> >> > >
>> >> > > Hi Nikolay,
>> >> > >
>> >> > > could you please announce that code freeze happened?
>> >> > >
>> >> > > Sincerely,
>> >> > > Dmitriy Pavlov
>> >> > >
>> >> > > пн, 1 окт. 2018 г. в 3:58, Saikat Maitra > >:
>> >> > >
>> >> > > > Hi Alex, Nicolay
>> >> > > >
>> >> > > > As discussed with Andrew the changes looks good. Would it be ok
>> to
>> >> merge
>> >> > > > this change to master considering the 2.7 release plan?
>> >> > > >
>> >> > > > Regards,
>> >> > > > Saikat
>> >> > > >
>> >> > > > On Fri, Sep 28, 2018 at 7:15 PM Saikat Maitra <
>> >> saikat.mai...@gmail.com>
>> >> > > > wrote:
>> >> > > >
>> >> > > > > Thank you Andrew
>> >> > > > >
>> >> > > > > Regards,
>> >> > > > > Saikat
>> >> > > > >
>> >> > > > > On Fri, Sep 28, 2018 at 7:00 PM Andrey Mashenkov <
>> >> > > > > andrey.mashen...@gmail.com> wrote:
>> >> > > > >
>> >> > > > >> Hi Saikat,
>> >> > > > >>
>> >> > > > >> Sorry for late answer. I've checked changes a day ago. Now,
>> >> looks good.
>> >> > > > >> Hope, it will be merged soon.
>> >> > > > >>
>> >> > > > >> Alex, would you please merge PR to master.
>> >> > > > >>
>> >> > > > >> сб, 29 сент. 2018 г., 2:29 Saikat Maitra <
>> >> saikat.mai...@gmail.com>:
>> >> > > > >>
>> >> > > > >> > Hi Andrew,
>> >> > > > >> >
>> >> > > > >> > I have updated the changes.
>> >> > > > >> >
>> >> > > > >> > Can you please review and share feedback.
>> >> > > > >> >
>> >> > > > >> > Regards
>> >> > > > >> > Saikat
>> >> > > > >> >
>> >> > > > >> > On Sat, Sep 22, 2018 at 2:23 PM Saikat Maitra <
>> >> > > > saikat.mai...@gmail.com>
>> >> > > > >> > wrote:
>> >> > > > >> >
>> >> > > > >> > > Hi Andrew
>> >> > > > >> > >
>> >> > > > >> > >
>> >> > > > >> > > I have updated the changes.
>> >> > > > >> > >
>> >> > > > >> > >
>> >> > > > >> > > Can you please revie

[MTCGA]: new failures in builds [2647030] needs to be handled

2018-12-25 Thread dpavlov . tasks
Hi Igniters,

 I've detected some new issue on TeamCity to be handled. You are more than 
welcomed to help.

 If your changes can lead to this failure(s): We're grateful that you were a 
volunteer to make the contribution to this project, but things change and you 
may no longer be able to finalize your contribution.
 Could you respond to this email and indicate if you wish to continue and fix 
test failures or step down and some committer may revert you commit. 

 *New test failure in master BigEntryQueryTest.testBigEntriesSelect 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-1071172963553561475&branch=%3Cdefault%3E&tab=testDetails
 Changes may lead to failure were done by 
 - andrey.mashenkov 
https://ci.ignite.apache.org/viewModification.html?modId=852237
 - kondakov87 
https://ci.ignite.apache.org/viewModification.html?modId=852172
 - dmitrievanthony 
https://ci.ignite.apache.org/viewModification.html?modId=852255
 - palmihal 
https://ci.ignite.apache.org/viewModification.html?modId=852011
 - kondakov87 
https://ci.ignite.apache.org/viewModification.html?modId=852149
 - kondakov87 
https://ci.ignite.apache.org/viewModification.html?modId=852180
 - ygerzhedovich 
https://ci.ignite.apache.org/viewModification.html?modId=852084
 - aplatonovv 
https://ci.ignite.apache.org/viewModification.html?modId=852263
 - saikat.maitra 
https://ci.ignite.apache.org/viewModification.html?modId=852321
 - sboikov 
https://ci.ignite.apache.org/viewModification.html?modId=851906
 - tledkov 
https://ci.ignite.apache.org/viewModification.html?modId=852194

 - Here's a reminder of what contributors were agreed to do 
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute 
 - Should you have any questions please contact dev@ignite.apache.org 

Best Regards,
Apache Ignite TeamCity Bot 
https://github.com/apache/ignite-teamcity-bot
Notification generated at 04:51:59 26-12-2018 


Re: Request for review : IGNITE-3303 Apache Flink Integration - Flink source

2018-12-25 Thread Saikat Maitra
Thank you so much Dmitriy, Andrew, Alexey and Anton

Warm Regards,
Saikat


On Tue, Dec 25, 2018 at 10:53 AM Dmitriy Pavlov  wrote:

> Hi All,
>
> Please check the commit
>
> https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=commit;h=3b9c415ee2c4b6461c18f87f46f1950f324f5662
>
> Thanks to everyone involved.
>
> Sincerely,
> Dmitriy Pavlov
>
> вт, 25 дек. 2018 г. в 19:38, Dmitriy Pavlov :
>
> > Hi Saikat,
> >
> > I'm going to slightly change the code before commit.
> >
> > First of all, I will add more code style and abbreviation rules related
> > changes.
> > Second of all, I add pom.xml variables usages where possible.
> > And last but not least, I'm going to change the implementation of
> > IgniteSource.
> >
> > AFAIK drainTo() method is not blocking so IgniteSource will be
> > continuously spinning if there are no events coming from a cache. So I'm
> > going to commit this variant:
> >
> > while (isRunning) {
> > // block here for some time if there is no events from source
> > CacheEvent firstEvt = evtBuf.poll(1, TimeUnit.SECONDS);
> >
> > if (firstEvt != null)
> > evts.add(firstEvt);
> >
> > if (evtBuf.drainTo(evts, evtBatchSize) > 0) {
> > synchronized (ctx.getCheckpointLock()) {
> > for (CacheEvent evt : evts)
> > ctx.collect(evt);
> >
> > evts.clear();
> > }
> > }
> > }
> >
> >
> > I hope you don't mind. This option allows us both to stop and does not
> spin in the while.
> >
> >
> > Sincerely,
> >
> > Dmitriy Pavlov
> >
> >
> > сб, 13 окт. 2018 г. в 20:37, Saikat Maitra :
> >
> >> Hi Alex ,
> >>
> >> Can you please review and let me know if the PR looks good to merge.
> >>
> >> I have completed the requested changes.
> >>
> >> Regards,
> >> Saikat
> >>
> >> On Mon, Oct 1, 2018 at 9:24 PM Saikat Maitra 
> >> wrote:
> >>
> >> > Thank you everyone for reviewing the changes. As discussed I have
> >> removed
> >> > the FlinkIgniteSourceSelfExample and also verified that
> >> FlinkIgniteSourceSelfTestSuite
> >> > is part of Ignite Streamers in Team City.
> >> >
> >> > Please let me know if any further changes are required.
> >> >
> >> > Alex , will you please review and let me know if the PR looks good?
> >> >
> >> > Regards,
> >> > Saikat
> >> >
> >> > On Mon, Oct 1, 2018 at 11:58 AM Nikolay Izhikov 
> >> > wrote:
> >> >
> >> >> Hello, Andrey.
> >> >>
> >> >> Yes, I know it.
> >> >> I've looked at the PR befor mailing :)
> >> >>
> >> >> Do you think we can include this improvement to the 2.7 release?
> >> >> Do you have time to assist Saikat with TC setup and other tasks?
> >> >>
> >> >>
> >> >> В Пн, 01/10/2018 в 19:54 +0300, Andrey Mashenkov пишет:
> >> >> > Dmitry, Nikolay,
> >> >> >
> >> >> > Ignite-3303 is a new Ignite module and there is no changes related
> to
> >> >> core or other existed modules.
> >> >> > So, PR should not affected existed functional ity and can be safely
> >> >> merged.
> >> >> >
> >> >> > Thanks.
> >> >> >
> >> >> >
> >> >> > пн, 1 окт. 2018 г., 16:04 Dmitriy Pavlov :
> >> >> > > Hi Saikat,
> >> >> > >
> >> >> > > I don't mind merging to master, but I have concern if it will go
> to
> >> >> 2.7. In
> >> >> > > the separate discussion, we agreed on code freeze should happen
> >> >> during last
> >> >> > > weekend, September, 30.
> >> >> > >
> >> >> > > So it is now up to community and release manager to decide if fix
> >> >> should go
> >> >> > > to the upcoming release. Usually, after the freeze, only bug/test
> >> >> fixes can
> >> >> > > be merged to release branch.
> >> >> > >
> >> >> > > Hi Nikolay,
> >> >> > >
> >> >> > > could you please announce that code freeze happened?
> >> >> > >
> >> >> > > Sincerely,
> >> >> > > Dmitriy Pavlov
> >> >> > >
> >> >> > > пн, 1 окт. 2018 г. в 3:58, Saikat Maitra <
> saikat.mai...@gmail.com
> >> >:
> >> >> > >
> >> >> > > > Hi Alex, Nicolay
> >> >> > > >
> >> >> > > > As discussed with Andrew the changes looks good. Would it be ok
> >> to
> >> >> merge
> >> >> > > > this change to master considering the 2.7 release plan?
> >> >> > > >
> >> >> > > > Regards,
> >> >> > > > Saikat
> >> >> > > >
> >> >> > > > On Fri, Sep 28, 2018 at 7:15 PM Saikat Maitra <
> >> >> saikat.mai...@gmail.com>
> >> >> > > > wrote:
> >> >> > > >
> >> >> > > > > Thank you Andrew
> >> >> > > > >
> >> >> > > > > Regards,
> >> >> > > > > Saikat
> >> >> > > > >
> >> >> > > > > On Fri, Sep 28, 2018 at 7:00 PM Andrey Mashenkov <
> >> >> > > > > andrey.mashen...@gmail.com> wrote:
> >> >> > > > >
> >> >> > > > >> Hi Saikat,
> >> >> > > > >>
> >> >> > > > >> Sorry for late answer. I've checked changes a day ago. Now,
> >> >> looks good.
> >> >> > > > >> Hope, it will be merged soon.
> >> >> > > > >>
> >> >> > > > >> Alex, would you please merge PR to master.
> >> >> > > > >>
> >> >> > > > >> сб, 29 сент. 2018 г., 2:29 Saikat Maitra <
> >> >> saikat.mai...@gmail.com>:
> >> >> > > > >>
> >> >> > > > >> > Hi Andrew,
> >> >> > > > >> >
> >> >> > > > >> > I have updated the changes.
> >> 

Re: Volunteer needed: AWS Elastic Load Balancer IP Findersimplemented

2018-12-25 Thread uday kale
Hi Stanislav,

I agree with your suggestions. Taking a step back I realised that, as a
user of an API, I don't want to spend my time refactoring the code due to
API name changes. I need more concrete reasons to do so. I will update the
PR with the required changes, to code and java-docs, for the review.

Regards,
Uday

On Sun, Dec 23, 2018 at 10:41 PM Stanislav Lukyanov 
wrote:

> OK, even though the name “ELB” applies to both the old and the new IpFinder
> in my opinion it isn’t really enough to change the name of the API classes
> now.
>
> BTW in the docs the features are called Application Load Balancer and
> Classic Load Balancer,
> so the abbreviations would be ALB and CLB, not ApplicationELB and
> ClassicELB.
>
> To make sure new users are not confused by the names of the classes we need
> to make the documentation clear:
> 1) Explain in the Javadoc of TcpDiscoveryElbIpFinder that it is for the
> Classic Load Balancers
> 2) Make sure that Javadocs of TcpDiscoveryElbIpFinder and
> TcpDiscoveryAlbIpFinder have
> each other in “@see” tags
> 3) Make sure both are described and clearly distinguished at readme.io
>
> Stan
>
> From: uday kale
> Sent: 21 декабря 2018 г. 18:30
> To: dev@ignite.apache.org
> Subject: Re: Volunteer needed: AWS Elastic Load Balancer IP
> Findersimplemented
>
> Hi,
>
> Regarding the name, it's not just about being a good name. I would point to
> the official AWS documentation [1], [2]. The current ELB does not stand for
> just the Classic Load balancer. Based on this documentation, the naming is
> good. I agree with your suggestion about extending
> TcpDiscoveryApplicationElbIpFinder from the TcpDiscoveryClassicElbIpFinder.
> I can make the required changes.
>
> [1]
>
> https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/introduction.html
>
> [2]
>
> https://docs.aws.amazon.com/elasticloadbalancing/latest/application/introduction.html
>
>
> Regards,
> Uday
>
>
>
>
> On Thu, Dec 20, 2018 at 12:09 AM Stanislav Lukyanov <
> stanlukya...@gmail.com>
> wrote:
>
> > Hi,
> >
> > Regarding the name - a not-so-good name isn’t always a sufficient
> > justification for renaming.
> > Public products such as Ignite have to also take into account convenience
> > for existing users.
> > Even in 3.0 when we technically can break compatibility I would avoid
> > breaking it just to
> > rename “ELB” to “Classic ELB”.
> >
> > Also, I believe the official names Amazon uses for these technologies are
> > ELB and ALB,
> > not “classic ELB” and “application ELB”.
> >
> > Regarding the code - `extends TcpDiscoveryClassicElbIpFinder` doesn’t
> > really solve it.
> > For example, now you have an unused loadBalancerName property in the
> > TcpDiscoveryApplicationElbIpFinder.
> >
> > I guess, to move this forward, let’s just add a new class,
> > TcpDiscoveryAlbIpFinder.
> > It doesn’t have to extend the existing TcpDiscoveryElbIpFinder – maybe we
> > could merge them, but let’s not be
> > stuck on this any longer.
> > Also let’s not change the existing TcpDiscoveryElbIpFinder – no need to
> > rename it or deprecate it. We can thing of the
> > two IpFinders as of just independent features.
> >
> > Thanks,
> > Stan
> >
> > From: uday kale
> > Sent: 9 октября 2018 г. 19:34
> > To: dev@ignite.apache.org
> > Subject: Re: Volunteer needed: AWS Elastic Load Balancer IP
> > Findersimplemented
> >
> > Hi Stanislav,
> >
> > I have removed extended the TcpDiscoveryApplicationElbIpFinder from
> > TcpDiscoveryClassicElbIpFinder to limi the code duplication. Please share
> > your feedback.
> >
> > Thanks,
> > Uday
> >
> > On Wed, Oct 3, 2018 at 1:12 AM Dmitriy Pavlov 
> > wrote:
> >
> > > Hi Uday,
> > >
> > > We should keep classes name as is within all minor releases (2.x). We
> can
> > > schedule rename only to 3.0. We need to keep both classes and methods
> > > naming as is to provide compilation guarantee. If someone used existing
> > > TcpDiscoveryElbIpFinder from 2.6 release than his or her code should
> > > compile and run well.
> > >
> > > I've updated checklist, thank you for pointing to this.
> > >
> > > So I suggest keeping existing naming of TcpDiscoveryElbIpFinder as it
> is
> > > part of API.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > вт, 2 окт. 2018 г. в 21:04, uday kale :
> > >
> > > > Thanks for the review Stan.
> > > > I renamed the existing class TcpDiscoveryElbIpFinder since the name
> is
> > > not
> > > > clear regarding the actual load balancer being used. The names
> > > > TcpDiscoveryClassicElbIpFinder
> > > > and TcpDiscoveryApplicationElbIpFinder provide a clear distinction.
> > > > I deprecated the existing class because of review checklist [1] point
> > 1.a
> > > > under checklist. It requires methods to be deprecated instead of
> being
> > > > renamed. I assumed this will extend to classes too.
> > > > Regarding the your suggestion for having the tests I agree. It will
> be
> > > > helpful to know whether TC can accept properties while initiating a
> > run.
> > > >
> > >

[jira] [Created] (IGNITE-10819) Test IgniteClientRejoinTest.testClientsReconnectAfterStart became flaky in master recently

2018-12-25 Thread Sergey Chugunov (JIRA)
Sergey Chugunov created IGNITE-10819:


 Summary: Test 
IgniteClientRejoinTest.testClientsReconnectAfterStart became flaky in master 
recently
 Key: IGNITE-10819
 URL: https://issues.apache.org/jira/browse/IGNITE-10819
 Project: Ignite
  Issue Type: Bug
Reporter: Sergey Chugunov
 Fix For: 2.8


As [test 
history|https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-21180267941031641&tab=testDetails&branch_IgniteTests24Java8=%3Cdefault%3E]
 in master branch shows the test has become flaky in master recently.

Test started failing when IGNITE-10555 was merged to master.

The reason of failure is timeout when *client4* node hangs waiting for PME to 
complete. Communication failures are emulated in the test and when all clients 
fail to init an exchange on a specific affinity topology version (major=7, 
minor=1) everything works fine.
But sometimes *client4* node manages to finish init the exchange and hangs 
forever.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-10818) GridQueryTypeDescriptor should have cacheName and alias field

2018-12-25 Thread Ray Liu (JIRA)
Ray Liu created IGNITE-10818:


 Summary: GridQueryTypeDescriptor should have cacheName and alias 
field
 Key: IGNITE-10818
 URL: https://issues.apache.org/jira/browse/IGNITE-10818
 Project: Ignite
  Issue Type: Improvement
Reporter: Ray Liu
Assignee: Ray Liu


Currently, GridQueryTypeDescriptor don't have cacheName and alias field.

We have to cast GridQueryTypeDescriptor to QueryTypeDescriptorImpl to get these 
two fields.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)