[MTCGA]: new failures in builds [1848300] needs to be handled

[dpavlov . tasks]

Hi Igniters,

 I've detected some new issue on TeamCity to be handled. You are more than 
welcomed to help.

 If your changes can lead to this failure(s): We're grateful that you were a 
volunteer to make the contribution to this project, but things change and you 
may no longer be able to finalize your contribution.
 Could you respond to this email and indicate if you wish to continue and fix 
test failures or step down and some committer may revert you commit. 

 *Recently contributed test failed in master 
org.apache.ignite.testsuites.IgniteIgfsLinuxAndMacOSTestSuite.initializationError
 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-4034437294452110435=%3Cdefault%3E=testDetails
 Changes may lead to failure were done by 
 - verbalab 
http://ci.ignite.apache.org/viewModification.html?modId=831346=false
 - verbalab 
http://ci.ignite.apache.org/viewModification.html?modId=831344=false
 - dmitriyff 
http://ci.ignite.apache.org/viewModification.html?modId=831341=false
 - verbalab 
http://ci.ignite.apache.org/viewModification.html?modId=831339=false

 - Here's a reminder of what contributors were agreed to do 
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute 
 - Should you have any questions please contact dev@ignite.apache.org 

Best Regards,
Apache Ignite TeamCity Bot 
https://github.com/apache/ignite-teamcity-bot
Notification generated at 08:31:40 27-09-2018 

Ignite and Spark on Yarn

[Dmitriy Setrakyan]

Can anyone familiar with the subject respond here?

https://stackoverflow.com/questions/52488073/ignite-yarn-and-hortonworks

D.

[jira] [Created] (IGNITE-9712) SQL: Select query benchmarks

[Pavel Kuznetsov (JIRA)]

Pavel Kuznetsov created IGNITE-9712:
---

 Summary: SQL: Select query benchmarks
 Key: IGNITE-9712
 URL: https://issues.apache.org/jira/browse/IGNITE-9712
 Project: Ignite
  Issue Type: Task
Reporter: Pavel Kuznetsov
Assignee: Pavel Kuznetsov


We need to create benchmark suite for select operations.
We already have benchmarks for selects, so we need to update this code to 
support 
1) select the single row with by PK; indexed field
2) select the range of rows filtered with PK; indexed field

Selects by non-indexed fields are out of scope.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: ML examples wrap logic in IgniteThread. Why?

[Юрий Бабак]

Denis,

Thanks for this notice, actually this is some kind of atavism. Run this
code inside IgniteThread was a requirement when we had a distributed
matrices. But now all our algorithms builds over distributed datasets and
we don't need it anymore.

I created JIRA ticket 
for this.

Thanks,
Yuriy

чт, 27 сент. 2018 г. в 0:20, Denis Magda :

> Yury, ML folks,
>
> I've mentioned a strange thing. Looks like every example we have wraps up
> its logic in the following block
>
> IgniteThread igniteThread = new
> IgniteThread(ignite.configuration().getIgniteInstanceName(),
> KMeansClusterizationExample.class.getSimpleName(), () -> {
>
>
> //ML specific stuff (training, predicting, calculations, etc.)
>
> });
>
> igniteThread.start();
> igniteThread.join();
>
>
> Why do we do that?
>
> Denis
>

[GitHub] ignite pull request #4846: IGNITE-9550 Get operation returns null for a lost...

[dgovorukhin]

GitHub user dgovorukhin opened a pull request:

https://github.com/apache/ignite/pull/4846

IGNITE-9550 Get operation returns null for a lost partition with READ_SAFE 
policy



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9550-dg

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4846.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4846


commit 9a66787932b86b865b1c4f42fd331b363e2e2a75
Author: Dmitriy Govorukhin 
Date:   2018-09-26T17:23:23Z

IGNITE-9550 change affReady future listen

commit a410671a6f80305dd9c19db42be5afd170c759b1
Author: Dmitriy Govorukhin 
Date:   2018-09-26T21:58:30Z

IGNITE-9550 Fix NPE on node stop




---

[GitHub] ignite pull request #4844: IGNITE-9550 change affReady future listen

[dgovorukhin]

Github user dgovorukhin closed the pull request at:

https://github.com/apache/ignite/pull/4844


---

ML examples wrap logic in IgniteThread. Why?

[Denis Magda]

Yury, ML folks,

I've mentioned a strange thing. Looks like every example we have wraps up
its logic in the following block

IgniteThread igniteThread = new
IgniteThread(ignite.configuration().getIgniteInstanceName(),
KMeansClusterizationExample.class.getSimpleName(), () -> {


//ML specific stuff (training, predicting, calculations, etc.)

});

igniteThread.start();
igniteThread.join();


Why do we do that?

Denis

Re: Apache Ignite project summary at people.apache.org

[Dmitriy Pavlov]

Ok, I will try ComDev list first and then incubator mailing lists. It seems
ComDev list contains some mentioning of people.apache.org.
ср, 26 сент. 2018 г. в 22:00, Denis Magda :

> Hi Dmitriy and thanks for putting this out,
>
> Ignite definition needs to be changed to this one
>
>
>
> *Memory-centric distributed database, caching, and processing platform
> fortransactional, analytical, and streaming workloads delivering
> in-memoryspeeds at petabyte scale*
>
> Have no idea how to update it here:
> http://people.apache.org/phonebook.html
>
> Please try to reach out to gene...@incubator.apache.org
>
> --
> Denis
>
> On Wed, Sep 26, 2018 at 9:04 AM Dmitriy Pavlov 
> wrote:
>
> > Hi Igniters,
> >
> > It seems to me, that the project description at
> > http://people.apache.org/phonebook.html is a little bit outdated.
> >
> > It says
> >   High-performance, integrated and distributed in-memory platform for
> > computing and transacting on large-scale data sets in real-time
> >
> > But our site says
> >  Memory-centric distributed database, caching, and processing platform
> for
> > transactional, analytical, and streaming workloads delivering in-memory
> > speeds at petabyte scale
> >
> > After persistent storage donation I think the second one is more correct,
> > so I suggest to update Apache info.
> >
> > Who can advise me how to update product description at the Apache site.
> Is
> > it easy to do?
> >
> > The same issue is related to
> > https://projects.apache.org/project.html?ignite
> > but here I can ask at the ComDev list to update.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
>

Re: Wrong off-heap size is reported for a node

[Dmitriy Pavlov]

Oops, my fault. It seems I forgot to cherry-pick to 2.7, but I resolved
ticket with 2.7 assigned.

I will fix it tomorrow.

ср, 26 сент. 2018 г. в 21:52, Denis Magda :

> Thanks, Pavel and the rest of the Igniters involved.
>
> That simple usability improvement is a big deal for those who use Ignite in
> production.
>
> Are we getting it in 2.7?
>
> --
> Denis
>
> On Wed, Sep 26, 2018 at 10:11 AM Dmitriy Pavlov 
> wrote:
>
> > Hi Igniters,
> >
> > Thanks to everyone, who has participated in this discussion and shared
> > their view and ideas.
> >
> > I've merged fix of changes related to logging only. Fixing of cluster
> > metrics can be done in a separate ticket/discussion.
> >
> > Pavel, thank you for your contribution and for answering my questions.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пн, 24 сент. 2018 г. в 18:52, Pavel Pereslegin :
> >
> > > Andrei,
> > > I totally agree with you and I think that "ClusterMetrics" should also
> > > be fixed, I'm just not sure that we should include this change in the
> > > same ticket.
> > > пн, 24 сент. 2018 г. в 18:43, aealexsandrov :
> > > >
> > > > Hi,
> > > >
> > > > OK, the user can use it to calculate the off-heap. But I think that
> the
> > > > reason for your changes to fix the calculation of the nonHeap used in
> > > Ignite
> > > > now. For example now REST return "-1" for nonHeapMemoryMaximum. I
> think
> > > that
> > > > it can't be used somehow. So REST possible should be updated as you
> did
> > > for
> > > > log metrics and it will require for the same logic.
> > > >
> > > > BR,
> > > > Andrei
> > > >
> > > >
> > > >
> > > > --
> > > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> > >
> >
>

Re: Critical worker threads liveness checking drawbacks

[Vyacheslav Daradur]

Andrey Gura, thank you for the answer!

I agree that wrapping of 'init' method reduces the profit of watchdog
service in case of PME worker, but in other cases, we should wrap all
possible long sections on GridDhtPartitionExchangeFuture. For example
'onCacheChangeRequest' method or
'cctx.affinity().onCacheChangeRequest' inside because it may take
significant time (reproducer attached).

I only want to point out a possible issue which may allow to end-user
halt the Ignite cluster accidentally.

I'm sure that PME experts know how to fix this issue properly.
On Wed, Sep 26, 2018 at 10:28 PM Andrey Gura  wrote:
>
> Vyacheslav,
>
> Exchange worker is strongly tied with
> GridDhtPartitionExchangeFuture#init and it is ok. Exchange worker also
> shouldn't be blocked for long time but in reality it happens.It also
> means that your change doesn't make sense.
>
> What actually make sense it is identification of places which
> intentionally blocking. May be some places/actions should be braced by
> blocking guards.
>
> If you have failing tests please make sure that your failureHandler is
> NoOpFailureHandler or any other handler with ignoreFailureTypes =
> [CRITICAL_WORKER_BLOCKED].
>
>
> On Wed, Sep 26, 2018 at 9:43 PM Vyacheslav Daradur  
> wrote:
> >
> > Hi Igniters!
> >
> > Thank you for this important improvement!
> >
> > I've looked through implementation and noticed that
> > GridDhtPartitionsExchangeFuture#init has not been wrapped in blocked
> > section. This means it easy to halt the node in case of longrunning
> > actions during PME, for example when we create a cache with
> > StoreFactrory which connect to 3rd party DB.
> >
> > I'm not sure that it is the right behavior.
> >
> > I filled the issue [1] and prepared the PR [2] with reproducer and possible 
> > fix.
> >
> > Andrey, could you please look at and confirm that it makes sense?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-9710
> > [2] https://github.com/apache/ignite/pull/4845
> > On Mon, Sep 24, 2018 at 9:46 PM Andrey Kuznetsov  wrote:
> > >
> > > Denis,
> > >
> > > I've created the ticket [1] with short description of the functionality.
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-9679
> > >
> > >
> > > пн, 24 сент. 2018 г. в 17:46, Denis Magda :
> > >
> > > > Andrey K. and G.,
> > > >
> > > > Thanks, do we have a documentation ticket created? Prachi (copied) can 
> > > > help
> > > > with the documentation.
> > > >
> > > > --
> > > > Denis
> > > >
> > > > On Mon, Sep 24, 2018 at 5:51 AM Andrey Gura  wrote:
> > > >
> > > > > Andrey,
> > > > >
> > > > > finally your change is merged to master branch. Congratulations and
> > > > > thank you very much! :)
> > > > >
> > > > > I think that the next step is feature that will allow signal about
> > > > > blocked threads to the monitoring tools via MXBean.
> > > > >
> > > > > I hope you will continue development of this feature and provide your
> > > > > vision in new JIRA issue.
> > > > >
> > > > >
> > > > > On Tue, Sep 11, 2018 at 6:54 PM Andrey Kuznetsov 
> > > > > wrote:
> > > > > >
> > > > > > David, Maxim!
> > > > > >
> > > > > > Thanks a lot for you ideas. Unfortunately, I can't adopt all of them
> > > > > right
> > > > > > now: the scope is much broader than the scope of the change I
> > > > implement.
> > > > > I
> > > > > > have had a talk to a group of Ignite commiters, and we agreed to
> > > > complete
> > > > > > the change as follows.
> > > > > > - Blocking instructions in system-critical which may resonably last
> > > > long
> > > > > > should be explicitly excluded from the monitoring.
> > > > > > - Failure handlers should have a setting to suppress some failures 
> > > > > > on
> > > > > > per-failure-type basis.
> > > > > > According to this I have updated the implementation: [1]
> > > > > >
> > > > > > [1] https://github.com/apache/ignite/pull/4089
> > > > > >
> > > > > > пн, 10 сент. 2018 г. в 22:35, David Harvey :
> > > > > >
> > > > > > > When I've done this before,I've needed to find the oldest  thread,
> > > > and
> > > > > kill
> > > > > > > the node running that.   From a language standpoint, Maxim's 
> > > > > > > "without
> > > > > > > progress" better than "heartbeat".   For example, what I'm most
> > > > > interested
> > > > > > > in on a distributed system is which thread started the work it has
> > > > not
> > > > > > > completed the earliest, and when did that thread last make forward
> > > > > > > process. You don't want to kill a node because a thread is
> > > > waiting
> > > > > on a
> > > > > > > lock held by a thread that went off-node and has not gotten a
> > > > response.
> > > > > > > If you don't understand the dependency relationships, you will 
> > > > > > > make
> > > > > > > incorrect recovery decisions.
> > > > > > >
> > > > > > > On Mon, Sep 10, 2018 at 4:08 AM Maxim Muzafarov 
> > > > > > > 
> > > > > > > wrote:
> > > > > > >
> > > > > > > > I think we should find exact answers to these questions:
> > > > > > > >  1. What 

Re: Critical worker threads liveness checking drawbacks

[Andrey Gura]

Vyacheslav,

Exchange worker is strongly tied with
GridDhtPartitionExchangeFuture#init and it is ok. Exchange worker also
shouldn't be blocked for long time but in reality it happens.It also
means that your change doesn't make sense.

What actually make sense it is identification of places which
intentionally blocking. May be some places/actions should be braced by
blocking guards.

If you have failing tests please make sure that your failureHandler is
NoOpFailureHandler or any other handler with ignoreFailureTypes =
[CRITICAL_WORKER_BLOCKED].


On Wed, Sep 26, 2018 at 9:43 PM Vyacheslav Daradur  wrote:
>
> Hi Igniters!
>
> Thank you for this important improvement!
>
> I've looked through implementation and noticed that
> GridDhtPartitionsExchangeFuture#init has not been wrapped in blocked
> section. This means it easy to halt the node in case of longrunning
> actions during PME, for example when we create a cache with
> StoreFactrory which connect to 3rd party DB.
>
> I'm not sure that it is the right behavior.
>
> I filled the issue [1] and prepared the PR [2] with reproducer and possible 
> fix.
>
> Andrey, could you please look at and confirm that it makes sense?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-9710
> [2] https://github.com/apache/ignite/pull/4845
> On Mon, Sep 24, 2018 at 9:46 PM Andrey Kuznetsov  wrote:
> >
> > Denis,
> >
> > I've created the ticket [1] with short description of the functionality.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-9679
> >
> >
> > пн, 24 сент. 2018 г. в 17:46, Denis Magda :
> >
> > > Andrey K. and G.,
> > >
> > > Thanks, do we have a documentation ticket created? Prachi (copied) can 
> > > help
> > > with the documentation.
> > >
> > > --
> > > Denis
> > >
> > > On Mon, Sep 24, 2018 at 5:51 AM Andrey Gura  wrote:
> > >
> > > > Andrey,
> > > >
> > > > finally your change is merged to master branch. Congratulations and
> > > > thank you very much! :)
> > > >
> > > > I think that the next step is feature that will allow signal about
> > > > blocked threads to the monitoring tools via MXBean.
> > > >
> > > > I hope you will continue development of this feature and provide your
> > > > vision in new JIRA issue.
> > > >
> > > >
> > > > On Tue, Sep 11, 2018 at 6:54 PM Andrey Kuznetsov 
> > > > wrote:
> > > > >
> > > > > David, Maxim!
> > > > >
> > > > > Thanks a lot for you ideas. Unfortunately, I can't adopt all of them
> > > > right
> > > > > now: the scope is much broader than the scope of the change I
> > > implement.
> > > > I
> > > > > have had a talk to a group of Ignite commiters, and we agreed to
> > > complete
> > > > > the change as follows.
> > > > > - Blocking instructions in system-critical which may resonably last
> > > long
> > > > > should be explicitly excluded from the monitoring.
> > > > > - Failure handlers should have a setting to suppress some failures on
> > > > > per-failure-type basis.
> > > > > According to this I have updated the implementation: [1]
> > > > >
> > > > > [1] https://github.com/apache/ignite/pull/4089
> > > > >
> > > > > пн, 10 сент. 2018 г. в 22:35, David Harvey :
> > > > >
> > > > > > When I've done this before,I've needed to find the oldest  thread,
> > > and
> > > > kill
> > > > > > the node running that.   From a language standpoint, Maxim's 
> > > > > > "without
> > > > > > progress" better than "heartbeat".   For example, what I'm most
> > > > interested
> > > > > > in on a distributed system is which thread started the work it has
> > > not
> > > > > > completed the earliest, and when did that thread last make forward
> > > > > > process. You don't want to kill a node because a thread is
> > > waiting
> > > > on a
> > > > > > lock held by a thread that went off-node and has not gotten a
> > > response.
> > > > > > If you don't understand the dependency relationships, you will make
> > > > > > incorrect recovery decisions.
> > > > > >
> > > > > > On Mon, Sep 10, 2018 at 4:08 AM Maxim Muzafarov 
> > > > > > wrote:
> > > > > >
> > > > > > > I think we should find exact answers to these questions:
> > > > > > >  1. What `critical` issue exactly is?
> > > > > > >  2. How can we find critical issues?
> > > > > > >  3. How can we handle critical issues?
> > > > > > >
> > > > > > > First,
> > > > > > >  - Ignore uninterruptable actions (e.g. worker\service shutdown)
> > > > > > >  - Long I/O operations (should be a configurable timeout for each
> > > > type of
> > > > > > > usage)
> > > > > > >  - Infinite loops
> > > > > > >  - Stalled\deadlocked threads (and\or too many parked threads,
> > > > exclude
> > > > > > I/O)
> > > > > > >
> > > > > > > Second,
> > > > > > >  - The working queue is without progress (e.g. disco, exchange
> > > > queues)
> > > > > > >  - Work hasn't been completed since the last heartbeat (checking
> > > > > > > milestones)
> > > > > > >  - Too many system resources used by a thread for the long period
> > > of
> > > > time
> > > > > > > (allocated memory, CPU)
> > > > > > >  

Re: Apache Ignite project summary at people.apache.org

[Denis Magda]

Hi Dmitriy and thanks for putting this out,

Ignite definition needs to be changed to this one



*Memory-centric distributed database, caching, and processing platform
fortransactional, analytical, and streaming workloads delivering
in-memoryspeeds at petabyte scale*

Have no idea how to update it here: http://people.apache.org/phonebook.html

Please try to reach out to gene...@incubator.apache.org

--
Denis

On Wed, Sep 26, 2018 at 9:04 AM Dmitriy Pavlov 
wrote:

> Hi Igniters,
>
> It seems to me, that the project description at
> http://people.apache.org/phonebook.html is a little bit outdated.
>
> It says
>   High-performance, integrated and distributed in-memory platform for
> computing and transacting on large-scale data sets in real-time
>
> But our site says
>  Memory-centric distributed database, caching, and processing platform for
> transactional, analytical, and streaming workloads delivering in-memory
> speeds at petabyte scale
>
> After persistent storage donation I think the second one is more correct,
> so I suggest to update Apache info.
>
> Who can advise me how to update product description at the Apache site. Is
> it easy to do?
>
> The same issue is related to
> https://projects.apache.org/project.html?ignite
> but here I can ask at the ComDev list to update.
>
> Sincerely,
> Dmitriy Pavlov
>

Re: Wrong off-heap size is reported for a node

[Denis Magda]

Thanks, Pavel and the rest of the Igniters involved.

That simple usability improvement is a big deal for those who use Ignite in
production.

Are we getting it in 2.7?

--
Denis

On Wed, Sep 26, 2018 at 10:11 AM Dmitriy Pavlov 
wrote:

> Hi Igniters,
>
> Thanks to everyone, who has participated in this discussion and shared
> their view and ideas.
>
> I've merged fix of changes related to logging only. Fixing of cluster
> metrics can be done in a separate ticket/discussion.
>
> Pavel, thank you for your contribution and for answering my questions.
>
> Sincerely,
> Dmitriy Pavlov
>
> пн, 24 сент. 2018 г. в 18:52, Pavel Pereslegin :
>
> > Andrei,
> > I totally agree with you and I think that "ClusterMetrics" should also
> > be fixed, I'm just not sure that we should include this change in the
> > same ticket.
> > пн, 24 сент. 2018 г. в 18:43, aealexsandrov :
> > >
> > > Hi,
> > >
> > > OK, the user can use it to calculate the off-heap. But I think that the
> > > reason for your changes to fix the calculation of the nonHeap used in
> > Ignite
> > > now. For example now REST return "-1" for nonHeapMemoryMaximum. I think
> > that
> > > it can't be used somehow. So REST possible should be updated as you did
> > for
> > > log metrics and it will require for the same logic.
> > >
> > > BR,
> > > Andrei
> > >
> > >
> > >
> > > --
> > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >
>

Re: Critical worker threads liveness checking drawbacks

[Vyacheslav Daradur]

Hi Igniters!

Thank you for this important improvement!

I've looked through implementation and noticed that
GridDhtPartitionsExchangeFuture#init has not been wrapped in blocked
section. This means it easy to halt the node in case of longrunning
actions during PME, for example when we create a cache with
StoreFactrory which connect to 3rd party DB.

I'm not sure that it is the right behavior.

I filled the issue [1] and prepared the PR [2] with reproducer and possible fix.

Andrey, could you please look at and confirm that it makes sense?

[1] https://issues.apache.org/jira/browse/IGNITE-9710
[2] https://github.com/apache/ignite/pull/4845
On Mon, Sep 24, 2018 at 9:46 PM Andrey Kuznetsov  wrote:
>
> Denis,
>
> I've created the ticket [1] with short description of the functionality.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-9679
>
>
> пн, 24 сент. 2018 г. в 17:46, Denis Magda :
>
> > Andrey K. and G.,
> >
> > Thanks, do we have a documentation ticket created? Prachi (copied) can help
> > with the documentation.
> >
> > --
> > Denis
> >
> > On Mon, Sep 24, 2018 at 5:51 AM Andrey Gura  wrote:
> >
> > > Andrey,
> > >
> > > finally your change is merged to master branch. Congratulations and
> > > thank you very much! :)
> > >
> > > I think that the next step is feature that will allow signal about
> > > blocked threads to the monitoring tools via MXBean.
> > >
> > > I hope you will continue development of this feature and provide your
> > > vision in new JIRA issue.
> > >
> > >
> > > On Tue, Sep 11, 2018 at 6:54 PM Andrey Kuznetsov 
> > > wrote:
> > > >
> > > > David, Maxim!
> > > >
> > > > Thanks a lot for you ideas. Unfortunately, I can't adopt all of them
> > > right
> > > > now: the scope is much broader than the scope of the change I
> > implement.
> > > I
> > > > have had a talk to a group of Ignite commiters, and we agreed to
> > complete
> > > > the change as follows.
> > > > - Blocking instructions in system-critical which may resonably last
> > long
> > > > should be explicitly excluded from the monitoring.
> > > > - Failure handlers should have a setting to suppress some failures on
> > > > per-failure-type basis.
> > > > According to this I have updated the implementation: [1]
> > > >
> > > > [1] https://github.com/apache/ignite/pull/4089
> > > >
> > > > пн, 10 сент. 2018 г. в 22:35, David Harvey :
> > > >
> > > > > When I've done this before,I've needed to find the oldest  thread,
> > and
> > > kill
> > > > > the node running that.   From a language standpoint, Maxim's "without
> > > > > progress" better than "heartbeat".   For example, what I'm most
> > > interested
> > > > > in on a distributed system is which thread started the work it has
> > not
> > > > > completed the earliest, and when did that thread last make forward
> > > > > process. You don't want to kill a node because a thread is
> > waiting
> > > on a
> > > > > lock held by a thread that went off-node and has not gotten a
> > response.
> > > > > If you don't understand the dependency relationships, you will make
> > > > > incorrect recovery decisions.
> > > > >
> > > > > On Mon, Sep 10, 2018 at 4:08 AM Maxim Muzafarov 
> > > > > wrote:
> > > > >
> > > > > > I think we should find exact answers to these questions:
> > > > > >  1. What `critical` issue exactly is?
> > > > > >  2. How can we find critical issues?
> > > > > >  3. How can we handle critical issues?
> > > > > >
> > > > > > First,
> > > > > >  - Ignore uninterruptable actions (e.g. worker\service shutdown)
> > > > > >  - Long I/O operations (should be a configurable timeout for each
> > > type of
> > > > > > usage)
> > > > > >  - Infinite loops
> > > > > >  - Stalled\deadlocked threads (and\or too many parked threads,
> > > exclude
> > > > > I/O)
> > > > > >
> > > > > > Second,
> > > > > >  - The working queue is without progress (e.g. disco, exchange
> > > queues)
> > > > > >  - Work hasn't been completed since the last heartbeat (checking
> > > > > > milestones)
> > > > > >  - Too many system resources used by a thread for the long period
> > of
> > > time
> > > > > > (allocated memory, CPU)
> > > > > >  - Timing fields associated with each thread status exceeded a
> > > maximum
> > > > > time
> > > > > > limit.
> > > > > >
> > > > > > Third (not too many options here),
> > > > > >  - `log everything` should be the default behaviour in all these
> > > cases,
> > > > > > since it may be difficult to find the cause after the restart.
> > > > > >  - Wait some interval of time and kill the hanging node (cluster
> > > should
> > > > > be
> > > > > > configured stable enough)
> > > > > >
> > > > > > Questions,
> > > > > >  - Not sure, but can workers miss their heartbeat deadlines if CPU
> > > loads
> > > > > up
> > > > > > to 80%-90%? Bursts of momentary overloads can be
> > > > > > expected behaviour as a normal part of system operations.
> > > > > >  - Why do we decide that critical thread should monitor each other?
> > > For
> > > > > > instance, if all 

[GitHub] ignite pull request #4845: IGNITE-9710 Ignite watchdog service handles longr...

[daradurvs]

GitHub user daradurvs opened a pull request:

https://github.com/apache/ignite/pull/4845

IGNITE-9710 Ignite watchdog service handles longrunning cache creation



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/daradurvs/ignite ignite-9710

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4845.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4845


commit 0ea5075eab46ffc58d08ceea50e2d9492a13bba0
Author: Vyacheslav Daradur 
Date:   2018-09-26T18:33:30Z

GridCachePartitionExchangeManager#init wrapped by blocking section




---

[jira] [Created] (IGNITE-9710) Ignite watchdog service handles cache creation

[Vyacheslav Daradur (JIRA)]

Vyacheslav Daradur created IGNITE-9710:
--

 Summary: Ignite watchdog service handles cache creation
 Key: IGNITE-9710
 URL: https://issues.apache.org/jira/browse/IGNITE-9710
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.6
Reporter: Vyacheslav Daradur
Assignee: Vyacheslav Daradur
 Fix For: 2.7


Ignite watchdog service introduced by IGNITE-6587 handles long running cache 
creation.

Action in {{GridDhtPartitionsExchangeFuture#init}} may take significant time 
and possibly should be covered by blocking section of warchdog service.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [ML] New features and improvement of ML module for 2.7 release

[Dmitriy Setrakyan]

Great stuff! Would be nice to see a series of blogs outlining Ignite ML
capabilities.This would give us a better momentum.

D.

On Wed, Sep 26, 2018 at 10:31 AM Юрий Бабак  wrote:

> Hello Igniters,
>
> I want to make up some overview of all features and major improvement of ML
> module for this release.
>
> So let me start from the one of our main feature for this release:
>
> *TensorFlow integration* <
> https://issues.apache.org/jira/browse/IGNITE-8670>
>
> This integration allows us to use Apache Ignite as a data source for
> TensorFlow. Also, this integration will allow creating and maintaining
> TensorFlow clusters over Apache Ignite and submit TF jobs to those
> clusters. More details in the related umbrella ticket.
>
> Also, for this release we have some new algorithms:
>
> * Random forest  
> * Gradient boosted trees <
> https://issues.apache.org/jira/browse/IGNITE-7149>
> * Logistic regression[binary
> ][multi-class
> ]
> * ANN 
>
> New features related with data preprocessing:
>
> * Pipeline 
> * L1,L2 normalization 
> * Data filtering for new datasets
> 
> * Encoding categorical features [OneHotEncoder
> ][OneOfKEncoder
> ]
> * Imputer and Binarizer  >
> * MaxAbsScaler 
> * Dataset splitting 
>
> New features for a model validation:
>
> * Model estimator 
> * k-fold cross-validation
> 
> * Param grid for tuning hyper-parameters in cross-validation
> 
>
> Other features and improvements:
>
> * Model updating 
> * ML tutorial 
> * Optional indexing for decision trees
> 
> * Learning context for trainers(local parallelizing and logging of training
> process) 
> * Unification of API for feature extractor
> 
> * Several tickets for removing old unused classes and improvements for code
> coverage and examples [1 <
> https://issues.apache.org/jira/browse/IGNITE-9124>
> ][2 ][3
> ][4
> ][5
> ][6
> ]
>
> Sincerely,
> Yuriy Babak
>

Re: [ML] New features and improvement of ML module for 2.7 release

[Denis Magda]

Enormous and outstanding addition!

Yuriy, I've talked to Akmal and he is happy to help with the documentation.
Please start documenting everything and reach out Akmal directly.

--
Denis

On Wed, Sep 26, 2018 at 10:31 AM Юрий Бабак  wrote:

> Hello Igniters,
>
> I want to make up some overview of all features and major improvement of ML
> module for this release.
>
> So let me start from the one of our main feature for this release:
>
> *TensorFlow integration* <
> https://issues.apache.org/jira/browse/IGNITE-8670>
>
> This integration allows us to use Apache Ignite as a data source for
> TensorFlow. Also, this integration will allow creating and maintaining
> TensorFlow clusters over Apache Ignite and submit TF jobs to those
> clusters. More details in the related umbrella ticket.
>
> Also, for this release we have some new algorithms:
>
> * Random forest  
> * Gradient boosted trees <
> https://issues.apache.org/jira/browse/IGNITE-7149>
> * Logistic regression[binary
> ][multi-class
> ]
> * ANN 
>
> New features related with data preprocessing:
>
> * Pipeline 
> * L1,L2 normalization 
> * Data filtering for new datasets
> 
> * Encoding categorical features [OneHotEncoder
> ][OneOfKEncoder
> ]
> * Imputer and Binarizer  >
> * MaxAbsScaler 
> * Dataset splitting 
>
> New features for a model validation:
>
> * Model estimator 
> * k-fold cross-validation
> 
> * Param grid for tuning hyper-parameters in cross-validation
> 
>
> Other features and improvements:
>
> * Model updating 
> * ML tutorial 
> * Optional indexing for decision trees
> 
> * Learning context for trainers(local parallelizing and logging of training
> process) 
> * Unification of API for feature extractor
> 
> * Several tickets for removing old unused classes and improvements for code
> coverage and examples [1 <
> https://issues.apache.org/jira/browse/IGNITE-9124>
> ][2 ][3
> ][4
> ][5
> ][6
> ]
>
> Sincerely,
> Yuriy Babak
>

[GitHub] asfgit closed pull request #21: IGNITE-9541 Fix bugs, refactor and add new info on html page

[GitBox]

asfgit closed pull request #21: IGNITE-9541 Fix bugs, refactor and add new info 
on html page
URL: https://github.com/apache/ignite-teamcity-bot/pull/21
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/IgnitePersistentTeamcity.java
 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/IgnitePersistentTeamcity.java
index 837a9dc..bbf96a2 100644
--- 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/IgnitePersistentTeamcity.java
+++ 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/IgnitePersistentTeamcity.java
@@ -408,14 +408,18 @@ private boolean isHistoryAgeLessThanSecs(SuiteInBranch 
key, int seconds, Expirab
 
 List buildRefs = loadBuildHistory(buildHistCache(), 90, 
suiteInBranch,
 (key, sinceBuildId) -> {
-buildsFromRest.addAll(teamcity.getFinishedBuilds(projectId, 
branch, sinceDate, untilDate, sinceBuildId));
+List reverseList = teamcity.getFinishedBuilds(projectId, 
branch, sinceDate, untilDate, sinceBuildId);
+
+Collections.reverse(reverseList);
+
+buildsFromRest.addAll(reverseList);
 
 return buildsFromRest;
 });
 
 if (sinceDate != null || untilDate != null) {
 if (!buildsFromRest.isEmpty() && sinceDate != null){
-int firstBuildId = 
buildRefs.indexOf(buildsFromRest.get(buildsFromRest.size() - 1));
+int firstBuildId = buildRefs.indexOf(buildsFromRest.get(0));
 
 if (firstBuildId == 0)
 return buildsFromRest;
@@ -457,7 +461,7 @@ private boolean isHistoryAgeLessThanSecs(SuiteInBranch key, 
int seconds, Expirab
 if (build == null || build.isFakeStub())
 return false;
 
-Date date = build.getFinishDate();
+Date date = build.getStartDate();
 
 if (sinceDate != null && untilDate != null)
 if ((date.after(sinceDate) || 
date.equals(sinceDate)) &&
diff --git a/ignite-tc-helper-web/src/main/webapp/comparison.html 
b/ignite-tc-helper-web/src/main/webapp/comparison.html
index 909ee44..5efe4e3 100644
--- a/ignite-tc-helper-web/src/main/webapp/comparison.html
+++ b/ignite-tc-helper-web/src/main/webapp/comparison.html
@@ -16,13 +16,19 @@
 
 
 
-
+
 
 DATE INTERVAL
-
+
 
 
 
+
+
+FEATURES
+
+
+
 TESTS
 COUNT
 
@@ -167,7 +173,11 @@
 return parseFloat(string.substring(string.indexOf("-") + 2, 
string.lastIndexOf("-") - 1));
 }
 
-function printStatistics(num, map) {
+function printStatistics(num, map, sinceDate, untilDate) {
+clearBackgroundFromAllDataCells();
+clearGraphs(num);
+
+const anotherNum = (num === 1) ? 2 : 1;
 const parseTime = d3.timeParse("%d-%m-%YT%H:%M:%S");
 
 let statistics = {};
@@ -187,15 +197,30 @@
 }
 }
 
-const anotherNum = (num === 1) ? 2 : 1;
+if (dates.length === 0) {
+printImportantMessage(num, "#ff", "No data for the selected 
period");
+fillAllDataCells(num, "");
+
+return;
+} else {
+let firstDate = 
moment(parseTime(map[0].startDate)).format("DD-MM-");
+let lastDate = moment(parseTime(map[map.length - 
1].startDate)).format("DD-MM-");
+
+if ((sinceDate.format("DD-MM-") !== firstDate) || 
(untilDate.format("DD-MM-") !== lastDate)) {
+printImportantMessage(num, "#ffb856", "Data for " +
+(firstDate === lastDate ? firstDate : ("the period from " 
+ firstDate + " to " + lastDate)) + "");
+} else {
+$("#info" + num).html("");
+if (document.getElementById("info" + anotherNum).innerHTML === 
"") {
+document.getElementById("showInfo").style.display = "none";
+}
+}
+}
+
 let anotherMedian;
 let result = {};
 
 for (let i = 0; i < prOcc.length; i++) {
-
-document.getElementById(prOcc[i] + 1).style.background = null;
-document.getElementById(prOcc[i] + 2).style.background = null;
-
 result = getMinMaxMedian(statistics[prOcc[i]]);
 anotherMedian = parseMedian(document.getElementById(prOcc[i] + 
anotherNum).innerHTML);
 
@@ -221,8 +246,8 @@
 }
 
 $(document).ready(function() {
-loadData(1, moment(oneWeekAgo).format("DDMM"), 
moment().format("DDMM"));
-loadData(2, moment(twoWeekAgo).format("DDMM"), 

[ML] New features and improvement of ML module for 2.7 release

[Юрий Бабак]

Hello Igniters,

I want to make up some overview of all features and major improvement of ML
module for this release.

So let me start from the one of our main feature for this release:

*TensorFlow integration* 

This integration allows us to use Apache Ignite as a data source for
TensorFlow. Also, this integration will allow creating and maintaining
TensorFlow clusters over Apache Ignite and submit TF jobs to those
clusters. More details in the related umbrella ticket.

Also, for this release we have some new algorithms:

* Random forest  
* Gradient boosted trees 
* Logistic regression[binary
][multi-class
]
* ANN 

New features related with data preprocessing:

* Pipeline 
* L1,L2 normalization 
* Data filtering for new datasets

* Encoding categorical features [OneHotEncoder
][OneOfKEncoder
]
* Imputer and Binarizer 
* MaxAbsScaler 
* Dataset splitting 

New features for a model validation:

* Model estimator 
* k-fold cross-validation

* Param grid for tuning hyper-parameters in cross-validation


Other features and improvements:

* Model updating 
* ML tutorial 
* Optional indexing for decision trees

* Learning context for trainers(local parallelizing and logging of training
process) 
* Unification of API for feature extractor

* Several tickets for removing old unused classes and improvements for code
coverage and examples [1 
][2 ][3
][4
][5
][6
]

Sincerely,
Yuriy Babak

[GitHub] ignite pull request #4844: IGNITE-9550 change affReady future listen

[dgovorukhin]

GitHub user dgovorukhin opened a pull request:

https://github.com/apache/ignite/pull/4844

IGNITE-9550 change affReady future listen



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignit4e-9550-dg

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4844.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4844






---

Re: Wrong off-heap size is reported for a node

[Dmitriy Pavlov]

Hi Igniters,

Thanks to everyone, who has participated in this discussion and shared
their view and ideas.

I've merged fix of changes related to logging only. Fixing of cluster
metrics can be done in a separate ticket/discussion.

Pavel, thank you for your contribution and for answering my questions.

Sincerely,
Dmitriy Pavlov

пн, 24 сент. 2018 г. в 18:52, Pavel Pereslegin :

> Andrei,
> I totally agree with you and I think that "ClusterMetrics" should also
> be fixed, I'm just not sure that we should include this change in the
> same ticket.
> пн, 24 сент. 2018 г. в 18:43, aealexsandrov :
> >
> > Hi,
> >
> > OK, the user can use it to calculate the off-heap. But I think that the
> > reason for your changes to fix the calculation of the nonHeap used in
> Ignite
> > now. For example now REST return "-1" for nonHeapMemoryMaximum. I think
> that
> > it can't be used somehow. So REST possible should be updated as you did
> for
> > log metrics and it will require for the same logic.
> >
> > BR,
> > Andrei
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

[GitHub] ignite pull request #4593: IGNITE-9305

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4593


---

[GitHub] zzzadruga opened a new pull request #21: IGNITE-9541 Fix bugs, refactor and add new info on html page

[GitBox]

zzzadruga opened a new pull request #21: IGNITE-9541 Fix bugs, refactor and add 
new info on html page
URL: https://github.com/apache/ignite-teamcity-bot/pull/21
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Apache Ignite project summary at people.apache.org

[Dmitriy Pavlov]

Hi Igniters,

It seems to me, that the project description at
http://people.apache.org/phonebook.html is a little bit outdated.

It says
  High-performance, integrated and distributed in-memory platform for
computing and transacting on large-scale data sets in real-time

But our site says
 Memory-centric distributed database, caching, and processing platform for
transactional, analytical, and streaming workloads delivering in-memory
speeds at petabyte scale

After persistent storage donation I think the second one is more correct,
so I suggest to update Apache info.

Who can advise me how to update product description at the Apache site. Is
it easy to do?

The same issue is related to https://projects.apache.org/project.html?ignite
but here I can ask at the ComDev list to update.

Sincerely,
Dmitriy Pavlov

[GitHub] ignite pull request #4843: IGN-11747 [IGNITE-9298] control.sh does not suppo...

[a-polyakov]

GitHub user a-polyakov opened a pull request:

https://github.com/apache/ignite/pull/4843

IGN-11747 [IGNITE-9298] control.sh does not support SSL in 5.2.1-p14



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite IGN-11747

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4843.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4843


commit 69d9272009cc6cbec07d8bbc99364ec777c851b5
Author: Sergey Chugunov 
Date:   2018-05-17T10:10:01Z

IGNITE-8474 Fixed WalStateNodeLeaveExchangeTask preventing exchange merge - 
Fixes #3990.

Signed-off-by: Alexey Goncharuk 

(cherry-picked from commit #ebd669e4c53cfd66708ff18dd59071e4aace38ae)

commit 9b80a49d642de27dd6a69daea3921d1c05919df6
Author: Evgeny Stanilovskiy 
Date:   2018-04-28T12:09:17Z

IGNITE-8401: errorneous WalPointers comparation. - Fixes #3931.

Signed-off-by: dspavlov 

commit 40ddce5d046e98b9da4c05a9a302326d4e15e08f
Author: Pavel Kovalenko 
Date:   2018-05-17T12:04:27Z

Merge branch 'ignite-2.5.1-master' of github.com:gridgain/apache-ignite 
into ignite-2.5.1-master

commit 31f8d7e445b1ef7986e3b58e700e119b8490c734
Author: Aleksei Scherbakov 
Date:   2018-05-17T17:11:41Z

IGNITE-8423 Control utility may block when joining node is watiting for 
partition map exchange. - Fixes #4018.

Signed-off-by: Alexey Goncharuk 

commit d9a80ff6c09ac92a78948fcdb292c5e5c28111d0
Author: Anton Kalashnikov 
Date:   2018-05-18T08:18:39Z

IGNITE-8464 Fixed race in WALIterator leading to skipped segments - Fixes 
#3982.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit fdad641)

commit 68bf746c47d445d5a469d29b7a4ae5ce040b3d9b
Author: Pavel Kovalenko 
Date:   2018-05-18T16:28:45Z

IGNITE-8531 Fixed NPE if checkpoint has no pages to write, but has 
partitions to destroy. - Fixes #4026.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit 5c8d9ff)

commit 18d913c6118b5132859fbc1e5bacfc97392f9bc2
Author: vd-pyatkov 
Date:   2018-05-18T13:59:14Z

IGNITE-8491 Add JMX flag: Is the node in baseline or not - Fixes #4010.

Signed-off-by: Ivan Rakov 
(cherry picked from commit f8ae30d)

commit 40bbe65df3cbd8b3f478812b0e204b2973b98d3d
Author: Alexey Goncharuk 
Date:   2018-05-21T12:13:37Z

IGNITE-8521 Do not attempt to unregister continuous query if query ID is 
null

commit affc9d47b986e054fb0c229dcf95e64a38cab13f
Author: Pavel Kovalenko 
Date:   2018-05-21T19:33:50Z

IGNITE-8544 Use exchange result topology version for local wal state 
management. - Fixes #4039.

Signed-off-by: Alexey Goncharuk 

commit 3f4b2ae4f594e8c73b5a9c4b86f2351c2bae591e
Author: Anton Kalashnikov 
Date:   2018-05-23T09:24:51Z

IGNITE-8561 SingleSegmentLogicalRecordsIterator is broken - Fixes #4045.

(cherry picked from commit 21678bc)

commit 49929b73a00f758ef099ea99f8c6f54373c30a95
Author: EdShangGG 
Date:   2018-04-28T16:27:27Z

IGNITE-7628 SqlQuery hangs indefinitely with additional not registered in 
baseline node.

Signed-off-by: Andrey Gura 

commit 95b749460b5fa32deecb946865944e854d9745d3
Author: Dmitriy Sorokin 
Date:   2018-05-24T13:03:41Z

IGNITE-8584 Provide ability to terminate any thread with enabled test 
features.

Signed-off-by: Andrey Gura 

commit 8ac69e68ecfc1c31b02004d33157d3f9b85a7b6e
Author: Andrey Gura 
Date:   2018-05-24T13:56:42Z

IGNITE-8563 Fixed WAL file archiver does not propagate file archiving error 
to error handler

commit ef63514bfa38c595017ebec62c42a93c90e0a062
Author: Dmitriy Govorukhin 
Date:   2018-05-24T15:04:02Z

IGNITE-8583 DataStorageMetricsMXBean.getOffHeapSize include checkpoint 
buffer size - Fixes #4054.

Signed-off-by: dpavlov 

(cherry picked from commit 86c1899)

commit 04d00be82369f633f26955656441147f53cf2da1
Author: Sergey Chugunov 
Date:   2018-05-24T15:11:47Z

IGNITE-8560 Update index validation utility to use statistical check 
approach - Fixes #4051.

Signed-off-by: Ivan Rakov 

(cherry-picked from commit #76e1fe754d7a8bd059d7ab64d17cbefa4913a702)

commit c457aa60084e1805b7fdbbc88d870ffdc36d14fa
Author: Pavel Kovalenko 
Date:   2018-04-06T07:35:17Z

IGNITE-7933 Checkpoing file markers should be written atomically - Fixes 
#3633.

Signed-off-by: Alexey Goncharuk 

(cherry-picked from commit #4a0695ceae2f99c4841e8382e723daff4580ea3d)

commit 6b28e8d76bc0119917af0a738d9f07ff794cde20
Author: Anton Kalashnikov 
Date:   2018-05-25T09:01:10Z

IGNITE-8540 Fast cleanup of PDS when joining node is not in baseline - 
Fixes #4037.

Signed-off-by: Alexey Goncharuk 

(cherry-picked from commit #3f14d2b35d7818196598e8541590186e06b8edbb)

commit 

Re: Volunteer needed: AWS Elastic Load Balancer IP Finders implemented

[Denis Magda]

Igniters,

Don't we have experts in our networking component? I do believe that's a
small improvement that can be verified and merged quickly.

--
Denis

On Wed, Sep 26, 2018 at 8:50 AM Dmitriy Pavlov 
wrote:

> Igniters, Friendly reminder.
>
> Denis, could you advice how to proceed in this case? It seems feature can
> be useful, but committers are not involved in a review.
>
> Should we send a proposal with lazy consensus and automatically merge? Any
> alternative ideas on how to proceed with issues stuck in the review process
> are strongly appreciated.
>
> Sincerely,
> Dmitriy Pavlov
>
> ср, 15 авг. 2018 г. в 19:38, Dmitriy Pavlov :
>
> > Hi Igniters,
> >
> > I took a look at the proposed contribution
> > https://issues.apache.org/jira/browse/IGNITE-8617 and PR
> > https://github.com/apache/ignite/pull/4076.
> >
> > I found this change reasonable and I can update code according to code
> > style.
> >
> > But I would ask someone from the community with experience with AWS ELB
> to
> > join the review. It would also benefit us if someone could try to run a
> > cluster with AWS ELB IP Finder.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
>

Re: Volunteer needed: AWS Elastic Load Balancer IP Finders implemented

[Dmitriy Pavlov]

Igniters, Friendly reminder.

Denis, could you advice how to proceed in this case? It seems feature can
be useful, but committers are not involved in a review.

Should we send a proposal with lazy consensus and automatically merge? Any
alternative ideas on how to proceed with issues stuck in the review process
are strongly appreciated.

Sincerely,
Dmitriy Pavlov

ср, 15 авг. 2018 г. в 19:38, Dmitriy Pavlov :

> Hi Igniters,
>
> I took a look at the proposed contribution
> https://issues.apache.org/jira/browse/IGNITE-8617 and PR
> https://github.com/apache/ignite/pull/4076.
>
> I found this change reasonable and I can update code according to code
> style.
>
> But I would ask someone from the community with experience with AWS ELB to
> join the review. It would also benefit us if someone could try to run a
> cluster with AWS ELB IP Finder.
>
> Sincerely,
> Dmitriy Pavlov
>

[GitHub] ignite pull request #4842: IGN-11747 [IGNITE-9298] control.sh does not suppo...

[a-polyakov]

Github user a-polyakov closed the pull request at:

https://github.com/apache/ignite/pull/4842


---

[jira] [Created] (IGNITE-9709) IgniteCache : Put/Get operations hang on deactivated cluster

[Vyacheslav Daradur (JIRA)]

Vyacheslav Daradur created IGNITE-9709:
--

 Summary: IgniteCache : Put/Get operations hang on deactivated 
cluster
 Key: IGNITE-9709
 URL: https://issues.apache.org/jira/browse/IGNITE-9709
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.6
Reporter: Vyacheslav Daradur


{{IgniteCache#put/get}} operations hang on inactive cluster.
Scenario:
 # Get instance of {{IgniteCache}} on active cluster;
 # Deactivate cluster;
 # Perform put/get operation using instance from step 1;
{code}
IgniteEx ignite = startGrid(0);

assertTrue(ignite.cluster().active());

IgniteCache cache = ignite.getOrCreateCache("testCache");

cache.put("key", "val");

ignite.cluster().active(false);

assertFalse(ignite.cluster().active());

IgniteInternalFuture fut = GridTestUtils.runAsync(() -> 
cache.get("key"));

ignite.cluster().active(true);

assertTrue(ignite.cluster().active());

assertNotNull(fut.get(10_000)); // Timeout was reached here
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9708) Node didn't detect corrup of file index.bin in FS

[ARomantsov (JIRA)]

ARomantsov created IGNITE-9708:
--

 Summary: Node didn't detect corrup of file index.bin in FS
 Key: IGNITE-9708
 URL: https://issues.apache.org/jira/browse/IGNITE-9708
 Project: Ignite
  Issue Type: Bug
  Components: persistence
Affects Versions: 2.5
Reporter: ARomantsov
 Fix For: 2.7


I run two tests
In first, i break part.bin file and node correctly find that
In second, i break index.bin file and all command (select, idle_verify, etc) 
not detected problem with it.

All corrupts run on correct stop node, and checks after node start



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4830: IGNITE-9649 Debug logs enhacement

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4830


---

[GitHub] ignite pull request #4840: IGNITE-9704

[devozerov]

Github user devozerov closed the pull request at:

https://github.com/apache/ignite/pull/4840


---

[jira] [Created] (IGNITE-9707) TouchedExpiryPolicy with persistent on atomic cache, update TTL without lock

[Vladislav Pyatkov (JIRA)]

Vladislav Pyatkov created IGNITE-9707:
-

 Summary: TouchedExpiryPolicy with persistent on atomic cache, 
update TTL without lock
 Key: IGNITE-9707
 URL: https://issues.apache.org/jira/browse/IGNITE-9707
 Project: Ignite
  Issue Type: Test
Reporter: Vladislav Pyatkov
 Attachments: AtomicCacheWithTtlTest.java

{noformat}

[2018-09-26 
18:06:29,882][ERROR][sys-stripe-0-#86%internal.AtomicCacheWithTtlTest2%][GridCacheIoManager]
 Failed to process message [senderId=949211e1-5e2a-41fe-b66b-195ae3300033, 
messageType=class 
o.a.i.i.processors.cache.distributed.near.GridNearSingleGetRequest]
java.lang.AssertionError
 at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1247)
 at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.invoke(GridCacheOffheapManager.java:1528)
 at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:352)
 at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.storeValue(GridCacheMapEntry.java:3605)
 at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.storeValue(GridCacheMapEntry.java:3581)
 at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.updateTtl(GridCacheMapEntry.java:2468)
 at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.updateTtl(GridCacheMapEntry.java:2444)
 at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerGet0(GridCacheMapEntry.java:680)
 at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerGetVersioned(GridCacheMapEntry.java:554)
 at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.getAllAsync0(GridCacheAdapter.java:1994)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.getDhtAllAsync(GridDhtCacheAdapter.java:781)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtGetSingleFuture.getAsync(GridDhtGetSingleFuture.java:360)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtGetSingleFuture.map0(GridDhtGetSingleFuture.java:254)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtGetSingleFuture.map(GridDhtGetSingleFuture.java:237)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtGetSingleFuture.init(GridDhtGetSingleFuture.java:161)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.getDhtSingleAsync(GridDhtCacheAdapter.java:878)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.processNearSingleGetRequest(GridDhtCacheAdapter.java:893)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$300(GridDhtAtomicCache.java:130)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$4.apply(GridDhtAtomicCache.java:252)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$4.apply(GridDhtAtomicCache.java:247)
 at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056)
 at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581)
 at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380)
 at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306)
 at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101)
 at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295)
 at 
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
 at 
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184)
 at 
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125)
 at 
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091)
 at 
org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:496)
 at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
 at java.lang.Thread.run(Thread.java:745)

{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9706) ML: Update ignite-tensorflow to support TensorFlow standalone client mode

[Anton Dmitriev (JIRA)]

Anton Dmitriev created IGNITE-9706:
--

 Summary: ML: Update ignite-tensorflow to support TensorFlow 
standalone client mode
 Key: IGNITE-9706
 URL: https://issues.apache.org/jira/browse/IGNITE-9706
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Affects Versions: 2.7
Reporter: Anton Dmitriev
Assignee: Anton Dmitriev
 Fix For: 2.7


We need to update "ignite-tensorflow" module to support TensorFlow standalone 
client mode described in in correspondent 
[documentation|https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/distribute#standalone-client-mode].
 To make it we need to run TensorFlow standard servers on workers and avoid 
using "TF_CONFIG" variable in user script (use "TF_CLUSTER" instead).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4842: IGN-11747 [IGNITE-9298] control.sh does not suppo...

[a-polyakov]

GitHub user a-polyakov opened a pull request:

https://github.com/apache/ignite/pull/4842

IGN-11747 [IGNITE-9298] control.sh does not support SSL  in 5.2.1-p14

(org.apache.ignite.internal.commandline.CommandHandler)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite IGN-11747

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4842.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4842


commit ded31b5668ff63993a002cfd7d86e9e4b744ba90
Author: a-polyakov 
Date:   2018-09-06T13:35:44Z

IGN-11747 [IGNITE-9298] control.sh does not support SSL 
(org.apache.ignite.internal.commandline.CommandHandler) in 5.2.1-p14




---

[GitHub] dspavlov opened a new pull request #20: Ignite 9645

[GitBox]

dspavlov opened a new pull request #20: Ignite 9645
URL: https://github.com/apache/ignite-teamcity-bot/pull/20
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ignite pull request #4841: IGNITE-9705 GridCacheDatabaseSharedManager#create...

[ibessonov]

GitHub user ibessonov opened a pull request:

https://github.com/apache/ignite/pull/4841

IGNITE-9705 GridCacheDatabaseSharedManager#createPageMemory flushes invalid 
byte buffer into snapshot manager.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9705

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4841.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4841


commit e732800145016008262513c950e969972ed4c29a
Author: ibessonov 
Date:   2018-09-26T14:38:52Z

IGNITE-9705 Added forgotten buffer rewinding.




---

[jira] [Created] (IGNITE-9705) GridCacheDatabaseSharedManager#createPageMemory flushes invalid byte buffer into snapshot manager

[Ivan Bessonov (JIRA)]

Ivan Bessonov created IGNITE-9705:
-

 Summary: GridCacheDatabaseSharedManager#createPageMemory flushes 
invalid byte buffer into snapshot manager
 Key: IGNITE-9705
 URL: https://issues.apache.org/jira/browse/IGNITE-9705
 Project: Ignite
  Issue Type: Improvement
Reporter: Ivan Bessonov
Assignee: Ivan Bessonov


org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager#createPageMemory
 executes two consecutive operations with the same byte buffer bot doesn't 
rewind it in between which may lead to inconsistent data.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4784: IGNITE-9381 On leaving node from cluster all clas...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4784


---

[GitHub] ignite pull request #4840: IGNITE-9704

[devozerov]

GitHub user devozerov opened a pull request:

https://github.com/apache/ignite/pull/4840

IGNITE-9704



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9704-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4840.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4840


commit 46c678af25af0f23a14d49dd1e430d862b48f819
Author: devozerov 
Date:   2018-09-12T08:00:00Z

Done.




---

[jira] [Created] (IGNITE-9704) JDBC thin: round-robin connection distribution

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-9704:
---

 Summary: JDBC thin: round-robin connection distribution
 Key: IGNITE-9704
 URL: https://issues.apache.org/jira/browse/IGNITE-9704
 Project: Ignite
  Issue Type: Task
  Components: jdbc
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov
 Fix For: 2.7


Currently every connection picks random IP address from connection string. This 
may lead to considerable imbalances between cluster nodes. Let's user 
round-robin instead of random.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9703) JDBC thick: do no throw exception from setReadOnly method

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-9703:
---

 Summary: JDBC thick: do no throw exception from setReadOnly method
 Key: IGNITE-9703
 URL: https://issues.apache.org/jira/browse/IGNITE-9703
 Project: Ignite
  Issue Type: Task
  Components: jdbc
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov
 Fix For: 2.7


This method is harmless and might be called from various {{DataSource}} 
implementations. E.g. connection pools such as Hikary. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9702) Make IGNITE_BINARY_SORT_OBJECT_FIELDS enabled by default

[Mikhail Cherkasov (JIRA)]

Mikhail Cherkasov created IGNITE-9702:
-

 Summary: Make IGNITE_BINARY_SORT_OBJECT_FIELDS enabled by default
 Key: IGNITE-9702
 URL: https://issues.apache.org/jira/browse/IGNITE-9702
 Project: Ignite
  Issue Type: Bug
Reporter: Mikhail Cherkasov
 Fix For: 3.0


Right now:

BinObjectImpl(a=1,b=2) != BinObjectImpl(a=2,b=1), so hash code of binary object 
depends of fields order and for ignite it's too different objects.

It's unclear and counter-intuitive for a user.

However, this can not be changed till 3.0, because it breaks compatibility with 
existent storage and requites migration with downtime and an util that will 
migrate binary objects to a new internal fields order.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] asfgit closed pull request #19: DI refactoring

[GitBox]

asfgit closed pull request #19: DI refactoring
URL: https://github.com/apache/ignite-teamcity-bot/pull/19
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] SomeFire commented on a change in pull request #19: DI refactoring

[GitBox]

SomeFire commented on a change in pull request #19: DI refactoring
URL: https://github.com/apache/ignite-teamcity-bot/pull/19#discussion_r220537287
 
 

 ##
 File path: 
ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/observer/ObserverTask.java
 ##
 @@ -17,43 +17,82 @@
 
 package org.apache.ignite.ci.observer;
 
+import java.util.HashSet;
 import java.util.Queue;
+import java.util.Set;
 import java.util.TimerTask;
 import java.util.concurrent.ConcurrentLinkedQueue;
+import javax.inject.Inject;
 import org.apache.ignite.ci.IAnalyticsEnabledTeamcity;
-import org.apache.ignite.ci.ITcHelper;
+import org.apache.ignite.ci.ITcServerProvider;
+import org.apache.ignite.ci.di.AutoProfiling;
+import org.apache.ignite.ci.di.MonitoredTask;
+import org.apache.ignite.ci.jira.IJiraIntegration;
 import org.apache.ignite.ci.tcmodel.result.Build;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 
 /**
  * Checks observed builds for finished status and comments JIRA ticket.
  */
 public class ObserverTask extends TimerTask {
+/** Logger. */
+private static final Logger logger = 
LoggerFactory.getLogger(ObserverTask.class);
+
+/** Helper. */
+@Inject private ITcServerProvider srvProvider;
+
 /** Helper. */
-private final ITcHelper helper;
+@Inject private IJiraIntegration jiraIntegration;
 
 /** Builds. */
 final Queue builds;
 
 /**
- * @param helper Helper.
  */
-ObserverTask(ITcHelper helper) {
-this.helper = helper;
+ObserverTask() {
 builds = new ConcurrentLinkedQueue<>();
 }
 
 /** {@inheritDoc} */
 @Override public void run() {
+try {
 
 Review comment:
   May be add `while(true)`?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (IGNITE-9701) Create pass-through identifier for every single Parrtition Map Exchange

[Andrey Kalinin (JIRA)]

Andrey Kalinin created IGNITE-9701:
--

 Summary: Create pass-through identifier for every single 
Parrtition Map Exchange
 Key: IGNITE-9701
 URL: https://issues.apache.org/jira/browse/IGNITE-9701
 Project: Ignite
  Issue Type: Improvement
Reporter: Andrey Kalinin


In order to improve the monitoring of PME, we can provide each message about 
this event in the logs with a unique identifier that will allow us to more 
clearly determine the start and end time, as well as to accurately identify 
events related to this PME.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] SomeFire commented on a change in pull request #19: DI refactoring

[GitBox]

SomeFire commented on a change in pull request #19: DI refactoring
URL: https://github.com/apache/ignite-teamcity-bot/pull/19#discussion_r220537287
 
 

 ##
 File path: 
ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/observer/ObserverTask.java
 ##
 @@ -17,43 +17,82 @@
 
 package org.apache.ignite.ci.observer;
 
+import java.util.HashSet;
 import java.util.Queue;
+import java.util.Set;
 import java.util.TimerTask;
 import java.util.concurrent.ConcurrentLinkedQueue;
+import javax.inject.Inject;
 import org.apache.ignite.ci.IAnalyticsEnabledTeamcity;
-import org.apache.ignite.ci.ITcHelper;
+import org.apache.ignite.ci.ITcServerProvider;
+import org.apache.ignite.ci.di.AutoProfiling;
+import org.apache.ignite.ci.di.MonitoredTask;
+import org.apache.ignite.ci.jira.IJiraIntegration;
 import org.apache.ignite.ci.tcmodel.result.Build;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 
 /**
  * Checks observed builds for finished status and comments JIRA ticket.
  */
 public class ObserverTask extends TimerTask {
+/** Logger. */
+private static final Logger logger = 
LoggerFactory.getLogger(ObserverTask.class);
+
+/** Helper. */
+@Inject private ITcServerProvider srvProvider;
+
 /** Helper. */
-private final ITcHelper helper;
+@Inject private IJiraIntegration jiraIntegration;
 
 /** Builds. */
 final Queue builds;
 
 /**
- * @param helper Helper.
  */
-ObserverTask(ITcHelper helper) {
-this.helper = helper;
+ObserverTask() {
 builds = new ConcurrentLinkedQueue<>();
 }
 
 /** {@inheritDoc} */
 @Override public void run() {
+try {
 
 Review comment:
   May be add `while(true)`?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ignite pull request #4836: Ignite 9683

[andrewmed]

Github user andrewmed closed the pull request at:

https://github.com/apache/ignite/pull/4836


---

[GitHub] ignite pull request #4839: Ignite 9683 2

[andrewmed]

GitHub user andrewmed opened a pull request:

https://github.com/apache/ignite/pull/4839

Ignite 9683 2



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite IGNITE-9683-2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4839.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4839


commit 9aa0a2217b3afe88bead03543add7a4f19f1781a
Author: Pavel Kovalenko 
Date:   2018-09-11T11:40:42Z

Zookeeper custom pinger.

commit fc962873f2d513f69d879a340e8dc6730fe1fe22
Author: AMedvedev 
Date:   2018-09-26T09:24:29Z

add ZooKeeper pinger configuration property

commit e0d789aefd36b66cfde90a1616eb171592f5b57b
Author: AMedvedev 
Date:   2018-09-26T09:55:36Z

codestyle




---

[GitHub] ignite pull request #4838: IGNITE-9700: Remove configurable values from meso...

[shroman]

GitHub user shroman opened a pull request:

https://github.com/apache/ignite/pull/4838

IGNITE-9700: Remove configurable values from mesos pom.xml.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/shroman/ignite IGNITE-9700

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4838.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4838


commit 4a5b1fd6f712eaad4c3cd8877c457102dd81dad0
Author: shroman 
Date:   2018-09-26T12:31:44Z

IGNITE-9700: Remove configurable values from mesos pom.xml.




---

[jira] [Created] (IGNITE-9700) Remove configurable values from mesos pom.xml

[Roman Shtykh (JIRA)]

Roman Shtykh created IGNITE-9700:


 Summary: Remove configurable values from mesos pom.xml
 Key: IGNITE-9700
 URL: https://issues.apache.org/jira/browse/IGNITE-9700
 Project: Ignite
  Issue Type: Bug
Reporter: Roman Shtykh
Assignee: Roman Shtykh


pom.xml contains urls and paths that can be configured when building, but it 
introduces git changes with every build.

Need to update the links in the file and remove them from build process, since 
the whole process depends on the the links and will not likely be changed in 
near future.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4837: Ignite 8717

[antonovsergey93]

GitHub user antonovsergey93 opened a pull request:

https://github.com/apache/ignite/pull/4837

Ignite 8717

Cache configuration was moved to meta store. Verification of cache 
configuration on node joining was added.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8717

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4837.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4837


commit 8aeefa775cac27e34fbebb3799a056d2f1ba282b
Author: Ivan Daschinskiy 
Date:   2018-06-20T09:01:00Z

IGNITE-8717: First steps in implementing storing cache configuration
in metastore.

commit 2e28aba5686f4f694c590487db1da9572f1cc78a
Author: Ivan Daschinskiy 
Date:   2018-06-21T08:30:26Z

IGNITE-8717: Second step. Tests passed.

commit 9c75281671e1a25afbbf9bc07ec623d0a6031f80
Author: Sergey Antonov 
Date:   2018-09-19T09:18:47Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into ignite-8717

commit 4e30a5707b39b650727a4512e0730f30155372f1
Author: Sergey Antonov 
Date:   2018-09-21T10:22:44Z

IGNITE-8717 WIP

commit 549073b14a43151ca5715e4d583ea31d07a7bf50
Author: Sergey Antonov 
Date:   2018-09-21T12:05:20Z

IGNITE-8717 WIP

commit 19be946aaec837b553f89b6d46528769f29df730
Author: Sergey Antonov 
Date:   2018-09-24T08:51:46Z

IGNITE-8717 Reproducer was added

commit 338a3fe1d56f3719f8b258c38d50f0fb8b069861
Author: Sergey Antonov 
Date:   2018-09-26T11:55:52Z

IGNITE-8717 first version of fix

commit 26b9e3e0bc39bc35f080e4c7e95ef8b2697d7cfc
Author: Sergey Antonov 
Date:   2018-09-26T12:00:24Z

IGNITE-8717 cody style changes in test




---

Re: mesos IgniteProvider improvement

[Roman Shtykh]

Ilya,
Sorry for getting back to you so late.
No idea, why it was necessary to make them configurable during the build 
process.I would go with this solution, since the whole process depends on the 
the links and will not likely be changed in near future.- Update the links in 
the file once and remove them from build process.
What do you think?
P.S. I moved the topic away from Ignite 2.7 release.

--
Roman 

On Friday, September 14, 2018, 7:22:27 p.m. GMT+9, Ilya Kasnacheev 
 wrote:  
 
 Hello!
So now there's an issue that this script makes source change after every build, 
show up in git status.
What we could do to it:- Commit the changes after the build, once. In hopes 
that it won't change very often. With benefit that we could do that right now, 
before the code freeze.
- Move these values to a properties file from both pom.xml and 
IgniteProvider.java. Any problems with this approach? We'll just read them from 
classpath properties file.
- Update the links in the file once and remove them from build process. Why 
were they added to build process in the first place - to make them configurable 
during build?

Regards,
-- 
Ilya Kasnacheev


вт, 11 сент. 2018 г. в 5:53, Roman Shtykh :

Ilya,
The "latest" version is the default, and resolved by 
https://ignite.apache.org/latest which is used by our web site when a user 
download the latest Ignite version. And I think this is the authority to judge 
of the latest official release (pom.xml you suggest can have SNAPSHOTs 
etc.).Also, as I explained during our review sessions, ignite-mesos-2.6.0 is a 
driver and doesn't mean you need to have Ignite 2.6.0. User can run any version 
of Ignite he/she specifies. By default, it's "latest" but a user can specify 
any version needed, even from a non-archive URL.
In short, what we have now1. mesos driver (ignite-mesos-x.x.x) will use 
"latest" version by default -> it will try to resolve the latest officially 
releases version of Apache Ignite, find the closest mirror and download Ignite 
in a minute. If the version resolution fails, we fall back to the slow apache 
archive (as you suggest; in my opinion we better fail-fast instead of waiting 
for hours to download, so the user can choose another download option (3))2. If 
the user specifies the version explicitly, it goes to the slow apache archive.
3. The user can put ignite zip file on his/her http server and provide the URL 
as a parameter to the driver, if options 1 and 2 don't work.
As you see, there are 3 options. And I just fix the 1st one with 
https://issues.apache.org/jira/browse/IGNITE-9388 and don't change the original 
logic (which I find reasonable) documented on our site -- I don't see how it 
blocks anything.

Roman Shtykh 

On Monday, September 10, 2018, 6:16:15 p.m. GMT+9, Ilya Kasnacheev 
 wrote:  
 
 Hello!
There's still two issues with the submission.
The first one is that we're downloading "latest" version from preferred mirror 
but a specified version, such as "2.6", we're also going to download from 
"slow" archive.apache.org/dist.That's a great limitation for this change, since 
most real deployments of Apache Ignite will have their Ignite version pegged to 
a specific release. But in this case there's no win in download speed.In my 
opinion it is a blocker.

The second one is that we can't download anything when we failed to resolve 
"latest". My idea is that we should try and download last known version in this 
case, which can be pushed to source from pom.xml, as we already do with URLs. 
So if you could not resolve "latest" you will download 2.7.0.
Buuut, maybe it's not necessary, maybe we should just discourage "latest", 
which is in my opinion almost always a bad idea.
WDYT?

Regards,
-- 
Ilya Kasnacheev


вс, 9 сент. 2018 г. в 5:47, Roman Shtykh :

Hi Ilya,
Sorry, missed that.
Added now.

--
Roman Shtykh 

On Thursday, September 6, 2018, 6:16:58 p.m. GMT+9, Ilya Kasnacheev 
 wrote:  
 
 Hello!
The last of my requests still standing is that we should fall-back to single 
URL download in case of error with 'latest' version. Everything else looks good 
to me.
Can we do that? I'm really worried that Apache API will go sour.

Regards,
-- 
Ilya Kasnacheev


чт, 6 сент. 2018 г. в 8:56, Roman Shtykh :

Hi Ilya,
Thanks again.
1) Done.2) Used catch() for latest version.

Please see my comments on github.
--
Roman Shtykh 

On Wednesday, September 5, 2018, 11:30:10 p.m. GMT+9, Ilya Kasnacheev 
 wrote:  
 
 Hello!
I've left a new wave of replies.
Basically, 1) let's keep DOWNLOAD_URL_PATTERN string value inlined so that it 
will work even if build process is broken (would be useful for e.g. developing 
out of IDE)
And also I urge you to catch() around new fragile Apache JSON API resolving, 
and download the 'current' version instead, as defined by ignite-mesos version.
This is because this module is not under continuouos scrutiny so extra care 
should be applied.-- 
Ilya Kasnacheev


вт, 4 сент. 2018 г. в 13:42, Roman Shtykh :

Re: Apache Ignite TeamCity Bot - wiki page available

[Dmitriy Pavlov]

Dmitriy, thank you for your efforts and for creating this page. A lot of
Igniters were asking about it.

ср, 26 сент. 2018 г. в 14:46, Dmitrii Ryabov :

> Hello, Igniters!
>
> You've all seen the messages with new failures from MTCGA.Bot.
>
> But the bot can also be used to analyze TeamCity runs and to print tests
> analysis results in the JIRA ticket.
>
> Now you can learn how to work with the bot at the wiki page [1].
>
> [1]
>
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot
>

[GitHub] ignite pull request #4773: IGNITE-9492 Ignore SingleMessage updated where th...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4773


---

Apache Ignite TeamCity Bot - wiki page available

[Dmitrii Ryabov]

Hello, Igniters!

You've all seen the messages with new failures from MTCGA.Bot.

But the bot can also be used to analyze TeamCity runs and to print tests
analysis results in the JIRA ticket.

Now you can learn how to work with the bot at the wiki page [1].

[1]
https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot

[jira] [Created] (IGNITE-9699) Web console: Mutual authentication on Web Console isn't supported

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-9699:
--

 Summary: Web console: Mutual authentication on Web Console isn't 
supported
 Key: IGNITE-9699
 URL: https://issues.apache.org/jira/browse/IGNITE-9699
 Project: Ignite
  Issue Type: New Feature
  Components: UI
Affects Versions: 2.6
Reporter: Andrey Aleksandrov
 Fix For: 2.8


Ignite has a possibility to configure SSL connection with Certificate 
Authority. But in the web console, we are able to use only one certificate used 
in web console agent. It could be a problem when a lot of users will use the 
same web console and have different certificates for validation.

We require to allow the user to pick his own certificate for SSL connection to 
the grid.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4836: Ignite 9683

[andrewmed]

GitHub user andrewmed opened a pull request:

https://github.com/apache/ignite/pull/4836

Ignite 9683



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite IGNITE-9683

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4836.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4836


commit a30682f9fc395ff88c84640effc9abd2515a79d5
Author: Pavel Kovalenko 
Date:   2018-09-11T11:40:42Z

Zookeeper custom pinger.

commit 231702f5ee81b4cfe6dff0617f6133f9c58485af
Author: AMedvedev 
Date:   2018-09-26T09:24:29Z

add ZooKeeper pinger configuration property

commit a4c4358f9a3803fcff171736737cdd71384036f5
Author: AMedvedev 
Date:   2018-09-26T09:55:36Z

codestyle




---

[GitHub] dspavlov opened a new pull request #19: DI refactoring

[GitBox]

dspavlov opened a new pull request #19: DI refactoring
URL: https://github.com/apache/ignite-teamcity-bot/pull/19
 
 
   Monitor observer tasks


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (IGNITE-9698) Add tests to control.sh with ssl authentication

[Sergey Kosarev (JIRA)]

Sergey Kosarev created IGNITE-9698:
--

 Summary: Add tests to  control.sh with ssl authentication
 Key: IGNITE-9698
 URL: https://issues.apache.org/jira/browse/IGNITE-9698
 Project: Ignite
  Issue Type: Improvement
Reporter: Sergey Kosarev
Assignee: Alexand Polyakov






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9697) [TC Bot] Autocomplete branch for TC field

[Ryabov Dmitrii (JIRA)]

Ryabov Dmitrii created IGNITE-9697:
--

 Summary: [TC Bot] Autocomplete branch for TC field
 Key: IGNITE-9697
 URL: https://issues.apache.org/jira/browse/IGNITE-9697
 Project: Ignite
  Issue Type: Bug
Reporter: Ryabov Dmitrii
Assignee: Ryabov Dmitrii


Get last updated PRs from GitHub and use them to autocomplete branch for TC 
fields.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4835: IGNITE-9695 WAL disabling prohibition in WalState...

[andrey-kuznetsov]

GitHub user andrey-kuznetsov opened a pull request:

https://github.com/apache/ignite/pull/4835

IGNITE-9695 WAL disabling prohibition in WalStateManager.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/andrey-kuznetsov/ignite ignite-9695

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4835.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4835


commit 48a00e37be36d3bab1291c852adf7709425551e9
Author: Andrey Kuznetsov 
Date:   2018-09-26T08:48:59Z

IGNITE-9695 WAL disabling prohibition in WalStateManager.




---

[GitHub] ignite pull request #4674: IGNITE-9419 Async wait for caches registration an...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4674


---

[GitHub] ignite pull request #4761: IGNITE-9558

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4761


---

[jira] [Created] (IGNITE-9696) SQL: Remove documentation about bad IN performance

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-9696:
---

 Summary: SQL: Remove documentation about bad IN performance
 Key: IGNITE-9696
 URL: https://issues.apache.org/jira/browse/IGNITE-9696
 Project: Ignite
  Issue Type: Task
  Components: documentation
Reporter: Vladimir Ozerov
Assignee: Artem Budnikov
 Fix For: 2.7


As IGNITE-4150 is merged, we should no longer have problems with IN statement. 
Let's remove it. 

See 
[https://apacheignite-sql.readme.io/docs/performance-and-debugging#sql-performance-and-usability-considerations]
 , point 2.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3956: IGNITE-4150

[devozerov]

Github user devozerov closed the pull request at:

https://github.com/apache/ignite/pull/3956


---

[GitHub] ignite pull request #4834: ignite-8507: Print a warning if SQL index inline ...

[ygerzhedovich]

GitHub user ygerzhedovich opened a pull request:

https://github.com/apache/ignite/pull/4834

ignite-8507: Print a warning if SQL index inline size is not sufficient.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8507

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4834.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4834






---

[GitHub] ignite pull request #4833: Ig 7196 retest

[Mmuzaf]

GitHub user Mmuzaf opened a pull request:

https://github.com/apache/ignite/pull/4833

Ig 7196 retest



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Mmuzaf/ignite ig-7196-retest

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4833.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4833


commit c352a51501cde064ff3e6e440a17d5b8bd70e405
Author: Maxim Muzafarov 
Date:   2018-09-26T07:22:56Z

IG-7196 run testDestroyCachesAbruptly 400 times

commit 41fe92daa38debd3a2c6df193061eb7d3216ea07
Author: Maxim Muzafarov 
Date:   2018-09-26T07:24:58Z

IG-7196 run testDestroyCachesAbruptly 400 times, more comment




---