Re: Grid hang on compute

[Dmitry Karachentsev]

Thanks!

Opened a ticket to support Yakov's proposal.
https://issues.apache.org/jira/browse/IGNITE-4395

On 08.12.2016 4:03, Dmitriy Setrakyan wrote:

Is there any way we can detect this and prevent from happening? Or perhaps
start rejecting jobs if they can potentially block the system?

On Wed, Dec 7, 2016 at 8:11 AM, Yakov Zhdanov  wrote:


Proper solution here is to have communication backpressure per policy -
SYSTEM or PUBLIC, but not single point as it is now. I think we can achieve
this having two queues per communication session or (which looks a bit
easier to implement) to have separate connections.

As a workaround you can increase the limit. Setting it to 0 may lead to a
potential OOME on sender or receiver sides.

--Yakov

2016-12-07 20:35 GMT+07:00 Dmitry Karachentsev 

[GitHub] ignite pull request #1331: Ignite 4395

[dkarachentsev]

GitHub user dkarachentsev opened a pull request:

https://github.com/apache/ignite/pull/1331

Ignite 4395



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4395

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1331.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1331


commit 37f22893e4aaf7f398928b95575a64b6280ab635
Author: tledkov-gridgain 
Date:   2016-08-10T09:21:13Z

Merge remote-tracking branch 'remotes/community/ignite-1.7.2'

commit a519c22cff94c8dfda23f8aa52ae0452f6eee260
Author: dkarachentsev 
Date:   2016-08-25T10:38:53Z

Merge branch 'master' of https://github.com/gridgain/apache-ignite

commit ca1381facd9a83b723fbeaed85a76672b05e5d25
Author: EdShangGG 
Date:   2016-08-30T11:53:46Z

Merge remote-tracking branch 'ignite-gg/ignite-1.7.2' into gg-master

commit ce94df568d278105f9132ab1a02b666ffcddacf5
Author: dkarachentsev 
Date:   2016-08-31T12:57:45Z

Merge branch 'master' of https://github.com/gridgain/apache-ignite

commit 1c8ac3161cf4474086ce6b66e0b0ac296f7a0ae4
Author: dkarachentsev 
Date:   2016-09-09T13:18:28Z

Merge branch 'master' of https://github.com/gridgain/apache-ignite

commit dd5db4669ecba81e1f4d9592e264175cb5f3616a
Author: dkarachentsev 
Date:   2016-09-12T07:54:20Z

Merge branch 'ignite-1.7.2'

commit c115bfefc1bc9294c8e61b7ee7932d92ec0661d8
Author: dkarachentsev 
Date:   2016-09-14T12:01:54Z

Merge branch 'master' of https://github.com/gridgain/apache-ignite

commit 7d50cd6f166c2d2ee77b7de150f1b2547f6efb63
Author: dkarachentsev 
Date:   2016-09-23T07:45:36Z

Merge branch 'master' of https://github.com/gridgain/apache-ignite

commit 720cf4a5d3fedc8c849e3a5c531f6a6f3d3d
Author: dkarachentsev 
Date:   2016-11-14T14:23:57Z

Merge branch 'master' of https://github.com/gridgain/apache-ignite

commit bfb00b6e61f9709718c30971997aeb0ac79e86b4
Author: Alexandr Kuramshin 
Date:   2016-11-18T20:12:28Z

IgniteTcpCommunicationBigClusterTest added

commit 02dd92e605b9b53f5a16c7ec5f8e7b5698b15ba4
Author: Alexandr Kuramshin 
Date:   2016-11-18T21:55:37Z

IgniteTcpCommunicationBigClusterTest update

commit 6acf193a3d356d1bad4c02a53ac76833ed1008d0
Author: Alexandr Kuramshin 
Date:   2016-11-19T09:55:45Z

Have got TcpCommunicationSpi error

commit 4fd39653d24f62f19f70b4dffba8497185cc46fb
Author: Alexandr Kuramshin 
Date:   2016-11-19T16:39:10Z

Some discovery have been done

commit c2c181922c7c24ea457577e32d2af897c8bec87f
Author: Alexandr Kuramshin 
Date:   2016-11-19T20:11:28Z

Prove that problem is not in the onFirstMessage hang

commit f8076edba097f6077229b2090ee3ff1a3369878c
Author: Alexandr Kuramshin 
Date:   2016-11-19T20:26:37Z

Revert: Prove that problem is not in the onFirstMessage hang

commit 6e1f2dfc2acb3dbb8f24aa51ed67b2ee447b4585
Author: Alexandr Kuramshin 
Date:   2016-11-21T08:55:09Z

Revert: pushing unnecessary changes to the master

commit ed794ca815f6bb1471af15779279d287576b39cc
Author: Alexandr Kuramshin 
Date:   2016-11-21T09:08:00Z

Revert: pushing unnecessary changes to the master

commit 1ce39beaa8fd9d90a450d0e8ec986b5dc115c721
Author: dkarachentsev 
Date:   2016-12-02T09:41:17Z

Merge branch 'master' of https://github.com/gridgain/apache-ignite

commit e919ae57a6226d0e3d12b2c3c2c9bdfe6d2e1da4
Author: dkarachentsev 
Date:   2016-12-08T07:31:18Z

IGNITE-4395 - Test that leads to grid hang.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-4395) Implement communication backpressure per policy - SYSTEM or PUBLIC

[Dmitry Karachentsev (JIRA)]

Dmitry Karachentsev created IGNITE-4395:
---

 Summary: Implement communication backpressure per policy - SYSTEM 
or PUBLIC
 Key: IGNITE-4395
 URL: https://issues.apache.org/jira/browse/IGNITE-4395
 Project: Ignite
  Issue Type: Improvement
  Components: cache, compute
Affects Versions: 1.7
Reporter: Dmitry Karachentsev


1) Start two data nodes with some cache.
2) From one node in async mode post some big number of jobs to another. That 
jobs do some cache operations.
3) Grid hangs almost immediately and all threads are sleeping except public 
ones, they are waiting for response.

This happens because all cache and job messages are queued on communication and 
limited with default number (1024). It looks like jobs are waiting for cache 
responses that could not be received due to this limit.

Proper solution here is to have communication backpressure per policy -
SYSTEM or PUBLIC, but not single point as it is now. It could be achieved
with having two queues per communication session or (which looks a bit
easier to implement) to have separate connections.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [VOTE] Apache Ignite 1.8.0 RC1

[Yakov Zhdanov]

+1 (binding)

--Yakov

2016-12-05 22:22 GMT+07:00 Vladimir Ozerov :

> Hello,
>
> We have uploaded the 1.8.0 release candidate to
> *https://dist.apache.org/repos/dist/dev/ignite/1.8.0-rc1/
> *
>
> Tag name is
> 1.8.0-rc1
>
> This release includes the following changes:
>
> Ignite:
> * SQL: Added DML operations support (INSERT, UPDATE, DELETE, MERGE)
> * SQL: Improved DISTINCT keyword handling in aggregates
> * Hadoop: Added MapR distribution support
> * Visor: Improved SQL statistics
> * Added Redis protocol support
> * Added transactions deadlock detection
> * Many stability and fault-tolerance fixes
>
> Ignite.NET:
> * ASP.NET session state store provider
> * Entity Framework second level cache
> * Custom loggers support: NLog, Apache log4Net
>
> ODBC driver:
> * Added DML operations support
> * Added distributed joins support
> * Added DSN support
> * Performance improvements
>
> Complete list of closed issues:
> https://issues.apache.org/jira/issues/?jql=
> *project%20%3D%20IGNITE%20AND%20fixVersion%20%3D%201.8%
> 20AND%20status%20in%20(resolved%2C%20closed)*
>
> DEVNOTES
> *https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=
> blob_plain;f=DEVNOTES.txt;hb=refs/tags/1.8.0-rc1
>  blob_plain;f=DEVNOTES.txt;hb=refs/tags/1.8.0-rc1>*
>
> RELEASE NOTES
> *https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=
> blob_plain;f=RELEASE_NOTES.txt;hb=refs/tags/1.8.0-rc1
>  blob_plain;f=RELEASE_NOTES.txt;hb=refs/tags/1.8.0-rc1>*
>
> Please start voting.
>
> +1 - to accept Apache Ignite 1.8.0-rc1
> 0 - don't care either way
> -1 - DO NOT accept Apache Ignite 1.8.0-rc1 (explain why)
>
> This vote will go for 72 hours.
>
> Vladimir.
>

[jira] [Created] (IGNITE-4394) Web console: memory leak when dialog "Connection to Ignite Node is not established" on the screen

[Pavel Konstantinov (JIRA)]

Pavel Konstantinov created IGNITE-4394:
--

 Summary: Web console: memory leak when dialog "Connection to 
Ignite Node is not established" on the screen
 Key: IGNITE-4394
 URL: https://issues.apache.org/jira/browse/IGNITE-4394
 Project: Ignite
  Issue Type: Bug
  Components: wizards
Reporter: Pavel Konstantinov


I've noticed memory leak in case when dialog is opened during long period.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] ignite pull request #1330: IGNITE-4387 Fix incorrect Karaf OSGI feature for ...

[SergiusSidorov]

GitHub user SergiusSidorov opened a pull request:

https://github.com/apache/ignite/pull/1330

IGNITE-4387 Fix incorrect Karaf OSGI feature for ignite-rest-http

IGNITE-4387 Fix

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/SergiusSidorov/ignite ignite-4387

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1330.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1330


commit 2b7e5766290c2f3ea666336b8ce86f8aaed38d1d
Author: Sergej Sidorov 
Date:   2016-12-08T06:32:00Z

IGNITE-4387 Fix incorrect Karaf OSGI feature for ignite-rest-http




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Major release 2.0 and compilation

[Rohit Mohta]

Agreed. Was just thinking from end user perspective - as Ignite is gaining 
traction and teams are building application, a news like breaking compatibility 
can cause some concerns. Something similar to AngularJS when they declared 
similar news.

 Regardless of the approach we take, excited about 2.0. 

Sent from my iPhone

> On Dec 7, 2016, at 19:56, Alexey Kuznetsov  wrote:
> 
> +1 for removing deprecated stuff in 2.0 and provide "Migration guide".
> 
> In any case we are going to break compatibility.
> 
> -- 
> Alexey Kuznetsov

[jira] [Created] (IGNITE-4393) Need to remove code putting cache operations to sequence

[Yakov Zhdanov (JIRA)]

Yakov Zhdanov created IGNITE-4393:
-

 Summary: Need to remove code putting cache operations to sequence
 Key: IGNITE-4393
 URL: https://issues.apache.org/jira/browse/IGNITE-4393
 Project: Ignite
  Issue Type: Improvement
Reporter: Yakov Zhdanov
Assignee: Semen Boikov
 Fix For: 1.9


Currently Ignite guarantees that async operations started from the same thread 
are applied in the same order. This code will be literally useless once we move 
to thread per partition model.

org.apache.ignite.internal.processors.cache.GridCacheAdapter#toggleAsync
org.apache.ignite.internal.processors.cache.GridCacheAdapter#asyncOpAcquire
org.apache.ignite.internal.processors.cache.GridCacheAdapter#asyncOpRelease
org.apache.ignite.internal.processors.cache.GridCacheAdapter#asyncOpsSem
org.apache.ignite.internal.processors.cache.GridCacheAdapter#lastFut
org.apache.ignite.internal.processors.cache.GridCacheAdapter.FutureHolder 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4392) Web Console: Implement support of management role

[Alexey Kuznetsov (JIRA)]

Alexey Kuznetsov created IGNITE-4392:


 Summary: Web Console: Implement support of management role
 Key: IGNITE-4392
 URL: https://issues.apache.org/jira/browse/IGNITE-4392
 Project: Ignite
  Issue Type: Task
  Components: wizards
Affects Versions: 1.8
Reporter: Alexey Kuznetsov
 Fix For: 1.9


In current implementation we have following roles: user and admins.
We need one more role: manager - this role will allow to view admin panel, but 
all other actions: delete users and giving admin rights to other users should 
be disabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4391) Web Console: Implement grouping by company in Admin panel

[Alexey Kuznetsov (JIRA)]

Alexey Kuznetsov created IGNITE-4391:


 Summary: Web Console: Implement grouping by company in Admin panel
 Key: IGNITE-4391
 URL: https://issues.apache.org/jira/browse/IGNITE-4391
 Project: Ignite
  Issue Type: Task
  Components: wizards
Affects Versions: 1.8
Reporter: Alexey Kuznetsov
 Fix For: 1.9


It will be useful to have different view on same data:
# list by name (current)
# group by company
# group by country



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Major release 2.0 and compilation

[Alexey Kuznetsov]

+1 for removing deprecated stuff in 2.0 and provide "Migration guide".

In any case we are going to break compatibility.

-- 
Alexey Kuznetsov

Re: Major release 2.0 and compilation

[Roman Shtykh]

Rohit,

I don't think this is a very clean approach to maintain multiple copies.

+1 for removing deprecated stuff in 2.0 with some exclusions and thoroughly 
documenting and sharing the changes as it was proposed above.

-Roman




On Thursday, December 8, 2016 10:47 AM, Rohit Mohta  
wrote:
Another thought

If there is going to be a breaking change to public API like remove deprecated 
method or  change signature for a public API
  - create a copy of this class, possibly keeping the same class name but 
change the package. Let's say we change the package name from org.ignite.cache 
to org.ignite.cache.ver2 or similar
  - move this class as-is to another module named ignite-deprecated or 
ignite-compatibility-1.x or similar
  - developers can *strongly try* to keep easy upgrade by extending ver2.class. 
The class in deprecated/compatibility will probably only have the public API to 
avoid compilation error and leverage most of the processing from ver2 class

We as developers probably maintain the deprecated/compatibility branch till 2.0 
is GA or a bit longer. And going forward, we can accept PR from others but 
developers cannot guarantee we will keep the compatibility latest.

This is a thought - and kinda practical- we have used it within our application 
for similar breaking changes. 



Sent from my iPhone


> On Dec 7, 2016, at 15:04, Denis Magda  wrote:
> 
> I would remove as much as possible and prepare a migration guide as Sergey K. 
> suggests below.
> 
> In any case, we will stick to the flexible approach. As the next step I would 
> split all the existed APIs in 2 groups. The first group will be for the APIs 
> that will be deleted and we will provide instructions in the migration guide 
> on how to migrate from them. The second group will be for those that will be 
> left.
> 
> Actually, there is an already existed ticket for this
> https://issues.apache.org/jira/browse/IGNITE-3469 
> 
> 
> —
> Denis
> 
>> On Dec 7, 2016, at 10:12 AM, Sergey Kozlov  wrote:
>> 
>> I would remind that Apache Ignite 1.0.0 has been released 20 months ago and
>> probably 2.0 will be rolled out close to the second anniversary of initial
>> release. It's right time to remove deprecated API and provide for users the
>> clear ways for migration 1.x to 2.0.
>> 
>> I think we could create a wiki page for users who can recompile their
>> applications, list all deprecated API calls and provide migration's
>> tips/tricks from deprecated features to new ones (something like the table
>> with columns "Deprecated API call for 1.x", "API call for 2.0/workaround").
>> 
>> 
>> 
>>> On Wed, Dec 7, 2016 at 11:36 AM, Yakov Zhdanov  wrote:
>>> 
>>> Agree with Vladimir that we should use flexible approach and should avoid
>>> any possible harm here, but cleaning out most of deprecations is the way to
>>> go, IMO. There are about 90 occurrences of "deprecated" in the project and
>>> most of them are not very hard to remove. Moreover, we have, for example,
>>> setRemoteFilter(CacheEntryEventSerializableFilter). Using this
>>> method
>>> is error prone. We should remove it so none can use it.
>>> 
>>> Cleaning up and renovating are good. This allows us to see what can be done
>>> better and also encourages users to move forward.
>>> 
>>> --Yakov
>>> 
>>> 2016-12-07 15:22 GMT+07:00 Vladimir Ozerov :
>>> 
 Igniters,
 
 Release Apache Ignite 2.0 is already planned and being discussed.
>>> Normally
 we do not brake compilation between minor releases, and if some method on
 public API should not be used anymore we mark it as *deprecated*. But we
>>> do
 not have such a rule for major releases. The question is whether we are
 going to break compilation and remove deprecated methods from public API
>>> in
 Apache ignite 2.0 or not. There are several extreme approaches to this.
 
 First, we can declare that we cleanup all deprecations and remove them.
 This will result in clean and consistent API and simplify further
 development. But it might slowdown adoption of Apache Ignite 2.0 because
 users will be reluctant switching to newer version because they will have
 to fix compilation, re-deploy their apps, etc..
 
 Second, we can say that we must avoid breaking compilation at all costs
>>> and
 retain deprecated methods. This approach might be better for users, for
 harder to maintain for Ignite developers. With this approach users will
 migrate to 2.0 quicker.
 
 My opinion is that we must choose flexible approach and decide whether to
 keep deprecation or not separately for every piece of API, depending on
 possible impact on both users and Ignite developers. But normally I would
 leave deprecation unless there is a strong reason to remove it.
 
 Thoughts?
 
 Vladimir.

DML Documentation Readiness

[Denis Magda]

Alexander,

How close are you to the finalization of the DML doc?
https://apacheignite.readme.io/docs/distributed-dml 


Since we’re approaching 1.8 release I would like to do a final review of it 
polishing whatever is needed tomorrow. 

As I see that only the limitations section is left. The first note is that I 
wouldn’t mention “JDBC Batching mode” for now because this is something that 
doesn’t affect the usability and SQL API's scope of support. 

—
Denis

Re: Major release 2.0 and compilation

[Rohit Mohta]

Another thought

If there is going to be a breaking change to public API like remove deprecated 
method or  change signature for a public API
  - create a copy of this class, possibly keeping the same class name but 
change the package. Let's say we change the package name from org.ignite.cache 
to org.ignite.cache.ver2 or similar
  - move this class as-is to another module named ignite-deprecated or 
ignite-compatibility-1.x or similar
  - developers can *strongly try* to keep easy upgrade by extending ver2.class. 
The class in deprecated/compatibility will probably only have the public API to 
avoid compilation error and leverage most of the processing from ver2 class

We as developers probably maintain the deprecated/compatibility branch till 2.0 
is GA or a bit longer. And going forward, we can accept PR from others but 
developers cannot guarantee we will keep the compatibility latest.

This is a thought - and kinda practical- we have used it within our application 
for similar breaking changes. 



Sent from my iPhone

> On Dec 7, 2016, at 15:04, Denis Magda  wrote:
> 
> I would remove as much as possible and prepare a migration guide as Sergey K. 
> suggests below.
> 
> In any case, we will stick to the flexible approach. As the next step I would 
> split all the existed APIs in 2 groups. The first group will be for the APIs 
> that will be deleted and we will provide instructions in the migration guide 
> on how to migrate from them. The second group will be for those that will be 
> left.
> 
> Actually, there is an already existed ticket for this
> https://issues.apache.org/jira/browse/IGNITE-3469 
> 
> 
> —
> Denis
> 
>> On Dec 7, 2016, at 10:12 AM, Sergey Kozlov  wrote:
>> 
>> I would remind that Apache Ignite 1.0.0 has been released 20 months ago and
>> probably 2.0 will be rolled out close to the second anniversary of initial
>> release. It's right time to remove deprecated API and provide for users the
>> clear ways for migration 1.x to 2.0.
>> 
>> I think we could create a wiki page for users who can recompile their
>> applications, list all deprecated API calls and provide migration's
>> tips/tricks from deprecated features to new ones (something like the table
>> with columns "Deprecated API call for 1.x", "API call for 2.0/workaround").
>> 
>> 
>> 
>>> On Wed, Dec 7, 2016 at 11:36 AM, Yakov Zhdanov  wrote:
>>> 
>>> Agree with Vladimir that we should use flexible approach and should avoid
>>> any possible harm here, but cleaning out most of deprecations is the way to
>>> go, IMO. There are about 90 occurrences of "deprecated" in the project and
>>> most of them are not very hard to remove. Moreover, we have, for example,
>>> setRemoteFilter(CacheEntryEventSerializableFilter). Using this
>>> method
>>> is error prone. We should remove it so none can use it.
>>> 
>>> Cleaning up and renovating are good. This allows us to see what can be done
>>> better and also encourages users to move forward.
>>> 
>>> --Yakov
>>> 
>>> 2016-12-07 15:22 GMT+07:00 Vladimir Ozerov :
>>> 
 Igniters,
 
 Release Apache Ignite 2.0 is already planned and being discussed.
>>> Normally
 we do not brake compilation between minor releases, and if some method on
 public API should not be used anymore we mark it as *deprecated*. But we
>>> do
 not have such a rule for major releases. The question is whether we are
 going to break compilation and remove deprecated methods from public API
>>> in
 Apache ignite 2.0 or not. There are several extreme approaches to this.
 
 First, we can declare that we cleanup all deprecations and remove them.
 This will result in clean and consistent API and simplify further
 development. But it might slowdown adoption of Apache Ignite 2.0 because
 users will be reluctant switching to newer version because they will have
 to fix compilation, re-deploy their apps, etc..
 
 Second, we can say that we must avoid breaking compilation at all costs
>>> and
 retain deprecated methods. This approach might be better for users, for
 harder to maintain for Ignite developers. With this approach users will
 migrate to 2.0 quicker.
 
 My opinion is that we must choose flexible approach and decide whether to
 keep deprecation or not separately for every piece of API, depending on
 possible impact on both users and Ignite developers. But normally I would
 leave deprecation unless there is a strong reason to remove it.
 
 Thoughts?
 
 Vladimir.
 
>>> 
>> 
>> 
>> 
>> -- 
>> Sergey Kozlov
>> GridGain Systems
>> www.gridgain.com
> 

Re: affinityCall in one distributed transaction

[Dmitriy Setrakyan]

Taras, is invokeAll() transactional? The javadoc is silent to this fact. If
it is indeed transactional, then we should update the javadoc.

D.

On Wed, Dec 7, 2016 at 5:32 AM, Taras Ledkov  wrote:

> Ignite compute has no relation to the cache's transaction.
>
> I think that IgniteCache.invokeAll() is appropriate for described case.
>
> On Wed, Dec 7, 2016 at 4:00 PM, Игорь Г  wrote:
>
> > Hi, igniters!
> >
> > Before openning JIRA ticket, I want to ask question about affinityCall or
> > affinityRun transactions.
> >
> > For example I have batch task to modify many values in someCache
> according
> > to someRule. I want to parallel this task to whole cluster and minimize
> > network traffic.
> > So the resonable choice is affinityCall feature.
> >
> > But I want all this changes to be in one transactoin. i.e. with at least
> > atomicity property (of ACID). And if for some reason my task will be
> > canceled or failed on one node - it should change nothing at all.
> >
> > So, can I achieve this with existing functionality, or how can I approach
> > to this task?
> >
>

Re: Grid hang on compute

[Dmitriy Setrakyan]

Is there any way we can detect this and prevent from happening? Or perhaps
start rejecting jobs if they can potentially block the system?

On Wed, Dec 7, 2016 at 8:11 AM, Yakov Zhdanov  wrote:

> Proper solution here is to have communication backpressure per policy -
> SYSTEM or PUBLIC, but not single point as it is now. I think we can achieve
> this having two queues per communication session or (which looks a bit
> easier to implement) to have separate connections.
>
> As a workaround you can increase the limit. Setting it to 0 may lead to a
> potential OOME on sender or receiver sides.
>
> --Yakov
>
> 2016-12-07 20:35 GMT+07:00 Dmitry Karachentsev  >:
>
> > Igniters!
> >
> > Recently faced with arguable issue, it looks like a bug. Scenario is
> > following:
> >
> > 1) Start two data nodes with some cache.
> >
> > 2) From one node in async mode post some big number of jobs to another.
> > That jobs do some cache operations.
> >
> > 3) Grid hangs almost immediately and all threads are sleeping except
> > public ones, they are waiting for response.
> >
> > This happens because all cache and job messages are queued on
> > communication and limited with default number (1024). It looks like jobs
> > are waiting for cache responses that could not be received due to this
> > limit. It's hard to diagnose and looks not convenient (as I know we have
> no
> > limitation in docs for using cache ops from compute jobs).
> >
> > So, my question is. Should we try to solve that or, may be, it's enough
> to
> > update documentation with recommendation to disable queue limit for such
> > cases?
> >
> >
>

Re: Major release 2.0 and compilation

[Denis Magda]

I would remove as much as possible and prepare a migration guide as Sergey K. 
suggests below.

In any case, we will stick to the flexible approach. As the next step I would 
split all the existed APIs in 2 groups. The first group will be for the APIs 
that will be deleted and we will provide instructions in the migration guide on 
how to migrate from them. The second group will be for those that will be left.

Actually, there is an already existed ticket for this
https://issues.apache.org/jira/browse/IGNITE-3469 


—
Denis

> On Dec 7, 2016, at 10:12 AM, Sergey Kozlov  wrote:
> 
> I would remind that Apache Ignite 1.0.0 has been released 20 months ago and
> probably 2.0 will be rolled out close to the second anniversary of initial
> release. It's right time to remove deprecated API and provide for users the
> clear ways for migration 1.x to 2.0.
> 
> I think we could create a wiki page for users who can recompile their
> applications, list all deprecated API calls and provide migration's
> tips/tricks from deprecated features to new ones (something like the table
> with columns "Deprecated API call for 1.x", "API call for 2.0/workaround").
> 
> 
> 
> On Wed, Dec 7, 2016 at 11:36 AM, Yakov Zhdanov  wrote:
> 
>> Agree with Vladimir that we should use flexible approach and should avoid
>> any possible harm here, but cleaning out most of deprecations is the way to
>> go, IMO. There are about 90 occurrences of "deprecated" in the project and
>> most of them are not very hard to remove. Moreover, we have, for example,
>> setRemoteFilter(CacheEntryEventSerializableFilter). Using this
>> method
>> is error prone. We should remove it so none can use it.
>> 
>> Cleaning up and renovating are good. This allows us to see what can be done
>> better and also encourages users to move forward.
>> 
>> --Yakov
>> 
>> 2016-12-07 15:22 GMT+07:00 Vladimir Ozerov :
>> 
>>> Igniters,
>>> 
>>> Release Apache Ignite 2.0 is already planned and being discussed.
>> Normally
>>> we do not brake compilation between minor releases, and if some method on
>>> public API should not be used anymore we mark it as *deprecated*. But we
>> do
>>> not have such a rule for major releases. The question is whether we are
>>> going to break compilation and remove deprecated methods from public API
>> in
>>> Apache ignite 2.0 or not. There are several extreme approaches to this.
>>> 
>>> First, we can declare that we cleanup all deprecations and remove them.
>>> This will result in clean and consistent API and simplify further
>>> development. But it might slowdown adoption of Apache Ignite 2.0 because
>>> users will be reluctant switching to newer version because they will have
>>> to fix compilation, re-deploy their apps, etc..
>>> 
>>> Second, we can say that we must avoid breaking compilation at all costs
>> and
>>> retain deprecated methods. This approach might be better for users, for
>>> harder to maintain for Ignite developers. With this approach users will
>>> migrate to 2.0 quicker.
>>> 
>>> My opinion is that we must choose flexible approach and decide whether to
>>> keep deprecation or not separately for every piece of API, depending on
>>> possible impact on both users and Ignite developers. But normally I would
>>> leave deprecation unless there is a strong reason to remove it.
>>> 
>>> Thoughts?
>>> 
>>> Vladimir.
>>> 
>> 
> 
> 
> 
> -- 
> Sergey Kozlov
> GridGain Systems
> www.gridgain.com

Re: What branch to use for Apache Ignite 2.0 development?

[Denis Magda]

What’s bad about this approach? 

Probably we can continue all development in *master*, and in case if we
decide to release another 1.x release, then create branch from ignite-1.8
tag and cherry-pick required fixes there.

Personally, it looks much more flexible to me. We can develop everything 
directly in master preparing it for 2.0 and avoiding management of several 
branches. 

Imagine that we’ve already released 2.0 and the master became 2.0 only after 
that event. But later on the community decides to release 1.9, 1.10 or other 
version and we will have to branch from 1.8 rather than from the master. So, 
it’s not feasible to predict everything but if we’re agreed that the next 
version will be 2.0 then it’s ok to develop in the master since there is 1.8 
branch that can be used for unpredictable situations.

—
Denis



> On Dec 7, 2016, at 2:04 AM, Yakov Zhdanov  wrote:
> 
>> Master must be periodically merged to 2.0. Looks simple enough, no?
> 
> This may not be always possible and may require to re-implement some
> features. However, I vote for this approach until it causes problems. Once
> we start to spend more time on merging, master should become 2.0
> development branch immediately.
> 
> --Yakov

Re: Major release 2.0 and compilation

[Dmitriy Setrakyan]

I would vote for removing the deprecated APIs, unless it may be absolutely
harmful to the community. For example, if we have deprecated configuration
properties, I would definitely remove them and force users to update to the
new configuration.

However, if there is some deprecation in a compute method, or, maybe,
cache.put(...) method, then removing such methods will be plain harmful. I
would keep them till Ignite 3.0 and print out a warning in the log that
this deprecated method will be removed in the next version of Ignite.

D.

On Wed, Dec 7, 2016 at 10:12 AM, Sergey Kozlov  wrote:

> I would remind that Apache Ignite 1.0.0 has been released 20 months ago and
> probably 2.0 will be rolled out close to the second anniversary of initial
> release. It's right time to remove deprecated API and provide for users the
> clear ways for migration 1.x to 2.0.
>
> I think we could create a wiki page for users who can recompile their
> applications, list all deprecated API calls and provide migration's
> tips/tricks from deprecated features to new ones (something like the table
> with columns "Deprecated API call for 1.x", "API call for 2.0/workaround").
>
>
>
> On Wed, Dec 7, 2016 at 11:36 AM, Yakov Zhdanov 
> wrote:
>
> > Agree with Vladimir that we should use flexible approach and should avoid
> > any possible harm here, but cleaning out most of deprecations is the way
> to
> > go, IMO. There are about 90 occurrences of "deprecated" in the project
> and
> > most of them are not very hard to remove. Moreover, we have, for example,
> > setRemoteFilter(CacheEntryEventSerializableFilter). Using this
> > method
> > is error prone. We should remove it so none can use it.
> >
> > Cleaning up and renovating are good. This allows us to see what can be
> done
> > better and also encourages users to move forward.
> >
> > --Yakov
> >
> > 2016-12-07 15:22 GMT+07:00 Vladimir Ozerov :
> >
> > > Igniters,
> > >
> > > Release Apache Ignite 2.0 is already planned and being discussed.
> > Normally
> > > we do not brake compilation between minor releases, and if some method
> on
> > > public API should not be used anymore we mark it as *deprecated*. But
> we
> > do
> > > not have such a rule for major releases. The question is whether we are
> > > going to break compilation and remove deprecated methods from public
> API
> > in
> > > Apache ignite 2.0 or not. There are several extreme approaches to this.
> > >
> > > First, we can declare that we cleanup all deprecations and remove them.
> > > This will result in clean and consistent API and simplify further
> > > development. But it might slowdown adoption of Apache Ignite 2.0
> because
> > > users will be reluctant switching to newer version because they will
> have
> > > to fix compilation, re-deploy their apps, etc..
> > >
> > > Second, we can say that we must avoid breaking compilation at all costs
> > and
> > > retain deprecated methods. This approach might be better for users, for
> > > harder to maintain for Ignite developers. With this approach users will
> > > migrate to 2.0 quicker.
> > >
> > > My opinion is that we must choose flexible approach and decide whether
> to
> > > keep deprecation or not separately for every piece of API, depending on
> > > possible impact on both users and Ignite developers. But normally I
> would
> > > leave deprecation unless there is a strong reason to remove it.
> > >
> > > Thoughts?
> > >
> > > Vladimir.
> > >
> >
>
>
>
> --
> Sergey Kozlov
> GridGain Systems
> www.gridgain.com
>

Re: Major release 2.0 and compilation

[Sergey Kozlov]

I would remind that Apache Ignite 1.0.0 has been released 20 months ago and
probably 2.0 will be rolled out close to the second anniversary of initial
release. It's right time to remove deprecated API and provide for users the
clear ways for migration 1.x to 2.0.

I think we could create a wiki page for users who can recompile their
applications, list all deprecated API calls and provide migration's
tips/tricks from deprecated features to new ones (something like the table
with columns "Deprecated API call for 1.x", "API call for 2.0/workaround").



On Wed, Dec 7, 2016 at 11:36 AM, Yakov Zhdanov  wrote:

> Agree with Vladimir that we should use flexible approach and should avoid
> any possible harm here, but cleaning out most of deprecations is the way to
> go, IMO. There are about 90 occurrences of "deprecated" in the project and
> most of them are not very hard to remove. Moreover, we have, for example,
> setRemoteFilter(CacheEntryEventSerializableFilter). Using this
> method
> is error prone. We should remove it so none can use it.
>
> Cleaning up and renovating are good. This allows us to see what can be done
> better and also encourages users to move forward.
>
> --Yakov
>
> 2016-12-07 15:22 GMT+07:00 Vladimir Ozerov :
>
> > Igniters,
> >
> > Release Apache Ignite 2.0 is already planned and being discussed.
> Normally
> > we do not brake compilation between minor releases, and if some method on
> > public API should not be used anymore we mark it as *deprecated*. But we
> do
> > not have such a rule for major releases. The question is whether we are
> > going to break compilation and remove deprecated methods from public API
> in
> > Apache ignite 2.0 or not. There are several extreme approaches to this.
> >
> > First, we can declare that we cleanup all deprecations and remove them.
> > This will result in clean and consistent API and simplify further
> > development. But it might slowdown adoption of Apache Ignite 2.0 because
> > users will be reluctant switching to newer version because they will have
> > to fix compilation, re-deploy their apps, etc..
> >
> > Second, we can say that we must avoid breaking compilation at all costs
> and
> > retain deprecated methods. This approach might be better for users, for
> > harder to maintain for Ignite developers. With this approach users will
> > migrate to 2.0 quicker.
> >
> > My opinion is that we must choose flexible approach and decide whether to
> > keep deprecation or not separately for every piece of API, depending on
> > possible impact on both users and Ignite developers. But normally I would
> > leave deprecation unless there is a strong reason to remove it.
> >
> > Thoughts?
> >
> > Vladimir.
> >
>



-- 
Sergey Kozlov
GridGain Systems
www.gridgain.com

Re: Grid hang on compute

[Yakov Zhdanov]

Proper solution here is to have communication backpressure per policy -
SYSTEM or PUBLIC, but not single point as it is now. I think we can achieve
this having two queues per communication session or (which looks a bit
easier to implement) to have separate connections.

As a workaround you can increase the limit. Setting it to 0 may lead to a
potential OOME on sender or receiver sides.

--Yakov

2016-12-07 20:35 GMT+07:00 Dmitry Karachentsev :

> Igniters!
>
> Recently faced with arguable issue, it looks like a bug. Scenario is
> following:
>
> 1) Start two data nodes with some cache.
>
> 2) From one node in async mode post some big number of jobs to another.
> That jobs do some cache operations.
>
> 3) Grid hangs almost immediately and all threads are sleeping except
> public ones, they are waiting for response.
>
> This happens because all cache and job messages are queued on
> communication and limited with default number (1024). It looks like jobs
> are waiting for cache responses that could not be received due to this
> limit. It's hard to diagnose and looks not convenient (as I know we have no
> limitation in docs for using cache ops from compute jobs).
>
> So, my question is. Should we try to solve that or, may be, it's enough to
> update documentation with recommendation to disable queue limit for such
> cases?
>
>

[GitHub] ignite pull request #1317: IGNITE-3292 Yardstick: add logging of preloading ...

[oleg-ostanin]

Github user oleg-ostanin closed the pull request at:

https://github.com/apache/ignite/pull/1317


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1328: Ignite 4003 1.7

[agura]

GitHub user agura opened a pull request:

https://github.com/apache/ignite/pull/1328

Ignite 4003 1.7



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/agura/incubator-ignite ignite-4003-1.7

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1328.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1328


commit 97bfee4dff807e3049b61fa473472a8395cdcb6a
Author: vozerov-gridgain 
Date:   2016-09-27T07:06:48Z

Fixing RAT.

commit 0dac458d2aa36b03986412726bd607877e53aa4f
Author: vozerov-gridgain 
Date:   2016-09-27T07:07:03Z

Merge remote-tracking branch 'upstream/ignite-1.6.9' into ignite-1.6.9

commit 68b0bcd83c295ce540aa9d9d0910abcf671671df
Author: Pavel Tupitsyn 
Date:   2016-09-27T09:08:46Z

IGNITE-3970 .NET: Fix Cyrillic 'C' letters in code

commit 41df266a232afd86ade91c5e18f082206913007c
Author: Pavel Tupitsyn 
Date:   2016-09-27T09:14:01Z

Merge remote-tracking branch 'remotes/community/ignite-1.6.9' into 
ignite-1.7.2

commit 48d4a9252536dd82811a10327b2df6ddbd1ff13a
Author: Alexey Kuznetsov 
Date:   2016-09-27T10:03:36Z

Web console beta-4.

commit 29acb33293c3d3130e16b7ff4d6b7ae260b7b78b
Author: Alexey Kuznetsov 
Date:   2016-09-27T10:15:38Z

Fixed typos.

commit 3f62596ec33026fb6104f7ed33979a0df46c5789
Author: Alexey Kuznetsov 
Date:   2016-09-27T10:19:00Z

Merge ignite-1.6.9 into ignite-1.7.2.

commit c2a3f11ca14cf9f9cf5bd2d6e2a87764f7cda5a7
Author: Andrey Martianov 
Date:   2016-09-20T14:41:49Z

ignite-3621 Use single ttl cleanup worker thread for all caches

(cherry picked from commit 1bc6058)

commit 39fc5477c19cbe2b2116aaf575a2d0a9c9a618b1
Author: tledkov-gridgain 
Date:   2016-09-27T11:48:18Z

IGNITE-3639: IGFS: Removed BufferedOutputStream from 
LocalIgfsSecondaryFileSystem because it doesn't give any performance benefit.

commit 5cffd3c3d6cb006e3745c314d6f85a066e6a0f06
Author: vozerov-gridgain 
Date:   2016-09-27T12:13:21Z

IGNITE-3661: First attempt to move ignored and flaky tests into a single 
suite. Applied to web-session module.

commit c8dc92ecc8a5d76e68d2d75f12158e0a581a0326
Author: sboikov 
Date:   2016-09-27T12:17:05Z

ignite-3973 In TcpDiscoveryMulticastIpFinder.requestAddresses wait full 
timeout for remote addresses

commit 82b44fe7e495b418bc4463036f4a6b7169bac6d6
Author: sboikov 
Date:   2016-09-27T12:17:54Z

Merge remote-tracking branch 'community/ignite-1.6.9' into ignite-1.6.9

commit 8ba2b947895cabdddb8633a39063c8739c18ad1b
Author: sboikov 
Date:   2016-09-27T13:07:52Z

ignite-3967 Do not use GridBoundedConcurrentOrderedMap.clear

commit 7f8281cd191ea576a8d6358b53fb13e4344cb9d5
Author: vozerov-gridgain 
Date:   2016-09-27T13:37:40Z

IGNITE-3978: Applied "IgniteIgnore" annotation to failing S3 tests. This 
closes #1123.

commit 8127667a14184f77bc0f686d21e3062ae19260a3
Author: vozerov-gridgain 
Date:   2016-09-27T13:37:56Z

Merge remote-tracking branch 'upstream/ignite-1.6.9' into ignite-1.6.9

commit 2bfa06dd75fa55ad01438fa59d14b864ec95834e
Author: Alexey Kuznetsov 
Date:   2016-09-28T01:22:40Z

Fixed typo.

commit c188c3c4a96eacb85ea8e08f0634288332432c1c
Author: Alexey Kuznetsov 
Date:   2016-09-28T01:46:23Z

IGNITE-3983 Fixed wrong cache load optimization. Test added.

commit 89c30c8b0be6915d2399be508ddcd9eb439a9aaa
Author: Alexey Kuznetsov 
Date:   2016-09-28T01:57:45Z

IGNITE-3965 @GridInternal tasks should run via standart LoadBalancingSpi. 
Added test.

commit a53c399e10926120106379d1c764edd7d3854e6a
Author: Alexey Kuznetsov 
Date:   2016-09-28T02:07:55Z

Merge ignite-1.6.9 into ignite-1.7.2.

commit ec9ddcd3d99d19403bf19e1172ede2afdab6c86f
Author: sboikov 
Date:   2016-09-28T09:05:28Z

Code style fixes.

commit 17c2fc0b69abd023b2a1e5da344e67951fd49408
Author: sboikov 
Date:   2016-09-28T09:56:17Z

ignite-2833 Need call 'touch' for cache entry if it was obtained using 
'entryEx'.

commit daf974d261efa525678d5fabc6191642c07f9ad4
Author: AKuznetsov 
Date:   2016-09-28T10:22:10Z

IGNITE-3965 Fixed issues found on review.

commit 53fbad7ddafdae7b368b0f207d06d16574978d62
Author: AKuznetsov 
Date:   2016-09-28T10:24:56Z

Merge branch ignite-1.6.9 into ignite-1.7.2.

commit 4ff19c20b169e0373eafc8025a838db8bfc61f27
Author: sboikov 

[jira] [Created] (IGNITE-4390) [IGNITE-4388] .NET: Improve LINQ error message when aggregates are used with multiple fields

[Semen Boikov (JIRA)]

Semen Boikov created IGNITE-4390:


 Summary: [IGNITE-4388] .NET: Improve LINQ error message when 
aggregates are used with multiple fields
 Key: IGNITE-4390
 URL: https://issues.apache.org/jira/browse/IGNITE-4390
 Project: Ignite
  Issue Type: Improvement
Reporter: Pavel Tupitsyn
Priority: Minor


Current message is "Aggregate functions do not support multiple fields". This 
happens when you try to apply aggregate function (such as COUNT) to a query 
which selects multiple fields (select new {X, Y}). LINQ allows that, but we 
can't express this in SQL.

* See what Entity Framework does
* Improve error message



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4389) [IGNITE-4386] Hadoop tests affect each other through IgniteHadoopClientProtocolProvider#cliMap

[Semen Boikov (JIRA)]

Semen Boikov created IGNITE-4389:


 Summary: [IGNITE-4386] Hadoop tests affect each other through 
IgniteHadoopClientProtocolProvider#cliMap
 Key: IGNITE-4389
 URL: https://issues.apache.org/jira/browse/IGNITE-4389
 Project: Ignite
  Issue Type: Bug
Reporter: Ivan Veselovsky


Tests affect each other through map
org.apache.ignite.hadoop.mapreduce.IgniteHadoopClientProtocolProvider#cliMap 
that never clears. 

For example, test 
org.apache.ignite.internal.processors.hadoop.impl.client.HadoopClientProtocolMultipleServersSelfTest#testSingleAddress
 sometimes fails if test 
org.apache.ignite.internal.processors.hadoop.impl.client.HadoopClientProtocolSelfTest#testJobCounters
 runs before it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4388) .NET: Improve LINQ error message when aggregates are used with multiple fields

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-4388:
--

 Summary: .NET: Improve LINQ error message when aggregates are used 
with multiple fields
 Key: IGNITE-4388
 URL: https://issues.apache.org/jira/browse/IGNITE-4388
 Project: Ignite
  Issue Type: Improvement
  Components: platforms
Affects Versions: 1.7, 1.8
Reporter: Pavel Tupitsyn
Priority: Trivial
 Fix For: 1.9


Current message is "Aggregate functions do not support multiple fields". This 
happens when you try to apply aggregate function (such as COUNT) to a query 
which selects multiple fields (select new {X, Y}). LINQ allows that, but we 
can't express this in SQL.

* See what Entity Framework does
* Improve error message



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] ignite pull request #1327: Ignite 4386

[iveselovskiy]

GitHub user iveselovskiy opened a pull request:

https://github.com/apache/ignite/pull/1327

Ignite 4386



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4386

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1327.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1327


commit bfb00b6e61f9709718c30971997aeb0ac79e86b4
Author: Alexandr Kuramshin 
Date:   2016-11-18T20:12:28Z

IgniteTcpCommunicationBigClusterTest added

commit 02dd92e605b9b53f5a16c7ec5f8e7b5698b15ba4
Author: Alexandr Kuramshin 
Date:   2016-11-18T21:55:37Z

IgniteTcpCommunicationBigClusterTest update

commit 6acf193a3d356d1bad4c02a53ac76833ed1008d0
Author: Alexandr Kuramshin 
Date:   2016-11-19T09:55:45Z

Have got TcpCommunicationSpi error

commit 4fd39653d24f62f19f70b4dffba8497185cc46fb
Author: Alexandr Kuramshin 
Date:   2016-11-19T16:39:10Z

Some discovery have been done

commit c2c181922c7c24ea457577e32d2af897c8bec87f
Author: Alexandr Kuramshin 
Date:   2016-11-19T20:11:28Z

Prove that problem is not in the onFirstMessage hang

commit f8076edba097f6077229b2090ee3ff1a3369878c
Author: Alexandr Kuramshin 
Date:   2016-11-19T20:26:37Z

Revert: Prove that problem is not in the onFirstMessage hang

commit 6e1f2dfc2acb3dbb8f24aa51ed67b2ee447b4585
Author: Alexandr Kuramshin 
Date:   2016-11-21T08:55:09Z

Revert: pushing unnecessary changes to the master

commit ed794ca815f6bb1471af15779279d287576b39cc
Author: Alexandr Kuramshin 
Date:   2016-11-21T09:08:00Z

Revert: pushing unnecessary changes to the master

commit e4357ca34b531a855379d841155e92919e69d42e
Author: iveselovskiy 
Date:   2016-12-07T14:07:44Z

ignite-4386




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1326: Ignite 3558

[tledkov-gridgain]

GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/1326

Ignite 3558



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-3558

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1326.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1326


commit 3df412e7f25aac8227b68cffe1577593a05d1431
Author: sboikov 
Date:   2016-12-07T09:25:32Z

ignite-2545 Optimization for GridCompoundFuture's futures iteration

commit f3db74f782c68c7f73ef3ef4390e010976493634
Author: Anton Vinogradov 
Date:   2016-12-07T10:15:37Z

IGNITE-4238: Added geospatial queries example (nolgpl examples hotfix)

commit 56efb10c40fd4481d6a9dc00928e7beee1f164c5
Author: Anton Vinogradov 
Date:   2016-12-07T11:25:53Z

IGNITE-4353 Parent Cassandra module deployed on maven repository (hotfix: 
deploy to custom maven repository)

commit 57a4d704bbe496b8d33d6f443cf4a85efcd22c02
Author: tledkov-gridgain 
Date:   2016-12-07T14:02:22Z

IGNITE-3558: use default equals for GridJobWorker




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-4387) Incorrect Karaf OSGI feature for ignite-rest-http

[Sergej Sidorov (JIRA)]

Sergej Sidorov created IGNITE-4387:
--

 Summary: Incorrect Karaf OSGI feature for ignite-rest-http
 Key: IGNITE-4387
 URL: https://issues.apache.org/jira/browse/IGNITE-4387
 Project: Ignite
  Issue Type: Bug
  Components: osgi, rest
Affects Versions: 1.7
Reporter: Sergej Sidorov
Assignee: Sergej Sidorov
Priority: Minor


In accordance with the commit f177d4312c47 (IGNITE-3277 Replaced outdated 
json-lib 2.4 to modern Jackson 2.7.5) dependencies in module ignite-rest-http 
have been updated. But Karaf OSGI features config has not been updated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4386) Hadoop tests affect each other through IgniteHadoopClientProtocolProvider#cliMap

[Ivan Veselovsky (JIRA)]

Ivan Veselovsky created IGNITE-4386:
---

 Summary: Hadoop tests affect each other through 
IgniteHadoopClientProtocolProvider#cliMap
 Key: IGNITE-4386
 URL: https://issues.apache.org/jira/browse/IGNITE-4386
 Project: Ignite
  Issue Type: Bug
  Components: hadoop
Affects Versions: 1.7
Reporter: Ivan Veselovsky
Assignee: Ivan Veselovsky
 Fix For: 1.8


Tests affect each other through map
org.apache.ignite.hadoop.mapreduce.IgniteHadoopClientProtocolProvider#cliMap 
that never clears. 

For example, test 
org.apache.ignite.internal.processors.hadoop.impl.client.HadoopClientProtocolMultipleServersSelfTest#testSingleAddress
 sometimes fails if test 
org.apache.ignite.internal.processors.hadoop.impl.client.HadoopClientProtocolSelfTest#testJobCounters
 runs before it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] ignite pull request #1325: IGNITE-4358: NPE Custom message

[rmohta]

GitHub user rmohta opened a pull request:

https://github.com/apache/ignite/pull/1325

IGNITE-4358: NPE Custom message

Validate non-null runnable.
Custom message to give more information, why is the user seeing NPE.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/rmohta/ignite improvement/IGNITE-4358

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1325.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1325


commit 2dce408b36113b91007975028ce985751c52fe99
Author: rmohta 
Date:   2016-12-05T18:25:32Z

IGNITE-4358: NPE Custom message

Validate non-null runnable.
Custom message to give more information, why is the user seeing NPE.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1316: Ignite 3558

[tledkov-gridgain]

Github user tledkov-gridgain closed the pull request at:

https://github.com/apache/ignite/pull/1316


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Grid hang on compute

[Dmitry Karachentsev]

Igniters!

Recently faced with arguable issue, it looks like a bug. Scenario is 
following:


1) Start two data nodes with some cache.

2) From one node in async mode post some big number of jobs to another. 
That jobs do some cache operations.


3) Grid hangs almost immediately and all threads are sleeping except 
public ones, they are waiting for response.


This happens because all cache and job messages are queued on 
communication and limited with default number (1024). It looks like jobs 
are waiting for cache responses that could not be received due to this 
limit. It's hard to diagnose and looks not convenient (as I know we have 
no limitation in docs for using cache ops from compute jobs).


So, my question is. Should we try to solve that or, may be, it's enough 
to update documentation with recommendation to disable queue limit for 
such cases?

affinityCall in one distributed transaction

[Игорь Г]

Hi, igniters!

Before openning JIRA ticket, I want to ask question about affinityCall or
affinityRun transactions.

For example I have batch task to modify many values in someCache according
to someRule. I want to parallel this task to whole cluster and minimize
network traffic.
So the resonable choice is affinityCall feature.

But I want all this changes to be in one transactoin. i.e. with at least
atomicity property (of ACID). And if for some reason my task will be
canceled or failed on one node - it should change nothing at all.

So, can I achieve this with existing functionality, or how can I approach
to this task?

[jira] [Created] (IGNITE-4385) .NET: LINQ requires IQueryable in joins

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-4385:
--

 Summary: .NET: LINQ requires IQueryable in joins
 Key: IGNITE-4385
 URL: https://issues.apache.org/jira/browse/IGNITE-4385
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Affects Versions: 1.7, 1.8
Reporter: Pavel Tupitsyn
Priority: Minor
 Fix For: 1.9


This does not work:
{code}
var q3 = from ic in GetCache().AsCacheQueryable()
from pp in GetCache().AsCacheQueryable()
select pp.Value;
{code}

Error:
{code}
Unexpected query source: GetCache().AsCacheQueryable()
{code}

And this does work:
{code}
var contracts = GetCache().AsCacheQueryable();
var paymentPlans = GetCache().AsCacheQueryable();
var q3 = from ic in contracts
from pp in paymentPlans
select pp.Value;
{code}

While it is usually possible to extract variables, this is an inconvenience. 
See if we can support such expressions (evaluate irrelevant part automatically).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4384) Web console: Invalid Optimazed marshaller configuration.

[Vasiliy Sisko (JIRA)]

Vasiliy Sisko created IGNITE-4384:
-

 Summary: Web console: Invalid Optimazed marshaller configuration.
 Key: IGNITE-4384
 URL: https://issues.apache.org/jira/browse/IGNITE-4384
 Project: Ignite
  Issue Type: Bug
  Components: wizards
Affects Versions: 1.9
Reporter: Vasiliy Sisko
 Fix For: 1.9


Wrong default value for require serializable field.
Missed ID mapper field.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] ignite pull request #1324: Ignite 3292 fixed

[oleg-ostanin]

GitHub user oleg-ostanin opened a pull request:

https://github.com/apache/ignite/pull/1324

Ignite 3292 fixed

log file looks like this:
...
<15:02:48> Preloading started.
<15:02:48> Preloading:atomic-index 
0(+0)
<15:02:48> Preloading:query
0(+0)
<15:02:48> Preloading:tx-fat-values
0(+0)
<15:02:48> Preloading:compute  
0(+0)
<15:02:49> Preloading:atomic-offheap-values
0(+0)
<15:02:49> Preloading:tx-offheap   
0(+0)
<15:02:49> Preloading:atomic-offheap   
0(+0)
<15:02:49> Preloading:orgCache 
0(+0)
<15:02:49> Preloading:atomic-fat-values
0(+0)
<15:02:49> Preloading:tx-offheap-values
0(+0)
...
<15:02:59> Preloading:atomic-index 
176841   (+176841)
<15:02:59> Preloading:query
188753   (+188753)
<15:02:59> Preloading:tx-fat-values
157049   (+157049)
<15:02:59> Preloading:compute  
185015   (+185015)
<15:02:59> Preloading:atomic-offheap-values
181587   (+181587)
<15:02:59> Preloading:tx-offheap   
150125   (+150125)
<15:02:59> Preloading:atomic-offheap   
149441   (+149441)
<15:02:59> Preloading:orgCache 
194663   (+194663)
<15:02:59> Preloading:atomic-fat-values
149060   (+149060)
<15:02:59> Preloading:tx-offheap-values
184565   (+184565)
...
<15:03:37> Preloading finished.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-3292-fixed

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1324.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1324


commit bfb00b6e61f9709718c30971997aeb0ac79e86b4
Author: Alexandr Kuramshin 
Date:   2016-11-18T20:12:28Z

IgniteTcpCommunicationBigClusterTest added

commit 02dd92e605b9b53f5a16c7ec5f8e7b5698b15ba4
Author: Alexandr Kuramshin 
Date:   2016-11-18T21:55:37Z

IgniteTcpCommunicationBigClusterTest update

commit 6acf193a3d356d1bad4c02a53ac76833ed1008d0
Author: Alexandr Kuramshin 
Date:   2016-11-19T09:55:45Z

Have got TcpCommunicationSpi error

commit 4fd39653d24f62f19f70b4dffba8497185cc46fb
Author: Alexandr Kuramshin 
Date:   2016-11-19T16:39:10Z

Some discovery have been done

commit c2c181922c7c24ea457577e32d2af897c8bec87f
Author: Alexandr Kuramshin 
Date:   2016-11-19T20:11:28Z

Prove that problem is not in the onFirstMessage hang

commit f8076edba097f6077229b2090ee3ff1a3369878c
Author: Alexandr Kuramshin 
Date:   2016-11-19T20:26:37Z

Revert: Prove that problem is not in the onFirstMessage hang

commit 6e1f2dfc2acb3dbb8f24aa51ed67b2ee447b4585
Author: Alexandr Kuramshin 
Date:   2016-11-21T08:55:09Z

Revert: pushing unnecessary changes to the master

commit ed794ca815f6bb1471af15779279d287576b39cc
Author: Alexandr Kuramshin 
Date:   2016-11-21T09:08:00Z

Revert: pushing unnecessary changes to the master

commit 924652f5d56427dfc9628870253d21c0f19d5035
Author: oleg-ostanin 
Date:   2016-12-07T11:42:29Z

added log of preloading process

commit f5b3c4a85c4a22cdb737c10150387e18fb2a5fe4
Author: oleg-ostanin 
Date:   2016-12-07T11:59:58Z

minor changes




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1323: IGNITE-4379: SQL: Local SQL field query broken in...

[AMashenkov]

GitHub user AMashenkov opened a pull request:

https://github.com/apache/ignite/pull/1323

IGNITE-4379: SQL: Local SQL field query broken in master

Fixed broken local SQLFieldsQuery.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4379

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1323.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1323


commit 312a64be2bcbf44e50e95595c3207a6c130dc486
Author: Andrey V. Mashenkov 
Date:   2016-12-07T10:50:37Z

Fixed broken local SQLFieldsQuery.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: What branch to use for Apache Ignite 2.0 development?

[Sergi Vladykin]

Vladimir,

Plans tend to change over time :) It is a good practice to avoid locking
ourselves and be flexible enough to tolerate any possible changes.

For me it is clear that everything related to a new storage model
(PageMemory and stuff) as well as other breaking changes have to go to 2.0.
Everything else goes to the current master. Master must be periodically
merged to 2.0. Looks simple enough, no?

Sergi

2016-12-07 12:44 GMT+03:00 Vladimir Ozerov :

> Sergi,
>
> The problem is that all current changes target 2.0, because we do not have
> 1.9 in plans at the moment. So it is not clear how to distinguish them.
> If 1.x is ever will be planned, most likely it will be bug-fix release. IMO
> it is better to continue all development in master and branch out of
> *ignite-1.8* tag for 1.x release if needed.
>
> On Wed, Dec 7, 2016 at 12:36 PM, Sergi Vladykin 
> wrote:
>
> > I think it is good idea to be able to continue releasing 1.X versions. It
> > may take quite some time to make Ignite 2.0 stable enough. Thus I'm for
> > keeping changes targeted exclusively to 2.0 in a separate branch for now.
> >
> > Sergi
> >
> > 2016-12-07 12:23 GMT+03:00 Vladimir Ozerov :
> >
> > > Igniters,
> > >
> > > We are moving towards Apache Ignite 2.0. This release will contain lots
> > of
> > > changes which break compilation on user side and change internal binary
> > > protocols. The question is - where to continue development of 2.0
> > features?
> > >
> > > We can do that in master branch. But what if we decide to release
> Apache
> > > Ignite 1.9 at some point? In this case we cannot use master branch
> > because
> > > it will contain incompatible API changes. As alternative we can create
> > > separate branch for 2.0 and merge all breaking changes there. But it
> will
> > > complicate development process.
> > >
> > > Probably we can continue all development in *master*, and in case if we
> > > decide to release another 1.x release, then create branch from
> ignite-1.8
> > > tag and cherry-pick required fixes there.
> > >
> > > Please share your thoughts.
> > >
> > > Vladimir.
> > >
> >
>
>
>
> --
> Vladimir Ozerov
> Senior Software Architect
> GridGain Systems
> www.gridgain.com
> *+7 (960) 283 98 40*
>

Re: What branch to use for Apache Ignite 2.0 development?

[Vladimir Ozerov]

Sergi,

The problem is that all current changes target 2.0, because we do not have
1.9 in plans at the moment. So it is not clear how to distinguish them.
If 1.x is ever will be planned, most likely it will be bug-fix release. IMO
it is better to continue all development in master and branch out of
*ignite-1.8* tag for 1.x release if needed.

On Wed, Dec 7, 2016 at 12:36 PM, Sergi Vladykin 
wrote:

> I think it is good idea to be able to continue releasing 1.X versions. It
> may take quite some time to make Ignite 2.0 stable enough. Thus I'm for
> keeping changes targeted exclusively to 2.0 in a separate branch for now.
>
> Sergi
>
> 2016-12-07 12:23 GMT+03:00 Vladimir Ozerov :
>
> > Igniters,
> >
> > We are moving towards Apache Ignite 2.0. This release will contain lots
> of
> > changes which break compilation on user side and change internal binary
> > protocols. The question is - where to continue development of 2.0
> features?
> >
> > We can do that in master branch. But what if we decide to release Apache
> > Ignite 1.9 at some point? In this case we cannot use master branch
> because
> > it will contain incompatible API changes. As alternative we can create
> > separate branch for 2.0 and merge all breaking changes there. But it will
> > complicate development process.
> >
> > Probably we can continue all development in *master*, and in case if we
> > decide to release another 1.x release, then create branch from ignite-1.8
> > tag and cherry-pick required fixes there.
> >
> > Please share your thoughts.
> >
> > Vladimir.
> >
>



-- 
Vladimir Ozerov
Senior Software Architect
GridGain Systems
www.gridgain.com
*+7 (960) 283 98 40*

Re: What branch to use for Apache Ignite 2.0 development?

[Alexey Goncharuk]

+1 to have a separate branch for 2.0 and keeping master clean from breaking
changes. I think 2.0 should be merged to master only after first 2.0
release, when it really becomes the main development branch.

2016-12-07 12:36 GMT+03:00 Sergi Vladykin :

> I think it is good idea to be able to continue releasing 1.X versions. It
> may take quite some time to make Ignite 2.0 stable enough. Thus I'm for
> keeping changes targeted exclusively to 2.0 in a separate branch for now.
>
> Sergi
>
> 2016-12-07 12:23 GMT+03:00 Vladimir Ozerov :
>
> > Igniters,
> >
> > We are moving towards Apache Ignite 2.0. This release will contain lots
> of
> > changes which break compilation on user side and change internal binary
> > protocols. The question is - where to continue development of 2.0
> features?
> >
> > We can do that in master branch. But what if we decide to release Apache
> > Ignite 1.9 at some point? In this case we cannot use master branch
> because
> > it will contain incompatible API changes. As alternative we can create
> > separate branch for 2.0 and merge all breaking changes there. But it will
> > complicate development process.
> >
> > Probably we can continue all development in *master*, and in case if we
> > decide to release another 1.x release, then create branch from ignite-1.8
> > tag and cherry-pick required fixes there.
> >
> > Please share your thoughts.
> >
> > Vladimir.
> >
>

[jira] [Created] (IGNITE-4383) Update section on binary marshaller in Apache Ignite Readme.io docs

[Alexander Paschenko (JIRA)]

Alexander Paschenko created IGNITE-4383:
---

 Summary: Update section on binary marshaller in Apache Ignite 
Readme.io docs
 Key: IGNITE-4383
 URL: https://issues.apache.org/jira/browse/IGNITE-4383
 Project: Ignite
  Issue Type: Task
  Components: binary, documentation
Reporter: Alexander Paschenko
Assignee: Alexander Paschenko
 Fix For: 1.8






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: What branch to use for Apache Ignite 2.0 development?

[Sergi Vladykin]

I think it is good idea to be able to continue releasing 1.X versions. It
may take quite some time to make Ignite 2.0 stable enough. Thus I'm for
keeping changes targeted exclusively to 2.0 in a separate branch for now.

Sergi

2016-12-07 12:23 GMT+03:00 Vladimir Ozerov :

> Igniters,
>
> We are moving towards Apache Ignite 2.0. This release will contain lots of
> changes which break compilation on user side and change internal binary
> protocols. The question is - where to continue development of 2.0 features?
>
> We can do that in master branch. But what if we decide to release Apache
> Ignite 1.9 at some point? In this case we cannot use master branch because
> it will contain incompatible API changes. As alternative we can create
> separate branch for 2.0 and merge all breaking changes there. But it will
> complicate development process.
>
> Probably we can continue all development in *master*, and in case if we
> decide to release another 1.x release, then create branch from ignite-1.8
> tag and cherry-pick required fixes there.
>
> Please share your thoughts.
>
> Vladimir.
>

What branch to use for Apache Ignite 2.0 development?

[Vladimir Ozerov]

Igniters,

We are moving towards Apache Ignite 2.0. This release will contain lots of
changes which break compilation on user side and change internal binary
protocols. The question is - where to continue development of 2.0 features?

We can do that in master branch. But what if we decide to release Apache
Ignite 1.9 at some point? In this case we cannot use master branch because
it will contain incompatible API changes. As alternative we can create
separate branch for 2.0 and merge all breaking changes there. But it will
complicate development process.

Probably we can continue all development in *master*, and in case if we
decide to release another 1.x release, then create branch from ignite-1.8
tag and cherry-pick required fixes there.

Please share your thoughts.

Vladimir.

Request to work on IGNITE-3959

[Максим Козлов]

Hi,

I would like to contribute to IGNITE-3959 https://issues.apache.org/
jira/browse/IGNITE-3959.
My username in JIRA is dreamx.

-- 
Best regards, Maks Kozlov

[jira] [Created] (IGNITE-4382) .NET: Documentation on calling Java services

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-4382:
--

 Summary: .NET: Documentation on calling Java services
 Key: IGNITE-4382
 URL: https://issues.apache.org/jira/browse/IGNITE-4382
 Project: Ignite
  Issue Type: Task
  Components: platforms
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 1.8


Calling Java services from .NET was implemented in IGNITE-2686, but not 
documented anywhere.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: NullPointerException on ScanQuery

[Alper Tekinalp]

Hi all.

It seems one method uses affinity:

 cctx.affinity().primaryPartitions(n.id(), topologyVersion());

other uses topology API.

cctx.topology().owners(part, topVer)

Are these two fully consistent?

On Tue, Dec 6, 2016 at 4:58 PM, Alper Tekinalp  wrote:

> Hi all.
>
> We have 2 servers and a cache X. On both servers a method is running
> reqularly and run a ScanQurey on that cache. We get partitions for that
> query via
>
> ignite.affinity(cacheName).primaryPartitions(ignite.cluster().localNode())
>
> and run the query on each partitions. When cache has been destroyed by
> master server on second server we get:
>
> javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException:
> null
> at org.apache.ignite.internal.processors.cache.
> IgniteCacheProxy.query(IgniteCacheProxy.java:740)
> at com.intellica.evam.engine.event.future.FutureEventWorker.
> processFutureEvents(FutureEventWorker.java:117)
> at com.intellica.evam.engine.event.future.FutureEventWorker.run(
> FutureEventWorker.java:66)
> Caused by: class org.apache.ignite.IgniteCheckedException: null
> at org.apache.ignite.internal.processors.query.GridQueryProcessor.
> executeQuery(GridQueryProcessor.java:1693)
> at org.apache.ignite.internal.processors.cache.
> IgniteCacheProxy.query(IgniteCacheProxy.java:494)
> at org.apache.ignite.internal.processors.cache.
> IgniteCacheProxy.query(IgniteCacheProxy.java:732)
> ... 2 more
> Caused by: java.lang.NullPointerException
> at org.apache.ignite.internal.processors.cache.query.
> GridCacheQueryAdapter$ScanQueryFallbackClosableIterator.init(
> GridCacheQueryAdapter.java:712)
> at org.apache.ignite.internal.processors.cache.query.
> GridCacheQueryAdapter$ScanQueryFallbackClosableIterator.(
> GridCacheQueryAdapter.java:677)
> at org.apache.ignite.internal.processors.cache.query.
> GridCacheQueryAdapter$ScanQueryFallbackClosableIterator.(
> GridCacheQueryAdapter.java:628)
> at org.apache.ignite.internal.processors.cache.query.
> GridCacheQueryAdapter.executeScanQuery(GridCacheQueryAdapter.java:548)
> at org.apache.ignite.internal.processors.cache.
> IgniteCacheProxy$2.applyx(IgniteCacheProxy.java:497)
> at org.apache.ignite.internal.processors.cache.
> IgniteCacheProxy$2.applyx(IgniteCacheProxy.java:495)
> at org.apache.ignite.internal.util.lang.IgniteOutClosureX.
> apply(IgniteOutClosureX.java:36)
> at org.apache.ignite.internal.processors.query.GridQueryProcessor.
> executeQuery(GridQueryProcessor.java:1670)
> ... 4 more
>
> for a while until cache is closed on that server too.
>
> The corresponding line is:
>
> 710:final ClusterNode node = nodes.poll();
> 711:
> 712:if (*node*.isLocal()) {
>
> Obviously node is null. nodes is a dequeue fill by following method:
>
> private Queue fallbacks(AffinityTopologyVersion
> topVer) {
> Deque fallbacks = new LinkedList<>();
> Collection owners = new HashSet<>();
>
> for (ClusterNode node : cctx.topology().owners(part, topVer)) {
> if (node.isLocal())
> fallbacks.addFirst(node);
> else
> fallbacks.add(node);
>
> owners.add(node);
> }
>
> for (ClusterNode node : cctx.topology().moving(part)) {
> if (!owners.contains(node))
> fallbacks.add(node);
> }
>
> return fallbacks;
> }
>
> There errors occurs before cache closed on second server. So checking if
> cache closed is not enough.
>
> Why when we take partitions for local node we get some partitions but
> ignite cant find any owner for that partition?
> Is our method for getting partitions wrong?
> Is there any way to avoid that?
>
> Best regards.
> --
> Alper Tekinalp
>
> Software Developer
> Evam Streaming Analytics
>
> Atatürk Mah. Turgut Özal Bulv.
> Gardenya 5 Plaza K:6 Ataşehir
> 34758 İSTANBUL
>
> Tel:  +90 216 455 01 53 Fax: +90 216 455 01 54
> www.evam.com.tr
> 
>



-- 
Alper Tekinalp

Software Developer
Evam Streaming Analytics

Atatürk Mah. Turgut Özal Bulv.
Gardenya 5 Plaza K:6 Ataşehir
34758 İSTANBUL

Tel:  +90 216 455 01 53 Fax: +90 216 455 01 54
www.evam.com.tr

[jira] [Created] (IGNITE-4381) Need ensure that internal threads do not execute blocking operations

[Semen Boikov (JIRA)]

Semen Boikov created IGNITE-4381:


 Summary: Need ensure that internal threads do not execute blocking 
operations
 Key: IGNITE-4381
 URL: https://issues.apache.org/jira/browse/IGNITE-4381
 Project: Ignite
  Issue Type: Task
  Components: general
Reporter: Semen Boikov
Assignee: Konstantin Dudkov
 Fix For: 2.0


If internal threads execute blocking operation this can cause starvation and 
hangs (example of issue https://issues.apache.org/jira/browse/IGNITE-4371).
Ideally we need a way to 'automatically' find all such places in code, 
straightforward idea is add assert in GridFutureAdapter.get - assert should 
fail if it is called by system thread and future is not finished. At least one 
issue here is that currently system threads can be blocked on operation on 
utility/marshaller cache, so assert should also take it into account.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Major release 2.0 and compilation

[Yakov Zhdanov]

Agree with Vladimir that we should use flexible approach and should avoid
any possible harm here, but cleaning out most of deprecations is the way to
go, IMO. There are about 90 occurrences of "deprecated" in the project and
most of them are not very hard to remove. Moreover, we have, for example,
setRemoteFilter(CacheEntryEventSerializableFilter). Using this method
is error prone. We should remove it so none can use it.

Cleaning up and renovating are good. This allows us to see what can be done
better and also encourages users to move forward.

--Yakov

2016-12-07 15:22 GMT+07:00 Vladimir Ozerov :

> Igniters,
>
> Release Apache Ignite 2.0 is already planned and being discussed. Normally
> we do not brake compilation between minor releases, and if some method on
> public API should not be used anymore we mark it as *deprecated*. But we do
> not have such a rule for major releases. The question is whether we are
> going to break compilation and remove deprecated methods from public API in
> Apache ignite 2.0 or not. There are several extreme approaches to this.
>
> First, we can declare that we cleanup all deprecations and remove them.
> This will result in clean and consistent API and simplify further
> development. But it might slowdown adoption of Apache Ignite 2.0 because
> users will be reluctant switching to newer version because they will have
> to fix compilation, re-deploy their apps, etc..
>
> Second, we can say that we must avoid breaking compilation at all costs and
> retain deprecated methods. This approach might be better for users, for
> harder to maintain for Ignite developers. With this approach users will
> migrate to 2.0 quicker.
>
> My opinion is that we must choose flexible approach and decide whether to
> keep deprecation or not separately for every piece of API, depending on
> possible impact on both users and Ignite developers. But normally I would
> leave deprecation unless there is a strong reason to remove it.
>
> Thoughts?
>
> Vladimir.
>

[GitHub] ignite pull request #1322: IGNITE 1.8.1

[devozerov]

GitHub user devozerov opened a pull request:

https://github.com/apache/ignite/pull/1322

IGNITE 1.8.1



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-1.8.1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1322.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1322


commit 9a6cfce659df40b0a4624f19fd91c217b74bafea
Author: nikolay_tikhonov 
Date:   2016-10-11T10:59:57Z

IGNITE-4014 Fixed "Transaction hangs if entry processor failed during 
serialization". This closes #1148.

commit 1938a17b01fac1e08c30011180bbcc3ed7374d83
Author: Andrey V. Mashenkov 
Date:   2016-10-11T11:50:18Z

IGNITE-3948: Fixed a bug causing TTL manager to continue tracking of 
evicted entries. This closes #1101.

commit 60d27547dfc6bd27c6d39cbcc523d0d1e872a821
Author: vozerov-gridgain 
Date:   2016-10-11T11:51:00Z

Merge remote-tracking branch 'upstream/ignite-1.6.10' into ignite-1.6.10

commit 359a392f1c53217c691c4c40762c51fd330666e2
Author: Valentin Kulichenko 
Date:   2016-01-15T06:58:41Z

Update notifier fixes

(cherry picked from commit a5c85ca)

commit 01ca6db70933fb7f50f161a80cde9647e68a3710
Author: dkarachentsev 
Date:   2016-10-11T13:18:40Z

Merge remote-tracking branch 'origin/ignite-1.6.10' into ignite-1.6.10

commit 0659bebe04dc9c0b0a163bc95061519c553e678c
Author: Andrey V. Mashenkov 
Date:   2016-10-12T11:49:36Z

IGNITE-3972: Fixed a bug causing continuous queries to be lost for ATOMIC 
cache when key's primary node leaves topology. This closes #1133.

commit f597aff1bdf65d3d430cf85c9932391a72c2d7dc
Author: Andrey V. Mashenkov 
Date:   2016-10-12T12:44:08Z

IGNITE-3875: Added separate thread pool for data streamer. This closes 
#1067.

commit 2ab094e08373dc6667af44d48a38b4f044953a79
Author: tledkov-gridgain 
Date:   2016-10-12T13:48:51Z

IGNITE-2355: Hadoop: added ability to configure multiple job tracker 
addresses. This closes #1153.

commit eaf8ae246cc799c1353332fcac05cb3a8efab02c
Author: Pavel Tupitsyn 
Date:   2016-10-12T16:57:09Z

IGNITE-4034 Get rid of specialized methods in platform targets

commit b1ec58f716ece3a5866dd654ebc707bef67caf57
Author: Alexey Kuznetsov 
Date:   2016-10-13T05:58:22Z

IGNITE-4066 Fixed NPE.

commit 447e07c0bb5af75bce6a14612606904e4e3d9705
Author: Anton Vinogradov 
Date:   2016-10-14T08:40:41Z

IGNITE-1924 Incomplete marshaller cache rebalancing causes Grid hangs under 
SSL

commit 7adfbcf1ee7bbe0beb95fa82749a2e04449de8fa
Author: tledkov-gridgain 
Date:   2016-10-14T14:48:14Z

IGNITE-4060: Fixed a bug in RoundRobinLoadBalancing API causing exception 
when running closures after client node reconnect. This closes #1157.

commit 80abd1b72e4fc7b0b95212e7f53c700c0fe21156
Author: Aleksei Scherbakov 
Date:   2016-10-14T16:33:07Z

GG-11360 - Implement SQL queries cancellation (#18)

* GG-11360 Merged IGNITE-2680 to ignite-1.6.3.

* GG-11360 Test cleanup.

* GG-11360 Fixing broken tests.

* GG-11360 Fixing test.

* GG-11360 Fixing test.

* GG-11360 Fixing broken tests.

* GG-11360 Added test for forever-running query cancellation on node 
restart.

* GG-11360 Fixing race.

* GG-11360 Added test for forever-running query cancellation on node stop.

* GG-11360 Cleanup.

* GG-11360 Support for local query cancellation/timeout.

* GG-11360 Increase test duration.

* GG-11360 Remove redundant catch block.

* GG-11360 Fix formatting.

* GG-11360 Fix formatting.

* GG-11360 Fix formatting.

* GG-11360 Fix formatting.

* GG-11360 Fix formatting.

* GG-11360 Fix formatting.

* GG-11360 Fix formatting.

* GG-11360 Fix formatting.

* GG-11360 Simplify test.

* GG-11360 Simplify test.

* GG-11360 Fixing issues.

* GG-11360 Fixing issues.

* GG-11360 Fixing issues.

* GG-11360 Fixing issues.

* GG-11360 Fixing issues.

* GG-11360 Fixing issues.

* GG-11360 Fixing issues.

* GG-11360 Fixing issues.

* GG-11360 Fixing issues.

* GG-11360 Fixing issues.

* GG-11360 Fixing issues.

* GG-11360 Fixing issues.

* GG-11360 Fixing issues.

* GG-11360 Fixing issues.

* GG-11360 Fixing issues.

* GG-11360 Fixing issues.

* Merge remote-tracking branch 

Major release 2.0 and compilation

[Vladimir Ozerov]

Igniters,

Release Apache Ignite 2.0 is already planned and being discussed. Normally
we do not brake compilation between minor releases, and if some method on
public API should not be used anymore we mark it as *deprecated*. But we do
not have such a rule for major releases. The question is whether we are
going to break compilation and remove deprecated methods from public API in
Apache ignite 2.0 or not. There are several extreme approaches to this.

First, we can declare that we cleanup all deprecations and remove them.
This will result in clean and consistent API and simplify further
development. But it might slowdown adoption of Apache Ignite 2.0 because
users will be reluctant switching to newer version because they will have
to fix compilation, re-deploy their apps, etc..

Second, we can say that we must avoid breaking compilation at all costs and
retain deprecated methods. This approach might be better for users, for
harder to maintain for Ignite developers. With this approach users will
migrate to 2.0 quicker.

My opinion is that we must choose flexible approach and decide whether to
keep deprecation or not separately for every piece of API, depending on
possible impact on both users and Ignite developers. But normally I would
leave deprecation unless there is a strong reason to remove it.

Thoughts?

Vladimir.

[jira] [Created] (IGNITE-4380) Cache invoke calls can be lost

[Semen Boikov (JIRA)]

Semen Boikov created IGNITE-4380:


 Summary: Cache invoke calls can be lost
 Key: IGNITE-4380
 URL: https://issues.apache.org/jira/browse/IGNITE-4380
 Project: Ignite
  Issue Type: Bug
  Components: cache
Reporter: Semen Boikov
Assignee: Semen Boikov
Priority: Critical
 Fix For: 2.0


Recently added test GridCacheAbstractFullApiSelfTest.testInvokeAllMultithreaded 
fails on TC in various configurations with transactional cache.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)