Re: Semaphore Stuck when no acquirers to assign permit

[Denis Magda]

Igniters,

Who can check out Tim’s fix for the semaphore?

pull request: https://github.com/apache/ignite/pull/3138 

jira: https://issues.apache.org/jira/browse/IGNITE-7090 


—
Denis

> On Jan 15, 2018, at 12:02 PM, Timay  wrote:
> 
> I saw a release date set for 2.4 but have not had any feedback on the jira so
> i wanted to check in on this. Can this make it into the 2.4 release?
> 
> Thanks
> Tim
> 
> 
> 
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: [SparkDataFrame] Query Optimization. Prototype

[Valentin Kulichenko]

Hi Nikolay,

I will review this sometime next week.

-Val

On Fri, Jan 19, 2018 at 2:44 AM, Nikolay Izhikov 
wrote:

> Hello, guys.
>
> I have done prototype of implementation of optimization query from Spark
> to Ignite [1].
>
> Please, take a look at PR [2].
>
> But still there are some issues I want to clarify with community:
>
> * Can I know scale and precision of DECIMAL column?
>   Example - [3]
>
> * Ignite have some limitation for a *distributed* join. For example,
> we can execute join only for indexed columns. Example - [4].
> * Do we have documentation for all limitations of distributed join
> implementation?
> * How can I know whether concrete syntactically correct SQL query
> with join can be executed by Ignite or not?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-7077
> [2] https://github.com/apache/ignite/pull/3397
> [3] https://gist.github.com/nizhikov/2d67ed68afd473d7251dab7d79135513
> [4] https://gist.github.com/nizhikov/a4389fd78636869dd38c13920b5baf2b
>

[jira] [Created] (IGNITE-7482) Cursor in TextQuery fetches all data in first call to next() or hasNext()

[Zbyszek B (JIRA)]

Zbyszek B created IGNITE-7482:
-

 Summary: Cursor in TextQuery fetches all data in first call to 
next() or hasNext()
 Key: IGNITE-7482
 URL: https://issues.apache.org/jira/browse/IGNITE-7482
 Project: Ignite
  Issue Type: Improvement
  Components: general
Affects Versions: 2.3
Reporter: Zbyszek B


In org.apache.ignite.internal.processors.query.h2.opt.GridLuceneIndex#query 
in line 285 one executes query 
docs = searcher.search(query, Integer.MAX_VALUE)  what causes all data to be 
fetched and deserialised regardless the size of the cursor.
TopDocs searchAfter(final ScoreDoc after, Query query, int numHits) is not used 
to implement cursor advancing. This causes the issues with 10 mln. entities  
when (depending 
on term) execution takes way above 7 sec. to retrieve first row, hence making 
this impossible to use TextQuery to implement auto-complete.

See 
[http://apache-ignite-users.70518.x6.nabble.com/Cursor-in-TextQuery-first-hasNex-is-slow-td19265.html]
 for more details.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Documentation for Binary Client Protocol

[Denis Magda]

Added the proposed callout to the top of the page:
https://cwiki.apache.org/confluence/display/IGNITE/IEP-9+Thin+Client+Protocol

> On Jan 19, 2018, at 9:36 AM, Denis Magda  wrote:
> 
> Pavel,
> 
> Do you plan to maintain this page? Frankly, since it’s an IEP and Prachi 
> already prepared an official documentation I would suggest us to but a teaser 
> on the top of the page saying that the most up-to-date and detailed 
> information about the protocol is on readme.io . Agree?
> 
> —
> Denis 
> 
>> On Jan 18, 2018, at 11:37 PM, Pavel Tupitsyn  wrote:
>> 
>> Hi everyone,
>> 
>> Just a reminder that initial spec is on wiki:
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-9+Thin+Client+Protocol
>> 
>> It is quite terse and may be harder to understand.
>> Prachi tried to document all of this in a more user-friendly manner with
>> examples and so on.
>> 
>> On Fri, Jan 19, 2018 at 6:30 AM, Denis Magda  wrote:
>> 
>>> Lucas, thanks for chiming in,
>>> 
>>> The protocol itself consists of all these magic constants (type and
>>> operations codes). See type codes table from this section:
>>> https://apacheignite.readme.io/docs/binary-client-
>>> protocol#section-data-objects >> io/docs/binary-client-protocol>
>>> 
>>> Hope that my explanation given to Alexey makes sense to both of you.
>>> 
>>> Initially, we didn’t plan to have the code snippets at all and wanted to
>>> document the protocol in a way similar to Mongo’s protocol:
>>> https://docs.mongodb.com/manual/reference/mongodb-wire-protocol/ <
>>> https://docs.mongodb.com/manual/reference/mongodb-wire-protocol/>
>>> 
>>> But after several iterations we agreed to simplify a life of those who
>>> will be developing thin clients on top of the protocol and filled out the
>>> doc with as many code snippets as possible. Again, the snippets are not a
>>> complete application and this is why we ignored some of the best practices
>>> used by Java developers.
>>> 
>>> —
>>> Denis
>>> 
 On Jan 18, 2018, at 7:11 PM, Lucas Beeler 
>>> wrote:
 
 Hi Igniters,
 
 I’m with Alexey here: hard-coded “magic” literals make code hard to read
>>> and neither communicate coding professionalism nor comport with published
>>> Sun/Oracle style guides. So I suggest that:
 For already defined symbolic constants, let’s use them instead of
>>> literal values
 If there are literal values for which symbolic constants haven’t yet
>>> been defined, let’s define symbolic constants for them, commit this change
>>> to master, and start using the newly defined symbolic constants
 Anyhoo, that’s just my two cents.
 
 As always, it’s a pleasure to be a part of Ignite development...
 
 Cheers guys & gals,
 Lucas
 
 --
 Lucas BEELER
 Technical Consultant, Professional Services
 GridGain Systems
 www.gridgain.com 
 
> On Jan 18, 2018, at 6:47 PM, Alexey Kuznetsov >> > wrote:
> 
> Prachi, Denis,
> 
> 
> It is OK that we are using numbers in code samples instead of named
> constants?
> 
> For example:
> writeByteLittleEndian(3, out);  // Integer type code
> 
> It could be smth. like this:
> writeByteLittleEndian(TYPE_INT, out);  // Integer type code
> 
> Where TYPE_INT is declared at some place:
> 
> public static final int TYPE_INT = 3;
> 
> 
> Make sense?
> 
> 
> On Fri, Jan 19, 2018 at 4:47 AM, Prachi Garg >> > wrote:
> 
>> Igniters,
>> 
>> The document for Binary Client Protocol (awaiting 2.4 release) is
>>> ready on
>> Apache Ignite readme.io . This document explains:
>> 
>> - How to connect to Ignite cluster
>> - Common message format (Request/Response header)
>> - Supported Data types and their format
>> - Request/Response format for each message type
>> 
>> Here are the links -
>> 
>> https://apacheignite.readme.io/docs/binary-client-protocol <
>>> https://apacheignite.readme.io/docs/binary-client-protocol>
>> 
>> https://apacheignite.readme.io/docs/binary-client-protocol-
>> key-value-operations
>> 
>> https://apacheignite.readme.io/docs/binary-client-
>>> protocol-sql-operations
>> 
>> https://apacheignite.readme.io/docs/binary-client-protocol-
>> binary-type-operations
>> 
>> https://apacheignite.readme.io/docs/binary-client-protocol-
>> cache-configuration-operations
>> 
>> 
>> Currently these pages are hidden and are accessible only with the link.
>> They will be active after the release.
>> 
>> -Prachi
>> 
> 
> 
> 
> --
> Alexey Kuznetsov
 
>>> 
>>> 
> 

Re: Documentation for Binary Client Protocol

[Denis Magda]

Pavel,

Do you plan to maintain this page? Frankly, since it’s an IEP and Prachi 
already prepared an official documentation I would suggest us to but a teaser 
on the top of the page saying that the most up-to-date and detailed information 
about the protocol is on readme.io . Agree?

—
Denis 

> On Jan 18, 2018, at 11:37 PM, Pavel Tupitsyn  wrote:
> 
> Hi everyone,
> 
> Just a reminder that initial spec is on wiki:
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-9+Thin+Client+Protocol
> 
> It is quite terse and may be harder to understand.
> Prachi tried to document all of this in a more user-friendly manner with
> examples and so on.
> 
> On Fri, Jan 19, 2018 at 6:30 AM, Denis Magda  wrote:
> 
>> Lucas, thanks for chiming in,
>> 
>> The protocol itself consists of all these magic constants (type and
>> operations codes). See type codes table from this section:
>> https://apacheignite.readme.io/docs/binary-client-
>> protocol#section-data-objects > io/docs/binary-client-protocol>
>> 
>> Hope that my explanation given to Alexey makes sense to both of you.
>> 
>> Initially, we didn’t plan to have the code snippets at all and wanted to
>> document the protocol in a way similar to Mongo’s protocol:
>> https://docs.mongodb.com/manual/reference/mongodb-wire-protocol/ <
>> https://docs.mongodb.com/manual/reference/mongodb-wire-protocol/>
>> 
>> But after several iterations we agreed to simplify a life of those who
>> will be developing thin clients on top of the protocol and filled out the
>> doc with as many code snippets as possible. Again, the snippets are not a
>> complete application and this is why we ignored some of the best practices
>> used by Java developers.
>> 
>> —
>> Denis
>> 
>>> On Jan 18, 2018, at 7:11 PM, Lucas Beeler 
>> wrote:
>>> 
>>> Hi Igniters,
>>> 
>>> I’m with Alexey here: hard-coded “magic” literals make code hard to read
>> and neither communicate coding professionalism nor comport with published
>> Sun/Oracle style guides. So I suggest that:
>>> For already defined symbolic constants, let’s use them instead of
>> literal values
>>> If there are literal values for which symbolic constants haven’t yet
>> been defined, let’s define symbolic constants for them, commit this change
>> to master, and start using the newly defined symbolic constants
>>> Anyhoo, that’s just my two cents.
>>> 
>>> As always, it’s a pleasure to be a part of Ignite development...
>>> 
>>> Cheers guys & gals,
>>> Lucas
>>> 
>>> --
>>> Lucas BEELER
>>> Technical Consultant, Professional Services
>>> GridGain Systems
>>> www.gridgain.com 
>>> 
 On Jan 18, 2018, at 6:47 PM, Alexey Kuznetsov > > wrote:
 
 Prachi, Denis,
 
 
 It is OK that we are using numbers in code samples instead of named
 constants?
 
 For example:
  writeByteLittleEndian(3, out);  // Integer type code
 
 It could be smth. like this:
  writeByteLittleEndian(TYPE_INT, out);  // Integer type code
 
 Where TYPE_INT is declared at some place:
 
 public static final int TYPE_INT = 3;
 
 
 Make sense?
 
 
 On Fri, Jan 19, 2018 at 4:47 AM, Prachi Garg > > wrote:
 
> Igniters,
> 
> The document for Binary Client Protocol (awaiting 2.4 release) is
>> ready on
> Apache Ignite readme.io . This document explains:
> 
>  - How to connect to Ignite cluster
>  - Common message format (Request/Response header)
>  - Supported Data types and their format
>  - Request/Response format for each message type
> 
> Here are the links -
> 
> https://apacheignite.readme.io/docs/binary-client-protocol <
>> https://apacheignite.readme.io/docs/binary-client-protocol>
> 
> https://apacheignite.readme.io/docs/binary-client-protocol-
> key-value-operations
> 
> https://apacheignite.readme.io/docs/binary-client-
>> protocol-sql-operations
> 
> https://apacheignite.readme.io/docs/binary-client-protocol-
> binary-type-operations
> 
> https://apacheignite.readme.io/docs/binary-client-protocol-
> cache-configuration-operations
> 
> 
> Currently these pages are hidden and are accessible only with the link.
> They will be active after the release.
> 
> -Prachi
> 
 
 
 
 --
 Alexey Kuznetsov
>>> 
>> 
>> 

[jira] [Created] (IGNITE-7481) Suspended transaction rollbacs incorrectly

[Alexey Kuznetsov (JIRA)]

Alexey Kuznetsov created IGNITE-7481:


 Summary: Suspended transaction rollbacs incorrectly  
 Key: IGNITE-7481
 URL: https://issues.apache.org/jira/browse/IGNITE-7481
 Project: Ignite
  Issue Type: Bug
Reporter: Alexey Kuznetsov
Assignee: Alexey Kuznetsov


When we suspend transaction, and then timeout occures , transaction would be 
rollbacked incorrectly.
One of incorrect behaviors : rollback\end tx metrics are not incremented.
Thre reason is that transactionMap is cleared when we suspend transaction.
We need not clear transactionMap.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3409: IGNITE-6711 DataRegionMetrics#totalAllocatedPages...

[anton-vinogradov]

GitHub user anton-vinogradov opened a pull request:

https://github.com/apache/ignite/pull/3409

IGNITE-6711 DataRegionMetrics#totalAllocatedPages is not valid after …

…node restart

Signed-off-by: Anton Vinogradov 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6711

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3409.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3409


commit 7ae5135ec7e5e496a10526b380f078316d527ab5
Author: Anton Vinogradov 
Date:   2018-01-19T16:27:50Z

IGNITE-6711 DataRegionMetrics#totalAllocatedPages is not valid after node 
restart

Signed-off-by: Anton Vinogradov 




---

[GitHub] ignite pull request #3408: IGNITE-7278-2

[DmitriyGovorukhin]

GitHub user DmitriyGovorukhin opened a pull request:

https://github.com/apache/ignite/pull/3408

IGNITE-7278-2



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7278-2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3408.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3408


commit 9755dd25022082be9ce7e175aeeab57e4a614373
Author: Dmitriy Govorukhin 
Date:   2018-01-19T15:12:54Z

IGNITE-7278 fix page corruption on node restart

commit 2c2345fb97a2e8d7c0da7711b3c5198ee076dc69
Author: Dmitriy Govorukhin 
Date:   2018-01-19T15:16:58Z

Merge branch 'master' into ignite-7278-2




---

[GitHub] ignite pull request #3407: IGNITE-7475 Improve VerifyBackupPartitionsTask to...

[glukos]

GitHub user glukos opened a pull request:

https://github.com/apache/ignite/pull/3407

IGNITE-7475 Improve VerifyBackupPartitionsTask to calculate partition 
hashes in multiple threads



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7475

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3407.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3407


commit 6192ef51d2b57c1c65f75a737a64050f5a0c
Author: Ivan Rakov 
Date:   2018-01-19T11:59:03Z

IGNITE-7475 Improve VerifyBackupPartitionsTask to calculate partition 
hashes in multiple threads

commit e852187626430b7e67a11ae15e59406db0406be4
Author: Ivan Rakov 
Date:   2018-01-19T13:07:29Z

IGNITE-7475 Fixed not returning emptyMap

commit 5ab65c4ff5d04e5c2b126c9546643b9ada2db5f5
Author: Ivan Rakov 
Date:   2018-01-19T14:01:17Z

IGNITE-7475 Added cancellation for failed computation




---

[GitHub] ignite pull request #3163: IGNITE-6341: Use direct IO for file page store

[dspavlov]

Github user dspavlov closed the pull request at:

https://github.com/apache/ignite/pull/3163


---

[jira] [Created] (IGNITE-7480) Select for BINARY type field returns 'strange' value.

[Alex Volkov (JIRA)]

Alex Volkov created IGNITE-7480:
---

 Summary: Select for BINARY type field returns 'strange' value.
 Key: IGNITE-7480
 URL: https://issues.apache.org/jira/browse/IGNITE-7480
 Project: Ignite
  Issue Type: Bug
  Components: sql
Affects Versions: 2.2
Reporter: Alex Volkov


Steps to reproduce:
 # Connect to grid using console client and dbc:ignite:thin driver.
 # Run these sql statements:

{code:java}
CREATE TABLE tmp_table_binary (key_field INT PRIMARY KEY,field1 BINARY);
INSERT INTO tmp_table_binary (key_field, field1) values(1, '00ffaa');
INSERT INTO tmp_table_binary (key_field, field1) values(2, 'ffccaa');
SELECT * FROM tmp_table_binary;
{code}
Here is output from H2:
{code:java}
SELECT * FROM tmp_table_binary;

KEY_FIELD   FIELD1  
1   00ffaa
2   ffccaa
{code}
And this one is from grid:
{code}
 0: jdbc:ignite:thin://127.0.0.1/> CREATE TABLE tmp_table_binary (key_field INT 
PRIMARY KEY,field1 BINARY);
No rows affected (0.155 seconds)
0: jdbc:ignite:thin://127.0.0.1/> INSERT INTO tmp_table_binary (key_field, 
field1) values(1, '00ffaa');
1 row affected (0.089 seconds)
0: jdbc:ignite:thin://127.0.0.1/> INSERT INTO tmp_table_binary (key_field, 
field1) values(2, 'ffccaa');
1 row affected (0.009 seconds)
0: jdbc:ignite:thin://127.0.0.1/> SELECT * FROM tmp_table_binary;
+++
|   KEY_FIELD| FIELD1 |
+++
| 1  | [B@6219232f|
| 2  | [B@605514f9|
+++
2 rows selected (0.09 seconds)
0: jdbc:ignite:thin://127.0.0.1/> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7479) NOT NULL constraints for complex PK

[Alex Volkov (JIRA)]

Alex Volkov created IGNITE-7479:
---

 Summary: NOT NULL constraints for complex PK
 Key: IGNITE-7479
 URL: https://issues.apache.org/jira/browse/IGNITE-7479
 Project: Ignite
  Issue Type: Bug
  Components: sql
Reporter: Alex Volkov


According to SQL ANSI 99 spec (11.7  point 5)

If the < unique specification > specifies PRIMARY KEY, then for each < column 
name > in the explicit or implicit < unique column list > for which NOT NULL is 
not specified, NOT NULL is implicit in the < column definition >.

But now we have violation:

 
{code:java}
0: jdbc:ignite:thin://127.0.0.1:10800/> create table 
tmp_table_age_name_wage_complex_pk (key_field INT,AGE INT,field1 VARCHAR,field2 
INT,PRIMARY KEY (key_field, AGE, field1));
SQL: create table tmp_table_age_name_wage_complex_pk (key_field INT,AGE 
INT,field1 VARCHAR,field2 INT,PRIMARY KEY (key_field, AGE, field1))
No rows affected (0.129 seconds)
0: jdbc:ignite:thin://127.0.0.1:10800/> insert into 
tmp_table_age_name_wage_complex_pk (key_field,AGE,field1,field2) values (11, 
null,'John',3);
SQL: insert into tmp_table_age_name_wage_complex_pk 
(key_field,AGE,field1,field2) values (11, null,'John',3)
1 row affected (0.121 seconds)
0: jdbc:ignite:thin://127.0.0.1:10800/> select * from 
tmp_table_age_name_wage_complex_pk;
SQL: select * from tmp_table_age_name_wage_complex_pk
IN IncrementalRows 4
+++++
|   KEY_FIELD|  AGE   | 
FIELD1 | FIELD2 |
+++++
| 11 | null   | John
   | 3  |
+++++
1 row selected (0.04 seconds)
0: jdbc:ignite:thin://127.0.0.1:10800/> 
{code}
The same SQL script with H2 BD got:

 
{code:java}
create table tmp_table_age_name_wage_complex_pk (key_field INT,AGE INT,field1 
VARCHAR,field2 INT,PRIMARY KEY (key_field, AGE, field1));
Update count: 0
(1 ms)

insert into tmp_table_age_name_wage_complex_pk (key_field,AGE,field1,field2) 
values (11, null,'John',3);
NULL not allowed for column "AGE"; SQL statement:
insert into tmp_table_age_name_wage_complex_pk (key_field,AGE,field1,field2) 
values (11, null,'John',3) [23502-195] 23502/23502
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3406: Ignite 2.1.11

[alamar]

GitHub user alamar opened a pull request:

https://github.com/apache/ignite/pull/3406

Ignite 2.1.11



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.1.11

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3406.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3406


commit 49a3631937b09c2fa795d29411635b3b5f82f15a
Author: Eduard Shangareev 
Date:   2017-09-21T15:55:00Z

IGNITE-6460 Fixed consistentId used in lightweight ClusterNode. Fixes #2707

commit fc34091c750cdaf759a54a989a84c76f8d9f9f7f
Author: Igor Sapego 
Date:   2017-09-21T17:02:33Z

IGNITE-6294: ODBC: implemented SQLSTATE management. This closes #2716.

commit ccc8c9b40d09e1d3314a519f1e9417e79c150d9d
Author: Alexey Kuznetsov 
Date:   2017-09-15T16:19:16Z

IGNITE-6120 Fixed version check for ignite-2.2.0.
(cherry picked from commit f4ead8b)

commit cd4e25ecb4912beb85a571b33e0f1096a24bbb7c
Author: Alexey Kuznetsov 
Date:   2017-09-22T03:10:52Z

Versions updated.

commit f08127d50357c4ea24217c096f00942189ed
Author: Alexey Kuznetsov 
Date:   2017-09-22T04:30:12Z

IGNITE-6476 Fixed "No suitable driver found" in demo mode.
(cherry picked from commit 27295f2)

commit 73e1578370c35f3428b097f2f185adf34dace881
Author: Dmitriy Govorukhin 
Date:   2017-09-22T09:11:15Z

IGNITE-6213 Removed locDepOwner flag and marked as deprecated - Fixes #2723.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit 1649c53)

commit 520c2e368baf4cb79912a14663ea3fd1c7da6487
Author: Ivan Rakov 
Date:   2017-09-22T09:40:22Z

IGNITE-6334 Throttle writing threads during ongoing checkpoint - Fixes 
#2710.

Signed-off-by: Alexey Goncharuk 

commit 24f904e6af7d17111bc9ec273d6555b70057e25c
Author: Alexey Goncharuk 
Date:   2017-09-22T10:12:15Z

IGNITE-6334 Fixed compilation after conflict

commit 1ccdc97898b37c7f2240cb222c98899603536fc0
Author: Dmitriy Govorukhin 
Date:   2017-09-22T10:14:57Z

IGNITE-6480 fix serverTopologyNodes method - Fixes #2726.

Signed-off-by: Alexey Goncharuk 

commit 98afbfdab47112cbae722a7317d1af39472e1f88
Author: dpavlov 
Date:   2017-09-22T10:47:14Z

IGNITE-6466 Fixed testGetForInitialWrite - Fixes #2721.

Signed-off-by: Alexey Goncharuk 

commit 212603e1a969b9320f40207cd9233bed9152b3e4
Author: Igor Seliverstov 
Date:   2017-08-09T16:29:39Z

IGNITE-5658 Optimizations for data streamer

(cherry picked from commit aa81dd1)

(cherry picked from commit 1ad4f14)

commit 71cd1e9188f59293c505dedd37c325687f3159f0
Author: Igor Seliverstov 
Date:   2017-08-09T15:46:53Z

ignite-5918 Adding and searching objects in index tree produces a lot of 
garbage

(cherry picked from commit 0e0c2c7)

(cherry picked from commit 20739d8)

commit 1faa8dbed4009b691ee0fd5920aead8ec3e6719c
Author: Yakov Zhdanov 
Date:   2017-08-21T14:28:25Z

IGNITE-6051 Improve future listeners model in DataStreamerImpl

(cherry picked from commit 18ca0b2)

(cherry picked from commit cec55c3)

commit 7daefd2701c7a566d759c2b7de4912ea3f5d37a8
Author: Ivan Rakov 
Date:   2017-09-18T08:35:17Z

IGNITE-6385 Offheap page eviction doesn't work if data is loaded without 
data streamer - Fixes #2666.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit 64b4abe)

(cherry picked from commit 7ca611f)

commit 9f5c9e385e258c48ce29ca4e9cec222f737eb910
Author: Ivan Rakov 
Date:   2017-09-18T13:58:10Z

IGNITE-6355 Calculating cache size during cache stop sporadically fails 
with ClusterGroupEmptyCheckedException - Fixes #2647.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit 0dd9755)

(cherry picked from commit 297ed38)

commit 2310417165d7fd663ff668359ee9a46c775feebf
Author: Igor Seliverstov 
Date:   2017-09-21T16:45:22Z

IGNITE-6101 Try to improve local scans performance

(cherry picked from commit 8d5e73c)

commit 25e4706501c1c0908df6d4cd070f84490c1ef6c4
Author: mcherkasov 
Date:   2017-08-31T09:11:01Z

Fixed NPE in case of null value for indexing.

(cherry picked from commit 6cf3b13)

(cherry picked from commit dd9804b)

commit ae9c6d603d04a46b4603d97d7db986bb33801228
Author: Dmitriy Govorukhin 
Date:   2017-09-22T12:33:37Z

IGNITE-6219 - IgniteCache#loadCache executes local load in caller thread

(cherry picked from commit e5f45fd)

(cherry picked from commit 2a6658e)

commit 21de1c56268c18685dd3620b7e3dc776ca2cf532
Author: Eduard Shangareev 
Date:   2017-09-22T13:17:42Z

IGNITE-6434 Fixed error in checkpointer during topology change. Fixes #2718

commit 6452201dea49d5583e61e7fb3d33fd8c096135bd
Author: Alexei Scherbakov 
Date:   2017-09-22T13:36:38Z

IGNITE-648

[GitHub] ignite pull request #3402: IGNITE-7117 .NET: Improve IgniteHome resolver

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3402


---

[jira] [Created] (IGNITE-7478) Too many HistoryAffinityAssignments in HistAffAssignmentsCache

[Alexander Belyak (JIRA)]

Alexander Belyak created IGNITE-7478:


 Summary: Too many HistoryAffinityAssignments in 
HistAffAssignmentsCache
 Key: IGNITE-7478
 URL: https://issues.apache.org/jira/browse/IGNITE-7478
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Alexander Belyak


Get throuble with GC, found over 26k instances of

org.apache.ignite.internal.processors.affinity.HistoryAffinityAssignment with 
about 12Gb of: ArrayList->Object[]->ArrayList->Object[] but can't find 
ClusterNode objects there!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7477) Ignite.NET does not work on Java 9

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-7477:
--

 Summary: Ignite.NET does not work on Java 9
 Key: IGNITE-7477
 URL: https://issues.apache.org/jira/browse/IGNITE-7477
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Affects Versions: 2.4
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 2.4


When jars are compiled with Java 9, Ignite.NET does not work properly.
Some additional JVM settings are required:

{code}
--add-exports java.base/jdk.internal.misc=ALL-UNNAMED --add-exports 
java.base/sun.nio.ch=ALL-UNNAMED --add-exports 
java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED --add-exports 
jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

cache operation failed after transaction rolled back due to deadlock

[ALEKSEY KUZNETSOV]

Hi, Igntrs!



When you have your transaction rolled back due to detected deadlock, you
are unabled to perform cache operations (in thread where tx was started and
rolled back), because it leads to TransactionTimeoutException.



The reason of such behavior is that tx thread map (txManager#threadMap) was
not cleared from tx when roll back occured.

In GridNearTxLocal#onTimeout you can find comment on that :

*// Note: if rollback asynchronously on timeout should not clear thread map*

*// since thread started tx still should be able to see this tx.*

Cache operation picks up tx from that map and throws exception.



So, one must create new thread in order to perform cache operations?


-- 

*Best Regards,*

*Kuznetsov Aleksey*

[GitHub] ignite pull request #3404: Ignite 7284 2.1 1

[slukyano]

GitHub user slukyano opened a pull request:

https://github.com/apache/ignite/pull/3404

Ignite 7284 2.1 1



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7284-2.1-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3404.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3404


commit e82f077df78972a58fdd06917839445711432705
Author: Dmitriy Govorukhin 
Date:   2017-09-21T15:56:40Z

Merge remote-tracking branch 'professional/ignite-2.1.5' into ignite-2.1.5

commit 49a3631937b09c2fa795d29411635b3b5f82f15a
Author: Eduard Shangareev 
Date:   2017-09-21T15:55:00Z

IGNITE-6460 Fixed consistentId used in lightweight ClusterNode. Fixes #2707

commit fc34091c750cdaf759a54a989a84c76f8d9f9f7f
Author: Igor Sapego 
Date:   2017-09-21T17:02:33Z

IGNITE-6294: ODBC: implemented SQLSTATE management. This closes #2716.

commit ccc8c9b40d09e1d3314a519f1e9417e79c150d9d
Author: Alexey Kuznetsov 
Date:   2017-09-15T16:19:16Z

IGNITE-6120 Fixed version check for ignite-2.2.0.
(cherry picked from commit f4ead8b)

commit cd4e25ecb4912beb85a571b33e0f1096a24bbb7c
Author: Alexey Kuznetsov 
Date:   2017-09-22T03:10:52Z

Versions updated.

commit f08127d50357c4ea24217c096f00942189ed
Author: Alexey Kuznetsov 
Date:   2017-09-22T04:30:12Z

IGNITE-6476 Fixed "No suitable driver found" in demo mode.
(cherry picked from commit 27295f2)

commit 73e1578370c35f3428b097f2f185adf34dace881
Author: Dmitriy Govorukhin 
Date:   2017-09-22T09:11:15Z

IGNITE-6213 Removed locDepOwner flag and marked as deprecated - Fixes #2723.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit 1649c53)

commit 520c2e368baf4cb79912a14663ea3fd1c7da6487
Author: Ivan Rakov 
Date:   2017-09-22T09:40:22Z

IGNITE-6334 Throttle writing threads during ongoing checkpoint - Fixes 
#2710.

Signed-off-by: Alexey Goncharuk 

commit 24f904e6af7d17111bc9ec273d6555b70057e25c
Author: Alexey Goncharuk 
Date:   2017-09-22T10:12:15Z

IGNITE-6334 Fixed compilation after conflict

commit 1ccdc97898b37c7f2240cb222c98899603536fc0
Author: Dmitriy Govorukhin 
Date:   2017-09-22T10:14:57Z

IGNITE-6480 fix serverTopologyNodes method - Fixes #2726.

Signed-off-by: Alexey Goncharuk 

commit 98afbfdab47112cbae722a7317d1af39472e1f88
Author: dpavlov 
Date:   2017-09-22T10:47:14Z

IGNITE-6466 Fixed testGetForInitialWrite - Fixes #2721.

Signed-off-by: Alexey Goncharuk 

commit 212603e1a969b9320f40207cd9233bed9152b3e4
Author: Igor Seliverstov 
Date:   2017-08-09T16:29:39Z

IGNITE-5658 Optimizations for data streamer

(cherry picked from commit aa81dd1)

(cherry picked from commit 1ad4f14)

commit 71cd1e9188f59293c505dedd37c325687f3159f0
Author: Igor Seliverstov 
Date:   2017-08-09T15:46:53Z

ignite-5918 Adding and searching objects in index tree produces a lot of 
garbage

(cherry picked from commit 0e0c2c7)

(cherry picked from commit 20739d8)

commit 1faa8dbed4009b691ee0fd5920aead8ec3e6719c
Author: Yakov Zhdanov 
Date:   2017-08-21T14:28:25Z

IGNITE-6051 Improve future listeners model in DataStreamerImpl

(cherry picked from commit 18ca0b2)

(cherry picked from commit cec55c3)

commit 7daefd2701c7a566d759c2b7de4912ea3f5d37a8
Author: Ivan Rakov 
Date:   2017-09-18T08:35:17Z

IGNITE-6385 Offheap page eviction doesn't work if data is loaded without 
data streamer - Fixes #2666.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit 64b4abe)

(cherry picked from commit 7ca611f)

commit 9f5c9e385e258c48ce29ca4e9cec222f737eb910
Author: Ivan Rakov 
Date:   2017-09-18T13:58:10Z

IGNITE-6355 Calculating cache size during cache stop sporadically fails 
with ClusterGroupEmptyCheckedException - Fixes #2647.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit 0dd9755)

(cherry picked from commit 297ed38)

commit 2310417165d7fd663ff668359ee9a46c775feebf
Author: Igor Seliverstov 
Date:   2017-09-21T16:45:22Z

IGNITE-6101 Try to improve local scans performance

(cherry picked from commit 8d5e73c)

commit 25e4706501c1c0908df6d4cd070f84490c1ef6c4
Author: mcherkasov 
Date:   2017-08-31T09:11:01Z

Fixed NPE in case of null value for indexing.

(cherry picked from commit 6cf3b13)

(cherry picked from commit dd9804b)

commit ae9c6d603d04a46b4603d97d7db986bb33801228
Author: Dmitriy Govorukhin 
Date:   2017-09-22T12:33:37Z

IGNITE-6219 - IgniteCache#loadCache executes local load in caller thread

(cherry picked from commit e5f45fd)

(cherry picked from commit 2a6658e)

commit 21de1c56268c18685dd3620b7e3dc776ca2cf532
Author: Eduard Shangareev 
Date:   2017-09-22T13:17:42Z

[GitHub] ignite pull request #3405: Ignite 7284 2.3 1

[slukyano]

GitHub user slukyano opened a pull request:

https://github.com/apache/ignite/pull/3405

Ignite 7284 2.3 1



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7284-2.3-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3405.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3405


commit be91bbf8bcb7e9c71fe6d3bea0f79763f9606558
Author: Krzysztof Chmielewski 
Date:   2017-10-10T14:50:59Z

Fixed "IGNITE-6234 Initialize schemaIds to empty set if schemas field is 
null during the deserialization".

Signed-off-by: nikolay_tikhonov 

commit 08389601728512dc4e7fa5b953f5afe34ae4506f
Author: AMRepo 
Date:   2017-10-10T08:57:20Z

IGNITE-6545: Failure during Ignite Service.cancel() can break normal 
shutdown process. This closes #2807.

(cherry picked from commit 8ffa109)

commit 57547b5afae059a0a6dfa13c08b2e0b6c0e96ebd
Author: devozerov 
Date:   2017-10-13T09:34:35Z

Merge branch 'ignite-2.3.1' into ignite-2.3.2

commit 08798f8e47bdfdd68a557385ed2ce98b4bb1609a
Author: devozerov 
Date:   2017-10-13T11:12:44Z

IGNITE-6605: SQL: common backup filter. This closes #2836.

commit 2b59a241de3935a338842b8fc3221aedc8e11e1d
Author: devozerov 
Date:   2017-10-16T07:33:36Z

IGNITE-6631: Minor improvements to GridH2KeyValueRowOnheap. This closes 
#2855.

commit 98438c954c5f9a08634cf3132361268456397864
Author: devozerov 
Date:   2017-10-16T09:38:54Z

IGNITE-6632: SQL: simplified GridH2Row inheritance tree. This closes #2856.

commit 95b7ab518dd3c3db6fcc5142c2ee85da2516c2b6
Author: devozerov 
Date:   2017-10-16T10:37:11Z

IGNITE-6634: Removed IgniteDistributedJoinTestSuite. It's tests are 
distributed between "Query" and "Query 2" suites. This closes #2857.

commit 9c91deff877ebc0eed84559d4abca71408e3cb0a
Author: devozerov 
Date:   2017-10-16T13:46:13Z

Merge branch 'ignite-2.3.1' into ignite-2.3.2

commit 911ab7ab7a8a6968d219b053adb2338477738506
Author: Alexey Popov 
Date:   2017-10-17T11:45:42Z

IGNITE-6627 .NET: Fix serialization of enums within generic collections

* Fix EnumEqualityComparer serialization
* Fix enum arrays serialization
* Fix empty objects missing metadata

This closes #2864

commit 3ba374c319ac7048a05871692060e2f143d6acdf
Author: Alexey Kuznetsov 
Date:   2017-10-06T17:11:37Z

IGNITE-6463 Web Console: Fixed output of big numbers in SQL query results.
(cherry picked from commit 35589a7)

commit b67feb0f175bfbd6ffbefe82a8d693c8ab7d4213
Author: Vasiliy Sisko 
Date:   2017-10-09T10:55:23Z

IGNITE-5767 Web console: Use byte array type instead of java.lang.Object 
for binary JDBC types.
(cherry picked from commit 3184437)

commit 8e1560322b87d79b3d3250832a3969ac4032d6fc
Author: Alexey Kuznetsov 
Date:   2017-10-06T18:10:08Z

IGNITE-6574 Remove pending requests in case STATUS_AUTH_FAILURE && 
credentials == null.
(cherry picked from commit 85261a3)

commit 7a0300ae35894c389b126e95615f720a99a3d360
Author: devozerov 
Date:   2017-10-18T11:18:08Z

Merge branch 'ignite-2.3.1' into ignite-2.3.2

commit ad01f9b099d0bf92537378859ad6d5a52de57748
Author: Alexey Kuznetsov 
Date:   2017-10-19T02:43:20Z

IGNITE-6647 Web Console: Implemented support of schema migration scripts.
(cherry picked from commit c65399c)

commit 0c66344bc752dac98b256dd140fcab95d1662862
Author: Pavel Tupitsyn 
Date:   2017-10-19T09:36:39Z

IGNITE-6627 .NET: Fix repeated known metadata updates

This closes #2876

commit 1b8abd214ed2afcd3fd1f6a4c71a19d6fe1a4b01
Author: Alexey Kuznetsov 
Date:   2017-10-20T04:23:23Z

IGNITE-6647 Added missing Mongo injector.
(cherry picked from commit 173ecef)

commit a221066b3d029afc392be704a810c0e830fc0c49
Author: Alexey Kuznetsov 
Date:   2017-10-20T14:15:02Z

IGNITE-6647 Web Console: Added folder for modules migrations.
(cherry picked from commit 3700717)

commit da8a9d5a968ba071697a28adb01bc59f80d1893c
Author: Pavel Tupitsyn 
Date:   2017-10-23T08:55:33Z

Merge branch 'ignite-2.3.1' into ignite-2.3.2

# Conflicts:
#   
modules/platforms/dotnet/Apache.Ignite.Core.Tests/Apache.Ignite.Core.Tests.csproj

commit 69fdac3acf768ecb9df80d4412c4de5ffd5bc4f5
Author: Dmitriy Shabalin 
Date:   2017-10-23T09:09:47Z

IGNITE-5909 Added list editable component.
(cherry picked from commit 01daee6)

commit 4a2c38333c112d4956d6394667672c1470503435
Author: apopov 
Date:   2017-10-24T08:56:33Z

IGNITE-6362 NPE in Log4J2Logger

commit 089ebecb3e5962c7a38afd01bd18c77feb23d155
Author: vsisko 
Date:   2017-10-25T04:23:11Z

IGNITE-6671 Web Agent: Fixed data type conversion for Oracle NUMBER(N) data 
types.
(cherry picked from commit 93be8ea)

commit 1e56de86525a79c895eba2d839b7887b61979b07
Author: Denis Mekhanikov

[GitHub] ignite pull request #3403: Ignite 7284 1.9 1

[slukyano]

GitHub user slukyano opened a pull request:

https://github.com/apache/ignite/pull/3403

Ignite 7284 1.9 1



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7284-1.9-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3403.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3403


commit 710db327c027d43c0984b7007447cecca71f3cca
Author: Andrey V. Mashenkov 
Date:   2017-06-28T08:47:21Z

Rethrow handshake exceptions as it done for ConnectionException.

(cherry picked from commit 7d52172)

commit 7c619f77009cc6876405839f9f6201d332d94b47
Author: Andrey V. Mashenkov 
Date:   2017-06-28T12:45:42Z

Fixed tests.

(cherry picked from commit 81cdea4)

commit a3717b0cc6d716e3bc966248854334e74b0614e2
Author: Andrey V. Mashenkov 
Date:   2017-06-29T10:06:06Z

Merge remote-tracking branch 'origin/ignite-1.7.12' into ignite-1.7.12

commit 7d41f5da867294befd6cca1de3c945bfd83ae597
Author: Andrey V. Mashenkov 
Date:   2017-06-29T10:15:57Z

Merge branch 'ignite-1.8.8' into ignite-1.9.4

commit 5219a35208b95a2d7dea3825dd4cb4edc74b3542
Author: Slava Koptilin 
Date:   2017-06-26T12:37:26Z

IGNITE-5076: improved multi-threaded start of nodes

commit aba9bf9982c6eba8cb57d64e6b1090914b8d020e
Author: Andrey V. Mashenkov 
Date:   2017-06-30T08:30:20Z

Merge branch 'ignite-1.8.8' into ignite-1.9.4

commit a4fc555b118d91ec0348154b88764f010dbbae52
Author: nikolay_tikhonov 
Date:   2017-06-30T11:38:54Z

Fixed "IGNITE-5424 GridServiceProxy does not unwraps exception message from 
InvocationTargetException." This closes #2168

Signed-off-by: nikolay_tikhonov 

commit 62876fefc8dbd334db9dc6741c2bc91b3409343d
Author: Andrey V. Mashenkov 
Date:   2017-06-30T11:45:18Z

IGNITE-5473 partial fix: Create ignite troubleshooting logger.

commit 75c442a8309d554f6894f30f63ed0b964e7a0aa1
Author: Slava Koptilin 
Date:   2017-06-26T12:37:26Z

Backported IGNITE-5076: improved multi-threaded start of nodes.

commit 551923e1e5139c484fcb716e7f10907fa71021d9
Author: nikolay_tikhonov 
Date:   2017-06-30T11:38:54Z

Backported "IGNITE-5424 GridServiceProxy does not unwraps exception message 
from InvocationTargetException."

commit 752b1368adbb5c77b5d5caca3c07a72decff5111
Author: Andrey V. Mashenkov 
Date:   2017-06-30T11:45:18Z

IGNITE-5473 partial fix: Create ignite troubleshooting logger.

commit fd3f947a1b49144a5b11aaf3f1a1b7a28ba961fd
Author: Andrey V. Mashenkov 
Date:   2017-06-30T15:36:50Z

Fixed service deployment tests.

commit 6ab152a85f70c05847823f65f8e095ab9eb6b1f7
Author: sboikov 
Date:   2017-04-19T09:46:31Z

Attempt to fix awaitPartitionMapExchange: wait for last exchange completion 
to avoid races with cache destroy.

(cherry picked from commit d383484)

commit 1525c6cf2cb015289392eb54fec4029e9b53b438
Author: Andrey V. Mashenkov 
Date:   2017-06-30T15:36:50Z

Fixed service deployment tests.

commit a24ca24dd7aaa34707d74a4e660d769d3d5b0ed8
Author: sboikov 
Date:   2017-04-19T09:46:31Z

Attempt to fix awaitPartitionMapExchange: wait for last exchange completion 
to avoid races with cache destroy.

(cherry picked from commit d383484)

commit 05fac7e8a0bf9e08cd758bed7bd35ec85b914592
Author: Evgenii Zhuravlev 
Date:   2017-04-19T11:01:21Z

Backported IGNITE-4925 Fix 
IgniteCacheBinaryObjectsScanSelfTest.testScanNoClasses - Fixes #1750.

(cherry picked from commit b47f29d)

commit ef0a874ceb5c8bfa53e16337f6fd1699afaf2a39
Author: nikolay_tikhonov 
Date:   2017-06-30T17:39:01Z

Fixed CacheSerializableTransactionsTest#testTxConflictRemoveWithOldValue 
test.

Signed-off-by: nikolay_tikhonov 

commit 4dce965ea86374cba7265cb5d22e975aeac7d480
Author: nikolay_tikhonov 
Date:   2017-06-30T18:36:02Z

Fixed org.jsr107.tck.PutTest tests.

Signed-off-by: nikolay_tikhonov 

commit 32f1e394c222a6f4a2c111d6b6284c8626442b68
Author: Andrey V. Mashenkov 
Date:   2017-07-03T09:28:12Z

Merge branch 'ignite-1.8.8' into ignite-1.9.4

commit 50887fed508e03a8b7df32569afb6d84ab3eb670
Author: Igor Sapego 
Date:   2017-07-04T17:01:01Z

IGNITE-5663: ODBC: Closing cursor do not reset prepared statement anymore

commit da290cee855ef45a90ad539515e039f2826a6c00
Author: Igor Sapego 
Date:   2017-07-05T10:21:12Z

IGNITE-5663: Fix for test

commit 024a01d6bf91b4f301c4aee7f4a747e810a9a30b
Author: nikolay_tikhonov 
Date:   2017-07-05T15:58:00Z

Merged 1.7.12 into 1.8.9

Signed-off-by: nikolay_tikhonov 

commit 3536a58982e4c264bb72b2ccc1953049d2b5c67f
Author: Alexey Kukushkin 
Date:   2017-07-05T16:36:41Z

IGNITE-4901 Decrease logging level for DataStremer retry

commit 6d3a3ff2d99697882232070e715928336a9180cd
Author: Alexey Kukushkin 
Date:   2017-07-05T17:05:02Z

[GitHub] ignite pull request #3393: IGNITE-7454 Wrong package in IgniteExamplesMLTest...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3393


---

[jira] [Created] (IGNITE-7476) Server node will join with failure gathering metrics

[Ilya Kasnacheev (JIRA)]

Ilya Kasnacheev created IGNITE-7476:
---

 Summary: Server node will join with failure gathering metrics
 Key: IGNITE-7476
 URL: https://issues.apache.org/jira/browse/IGNITE-7476
 Project: Ignite
  Issue Type: Bug
Reporter: Ilya Kasnacheev


Sometimes server node will fail with the following trace:
{code:java}
SEVERE: TcpDiscoverSpi's message worker thread failed abnormally. Stopping the 
node in order to prevent cluster wide instability.
java.lang.NullPointerException
    at 
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$7.cacheMetrics(GridDiscoveryManager.java:1149)
    at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMetricsUpdateMessage(ServerImpl.java:5022)
    at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2690)
    at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2491)
    at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6675)
    at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2574)
    at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62){code}
Two problems here:
 * Uncaught exception in cacheMetrics() leads to unconditional failure of node, 
because it happens to be in discovery thread. Should probably wrap all 
non-trivial code include try-catch.
 * Lack of proper locking when destroying cache (see also IGNITE-6423 and 
IGNITE-7165)

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7475) Improve VerifyBackupPartitionsTask to calculate partition hashes in multiple threads

[Ivan Rakov (JIRA)]

Ivan Rakov created IGNITE-7475:
--

 Summary: Improve VerifyBackupPartitionsTask to calculate partition 
hashes in multiple threads
 Key: IGNITE-7475
 URL: https://issues.apache.org/jira/browse/IGNITE-7475
 Project: Ignite
  Issue Type: Improvement
Reporter: Ivan Rakov
Assignee: Ivan Rakov


Currently, compute task VerifyBackupPartitionsTask calculates all hashes in 
ComputeJob#execute caller thread. Using multiple threads can bring significant 
speedup.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7474) Exception on accessing sys cache when node joins topology with service

[Ilya Kasnacheev (JIRA)]

Ilya Kasnacheev created IGNITE-7474:
---

 Summary: Exception on accessing sys cache when node joins topology 
with service
 Key: IGNITE-7474
 URL: https://issues.apache.org/jira/browse/IGNITE-7474
 Project: Ignite
  Issue Type: Bug
  Components: compute
Affects Versions: 2.3
Reporter: Ilya Kasnacheev


Sometimes when node joins to a cluster with service configured, it will get the 
following exception:
{code:java}
Exception in thread "sys-#1071" java.lang.IllegalArgumentException: Cache is 
not configured: ignite-sys-cache
    at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.jcache(GridCacheProcessor.java:3423)
    at 
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.handleEvent(CacheContinuousQueryHandler.java:770)
    at 
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.notifyCallback0(CacheContinuousQueryHandler.java:742)
    at 
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.notifyCallback(CacheContinuousQueryHandler.java:696)
    at 
org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.processNotification(GridContinuousProcessor.java:1110)
    at 
org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.access$1600(GridContinuousProcessor.java:103)
    at 
org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$7.onMessage(GridContinuousProcessor.java:755)
    at 
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555)
    at 
org.apache.ignite.internal.managers.communication.GridIoManager.access$4100(GridIoManager.java:126)
    at 
org.apache.ignite.internal.managers.communication.GridIoManager$GridCommunicationMessageSet.unwind(GridIoManager.java:2751)
    at 
org.apache.ignite.internal.managers.communication.GridIoManager.unwindMessageSet(GridIoManager.java:1515)
    at 
org.apache.ignite.internal.managers.communication.GridIoManager.access$4400(GridIoManager.java:126)
    at 
org.apache.ignite.internal.managers.communication.GridIoManager$10.run(GridIoManager.java:1484)
    at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
{code}
Seems to be a race condition. I will try to provide test case later.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Transport compression (not store compression)

[Nikita Amelchev]

Hello, Igniters!

I implement network compression as described earlier [1]. I implemented
Zstd, LZ4, and Deflater [2, 3]. LZ4 have good speed (1.2 G/s per
connection) but compression ratio is not more than 2. Zstd has good
compression ratio, up to 5, but relatively slow throughputs (0.28 G/s per
connection). In addition, we boost SSL throughput from 0.8 G/s up to 1.0
G/s, because we encrypt fewer data.

Alexander (sharple...@gmail.com) has developed benchmarks for a network.
They measure effective and real throughput, compression ratio, and latency.
The test generates data with a structure similar to natural language. At
first, generated "words" of lengths from the Poisson distribution. Each
word is a byte array filled randomly. These "words" form a "language" that
is being added to the buffer by Zipf's law distribution until the buffer is
full.

In case SSL is turned on results follow:

On 500kb messages, Zstd shows best results per thread (Real throughput
0.202 G/s, compression ratio 5), LZ4 is second (Real throughput 0.498 G/s,
compression ratio 1.92). A base throughput of SSL is 0.76 G/s. Enabling
Zstd compression boosts effective throughput up to 30%. LZ4 - to 26%.
On 1kb messages, Zstd shows best results per thread (Real throughput 0.042
G/s, compression ratio 2.5), LZ4 is second (Real throughput 0.065 G/s,
compression ratio 1.74). A real throughput of SSL is 0.107 G/s. Enabling
LZ4 compression boosts effective throughput up to 6%. Zstd only compresses
the data.

These are good results even if a network is not fully utilized. Throughput
can be boosted up to several times if network speed is limited. I wrote the
result into comments of the issue [1].
Now I cover code with tests.

Any thoughts?

1. https://issues.apache.org/jira/browse/IGNITE-7024
2. https://reviews.ignite.apache.org/ignite/review/IGNT-CR-467
3. https://github.com/apache/ignite/pull/3101/files

2017-11-30 13:00 GMT+03:00 Vladimir Ozerov :

> I would start with communication only, and put discovery and clients aside
> for now. Let's confirm that communication works well with compression first
> in terms of performance.
>
> On Thu, Nov 30, 2017 at 10:36 AM, Nikita Amelchev 
> wrote:
>
> > Hello, everybody.
> > I propose the following design for network compression.
> >
> > I suggest to implement it like SSL implementation that already works. I
> add
> > compression filter to a chain of NIO filters. It changes the logic of
> > TcpCommunicationSpi.safeTcpHandshake() and
> > DirectNioClientWorker.processWrite() methods, where depending on whether
> > SSL and/or compression are turned on we compress and/or encrypt data and
> > messages using compress handler. Also, I will use application buffer
> > (implemented in "GridNioCompressHandler") for decompressing.
> >
> > For compression verification, I have to write the header with a length of
> > compressed data (or get it from an algorithm headers) and a flag of
> > compression disabled (for small messages). When we are reading compressed
> > data from a channel, we check length, the same way as an isInboundDone
> flag
> > is checked in SSL implementation. It is conveniently to implement this
> flag
> > in "CompressEngine" with wrap and unwrap methods for compress and
> > decompress to byte buffer. Compression settings should be placed in a
> > configuration (Ignite configuration/discovery SPI and
> > GridClientConfiguration).
> >
> > Any thoughts?
> >
> >
> > 2017-11-28 10:39 GMT+03:00 Nikita Amelchev :
> >
> > > Hi,
> > > I've filed a ticket [1]. I'll try to share design details in a couple
> of
> > > days.
> > >
> > > 1. https://issues.apache.org/jira/browse/IGNITE-7024
> > >
> > > 2017-11-23 18:31 GMT+03:00 Denis Magda :
> > >
> > >> Nikita,
> > >>
> > >> Sounds like a good plan. Please share the design details prior getting
> > >> down to the implementation.
> > >>
> > >> —
> > >> Denis
> > >>
> > >> > On Nov 23, 2017, at 4:38 AM, Nikita Amelchev 
> > >> wrote:
> > >> >
> > >> > Hi Igniters!
> > >> >
> > >> > I’m working on the similar feature for my own project.
> > >> > I would like to suggest use in-line compression and write encoded
> > bytes
> > >> in
> > >> > network channel by bytes array buffer. It allows us avoiding
> expensive
> > >> > memory allocation.
> > >> > The described design may be implemented in TcpCommunicationSpi
> level.
> > We
> > >> > can introduce pluggable compressor on TCP level where we will be
> able
> > to
> > >> > describe our compression strategy, for example, exclude some small
> > >> messages
> > >> > and many other.
> > >> >
> > >> > If the community doesn't mind I will file the ticket and will start
> > >> > implementing it.
> > >> > Any thoughts?
> > >> >
> > >> > 2017-11-23 12:06 GMT+03:00 Vladimir Ozerov :
> > >> >
> > >> >> Denis,
> > >> >>
> > >> >> Regarding zipped marshaller - this would be inefficient, because
> > >> >> compression rate will be lower.
> > >> >>
> > >> >> On Thu, Nov 23, 2017 at 1:01 AM, Denis Magda 
> > >> wrote:
> > >> >>
> > >

[SparkDataFrame] Query Optimization. Prototype

[Nikolay Izhikov]

Hello, guys.

I have done prototype of implementation of optimization query from Spark to 
Ignite [1].

Please, take a look at PR [2].

But still there are some issues I want to clarify with community:

* Can I know scale and precision of DECIMAL column? 
  Example - [3]

* Ignite have some limitation for a *distributed* join. For example, we can 
execute join only for indexed columns. Example - [4]. 
* Do we have documentation for all limitations of distributed join 
implementation?
* How can I know whether concrete syntactically correct SQL query with 
join can be executed by Ignite or not?

[1] https://issues.apache.org/jira/browse/IGNITE-7077
[2] https://github.com/apache/ignite/pull/3397
[3] https://gist.github.com/nizhikov/2d67ed68afd473d7251dab7d79135513
[4] https://gist.github.com/nizhikov/a4389fd78636869dd38c13920b5baf2b

[GitHub] ignite pull request #3101: IGNITE-7024

[NSAmelchev]

GitHub user NSAmelchev reopened a pull request:

https://github.com/apache/ignite/pull/3101

IGNITE-7024



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/NSAmelchev/ignite comress-with-ssl

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3101.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3101


commit 11424e9eeefc8fe1b8575623c18a8d565a376ab9
Author: NSAmelchev 
Date:   2017-11-28T14:25:13Z

Merge remote-tracking branch 'refs/remotes/apache/master'

commit ba4086e282b1a78fac6e0e9832eb02d6f428250f
Author: NSAmelchev 
Date:   2017-11-10T07:42:24Z

draft

commit 15566a4f91bdcad117ef2b76b202531ae2ad91e4
Author: NSAmelchev 
Date:   2017-11-20T13:15:40Z

compressV2.

commit f7162f96b36f2756ecf63000192ee7facbce0e77
Author: NSAmelchev 
Date:   2017-11-21T13:24:46Z

change size of buffers

commit bcc43b501700958fce927250b1b76bfb2511a328
Author: NSAmelchev 
Date:   2017-11-28T11:07:38Z

draft

commit 6a5df699fea2a5cf53a097571c7206720fbbcd29
Author: NSAmelchev 
Date:   2017-11-30T07:37:52Z

some fix

commit 6343f08f2e74054cbe0c656c4d7a933a165e023d
Author: NSAmelchev 
Date:   2017-12-06T16:54:58Z

some fix

commit ea687fb1b299cacc432f96c1b894a40108a4a6b8
Author: NSAmelchev 
Date:   2017-12-07T11:20:44Z

some fix

commit f0496708fc39de787b0eedf5343bc8565fb46e68
Author: NSAmelchev 
Date:   2017-12-07T11:48:44Z

for tests

commit ceb0ce44e5882f295c9b75ee2e7d5feae7ae519c
Author: NSAmelchev 
Date:   2017-12-08T07:18:43Z

proceedSessionOpened on not_handshaking

commit 655a85a7f72fdac5f98dab241140359da303bfdd
Author: NSAmelchev 
Date:   2017-12-08T14:30:48Z

default disable compresss and ssl

commit 106d0608bace7b80e0d328abef4f0b53068a1930
Author: NSAmelchev 
Date:   2017-12-11T16:20:06Z

refactor TcpCommunicationSpi

commit db037e2bca7db4c2d34395e537c7d7c22f9e06b1
Author: Alexander Menshikov 
Date:   2017-12-06T14:38:13Z

remove some null checks

commit c8d2f2fb16f501c34fe6fa079e5518035bfe35d3
Author: Alexander Menshikov 
Date:   2017-12-07T17:34:01Z

refactoring...

commit 0e58441a820974629b08b03f9e14d78d7e234806
Author: Alexander Menshikov 
Date:   2017-12-07T17:59:13Z

refactoring

commit db5b8b1c9cb44f361b75d55fd2a5c49b5f5ca1c7
Author: Alexander Menshikov 
Date:   2017-12-07T18:57:31Z

pam-pam)

commit 87f708013498e52af26afb47a92024e4cc86ad52
Author: Alexander Menshikov 
Date:   2017-12-08T09:46:33Z

add optimization for no-filters mode

commit 8f155640ffc9fd039efc5ff756338ed2fb911777
Author: Alexander Menshikov 
Date:   2017-12-08T10:58:02Z

fix bug

commit 5a69ae1cb0fb32cc607dc42c6c7944677a27eb99
Author: Alexander Menshikov 
Date:   2017-12-11T18:12:02Z

Fix bugs

commit a5fb4a0e0b55545005008c32a4fa5242192c5fc4
Author: Alexander Menshikov 
Date:   2017-12-12T12:11:56Z

refactoring

commit ac86ba1661705d17fa8dd7f15042c762c4ba7a31
Author: NSAmelchev 
Date:   2017-12-12T12:18:16Z

Merge pull request #17 from NSAmelchev/ignite-net-compress-2

Ignite net compress 2

commit 745f518963ede54e17f563a961d4cf0e5d783393
Author: NSAmelchev 
Date:   2017-12-12T13:36:46Z

some refactor

commit 3e1a372b6ec75cd10e99890977f6bcaedf15479a
Author: NSAmelchev 
Date:   2017-12-13T10:00:21Z

workaround for ssl deadlock on close and receive

commit 56f01874eb4fe1efbf9cf6579b12593660a81ceb
Author: NSAmelchev 
Date:   2017-12-13T15:48:49Z

use new format in deflater. Add small msg flag to gzip

commit 111d4e0caddda948a1be484b25f227d1e01e2add
Author: Alexander Menshikov 
Date:   2017-12-13T16:27:20Z

add benchmarks

commit f14aa409f8326b088caab84979cfc6d41d1b18aa
Author: NSAmelchev 
Date:   2017-12-14T08:34:41Z

Merge remote-tracking branch 'origin/test-comress-with-ssl' into 
comress-with-ssl

commit 054e14caa0df0875f642d10c74a6a65551a914db
Author: NSAmelchev 
Date:   2017-12-14T12:45:58Z

reuse deflateBaos

commit deb771ce916a53a6380892fc97af2eba2370ec4f
Author: NSAmelchev 
Date:   2017-12-14T13:14:00Z

optimize gzip

commit 0eb9f6e4af822d07c4ebd250e7106e4d617618fa
Author: NSAmelchev 
Date:   2017-12-14T16:00:25Z

correct expand out buf

commit 9ae441bf0f04c55b0a622f8a9e00af645a913464
Author: NSAmelchev 
Date:   2017-12-15T08:07:25Z

correct BUFFER_OVERFLOW for BlockingCompressHandler




---

[GitHub] ignite pull request #3101: NetworkCompression

[NSAmelchev]

Github user NSAmelchev closed the pull request at:

https://github.com/apache/ignite/pull/3101


---

[jira] [Created] (IGNITE-7473) .NET: "Could not load file or assembly ConfigurationManager" error on Linux

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-7473:
--

 Summary: .NET: "Could not load file or assembly 
ConfigurationManager" error on Linux
 Key: IGNITE-7473
 URL: https://issues.apache.org/jira/browse/IGNITE-7473
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Affects Versions: 2.4
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 2.4


Steps to reproduce:

{code}
dotnet new console
dotnet add package Apache.Ignite
nano Program.cs  # Add Ignition.Start() line and a namespace import
dotnet run
{code}

Error:
{code}
Unhandled Exception: System.IO.FileNotFoundException: Could not load file or 
assembly 'System.Configuration.ConfigurationManager, Version=0.0.0.0, 
Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the 
file specified.
{code}

Works on Windows, does not work on Linux.
{{}} is required to fix the issue.

However, {{ConfigurationManager}} is not necessary for code configuration, we 
should be able to start without it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7472) Exchange initiated by custom disco events could hang in case of coordinator leave

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-7472:
---

 Summary: Exchange initiated by custom disco events could hang in 
case of coordinator leave
 Key: IGNITE-7472
 URL: https://issues.apache.org/jira/browse/IGNITE-7472
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.4
Reporter: Vladimir Ozerov
 Fix For: 2.5


Reproducer: {{WalModeChangeAdvancedSelfTest.testServerRestartCoordinator}}

WAL mode change operation initiates an exchange through custom discovery event. 
The test does the following:
1) Initiate constant flow of such events from one node
2) Constantly stops current coordinator node and start another one sever node 

Debug shows the following:
1) Remaining client nodes is on exchange X, waiting for new coordinator to send 
affinity message
2) New coordinator is on exchange [X+1], waiting for client to send partition 
message



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3402: IGNITE-7117 .NET: Improve IgniteHome resolver

[ptupitsyn]

GitHub user ptupitsyn opened a pull request:

https://github.com/apache/ignite/pull/3402

IGNITE-7117 .NET: Improve IgniteHome resolver



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ptupitsyn/ignite ignite-7117

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3402.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3402


commit e575d58f1bccd0a928e0a281c397a4dbaa737f8f
Author: Pavel Tupitsyn 
Date:   2018-01-19T08:40:18Z

IGNITE-7117 .NET: Classpath resolver relies on Java examples

commit 899295efc9c63b7876c143e37d05d6fe28e47392
Author: Pavel Tupitsyn 
Date:   2018-01-19T09:05:09Z

wip

commit 177fc20087f450fce1e00e1e8b51b3ea0b6b0252
Author: Pavel Tupitsyn 
Date:   2018-01-19T09:18:51Z

Add source release resolver




---

[GitHub] ignite pull request #3384: IGNITE-7248 "Schema not found" error when setting...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3384


---

[GitHub] ignite pull request #3342: IGNITE-6772: SQL exception messages made more inf...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3342


---

[jira] [Created] (IGNITE-7471) Use soft reference for checkpoint entry contents to avoid excessive memory usage

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-7471:


 Summary: Use soft reference for checkpoint entry contents to avoid 
excessive memory usage
 Key: IGNITE-7471
 URL: https://issues.apache.org/jira/browse/IGNITE-7471
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexey Goncharuk






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3309: IGNITE-6456: Added opportunity to enable/disable ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3309


---