Re: Problem with reading incomplete payload - IGNITE-7153

[Michael Fong]

bump :)

On Fri, Oct 26, 2018 at 4:00 PM Michael Fong  wrote:

> Hi,
>
> Thanks for your reply. I think current implementation (based on R2.6) does
> exactly what you mentioned as of in GridNioServer
> .
> The problem is that in redis protocol defines message length right before
> the message context. If a huge message comes in and unfortunately Selector
> / NioServer has not read the whole payload fully. Furthermore, the
> incomplete message is passed to Redis packet parser, then logic error would
> occur - in this case BufferUnderFlowError. I wonder if this is expected
> behavior of how Ignite receive data from server perspective?
>
> Regards,
>
> Michael
>
> On Fri, Oct 26, 2018 at 1:13 PM Jörn Franke  wrote:
>
>> I have not checked the exact routine, but you need to reexecute read
>> until it returns -1 or if known that all bytes that are expected have been
>> read (if sockCh is based on some kind of input stream)
>>
>> > Am 26.10.2018 um 04:24 schrieb Michael Fong :
>> >
>> > Bump!
>> >
>> > Anyone who could help me solve this random issue? Thanks!
>> >
>> > Regards,
>> >
>> >
>> > Michael
>> >
>> >> On Wed, Oct 24, 2018 at 10:32 PM Michael Fong 
>> wrote:
>> >>
>> >> Hi, all,
>> >>
>> >>
>> >> I was trying to fix  IGNITE-7153 which relates to parsing incomplete
>> REDIS
>> >> packet larger than 8192 bytes. However, I found a random problem
>> which is
>> >> reproducible on TC as well.
>> >> That said, GridNioServerRead.processRead() :
>> >>  - int cnt = sockCh.read(readBuf);
>> >>
>> >> sometimes does not read the payload fully as the length field in the
>> >> header is larger than the ByteBuffer.limit(). As the result, the
>> >> BufferUnderFlowException will be thrown.
>> >>
>> >> For example, in a erroneous round run with my IDE, a REDIS payload
>> (sent
>> >> by jedis client) looks like the following:
>> >>
>> >> 2a 33 d a 24 33 d a 53 45 54 d a 24 32 d a 62 31 d a 24 {38 31 39 32}
>> d a | 65
>> >> d a ...etc
>> >>
>> >> GridRedisProtocolParser.readBulkStr(buf) invokes elCnt(buf) which gets
>> >> {8192}. However, the limit of buf is 28 which ends at | position.
>> Obviously,
>> >> 8192 < limit(), therefore, the logic throws BufferUnderFlow soon after.
>> >>
>> >> I traced back and found  sockCh.read(readBuf) only read 28 bytes for
>> unknown reason
>> >>
>> >> The problem seems totally random. I wonder if this has anything to do
>> with other NioWorker
>> >> or Selector setting.
>> >>
>> >>
>> >> Any help would be appreciated!
>> >>
>> >>
>> >> Regards,
>> >>
>> >>
>> >> Michael
>> >>
>> >>
>>
>

[GitHub] ignite pull request #5156: IGNITE-10033: fix logging according coding guidel...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5156


---

[GitHub] ignite pull request #5205: IGNITE-10053

[pavel-kuznetsov]

GitHub user pavel-kuznetsov opened a pull request:

https://github.com/apache/ignite/pull/5205

IGNITE-10053



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10053

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5205.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5205


commit 0e9d30a0dedfc0903712b5a7285cb8fb4f446e57
Author: Pavel Kuznetsov 
Date:   2018-10-29T16:15:12Z

ignite-10053: Fixed npe.

Earlier joined client node could know nothing what happened to the
cluster before it joined. This used to cause NPE.

commit ff76e428004a00aaf5241d2a0cb0e9a3c98d81cb
Author: Pavel Kuznetsov 
Date:   2018-10-29T16:19:03Z

ignite-10053: Fix




---

[GitHub] ignite pull request #5204: IGNITE-9747 Add Bernoulli naive Bayes

[dehasi]

GitHub user dehasi opened a pull request:

https://github.com/apache/ignite/pull/5204

IGNITE-9747 Add Bernoulli naive Bayes



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dehasi/ignite 
feature/ignite-9747-add-bernoulli-bayes

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5204.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5204


commit 82e2cdf780a0ddc61b85b09509c3b62431f73312
Author: dehasi 
Date:   2018-10-24T16:37:39Z

IGNITE-9747: Add bernoulli package

commit aaec115f7692d925c4bc84db0f0532a672883022
Author: dehasi 
Date:   2018-10-25T16:19:01Z

IGNITE-9747: Add bernoulli model

commit 601db0f794cbbc84ce4f7629b20e87bb3cf4ab97
Author: dehasi 
Date:   2018-10-25T16:59:27Z

IGNITE-9747: Use log sum instead of mult

commit 24a7494012b8e4ae94a7f48cd32ad4a7d35c5977
Author: dehasi 
Date:   2018-10-25T17:01:55Z

IGNITE-9747: Use log sum instead of mult

commit b21c1443d7931595cc097afd5835b94c255c6882
Author: dehasi 
Date:   2018-10-26T16:58:11Z

IGNITE-9747: Update javadoc

commit c5a11244fd82875a47f79118525459a9f8d8cc7a
Author: dehasi 
Date:   2018-10-26T17:03:24Z

IGNITE-9747: Add Bernoulli trainer template

commit 573175c2d8da10cfd49b842095788332366c0cff
Author: dehasi 
Date:   2018-10-26T17:13:26Z

IGNITE-9747: Add unit test

commit 41cff0ace9525283dead37da40850209b8d74c75
Author: dehasi 
Date:   2018-10-26T17:20:26Z

IGNITE-9747: Add update model template

commit fe8e6d2b21296f16c3164d859b985f9d8c146d1c
Author: dehasi 
Date:   2018-10-26T17:31:58Z

IGNITE-9747: Add update model

commit fa3befa64fba3202c0d3a9d6a4431c092348ceaa
Author: dehasi 
Date:   2018-10-26T17:36:14Z

IGNITE-9747: Add trainer

commit 1e9608ac89d45520f27cef97365f76d58971cb05
Author: dehasi 
Date:   2018-10-26T17:48:08Z

IGNITE-9747: Test preset probabilities

commit 01fff4fc808399fc888bc90b2f74adb9f6449d85
Author: dehasi 
Date:   2018-10-26T17:52:07Z

IGNITE-9747: Test count probabilities

commit c218b2d07c6dde4fdaa2bb6b9567d2973b9165eb
Author: dehasi 
Date:   2018-10-26T17:58:57Z

IGNITE-9747: Test integration test

commit 38ac517c66fcacb1db49ff4e5f766214ef27be81
Author: dehasi 
Date:   2018-10-28T17:36:32Z

IGNITE-9747: Rid off exp

commit 52941a515c04eee78d8914ee34938cde660ea69a
Author: dehasi 
Date:   2018-10-28T17:39:50Z

IGNITE-9747: Rename test

commit 632a27abb15e6af228dae0ced3f818cca6eb8325
Author: dehasi 
Date:   2018-10-28T17:45:34Z

IGNITE-9747: Add java doc

commit 73dee910306975a52375a15ed17f200a8bb9fb9b
Author: dehasi 
Date:   2018-10-28T17:48:07Z

Merge branch 'master' into feature/ignite-9747-add-bernoulli-bayes

commit 5d822ad1174bfc17bff0734ed264e3ff8af82f3b
Author: dehasi 
Date:   2018-10-28T18:52:53Z

IGNITE-8292: Add Bernoulli example

commit 5d7f38e19f14bde197a05c5a7fe7358b63647f9b
Author: dehasi 
Date:   2018-10-28T19:17:59Z

IGNITE-8292: Fix dataset path




---

[jira] [Created] (IGNITE-10056) Attemp to create MVCC cache with TTL causes full cluster halt

[Sergey Kozlov (JIRA)]

1, intOrder=1, lastExchangeTime=1540838375187, loc=false, 
ver=2.7.1#20181029-sha1:83c27088, isClient=false], ackSent=true, 
super=CompletableLatch [id=exchange, topVer=AffinityTopologyVersion [topVer=2, 
minorTopVer=2]]]
[21:40:03,696][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] 
Finished waiting for partition release future [topVer=AffinityTopologyVersion 
[topVer=2, minorTopVer=2], waitTime=0ms, futInfo=NA, mode=LOCAL]
[21:40:03,721][INFO][exchange-worker-#43][GridCacheDatabaseSharedManager] 
Logical recovery performed in 23 ms.
[21:40:03,727][SEVERE][exchange-worker-#43][GridDhtPartitionsExchangeFuture] 
Failed to reinitialize local partitions (rebalancing will be stopped): 
GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=2, 
minorTopVer=2], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode 
[id=8a320d9a-c057-45e5-958a-6fff470c8485, addrs=ArrayList [127.0.0.1], 
sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, 
lastExchangeTime=1540838375187, loc=false, ver=2.7.1#20181029-sha1:83c27088, 
isClient=false], topVer=2, nodeId8=0c6a653d, msg=null, 
type=DISCOVERY_CUSTOM_EVT, tstamp=1540838403586]DiscoveryCustomEvent 
[customMsg=DynamicCacheChangeBatch 
[id=1876f11c661-41abad2f-0c12-4bcb-ad8c-f37aea8f7c42, reqs=ArrayList 
[DynamicCacheChangeRequest [cacheName=SQL_PUBLIC_T1, hasCfg=true, 
nodeId=8a320d9a-c057-45e5-958a-6fff470c8485, clientStartOnly=false, stop=false, 
destroy=false, disabledAfterStartfalse]], exchangeActions=ExchangeActions 
[startCaches=[SQL_PUBLIC_T1], stopCaches=null, startGrps=[SQL_PUBLIC_T1], 
stopGrps=[], resetParts=null, stateChangeRequest=null], startCaches=false], 
affTopVer=AffinityTopologyVersion [topVer=2, minorTopVer=2], super=], 
nodeId=8a320d9a, evt=DISCOVERY_CUSTOM_EVT]
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:657)
at java.util.ArrayList.get(ArrayList.java:433)
at 
org.apache.ignite.internal.processors.cache.CacheGroupContext.singleCacheContext(CacheGroupContext.java:387)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.(GridDhtLocalPartition.java:200)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl.getOrCreatePartition(GridDhtPartitionTopologyImpl.java:853)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl.initPartitions(GridDhtPartitionTopologyImpl.java:406)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl.beforeExchange(GridDhtPartitionTopologyImpl.java:585)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.distributedExchange(GridDhtPartitionsExchangeFuture.java:1470)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:806)
at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2667)
at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2539)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Thread.java:748)
[21:40:03,729][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] 
Finish exchange future [startVer=AffinityTopologyVersion [topVer=2, 
minorTopVer=2], resVer=null, err=java.lang.IndexOutOfBoundsException: Index: 0, 
Size: 0]
[21:40:03,734][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] 
Completed partition exchange [localNode=0c6a653d-b151-46b5-bd0c-7fea4b94ca26, 
exchange=GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion 
[topVer=2, minorTopVer=2], evt=DISCOVERY_CUSTOM_EVT, evtNode=TcpDiscoveryNode 
[id=8a320d9a-c057-45e5-958a-6fff470c8485, addrs=ArrayList [127.0.0.1], 
sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, 
lastExchangeTime=1540838375187, loc=false, ver=2.7.1#20181029-sha1:83c27088, 
isClient=false], done=true], topVer=null, durationFromInit=140]
[21:40:03,739][SEVERE][exchange-worker-#43][GridCachePartitionExchangeManager] 
Failed to wait for completion of partition map exchange (preloading will not 
start): GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent 
[evtNode=TcpDiscoveryNode [id=8a320d9a-c057-45e5-958a-6fff470c8485, 
addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], 
discPort=47500, order=1, intOrder=1, lastExchangeTime=1540838375187, loc=false, 
ver=2.7.1#20181029-sha1:83c27088, isClient=false], topVer=2, nodeId8=0c6a653d, 
msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1540838403586]DiscoveryCustomEvent 
[customMsg=null, affTopVer=AffinityTopologyVersion [topVer=2

[GitHub] dspavlov opened a new pull request #54: IGNITE-9848 Chain run report using pre-fetched builds instead of REST-cache

[GitBox]

dspavlov opened a new pull request #54: IGNITE-9848 Chain run report using 
pre-fetched builds instead of REST-cache
URL: https://github.com/apache/ignite-teamcity-bot/pull/54
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ignite pull request #5203: TX DR test pull-request

[sk0x50]

GitHub user sk0x50 opened a pull request:

https://github.com/apache/ignite/pull/5203

TX DR test pull-request



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.8.1-master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5203.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5203


commit 9d2cdcdf8facb1a358699fc4da1a7e1f1f6cebab
Author: Ivan Rakov 
Date:   2018-07-18T20:55:36Z

TDR-4: Refactor consistent cut logic as a separate entity

Signed-off-by: Ivan Rakov 

commit f290da12c6a607d8160ef668bbb646bb595f7d9a
Author: Slava Koptilin 
Date:   2018-07-20T15:53:19Z

txdr: keys that are participating in write operations should be included in 
writeKeys collection

commit 323f27d459abff222cfc0cc797331fdb17ac661d
Author: Ivan Rakov 
Date:   2018-07-20T16:54:36Z

Merge branch 'master' into txdr-master

commit 2ad80b64076613c837ae2d3a19419f19c5984100
Author: Ivan Rakov 
Date:   2018-07-23T09:29:56Z

Merge branch 'master' into txdr-master

commit 8557ae7134c93715ecd867da0366d301954c485b
Author: Dmitriy Sorokin 
Date:   2018-07-25T11:49:22Z

TXDR: Memory leak with unlimited growth of 
LocalPendingTransactionsTracker.preparedCommittedTxsCounters fixed.

commit 3a9e50d6523922c591985d1cea1b2fc146938697
Author: Ivan Rakov 
Date:   2018-07-25T13:52:59Z

Merge branch 'master' into txdr-master

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/GridDistributedTxRemoteAdapter.java

commit 4e63980ddeb20243d349acc84e54156d4c9fc911
Author: Aleksey Plekhanov 
Date:   2018-07-27T16:25:04Z

System property for local transaction tracker

commit 9bbb9e0666ae7f60e418fae09168ef292383fc5e
Author: Ivan Rakov 
Date:   2018-08-01T09:38:24Z

Merge branch 'master' into txdr-master

commit 62acf55b3af0d8fa672ca4876556b8a53afdeb6b
Author: Aleksey Plekhanov 
Date:   2018-08-01T13:54:52Z

System property flor local transaction tracker

commit 7ce7c30700293f949fbef85b3aeb3f05a227de6e
Author: Ivan Rakov 
Date:   2018-08-03T13:31:32Z

TDR-13 Create public API interface for transactional replication

Signed-off-by: Ivan Rakov 

commit d7c559880cd1ecac6729ad96ecded8ef4f24d56d
Author: Slava Koptilin 
Date:   2018-08-05T21:49:29Z

TDR-6 Build CC dependent transaction graph using key hashes instead of keys

Signed-off-by: Ivan Rakov 

commit 3badc1ae77d3cd87271089e75338dc4419f42c16
Author: Aleksey Plekhanov 
Date:   2018-08-05T21:59:33Z

TDR-15 Create test that will detect memory leaks in 
LocalPendingTransactionsTracker

Signed-off-by: Ivan Rakov 

commit 3f12544c6cb629f40c3c740e2483075c72c34103
Author: Dmitriy Sorokin 
Date:   2018-08-05T22:56:59Z

TDR-11 Implement handling of already COMMITTING transactions

Signed-off-by: Ivan Rakov 

commit 8c41acae041494cc834f5f07f491232a73a1ec9a
Author: Ivan Rakov 
Date:   2018-08-08T19:19:51Z

TDR-24 Refactor replication manager into GridPluginProcessor with 
ExchangeAware interface

Signed-off-by: Ivan Rakov 

commit 2a289afb482a25b8d6e80393285d8b12b888282b
Author: Ivan Rakov 
Date:   2018-08-09T13:52:13Z

Merge branch 'master' into txdr-master

commit 79ae004afe969d4acd86836912b3fa625ba34540
Author: Aleksey Plekhanov 
Date:   2018-08-09T15:51:38Z

TDR-22 Сonistent cuts watcher mechanism for replica cluster

commit 4b09f481d5bd5b735c196ab2edffb9d123a2b450
Author: Ivan Rakov 
Date:   2018-08-15T11:09:34Z

Merge branch 'master' into txdr-master

# Conflicts:
#   
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/wal/IgniteWalIteratorSwitchSegmentTest.java

commit e3615cc194d5ad3ec30adb06457b5d06a3b597d7
Author: Ivan Rakov 
Date:   2018-08-17T15:20:39Z

Merge branch 'master' into txdr-master

commit cd034c4dbab1174e1ee95d8d937b8260a04e203d
Author: Ivan Rakov 
Date:   2018-08-20T13:21:27Z

Merge branch 'master' into txdr-master

commit 1488751b1595be50a6cef24320dd1062a057bd54
Author: Andrey Gura 
Date:   2018-08-20T21:24:02Z

TDR-9 Applying consistent cuts.

commit ebdd6d03d14a8af40ed9e8b9d83b6fbc95f24f35
Author: Slava Koptilin 
Date:   2018-08-22T17:09:44Z

TDR-16: bootstrap master cluster

commit 620a657a84fcb2d040f0053adb17a06a092166c8
Author: Ivan Rakov 
Date:   2018-08-27T18:13:46Z

TDR-43 Create TEST_SNAPSHOT_CREATE command for purposes of consistent cut 
testing

commit 5b4eb8cf079373fad60312a66dee5cb95f3b42cc
Author: Ivan Rakov 
Date:   2018-08-29T09:08:28Z

Added package-info

commit c1d07ccc6f4c1b42ffac7c192058b622aecb56bb
Author: Ivan Rakov 
Date:   2018-08-29T12:12:34Z

Added package description for Spring Data 2.0

commit a15a4fff0c310327217d698dbf56256964bc0389
Author: Ivan Rakov 

[GitHub] ignite pull request #5202: IGNITE-10055

[devozerov]

GitHub user devozerov opened a pull request:

https://github.com/apache/ignite/pull/5202

IGNITE-10055



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10055

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5202.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5202


commit 196b02c5bdf1b5bd181d6eafd90efbad042cd9bc
Author: devozerov 
Date:   2018-10-29T17:35:48Z

Fixed.




---

[GitHub] ignite pull request #5192: IGNITE-10024: MVCC TX: Stackoverflow during DhtEn...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5192


---

[jira] [Created] (IGNITE-10055) MVCC: incorrect fields count in PartitionUpdateCountersMessage

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-10055:


 Summary: MVCC: incorrect fields count in 
PartitionUpdateCountersMessage
 Key: IGNITE-10055
 URL: https://issues.apache.org/jira/browse/IGNITE-10055
 Project: Ignite
  Issue Type: Task
  Components: mvcc
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov
 Fix For: 2.7


Should be 2.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-10054) sqlline hangs with concurrent operations

[Sergey Kozlov (JIRA)]

Sergey Kozlov created IGNITE-10054:
--

 Summary: sqlline hangs with concurrent operations
 Key: IGNITE-10054
 URL: https://issues.apache.org/jira/browse/IGNITE-10054
 Project: Ignite
  Issue Type: Bug
  Components: mvcc
Affects Versions: 2.7
Reporter: Sergey Kozlov
 Fix For: 2.7


1. Start 2 nodes cluster with PDS (wal_mode=FSYNC), activate
2. Start 1st {{bin/sqlline}} with following commands (in batch file):
{noformat}
create table t1 (a int, b int, primary key(a)) with 
"atomicity=TRANSACTIONAL_SNAPSHOT,backups=1";
insert into t1 values (1,1);
insert into t1 values (2,1);
insert into t1 values (3,1);
insert into t1 values (4,1);
insert into t1 values (5,1);
insert into t1 values (6,1);
insert into t1 values (7,1);
insert into t1 values (8,1);
insert into t1 values (9,1);
insert into t1 values (10,1);
begin;
update t1 set b=b+1;
commit;
begin;
update t1 set b=b+1;
commit;
begin;
update t1 set b=b+1;
commit;
begin;
update t1 set b=b+1;
commit;
...repeat TX many times...
{noformat}
3. Start 2nd {{bin/sqlline}} with following commands (in batch file):
{noformat}
begin;
update t1 set b=b-1;
commit;
begin;
update t1 set b=b-1;
commit;
begin;
update t1 set b=b-1;
commit;
...repeat TX many times...
{noformat}

4. After some time both {{sqlline}} hang:
{noformat}
611/113163   commit;
Error: Failed to execute DDL statement [stmt=commit, err=Failed to finish 
transaction because it has been rolled back [t
imeout=0, 
tx=GridNearTxLocal[xid=f91ff90c661--0914-10f6--0001, 
xidVersion=GridCacheVersion [topVer=1
52310006, order=154082519, nodeOrder=1], concurrency=PESSIMISTIC, 
isolation=REPEATABLE_READ, state=MARKED_ROLLBACK,
invalidate=false, rollbackOnly=true, 
nodeId=c9ddebdb-c328-4283-97c5-a663887a, timeout=0, duration=11, 
label=null]]]
(state=5,code=1)
java.sql.SQLException: Failed to execute DDL statement [stmt=commit, err=Failed 
to finish transaction because it has bee
n rolled back [timeout=0, 
tx=GridNearTxLocal[xid=f91ff90c661--0914-10f6--0001, 
xidVersion=GridCacheV
ersion [topVer=152310006, order=154082519, nodeOrder=1], 
concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, state=M
ARKED_ROLLBACK, invalidate=false, rollbackOnly=true, 
nodeId=c9ddebdb-c328-4283-97c5-a663887a, timeout=0, duration=11
, label=null]]]
at 
org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.sendRequest(JdbcThinConnection.java:750)
at 
org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute0(JdbcThinStatement.java:212)
at 
org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute(JdbcThinStatement.java:475)
at sqlline.Commands.execute(Commands.java:823)
at sqlline.Commands.sql(Commands.java:733)
at sqlline.SqlLine.dispatch(SqlLine.java:795)
at sqlline.SqlLine.runCommands(SqlLine.java:1706)
at sqlline.Commands.run(Commands.java:1317)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
at sqlline.SqlLine.dispatch(SqlLine.java:791)
at sqlline.SqlLine.initArgs(SqlLine.java:595)
at sqlline.SqlLine.begin(SqlLine.java:643)
at sqlline.SqlLine.start(SqlLine.java:373)
at sqlline.SqlLine.main(SqlLine.java:265)
612/113163   begin;
No rows affected (0 seconds)
613/113163   update t1 set b=b+1;
{noformat}

{noformat}
Error: Failed to execute DDL statement [stmt=commit, err=Failed to finish 
transaction because it has been rolled back [t
imeout=0, 
tx=GridNearTxLocal[xid=191ff90c661--0914-10f6--0001, 
xidVersion=GridCacheVersion [topVer=1
52310006, order=154082505, nodeOrder=1], concurrency=PESSIMISTIC, 
isolation=REPEATABLE_READ, state=MARKED_ROLLBACK,
invalidate=false, rollbackOnly=true, 
nodeId=c9ddebdb-c328-4283-97c5-a663887a, timeout=0, duration=11, 
label=null]]]
(state=5,code=1)
java.sql.SQLException: Failed to execute DDL statement [stmt=commit, err=Failed 
to finish transaction because it has bee
n rolled back [timeout=0, 
tx=GridNearTxLocal[xid=191ff90c661--0914-10f6--0001, 
xidVersion=GridCacheV
ersion [topVer=152310006, order=154082505, nodeOrder=1], 
concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, state=M
ARKED_ROLLBACK, invalidate=false, rollbackOnly=true, 
nodeId=c9ddebdb-c328-4283-97c5-a663887a, timeout=0, duration=11
, label=null]]]
at 
org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.sendRequest(JdbcThinConnection.java:750)
at 

[jira] [Created] (IGNITE-10052) Restart node during TX causes vacuum error

[Sergey Kozlov (JIRA)]

ageHandler.run(BPlusTree.java:5600)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.readPage(PageHandler.java:159)
at 
org.apache.ignite.internal.processors.cache.persistence.DataStructure.read(DataStructure.java:334)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visitDown(BPlusTree.java:2585)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doVisit(BPlusTree.java:2552)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.access$8200(BPlusTree.java:90)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.visit(BPlusTree.java:3148)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.access$5100(BPlusTree.java:2908)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visit(BPlusTree.java:1058)
... 23 more
{noformat}

{noformat}
[18:46:20,439][SEVERE][vacuum-scheduler-#47][MvccProcessorImpl] Vacuum error.
class 
org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedException: 
Unexpected state: [state=0, rowVer=1540827914710:6:536870913, 
localNodeId=4d66a9d5-4352-495c-9c64-405ccdf8a079]
at 
org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:310)
at 
org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:281)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3206)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3180)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.writePage(PageHandler.java:279)
at 
org.apache.ignite.internal.processors.cache.persistence.DataStructure.write(DataStructure.java:256)
at 
org.apache.ignite.internal.processors.cache.persistence.freelist.AbstractFreeList.updateDataRow(AbstractFreeList.java:555)
at 
org.apache.ignite.internal.processors.cache.persistence.RowStore.updateDataRow(RowStore.java:143)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.updateTxState(IgniteCacheOffheapManagerImpl.java:2449)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.updateTxState(GridCacheOffheapManager.java:1917)
at 
org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.cleanup(MvccProcessorImpl.java:2415)
at 
org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.processPartition(MvccProcessorImpl.java:2300)
at 
org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.body(MvccProcessorImpl.java:2198)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Thread.java:748)
[18:46:20,582][INFO][db-checkpoint-thread-#51][GridCacheDatabaseSharedManager] 
Skipping checkpoint (no pages were modified) [checkpointLockWait=0ms, 
checkpointLockHoldTime=12ms, reason='timeout']
[18:46:20,988][INFO][exchange-worker-#43][time] Started exchange init 
[topVer=AffinityTopologyVersion [topVer=4, minorTopVer=1], 
mvccCrd=MvccCoordinator [nodeId=319a2fda-1315-4bdf-8647-7ddee2d3342e, 
crdVer=1540827914710, topVer=AffinityTopologyVersion [topVer=4, 
minorTopVer=0]], mvccCrdChange=false, crd=false, evt=DISCOVERY_CUSTOM_EVT, 
evtNode=319a2fda-1315-4bdf-8647-7ddee2d3342e, 
customEvt=CacheAffinityChangeMessage 
[id=f4aff70c661-30eb588f-bc2f-4094-86c4-e7070c6fcb1e, 
topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], exchId=null, 
partsMsg=null, exchangeNeeded=true], allowMerge=false]
[18:46:20,998][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] 
Finished waiting for partition release future [topVer=AffinityTopologyVersion 
[topVer=4, minorTopVer=1], waitTime=0ms, futInfo=NA, mode=DISTRIBUTED]
[18:46:21,004][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] 
Finished waiting for partitions release latch: ClientLatch 
[coordinator=TcpDiscoveryNode [id=319a2fda-1315-4bdf-8647-7ddee2d3342e, 
addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], 
discPort=47500, order=1, intOrder=1, lastExchangeTime=1540827969985, loc=false, 
ver=2.7.1#20181029-sha1:83c27088, isClient=false], ackSent=true, 
super=CompletableLatch [id=exchange, topVer=AffinityTopologyVersion [topVer=4, 
minorTopVer=1]]]
[18:46:21,004][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] 
Finished waiting for partition release future [topVer=AffinityTopologyVersion 
[topVer=4, minorTopVer=1], waitTime=0ms, futInfo=NA, mode=LOCAL]
[18:46:21,005][INFO][exchange-worker-#43][GridCacheDatabaseSharedManager] 
Logical recovery performed in 0

[jira] [Created] (IGNITE-10051) Kubernetes documentation improvements

[Maxim Pudov (JIRA)]

Maxim Pudov created IGNITE-10051:


 Summary: Kubernetes documentation improvements
 Key: IGNITE-10051
 URL: https://issues.apache.org/jira/browse/IGNITE-10051
 Project: Ignite
  Issue Type: Task
  Components: documentation
Affects Versions: 2.6
Reporter: Maxim Pudov
 Fix For: 2.8
 Attachments: kube-dashboard-access.yaml, mongodb-deployment.yaml

I verified the deployment instructions of a kubernetes cluster with apache 
ignite and found some issues.

1. AWS deployment:

[https://apacheignite.readme.io/docs/amazon-web-services-eks-deployment]

miss print:
{quote}{{kubbectl get pods}}
{quote}
has to be: 
{quote}{{kubectl get pods}}
{quote}
2. Google Cloud deployment

[https://apacheignite.readme.io/docs/google-cloud-deployment]

Screenshots seem to be outdated

3. Microsoft Azure deployment

[https://apacheignite.readme.io/docs/microsoft-azure-deployment#section-access-to-your-kubernetes-cluster]

3.1 At first I had no access to Kubernetes dashboard having this error:
azure kubernetes dashboard namespaces is forbidden: User 
"system:serviceaccount:kube-system:kubernetes-dashboard" cannot list namespaces 
at the cluster scope

This problem was solved by adding this config:
{quote}kubectl create -f [^kube-dashboard-access.yaml]
{quote}
3.2 Screenshots seem to be outdated for azure too.

4. Stateless deployment:

[https://apacheignite.readme.io/docs/stateless-deployment]

4.1 At first we create example-cube-rbac.xml locally, but then we don't use it.

4.2 Instead the configuration file ignite-deployment.yaml 
([https://apacheignite.readme.io/docs/stateless-deployment#section-ignite-pods-deployment])
 points to the wrong external configuration file:
{quote}{{https://raw.githubusercontent.com/apache/ignite/master/modules/kubernetes/config/example-kube-persistence.xml}}
{quote}
I believe, it has to point to 
{quote}[{{https://raw.githubusercontent.com/apache/ignite/master/modules/kubernetes/config/example-kube-rback.xml}}|https://raw.githubusercontent.com/apache/ignite/master/modules/kubernetes/config/example-kube-rbac.xml]
{quote}
4.3 In the same configuration file ignite-deployment.yaml there is a row
{quote}{{serviceAccountName: ignite}}
{quote}
with Tab symbol, which has to be replaced with double Space.

5. Web console kubernetes installation.

[https://apacheignite-tools.readme.io/docs/kubernetes-installation]

5.1 mongodb-deployment.yaml doesn't work as is. I had to modified it (check the 
attachment [^mongodb-deployment.yaml])

5.2 web-console-deployment.yaml - default image values aren't specified for 
frontend and backend (apacheignite/web-console-frontend, 
apacheignite/web-console-backend)

5.3 web-agent-deployment.yaml - serviceAccountName: "ignite-cluster" is 
incorrect. I guess "ignite" is the right one.

5.4 No such docker image as apacheignite/web-agent:mytag3.

6. General suggestions for kubernetes cluster deployment on GCC, AWS, Azure.

6.1 Create cluster via AWS Console, not command line. It's easier to pick 
values from drop down lists, and we don't provide complete installation via 
command line anyway.

6.2 When we suggest to use our configuration files for setting up services and 
pods, it is worth mentioning that our configs contain region parameter (zones: 
eu-west-1a) and user has to modify it himself.

6.3 The link to download configuration files points to the master branch, which 
could cause problems in future. For AWS it's 
[https://github.com/apache/ignite/tree/master/modules/kubernetes/config/eks]

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #5201: IGNITE-10007: Deactivation hangs if an open trans...

[pavlukhin]

GitHub user pavlukhin opened a pull request:

https://github.com/apache/ignite/pull/5201

IGNITE-10007: Deactivation hangs if an open transaction exists



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10007

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5201.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5201


commit 5a846d4ae40e6dab7c8a1499f27044aa16c82aa0
Author: ipavlukhin 
Date:   2018-10-29T15:19:20Z

allow finish (commit/rollback) transaction on deactivated cluster




---

[GitHub] ignite pull request #5200: Page compression for Ignite persistent storage.

[svladykin]

GitHub user svladykin opened a pull request:

https://github.com/apache/ignite/pull/5200

Page compression for Ignite persistent storage.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/svladykin/ignite page-compress

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5200.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5200


commit 7309c1d831553c14bfd07d69a1f4cb52674c4f9a
Author: Sergi Vladykin 
Date:   2018-09-21T15:04:42Z

Compress module

commit 5f9d471f2854d9ac5dc2c2b9221befa01b6fc840
Author: Sergi Vladykin 
Date:   2018-09-21T15:11:14Z

OSS Licenses

commit 570411b384dee1121a67581c73ae32a3c9631b3d
Author: Sergi Vladykin 
Date:   2018-09-21T15:36:38Z

File system stat.

commit 299238239b05ac0bbaef01d189efb1eb2aa5e86e
Author: Sergi Vladykin 
Date:   2018-10-05T15:51:24Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into page-compress

commit c3d14b74113776b7612b002a1bee61b43c145ae4
Author: Sergi Vladykin 
Date:   2018-10-11T07:59:31Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into page-compress

commit 122929efa8b8cf297f84db4e4ff12ac039bc7fcd
Author: Sergi Vladykin 
Date:   2018-10-11T11:28:59Z

Proc

commit b6dcfff4d138b34ab8fbda23c7adb49f3ad04404
Author: Sergi Vladykin 
Date:   2018-10-15T11:40:13Z

wip

commit 852adeda596555dfaf56e83d7c2d13ab2d4f177f
Author: Sergi Vladykin 
Date:   2018-10-15T12:20:17Z

wip

commit 3ae9d5e876704913347f8c0392013c073c23ebe5
Author: Sergi Vladykin 
Date:   2018-10-15T12:55:51Z

wip

commit 963dfdc9ae2a237eb829526b386eedc09b473b0c
Author: Sergi Vladykin 
Date:   2018-10-16T16:51:07Z

wip

commit e888935f30514ebd4b7e4c697c0d5c1e7f073cb0
Author: Sergi Vladykin 
Date:   2018-10-17T06:01:43Z

wip

commit dfd55cda30e8d0e5fd68b4851698b21b743d77ef
Author: Sergi Vladykin 
Date:   2018-10-17T06:19:10Z

bplusio

commit 2bdfd2bb0bb15bdbcdb652641bddfa734aaa7fc1
Author: Sergi Vladykin 
Date:   2018-10-22T08:22:27Z

wip

commit 8bca890b703e66220ff4792e7ea4c3fd6ace539a
Author: Sergi Vladykin 
Date:   2018-10-22T12:20:42Z

minor

commit 74db48b32053f8cdce07e59fc067def46af38556
Author: Sergi Vladykin 
Date:   2018-10-22T13:09:10Z

rename

commit 66d4382b9f9a7136e52acd4ddc457e2457bc3d73
Author: Sergi Vladykin 
Date:   2018-10-22T13:22:57Z

merge

commit 19d571ac544e549e292c6137ac2ba6c1a988c84c
Author: Sergi Vladykin 
Date:   2018-10-22T13:26:45Z

proc start

commit 15b3ea11fe55b0ed596ea91e3267e19484abf5be
Author: Sergi Vladykin 
Date:   2018-10-22T19:21:26Z

punch

commit f90406bc217ade86d379c12ca02194aea57d5da2
Author: Sergi Vladykin 
Date:   2018-10-23T12:01:55Z

punch

commit acf9752b8596f423e1a39190365bf41af87f08ed
Author: Sergi Vladykin 
Date:   2018-10-23T12:53:58Z

punch

commit 82d88c5f068450778dd2dc9bdadb1ddff1b1c669
Author: Sergi Vladykin 
Date:   2018-10-23T13:27:51Z

minor

commit 2f71113fc696abae425753a63ce9241ed2c97de7
Author: Sergi Vladykin 
Date:   2018-10-23T13:44:01Z

minor

commit a10bededce9392a91eb39dba1c3be70796d514ac
Author: Sergi Vladykin 
Date:   2018-10-23T13:47:39Z

compilation

commit 38a407d578b3123671a58e9d61d2f3c0f5c70115
Author: Sergi Vladykin 
Date:   2018-10-23T14:05:09Z

todo

commit cff9233da7b52d175ca4c6a880751fe27a6ffdc0
Author: Sergi Vladykin 
Date:   2018-10-23T14:08:41Z

minor

commit dcbc433cfe3df6fa29f579414c41d5d3a49cfd04
Author: Sergi Vladykin 
Date:   2018-10-23T14:09:39Z

minor

commit 72fd266b28430863b2426852243342652d6f22a7
Author: Sergi Vladykin 
Date:   2018-10-23T14:36:15Z

fd

commit da8aefc5f234a783b3da9755717bc1db86484d5b
Author: Sergi Vladykin 
Date:   2018-10-24T08:36:02Z

refactor

commit ed4ad820310caf1bf923f2a3ed959e1f57b9ec89
Author: Sergi Vladykin 
Date:   2018-10-24T13:54:37Z

refactor

commit 507ac68abb5ad252d37ff51f81a97f5c835449ea
Author: Sergi Vladykin 
Date:   2018-10-24T14:19:20Z

refactor




---

[GitHub] ignite pull request #5199: Sctrict bounds check

[SpiderRus]

GitHub user SpiderRus opened a pull request:

https://github.com/apache/ignite/pull/5199

Sctrict bounds check



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10045

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5199.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5199


commit 620b1c4680a31c361a4f5e4e95aaebc794b7c139
Author: Alexey Stelmak 
Date:   2018-10-29T15:05:27Z

Sctrict bounds check




---

[jira] [Created] (IGNITE-10050) MVCC: Create "Cache 5" test suite for MVCC mode.

[Andrew Mashenkov (JIRA)]

Andrew Mashenkov created IGNITE-10050:
-

 Summary: MVCC: Create "Cache 5" test suite for MVCC mode.
 Key: IGNITE-10050
 URL: https://issues.apache.org/jira/browse/IGNITE-10050
 Project: Ignite
  Issue Type: Bug
Reporter: Andrew Mashenkov






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-10049) MVCC: Create "Cache 4" test suite for MVCC mode.

[Andrew Mashenkov (JIRA)]

Andrew Mashenkov created IGNITE-10049:
-

 Summary: MVCC: Create "Cache 4" test suite for MVCC mode.
 Key: IGNITE-10049
 URL: https://issues.apache.org/jira/browse/IGNITE-10049
 Project: Ignite
  Issue Type: Sub-task
  Components: mvcc
Reporter: Andrew Mashenkov






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #5198: IGNITE-10001: MVCC: Make existed tests run in MVC...

[AMashenkov]

GitHub user AMashenkov opened a pull request:

https://github.com/apache/ignite/pull/5198

IGNITE-10001: MVCC: Make existed tests run in MVCC mode as well.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10002

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5198.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5198


commit ce0fdab9d885d42a7b9e2005831e25ffd13dc06a
Author: Andrey V. Mashenkov 
Date:   2018-10-25T14:55:01Z

IGNITE-10002: Add MvccCacheTestSuite2. Add flag to force mvcc mode.

commit 1613f174ca5a92f21d614b867a7ab00a875a09ae
Author: Andrey V. Mashenkov 
Date:   2018-10-25T16:21:24Z

IGNITE-10002: Ignore AtomicCache tests.
Mute NearCache tests.

commit 7b5b2f579d6f5276fb35388dfd604cd51fc7d2c6
Author: Andrey V. Mashenkov 
Date:   2018-10-26T18:02:55Z

IGNITE-10004: Mute NearCache and CacheStore tests.

commit 54e12471cc30b032b12d4e7dd5546240b32b9c28
Author: Andrey V. Mashenkov 
Date:   2018-10-26T18:17:39Z

IGNITE-10004: WIP.

commit 980175501dd979adf62d3158e497ce042d1150b4
Author: Andrey V. Mashenkov 
Date:   2018-10-26T18:32:33Z

IGNITE-10004: Mute unsupported cases.

commit 0d57e4b8ee1204ff2a8638a52c3590e02494856c
Author: Andrey V. Mashenkov 
Date:   2018-10-29T12:26:39Z

IGNITE-10004: WIP.

commit 24b135ea45595ae663ca336fe1d5aa18b2fc5f89
Author: Andrey V. Mashenkov 
Date:   2018-10-29T13:40:37Z

IGNITE-10004: WIP.

commit 4169c53d3ab709f42867012e8823726eccb89fb6
Author: Andrey V. Mashenkov 
Date:   2018-10-29T13:41:27Z

IGNITE-10004: WIP.

commit 137ae8272b46991c52d39f5c491e55fb3a2773fc
Author: Andrey V. Mashenkov 
Date:   2018-10-29T13:53:40Z

IGNITE-10004: WIP.

commit bead55f56ddecd61190a33799c1318c98dbef3da
Author: Andrey V. Mashenkov 
Date:   2018-10-29T14:20:37Z

IGNITE-10004: WIP.




---

[GitHub] ignite pull request #5197: IGNITE-10048 Bounded iteration in standalone WAL ...

[glukos]

GitHub user glukos opened a pull request:

https://github.com/apache/ignite/pull/5197

IGNITE-10048 Bounded iteration in standalone WAL iterator with compac…

…tion enabled may skip records

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10048

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5197.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5197


commit 8185c620a57bb6869b2068eee2ba1a7fec0d7170
Author: Ivan Rakov 
Date:   2018-10-29T14:17:29Z

IGNITE-10048 Bounded iteration in standalone WAL iterator with compaction 
enabled may skip records




---

[jira] [Created] (IGNITE-10048) Bounded iteration in standalone WAL iterator with compaction enabled may skip records

[Ivan Rakov (JIRA)]

Ivan Rakov created IGNITE-10048:
---

 Summary: Bounded iteration in standalone WAL iterator with 
compaction enabled may skip records
 Key: IGNITE-10048
 URL: https://issues.apache.org/jira/browse/IGNITE-10048
 Project: Ignite
  Issue Type: Improvement
Reporter: Ivan Rakov
 Fix For: 2.8


Bounded iteration with non-zero start/end offsets may skip some records in 
intermediate segments. Reproducer (wal compaction should be enabled):
{noformat}
/**
 *
 */
public void testBoundedIterationOverSeveralSegments() throws Exception {
walCompactionEnabled = true;

IgniteEx ig = (IgniteEx)startGrid();

String archiveWalDir = getArchiveWalDirPath(ig);

ig.cluster().active(true);

IgniteCache cache = ig.getOrCreateCache(
new CacheConfiguration<>().setName("c-n").setAffinity(new 
RendezvousAffinityFunction(false, 32)));

IgniteCacheDatabaseSharedManager sharedMgr = 
ig.context().cache().context().database();

IgniteWriteAheadLogManager walMgr = 
ig.context().cache().context().wal();

WALPointer fromPtr = null;

int recordsCnt = WAL_SEGMENT_SIZE / 8 /* record size */ * 5;

for (int i = 0; i < recordsCnt; i++) {
WALPointer ptr = walMgr.log(new PartitionDestroyRecord(i, i));

if (i == 100)
fromPtr = ptr;
}

assertNotNull(fromPtr);

cache.put(1, 1);

forceCheckpoint();

// Generate WAL segments for filling WAL archive folder.
for (int i = 0; i < 2 * 
ig.configuration().getDataStorageConfiguration().getWalSegments(); i++) {
sharedMgr.checkpointReadLock();

try {
walMgr.log(new SnapshotRecord(i, false), 
RolloverType.NEXT_SEGMENT);
}
finally {
sharedMgr.checkpointReadUnlock();
}
}

cache.put(2, 2);

forceCheckpoint();

U.sleep(5000);

stopGrid();

WALIterator it = new IgniteWalIteratorFactory(log)
.iterator(new 
IteratorParametersBuilder().from((FileWALPointer)fromPtr).filesOrDirs(archiveWalDir));

TreeSet foundCounters = new TreeSet<>();

it.forEach(x -> {
WALRecord rec = x.get2();

if (rec instanceof PartitionDestroyRecord)
foundCounters.add(((WalRecordCacheGroupAware)rec).groupId());
});

assertEquals(new Integer(100), foundCounters.first());
assertEquals(new Integer(recordsCnt - 1), foundCounters.last());
assertEquals(recordsCnt - 100, foundCounters.size());
}

{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #5089: IGNITE-8902 One-phase commit race on tx timeout

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5089


---

[GitHub] ignite pull request #5196: IGNITE-9972 Deleted entries remain in local parti...

[ibessonov]

GitHub user ibessonov opened a pull request:

https://github.com/apache/ignite/pull/5196

IGNITE-9972 Deleted entries remain in local partition internal map



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9972

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5196.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5196


commit a9b776ceb4e120a0fff153763557acb1664d7b1f
Author: ibessonov 
Date:   2018-10-29T14:09:26Z

IGNITE-9972 Add entry to deferred delete queue only when transaction is 
finished.




---

[jira] [Created] (IGNITE-10047) MVCC: Implicit transaction can hung when started on client node on unstable topology

[Roman Kondakov (JIRA)]

Roman Kondakov created IGNITE-10047:
---

 Summary: MVCC: Implicit transaction can hung when started on 
client node on unstable topology
 Key: IGNITE-10047
 URL: https://issues.apache.org/jira/browse/IGNITE-10047
 Project: Ignite
  Issue Type: Bug
  Components: mvcc
Reporter: Roman Kondakov
 Fix For: 2.7


Reproducer: 
{{CacheContinuousQueryFailoverMvccTxSelfTest#testLeftPrimaryAndBackupNodes}}. 
This test can sporadically hangs when topology is unstable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4709: IGNITE-9342: Fix two-step queries cache for multi...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4709


---

[GitHub] ignite pull request #5062: IGNITE-9917 Write proper tests for start/stop cli...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5062


---

[GitHub] ignite pull request #5088: IGNITE-10004: Add test.

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5088


---

[GitHub] ignite pull request #5187: IGNITE-10022: JS, PHP thin clients: a more meanin...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5187


---

Re: How to run TDE tests locally?

[Nikolay Izhikov]

Hello, Pavel.

Found your environment in the end of the mail.

> My environment: JDK 1.8.0_151-b12, Ubuntu 17.10

Seems, you should enable JCE [1], [2] or update your JDK.
Actually, I don't executed any manual steps on my local environment.

host:~/src/ignite:[master]$ java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)
host:~/src/ignite:[master]$ uname -a
Linux Host 4.15.0-36-generic #39-Ubuntu SMP Mon Sep 24 16:19:09 UTC 2018 x86_64 
x86_64 x86_64 GNU/Linux

[1] https://www.oracle.com/technetwork/java/javase/8u151-relnotes-3850493.html, 
section "New Security property to control crypto policy"
[2] 
https://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html

В Пн, 29/10/2018 в 15:45 +0300, Pavel Kovalenko пишет:
> Nikolay,
> 
> JDK 1.8.0_151-b12
> 
> пн, 29 окт. 2018 г. в 15:31, Nikolay Izhikov :
> 
> > Hello, Pavel.
> > 
> > What jdk version do you use?
> > 
> > В Пн, 29/10/2018 в 15:30 +0300, Pavel Kovalenko пишет:
> > > Hello Igniters,
> > > 
> > > I have a problem with running TDE tests on my local machine.
> > > 
> > > When I start e.g. EncryptedCacheDestroyTest I get following exception:
> > > 
> > > > javax.cache.CacheException: class
> > > > > org.apache.ignite.IgniteCheckedException: Illegal key size
> > > > 
> > > > 
> > > > > at
> > > > > 
> > 
> > org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1337)
> > > > 
> > > > at
> > > > > 
> > 
> > org.apache.ignite.internal.IgniteKernal.createCache(IgniteKernal.java:2942)
> > > > 
> > > > at
> > > > > 
> > 
> > org.apache.ignite.internal.encryption.AbstractEncryptionTest.createEncryptedCache(AbstractEncryptionTest.java:173)
> > > > 
> > > > at
> > > > > 
> > 
> > org.apache.ignite.internal.encryption.AbstractEncryptionTest.createEncryptedCache(AbstractEncryptionTest.java:162)
> > > > 
> > > > at
> > > > > 
> > 
> > org.apache.ignite.internal.encryption.EncryptedCacheDestroyTest.testEncryptedCacheFromGroupDestroy(EncryptedCacheDestroyTest.java:72)
> > > > 
> > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > > 
> > > > at
> > > > > 
> > 
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > > > 
> > > > at
> > > > > 
> > 
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > > 
> > > > at java.lang.reflect.Method.invoke(Method.java:498)
> > > > 
> > > > at junit.framework.TestCase.runTest(TestCase.java:176)
> > > > 
> > > > at
> > > > > 
> > 
> > org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2206)
> > > > 
> > > > at
> > > > > 
> > 
> > org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:144)
> > > > 
> > > > at
> > > > > 
> > 
> > org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:2121)
> > > > 
> > > > at java.lang.Thread.run(Thread.java:748)
> > > > 
> > > > Caused by: class org.apache.ignite.IgniteCheckedException: Illegal key
> > 
> > size
> > > > 
> > > > at
> > 
> > org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7431)
> > > > 
> > > > at
> > > > > 
> > 
> > org.apache.ignite.internal.util.future.GridFinishedFuture.get(GridFinishedFuture.java:102)
> > > > 
> > > > at
> > > > > 
> > 
> > org.apache.ignite.internal.IgniteKernal.createCache(IgniteKernal.java:2937)
> > > > 
> > > > ... 12 more
> > > > 
> > > > Caused by: class org.apache.ignite.spi.IgniteSpiException: Illegal key
> > 
> > size
> > > > 
> > > > at
> > > > > 
> > 
> > org.apache.ignite.spi.encryption.keystore.KeystoreEncryptionSpi.doEncryption(KeystoreEncryptionSpi.java:288)
> > > > 
> > > > at
> > > > > 
> > 
> > org.apache.ignite.spi.encryption.keystore.KeystoreEncryptionSpi.encrypt(KeystoreEncryptionSpi.java:211)
> > > > 
> > > > at
> > > > > 
> > 
> > org.apache.ignite.spi.encryption.keystore.KeystoreEncryptionSpi.encryptKey(KeystoreEncryptionSpi.java:300)
> > > > 
> > > > at
> > > > > 
> > 
> > org.apache.ignite.internal.managers.encryption.GridEncryptionManager.createKeys(GridEncryptionManager.java:753)
> > > > 
> > > > at
> > > > > 
> > 
> > org.apache.ignite.internal.managers.encryption.GridEncryptionManager.generateKeys(GridEncryptionManager.java:608)
> > > > 
> > > > at
> > > > > 
> > 
> > org.apache.ignite.internal.processors.cache.GridCacheProcessor.generateEncryptionKeysAndStartCacheAfter(GridCacheProcessor.java:3650)
> > > > 
> > > > at
> > > > > 
> > 
> > org.apache.ignite.internal.processors.cache.GridCacheProcessor.dynamicStartCache(GridCacheProcessor.java:3632)
> > > > 
> > > > at
> > > > > 
> > 
> > org.apache.ignite.internal.processors.cache.GridCacheProcessor.dynamicStartCache(GridCacheProcessor.java:3543)
> > > > 
> > > > at
> > > > > 
> > 
> > org.apache.ignite.internal.IgniteKernal.createCache(IgniteKernal.java:2932)
> > > > 
> > > > ... 12 more
> 

Re: How to run TDE tests locally?

[Pavel Kovalenko]

Nikolay,

JDK 1.8.0_151-b12

пн, 29 окт. 2018 г. в 15:31, Nikolay Izhikov :

> Hello, Pavel.
>
> What jdk version do you use?
>
> В Пн, 29/10/2018 в 15:30 +0300, Pavel Kovalenko пишет:
> > Hello Igniters,
> >
> > I have a problem with running TDE tests on my local machine.
> >
> > When I start e.g. EncryptedCacheDestroyTest I get following exception:
> >
> > > javax.cache.CacheException: class
> > > > org.apache.ignite.IgniteCheckedException: Illegal key size
> > >
> > >
> > > > at
> > > >
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1337)
> > >
> > > at
> > > >
> org.apache.ignite.internal.IgniteKernal.createCache(IgniteKernal.java:2942)
> > >
> > > at
> > > >
> org.apache.ignite.internal.encryption.AbstractEncryptionTest.createEncryptedCache(AbstractEncryptionTest.java:173)
> > >
> > > at
> > > >
> org.apache.ignite.internal.encryption.AbstractEncryptionTest.createEncryptedCache(AbstractEncryptionTest.java:162)
> > >
> > > at
> > > >
> org.apache.ignite.internal.encryption.EncryptedCacheDestroyTest.testEncryptedCacheFromGroupDestroy(EncryptedCacheDestroyTest.java:72)
> > >
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >
> > > at
> > > >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > >
> > > at
> > > >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > >
> > > at java.lang.reflect.Method.invoke(Method.java:498)
> > >
> > > at junit.framework.TestCase.runTest(TestCase.java:176)
> > >
> > > at
> > > >
> org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2206)
> > >
> > > at
> > > >
> org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:144)
> > >
> > > at
> > > >
> org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:2121)
> > >
> > > at java.lang.Thread.run(Thread.java:748)
> > >
> > > Caused by: class org.apache.ignite.IgniteCheckedException: Illegal key
> size
> > >
> > > at
> org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7431)
> > >
> > > at
> > > >
> org.apache.ignite.internal.util.future.GridFinishedFuture.get(GridFinishedFuture.java:102)
> > >
> > > at
> > > >
> org.apache.ignite.internal.IgniteKernal.createCache(IgniteKernal.java:2937)
> > >
> > > ... 12 more
> > >
> > > Caused by: class org.apache.ignite.spi.IgniteSpiException: Illegal key
> size
> > >
> > > at
> > > >
> org.apache.ignite.spi.encryption.keystore.KeystoreEncryptionSpi.doEncryption(KeystoreEncryptionSpi.java:288)
> > >
> > > at
> > > >
> org.apache.ignite.spi.encryption.keystore.KeystoreEncryptionSpi.encrypt(KeystoreEncryptionSpi.java:211)
> > >
> > > at
> > > >
> org.apache.ignite.spi.encryption.keystore.KeystoreEncryptionSpi.encryptKey(KeystoreEncryptionSpi.java:300)
> > >
> > > at
> > > >
> org.apache.ignite.internal.managers.encryption.GridEncryptionManager.createKeys(GridEncryptionManager.java:753)
> > >
> > > at
> > > >
> org.apache.ignite.internal.managers.encryption.GridEncryptionManager.generateKeys(GridEncryptionManager.java:608)
> > >
> > > at
> > > >
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.generateEncryptionKeysAndStartCacheAfter(GridCacheProcessor.java:3650)
> > >
> > > at
> > > >
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.dynamicStartCache(GridCacheProcessor.java:3632)
> > >
> > > at
> > > >
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.dynamicStartCache(GridCacheProcessor.java:3543)
> > >
> > > at
> > > >
> org.apache.ignite.internal.IgniteKernal.createCache(IgniteKernal.java:2932)
> > >
> > > ... 12 more
> > >
> > > Caused by: java.security.InvalidKeyException: Illegal key size
> > >
> > > at javax.crypto.Cipher.checkCryptoPerm(Cipher.java:1039)
> > >
> > > at javax.crypto.Cipher.init(Cipher.java:1393)
> > >
> > > at javax.crypto.Cipher.init(Cipher.java:1327)
> > >
> > > at
> > > >
> org.apache.ignite.spi.encryption.keystore.KeystoreEncryptionSpi.doEncryption(KeystoreEncryptionSpi.java:282)
> > >
> > > ... 20 more
> > >
> > >
> > > >
> >
> > What I should do to avoid the exception above?
> >
> > My environment: JDK 1.8.0_151-b12, Ubuntu 17.10
>

Re: Welcome Message

[Dmitriy Pavlov]

Hi Alexander,

I've added you to the list of contributors. Now you can assign an issue to
yourself.

Should you have any question, please do not hesitate to ask.

Welcome to the Apache Ignite Community!

Sincerely,
Dmitriy Pavlov

P. S. links that can help you to onboard.
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines

пн, 29 окт. 2018 г. в 15:09, Alexander Lapin :

> Hello, Ignite Community!
>
> My name is Alexander Lapin. I want to contribute to Apache Ignite.
> my JIRA user name is alapin. Any help on this will be appreciated.
>
> Thanks!
>

[jira] [Created] (IGNITE-10046) MVCC: IgniteCachePrimaryNodeFailureRecoveryAbstractTest.testPessimisticPrimaryNodeFailureRollback hangs sometimes.

[Ilya Lantukh (JIRA)]

Ilya Lantukh created IGNITE-10046:
-

 Summary: MVCC: 
IgniteCachePrimaryNodeFailureRecoveryAbstractTest.testPessimisticPrimaryNodeFailureRollback
 hangs sometimes.
 Key: IGNITE-10046
 URL: https://issues.apache.org/jira/browse/IGNITE-10046
 Project: Ignite
  Issue Type: Improvement
Reporter: Ilya Lantukh
 Fix For: 2.7
 Attachments: 
Ignite_Tests_2.4_Java_8_9_10_11_MVCC_Queries_1323.log(1).zip

The following exception can be found in log before the hangup:
{noformat}
[14:51:43]W: [org.apache.ignite:ignite-indexing] 
java.lang.NullPointerException
[14:51:43]W: [org.apache.ignite:ignite-indexing]at 
java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
[14:51:43]W: [org.apache.ignite:ignite-indexing]at 
org.apache.ignite.internal.processors.cache.GridCacheMvccManager.future(GridCacheMvccManager.java:754)
[14:51:43]W: [org.apache.ignite:ignite-indexing]at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processPartitionCountersResponse(IgniteTxHandler.java:2204)
[14:51:43]W: [org.apache.ignite:ignite-indexing]at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$1100(IgniteTxHandler.java:120)
[14:51:43]W: [org.apache.ignite:ignite-indexing]at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$13.apply(IgniteTxHandler.java:276)
[14:51:43]W: [org.apache.ignite:ignite-indexing]at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$13.apply(IgniteTxHandler.java:274)
[14:51:43]W: [org.apache.ignite:ignite-indexing]at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1060)
[14:51:43]W: [org.apache.ignite:ignite-indexing]at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:585)
[14:51:43]W: [org.apache.ignite:ignite-indexing]at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:384)
[14:51:43]W: [org.apache.ignite:ignite-indexing]at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:310)
[14:51:43]W: [org.apache.ignite:ignite-indexing]at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:100)
[14:51:43]W: [org.apache.ignite:ignite-indexing]at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:299)
[14:51:43]W: [org.apache.ignite:ignite-indexing]at 
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
[14:51:43]W: [org.apache.ignite:ignite-indexing]at 
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
[14:51:43]W: [org.apache.ignite:ignite-indexing]at 
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
[14:51:43]W: [org.apache.ignite:ignite-indexing]at 
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
[14:51:43]W: [org.apache.ignite:ignite-indexing]at 
org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:505)
[14:51:43]W: [org.apache.ignite:ignite-indexing]at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
[14:51:43]W: [org.apache.ignite:ignite-indexing]at 
java.lang.Thread.run(Thread.java:748)
{noformat}
Full log is attached.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Welcome Message

[Alexander Lapin]

Hello, Ignite Community!

My name is Alexander Lapin. I want to contribute to Apache Ignite.
my JIRA user name is alapin. Any help on this will be appreciated.

Thanks!

[GitHub] ignite pull request #5195: IGNITE-9455

[xtern]

GitHub user xtern opened a pull request:

https://github.com/apache/ignite/pull/5195

IGNITE-9455



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xtern/ignite IGNITE-9455

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5195.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5195


commit 2993f76b982ddf8c2a71e2f5b7bc44c45613ad5c
Author: pereslegin-pa 
Date:   2018-10-29T12:06:15Z

IGNITE-9455 Replaced NO_OP allocation tracker for metastore region.




---

[jira] [Created] (IGNITE-10045) Add fail-fast mode to bounded iteration of StandaloneWalRecordsIterator

[Ivan Rakov (JIRA)]

Ivan Rakov created IGNITE-10045:
---

 Summary: Add fail-fast mode to bounded iteration of 
StandaloneWalRecordsIterator
 Key: IGNITE-10045
 URL: https://issues.apache.org/jira/browse/IGNITE-10045
 Project: Ignite
  Issue Type: Improvement
Reporter: Ivan Rakov
 Fix For: 2.8


Since IGNITE-9294 StandaloneWalRecordsIterator supports bounded iteration. That 
means we can specify "from" and "to" WAL pointers and iterator will return 
records only between given bounds. 
The problem is that in current implementation StandaloneWalRecordsIterator just 
skips segments if they are missing. For example: if we'll specify fromIdx=0, 
toIdx = 10 and segments with indexes=[9, 10] will be missing, we'll just 
silently finish iteration on idx=8.
To prevent that, we should be able to switch on fail-fast mode, in which 
StandaloneWalRecordsIterator will throw error unless iteration is really 
started from left bound and ended on right bound.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-10044) LOST partition is marked as OWNING after the owner rejoins with existing persistent data

[Stanislav Lukyanov (JIRA)]

Stanislav Lukyanov created IGNITE-10044:
---

 Summary: LOST partition is marked as OWNING after the owner 
rejoins with existing persistent data
 Key: IGNITE-10044
 URL: https://issues.apache.org/jira/browse/IGNITE-10044
 Project: Ignite
  Issue Type: Bug
Reporter: Stanislav Lukyanov


When persistence is enabled LOST partition may become OWNING without a call to 
resetLostPartitions.

If a partition is marked as LOST, and a node with that partition in persistence 
joins, the partition becomes OWNING. IgniteCache::lostPartitions doesn't return 
that partition anymore.

Apparently, it only affects the lost partitions list, while needsRecovery flag 
stays set until resetLostPartitions is called.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #5194: WAL archiever optimization

[SpiderRus]

GitHub user SpiderRus opened a pull request:

https://github.com/apache/ignite/pull/5194

WAL archiever optimization



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9903

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5194.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5194


commit 188285a766b155d71397fe18830b5a920430e9b7
Author: Alexey Stelmak 
Date:   2018-10-29T11:56:14Z

WAL archiever optimization




---

[jira] [Created] (IGNITE-10043) Lost partitions list is reset when only one server is alive in the cluster

[Stanislav Lukyanov (JIRA)]

Stanislav Lukyanov created IGNITE-10043:
---

 Summary: Lost partitions list is reset when only one server is 
alive in the cluster
 Key: IGNITE-10043
 URL: https://issues.apache.org/jira/browse/IGNITE-10043
 Project: Ignite
  Issue Type: Bug
Reporter: Stanislav Lukyanov


When server nodes are killed one by one IgniteCache.lostPartitions() returns 
non-empty list up until there is just one node left. When only one node is 
alive, the lostPartitions() returns an empty collection even though the 
partition loss events has been fired and the partitions are missing.

This only affects cluster with persistence enabled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [DISCUSSION] TDE. Phase-2. Master key rotation.

[Nikolay Izhikov]

Hello, Igniters.

We had private discussion of this design with Anton Vinogradov, Vladimir 
Ozerov, Alexey Goncharyuk.
Design need to be improved in the following ways:

1. Clear recovery logic should be written.

2. Clear process of atomic cache keys and master key change should be written.

3. We should design master key rotation process in the way it would be easy to 
implement EncryptionSPI for some enterprise encryption provider(AWS Key 
Management Service as an example)
   Which aspect of EncryptionSPI should be improved to make this process 
obvious.


В Ср, 24/10/2018 в 17:19 +0300, Nikolay Izhikov пишет:
> Hello.
> 
> Deisgn updated [1]
> 
> Please, share your feedback
> 
> [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652381
> 
> 
> В Вт, 23/10/2018 в 21:49 +0300, Nikolay Izhikov пишет:
> > Hello, Anton.
> > 
> > Thank you for your very usefull feedback!
> > 
> > I accept your proposals.
> > Seems it makes this feature more robust and clear.
> > 
> > Will update design in confluence in a couple of hours.
> > 
> > В Вт, 23/10/2018 в 19:18 +0300, Anton Vinogradov пишет:
> > > Nikolay,
> > > 
> > > I have some comments.
> > > 
> > > 1) Master key setup and removal is a responsibility of system 
> > > administrator.
> > > No matter how he will set a new master key or remove an old.
> > > The only need it to have both keys, new and old, installed before starting
> > > the rotation process.
> > > 
> > > 2) Master Key rotation is a process of cache's keys re-encryption.
> > > 
> > > So, we should send a message contains a new master key id only.
> > > We also have to check that "Master Key rotation" allowed to the user by
> > > checking it has SecurityPermission#ADMIN_OPS permission.
> > > 
> > > During this message handling, we have to re-encrypt keys and to set new
> > > master key id.
> > > 
> > > 3) We should provide recovery mode for nodes unexpectedly leaved cluster
> > > during "Master Key rotation" process.
> > > We have to have a special "node start" command allows to change node's
> > > master key before joining the cluster.
> > > 
> > > пн, 22 окт. 2018 г. в 22:39, Nikolay Izhikov :
> > > 
> > > > Hello, Igniters.
> > > > 
> > > > As you may know, we successfully implement TDE. Phase-1 feature. [1]
> > > > This improvement allows users to use an encrypted cache.
> > > > 
> > > > To make TDE production ready I propose to extend it with two things:
> > > > 
> > > > * Master key rotation.
> > > > * Cache key rotation.
> > > > 
> > > > Such features required by many security standards such as PCI DSS [2] 
> > > > and
> > > > GDPR [3]
> > > > 
> > > > I think it would be easier to discuss, implement and review both 
> > > > features
> > > > separately.
> > > > So my plan is the following:
> > > > 
> > > > * TDE. Phase-2 - Master key rotation [4]
> > > > * TDE. Phase-3 - Cache key rotation. [5]
> > > > 
> > > > I prepared designs for both parts.
> > > > I want to specifically discuss Phase-2 design.
> > > > Phase-3 design state is [EARLY DRAFT].
> > > > I propose to use Phase-3 design as a reference to make sure we have a
> > > > consistent view of all aspects of TDE
> > > > and can be implemented without significant changes in earlier parts.
> > > > 
> > > > Below, my design.
> > > > Following changes will be made in confluence [4].
> > > > Please, share your feedback.
> > > > 
> > > > *TDE. PHASE-2. MASTER KEY ROTATION*
> > > > Key rotation required in case of it compromising or at the end 
> > > > of
> > > > crypto period(key validity period).
> > > > 
> > > > Goal:
> > > > To implement the ability to rotate master encryption key.
> > > > 
> > > > New processes:
> > > > 1. Master key rotation.
> > > > 2. Removal of a master key.
> > > > 
> > > > New administrator commands:
> > > > 1. Master keys view: node -> master key hash
> > > > 2. Cache group keys view: node -> group name -> encryption key
> > > > hash
> > > > 
> > > > MASTER KEY ROTATION:
> > > > Process start:
> > > > Administrator initiates key rotation via  some kind of
> > > > user interface(CLI, Visor, Web Console, JMX, etc).
> > > > 
> > > > Process description:
> > > > Message is sent by discovery.
> > > > A Message should contain:
> > > > * Master cache key encrypted with the current
> > > > master key.
> > > > 
> > > > When server node processed message following actions are
> > > > executed:
> > > > * Blocks creation of encrypted cache key.
> > > > * Encrypt cache group keys with new master key.
> > > > * Unblock creation of encrypted cache key.
> > > > 
> > > > New joining node should also change the current master 
> > > > key
> > > > with the new one.
> > > > 
> > > > Process completion:
> > > > The 

[GitHub] asfgit closed pull request #50: IGNITE-10030 Background upload of changes and references to changes

[GitBox]

asfgit closed pull request #50: IGNITE-10030 Background upload of changes and 
references to changes
URL: https://github.com/apache/ignite-teamcity-bot/pull/50
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ignite pull request #5193: IGNITE-10042: Fix wrong TxLog root page type.

[AMashenkov]

GitHub user AMashenkov opened a pull request:

https://github.com/apache/ignite/pull/5193

IGNITE-10042: Fix wrong TxLog root page type.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10042

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5193.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5193


commit 13c108c16dc809d53ef306029a37b737657f52ec
Author: Andrey V. Mashenkov 
Date:   2018-10-29T10:45:34Z

IGNITE-10042: Fix wrong TxLog root page type.




---

[jira] [Created] (IGNITE-10042) Wrong TxLog root page type.

[Andrew Mashenkov (JIRA)]

Andrew Mashenkov created IGNITE-10042:
-

 Summary: Wrong TxLog root page type.
 Key: IGNITE-10042
 URL: https://issues.apache.org/jira/browse/IGNITE-10042
 Project: Ignite
  Issue Type: Bug
  Components: mvcc
Reporter: Andrew Mashenkov
 Fix For: 2.7


TxLog tries root pages has 'data page' type.
This is a bug as tree structures root pages has to be a type of 'index page'.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4824: IGNITE-9682 changed updatePartitionFullMap() meth...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4824


---

[jira] [Created] (IGNITE-10041) Partition loss policies work incorrectly with BLT

[Stanislav Lukyanov (JIRA)]

Stanislav Lukyanov created IGNITE-10041:
---

 Summary: Partition loss policies work incorrectly with BLT
 Key: IGNITE-10041
 URL: https://issues.apache.org/jira/browse/IGNITE-10041
 Project: Ignite
  Issue Type: Bug
Reporter: Stanislav Lukyanov


Current partition loss handling (initially designed before persistence and BLT 
were introduced) aligns poorly with BLT semantics. The general problem is that 
partition loss handling assumes that rebalancing always happens on topology 
change - which is not the case with BLT.

The features with semantics conflicting with BLT are:
- PartitionLossPolicy.READ_ONLY_ALL and PartitionLossPolicy.READ_WRITE_ALL
READ_*_ALL policies assume that there is "some" partition created in the 
cluster which we could read or write, even though it is marked as LOST. With 
BLT the LOST partition only exists on an offline node, so there is no place to 
route the requests to.

- PartitionLossPolicy.IGNORE
Somewhat similar to the READ_*_ALL, but IGNORE also expects that the lost 
partition is immediately recreated on another node - which is impossible 
without changing affinity, i.e. without resetting BLT.

- IgniteCache::withPartitionRecover
withPartitionRecover is supposed to ignore partition loss policies and allow 
all operations, but when a partition is lost in a BLT cluster it's actually 
unavailable, there is straightforward way to ignore it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #5190: IGNITE-9790 Fixed delayed full message handling i...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5190


---

[GitHub] ignite pull request #5072: IGNITE-9941 Web Console: Add option to disable se...

[akuznetsov-gridgain]

Github user akuznetsov-gridgain closed the pull request at:

https://github.com/apache/ignite/pull/5072


---

[GitHub] ignite pull request #5095: IGNITE-10031 REST: Added "excludeCaches" paramete...

[akuznetsov-gridgain]

Github user akuznetsov-gridgain closed the pull request at:

https://github.com/apache/ignite/pull/5095


---

[jira] [Created] (IGNITE-10040) Auto rebalance throttling

[Alexei Scherbakov (JIRA)]

Alexei Scherbakov created IGNITE-10040:
--

 Summary: Auto rebalance throttling
 Key: IGNITE-10040
 URL: https://issues.apache.org/jira/browse/IGNITE-10040
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexei Scherbakov


Currently we provide a few options to control rebalance overhead, most important

org.apache.ignite.configuration.CacheConfiguration#setRebalanceThrottle

org.apache.ignite.configuration.IgniteConfiguration#setRebalanceThreadPoolSize

In general proper option values could be only derived from load testing, which 
is very inconvenient. Moreover, changing the settings requires grid restart.

It's desirable to implement automatic rebalance throttling defined by user 
configuration option, in terms of ratio between dirty pages produced by 
rebalance and dirty pages produced by user activity.





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #5086: IGNITE-9982: SQLLine: can't run with option --aut...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5086


---

[GitHub] ignite pull request #5063: IGNITE-9753 Several optimization of validate_inde...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5063


---

[jira] [Created] (IGNITE-10039) Small data region may lead IgniteOOM

[Sergey Kozlov (JIRA)]

Sergey Kozlov created IGNITE-10039:
--

 Summary: Small data region may lead IgniteOOM
 Key: IGNITE-10039
 URL: https://issues.apache.org/jira/browse/IGNITE-10039
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 2.7
Reporter: Sergey Kozlov
 Fix For: 2.7


For small data region(s) the node restart may lead IgniteOOM. We should 
calculate the minimal required size and show a warning message on node output.

{noformat}
[21:36:06,648][SEVERE][exchange-worker-#63][] Critical system error detected. 
Will be handled accordingly to configured handler 
[hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, 
super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet 
[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext [type=CRITICAL_ERROR, 
err=class o.a.i.i.mem.IgniteOutOfMemoryException: Failed to find a page for 
eviction [segmentCapacity=3952, loaded=1549, maxDirtyPages=1162, 
dirtyPages=1549, cpPages=0, pinnedInSegment=0, failedToPrepare=1549]
Out of memory in data region [name=default, initSize=50.0 MiB, maxSize=100.0 
MiB, persistenceEnabled=true] Try the following:
  ^-- Increase maximum off-heap memory size (DataRegionConfiguration.maxSize)
  ^-- Enable Ignite persistence (DataRegionConfiguration.persistenceEnabled)
  ^-- Enable eviction or expiration policies]]
class org.apache.ignite.internal.mem.IgniteOutOfMemoryException: Failed to find 
a page for eviction [segmentCapacity=3952, loaded=1549, maxDirtyPages=1162, 
dirtyPages=1549, cpPages=0, pinnedInSegment=0, failedToPrepare=1549]
Out of memory in data region [name=default, initSize=50.0 MiB, maxSize=100.0 
MiB, persistenceEnabled=true] Try the following:
  ^-- Increase maximum off-heap memory size (DataRegionConfiguration.maxSize)
  ^-- Enable Ignite persistence (DataRegionConfiguration.persistenceEnabled)
  ^-- Enable eviction or expiration policies
at 
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl$Segment.tryToFindSequentially(PageMemoryImpl.java:2361)
at 
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl$Segment.removePageForReplacement(PageMemoryImpl.java:2255)
at 
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl$Segment.access$900(PageMemoryImpl.java:1864)
at 
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:696)
at 
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:637)
at 
org.apache.ignite.internal.processors.cache.persistence.DataStructure.acquirePage(DataStructure.java:144)
at 
org.apache.ignite.internal.processors.cache.persistence.freelist.PagesList.init(PagesList.java:169)
at 
org.apache.ignite.internal.processors.cache.persistence.freelist.AbstractFreeList.(AbstractFreeList.java:371)
at 
org.apache.ignite.internal.processors.cache.persistence.freelist.CacheFreeListImpl.(CacheFreeListImpl.java:49)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore$1.(GridCacheOffheapManager.java:1339)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.init0(GridCacheOffheapManager.java:1339)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.updateCounter(GridCacheOffheapManager.java:1631)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.updateCounter(GridDhtLocalPartition.java:991)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.(GridDhtLocalPartition.java:224)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl.forceCreatePartition(GridDhtPartitionTopologyImpl.java:884)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restorePartitionStates(GridCacheDatabaseSharedManager.java:2444)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.applyLastUpdates(GridCacheDatabaseSharedManager.java:2382)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreState(GridCacheDatabaseSharedManager.java:1624)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.beforeExchange(GridCacheDatabaseSharedManager.java:1298)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.distributedExchange(GridDhtPartitionsExchangeFuture.java:1453)
at 

[jira] [Created] (IGNITE-10038) Enable typeahed pug mixin to work with objects

[Alexander Kalinin (JIRA)]

Alexander Kalinin created IGNITE-10038:
--

 Summary: Enable typeahed pug mixin to work with objects
 Key: IGNITE-10038
 URL: https://issues.apache.org/jira/browse/IGNITE-10038
 Project: Ignite
  Issue Type: Improvement
  Components: wizards
Reporter: Alexander Kalinin
Assignee: Alexander Kalinin


Currently form-field__typeahed mixin works with plaing arrays of strings, but 
we also need this mixin to support arrays of objects.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #5192: IGNITE-10024: MVCC TX: Stackoverflow during DhtEn...

[pavlukhin]

GitHub user pavlukhin opened a pull request:

https://github.com/apache/ignite/pull/5192

IGNITE-10024: MVCC TX: Stackoverflow during DhtEnlistFuture mapping



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10024

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5192.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5192


commit 98a2785dbcbb8c62f059f1d0e71e77aeee2f73df
Author: ipavlukhin 
Date:   2018-10-29T08:43:09Z

GridTimeoutProcessor signals timeout==true to closure when expired (-1) 
timeout passed




---

[jira] [Created] (IGNITE-10037) Cache 2 tests optimization

[Alexey Platonov (JIRA)]

Alexey Platonov created IGNITE-10037:


 Summary: Cache 2 tests optimization
 Key: IGNITE-10037
 URL: https://issues.apache.org/jira/browse/IGNITE-10037
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexey Platonov
Assignee: Alexey Platonov


We need to investigate how to optimize these tests:

GridCachePartitionNotLoadedEventSelfTest.testPrimaryAndBackupDead
 
CacheTxLoadingConcurrentGridStartSelfTestAllowOverwrite.testLoadCacheWithDataStreamerSequentialWithConfigAndRestarts
 IgniteCacheEntryProcessorNodeJoinTest.testEntryProcessorNodeLeave
 
CacheTxLoadingConcurrentGridStartSelfTestAllowOverwrite.testLoadCacheWithDataStreamerSequentialWithConfig
 
CacheTxLoadingConcurrentGridStartSelfTestAllowOverwrite.testLoadCacheWithDataStreamerSequentialClientWithConfig

 

and optimize them.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #5191: IGNITE-10023

[NSAmelchev]

GitHub user NSAmelchev opened a pull request:

https://github.com/apache/ignite/pull/5191

IGNITE-10023

Improve ListeningTestLogger for wait conditions.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/NSAmelchev/ignite ignite-10023

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5191.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5191


commit ee24281b9bfd0affc15f2dc2efaabf95e8062cdf
Author: NSAmelchev 
Date:   2018-10-29T08:39:28Z

Improve test logger




---

[jira] [Created] (IGNITE-10036) node_startup_by_ssh.sample.ini move to /config

[Sergey Kozlov (JIRA)]

Sergey Kozlov created IGNITE-10036:
--

 Summary: node_startup_by_ssh.sample.ini move to /config
 Key: IGNITE-10036
 URL: https://issues.apache.org/jira/browse/IGNITE-10036
 Project: Ignite
  Issue Type: Bug
  Components: build
Affects Versions: 2.7
Reporter: Sergey Kozlov
 Fix For: 2.7


The configuration file for IGNITE-1023 has been added to location
{{bin/include/visorcmd/node_startup_by_ssh.sample.ini}}.

But the more convinient directory is {{/config}} where already collected a few 
various  confuration files.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Apache Ignite 2.7. Last Mile

[Nikolay Izhikov]

Hello, guys.

For today we have 11 tickets mapped to 2.7

IGNITE-10010 Alexey Goncharuk Node halted if second node was stopped, then
cache destroyed, then second node returned
IGNITE-10015 Alexey Goncharuk Sporadic JVM crash due to restart nodes
IGNITE-10013 Unassigned Node restart may lead to NPE in
GridDhtPartitionsExchangeFuture
IGNITE-9928 Igor Seliverstov MVCC TX: Late affinity assignment support.
IGNITE-9985 Igor Seliverstov MVCC TX: fix backup mappings
IGNITE-10007 Sergey Kozlov Deactivation hangs if an open transaction exists
IGNITE-10004 Andrew Mashenkov Parse error leads to leave the transaction
IGNITE-10024 Ivan Pavlukhin MVCC TX: Stackoverflow during DhtEnlistFuture
mapping
IGNITE-9996 Alexey Goncharuk Investigate possible performance drop in FSYNC
mode for ignite-2.7 compared to ignite-2.6
IGNITE-9982 Ivan Pavlukhin SQLLine: can't run with option
--autoCommit=false or true
IGNITE-9828 Roman Kondakov MVCC: Continuous query failover.


пт, 26 окт. 2018 г. в 9:59, Vladimir Ozerov :

> Hi Nikolay,
>
> I do not know. We need to investigate them first. These are all
> regressions, so decision about impact and urgency should be made separately
> for every ticket.
>
> On Fri, Oct 26, 2018 at 9:57 AM Nikolay Izhikov 
> wrote:
>
> > Hello, Igniters.
> >
> > We have *9* tickets mapped to 2.7 today
> >
> > Vladimir, do you think 1 week delay will be enough to resolve all this
> > tickets?
> >
> > IGNITE-9828 Roman Kondakov  MVCC: Continuous query failover.
> > IGNITE-9928 Igor SeliverstovMVCC TX: Late affinity assignment
> > support.
> > IGNITE-9985 Igor SeliverstovMVCC TX: fix backup mappings
> > IGNITE-9982 Ivan Pavlukhin  SQLLine: can't run with option
> > --autoCommit=false or true
> > IGNITE-10010Unassigned  Node halted if table was dropped
> > IGNITE-10013Unassigned  Node restart may lead to NPE in
> > GridDhtPartitionsExchangeFuture
> > IGNITE-9996 Unassigned  Investigate possible performance
> > drop in FSYNC mode for ignite-2.7 compared to ignite-2.6
> > IGNITE-10007Unassigned  Deactivation hangs if an open
> > transaction exists
> > IGNITE-10004Unassigned  Parse error leads to leave the
> > transaction
> >
> > В Ср, 24/10/2018 в 12:30 +0300, Nikolay Izhikov пишет:
> > > Hello, Igniters.
> > >
> > > We have 3 ticket mapped to 2.7 today:
> > >
> > > Igor Seliverstov - IGNITE-9892 - MVCC: Exchange hangs on mvcc
> > coordinator fail
> > > Roman Kondakov   - IGNITE-9828 - MVCC: Continuous query failover.
> > > Roman Kondakov   - IGNITE-9928 - MVCC TX: Bug in SQL query mapping.
> > >
> > > В Вт, 23/10/2018 в 15:01 +0300, Nikolay Izhikov пишет:
> > > > Hello, Dmitriy.
> > > >
> > > > I'm OK with including this patch to 2.7.
> > > >
> > > > Can you ensure it completely fix the issue?
> > > > I left comment in ticket.
> > > >
> > > >
> >
> https://issues.apache.org/jira/browse/IGNITE-9854?focusedCommentId=16660516=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16660516
> > > >
> > > >
> > > > В Вт, 23/10/2018 в 13:10 +0300, Dmitriy Govorukhin пишет:
> > > > > Nikolay,
> > > > >
> > > > > I have an issue which I want to include in 2.7 release,
> > > > > https://issues.apache.org/jira/browse/IGNITE-9854
> > > > > It is a very small fix but very important, it protects us from NPE
> > in some
> > > > > race scenario.
> > > > > Changes already in master, but the issue still not resolve, need
> your
> > > > > approval for cherry-picking changes to ignite-2.7.
> > > > >
> > > > > On Tue, Oct 23, 2018 at 10:51 AM Vladimir Ozerov <
> > voze...@gridgain.com>
> > > > > wrote:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > There are still tickets in the scope as we continue finding new
> > issues
> > > > > > during QA. I propose the following plan: if there are still
> opened
> > issues
> > > > > > by Friday, then shift vote date for 1 week, to 2nd November. This
> > is needed
> > > > > > to ensure that product quality is sufficient. But if the backlog
> > is empty
> > > > > > by Friday, we can go ahead with vote.
> > > > > >
> > > > > > Thoughts?
> > > > > >
> > > > > > On Tue, Oct 23, 2018 at 10:41 AM Nikolay Izhikov <
> > nizhi...@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > Hello, Igniters.
> > > > > > >
> > > > > > > We have 7 tickets mapped to 2.7 today.
> > > > > > >
> > > > > > > Igov Seliverstov  - IGNITE-9892 - MVCC: Exchange hangs on mvcc
> > > > > >
> > > > > > coordinator
> > > > > > > fail
> > > > > > > Igor Seliverstov  - IGNITE-9911 -
> > > > > > >
> > > > > >
> > > > > >
> >
> CacheMvccSelectForUpdateQueryAbstractTest#testSelectForUpdateAfterAbortedTx
> > > > > > > periodically hangs
> > > > > > > Roman Kondakov- IGNITE-9928 - MVCC TX: Bug in SQL query
> > mapping.
> > > > > > > Roman Kondakov- IGNITE-9663 - MVCC: Data node failure can
> > cause TX
> > > > > > > hanging.
> > > > > > > Vladimir Ozerov   - IGNITE-9960 - 

Re: Exchange stucks while node restoring state from WAL

[Maxim Muzafarov]

Pavel, Dmitry G.,

Thank you for the help and review this improvement!

Igniters,

I'd like to finalize this topic with the following changes [1] `Update
Grid(Ignite)CacheDatabaseSharedManager
according to accepted coding guidelines`. I've consciously excluded them
from the original PR not to overcomplicate review.

Please, consider these changes to be included in the master branch too:
 - createDataRegionConfiguration have an unobvious name – it creates
metastore dataregion config
 - startMemoryPolicies rename to startDataRegion\or remove
 - output messages format fix code style
 - register\unregister method for MBean can be simplified

I've prepared everything required for the review.

[1] https://issues.apache.org/jira/browse/IGNITE-10033
[2] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-927
[3] https://github.com/apache/ignite/pull/5156
[4]
https://ci.ignite.apache.org/viewLog.html?buildId=2184711=buildResultsDiv=IgniteTests24Java8_RunAll



On Tue, 18 Sep 2018 at 18:02 Maxim Muzafarov  wrote:

> Igniters,
>
> I would like this issue to be part of release 2.7. There is not much time
> left
> considering that I will have to correct comments after the review.
>
> Will anyone help with review?
>
>
> PR: https://github.com/apache/ignite/pull/4520/files
> Upsource: https://reviews.ignite.apache.org/ignite/review/IGNT-CR-727
> JIRA: https://issues.apache.org/jira/browse/IGNITE-7196
>
> On Thu, 13 Sep 2018 at 11:27 Maxim Muzafarov  wrote:
>
>> Igniters,
>>
>> I need your help with the review of this patch. In general, it will help
>> to reduce PME duration. I've moved binary
>> memory recovery at the moment of node startup as we've discussed it with
>> Pavel previously this topic.
>>
>> Changes are relatively small (+299 −95) and they are ready. I've left a
>> comment in JIRA with high-level
>> implementation details. Hope, this will help with the review [1].
>>
>> Please, take a look.
>> PR: https://github.com/apache/ignite/pull/4520/files
>> Upsource: https://reviews.ignite.apache.org/ignite/review/IGNT-CR-727
>> JIRA: https://issues.apache.org/jira/browse/IGNITE-7196
>> TC Run All: [2]
>>
>>
>> [1]
>> https://issues.apache.org/jira/browse/IGNITE-7196?focusedCommentId=16613175=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16613175
>> [2]
>> https://ci.ignite.apache.org/viewLog.html?buildId=1835822=buildResultsDiv=IgniteTests24Java8_RunAll
>>
>>
>> On Tue, 28 Aug 2018 at 17:59 Pavel Kovalenko  wrote:
>>
>>> Hello Maxim,
>>>
>>> I think you're going in the right direction, but why you perform binary
>>> recovery only in the case when a node in the baseline?
>>> I think that node can also perform binary recovery before the first
>>> exchange is started in case of
>>> 1) First activation
>>> 2) Baseline change
>>>
>>> In all of these cases, you need just local available CacheDescriptors to
>>> restore binary state from WAL.
>>> I would like also see a test when Ignite is stopped in the middle of a
>>> checkpoint and binary recovery succeed before PME in that case also.
>>>
>>>
>>> ср, 22 авг. 2018 г. в 15:31, Pavel Kovalenko :
>>>
>>> > Hello Maxim,
>>> >
>>> > Thank you for your work. I will review your changes within the next
>>> > several days.
>>> >
>>> > 2018-08-22 11:12 GMT+03:00 Maxim Muzafarov :
>>> >
>>> >> Pavel,
>>> >>
>>> >> Thank you for so detailed introduction into the root of the problem of
>>> >> ticket IGNITE-7196.
>>> >> As you mentioned before, we can divide this ticket into two
>>> sub-tasks. So,
>>> >> I will
>>> >> file new issue for `Logical consistency recovery`. I think it's more
>>> >> convenient way
>>> >> for reviewing and merging each of these high level Steps (PRs).
>>> >>
>>> >> Currently, let's focus on `Physical(binary) consistency recovery` as
>>> the
>>> >> most critical
>>> >> part of this issue and synchronize our vision and vision of other
>>> >> community
>>> >> members
>>> >> on implementation details of Step 1. From this point everything what
>>> I'm
>>> >> saying is about
>>> >> only binary recovery.
>>> >>
>>> >>
>>> >> I think PR is almost ready (I need to check TC carefully).
>>> >> Can you look at my changes? Am I going the right way?
>>> >>
>>> >> PR: https://github.com/apache/ignite/pull/4520
>>> >> Upsource: https://reviews.ignite.apache.org/ignite/review/IGNT-CR-727
>>> >> JIRA: https://issues.apache.org/jira/browse/IGNITE-7196
>>> >>
>>> >>
>>> >> These are my milestones which I've tried to solve:
>>> >> 1. On first time cluster activation no changes required. We should
>>> keep
>>> >> this behavior.
>>> >>
>>> >> 2. `startMemoryPolicies` earlier if need.
>>> >> Now at the master branch data regions starts at onActivate method
>>> called.
>>> >> If we are  trying
>>> >> to restore binary state on node reconnects to cluster we should start
>>> >> regions earlier.
>>> >>
>>> >> 3. `suspendLogging` method added to switch off handlers.
>>> >> Keep fast PDS cleanup when joining is not in BLT (or belongs to
>>> another
>>> >>