[jira] [Updated] (IGNITE-7254) Add more ways to activate a cluster to the error message for an inactive cluster

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-7254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-7254:
--
Fix Version/s: (was: 2.9)
   2.10

> Add more ways to activate a cluster to the error message for an inactive 
> cluster
> 
>
> Key: IGNITE-7254
> URL: https://issues.apache.org/jira/browse/IGNITE-7254
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexey Goncharuk
>Assignee: Alexey Goncharuk
>Priority: Minor
> Fix For: 2.10
>
>
> Need to mention REST API and control.sh script (and Visorcmd, if activation 
> is supported)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-7204) Unexpected behavior if passing null to binaryObject.field method

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-7204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-7204:
--
Fix Version/s: (was: 2.9)
   2.10

> Unexpected behavior if passing null to binaryObject.field method
> 
>
> Key: IGNITE-7204
> URL: https://issues.apache.org/jira/browse/IGNITE-7204
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.3
>Reporter: Alexey Scherbakov
>Assignee: Dmitry Vinnik
>Priority: Major
>  Labels: newbie
> Fix For: 2.10
>
>
> If assertions are disabled, when first field value will be returned.
> If not, an AssertionError will be thrown.
> Reproducer:
> {noformat}
> public void testNullField() throws Exception {
> try {
> final IgniteEx ex = startGrid(0);
> final IgniteCache test = 
> ex.cache("test").withKeepBinary();
> final BinaryObjectBuilder bldr = ex.binary().builder("obj");
> bldr.setField("x", 1);
> test.put(0, bldr.build());
> test.query(new ScanQuery<>(new IgniteBiPredicate BinaryObject>() {
> @Override public boolean apply(Integer o, BinaryObject o2) {
> final Object q = o2.field(null);
> return false;
> }
> })).getAll();
> }
> finally {
> stopAllGrids();
> }
> }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-7115) .NET: Thin client: performance: use buffer pool

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-7115:
--
Fix Version/s: (was: 2.9)
   2.10

> .NET: Thin client: performance: use buffer pool
> ---
>
> Key: IGNITE-7115
> URL: https://issues.apache.org/jira/browse/IGNITE-7115
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms, thin client
>Affects Versions: 2.4
>Reporter: Pavel Tupitsyn
>Priority: Major
>  Labels: .NET, performance
> Fix For: 2.10
>
>
> Currently we allocate a lot of {{byte[]}} arrays for socket operations, which 
> is suboptimal, especially for large arrays which go to Large Object Heap.
> Use a shared buffer pool instead, see {{PlatformMemoryPool}} for reference.
> Make sure to maintain correct detachment semantics: 
> {{BinaryReader.ReadAsBinary}} in some cases may pass streamed array to user 
> code.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-7039) SQL: local query should pin affected partitions

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-7039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-7039:
--
Fix Version/s: (was: 2.9)
   2.10

> SQL: local query should pin affected partitions
> ---
>
> Key: IGNITE-7039
> URL: https://issues.apache.org/jira/browse/IGNITE-7039
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Reporter: Vladimir Ozerov
>Priority: Major
>  Labels: sql-stability
> Fix For: 2.10
>
> Attachments: 3194.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When distributed query is executed, we pin cache partitions for particular 
> topology version on map nodes [1]. However, it seems that we do no do that 
> for local queries. This is a bug because partition with required data could 
> be evicted from local node at any time, leading to incorrect results.
> [1] 
> https://github.com/apache/ignite/blob/ignite-2.3/modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/twostep/GridMapQueryExecutor.java#L288



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-7065) Cannot use multiple Ignite Kafka Sink Connectors

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-7065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-7065:
--
Fix Version/s: (was: 2.9)
   2.10

> Cannot use multiple Ignite Kafka Sink Connectors 
> -
>
> Key: IGNITE-7065
> URL: https://issues.apache.org/jira/browse/IGNITE-7065
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.3
>Reporter: Alexey Kukushkin
>Priority: Major
> Fix For: 2.10
>
>
> Stopping an Ignite Kafka sink connector makes other connectors deployed in 
> the same worker unusable. As a result we cannot stream different topics in 
> different caches since connector support streaming to single cache only.
> See 
> http://apache-ignite-users.70518.x6.nabble.com/Issues-with-multiple-kafka-connectors-questions-regarding-ignite-caches-tc18297.html
>  for more details. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6956) Reduce excessive iterator allocations during GridDhtPartitionMap instantiation

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6956:
--
Fix Version/s: (was: 2.9)
   2.10

> Reduce excessive iterator allocations during GridDhtPartitionMap instantiation
> --
>
> Key: IGNITE-6956
> URL: https://issues.apache.org/jira/browse/IGNITE-6956
> Project: Ignite
>  Issue Type: Task
>Affects Versions: 2.1
>Reporter: Alexey Goncharuk
>Priority: Major
> Fix For: 2.10
>
> Attachments: 2017-11-20_12-02-04.png
>
>
> We allocate an iterator during map.values() call, see attached allocation 
> profile.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6910) Introduce a force join parameter to clear PDS after branching pointing

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6910:
--
Fix Version/s: (was: 2.9)
   2.10

> Introduce a force join parameter to clear PDS after branching pointing
> --
>
> Key: IGNITE-6910
> URL: https://issues.apache.org/jira/browse/IGNITE-6910
> Project: Ignite
>  Issue Type: New Feature
>  Components: persistence
>Reporter: Sergey Puchnin
>Priority: Major
>  Labels: IEP-4, Phase-2
> Fix For: 2.10
>
>
> Need to give an opportunity to clean a PDS on a node that is trying to join 
> after branching pointing. It may be a "force-join" or a "force-clean" 
> parameter



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-7028) Memcached does not set type flags for response

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-7028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-7028:
--
Fix Version/s: (was: 2.9)
   2.10

> Memcached does not set type flags for response
> --
>
> Key: IGNITE-7028
> URL: https://issues.apache.org/jira/browse/IGNITE-7028
> Project: Ignite
>  Issue Type: Bug
>  Components: rest
>Affects Versions: 2.3
>Reporter: Mikhail Cherkasov
>Priority: Major
> Fix For: 2.10
>
>
> Memcached does not set type flags for response:
> http://apache-ignite-users.70518.x6.nabble.com/Memcached-doesn-t-store-flags-td18403.html



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6862) SparseDistributedMatrixStorage cache config possibly allows read of old state of matrix

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6862:
--
Fix Version/s: (was: 2.9)
   2.10

> SparseDistributedMatrixStorage cache config possibly allows  read of old 
> state of matrix
> 
>
> Key: IGNITE-6862
> URL: https://issues.apache.org/jira/browse/IGNITE-6862
> Project: Ignite
>  Issue Type: Bug
>Reporter: Artem Malykh
>Priority: Major
> Fix For: 2.10
>
>
> Because synchronization mode is PRIMARY_SYNC, but by default we have 
> readFromBackup=true, we can read old state of matrix from backups. It seem to 
> be an issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6857) Multi stage start cache

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6857:
--
Fix Version/s: (was: 2.9)
   2.10

> Multi stage start cache
> ---
>
> Key: IGNITE-6857
> URL: https://issues.apache.org/jira/browse/IGNITE-6857
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Reporter: Dmitriy Govorukhin
>Priority: Major
> Fix For: 2.10
>
>
> In current implementation, cache start in one stage (cache start exchange). 
> We must provide ability to start cache more that one stage (for internal 
> operation, like transaction recovery)
> No cache operation should occur between the start stages.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6854) Enabling Persistent Memory for Ignite

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6854:
--
Fix Version/s: (was: 2.9)
   2.10

> Enabling Persistent Memory for Ignite
> -
>
> Key: IGNITE-6854
> URL: https://issues.apache.org/jira/browse/IGNITE-6854
> Project: Ignite
>  Issue Type: New Feature
>Affects Versions: 2.3
>Reporter: Mulugeta Mammo
>Priority: Major
> Fix For: 2.10
>
>
> Ignite, when persistence mode is enabled, stores data and indexes on disk. To 
> minimize the latency of disks, several tuning options can be applied. Setting 
> the page size of a memory region to match the page size of the underlying 
> storage, using a separate disk for the WAL, and using production-level SSDs 
> are just a few of them [ 
> https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning
>  ]. 
>  
> A persistent memory store with low latency and high capacity offers a viable 
> alternative to disks. In light of this, we are proposing to make use of our 
> Low Level Persistent Library (LLPL), 
> https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory 
> storage for Ignite. 
>  
> At this point, we envision two distinct implementation options:
> # Data and indexes will continue to be stored in the off-heap memory but the 
> disk will be replaced by a persistent memory. Since persistence memory in 
> this option is not a file system, the logic currently offered by WAL file and 
> the partition files would have to be implemented from scratch.
> # In this option, we eliminate the current check-point process and the WAL 
> file. We will use a memory region defined by LLPL to store data and indexes. 
> There will be no off-heap memory. DRAM will be exclusively used to store hot 
> cache entries just like the on-heap cache is in the current implementation. 
>   
>  
> In both cases, there are more details and subtleties that have to handled – 
> e.g. the atomic and transactional guarantees offered. More clarifications 
> will be given as we go along.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6826) Change default DiscoverySpi ipFinder type for examples

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6826:
--
Fix Version/s: (was: 2.9)
   2.10

> Change default DiscoverySpi ipFinder type for examples
> --
>
> Key: IGNITE-6826
> URL: https://issues.apache.org/jira/browse/IGNITE-6826
> Project: Ignite
>  Issue Type: Improvement
>  Components: examples
>Affects Versions: 2.3
>Reporter: Alexey Popov
>Assignee: Ryabov Dmitrii
>Priority: Major
>  Labels: newbie
> Fix For: 2.10
>
>
> It is better to change multicast ipFinder to static (vm) ipFinder for java 
> examples, .NET examples and C++ examples.
> http://apache-ignite-developers.2346864.n4.nabble.com/ipFinder-configuration-for-Samples-td23818.html



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6793) NPE in InitNewCoordinatorFuture leading to Cache3 suite hang

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6793:
--
Fix Version/s: (was: 2.9)
   2.10

> NPE in InitNewCoordinatorFuture leading to Cache3 suite hang
> 
>
> Key: IGNITE-6793
> URL: https://issues.apache.org/jira/browse/IGNITE-6793
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.3
>Reporter: Alexey Goncharuk
>Assignee: Vitaliy Biryukov
>Priority: Major
>  Labels: MakeTeamcityGreenAgain
> Fix For: 2.10
>
>
> Got the following exception in the IgniteCacheGroupsTest run:
> {code}
> [17:59:25]W:   [org.apache.ignite:ignite-core] [2017-10-30 
> 14:59:25,159][ERROR][sys-#35070%cache.IgniteCacheGroupsTest2%][GridCacheIoManager]
>  Failed processing message [senderId=9a523ce6-a252-457f-8175-7246b6c4, 
> msg=GridDhtPartitionsSingleMessage [parts={3181548=GridDhtPartitionMap 
> [moving=0, top=AffinityTopologyVersion [topVer=42, minorTopVer=2], 
> updateSeq=1098, size=191], -2100569601=GridDhtPartitionMap [moving=0, 
> top=AffinityTopologyVersion [topVer=42, minorTopVer=2], updateSeq=654, 
> size=100]}, 
> partCntrs={3181548=o.a.i.i.processors.cache.distributed.dht.preloader.CachePartitionPartialCountersMap@78b1774,
>  
> -2100569601=o.a.i.i.processors.cache.distributed.dht.preloader.CachePartitionPartialCountersMap@3cb0c48a},
>  partHistCntrs=null, err=null, client=false, compress=false, 
> finishMsg=GridDhtPartitionsFullMessage [parts=null, partCntrs=null, 
> partCntrs2=null, partHistSuppliers=null, partsToReload=null, 
> topVer=AffinityTopologyVersion [topVer=42, minorTopVer=2], errs=null, 
> compress=false, resTopVer=null, partCnt=0, 
> super=GridDhtPartitionsAbstractMessage [exchId=GridDhtPartitionExchangeId 
> [topVer=AffinityTopologyVersion [topVer=42, minorTopVer=2], discoEvt=null, 
> nodeId=33c058c3, evt=DISCOVERY_CUSTOM_EVT], lastVer=GridCacheVersion 
> [topVer=120855515, order=1509375572161, nodeOrder=32], super=GridCacheMessage 
> [msgId=4153599, depInfo=null, err=null, skipPrepare=false]]], 
> super=GridDhtPartitionsAbstractMessage [exchId=GridDhtPartitionExchangeId 
> [topVer=AffinityTopologyVersion [topVer=42, minorTopVer=2], discoEvt=null, 
> nodeId=33c058c3, evt=DISCOVERY_CUSTOM_EVT], lastVer=GridCacheVersion 
> [topVer=120855515, order=1509375572153, nodeOrder=38], super=GridCacheMessage 
> [msgId=4153605, depInfo=null, err=null, skipPrepare=false
> [17:59:25]W:   [org.apache.ignite:ignite-core] 
> java.lang.NullPointerException
> [17:59:25]W:   [org.apache.ignite:ignite-core]at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.InitNewCoordinatorFuture.onMessage(InitNewCoordinatorFuture.java:238)
> [17:59:25]W:   [org.apache.ignite:ignite-core]at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onReceiveSingleMessage(GridDhtPartitionsExchangeFuture.java:1749)
> [17:59:25]W:   [org.apache.ignite:ignite-core]at 
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.processSinglePartitionUpdate(GridCachePartitionExchangeManager.java:1484)
> [17:59:25]W:   [org.apache.ignite:ignite-core]at 
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.access$1000(GridCachePartitionExchangeManager.java:131)
> [17:59:25]W:   [org.apache.ignite:ignite-core]at 
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$2.onMessage(GridCachePartitionExchangeManager.java:327)
> [17:59:25]W:   [org.apache.ignite:ignite-core]at 
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$2.onMessage(GridCachePartitionExchangeManager.java:307)
> [17:59:25]W:   [org.apache.ignite:ignite-core]at 
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCachePartitionExchangeManager.java:2627)
> [17:59:25]W:   [org.apache.ignite:ignite-core]at 
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCachePartitionExchangeManager.java:2606)
> [17:59:25]W:   [org.apache.ignite:ignite-core]at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1060)
> [17:59:25]W:   [org.apache.ignite:ignite-core]at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:579)
> [17:59:25]W:   [org.apache.ignite:ignite-core]at 
> 

[jira] [Updated] (IGNITE-6820) Add data regions to data structures configuration

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6820:
--
Fix Version/s: (was: 2.9)
   2.10

> Add data regions to data structures configuration
> -
>
> Key: IGNITE-6820
> URL: https://issues.apache.org/jira/browse/IGNITE-6820
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.3
>Reporter: Alexey Goncharuk
>Priority: Major
>  Labels: newbie
> Fix For: 2.10
>
>
> Data structures configuration has cache group name but misses data region 
> name. This makes it tricky to move data structures to a different data 
> region. More specifically, it's hard to have default data region with 
> persistence disabled but configure data structures to be persistent.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6712) Add data volume metrics to data region metrics

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6712:
--
Fix Version/s: (was: 2.9)
   2.10

> Add data volume metrics to data region metrics
> --
>
> Key: IGNITE-6712
> URL: https://issues.apache.org/jira/browse/IGNITE-6712
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 2.2
>Reporter: Alexey Goncharuk
>Priority: Major
> Fix For: 2.10
>
>
> Current DataRegion metrics lack information about how much space is used for 
> data or indexes. 
> We need to add the following metrics to the DataRegionMetrics interface:
> * Total data/index pages
> * Physical (loaded) data/index pages
> * Evict rate/age for data/index pages
> To the Persistence metrics:
> * Evict rate/age for data/index pages
> To cache metrics:
> * Effective space used in data pages (key-value, PK index)
> * Effective space used in index pages (secondary indexes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6659) Check that public API is not called in sensitive system threads

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6659:
--
Fix Version/s: (was: 2.9)
   2.10

> Check that public API is not called in sensitive system threads
> ---
>
> Key: IGNITE-6659
> URL: https://issues.apache.org/jira/browse/IGNITE-6659
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.2
>Reporter: Alexey Goncharuk
>Priority: Major
> Fix For: 2.10
>
>
> We need to add a check that public (or semi-public) blocking API is not 
> called from sensitive system threads.
> For example, cache operations, service deploy must not be called in disco* 
> threads, exchange-worker, system thread pool because this will cause a 
> cluster-wide deadlock.
> Need to consider which other threads and APIs are affected.
> Any invalid call must throw an exception.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6665) Client node re-joins only to the list from disco configuration and ignores the rest nodes

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6665:
--
Fix Version/s: (was: 2.9)
   2.10

> Client node re-joins only to the list from disco configuration and ignores 
> the rest nodes
> -
>
> Key: IGNITE-6665
> URL: https://issues.apache.org/jira/browse/IGNITE-6665
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Affects Versions: 2.2
>Reporter: Mikhail Cherkasov
>Priority: Major
> Fix For: 2.10
>
> Attachments: ClientReJoinTest.java
>
>
> Client node re-joins only to the list from disco configuration and ignores 
> the rest nodes.
> if we have a cluster with 3 server nodes and in client discovery 
> configuration only 1 is mentioned and this server node left cluster, client 
> node will try to re-join only to this one and will ignore the rest 2 server 
> nodes.
> Reproducer is attached.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6664) Memcached example refactoring

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6664:
--
Fix Version/s: (was: 2.9)
   2.10

> Memcached example refactoring
> -
>
> Key: IGNITE-6664
> URL: https://issues.apache.org/jira/browse/IGNITE-6664
> Project: Ignite
>  Issue Type: Task
>  Components: examples
>Affects Versions: 2.2
>Reporter: Sergey Kozlov
>Priority: Major
>  Labels: newbie
> Fix For: 2.10
>
>
> The memcached example {{examples/memcached/memcached-example.php}} looks like 
> outpdated and doesn't work for php 7.0/7.1.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6629) Make service automatic redeployment configurable in ServiceConfiguration

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6629:
--
Fix Version/s: (was: 2.9)
   2.10

> Make service automatic redeployment configurable in ServiceConfiguration
> 
>
> Key: IGNITE-6629
> URL: https://issues.apache.org/jira/browse/IGNITE-6629
> Project: Ignite
>  Issue Type: Improvement
>  Components: managed services
>Affects Versions: 2.3
>Reporter: Ivan Rakov
>Priority: Major
>  Labels: Muted_test
> Fix For: 2.10
>
>
> Before 2.3, if persistence was enabled globally, services were recovered 
> along with system cache. But in 2.3, persistence can be enabled for per data 
> region (IGNITE-6030), and system data region is not persistent.
> We should add feaure to configure service redeployment after restart. 
> Service-related information should be stored in Metastore instead of system 
> cache.
> IgniteChangeGlobalStateServiceTest#testDeployService should be fixed under 
> this ticket.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6628) Make possible to rebuild all SQL indexes programmatically with enabled persistence.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6628:
--
Fix Version/s: (was: 2.9)
   2.10

> Make possible to rebuild all SQL indexes programmatically with enabled 
> persistence.
> ---
>
> Key: IGNITE-6628
> URL: https://issues.apache.org/jira/browse/IGNITE-6628
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.0
>Reporter: Alexey Scherbakov
>Priority: Major
> Fix For: 2.10
>
>
> We have unofficial way for rebuilding indexes, which is called on activation 
> if index.bin is removed from PDS directory.
> Code is located here [1]
> I think it's ok to make it public for several cases: model is changed, index 
> is damaged, etc...
> Also current impl has a bug: CacheEntry in [2] is not touched, polluting heap 
> and leading to OOM.
> [1] 
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager#beforeExchange
> [2] 
> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing#rebuildIndexesFromHash



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6546) Update faveicon.ico in rest-htttp module

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6546:
--
Fix Version/s: (was: 2.9)
   2.10

> Update faveicon.ico in rest-htttp module
> 
>
> Key: IGNITE-6546
> URL: https://issues.apache.org/jira/browse/IGNITE-6546
> Project: Ignite
>  Issue Type: Improvement
>  Components: rest
>Reporter: Alexey Kuznetsov
>Assignee: Peter Ivanov
>Priority: Major
> Fix For: 2.10
>
>
> Correct icon: https://svn.apache.org/repos/asf/ignite/site/trunk/favicon.ico



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6544) Can't switch WalMode from LOG_ONLY/BACKGROUND to DEFAULT

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6544:
--
Fix Version/s: 2.10

> Can't switch WalMode from LOG_ONLY/BACKGROUND to DEFAULT
> 
>
> Key: IGNITE-6544
> URL: https://issues.apache.org/jira/browse/IGNITE-6544
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Alexander Belyak
>Priority: Major
> Fix For: 2.10
>
>
> To reproduce:
> 1) Start ignite with persistence with LOG_ONLY/BACKGROUND log mode
> 2) Stop and start with DEFAULT log mode
> Exception is:
> {noformat}
> Exception in thread "main" class org.apache.ignite.IgniteException: Failed to 
> start processor: GridProcessorAdapter []
>   at 
> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:966)
>   at org.apache.ignite.Ignition.start(Ignition.java:325)
>   at 
> org.apache.ignite.examples.datagrid.CacheApiExample.main(CacheApiExample.java:59)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to start 
> processor: GridProcessorAdapter []
>   at 
> org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1813)
>   at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:931)
>   at 
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1904)
>   at 
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1646)
>   at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1074)
>   at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:594)
>   at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:519)
>   at org.apache.ignite.Ignition.start(Ignition.java:322)
>   ... 1 more
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to 
> initialize WAL log segment (WAL segment size change is not 
> supported):/tmp/s1/wal/0_0_0_0_0_0_0_1_lo_10_0_3_1_10_42_1_107_127_0_0_1_172_17_0_1_47500/.wal
>   at 
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.checkFiles(FileWriteAheadLogManager.java:1420)
>   at 
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.checkOrPrepareFiles(FileWriteAheadLogManager.java:934)
>   at 
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.start0(FileWriteAheadLogManager.java:274)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter.start(GridCacheSharedManagerAdapter.java:61)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.start(GridCacheProcessor.java:614)
>   at 
> org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1810)
>   ... 8 more
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6544) Can't switch WalMode from LOG_ONLY/BACKGROUND to DEFAULT

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6544:
--
Fix Version/s: (was: 2.9)

> Can't switch WalMode from LOG_ONLY/BACKGROUND to DEFAULT
> 
>
> Key: IGNITE-6544
> URL: https://issues.apache.org/jira/browse/IGNITE-6544
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Alexander Belyak
>Priority: Major
>
> To reproduce:
> 1) Start ignite with persistence with LOG_ONLY/BACKGROUND log mode
> 2) Stop and start with DEFAULT log mode
> Exception is:
> {noformat}
> Exception in thread "main" class org.apache.ignite.IgniteException: Failed to 
> start processor: GridProcessorAdapter []
>   at 
> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:966)
>   at org.apache.ignite.Ignition.start(Ignition.java:325)
>   at 
> org.apache.ignite.examples.datagrid.CacheApiExample.main(CacheApiExample.java:59)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to start 
> processor: GridProcessorAdapter []
>   at 
> org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1813)
>   at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:931)
>   at 
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1904)
>   at 
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1646)
>   at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1074)
>   at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:594)
>   at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:519)
>   at org.apache.ignite.Ignition.start(Ignition.java:322)
>   ... 1 more
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to 
> initialize WAL log segment (WAL segment size change is not 
> supported):/tmp/s1/wal/0_0_0_0_0_0_0_1_lo_10_0_3_1_10_42_1_107_127_0_0_1_172_17_0_1_47500/.wal
>   at 
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.checkFiles(FileWriteAheadLogManager.java:1420)
>   at 
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.checkOrPrepareFiles(FileWriteAheadLogManager.java:934)
>   at 
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.start0(FileWriteAheadLogManager.java:274)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter.start(GridCacheSharedManagerAdapter.java:61)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.start(GridCacheProcessor.java:614)
>   at 
> org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1810)
>   ... 8 more
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6532) Introduce preallocation in LFS files to avoid high fragmentation on filesystem level

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6532:
--
Fix Version/s: (was: 2.9)
   2.10

> Introduce preallocation in LFS files to avoid high fragmentation on 
> filesystem level
> 
>
> Key: IGNITE-6532
> URL: https://issues.apache.org/jira/browse/IGNITE-6532
> Project: Ignite
>  Issue Type: Bug
>  Components: persistence
>Affects Versions: 2.2
>Reporter: Ivan Rakov
>Priority: Major
> Fix For: 2.10
>
>
> Modern databases (Oracle, MySql) work with storage drive on physical level, 
> creating their own partition table and filesystem.
> Ignite Persistent Store work with regular files. It appends new pages to 
> partition file once new pages are allocated and written on checkpoint. These 
> new pages can form one or several fragments on filesystem level.
> As a result, after weeks of uptime, partition files can contain huge amount 
> of fragments. There were reports about 120 fragments in index.bin file on 
> XFS filesystem. 
> We can work this around by preallocating files in bigger chunks, e.g. 1000 
> pages at a time. On the other hand, early allocation will increase LFS size 
> overhead, so we should consider reasonable heuristic for allocation.
> Allocation should be performed on native level. Just writing a byte at 
> position (file_size + page_size * 1000) won't do it because XFS (and other 
> filesystems as well) has an optimization for that case. Missing range will be 
> just skipped.
> Related article about filesystem internals: 
> https://blog.codecentric.de/en/2017/04/xfs-possible-memory-allocation-deadlock-kmem_alloc/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6527) Deadlock detection works incorrectly with some timeouts that haven't caused by deadlocks.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6527:
--
Fix Version/s: (was: 2.9)
   2.10

> Deadlock detection works incorrectly with some timeouts that haven't caused 
> by deadlocks.
> -
>
> Key: IGNITE-6527
> URL: https://issues.apache.org/jira/browse/IGNITE-6527
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.3
>Reporter: Vitaliy Biryukov
>Assignee: Andrey N. Gura
>Priority: Major
> Fix For: 2.10
>
> Attachments: TxOptimisticDeadlockDetectionIncorrectMessageTest.java
>
>
> Deadlock detection works incorrectly with timeouts that haven't caused by 
> deadlocks. In case of a deadlock in future. Or can detect another deadlock 
> which was not the cause of timeout.
> *requested keys:* keys primary for the same node and blocking in sequential 
> order during the timeout (or all keys that haven't locked by an optimistic 
> transaction in case of near cache).
> *candidates:* keys candidates to be locked on a primary node (entries 
> contains in  GridDhtTxLocal). 
> In the process of updating the Wait-For-Graph requested keys used as 
> candidates.  But "TxDeadlock.toString" method use candidates which were 
> received from messages. 
> 1) It causes an incorrect error message.
> Example: 
> K1: TX1 holds lock, TX2 waits lock.
> K2: TX3 holds lock, TX1 waits lock.
> Transactions:
> TX1 [txId=GridCacheVersion [topVer=118090802, order=1506610794980, 
> nodeOrder=1], nodeId=f03b1ae3-a100-479c-9671-11d5cef0, threadId=455]
> TX2 [txId=GridCacheVersion [topVer=118090802, order=1506610794980, 
> nodeOrder=2], nodeId=2c0c0e78-cab2-4b23-a985-4965e421, threadId=456]
> TX3 [txId=GridCacheVersion [topVer=118090802, order=1506610794980, 
> nodeOrder=3], nodeId=3340dc48-f1a1-4ea8-8742-19b31432, threadId=457]
> Keys:
> K1 [key=6, cache=cache]
> K2 [key=1, cache=cache]
> 2) DD can detect another deadlock which was not the cause of timeout but it 
> would be the cause if the current deadlock did not happen.
> These are very rare situations, but they can happen.
> I see several solutions:
> * Just make a correct message.
> * log warn and continue detecting.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-6324) Transactional cache data partially available after crash.

2020-06-26 Thread Aleksey Plekhanov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146378#comment-17146378
 ] 

Aleksey Plekhanov commented on IGNITE-6324:
---

[~zstan], [~DmitriyGovorukhin],

Do we have a chance to resolve this issue in 2.9 release? Can we move it to the 
next release?

> Transactional cache data partially available after crash.
> -
>
> Key: IGNITE-6324
> URL: https://issues.apache.org/jira/browse/IGNITE-6324
> Project: Ignite
>  Issue Type: Bug
>  Components: persistence
>Affects Versions: 1.9, 2.1
>Reporter: Stanilovsky Evgeny
>Priority: Critical
> Fix For: 2.9
>
> Attachments: InterruptCommitedThreadTest.java
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> If InterruptedException raise in client code during pds store operations we 
> can obtain inconsistent cache after restart. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-13190) Core defragmentation functions

2020-06-26 Thread Sergey Chugunov (Jira)
Sergey Chugunov created IGNITE-13190:


 Summary: Core defragmentation functions
 Key: IGNITE-13190
 URL: https://issues.apache.org/jira/browse/IGNITE-13190
 Project: Ignite
  Issue Type: Sub-task
Reporter: Sergey Chugunov


The following set of functions covering defragmentation happy-case needed:
 * Initialization of defragmentation manager when node is started in 
maintenance mode.
 * Information about partition files is gathered by defrag mgr.
 * For each partition file corresponding file of defragmented partition is 
created and initialized.
 * Keys are transferred from old partitions to new partitions.
 * Checkpointer is aware of new partition files and flushes defragmented memory 
to new partition files.

 

No fault-tolerance code nor index defragmentation mappings are needed in this 
task.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6504) Very quick checkpoint can cause AssertionError on next start from LFS

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6504:
--
Fix Version/s: (was: 2.9)
   2.10

> Very quick checkpoint can cause AssertionError on next start from LFS
> -
>
> Key: IGNITE-6504
> URL: https://issues.apache.org/jira/browse/IGNITE-6504
> Project: Ignite
>  Issue Type: Bug
>  Components: persistence
>Affects Versions: 2.1
>Reporter: Ivan Rakov
>Assignee: Ivan Rakov
>Priority: Major
>  Labels: newbie
> Fix For: 2.10
>
>
> Checkpoint markers are compared using their timestamps. If checkpoint took 
> less than 1 millisecond, two subsequent markers will have same timestamp, 
> which will lead to error:
> {noformat}
> java.lang.AssertionError: 
> o1=/data/teamcity/tmpfs/work/db/127_0_0_1_47503/cp/1506338145591-c4f23411-e1b1-4468-856a-4419003bba93-END.bin,
>  
> o2=/data/teamcity/tmpfs/work/db/127_0_0_1_47503/cp/1506338145591-f76c023b-9982-40d7-a1eb-855a33b710f2-END.bin
> at 
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$4.compare(GridCacheDatabaseSharedManager.java:216)
> at 
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$4.compare(GridCacheDatabaseSharedManager.java:195)
> at java.util.TimSort.binarySort(TimSort.java:265)
> at java.util.TimSort.sort(TimSort.java:208)
> at java.util.TimSort.sort(TimSort.java:173)
> at java.util.Arrays.sort(Arrays.java:659)
> at 
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$CheckpointHistory.loadHistory(GridCacheDatabaseSharedManager.java:2704)
> at 
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$CheckpointHistory.access$2600(GridCacheDatabaseSharedManager.java:2685)
> at 
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1468)
> at 
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readCheckpointAndRestoreMemory(GridCacheDatabaseSharedManager.java:562)
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:722)
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:613)
> at 
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2289)
> at 
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6444) Validate that copyOnRead flag is configured with on-heap cache enabled

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6444:
--
Fix Version/s: (was: 2.9)
   2.10

> Validate that copyOnRead flag is configured with on-heap cache enabled
> --
>
> Key: IGNITE-6444
> URL: https://issues.apache.org/jira/browse/IGNITE-6444
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.0
>Reporter: Alexey Goncharuk
>Priority: Major
>  Labels: usability
> Fix For: 2.10
>
>
> Link to the user-list discussion:
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-2-1-0-CopyOnRead-Problem-td17009.html
> It makes sense to validate the flag and print out a warning if on-heap cache 
> is disabled. I do not think that we should prevent node from startup because 
> this may break existing deployments.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6445) IgniteTxManager.txLocksInfo method misses locks

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6445:
--
Fix Version/s: (was: 2.9)
   2.10

> IgniteTxManager.txLocksInfo method misses locks
> ---
>
> Key: IGNITE-6445
> URL: https://issues.apache.org/jira/browse/IGNITE-6445
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Vitaliy Biryukov
>Assignee: Vitaliy Biryukov
>Priority: Critical
> Fix For: 2.10
>
>
> In some cases "IgniteTxManager.txLocksInfo" method (searches for locks) 
> misses locks.
> For example:
> # In case of a configuration with near cache, entries are created for the 
> near cache and for the ordinal cache. For each entry, their own MVCC 
> candidates are created.
> # For non-custom objects of type (Integer, etc.), the entry stored in 
> "GridNearTxLocal" is not associated with MVCC candidates with which the same 
> entity is associated in another format stored in "GridDhtTxLocal"



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6405) Deadlock is not detected if timed out on client.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6405:
--
Fix Version/s: (was: 2.9)
   2.10

> Deadlock is not detected if timed out on client.
> 
>
> Key: IGNITE-6405
> URL: https://issues.apache.org/jira/browse/IGNITE-6405
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Alexey Scherbakov
>Assignee: Andrey N. Gura
>Priority: Minor
> Fix For: 2.10
>
>
> Timeout exception is thrown instead.
> Reproducer:
> {noformat}
> /*
>  * Licensed to the Apache Software Foundation (ASF) under one or more
>  * contributor license agreements.  See the NOTICE file distributed with
>  * this work for additional information regarding copyright ownership.
>  * The ASF licenses this file to You under the Apache License, Version 2.0
>  * (the "License"); you may not use this file except in compliance with
>  * the License.  You may obtain a copy of the License at
>  *
>  *  http://www.apache.org/licenses/LICENSE-2.0
>  *
>  * Unless required by applicable law or agreed to in writing, software
>  * distributed under the License is distributed on an "AS IS" BASIS,
>  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>  * See the License for the specific language governing permissions and
>  * limitations under the License.
>  */
> package org.apache.ignite.internal.processors.cache.transactions;
> import java.util.Collections;
> import java.util.concurrent.CountDownLatch;
> import javax.cache.CacheException;
> import org.apache.ignite.Ignite;
> import org.apache.ignite.configuration.CacheConfiguration;
> import org.apache.ignite.configuration.IgniteConfiguration;
> import org.apache.ignite.configuration.TransactionConfiguration;
> import org.apache.ignite.internal.IgniteInternalFuture;
> import org.apache.ignite.internal.util.typedef.internal.U;
> import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
> import org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;
> import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest;
> import org.apache.ignite.transactions.Transaction;
> import org.apache.ignite.transactions.TransactionDeadlockException;
> import static org.apache.ignite.cache.CacheAtomicityMode.TRANSACTIONAL;
> import static 
> org.apache.ignite.transactions.TransactionConcurrency.PESSIMISTIC;
> import static 
> org.apache.ignite.transactions.TransactionIsolation.REPEATABLE_READ;
> /**
>  * Tests an ability to eagerly rollback timed out transactions.
>  */
> public class TxPessimisticDeadlockDetectionClient extends 
> GridCommonAbstractTest {
> /** */
> private static final long TX_MIN_TIMEOUT = 1;
> /** */
> private static final long TX_TIMEOUT = 300;
> /** */
> private static final long TX_DEFAULT_TIMEOUT = 3_000;
> /** */
> private static final String CACHE_NAME = "test";
> /** IP finder. */
> private static final TcpDiscoveryVmIpFinder IP_FINDER = new 
> TcpDiscoveryVmIpFinder(true);
> /** */
> private static final int GRID_CNT = 3;
> /** */
> private final CountDownLatch blocked = new CountDownLatch(1);
> /** */
> private final CountDownLatch unblocked = new CountDownLatch(1);
> /** {@inheritDoc} */
> @Override protected IgniteConfiguration getConfiguration(String 
> igniteInstanceName) throws Exception {
> IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
> cfg.setClientMode("client".equals(igniteInstanceName));
> ((TcpDiscoverySpi)cfg.getDiscoverySpi()).setIpFinder(IP_FINDER);
> TransactionConfiguration txCfg = new TransactionConfiguration();
> txCfg.setDefaultTxTimeout(TX_DEFAULT_TIMEOUT);
> cfg.setTransactionConfiguration(txCfg);
> CacheConfiguration ccfg = new CacheConfiguration(CACHE_NAME);
> ccfg.setAtomicityMode(TRANSACTIONAL);
> ccfg.setBackups(2);
> cfg.setCacheConfiguration(ccfg);
> return cfg;
> }
> /** {@inheritDoc} */
> @Override protected void beforeTest() throws Exception {
> super.beforeTest();
> startGridsMultiThreaded(GRID_CNT);
> }
> /** {@inheritDoc} */
> @Override protected void afterTest() throws Exception {
> super.afterTest();
> stopAllGrids();
> }
> /** */
> protected void validateException(Exception e) {
> assertEquals("Deadlock report is expected",
> TransactionDeadlockException.class, 
> e.getCause().getCause().getClass());
> }
> /**
>  * Tests if deadlock is resolved on timeout with correct message.
>  *
>  * @throws Exception If failed.
>  */
> public void 

[jira] [Updated] (IGNITE-6243) RazorSQL crash on try edit, describe and another actions with table.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6243:
--
Fix Version/s: (was: 2.9)
   2.10

> RazorSQL crash on try edit, describe and another actions with table.
> 
>
> Key: IGNITE-6243
> URL: https://issues.apache.org/jira/browse/IGNITE-6243
> Project: Ignite
>  Issue Type: Bug
>  Components: odbc
>Affects Versions: 2.1
>Reporter: Aleksey Chetaev
>Assignee: Igor Sapego
>Priority: Major
> Fix For: 2.10
>
>
> 1. Install ODBC on Windows.
> 2. Crete DSN for Ignite ODBC Driver.
> 3. Connect by RazorSQL using ODBC.
> 4. Create new table. 
> 5. Try to edit table, using right click.
> RazorSQL crashed without error.
> Need check that it's not crash of Apache Ignite ODBC driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-6224) Node stoping does not wait all transactions completion

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-6224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-6224:
--
Fix Version/s: (was: 2.9)
   2.10

> Node stoping does not wait all transactions completion
> --
>
> Key: IGNITE-6224
> URL: https://issues.apache.org/jira/browse/IGNITE-6224
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Vladislav Pyatkov
>Assignee: Vitaliy Biryukov
>Priority: Major
> Fix For: 2.10
>
> Attachments: TransactionBehindStopNodeTest.java
>
>
> I have started grid node and executing transaction over some cache. After I 
> stopped the node in the middle execution of transaction. I got transaction 
> execution exception:
> {noformat}
> java.lang.IllegalStateException: class 
> org.apache.ignite.internal.processors.cache.CacheStoppedException: Failed to 
> perform cache operation (cache is stopped): cache
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheGateway.enter(GridCacheGateway.java:164)
>   at 
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.onEnter(GatewayProtectedCacheProxy.java:1656)
>   at 
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:869)
>   at 
> org.apache.ignite.TransactionBehindStopNodeTest.testOneNode(TransactionBehindStopNodeTest.java:56)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at junit.framework.TestCase.runTest(TestCase.java:176)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2000)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:132)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1915)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}
> although I stopped node with _false_ {{cancel}} flag.
> {code}
> G.stop(getTestIgniteInstanceName(0), false);
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-5968) Test fail in Ignite Cache 2: GridCachePartitionNotLoadedEventSelfTest.testPrimaryAndBackupDead

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-5968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-5968:
--
Fix Version/s: (was: 2.9)
   2.10

> Test fail in Ignite Cache 2: 
> GridCachePartitionNotLoadedEventSelfTest.testPrimaryAndBackupDead
> --
>
> Key: IGNITE-5968
> URL: https://issues.apache.org/jira/browse/IGNITE-5968
> Project: Ignite
>  Issue Type: Test
>Affects Versions: 2.1
>Reporter: Dmitriy Govorukhin
>Priority: Major
>  Labels: MakeTeamcityGreenAgain
> Fix For: 2.10
>
>
> java.util.concurrent.TimeoutException: Test has been timed out 
> [test=testPrimaryAndBackupDead, timeout=30]
> at 
> org.apache.ignite.testframework.junits.GridAbstractTest.runTest(GridAbstractTest.java:1949)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at 
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (IGNITE-13189) Maintenance mode switch and defragmentation process initialization

2020-06-26 Thread Sergey Chugunov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-13189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Chugunov reassigned IGNITE-13189:


Assignee: Sergey Chugunov

> Maintenance mode switch and defragmentation process initialization
> --
>
> Key: IGNITE-13189
> URL: https://issues.apache.org/jira/browse/IGNITE-13189
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Sergey Chugunov
>Assignee: Sergey Chugunov
>Priority: Major
>
> As described in IEP-47 defragmentation is performed when a node enters a 
> special mode called maintenance mode.
> Discussion on dev-list clarifies algorithm to enter maintenance mode:
>  # Special key is written to local metastorage.
>  # Node is restarted.
>  # Node observes the key on startup and enters maintenance mode.
> Node should be fully-functioning in that mode but should not join the rest of 
> the cluster and participate in any regular activity like handling cache 
> operations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-5853) Provide a way to determine which user attributes are used in affinity calculation

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-5853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-5853:
--
Fix Version/s: (was: 2.9)
   2.10

> Provide a way to determine which user attributes are used in affinity 
> calculation
> -
>
> Key: IGNITE-5853
> URL: https://issues.apache.org/jira/browse/IGNITE-5853
> Project: Ignite
>  Issue Type: New Feature
>  Components: persistence
>Affects Versions: 2.1
>Reporter: Alexey Goncharuk
>Priority: Major
>  Labels: IEP-4, Phase-2
> Fix For: 2.10
>
>
> Since an affinity function may use user attributes to calculate affinity 
> distribution, we need to save these attributes to the metastore. However, 
> storing all the attributes is not very effective, so we need to have a way to 
> determine which attributes should be stored.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-5829) Writing entry contents to WAL only single time

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-5829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-5829:
--
Fix Version/s: (was: 2.9)
   2.10

> Writing entry contents to WAL only single time
> --
>
> Key: IGNITE-5829
> URL: https://issues.apache.org/jira/browse/IGNITE-5829
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.1
>Reporter: Pavel Kovalenko
>Assignee: Pavel Kovalenko
>Priority: Major
> Fix For: 2.10
>
>
> Currently we write entry contents 2 times: once in logical record and once 
> again when we write data page update records. We should do that only once. In 
> data page updates we can write only a reference to the logical update record 
> but not the whole entry contents.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-5580) Improve node failure cause information

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-5580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-5580:
--
Fix Version/s: (was: 2.9)
   2.10

> Improve node failure cause information
> --
>
> Key: IGNITE-5580
> URL: https://issues.apache.org/jira/browse/IGNITE-5580
> Project: Ignite
>  Issue Type: Improvement
>  Components: general
>Affects Versions: 1.7
>Reporter: Alexey Goncharuk
>Priority: Major
>  Labels: observability
> Fix For: 2.10
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When a node fails, we do not print out any information about the root cause 
> of this failure. This makes it extremely hard to investigate the failure 
> causes - I need to find a previous node for the failed node and check the 
> logs on the previous node.
> I suggest that we add extensive information about the reason of the node 
> failure and the sequence of events that led to this, e.g.:
> [time] [NODE] Sending a message to next node - failed _because_ - write 
> timeout, read timeout, ...?
> [time] [NODE] Connection check - failed - why? Connection refused, handshake 
> timed out, ...?
> ...
> [time] [NODE] Decided to drop the node because of the sequence above
> Maybe we do not need to print out this information always, but we do need 
> this when troubleshooting logger is enabled.
> Also, DiscoverySpi should collect a set of latest important events and dump 
> these events in case of local node segmentation. This will allow users to 
> match the events in the cluster and events on local node and get to the 
> bottom of the failure.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-5579) Make sure identical binary metadata updates from the same node do not happen twice

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-5579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-5579:
--
Fix Version/s: (was: 2.9)
   2.10

> Make sure identical binary metadata updates from the same node do not happen 
> twice
> --
>
> Key: IGNITE-5579
> URL: https://issues.apache.org/jira/browse/IGNITE-5579
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 1.7
>Reporter: Alexey Goncharuk
>Priority: Major
> Fix For: 2.10
>
>
> It is possible that multiple user threads attempt identical binary metadata 
> update concurrently. In this case, a node will just increase contention on 
> the cache key, but once the lock is acquired there will be nothing changed. 
> On large topologies, this may lead to nodes waiting for the metadata update 
> for hours (!).
> We should work out a way to identify identical metadata updates and allow 
> only one thread to proceed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (IGNITE-5560) A failed service must be redeployed when possible

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov resolved IGNITE-5560.
---
Resolution: Won't Fix

> A failed service must be redeployed when possible
> -
>
> Key: IGNITE-5560
> URL: https://issues.apache.org/jira/browse/IGNITE-5560
> Project: Ignite
>  Issue Type: Improvement
>  Components: managed services
>Affects Versions: 1.7
>Reporter: Alexey Goncharuk
>Priority: Major
> Fix For: 2.8
>
>
> I observed the following behavior: when a service deployment (or run) fails 
> with an unexpected exception, Ignite just outputs a warning to the console 
> and does not attempt anything to recover. 
> In our deployment, we rely on a cluster singleton to be always present in a 
> cluster. 
> If a service fails, Ignite should attempt to failover this service to some 
> other node, if possible



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-5560) A failed service must be redeployed when possible

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-5560:
--
Fix Version/s: (was: 2.9)
   2.8

> A failed service must be redeployed when possible
> -
>
> Key: IGNITE-5560
> URL: https://issues.apache.org/jira/browse/IGNITE-5560
> Project: Ignite
>  Issue Type: Improvement
>  Components: managed services
>Affects Versions: 1.7
>Reporter: Alexey Goncharuk
>Priority: Major
> Fix For: 2.8
>
>
> I observed the following behavior: when a service deployment (or run) fails 
> with an unexpected exception, Ignite just outputs a warning to the console 
> and does not attempt anything to recover. 
> In our deployment, we rely on a cluster singleton to be always present in a 
> cluster. 
> If a service fails, Ignite should attempt to failover this service to some 
> other node, if possible



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-5560) A failed service must be redeployed when possible

2020-06-26 Thread Aleksey Plekhanov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146346#comment-17146346
 ] 

Aleksey Plekhanov commented on IGNITE-5560:
---

Service Grid functionality was reworked by IEP-17. I believe this ticket is not 
actual anymore. Feel free to reopen it if it's still actual.

> A failed service must be redeployed when possible
> -
>
> Key: IGNITE-5560
> URL: https://issues.apache.org/jira/browse/IGNITE-5560
> Project: Ignite
>  Issue Type: Improvement
>  Components: managed services
>Affects Versions: 1.7
>Reporter: Alexey Goncharuk
>Priority: Major
> Fix For: 2.8
>
>
> I observed the following behavior: when a service deployment (or run) fails 
> with an unexpected exception, Ignite just outputs a warning to the console 
> and does not attempt anything to recover. 
> In our deployment, we rely on a cluster singleton to be always present in a 
> cluster. 
> If a service fails, Ignite should attempt to failover this service to some 
> other node, if possible



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-5427) Add cluster activation/deactivation lifecycle events

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-5427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-5427:
--
Fix Version/s: (was: 2.9)
   2.10

> Add cluster activation/deactivation lifecycle events
> 
>
> Key: IGNITE-5427
> URL: https://issues.apache.org/jira/browse/IGNITE-5427
> Project: Ignite
>  Issue Type: Improvement
>  Components: general
>Affects Versions: 2.0
>Reporter: Alexey Goncharuk
>Assignee: Sergey Dorozhkin
>Priority: Major
> Fix For: 2.10
>
>
> We should add AFTER_ACTIVATE and BEFORE_DEACTIVATE lifecycle event types.
> Add methods for these event to LifecycleListener interface.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-5473) Create ignite troubleshooting logger

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-5473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-5473:
--
Fix Version/s: (was: 2.9)
   2.10

> Create ignite troubleshooting logger
> 
>
> Key: IGNITE-5473
> URL: https://issues.apache.org/jira/browse/IGNITE-5473
> Project: Ignite
>  Issue Type: Improvement
>  Components: general
>Affects Versions: 2.0
>Reporter: Alexey Goncharuk
>Priority: Critical
>  Labels: important, observability
> Fix For: 3.0, 2.10
>
>
> Currently, we have two extremes of logging - either INFO wich logs almost 
> nothing, or DEBUG, which will pollute logs with too verbose messages.
> We should create a 'troubleshooting' logger, which should be easily enabled 
> (via a system property, for example) and log all stability-critical node and 
> cluster events:
>  * Connection events (both communication and discovery), handshake status
>  * ALL ignored messages and skipped actions (even those we assume are safe to 
> ignore)
>  * Partition exchange stages and timings
>  * Verbose discovery state changes (this should make it easy to understand 
> the reason for 'Node has not been connected to the topology')
>  * Transaction failover stages and actions
>  * All unlogged exceptions
>  * Responses that took more than N milliseconds when in normal they should 
> return right away
>  * Long discovery SPI messages processing times
>  * Managed service deployment stages
>  * Marshaller mappings registration and notification
>  * Binary metadata registration and notification
>  * Continuous query registration / notification
> (add more)
> The amount of logging should be chosen accurately so that it would be safe to 
> enable this logger in production clusters.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-5286) Reconsider deferredDelete implementation

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-5286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-5286:
--
Fix Version/s: (was: 2.9)
   2.10

> Reconsider deferredDelete implementation
> 
>
> Key: IGNITE-5286
> URL: https://issues.apache.org/jira/browse/IGNITE-5286
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.0
>Reporter: Alexey Goncharuk
>Priority: Major
> Fix For: 2.10
>
>
> In current implementation entries marked for 'deferredDelete' should be 
> stored on heap (otherwise information about remove is lost). This potentially 
> can be an issue if there are a lot of removes.
> (note: in current 'deferredDelete' implementation in Ignite there is a bug - 
> https://issues.apache.org/jira/browse/IGNITE-3299).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-5300) .NET: LINQ RemoveAll and UpdateAll examples

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-5300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-5300:
--
Fix Version/s: (was: 2.9)
   2.10

> .NET: LINQ RemoveAll and UpdateAll examples
> ---
>
> Key: IGNITE-5300
> URL: https://issues.apache.org/jira/browse/IGNITE-5300
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms
>Affects Versions: 2.1
>Reporter: Pavel Tupitsyn
>Priority: Major
>  Labels: .NET, LINQ
> Fix For: 2.10
>
>
> Update examples (normal and LinqPad) to demonstrate {{RemoveAll}} and 
> {{UpdateAll}} LINQ bulk operations (IGNITE-4904, IGNITE-5298).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-5110) Topology version should be included in synchronous continuous message

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-5110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-5110:
--
Fix Version/s: (was: 2.9)
   2.10

> Topology version should be included in synchronous continuous message
> -
>
> Key: IGNITE-5110
> URL: https://issues.apache.org/jira/browse/IGNITE-5110
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 2.0
>Reporter: Alexey Goncharuk
>Priority: Major
>  Labels: MakeTeamcityGreenAgain
> Fix For: 2.10
>
>
> I've observed the following situation on TC:
> {code}
> "sys-stripe-11-#12316%continuous.GridCacheContinuousQueryConcurrentTest1%" 
> prio=10 tid=0x7f8e1888f800 nid=0x44ad waiting on condition 
> [0x7f91050fa000]
>java.lang.Thread.State: WAITING (parking)
>   at sun.misc.Unsafe.park(Native Method)
>   at java.util.concurrent.locks.LockSupport.park(LockSupport.java:315)
>   at 
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:176)
>   at 
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:139)
>   at 
> org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.addNotification(GridContinuousProcessor.java:930)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.onEntryUpdate(CacheContinuousQueryHandler.java:817)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.access$800(CacheContinuousQueryHandler.java:92)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler$1.onEntryUpdated(CacheContinuousQueryHandler.java:420)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.onEntryUpdated(CacheContinuousQueryManager.java:346)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerSet(GridCacheMapEntry.java:1029)
>   - locked <0x000766998a50> (a 
> org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedCacheEntry)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:663)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.localFinish(GridDhtTxLocalAdapter.java:772)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.localFinish(GridDhtTxLocal.java:580)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.finishTx(GridDhtTxLocal.java:466)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.commitDhtLocalAsync(GridDhtTxLocal.java:514)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:841)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:720)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:676)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$200(IgniteTxHandler.java:95)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:153)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:151)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:863)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:386)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:308)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:100)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:253)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1257)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:885)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$2100(GridIoManager.java:114)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager$7.run(GridIoManager.java:802)
>   at 
> org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:483)
>   at 

[jira] [Updated] (IGNITE-5107) Remove nonHeapMemoryCommitted and nonHeapMemoryUsed from cluster metrics

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-5107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-5107:
--
Fix Version/s: (was: 2.9)
   3.0

> Remove nonHeapMemoryCommitted and nonHeapMemoryUsed from cluster metrics
> 
>
> Key: IGNITE-5107
> URL: https://issues.apache.org/jira/browse/IGNITE-5107
> Project: Ignite
>  Issue Type: Improvement
>  Components: general
>Affects Versions: 2.0
>Reporter: Alexey Goncharuk
>Priority: Major
> Fix For: 3.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-4959) Possible slight memory leak in free list

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-4959:
--
Fix Version/s: (was: 2.9)
   2.10

> Possible slight memory leak in free list
> 
>
> Key: IGNITE-4959
> URL: https://issues.apache.org/jira/browse/IGNITE-4959
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 2.0
>Reporter: Ivan Rakov
>Assignee: Alexey Goncharuk
>Priority: Major
> Fix For: 2.10
>
>
> To reproduce, run PageEvictionMultinodeTest (any eviction mode), set ENTRIES 
> to Integer.MAX_VALUE.
> Observations:
> 1) After a few minutes of test running, number of allocated pages looks like 
> a constant (a bit more than eviciton threshold, 90% by default). This is 
> expected behaviour with enabled page eviction.
> 2) More precise measurement shows that there's slow linear growth of 
> allocated pages number, literally 10-20 pages per minute.
> 3) Number of pages with type T_PAGE_LIST_NODE grows, number of all other 
> pages remains constant.
> 4) Though, total number of data pages in free list remains constant (with 
> minor fluctuations).
> We have to find out whether this process has a saturation point, after which 
> pages number stops growing. Otherwise, it's a memory leak and should be fixed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-4918) ScanQuery filter class code is not updated on server once it has been changed on client.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-4918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-4918:
--
Fix Version/s: (was: 2.9)
   2.10

> ScanQuery filter class code is not updated on server once it has been changed 
> on client.
> 
>
> Key: IGNITE-4918
> URL: https://issues.apache.org/jira/browse/IGNITE-4918
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Affects Versions: 1.9
>Reporter: Andrey Mashenkov
>Priority: Major
> Fix For: 2.10
>
> Attachments: ScanQueryFilter.java
>
>
> PFA reproducer attached. Peer class loading should be enabled on both, server 
> and client.
> Server should NOT has scan query filter class in its classpath.
> Steps to reproduce:
> - start standalone server
> - run repro, it should work fine.
> - run repro with changed filter code, it will return same results as on 
> previous step.
> Looks like filter code is cached on server and is never being updated.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-4554) Optimize integer sets.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-4554:
--
Fix Version/s: (was: 2.9)
   2.10

> Optimize integer sets.
> --
>
> Key: IGNITE-4554
> URL: https://issues.apache.org/jira/browse/IGNITE-4554
> Project: Ignite
>  Issue Type: Improvement
>  Components: general
>Affects Versions: 1.6
>Reporter: Alexey Scherbakov
>Assignee: Alexey Scherbakov
>Priority: Major
> Fix For: 2.10
>
>
> Ignite has many uses of data structures like Set, IntArray etc.
> This is not most efficient way to represent integer sets. The best way is to 
> use compressed bit sets. This should save a lot of heap space.
> We should optimize integer sets whenever possible.
> The most obvious place to start is GridAffinityAssignment.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-4682) Need to finish transition to thread-per-partition model

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-4682:
--
Fix Version/s: (was: 2.9)
   2.10

> Need to finish transition to thread-per-partition model
> ---
>
> Key: IGNITE-4682
> URL: https://issues.apache.org/jira/browse/IGNITE-4682
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Reporter: Yakov Zhdanov
>Assignee: Pavel Kovalenko
>Priority: Major
>  Labels: thread-per-partition
> Fix For: 2.10
>
>
> 1) Investigate performance on switching to tpp model and choose best solution 
> - https://issues.apache.org/jira/browse/IGNITE-9270
> 2) Implement transactions commit using tpp model - 
> https://issues.apache.org/jira/browse/IGNITE-9271
> 3) Implement putAll, removeAll on atomic caches using tpp model - 
> https://issues.apache.org/jira/browse/IGNITE-4680
> 4) Rebalance should works through tpp
> 5) Get rid of deleted entries buffer from DhtLocalPartition



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-4221) Document ComputeJobMasterLeaveAware interface usage

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-4221:
--
Fix Version/s: (was: 2.9)
   2.10

> Document ComputeJobMasterLeaveAware interface usage
> ---
>
> Key: IGNITE-4221
> URL: https://issues.apache.org/jira/browse/IGNITE-4221
> Project: Ignite
>  Issue Type: Task
>  Components: documentation
>Reporter: Denis A. Magda
>Assignee: Prachi Garg
>Priority: Major
> Fix For: 2.10
>
>
> The usage and applicability of `ComputeJobMasterLeaveAware` have to be 
> documented on Apache Ignite Readme.io which will help out to avoid discussion 
> like that [1]. The new page has to be created for the topic and placed here 
> [2].
> In advance, the following example has to be contributed to Apache Ignite
> https://github.com/gridgain/gridgain-advanced-examples/blob/master/src/main/java/org/gridgain/examples/compute/masterleave/ComputeMasterLeaveAwareExample.java
>  
> [1] 
> http://apache-ignite-users.70518.x6.nabble.com/Remote-Server-Thread-Not-exit-when-Job-finished-Cause-out-of-memory-tp8934p8947.html
> [2] https://apacheignite.readme.io/docs/compute-grid#section-features



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-4380) Cache invoke calls can be lost

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-4380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-4380:
--
Fix Version/s: (was: 2.9)
   2.10

> Cache invoke calls can be lost
> --
>
> Key: IGNITE-4380
> URL: https://issues.apache.org/jira/browse/IGNITE-4380
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 2.0
>Reporter: Semen Boikov
>Priority: Critical
>  Labels: MakeTeamcityGreenAgain
> Fix For: 2.10
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> * Recently added test 
> GridCacheAbstractFullApiSelfTest.testInvokeAllMultithreaded fails on TC in 
> various configurations with transactional cache.
> Example of failure 
> GridCacheReplicatedOffHeapTieredMultiNodeFullApiSelfTest.testInvokeAllMultithreaded:
> {noformat}
> junit.framework.AssertionFailedError: expected:<2> but was:<10868>
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.failNotEquals(Assert.java:329)
> at junit.framework.Assert.assertEquals(Assert.java:78)
> at junit.framework.Assert.assertEquals(Assert.java:234)
> at junit.framework.Assert.assertEquals(Assert.java:241)
> at junit.framework.TestCase.assertEquals(TestCase.java:409)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheAbstractFullApiSelfTest.testInvokeAllMultithreaded(GridCacheAbstractFullApiSelfTest.java:342)
> at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at 
> org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:1803)
> at 
> org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:118)
> at 
> org.apache.ignite.testframework.junits.GridAbstractTest$4.run(GridAbstractTest.java:1718)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-13189) Maintenance mode switch and defragmentation process initialization

2020-06-26 Thread Sergey Chugunov (Jira)
Sergey Chugunov created IGNITE-13189:


 Summary: Maintenance mode switch and defragmentation process 
initialization
 Key: IGNITE-13189
 URL: https://issues.apache.org/jira/browse/IGNITE-13189
 Project: Ignite
  Issue Type: Sub-task
Reporter: Sergey Chugunov


As described in IEP-47 defragmentation is performed when a node enters a 
special mode called maintenance mode.

Discussion on dev-list clarifies algorithm to enter maintenance mode:
 # Special key is written to local metastorage.
 # Node is restarted.
 # Node observes the key on startup and enters maintenance mode.

Node should be fully-functioning in that mode but should not join the rest of 
the cluster and participate in any regular activity like handling cache 
operations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13187) Jar hell in classpath leads to failed tests in C++ and .NET suites

2020-06-26 Thread Igor Sapego (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146289#comment-17146289
 ] 

Igor Sapego commented on IGNITE-13187:
--

[~ivandasch] looks good.

> Jar hell in classpath leads to failed tests in C++ and .NET suites
> --
>
> Key: IGNITE-13187
> URL: https://issues.apache.org/jira/browse/IGNITE-13187
> Project: Ignite
>  Issue Type: Test
>  Components: platforms
>Affects Versions: 2.8.1
> Environment: Apache Ignite TC.
>Reporter: Ivan Daschinskiy
>Assignee: Ivan Daschinskiy
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> On some agents tests and examples start failing with this calltrace:
> {code:java}
> [13:53:52]java.lang.NoSuchFieldError: logger
> [13:53:52]at 
> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:723)
> [13:53:52]at 
> org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:881)
> [13:53:52]at 
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:551)
> [13:53:52]at 
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:381)
> [13:53:52]at 
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:104)
> [13:53:52]at 
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:98)
> [13:53:52]at 
> org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:709)
> [13:53:52]at 
> org.apache.ignite.internal.IgnitionEx.loadConfiguration(IgnitionEx.java:767)
> [13:53:52]at 
> org.apache.ignite.internal.processors.platform.PlatformIgnition.configuration(PlatformIgnition.java:152)
> [13:53:52]at 
> org.apache.ignite.internal.processors.platform.PlatformIgnition.start(PlatformIgnition.java:67)
> {code}
> The main reason of failure is jar-hell. When .NET or C++ tests are started, 
> if IGNITE_NATIVE_TEST_CLASSPATH is set to true, source directory is iterated 
> and files libs, target/classes etc.are added to classpath. But neither 
> readdir(), FindNextFileA() or Directory.EnumerateDirectories() do guarantee 
> any ordering. But in spring-data-2.0 and spring-data-2.2 there are different 
> version of spring. So jar hell occurs and tests fails. 
> I created small patch that excluded spring-data from directories, that create 
> classpath and tests start working.
> See links attached.
> {code:java}
> diff --git a/modules/platforms/cpp/jni/os/linux/src/utils.cpp 
> b/modules/platforms/cpp/jni/os/linux/src/utils.cpp
> index 1fcdb317d2..586c37d008 100644
> --- a/modules/platforms/cpp/jni/os/linux/src/utils.cpp
> +++ b/modules/platforms/cpp/jni/os/linux/src/utils.cpp
> @@ -183,7 +183,7 @@ namespace ignite
>  {
>  std::string res;
>  
> -if (FileExists(path))
> +if (FileExists(path) && (path.find("spring-data") == 
> std::string::npos))
>  {
>  // 1. Append "target\classes".
>  std::string classesPath = path + "/target/classes";
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-4210) CacheLoadingConcurrentGridStartSelfTest.testLoadCacheFromStore() test lose data.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-4210:
--
Fix Version/s: (was: 2.9)
   2.10

> CacheLoadingConcurrentGridStartSelfTest.testLoadCacheFromStore() test lose 
> data.
> 
>
> Key: IGNITE-4210
> URL: https://issues.apache.org/jira/browse/IGNITE-4210
> Project: Ignite
>  Issue Type: Bug
>Reporter: Anton Vinogradov
>Assignee: Alexey Kuznetsov
>Priority: Major
>  Labels: MakeTeamcityGreenAgain
> Fix For: 2.10
>
>
> org.apache.ignite.internal.processors.cache.distributed.CacheLoadingConcurrentGridStartSelfTest#testLoadCacheFromStore
>  sometimes have failures.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-4188) Savepoints support inside of Ignite Transactions

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-4188:
--
Fix Version/s: (was: 2.9)
   2.10

> Savepoints support inside of Ignite Transactions
> 
>
> Key: IGNITE-4188
> URL: https://issues.apache.org/jira/browse/IGNITE-4188
> Project: Ignite
>  Issue Type: Task
>Reporter: Denis A. Magda
>Priority: Major
> Fix For: 2.10
>
>
> A savepoint is a special mark inside a transaction that allows all commands 
> that are executed after it was established to be rolled back, restoring the 
> transaction state to what it was at the time of the savepoint.
> Here is a reference to the similar functionality implemented by some of RDBMs 
> vendors.
> https://www.postgresql.org/docs/8.1/static/sql-savepoint.html
> https://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_10001.htm
> http://dev.mysql.com/doc/refman/5.7/en/savepoint.html
> Consider the following example.
> {code}
> BEGIN;
> INSERT INTO table1 VALUES (1); 
> SAVEPOINT my_savepoint; 
> INSERT INTO table1 VALUES (2); 
> ROLLBACK TO SAVEPOINT my_savepoint; 
> INSERT INTO table1 VALUES (3); 
> COMMIT;
> {code}
> The execution result must guarantee that only values 1 and 3 are inserted 
> into table1.
> In Ignite, it should be supported this way (preserving the same behavior as 
> above).
> {code}
> Ignite ignite = ;
> IgniteCache c = ;
> try (Transaction tx = ignite.transactions().txStart()) {
> c.put(1, 1);
> 
> tx.savepoint("mysavepoint");
> 
> c.put(2, 2);
> 
> tx.rollbackToSavepoint("mysavepoint");
> 
> c.put(3, 3);
> 
> tx.commit();
> }
> {code}
> As a summary the following has to be supported on Ignite side:
> - The {{savepoint}} method which will set a named transaction savepoint with 
> a name of an identifier.
> - Multiple savepoints defined within a transaction. The names of the 
> savepoints have to differ from each other. If the current transaction has a 
> savepoint with the same name, the old savepoint is deleted and a new one is 
> set.
> - The {{rollbackToSavepoint}} method that will roll back all the changes done 
> after a specific checkpoint establishment.
> - The {{releaseCheckpoint}} method that will destroy a savepoint, keeping the 
> effects of commands executed after it was established.
> - Full support of the behavior listed above at the level of ODBC and JDBC 
> drivers and DML (will be handled under separate tickets).
> - The behavior has to be support for all transactional modes.
> Original proposal on the dev list:
> http://apache-ignite-developers.2346864.n4.nabble.com/TX-savepoints-td12041.html
> Deadlock detection should be addressed and possibly updated to work properly 
> with presence of savepoints. See linked ticket.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-3471) Do not send previous value to client node for invoke() when possible

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-3471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-3471:
--
Fix Version/s: (was: 2.9)
   2.10

> Do not send previous value to client node for invoke() when possible
> 
>
> Key: IGNITE-3471
> URL: https://issues.apache.org/jira/browse/IGNITE-3471
> Project: Ignite
>  Issue Type: Task
>  Components: cache
>Affects Versions: 1.4
>Reporter: Alexey Goncharuk
>Priority: Major
> Fix For: 2.10
>
> Attachments: CacheEntryProcessorTxSelfTest.java
>
>
> Currently for invoke() or invokeAll() methods we send previous cache value to 
> near node and apply EntryProcessor locally to get a return value. This can 
> induce a significant overhead when cache value is much larger than entry 
> processor result.
> For many cases this can be avoided, e.g.
> {code}
> try (tx = txStart()) {
> cache.invoke(key, EP); // No need to send previous value to client in 
> this case.
> tx.commit();
> }
> {code}
> Note that we need to add additional handling of such a case:
> {code}
> try (tx = txStart()) {
> cache.invoke(key, EP); // No need to send previous value to client in 
> this case.
> cache.get(key); // This should actually get the current cache value from 
> primary node and apply an entry processor locally to get the updated value.
> tx.commit();
> }
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-1678) IgniteAtomicSequence: make following reservations in advance

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-1678:
--
Fix Version/s: (was: 2.9)
   2.10

> IgniteAtomicSequence: make following reservations in advance
> 
>
> Key: IGNITE-1678
> URL: https://issues.apache.org/jira/browse/IGNITE-1678
> Project: Ignite
>  Issue Type: Improvement
>  Components: data structures
>Affects Versions: ignite-1.4
>Reporter: Denis A. Magda
>Assignee: Dmitriy Govorukhin
>Priority: Major
> Fix For: 2.10
>
> Attachments: Screenshot from 2016-12-15 10-50-22.png
>
>
> In current implementation a new reservation is made when the current local 
> sequence boundary is exceeded.
> In cases when there are many nodes that use the same atomic sequence there 
> can be a situation when all the nodes start doing a new reservation at the 
> same time. This can lead to performance drops.
> As a performance optimization it makes sense to start reserving new sequence 
> slot in advance (in background), like when around 80% of current reservation 
> has already been used. Probably we should add a special parameter to 
> {{AtomicConfiguration}} that will manage such behavior.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12493) Test refactoring. Explicit method for starting client nodes

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12493:
---
Fix Version/s: 2.9

> Test refactoring. Explicit method for starting client nodes
> ---
>
> Key: IGNITE-12493
> URL: https://issues.apache.org/jira/browse/IGNITE-12493
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.7.6
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: newbie
> Fix For: 2.9
>
>  Time Spent: 12h 20m
>  Remaining Estimate: 0h
>
> Right now there is almost 500 explicit usage of {{setClientMode}} in tests.
> Seems we should support the starting of client nodes in test framework.
> We should refactor tests to use {{startClientNode(String name)}}.
> This will simplify tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12641) IgniteSequenceInternalCleanupTest flacky after IGNITE-12598

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12641:
---
Fix Version/s: 2.9

> IgniteSequenceInternalCleanupTest flacky after IGNITE-12598
> ---
>
> Key: IGNITE-12641
> URL: https://issues.apache.org/jira/browse/IGNITE-12641
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.9
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> IgniteSequenceInternalCleanupTest#deactivate is flacky after IGNITE-12598
> The test expects that cache metrics will be reset after cluster deactivation 
> and subsequent cache stop.
> But this was changed intentionally in IGNITE-12598.
> Test not always fail because it asserts cluster metrics value which updated 
> concurrently with the test thread. 
> We should fix the assert.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-12903) Fix ML + SQL examples

2020-06-26 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146263#comment-17146263
 ] 

Ignite TC Bot commented on IGNITE-12903:


{panel:title=Branch: [pull/7965/head] Base: [master] : Possible Blockers 
(116)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}MVCC Cache 7{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419229]]

{color:#d04437}PDS 4{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419208]]

{color:#d04437}PDS (Indexing){color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419203]]

{color:#d04437}ZooKeeper (Discovery) 1{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419162]]

{color:#d04437}Platform .NET (Long Running){color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419213]]

{color:#d04437}Queries 1{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419215]]

{color:#d04437}MVCC PDS 4{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419235]]

{color:#d04437}Cache 5{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419192]]

{color:#d04437}PDS 2{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419206]]

{color:#d04437}PDS 1{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419205]]

{color:#d04437}MVCC PDS 2{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419233]]

{color:#d04437}Platform C++ CMake (Linux Clang){color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419240]]

{color:#d04437}PDS 3{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419207]]

{color:#d04437}SPI{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419155]]

{color:#d04437}ZooKeeper (Discovery) 2{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419163]]

{color:#d04437}Cache (Expiry Policy){color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419178]]

{color:#d04437}Cache 7{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419194]]

{color:#d04437}Cache 6{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419193]]

{color:#d04437}Platform .NET{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419209]]

{color:#d04437}Platform C++ CMake (Linux){color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419241]]

{color:#d04437}Examples{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419136]]

{color:#d04437}Streamers{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419153]]

{color:#d04437}Continuous Query 3{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419198]]

{color:#d04437}ZooKeeper (Discovery) 3{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419217]]

{color:#d04437}Scala (Examples){color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419160]]

{color:#d04437}Cache 2{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419189]]

{color:#d04437}Cache (Restarts) 1{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419185]]

{color:#d04437}MVCC Queries{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419173]]

{color:#d04437}Thin Client: Java{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419146]]

{color:#d04437}Platform .NET (Core Linux){color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419210]]

{color:#d04437}Cache 1{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419188]]

{color:#d04437}Thin client: Node.js{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419166]]

{color:#d04437}Cache 8{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419195]]

{color:#d04437}MVCC PDS 1{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419232]]

{color:#d04437}Thin client: PHP{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419174]]

{color:#d04437}Basic 3{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419168]]

{color:#d04437}Cache 3{color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419190]]

{color:#d04437}Cache (Deadlock Detection){color} [[tests 0 
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=5419177]]

{color:#d04437}Basic 1{color} [[tests 0 

[jira] [Updated] (IGNITE-12687) Sqline throws unexpected exceptions on few specific arguments to !describe and !primarykeys

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12687:
---
Fix Version/s: 2.9

> Sqline throws unexpected exceptions on few specific arguments to !describe 
> and !primarykeys
> ---
>
> Key: IGNITE-12687
> URL: https://issues.apache.org/jira/browse/IGNITE-12687
> Project: Ignite
>  Issue Type: Bug
>Reporter: Konstantin Orlov
>Assignee: Konstantin Orlov
>Priority: Major
> Fix For: 2.9
>
>
> %subj 
> {noformat}
> 0: jdbc:ignite:thin://127.0.0.1/> !describe '
> java.lang.StringIndexOutOfBoundsException: String index out of range: -1
>   at java.lang.String.substring(String.java:1967)
>   at sqlline.SqlLine.dequote(SqlLine.java:1322)
>   at sqlline.SqlLine.split(SqlLine.java:1335)
>   at sqlline.SqlLine.split(SqlLine.java:1154)
>   at 
> sqlline.AbstractCommandHandler.matches(AbstractCommandHandler.java:65)
>   at sqlline.SqlLine.dispatch(SqlLine.java:774)
>   at sqlline.SqlLine.begin(SqlLine.java:668)
>   at sqlline.SqlLine.start(SqlLine.java:373)
>   at sqlline.SqlLine.main(SqlLine.java:265)
> {noformat}
> when at least one table was created, following command also fails:
> {noformat}
> 0: jdbc:ignite:thin://127.0.0.1/> !describe ?
> [17:19:31,763][SEVERE][client-connector-#110][JdbcRequestHandler] Failed to 
> get columns metadata [reqId=108, req=JdbcMetaColumnsRequest [schemaName=null, 
> tblName=?, colName=%]]
> java.util.regex.PatternSyntaxException: Dangling meta character '?' near 
> index 0
> ?
> ^
>   at java.util.regex.Pattern.error(Pattern.java:1955)
>   at java.util.regex.Pattern.sequence(Pattern.java:2123)
>   at java.util.regex.Pattern.expr(Pattern.java:1996)
>   at java.util.regex.Pattern.compile(Pattern.java:1696)
>   at java.util.regex.Pattern.(Pattern.java:1351)
>   at java.util.regex.Pattern.compile(Pattern.java:1028)
>   at java.util.regex.Pattern.matches(Pattern.java:1133)
>   at java.lang.String.matches(String.java:2121)
>   at 
> org.apache.ignite.internal.processors.query.QueryUtils.matches(QueryUtils.java:1547)
>   at 
> org.apache.ignite.internal.processors.odbc.jdbc.JdbcMetadataInfo.getColumnsMeta(JdbcMetadataInfo.java:170)
>   at 
> org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.getColumnsMeta(JdbcRequestHandler.java:1089)
>   at 
> org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.doHandle(JdbcRequestHandler.java:332)
>   at 
> org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.handle(JdbcRequestHandler.java:241)
>   at 
> org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:180)
>   at 
> org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:47)
>   at 
> org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:278)
>   at 
> org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:108)
>   at 
> org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:96)
>   at 
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
>   at 
> org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:69)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:748)
> Error: Dangling meta character '?' near index 0
> ?
> ^ (state=5,code=1)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12723) Test refactoring. Explicit method to get MXBean instance

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12723:
---
Fix Version/s: 2.9

> Test refactoring. Explicit method to get MXBean instance
> 
>
> Key: IGNITE-12723
> URL: https://issues.apache.org/jira/browse/IGNITE-12723
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Minor
> Fix For: 2.9
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Test code contains several dozen of copy-paste code to get different MXBeans.
> We can reduce code base by introducing common method for it.
> Example of copy pasted code:
> {code:java}
> /** */
> private FailureHandlingMxBean getMBean() throws Exception {
> ObjectName name = U.makeMBeanName(getTestIgniteInstanceName(0), 
> "Kernal",
> FailureHandlingMxBeanImpl.class.getSimpleName());
> MBeanServer srv = ManagementFactory.getPlatformMBeanServer();
> assertTrue(srv.isRegistered(name));
> return MBeanServerInvocationHandler.newProxyInstance(srv, name, 
> FailureHandlingMxBean.class, true);
> }
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12763) OpenCensus integration example.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12763:
---
Fix Version/s: 2.9

> OpenCensus integration example.
> ---
>
> Key: IGNITE-12763
> URL: https://issues.apache.org/jira/browse/IGNITE-12763
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Fix For: 2.9
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> We should provide to the user simple self-explaining example of the 
> integration with the opencensus.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12811) [IEP-39] Management API to cancel services.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12811:
---
Fix Version/s: 2.9

> [IEP-39] Management API to cancel services.
> ---
>
> Key: IGNITE-12811
> URL: https://issues.apache.org/jira/browse/IGNITE-12811
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-39
> Fix For: 2.9
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Ignite provides many API to deploy and execute user-provided code on the 
> server nodes inside the same JVM as the Ignite process runs.
> Ignite has many APIs that allocate many resources on the server nodes, also. 
> In case of some buggy code that consumes many system resources(CPU, RAM, 
> flood network) or heavy query the whole cluster can become unstable.
> We should provide to the cluster administrator the ability to stop any user 
> deployed task.
> JMX beans to cancel listed tasks should be introduced:
> * Services
> In the scope of IEP-35 System view API introduced.
> A new API should use the same identifier that is used in corresponding System 
> View.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12812) [IEP-39] Management API to cancel transaction.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12812:
---
Fix Version/s: 2.9

> [IEP-39] Management API to cancel transaction.
> --
>
> Key: IGNITE-12812
> URL: https://issues.apache.org/jira/browse/IGNITE-12812
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-39
> Fix For: 2.9
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Ignite provides many API to deploy and execute user-provided code on the 
> server nodes inside the same JVM as the Ignite process runs.
> Ignite has many APIs that allocate many resources on the server nodes, also. 
> In case of some buggy code that consumes many system resources(CPU, RAM, 
> flood network) or heavy query the whole cluster can become unstable.
> We should provide to the cluster administrator the ability to stop any user 
> deployed task.
> JMX beans to cancel listed tasks should be introduced:
> * Transaction
> In the scope of IEP-35 System view API introduced.
> A new API should use the same identifier that is used in corresponding System 
> View.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12814) [IEP-39] Management API to cancel continuous queries.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12814:
---
Fix Version/s: 2.9

> [IEP-39] Management API to cancel continuous queries.
> -
>
> Key: IGNITE-12814
> URL: https://issues.apache.org/jira/browse/IGNITE-12814
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-39
> Fix For: 2.9
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Ignite provides many API to deploy and execute user-provided code on the 
> server nodes inside the same JVM as the Ignite process runs.
> Ignite has many APIs that allocate many resources on the server nodes, also. 
> In case of some buggy code that consumes many system resources(CPU, RAM, 
> flood network) or heavy query the whole cluster can become unstable.
> We should provide to the cluster administrator the ability to stop any user 
> deployed task.
> JMX beans to cancel listed tasks should be introduced:
> * Continuous queries
> In the scope of IEP-35 System view API introduced.
> A new API should use the same identifier that is used in corresponding System 
> View.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12815) [IEP-39] Management API to cancel SQL queries.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12815:
---
Fix Version/s: 2.9

> [IEP-39] Management API to cancel SQL queries.
> --
>
> Key: IGNITE-12815
> URL: https://issues.apache.org/jira/browse/IGNITE-12815
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-39
> Fix For: 2.9
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Ignite provides many API to deploy and execute user-provided code on the 
> server nodes inside the same JVM as the Ignite process runs.
> Ignite has many APIs that allocate many resources on the server nodes, also. 
> In case of some buggy code that consumes many system resources(CPU, RAM, 
> flood network) or heavy query the whole cluster can become unstable.
> We should provide to the cluster administrator the ability to stop any user 
> deployed task.
> JMX beans to cancel listed tasks should be introduced:
> * SQL queries.
> In the scope of IEP-35 System view API introduced.
> A new API should use the same identifier that is used in corresponding System 
> View.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12813) [IEP-39] Management API to cancel scan queries.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12813:
---
Fix Version/s: 2.9

> [IEP-39] Management API to cancel scan queries.
> ---
>
> Key: IGNITE-12813
> URL: https://issues.apache.org/jira/browse/IGNITE-12813
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-39
> Fix For: 2.9
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Ignite provides many API to deploy and execute user-provided code on the 
> server nodes inside the same JVM as the Ignite process runs.
> Ignite has many APIs that allocate many resources on the server nodes, also. 
> In case of some buggy code that consumes many system resources(CPU, RAM, 
> flood network) or heavy query the whole cluster can become unstable.
> We should provide to the cluster administrator the ability to stop any user 
> deployed task.
> JMX beans to cancel listed tasks should be introduced:
> * scan queries
> In the scope of IEP-35 System view API introduced.
> A new API should use the same identifier that is used in corresponding System 
> View.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12632) [IEP-39] Management API to cancel user provided tasks and queries.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12632:
---
Fix Version/s: 2.9

> [IEP-39] Management API to cancel user provided tasks and queries.
> --
>
> Key: IGNITE-12632
> URL: https://issues.apache.org/jira/browse/IGNITE-12632
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-39, important
> Fix For: 2.9
>
>  Time Spent: 4h 10m
>  Remaining Estimate: 0h
>
> Ignite provides many API to deploy and execute user-provided code on the 
> server nodes inside the same JVM as the Ignite process runs.
> Ignite has many APIs that allocate many resources on the server nodes, also. 
> In case of some buggy code that consumes many system resources(CPU, RAM, 
> flood network) or heavy query the whole cluster can become unstable.
> We should provide to the cluster administrator the ability to stop any user 
> deployed task.
> JMX beans to cancel listed tasks should be introduced:
> * Compute task
> * Service
> * Continuous query
> * Transactions
> * Queries(SQL, Scan, Text)
> In the scope of IEP-35 System view API introduced.
> A new API should use the same identifier that is used in corresponding System 
> View.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12632) [IEP-39] Management API to cancel user provided tasks and queries.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12632:
---
Labels: IEP-39 important  (was: IEP-39)

> [IEP-39] Management API to cancel user provided tasks and queries.
> --
>
> Key: IGNITE-12632
> URL: https://issues.apache.org/jira/browse/IGNITE-12632
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-39, important
>  Time Spent: 4h 10m
>  Remaining Estimate: 0h
>
> Ignite provides many API to deploy and execute user-provided code on the 
> server nodes inside the same JVM as the Ignite process runs.
> Ignite has many APIs that allocate many resources on the server nodes, also. 
> In case of some buggy code that consumes many system resources(CPU, RAM, 
> flood network) or heavy query the whole cluster can become unstable.
> We should provide to the cluster administrator the ability to stop any user 
> deployed task.
> JMX beans to cancel listed tasks should be introduced:
> * Compute task
> * Service
> * Continuous query
> * Transactions
> * Queries(SQL, Scan, Text)
> In the scope of IEP-35 System view API introduced.
> A new API should use the same identifier that is used in corresponding System 
> View.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12810) [IEP-39] Management API to cancel compute tasks.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12810:
---
Labels: IEP-39  (was: IEP-39 important)

> [IEP-39] Management API to cancel compute tasks.
> 
>
> Key: IGNITE-12810
> URL: https://issues.apache.org/jira/browse/IGNITE-12810
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-39
> Fix For: 2.9
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Ignite provides many API to deploy and execute user-provided code on the 
> server nodes inside the same JVM as the Ignite process runs.
> Ignite has many APIs that allocate many resources on the server nodes, also. 
> In case of some buggy code that consumes many system resources(CPU, RAM, 
> flood network) or heavy query the whole cluster can become unstable.
> We should provide to the cluster administrator the ability to stop any user 
> deployed task.
> JMX beans to cancel listed tasks should be introduced:
> * Compute task
> In the scope of IEP-35 System view API introduced.
> A new API should use the same identifier that is used in corresponding System 
> View.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12872) Check of correct work of explicit security permissions

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12872:
---
Fix Version/s: 2.9

> Check of correct work of explicit security permissions
> --
>
> Key: IGNITE-12872
> URL: https://issues.apache.org/jira/browse/IGNITE-12872
> Project: Ignite
>  Issue Type: Test
>Reporter: Nikolay Izhikov
>Assignee: Denis Garus
>Priority: Major
> Fix For: 2.9
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> We need to add 3 tests that checks explicit security permission for the cache.
> 1. If a user has only CACHE_READ permission for some cache.
>  * get(and other read operations are permitted)
>  * put, remove (and other write operations are prohibited).
> 2. If a user has only CACHE_REMOVE permission for some cache.
>  * put, get(and other reads/write operations are prohibited)
>  * remove (and other remove operations are permitted).
> 3. If a user has only CACHE_PUT permission for some cache.
>  * get, remove(and other reads/remove operations are prohibited)
>  * put (and other write operations are permitted).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12841) @Override must be on the same line as a method

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12841:
---
Fix Version/s: 2.9

> @Override must be on the same line as a method
> --
>
> Key: IGNITE-12841
> URL: https://issues.apache.org/jira/browse/IGNITE-12841
> Project: Ignite
>  Issue Type: Bug
>Reporter: Nikolay Izhikov
>Assignee: Oleg Ostanin
>Priority: Trivial
>  Labels: newbie
> Fix For: 2.9
>
> Attachments: Снимок экрана 2020-04-12 в 11.12.11 (2).png
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> Right now, there are many places where codestyle broken.
> {noformat}
> /** {@inheritDoc} */
> @Override
> public boolean registerClassName(byte platformId, int typeId, String 
> clsName) throws IgniteCheckedException {
> return registerClassName(platformId, typeId, clsName, false);
> }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12879) Refactor test configuration of discovery messages interceptors.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12879:
---
Fix Version/s: 2.9

> Refactor test configuration of discovery messages interceptors.
> ---
>
> Key: IGNITE-12879
> URL: https://issues.apache.org/jira/browse/IGNITE-12879
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Mikhail Petrov
>Assignee: Mikhail Petrov
>Priority: Minor
> Fix For: 2.9
>
>  Time Spent: 3h
>  Remaining Estimate: 0h
>
> It's needed to change DiscoveryHook class method naming to the following:
> {code:java}
> public void beforeDiscovery(DiscoverySpiCustomMessage msg)
> public void afterDiscovery(DiscoverySpiCustomMessage msg)
> {code}
> It will help to clarify the purpose of the methods.
> It's needed to add the ability to configure DiscoveryHook instance through 
> TestTcpDiscoverySpi for discovery messages interception. It helps to avoid 
> redefinition of the TestTcpDiscoverySpi and its reconfiguration. The current 
> approach is as follows:
> {code:java}
> TcpDiscoverySpi spi = new TestTcpDiscoverySpi() {
> @Override public void setListener(@Nullable DiscoverySpiListener lsnr) {
> super.setListener(DiscoverySpiListenerWrapper.wrap(lsnr, 
> discoveryHook));
> }
> };
> spi.setIpFinder(((TcpDiscoverySpi)cfg.getDiscoverySpi()).getIpFinder());
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12856) Fix two corruptedIndexPartition flaky tests.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12856:
---
Fix Version/s: 2.9

> Fix two corruptedIndexPartition flaky tests.
> 
>
> Key: IGNITE-12856
> URL: https://issues.apache.org/jira/browse/IGNITE-12856
> Project: Ignite
>  Issue Type: Improvement
>  Components: general
>Affects Versions: 2.8
>Reporter: Stanilovsky Evgeny
>Assignee: Stanilovsky Evgeny
>Priority: Major
> Fix For: 2.9
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> TC highlights two new flack tests: 
> org.apache.ignite.util.GridCommandHandlerIndexingTest#testCorruptedIndexPartitionShouldFailValidationWithCrc
>  
> org.apache.ignite.util.GridCommandHandlerIndexingTest#testCorruptedIndexPartitionShouldFailValidationWithoutCrc



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12980) Use Apache Ignite logo with registered trade mark symbol in README.md

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12980:
---
Fix Version/s: 2.9

> Use Apache Ignite logo with registered trade mark symbol in README.md
> -
>
> Key: IGNITE-12980
> URL: https://issues.apache.org/jira/browse/IGNITE-12980
> Project: Ignite
>  Issue Type: Task
>Reporter: Ivan Pavlukhin
>Assignee: Ivan Pavlukhin
>Priority: Major
> Fix For: 2.9
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Logo in the top of https://github.com/apache/ignite/blob/master/README.md 
> should have registered trademark symbol ® instead of unregistered one ™.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12987) Remove TC badge from README.md

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12987:
---
Fix Version/s: 2.9

> Remove TC badge from README.md
> --
>
> Key: IGNITE-12987
> URL: https://issues.apache.org/jira/browse/IGNITE-12987
> Project: Ignite
>  Issue Type: Task
>Reporter: Ivan Pavlukhin
>Assignee: Ivan Pavlukhin
>Priority: Major
> Fix For: 2.9
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Currently TC badge in main 
> [README|https://github.com/apache/ignite/blob/master/README.md] seems to be 
> not very useful because a build status is not seen directly (is says "no 
> permissions to get data"). Moreover TC results are not very representative 
> because integration tests are not stable by nature and we rely on TC bot 
> checks instead of plain TC status.
> Let's remove this badge.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12965) Redirect ignite-website GitHub notifications

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12965:
---
Fix Version/s: 2.9

> Redirect ignite-website GitHub notifications
> 
>
> Key: IGNITE-12965
> URL: https://issues.apache.org/jira/browse/IGNITE-12965
> Project: Ignite
>  Issue Type: Task
>Reporter: Ivan Pavlukhin
>Assignee: Ivan Pavlukhin
>Priority: Major
> Fix For: 2.9
>
>
> GitHub notifications for all Ignite repositories are sent to 
> notificati...@ignite.apache.org after INFRA-17351. But notifications for a 
> new ignite-website repository (https://github.com/apache/ignite-website) are 
> sent to d...@ignite.apache.org. These notifications should be sent to 
> notificati...@ignite.apache.org as well.
> Apparently we can solve it on our side by editing .asf.yaml file according to 
> https://cwiki.apache.org/confluence/display/INFRA/.asf.yaml+features+for+git+repositories



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12993) Github notification integrations on new ASF infra feature

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12993:
---
Fix Version/s: 2.9

> Github notification integrations on new ASF infra feature
> -
>
> Key: IGNITE-12993
> URL: https://issues.apache.org/jira/browse/IGNITE-12993
> Project: Ignite
>  Issue Type: Task
>Reporter: Pavel Pereslegin
>Assignee: Pavel Pereslegin
>Priority: Major
> Fix For: 2.9
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Recent github pull requests are not automatically linked to the jira tickets.
> Infra 
> [advises|https://issues.apache.org/jira/browse/INFRA-20177?focusedCommentId=17090758=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17090758]
>  creating a yaml configuration in the root of the repository with the 
> settings for the github bot.
> Feature description: 
> [https://cwiki.apache.org/confluence/display/INFRA/.asf.yaml+features+for+git+repositories]
> According to the current github setup, I suggest the following initial 
> configuration:
> {code:yaml}
> github: 
>   description: "Apache Ignite"
>   homepage: https://ignite.apache.org/
>   labels: 
> - distributed-sql-database 
> - iot
> - osgi
> - network-client
> - ignite
> - data-management-platform
> - big-data
> - cloud
> - database
> - network-server
> - hadoop
> - sql
>   features: 
> wiki: false
> issues: false
> projects: false
>   enabled_merge_buttons: 
> squash:  true
> merge:   false
> rebase:  false
> notifications: 
> commits:  comm...@ignite.apache.org
> issues:   notificati...@ignite.apache.org
> pullrequests: notificati...@ignite.apache.org
> jira_options: link label worklog
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12992) Fix multijvm failing tests in ZookeeperDiscoverySpiTestSuite3

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12992:
---
Fix Version/s: 2.9

> Fix multijvm failing tests in ZookeeperDiscoverySpiTestSuite3
> -
>
> Key: IGNITE-12992
> URL: https://issues.apache.org/jira/browse/IGNITE-12992
> Project: Ignite
>  Issue Type: Test
>Reporter: Ivan Daschinskiy
>Assignee: Ivan Daschinskiy
>Priority: Minor
> Fix For: 2.9
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12998) Fix a typo in CONTRIBUTING.md

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12998:
---
Fix Version/s: 2.9

> Fix a typo in CONTRIBUTING.md
> -
>
> Key: IGNITE-12998
> URL: https://issues.apache.org/jira/browse/IGNITE-12998
> Project: Ignite
>  Issue Type: Task
>Reporter: Ivan Pavlukhin
>Assignee: Ivan Pavlukhin
>Priority: Major
> Fix For: 2.9
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> CoPDoC abbreviation should be spelled properly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-13009) NoopTimeBag should be used if logging disabled

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-13009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-13009:
---
Fix Version/s: 2.9

> NoopTimeBag should be used if logging disabled
> --
>
> Key: IGNITE-13009
> URL: https://issues.apache.org/jira/browse/IGNITE-13009
> Project: Ignite
>  Issue Type: Task
>Reporter: Anton Vinogradov
>Assignee: Anton Vinogradov
>Priority: Major
> Fix For: 2.9
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> TimeBag uses performs useless operations on finishGlobalStage if logging 
> disabled.
> Better case is to use no-op realization instead.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-13012) Fix failure detection timeout. Simplify node ping routine.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-13012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-13012:
---
Fix Version/s: 2.9

> Fix failure detection timeout. Simplify node ping routine.
> --
>
> Key: IGNITE-13012
> URL: https://issues.apache.org/jira/browse/IGNITE-13012
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.8.1
>Reporter: Vladimir Steshin
>Assignee: Vladimir Steshin
>Priority: Major
>  Labels: iep-45
> Fix For: 2.9
>
> Attachments: IGNITE-13012-patch.patch
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Connection failure may not be detected within 
> IgniteConfiguration.failureDetectionTimeout. Actual worst delay is: 
> ServerImpl.CON_CHECK_INTERVAL + IgniteConfiguration.failureDetectionTimeout. 
> Node ping routine is duplicated.
> We should fix:
> 1. Failure detection timeout should take in account last sent message. 
> Current ping is bound to own time:
> {code:java}ServerImpl. RingMessageWorker.lastTimeConnCheckMsgSent{code}
> This is weird because any discovery message check connection. 
> 2. Make connection check interval depend on failure detection timeout (FTD). 
> Current value is a constant:
> {code:java}static int ServerImpls.CON_CHECK_INTERVAL = 500{code}
> 3. Remove additional, quickened connection checking.  Once we do fix 1, this 
> will become even more useless.
> Despite TCP discovery has a period of connection checking, it may send ping 
> before this period exhausts. This premature ping relies also on the time of 
> any received message for some reason. 
> 4. Do not worry user with “Node seems disconnected” when everything is OK. 
> Once we do fix 1 and 3, this will become even more useless. 
> Node may log on INFO: “Local node seems to be disconnected from topology …” 
> whereas it is not actually disconnected at all.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-13166) Flaky H2RowCachePageEvictionTest and IgniteCacheQueryH2IndexingLeakTest tests

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-13166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-13166:
---
Fix Version/s: 2.9

> Flaky H2RowCachePageEvictionTest and IgniteCacheQueryH2IndexingLeakTest tests 
> --
>
> Key: IGNITE-13166
> URL: https://issues.apache.org/jira/browse/IGNITE-13166
> Project: Ignite
>  Issue Type: Task
>Reporter: Konstantin Orlov
>Assignee: Konstantin Orlov
>Priority: Major
> Fix For: 2.9
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Currently we have several tests on TC which considered flaky. Need to 
> investigate and fix them.
>  
> [https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-4422957309644246466=testDetails]
> [https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=8791646325227962211=testDetails]
> [https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-7378154946319099847=testDetails]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12085) ThreadPool metrics register after all components start

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12085:
---
Fix Version/s: 2.9

> ThreadPool metrics register after all components start
> --
>
> Key: IGNITE-12085
> URL: https://issues.apache.org/jira/browse/IGNITE-12085
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Mikhail Petrov
>Priority: Major
>  Labels: IEP-35, await
> Fix For: 2.9
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> For now, thread pool metrics register after all {{GridComponent}} starts.
> But there are specific scenarios when some component blocks {{onKernalStart}} 
> execution for a long time. {{GridCacheProcessor}} can be taken as an example.
> This leads to the situation when some metric info is lost.
> Seems, we can register thread pool metrics right after only **required** 
> components are started and don't wait for all components.
> {code:java}
> // Callbacks.
> for (GridComponent comp : ctx) {
> comp.onKernalStart(active);
> }
> // Start plugins.
> for (PluginProvider provider : ctx.plugins().allProviders())
> provider.onIgniteStart();
> ctx.metric().registerThreadPools(utilityCachePool, execSvc, 
> svcExecSvc, sysExecSvc, stripedExecSvc,
> p2pExecSvc, mgmtExecSvc, igfsExecSvc, dataStreamExecSvc, 
> restExecSvc, affExecSvc, idxExecSvc,
> callbackExecSvc, qryExecSvc, schemaExecSvc, rebalanceExecSvc, 
> rebalanceStripedExecSvc, customExecSvcs);
> // Register MBeans.
> mBeansMgr.registerAllMBeans(utilityCachePool, execSvc, 
> svcExecSvc, sysExecSvc, stripedExecSvc, p2pExecSvc,
> mgmtExecSvc, igfsExecSvc, dataStreamExecSvc, restExecSvc, 
> affExecSvc, idxExecSvc, callbackExecSvc,
> qryExecSvc, schemaExecSvc, rebalanceExecSvc, 
> rebalanceStripedExecSvc, customExecSvcs, ctx.workersRegistry());
> {code}
> {code:java}
> public class GridCacheProcessor {
> @Override public void onKernalStart(boolean active) throws 
> IgniteCheckedException {
> //.
> final List syncFuts = new 
> ArrayList<>(caches.size());
> sharedCtx.forAllCaches(new CIX1() {
> @Override public void applyx(GridCacheContext cctx) {
> CacheConfiguration cfg = cctx.config();
> if (cctx.affinityNode() &&
> cfg.getRebalanceMode() == SYNC &&
> startTopVer.equals(cctx.startTopologyVersion())) {
> CacheMode cacheMode = cfg.getCacheMode();
> if (cacheMode == REPLICATED || (cacheMode == PARTITIONED 
> && cfg.getRebalanceDelay() >= 0))
> // Need to wait outside to avoid a deadlock
> syncFuts.add(cctx.preloader().syncFuture());
> }
> }
> });
> for (int i = 0, size = syncFuts.size(); i < size; i++)
> syncFuts.get(i).get();
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12808) Allow create tables for existing caches

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12808:
---
Fix Version/s: 2.9

> Allow create tables for existing caches
> ---
>
> Key: IGNITE-12808
> URL: https://issues.apache.org/jira/browse/IGNITE-12808
> Project: Ignite
>  Issue Type: New Feature
>  Components: sql
>Reporter: Mikhail Cherkasov
>Assignee: Ivan Daschinskiy
>Priority: Major
> Fix For: 2.9
>
>  Time Spent: 12.5h
>  Remaining Estimate: 0h
>
> If you have a big cache with a lot of data and you need to index it, right 
> now you have to destroy cache and create a new one to index your data.  Or 
> create a new cache with a table and reload it to  data to the new cache which 
> definitely is time-consuming and super inconvenient.
> I believe we can allow users to create tables for existing caches.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-11073) Persistence cache snapshot

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-11073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-11073:
---
Labels: iep-28 iep-43 important  (was: iep-28 iep-43)

> Persistence cache snapshot
> --
>
> Key: IGNITE-11073
> URL: https://issues.apache.org/jira/browse/IGNITE-11073
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
>  Labels: iep-28, iep-43, important
> Fix For: 2.9
>
>  Time Spent: 52h 20m
>  Remaining Estimate: 0h
>
> *Snapshot requirements*
> # Users must have the ability to create a snapshot of persisted user data 
> (in-memory is out of the scope).
> # Users must have the ability to create a snapshot from the cluster under the 
> load without cluster deactivation.
> # The snapshot process must not block for a long time any of the user 
> transactions (short-time blocks are acceptable).
> # The snapshot process must allow creating a data snapshot on each node and 
> transfer it to any of the remote nodes for internal cluster needs.
> # The created snapshot at the cluster-level must be fully consistent from 
> cluster-wide terms, there should not be any incomplete transactions inside.
> # The snapshot of each node must be consistent – cache partitions, binary 
> meta, etc. must not have unnecessary changes.
> *The following API must be available:*
> # [public] Java API
> # [public] JMX MBean
> # [internal] File Transmission



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-13033) Java thin client: Service invocation

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-13033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-13033:
---
Labels: iep-46 important  (was: iep-46)

> Java thin client: Service invocation
> 
>
> Key: IGNITE-13033
> URL: https://issues.apache.org/jira/browse/IGNITE-13033
> Project: Ignite
>  Issue Type: New Feature
>  Components: thin client
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
>  Labels: iep-46, important
> Fix For: 2.9
>
>  Time Spent: 5h 50m
>  Remaining Estimate: 0h
>
> Provide an API to invoke Ignite Services from java thin client.
> Protocol changes and all implementation details described in IEP-46.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12835) Thin client: compute support

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12835:
---
Labels: iep-42 important  (was: iep-42)

> Thin client: compute support
> 
>
> Key: IGNITE-12835
> URL: https://issues.apache.org/jira/browse/IGNITE-12835
> Project: Ignite
>  Issue Type: New Feature
>  Components: thin client
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
>  Labels: iep-42, important
> Fix For: 2.9
>
>  Time Spent: 3h
>  Remaining Estimate: 0h
>
> Add compute grid functionality to thin clients.
> As a first step execution of task by task name should be added.
> Implement a compute facade in java thin client.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12385) .NET Thin Client: introduce ClusterGroup API

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12385:
---
Labels: important  (was: )

> .NET Thin Client: introduce ClusterGroup API
> 
>
> Key: IGNITE-12385
> URL: https://issues.apache.org/jira/browse/IGNITE-12385
> Project: Ignite
>  Issue Type: New Feature
>  Components: thin client
>Affects Versions: 2.7.6
>Reporter: Alexandr Shapkin
>Assignee: Alexandr Shapkin
>Priority: Major
>  Labels: important
> Fix For: 2.9
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> We need to implement IClusterGroup API for thin clients.
> Let's start with following methods:
>  * IClusterGroup.ForDotNet()
>  * IClusterGroup.ForAttributes()
>  * IClusterGroup.GetNodes()



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12932) Thin client cluster discovery

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12932:
---
Labels: important  (was: )

> Thin client cluster discovery
> -
>
> Key: IGNITE-12932
> URL: https://issues.apache.org/jira/browse/IGNITE-12932
> Project: Ignite
>  Issue Type: New Feature
>  Components: thin client
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: important
> Fix For: 2.9
>
>   Original Estimate: 336h
>  Time Spent: 20m
>  Remaining Estimate: 3h 50m
>
> Thin clients should be able to discover all server nodes automatically when 
> connected to any of them, and maintain an up to date list of servers at all 
> times.
> See 
> [IEP-44|https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery]
>  for more details.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12186) TDE - Phase-2. Master key rotation.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12186:
---
Labels: IEP-18 important  (was: IEP-18)

> TDE - Phase-2. Master key rotation.
> ---
>
> Key: IGNITE-12186
> URL: https://issues.apache.org/jira/browse/IGNITE-12186
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
>  Labels: IEP-18, important
> Fix For: 2.9
>
>  Time Spent: 19.5h
>  Remaining Estimate: 0h
>
> Need to implement master key rotation process. Master key(MK) rotation 
> required in case of it compromising or at the end of crypto period(key 
> validity period). 
> [Design 
> (cwiki).|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652381]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12810) [IEP-39] Management API to cancel compute tasks.

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-12810:
---
Labels: IEP-39 important  (was: IEP-39)

> [IEP-39] Management API to cancel compute tasks.
> 
>
> Key: IGNITE-12810
> URL: https://issues.apache.org/jira/browse/IGNITE-12810
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-39, important
> Fix For: 2.9
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Ignite provides many API to deploy and execute user-provided code on the 
> server nodes inside the same JVM as the Ignite process runs.
> Ignite has many APIs that allocate many resources on the server nodes, also. 
> In case of some buggy code that consumes many system resources(CPU, RAM, 
> flood network) or heavy query the whole cluster can become unstable.
> We should provide to the cluster administrator the ability to stop any user 
> deployed task.
> JMX beans to cancel listed tasks should be introduced:
> * Compute task
> In the scope of IEP-35 System view API introduced.
> A new API should use the same identifier that is used in corresponding System 
> View.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-11410) Sandbox for user-defined code

2020-06-26 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-11410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-11410:
---
Labels: iep-38 important  (was: iep-38)

> Sandbox for user-defined code
> -
>
> Key: IGNITE-11410
> URL: https://issues.apache.org/jira/browse/IGNITE-11410
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Anton Vinogradov
>Assignee: Denis Garus
>Priority: Critical
>  Labels: iep-38, important
> Fix For: 2.9
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> We should provide a restricted environment (sandbox) in which to run 
> user-defined code securely. To get it done, we would use the java sandbox 
> model.
>  The java sandbox model allows restricting access from user-defined code to 
> the system resources or security-sensitive feature of java, for example, 
> reflection.
> The user-defined code contains:
>  - StreamReceiver for DataStreamer:
>  - EntryProcessor;
>  - ComputeJob;
>  - filter and transformer for ScanQuery.
> The user-defined code will get permissions from GridSecuerityProcessor 
> (security plugin).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12274) [ML] DecisionTree works incorrectly if maxDeep > amount of features

2020-06-26 Thread Alexey Zinoviev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Zinoviev updated IGNITE-12274:
-
Affects Version/s: (was: 2.8)

> [ML] DecisionTree works incorrectly if maxDeep > amount of features
> ---
>
> Key: IGNITE-12274
> URL: https://issues.apache.org/jira/browse/IGNITE-12274
> Project: Ignite
>  Issue Type: Bug
>  Components: ml
>Reporter: Alexey Zinoviev
>Assignee: Alexey Zinoviev
>Priority: Blocker
> Fix For: 2.10
>
>
> We have a problem in two places: 
> null nodes could be created here *MeanDecisionTreeLeafBuilder.createLeafNode* 
> method in the row *return aa != null ? new DecisionTreeLeafNode(aa[0]) : 
> null;*
> Probably, this situation is arising then the amount of features is smaller 
> than maxDeep



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12274) [ML] DecisionTree works incorrectly if maxDeep > amount of features

2020-06-26 Thread Alexey Zinoviev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Zinoviev updated IGNITE-12274:
-
Priority: Major  (was: Blocker)

> [ML] DecisionTree works incorrectly if maxDeep > amount of features
> ---
>
> Key: IGNITE-12274
> URL: https://issues.apache.org/jira/browse/IGNITE-12274
> Project: Ignite
>  Issue Type: Bug
>  Components: ml
>Reporter: Alexey Zinoviev
>Assignee: Alexey Zinoviev
>Priority: Major
> Fix For: 2.10
>
>
> We have a problem in two places: 
> null nodes could be created here *MeanDecisionTreeLeafBuilder.createLeafNode* 
> method in the row *return aa != null ? new DecisionTreeLeafNode(aa[0]) : 
> null;*
> Probably, this situation is arising then the amount of features is smaller 
> than maxDeep



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12274) [ML] DecisionTree works incorrectly if maxDeep > amount of features

2020-06-26 Thread Alexey Zinoviev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Zinoviev updated IGNITE-12274:
-
Fix Version/s: (was: 2.9)
   2.10

> [ML] DecisionTree works incorrectly if maxDeep > amount of features
> ---
>
> Key: IGNITE-12274
> URL: https://issues.apache.org/jira/browse/IGNITE-12274
> Project: Ignite
>  Issue Type: Bug
>  Components: ml
>Affects Versions: 2.8
>Reporter: Alexey Zinoviev
>Assignee: Alexey Zinoviev
>Priority: Blocker
> Fix For: 2.10
>
>
> We have a problem in two places: 
> null nodes could be created here *MeanDecisionTreeLeafBuilder.createLeafNode* 
> method in the row *return aa != null ? new DecisionTreeLeafNode(aa[0]) : 
> null;*
> Probably, this situation is arising then the amount of features is smaller 
> than maxDeep



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


<    1   2   3   >