date:20200818



[ 
https://issues.apache.org/jira/browse/IGNITE-13349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179490#comment-17179490
 ] 

Ignite TC Bot commented on IGNITE-13349:


{panel:title=Branch: [pull/8145/head] Base: [master] : No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/8145/head] Base: [master] : No new tests 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5549807&buildTypeId=IgniteTests24Java8_RunAll]

> Migrate TcpDiscoveryStatistics to new metrics framework
> ---
>
> Key: IGNITE-13349
> URL: https://issues.apache.org/jira/browse/IGNITE-13349
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.10
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> TcpDiscoveryStatistics should be migrated to the new metrics framework.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (IGNITE-13040) Remove unused parameter from TcpDiscoverySpi.writeToSocket()

2020-08-18 Thread Vladimir Steshin (Jira)



[ 
https://issues.apache.org/jira/browse/IGNITE-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179492#comment-17179492
 ] 

Vladimir Steshin commented on IGNITE-13040:
---

[~Kurinov], you still need visa for this ticket. All the tests passed?

> Remove unused parameter from TcpDiscoverySpi.writeToSocket()
> 
>
> Key: IGNITE-13040
> URL: https://issues.apache.org/jira/browse/IGNITE-13040
> Project: Ignite
>  Issue Type: Improvement
> Environment:  
>Reporter: Vladimir Steshin
>Assignee: Aleksey Kurinov
>Priority: Trivial
>  Labels: newbie
>
> Unused parameter
> {code:java}
> TcpDiscoveryAbstractMessage msg{code}
> should be removed from
> {code:java}
> TcpDiscoverySpi.writeToSocket(Socket sock, TcpDiscoveryAbstractMessage msg, 
> byte[] data, long timeout){code}
> This method seems to send raw data, not a message.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (IGNITE-13224) Add metrics to OpenCensus exporter

2020-08-18 Thread Nikolay Izhikov (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-13224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikolay Izhikov resolved IGNITE-13224.
--
Resolution: Implemented

Covered by the IGNITE-13347, IGNITE-13349, IGNITE-13354 

> Add metrics to OpenCensus exporter
> --
>
> Key: IGNITE-13224
> URL: https://issues.apache.org/jira/browse/IGNITE-13224
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.8, 2.8.1
>Reporter: Dmitry Frolov
>Priority: Major
>  Labels: IEP-35
>
> The metrics below are available only via JMX now. The ticket is a suggestion 
> to add these  metrics to Opencencus as well:
> *ClusterMetricsMXBeanImpl:*
> ActiveBaselineNodes
> TopologyVersion
> TotalBaselineNodes
> TotalClientNodes
> TotalNodes
> TotalServerNodes
>  
> *Ignite clients count:*
> Connections
>  
> *TcpDiscoverySpi:*
> MessageWorkerQueueSize
>  
> *ClusterLocalNodeMetricsMXBeanImpl:*
> HeapMemoryTotal
>  
> *Data Region Metrics:*
> MaxSize
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (IGNITE-13361) SQL Select query hangs during cursor iteration.

2020-08-18 Thread Mikhail Petrov (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-13361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Petrov updated IGNITE-13361:

Description: 
The following test hangs intermittently (once for 4-5 runs) on my laptop 
(Ubuntu 20.04, i7-8565u, 16gb RAM, JDK 1.8.0_251). The cursor iteration 
randomly hangs on the stage of waiting for the next page from the remote node.
{code:java}
/** */
public static final int NODES_CNT = 2;

/** */
public static final int TABLE_POPULATION = 2000;

/** */
public static final int SELECT_RANGE = 1000;

/** */
public static final int QRY_PAGE_SIZE = 5;

/** */
@Test
public void test() throws Exception {
for (int i = 0; i < NODES_CNT; i++)
startGrid(i, false);

IgniteEx cli = startGrid(NODES_CNT, true);

GridQueryProcessor qryProc = cli.context().query();

qryProc.querySqlFields(
new SqlFieldsQuery("CREATE TABLE test_table (id LONG PRIMARY KEY, 
val LONG)"), false);

qryProc.querySqlFields(new SqlFieldsQuery("CREATE INDEX val_idx ON 
test_table (val)"), false);

for (long l = 0; l < TABLE_POPULATION; ++l) {
qryProc.querySqlFields(
new SqlFieldsQuery("INSERT INTO test_table (id, val) VALUES (?, 
?)").setArgs(l, l),
true
);
}

for (int i = 0; i < 1 ; i++) {
long lowId = ThreadLocalRandom.current().nextLong(TABLE_POPULATION 
- SELECT_RANGE);

long highId = lowId + SELECT_RANGE;

try (
FieldsQueryCursor> cursor = cli
.context().query().querySqlFields(
new SqlFieldsQuery("SELECT id, val FROM test_table 
WHERE id BETWEEN ? and ?")
.setArgs(lowId, highId)
.setPageSize(QRY_PAGE_SIZE),
false
)
) {
cursor.iterator().forEachRemaining(val -> {});
}
}
}

/** */
private IgniteEx startGrid(int idx, boolean clientMode) throws Exception {
return (IgniteEx) Ignition.start(new IgniteConfiguration()
.setIgniteInstanceName("node-" + idx)
.setGridLogger(new 
Log4JLogger("modules/core/src/test/config/log4j-test.xml"))
.setClientMode(clientMode));
}
{code}

UPD It seems that IGNITE-12845 is responsible for the behavior described above. 
Commit which is related to this ticket is the first since which the code 
mentioned above started to hang. 
Cursor iteration hangs due to GridQueryNextPageRequest in some cases are not 
sent correctly from the client node.

  was:
The following test hangs intermittently (once for 4-5 runs) on my laptop 
(Ubuntu 20.04, i7-8565u, 16gb RAM). The cursor iteration randomly hangs on the 
stage of waiting for the next page from the remote node.
{code:java}
/** */
public static final int NODES_CNT = 2;

/** */
public static final int TABLE_POPULATION = 2000;

/** */
public static final int SELECT_RANGE = 1000;

/** */
public static final int QRY_PAGE_SIZE = 5;

/** */
@Test
public void test() throws Exception {
for (int i = 0; i < NODES_CNT; i++)
startGrid(i, false);

IgniteEx cli = startGrid(NODES_CNT, true);

GridQueryProcessor qryProc = cli.context().query();

qryProc.querySqlFields(
new SqlFieldsQuery("CREATE TABLE test_table (id LONG PRIMARY KEY, 
val LONG)"), false);

qryProc.querySqlFields(new SqlFieldsQuery("CREATE INDEX val_idx ON 
test_table (val)"), false);

for (long l = 0; l < TABLE_POPULATION; ++l) {
qryProc.querySqlFields(
new SqlFieldsQuery("INSERT INTO test_table (id, val) VALUES (?, 
?)").setArgs(l, l),
true
);
}

for (int i = 0; i < 1 ; i++) {
long lowId = ThreadLocalRandom.current().nextLong(TABLE_POPULATION 
- SELECT_RANGE);

long highId = lowId + SELECT_RANGE;

try (
FieldsQueryCursor> cursor = cli
.context().query().querySqlFields(
new SqlFieldsQuery("SELECT id, val FROM test_table 
WHERE id BETWEEN ? and ?")
.setArgs(lowId, highId)
.setPageSize(QRY_PAGE_SIZE),
false
)
) {
cursor.iterator().forEachRemaining(val -> {});
}
}
}

/** */
private IgniteEx startGrid(int idx, boolean clientMode) throws Exception {
return (IgniteEx) Ignition.start(new IgniteConfiguration()
.setIgniteInstanceName("node-" + idx)
.setGridLogger(new 
Log4JLogger("modules/core/src/test/config/log4j-test.xml"))
.setClientMode(clientMode));

[jira] [Commented] (IGNITE-13280) Improper index usage, fields enumeration not used with pk index creation.

2020-08-18 Thread Taras Ledkov (Jira)



[ 
https://issues.apache.org/jira/browse/IGNITE-13280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179508#comment-17179508
 ] 

Taras Ledkov commented on IGNITE-13280:
---

[~zstan], I see no reason not to use one cost function for all indexes. This 
can save you from making mistakes when fixing the cost function later. If you 
know a reason for using different cost functions please specify them.

> Improper index usage, fields enumeration not used with pk index creation.
> -
>
> Key: IGNITE-13280
> URL: https://issues.apache.org/jira/browse/IGNITE-13280
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.8.1
>Reporter: Stanilovsky Evgeny
>Assignee: Stanilovsky Evgeny
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> For example:
> {code:java}
> CREATE TABLE PUBLIC.TEST_TABLE (FIRST_NAME VARCHAR, LAST_NAME VARCHAR, 
> ADDRESS VARCHAR, LANG VARCHAR,  CONSTRAINT PK_PERSON PRIMARY KEY (FIRST_NAME, 
> LAST_NAME));
> CREATE INDEX "idx2" ON PUBLIC.TEST_TABLE (LANG, ADDRESS);
> {code}
> and further explain:
> {code:java}
> SELECT
> "__Z0"."FIRST_NAME" AS "__C0_0",
> "__Z0"."LAST_NAME" AS "__C0_1",
> "__Z0"."ADDRESS" AS "__C0_2",
> "__Z0"."LANG" AS "__C0_3"
> FROM "PUBLIC"."TEST_TABLE" "__Z0"
> /* PUBLIC.IDX2: ADDRESS > 0 */
> WHERE "__Z0"."ADDRESS" > 0
> {code}
> this is erroneous  to use "idx2" here, because first index field LANG not 
> equals to predicate ADDRESS.
> Looks like this bug brings this ticket [1]
> [1] https://issues.apache.org/jira/browse/IGNITE-10217



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (IGNITE-13280) Improper index usage, fields enumeration not used with pk index creation.

2020-08-18 Thread Taras Ledkov (Jira)



[ 
https://issues.apache.org/jira/browse/IGNITE-13280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179519#comment-17179519
 ] 

Taras Ledkov commented on IGNITE-13280:
---

[~zstan] test to reproduce (for class {{BasicIndexTest}}):
{code}
/**
 *  Checks index usage for full coverage.
 */
@Test
public void testConditionsWithoutIndexes2() throws Exception {
inlineSize = 10;

srvLog = new ListeningTestLogger(false, log);

IgniteEx ig0 = startGrid(0);

GridQueryProcessor qryProc = ig0.context().query();

populateTable(qryProc, TEST_TBL_NAME, 1, "ID", "VAL0", "VAL1");

String sqlIdx = String.format("create index \"idx1\" on %s(VAL0, 
VAL1)", TEST_TBL_NAME);

qryProc.querySqlFields(new SqlFieldsQuery(sqlIdx), true).getAll();

assertFalse(checkIdxUsed(qryProc, null, TEST_TBL_NAME, "VAL1"));

assertTrue(checkIdxUsed(qryProc, "idx1", TEST_TBL_NAME, "VAL0"));
}


{code}

> Improper index usage, fields enumeration not used with pk index creation.
> -
>
> Key: IGNITE-13280
> URL: https://issues.apache.org/jira/browse/IGNITE-13280
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.8.1
>Reporter: Stanilovsky Evgeny
>Assignee: Stanilovsky Evgeny
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> For example:
> {code:java}
> CREATE TABLE PUBLIC.TEST_TABLE (FIRST_NAME VARCHAR, LAST_NAME VARCHAR, 
> ADDRESS VARCHAR, LANG VARCHAR,  CONSTRAINT PK_PERSON PRIMARY KEY (FIRST_NAME, 
> LAST_NAME));
> CREATE INDEX "idx2" ON PUBLIC.TEST_TABLE (LANG, ADDRESS);
> {code}
> and further explain:
> {code:java}
> SELECT
> "__Z0"."FIRST_NAME" AS "__C0_0",
> "__Z0"."LAST_NAME" AS "__C0_1",
> "__Z0"."ADDRESS" AS "__C0_2",
> "__Z0"."LANG" AS "__C0_3"
> FROM "PUBLIC"."TEST_TABLE" "__Z0"
> /* PUBLIC.IDX2: ADDRESS > 0 */
> WHERE "__Z0"."ADDRESS" > 0
> {code}
> this is erroneous  to use "idx2" here, because first index field LANG not 
> equals to predicate ADDRESS.
> Looks like this bug brings this ticket [1]
> [1] https://issues.apache.org/jira/browse/IGNITE-10217



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13370) Missed README.txt in ignite-control-utility module

2020-08-18 Thread Kirill Tkalenko (Jira)

Kirill Tkalenko created IGNITE-13370:


 Summary: Missed README.txt in ignite-control-utility module
 Key: IGNITE-13370
 URL: https://issues.apache.org/jira/browse/IGNITE-13370
 Project: Ignite
  Issue Type: Improvement
  Components: control.sh
Reporter: Kirill Tkalenko
Assignee: Kirill Tkalenko
 Fix For: 2.10


Missed README.txt in ignite-control-utility module



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (IGNITE-13253) Advanced heuristics for historical rebalance



[ 
https://issues.apache.org/jira/browse/IGNITE-13253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179550#comment-17179550
 ] 

Ivan Rakov commented on IGNITE-13253:
-

[~v.pyatkov] Please take a look.

> Advanced heuristics for historical rebalance
> 
>
> Key: IGNITE-13253
> URL: https://issues.apache.org/jira/browse/IGNITE-13253
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vladislav Pyatkov
>Assignee: Ivan Rakov
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Before, cluster detects partitions that have not to rebalance by history, by 
> them size. This threshold might be set through a system property 
> IGNITE_PDS_WAL_REBALANCE_THRESHOLD. But it is not fair deciding which 
> partitions will be rebalanced by WAL only by them size. WAL can have much 
> more records than size of a partition (many update by one key) and that 
> rebalance required more data than full transferring by network.
> Need to implement a heuristic, that might to estimate data size.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (IGNITE-13253) Advanced heuristics for historical rebalance



[ 
https://issues.apache.org/jira/browse/IGNITE-13253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179549#comment-17179549
 ] 

Ignite TC Bot commented on IGNITE-13253:


{panel:title=Branch: [pull/8160/head] Base: [master] : No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/8160/head] Base: [master] : New Tests 
(4)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}MVCC PDS 2{color} [[tests 
2|https://ci.ignite.apache.org/viewLog.html?buildId=5547062]]
* {color:#013220}IgnitePdsMvccTestSuite2: 
HistoricalRebalanceHeuristicsTest.testPutRemoveReorderingConsistency[historical 
= false] - PASSED{color}
* {color:#013220}IgnitePdsMvccTestSuite2: 
HistoricalRebalanceHeuristicsTest.testPutRemoveReorderingConsistency[historical 
= true] - PASSED{color}

{color:#8b}PDS 2{color} [[tests 
2|https://ci.ignite.apache.org/viewLog.html?buildId=5547035]]
* {color:#013220}IgnitePdsTestSuite2: 
HistoricalRebalanceHeuristicsTest.testPutRemoveReorderingConsistency[historical 
= false] - PASSED{color}
* {color:#013220}IgnitePdsTestSuite2: 
HistoricalRebalanceHeuristicsTest.testPutRemoveReorderingConsistency[historical 
= true] - PASSED{color}

{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5547073&buildTypeId=IgniteTests24Java8_RunAll]

> Advanced heuristics for historical rebalance
> 
>
> Key: IGNITE-13253
> URL: https://issues.apache.org/jira/browse/IGNITE-13253
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vladislav Pyatkov
>Assignee: Ivan Rakov
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Before, cluster detects partitions that have not to rebalance by history, by 
> them size. This threshold might be set through a system property 
> IGNITE_PDS_WAL_REBALANCE_THRESHOLD. But it is not fair deciding which 
> partitions will be rebalanced by WAL only by them size. WAL can have much 
> more records than size of a partition (many update by one key) and that 
> rebalance required more data than full transferring by network.
> Need to implement a heuristic, that might to estimate data size.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (IGNITE-13253) Advanced heuristics for historical rebalance



 [ 
https://issues.apache.org/jira/browse/IGNITE-13253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Rakov updated IGNITE-13253:

Reviewer: Vladislav Pyatkov

> Advanced heuristics for historical rebalance
> 
>
> Key: IGNITE-13253
> URL: https://issues.apache.org/jira/browse/IGNITE-13253
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vladislav Pyatkov
>Assignee: Ivan Rakov
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Before, cluster detects partitions that have not to rebalance by history, by 
> them size. This threshold might be set through a system property 
> IGNITE_PDS_WAL_REBALANCE_THRESHOLD. But it is not fair deciding which 
> partitions will be rebalanced by WAL only by them size. WAL can have much 
> more records than size of a partition (many update by one key) and that 
> rebalance required more data than full transferring by network.
> Need to implement a heuristic, that might to estimate data size.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (IGNITE-10789) CacheInterceptor on server node get BinaryObject if put was invoked by ClientCache.

2020-08-18 Thread Stanilovsky Evgeny (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-10789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stanilovsky Evgeny reassigned IGNITE-10789:
---

Assignee: Stanilovsky Evgeny

> CacheInterceptor on server node get BinaryObject if put was invoked by 
> ClientCache.
> ---
>
> Key: IGNITE-10789
> URL: https://issues.apache.org/jira/browse/IGNITE-10789
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 2.7
>Reporter: Sergey Antonov
>Assignee: Stanilovsky Evgeny
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Cache interceptor on server node expects deserialized object, but gets 
> BinaryObject in case of put was from thin client.
> The exception is following:
> {noformat}
> [2018-12-21 
> 17:25:08,213][ERROR][client-connector-#53%cache.Test20%][ClientListenerNioListener]
>  Failed to process client request 
> [req=o.a.i.i.processors.platform.client.cache.ClientCachePutRequest@72009659]
> org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys 
> (retry update if possible).: [key]
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1313)
>   at 
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:1754)
>   at 
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1107)
>   at 
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:820)
>   at 
> org.apache.ignite.internal.processors.platform.client.cache.ClientCachePutRequest.process(ClientCachePutRequest.java:40)
>   at 
> org.apache.ignite.internal.processors.platform.client.ClientRequestHandler.handle(ClientRequestHandler.java:52)
>   at 
> org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:174)
>   at 
> org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:48)
>   at 
> org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279)
>   at 
> org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
>   at 
> org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:97)
>   at 
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
>   at 
> org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:70)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: class 
> org.apache.ignite.internal.processors.cache.CachePartialUpdateCheckedException:
>  Failed to update keys (retry update if possible).: [key]
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.onPrimaryError(GridNearAtomicAbstractUpdateFuture.java:397)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.onPrimaryResponse(GridNearAtomicSingleUpdateFuture.java:252)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture$1.apply(GridNearAtomicAbstractUpdateFuture.java:304)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture$1.apply(GridNearAtomicAbstractUpdateFuture.java:301)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture.map(GridDhtAtomicAbstractUpdateFuture.java:394)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1942)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1671)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:300)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:482)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:442)
>   at 
> org.apache.ignite.internal.proces

[jira] [Commented] (IGNITE-13253) Advanced heuristics for historical rebalance

2020-08-18 Thread Vladislav Pyatkov (Jira)



[ 
https://issues.apache.org/jira/browse/IGNITE-13253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179563#comment-17179563
 ] 

Vladislav Pyatkov commented on IGNITE-13253:


LGTM

> Advanced heuristics for historical rebalance
> 
>
> Key: IGNITE-13253
> URL: https://issues.apache.org/jira/browse/IGNITE-13253
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vladislav Pyatkov
>Assignee: Ivan Rakov
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Before, cluster detects partitions that have not to rebalance by history, by 
> them size. This threshold might be set through a system property 
> IGNITE_PDS_WAL_REBALANCE_THRESHOLD. But it is not fair deciding which 
> partitions will be rebalanced by WAL only by them size. WAL can have much 
> more records than size of a partition (many update by one key) and that 
> rebalance required more data than full transferring by network.
> Need to implement a heuristic, that might to estimate data size.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (IGNITE-12754) .NET: Thin Client: Service invocation



[ 
https://issues.apache.org/jira/browse/IGNITE-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179566#comment-17179566
 ] 

Ignite TC Bot commented on IGNITE-12754:


{panel:title=Branch: [pull/8151/head] Base: [master] : Possible Blockers 
(8)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}MVCC Cache 7{color} [[tests 0 TIMEOUT , Exit Code 
|https://ci.ignite.apache.org/viewLog.html?buildId=5550286]]

{color:#d04437}Cache (Restarts) 1{color} [[tests 0 TIMEOUT , Exit Code 
|https://ci.ignite.apache.org/viewLog.html?buildId=5550242]]

{color:#d04437}Cache 1{color} [[tests 0 Exit Code 
|https://ci.ignite.apache.org/viewLog.html?buildId=5550245]]

{color:#d04437}JDBC Driver{color} [[tests 
1|https://ci.ignite.apache.org/viewLog.html?buildId=5550206]]
* IgniteJdbcDriverTestSuite: 
JdbcThinJdbcToCacheDataTypesCoverageTest.testBigIntDataType[atomicityMode=ATOMIC,
 cacheMode=PARTITIONED, ttlFactory=null, backups=0, evictionFactory=null, 
onheapCacheEnabled=false, writeSyncMode=FULL_SYNC, persistenceEnabled=false] - 
Test has low fail rate in base branch 0,0% and is not flaky

{color:#d04437}Cache 9{color} [[tests 0 TIMEOUT , Exit Code 
|https://ci.ignite.apache.org/viewLog.html?buildId=5550253]]

{color:#d04437}ZooKeeper (Discovery) 2{color} [[tests 
1|https://ci.ignite.apache.org/viewLog.html?buildId=5550222]]
* ZookeeperDiscoverySpiTestSuite2: 
IgniteClientReconnectCacheTest.testReconnectExchangeInProgress - Test has low 
fail rate in base branch 0,0% and is not flaky

{color:#d04437}Interceptor Cache (Full API Config Variations / Basic)*{color} 
[[tests 1|https://ci.ignite.apache.org/viewLog.html?buildId=5550203]]
* InterceptorCacheConfigVariationsFullApiTestSuite: 
InterceptorCacheConfigVariationsFullApiTest_5.testSize - Test has low fail rate 
in base branch 0,0% and is not flaky

{color:#d04437}Cache (Full API){color} [[tests 
1|https://ci.ignite.apache.org/viewLog.html?buildId=5550241]]
* IgniteCacheFullApiSelfTestSuite: 
GridCachePartitionedOnheapMultiNodeFullApiSelfTest.testWithSkipStoreTx - Test 
has low fail rate in base branch 0,0% and is not flaky

{panel}
{panel:title=Branch: [pull/8151/head] Base: [master] : New Tests 
(35)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Platform .NET (Core Linux){color} [[tests 
18|https://ci.ignite.apache.org/viewLog.html?buildId=5550267]]
* {color:#013220}dll: ServicesClientTest.TestJavaServiceCall - PASSED{color}
* {color:#013220}dll: 
ServicesClientTest.TestExceptionInServiceIsPropagatedToClient - PASSED{color}
* {color:#013220}dll: ServicesClientTest.TestEmptyClusterGroupThrowsError - 
PASSED{color}
* {color:#013220}dll: 
ServicesClientTest.TestClusterGroupWithoutMatchingServiceNodesThrowsError - 
PASSED{color}
* {color:#013220}dll: 
ServicesClientTest.TestClientKeepBinaryReturnsServiceInvocationResultInBinaryMode
 - PASSED{color}
* {color:#013220}dll: ServicesClientTest.TestBasicServiceCall - PASSED{color}
* {color:#013220}dll: 
ServicesClientTest.TestServerKeepBinaryPassesServerSideArgumentsInBinaryMode - 
PASSED{color}
* {color:#013220}dll: 
ServicesClientTest.TestServerAndClientKeepBinaryPassesBinaryObjectsOnServerAndClient
 - PASSED{color}
* {color:#013220}dll: ServicesClientTest.TestPropertyCalls - PASSED{color}
* {color:#013220}dll: ServicesClientTest.TestOverloadResolution - PASSED{color}
* {color:#013220}dll: ServicesClientTest.TestObjectMethodCall - PASSED{color}
... and 7 new tests

{color:#8b}Platform .NET{color} [[tests 
17|https://ci.ignite.apache.org/viewLog.html?buildId=5550266]]
* {color:#013220}exe: 
ServicesClientTest.TestServicesWithCustomClusterGroupInvokeOnSpecifiedNodes - 
PASSED{color}
* {color:#013220}exe: 
ServicesClientTest.TestServerKeepBinaryPassesServerSideArgumentsInBinaryMode - 
PASSED{color}
* {color:#013220}exe: 
ServicesClientTest.TestServerAndClientKeepBinaryPassesBinaryObjectsOnServerAndClient
 - PASSED{color}
* {color:#013220}exe: ServicesClientTest.TestPropertyCalls - PASSED{color}
* {color:#013220}exe: ServicesClientTest.TestOverloadResolution - PASSED{color}
* {color:#013220}exe: ServicesClientTest.TestObjectMethodCall - PASSED{color}
* {color:#013220}exe: ServicesClientTest.TestObjectArrayBinary - PASSED{color}
* {color:#013220}exe: ServicesClientTest.TestVoidMethodCall - PASSED{color}
* {color:#013220}exe: 
ServicesClientTest.TestClientKeepBinaryReturnsServiceInvocationResultInBinaryMode
 - PASSED{color}
* {color:#013220}exe: ServicesClientTest.TestBasicServiceCall - PASSED{color}
* {color:#013220}exe: ServicesClientTest.TestAllArgumentTypes - PASSED{color}
... and 6 new tests

{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5550301&buildTypeId=IgniteTests24Java8_RunAll]

> .NET: Thin Client: Service invocation
> -
>
> Key: IGNITE-12754
> URL: https://issues.a

[jira] [Commented] (IGNITE-13183) Query timeout redesign

2020-08-18 Thread Konstantin Orlov (Jira)



[ 
https://issues.apache.org/jira/browse/IGNITE-13183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179613#comment-17179613
 ] 

Konstantin Orlov commented on IGNITE-13183:
---

[~tledkov-gridgain], LGTM!

> Query timeout redesign
> --
>
> Key: IGNITE-13183
> URL: https://issues.apache.org/jira/browse/IGNITE-13183
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Affects Versions: 2.8.1
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> *Motivation:*
> Now the query timeout is set up for each node separately by the node 
> configuration. This property isn't propagated for the all nodes of cluster. 
> Also user cannot change / disable query timeout without restart all nodes of 
> the cluster.
> *Proposal fix:*
> - Adds the default query timeout property to the 
> {{DistributedSqlConfiguration}}. Use distributed metastore to store and 
> manage the property.
> - Deprecates the {{SqlConfiguration#defaultQueryTimeout}} property and use it 
> to set up initial value of new property 
> {{DistributedSqlConfiguration#defaultQueryTimeout}}
> - Adds info about explicit query timeout to {{GridH2QueryRequest}} (boolean 
> flag {{explicitTimeout=false}} by default). This is necessary so that the 
> default timeout may be used for queries from old nodes.
> - When query timeout is set to zero by old node ({{explicitTimeout=false}}) 
> we assume this is the default value and use default timeout for this queries.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (IGNITE-13253) Advanced heuristics for historical rebalance



 [ 
https://issues.apache.org/jira/browse/IGNITE-13253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Rakov updated IGNITE-13253:

Release Note: Introduced heuristic for using historical rebalancing 
automatically if it's more efficient. Property 
-DIGNITE_PDS_WAL_REBALANCE_THRESHOLD is still present, but its value changed to 
500. Heuristic is applied only for larger partitions.

> Advanced heuristics for historical rebalance
> 
>
> Key: IGNITE-13253
> URL: https://issues.apache.org/jira/browse/IGNITE-13253
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vladislav Pyatkov
>Assignee: Ivan Rakov
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Before, cluster detects partitions that have not to rebalance by history, by 
> them size. This threshold might be set through a system property 
> IGNITE_PDS_WAL_REBALANCE_THRESHOLD. But it is not fair deciding which 
> partitions will be rebalanced by WAL only by them size. WAL can have much 
> more records than size of a partition (many update by one key) and that 
> rebalance required more data than full transferring by network.
> Need to implement a heuristic, that might to estimate data size.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13371) Sporadic partition inconsistency after historical rebalancing of updates with same key put-remove pattern

Ivan Rakov created IGNITE-13371:
---

 Summary: Sporadic partition inconsistency after historical 
rebalancing of updates with same key put-remove pattern
 Key: IGNITE-13371
 URL: https://issues.apache.org/jira/browse/IGNITE-13371
 Project: Ignite
  Issue Type: Bug
Reporter: Ivan Rakov
Assignee: Ivan Rakov
 Fix For: 2.10


h4. scenario
# start 3 servers 3 clients, create caches
# clients start combined put + 1% remove of data in transactions 
PESSIMISTIC/REPEATABLE_READ
## kill one node
## restart one node
# ensure all transactions completed
# run idle_verify

Expected: no conflicts found
Actual:
{noformat}
[12:03:18][:55 :230] Control utility --cache idle_verify --skip-zeros 
--cache-filter PERSISTENT
[12:03:20][:55 :230] Control utility [ver. 8.7.13#20200228-sha1:7b016d63]
[12:03:20][:55 :230] 2020 Copyright(C) GridGain Systems, Inc. and Contributors
[12:03:20][:55 :230] User: prtagent
[12:03:20][:55 :230] Time: 2020-03-03T12:03:19.836
[12:03:20][:55 :230] Command [CACHE] started
[12:03:20][:55 :230] Arguments: --host 172.25.1.11 --port 11211 --cache 
idle_verify --skip-zeros --cache-filter PERSISTENT 
[12:03:20][:55 :230] 

[12:03:20][:55 :230] idle_verify task was executed with the following args: 
caches=[], excluded=[], cacheFilter=[PERSISTENT]
[12:03:20][:55 :230] idle_verify check has finished, found 1 conflict 
partitions: [counterConflicts=0, hashConflicts=1]
[12:03:20][:55 :230] Hash conflicts:
[12:03:20][:55 :230] Conflict partition: PartitionKeyV2 [grpId=1338167321, 
grpName=cache_group_3_088_1, partId=24]
[12:03:20][:55 :230] Partition instances: [PartitionHashRecordV2 
[isPrimary=false, consistentId=node_1_2, updateCntr=172349, 
partitionState=OWNING, size=6299, partHash=157875238], PartitionHashRecordV2 
[isPrimary=true, consistentId=node_1_1, updateCntr=172349, 
partitionState=OWNING, size=6299, partHash=157875238], PartitionHashRecordV2 
[isPrimary=false, consistentId=node_1_4, updateCntr=172349, 
partitionState=OWNING, size=6300, partHash=-944532882]]
[12:03:20][:55 :230] Command [CACHE] finished with code: 0
[12:03:20][:55 :230] Control utility has completed execution at: 
2020-03-03T12:03:20.593
[12:03:20][:55 :230] Execution time: 757 ms
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (IGNITE-12080) Add extended logging for rebalance



[ 
https://issues.apache.org/jira/browse/IGNITE-12080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179647#comment-17179647
 ] 

Ignite TC Bot commented on IGNITE-12080:


{panel:title=Branch: [pull/7705/head] Base: [master] : Possible Blockers 
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Cache (Restarts) 1{color} [[tests 0 TIMEOUT , Exit Code 
|https://ci.ignite.apache.org/viewLog.html?buildId=5550503]]

{panel}
{panel:title=Branch: [pull/7705/head] Base: [master] : New Tests 
(4)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}MVCC Cache 9{color} [[tests 
1|https://ci.ignite.apache.org/viewLog.html?buildId=5550549]]
* {color:#013220}IgniteCacheMvccTestSuite9: 
RebalanceStatisticsTest.testRebalanceStatistics - PASSED{color}

{color:#8b}Cache 9{color} [[tests 
1|https://ci.ignite.apache.org/viewLog.html?buildId=5550514]]
* {color:#013220}IgniteCacheTestSuite9: 
RebalanceStatisticsTest.testRebalanceStatistics - PASSED{color}

{color:#8b}Basic 1{color} [[tests 
2|https://ci.ignite.apache.org/viewLog.html?buildId=5550602]]
* {color:#013220}IgniteBasicTestSuite: 
IgniteUtilsSelfTest.testHumanReadableDuration - PASSED{color}
* {color:#013220}IgniteBasicTestSuite: 
IgniteUtilsSelfTest.testHumanReadableByteCount - PASSED{color}

{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5550562&buildTypeId=IgniteTests24Java8_RunAll]

> Add extended logging for rebalance
> --
>
> Key: IGNITE-12080
> URL: https://issues.apache.org/jira/browse/IGNITE-12080
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Kirill Tkalenko
>Assignee: Kirill Tkalenko
>Priority: Major
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> We should log information about finished rebalance on demander node.
>  I'd have in log:
> h3. Total information:
>  # Rebalance duration, rebalance start time/rebalance finish time
>  # How many nodes were suppliers in rebalance (nodeId, number of supplied 
> paritions, number of supplied entries, number of bytes, duraton of getting 
> and processing partitions from supplier)
> h3. Information per cache group:
>  # Rebalance duration, rebalance start time/rebalance finish time
>  # How many nodes were suppliers in rebalance (nodeId, number of supplied 
> paritions, list of partition ids with PRIMARY/BACKUP flag, number of supplied 
> entries, number of bytes, duraton of getting and processing partitions from 
> supplier)
>  # Information about each partition distribution (list of nodeIds with 
> primary/backup flag and marked supplier nodeId)
> h3. Information per supplier node:
>  # How many paritions were requested:
>  ** Total number
>  ** Primary/backup distribution (number of primary partitions, number of 
> backup partitions)
>  ** Total number of entries
>  ** Total size partitions in bytes
>  # How many paritions were requested per cache group:
>  ** Number of requested partitions
>  ** Number of entries in partitions
>  ** Total size of partitions in bytes
>  ** List of requested partitions with size in bytes, count entries, primary 
> or backup partition flag



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (IGNITE-12080) Add extended logging for rebalance

2020-08-18 Thread Kirill Tkalenko (Jira)



[ 
https://issues.apache.org/jira/browse/IGNITE-12080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179662#comment-17179662
 ] 

Kirill Tkalenko commented on IGNITE-12080:
--

[~ascherbakov] I corrected code by your comments and left a couple of comments, 
1 possible blocker with a high fail rate, please make code review.

> Add extended logging for rebalance
> --
>
> Key: IGNITE-12080
> URL: https://issues.apache.org/jira/browse/IGNITE-12080
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Kirill Tkalenko
>Assignee: Kirill Tkalenko
>Priority: Major
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> We should log information about finished rebalance on demander node.
>  I'd have in log:
> h3. Total information:
>  # Rebalance duration, rebalance start time/rebalance finish time
>  # How many nodes were suppliers in rebalance (nodeId, number of supplied 
> paritions, number of supplied entries, number of bytes, duraton of getting 
> and processing partitions from supplier)
> h3. Information per cache group:
>  # Rebalance duration, rebalance start time/rebalance finish time
>  # How many nodes were suppliers in rebalance (nodeId, number of supplied 
> paritions, list of partition ids with PRIMARY/BACKUP flag, number of supplied 
> entries, number of bytes, duraton of getting and processing partitions from 
> supplier)
>  # Information about each partition distribution (list of nodeIds with 
> primary/backup flag and marked supplier nodeId)
> h3. Information per supplier node:
>  # How many paritions were requested:
>  ** Total number
>  ** Primary/backup distribution (number of primary partitions, number of 
> backup partitions)
>  ** Total number of entries
>  ** Total size partitions in bytes
>  # How many paritions were requested per cache group:
>  ** Number of requested partitions
>  ** Number of entries in partitions
>  ** Total size of partitions in bytes
>  ** List of requested partitions with size in bytes, count entries, primary 
> or backup partition flag



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (IGNITE-13334) Document initial tracing implementation

2020-08-18 Thread Artem Budnikov (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-13334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Artem Budnikov reassigned IGNITE-13334:
---

Assignee: Artem Budnikov

> Document initial tracing implementation
> ---
>
> Key: IGNITE-13334
> URL: https://issues.apache.org/jira/browse/IGNITE-13334
> Project: Ignite
>  Issue Type: Task
>  Components: documentation
>Reporter: Aleksey Plekhanov
>Assignee: Artem Budnikov
>Priority: Major
>  Labels: important
> Fix For: 2.9
>
>
> Document initial tracing implementation



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13372) Operations block when cluster becomes inactive.

2020-08-18 Thread None none (Jira)

None none created IGNITE-13372:
--

Summary: Operations block when cluster becomes inactive.
Key: IGNITE-13372
URL: https://issues.apache.org/jira/browse/IGNITE-13372
Project: Ignite
Issue Type: Improvement
Affects Versions: 2.8, 2.7
Environment: 3 server nodes using I.P discovery and multiple client
nodes (client =true)
Server nodes are 32GB total of which each is configure with 6GB heap and 10GB
native persistence.
Reporter: None none

Operations block indefinitely when the cluster state becomes inactive. Initial
discussion is here:
[http://apache-ignite-users.70518.x6.nabble.com/Operation-block-on-Cluster-recovery-rebalance-td33579.html]
# Start a server cluster with persistence.
# Start thick client (client = true) application with Ignition.start(), create
IgniteCache instance.
# IgniteCache instance is initialized once at beginning of application and
shared between "threads". This particular application is a REST API and uses
the IgniteCache instance when HTTP request is made.
## When HTTP request comes in first execute cache.query()
## If successful do cache.put()
## Reply back to client.
# Shut off some nodes and bring back some to ensure that the cluster is in
inactive state.
# As the application continues to operate and since the IgniteCache instance
was initialized once at the beginning of the application, the operation blocks
indefinitely and renders the application unresponsive.
# As per the discussion thread it was advised to listen for node events and
re-initialize the IgniteCache instance. This is not user friendly or intuitive
as the client should handle all failover scenarios it's better that the
operation fail with exception so at least the application can respond and the
IgniteCache instance should somehow recover when the cluster is back to active.

--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (IGNITE-13327) Add a metric for processed keys when rebuilding indexes.



[ 
https://issues.apache.org/jira/browse/IGNITE-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179737#comment-17179737
 ] 

Ivan Rakov commented on IGNITE-13327:
-

[~ktkale...@gridgain.com] Thanks! Merged to master.

> Add a metric for processed keys when rebuilding indexes.
> 
>
> Key: IGNITE-13327
> URL: https://issues.apache.org/jira/browse/IGNITE-13327
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Kirill Tkalenko
>Assignee: Kirill Tkalenko
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> It would be useful to understand how long it will take to rebuild indexes, 
> since there can be a lot of data and indexes. Now there are following metrics 
> that allow to estimate approximately how many indexes are left to rebuild:
> # IsIndexRebuildInProgress - rebuilding cache indexes in the process;
> # IndexBuildCountPartitionsLeft - remaining number of partitions (by cache 
> group) to rebuild indexes for.
> For a more accurate estimate, I suggest adding a metric for caches "Number of 
> keys processed when rebuilding indexes" with the name 
> "IndexRebuildKeyProcessed". This way we can estimate for cache how much index 
> rebuilding will take. To do this, we can get "CacheSize" and use new metric 
> to find out how many keys are left to process.
> I also suggest adding methods:
> # org.apache.ignite.cache.CacheMetrics#getIndexRebuildKeyProcessed
> # org.apache.ignite.cache.CacheMetrics#IsIndexRebuildInProgress



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (IGNITE-13372) Operations block when cluster becomes inactive.

2020-08-18 Thread Denis A. Magda (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-13372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denis A. Magda updated IGNITE-13372:

Fix Version/s: 2.10

> Operations block when cluster becomes inactive.
> ---
>
> Key: IGNITE-13372
> URL: https://issues.apache.org/jira/browse/IGNITE-13372
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.7, 2.8
> Environment: 3 server nodes using I.P discovery and multiple client 
> nodes (client =true)
> Server nodes are 32GB total of which each is configure with 6GB heap and 10GB 
> native persistence.
>Reporter: None none
>Priority: Major
> Fix For: 2.10
>
>
> Operations block indefinitely when the cluster state becomes inactive. 
> Initial discussion is here: 
> [http://apache-ignite-users.70518.x6.nabble.com/Operation-block-on-Cluster-recovery-rebalance-td33579.html]
>  # Start a server cluster with persistence.
>  # Start thick client (client = true) application with Ignition.start(), 
> create IgniteCache instance.
>  # IgniteCache instance is initialized once at beginning of application and 
> shared between "threads". This particular application is a REST API and uses 
> the IgniteCache instance when HTTP request is made.
>  ## When HTTP request comes in first execute cache.query()
>  ## If successful do cache.put() 
>  ## Reply back to client.
>  # Shut off some nodes and bring back some to ensure that the cluster is in 
> inactive state.
>  # As the application continues to operate and since the IgniteCache instance 
> was initialized once at the beginning of the application, the operation 
> blocks indefinitely and renders the application unresponsive.
>  # As per the discussion thread it was advised to listen for node events and 
> re-initialize the IgniteCache instance. This is not user friendly or 
> intuitive as the client should handle all failover scenarios it's better that 
> the operation fail with exception so at least the application can respond and 
> the IgniteCache instance should somehow recover when the cluster is back to 
> active.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (IGNITE-13372) Operations block when cluster becomes inactive.

2020-08-18 Thread Denis A. Magda (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-13372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denis A. Magda updated IGNITE-13372:

Issue Type: Bug  (was: Improvement)

> Operations block when cluster becomes inactive.
> ---
>
> Key: IGNITE-13372
> URL: https://issues.apache.org/jira/browse/IGNITE-13372
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.7, 2.8
> Environment: 3 server nodes using I.P discovery and multiple client 
> nodes (client =true)
> Server nodes are 32GB total of which each is configure with 6GB heap and 10GB 
> native persistence.
>Reporter: None none
>Priority: Major
> Fix For: 2.10
>
>
> Operations block indefinitely when the cluster state becomes inactive. 
> Initial discussion is here: 
> [http://apache-ignite-users.70518.x6.nabble.com/Operation-block-on-Cluster-recovery-rebalance-td33579.html]
>  # Start a server cluster with persistence.
>  # Start thick client (client = true) application with Ignition.start(), 
> create IgniteCache instance.
>  # IgniteCache instance is initialized once at beginning of application and 
> shared between "threads". This particular application is a REST API and uses 
> the IgniteCache instance when HTTP request is made.
>  ## When HTTP request comes in first execute cache.query()
>  ## If successful do cache.put() 
>  ## Reply back to client.
>  # Shut off some nodes and bring back some to ensure that the cluster is in 
> inactive state.
>  # As the application continues to operate and since the IgniteCache instance 
> was initialized once at the beginning of the application, the operation 
> blocks indefinitely and renders the application unresponsive.
>  # As per the discussion thread it was advised to listen for node events and 
> re-initialize the IgniteCache instance. This is not user friendly or 
> intuitive as the client should handle all failover scenarios it's better that 
> the operation fail with exception so at least the application can respond and 
> the IgniteCache instance should somehow recover when the cluster is back to 
> active.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (IGNITE-13364) Improve index inline defaults



[ 
https://issues.apache.org/jira/browse/IGNITE-13364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17180175#comment-17180175
 ] 

Ignite TC Bot commented on IGNITE-13364:


{panel:title=Branch: [pull/8161/head] Base: [master] : No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/8161/head] Base: [master] : New Tests 
(7)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Cache 5{color} [[tests 
7|https://ci.ignite.apache.org/viewLog.html?buildId=5549471]]
* {color:#013220}IgniteCacheWithIndexingTestSuite: 
H2ComputeInlineSizeTest.testDefaultSizeForByteArray - PASSED{color}
* {color:#013220}IgniteCacheWithIndexingTestSuite: 
H2ComputeInlineSizeTest.testDefaultSizeForLargeIndex - PASSED{color}
* {color:#013220}IgniteCacheWithIndexingTestSuite: 
H2ComputeInlineSizeTest.testDefaultSizeForCompositeIndex - PASSED{color}
* {color:#013220}IgniteCacheWithIndexingTestSuite: 
H2ComputeInlineSizeTest.testDefaultSizeForString - PASSED{color}
* {color:#013220}IgniteCacheWithIndexingTestSuite: 
H2ComputeInlineSizeTest.testDefaultSizeForStringWithDefinedLength - 
PASSED{color}
* {color:#013220}IgniteCacheWithIndexingTestSuite: 
H2ComputeInlineSizeTest.testDefaultSizeForStringWithIncorrectSql - PASSED{color}
* {color:#013220}IgniteCacheWithIndexingTestSuite: 
H2ComputeInlineSizeTest.testDefaultSizeForBytesWithDefinedLength - PASSED{color}

{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5549457&buildTypeId=IgniteTests24Java8_RunAll]

>  Improve index inline defaults
> --
>
> Key: IGNITE-13364
> URL: https://issues.apache.org/jira/browse/IGNITE-13364
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Reporter: Evgeniy Rudenko
>Assignee: Evgeniy Rudenko
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> We need to improve how inline size is calculated by default for 
> variable-length types.
> Currently if a varlength type is encountered inline size just defaults to 10, 
> which is almost always not enough.
> A more sensible behavior would be the following:
> 1. Add a fixed default to the inline size calculation for every 
> variable-length type. For example, if the default inlined size for a string 
> is 10 then an index like (INT, VARCHAR, VARCHAR, INT) should have inline size 
> default as 5 + 10 + 10 + 5 = 30 (5 for each int, 10 for each string).
> 2. Add special support for VARCHAR_FIXED - if a VARCHAR has known length then 
> that length is used for inline size calculation



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13373) WAL segmentns do not released on releaseHistoryForPreloading()

Alexander Lapin created IGNITE-13373:


 Summary: WAL segmentns do not released on 
releaseHistoryForPreloading()
 Key: IGNITE-13373
 URL: https://issues.apache.org/jira/browse/IGNITE-13373
 Project: Ignite
  Issue Type: Bug
Reporter: Alexander Lapin
Assignee: Alexander Lapin


* Reserve/releaseHistoryForPreloading() was reworked, now we store oldest 
WALPointer that was reserved during reserveHistoryForPreloading in 
reservedForPreloading field. As a result it's possible to release wall 
reservation on releaseHIstoryForPreloading().

 * searchAndReserveCheckpoints() was slightly refactored: now it returns not 
only an earliestValidCheckpoints but also an oldest reservedCheckpoint, so that 
there’s no need to recalculate it within reserveHistoryForExchange().

 *



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (IGNITE-13373) WAL segmentns do not released on releaseHistoryForPreloading()



 [ 
https://issues.apache.org/jira/browse/IGNITE-13373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-13373:
-
Fix Version/s: 2.10

> WAL segmentns do not released on releaseHistoryForPreloading()
> --
>
> Key: IGNITE-13373
> URL: https://issues.apache.org/jira/browse/IGNITE-13373
> Project: Ignite
>  Issue Type: Bug
>Reporter: Alexander Lapin
>Assignee: Alexander Lapin
>Priority: Major
> Fix For: 2.10
>
>
> * Reserve/releaseHistoryForPreloading() was reworked, now we store oldest 
> WALPointer that was reserved during reserveHistoryForPreloading in 
> reservedForPreloading field. As a result it's possible to release wall 
> reservation on releaseHIstoryForPreloading().
>  * searchAndReserveCheckpoints() was slightly refactored: now it returns not 
> only an earliestValidCheckpoints but also an oldest reservedCheckpoint, so 
> that there’s no need to recalculate it within reserveHistoryForExchange().
>  *



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (IGNITE-13373) WAL segmentns do not released on releaseHistoryForPreloading()



 [ 
https://issues.apache.org/jira/browse/IGNITE-13373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-13373:
-
Fix Version/s: (was: 2.10)

> WAL segmentns do not released on releaseHistoryForPreloading()
> --
>
> Key: IGNITE-13373
> URL: https://issues.apache.org/jira/browse/IGNITE-13373
> Project: Ignite
>  Issue Type: Bug
>Reporter: Alexander Lapin
>Assignee: Alexander Lapin
>Priority: Major
>
> * Reserve/releaseHistoryForPreloading() was reworked, now we store oldest 
> WALPointer that was reserved during reserveHistoryForPreloading in 
> reservedForPreloading field. As a result it's possible to release wall 
> reservation on releaseHIstoryForPreloading().
>  * searchAndReserveCheckpoints() was slightly refactored: now it returns not 
> only an earliestValidCheckpoints but also an oldest reservedCheckpoint, so 
> that there’s no need to recalculate it within reserveHistoryForExchange().
>  *



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (IGNITE-13373) WAL segmentns do not released on releaseHistoryForPreloading()