[jira] [Commented] (IGNITE-14658) [IEP-35] SSL metrics

2021-06-11 Thread Mikhail Petrov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-14658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17362242#comment-17362242
 ] 

Mikhail Petrov commented on IGNITE-14658:
-

[~NSAmelchev], [~nizhikov] Thanks a lot for the review.

> [IEP-35] SSL metrics
> 
>
> Key: IGNITE-14658
> URL: https://issues.apache.org/jira/browse/IGNITE-14658
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Mikhail Petrov
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.12
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> The following SSL metrics required:
> * Count of SSL sessions.
> * Count of rejected SSL sessions.
> * Handshake time metric.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (IGNITE-14900) Python "Can not query binary type" error on collections of objects

2021-06-11 Thread Bojidar Marinov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bojidar Marinov resolved IGNITE-14900.
--
Fix Version/s: python-0.5.0
   Resolution: Duplicate

Apologies, did not notice the duplicate issue..

> Python "Can not query binary type" error on collections of objects
> --
>
> Key: IGNITE-14900
> URL: https://issues.apache.org/jira/browse/IGNITE-14900
> Project: Ignite
>  Issue Type: Bug
>  Components: python
>Affects Versions: python-0.4.0
> Environment: {noformat}
> pyignite==0.4.0
> org.apache.ignite:ignite-core:2.10.0{noformat}
>  
>Reporter: Bojidar Marinov
>Priority: Major
> Fix For: python-0.5.0
>
> Attachments: repro.py
>
>
> When reading an object which has a CollectionObject field containing binary 
> objects, the Python thin client fails to read the object with e.g.
>  
> {noformat}
> pyignite.exceptions.ParseError: Can not query binary type -1322970774
> {noformat}
> The same occurs with ObjectArrayObject and MapObject fields.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-14900) Python "Can not query binary type" error on collections of objects

2021-06-11 Thread Bojidar Marinov (Jira)
Bojidar Marinov created IGNITE-14900:


 Summary: Python "Can not query binary type" error on collections 
of objects
 Key: IGNITE-14900
 URL: https://issues.apache.org/jira/browse/IGNITE-14900
 Project: Ignite
  Issue Type: Bug
  Components: python
Affects Versions: python-0.4.0
 Environment: {noformat}
pyignite==0.4.0
org.apache.ignite:ignite-core:2.10.0{noformat}
 
Reporter: Bojidar Marinov
 Attachments: repro.py

When reading an object which has a CollectionObject field containing binary 
objects, the Python thin client fails to read the object with e.g.

 
{noformat}
pyignite.exceptions.ParseError: Can not query binary type -1322970774
{noformat}
The same occurs with ObjectArrayObject and MapObject fields.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-14658) [IEP-35] SSL metrics

2021-06-11 Thread Amelchev Nikita (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amelchev Nikita updated IGNITE-14658:
-
Fix Version/s: 2.12

> [IEP-35] SSL metrics
> 
>
> Key: IGNITE-14658
> URL: https://issues.apache.org/jira/browse/IGNITE-14658
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Mikhail Petrov
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.12
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> The following SSL metrics required:
> * Count of SSL sessions.
> * Count of rejected SSL sessions.
> * Handshake time metric.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-14812) SQL statistics

2021-06-11 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-14812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361847#comment-17361847
 ] 

Ignite TC Bot commented on IGNITE-14812:


{panel:title=Branch: [pull/9145/head] Base: [master] : No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/9145/head] Base: [master] : New Tests 
(176)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Queries 1{color} [[tests 
176|https://ci.ignite.apache.org/viewLog.html?buildId=6045172]]
* {color:#013220}IgniteBinaryCacheQueryTestSuite: 
StatisticsViewsPersistenceTest.testConfigurationViewDeletion - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite: 
StatisticsViewsInMemoryTest.testConfigurationViewDeletion - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite: 
StatisticsConfigurationTest.dropUpdate[persist=true] - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite: 
StatisticsConfigurationTest.updateStatisticsOnChangeTopology[persist=false] - 
PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite: 
StatisticsConfigurationTest.updateStatisticsOnChangeTopology[persist=true] - 
PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite: 
StatisticsConfigurationTest.dropSingleColumnStatisticWhileNodeDown[persist=true]
 - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite: 
StatisticsConfigurationTest.dropSingleColumnStatisticWhileNodeDown[persist=false]
 - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite: 
StatisticsConfigurationTest.dropUpdate[persist=false] - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite: 
StatisticsViewsPersistenceTest.testEnforceStatisticValues - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite: 
StatisticsViewsInMemoryTest.testEnforceStatisticValues - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite: 
StatisticsConfigurationTest.dropColumnWhileNodeDown[persist=false] - 
PASSED{color}
... and 165 new tests

{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6044375&buildTypeId=IgniteTests24Java8_RunAll]

> SQL statistics
> --
>
> Key: IGNITE-14812
> URL: https://issues.apache.org/jira/browse/IGNITE-14812
> Project: Ignite
>  Issue Type: New Feature
>  Components: sql
>Reporter: Alexander Belyak
>Assignee: Alexander Belyak
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Add statistics collection and usage.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-14899) PagesWriteSpeedBasedThrottle wrong appliance

2021-06-11 Thread Semyon Danilov (Jira)
Semyon Danilov created IGNITE-14899:
---

 Summary: PagesWriteSpeedBasedThrottle wrong appliance
 Key: IGNITE-14899
 URL: https://issues.apache.org/jira/browse/IGNITE-14899
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.10
Reporter: Semyon Danilov
Assignee: Semyon Danilov


During the investigation of the throttling algorithm I found out that it 
doesn’t exactly match the description provided in readme 
([https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/pagemem/README.md]).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-14658) [IEP-35] SSL metrics

2021-06-11 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-14658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361813#comment-17361813
 ] 

Ignite TC Bot commented on IGNITE-14658:


{panel:title=Branch: [pull/9132/head] Base: [master] : Possible Blockers 
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}RDD{color} [[tests 0 Exit Code 
|https://ci.ignite.apache.org/viewLog.html?buildId=6045057]]

{panel}
{panel:title=Branch: [pull/9132/head] Base: [master] : New Tests 
(6)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Java Client{color} [[tests 
6|https://ci.ignite.apache.org/viewLog.html?buildId=6044719]]
* {color:#013220}IgniteClientTestSuite: 
NodeSslConnectionMetricTest.testDiscovery - PASSED{color}
* {color:#013220}IgniteClientTestSuite: 
NodeSslConnectionMetricTest.testCommunication - PASSED{color}
* {color:#013220}IgniteClientTestSuite: 
NodeSslConnectionMetricTest.testClientConnector - PASSED{color}
* {color:#013220}IgniteClientTestSuite: 
NodeSslConnectionMetricTest.testSslDisabled - PASSED{color}
* {color:#013220}IgniteClientTestSuite: NodeSslConnectionMetricTest.testJdbc - 
PASSED{color}
* {color:#013220}IgniteClientTestSuite: 
NodeSslConnectionMetricTest.testRestClientConnector - PASSED{color}

{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6044801&buildTypeId=IgniteTests24Java8_RunAll]

> [IEP-35] SSL metrics
> 
>
> Key: IGNITE-14658
> URL: https://issues.apache.org/jira/browse/IGNITE-14658
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Mikhail Petrov
>Priority: Major
>  Labels: IEP-35
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> The following SSL metrics required:
> * Count of SSL sessions.
> * Count of rejected SSL sessions.
> * Handshake time metric.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-14851) Enable partition awareness by default in python thin client

2021-06-11 Thread Igor Sapego (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-14851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361806#comment-17361806
 ] 

Igor Sapego commented on IGNITE-14851:
--

[~ivandasch] looks good to me.

> Enable partition awareness by default in python thin client
> ---
>
> Key: IGNITE-14851
> URL: https://issues.apache.org/jira/browse/IGNITE-14851
> Project: Ignite
>  Issue Type: Task
>  Components: python, thin client
>Reporter: Ivan Daschinsky
>Assignee: Ivan Daschinsky
>Priority: Major
> Fix For: python-0.5.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Needs to enable partition awareness by default, as in C++, Java and .NET



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-14895) Replace reflection with method handles in GridUnsage

2021-06-11 Thread Ivan Bessonov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Bessonov updated IGNITE-14895:
---
Fix Version/s: 3.0.0-alpha3

> Replace reflection with method handles in GridUnsage
> 
>
> Key: IGNITE-14895
> URL: https://issues.apache.org/jira/browse/IGNITE-14895
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 3.0.0-alpha2
>Reporter: Ivan Bessonov
>Assignee: Ivan Bessonov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha3
>
>
> Reflective operations are notoriously slow, there's a better way to invoke 
> methods dynamically.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-14886) Binary metadata registration in EntryProcessor fails after CREATE TABLE with same type

2021-06-11 Thread Veena Mithare (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361693#comment-17361693
 ] 

Veena Mithare commented on IGNITE-14886:


 As per the definition of entry processor - it is used to execute updates on 
entries on
the nodes that store it - currently this is not the case. It runs on the client 
side as well. 

 

Will this issue not be resolved if the entry processor ran only on the server 
nodes.

 

> Binary metadata registration in EntryProcessor fails after CREATE TABLE with 
> same type
> --
>
> Key: IGNITE-14886
> URL: https://issues.apache.org/jira/browse/IGNITE-14886
> Project: Ignite
>  Issue Type: Bug
>Reporter: Anton Kurbanov
>Priority: Major
> Attachments: BinaryObjectEntryProcessorTest.java
>
>
> https://ignite.apache.org/docs/latest/key-value-api/binary-objects#creating-and-modifying-binary-objects
> This part of documentation assumes that binary object can be 
> processed/created/modified within the EntryProcessor, however in this case it 
> is no longer working unless the type is manually registered before executing 
> the invoke or create table is removed, which renders this documentation part 
> incorrect.
> This situation happens since the attempt to fix metadata registration 
> deadlock here: https://issues.apache.org/jira/browse/IGNITE-11313
> Failing test attached.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-14894) Ignite RDD test suite is broken

2021-06-11 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-14894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361691#comment-17361691
 ] 

Ignite TC Bot commented on IGNITE-14894:


{panel:title=Branch: [pull/9169/head] Base: [master] : No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/9169/head] Base: [master] : No new tests 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6044493&buildTypeId=IgniteTests24Java8_RunAll]

> Ignite RDD test suite is broken
> ---
>
> Key: IGNITE-14894
> URL: https://issues.apache.org/jira/browse/IGNITE-14894
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.10
>Reporter: Pavel Pereslegin
>Assignee: Pavel Pereslegin
>Priority: Blocker
> Fix For: 2.11
>
>
> RDD starts to fail constantly after IGNITE-14424.
> Stacktrace that may lead to this failure
> {noformat}
> [04:45:12]W:   [org.apache.ignite:ignite-spark] [2021-06-08 
> 04:45:12,601][ERROR][spark-listener-group-shared][IgniteKernal] Failed to 
> pre-stop processor: GridProcessorAdapter []
> [04:45:12]W:   [org.apache.ignite:ignite-spark] 
> java.lang.NullPointerException
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.localJoinExchange(GridDhtPartitionsExchangeFuture.java:2137)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.managers.encryption.GridEncryptionManager.onDoneAfterTopologyUnlock(GridEncryptionManager.java:1118)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:2627)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:157)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:478)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.onKernalStop0(GridCachePartitionExchangeManager.java:1001)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter.onKernalStop(GridCacheSharedManagerAdapter.java:135)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.onKernalStop(GridCacheProcessor.java:824)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2578)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2526)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2864)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2687)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.IgnitionEx.stop(IgnitionEx.java:362)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.Ignition.stop(Ignition.java:230)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.spark.IgniteContext.org$apache$ignite$spark$IgniteContext$$doClose(IgniteContext.scala:186)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.spark.IgniteContext.close(IgniteContext.scala:179)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.spark.IgniteContext$$anon$1.onApplicationEnd(IgniteContext.scala:69)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.spark.scheduler.SparkListenerBus$class.doPostEvent(SparkListenerBus.scala:57)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.spark.util.ListenerBus$class.postToAll(List

[jira] [Updated] (IGNITE-14898) IndexOutOfBoundException in flusher selection logic in GridCacheWriteBehindStore

2021-06-11 Thread Ilya Korol (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Korol updated IGNITE-14898:

Description: 
There i a bug in GridCacheWriteBehindStore method for selecting which flusher 
should be used for current data write by specified key:

{code:java}
/**
 * Return flusher by by key.
 *
 * @param key Key for search.
 * @return flusher.
 */
private Flusher flusher(K key) {
int h, idx;

if (flushThreadCntIsPowerOfTwo)
idx = ((h = key.hashCode()) ^ (h >>> 16)) & (flushThreadCnt - 1);
else
idx = ((h = key.hashCode()) ^ (h >>> 16)) % flushThreadCnt;

return flushThreads[idx];
}
{code}

In case when flushThreadCount is not a power of 2 and incoming key.hashCode() < 
0 (e.g. for UUID string) we will get IndexOutOfBoundException.


http://apache-ignite-users.70518.x6.nabble.com/Bug-in-GridCacheWriteBehindStore-td36189.html

  was:
There i a bug in GridCacheWriteBehindStore method for selecting which flusher 
should be used for current data write by specified key:

{code:java}
/**
 * Return flusher by by key.
 *
 * @param key Key for search.
 * @return flusher.
 */
private Flusher flusher(K key) {
int h, idx;

if (flushThreadCntIsPowerOfTwo)
idx = ((h = key.hashCode()) ^ (h >>> 16)) & (flushThreadCnt - 1);
else
idx = ((h = key.hashCode()) ^ (h >>> 16)) % flushThreadCnt;

return flushThreads[idx];
}
{code}

In case when flushThreadCount is not a power of 2 and incoming key.hashCode() < 
0 (e.g. for UUID string) we will get IndexOutOfBoundException.



> IndexOutOfBoundException in flusher selection logic in 
> GridCacheWriteBehindStore
> 
>
> Key: IGNITE-14898
> URL: https://issues.apache.org/jira/browse/IGNITE-14898
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Affects Versions: 2.10
>Reporter: Ilya Korol
>Assignee: Ilya Korol
>Priority: Major
>
> There i a bug in GridCacheWriteBehindStore method for selecting which flusher 
> should be used for current data write by specified key:
> {code:java}
> /**
>  * Return flusher by by key.
>  *
>  * @param key Key for search.
>  * @return flusher.
>  */
> private Flusher flusher(K key) {
> int h, idx;
> if (flushThreadCntIsPowerOfTwo)
> idx = ((h = key.hashCode()) ^ (h >>> 16)) & (flushThreadCnt - 1);
> else
> idx = ((h = key.hashCode()) ^ (h >>> 16)) % flushThreadCnt;
> return flushThreads[idx];
> }
> {code}
> In case when flushThreadCount is not a power of 2 and incoming key.hashCode() 
> < 0 (e.g. for UUID string) we will get IndexOutOfBoundException.
> http://apache-ignite-users.70518.x6.nabble.com/Bug-in-GridCacheWriteBehindStore-td36189.html



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-14898) IndexOutOfBoundException in flusher selection logic in GridCacheWriteBehindStore

2021-06-11 Thread Ilya Korol (Jira)
Ilya Korol created IGNITE-14898:
---

 Summary: IndexOutOfBoundException in flusher selection logic in 
GridCacheWriteBehindStore
 Key: IGNITE-14898
 URL: https://issues.apache.org/jira/browse/IGNITE-14898
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 2.10
Reporter: Ilya Korol
Assignee: Ilya Korol


There i a bug in GridCacheWriteBehindStore method for selecting which flusher 
should be used for current data write by specified key:

{code:java}
/**
 * Return flusher by by key.
 *
 * @param key Key for search.
 * @return flusher.
 */
private Flusher flusher(K key) {
int h, idx;

if (flushThreadCntIsPowerOfTwo)
idx = ((h = key.hashCode()) ^ (h >>> 16)) & (flushThreadCnt - 1);
else
idx = ((h = key.hashCode()) ^ (h >>> 16)) % flushThreadCnt;

return flushThreads[idx];
}
{code}

In case when flushThreadCount is not a power of 2 and incoming key.hashCode() < 
0 (e.g. for UUID string) we will get IndexOutOfBoundException.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Comment Edited] (IGNITE-14854) Fix tests IgniteCacheLocalQueryDefaultTimeoutSelfTest for lazy=true mode

2021-06-11 Thread Yury Gerzhedovich (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-14854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361582#comment-17361582
 ] 

Yury Gerzhedovich edited comment on IGNITE-14854 at 6/11/21, 11:43 AM:
---

[~tledkov-gridgain], just a minor comment, otherwise good.


was (Author: jooger):
[~tledkov-gridgain], just minor comment, otherwise it's good.

> Fix tests IgniteCacheLocalQueryDefaultTimeoutSelfTest for lazy=true mode
> 
>
> Key: IGNITE-14854
> URL: https://issues.apache.org/jira/browse/IGNITE-14854
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Fix For: 2.12
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Tests are not stable:
> {{IgniteCacheLocalQueryDefaultTimeoutSelfTest.testQueryTimeout}}
> {{IgniteCacheLocalQueryDefaultTimeoutSelfTest.testQueryDefaultTimeout}}
> The local runs are OK. So I guess the timeout works OK. Looks like the tests 
> must be stabilized for TC.
> Root cause: the test isn't adopted for lazy mode (results aren't fetched, 
> only iterator is opened)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-14819) ConnectionManager supresses exceptions from the HandshakeManager

2021-06-11 Thread Semyon Danilov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-14819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361667#comment-17361667
 ] 

Semyon Danilov commented on IGNITE-14819:
-

Looks good to me!

> ConnectionManager supresses exceptions from the HandshakeManager
> 
>
> Key: IGNITE-14819
> URL: https://issues.apache.org/jira/browse/IGNITE-14819
> Project: Ignite
>  Issue Type: Bug
>  Components: networking
>Reporter: Aleksandr Polovtcev
>Assignee: Aleksandr Polovtcev
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha3
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> If an exception is thrown during the handshake, for example, because of a 
> missing serialization factory, no errors are logged and no exceptions are 
> thrown, \{{ClientManager#connect}} future simply never completes.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (IGNITE-14258) Ducktests code de-duplication and simplification

2021-06-11 Thread Anton Vinogradov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anton Vinogradov resolved IGNITE-14258.
---
Resolution: Fixed

> Ducktests code de-duplication and simplification
> 
>
> Key: IGNITE-14258
> URL: https://issues.apache.org/jira/browse/IGNITE-14258
> Project: Ignite
>  Issue Type: Bug
>Reporter: Anton Vinogradov
>Assignee: Anton Vinogradov
>Priority: Major
>
> The goal is to perform pre-merge review of the whole ducktests code and 
> refactor it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-14897) Separate logs for different Ignite nodes.

2021-06-11 Thread Andrey Mashenkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Mashenkov updated IGNITE-14897:
--
Summary: Separate logs for different Ignite nodes.  (was: Ignite node 
logger.)

> Separate logs for different Ignite nodes.
> -
>
> Key: IGNITE-14897
> URL: https://issues.apache.org/jira/browse/IGNITE-14897
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Andrey Mashenkov
>Priority: Major
>  Labels: ignite-3
>
> h3. Motivation
> For now we use statically initialized loggers in classes. This can leads to 
> mess in logs when user starts 2+ nodes in the same JVM and therefore, to 
> harder or even impossible debugging.
> Ignite-2 is not affected with this, because every node has a separate logger 
> instance and can write logs into a separate file.
> h3. Description
> To resolve this we should use separate logger for each node instance.
> Possible solutions
> # Avoid static logger usage.
> # Use static wrappers for Thread-local loggers.
> With the first approach, we have to pass loggers to every component via 
> constructor or use dependency injection or create logger manually. Also, we 
> should bother about passing correct logger category to the object that may 
> use it and logger instance creation rate (if it is created in for every class 
> instance)
> With the second one, we can use a default single logger instance for user 
> threads
> and use thread-local like wrappers for Ignite node threads.
> Instead of using a ThreadLocal class directly and avoid hash-table lookups, 
> we can introduce IgniteThread class with a Logger field.
> The logger may either add a node prefix to all messages or delegate calls to 
> other logger of certain category (if we want to or will be able to support 
> different logger configurations for different nodes).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-14897) Separate loggers for different Ignite nodes.

2021-06-11 Thread Andrey Mashenkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Mashenkov updated IGNITE-14897:
--
Summary: Separate loggers for different Ignite nodes.  (was: Separate logs 
for different Ignite nodes.)

> Separate loggers for different Ignite nodes.
> 
>
> Key: IGNITE-14897
> URL: https://issues.apache.org/jira/browse/IGNITE-14897
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Andrey Mashenkov
>Priority: Major
>  Labels: ignite-3
>
> h3. Motivation
> For now we use statically initialized loggers in classes. This can leads to 
> mess in logs when user starts 2+ nodes in the same JVM and therefore, to 
> harder or even impossible debugging.
> Ignite-2 is not affected with this, because every node has a separate logger 
> instance and can write logs into a separate file.
> h3. Description
> To resolve this we should use separate logger for each node instance.
> Possible solutions
> # Avoid static logger usage.
> # Use static wrappers for Thread-local loggers.
> With the first approach, we have to pass loggers to every component via 
> constructor or use dependency injection or create logger manually. Also, we 
> should bother about passing correct logger category to the object that may 
> use it and logger instance creation rate (if it is created in for every class 
> instance)
> With the second one, we can use a default single logger instance for user 
> threads
> and use thread-local like wrappers for Ignite node threads.
> Instead of using a ThreadLocal class directly and avoid hash-table lookups, 
> we can introduce IgniteThread class with a Logger field.
> The logger may either add a node prefix to all messages or delegate calls to 
> other logger of certain category (if we want to or will be able to support 
> different logger configurations for different nodes).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-14897) Ignite node logger.

2021-06-11 Thread Andrey Mashenkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Mashenkov updated IGNITE-14897:
--
Description: 
h3. Motivation
For now we use statically initialized loggers in classes. This can leads to 
mess in logs when user starts 2+ nodes in the same JVM and therefore, to harder 
or even impossible debugging.
Ignite-2 is not affected with this, because every node has a separate logger 
instance and can write logs into a separate file.

h3. Description
To resolve this we should use separate logger for each node instance.

Possible solutions
# Avoid static logger usage.
# Use static wrappers for Thread-local loggers.

With the first approach, we have to pass loggers to every component via 
constructor or use dependency injection or create logger manually. Also, we 
should bother about passing correct logger category to the object that may use 
it and logger instance creation rate (if it is created in for every class 
instance)

With the second one, we can use a default single logger instance for user 
threads
and use thread-local like wrappers for Ignite node threads.
Instead of using a ThreadLocal class directly and avoid hash-table lookups, we 
can introduce IgniteThread class with a Logger field.
The logger may either add a node prefix to all messages or delegate calls to 
other logger of certain category (if we want to or will be able to support 
different logger configurations for different nodes).


  was:
h3. Motivation
For now we use statically initialized loggers in classes. This can leads to 
mess in logs when user starts 2+ nodes in the same JVM and therefore, to harder 
or even impossible debugging.
Ignite-2 is not affected with this, because every node has a separate logger 
instance and can write logs into a separate file.

3. Description
To resolve this we should use separate logger for each node instance.

Possible solutions
# Avoid static logger usage.
# Use static wrappers for Thread-local loggers.

With the first approach, we have to pass loggers to every component via 
constructor or use dependency injection or create logger manually. Also, we 
should bother about passing correct logger category to the object that may use 
it and logger instance creation rate (if it is created in for every class 
instance)

With the second one, we can use a default single logger instance for user 
threads
and use thread-local like wrappers for Ignite node threads.
Instead of using a ThreadLocal class directly and avoid hash-table lookups, we 
can introduce IgniteThread class with a Logger field.
The logger may either add a node prefix to all messages or delegate calls to 
other logger of certain category (if we want to or will be able to support 
different logger configurations for different nodes).



> Ignite node logger.
> ---
>
> Key: IGNITE-14897
> URL: https://issues.apache.org/jira/browse/IGNITE-14897
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Andrey Mashenkov
>Priority: Major
>  Labels: ignite-3
>
> h3. Motivation
> For now we use statically initialized loggers in classes. This can leads to 
> mess in logs when user starts 2+ nodes in the same JVM and therefore, to 
> harder or even impossible debugging.
> Ignite-2 is not affected with this, because every node has a separate logger 
> instance and can write logs into a separate file.
> h3. Description
> To resolve this we should use separate logger for each node instance.
> Possible solutions
> # Avoid static logger usage.
> # Use static wrappers for Thread-local loggers.
> With the first approach, we have to pass loggers to every component via 
> constructor or use dependency injection or create logger manually. Also, we 
> should bother about passing correct logger category to the object that may 
> use it and logger instance creation rate (if it is created in for every class 
> instance)
> With the second one, we can use a default single logger instance for user 
> threads
> and use thread-local like wrappers for Ignite node threads.
> Instead of using a ThreadLocal class directly and avoid hash-table lookups, 
> we can introduce IgniteThread class with a Logger field.
> The logger may either add a node prefix to all messages or delegate calls to 
> other logger of certain category (if we want to or will be able to support 
> different logger configurations for different nodes).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-14895) Replace reflection with method handles in GridUnsage

2021-06-11 Thread Aleksandr Polovtcev (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-14895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361603#comment-17361603
 ] 

Aleksandr Polovtcev commented on IGNITE-14895:
--

The patch looks good!

> Replace reflection with method handles in GridUnsage
> 
>
> Key: IGNITE-14895
> URL: https://issues.apache.org/jira/browse/IGNITE-14895
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 3.0.0-alpha2
>Reporter: Ivan Bessonov
>Assignee: Ivan Bessonov
>Priority: Major
>  Labels: ignite-3
>
> Reflective operations are notoriously slow, there's a better way to invoke 
> methods dynamically.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13364) Improve index inline defaults

2021-06-11 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361601#comment-17361601
 ] 

Ignite TC Bot commented on IGNITE-13364:


{panel:title=Branch: [pull/9167/head] Base: [master] : No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/9167/head] Base: [master] : New Tests 
(2)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Cache 5{color} [[tests 
2|https://ci.ignite.apache.org/viewLog.html?buildId=6044383]]
* {color:#013220}IgniteCacheWithIndexingTestSuite: 
ComputeInlineSizeTest.testSQLIndexes - PASSED{color}
* {color:#013220}IgniteCacheWithIndexingTestSuite: 
ComputeInlineSizeTest.testAnnotaitionPrecision - PASSED{color}

{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6043361&buildTypeId=IgniteTests24Java8_RunAll]

>  Improve index inline defaults
> --
>
> Key: IGNITE-13364
> URL: https://issues.apache.org/jira/browse/IGNITE-13364
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Reporter: Evgeniy Rudenko
>Assignee: Maksim Timonin
>Priority: Major
> Fix For: 2.12
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> We need to improve how inline size is calculated by default for 
> variable-length types.
> Currently if a varlength type is encountered inline size just defaults to 10, 
> which is almost always not enough.
> A more sensible behavior would be the following:
> 1. Add a fixed default to the inline size calculation for every 
> variable-length type. For example, if the default inlined size for a string 
> is 10 then an index like (INT, VARCHAR, VARCHAR, INT) should have inline size 
> default as 5 + 10 + 10 + 5 = 30 (5 for each int, 10 for each string).
> 2. Add special support for VARCHAR_FIXED - if a VARCHAR has known length then 
> that length is used for inline size calculation



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-14897) Ignite node logger.

2021-06-11 Thread Andrey Mashenkov (Jira)
Andrey Mashenkov created IGNITE-14897:
-

 Summary: Ignite node logger.
 Key: IGNITE-14897
 URL: https://issues.apache.org/jira/browse/IGNITE-14897
 Project: Ignite
  Issue Type: New Feature
Reporter: Andrey Mashenkov


h3. Motivation
For now we use statically initialized loggers in classes. This can leads to 
mess in logs when user starts 2+ nodes in the same JVM and therefore, to harder 
or even impossible debugging.
Ignite-2 is not affected with this, because every node has a separate logger 
instance and can write logs into a separate file.

3. Description
To resolve this we should use separate logger for each node instance.

Possible solutions
# Avoid static logger usage.
# Use static wrappers for Thread-local loggers.

With the first approach, we have to pass loggers to every component via 
constructor or use dependency injection or create logger manually. Also, we 
should bother about passing correct logger category to the object that may use 
it and logger instance creation rate (if it is created in for every class 
instance)

With the second one, we can use a default single logger instance for user 
threads
and use thread-local like wrappers for Ignite node threads.
Instead of using a ThreadLocal class directly and avoid hash-table lookups, we 
can introduce IgniteThread class with a Logger field.
The logger may either add a node prefix to all messages or delegate calls to 
other logger of certain category (if we want to or will be able to support 
different logger configurations for different nodes).




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-14854) Fix tests IgniteCacheLocalQueryDefaultTimeoutSelfTest for lazy=true mode

2021-06-11 Thread Yury Gerzhedovich (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-14854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361582#comment-17361582
 ] 

Yury Gerzhedovich commented on IGNITE-14854:


[~tledkov-gridgain], just minor comment, otherwise it's good.

> Fix tests IgniteCacheLocalQueryDefaultTimeoutSelfTest for lazy=true mode
> 
>
> Key: IGNITE-14854
> URL: https://issues.apache.org/jira/browse/IGNITE-14854
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Fix For: 2.12
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Tests are not stable:
> {{IgniteCacheLocalQueryDefaultTimeoutSelfTest.testQueryTimeout}}
> {{IgniteCacheLocalQueryDefaultTimeoutSelfTest.testQueryDefaultTimeout}}
> The local runs are OK. So I guess the timeout works OK. Looks like the tests 
> must be stabilized for TC.
> Root cause: the test isn't adopted for lazy mode (results aren't fetched, 
> only iterator is opened)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-14866) Calcite. Check SQL function works

2021-06-11 Thread Yury Gerzhedovich (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yury Gerzhedovich updated IGNITE-14866:
---
Description: We need to write tests on existing simple SQL functions (nor 
aggregation or window functions) and have a list of supported functions.  (was: 
emphasized textWe need to write tests on existing simple SQL function (nor 
aggregation or window functions) and have a list of supported functions.)

> Calcite. Check SQL function works
> -
>
> Key: IGNITE-14866
> URL: https://issues.apache.org/jira/browse/IGNITE-14866
> Project: Ignite
>  Issue Type: New Feature
>  Components: sql
>Reporter: Yury Gerzhedovich
>Priority: Major
>
> We need to write tests on existing simple SQL functions (nor aggregation or 
> window functions) and have a list of supported functions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-14859) Schema update. Instant row write-back after upgrade to the latest schema.

2021-06-11 Thread Andrey Mashenkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Mashenkov updated IGNITE-14859:
--
Summary: Schema update. Instant row write-back after upgrade to the latest 
schema.  (was: Instant row write-back after upgrade to the latest schema.)

> Schema update. Instant row write-back after upgrade to the latest schema.
> -
>
> Key: IGNITE-14859
> URL: https://issues.apache.org/jira/browse/IGNITE-14859
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Reporter: Andrey Mashenkov
>Priority: Major
>  Labels: iep-54, ignite-3
> Fix For: 3.0.0-alpha3
>
>
> Let's implement row write-back to store when read and upgradeed a row of 
> outdated version.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (IGNITE-14863) Schema update. Add and remove column.

2021-06-11 Thread Andrey Mashenkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Mashenkov reassigned IGNITE-14863:
-

Assignee: Andrey Mashenkov

> Schema update. Add and remove column.
> -
>
> Key: IGNITE-14863
> URL: https://issues.apache.org/jira/browse/IGNITE-14863
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Reporter: Andrey Mashenkov
>Assignee: Andrey Mashenkov
>Priority: Major
>  Labels: iep-54, ignite-3
> Fix For: 3.0.0-alpha3
>
>
> Implement evolution converters for add/remove column schema operations.
> Configuration changes should trigger a new schema version adding to schema 
> history.
> Assumes, all nodes will use a new schema after upgrade.
> Old row can be upgraded on fly using evolution converters.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-14556) Live Schema. Add Tuple validation.

2021-06-11 Thread Andrey Mashenkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Mashenkov updated IGNITE-14556:
--
Summary: Live Schema. Add Tuple validation.  (was: Add Tuple validation.)

> Live Schema. Add Tuple validation.
> --
>
> Key: IGNITE-14556
> URL: https://issues.apache.org/jira/browse/IGNITE-14556
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Mashenkov
>Priority: Major
>  Labels: iep-54, ignite-3
> Fix For: 3.0.0-alpha3
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> h3. Motivation.
> At a point of Table public method is called by a user, we need to validate 
> user input (for LIVE-schema purposes at least).
> h3. Description.
> We can add this logic to check if value fields match the current schema 
> version (no new fields).
>  * For LIVE-Schema. If Tuple has one or more additional columns, then we 
> should try to register a new schema first, then proceed with the user 
> operation.
>  * For STRICT-Schema.  If Tuple has one or more additional columns, then we 
> should fail the user operation.
>  * For KeyValueView, we should validate key Tuple as well, and fail if there 
> are unknown columns. Because a key column span is immutable. The only 
> exception may be if a user creates a schemaless table, then a schema of the 
> 1-st version should be registered instantly.
> Assumed, any column type mismatch or missed Non-Nullable columns will be 
> caught and processed by RowAssembler.
> h4. Possible optimization.
> It is possible to add the validation into a TupleBuilder and then just check 
> the Tuple instance class (should be a builder). 
>  For any Tuple of unknown type or if a schema was changed concurrently 
> (TupleBuilder validated input against outdated schema version), then fallback 
> to default logic and re-validate input against the latest schema.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Comment Edited] (IGNITE-13668) Implement Number(n) and Decimal native types

2021-06-11 Thread Andrey Mashenkov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315452#comment-17315452
 ] 

Andrey Mashenkov edited comment on IGNITE-13668 at 6/11/21, 9:16 AM:
-

1. Does it make sense to has real varlen Numerics for {{Number[n]}} where n<8 ?
With a suggested approach, we could save a single byte only in the case of 
Numeric[5], other cases will cost more or equal memory if they are encoded as 
fixlen (short, int, long).
2. Is it possible to serialize decimal into byte[] preserving the ordering to 
avoid serialization/deserialization due to performance reasons. 
This may be very useful for SQL indices.


was (Author: amashenkov):
1. Does it make sense to has real varlen Numerics for {{Number(n)}} where n<8 ?
With a suggested approach, we could save a single byte only in the case of 
Numeric(5), other cases will cost more or equal memory if they are encoded as 
fixlen (short, int, long).
2. Is it possible to serialize decimal into byte[] preserving the ordering to 
avoid serialization/deserialization due to performance reasons. 
This may be very useful for SQL indices.

> Implement Number(n) and Decimal native types
> 
>
> Key: IGNITE-13668
> URL: https://issues.apache.org/jira/browse/IGNITE-13668
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexey Goncharuk
>Priority: Major
>  Labels: iep-54, ignite-3
> Fix For: 3.0.0-alpha3
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Let's extend native support for Numeric types.
> * Number( n ) is an {{n}}-bytes two-complement integer signed value encoded 
> in the varlong style 
> (so that Number(4) can be mapped to integer and Number(8) can be mapped to 
> long during (de)serialization). 
> * Larger numbers can be represented as {{BigInteger}}.
> * The Number( n ) is a varlen type, so it will take two additional bytes in 
> the varlen table, so types smaller than Number(4) are better represented by 
> {{byte}} and {{short}} and {{int}} types as their fixlen encoding takes 
> exactly 1, 2, 4 bytes respectively.
> * Decimal is a direct mapping to BigDecimal value.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-14479) Column default values serialization.

2021-06-11 Thread Andrey Mashenkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Mashenkov updated IGNITE-14479:
--
Description: 
# The Column default value is a part of schema configuration and intended to be 
transferred among nodes, incl. server nodes where can be no user classes.
Thus, the default value MUST be of the natively supported type or a byte[] with 
a serialized value.
# We should be able to write default to a row in raw format (as already 
serialized into byte[])  during a marshaling stage.
# We should always write defaults to Row during a marshaling stage as Row has 
no markers that a default value is stored for the column in the row.


  was:
h3. Motivation.
Let's add default values support that is widely used in SQL.
This will allow a user to work with Table binary views more efficiently 
specifying only a subset of columns in some cases.


h3. Description.
This task is related to Table binary views as well as non-binary views with 
truncated classes.

# The Column default value is a part of schema configuration and intended to be 
transferred among nodes, incl. server nodes where can be no user classes.
Thus, the value MUST be of the natively supported type or a byte[] with a 
serialized value that can be compared as byte[].
# Tuple has no default-value-map support (like null-map), so we should always 
write defaults (specified in the current version of schema) to Tuple during a 
marshaling stage.
Thus, we should be able to answer if a 'null' value was set or a value was not 
set to Tuple and write a default column value to Row explicitly if it was not 
specified in Tuple.
Seems, a Tuple contract needs to be extended.






> Column default values serialization.
> 
>
> Key: IGNITE-14479
> URL: https://issues.apache.org/jira/browse/IGNITE-14479
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Andrey Mashenkov
>Priority: Major
>  Labels: iep-54, ignite-3
> Fix For: 3.0.0-alpha3
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> # The Column default value is a part of schema configuration and intended to 
> be transferred among nodes, incl. server nodes where can be no user classes.
> Thus, the default value MUST be of the natively supported type or a byte[] 
> with a serialized value.
> # We should be able to write default to a row in raw format (as already 
> serialized into byte[])  during a marshaling stage.
> # We should always write defaults to Row during a marshaling stage as Row has 
> no markers that a default value is stored for the column in the row.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-14743) Support Row with large values.

2021-06-11 Thread Andrey Mashenkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Mashenkov updated IGNITE-14743:
--
Description: 
For now, TupleAssembler writes offsets for varlen columns as 2-byte \{{short 
}}type.
 This implicitly restricts key/value sizes down to 64 kB in total.

Let's use 4 byte types (byte, short, int) for offsets.

  was:
For now, TupleAssembler writes offsets for varlen columns as 2-byte \{{short 
}}type.
 This implicitly restricts key/value sizes down to 64 kB in total.

Let's use adaptive 1-2-4 byte types (byte, short, int) for offsets.


> Support Row with large values.
> --
>
> Key: IGNITE-14743
> URL: https://issues.apache.org/jira/browse/IGNITE-14743
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Mashenkov
>Assignee: Andrey Mashenkov
>Priority: Major
>  Labels: iep-54, ignite-3
> Fix For: 3.0.0-alpha3
>
>   Original Estimate: 168h
>  Time Spent: 10m
>  Remaining Estimate: 167h 50m
>
> For now, TupleAssembler writes offsets for varlen columns as 2-byte \{{short 
> }}type.
>  This implicitly restricts key/value sizes down to 64 kB in total.
> Let's use 4 byte types (byte, short, int) for offsets.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-14896) Schema update. Changing default value.

2021-06-11 Thread Andrey Mashenkov (Jira)
Andrey Mashenkov created IGNITE-14896:
-

 Summary: Schema update. Changing default value.
 Key: IGNITE-14896
 URL: https://issues.apache.org/jira/browse/IGNITE-14896
 Project: Ignite
  Issue Type: Improvement
Reporter: Andrey Mashenkov


Implement evolution converters for changing column default value operation.

Configuration changes should trigger a new schema version adding to schema 
history.
Assumes, all nodes will use a new schema after upgrade.
Old row can be upgraded on fly using evolution converters.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-14896) Schema update. Changing default value.

2021-06-11 Thread Andrey Mashenkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Mashenkov updated IGNITE-14896:
--
Labels: iep-54 ignite-3  (was: )

> Schema update. Changing default value.
> --
>
> Key: IGNITE-14896
> URL: https://issues.apache.org/jira/browse/IGNITE-14896
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Mashenkov
>Priority: Major
>  Labels: iep-54, ignite-3
>
> Implement evolution converters for changing column default value operation.
> Configuration changes should trigger a new schema version adding to schema 
> history.
> Assumes, all nodes will use a new schema after upgrade.
> Old row can be upgraded on fly using evolution converters.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-14479) Column default values serialization.

2021-06-11 Thread Andrey Mashenkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Mashenkov updated IGNITE-14479:
--
Summary: Column default values serialization.  (was: Add column default 
values support.)

> Column default values serialization.
> 
>
> Key: IGNITE-14479
> URL: https://issues.apache.org/jira/browse/IGNITE-14479
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Andrey Mashenkov
>Priority: Major
>  Labels: iep-54, ignite-3
> Fix For: 3.0.0-alpha3
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> h3. Motivation.
> Let's add default values support that is widely used in SQL.
> This will allow a user to work with Table binary views more efficiently 
> specifying only a subset of columns in some cases.
> h3. Description.
> This task is related to Table binary views as well as non-binary views with 
> truncated classes.
> # The Column default value is a part of schema configuration and intended to 
> be transferred among nodes, incl. server nodes where can be no user classes.
> Thus, the value MUST be of the natively supported type or a byte[] with a 
> serialized value that can be compared as byte[].
> # Tuple has no default-value-map support (like null-map), so we should always 
> write defaults (specified in the current version of schema) to Tuple during a 
> marshaling stage.
> Thus, we should be able to answer if a 'null' value was set or a value was 
> not set to Tuple and write a default column value to Row explicitly if it was 
> not specified in Tuple.
> Seems, a Tuple contract needs to be extended.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-14895) Replace reflection with method handles in GridUnsage

2021-06-11 Thread Ivan Bessonov (Jira)
Ivan Bessonov created IGNITE-14895:
--

 Summary: Replace reflection with method handles in GridUnsage
 Key: IGNITE-14895
 URL: https://issues.apache.org/jira/browse/IGNITE-14895
 Project: Ignite
  Issue Type: Bug
Affects Versions: 3.0.0-alpha2
Reporter: Ivan Bessonov
Assignee: Ivan Bessonov


Reflective operations are notoriously slow, there's a better way to invoke 
methods dynamically.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13587) Calcite improvements. Append tests for unstable topology checking.

2021-06-11 Thread Stanilovsky Evgeny (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361557#comment-17361557
 ] 

Stanilovsky Evgeny commented on IGNITE-13587:
-

[~korlov] [~tledkov-gridgain] i think this test is still actual, wdyt ? 

> Calcite improvements. Append tests for unstable topology checking.
> --
>
> Key: IGNITE-13587
> URL: https://issues.apache.org/jira/browse/IGNITE-13587
> Project: Ignite
>  Issue Type: Test
>  Components: sql
>Affects Versions: 2.8.1
>Reporter: Stanilovsky Evgeny
>Assignee: Stanilovsky Evgeny
>Priority: Major
>  Labels: calcite
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> After [1] was found in h2 realization we probably need for such kind of tests 
> with calcite.
> [1] https://issues.apache.org/jira/browse/IGNITE-13572 (Duplicates in select 
> query during partition eviction.)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-14854) Fix tests IgniteCacheLocalQueryDefaultTimeoutSelfTest for lazy=true mode

2021-06-11 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-14854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361543#comment-17361543
 ] 

Ignite TC Bot commented on IGNITE-14854:


{panel:title=Branch: [pull/9161/head] Base: [master] : Possible Blockers 
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}RDD{color} [[tests 0 Exit Code 
|https://ci.ignite.apache.org/viewLog.html?buildId=6043943]]

{panel}
{panel:title=Branch: [pull/9161/head] Base: [master] : New Tests 
(72)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Queries 2{color} [[tests 
72|https://ci.ignite.apache.org/viewLog.html?buildId=6041371]]
* {color:#013220}IgniteBinaryCacheQueryTestSuite2: 
DefaultQueryTimeoutThickJavaTest.testConcurrent[lazy=true, update=false, 
local=true] - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite2: 
DefaultQueryTimeoutThickJavaTest.testNoExplicitTimeout3[lazy=true, 
update=false, local=true] - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite2: 
DefaultQueryTimeoutThickJavaTest.testExplicitTimeout2[lazy=true, update=false, 
local=false] - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite2: 
DefaultQueryTimeoutThickJavaTest.testExplicitTimeout1[lazy=true, update=false, 
local=false] - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite2: 
DefaultQueryTimeoutThickJavaTest.testExplicitTimeout4[lazy=true, update=false, 
local=false] - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite2: 
DefaultQueryTimeoutThickJavaTest.testExplicitTimeout3[lazy=true, update=false, 
local=false] - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite2: 
DefaultQueryTimeoutThickJavaTest.testExplicitTimeout6[lazy=true, update=false, 
local=false] - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite2: 
DefaultQueryTimeoutThickJavaTest.testExplicitTimeout5[lazy=true, update=false, 
local=false] - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite2: 
DefaultQueryTimeoutThickJavaTest.testExplicitTimeout4[lazy=true, update=false, 
local=true] - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite2: 
DefaultQueryTimeoutThickJavaTest.testExplicitTimeout3[lazy=true, update=false, 
local=true] - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite2: 
DefaultQueryTimeoutThickJavaTest.testExplicitTimeout6[lazy=true, update=false, 
local=true] - PASSED{color}
... and 61 new tests

{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6041446&buildTypeId=IgniteTests24Java8_RunAll]

> Fix tests IgniteCacheLocalQueryDefaultTimeoutSelfTest for lazy=true mode
> 
>
> Key: IGNITE-14854
> URL: https://issues.apache.org/jira/browse/IGNITE-14854
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Fix For: 2.12
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Tests are not stable:
> {{IgniteCacheLocalQueryDefaultTimeoutSelfTest.testQueryTimeout}}
> {{IgniteCacheLocalQueryDefaultTimeoutSelfTest.testQueryDefaultTimeout}}
> The local runs are OK. So I guess the timeout works OK. Looks like the tests 
> must be stabilized for TC.
> Root cause: the test isn't adopted for lazy mode (results aren't fetched, 
> only iterator is opened)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-14894) Ignite RDD test suite is broken

2021-06-11 Thread Pavel Pereslegin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Pereslegin updated IGNITE-14894:
--
Fix Version/s: 2.11

> Ignite RDD test suite is broken
> ---
>
> Key: IGNITE-14894
> URL: https://issues.apache.org/jira/browse/IGNITE-14894
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.10
>Reporter: Pavel Pereslegin
>Assignee: Pavel Pereslegin
>Priority: Blocker
> Fix For: 2.11
>
>
> RDD starts to fail constantly after IGNITE-14424.
> Stacktrace that may lead to this failure
> {noformat}
> [04:45:12]W:   [org.apache.ignite:ignite-spark] [2021-06-08 
> 04:45:12,601][ERROR][spark-listener-group-shared][IgniteKernal] Failed to 
> pre-stop processor: GridProcessorAdapter []
> [04:45:12]W:   [org.apache.ignite:ignite-spark] 
> java.lang.NullPointerException
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.localJoinExchange(GridDhtPartitionsExchangeFuture.java:2137)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.managers.encryption.GridEncryptionManager.onDoneAfterTopologyUnlock(GridEncryptionManager.java:1118)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:2627)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:157)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:478)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.onKernalStop0(GridCachePartitionExchangeManager.java:1001)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter.onKernalStop(GridCacheSharedManagerAdapter.java:135)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.onKernalStop(GridCacheProcessor.java:824)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2578)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2526)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2864)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2687)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.internal.IgnitionEx.stop(IgnitionEx.java:362)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.Ignition.stop(Ignition.java:230)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.spark.IgniteContext.org$apache$ignite$spark$IgniteContext$$doClose(IgniteContext.scala:186)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.spark.IgniteContext.close(IgniteContext.scala:179)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.ignite.spark.IgniteContext$$anon$1.onApplicationEnd(IgniteContext.scala:69)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.spark.scheduler.SparkListenerBus$class.doPostEvent(SparkListenerBus.scala:57)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.spark.util.ListenerBus$class.postToAll(ListenerBus.scala:82)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.spark.scheduler.AsyncEventQueue.org$apache$spark$scheduler$AsyncEventQueue$$super$postToAll(AsyncEventQueue.scala:89)
> [04:45:12]W:   [org.apache.ignite:ignite-spark]   at 
> org.apache.spark.scheduler.AsyncEventQueue$$anonfun$org$apache$spark$scheduler$AsyncEventQueue$$dispatch$1.apply(AsyncEventQueue.scala:89)
> [04:45:12]W:  

[jira] [Commented] (IGNITE-14120) select count * returns multiple rows

2021-06-11 Thread Vladimir Ermakov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-14120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361538#comment-17361538
 ] 

Vladimir Ermakov commented on IGNITE-14120:
---

My changes are not related to a possible blocker. As I see in teamcity, RDD 
tests will fail almost universally with exit code 130. The problem is mentioned 
in devlist. 

> select count * returns multiple rows
> 
>
> Key: IGNITE-14120
> URL: https://issues.apache.org/jira/browse/IGNITE-14120
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.8.1
>Reporter: Isaac Zhu
>Assignee: Vladimir Ermakov
>Priority: Major
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> I have a partitioned table which has 1 backup, the *queryParallelism* is set 
> to 4.
> The table primary key is column "ID", 
> If I do this query:
>         select count( * ) from my_table where ID = 1000;
> It will return 4 rows:
>         1
>          0
>          0
>          0
>  
> If I query by other not primary-key columns of this table, the result is 
> good, like:
>         select count( *) from my_table where name = 'abcd'
> result is:
>         0



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-14120) select count * returns multiple rows

2021-06-11 Thread Konstantin Orlov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-14120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361534#comment-17361534
 ] 

Konstantin Orlov commented on IGNITE-14120:
---

[~vermakov], LGTM!

[~tledkov-gridgain], could you please have a look too and merge it if 
everything is ok?

> select count * returns multiple rows
> 
>
> Key: IGNITE-14120
> URL: https://issues.apache.org/jira/browse/IGNITE-14120
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.8.1
>Reporter: Isaac Zhu
>Assignee: Vladimir Ermakov
>Priority: Major
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> I have a partitioned table which has 1 backup, the *queryParallelism* is set 
> to 4.
> The table primary key is column "ID", 
> If I do this query:
>         select count( * ) from my_table where ID = 1000;
> It will return 4 rows:
>         1
>          0
>          0
>          0
>  
> If I query by other not primary-key columns of this table, the result is 
> good, like:
>         select count( *) from my_table where name = 'abcd'
> result is:
>         0



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-14894) Ignite RDD test suite is broken

2021-06-11 Thread Pavel Pereslegin (Jira)
Pavel Pereslegin created IGNITE-14894:
-

 Summary: Ignite RDD test suite is broken
 Key: IGNITE-14894
 URL: https://issues.apache.org/jira/browse/IGNITE-14894
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.10
Reporter: Pavel Pereslegin
Assignee: Pavel Pereslegin


RDD starts to fail constantly after IGNITE-14424.

Stacktrace that may lead to this failure
{noformat}
[04:45:12]W: [org.apache.ignite:ignite-spark] [2021-06-08 
04:45:12,601][ERROR][spark-listener-group-shared][IgniteKernal] Failed to 
pre-stop processor: GridProcessorAdapter []
[04:45:12]W: [org.apache.ignite:ignite-spark] 
java.lang.NullPointerException
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.localJoinExchange(GridDhtPartitionsExchangeFuture.java:2137)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.ignite.internal.managers.encryption.GridEncryptionManager.onDoneAfterTopologyUnlock(GridEncryptionManager.java:1118)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:2627)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:157)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:478)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.onKernalStop0(GridCachePartitionExchangeManager.java:1001)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter.onKernalStop(GridCacheSharedManagerAdapter.java:135)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.onKernalStop(GridCacheProcessor.java:824)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2578)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2526)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2864)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2687)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.ignite.internal.IgnitionEx.stop(IgnitionEx.java:362)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.ignite.Ignition.stop(Ignition.java:230)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.ignite.spark.IgniteContext.org$apache$ignite$spark$IgniteContext$$doClose(IgniteContext.scala:186)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.ignite.spark.IgniteContext.close(IgniteContext.scala:179)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.ignite.spark.IgniteContext$$anon$1.onApplicationEnd(IgniteContext.scala:69)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.spark.scheduler.SparkListenerBus$class.doPostEvent(SparkListenerBus.scala:57)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.spark.util.ListenerBus$class.postToAll(ListenerBus.scala:82)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.spark.scheduler.AsyncEventQueue.org$apache$spark$scheduler$AsyncEventQueue$$super$postToAll(AsyncEventQueue.scala:89)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.spark.scheduler.AsyncEventQueue$$anonfun$org$apache$spark$scheduler$AsyncEventQueue$$dispatch$1.apply(AsyncEventQueue.scala:89)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
[04:45:12]W: [org.apache.ignite:ignite-spark]   at 
org.apache.spark.scheduler.AsyncEventQueue.org$apache$spark$scheduler$AsyncEventQueue$$dispatch(AsyncEventQueu

[jira] [Commented] (IGNITE-14808) Calcite. RIGHT|FULL Join operations are lost nulls sort ordering.

2021-06-11 Thread Stanilovsky Evgeny (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-14808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361524#comment-17361524
 ] 

Stanilovsky Evgeny commented on IGNITE-14808:
-

[~tledkov-gridgain] check it plz ?

> Calcite. RIGHT|FULL Join operations are lost nulls sort ordering.
> -
>
> Key: IGNITE-14808
> URL: https://issues.apache.org/jira/browse/IGNITE-14808
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Reporter: Stanilovsky Evgeny
>Assignee: Stanilovsky Evgeny
>Priority: Major
>  Labels: calcite
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> queries like 
> {noformat}
> FROM integers FULL OUTER JOIN integers2 ON integers.i=integers2.k ORDER BY i 
> NULLS FIRST
> {noformat} will bring to erroneous results. Also some tests from /sql/join* 
> have no ordering thus results are not equal to pattern.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13744) Calcite. Use TableSpool for IgniteNestedLoopJoin

2021-06-11 Thread Stanilovsky Evgeny (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361506#comment-17361506
 ] 

Stanilovsky Evgeny commented on IGNITE-13744:
-

[~korlov] [~tledkov-gridgain] ready for review.

> Calcite. Use TableSpool for IgniteNestedLoopJoin
> 
>
> Key: IGNITE-13744
> URL: https://issues.apache.org/jira/browse/IGNITE-13744
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Reporter: Taras Ledkov
>Assignee: Stanilovsky Evgeny
>Priority: Minor
>  Labels: calcite
>
> Now {{NestedLoopJoinNode}} uses internal buffer to save all rows of the right 
> input.
> We have to do refactoring {{IgniteNestedLoopJoin}} to use rewind of the right 
> input and use TableSpool for not rewindable inputs.
> This refactoring separates implementation the join logic from materialization 
> of the right input if it is needed. In the future we can use disk offload for 
> TableSpool etc.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-13744) Calcite. Use TableSpool for IgniteNestedLoopJoin

2021-06-11 Thread Stanilovsky Evgeny (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-13744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stanilovsky Evgeny updated IGNITE-13744:

Ignite Flags:   (was: Docs Required,Release Notes Required)
  Labels: calcite  (was: )

> Calcite. Use TableSpool for IgniteNestedLoopJoin
> 
>
> Key: IGNITE-13744
> URL: https://issues.apache.org/jira/browse/IGNITE-13744
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Reporter: Taras Ledkov
>Assignee: Stanilovsky Evgeny
>Priority: Minor
>  Labels: calcite
>
> Now {{NestedLoopJoinNode}} uses internal buffer to save all rows of the right 
> input.
> We have to do refactoring {{IgniteNestedLoopJoin}} to use rewind of the right 
> input and use TableSpool for not rewindable inputs.
> This refactoring separates implementation the join logic from materialization 
> of the right input if it is needed. In the future we can use disk offload for 
> TableSpool etc.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-13744) Calcite. Use TableSpool for IgniteNestedLoopJoin

2021-06-11 Thread Stanilovsky Evgeny (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-13744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stanilovsky Evgeny updated IGNITE-13744:

Summary: Calcite. Use TableSpool for IgniteNestedLoopJoin  (was: Use 
TableSpool for IgniteNestedLoopJoin)

> Calcite. Use TableSpool for IgniteNestedLoopJoin
> 
>
> Key: IGNITE-13744
> URL: https://issues.apache.org/jira/browse/IGNITE-13744
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Reporter: Taras Ledkov
>Assignee: Stanilovsky Evgeny
>Priority: Minor
>
> Now {{NestedLoopJoinNode}} uses internal buffer to save all rows of the right 
> input.
> We have to do refactoring {{IgniteNestedLoopJoin}} to use rewind of the right 
> input and use TableSpool for not rewindable inputs.
> This refactoring separates implementation the join logic from materialization 
> of the right input if it is needed. In the future we can use disk offload for 
> TableSpool etc.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (IGNITE-13744) Use TableSpool for IgniteNestedLoopJoin

2021-06-11 Thread Stanilovsky Evgeny (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-13744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stanilovsky Evgeny reassigned IGNITE-13744:
---

Assignee: Stanilovsky Evgeny  (was: Taras Ledkov)

> Use TableSpool for IgniteNestedLoopJoin
> ---
>
> Key: IGNITE-13744
> URL: https://issues.apache.org/jira/browse/IGNITE-13744
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Reporter: Taras Ledkov
>Assignee: Stanilovsky Evgeny
>Priority: Minor
>
> Now {{NestedLoopJoinNode}} uses internal buffer to save all rows of the right 
> input.
> We have to do refactoring {{IgniteNestedLoopJoin}} to use rewind of the right 
> input and use TableSpool for not rewindable inputs.
> This refactoring separates implementation the join logic from materialization 
> of the right input if it is needed. In the future we can use disk offload for 
> TableSpool etc.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-14120) select count * returns multiple rows

2021-06-11 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-14120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361502#comment-17361502
 ] 

Ignite TC Bot commented on IGNITE-14120:


{panel:title=Branch: [pull/9164/head] Base: [master] : Possible Blockers 
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}RDD{color} [[tests 0 Exit Code 
|https://ci.ignite.apache.org/viewLog.html?buildId=6043387]]

{panel}
{panel:title=Branch: [pull/9164/head] Base: [master] : New Tests 
(4)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Continuous Query 4{color} [[tests 
4|https://ci.ignite.apache.org/viewLog.html?buildId=6043164]]
* {color:#013220}IgniteCacheQuerySelfTestSuite6: 
IgniteSqlSinglePartitionMultiParallelismTest.assertSimpleCountQuery - 
PASSED{color}
* {color:#013220}IgniteCacheQuerySelfTestSuite6: 
IgniteSqlSinglePartitionMultiParallelismTest.assertWhereCountFirstPartitionQuery
 - PASSED{color}
* {color:#013220}IgniteCacheQuerySelfTestSuite6: 
IgniteSqlSinglePartitionMultiParallelismTest.assertWhereCountMultiPartitionsQuery
 - PASSED{color}
* {color:#013220}IgniteCacheQuerySelfTestSuite6: 
IgniteSqlSinglePartitionMultiParallelismTest.assertWhereCountAnotherPartitionQuery
 - PASSED{color}

{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6043199&buildTypeId=IgniteTests24Java8_RunAll]

> select count * returns multiple rows
> 
>
> Key: IGNITE-14120
> URL: https://issues.apache.org/jira/browse/IGNITE-14120
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.8.1
>Reporter: Isaac Zhu
>Assignee: Vladimir Ermakov
>Priority: Major
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> I have a partitioned table which has 1 backup, the *queryParallelism* is set 
> to 4.
> The table primary key is column "ID", 
> If I do this query:
>         select count( * ) from my_table where ID = 1000;
> It will return 4 rows:
>         1
>          0
>          0
>          0
>  
> If I query by other not primary-key columns of this table, the result is 
> good, like:
>         select count( *) from my_table where name = 'abcd'
> result is:
>         0



--
This message was sent by Atlassian Jira
(v8.3.4#803005)