[jira] [Updated] (IGNITE-12441) Refactor SystemViewRowAttributeWalker registration for system views
[ https://issues.apache.org/jira/browse/IGNITE-12441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-12441: --- Description: Currently, to register system view we need at least to actions: # Register walker for a system view class # Register system view itself for a given class There is a map maintained to bind the attribute walker and system view class. Sometimes walker registration code and system view registration code are located in different parts of Ignite. But in most cases, there is 1:1 relation between attribute walkers and system views. We don't need to maintain additional structures and do additional actions to find a walker by system view class, we can bind walker to system view itself and can do it atomically. Moreover, in the current implementation, there is impossible to use different walkers for one system view class (for example when we want to create two views with the same system view class, but with different columns set) was: Currently, to register system view we need at least to actions: # Register walker for a system view class # Register system view itself for a given class There is a map maintained to bind the attribute walker and system view class. Sometimes walker registration code and system view registration code are located in different parts of Ignite. But in most cases, there is 1:1 relation between attribute walkers and system views. We don't need to maintain additional structures and do additional actions to find a walker by system view class, we can bind walker to system view itself. Moreover, in the current implementation, there is impossible to use different walkers for one system view class (for example when we want to create two views with the same system view class, but with different columns set) > Refactor SystemViewRowAttributeWalker registration for system views > --- > > Key: IGNITE-12441 > URL: https://issues.apache.org/jira/browse/IGNITE-12441 > Project: Ignite > Issue Type: Task >Reporter: Aleksey Plekhanov >Assignee: Aleksey Plekhanov >Priority: Major > Labels: IEP-35 > Time Spent: 10m > Remaining Estimate: 0h > > Currently, to register system view we need at least to actions: > # Register walker for a system view class > # Register system view itself for a given class > There is a map maintained to bind the attribute walker and system view class. > Sometimes walker registration code and system view registration code are > located in different parts of Ignite. > But in most cases, there is 1:1 relation between attribute walkers and system > views. We don't need to maintain additional structures and do additional > actions to find a walker by system view class, we can bind walker to system > view itself and can do it atomically. > Moreover, in the current implementation, there is impossible to use different > walkers for one system view class (for example when we want to create two > views with the same system view class, but with different columns set) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12441) Refactor SystemViewRowAttributeWalker registration for system views
Aleksey Plekhanov created IGNITE-12441: -- Summary: Refactor SystemViewRowAttributeWalker registration for system views Key: IGNITE-12441 URL: https://issues.apache.org/jira/browse/IGNITE-12441 Project: Ignite Issue Type: Task Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov Currently, to register system view we need at least to actions: # Register walker for a system view class # Register system view itself for a given class There is a map maintained to bind the attribute walker and system view class. Sometimes walker registration code and system view registration code are located in different parts of Ignite. But in most cases, there is 1:1 relation between attribute walkers and system views. We don't need to maintain additional structures and do additional actions to find a walker by system view class, we can bind walker to system view itself. Moreover, in the current implementation, there is impossible to use different walkers for one system view class (for example when we want to create two views with the same system view class, but with different columns set) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12424) Fix default query timeout behavior for thin JDBC
[ https://issues.apache.org/jira/browse/IGNITE-12424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995239#comment-16995239 ] Saikat Maitra commented on IGNITE-12424: [~Pavlukhin] Thank you for sharing the details. The changes in PR looks good. Regards, Saikat > Fix default query timeout behavior for thin JDBC > > > Key: IGNITE-12424 > URL: https://issues.apache.org/jira/browse/IGNITE-12424 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Ivan Pavlukhin >Priority: Blocker > Fix For: 2.8 > > > After IGNITE-7285 there appeared a buggy behavior for thin JDBC driver. > Thin JDBC handles explicit query timeout on a client side. Default query > timeout is tracked on a server side. As a server is not aware of explicit > client timeout it is not possible to override a default timeout with longer > explicit timeout (effectively a query will be cancelled after a default > timeout expiration). > The expected behavior is that an explicit query timeout always overrides a > default one. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (IGNITE-12383) [ML] Add more distances between two Vectors
[ https://issues.apache.org/jira/browse/IGNITE-12383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravil Galeyev reassigned IGNITE-12383: -- Assignee: Ravil Galeyev > [ML] Add more distances between two Vectors > --- > > Key: IGNITE-12383 > URL: https://issues.apache.org/jira/browse/IGNITE-12383 > Project: Ignite > Issue Type: Sub-task > Components: ml >Reporter: Alexey Zinoviev >Assignee: Ravil Galeyev >Priority: Minor > Labels: newbie > > Currently, we support > * Hamming > * Manhattan > * Euclidean > Please have a look and propose the next implementations for distance > calculation between two points as > # [http://en.wikipedia.org/wiki/Chebyshev_distance] > # [http://en.wikipedia.org/wiki/Cosine_similarity] > # [http://en.wikipedia.org/wiki/Minkowski_distance] (the common case for > Manhattan and Euclidean) > # [http://en.wikipedia.org/wiki/Jaccard_index] (Tanimoto) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12412) Incomplete check for ABA problem in PageMemoryImpl#PagePool
[
https://issues.apache.org/jira/browse/IGNITE-12412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994916#comment-16994916
]
Alexey Goncharuk commented on IGNITE-12412:
---
Done, the visa is acquired on a separate refactoring + fix PR.
> Incomplete check for ABA problem in PageMemoryImpl#PagePool
> ---
>
> Key: IGNITE-12412
> URL: https://issues.apache.org/jira/browse/IGNITE-12412
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Goncharuk
>Assignee: Alexey Goncharuk
>Priority: Critical
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> In current implementation, {{PagePool#releasePage}} clears the counter part
> of the returned page ID, which effectively disables the ABA check intended in
> the class. This issue can be rarely reproduced on zOS during checkpoints
> (when pages are being taken and returned to the checkpoint pages pool).
> I managed to write a unit-test to reproduce this issue on x86.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12412) Incomplete check for ABA problem in PageMemoryImpl#PagePool
[
https://issues.apache.org/jira/browse/IGNITE-12412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994915#comment-16994915
]
Ignite TC Bot commented on IGNITE-12412:
{panel:title=Branch: [pull/7098/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4826256buildTypeId=IgniteTests24Java8_RunAll]
> Incomplete check for ABA problem in PageMemoryImpl#PagePool
> ---
>
> Key: IGNITE-12412
> URL: https://issues.apache.org/jira/browse/IGNITE-12412
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Goncharuk
>Assignee: Alexey Goncharuk
>Priority: Critical
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> In current implementation, {{PagePool#releasePage}} clears the counter part
> of the returned page ID, which effectively disables the ABA check intended in
> the class. This issue can be rarely reproduced on zOS during checkpoints
> (when pages are being taken and returned to the checkpoint pages pool).
> I managed to write a unit-test to reproduce this issue on x86.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12395) Client nodes fail on SPARC: No session found at TcpCommunicationSpi.createNioSession
[ https://issues.apache.org/jira/browse/IGNITE-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994860#comment-16994860 ] Vyacheslav Koptilin commented on IGNITE-12395: -- Hello [~ilyak], The proposed fix looks good to me. Please proceed with the merge. Thank you! > Client nodes fail on SPARC: No session found at > TcpCommunicationSpi.createNioSession > > > Key: IGNITE-12395 > URL: https://issues.apache.org/jira/browse/IGNITE-12395 > Project: Ignite > Issue Type: Bug > Components: general >Reporter: Ilya Kasnacheev >Assignee: Ilya Kasnacheev >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > This happens when running client nodes from tests with startClient(), since > it does not do optimize() and this causes socket binding issues on Sparc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11815) Get rid of GridTestUtils.retryAssert method.
[ https://issues.apache.org/jira/browse/IGNITE-11815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-11815: -- Priority: Minor (was: Major) > Get rid of GridTestUtils.retryAssert method. > > > Key: IGNITE-11815 > URL: https://issues.apache.org/jira/browse/IGNITE-11815 > Project: Ignite > Issue Type: Test >Reporter: Andrey Mashenkov >Assignee: Diana Iakovleva >Priority: Minor > Labels: MakeTeamcityGreenAgain, newbie > Time Spent: 1h 50m > Remaining Estimate: 0h > > For now we have GridTestUtils.retryAssert() method which runs a closure 'n' > times to check if some invariantes become ok, eventually. > This method catch assertion error (this looks like a very bad idea) and can > print them to log many times even if assertion is acceptable for the moment. > Also, it is possible to miss an assertion is not related to those ones that > closure checks (e.g. assertion error thrown from ignite internals). > Let's replace retryAssert with GridTestUtils.waitForCondition() usage to make > logs clearer and to avoid possible false positive results. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12395) Client nodes fail on SPARC: No session found at TcpCommunicationSpi.createNioSession
[ https://issues.apache.org/jira/browse/IGNITE-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Kasnacheev updated IGNITE-12395: - Reviewer: Vyacheslav Koptilin > Client nodes fail on SPARC: No session found at > TcpCommunicationSpi.createNioSession > > > Key: IGNITE-12395 > URL: https://issues.apache.org/jira/browse/IGNITE-12395 > Project: Ignite > Issue Type: Bug > Components: general >Reporter: Ilya Kasnacheev >Assignee: Ilya Kasnacheev >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > This happens when running client nodes from tests with startClient(), since > it does not do optimize() and this causes socket binding issues on Sparc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12395) Client nodes fail on SPARC: No session found at TcpCommunicationSpi.createNioSession
[ https://issues.apache.org/jira/browse/IGNITE-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994715#comment-16994715 ] Ilya Kasnacheev commented on IGNITE-12395: -- [~slava.koptilin] please review proposed fix. I believe the failure is random. > Client nodes fail on SPARC: No session found at > TcpCommunicationSpi.createNioSession > > > Key: IGNITE-12395 > URL: https://issues.apache.org/jira/browse/IGNITE-12395 > Project: Ignite > Issue Type: Bug > Components: general >Reporter: Ilya Kasnacheev >Assignee: Ilya Kasnacheev >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > This happens when running client nodes from tests with startClient(), since > it does not do optimize() and this causes socket binding issues on Sparc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12395) Client nodes fail on SPARC: No session found at TcpCommunicationSpi.createNioSession
[
https://issues.apache.org/jira/browse/IGNITE-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994712#comment-16994712
]
Ignite TC Bot commented on IGNITE-12395:
{panel:title=Branch: [pull/7112/head] Base: [master] : Possible Blockers
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}PDS (Indexing){color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=4832619]]
* IgnitePdsWithIndexingCoreTestSuite:
IgniteLogicalRecoveryTest.testRecoveryOnJoinToActiveCluster - Test has low fail
rate in base branch 0,0% and is not flaky
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4823548buildTypeId=IgniteTests24Java8_RunAll]
> Client nodes fail on SPARC: No session found at
> TcpCommunicationSpi.createNioSession
>
>
> Key: IGNITE-12395
> URL: https://issues.apache.org/jira/browse/IGNITE-12395
> Project: Ignite
> Issue Type: Bug
> Components: general
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Kasnacheev
>Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> This happens when running client nodes from tests with startClient(), since
> it does not do optimize() and this causes socket binding issues on Sparc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12440) Sensetive data leak in partition release future messages
[ https://issues.apache.org/jira/browse/IGNITE-12440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mirza Aliev updated IGNITE-12440: - Description: There is sensitive data leak in partition release future messages. We should hide sensitive data when option -DIGNITE_TO_STRING_INCLUDE_SENSITIVE=false is used 2019-10-04 06:51:40.328[WARN ]exchange-worker-#161%GRID%GridNodeName%[o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture] Partition release future: PartitionReleaseFuture was: There is sensitive data lick in partition release future messages. We should hide sensitive data when option -DIGNITE_TO_STRING_INCLUDE_SENSITIVE=false is used 2019-10-04 06:51:40.328[WARN ]exchange-worker-#161%GRID%GridNodeName%[o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture] Partition release future: PartitionReleaseFuture > Sensetive data leak in partition release future messages > > > Key: IGNITE-12440 > URL: https://issues.apache.org/jira/browse/IGNITE-12440 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > There is sensitive data leak in partition release future messages. > We should hide sensitive data when option > -DIGNITE_TO_STRING_INCLUDE_SENSITIVE=false is used > 2019-10-04 06:51:40.328[WARN > ]exchange-worker-#161%GRID%GridNodeName%[o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture] > Partition release future: PartitionReleaseFuture -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12440) Sensitive data leak in partition release future messages
[ https://issues.apache.org/jira/browse/IGNITE-12440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mirza Aliev updated IGNITE-12440: - Summary: Sensitive data leak in partition release future messages (was: Sensetive data leak in partition release future messages) > Sensitive data leak in partition release future messages > > > Key: IGNITE-12440 > URL: https://issues.apache.org/jira/browse/IGNITE-12440 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > There is sensitive data leak in partition release future messages. > We should hide sensitive data when option > -DIGNITE_TO_STRING_INCLUDE_SENSITIVE=false is used > 2019-10-04 06:51:40.328[WARN > ]exchange-worker-#161%GRID%GridNodeName%[o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture] > Partition release future: PartitionReleaseFuture -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12440) Sensetive data leak in partition release future messages
[ https://issues.apache.org/jira/browse/IGNITE-12440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Tkalenko updated IGNITE-12440: - Ignite Flags: (was: Docs Required,Release Notes Required) > Sensetive data leak in partition release future messages > > > Key: IGNITE-12440 > URL: https://issues.apache.org/jira/browse/IGNITE-12440 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Fix For: 2.8 > > > There is sensitive data lick in partition release future messages. > We should hide sensitive data when option > -DIGNITE_TO_STRING_INCLUDE_SENSITIVE=false is used > 2019-10-04 06:51:40.328[WARN > ]exchange-worker-#161%GRID%GridNodeName%[o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture] > Partition release future: PartitionReleaseFuture -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12440) Sensetive data leak in partition release future messages
Kirill Tkalenko created IGNITE-12440: Summary: Sensetive data leak in partition release future messages Key: IGNITE-12440 URL: https://issues.apache.org/jira/browse/IGNITE-12440 Project: Ignite Issue Type: Improvement Reporter: Kirill Tkalenko Assignee: Kirill Tkalenko Fix For: 2.8 There is sensitive data lick in partition release future messages. We should hide sensitive data when option -DIGNITE_TO_STRING_INCLUDE_SENSITIVE=false is used 2019-10-04 06:51:40.328[WARN ]exchange-worker-#161%GRID%GridNodeName%[o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture] Partition release future: PartitionReleaseFuture -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12439) More descriptive message in situation of IgniteOutOfMemoryException, warning message if risk of IOOME is found
Sergey Chugunov created IGNITE-12439: Summary: More descriptive message in situation of IgniteOutOfMemoryException, warning message if risk of IOOME is found Key: IGNITE-12439 URL: https://issues.apache.org/jira/browse/IGNITE-12439 Project: Ignite Issue Type: Improvement Reporter: Sergey Chugunov Assignee: Sergey Chugunov Fix For: 2.9 In persistent mode starting many caches in a data region of a small size may lead to IgniteOutOfMemoryException being thrown. The root cause is that each partition requires allocation of one or more metapages that should be stored during checkpoint and cannot be replaced by other types of pages. As a result when too many metapages occupy significant portion of data region's space a request to replace a page in memory (with one on disk) may not be able to find clean page for replacement. In this situation IgniteOutOfMemoryException is thrown. It is not easy to prevent IOOME in general case, but we should provide more descriptive message when the exception is thrown and/or print out warning to logs when too many caches (or one cache with huge number of partitions) are started in the same data region. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12247) [Spark] Add initial support of Spark 2.4
[
https://issues.apache.org/jira/browse/IGNITE-12247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994600#comment-16994600
]
Maxim Muzafarov edited comment on IGNITE-12247 at 12/12/19 12:23 PM:
-
[~zaleslaw] I've fixed the version of the newly added module (changed from
{{2.9.0-SNAPSHOT}} to {{2.8.0-SNAPSHOT}}) for the {{ignite-2.8}} branch.
was (Author: mmuzaf):
[~zaleslaw] I've fixed the version of the newly added module (changed from
{{2.9.0-SNAPSHOT}} to {{2.8.0-SNAPSHOT}}).
> [Spark] Add initial support of Spark 2.4
>
>
> Key: IGNITE-12247
> URL: https://issues.apache.org/jira/browse/IGNITE-12247
> Project: Ignite
> Issue Type: Sub-task
> Components: spark
>Affects Versions: 2.8
>Reporter: Alexey Zinoviev
>Assignee: Alexey Zinoviev
>Priority: Critical
> Labels: await, important
> Fix For: 2.8
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> This solution provides the initial support of spark 2.4 with copied codebase
> from spark and with initial support of ExternalCatalog refactoring
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12247) [Spark] Add initial support of Spark 2.4
[
https://issues.apache.org/jira/browse/IGNITE-12247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994600#comment-16994600
]
Maxim Muzafarov commented on IGNITE-12247:
--
[~zaleslaw] I've fixed the version of the newly added module (changed from
{{2.9.0-SNAPSHOT}} to {{2.8.0-SNAPSHOT}}).
> [Spark] Add initial support of Spark 2.4
>
>
> Key: IGNITE-12247
> URL: https://issues.apache.org/jira/browse/IGNITE-12247
> Project: Ignite
> Issue Type: Sub-task
> Components: spark
>Affects Versions: 2.8
>Reporter: Alexey Zinoviev
>Assignee: Alexey Zinoviev
>Priority: Critical
> Labels: await, important
> Fix For: 2.8
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> This solution provides the initial support of spark 2.4 with copied codebase
> from spark and with initial support of ExternalCatalog refactoring
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12428) Cache configuration cacheLoader error after cfgs serialization changes
[
https://issues.apache.org/jira/browse/IGNITE-12428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Kasnacheev updated IGNITE-12428:
-
Ignite Flags: (was: Release Notes Required)
> Cache configuration cacheLoader error after cfgs serialization changes
> --
>
> Key: IGNITE-12428
> URL: https://issues.apache.org/jira/browse/IGNITE-12428
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.8
>Reporter: Ilya Kasnacheev
>Priority: Blocker
> Attachments: CacheStoreInitializationTest.java
>
>
> The attached test will work on 2.7 but will fail on master
> {code}
> [19:37:44,606][SEVERE][exchange-worker-#43][GridDhtPartitionsExchangeFuture]
> Failed to initialize cache(s) (will try to rollback)
> [exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=1,
> minorTopVer=1], discoEvt=DiscoveryCustomEvent
> [customMsg=DynamicCacheChangeBatch
> [id=18f068bee61-a388ef9b-b1b7-435f-897f-2397f3af9cff, reqs=ArrayList
> [DynamicCacheChangeRequest [cacheName=foo-0, hasCfg=true,
> nodeId=895176bf-6687-4a05-a7b3-3a1bfa81b728, clientStartOnly=false,
> stop=false, destroy=false, disabledAfterStartfalse]],
> exchangeActions=ExchangeActions [startCaches=[foo-0], stopCaches=null,
> startGrps=[foo-0], stopGrps=[], resetParts=null, stateChangeRequest=null],
> startCaches=false], affTopVer=AffinityTopologyVersion [topVer=1,
> minorTopVer=1], super=DiscoveryEvent [evtNode=TcpDiscoveryNode
> [id=895176bf-6687-4a05-a7b3-3a1bfa81b728,
> consistentId=0:0:0:0:0:0:0:1%lo,127.0.0.1,172.17.0.1,192.168.1.2:47500,
> addrs=ArrayList [0:0:0:0:0:0:0:1%lo, 127.0.0.1, 172.17.0.1, 192.168.1.2],
> sockAddrs=HashSet [/172.17.0.1:47500, /0:0:0:0:0:0:0:1%lo:47500,
> /127.0.0.1:47500, /192.168.1.2:47500], discPort=47500, order=1, intOrder=1,
> lastExchangeTime=1575909462941, loc=true, ver=2.7.0#20191209-sha1:,
> isClient=false], topVer=1, nodeId8=895176bf, msg=null,
> type=DISCOVERY_CUSTOM_EVT, tstamp=1575909464589]], nodeId=895176bf,
> evt=DISCOVERY_CUSTOM_EVT],
> caches=[o.a.i.i.processors.cache.ExchangeActions$CacheGroupActionData@76180c2b]]
> class org.apache.ignite.IgniteCheckedException: Cannot enable read-through
> (loader or store is not provided) for cache: foo-0
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validate(GridCacheProcessor.java:609)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.createCacheContext(GridCacheProcessor.java:1616)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheContext(GridCacheProcessor.java:2398)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:2333)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.lambda$prepareStartCaches$55a0e703$1(GridCacheProcessor.java:2208)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.lambda$prepareStartCachesIfPossible$6(GridCacheProcessor.java:2178)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareStartCaches(GridCacheProcessor.java:2205)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareStartCachesIfPossible(GridCacheProcessor.java:2176)
> at
> org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.processCacheStartRequests(CacheAffinitySharedManager.java:953)
> at
> org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:839)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchangeFuture.java:1270)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:793)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:3031)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2880)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
> at java.lang.Thread.run(Thread.java:748)
> javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException:
> Failed to complete exchange process.
> at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1318)
> at
> org.apache.ignite.internal.IgniteKernal.createCache(IgniteKernal.java:2980)
> at
>
[jira] [Commented] (IGNITE-8550) CacheAbstractJdbcStore expects merge to always return 1 but MySQL may also return 2 or 0
[
https://issues.apache.org/jira/browse/IGNITE-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994556#comment-16994556
]
Ilya Kasnacheev commented on IGNITE-8550:
-
We should definitely not write entry to log, btw, since it may contain
sensitive data.
> CacheAbstractJdbcStore expects merge to always return 1 but MySQL may also
> return 2 or 0
>
>
> Key: IGNITE-8550
> URL: https://issues.apache.org/jira/browse/IGNITE-8550
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Reporter: Stanislav Lukyanov
>Assignee: Moldachev Sergey
>Priority: Minor
> Labels: newbie
> Time Spent: 40m
> Remaining Estimate: 0h
>
> CacheAbstractJdbcStore.write attempts to execute a merge update if it is
> available, and expects the merge to always return 1 (as the number of updated
> entries is always 1).
> However, MySQL's `INSERT ... ON DUPLICATE KEY UPDATE`
> (https://dev.mysql.com/doc/refman/8.0/en/insert-on-duplicate.html) may return
> 0 or 2, depending on what was updated:
> {quote}With ON DUPLICATE KEY UPDATE, the affected-rows value per row is 1 if
> the row is inserted as a new row, 2 if an existing row is updated, and 0 if
> an existing row is set to its current values.{quote}
> Because of that, CacheAbstractJdbcStore may report a false warning.
> Need to consider either removing the warning or special-case the MySQL
> dialect to allow to return values other than 1.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12438) Extend communication protocol to establish client-server connection
Alexey Goncharuk created IGNITE-12438: - Summary: Extend communication protocol to establish client-server connection Key: IGNITE-12438 URL: https://issues.apache.org/jira/browse/IGNITE-12438 Project: Ignite Issue Type: Improvement Reporter: Alexey Goncharuk Recently there was quite a lot of questions related to thick clients connectivity issues when the clients are deployed in a k8s pod [1]. The general issue here is clients reporting network address which are not reachable from server nodes. At the same time, the clients can connect to server nodes. An idea of how to fix this is as follows: * Make sure that think clients discovery SPI always maintains a connection to a server node (this should be already implemented) * (Optionally) detect when a client has only one-way connectivity with the server nodes. This part should be investigated. We need this to avoid server nodes attempt to connect to a client and send communication request to the client node faster * When a server attempts to establish a connection with a client, check if client is unreachable or the previous connection attempt failed. If so, send a discovery message to the client to force a client-server connection. In this case, server will be able to send the original message via the newly established connection. [1] https://stackoverflow.com/questions/59192075/ignite-communicationspi-questions-in-paas-environment/59232504 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12424) Fix default query timeout behavior for thin JDBC
[
https://issues.apache.org/jira/browse/IGNITE-12424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994438#comment-16994438
]
Ivan Pavlukhin commented on IGNITE-12424:
-
[~samaitra], in your example, an expected result for me is that query is
aborted after 30s (explicit timeout). If we turn a _default timeout_
effectively to _max allowed timeout_ we will loose the ability to increase
timeouts in special cases. I suppose _max allowed timeout_ can be useful as
well, but it is a slightly different thing. Moreover we already have the logic
"explicit timeout overrides default one" for {{IgniteCache.query}} API. It is
good to be consistent among different APIs.
{quote}
Also, I had another question on SqlFieldsQuery.timeout, Is this also not
available for JDBC client users?
{quote}
If you assume so-called JDBCv2, then we do not support explicit timeout here
(https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/jdbc2/JdbcStatement.java#L294).
But a default timeout feature should work.
> Fix default query timeout behavior for thin JDBC
>
>
> Key: IGNITE-12424
> URL: https://issues.apache.org/jira/browse/IGNITE-12424
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Ivan Pavlukhin
>Priority: Blocker
> Fix For: 2.8
>
>
> After IGNITE-7285 there appeared a buggy behavior for thin JDBC driver.
> Thin JDBC handles explicit query timeout on a client side. Default query
> timeout is tracked on a server side. As a server is not aware of explicit
> client timeout it is not possible to override a default timeout with longer
> explicit timeout (effectively a query will be cancelled after a default
> timeout expiration).
> The expected behavior is that an explicit query timeout always overrides a
> default one.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-9913) Prevent data updates blocking in case of backup BLT server node leave
[ https://issues.apache.org/jira/browse/IGNITE-9913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994406#comment-16994406 ] Alexei Scherbakov edited comment on IGNITE-9913 at 12/12/19 9:27 AM: - [~avinogradov] 1. I've came to a conclusion having rebalanced state calculated on a coordinator is the most robust way to say the grid is rebalanced. Let's keep it. 2. I've left two comments in your PR regarding the change. 3. ok. was (Author: ascherbakov): [~avinogradov] 1. I've came to a conclusion having rebalanced state calculated on coordinator is the most robust way to say the grid is rebalanced. Let's keep it. 2. I've left two comments in your PR regarding the change. 3. ok. > Prevent data updates blocking in case of backup BLT server node leave > - > > Key: IGNITE-9913 > URL: https://issues.apache.org/jira/browse/IGNITE-9913 > Project: Ignite > Issue Type: Improvement > Components: general >Reporter: Ivan Rakov >Assignee: Anton Vinogradov >Priority: Major > Attachments: 9913_yardstick.png, master_yardstick.png > > Time Spent: 9h 10m > Remaining Estimate: 0h > > Ignite cluster performs distributed partition map exchange when any server > node leaves or joins the topology. > Distributed PME blocks all updates and may take a long time. If all > partitions are assigned according to the baseline topology and server node > leaves, there's no actual need to perform distributed PME: every cluster node > is able to recalculate new affinity assigments and partition states locally. > If we'll implement such lightweight PME and handle mapping and lock requests > on new topology version correctly, updates won't be stopped (except updates > of partitions that lost their primary copy). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12247) [Spark] Add initial support of Spark 2.4
[ https://issues.apache.org/jira/browse/IGNITE-12247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-12247: - Labels: await important (was: await) > [Spark] Add initial support of Spark 2.4 > > > Key: IGNITE-12247 > URL: https://issues.apache.org/jira/browse/IGNITE-12247 > Project: Ignite > Issue Type: Sub-task > Components: spark >Affects Versions: 2.8 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Critical > Labels: await, important > Fix For: 2.8 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > This solution provides the initial support of spark 2.4 with copied codebase > from spark and with initial support of ExternalCatalog refactoring -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12054) [Umbrella][Spark] Upgrade Spark module to 2.4
[ https://issues.apache.org/jira/browse/IGNITE-12054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-12054: - Labels: important (was: ) > [Umbrella][Spark] Upgrade Spark module to 2.4 > - > > Key: IGNITE-12054 > URL: https://issues.apache.org/jira/browse/IGNITE-12054 > Project: Ignite > Issue Type: New Feature > Components: spark >Reporter: Denis A. Magda >Assignee: Alexey Zinoviev >Priority: Blocker > Labels: important > Fix For: 2.9 > > Attachments: ignite-spark-patch-new.diff > > Time Spent: 20m > Remaining Estimate: 0h > > Users can't use APIs that are already available in Spark 2.4: > https://stackoverflow.com/questions/57392143/persisting-spark-dataframe-to-ignite > Let's upgrade Spark from 2.3 to 2.4 until we extract the Spark Integration as > a separate module that can support multiple Spark versions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12054) [Umbrella][Spark] Upgrade Spark module to 2.4
[ https://issues.apache.org/jira/browse/IGNITE-12054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994415#comment-16994415 ] Alexey Zinoviev commented on IGNITE-12054: -- The initial support will be presented in 2.8, known issues are open in Jira under this umbrella ticket > [Umbrella][Spark] Upgrade Spark module to 2.4 > - > > Key: IGNITE-12054 > URL: https://issues.apache.org/jira/browse/IGNITE-12054 > Project: Ignite > Issue Type: New Feature > Components: spark >Reporter: Denis A. Magda >Assignee: Alexey Zinoviev >Priority: Blocker > Fix For: 2.9 > > Attachments: ignite-spark-patch-new.diff > > Time Spent: 20m > Remaining Estimate: 0h > > Users can't use APIs that are already available in Spark 2.4: > https://stackoverflow.com/questions/57392143/persisting-spark-dataframe-to-ignite > Let's upgrade Spark from 2.3 to 2.4 until we extract the Spark Integration as > a separate module that can support multiple Spark versions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12054) [Umbrella][Spark] Upgrade Spark module to 2.4
[ https://issues.apache.org/jira/browse/IGNITE-12054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Zinoviev updated IGNITE-12054: - Labels: (was: await important) > [Umbrella][Spark] Upgrade Spark module to 2.4 > - > > Key: IGNITE-12054 > URL: https://issues.apache.org/jira/browse/IGNITE-12054 > Project: Ignite > Issue Type: New Feature > Components: spark >Reporter: Denis A. Magda >Assignee: Alexey Zinoviev >Priority: Blocker > Fix For: 2.9 > > Attachments: ignite-spark-patch-new.diff > > Time Spent: 20m > Remaining Estimate: 0h > > Users can't use APIs that are already available in Spark 2.4: > https://stackoverflow.com/questions/57392143/persisting-spark-dataframe-to-ignite > Let's upgrade Spark from 2.3 to 2.4 until we extract the Spark Integration as > a separate module that can support multiple Spark versions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12054) [Umbrella][Spark] Upgrade Spark module to 2.4
[ https://issues.apache.org/jira/browse/IGNITE-12054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Zinoviev updated IGNITE-12054: - Fix Version/s: (was: 2.8) 2.9 > [Umbrella][Spark] Upgrade Spark module to 2.4 > - > > Key: IGNITE-12054 > URL: https://issues.apache.org/jira/browse/IGNITE-12054 > Project: Ignite > Issue Type: New Feature > Components: spark >Affects Versions: 2.8 >Reporter: Denis A. Magda >Assignee: Alexey Zinoviev >Priority: Blocker > Labels: await, important > Fix For: 2.9 > > Attachments: ignite-spark-patch-new.diff > > Time Spent: 20m > Remaining Estimate: 0h > > Users can't use APIs that are already available in Spark 2.4: > https://stackoverflow.com/questions/57392143/persisting-spark-dataframe-to-ignite > Let's upgrade Spark from 2.3 to 2.4 until we extract the Spark Integration as > a separate module that can support multiple Spark versions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12054) [Umbrella][Spark] Upgrade Spark module to 2.4
[ https://issues.apache.org/jira/browse/IGNITE-12054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Zinoviev updated IGNITE-12054: - Affects Version/s: (was: 2.8) > [Umbrella][Spark] Upgrade Spark module to 2.4 > - > > Key: IGNITE-12054 > URL: https://issues.apache.org/jira/browse/IGNITE-12054 > Project: Ignite > Issue Type: New Feature > Components: spark >Reporter: Denis A. Magda >Assignee: Alexey Zinoviev >Priority: Blocker > Labels: await, important > Fix For: 2.9 > > Attachments: ignite-spark-patch-new.diff > > Time Spent: 20m > Remaining Estimate: 0h > > Users can't use APIs that are already available in Spark 2.4: > https://stackoverflow.com/questions/57392143/persisting-spark-dataframe-to-ignite > Let's upgrade Spark from 2.3 to 2.4 until we extract the Spark Integration as > a separate module that can support multiple Spark versions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12247) [Spark] Add initial support of Spark 2.4
[ https://issues.apache.org/jira/browse/IGNITE-12247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994413#comment-16994413 ] Alexey Zinoviev commented on IGNITE-12247: -- [~shensonj] [~nizhikov] This is issue merged both: master branch and ignite-2.8 > [Spark] Add initial support of Spark 2.4 > > > Key: IGNITE-12247 > URL: https://issues.apache.org/jira/browse/IGNITE-12247 > Project: Ignite > Issue Type: Sub-task > Components: spark >Affects Versions: 2.8 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Critical > Labels: await > Fix For: 2.8 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > This solution provides the initial support of spark 2.4 with copied codebase > from spark and with initial support of ExternalCatalog refactoring -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-9913) Prevent data updates blocking in case of backup BLT server node leave
[ https://issues.apache.org/jira/browse/IGNITE-9913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994406#comment-16994406 ] Alexei Scherbakov commented on IGNITE-9913: --- [~avinogradov] 1. I've came to a conclusion having rebalanced state calculated on coordinator is the most robust way to say the grid is rebalanced. Let's keep it. 2. I've left two comments in your PR regarding the change. 3. ok. > Prevent data updates blocking in case of backup BLT server node leave > - > > Key: IGNITE-9913 > URL: https://issues.apache.org/jira/browse/IGNITE-9913 > Project: Ignite > Issue Type: Improvement > Components: general >Reporter: Ivan Rakov >Assignee: Anton Vinogradov >Priority: Major > Attachments: 9913_yardstick.png, master_yardstick.png > > Time Spent: 9h 10m > Remaining Estimate: 0h > > Ignite cluster performs distributed partition map exchange when any server > node leaves or joins the topology. > Distributed PME blocks all updates and may take a long time. If all > partitions are assigned according to the baseline topology and server node > leaves, there's no actual need to perform distributed PME: every cluster node > is able to recalculate new affinity assigments and partition states locally. > If we'll implement such lightweight PME and handle mapping and lock requests > on new topology version correctly, updates won't be stopped (except updates > of partitions that lost their primary copy). -- This message was sent by Atlassian Jira (v8.3.4#803005)
