[jira] [Created] (IGNITE-13099) .net thin client - node selection on start up incorrect - wrong node chosen in ClientFailoverSocket GetNextSocket
david created IGNITE-13099:
--
Summary: .net thin client - node selection on start up incorrect -
wrong node chosen in ClientFailoverSocket GetNextSocket
Key: IGNITE-13099
URL: https://issues.apache.org/jira/browse/IGNITE-13099
Project: Ignite
Issue Type: Bug
Components: thin client
Affects Versions: 2.8.1, 2.8
Environment: .net core 3.1 / windows server 2019 / c#
Reporter: david
To reproduce
var cfg = new IgniteClientConfiguration()
{
Endpoints = new List()
{
"localhost","ignite01","ignite02","ignite03","ignite04"
}
};
Ignition.StartClient(cfg)
Log show ignite Thin client connecting to "ignite01" and not "localhost" first
- the first elment in the array?
if Port randes are used: eg:
Endpoints = new List()
{
"localhost:10800..10810","ignite01:10800..10810","ignite02:10800..10810" //snip
}
};
logs show connecting to localhost:10801 and localhost:10800 is skipped
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13050) ClusterGroup that is recomputed on topology change
[ https://issues.apache.org/jira/browse/IGNITE-13050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Kasnacheev updated IGNITE-13050: - Component/s: compute > ClusterGroup that is recomputed on topology change > -- > > Key: IGNITE-13050 > URL: https://issues.apache.org/jira/browse/IGNITE-13050 > Project: Ignite > Issue Type: Improvement > Components: compute >Reporter: Ivan Bessonov >Assignee: Ivan Bessonov >Priority: Major > Fix For: 2.9 > > Time Spent: 10m > Remaining Estimate: 0h > > Currently, ClusterGroup comes in two favors: > One is a static set of UUIDs which will not change, second is predicate that > is recomputed over ALL nodes on EVERY operation. This has bitten our client > because recomputing of ClusterGroup happens in tcp-communication thread > clogging it and delaying every operation in cluster. This is a major problem. > It would be nice if there was a ClusterGroup with predicate which would > recompute once per topology affinity change. Bonus points if it precisely > tracks current topology with zero delay or overrun. > Would be nice to upgrade firstNode/lastNode predicates to that mechanism > since now they are static - topology changes but firstNode/lastNode > projections don't, they may point to absent node. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13050) ClusterGroup that is recomputed on topology change
[ https://issues.apache.org/jira/browse/IGNITE-13050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120811#comment-17120811 ] Ilya Kasnacheev commented on IGNITE-13050: -- Thank you for this improvement, I have merged it to master! > ClusterGroup that is recomputed on topology change > -- > > Key: IGNITE-13050 > URL: https://issues.apache.org/jira/browse/IGNITE-13050 > Project: Ignite > Issue Type: Improvement > Components: compute >Reporter: Ivan Bessonov >Assignee: Ivan Bessonov >Priority: Major > Fix For: 2.9 > > Time Spent: 10m > Remaining Estimate: 0h > > Currently, ClusterGroup comes in two favors: > One is a static set of UUIDs which will not change, second is predicate that > is recomputed over ALL nodes on EVERY operation. This has bitten our client > because recomputing of ClusterGroup happens in tcp-communication thread > clogging it and delaying every operation in cluster. This is a major problem. > It would be nice if there was a ClusterGroup with predicate which would > recompute once per topology affinity change. Bonus points if it precisely > tracks current topology with zero delay or overrun. > Would be nice to upgrade firstNode/lastNode predicates to that mechanism > since now they are static - topology changes but firstNode/lastNode > projections don't, they may point to absent node. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (IGNITE-13047) Ignite node must return Ignition#KILL_EXIT_CODE exit code, if node had stopped by StopNodeFailureHandler or StopNodeOrHaltFailureHandler
[ https://issues.apache.org/jira/browse/IGNITE-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Tkalenko reassigned IGNITE-13047: Assignee: Kirill Tkalenko > Ignite node must return Ignition#KILL_EXIT_CODE exit code, if node had > stopped by StopNodeFailureHandler or StopNodeOrHaltFailureHandler > > > Key: IGNITE-13047 > URL: https://issues.apache.org/jira/browse/IGNITE-13047 > Project: Ignite > Issue Type: Improvement >Reporter: Sergey Antonov >Assignee: Kirill Tkalenko >Priority: Major > > Ignite process must be finished with exit code 130 [1] if the node had > stopped by StopNodeFailureHandler or StopNodeOrHaltFailureHanlder. > [1] > https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/Ignition.html#KILL_EXIT_CODE -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12780) Deadlock between db-checkpoint-thread and checkpoint-runner
[
https://issues.apache.org/jira/browse/IGNITE-12780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Goncharuk updated IGNITE-12780:
--
Ignite Flags: (was: Docs Required,Release Notes Required)
> Deadlock between db-checkpoint-thread and checkpoint-runner
> ---
>
> Key: IGNITE-12780
> URL: https://issues.apache.org/jira/browse/IGNITE-12780
> Project: Ignite
> Issue Type: Bug
>Reporter: Vladislav Pyatkov
>Assignee: Anton Kalashnikov
>Priority: Critical
> Labels: MakeTeamcityGreenAgain
> Fix For: 2.9
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Look at this run:
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_PdsIndexing/5121878?buildTab=log&focusLine=3
> {noformat}
> "db-checkpoint-thread-#46926%db.IgniteSequentialNodeCrashRecoveryTest0%"
> #55580 prio=5 os_prio=0 tid=0x7efb2000c800 nid=0x77e waiting on condition
> [0x7eff31add000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:178)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.fillCacheGroupState(GridCacheDatabaseSharedManager.java:4367)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.markCheckpointBegin(GridCacheDatabaseSharedManager.java:4147)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.doCheckpoint(GridCacheDatabaseSharedManager.java:3728)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.body(GridCacheDatabaseSharedManager.java:3617)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
> at java.lang.Thread.run(Thread.java:748)
>
>
> "checkpoint-runner-#46927%db.IgniteSequentialNodeCrashRecoveryTest0%" #55581
> prio=5 os_prio=0 tid=0x7efbd4009000 nid=0x77f waiting on condition
> [0x7eff317da000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xe5c23ed8> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.checkpointReadLock(GridCacheDatabaseSharedManager.java:1645)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.init0(GridCacheOffheapManager.java:1688)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.fullSize(GridCacheOffheapManager.java:2061)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.lambda$fillCacheGroupState$1(GridCacheDatabaseSharedManager.java:4336)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer$$Lambda$565/253081186.run(Unknown
> Source)
> at
> org.apache.ignite.internal.util.IgniteUtils.lambda$wrapIgniteFuture$3(IgniteUtils.java:11392)
> at
> org.apache.ignite.internal.util.IgniteUtils$$Lambda$561/471384364.run(Unknown
> Source)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> I suspect this issue happening due to IgniteSequentialNodeCrashRecoveryTest
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12712) NPE in checkpoint thread
[
https://issues.apache.org/jira/browse/IGNITE-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Goncharuk updated IGNITE-12712:
--
Fix Version/s: 2.9
> NPE in checkpoint thread
>
>
> Key: IGNITE-12712
> URL: https://issues.apache.org/jira/browse/IGNITE-12712
> Project: Ignite
> Issue Type: Bug
>Reporter: Anton Kalashnikov
>Assignee: Anton Kalashnikov
>Priority: Major
> Fix For: 2.9
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> NPE occured in checkpoint thread (rare reproducing):
> {noformat}
> [2019-11-04 20:54:58,018][INFO ][sys-#50][GridDhtPartitionsExchangeFuture]
> Received full message, will finish exchange
> [node=1784645d-3bef-44fe-8288-e0c16202f5e3, resVer=AffinityTopologyVersion
> [topVer=4, minorTopVer=9]]
> [2019-11-04 20:54:58,023][INFO ][sys-#50][GridDhtPartitionsExchangeFuture]
> Finish exchange future [startVer=AffinityTopologyVersion [topVer=4,
> minorTopVer=9], resVer=AffinityTopologyVersion [topVer=4, minorTopVer=9],
> err=null]
> [2019-11-04 20:54:58,029][INFO ][sys-#50][GridCacheProcessor] Finish proxy
> initialization, cacheName=SQL_PUBLIC_T8,
> localNodeId=5b153e14-70f2-4408-a125-584752532ebd
> [2019-11-04 20:54:58,030][INFO ][sys-#50][GridDhtPartitionsExchangeFuture]
> Completed partition exchange [localNode=5b153e14-70f2-4408-a125-584752532ebd,
> exchange=GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion
> [topVer=4, minorTopVer=9], evt=DISCOVERY_CUSTOM_EVT, evtNode=TcpDiscoveryNode
> [id=1784645d-3bef-44fe-8288-e0c16202f5e3, consistentId=1, addrs=ArrayList
> [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1,
> intOrder=1, lastExchangeTime=1572890071469, loc=false,
> ver=8.7.8#20191101-sha1:e344ed04, isClient=false], done=true,
> newCrdFut=null], topVer=AffinityTopologyVersion [topVer=4, minorTopVer=9]]
> [2019-11-04 20:54:58,030][INFO ][sys-#50][GridDhtPartitionsExchangeFuture]
> Exchange timings [startVer=AffinityTopologyVersion [topVer=4, minorTopVer=9],
> resVer=AffinityTopologyVersion [topVer=4, minorTopVer=9], stage="Waiting in
> exchange queue" (0 ms), stage="Exchange parameters initialization" (0 ms),
> stage="Update caches registry" (0 ms), stage="Start caches" (52 ms),
> stage="Affinity initialization on cache group start" (1 ms), stage="Determine
> exchange type" (0 ms), stage="Preloading notification" (0 ms), stage="WAL
> history reservation" (0 ms), stage="Wait partitions release" (1 ms),
> stage="Wait partitions release latch" (5 ms), stage="Wait partitions release"
> (0 ms), stage="Restore partition states" (7 ms), stage="After states restored
> callback" (10 ms), stage="Waiting for Full message" (59 ms), stage="Affinity
> recalculation" (0 ms), stage="Full map updating" (4 ms), stage="Exchange
> done" (7 ms), stage="Total time" (146 ms)]
> [2019-11-04 20:54:58,030][INFO ][sys-#50][GridDhtPartitionsExchangeFuture]
> Exchange longest local stages [startVer=AffinityTopologyVersion [topVer=4,
> minorTopVer=9], resVer=AffinityTopologyVersion [topVer=4, minorTopVer=9],
> stage="Affinity initialization on cache group start [grp=SQL_PUBLIC_T8]" (1
> ms) (parent=Affinity initialization on cache group start), stage="Restore
> partition states [grp=SQL_PUBLIC_T8]" (6 ms) (parent=Restore partition
> states), stage="Restore partition states [grp=ignite-sys-cache]" (3 ms)
> (parent=Restore partition states), stage="Restore partition states
> [grp=cache_group_3]" (0 ms) (parent=Restore partition states)]
> [2019-11-04 20:54:58,037][INFO
> ][exchange-worker-#45][GridCachePartitionExchangeManager] Skipping
> rebalancing (nothing scheduled) [top=AffinityTopologyVersion [topVer=4,
> minorTopVer=9], force=false, evt=DISCOVERY_CUSTOM_EVT,
> node=1784645d-3bef-44fe-8288-e0c16202f5e3]
> [2019-11-04 20:54:58,713][INFO
> ][db-checkpoint-thread-#53][GridCacheDatabaseSharedManager] Checkpoint
> started [checkpointId=82969270-b1a5-4480-9513-3af65bab0e17,
> startPtr=FileWALPointer [idx=0, fileOff=3550077, len=12350],
> checkpointBeforeLockTime=8ms, checkpointLockWait=4ms,
> checkpointListenersExecuteTime=56ms, checkpointLockHoldTime=61ms,
> walCpRecordFsyncDuration=4ms, writeCheckpointEntryDuration=8ms,
> splitAndSortCpPagesDuration=1ms, pages=178, reason='timeout']
> [2019-11-04 20:54:58,914][INFO ][exchange-worker-#45][time] Started exchange
> init [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=10], crd=false,
> evt=DISCOVERY_CUSTOM_EVT, evtNode=1784645d-3bef-44fe-8288-e0c16202f5e3,
> customEvt=DynamicCacheChangeBatch
> [id=8b06d873e61-af9e27a6-8fe9-4da1-bc0a-d19cd0eabd36, reqs=ArrayList
> [DynamicCacheChangeRequest [cacheName=SQL_PUBLIC_T9, hasCfg=true,
> nodeId=1784645d-3bef-44fe-8288-e0c16202f5e3, clientStartOnly=false,
> stop=false, destroy=false, disabledAfterStartfalse]
[jira] [Updated] (IGNITE-12712) NPE in checkpoint thread
[
https://issues.apache.org/jira/browse/IGNITE-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Goncharuk updated IGNITE-12712:
--
Ignite Flags: (was: Docs Required,Release Notes Required)
> NPE in checkpoint thread
>
>
> Key: IGNITE-12712
> URL: https://issues.apache.org/jira/browse/IGNITE-12712
> Project: Ignite
> Issue Type: Bug
>Reporter: Anton Kalashnikov
>Assignee: Anton Kalashnikov
>Priority: Major
> Fix For: 2.9
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> NPE occured in checkpoint thread (rare reproducing):
> {noformat}
> [2019-11-04 20:54:58,018][INFO ][sys-#50][GridDhtPartitionsExchangeFuture]
> Received full message, will finish exchange
> [node=1784645d-3bef-44fe-8288-e0c16202f5e3, resVer=AffinityTopologyVersion
> [topVer=4, minorTopVer=9]]
> [2019-11-04 20:54:58,023][INFO ][sys-#50][GridDhtPartitionsExchangeFuture]
> Finish exchange future [startVer=AffinityTopologyVersion [topVer=4,
> minorTopVer=9], resVer=AffinityTopologyVersion [topVer=4, minorTopVer=9],
> err=null]
> [2019-11-04 20:54:58,029][INFO ][sys-#50][GridCacheProcessor] Finish proxy
> initialization, cacheName=SQL_PUBLIC_T8,
> localNodeId=5b153e14-70f2-4408-a125-584752532ebd
> [2019-11-04 20:54:58,030][INFO ][sys-#50][GridDhtPartitionsExchangeFuture]
> Completed partition exchange [localNode=5b153e14-70f2-4408-a125-584752532ebd,
> exchange=GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion
> [topVer=4, minorTopVer=9], evt=DISCOVERY_CUSTOM_EVT, evtNode=TcpDiscoveryNode
> [id=1784645d-3bef-44fe-8288-e0c16202f5e3, consistentId=1, addrs=ArrayList
> [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1,
> intOrder=1, lastExchangeTime=1572890071469, loc=false,
> ver=8.7.8#20191101-sha1:e344ed04, isClient=false], done=true,
> newCrdFut=null], topVer=AffinityTopologyVersion [topVer=4, minorTopVer=9]]
> [2019-11-04 20:54:58,030][INFO ][sys-#50][GridDhtPartitionsExchangeFuture]
> Exchange timings [startVer=AffinityTopologyVersion [topVer=4, minorTopVer=9],
> resVer=AffinityTopologyVersion [topVer=4, minorTopVer=9], stage="Waiting in
> exchange queue" (0 ms), stage="Exchange parameters initialization" (0 ms),
> stage="Update caches registry" (0 ms), stage="Start caches" (52 ms),
> stage="Affinity initialization on cache group start" (1 ms), stage="Determine
> exchange type" (0 ms), stage="Preloading notification" (0 ms), stage="WAL
> history reservation" (0 ms), stage="Wait partitions release" (1 ms),
> stage="Wait partitions release latch" (5 ms), stage="Wait partitions release"
> (0 ms), stage="Restore partition states" (7 ms), stage="After states restored
> callback" (10 ms), stage="Waiting for Full message" (59 ms), stage="Affinity
> recalculation" (0 ms), stage="Full map updating" (4 ms), stage="Exchange
> done" (7 ms), stage="Total time" (146 ms)]
> [2019-11-04 20:54:58,030][INFO ][sys-#50][GridDhtPartitionsExchangeFuture]
> Exchange longest local stages [startVer=AffinityTopologyVersion [topVer=4,
> minorTopVer=9], resVer=AffinityTopologyVersion [topVer=4, minorTopVer=9],
> stage="Affinity initialization on cache group start [grp=SQL_PUBLIC_T8]" (1
> ms) (parent=Affinity initialization on cache group start), stage="Restore
> partition states [grp=SQL_PUBLIC_T8]" (6 ms) (parent=Restore partition
> states), stage="Restore partition states [grp=ignite-sys-cache]" (3 ms)
> (parent=Restore partition states), stage="Restore partition states
> [grp=cache_group_3]" (0 ms) (parent=Restore partition states)]
> [2019-11-04 20:54:58,037][INFO
> ][exchange-worker-#45][GridCachePartitionExchangeManager] Skipping
> rebalancing (nothing scheduled) [top=AffinityTopologyVersion [topVer=4,
> minorTopVer=9], force=false, evt=DISCOVERY_CUSTOM_EVT,
> node=1784645d-3bef-44fe-8288-e0c16202f5e3]
> [2019-11-04 20:54:58,713][INFO
> ][db-checkpoint-thread-#53][GridCacheDatabaseSharedManager] Checkpoint
> started [checkpointId=82969270-b1a5-4480-9513-3af65bab0e17,
> startPtr=FileWALPointer [idx=0, fileOff=3550077, len=12350],
> checkpointBeforeLockTime=8ms, checkpointLockWait=4ms,
> checkpointListenersExecuteTime=56ms, checkpointLockHoldTime=61ms,
> walCpRecordFsyncDuration=4ms, writeCheckpointEntryDuration=8ms,
> splitAndSortCpPagesDuration=1ms, pages=178, reason='timeout']
> [2019-11-04 20:54:58,914][INFO ][exchange-worker-#45][time] Started exchange
> init [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=10], crd=false,
> evt=DISCOVERY_CUSTOM_EVT, evtNode=1784645d-3bef-44fe-8288-e0c16202f5e3,
> customEvt=DynamicCacheChangeBatch
> [id=8b06d873e61-af9e27a6-8fe9-4da1-bc0a-d19cd0eabd36, reqs=ArrayList
> [DynamicCacheChangeRequest [cacheName=SQL_PUBLIC_T9, hasCfg=true,
> nodeId=1784645d-3bef-44fe-8288-e0c16202f5e3, clientStartOnly=false,
> stop=false
[jira] [Updated] (IGNITE-13100) ClassCastException in cache group metrics on client nodes
[
https://issues.apache.org/jira/browse/IGNITE-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Goncharuk updated IGNITE-13100:
--
Fix Version/s: 2.9
> ClassCastException in cache group metrics on client nodes
> -
>
> Key: IGNITE-13100
> URL: https://issues.apache.org/jira/browse/IGNITE-13100
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexey Goncharuk
>Assignee: Alexey Goncharuk
>Priority: Major
> Fix For: 2.9
>
>
> The following exception can be observed when reading cache group metrics on
> client nodes with persistence-enabled config:
> {code}
> java.lang.ClassCastException:
> org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager
> cannot be cast to
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager
> at
> org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.database(CacheGroupMetricsImpl.java:506)
> at
> org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.lambda$new$0(CacheGroupMetricsImpl.java:103)
> at
> org.apache.ignite.internal.util.lang.GridFunc.lambda$nonThrowableSupplier$3(GridFunc.java:3341)
> at
> org.apache.ignite.internal.processors.metric.impl.LongGauge.value(LongGauge.java:45)
> at
> org.apache.ignite.spi.metric.LongMetric.getAsString(LongMetric.java:29)
> {code}
> The reason is an incomplete check for persistence enabled in
> {{CacheGroupMetricsImpl}}: we should also check for client nodes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13100) ClassCastException in cache group metrics on client nodes
[
https://issues.apache.org/jira/browse/IGNITE-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Goncharuk updated IGNITE-13100:
--
Ignite Flags: (was: Docs Required,Release Notes Required)
> ClassCastException in cache group metrics on client nodes
> -
>
> Key: IGNITE-13100
> URL: https://issues.apache.org/jira/browse/IGNITE-13100
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexey Goncharuk
>Assignee: Alexey Goncharuk
>Priority: Major
>
> The following exception can be observed when reading cache group metrics on
> client nodes with persistence-enabled config:
> {code}
> java.lang.ClassCastException:
> org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager
> cannot be cast to
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager
> at
> org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.database(CacheGroupMetricsImpl.java:506)
> at
> org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.lambda$new$0(CacheGroupMetricsImpl.java:103)
> at
> org.apache.ignite.internal.util.lang.GridFunc.lambda$nonThrowableSupplier$3(GridFunc.java:3341)
> at
> org.apache.ignite.internal.processors.metric.impl.LongGauge.value(LongGauge.java:45)
> at
> org.apache.ignite.spi.metric.LongMetric.getAsString(LongMetric.java:29)
> {code}
> The reason is an incomplete check for persistence enabled in
> {{CacheGroupMetricsImpl}}: we should also check for client nodes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (IGNITE-13100) ClassCastException in cache group metrics on client nodes
[
https://issues.apache.org/jira/browse/IGNITE-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Goncharuk reassigned IGNITE-13100:
-
Assignee: Alexey Goncharuk
> ClassCastException in cache group metrics on client nodes
> -
>
> Key: IGNITE-13100
> URL: https://issues.apache.org/jira/browse/IGNITE-13100
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexey Goncharuk
>Assignee: Alexey Goncharuk
>Priority: Major
>
> The following exception can be observed when reading cache group metrics on
> client nodes with persistence-enabled config:
> {code}
> java.lang.ClassCastException:
> org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager
> cannot be cast to
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager
> at
> org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.database(CacheGroupMetricsImpl.java:506)
> at
> org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.lambda$new$0(CacheGroupMetricsImpl.java:103)
> at
> org.apache.ignite.internal.util.lang.GridFunc.lambda$nonThrowableSupplier$3(GridFunc.java:3341)
> at
> org.apache.ignite.internal.processors.metric.impl.LongGauge.value(LongGauge.java:45)
> at
> org.apache.ignite.spi.metric.LongMetric.getAsString(LongMetric.java:29)
> {code}
> The reason is an incomplete check for persistence enabled in
> {{CacheGroupMetricsImpl}}: we should also check for client nodes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13100) ClassCastException in cache group metrics on client nodes
Alexey Goncharuk created IGNITE-13100:
-
Summary: ClassCastException in cache group metrics on client nodes
Key: IGNITE-13100
URL: https://issues.apache.org/jira/browse/IGNITE-13100
Project: Ignite
Issue Type: Bug
Reporter: Alexey Goncharuk
The following exception can be observed when reading cache group metrics on
client nodes with persistence-enabled config:
{code}
java.lang.ClassCastException:
org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager
cannot be cast to
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager
at
org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.database(CacheGroupMetricsImpl.java:506)
at
org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.lambda$new$0(CacheGroupMetricsImpl.java:103)
at
org.apache.ignite.internal.util.lang.GridFunc.lambda$nonThrowableSupplier$3(GridFunc.java:3341)
at
org.apache.ignite.internal.processors.metric.impl.LongGauge.value(LongGauge.java:45)
at
org.apache.ignite.spi.metric.LongMetric.getAsString(LongMetric.java:29)
{code}
The reason is an incomplete check for persistence enabled in
{{CacheGroupMetricsImpl}}: we should also check for client nodes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-13094) java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
[
https://issues.apache.org/jira/browse/IGNITE-13094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120980#comment-17120980
]
Ivan Pavlukhin commented on IGNITE-13094:
-
[~AK47Sonic], lombok _provided_ scope in pom.xml looks suspicious to me. Can we
be really sure that logback is on classpath?
> java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
>
>
> Key: IGNITE-13094
> URL: https://issues.apache.org/jira/browse/IGNITE-13094
> Project: Ignite
> Issue Type: Bug
> Components: compute
>Affects Versions: 2.8.1
> Environment: Ignite 2.8.0
> Spring Boot 2.1.0.RELEASE
> Java 1.8
>Reporter: AK47Sonic
>Priority: Major
>
> I try to use spring boot (2.1.0.RELEASE) to run ignite.compute. As spring
> boot + lombok use logback as default, the following codes always throw
> exceptions (2020-05-28 07:00:48.979 WARN 8460 --- [ p2p-#49]
> o.a.i.i.m.d.GridDeploymentCommunication : Failed to resolve class:
> ch.qos.logback). The result (28) could be returned. Seems we see log4j and
> log4j2 on official website. I try to use log4j2 to replace logback. And it
> works fine. Does ignite support logback for distributed compute? Or It is
> just a bug?
> {panel:title=pom.xml}
>
>
> org.springframework.boot
> spring-boot-starter-web
>
>
> org.springframework.boot
> spring-boot-starter-log4j
>
>
>
>
> org.apache.ignite
> ignite-core
> 2.8.0
>
>
> org.apache.ignite
> ignite-spring
> 2.8.0
>
>
> org.projectlombok
> lombok
> 1.18.12
> provided
>
>
> {panel}
> {panel:title=config.xml}
>
> http://www.springframework.org/schema/beans";
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>xmlns:util="http://www.springframework.org/schema/util";
>xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd
> http://www.springframework.org/schema/util
> http://www.springframework.org/schema/util/spring-util.xsd";>
> class="org.apache.ignite.configuration.IgniteConfiguration">
>
>
>
>
>
>
>
> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>
>
> 127.0.0.1:48500..48520
>
>
>
>
>
>
>
> class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
>
>
>
>
>
> {panel}
> {code:java}
> @Configuration
> @ImportResource(locations={"classpath:default-config.xml"})
> public class IgniteConfig {
> @Bean
> public Ignite ignite(IgniteConfiguration igniteConfiguration) {
> Ignite ignite = Ignition.start(igniteConfiguration);
> Ignition.setClientMode(true);
> return ignite;
> }
> }
> @Slf4j
> @RestController
> public class WebController {
> @Autowired
> private Ignite ignite;
> @GetMapping("/compute")
> public void compute() {
> Collection> calls = new ArrayList<>();
> // Iterate through all the words in the sentence and create Callable
> jobs.
> for (final String word : "Count characters using callable".split(" "))
> calls.add(word::length);
> // Execute collection of Callables on the grid.
> Collection res = ignite.compute().call(calls);
> // Add up all the results.
> int sum = res.stream().mapToInt(Integer::intValue).sum();
> log.info("Total number of characters is '" + sum + "'.");
> }
> }
> {code}
> {panel:title=Exception}
> 2020-05-28 07:00:48.786 INFO 8460 --- [nio-8080-exec-1]
> o.a.i.i.m.d.GridDeploymentLocalStore : Class locally deployed: class
> com.sonic.sample.springboot.controller.WebController
> 2020-05-28 07:00:48.890 WARN 8460 --- [p2p-#49]
> o.a.i.i.m.d.GridDeploymentCommunication : Failed to resolve class:
> ch.qos.logback
> java.lang.ClassNotFoundException: ch.qos.logback
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> ~[na:1.8.0_121]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_121]
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> ~[na:1.8.0_121]
> at java.lang
[jira] [Commented] (IGNITE-13094) java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
[
https://issues.apache.org/jira/browse/IGNITE-13094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121006#comment-17121006
]
AK47Sonic commented on IGNITE-13094:
Hi Pavlukhin,
I removed the provided scope, but that was the same exception.
I also submitted my codes into github. You can pull, run and find the issue.
(https://github.com/AK47Sonic/ignite_demo)
1. Run com.sonic.sample.springboot.WebBootstrap
2. Access http://localhost:8080/compute
> java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
>
>
> Key: IGNITE-13094
> URL: https://issues.apache.org/jira/browse/IGNITE-13094
> Project: Ignite
> Issue Type: Bug
> Components: compute
>Affects Versions: 2.8.1
> Environment: Ignite 2.8.0
> Spring Boot 2.1.0.RELEASE
> Java 1.8
>Reporter: AK47Sonic
>Priority: Major
>
> I try to use spring boot (2.1.0.RELEASE) to run ignite.compute. As spring
> boot + lombok use logback as default, the following codes always throw
> exceptions (2020-05-28 07:00:48.979 WARN 8460 --- [ p2p-#49]
> o.a.i.i.m.d.GridDeploymentCommunication : Failed to resolve class:
> ch.qos.logback). The result (28) could be returned. Seems we see log4j and
> log4j2 on official website. I try to use log4j2 to replace logback. And it
> works fine. Does ignite support logback for distributed compute? Or It is
> just a bug?
> {panel:title=pom.xml}
>
>
> org.springframework.boot
> spring-boot-starter-web
>
>
> org.springframework.boot
> spring-boot-starter-log4j
>
>
>
>
> org.apache.ignite
> ignite-core
> 2.8.0
>
>
> org.apache.ignite
> ignite-spring
> 2.8.0
>
>
> org.projectlombok
> lombok
> 1.18.12
> provided
>
>
> {panel}
> {panel:title=config.xml}
>
> http://www.springframework.org/schema/beans";
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>xmlns:util="http://www.springframework.org/schema/util";
>xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd
> http://www.springframework.org/schema/util
> http://www.springframework.org/schema/util/spring-util.xsd";>
> class="org.apache.ignite.configuration.IgniteConfiguration">
>
>
>
>
>
>
>
> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>
>
> 127.0.0.1:48500..48520
>
>
>
>
>
>
>
> class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
>
>
>
>
>
> {panel}
> {code:java}
> @Configuration
> @ImportResource(locations={"classpath:default-config.xml"})
> public class IgniteConfig {
> @Bean
> public Ignite ignite(IgniteConfiguration igniteConfiguration) {
> Ignite ignite = Ignition.start(igniteConfiguration);
> Ignition.setClientMode(true);
> return ignite;
> }
> }
> @Slf4j
> @RestController
> public class WebController {
> @Autowired
> private Ignite ignite;
> @GetMapping("/compute")
> public void compute() {
> Collection> calls = new ArrayList<>();
> // Iterate through all the words in the sentence and create Callable
> jobs.
> for (final String word : "Count characters using callable".split(" "))
> calls.add(word::length);
> // Execute collection of Callables on the grid.
> Collection res = ignite.compute().call(calls);
> // Add up all the results.
> int sum = res.stream().mapToInt(Integer::intValue).sum();
> log.info("Total number of characters is '" + sum + "'.");
> }
> }
> {code}
> {panel:title=Exception}
> 2020-05-28 07:00:48.786 INFO 8460 --- [nio-8080-exec-1]
> o.a.i.i.m.d.GridDeploymentLocalStore : Class locally deployed: class
> com.sonic.sample.springboot.controller.WebController
> 2020-05-28 07:00:48.890 WARN 8460 --- [p2p-#49]
> o.a.i.i.m.d.GridDeploymentCommunication : Failed to resolve class:
> ch.qos.logback
> java.lang.ClassNotFoundException: ch.qos.logback
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> ~[na:1.8.0_121]
> at java.lang.ClassLoader.loa
[jira] [Assigned] (IGNITE-13026) Calcite integration. Fix metadata for IgniteTable when it has embedded filters.
[
https://issues.apache.org/jira/browse/IGNITE-13026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Taras Ledkov reassigned IGNITE-13026:
-
Assignee: Konstantin Orlov (was: Taras Ledkov)
> Calcite integration. Fix metadata for IgniteTable when it has embedded
> filters.
> ---
>
> Key: IGNITE-13026
> URL: https://issues.apache.org/jira/browse/IGNITE-13026
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Roman Kondakov
>Assignee: Konstantin Orlov
>Priority: Major
>
> By default {{RelMdRowCount}} return the actual row count for
> {{IgniteTables}}. But current implementation of {{IgniteTableScan}} may
> contain embedded filters and rows that are filtered out by these filters are
> not taken into account by the metadata subsystem.
> For example, table emps contains 1000 rows. Filter name='Vasya' filters out
> 95% of rows. For now we end up with situation
> {code:java}
> IgniteFilter(name='Vasya') <- estimated cardinality = 50 rows
> IgniteTableScan(filters=null) <- estimated cardinality = 1000 rows.
> {code}
> but after merging {{Filter}} into {{Scan}} we get the wrong estimation
> {code:java}
> IgniteTableScan(filters={name='Vasya'}) <- estimated cardinality = 1000 rows.
> But should be 50 rows.
> {code}
> We need to add a special case in order to compute the actual cardinality
> returned by {{IgniteTableScan}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (IGNITE-13025) Calcite integration. Limit and offset support.
[
https://issues.apache.org/jira/browse/IGNITE-13025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Taras Ledkov reassigned IGNITE-13025:
-
Assignee: Taras Ledkov
> Calcite integration. Limit and offset support.
> --
>
> Key: IGNITE-13025
> URL: https://issues.apache.org/jira/browse/IGNITE-13025
> Project: Ignite
> Issue Type: New Feature
> Components: sql
>Reporter: Roman Kondakov
>Assignee: Taras Ledkov
>Priority: Major
>
> We can exploit {{Sort}} operator for this purpose or introduce a new operator
> {{Limit}} by analogy with {{EnumerableLimit}}. Some research is needed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (IGNITE-13021) Calcite integration. Avoid full scans for disjunctive queries.
[
https://issues.apache.org/jira/browse/IGNITE-13021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yury Gerzhedovich reassigned IGNITE-13021:
--
Assignee: Andrey Mashenkov
> Calcite integration. Avoid full scans for disjunctive queries.
> --
>
> Key: IGNITE-13021
> URL: https://issues.apache.org/jira/browse/IGNITE-13021
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Assignee: Andrey Mashenkov
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently a full table scan will be executed in the case of disjunctive
> predicate even if predicate fields are indexed. For example:
> {code:java}
> SELECT * FROM emps WHERE name='A' OR surname='B'
> {code}
> This is caused by the nature of indexes: they can return cursor bounded by
> lower and upper bounds. We can cope with it by implementing a logical rule
> for rewriting {{OR}} query to a {{UNION ALL}} query:
> {code:java}
> SELECT * FROM emps WHERE name='A'
> UNION ALL
> SELECT * FROM emps WHERE surname='B' AND LNNVL(name='A')
> {code}
> where {{LNNVL()}} function has semantics
> {code:java}
> LNNVL(name='A') == name!='A' OR name=NULL.
> {code}
> It is used to avoid expensive deduplication. This name is taken from Oracle,
> we can think of more meaningful name, or find the analog in Calcite or H2.
> See, for example, this blog post:
> [https://blogs.oracle.com/optimizer/optimizer-transformations:-or-expansion]
> for details.
> Also it is needed to check this works for {{IN}} clause with small number of
> literals (AFAIK Calcite converts large {{IN}} clauses to a join with
> {{Values}} table where N > 20).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13101) Metastore may leave uncompleted write futures during node stop
Alexey Goncharuk created IGNITE-13101:
-
Summary: Metastore may leave uncompleted write futures during node
stop
Key: IGNITE-13101
URL: https://issues.apache.org/jira/browse/IGNITE-13101
Project: Ignite
Issue Type: Bug
Reporter: Alexey Goncharuk
I've got the following thread-dump (only relevant parts are retained) during
one of the teamcity runs:
{code}
"sys-#103862%baseline.IgniteStableBaselineBinObjFieldsQuerySelfTest0%" #107048
prio=5 os_prio=0 tid=0x7fa2d8009800 nid=0x480d waiting on condition
[0x7fa1d1cdc000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:178)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141)
at
org.apache.ignite.internal.processors.metric.GridMetricManager.remove(GridMetricManager.java:411)
at
org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.remove(CacheGroupMetricsImpl.java:497)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.cleanup(GridCacheProcessor.java:512)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.stopCacheGroup(GridCacheProcessor.java:2901)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.stopCacheGroup(GridCacheProcessor.java:2889)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.processCacheStopRequestOnExchangeDone(GridCacheProcessor.java:2781)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.onExchangeDone(GridCacheProcessor.java:2878)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:2431)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.finishExchangeOnCoordinator(GridDhtPartitionsExchangeFuture.java:3832)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onAllReceived(GridDhtPartitionsExchangeFuture.java:3608)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.processSingleMessage(GridDhtPartitionsExchangeFuture.java:3207)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.access$200(GridDhtPartitionsExchangeFuture.java:154)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$2.apply(GridDhtPartitionsExchangeFuture.java:2994)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$2.apply(GridDhtPartitionsExchangeFuture.java:2982)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:399)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:354)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onReceiveSingleMessage(GridDhtPartitionsExchangeFuture.java:2982)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.processSinglePartitionUpdate(GridCachePartitionExchangeManager.java:1989)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.preprocessSingleMessage(GridCachePartitionExchangeManager.java:524)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.access$1100(GridCachePartitionExchangeManager.java:182)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$2.onMessage(GridCachePartitionExchangeManager.java:407)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$2.onMessage(GridCachePartitionExchangeManager.java:389)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCachePartitionExchangeManager.java:3715)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCachePartitionExchangeManager.java:3694)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1142)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:591)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java
[jira] [Assigned] (IGNITE-11687) Concurrent WAL replay & log may fail with CRC error on read
[
https://issues.apache.org/jira/browse/IGNITE-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey N. Gura reassigned IGNITE-11687:
---
Assignee: Anton Kalashnikov (was: Dmitriy Govorukhin)
> Concurrent WAL replay & log may fail with CRC error on read
> ---
>
> Key: IGNITE-11687
> URL: https://issues.apache.org/jira/browse/IGNITE-11687
> Project: Ignite
> Issue Type: Bug
> Components: persistence
>Reporter: Alexey Goncharuk
>Assignee: Anton Kalashnikov
>Priority: Critical
> Fix For: 2.9
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The cause is the way {{end}} is calculated for WAL iterator:
> {code}
> if (hnd != null)
> end = hnd.position();
> {code}
> {code}
> @Override public FileWALPointer position() {
> lock.lock();
> try {
> return new FileWALPointer(getSegmentId(), (int)written, 0);
> }
> finally {
> lock.unlock();
> }
> }
> {code}
> Consider a partially written entry. In this case, {{written}} has been
> already updated, concurrent WAL replay will attempt to read the incompletely
> written record and since {{end}} is not null, iterator will fail with CRC
> error.
> The issue may be rarely reproduced by {{IgniteWalSerializerVersionTest}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12845) GridNioServer can infinitely lose some events
[
https://issues.apache.org/jira/browse/IGNITE-12845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121055#comment-17121055
]
Aleksey Plekhanov commented on IGNITE-12845:
[~ptupitsyn], can you please check locally your case against the attached
pull-request?
> GridNioServer can infinitely lose some events
> --
>
> Key: IGNITE-12845
> URL: https://issues.apache.org/jira/browse/IGNITE-12845
> Project: Ignite
> Issue Type: Bug
>Reporter: Aleksey Plekhanov
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> With enabled optimization (IGNITE_NO_SELECTOR_OPTS = false, by default)
> {{GridNioServer}} can lose some events for a channel (depending on JDK
> version and OS). It can lead to connected applications hang. Reproducer:
> {code:java}
> public void testConcurrentLoad() throws Exception {
> startGrid(0);
> try (IgniteClient client = Ignition.startClient(new
> ClientConfiguration().setAddresses("127.0.0.1:10800"))) {
> ClientCache cache =
> client.getOrCreateCache(DEFAULT_CACHE_NAME);
> GridTestUtils.runMultiThreaded(
> () -> {
> for (int i = 0; i < 1000; i++)
> cache.put(i, i);
> }, 5, "run-async");
> }
> }
> {code}
> This reproducer hangs eventually on MacOS (tested with JDK 8, 11, 12, 13,
> 14), hangs on some Linux environments (for example passed more than 100 times
> on desktop Linux system with JDK 8, but hangs on team-city agents with JDK 8,
> 11) and never hanged (passed more than 100 times) on windows system, but
> passes on all systems and JDK versions when system property
> {{IGNITE_NO_SELECTOR_OPTS = true}} is set.
>
> The root cause: optimized {{SelectedSelectionKeySet}} always returns
> {{false}} for {{contains()}} method. The {{contains()}} method used by
> {{sun.nio.ch.SelectorImpl.processReadyEvents()}} method:
> {code:java}
> if (selectedKeys.contains(ski)) {
> if (ski.translateAndUpdateReadyOps(rOps)) {
> return 1;
> }
> } else {
> ski.translateAndSetReadyOps(rOps);
> if ((ski.nioReadyOps() & ski.nioInterestOps()) != 0) {
> selectedKeys.add(ski);
> return 1;
> }
> }
> {code}
> So, for fair implementation, if a selection key is contained in the selected
> keys set, then ready operations flags are updated, but for
> {{SelectedSelectionKeySet}} ready operations flags will be always overridden
> and new selector key will be added even if it's already contained in the set.
> Some {{SelectorImpl}} implementations can pass several events for one
> selector key to {{processReadyEvents}} method (for example, MacOs
> implementation {{KQueueSelectorImpl}} works in such a way). In this case,
> duplicated selector keys will be added to {{selectedKeys}} and all events
> except last will be lost.
> Two bad things happen in {{GridNioServer}} due to described above reasons:
> # Some event flags are lost and the worker doesn't process corresponding
> action (for attached reproducer "channel is ready for reading" event is lost
> and the workers never read the channel after some point in time).
> # Duplicated selector keys with the same event flags (for attached
> reproducer it's "channel is ready for writing" event, this duplication leads
> to wrong processing of {{GridSelectorNioSessionImpl#procWrite}} flag, which
> will be {{false}} in some cases, but at the same time selector key's
> {{interestedOps}} will contain {{OP_WRITE}} operation and this operation
> never be excluded)
> Possible solutions:
> * Fair implementation of {{SelectedSelectionKeySet.contains}} method (this
> will solve all problems but can be resource consuming)
> * Always set {{GridSelectorNioSessionImpl#procWrite}} to {{true}} when
> adding {{OP_WRITE}} to {{interestedOps}} (for example in
> {{AbstractNioClientWorker.registerWrite()}} method). In this case, some
> "channel is ready for reading" events (but not data) still can be lost, but
> not infinitely, and eventually data will be read. If events will be reordered
> (first "channel is ready for writing", after it "channel is ready for
> reading") then write to the channel will be only processed after all data
> will be read.
> * Exclude {{OP_WRITE}} from {{interestedOps}} even if
> {{GridSelectorNioSessionImpl#procWrite}} is {{false}} when there are no write
> requests in the queue (see {{GridNioServer.stopPollingForWrite()}} method).
> This solution has the same shortcomings as the previous one.
> * Hybrid approach. Use some probabilistic implementation for {{contains}}
> method (bloom filter or just check the last element) and use one of two
> previous solutions as a workaround, for cases when we incorrectly return
> {{fa
[jira] [Commented] (IGNITE-13094) java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
[
https://issues.apache.org/jira/browse/IGNITE-13094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121062#comment-17121062
]
Ivan Pavlukhin commented on IGNITE-13094:
-
[~AK47Sonic], thank you for sharing the code. I tried to launch it and
everything worked fine. One unclear thing is how the Ignite server (or cluster)
is started and configured. I used the same xml config as _default-config.xml_
with enabled server mode.
> java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
>
>
> Key: IGNITE-13094
> URL: https://issues.apache.org/jira/browse/IGNITE-13094
> Project: Ignite
> Issue Type: Bug
> Components: compute
>Affects Versions: 2.8.1
> Environment: Ignite 2.8.0
> Spring Boot 2.1.0.RELEASE
> Java 1.8
>Reporter: AK47Sonic
>Priority: Major
>
> I try to use spring boot (2.1.0.RELEASE) to run ignite.compute. As spring
> boot + lombok use logback as default, the following codes always throw
> exceptions (2020-05-28 07:00:48.979 WARN 8460 --- [ p2p-#49]
> o.a.i.i.m.d.GridDeploymentCommunication : Failed to resolve class:
> ch.qos.logback). The result (28) could be returned. Seems we see log4j and
> log4j2 on official website. I try to use log4j2 to replace logback. And it
> works fine. Does ignite support logback for distributed compute? Or It is
> just a bug?
> {panel:title=pom.xml}
>
>
> org.springframework.boot
> spring-boot-starter-web
>
>
> org.springframework.boot
> spring-boot-starter-log4j
>
>
>
>
> org.apache.ignite
> ignite-core
> 2.8.0
>
>
> org.apache.ignite
> ignite-spring
> 2.8.0
>
>
> org.projectlombok
> lombok
> 1.18.12
> provided
>
>
> {panel}
> {panel:title=config.xml}
>
> http://www.springframework.org/schema/beans";
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>xmlns:util="http://www.springframework.org/schema/util";
>xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd
> http://www.springframework.org/schema/util
> http://www.springframework.org/schema/util/spring-util.xsd";>
> class="org.apache.ignite.configuration.IgniteConfiguration">
>
>
>
>
>
>
>
> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>
>
> 127.0.0.1:48500..48520
>
>
>
>
>
>
>
> class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
>
>
>
>
>
> {panel}
> {code:java}
> @Configuration
> @ImportResource(locations={"classpath:default-config.xml"})
> public class IgniteConfig {
> @Bean
> public Ignite ignite(IgniteConfiguration igniteConfiguration) {
> Ignite ignite = Ignition.start(igniteConfiguration);
> Ignition.setClientMode(true);
> return ignite;
> }
> }
> @Slf4j
> @RestController
> public class WebController {
> @Autowired
> private Ignite ignite;
> @GetMapping("/compute")
> public void compute() {
> Collection> calls = new ArrayList<>();
> // Iterate through all the words in the sentence and create Callable
> jobs.
> for (final String word : "Count characters using callable".split(" "))
> calls.add(word::length);
> // Execute collection of Callables on the grid.
> Collection res = ignite.compute().call(calls);
> // Add up all the results.
> int sum = res.stream().mapToInt(Integer::intValue).sum();
> log.info("Total number of characters is '" + sum + "'.");
> }
> }
> {code}
> {panel:title=Exception}
> 2020-05-28 07:00:48.786 INFO 8460 --- [nio-8080-exec-1]
> o.a.i.i.m.d.GridDeploymentLocalStore : Class locally deployed: class
> com.sonic.sample.springboot.controller.WebController
> 2020-05-28 07:00:48.890 WARN 8460 --- [p2p-#49]
> o.a.i.i.m.d.GridDeploymentCommunication : Failed to resolve class:
> ch.qos.logback
> java.lang.ClassNotFoundException: ch.qos.logback
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> ~[na:1.8.0_121]
> at java.lang.ClassLoader.loadClass(ClassLoader.j
[jira] [Created] (IGNITE-13102) IgniteCache#isClosed() returns false on server node even if the cache had closed before.
Sergey Antonov created IGNITE-13102:
---
Summary: IgniteCache#isClosed() returns false on server node even
if the cache had closed before.
Key: IGNITE-13102
URL: https://issues.apache.org/jira/browse/IGNITE-13102
Project: Ignite
Issue Type: Bug
Affects Versions: 2.8.1, 2.8
Reporter: Sergey Antonov
Fix For: 2.9
IgniteCache#isClosed() still returns {{false}} even after
{{IgniteCache#close()}}. Only server nodes affect by this problem.
Simple reproducer:
{code:java}
@Test
public void test() throws Exception {
IgniteEx node = startGrid(0);
IgniteCache cache =
node.getOrCreateCache(DEFAULT_CACHE_NAME);
assertFalse(cache.isClosed());
cache.close();
// java.lang.AssertionError
assertTrue(cache.isClosed());
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13102) IgniteCache#isClosed() returns false on server node even if the cache had been closed before.
[
https://issues.apache.org/jira/browse/IGNITE-13102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Antonov updated IGNITE-13102:
Summary: IgniteCache#isClosed() returns false on server node even if the
cache had been closed before. (was: IgniteCache#isClosed() returns false on
server node even if the cache had closed before.)
> IgniteCache#isClosed() returns false on server node even if the cache had
> been closed before.
> -
>
> Key: IGNITE-13102
> URL: https://issues.apache.org/jira/browse/IGNITE-13102
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.8, 2.8.1
>Reporter: Sergey Antonov
>Priority: Major
> Fix For: 2.9
>
>
> IgniteCache#isClosed() still returns {{false}} even after
> {{IgniteCache#close()}}. Only server nodes affect by this problem.
> Simple reproducer:
> {code:java}
> @Test
> public void test() throws Exception {
> IgniteEx node = startGrid(0);
> IgniteCache cache =
> node.getOrCreateCache(DEFAULT_CACHE_NAME);
> assertFalse(cache.isClosed());
> cache.close();
> // java.lang.AssertionError
> assertTrue(cache.isClosed());
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13026) Calcite integration. Fix metadata for IgniteTable when it has embedded filters.
[
https://issues.apache.org/jira/browse/IGNITE-13026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-13026:
Priority: Critical (was: Major)
> Calcite integration. Fix metadata for IgniteTable when it has embedded
> filters.
> ---
>
> Key: IGNITE-13026
> URL: https://issues.apache.org/jira/browse/IGNITE-13026
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Roman Kondakov
>Assignee: Konstantin Orlov
>Priority: Critical
>
> By default {{RelMdRowCount}} return the actual row count for
> {{IgniteTables}}. But current implementation of {{IgniteTableScan}} may
> contain embedded filters and rows that are filtered out by these filters are
> not taken into account by the metadata subsystem.
> For example, table emps contains 1000 rows. Filter name='Vasya' filters out
> 95% of rows. For now we end up with situation
> {code:java}
> IgniteFilter(name='Vasya') <- estimated cardinality = 50 rows
> IgniteTableScan(filters=null) <- estimated cardinality = 1000 rows.
> {code}
> but after merging {{Filter}} into {{Scan}} we get the wrong estimation
> {code:java}
> IgniteTableScan(filters={name='Vasya'}) <- estimated cardinality = 1000 rows.
> But should be 50 rows.
> {code}
> We need to add a special case in order to compute the actual cardinality
> returned by {{IgniteTableScan}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12747) Calcite integration. Correlated queries support.
[ https://issues.apache.org/jira/browse/IGNITE-12747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12747: Priority: Critical (was: Major) > Calcite integration. Correlated queries support. > > > Key: IGNITE-12747 > URL: https://issues.apache.org/jira/browse/IGNITE-12747 > Project: Ignite > Issue Type: New Feature >Reporter: Igor Seliverstov >Priority: Critical > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12455) Calcite integration. Partition pruning.
[ https://issues.apache.org/jira/browse/IGNITE-12455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12455: Priority: Minor (was: Major) > Calcite integration. Partition pruning. > --- > > Key: IGNITE-12455 > URL: https://issues.apache.org/jira/browse/IGNITE-12455 > Project: Ignite > Issue Type: Improvement >Reporter: Igor Seliverstov >Priority: Minor > > In scope of fragment info calculation we're able to prune partitions on the > basis of filter condition, query parameters, affinity and tables distribution. > This optimization may dramatically reduce query execution time/needed > resources. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12868) Calcite integration. LEFT, RIGHT join support.
[ https://issues.apache.org/jira/browse/IGNITE-12868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12868: Priority: Critical (was: Major) > Calcite integration. LEFT, RIGHT join support. > -- > > Key: IGNITE-12868 > URL: https://issues.apache.org/jira/browse/IGNITE-12868 > Project: Ignite > Issue Type: New Feature >Reporter: Igor Seliverstov >Priority: Critical > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13025) Calcite integration. Limit and offset support.
[
https://issues.apache.org/jira/browse/IGNITE-13025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-13025:
Priority: Critical (was: Major)
> Calcite integration. Limit and offset support.
> --
>
> Key: IGNITE-13025
> URL: https://issues.apache.org/jira/browse/IGNITE-13025
> Project: Ignite
> Issue Type: New Feature
> Components: sql
>Reporter: Roman Kondakov
>Assignee: Taras Ledkov
>Priority: Critical
>
> We can exploit {{Sort}} operator for this purpose or introduce a new operator
> {{Limit}} by analogy with {{EnumerableLimit}}. Some research is needed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13103) Investigate marshalling errors when changing fields of lambda's capturingClass
Ilya Kasnacheev created IGNITE-13103:
Summary: Investigate marshalling errors when changing fields of
lambda's capturingClass
Key: IGNITE-13103
URL: https://issues.apache.org/jira/browse/IGNITE-13103
Project: Ignite
Issue Type: Bug
Components: binary, compute
Affects Versions: 2.8.1
Reporter: Ilya Kasnacheev
Assignee: Ilya Kasnacheev
Trying to execute static lambda whose outer type's fields changed leads to the
following error:
{code}
Exception in thread "main" class org.apache.ignite.IgniteException: Remote job
threw user exception (override or implement ComputeTask.result(..) method if
you would like to have automatic failover for this exception): Failed to
serialize object
[typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C4]
at
org.apache.ignite.compute.ComputeTaskAdapter.result(ComputeTaskAdapter.java:102)
at
org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1062)
at
org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1055)
at
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7037)
at
org.apache.ignite.internal.processors.task.GridTaskWorker.result(GridTaskWorker.java:1055)
at
org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:862)
at
org.apache.ignite.internal.processors.task.GridTaskWorker.processDelayedResponses(GridTaskWorker.java:711)
at
org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:542)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at
org.apache.ignite.internal.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:829)
at
org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:497)
at
org.apache.ignite.internal.processors.closure.GridClosureProcessor.runAsync(GridClosureProcessor.java:244)
at
org.apache.ignite.internal.processors.closure.GridClosureProcessor.runAsync(GridClosureProcessor.java:216)
at
org.apache.ignite.internal.IgniteComputeImpl.runAsync0(IgniteComputeImpl.java:702)
at
org.apache.ignite.internal.IgniteComputeImpl.run(IgniteComputeImpl.java:678)
at
com.gridgain.deployer.c.ComputeCallerStarter.start(ComputeCallerStarter.java:19)
at
com.gridgain.deployer.c.ComputeCallerStarter.main(ComputeCallerStarter.java:13)
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
serialize object
[typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C4]
at
org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:853)
at
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:232)
at
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:165)
at
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:152)
at
org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:251)
at
org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:84)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:57)
at
org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10386)
at
org.apache.ignite.internal.processors.task.GridTaskWorker.sendRequest(GridTaskWorker.java:1397)
at
org.apache.ignite.internal.processors.task.GridTaskWorker.processMappedJobs(GridTaskWorker.java:664)
at
org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:536)
... 9 more
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
serialize object [typeName=java.lang.invoke.SerializedLambda]
at
org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:853)
at
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:232)
at
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:165)
at
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:227)
at
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:165)
at
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:152)
at
org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:524)
at
org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObject(BinaryWriterExImpl.java:1503)
at
org.apache
[jira] [Updated] (IGNITE-12585) Calcite integration. Splitter improvements.
[ https://issues.apache.org/jira/browse/IGNITE-12585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12585: Priority: Minor (was: Major) > Calcite integration. Splitter improvements. > --- > > Key: IGNITE-12585 > URL: https://issues.apache.org/jira/browse/IGNITE-12585 > Project: Ignite > Issue Type: Improvement >Reporter: Igor Seliverstov >Assignee: Nikolay Izhikov >Priority: Minor > > Currently in case a head fragment of a query plan have broadcast distribution > it causes OptimisticPlanningException and additional fragment split each time > it's mapped on topology on a client node. Seems it's quite usual case, so, we > should cover it properly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12586) Calcite integration. NodesMapping simplification.
[
https://issues.apache.org/jira/browse/IGNITE-12586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-12586:
Priority: Minor (was: Major)
> Calcite integration. NodesMapping simplification.
> -
>
> Key: IGNITE-12586
> URL: https://issues.apache.org/jira/browse/IGNITE-12586
> Project: Ignite
> Issue Type: Improvement
>Reporter: Igor Seliverstov
>Priority: Minor
>
> Seems turning {{List assignments}} to {{Map
> assignments}} may significantly reduce occupied memory.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12692) Calcite integration. DML. Skip reducer optimization.
[ https://issues.apache.org/jira/browse/IGNITE-12692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12692: Priority: Minor (was: Major) > Calcite integration. DML. Skip reducer optimization. > > > Key: IGNITE-12692 > URL: https://issues.apache.org/jira/browse/IGNITE-12692 > Project: Ignite > Issue Type: Improvement >Reporter: Igor Seliverstov >Priority: Minor > > Having plan tree we easily can check whether a final modification may be > executed on data nodes directly or not. We should implement such kind of > optimization. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13103) Investigate marshalling errors when changing fields of lambda's capturingClass
[
https://issues.apache.org/jira/browse/IGNITE-13103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Kasnacheev updated IGNITE-13103:
-
Attachment: deployer.zip
> Investigate marshalling errors when changing fields of lambda's capturingClass
> --
>
> Key: IGNITE-13103
> URL: https://issues.apache.org/jira/browse/IGNITE-13103
> Project: Ignite
> Issue Type: Bug
> Components: binary, compute
>Affects Versions: 2.8.1
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Kasnacheev
>Priority: Major
> Attachments: deployer.zip
>
>
> Trying to execute static lambda whose outer type's fields changed leads to
> the following error:
> {code}
> Exception in thread "main" class org.apache.ignite.IgniteException: Remote
> job threw user exception (override or implement ComputeTask.result(..) method
> if you would like to have automatic failover for this exception): Failed to
> serialize object
> [typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C4]
> at
> org.apache.ignite.compute.ComputeTaskAdapter.result(ComputeTaskAdapter.java:102)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1062)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1055)
> at
> org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7037)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.result(GridTaskWorker.java:1055)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:862)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.processDelayedResponses(GridTaskWorker.java:711)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:542)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:829)
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:497)
> at
> org.apache.ignite.internal.processors.closure.GridClosureProcessor.runAsync(GridClosureProcessor.java:244)
> at
> org.apache.ignite.internal.processors.closure.GridClosureProcessor.runAsync(GridClosureProcessor.java:216)
> at
> org.apache.ignite.internal.IgniteComputeImpl.runAsync0(IgniteComputeImpl.java:702)
> at
> org.apache.ignite.internal.IgniteComputeImpl.run(IgniteComputeImpl.java:678)
> at
> com.gridgain.deployer.c.ComputeCallerStarter.start(ComputeCallerStarter.java:19)
> at
> com.gridgain.deployer.c.ComputeCallerStarter.main(ComputeCallerStarter.java:13)
> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
> serialize object
> [typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C4]
> at
> org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:853)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:232)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:165)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:152)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:251)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:84)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:57)
> at
> org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10386)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.sendRequest(GridTaskWorker.java:1397)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.processMappedJobs(GridTaskWorker.java:664)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:536)
> ... 9 more
> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
> serialize object [typeName=java.lang.invoke.SerializedLambda]
> at
> org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:853)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:232)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:165)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:227)
> at
> org.apache.ignite.internal.binary.BinaryWrit
[jira] [Updated] (IGNITE-12633) Calcite integration. Query cancel purpose.
[ https://issues.apache.org/jira/browse/IGNITE-12633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12633: Priority: Minor (was: Major) > Calcite integration. Query cancel purpose. > -- > > Key: IGNITE-12633 > URL: https://issues.apache.org/jira/browse/IGNITE-12633 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Igor Seliverstov >Assignee: Ravil Galeyev >Priority: Minor > > Currently on a query participant node left whole query is cancelled, but the > end user doesn't know the purpose. We should pass causing cancel exception to > end user. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13103) Investigate marshalling errors when changing fields of lambda's capturingClass
[
https://issues.apache.org/jira/browse/IGNITE-13103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121097#comment-17121097
]
Ilya Kasnacheev commented on IGNITE-13103:
--
Please see attached reproducer project! Just run ServerNodeStarter then
ComputeCallerStarter's from subdirectories, the third one will fail.
> Investigate marshalling errors when changing fields of lambda's capturingClass
> --
>
> Key: IGNITE-13103
> URL: https://issues.apache.org/jira/browse/IGNITE-13103
> Project: Ignite
> Issue Type: Bug
> Components: binary, compute
>Affects Versions: 2.8.1
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Kasnacheev
>Priority: Major
> Attachments: deployer.zip
>
>
> Trying to execute static lambda whose outer type's fields changed leads to
> the following error:
> {code}
> Exception in thread "main" class org.apache.ignite.IgniteException: Remote
> job threw user exception (override or implement ComputeTask.result(..) method
> if you would like to have automatic failover for this exception): Failed to
> serialize object
> [typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C4]
> at
> org.apache.ignite.compute.ComputeTaskAdapter.result(ComputeTaskAdapter.java:102)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1062)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1055)
> at
> org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7037)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.result(GridTaskWorker.java:1055)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:862)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.processDelayedResponses(GridTaskWorker.java:711)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:542)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:829)
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:497)
> at
> org.apache.ignite.internal.processors.closure.GridClosureProcessor.runAsync(GridClosureProcessor.java:244)
> at
> org.apache.ignite.internal.processors.closure.GridClosureProcessor.runAsync(GridClosureProcessor.java:216)
> at
> org.apache.ignite.internal.IgniteComputeImpl.runAsync0(IgniteComputeImpl.java:702)
> at
> org.apache.ignite.internal.IgniteComputeImpl.run(IgniteComputeImpl.java:678)
> at
> com.gridgain.deployer.c.ComputeCallerStarter.start(ComputeCallerStarter.java:19)
> at
> com.gridgain.deployer.c.ComputeCallerStarter.main(ComputeCallerStarter.java:13)
> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
> serialize object
> [typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C4]
> at
> org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:853)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:232)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:165)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:152)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:251)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:84)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:57)
> at
> org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10386)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.sendRequest(GridTaskWorker.java:1397)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.processMappedJobs(GridTaskWorker.java:664)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:536)
> ... 9 more
> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
> serialize object [typeName=java.lang.invoke.SerializedLambda]
> at
> org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:853)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:232)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.j
[jira] [Updated] (IGNITE-12913) Calcite integration: Add semi join remove rule to the planner
[
https://issues.apache.org/jira/browse/IGNITE-12913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-12913:
Priority: Minor (was: Major)
> Calcite integration: Add semi join remove rule to the planner
> -
>
> Key: IGNITE-12913
> URL: https://issues.apache.org/jira/browse/IGNITE-12913
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Priority: Minor
>
> We need to add next rules to planner
> * FilterJoinRule,
> * JoinAddRedundantSemiJoinRule,
> * SemiJoinRemoveRule
> In order to be able to make this transformation for the query:
> {noformat}
> "select e.ename from emp e, dept d\n"
> + "where e.deptno = d.deptno"
> BEFORE=
> LogicalProject(ENAME=[$1])
> LogicalFilter(condition=[=($3, $5)])
> LogicalJoin(condition=[true], joinType=[inner])
> IgniteTableScan(table=[[PUBLIC, EMP]])
> IgniteTableScan(table=[[PUBLIC, DEPT]])
> AFTER=
> IgniteProject(ENAME=[$1])
> IgniteJoin(condition=[=($3, $5)], joinType=[inner])
> IgniteTableScan(table=[[PUBLIC, EMP]])
> IgniteTableScan(table=[[PUBLIC, DEPT]])
> {noformat}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12914) Calcite integration: Add aggregate project merge rule to the planner
[
https://issues.apache.org/jira/browse/IGNITE-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-12914:
Priority: Minor (was: Major)
> Calcite integration: Add aggregate project merge rule to the planner
>
>
> Key: IGNITE-12914
> URL: https://issues.apache.org/jira/browse/IGNITE-12914
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Priority: Minor
>
> We need to add next rules to planner
> * AggregateProjectMergeRule
> In order to be able to make this transformation for the query:
> {noformat}
> "select x, sum(z), y from (\n"
> + " select deptno as x, empno as y, sal as z, sal * 2 as zz\n"
> + " from emp)\n"
> + "group by x, y"
> BEFORE=
> LogicalProject(X=[$0], EXPR$1=[$2], Y=[$1])
> LogicalAggregate(group=[{0, 1}], EXPR$1=[SUM($2)])
> LogicalProject(X=[$3], Y=[$0], Z=[$2])
> IgniteTableScan(table=[[PUBLIC, EMP]])
> AFTER=
> IgniteProject(X=[$0], EXPR$1=[$2], Y=[$1])
> IgniteProject(DEPTNO=[$1], EMPNO=[$0], EXPR$1=[$2])
> IgniteAggregate(group=[{0, 3}], EXPR$1=[SUM($2)])
> IgniteTableScan(table=[[PUBLIC, EMP]])
> {noformat}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12991) Calcite integration. Pass cancel flag to VolcanoPlanner
[
https://issues.apache.org/jira/browse/IGNITE-12991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-12991:
Priority: Minor (was: Major)
> Calcite integration. Pass cancel flag to VolcanoPlanner
> ---
>
> Key: IGNITE-12991
> URL: https://issues.apache.org/jira/browse/IGNITE-12991
> Project: Ignite
> Issue Type: Improvement
>Reporter: Igor Seliverstov
>Priority: Minor
>
> see {{AbstractRelOptPlanner.java:91}}, here {{CancelFlag}} is used to cancel
> planning loop. We need to pass it into a newly created context and bind its
> state with {{PlanningContext#queryCancel}} to break possible infinite
> planning loop. See also {{PlanningContext#unwrap}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13022) Calcite integration. Merge index conditions for the same field.
[
https://issues.apache.org/jira/browse/IGNITE-13022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-13022:
Priority: Minor (was: Major)
> Calcite integration. Merge index conditions for the same field.
> ---
>
> Key: IGNITE-13022
> URL: https://issues.apache.org/jira/browse/IGNITE-13022
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Priority: Minor
>
> Index scans should be able to merge index conditions. For example query
> {code:java}
> SELECT * FROM tbl WHERE a<5 AND a<10
> {code}
> should be reduced to
>
> {code:java}
> SELECT * FROM tbl WHERE a<5
> {code}
> Parameters should be handled in a more tricky way:
> {code:java}
> SELECT * FROM tbl WHERE a {code}
> can be rewritten as
> {code:java}
> SELECT * FROM tbl WHERE a where the expression {{MIN(?1, ?2)}} should be evaluated right before the
> execution when parameters are known.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13023) Calcite integration. Page scan support.
[ https://issues.apache.org/jira/browse/IGNITE-13023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-13023: Priority: Minor (was: Major) > Calcite integration. Page scan support. > --- > > Key: IGNITE-13023 > URL: https://issues.apache.org/jira/browse/IGNITE-13023 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Roman Kondakov >Priority: Minor > > We need to evaluate the advantage of using data page scan (see IGNITE-10798) > in Calcite engine and support it if needed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13030) Calcite integration. Push projections to scans and avoid reading full row when possible
[
https://issues.apache.org/jira/browse/IGNITE-13030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-13030:
Priority: Minor (was: Major)
> Calcite integration. Push projections to scans and avoid reading full row
> when possible
> ---
>
> Key: IGNITE-13030
> URL: https://issues.apache.org/jira/browse/IGNITE-13030
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Roman Kondakov
>Priority: Minor
>
> Currently {{IngiteTableScan}} supports only embedded filters. If it supports
> also embedded projections (see {{ProjectableFilterableTable}} in Calcite), it
> will be able to push these projections down to page memory scans in order to
> avoid full row reading.
> For example for the query
> {code:java}
> SELECT _val FROM tbl
> {code}
> we can read only values.
> Moreover, we can establish index-only scans in some cases. For example, if
> the column {{depId}} is indexed, we can the execute query
> {code:java}
> SELECT depId FROM emp
> {code}
> using only index tree, without reading rows at all.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13027) Calcite integration. Query parallelism support.
[ https://issues.apache.org/jira/browse/IGNITE-13027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-13027: Priority: Minor (was: Major) > Calcite integration. Query parallelism support. > --- > > Key: IGNITE-13027 > URL: https://issues.apache.org/jira/browse/IGNITE-13027 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Roman Kondakov >Priority: Minor > > We need to add support for segmented indexes in Calcite engine. Currently > engine is able to execute queries over index with only one segment. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13035) Calcite integration. Support fallback on indexes rebuild
[ https://issues.apache.org/jira/browse/IGNITE-13035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-13035: Priority: Minor (was: Major) > Calcite integration. Support fallback on indexes rebuild > > > Key: IGNITE-13035 > URL: https://issues.apache.org/jira/browse/IGNITE-13035 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Roman Kondakov >Priority: Minor > > When indexes are being rebuilt we need to use CacheTree for index scans as it > happens in legacy engine. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12563) Calcite integration. Reset and remap queries on topology change.
[ https://issues.apache.org/jira/browse/IGNITE-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12563: Priority: Minor (was: Major) > Calcite integration. Reset and remap queries on topology change. > > > Key: IGNITE-12563 > URL: https://issues.apache.org/jira/browse/IGNITE-12563 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Igor Seliverstov >Priority: Minor > > A query should: > * be silently remapped in case an actual topology version at query plan > materialization time differs from a topology version at the time locations > was calculated > * be cancelled with appropriate exception in case one of responsible nodes > left a cluster after execution is started. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12819) Calcite integration. Cost system.
[ https://issues.apache.org/jira/browse/IGNITE-12819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12819: Priority: Minor (was: Major) > Calcite integration. Cost system. > - > > Key: IGNITE-12819 > URL: https://issues.apache.org/jira/browse/IGNITE-12819 > Project: Ignite > Issue Type: Task >Reporter: Igor Seliverstov >Priority: Minor > > Current implementation doesn't suit our needs for particular reasons. We need > to introduce our own one taking into account such parameters as IO > operations, memory usage, CPU usage, disk usage etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12899) Calcite integration. Distribution multitrait
[ https://issues.apache.org/jira/browse/IGNITE-12899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12899: Priority: Minor (was: Major) > Calcite integration. Distribution multitrait > > > Key: IGNITE-12899 > URL: https://issues.apache.org/jira/browse/IGNITE-12899 > Project: Ignite > Issue Type: Task >Reporter: Igor Seliverstov >Priority: Minor > > Currently Ignite nodes have single distribution value which isn't true in > several cases like: > # a partitioned table with a key alias should have 2 distribution traits: by > _key and by key alias (id for example) - without distribution by _key column > almost impossible to implement IGNITE-12692 > # After inner / cross join the result should have several hash distributions > (by left and by right keys) - it may be important on join transpose > optimization > # On project a source distribution key may appear more than once, so the > result should contain a distribution for each distribution key position. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12909) Calcite integration. Explain command syntax change
[ https://issues.apache.org/jira/browse/IGNITE-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-12909: Summary: Calcite integration. Explain command syntax change (was: Calcite integration. Explain command support) > Calcite integration. Explain command syntax change > -- > > Key: IGNITE-12909 > URL: https://issues.apache.org/jira/browse/IGNITE-12909 > Project: Ignite > Issue Type: Task >Reporter: Igor Seliverstov >Priority: Major > > Support EXPLAIN command -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12909) Calcite integration. Explain command syntax change
[
https://issues.apache.org/jira/browse/IGNITE-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-12909:
Priority: Minor (was: Major)
> Calcite integration. Explain command syntax change
> --
>
> Key: IGNITE-12909
> URL: https://issues.apache.org/jira/browse/IGNITE-12909
> Project: Ignite
> Issue Type: Task
>Reporter: Igor Seliverstov
>Priority: Minor
>
> We need to change the default Calcite syntax from {{EXPLAIN PLAN FOR}} to
> {{EXPLAIN}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12909) Calcite integration. Explain command syntax change
[
https://issues.apache.org/jira/browse/IGNITE-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Kondakov updated IGNITE-12909:
Description: We need to change the default Calcite syntax from {{EXPLAIN
PLAN FOR}} to {{EXPLAIN}}. (was: Support EXPLAIN command)
> Calcite integration. Explain command syntax change
> --
>
> Key: IGNITE-12909
> URL: https://issues.apache.org/jira/browse/IGNITE-12909
> Project: Ignite
> Issue Type: Task
>Reporter: Igor Seliverstov
>Priority: Major
>
> We need to change the default Calcite syntax from {{EXPLAIN PLAN FOR}} to
> {{EXPLAIN}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13104) Spring data 2.0 IgniteRepositoryImpl#deleteAllById contains wrong code
Alexey Kuznetsov created IGNITE-13104:
-
Summary: Spring data 2.0 IgniteRepositoryImpl#deleteAllById
contains wrong code
Key: IGNITE-13104
URL: https://issues.apache.org/jira/browse/IGNITE-13104
Project: Ignite
Issue Type: Improvement
Components: springdata
Affects Versions: 2.8.1
Reporter: Alexey Kuznetsov
Assignee: Alexey Kuznetsov
Fix For: 2.9
{code}
/** {@inheritDoc} */
@Override public void deleteAllById(Iterable ids) {
if (ids instanceof Set)
cache.removeAll((Set)ids);
if (ids instanceof Collection)
cache.removeAll(new HashSet<>((Collection)ids));
TreeSet keys = new TreeSet<>();
for (ID id : ids)
keys.add(id);
cache.removeAll(keys);
}
{code}
As you can see cache.removeAll may be executed THREE times in some situations.
Also this method can throw ClassCast exception if ids collection contains
objects that are not implement Comparable interface.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (IGNITE-13094) java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
[
https://issues.apache.org/jira/browse/IGNITE-13094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Kasnacheev reassigned IGNITE-13094:
Assignee: Ilya Kasnacheev
> java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
>
>
> Key: IGNITE-13094
> URL: https://issues.apache.org/jira/browse/IGNITE-13094
> Project: Ignite
> Issue Type: Bug
> Components: compute
>Affects Versions: 2.8.1
> Environment: Ignite 2.8.0
> Spring Boot 2.1.0.RELEASE
> Java 1.8
>Reporter: AK47Sonic
>Assignee: Ilya Kasnacheev
>Priority: Major
>
> I try to use spring boot (2.1.0.RELEASE) to run ignite.compute. As spring
> boot + lombok use logback as default, the following codes always throw
> exceptions (2020-05-28 07:00:48.979 WARN 8460 --- [ p2p-#49]
> o.a.i.i.m.d.GridDeploymentCommunication : Failed to resolve class:
> ch.qos.logback). The result (28) could be returned. Seems we see log4j and
> log4j2 on official website. I try to use log4j2 to replace logback. And it
> works fine. Does ignite support logback for distributed compute? Or It is
> just a bug?
> {panel:title=pom.xml}
>
>
> org.springframework.boot
> spring-boot-starter-web
>
>
> org.springframework.boot
> spring-boot-starter-log4j
>
>
>
>
> org.apache.ignite
> ignite-core
> 2.8.0
>
>
> org.apache.ignite
> ignite-spring
> 2.8.0
>
>
> org.projectlombok
> lombok
> 1.18.12
> provided
>
>
> {panel}
> {panel:title=config.xml}
>
> http://www.springframework.org/schema/beans";
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>xmlns:util="http://www.springframework.org/schema/util";
>xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd
> http://www.springframework.org/schema/util
> http://www.springframework.org/schema/util/spring-util.xsd";>
> class="org.apache.ignite.configuration.IgniteConfiguration">
>
>
>
>
>
>
>
> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>
>
> 127.0.0.1:48500..48520
>
>
>
>
>
>
>
> class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
>
>
>
>
>
> {panel}
> {code:java}
> @Configuration
> @ImportResource(locations={"classpath:default-config.xml"})
> public class IgniteConfig {
> @Bean
> public Ignite ignite(IgniteConfiguration igniteConfiguration) {
> Ignite ignite = Ignition.start(igniteConfiguration);
> Ignition.setClientMode(true);
> return ignite;
> }
> }
> @Slf4j
> @RestController
> public class WebController {
> @Autowired
> private Ignite ignite;
> @GetMapping("/compute")
> public void compute() {
> Collection> calls = new ArrayList<>();
> // Iterate through all the words in the sentence and create Callable
> jobs.
> for (final String word : "Count characters using callable".split(" "))
> calls.add(word::length);
> // Execute collection of Callables on the grid.
> Collection res = ignite.compute().call(calls);
> // Add up all the results.
> int sum = res.stream().mapToInt(Integer::intValue).sum();
> log.info("Total number of characters is '" + sum + "'.");
> }
> }
> {code}
> {panel:title=Exception}
> 2020-05-28 07:00:48.786 INFO 8460 --- [nio-8080-exec-1]
> o.a.i.i.m.d.GridDeploymentLocalStore : Class locally deployed: class
> com.sonic.sample.springboot.controller.WebController
> 2020-05-28 07:00:48.890 WARN 8460 --- [p2p-#49]
> o.a.i.i.m.d.GridDeploymentCommunication : Failed to resolve class:
> ch.qos.logback
> java.lang.ClassNotFoundException: ch.qos.logback
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> ~[na:1.8.0_121]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_121]
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> ~[na:1.8.0_121]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_121]
> at java.lang.Class.forName0(Native Meth
[jira] [Commented] (IGNITE-13094) java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
[
https://issues.apache.org/jira/browse/IGNITE-13094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121135#comment-17121135
]
Ilya Kasnacheev commented on IGNITE-13094:
--
It turns out Ignite cannot peer class load from 'classic'
I kid you not, consider GridDeploymentCommunication.class:
{code}
int idx = clsName.indexOf(".class");
if (idx >= 0) {
clsName = clsName.substring(0, idx);
}
{code}
> java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
>
>
> Key: IGNITE-13094
> URL: https://issues.apache.org/jira/browse/IGNITE-13094
> Project: Ignite
> Issue Type: Bug
> Components: compute
>Affects Versions: 2.8.1
> Environment: Ignite 2.8.0
> Spring Boot 2.1.0.RELEASE
> Java 1.8
>Reporter: AK47Sonic
>Priority: Major
>
> I try to use spring boot (2.1.0.RELEASE) to run ignite.compute. As spring
> boot + lombok use logback as default, the following codes always throw
> exceptions (2020-05-28 07:00:48.979 WARN 8460 --- [ p2p-#49]
> o.a.i.i.m.d.GridDeploymentCommunication : Failed to resolve class:
> ch.qos.logback). The result (28) could be returned. Seems we see log4j and
> log4j2 on official website. I try to use log4j2 to replace logback. And it
> works fine. Does ignite support logback for distributed compute? Or It is
> just a bug?
> {panel:title=pom.xml}
>
>
> org.springframework.boot
> spring-boot-starter-web
>
>
> org.springframework.boot
> spring-boot-starter-log4j
>
>
>
>
> org.apache.ignite
> ignite-core
> 2.8.0
>
>
> org.apache.ignite
> ignite-spring
> 2.8.0
>
>
> org.projectlombok
> lombok
> 1.18.12
> provided
>
>
> {panel}
> {panel:title=config.xml}
>
> http://www.springframework.org/schema/beans";
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>xmlns:util="http://www.springframework.org/schema/util";
>xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd
> http://www.springframework.org/schema/util
> http://www.springframework.org/schema/util/spring-util.xsd";>
> class="org.apache.ignite.configuration.IgniteConfiguration">
>
>
>
>
>
>
>
> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>
>
> 127.0.0.1:48500..48520
>
>
>
>
>
>
>
> class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
>
>
>
>
>
> {panel}
> {code:java}
> @Configuration
> @ImportResource(locations={"classpath:default-config.xml"})
> public class IgniteConfig {
> @Bean
> public Ignite ignite(IgniteConfiguration igniteConfiguration) {
> Ignite ignite = Ignition.start(igniteConfiguration);
> Ignition.setClientMode(true);
> return ignite;
> }
> }
> @Slf4j
> @RestController
> public class WebController {
> @Autowired
> private Ignite ignite;
> @GetMapping("/compute")
> public void compute() {
> Collection> calls = new ArrayList<>();
> // Iterate through all the words in the sentence and create Callable
> jobs.
> for (final String word : "Count characters using callable".split(" "))
> calls.add(word::length);
> // Execute collection of Callables on the grid.
> Collection res = ignite.compute().call(calls);
> // Add up all the results.
> int sum = res.stream().mapToInt(Integer::intValue).sum();
> log.info("Total number of characters is '" + sum + "'.");
> }
> }
> {code}
> {panel:title=Exception}
> 2020-05-28 07:00:48.786 INFO 8460 --- [nio-8080-exec-1]
> o.a.i.i.m.d.GridDeploymentLocalStore : Class locally deployed: class
> com.sonic.sample.springboot.controller.WebController
> 2020-05-28 07:00:48.890 WARN 8460 --- [p2p-#49]
> o.a.i.i.m.d.GridDeploymentCommunication : Failed to resolve class:
> ch.qos.logback
> java.lang.ClassNotFoundException: ch.qos.logback
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> ~[na:1.8.0_121]
>
[jira] [Created] (IGNITE-13105) Spring data 2.0 IgniteRepositoryQuery#transformQueryCursor contains cursor leak
Alexey Kuznetsov created IGNITE-13105:
-
Summary: Spring data 2.0
IgniteRepositoryQuery#transformQueryCursor contains cursor leak
Key: IGNITE-13105
URL: https://issues.apache.org/jira/browse/IGNITE-13105
Project: Ignite
Issue Type: Improvement
Components: springdata
Affects Versions: 2.8.1
Reporter: Alexey Kuznetsov
Assignee: Alexey Kuznetsov
Fix For: 2.9
This code produce cursor leak in RunningQueryManager:
If result set contains one ore more rows.
{code}
case ONE_VALUE:
Iterator iter = qryIter.iterator();
if (iter.hasNext())
return iter.next().get(0);
return null;
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-10959) Memory leaks in continuous query handlers
[ https://issues.apache.org/jira/browse/IGNITE-10959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121138#comment-17121138 ] Maxim Muzafarov commented on IGNITE-10959: -- Folks, I can confirm the issue with the `pending` map overflow mentioned above. PR with reproducer (PR #7881) attached. I will try to fix it. > Memory leaks in continuous query handlers > - > > Key: IGNITE-10959 > URL: https://issues.apache.org/jira/browse/IGNITE-10959 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.7 >Reporter: Denis Mekhanikov >Assignee: Maxim Muzafarov >Priority: Critical > Attachments: CacheContinuousQueryMemoryUsageTest.java, > CacheContinuousQueryMemoryUsageTest.result, > CacheContinuousQueryMemoryUsageTest2.java, > Memory_blowup_in_Ignite_CacheContinuousQueryHandler.txt, > Memory_blowup_in_Ignite_CacheContinuousQueryHandler.txt, > Memory_blowup_in_Ignite_CacheContinuousQueryHandler.txt, > continuousquery_leak_profile.png, referencepath.PNG > > Time Spent: 10m > Remaining Estimate: 0h > > Continuous query handlers don't clear internal data structures after cache > events are processed. > A test, that reproduces the problem, is attached. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-13094) java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
[
https://issues.apache.org/jira/browse/IGNITE-13094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121135#comment-17121135
]
Ilya Kasnacheev edited comment on IGNITE-13094 at 6/1/20, 4:23 PM:
---
It turns out Ignite cannot peer class load from 'classic'
I kid you not, consider GridDeploymentCommunication.java:
{code}
int idx = clsName.indexOf(".class");
if (idx >= 0) {
clsName = clsName.substring(0, idx);
}
{code}
was (Author: ilyak):
It turns out Ignite cannot peer class load from 'classic'
I kid you not, consider GridDeploymentCommunication.class:
{code}
int idx = clsName.indexOf(".class");
if (idx >= 0) {
clsName = clsName.substring(0, idx);
}
{code}
> java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
>
>
> Key: IGNITE-13094
> URL: https://issues.apache.org/jira/browse/IGNITE-13094
> Project: Ignite
> Issue Type: Bug
> Components: compute
>Affects Versions: 2.8.1
> Environment: Ignite 2.8.0
> Spring Boot 2.1.0.RELEASE
> Java 1.8
>Reporter: AK47Sonic
>Assignee: Ilya Kasnacheev
>Priority: Major
>
> I try to use spring boot (2.1.0.RELEASE) to run ignite.compute. As spring
> boot + lombok use logback as default, the following codes always throw
> exceptions (2020-05-28 07:00:48.979 WARN 8460 --- [ p2p-#49]
> o.a.i.i.m.d.GridDeploymentCommunication : Failed to resolve class:
> ch.qos.logback). The result (28) could be returned. Seems we see log4j and
> log4j2 on official website. I try to use log4j2 to replace logback. And it
> works fine. Does ignite support logback for distributed compute? Or It is
> just a bug?
> {panel:title=pom.xml}
>
>
> org.springframework.boot
> spring-boot-starter-web
>
>
> org.springframework.boot
> spring-boot-starter-log4j
>
>
>
>
> org.apache.ignite
> ignite-core
> 2.8.0
>
>
> org.apache.ignite
> ignite-spring
> 2.8.0
>
>
> org.projectlombok
> lombok
> 1.18.12
> provided
>
>
> {panel}
> {panel:title=config.xml}
>
> http://www.springframework.org/schema/beans";
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>xmlns:util="http://www.springframework.org/schema/util";
>xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd
> http://www.springframework.org/schema/util
> http://www.springframework.org/schema/util/spring-util.xsd";>
> class="org.apache.ignite.configuration.IgniteConfiguration">
>
>
>
>
>
>
>
> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>
>
> 127.0.0.1:48500..48520
>
>
>
>
>
>
>
> class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
>
>
>
>
>
> {panel}
> {code:java}
> @Configuration
> @ImportResource(locations={"classpath:default-config.xml"})
> public class IgniteConfig {
> @Bean
> public Ignite ignite(IgniteConfiguration igniteConfiguration) {
> Ignite ignite = Ignition.start(igniteConfiguration);
> Ignition.setClientMode(true);
> return ignite;
> }
> }
> @Slf4j
> @RestController
> public class WebController {
> @Autowired
> private Ignite ignite;
> @GetMapping("/compute")
> public void compute() {
> Collection> calls = new ArrayList<>();
> // Iterate through all the words in the sentence and create Callable
> jobs.
> for (final String word : "Count characters using callable".split(" "))
> calls.add(word::length);
> // Execute collection of Callables on the grid.
> Collection res = ignite.compute().call(calls);
> // Add up all the results.
> int sum = res.stream().mapToInt(Integer::intValue).sum();
> log.info("Total number of characters is '" + sum + "'.");
> }
> }
> {code}
> {panel:title=Exception}
> 2020-05-28 07:00:48.786 INFO 8460 --- [nio-8080-exec-1]
[jira] [Commented] (IGNITE-13093) Unidentified Apache Ignite worker blocked when inserting large amount of records to the persistent storage
[ https://issues.apache.org/jira/browse/IGNITE-13093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121169#comment-17121169 ] Tomasz Grygo commented on IGNITE-13093: --- I tried your suggestions and exception did not happen :). I changed maxSize to 4GB ,-Xmx30g and adjusted config settings as you proposed. But why did it help exactly? What criteria should I use for sizing maxCache and heap? What value should I set if i have 2x or 4x the data? I understood that maxSize is the maximum size of cache in memory. I have 130 caches and most of them will not fit in memory anyways so didn't want to unnecessarily stress the garbage collector. Also, i have some additional questions about setting up Ignite. Is here a good place to ask them or is there some other, better place? > Unidentified Apache Ignite worker blocked when inserting large amount of > records to the persistent storage > -- > > Key: IGNITE-13093 > URL: https://issues.apache.org/jira/browse/IGNITE-13093 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.8.1 > Environment: Java 1.8.0_231 > Apache Ignite 2.8.1 > Windows 10, 64G memory > Java settings > -Xms1024m -Xmx50g -Xss1024m > -Xverify:none > -server > -DIGNITE_QUIET=true > -XX:+UseG1GC > -XX:+DisableExplicitGC > -Djava.net.preferIPv4Stack=true > -XX:+AlwaysPreTouch > -XX:+ScavengeBeforeFullGC > -XX:+AggressiveOpts >Reporter: Tomasz Grygo >Priority: Blocker > Attachments: 20_05_29__14_51.out.log, > PureIgniteDynamicRowStorage.java, PureIgniteUtils.java, ignite.xml, > log4j2.xml, thread_dump.txt > > > I'm looking at Apache Ignite to use as a fast database. Performance is very > important, I need to build it as fast as possible with resources available. > First I copy all (450M) records from my original test database to Ignite > caches through IgniteDataStreams using PK as a key. Database does not fit in > memory so I have disk persistence enabled and eviction disabled. Data is > inserted in parallel using 8 threads. I have only one but fairly powerful > Windows PC doing all the work, no separate Ignite cluster. I'm not interested > in cache recovery so WAL is disabled. Everything goes well until I hit around > 310 million entries (2 hours of work). At this point Ignite starts to choke, > inserts slow down and then stop with exceptions. Exception is triggered by > systemWorkerBlockedTimeout setting set to 5 minutes. Extending this time does > not help at all. Based on heap dump I tried adding > -DIGNITE_PAGES_LIST_DISABLE_ONHEAP_CACHING=true and it failed slightly later > but still could not finish the job. I read the performance guides and I tried > tweaking other Ignite settings too but didn't see any impact. How can if find > which worker is being blocked and why? > 2020-05-27 21:54:26,176 [Storage2 ] [ERROR] - DTR_0030 worker Storage2 had > error: FATAL ERROR java.lang.IllegalStateException: Data streamer has been > closed. > java.lang.IllegalStateException: Data streamer has been closed. > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.closedException(DataStreamerImpl.java:1095) > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.lock(DataStreamerImpl.java:446) > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addDataInternal(DataStreamerImpl.java:646) > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addDataInternal(DataStreamerImpl.java:631) > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:753) > at > com.sc.extr.cache.PureIgniteDynamicRowStorage.putIfAbsent(PureIgniteDynamicRowStorage.java:83) > at > com.sc.extr.cache.PureIgniteDynamicRowStorage.addRowOnKey(PureIgniteDynamicRowStorage.java:160) > at > com.sc.extr.tree.MultiCacheTreeBuilder.addRootRowToCache(MultiCacheTreeBuilder.java:409) > at > com.sc.extr.tree.MultiCacheTreeBuilder.parentRev1to1(MultiCacheTreeBuilder.java:237) > at > com.sc.extr.tree.MultiCacheTreeBuilder.addRowToCache(MultiCacheTreeBuilder.java:333) > at > com.sc.extr.tree.MultiCacheTreeBuilder.parentRev(MultiCacheTreeBuilder.java:274) > at > com.sc.extr.tree.MultiCacheTreeBuilder.addRow(MultiCacheTreeBuilder.java:379) > at > com.sc.extr.tree.MultiCacheTreeBuilder.process(MultiCacheTreeBuilder.java:206) > at com.sc.bi.workflow.WorkTransformer.processOne(WorkTransformer.java:84) > at com.sc.bi.workflow.WorkTransformer.doWork(WorkTransformer.java:145) > at > com.sc.bi.workflow.WorkTransformer.processQueue(WorkTransformer.java:210) > at com.sc.bi.workflow.WorkTransformer.run(WorkTransformer.java:16
[jira] [Resolved] (IGNITE-13093) Unidentified Apache Ignite worker blocked when inserting large amount of records to the persistent storage
[
https://issues.apache.org/jira/browse/IGNITE-13093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stanilovsky Evgeny resolved IGNITE-13093.
-
Resolution: Not A Problem
> Unidentified Apache Ignite worker blocked when inserting large amount of
> records to the persistent storage
> --
>
> Key: IGNITE-13093
> URL: https://issues.apache.org/jira/browse/IGNITE-13093
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.8.1
> Environment: Java 1.8.0_231
> Apache Ignite 2.8.1
> Windows 10, 64G memory
> Java settings
> -Xms1024m -Xmx50g -Xss1024m
> -Xverify:none
> -server
> -DIGNITE_QUIET=true
> -XX:+UseG1GC
> -XX:+DisableExplicitGC
> -Djava.net.preferIPv4Stack=true
> -XX:+AlwaysPreTouch
> -XX:+ScavengeBeforeFullGC
> -XX:+AggressiveOpts
>Reporter: Tomasz Grygo
>Priority: Blocker
> Attachments: 20_05_29__14_51.out.log,
> PureIgniteDynamicRowStorage.java, PureIgniteUtils.java, ignite.xml,
> log4j2.xml, thread_dump.txt
>
>
> I'm looking at Apache Ignite to use as a fast database. Performance is very
> important, I need to build it as fast as possible with resources available.
> First I copy all (450M) records from my original test database to Ignite
> caches through IgniteDataStreams using PK as a key. Database does not fit in
> memory so I have disk persistence enabled and eviction disabled. Data is
> inserted in parallel using 8 threads. I have only one but fairly powerful
> Windows PC doing all the work, no separate Ignite cluster. I'm not interested
> in cache recovery so WAL is disabled. Everything goes well until I hit around
> 310 million entries (2 hours of work). At this point Ignite starts to choke,
> inserts slow down and then stop with exceptions. Exception is triggered by
> systemWorkerBlockedTimeout setting set to 5 minutes. Extending this time does
> not help at all. Based on heap dump I tried adding
> -DIGNITE_PAGES_LIST_DISABLE_ONHEAP_CACHING=true and it failed slightly later
> but still could not finish the job. I read the performance guides and I tried
> tweaking other Ignite settings too but didn't see any impact. How can if find
> which worker is being blocked and why?
> 2020-05-27 21:54:26,176 [Storage2 ] [ERROR] - DTR_0030 worker Storage2 had
> error: FATAL ERROR java.lang.IllegalStateException: Data streamer has been
> closed.
> java.lang.IllegalStateException: Data streamer has been closed.
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.closedException(DataStreamerImpl.java:1095)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.lock(DataStreamerImpl.java:446)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addDataInternal(DataStreamerImpl.java:646)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addDataInternal(DataStreamerImpl.java:631)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:753)
> at
> com.sc.extr.cache.PureIgniteDynamicRowStorage.putIfAbsent(PureIgniteDynamicRowStorage.java:83)
> at
> com.sc.extr.cache.PureIgniteDynamicRowStorage.addRowOnKey(PureIgniteDynamicRowStorage.java:160)
> at
> com.sc.extr.tree.MultiCacheTreeBuilder.addRootRowToCache(MultiCacheTreeBuilder.java:409)
> at
> com.sc.extr.tree.MultiCacheTreeBuilder.parentRev1to1(MultiCacheTreeBuilder.java:237)
> at
> com.sc.extr.tree.MultiCacheTreeBuilder.addRowToCache(MultiCacheTreeBuilder.java:333)
> at
> com.sc.extr.tree.MultiCacheTreeBuilder.parentRev(MultiCacheTreeBuilder.java:274)
> at
> com.sc.extr.tree.MultiCacheTreeBuilder.addRow(MultiCacheTreeBuilder.java:379)
> at
> com.sc.extr.tree.MultiCacheTreeBuilder.process(MultiCacheTreeBuilder.java:206)
> at com.sc.bi.workflow.WorkTransformer.processOne(WorkTransformer.java:84)
> at com.sc.bi.workflow.WorkTransformer.doWork(WorkTransformer.java:145)
> at
> com.sc.bi.workflow.WorkTransformer.processQueue(WorkTransformer.java:210)
> at com.sc.bi.workflow.WorkTransformer.run(WorkTransformer.java:169)
> Caused by: class org.apache.ignite.IgniteCheckedException: Data streamer has
> been cancelled: DataStreamerImpl [bufLdrSzPerThread=4096,
> rcvr=org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$IsolatedUpdater@381b03ed,
> ioPlcRslvr=null, cacheName=PERSON.PTINTN, bufSize=512, parallelOps=0,
> timeout=-1, autoFlushFreq=0, bufMappings=ConcurrentHashMap
> {03e74462-12ec-4140-b9fb-a975572ac3bb=Buffer [node=TcpDiscoveryNode
> [id=03e74462-12ec-4140-b9fb-a975572ac3bb,
> consistentId=b01eb38b-7728-4e43-a697-0bc52f872e44, addrs=ArrayList
> [127.0.0.1, 172.27.179.112], sockAddrs=HashSet
> [SOFTBI-
[jira] [Commented] (IGNITE-13093) Unidentified Apache Ignite worker blocked when inserting large amount of records to the persistent storage
[ https://issues.apache.org/jira/browse/IGNITE-13093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121207#comment-17121207 ] Stanilovsky Evgeny commented on IGNITE-13093: - ok, i close it with "not a problem" resolution. I hope you need to proceed with further questions on dev list [1]. I try briefly question here: 1. If it is not dictated of your business logic you need to decrease heap for 2-4 Gb, ignite not need such huge heap at all. 2. give released memory to dataregion size. 3. give maximum resources you can, if data will not fit into memory it will be replaced to disk, no additional settings need here. 4. if you not fit it would be about disk io operations, not about heap pressure. [1] https://ignite.apache.org/community/resources.html > Unidentified Apache Ignite worker blocked when inserting large amount of > records to the persistent storage > -- > > Key: IGNITE-13093 > URL: https://issues.apache.org/jira/browse/IGNITE-13093 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.8.1 > Environment: Java 1.8.0_231 > Apache Ignite 2.8.1 > Windows 10, 64G memory > Java settings > -Xms1024m -Xmx50g -Xss1024m > -Xverify:none > -server > -DIGNITE_QUIET=true > -XX:+UseG1GC > -XX:+DisableExplicitGC > -Djava.net.preferIPv4Stack=true > -XX:+AlwaysPreTouch > -XX:+ScavengeBeforeFullGC > -XX:+AggressiveOpts >Reporter: Tomasz Grygo >Priority: Blocker > Attachments: 20_05_29__14_51.out.log, > PureIgniteDynamicRowStorage.java, PureIgniteUtils.java, ignite.xml, > log4j2.xml, thread_dump.txt > > > I'm looking at Apache Ignite to use as a fast database. Performance is very > important, I need to build it as fast as possible with resources available. > First I copy all (450M) records from my original test database to Ignite > caches through IgniteDataStreams using PK as a key. Database does not fit in > memory so I have disk persistence enabled and eviction disabled. Data is > inserted in parallel using 8 threads. I have only one but fairly powerful > Windows PC doing all the work, no separate Ignite cluster. I'm not interested > in cache recovery so WAL is disabled. Everything goes well until I hit around > 310 million entries (2 hours of work). At this point Ignite starts to choke, > inserts slow down and then stop with exceptions. Exception is triggered by > systemWorkerBlockedTimeout setting set to 5 minutes. Extending this time does > not help at all. Based on heap dump I tried adding > -DIGNITE_PAGES_LIST_DISABLE_ONHEAP_CACHING=true and it failed slightly later > but still could not finish the job. I read the performance guides and I tried > tweaking other Ignite settings too but didn't see any impact. How can if find > which worker is being blocked and why? > 2020-05-27 21:54:26,176 [Storage2 ] [ERROR] - DTR_0030 worker Storage2 had > error: FATAL ERROR java.lang.IllegalStateException: Data streamer has been > closed. > java.lang.IllegalStateException: Data streamer has been closed. > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.closedException(DataStreamerImpl.java:1095) > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.lock(DataStreamerImpl.java:446) > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addDataInternal(DataStreamerImpl.java:646) > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addDataInternal(DataStreamerImpl.java:631) > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:753) > at > com.sc.extr.cache.PureIgniteDynamicRowStorage.putIfAbsent(PureIgniteDynamicRowStorage.java:83) > at > com.sc.extr.cache.PureIgniteDynamicRowStorage.addRowOnKey(PureIgniteDynamicRowStorage.java:160) > at > com.sc.extr.tree.MultiCacheTreeBuilder.addRootRowToCache(MultiCacheTreeBuilder.java:409) > at > com.sc.extr.tree.MultiCacheTreeBuilder.parentRev1to1(MultiCacheTreeBuilder.java:237) > at > com.sc.extr.tree.MultiCacheTreeBuilder.addRowToCache(MultiCacheTreeBuilder.java:333) > at > com.sc.extr.tree.MultiCacheTreeBuilder.parentRev(MultiCacheTreeBuilder.java:274) > at > com.sc.extr.tree.MultiCacheTreeBuilder.addRow(MultiCacheTreeBuilder.java:379) > at > com.sc.extr.tree.MultiCacheTreeBuilder.process(MultiCacheTreeBuilder.java:206) > at com.sc.bi.workflow.WorkTransformer.processOne(WorkTransformer.java:84) > at com.sc.bi.workflow.WorkTransformer.doWork(WorkTransformer.java:145) > at > com.sc.bi.workflow.WorkTransformer.processQueue(WorkTransformer.java:210) > at com.sc.bi.workflow.WorkTransformer.run(WorkTransformer.java:169) > Caused by:
[jira] [Comment Edited] (IGNITE-13093) Unidentified Apache Ignite worker blocked when inserting large amount of records to the persistent storage
[ https://issues.apache.org/jira/browse/IGNITE-13093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121207#comment-17121207 ] Stanilovsky Evgeny edited comment on IGNITE-13093 at 6/1/20, 5:48 PM: -- ok, i close it with "not a problem" resolution. I hope you need to proceed with further questions on dev list [1]. I try briefly request here: 1. If it is not dictated of your business logic you need to decrease heap for 2-4 Gb, ignite not need such huge heap at all. 2. give released memory to dataregion size. 3. give maximum resources you can, if data will not fit into memory it will be replaced to disk, no additional settings need here. 4. if you not fit it would be about disk io operations, not about heap pressure. [1] https://ignite.apache.org/community/resources.html was (Author: zstan): ok, i close it with "not a problem" resolution. I hope you need to proceed with further questions on dev list [1]. I try briefly question here: 1. If it is not dictated of your business logic you need to decrease heap for 2-4 Gb, ignite not need such huge heap at all. 2. give released memory to dataregion size. 3. give maximum resources you can, if data will not fit into memory it will be replaced to disk, no additional settings need here. 4. if you not fit it would be about disk io operations, not about heap pressure. [1] https://ignite.apache.org/community/resources.html > Unidentified Apache Ignite worker blocked when inserting large amount of > records to the persistent storage > -- > > Key: IGNITE-13093 > URL: https://issues.apache.org/jira/browse/IGNITE-13093 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.8.1 > Environment: Java 1.8.0_231 > Apache Ignite 2.8.1 > Windows 10, 64G memory > Java settings > -Xms1024m -Xmx50g -Xss1024m > -Xverify:none > -server > -DIGNITE_QUIET=true > -XX:+UseG1GC > -XX:+DisableExplicitGC > -Djava.net.preferIPv4Stack=true > -XX:+AlwaysPreTouch > -XX:+ScavengeBeforeFullGC > -XX:+AggressiveOpts >Reporter: Tomasz Grygo >Priority: Blocker > Attachments: 20_05_29__14_51.out.log, > PureIgniteDynamicRowStorage.java, PureIgniteUtils.java, ignite.xml, > log4j2.xml, thread_dump.txt > > > I'm looking at Apache Ignite to use as a fast database. Performance is very > important, I need to build it as fast as possible with resources available. > First I copy all (450M) records from my original test database to Ignite > caches through IgniteDataStreams using PK as a key. Database does not fit in > memory so I have disk persistence enabled and eviction disabled. Data is > inserted in parallel using 8 threads. I have only one but fairly powerful > Windows PC doing all the work, no separate Ignite cluster. I'm not interested > in cache recovery so WAL is disabled. Everything goes well until I hit around > 310 million entries (2 hours of work). At this point Ignite starts to choke, > inserts slow down and then stop with exceptions. Exception is triggered by > systemWorkerBlockedTimeout setting set to 5 minutes. Extending this time does > not help at all. Based on heap dump I tried adding > -DIGNITE_PAGES_LIST_DISABLE_ONHEAP_CACHING=true and it failed slightly later > but still could not finish the job. I read the performance guides and I tried > tweaking other Ignite settings too but didn't see any impact. How can if find > which worker is being blocked and why? > 2020-05-27 21:54:26,176 [Storage2 ] [ERROR] - DTR_0030 worker Storage2 had > error: FATAL ERROR java.lang.IllegalStateException: Data streamer has been > closed. > java.lang.IllegalStateException: Data streamer has been closed. > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.closedException(DataStreamerImpl.java:1095) > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.lock(DataStreamerImpl.java:446) > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addDataInternal(DataStreamerImpl.java:646) > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addDataInternal(DataStreamerImpl.java:631) > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:753) > at > com.sc.extr.cache.PureIgniteDynamicRowStorage.putIfAbsent(PureIgniteDynamicRowStorage.java:83) > at > com.sc.extr.cache.PureIgniteDynamicRowStorage.addRowOnKey(PureIgniteDynamicRowStorage.java:160) > at > com.sc.extr.tree.MultiCacheTreeBuilder.addRootRowToCache(MultiCacheTreeBuilder.java:409) > at > com.sc.extr.tree.MultiCacheTreeBuilder.parentRev1to1(MultiCacheTreeBuilder.java:237) > at > com.sc.extr.tree.MultiCacheTree
[jira] [Commented] (IGNITE-13100) ClassCastException in cache group metrics on client nodes
[
https://issues.apache.org/jira/browse/IGNITE-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121214#comment-17121214
]
Ignite TC Bot commented on IGNITE-13100:
{panel:title=Branch: [pull/7875/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *--> Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5354517&buildTypeId=IgniteTests24Java8_RunAll]
> ClassCastException in cache group metrics on client nodes
> -
>
> Key: IGNITE-13100
> URL: https://issues.apache.org/jira/browse/IGNITE-13100
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexey Goncharuk
>Assignee: Alexey Goncharuk
>Priority: Major
> Fix For: 2.9
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The following exception can be observed when reading cache group metrics on
> client nodes with persistence-enabled config:
> {code}
> java.lang.ClassCastException:
> org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager
> cannot be cast to
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager
> at
> org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.database(CacheGroupMetricsImpl.java:506)
> at
> org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.lambda$new$0(CacheGroupMetricsImpl.java:103)
> at
> org.apache.ignite.internal.util.lang.GridFunc.lambda$nonThrowableSupplier$3(GridFunc.java:3341)
> at
> org.apache.ignite.internal.processors.metric.impl.LongGauge.value(LongGauge.java:45)
> at
> org.apache.ignite.spi.metric.LongMetric.getAsString(LongMetric.java:29)
> {code}
> The reason is an incomplete check for persistence enabled in
> {{CacheGroupMetricsImpl}}: we should also check for client nodes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13100) ClassCastException in cache group metrics on client nodes
[
https://issues.apache.org/jira/browse/IGNITE-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Goncharuk updated IGNITE-13100:
--
Description:
The following exception can be observed when reading cache group metrics on
client nodes with persistence-enabled config:
{code}
java.lang.ClassCastException:
org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager
cannot be cast to
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager
at
org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.database(CacheGroupMetricsImpl.java:506)
at
org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.lambda$new$0(CacheGroupMetricsImpl.java:103)
at
org.apache.ignite.internal.util.lang.GridFunc.lambda$nonThrowableSupplier$3(GridFunc.java:3341)
at
org.apache.ignite.internal.processors.metric.impl.LongGauge.value(LongGauge.java:45)
at
org.apache.ignite.spi.metric.LongMetric.getAsString(LongMetric.java:29)
{code}
The reason is an incomplete check for persistence enabled in
{{CacheGroupMetricsImpl}}: we should also check for client nodes.
Suggested solution: include the check for client node mode for database metrics
readings
was:
The following exception can be observed when reading cache group metrics on
client nodes with persistence-enabled config:
{code}
java.lang.ClassCastException:
org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager
cannot be cast to
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager
at
org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.database(CacheGroupMetricsImpl.java:506)
at
org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.lambda$new$0(CacheGroupMetricsImpl.java:103)
at
org.apache.ignite.internal.util.lang.GridFunc.lambda$nonThrowableSupplier$3(GridFunc.java:3341)
at
org.apache.ignite.internal.processors.metric.impl.LongGauge.value(LongGauge.java:45)
at
org.apache.ignite.spi.metric.LongMetric.getAsString(LongMetric.java:29)
{code}
The reason is an incomplete check for persistence enabled in
{{CacheGroupMetricsImpl}}: we should also check for client nodes.
> ClassCastException in cache group metrics on client nodes
> -
>
> Key: IGNITE-13100
> URL: https://issues.apache.org/jira/browse/IGNITE-13100
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexey Goncharuk
>Assignee: Alexey Goncharuk
>Priority: Major
> Fix For: 2.9
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The following exception can be observed when reading cache group metrics on
> client nodes with persistence-enabled config:
> {code}
> java.lang.ClassCastException:
> org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager
> cannot be cast to
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager
> at
> org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.database(CacheGroupMetricsImpl.java:506)
> at
> org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.lambda$new$0(CacheGroupMetricsImpl.java:103)
> at
> org.apache.ignite.internal.util.lang.GridFunc.lambda$nonThrowableSupplier$3(GridFunc.java:3341)
> at
> org.apache.ignite.internal.processors.metric.impl.LongGauge.value(LongGauge.java:45)
> at
> org.apache.ignite.spi.metric.LongMetric.getAsString(LongMetric.java:29)
> {code}
> The reason is an incomplete check for persistence enabled in
> {{CacheGroupMetricsImpl}}: we should also check for client nodes.
> Suggested solution: include the check for client node mode for database
> metrics readings
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-5214) ConcurrentModificationException with enable DEBUG log level
[
https://issues.apache.org/jira/browse/IGNITE-5214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121230#comment-17121230
]
Ignite TC Bot commented on IGNITE-5214:
---
{panel:title=Branch: [pull/7874/head] Base: [master] : Possible Blockers
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Cache 6{color} [[tests 0 Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=5354906]]
{panel}
[TeamCity *--> Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5348846&buildTypeId=IgniteTests24Java8_RunAll]
> ConcurrentModificationException with enable DEBUG log level
> ---
>
> Key: IGNITE-5214
> URL: https://issues.apache.org/jira/browse/IGNITE-5214
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.0
>Reporter: Nikolay Tikhonov
>Assignee: Ilya Kasnacheev
>Priority: Major
> Attachments: IGNITE_5214.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> ConcurrentModificationException with
> org.apache.ignite.continuous.query=DEBUG
> {noformat}
> Unexpected exception during cache update
> java.util.ConcurrentModificationException: null
> at java.util.TreeMap$PrivateEntryIterator.nextEntry(TreeMap.java:1211)
> at java.util.TreeMap$EntryIterator.next(TreeMap.java:1247)
> at java.util.TreeMap$EntryIterator.next(TreeMap.java:1242)
> at java.util.AbstractMap.toString(AbstractMap.java:554)
> at java.lang.String.valueOf(String.java:2994)
> at java.lang.StringBuilder.append(StringBuilder.java:131)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler$PartitionRecovery.collectEntries(CacheContinuousQueryHandler.java:1132)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.handleEvent(CacheContinuousQueryHandler.java:739)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.onEntryUpdate(CacheContinuousQueryHandler.java:792)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.access$800(CacheContinuousQueryHandler.java:91)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler$1.onEntryUpdated(CacheContinuousQueryHandler.java:419)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.onEntryUpdated(CacheContinuousQueryManager.java:347)
> at
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:2669)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2390)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1792)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1632)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.mapSingle(GridNearAtomicAbstractUpdateFuture.java:263)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:494)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:436)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:208)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$23.apply(GridDhtAtomicCache.java:1152)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$23.apply(GridDhtAtomicCache.java:1150)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.asyncOp(GridDhtAtomicCache.java:847)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAsync0(GridDhtAtomicCache.java:1150)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.putAsync0(GridDhtAtomicCache.java:619)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.putAsync(GridCacheAdapter.java:2574)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.putIfAbsentAsync(GridDhtAtomicCache.java:664)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.p
[jira] [Commented] (IGNITE-5214) ConcurrentModificationException with enable DEBUG log level
[
https://issues.apache.org/jira/browse/IGNITE-5214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121234#comment-17121234
]
Ilya Kasnacheev commented on IGNITE-5214:
-
[~akorensh] please review proposed fix.
> ConcurrentModificationException with enable DEBUG log level
> ---
>
> Key: IGNITE-5214
> URL: https://issues.apache.org/jira/browse/IGNITE-5214
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.0
>Reporter: Nikolay Tikhonov
>Assignee: Ilya Kasnacheev
>Priority: Major
> Attachments: IGNITE_5214.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> ConcurrentModificationException with
> org.apache.ignite.continuous.query=DEBUG
> {noformat}
> Unexpected exception during cache update
> java.util.ConcurrentModificationException: null
> at java.util.TreeMap$PrivateEntryIterator.nextEntry(TreeMap.java:1211)
> at java.util.TreeMap$EntryIterator.next(TreeMap.java:1247)
> at java.util.TreeMap$EntryIterator.next(TreeMap.java:1242)
> at java.util.AbstractMap.toString(AbstractMap.java:554)
> at java.lang.String.valueOf(String.java:2994)
> at java.lang.StringBuilder.append(StringBuilder.java:131)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler$PartitionRecovery.collectEntries(CacheContinuousQueryHandler.java:1132)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.handleEvent(CacheContinuousQueryHandler.java:739)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.onEntryUpdate(CacheContinuousQueryHandler.java:792)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.access$800(CacheContinuousQueryHandler.java:91)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler$1.onEntryUpdated(CacheContinuousQueryHandler.java:419)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.onEntryUpdated(CacheContinuousQueryManager.java:347)
> at
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:2669)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2390)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1792)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1632)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.mapSingle(GridNearAtomicAbstractUpdateFuture.java:263)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:494)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:436)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:208)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$23.apply(GridDhtAtomicCache.java:1152)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$23.apply(GridDhtAtomicCache.java:1150)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.asyncOp(GridDhtAtomicCache.java:847)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAsync0(GridDhtAtomicCache.java:1150)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.putAsync0(GridDhtAtomicCache.java:619)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.putAsync(GridCacheAdapter.java:2574)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.putIfAbsentAsync(GridDhtAtomicCache.java:664)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.putIfAbsent(GridDhtAtomicCache.java:657)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxy.putIfAbsent(IgniteCacheProxy.java:1451)
> at
> com.workday.fabric.ignite.management.IgniteManagementService.doExecute(IgniteManagementService.java:174)
> at
> com.workday.fabric.ignite.service.AbstractIgniteService.ex
[jira] [Commented] (IGNITE-13100) ClassCastException in cache group metrics on client nodes
[
https://issues.apache.org/jira/browse/IGNITE-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121289#comment-17121289
]
Andrey N. Gura commented on IGNITE-13100:
-
[~agoncharuk] LGTM! With one little comment. Please proceed with merge.
> ClassCastException in cache group metrics on client nodes
> -
>
> Key: IGNITE-13100
> URL: https://issues.apache.org/jira/browse/IGNITE-13100
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexey Goncharuk
>Assignee: Alexey Goncharuk
>Priority: Major
> Fix For: 2.9
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The following exception can be observed when reading cache group metrics on
> client nodes with persistence-enabled config:
> {code}
> java.lang.ClassCastException:
> org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager
> cannot be cast to
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager
> at
> org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.database(CacheGroupMetricsImpl.java:506)
> at
> org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.lambda$new$0(CacheGroupMetricsImpl.java:103)
> at
> org.apache.ignite.internal.util.lang.GridFunc.lambda$nonThrowableSupplier$3(GridFunc.java:3341)
> at
> org.apache.ignite.internal.processors.metric.impl.LongGauge.value(LongGauge.java:45)
> at
> org.apache.ignite.spi.metric.LongMetric.getAsString(LongMetric.java:29)
> {code}
> The reason is an incomplete check for persistence enabled in
> {{CacheGroupMetricsImpl}}: we should also check for client nodes.
> Suggested solution: include the check for client node mode for database
> metrics readings
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13106) Java thin client: Race between response and notification for compute tasks
Aleksey Plekhanov created IGNITE-13106:
--
Summary: Java thin client: Race between response and notification
for compute tasks
Key: IGNITE-13106
URL: https://issues.apache.org/jira/browse/IGNITE-13106
Project: Ignite
Issue Type: Bug
Components: thin client
Affects Versions: 2.9
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
Java thin client {{ClientCompute.execute()}} method can hang due to race
between processing of COMPUTE_TASK_EXECUTE response and COMPUTE_TASK_FINISHED
notification.
These messages are strongly ordered on the server-side. But on the client-side
response and notification are processed by different threads. If notification
will be processed before response, task future will never be completed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12845) GridNioServer can infinitely lose some events
[
https://issues.apache.org/jira/browse/IGNITE-12845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121352#comment-17121352
]
Aleksey Plekhanov commented on IGNITE-12845:
[~ptupitsyn], most likely NIO bug should not affect Linux systems (but it
certainly affects MacOS). I found another bug in java thin client compute
implementation (IGNITE-13106). And now I think that team-city hangs (which I
mention in original ticket description) were due to compute bug, but not NIO
bug (originally I've tested compute, but later wrote simplified reproducer with
cache.put). Please have a look at IGNITE-13106, perhaps .Net client has the
same problems.
> GridNioServer can infinitely lose some events
> --
>
> Key: IGNITE-12845
> URL: https://issues.apache.org/jira/browse/IGNITE-12845
> Project: Ignite
> Issue Type: Bug
>Reporter: Aleksey Plekhanov
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> With enabled optimization (IGNITE_NO_SELECTOR_OPTS = false, by default)
> {{GridNioServer}} can lose some events for a channel (depending on JDK
> version and OS). It can lead to connected applications hang. Reproducer:
> {code:java}
> public void testConcurrentLoad() throws Exception {
> startGrid(0);
> try (IgniteClient client = Ignition.startClient(new
> ClientConfiguration().setAddresses("127.0.0.1:10800"))) {
> ClientCache cache =
> client.getOrCreateCache(DEFAULT_CACHE_NAME);
> GridTestUtils.runMultiThreaded(
> () -> {
> for (int i = 0; i < 1000; i++)
> cache.put(i, i);
> }, 5, "run-async");
> }
> }
> {code}
> This reproducer hangs eventually on MacOS (tested with JDK 8, 11, 12, 13,
> 14), hangs on some Linux environments (for example passed more than 100 times
> on desktop Linux system with JDK 8, but hangs on team-city agents with JDK 8,
> 11) and never hanged (passed more than 100 times) on windows system, but
> passes on all systems and JDK versions when system property
> {{IGNITE_NO_SELECTOR_OPTS = true}} is set.
>
> The root cause: optimized {{SelectedSelectionKeySet}} always returns
> {{false}} for {{contains()}} method. The {{contains()}} method used by
> {{sun.nio.ch.SelectorImpl.processReadyEvents()}} method:
> {code:java}
> if (selectedKeys.contains(ski)) {
> if (ski.translateAndUpdateReadyOps(rOps)) {
> return 1;
> }
> } else {
> ski.translateAndSetReadyOps(rOps);
> if ((ski.nioReadyOps() & ski.nioInterestOps()) != 0) {
> selectedKeys.add(ski);
> return 1;
> }
> }
> {code}
> So, for fair implementation, if a selection key is contained in the selected
> keys set, then ready operations flags are updated, but for
> {{SelectedSelectionKeySet}} ready operations flags will be always overridden
> and new selector key will be added even if it's already contained in the set.
> Some {{SelectorImpl}} implementations can pass several events for one
> selector key to {{processReadyEvents}} method (for example, MacOs
> implementation {{KQueueSelectorImpl}} works in such a way). In this case,
> duplicated selector keys will be added to {{selectedKeys}} and all events
> except last will be lost.
> Two bad things happen in {{GridNioServer}} due to described above reasons:
> # Some event flags are lost and the worker doesn't process corresponding
> action (for attached reproducer "channel is ready for reading" event is lost
> and the workers never read the channel after some point in time).
> # Duplicated selector keys with the same event flags (for attached
> reproducer it's "channel is ready for writing" event, this duplication leads
> to wrong processing of {{GridSelectorNioSessionImpl#procWrite}} flag, which
> will be {{false}} in some cases, but at the same time selector key's
> {{interestedOps}} will contain {{OP_WRITE}} operation and this operation
> never be excluded)
> Possible solutions:
> * Fair implementation of {{SelectedSelectionKeySet.contains}} method (this
> will solve all problems but can be resource consuming)
> * Always set {{GridSelectorNioSessionImpl#procWrite}} to {{true}} when
> adding {{OP_WRITE}} to {{interestedOps}} (for example in
> {{AbstractNioClientWorker.registerWrite()}} method). In this case, some
> "channel is ready for reading" events (but not data) still can be lost, but
> not infinitely, and eventually data will be read. If events will be reordered
> (first "channel is ready for writing", after it "channel is ready for
> reading") then write to the channel will be only processed after all data
> will be read.
> * Exclude {{OP_WRITE}} from {{interestedOps}} even if
> {{GridSelectorNioSessionImpl#procWrite}} is {{false}} when there a
[jira] [Commented] (IGNITE-13094) java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
[
https://issues.apache.org/jira/browse/IGNITE-13094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121377#comment-17121377
]
AK47Sonic commented on IGNITE-13094:
Hi Pavlukhin,
It is strange. My team member and I have the same issue in local machine.
Everything works fine for you. It make me confused. :(
Detailed steps:
1. I download ignite(BINARY RELEASES) from
https://ignite.apache.org/download.cgi
2.8.0 guide, javadoc, scaladocrelease notes 2020-03-03
apache-ignite-2.8.0-bin.zip (pgp, sha512)
2. Change default-config.xml
http://www.springframework.org/schema/beans";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd";>
127.0.0.1:48500..48520
3. Run ignite.bat in the bin directory
> java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
>
>
> Key: IGNITE-13094
> URL: https://issues.apache.org/jira/browse/IGNITE-13094
> Project: Ignite
> Issue Type: Bug
> Components: compute
>Affects Versions: 2.8.1
> Environment: Ignite 2.8.0
> Spring Boot 2.1.0.RELEASE
> Java 1.8
>Reporter: AK47Sonic
>Assignee: Ilya Kasnacheev
>Priority: Major
>
> I try to use spring boot (2.1.0.RELEASE) to run ignite.compute. As spring
> boot + lombok use logback as default, the following codes always throw
> exceptions (2020-05-28 07:00:48.979 WARN 8460 --- [ p2p-#49]
> o.a.i.i.m.d.GridDeploymentCommunication : Failed to resolve class:
> ch.qos.logback). The result (28) could be returned. Seems we see log4j and
> log4j2 on official website. I try to use log4j2 to replace logback. And it
> works fine. Does ignite support logback for distributed compute? Or It is
> just a bug?
> {panel:title=pom.xml}
>
>
> org.springframework.boot
> spring-boot-starter-web
>
>
> org.springframework.boot
> spring-boot-starter-log4j
>
>
>
>
> org.apache.ignite
> ignite-core
> 2.8.0
>
>
> org.apache.ignite
> ignite-spring
> 2.8.0
>
>
> org.projectlombok
> lombok
> 1.18.12
> provided
>
>
> {panel}
> {panel:title=config.xml}
>
> http://www.springframework.org/schema/beans";
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>xmlns:util="http://www.springframework.org/schema/util";
>xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd
> http://www.springframework.org/schema/util
> http://www.springframework.org/schema/util/spring-util.xsd";>
> class="org.apache.ignite.configuration.IgniteConfiguration">
>
>
>
>
>
>
>
> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>
>
> 127.0.0.1:48500..48520
>
>
>
>
>
>
>
> class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
>
>
>
>
>
> {panel}
> {code:java}
> @Configuration
> @ImportResource(locations={"classpath:default-config.xml"})
> public class IgniteConfig {
> @Bean
> public Ignite ignite(IgniteConfiguration igniteConfiguration) {
> Ignite ignite = Ignition.start(igniteConfiguration);
> Ignition.setClientMode(true);
> return ignite;
> }
> }
> @Slf4j
> @RestController
> public class WebController {
> @Autowired
> privat
[jira] [Comment Edited] (IGNITE-13094) java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
[
https://issues.apache.org/jira/browse/IGNITE-13094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121377#comment-17121377
]
AK47Sonic edited comment on IGNITE-13094 at 6/1/20, 10:22 PM:
--
Hi Pavlukhin,
It is strange. My team member and I have the same issue in local machine.
Everything works fine for you. And that makes me confused. :(
Detailed steps:
1. I download ignite(BINARY RELEASES) from
https://ignite.apache.org/download.cgi
2.8.0 guide, javadoc, scaladocrelease notes 2020-03-03
apache-ignite-2.8.0-bin.zip (pgp, sha512)
2. Change default-config.xml
http://www.springframework.org/schema/beans";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd";>
127.0.0.1:48500..48520
3. Run ignite.bat in the bin directory
was (Author: ak47sonic):
Hi Pavlukhin,
It is strange. My team member and I have the same issue in local machine.
Everything works fine for you. It make me confused. :(
Detailed steps:
1. I download ignite(BINARY RELEASES) from
https://ignite.apache.org/download.cgi
2.8.0 guide, javadoc, scaladocrelease notes 2020-03-03
apache-ignite-2.8.0-bin.zip (pgp, sha512)
2. Change default-config.xml
http://www.springframework.org/schema/beans";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd";>
127.0.0.1:48500..48520
3. Run ignite.bat in the bin directory
> java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
>
>
> Key: IGNITE-13094
> URL: https://issues.apache.org/jira/browse/IGNITE-13094
> Project: Ignite
> Issue Type: Bug
> Components: compute
>Affects Versions: 2.8.1
> Environment: Ignite 2.8.0
> Spring Boot 2.1.0.RELEASE
> Java 1.8
>Reporter: AK47Sonic
>Assignee: Ilya Kasnacheev
>Priority: Major
>
> I try to use spring boot (2.1.0.RELEASE) to run ignite.compute. As spring
> boot + lombok use logback as default, the following codes always throw
> exceptions (2020-05-28 07:00:48.979 WARN 8460 --- [ p2p-#49]
> o.a.i.i.m.d.GridDeploymentCommunication : Failed to resolve class:
> ch.qos.logback). The result (28) could be returned. Seems we see log4j and
> log4j2 on official website. I try to use log4j2 to replace logback. And it
> works fine. Does ignite support logback for distributed compute? Or It is
> just a bug?
> {panel:title=pom.xml}
>
>
> org.springframework.boot
> spring-boot-starter-web
>
>
> org.springframework.boot
> spring-boot-starter-log4j
>
>
>
>
> org.apache.ignite
> ignite-core
> 2.8.0
>
>
> org.apache.ignite
> ignite-spring
> 2.8.0
>
>
> org.projectlombok
> lombok
> 1.18.12
> provided
>
>
> {panel}
> {panel:title=config.xml}
>
> http://www.springframework.org/schema/beans";
>
[jira] [Comment Edited] (IGNITE-13094) java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
[
https://issues.apache.org/jira/browse/IGNITE-13094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121377#comment-17121377
]
AK47Sonic edited comment on IGNITE-13094 at 6/1/20, 10:24 PM:
--
Hi Pavlukhin,
It is strange. My team member and I have the same issue in local machine.
Everything works fine for you. And that makes me confused. :(
Detailed steps:
1. I download ignite(BINARY RELEASES) from
https://ignite.apache.org/download.cgi
2.8.0 guide, javadoc, scaladocrelease notes 2020-03-03
apache-ignite-2.8.0-bin.zip (pgp, sha512)
2. Change default-config.xml in the config folder
http://www.springframework.org/schema/beans";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd";>
127.0.0.1:48500..48520
3. Run ignite.bat in the bin folder
was (Author: ak47sonic):
Hi Pavlukhin,
It is strange. My team member and I have the same issue in local machine.
Everything works fine for you. And that makes me confused. :(
Detailed steps:
1. I download ignite(BINARY RELEASES) from
https://ignite.apache.org/download.cgi
2.8.0 guide, javadoc, scaladocrelease notes 2020-03-03
apache-ignite-2.8.0-bin.zip (pgp, sha512)
2. Change default-config.xml
http://www.springframework.org/schema/beans";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd";>
127.0.0.1:48500..48520
3. Run ignite.bat in the bin directory
> java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
>
>
> Key: IGNITE-13094
> URL: https://issues.apache.org/jira/browse/IGNITE-13094
> Project: Ignite
> Issue Type: Bug
> Components: compute
>Affects Versions: 2.8.1
> Environment: Ignite 2.8.0
> Spring Boot 2.1.0.RELEASE
> Java 1.8
>Reporter: AK47Sonic
>Assignee: Ilya Kasnacheev
>Priority: Major
>
> I try to use spring boot (2.1.0.RELEASE) to run ignite.compute. As spring
> boot + lombok use logback as default, the following codes always throw
> exceptions (2020-05-28 07:00:48.979 WARN 8460 --- [ p2p-#49]
> o.a.i.i.m.d.GridDeploymentCommunication : Failed to resolve class:
> ch.qos.logback). The result (28) could be returned. Seems we see log4j and
> log4j2 on official website. I try to use log4j2 to replace logback. And it
> works fine. Does ignite support logback for distributed compute? Or It is
> just a bug?
> {panel:title=pom.xml}
>
>
> org.springframework.boot
> spring-boot-starter-web
>
>
> org.springframework.boot
> spring-boot-starter-log4j
>
>
>
>
> org.apache.ignite
> ignite-core
> 2.8.0
>
>
> org.apache.ignite
> ignite-spring
> 2.8.0
>
>
> org.projectlombok
> lombok
> 1.18.12
> provided
>
>
> {panel}
> {panel:title=config.xml}
>
> http://www.springframework.
[jira] [Comment Edited] (IGNITE-13094) java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
[
https://issues.apache.org/jira/browse/IGNITE-13094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121377#comment-17121377
]
AK47Sonic edited comment on IGNITE-13094 at 6/1/20, 10:25 PM:
--
Hi Pavlukhin,
It is strange. My team member and I have the same issue in local machine.
Everything works fine for you. And that makes me confused. :(
Detailed steps:
1. I download ignite(BINARY RELEASES) from
https://ignite.apache.org/download.cgi
2.8.0 guide, javadoc, scaladocrelease notes 2020-03-03
apache-ignite-2.8.0-bin.zip (pgp, sha512)
2. Change default-config.xml in the config folder
{panel:title=default-config.xml}
http://www.springframework.org/schema/beans";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd";>
127.0.0.1:48500..48520
{panel}
3. Run ignite.bat in the bin folder
was (Author: ak47sonic):
Hi Pavlukhin,
It is strange. My team member and I have the same issue in local machine.
Everything works fine for you. And that makes me confused. :(
Detailed steps:
1. I download ignite(BINARY RELEASES) from
https://ignite.apache.org/download.cgi
2.8.0 guide, javadoc, scaladocrelease notes 2020-03-03
apache-ignite-2.8.0-bin.zip (pgp, sha512)
2. Change default-config.xml in the config folder
http://www.springframework.org/schema/beans";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd";>
127.0.0.1:48500..48520
3. Run ignite.bat in the bin folder
> java.lang.ClassNotFoundException: ch.qos.logback for distributed compute
>
>
> Key: IGNITE-13094
> URL: https://issues.apache.org/jira/browse/IGNITE-13094
> Project: Ignite
> Issue Type: Bug
> Components: compute
>Affects Versions: 2.8.1
> Environment: Ignite 2.8.0
> Spring Boot 2.1.0.RELEASE
> Java 1.8
>Reporter: AK47Sonic
>Assignee: Ilya Kasnacheev
>Priority: Major
>
> I try to use spring boot (2.1.0.RELEASE) to run ignite.compute. As spring
> boot + lombok use logback as default, the following codes always throw
> exceptions (2020-05-28 07:00:48.979 WARN 8460 --- [ p2p-#49]
> o.a.i.i.m.d.GridDeploymentCommunication : Failed to resolve class:
> ch.qos.logback). The result (28) could be returned. Seems we see log4j and
> log4j2 on official website. I try to use log4j2 to replace logback. And it
> works fine. Does ignite support logback for distributed compute? Or It is
> just a bug?
> {panel:title=pom.xml}
>
>
> org.springframework.boot
> spring-boot-starter-web
>
>
> org.springframework.boot
> spring-boot-starter-log4j
>
>
>
>
> org.apache.ignite
> ignite-core
> 2.8.0
>
>
> org.apache.ignite
> ignite-spring
> 2.8.0
>
>
> org.projectlombok
> lombok
> 1.18.12
> provided
>
>
> {panel}
[jira] [Commented] (IGNITE-12702) Print warning when a cache value contains @AffinityKeyMapped annotation
[
https://issues.apache.org/jira/browse/IGNITE-12702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17123350#comment-17123350
]
Kartik Somani commented on IGNITE-12702:
Pull request for this
https://issues.apache.org/jira/browse/IGNITE-12702?jql=project%20%3D%20IGNITE%20AND%20labels%20in%20(newbie)%20and%20status%20%3D%20OPEN
> Print warning when a cache value contains @AffinityKeyMapped annotation
> ---
>
> Key: IGNITE-12702
> URL: https://issues.apache.org/jira/browse/IGNITE-12702
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Reporter: Denis Mekhanikov
>Assignee: Kartik Somani
>Priority: Major
> Labels: newbie
>
> Consider the following code snippet:
> {code:java}
> public class WrongAffinityExample {
> public static void main(String[] args) {
> Ignite ignite = Ignition.start("config/ignite.xml");
> IgniteCache cache =
> ignite.getOrCreateCache("employees");
> EmployeeKey key = new EmployeeKey(1);
> EmployeeValue value = new EmployeeValue(1, "Denis");
> cache.put(key, value);
> }
> public static class EmployeeKey {
> private int id;
> public EmployeeKey(int id) {
> this.id = id;
> }
> }
> public static class EmployeeValue {
> @AffinityKeyMapped
> int departmentId;
> String name;
> public EmployeeValue(int departmentId, String name) {
> this.departmentId = departmentId;
> this.name = name;
> }
> }
> }
> {code}
> Note, that {{EmployeeValue}} contains an {{@AffinityKeyMapped}} annotation,
> which doesn't have any effect, since it's specified in a value, and not in a
> key.
> Such mistake is simple to make and pretty hard to track down.
> This configuration should trigger a warning message printed in log to let
> the user know that this affinity key configuration is not applied.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12702) Print warning when a cache value contains @AffinityKeyMapped annotation
[
https://issues.apache.org/jira/browse/IGNITE-12702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17123354#comment-17123354
]
Kartik Somani commented on IGNITE-12702:
[~PetrovMikhail]I've updated the pull request here:
[https://github.com/apache/ignite/pull/7882]
please review
> Print warning when a cache value contains @AffinityKeyMapped annotation
> ---
>
> Key: IGNITE-12702
> URL: https://issues.apache.org/jira/browse/IGNITE-12702
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Reporter: Denis Mekhanikov
>Assignee: Kartik Somani
>Priority: Major
> Labels: newbie
>
> Consider the following code snippet:
> {code:java}
> public class WrongAffinityExample {
> public static void main(String[] args) {
> Ignite ignite = Ignition.start("config/ignite.xml");
> IgniteCache cache =
> ignite.getOrCreateCache("employees");
> EmployeeKey key = new EmployeeKey(1);
> EmployeeValue value = new EmployeeValue(1, "Denis");
> cache.put(key, value);
> }
> public static class EmployeeKey {
> private int id;
> public EmployeeKey(int id) {
> this.id = id;
> }
> }
> public static class EmployeeValue {
> @AffinityKeyMapped
> int departmentId;
> String name;
> public EmployeeValue(int departmentId, String name) {
> this.departmentId = departmentId;
> this.name = name;
> }
> }
> }
> {code}
> Note, that {{EmployeeValue}} contains an {{@AffinityKeyMapped}} annotation,
> which doesn't have any effect, since it's specified in a value, and not in a
> key.
> Such mistake is simple to make and pretty hard to track down.
> This configuration should trigger a warning message printed in log to let
> the user know that this affinity key configuration is not applied.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
