Hi Mikhail,
Unfortunately, the problem has repeated itself on ignite-core-2.3.3
27.02.18 00:27:55 ERROR GridCacheIoManager - Failed to process message
[senderId=8f99c887-cd4b-4c38-a649-ca430040d535, messageType=class
o.a.i.i.processors.cache.distributed.dht.atomic.
GridNearAtomicUpdateResponse]
org.apache.ignite.IgniteException: Runtime failure on bounds: [lower=null,
upper=PendingRow []]
at org.apache.ignite.internal.processors.cache.persistence.
tree.BPlusTree.find(BPlusTree.java:954) ~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.persistence.
tree.BPlusTree.find(BPlusTree.java:933) ~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.
IgniteCacheOffheapManagerImpl.expire(IgniteCacheOffheapManagerImpl.java:979)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.
*GridCacheTtlManager.expire*(GridCacheTtlManager.java:197)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.
GridCacheUtils.unwindEvicts(GridCacheUtils.java:833)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.
onMessageProcessed(GridCacheIoManager.java:1099)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.
processMessage(GridCacheIoManager.java:1072) ~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.
GridCacheIoManager.onMessage0(GridCacheIoManager.java:579)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.
handleMessage(GridCacheIoManager.java:378) ~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.
handleMessage(GridCacheIoManager.java:304) ~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.
GridCacheIoManager.access$100(GridCacheIoManager.java:99)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.
onMessage(GridCacheIoManager.java:293) ~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.managers.communication.
GridIoManager.invokeListener(GridIoManager.java:1555)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.managers.communication.GridIoManager.
processRegularMessage0(GridIoManager.java:1183)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.managers.communication.
GridIoManager.access$4200(GridIoManager.java:126)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.managers.communication.
GridIoManager$9.run(GridIoManager.java:1090) ~[ignite-core-2.3.3.jar:2.3.3]
at
org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:505)
~[ignite-core-2.3.3.jar:2.3.3]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
Caused by: java.lang.IllegalStateException: Failed to get page IO instance
(page content is corrupted)
at org.apache.ignite.internal.processors.cache.persistence.
tree.io.IOVersions.forVersion(IOVersions.java:83)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.persistence.
tree.io.IOVersions.forPage(IOVersions.java:95)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.persistence.
CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:148)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.persistence.
CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:102)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.tree.
PendingRow.initKey(PendingRow.java:72) ~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.tree.
PendingEntriesTree.getRow(PendingEntriesTree.java:118)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.tree.
PendingEntriesTree.getRow(PendingEntriesTree.java:31)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.persistence.
tree.BPlusTree$ForwardCursor.fillFromBuffer(BPlusTree.java:4539)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.persistence.
tree.BPlusTree$ForwardCursor.init(BPlusTree.java:4441)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.persistence.
tree.BPlusTree$ForwardCursor.access$5300(BPlusTree.java:4380)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.persistence.
tree.BPlusTree.findLowerUnbounded(BPlusTree.java:910)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.persistence.
tree.BPlusTree.find(BPlusTree.java:942) ~[ignite-core-2.3.3.jar:2.3.3]
... 17 more
And if we are reading from cache...
27.02.18 00:27:56 ERROR MessagePartProcessingHandler - error processing
incoming messagePartDeliveryReceipt=MessagePartDeliveryReceipt{id=null,
abonent=7****, externalId='7586F73B'}
org.springframework.dao.DataAccessResourceFailureException: Runtime failure
on bounds: [lower=null, upper=PendingRow []]; nested exception is class
org.apache.ignite.IgniteException: Runtime failure on bounds: [lower=null,
upper=PendingRow []]
at com.a1s.sdp.data.dao.ignite.IgniteExceptionTranslator.
translateExceptionIfPossible(IgniteExceptionTranslator.java:35)
~[core-data-1.5-SNAPSHOT.jar:3752]
at
com.a1s.sdp.data.dao.ignite.IgniteDaoHelper.wrapIgnite(IgniteDaoHelper.java:41)
~[core-data-1.5-SNAPSHOT.jar:3752]
at com.a1s.sdp.data.dao.ignite.BaseIgniteCassandraRepository.
findOne(BaseIgniteCassandraRepository.java:136)
~[core-data-1.5-SNAPSHOT.jar:3752]
at com.a1s.sdp.common.service.TransportServiceImpl.
processMessagePartDelivered(TransportServiceImpl.java:213)
~[core-common-1.5-SNAPSHOT.jar:3752]
at com.a1s.sdp.common.service.TransportServiceImpl.
messagePartDelivered(TransportServiceImpl.java:202)
~[core-common-1.5-SNAPSHOT.jar:3752]
at com.a1s.sdp.common.transport.handler.
MessagePartProcessingHandler.lambda$channelReadMessagePartDeliveryReceipt$1(
MessagePartProcessingHandler.java:91) ~[core-common-1.5-SNAPSHOT.jar:3752]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[?:1.8.0_151]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[?:1.8.0_151]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
~[?:1.8.0_151]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
~[?:1.8.0_151]
at java.lang.Thread.run(Thread.java:748) [?:*1.8.0_151*]
Caused by: org.apache.ignite.IgniteException: *Runtime failure on bounds:
[lower=null, upper=PendingRow []]*
at org.apache.ignite.internal.processors.cache.persistence.
tree.BPlusTree.find(BPlusTree.java:954) ~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.persistence.
tree.BPlusTree.find(BPlusTree.java:933) ~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.
*IgniteCacheOffheapManagerImpl.**expire*(IgniteCacheOffheapManagerImpl.java:979)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.
GridCacheTtlManager.expire(GridCacheTtlManager.java:197)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.
GridCacheUtils.unwindEvicts(GridCacheUtils.java:833)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.
GridCacheGateway.leaveNoLock(GridCacheGateway.java:240)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.
GridCacheGateway.leave(GridCacheGateway.java:225)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.
GatewayProtectedCacheProxy.onLeave(GatewayProtectedCacheProxy.java:1510)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.
GatewayProtectedCacheProxy.get(GatewayProtectedCacheProxy.java:608)
~[ignite-core-2.3.3.jar:2.3.3]
at com.a1s.sdp.data.dao.ignite.BaseIgniteCassandraRepository.
lambda$findOne$7(BaseIgniteCassandraRepository.java:136)
~[core-data-1.5-SNAPSHOT.jar:3752]
at
com.a1s.sdp.data.dao.ignite.IgniteDaoHelper.wrapIgnite(IgniteDaoHelper.java:39)
~[core-data-1.5-SNAPSHOT.jar:3752]
... 9 more
Caused by: java.lang.IllegalStateException: Failed to get page IO instance
(page content is corrupted)
at org.apache.ignite.internal.processors.cache.persistence.
tree.io.IOVersions.forVersion(IOVersions.java:83)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.persistence.
tree.io.IOVersions.forPage(IOVersions.java:95)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.persistence.
CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:148)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.persistence.
CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:102)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.tree.
PendingRow.initKey(PendingRow.java:72) ~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.tree.
PendingEntriesTree.getRow(PendingEntriesTree.java:118)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.tree.
PendingEntriesTree.getRow(PendingEntriesTree.java:31)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.persistence.
tree.BPlusTree$ForwardCursor.fillFromBuffer(BPlusTree.java:4539)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.persistence.
tree.BPlusTree$ForwardCursor.init(BPlusTree.java:4441)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.persistence.
tree.BPlusTree$ForwardCursor.access$5300(BPlusTree.java:4380)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.persistence.
tree.BPlusTree.findLowerUnbounded(BPlusTree.java:910)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.persistence.
tree.BPlusTree.find(BPlusTree.java:942) ~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.persistence.
tree.BPlusTree.find(BPlusTree.java:933) ~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.
*IgniteCacheOffheapManagerImpl.**expire*(IgniteCacheOffheapManagerImpl.java:979)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.
GridCacheTtlManager.expire(GridCacheTtlManager.java:197)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.
GridCacheUtils.unwindEvicts(GridCacheUtils.java:833)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.
GridCacheGateway.leaveNoLock(GridCacheGateway.java:240)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.
GridCacheGateway.leave(GridCacheGateway.java:225)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.
GatewayProtectedCacheProxy.onLeave(GatewayProtectedCacheProxy.java:1510)
~[ignite-core-2.3.3.jar:2.3.3]
at org.apache.ignite.internal.processors.cache.
GatewayProtectedCacheProxy.get(GatewayProtectedCacheProxy.java:608)
~[ignite-core-2.3.3.jar:2.3.3]
at com.a1s.sdp.data.dao.ignite.BaseIgniteCassandraRepository.
lambda$findOne$7(BaseIgniteCassandraRepository.java:136)
~[core-data-1.5-SNAPSHOT.jar:3752]
at
com.a1s.sdp.data.dao.ignite.IgniteDaoHelper.wrapIgnite(IgniteDaoHelper.java:39)
~[core-data-1.5-SNAPSHOT.jar:3752]
... 9 more
The attachment (https://yadi.sk/d/u7fSK8Y53SpCpA) contains a part of cache
obtained from node of cluster - /var/ignite/***/db/
node00-66fd86f5-4e2d-4f1b-8fe1-993bf5937d99/cache-MessagePartSent.
This cache contains objects of class MessagePartSent:
@Table("message_part_sent_external")
public final class MessagePartSent {
@Id
private MessagePartSentPk id;
private MessagePartPk partId;
public MessagePartSentPk getId() {
return id;
}
public void setId(MessagePartSentPk id) {
this.id = id;
}
public MessagePartPk getPartId() {
return partId;
}
public void setPartId(MessagePartPk partId) {
this.partId = partId;
}
@Override
public String toString() {
final StringBuilder sb = new StringBuilder("MessagePartSent{");
sb.append("id=").append(id);
sb.append(", partId=").append(partId);
sb.append('}');
return sb.toString();
}
public static final class MessagePartSentPk implements Serializable {
private Integer linkId;
private String externalId;
public MessagePartSentPk() {
//
}
public MessagePartSentPk(Integer linkId, String externalId) {
this.linkId = linkId;
this.externalId = externalId;
}
public Integer getLinkId() {
return linkId;
}
public void setLinkId(Integer linkId) {
this.linkId = linkId;
}
public String getExternalId() {
return externalId;
}
public void setExternalId(String externalId) {
this.externalId = externalId;
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
if (!(o instanceof MessagePartSentPk)) return false;
final MessagePartSentPk that = (MessagePartSentPk) o;
if (externalId != null ? !externalId.equals(that.externalId) :
that.externalId != null) return false;
if (linkId != null ? !linkId.equals(that.linkId) : that.linkId
!= null) return false;
return true;
}
@Override
public int hashCode() {
int result = linkId != null ? linkId.hashCode() : 0;
result = 31 * result + (externalId != null ?
externalId.hashCode() : 0);
return result;
}
@Override
public String toString() {
final StringBuilder sb = new StringBuilder("
MessagePartSentPk{");
sb.append("linkId=").append(linkId);
sb.append(", externalId='").append(externalId).append('\'');
sb.append('}');
return sb.toString();
}
}
}
public final class MessagePartPk implements Serializable {
private UUID transactionId;
private Short partNumber;
public MessagePartPk() {
//default
}
public MessagePartPk(UUID transactionId, Short partNumber) {
this.transactionId = transactionId;
this.partNumber = partNumber;
}
public UUID getTransactionId() {
return transactionId;
}
public void setTransactionId(UUID transactionId) {
this.transactionId = transactionId;
}
public Short getPartNumber() {
return partNumber;
}
public void setPartNumber(Short partNumber) {
this.partNumber = partNumber;
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
if (!(o instanceof MessagePartPk)) return false;
final MessagePartPk messagePartPk = (MessagePartPk) o;
if (partNumber != null ? !partNumber.equals(messagePartPk.partNumber)
:
messagePartPk.partNumber != null) return false;
if (transactionId != null ?
!transactionId.equals(messagePartPk.transactionId)
:
messagePartPk.transactionId != null)
return false;
return true;
}
@Override
public int hashCode() {
int result = transactionId != null ? transactionId.hashCode() : 0;
result = 31 * result + (partNumber != null ? partNumber.hashCode()
: 0);
return result;
}
@Override
public String toString() {
final StringBuilder sb = new StringBuilder("MessagePartPk{");
sb.append("transactionId=").append(transactionId);
sb.append(", partNumber=").append(partNumber);
sb.append('}');
return sb.toString();
}
On Thu, Mar 8, 2018 at 11:03 PM, Dmitriy Setrakyan <[email protected]>
wrote:
> Hi Sergey,
>
> The 2.4 release is about to be voted for. You can use the RC1 in the mean
> time:
> http://apache-ignite-developers.2346864.n4.nabble.
> com/VOTE-Apache-Ignite-2-4-0-RC1-td27687.html
>
> D.
>
> On Mon, Feb 19, 2018 at 6:43 AM, Mikhail <[email protected]>
> wrote:
>
>> Hi Sergey,
>>
>> The release of 2.4 should be soon, in a week or couple, however, there's
>> no
>> strong schedule for Apache releases.
>>
>> Could you please share a reproducer for the issue? Might be you can share
>> a
>> storage on which the issue can be reproduced?
>>
>> Thanks,
>> Mike.
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
>
--
Best regards,
Sergey S. Sergeev (aka SSSerj)
Notes: https://twitter.com/ssserj
exit(0);
