Re: Support for Java records in cache

2024-04-15 Thread Humphrey Lopez 
We have faced the same yesterday, I’m also curious if that will be supported in 
the future.

Humphrey 

> On 11 Apr 2024, at 20:12, Victor  wrote:
> 
> Hi
> 
> Using Ignite 2.16, it seems that it's unsupported yet to have a cache of 
> record classes. If so, two questions:
> 
> - is there ETA to support this out-of-the box?
> 
> - there's related github issue https://github.com/apache/ignite/issues/10846, 
> but how can I re-use this code -- not clear which Ignite API this code should 
> be using.
> 
> Thanks,
> 
> Victor
>

Re: Failed to process selector key

2023-11-14 Thread Humphrey Lopez 
Yes kubernetes is using liveness probes but not the communication port.
47100 (or the 47500 dicovery). We are using TcpDiscoveryKubernetesIpFinder
for the nodes.

  livenessProbe:
failureThreshold: 5
httpGet:
  port: 8081
  path: /actuator/liveness
  scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 5
timeoutSeconds: 3
  readinessProbe:
failureThreshold: 5
httpGet:
  port: 8081
  path: /actuator/readiness
  scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 5

Above the liveness and readiness probes.

Op di 14 nov 2023 om 16:26 schreef Jeremy McMillan <
jeremy.mcmil...@gridgain.com>:

> Is Kubernetes running liveness probes that connect to GridGain ports? That
> could be your culprit.
>
> On Tue, Nov 14, 2023 at 4:57 AM Humphrey Lopez  wrote:
>
>> We have several server nodes and thick client nodes, we have Ignite
>> embedded in spring boot. The IP address of the remote node is indeed of our
>> client. And as we have deployed everything in one namespace in kubernetes I
>> don't think something else is sending data to the server node from that IP.
>> However we are using IgniteDataStreamer to stream data to the server.
>> Before we were using Ignite 2.11 for a long time (JDK 8 and later 11), and
>> now just upgraded to JDK 17 and Ignite 2.15. Only thing we have changed
>> lately is to set the IgniteDataStreamer to allowOverwrite =  true, but I
>> don't think this is the cause. We were getting a lot of warnings when it
>> was set to false when using 2.15 of ignite.
>>
>> If I take one example:
>> (Server Node timestamp: November 14th 2023, 11:11:30.707)
>> Error:
>> <#fb58e00b> o.a.i.IgniteException: Invalid message type: 2057
>> at
>> o.a.i.i.m.c.IgniteMessageFactoryImpl.create(IgniteMessageFactoryImpl.java:133)
>> at
>> o.a.i.s.c.t.i.GridNioServerWrapper$2.create(GridNioServerWrapper.java:813)
>> at o.a.i.i.u.n.GridDirectParser.decode(GridDirectParser.java:81)
>> at
>> o.a.i.i.u.n.GridNioCodecFilter.onMessageReceived(GridNioCodecFilter.java:113)
>> at
>> o.a.i.i.u.n.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
>> at
>> o.a.i.i.u.n.GridConnectionBytesVerifyFilter.onMessageReceived(GridConnectionBytesVerifyFilter.java:133)
>> at
>> o.a.i.i.u.n.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
>> at
>> o.a.i.i.u.n.GridNioServer$HeadFilter.onMessageReceived(GridNioServer.java:3752)
>> at
>> o.a.i.i.u.n.GridNioFilterChain.onMessageReceived(GridNioFilterChain.java:175)
>> at
>> o.a.i.i.u.n.GridNioServer$DirectNioClientWorker.processRead(GridNioServer.java:1379)
>> at
>> o.a.i.i.u.n.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNioServer.java:2526)
>> at
>> o.a.i.i.u.n.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2281)
>> at
>> o.a.i.i.u.n.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1910)
>> at o.a.i.i.u.w.GridWorker.run(GridWorker.java:125)
>> at java.lang.Thread.run(Thread.java:833)
>>
>> Message:
>> Failed to read message [msg=null, buf=java.nio.DirectByteBuffer[pos=9
>> lim=334 cap=32768], reader=DirectMessageReader [state=DirectMessageState
>> [pos=0, stack=[StateItem [stream=DirectByteBufferStreamImplV2
>> [baseOff=139733741570896, arrOff=-1, tmpArrOff=0, valReadBytes=0,
>> tmpArrBytes=0, msgTypeDone=false, msg=null, mapIt=null, it=null, arrPos=-1,
>> keyDone=false, readSize=-1, readItems=0, prim=0, primShift=0, uuidState=0,
>> uuidMost=0, uuidLeast=0, uuidLocId=0], state=0], StateItem
>> [stream=DirectByteBufferStreamImplV2 [baseOff=139733741570896, arrOff=-1,
>> tmpArrOff=0, valReadBytes=0, tmpArrBytes=0, msgTypeDone=false, msg=null,
>> mapIt=null, it=null, arrPos=-1, keyDone=false, readSize=-1, readItems=0,
>> prim=0, primShift=0, uuidState=0, uuidMost=0, uuidLeast=0, uuidLocId=0],
>> state=0], StateItem [stream=DirectByteBufferStreamImplV2
>> [baseOff=139733741570896, arrOff=-1, tmpArrOff=0, valReadBytes=0,
>> tmpArrBytes=0, msgTypeDone=false, msg=null, mapIt=null, it=null, arrPos=-1,
>> keyDone=false, readSize=-1, readItems=0, prim=0, primShift=0, uuidState=0,
>> uuidMost=0, uuidLeast=0, uuidLocId=0], state=0], null, null, null, null,
>> null, null, null]], protoVer=3, lastRead=true],
>> ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker
>> [super=AbstractNioClientWorker [idx=0, bytesRcvd=107237239386,
>> bytesSent=43987266268, bytesRcvd0=3776389, bytesSent0=1

Re: Failed to process selector key

2023-11-14 Thread Humphrey Lopez 
super=GridNioSessionImpl [locAddr=/server_ip:47100,
rmtAddr=/client_ip:34662, createTime=1699700035875, closeTime=0,
bytesSent=5758138431, bytesRcvd=47248795615, bytesSent0=195008,
bytesRcvd0=1614751, sndSchedTime=1699937197520, lastSndTime=1699956690669,
lastRcvTime=1699956690699, readsPaused=false,
filterChain=FilterChain[filters=[GridNioCodecFilter
[parser=o.a.i.i.util.nio.GridDirectParser@7141a1d9, directMode=true],
GridConnectionBytesVerifyFilter], accepted=true, markedForClose=false]]]

On the client side I don't see any errors happening around that time, I
have also searched for warnings, but nothing.

I've seen this post where they are also using the streaming api and getting
similar errors.

https://lists.apache.org/thread/jgf2jrp231jd5rhdbh7f5sb8gnclocl8

My guess maybe it has to do with the datastreamer somehow?

Humphrey

Op ma 13 nov 2023 om 21:10 schreef Jeremy McMillan <
jeremy.mcmil...@gridgain.com>:

> These errors look like something which does not speak Ignite binary
> protocol is connecting and sending useless stuff to your Ignite cluster.
>
> IgniteException: Invalid message type: 2057
>
>
> Check the configuration of the client if the host generating this traffic
> is known, and check firewalls or monitoring tools if not.
>
> On Mon, Nov 13, 2023 at 8:04 AM Humphrey Lopez  wrote:
>
>> Other errors we are seeing:
>>
>> Failed to read message [msg=null, buf=java.nio.DirectByteBuffer[pos=2
>> lim=162 cap=32768], reader=DirectMessageReader [state=DirectMessageState
>> [pos=0, stack=[StateItem [stream=DirectByteBufferStreamImplV2
>> [baseOff=140381476511056, arrOff=-1, tmpArrOff=0, valReadBytes=0,
>> tmpArrBytes=0, msgTypeDone=false, msg=null, mapIt=null, it=null, arrPos=-1,
>> keyDone=false, readSize=-1, readItems=0, prim=0, primShift=0, uuidState=0,
>> uuidMost=0, uuidLeast=0, uuidLocId=0], state=0], StateItem
>> [stream=DirectByteBufferStreamImplV2 [baseOff=140381476511056, arrOff=-1,
>> tmpArrOff=0, valReadBytes=0, tmpArrBytes=0, msgTypeDone=false, msg=null,
>> mapIt=null, it=null, arrPos=-1, keyDone=false, readSize=-1, readItems=0,
>> prim=0, primShift=0, uuidState=0, uuidMost=0, uuidLeast=0, uuidLocId=0],
>> state=0], StateItem [stream=DirectByteBufferStreamImplV2
>> [baseOff=140381476511056, arrOff=-1, tmpArrOff=0, valReadBytes=0,
>> tmpArrBytes=0, msgTypeDone=false, msg=null, mapIt=null, it=null, arrPos=-1,
>> keyDone=false, readSize=-1, readItems=0, prim=0, primShift=0, uuidState=0,
>> uuidMost=0, uuidLeast=0, uuidLocId=0], state=0], null, null, null, null,
>> null, null, null]], protoVer=3, lastRead=true],
>> ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker
>> [super=AbstractNioClientWorker [idx=1, bytesRcvd=6506344847,
>> bytesSent=5800573007, bytesRcvd0=5461705, bytesSent0=197830, select=true,
>> super=GridWorker [name=grid-nio-worker-tcp-comm-1,
>> igniteInstanceName=TcpCommunicationSpi, finished=false,
>> heartbeatTs=1699706651957, hashCode=2094994491, interrupted=false,
>> runner=grid-nio-worker-tcp-comm-1-#48%TcpCommunicationSpi%]]],
>> writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768],
>> readBuf=java.nio.DirectByteBuffer[pos=2 lim=162 cap=32768],
>> inRecovery=GridNioRecoveryDescriptor [acked=47232, resendCnt=0,
>> rcvCnt=53951, sentCnt=47247, reserved=true, lastAck=53920, nodeLeft=false,
>> node=TcpDiscoveryNode [id=34cfcc64-d369-415b-b14f-6ac222087232,
>> consistentId=34cfcc64-d369-415b-b14f-6ac222087232, addrs=ArrayList
>> [xx.xxx.xx.xxx, 127.0.0.1], sockAddrs=null, discPort=0, order=24,
>> intOrder=24, lastExchangeTime=1699691906215, loc=false,
>> ver=2.15.0#20230425-sha1:f98f7f35, isClient=true], connected=true,
>> connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false],
>> outRecovery=GridNioRecoveryDescriptor [acked=47232, resendCnt=0,
>> rcvCnt=53951, sentCnt=47247, reserved=true, lastAck=53920, nodeLeft=false,
>> node=TcpDiscoveryNode [id=34cfcc64-d369-415b-b14f-6ac222087232,
>> consistentId=34cfcc64-d369-415b-b14f-6ac222087232, addrs=ArrayList
>> [xx.xxx.xx.xxx, 127.0.0.1], sockAddrs=null, discPort=0, order=24,
>> intOrder=24, lastExchangeTime=1699691906215, loc=false,
>> ver=2.15.0#20230425-sha1:f98f7f35, isClient=true], connected=true,
>> connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false],
>> closeSocket=true,
>> outboundMessagesQueueSizeMetric=o.a.i.i.processors.metric.impl.LongAdderMetric@69a257d1,
>> super=GridNioSessionImpl [locAddr=/xx.xxx.xx.xx:47100,
>> rmtAddr=/xx.xxx.xx.xxx:35492, createTime=1699700043744, closeTime=0,
>> bytesSent=74190856, bytesRcvd=167712723, bytesSent0=0, bytesRcvd0=5260541,
>> sndSchedTime=1699700043744, lastSndTime=1699706650143,
>> lastRcvTime=1

Re: Failed to process selector key

2023-11-13 Thread Humphrey Lopez 
Other errors we are seeing:

Failed to read message [msg=null, buf=java.nio.DirectByteBuffer[pos=2
lim=162 cap=32768], reader=DirectMessageReader [state=DirectMessageState
[pos=0, stack=[StateItem [stream=DirectByteBufferStreamImplV2
[baseOff=140381476511056, arrOff=-1, tmpArrOff=0, valReadBytes=0,
tmpArrBytes=0, msgTypeDone=false, msg=null, mapIt=null, it=null, arrPos=-1,
keyDone=false, readSize=-1, readItems=0, prim=0, primShift=0, uuidState=0,
uuidMost=0, uuidLeast=0, uuidLocId=0], state=0], StateItem
[stream=DirectByteBufferStreamImplV2 [baseOff=140381476511056, arrOff=-1,
tmpArrOff=0, valReadBytes=0, tmpArrBytes=0, msgTypeDone=false, msg=null,
mapIt=null, it=null, arrPos=-1, keyDone=false, readSize=-1, readItems=0,
prim=0, primShift=0, uuidState=0, uuidMost=0, uuidLeast=0, uuidLocId=0],
state=0], StateItem [stream=DirectByteBufferStreamImplV2
[baseOff=140381476511056, arrOff=-1, tmpArrOff=0, valReadBytes=0,
tmpArrBytes=0, msgTypeDone=false, msg=null, mapIt=null, it=null, arrPos=-1,
keyDone=false, readSize=-1, readItems=0, prim=0, primShift=0, uuidState=0,
uuidMost=0, uuidLeast=0, uuidLocId=0], state=0], null, null, null, null,
null, null, null]], protoVer=3, lastRead=true],
ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker
[super=AbstractNioClientWorker [idx=1, bytesRcvd=6506344847,
bytesSent=5800573007, bytesRcvd0=5461705, bytesSent0=197830, select=true,
super=GridWorker [name=grid-nio-worker-tcp-comm-1,
igniteInstanceName=TcpCommunicationSpi, finished=false,
heartbeatTs=1699706651957, hashCode=2094994491, interrupted=false,
runner=grid-nio-worker-tcp-comm-1-#48%TcpCommunicationSpi%]]],
writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768],
readBuf=java.nio.DirectByteBuffer[pos=2 lim=162 cap=32768],
inRecovery=GridNioRecoveryDescriptor [acked=47232, resendCnt=0,
rcvCnt=53951, sentCnt=47247, reserved=true, lastAck=53920, nodeLeft=false,
node=TcpDiscoveryNode [id=34cfcc64-d369-415b-b14f-6ac222087232,
consistentId=34cfcc64-d369-415b-b14f-6ac222087232, addrs=ArrayList
[xx.xxx.xx.xxx, 127.0.0.1], sockAddrs=null, discPort=0, order=24,
intOrder=24, lastExchangeTime=1699691906215, loc=false,
ver=2.15.0#20230425-sha1:f98f7f35, isClient=true], connected=true,
connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false],
outRecovery=GridNioRecoveryDescriptor [acked=47232, resendCnt=0,
rcvCnt=53951, sentCnt=47247, reserved=true, lastAck=53920, nodeLeft=false,
node=TcpDiscoveryNode [id=34cfcc64-d369-415b-b14f-6ac222087232,
consistentId=34cfcc64-d369-415b-b14f-6ac222087232, addrs=ArrayList
[xx.xxx.xx.xxx, 127.0.0.1], sockAddrs=null, discPort=0, order=24,
intOrder=24, lastExchangeTime=1699691906215, loc=false,
ver=2.15.0#20230425-sha1:f98f7f35, isClient=true], connected=true,
connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false],
closeSocket=true,
outboundMessagesQueueSizeMetric=o.a.i.i.processors.metric.impl.LongAdderMetric@69a257d1,
super=GridNioSessionImpl [locAddr=/xx.xxx.xx.xx:47100,
rmtAddr=/xx.xxx.xx.xxx:35492, createTime=1699700043744, closeTime=0,
bytesSent=74190856, bytesRcvd=167712723, bytesSent0=0, bytesRcvd0=5260541,
sndSchedTime=1699700043744, lastSndTime=1699706650143,
lastRcvTime=1699706651957, readsPaused=false,
filterChain=FilterChain[filters=[GridNioCodecFilter
[parser=o.a.i.i.util.nio.GridDirectParser@6c311b05, directMode=true],
GridConnectionBytesVerifyFilter], accepted=true, markedForClose=false]]]
<#fb58e00b> o.a.i.IgniteException: Invalid message type: 2057 at
o.a.i.i.m.c.IgniteMessageFactoryImpl.create(IgniteMessageFactoryImpl.java:133)
at
o.a.i.s.c.t.i.GridNioServerWrapper$2.create(GridNioServerWrapper.java:813)
at o.a.i.i.u.n.GridDirectParser.decode(GridDirectParser.java:81) at
o.a.i.i.u.n.GridNioCodecFilter.onMessageReceived(GridNioCodecFilter.java:113)
at
o.a.i.i.u.n.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
at
o.a.i.i.u.n.GridConnectionBytesVerifyFilter.onMessageReceived(GridConnectionBytesVerifyFilter.java:133)
at
o.a.i.i.u.n.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
at
o.a.i.i.u.n.GridNioServer$HeadFilter.onMessageReceived(GridNioServer.java:3752)
at
o.a.i.i.u.n.GridNioFilterChain.onMessageReceived(GridNioFilterChain.java:175)
at
o.a.i.i.u.n.GridNioServer$DirectNioClientWorker.processRead(GridNioServer.java:1379)
at
o.a.i.i.u.n.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNioServer.java:2526)
at
o.a.i.i.u.n.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2281)
at
o.a.i.i.u.n.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1910)
at o.a.i.i.u.w.GridWorker.run(GridWorker.java:125) at
java.lang.Thread.run(Thread.java:833)

Op ma 13 nov 2023 om 14:33 schreef Humphrey Lopez :

> Hello Ignite community.
>
> We are running Ignite 2.15 in production with JDK 17. We are seeing the
> following errors and have no idea what is causing it.
>
> Failed to process selector key [ses=GridSelector

Failed to process selector key

2023-11-13 Thread Humphrey Lopez 
Hello Ignite community.

We are running Ignite 2.15 in production with JDK 17. We are seeing the
following errors and have no idea what is causing it.

Failed to process selector key [ses=GridSelectorNioSessionImpl
[worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=3,
bytesRcvd=64732383766, bytesSent=30081901336, bytesRcvd0=0, bytesSent0=0,
select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-3,
igniteInstanceName=TcpCommunicationSpi, finished=false,
heartbeatTs=1699879571052, hashCode=475467093, interrupted=false,
runner=grid-nio-worker-tcp-comm-3-#50%TcpCommunicationSpi%]]],
writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768],
readBuf=java.nio.DirectByteBuffer[pos=10 lim=282 cap=32768],
inRecovery=GridNioRecoveryDescriptor [acked=209920, resendCnt=0,
rcvCnt=229599, sentCnt=209938, reserved=true, lastAck=229568,
nodeLeft=false, node=TcpDiscoveryNode
[id=c8353de9-9cd2-4ae5-bc48-3271c47fffae,
consistentId=c8353de9-9cd2-4ae5-bc48-3271c47fffae, addrs=ArrayList
[machine1, 127.0.0.1], sockAddrs=null, discPort=0, order=26, intOrder=26,
lastExchangeTime=1699691907837, loc=false,
ver=2.15.0#20230425-sha1:f98f7f35, isClient=true], connected=true,
connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false],
outRecovery=GridNioRecoveryDescriptor [acked=209920, resendCnt=0,
rcvCnt=229599, sentCnt=209938, reserved=true, lastAck=229568,
nodeLeft=false, node=TcpDiscoveryNode
[id=c8353de9-9cd2-4ae5-bc48-3271c47fffae,
consistentId=c8353de9-9cd2-4ae5-bc48-3271c47fffae, addrs=ArrayList
[machine1, 127.0.0.1], sockAddrs=null, discPort=0, order=26, intOrder=26,
lastExchangeTime=1699691907837, loc=false,
ver=2.15.0#20230425-sha1:f98f7f35, isClient=true], connected=true,
connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false],
closeSocket=true,
outboundMessagesQueueSizeMetric=o.a.i.i.processors.metric.impl.LongAdderMetric@69a257d1,
super=GridNioSessionImpl [locAddr=/10.129.34.235:47100,
rmtAddr=/machine1:42492, createTime=1699700047871, closeTime=0,
bytesSent=3311788770, bytesRcvd=23387281236, bytesSent0=0, bytesRcvd0=0,
sndSchedTime=1699852982273, lastSndTime=1699879565624,
lastRcvTime=1699879571052, readsPaused=false,
filterChain=FilterChain[filters=[GridNioCodecFilter
[parser=o.a.i.i.util.nio.GridDirectParser@f6e5016, directMode=true],
GridConnectionBytesVerifyFilter], accepted=true, markedForClose=false]]]

And there is also this stacktrace:
j.l.NullPointerException: Cannot invoke "Object.hashCode()" because "key"
is null
at j.u.c.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
at o.a.i.i.m.c.GridIoManager.processOrderedMessage(GridIoManager.java:1707)
at o.a.i.i.m.c.GridIoManager.onMessage0(GridIoManager.java:1328)
at o.a.i.i.m.c.GridIoManager.access$300(GridIoManager.java:243)
at o.a.i.i.m.c.GridIoManager$2.onMessage(GridIoManager.java:509)
at
o.a.i.s.c.t.TcpCommunicationSpi.notifyListener(TcpCommunicationSpi.java:1220)
at o.a.i.s.c.t.TcpCommunicationSpi$1.onMessage(TcpCommunicationSpi.java:689)
at o.a.i.s.c.t.TcpCommunicationSpi$1.onMessage(TcpCommunicationSpi.java:687)
at
o.a.i.s.c.t.i.InboundConnectionHandler.onMessage(InboundConnectionHandler.java:392)
at
o.a.i.s.c.t.i.InboundConnectionHandler.onMessage(InboundConnectionHandler.java:78)
at
o.a.i.i.u.n.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279)
at
o.a.i.i.u.n.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
at
o.a.i.i.u.n.GridNioCodecFilter.onMessageReceived(GridNioCodecFilter.java:116)
at
o.a.i.i.u.n.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
at
o.a.i.i.u.n.GridConnectionBytesVerifyFilter.onMessageReceived(GridConnectionBytesVerifyFilter.java:88)
at
o.a.i.i.u.n.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
at
o.a.i.i.u.n.GridNioServer$HeadFilter.onMessageReceived(GridNioServer.java:3752)
at
o.a.i.i.u.n.GridNioFilterChain.onMessageReceived(GridNioFilterChain.java:175)
at
o.a.i.i.u.n.GridNioServer$DirectNioClientWorker.processRead(GridNioServer.java:1379)
at
o.a.i.i.u.n.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNioServer.java:2526)
at
o.a.i.i.u.n.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2281)
at
o.a.i.i.u.n.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1910)
at o.a.i.i.u.w.GridWorker.run(GridWorker.java:125)
at java.lang.Thread.run(Thread.java:833)

We are using the following flags:

"-XX:+AlwaysPreTouch",
"-XX:+UseG1GC",
"-XX:+ScavengeBeforeFullGC",
"-XX:+DisableExplicitGC",
"-XX:MaxMetaspaceSize=640m",
"-Djava.net.preferIPv4Stack=true",
"-DIGNITE_QUIET=false",
"-DIGNITE_UPDATE_NOTIFIER=false",
"-DIGNITE_WAIT_FOR_BACKUPS_ON_SHUTDOWN=true",
"--add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED",
"--add-opens=java.base/jdk.internal.misc=ALL-UNNAMED",
"--add-opens=java.base/sun.nio.ch=ALL-UNNAMED",
"--add-opens=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED",

Ignite website shows downloading version 2.11 as latest version

2023-09-26 Thread Humphrey Lopez 
If you narrow the page (or you are on your phone) you get a menu, when
clicking on the menu it shows below an older version of apache ignite to
download. See screenshot below.

[image: image.png]

Kubernetes liveness and readiness probes

2023-08-16 Thread Humphrey Lopez 
Has anyone deploy ignite in kubernetes and supply a readiness probe? If I try 
to start up like 10 nodes at the same time with the readiness probe they all 
start up as separate cluster, whereas if I don’t supply the probes they are 
finding each other and I’ll have one cluster. 
I’m using the kubernetes IpFinder.
Used version 2.15 of ignite (Java/Kotlin) and spring boot.


Humphrey

ignite-http-rest

2023-08-13 Thread Humphrey Lopez 
When running ignite http rest with spring boot 3, i'm getting the following
error:

java.lang.AbstractMethodError: Receiver class
org.apache.ignite.internal.processors.rest.protocols.http.jetty.GridJettyRestHandler
does not define or inherit an implementation of the resolved method
'abstract void handle(java.lang.String,
org.eclipse.jetty.server.Request,
jakarta.servlet.http.HttpServletRequest,
jakarta.servlet.http.HttpServletResponse)' of interface
org.eclipse.jetty.server.Handler. Selected method is 'abstract void
org.eclipse.jetty.server.handler.AbstractHandler.handle(java.lang.String,
org.eclipse.jetty.server.Request,
jakarta.servlet.http.HttpServletRequest,
jakarta.servlet.http.HttpServletResponse)'.
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
at org.eclipse.jetty.server.Server.handle(Server.java:563)
at 
org.eclipse.jetty.server.HttpChannel.lambda$handle$0(HttpChannel.java:505)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:762)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:497)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:282)
at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
at 
org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
at java.base/java.lang.Thread.run(Thread.java:1589)


I've looked in the code, it only has method for javax
(javax.servlet.http.HttpServletRequest) and not for the jakarta
(jakarta.servlet.http.HttpServletRequest) interfaces. Will this be picked
up any time soon, as the javax is dead, and everyone is using jakarta now?
(Tested with Ignite version 2.15)

Query produced big result set.

2023-07-25 Thread Humphrey Lopez 
Hello all,

When we are executing some query on Ignite we are seeing the warning
message:
"Query produced big result set."
fetched=10, duration=676ms, type=LOCAL, distributedJoin=false,
enforceJoinOrder=false, lazy=false

Diving into the code I found it here:
org.apache.ignite.internal.processors.query.running.HeavyQueriesTracker

It has a default threshold of 100_000, and when our queries are returning
more than 100_000 results we get that log message.

The code snippet is:
/**
* Print warning message to log when query result size fetch count is bigger
than specified threshold.
* Threshold may be recalculated with multiplier.
*/
public void checkOnFetchNext() {
++fetchedSize;

if (threshold > 0 && fetchedSize >= threshold) {
LT.warn(log, BIG_RESULT_SET_MSG + qryInfo.queryInfo("fetched=" + fetchedSize
));

if (thresholdMult > 1)
threshold *= thresholdMult;
else
threshold = 0;

bigResults = true;
}
}

But I don't see a place where I can define a threshold in Ignite
(IgniteConfiguration, CacheConfiguration or SqlConfiguration etc) I would
like to set it on the query or on the cache or ignite configuration.

I do see a usage in
org.apache.ignite.internal.processors.query.running.SqlQueryMXBeanImpl
/** {@inheritDoc} */
@Override public void setResultSetSizeThreshold(long rsSizeThreshold) {
heavyQrysTracker.setResultSetSizeThreshold(rsSizeThreshold);
}

But this is not a configuration item.

- Where can I (set / configure) this Threshold in Ignite to a higher value
so we don't get those warnings any longer?
- if we increase the number of nodes, will it help (less data on one node)?

Thanks.

Humphrey

Re: Spring (Boot) version

2023-04-16 Thread Humphrey Lopez 
I'm sure it works well with 2.x versions. I'm wondering with 3.x? Or is
Ignite only using generic Spring API that will not change with new
versions? I'm sure I can try it out myself (doing so with example projects)
but not sure if the developers have tested it.

Humphrey

Op do 13 apr 2023 om 09:04 schreef f cad :

> we used with springboot 2.6.7. it works fine
>
> Humphrey Lopez  于2023年4月7日周五 14:47写道：
>
>> Currently we are running Ignite with spring boot 2.x version. Is any
>> version of Spring (Boot) supported? Can we for example want to upgrade to
>> spring boot 3.x ?
>>
>> Humphrey
>>
>

Spring (Boot) version

2023-04-07 Thread Humphrey Lopez 
Currently we are running Ignite with spring boot 2.x version. Is any
version of Spring (Boot) supported? Can we for example want to upgrade to
spring boot 3.x ?

Humphrey

Re: Key1 equals Key2 but not found in cache

2023-03-22 Thread Humphrey Lopez 
Can we query the keys as well? Adding QuerySqlField? I think this only
applies to the values right?

Op wo 22 mrt 2023 om 16:08 schreef Stephen Darlington <
stephen.darling...@gridgain.com>:

> You don’t want to look through all the keys. That’s why Ignite has SQL.
>
> On 22 Mar 2023, at 14:44, Humphrey Lopez  wrote:
>
> We have FAT keys that contain information about the Values we store.
> KEY
> - Object A
> - Object B
>
> VALUE
> - Object Y
>
> The KEY we are using contains several (small) objects as fields. We have
> added to Object A a new field A.TYPE so when searching through the cache we
> could filter on that type.
> I guess we will need to first get all the keys that match that A.TYPE and
> then get the objects with those objects, I think looping through the keys
> will be quicker than looping through all the values.
> Maybe we can query/filter the keys in BinaryMode, and then get all
> corresponding values? Just thinking out loud. We don't want to be
> deserializing unnecessarily values (Object.Y). Maybe with QuerySqlFields on
> the key?
>
> Humphrey
>
> Op wo 22 mrt 2023 om 15:18 schreef Rick Lee :
>
>>
>> Why don’t u put the type to value and leave the id as the key?
>> 「Humphrey Lopez 」在 2023年3月22日 週三，下午10:16 寫道：
>>
>>> Okay transient is also not the way to go, cause we want to be able to
>>> filter sometimes on that field, so it should be there in the cache. But
>>> thanks for clarifying that the Equals and HashCode is not being used on
>>> BinaryObjects.
>>>
>>> Op wo 22 mrt 2023 om 14:51 schreef Humphrey Lopez :
>>>
>>>> I see marking the field as Transient Ignite won't serialize it to
>>>> BinaryObject, is that the way to go?
>>>>
>>>> Humphrey
>>>>
>>>> Op wo 22 mrt 2023 om 14:37 schreef Humphrey Lopez :
>>>>
>>>>> Thanks for clarifying that. Is there a way to mark a property/field to
>>>>> be excluded when storing?
>>>>>
>>>>> Humphrey
>>>>>
>>>>> Op wo 22 mrt 2023 om 14:20 schreef Stephen Darlington <
>>>>> stephen.darling...@gridgain.com>:
>>>>>
>>>>>> Ignite doesn’t use your equals or hashCode implementation. Data is
>>>>>> stored as a BinaryObject, and it’s that that is compared for equality.
>>>>>>
>>>>>> On 22 Mar 2023, at 12:14, Humphrey Lopez  wrote:
>>>>>>
>>>>>> They are in the example only checking the first field when overriding
>>>>>> the equals. And hashCode always returns 1.
>>>>>>
>>>>>> Op wo 22 mrt 2023 om 13:06 schreef Prigoreanu, Alexandru <
>>>>>> prigoreanu.alexan...@anteash.com>:
>>>>>>
>>>>>>> hashCode and equals should depend on the same fields.
>>>>>>>
>>>>>>> On Wed, Mar 22, 2023 at 8:02 AM Humphrey Lopez 
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hello, when having a key which equals another key, when trying to
>>>>>>>> retrieve from cache it does not return the expected value. Is this a 
>>>>>>>> bug?
>>>>>>>>
>>>>>>>> I have a reproducible below in kotlin but in java we get the same
>>>>>>>> result (test with Ignite 2.10 and  2.14) and java 11 and 19.
>>>>>>>>
>>>>>>>>
>>>>>>>> import org.apache.ignite.Ignition
>>>>>>>> import org.apache.ignite.configuration.CacheConfiguration
>>>>>>>> import org.assertj.core.api.SoftAssertions
>>>>>>>> import org.junit.jupiter.api.Test
>>>>>>>>
>>>>>>>> class MyTest {
>>>>>>>>
>>>>>>>> private val key1 = MyKey("ABC", "DEF")
>>>>>>>> private val key2 = MyKey("ABC", "xxx")
>>>>>>>>
>>>>>>>> @Test
>>>>>>>> fun testEquals() {
>>>>>>>> SoftAssertions.assertSoftly {
>>>>>>>> it.assertThat(key1).isEqualTo(key2)
>>>>>>>> it.assertThat(key1 == key2).isTrue
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> @Test
>>>>>>>> fun testWithMap() {
>>>>>>>> val map = mapOf(Pair(key1, "key1"))
>>>>>>>>
>>>>>>>> SoftAssertions.assertSoftly {
>>>>>>>> it.assertThat(map.containsKey(key1)).isTrue
>>>>>>>> it.assertThat(map.containsKey(key2)).isTrue
>>>>>>>> }
>>>>>>>>
>>>>>>>> }
>>>>>>>>
>>>>>>>> @Test
>>>>>>>> fun testWithIgnite() {
>>>>>>>> val ignite = Ignition.start();
>>>>>>>> val cache = ignite.createCache(CacheConfiguration>>>>>>> String>("mycache"))
>>>>>>>>
>>>>>>>> cache.put(key1, "key1")
>>>>>>>> SoftAssertions.assertSoftly {
>>>>>>>> it.assertThat(cache.containsKey(key1)).isTrue
>>>>>>>> it.assertThat(cache.containsKey(key2)).isTrue
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> private data class MyKey(val id: String, val type: String) {
>>>>>>>> override fun equals(other: Any?): Boolean {
>>>>>>>> if (other is MyKey)
>>>>>>>> return id == other.id
>>>>>>>> return false
>>>>>>>> }
>>>>>>>>
>>>>>>>> override fun hashCode(): Int {
>>>>>>>> return 1
>>>>>>>> }
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>
>

Re: Key1 equals Key2 but not found in cache

2023-03-22 Thread Humphrey Lopez 
We have FAT keys that contain information about the Values we store.
KEY
- Object A
- Object B

VALUE
- Object Y

The KEY we are using contains several (small) objects as fields. We have
added to Object A a new field A.TYPE so when searching through the cache we
could filter on that type.
I guess we will need to first get all the keys that match that A.TYPE and
then get the objects with those objects, I think looping through the keys
will be quicker than looping through all the values.
Maybe we can query/filter the keys in BinaryMode, and then get all
corresponding values? Just thinking out loud. We don't want to be
deserializing unnecessarily values (Object.Y). Maybe with QuerySqlFields on
the key?

Humphrey

Op wo 22 mrt 2023 om 15:18 schreef Rick Lee :

>
> Why don’t u put the type to value and leave the id as the key?
> 「Humphrey Lopez 」在 2023年3月22日 週三，下午10:16 寫道：
>
>> Okay transient is also not the way to go, cause we want to be able to
>> filter sometimes on that field, so it should be there in the cache. But
>> thanks for clarifying that the Equals and HashCode is not being used on
>> BinaryObjects.
>>
>> Op wo 22 mrt 2023 om 14:51 schreef Humphrey Lopez :
>>
>>> I see marking the field as Transient Ignite won't serialize it to
>>> BinaryObject, is that the way to go?
>>>
>>> Humphrey
>>>
>>> Op wo 22 mrt 2023 om 14:37 schreef Humphrey Lopez :
>>>
>>>> Thanks for clarifying that. Is there a way to mark a property/field to
>>>> be excluded when storing?
>>>>
>>>> Humphrey
>>>>
>>>> Op wo 22 mrt 2023 om 14:20 schreef Stephen Darlington <
>>>> stephen.darling...@gridgain.com>:
>>>>
>>>>> Ignite doesn’t use your equals or hashCode implementation. Data is
>>>>> stored as a BinaryObject, and it’s that that is compared for equality.
>>>>>
>>>>> On 22 Mar 2023, at 12:14, Humphrey Lopez  wrote:
>>>>>
>>>>> They are in the example only checking the first field when overriding
>>>>> the equals. And hashCode always returns 1.
>>>>>
>>>>> Op wo 22 mrt 2023 om 13:06 schreef Prigoreanu, Alexandru <
>>>>> prigoreanu.alexan...@anteash.com>:
>>>>>
>>>>>> hashCode and equals should depend on the same fields.
>>>>>>
>>>>>> On Wed, Mar 22, 2023 at 8:02 AM Humphrey Lopez 
>>>>>> wrote:
>>>>>>
>>>>>>> Hello, when having a key which equals another key, when trying to
>>>>>>> retrieve from cache it does not return the expected value. Is this a 
>>>>>>> bug?
>>>>>>>
>>>>>>> I have a reproducible below in kotlin but in java we get the same
>>>>>>> result (test with Ignite 2.10 and  2.14) and java 11 and 19.
>>>>>>>
>>>>>>>
>>>>>>> import org.apache.ignite.Ignition
>>>>>>> import org.apache.ignite.configuration.CacheConfiguration
>>>>>>> import org.assertj.core.api.SoftAssertions
>>>>>>> import org.junit.jupiter.api.Test
>>>>>>>
>>>>>>> class MyTest {
>>>>>>>
>>>>>>> private val key1 = MyKey("ABC", "DEF")
>>>>>>> private val key2 = MyKey("ABC", "xxx")
>>>>>>>
>>>>>>> @Test
>>>>>>> fun testEquals() {
>>>>>>> SoftAssertions.assertSoftly {
>>>>>>> it.assertThat(key1).isEqualTo(key2)
>>>>>>> it.assertThat(key1 == key2).isTrue
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> @Test
>>>>>>> fun testWithMap() {
>>>>>>> val map = mapOf(Pair(key1, "key1"))
>>>>>>>
>>>>>>> SoftAssertions.assertSoftly {
>>>>>>> it.assertThat(map.containsKey(key1)).isTrue
>>>>>>> it.assertThat(map.containsKey(key2)).isTrue
>>>>>>> }
>>>>>>>
>>>>>>> }
>>>>>>>
>>>>>>> @Test
>>>>>>> fun testWithIgnite() {
>>>>>>> val ignite = Ignition.start();
>>>>>>> val cache = ignite.createCache(CacheConfiguration>>>>>> String>("mycache"))
>>>>>>>
>>>>>>> cache.put(key1, "key1")
>>>>>>> SoftAssertions.assertSoftly {
>>>>>>> it.assertThat(cache.containsKey(key1)).isTrue
>>>>>>> it.assertThat(cache.containsKey(key2)).isTrue
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> private data class MyKey(val id: String, val type: String) {
>>>>>>> override fun equals(other: Any?): Boolean {
>>>>>>> if (other is MyKey)
>>>>>>> return id == other.id
>>>>>>> return false
>>>>>>> }
>>>>>>>
>>>>>>> override fun hashCode(): Int {
>>>>>>> return 1
>>>>>>> }
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>

Re: Key1 equals Key2 but not found in cache

2023-03-22 Thread Humphrey Lopez 
Okay transient is also not the way to go, cause we want to be able to
filter sometimes on that field, so it should be there in the cache. But
thanks for clarifying that the Equals and HashCode is not being used on
BinaryObjects.

Op wo 22 mrt 2023 om 14:51 schreef Humphrey Lopez :

> I see marking the field as Transient Ignite won't serialize it to
> BinaryObject, is that the way to go?
>
> Humphrey
>
> Op wo 22 mrt 2023 om 14:37 schreef Humphrey Lopez :
>
>> Thanks for clarifying that. Is there a way to mark a property/field to be
>> excluded when storing?
>>
>> Humphrey
>>
>> Op wo 22 mrt 2023 om 14:20 schreef Stephen Darlington <
>> stephen.darling...@gridgain.com>:
>>
>>> Ignite doesn’t use your equals or hashCode implementation. Data is
>>> stored as a BinaryObject, and it’s that that is compared for equality.
>>>
>>> On 22 Mar 2023, at 12:14, Humphrey Lopez  wrote:
>>>
>>> They are in the example only checking the first field when overriding
>>> the equals. And hashCode always returns 1.
>>>
>>> Op wo 22 mrt 2023 om 13:06 schreef Prigoreanu, Alexandru <
>>> prigoreanu.alexan...@anteash.com>:
>>>
>>>> hashCode and equals should depend on the same fields.
>>>>
>>>> On Wed, Mar 22, 2023 at 8:02 AM Humphrey Lopez 
>>>> wrote:
>>>>
>>>>> Hello, when having a key which equals another key, when trying to
>>>>> retrieve from cache it does not return the expected value. Is this a bug?
>>>>>
>>>>> I have a reproducible below in kotlin but in java we get the same
>>>>> result (test with Ignite 2.10 and  2.14) and java 11 and 19.
>>>>>
>>>>>
>>>>> import org.apache.ignite.Ignition
>>>>> import org.apache.ignite.configuration.CacheConfiguration
>>>>> import org.assertj.core.api.SoftAssertions
>>>>> import org.junit.jupiter.api.Test
>>>>>
>>>>> class MyTest {
>>>>>
>>>>> private val key1 = MyKey("ABC", "DEF")
>>>>> private val key2 = MyKey("ABC", "xxx")
>>>>>
>>>>> @Test
>>>>> fun testEquals() {
>>>>> SoftAssertions.assertSoftly {
>>>>> it.assertThat(key1).isEqualTo(key2)
>>>>> it.assertThat(key1 == key2).isTrue
>>>>> }
>>>>> }
>>>>>
>>>>> @Test
>>>>> fun testWithMap() {
>>>>> val map = mapOf(Pair(key1, "key1"))
>>>>>
>>>>> SoftAssertions.assertSoftly {
>>>>> it.assertThat(map.containsKey(key1)).isTrue
>>>>> it.assertThat(map.containsKey(key2)).isTrue
>>>>> }
>>>>>
>>>>> }
>>>>>
>>>>> @Test
>>>>> fun testWithIgnite() {
>>>>> val ignite = Ignition.start();
>>>>> val cache = ignite.createCache(CacheConfiguration>>>> String>("mycache"))
>>>>>
>>>>> cache.put(key1, "key1")
>>>>> SoftAssertions.assertSoftly {
>>>>> it.assertThat(cache.containsKey(key1)).isTrue
>>>>> it.assertThat(cache.containsKey(key2)).isTrue
>>>>> }
>>>>> }
>>>>>
>>>>> private data class MyKey(val id: String, val type: String) {
>>>>> override fun equals(other: Any?): Boolean {
>>>>> if (other is MyKey)
>>>>> return id == other.id
>>>>> return false
>>>>> }
>>>>>
>>>>> override fun hashCode(): Int {
>>>>> return 1
>>>>> }
>>>>> }
>>>>> }
>>>>>
>>>>>
>>>

Re: Key1 equals Key2 but not found in cache

2023-03-22 Thread Humphrey Lopez 
I see marking the field as Transient Ignite won't serialize it to
BinaryObject, is that the way to go?

Humphrey

Op wo 22 mrt 2023 om 14:37 schreef Humphrey Lopez :

> Thanks for clarifying that. Is there a way to mark a property/field to be
> excluded when storing?
>
> Humphrey
>
> Op wo 22 mrt 2023 om 14:20 schreef Stephen Darlington <
> stephen.darling...@gridgain.com>:
>
>> Ignite doesn’t use your equals or hashCode implementation. Data is stored
>> as a BinaryObject, and it’s that that is compared for equality.
>>
>> On 22 Mar 2023, at 12:14, Humphrey Lopez  wrote:
>>
>> They are in the example only checking the first field when overriding the
>> equals. And hashCode always returns 1.
>>
>> Op wo 22 mrt 2023 om 13:06 schreef Prigoreanu, Alexandru <
>> prigoreanu.alexan...@anteash.com>:
>>
>>> hashCode and equals should depend on the same fields.
>>>
>>> On Wed, Mar 22, 2023 at 8:02 AM Humphrey Lopez 
>>> wrote:
>>>
>>>> Hello, when having a key which equals another key, when trying to
>>>> retrieve from cache it does not return the expected value. Is this a bug?
>>>>
>>>> I have a reproducible below in kotlin but in java we get the same
>>>> result (test with Ignite 2.10 and  2.14) and java 11 and 19.
>>>>
>>>>
>>>> import org.apache.ignite.Ignition
>>>> import org.apache.ignite.configuration.CacheConfiguration
>>>> import org.assertj.core.api.SoftAssertions
>>>> import org.junit.jupiter.api.Test
>>>>
>>>> class MyTest {
>>>>
>>>> private val key1 = MyKey("ABC", "DEF")
>>>> private val key2 = MyKey("ABC", "xxx")
>>>>
>>>> @Test
>>>> fun testEquals() {
>>>> SoftAssertions.assertSoftly {
>>>> it.assertThat(key1).isEqualTo(key2)
>>>> it.assertThat(key1 == key2).isTrue
>>>> }
>>>> }
>>>>
>>>> @Test
>>>> fun testWithMap() {
>>>> val map = mapOf(Pair(key1, "key1"))
>>>>
>>>> SoftAssertions.assertSoftly {
>>>> it.assertThat(map.containsKey(key1)).isTrue
>>>> it.assertThat(map.containsKey(key2)).isTrue
>>>> }
>>>>
>>>> }
>>>>
>>>> @Test
>>>> fun testWithIgnite() {
>>>> val ignite = Ignition.start();
>>>> val cache = ignite.createCache(CacheConfiguration>>> String>("mycache"))
>>>>
>>>> cache.put(key1, "key1")
>>>> SoftAssertions.assertSoftly {
>>>> it.assertThat(cache.containsKey(key1)).isTrue
>>>> it.assertThat(cache.containsKey(key2)).isTrue
>>>> }
>>>> }
>>>>
>>>> private data class MyKey(val id: String, val type: String) {
>>>> override fun equals(other: Any?): Boolean {
>>>> if (other is MyKey)
>>>> return id == other.id
>>>> return false
>>>> }
>>>>
>>>> override fun hashCode(): Int {
>>>> return 1
>>>> }
>>>> }
>>>> }
>>>>
>>>>
>>

Re: Key1 equals Key2 but not found in cache

2023-03-22 Thread Humphrey Lopez 
Thanks for clarifying that. Is there a way to mark a property/field to be
excluded when storing?

Humphrey

Op wo 22 mrt 2023 om 14:20 schreef Stephen Darlington <
stephen.darling...@gridgain.com>:

> Ignite doesn’t use your equals or hashCode implementation. Data is stored
> as a BinaryObject, and it’s that that is compared for equality.
>
> On 22 Mar 2023, at 12:14, Humphrey Lopez  wrote:
>
> They are in the example only checking the first field when overriding the
> equals. And hashCode always returns 1.
>
> Op wo 22 mrt 2023 om 13:06 schreef Prigoreanu, Alexandru <
> prigoreanu.alexan...@anteash.com>:
>
>> hashCode and equals should depend on the same fields.
>>
>> On Wed, Mar 22, 2023 at 8:02 AM Humphrey Lopez 
>> wrote:
>>
>>> Hello, when having a key which equals another key, when trying to
>>> retrieve from cache it does not return the expected value. Is this a bug?
>>>
>>> I have a reproducible below in kotlin but in java we get the same result
>>> (test with Ignite 2.10 and  2.14) and java 11 and 19.
>>>
>>>
>>> import org.apache.ignite.Ignition
>>> import org.apache.ignite.configuration.CacheConfiguration
>>> import org.assertj.core.api.SoftAssertions
>>> import org.junit.jupiter.api.Test
>>>
>>> class MyTest {
>>>
>>> private val key1 = MyKey("ABC", "DEF")
>>> private val key2 = MyKey("ABC", "xxx")
>>>
>>> @Test
>>> fun testEquals() {
>>> SoftAssertions.assertSoftly {
>>> it.assertThat(key1).isEqualTo(key2)
>>> it.assertThat(key1 == key2).isTrue
>>> }
>>> }
>>>
>>> @Test
>>> fun testWithMap() {
>>> val map = mapOf(Pair(key1, "key1"))
>>>
>>> SoftAssertions.assertSoftly {
>>> it.assertThat(map.containsKey(key1)).isTrue
>>> it.assertThat(map.containsKey(key2)).isTrue
>>> }
>>>
>>> }
>>>
>>> @Test
>>> fun testWithIgnite() {
>>> val ignite = Ignition.start();
>>> val cache = ignite.createCache(CacheConfiguration>> String>("mycache"))
>>>
>>> cache.put(key1, "key1")
>>> SoftAssertions.assertSoftly {
>>> it.assertThat(cache.containsKey(key1)).isTrue
>>> it.assertThat(cache.containsKey(key2)).isTrue
>>> }
>>> }
>>>
>>> private data class MyKey(val id: String, val type: String) {
>>> override fun equals(other: Any?): Boolean {
>>> if (other is MyKey)
>>> return id == other.id
>>> return false
>>> }
>>>
>>> override fun hashCode(): Int {
>>> return 1
>>> }
>>> }
>>> }
>>>
>>>
>

Re: Key1 equals Key2 but not found in cache

2023-03-22 Thread Humphrey Lopez 
They are in the example only checking the first field when overriding the
equals. And hashCode always returns 1.

Op wo 22 mrt 2023 om 13:06 schreef Prigoreanu, Alexandru <
prigoreanu.alexan...@anteash.com>:

> hashCode and equals should depend on the same fields.
>
> On Wed, Mar 22, 2023 at 8:02 AM Humphrey Lopez  wrote:
>
>> Hello, when having a key which equals another key, when trying to
>> retrieve from cache it does not return the expected value. Is this a bug?
>>
>> I have a reproducible below in kotlin but in java we get the same result
>> (test with Ignite 2.10 and  2.14) and java 11 and 19.
>>
>>
>> import org.apache.ignite.Ignition
>> import org.apache.ignite.configuration.CacheConfiguration
>> import org.assertj.core.api.SoftAssertions
>> import org.junit.jupiter.api.Test
>>
>> class MyTest {
>>
>> private val key1 = MyKey("ABC", "DEF")
>> private val key2 = MyKey("ABC", "xxx")
>>
>> @Test
>> fun testEquals() {
>> SoftAssertions.assertSoftly {
>> it.assertThat(key1).isEqualTo(key2)
>> it.assertThat(key1 == key2).isTrue
>> }
>> }
>>
>> @Test
>> fun testWithMap() {
>> val map = mapOf(Pair(key1, "key1"))
>>
>> SoftAssertions.assertSoftly {
>> it.assertThat(map.containsKey(key1)).isTrue
>> it.assertThat(map.containsKey(key2)).isTrue
>> }
>>
>> }
>>
>> @Test
>> fun testWithIgnite() {
>> val ignite = Ignition.start();
>> val cache = ignite.createCache(CacheConfiguration> String>("mycache"))
>>
>> cache.put(key1, "key1")
>> SoftAssertions.assertSoftly {
>> it.assertThat(cache.containsKey(key1)).isTrue
>> it.assertThat(cache.containsKey(key2)).isTrue
>> }
>> }
>>
>> private data class MyKey(val id: String, val type: String) {
>> override fun equals(other: Any?): Boolean {
>> if (other is MyKey)
>> return id == other.id
>> return false
>> }
>>
>> override fun hashCode(): Int {
>> return 1
>> }
>> }
>> }
>>
>>

Key1 equals Key2 but not found in cache

2023-03-22 Thread Humphrey Lopez 
Hello, when having a key which equals another key, when trying to retrieve
from cache it does not return the expected value. Is this a bug?

I have a reproducible below in kotlin but in java we get the same result
(test with Ignite 2.10 and  2.14) and java 11 and 19.


import org.apache.ignite.Ignition
import org.apache.ignite.configuration.CacheConfiguration
import org.assertj.core.api.SoftAssertions
import org.junit.jupiter.api.Test

class MyTest {

private val key1 = MyKey("ABC", "DEF")
private val key2 = MyKey("ABC", "xxx")

@Test
fun testEquals() {
SoftAssertions.assertSoftly {
it.assertThat(key1).isEqualTo(key2)
it.assertThat(key1 == key2).isTrue
}
}

@Test
fun testWithMap() {
val map = mapOf(Pair(key1, "key1"))

SoftAssertions.assertSoftly {
it.assertThat(map.containsKey(key1)).isTrue
it.assertThat(map.containsKey(key2)).isTrue
}

}

@Test
fun testWithIgnite() {
val ignite = Ignition.start();
val cache = ignite.createCache(CacheConfiguration("mycache"))

cache.put(key1, "key1")
SoftAssertions.assertSoftly {
it.assertThat(cache.containsKey(key1)).isTrue
it.assertThat(cache.containsKey(key2)).isTrue
}
}

private data class MyKey(val id: String, val type: String) {
override fun equals(other: Any?): Boolean {
if (other is MyKey)
return id == other.id
return false
}

override fun hashCode(): Int {
return 1
}
}
}

Re: Create a Blog with Example code

2023-03-08 Thread Humphrey Lopez 
Thanks, that looks good. I shall take a closer look at it later this week.

Op wo 8 mrt 2023 om 04:34 schreef Surinder Mehra :

> Hello,
> Hope this helps you with some more examples. This is my GitHub repo which
> has examples on various features of ignite.
>
> https://github.com/Rednirus/apache-ignite-masterclass
>
> Cheers !
>
>
> On Mon, 6 Mar 2023, 13:47 Humphrey Lopez,  wrote:
>
>> What I need is a nice example where I can demonstrate Affinity
>> Collocation. Would like to populate data (maybe stream) from a Rest Api
>> outside kubernetes, what also would be nice is to fetch some data from the
>> internet using a CacheStoreAdapter.
>> Primarily using ignite only in Memory Mode (with backups 1 or 2). And
>> maybe later I could dive further into using persistent with a StateFullSet
>> deployment. I think it would be a series of several blogs which we will
>> build step by step from scratch. But for that I would like to have the
>> final project first worked out and then create the blog. Any input or
>> reference to other examples that covers the first part (in memory grid,
>> affinity collocation). I need an example of a use case (Company with
>> Employee, or Cars) or something like that.
>>
>> Humphrey
>>
>> Op za 4 mrt 2023 om 18:02 schreef Kseniya Romanova > >:
>>
>>> Humphrey as an inspiration you can also check Stack Overflow questions
>>> tagged "ignite". Many of them are about working with Ignite and Spring.
>>> Step by step tutorial following your experience sounds good too.
>>>
>>> On Sat, Mar 4, 2023 at 8:16 AM  wrote:
>>>
>>>> Hi Humphrey,
>>>>
>>>> Here is my own contribution in sharing experience of Apache Ignite.
>>>> Back in those times I was starting a new project and spent a few weeks in
>>>> solving data load perfomance problems. I was inspired by the post [2], it
>>>> was very helpful that I can load, compile and debug complete project code
>>>> from github. So I spent another few weeks and wrote my own post [1], also
>>>> with compilable project inside.
>>>>
>>>> As for your question, I think a post about monitoring the cashes with
>>>> grafana will be usefull
>>>>
>>>> PS
>>>> Many thanks to Kseniya Romanova, Denis Magda and Susan Ledford for
>>>> their help in my work
>>>>
>>>> Vladimir
>>>>
>>>> [1]
>>>> https://www.gridgain.com/resources/blog/how-fast-load-large-datasets-apache-ignite-using-key-value-api
>>>> [2]
>>>> https://www.gridgain.com/resources/blog/implementing-microservices-apache-ignite-service-apis-part-i-0
>>>> 00:41, 4 марта 2023 г., Humphrey Lopez :
>>>>
>>>> I’ve no idea yet. Lately working with spring boot and ignite again
>>>> think maybe more people could use some help getting started. Would also
>>>> like to setup local K8s cluster with microk8s and deploy the service and
>>>> some nodes (client and server) and use rest endpoint to populate and
>>>> monitor the cashes with grafana.
>>>>
>>>> Humphrey
>>>>
>>>> On 3 Mar 2023, at 10:57, Kseniya Romanova 
>>>> wrote:
>>>>
>>>> 
>>>> Hi Humphrey! Good idea! If you need a review before publishing,
>>>> please let me know and I'll find a comitter who could help with this.
>>>> Where do you plan to publish your blog? I know that many igniters
>>>> prefer dzone or dev.to, but personal github or medium blogs work as
>>>> well.
>>>>
>>>> Cheers,
>>>> Kseniya
>>>>
>>>> On Fri, Mar 3, 2023 at 10:21 AM Humphrey  wrote:
>>>>
>>>> Hi,
>>>>
>>>> I would like to create a blog with code and share experience of Apache
>>>> Ignite. What’s the best way to do that?
>>>>
>>>> Greetings Humphrey
>>>>
>>>>
>>>>
>>>> --
>>>> Отправлено из мобильного приложения Яндекс Почты
>>>
>>>

Re: Create a Blog with Example code

2023-03-06 Thread Humphrey Lopez 
What I need is a nice example where I can demonstrate Affinity Collocation.
Would like to populate data (maybe stream) from a Rest Api outside
kubernetes, what also would be nice is to fetch some data from the internet
using a CacheStoreAdapter.
Primarily using ignite only in Memory Mode (with backups 1 or 2). And maybe
later I could dive further into using persistent with a StateFullSet
deployment. I think it would be a series of several blogs which we will
build step by step from scratch. But for that I would like to have the
final project first worked out and then create the blog. Any input or
reference to other examples that covers the first part (in memory grid,
affinity collocation). I need an example of a use case (Company with
Employee, or Cars) or something like that.

Humphrey

Op za 4 mrt 2023 om 18:02 schreef Kseniya Romanova :

> Humphrey as an inspiration you can also check Stack Overflow questions
> tagged "ignite". Many of them are about working with Ignite and Spring.
> Step by step tutorial following your experience sounds good too.
>
> On Sat, Mar 4, 2023 at 8:16 AM  wrote:
>
>> Hi Humphrey,
>>
>> Here is my own contribution in sharing experience of Apache Ignite. Back
>> in those times I was starting a new project and spent a few weeks in
>> solving data load perfomance problems. I was inspired by the post [2], it
>> was very helpful that I can load, compile and debug complete project code
>> from github. So I spent another few weeks and wrote my own post [1], also
>> with compilable project inside.
>>
>> As for your question, I think a post about monitoring the cashes with
>> grafana will be usefull
>>
>> PS
>> Many thanks to Kseniya Romanova, Denis Magda and Susan Ledford for their
>> help in my work
>>
>> Vladimir
>>
>> [1]
>> https://www.gridgain.com/resources/blog/how-fast-load-large-datasets-apache-ignite-using-key-value-api
>> [2]
>> https://www.gridgain.com/resources/blog/implementing-microservices-apache-ignite-service-apis-part-i-0
>> 00:41, 4 марта 2023 г., Humphrey Lopez :
>>
>> I’ve no idea yet. Lately working with spring boot and ignite again think
>> maybe more people could use some help getting started. Would also like to
>> setup local K8s cluster with microk8s and deploy the service and some nodes
>> (client and server) and use rest endpoint to populate and monitor the
>> cashes with grafana.
>>
>> Humphrey
>>
>> On 3 Mar 2023, at 10:57, Kseniya Romanova  wrote:
>>
>> 
>> Hi Humphrey! Good idea! If you need a review before publishing,
>> please let me know and I'll find a comitter who could help with this.
>> Where do you plan to publish your blog? I know that many igniters prefer
>> dzone or dev.to, but personal github or medium blogs work as well.
>>
>> Cheers,
>> Kseniya
>>
>> On Fri, Mar 3, 2023 at 10:21 AM Humphrey  wrote:
>>
>> Hi,
>>
>> I would like to create a blog with code and share experience of Apache
>> Ignite. What’s the best way to do that?
>>
>> Greetings Humphrey
>>
>>
>>
>> --
>> Отправлено из мобильного приложения Яндекс Почты
>
>

Re: Create a Blog with Example code

2023-03-03 Thread Humphrey Lopez 
I’ve no idea yet. Lately working with spring boot and ignite again think maybe more people could use some help getting started. Would also like to setup local K8s cluster with microk8s and deploy the service and some nodes (client and server) and use rest endpoint to populate and monitor the cashes with grafana. Humphrey On 3 Mar 2023, at 10:57, Kseniya Romanova  wrote:Hi Humphrey! Good idea! If you need a review before publishing, please let me know and I'll find a comitter who could help with this. Where do you plan to publish your blog? I know that many igniters prefer dzone or dev.to, but personal github or medium blogs work as well. Cheers,KseniyaOn Fri, Mar 3, 2023 at 10:21 AM Humphrey  wrote:Hi,

I would like to create a blog with code and share experience of Apache Ignite. What’s the best way to do that?

Greetings Humphrey

Re: H2database dependency in Apache Ignite

2023-01-27 Thread Humphrey Lopez 
As of version 2.13 I think you can use Apache-Calcite instead of H2, and
get rid of H2 from your dependency list.

https://ignite.apache.org/docs/latest/SQL/sql-calcite

Regards.

Op vrijdag 20 januari 2023 schreef Andrey Mashenkov <
andrey.mashen...@gmail.com>:

> Hi,
>
> Ignite uses H2 as one of 2 available execution engines. Module
> 'ignite-indexing' depends on H2.
> Long story short, we can't bump H2 version anymore as some integration
> points were dropped in higher H2 versions.
> However, this CVE along with other known issues[1] do not affect the
> Ignite.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-15241
>
> On Fri, Jan 20, 2023 at 1:06 PM Gianluca Bonetti <
> gianluca.bone...@gmail.com> wrote:
>
>> Hello
>>
>> I am also using Apache Ignite for some projects, but I don't see any
>> dependency on h2 in my projects.
>> I think h2 dependency is coming from somewhere else.
>> Can you run a "mvn dependency:tree" and share the results?
>>
>> Cheers
>> Gianluca
>>
>> On Fri, 20 Jan 2023 at 09:56, David Cussen 
>> wrote:
>>
>>> Hi,
>>>
>>> I am an employee in Workday and our team uses Apache Ignite for one of
>>> our products. There is a dependency on com.h2database:h2:jar:1.4.197 :
>>> https://github.com/apache/ignite/blob/master/parent/pom.xml#L92
>>>
>>>
>>>
>>> We are wondering if there is a plan to upgrade this dependency to
>>> remediate CVE-2021-42392
>>>  and if
>>> so, do you have an ETA on when this would be available?
>>>
>>>
>>>
>>> Thank you.
>>>
>>>
>>>
>>> Kind regards,
>>>
>>> David Cussen
>>>
>>> Workday
>>>
>>>
>>>
>>
>
> --
> Best regards,
> Andrey V. Mashenkov
>

Re: DataRegion not releasing memory back to OS

2022-12-23 Thread Humphrey Lopez 
Is there a way I can see how much of that claimed memory of Ignite is
available when caches are cleared? Currently we are in production and there
we see that the memory consumed by our pods are passing above 90%.
But we are clearing caches and filling them with new data to do new
calculations. I hope when clearing the data that memory becomes available
for new data.

Humphrey

Op vr 23 dec. 2022 om 09:56 schreef Ivan Daschinsky :

> Hi, but Ignite doesn't and never did release memory back to the OS, except
> only on deactivation or shutting down.
>
> пт, 23 дек. 2022 г. в 10:46, Humphrey Lopez :
>
>> Hello,
>>
>> I've been doing some test with Ignite 2.14, creating one cache filling it
>> with data and then clearing back the cache, eventually destroy the cache.
>> I'm using the OpenCensus metrics to get the statistics:
>> - TotalAllocatedSize
>> When I test it with persistence Enabled I see that TotalAllocatedSize
>> drops back to 0 when I invoke the destroy cache method.
>> But when I do the same with persistence Disabled, the TotalAllocatedSize
>> doesn't drop back.
>>
>> I was hoping that when clearing the cache the memory drops, or at least
>> when I do destroy(). My question is why is this memory given back when
>> Persistence is Enabled and not when Persistence is disabled, I was hoping
>> to see the same effect.
>>
>> I can create a reproducer for this, but maybe it's a known thing.
>>
>> Humphrey
>>
>
>
> --
> Sincerely yours, Ivan Daschinskiy
>

DataRegion not releasing memory back to OS

2022-12-22 Thread Humphrey Lopez 
Hello,

I've been doing some test with Ignite 2.14, creating one cache filling it
with data and then clearing back the cache, eventually destroy the cache.
I'm using the OpenCensus metrics to get the statistics:
- TotalAllocatedSize
When I test it with persistence Enabled I see that TotalAllocatedSize drops
back to 0 when I invoke the destroy cache method.
But when I do the same with persistence Disabled, the TotalAllocatedSize
doesn't drop back.

I was hoping that when clearing the cache the memory drops, or at least
when I do destroy(). My question is why is this memory given back when
Persistence is Enabled and not when Persistence is disabled, I was hoping
to see the same effect.

I can create a reproducer for this, but maybe it's a known thing.

Humphrey

Re: Ignite 2.0: CacheWriteSynchronizationMode = FULL_ASYNC when using cache.putAsync

2017-05-15 Thread Humphrey Lopez 
https://apacheignite.readme.io/v2.0/docs/primary-and-backup-copies#synchronous-and-asynchronous-backups

Humphrey 

> On 16 May 2017, at 03:14, Gordon Reid (Nine Mile) 
>  wrote:
> 
> Thanks Humphrey. And in the case of PRIMARY_SYNC, how is the primary node 
> determined for the entity? Will it be the node where the entity was created? 
> (we are in REPLICATED)
> 
> Thanks,.
> 
> -Original Message-
> From: Humphrey [mailto:hmmlo...@gmail.com]
> Sent: Monday, 15 May 2017 5:19 PM
> To: user@ignite.apache.org
> Subject: Re: Ignite 2.0: CacheWriteSynchronizationMode = FULL_ASYNC when 
> using cache.putAsync
> 
> Hi Gordon,
> 
> FULL_ASYNC: Do not wait for any server node to acknowledge
> 
> So it will fire and forget, and will not wait for all nodes in cluster to be 
> updated. That will happen in the background.
> 
> Humphrey
> 
> 
> 
> --
> View this message in context: 
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-2-0-CacheWriteSynchronizationMode-FULL-ASYNC-when-using-cache-putAsync-tp12837p12839.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Kindly tell me where to find these jar files.

2017-05-12 Thread Humphrey Lopez 
I'm not able to help you out today but Monday I can send you some code. I used 
the ignite Kafka streamer to get the messages from Kafka. So ignite is pulling 
from Kafka rather than Kafka pushing to ignite. 

Humphrey 

> On 12 May 2017, at 10:15, blasteralfred  wrote:
> 
> Hi Humphrey, Thanks for your reply. I am trying to use Kafka Connect for 
> pushing to Kafka, and from there, to Ignite, and that's why I have asked for 
> jar files. Its okay, that I can try the Stream also. I have no idea on how to 
> use streams, and the examples I have found were, not in a beginner level (for 
> me to understand). Could you be kind enough to show me a sample code to push 
> data to ignite, from kafka? I have pushed data to a kafka topic using Kafka 
> connect's file stream source, and in Kafka, the data reside like this; 
> {"schema":{"type":"string","optional":false},"payload":"hello world 6"} 
> {"schema":{"type":"string","optional":false},"payload":"hello world 7"} 
> {"schema":{"type":"string","optional":false},"payload":"hello world 8"} 
> {"schema":{"type":"string","optional":false},"payload":"hello world 9"} 
> {"schema":{"type":"string","optional":false},"payload":"hello world 10"} Can 
> you show me a code to push this data to Ignite? Thanks in advance.. 
> View this message in context: Re: Kindly tell me where to find these jar 
> files.
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.