Re: Does apache ignite support Spring 5.x
I don't know if this relates and we have not fully understood where the problem is but after we upgraded to SpringBoot 2.2.0 the Kubernetes TCP finder "seemed" to be failing on the "logger" which was odd because everything else seemed to work, even logging in other pieces (it worked previously). This started out because our nodes were not joining each other. As we investigated the problem it "appeared" the logger was null and throwing the exception. We copied your implementation into our own package and removed the logger and it seemed to work, the nodes joined each other. We are still looking into this but nothing on our end has changed except the SpringBoot upgrade. - KJQ -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Events and selective marshalling between nodes?
Hello everyone, I am not even sure how to phrase this so I will try to describe the environment... We have numerous embedded Ignite instances (server mode) that deploy into a K8S cluster. Between those instances we have Ignite events being raised for PUT and EXPIRED. We have at least one instance, also a part of the Ignite cluster, which uses Drools and distributed compute (this works good so far). The problem we are having is when the distributed compute node raises the cache events to other nodes, they do not have those classes there. For example, Drools caches some stuff specific to it and when that event gets raised to other nodes (who have no knowledge of Drools) an exception is of course thrown. So my questions are: - Can I selectively choose what to raise the event for based on the type? For example, check the event "value" and ignore if something I am not concerned with and not raise it. - Is the only way to share classes between nodes, whose Java applications may not have knowledge of each other, through peer-classloading or downloading a jar? For example, projectB depends on projectA so it is aware of A's classes but projectA has no knowledge of projectB in the event B raises an event from its cached object to A. - How does all of this play into Spring? If I do somehow get the classes into each node, do they have to be within the SpringContext (IoC configured) or is having the classes available good enough? This applies to using the @SpringResource in something like a callable. Any help or guidance is greatly appreciated. - KJQ -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Confusion with Events
Ivan, thanks for all of the help! I think we have it working based on your advice. I cannot seem to really reproduce it without following a very specific scenario outside of a test case. The test case works correctly - one change that I made was to change the predicate from looking for nodes with a specific attribute to all server nodes. What has been frustrating, but nothing to do with Ignite, is the way working in Windows WSL vs. DOS. Oddly, WSL does not seem to respect Windows ports and it led us down a red herring thinking our configuration was messed up somewhere. - KJQ -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Confusion with Events
Thank you for your help! I "think" this is working.
*One thing I did notice in my test environment is that the events can be
raised in drastically different times? *
It is not a huge deal since getting the item (providing the near cache has
expired it) will trigger the expiration on Ignite which will in turn raise
the events but I was just curious why such the delay especially in a local
test environment...
What is really odd to me is that no matter what I do, what order I put items
in, what order the cache gets initially created, anything in "cache1" always
seems to raise the event in a timely manner and both caches have the exact
same configuration. My test case, which uses sleep() to simulate the
expiration, seems to show more consistent results with the events being
raised more timely.
*For example:*
1) Spin up a single Ignite node (embedded / server / partitioned /
EVT_CACHE_OBJECT_EXPIRED )
a) cache1 defined via configuration (eager TTL = true)
b) cache2 defined via configuration (eager TTL = true)
I assign a node attribute to each cache (cache = true) and then use a
predicate to filter only those specific nodes in the listener.
2) Using a dummy REST resource (in same instance) put entries into the
near-cache (caffeine) which writes through to the far-cache (ignite).
Caffeine expires in 30s / Ignite expires in 60s
On the first call to caffeine (which would be per Ignite node as well), if I
have not initialized the ignite side, attach the listener (so, this happens
just once on the node). Idea is to listen on any "expired" events and raise
the action to the local near-cache.
Then...
a) put k1/aaa into cache1
b) put k2/bbb into cache2
c) put k3/ccc into cache2
/Output looks something like (9:35):/
[DEBUG] CaffeineConfigurableCacheManager : Created cache cache1
[ WARN] CaffeineConfigurableCacheManager : Initialized
[DEBUG] IgniteCacheBuilder : Writing aaa to cache1
[DEBUG] CaffeineConfigurableCacheManager : Created cache cache2
[DEBUG] IgniteCacheBuilder : Writing bbb to cache2
[DEBUG] IgniteCacheBuilder : Writing ccc to cache2
3) Sit and wait (just looking at the events only, not if I get the record
which forces the expiry on expired items)...
/Within the ~"60s" expiration policy on Ignite I see (9:36):/
[DEBUG] IgniteCacheBuilder : Received remote event cache1
CACHE_OBJECT_EXPIRED aaa
[DEBUG] IgniteCacheBuilder : Eviction callback cache1 CACHE_OBJECT_EXPIRED
aaa
/Much later I see (9:52):/
[DEBUG] IgniteCacheBuilder : Received remote event cache2
CACHE_OBJECT_EXPIRED bbb
[DEBUG] IgniteCacheBuilder : Eviction callback cache2 CACHE_OBJECT_EXPIRED
bbb
[DEBUG] IgniteCacheBuilder : Received remote event cache2
CACHE_OBJECT_EXPIRED ccc
[DEBUG] IgniteCacheBuilder : Eviction callback cache2 CACHE_OBJECT_EXPIRED
ccc
*My current code looks something like this:*
ignite.events(ignite.cluster().forPredicate(new
AttributeNodeFilter<>(CACHE_NODE_ATTRIBUTE_KEY, true)))
.remoteListen(
null,
new IgnitePredicate()
{
@IgniteInstanceResource
private transient Ignite ignite;
@SpringResource(resourceClass = CacheManager.class)
private transient CacheManager cacheManager;
@Override
public boolean apply(final CacheEvent event)
{
log.debug("Received remote event {} {} {} / {} {} ",
event.cacheName(),
event.name(),
event.key(),
event.node().id(),
event.eventNode().id());
//Broadcase the callback
ignite.compute().broadcastAsync((IgniteCallable)() -> {
try
{
log.debug("Eviction callback {} {} {} / {} {} ",
event.cacheName(),
event.name(),
event.key(),
event.node().id(),
event.eventNode().id());
//Evict item from cache
cacheManager
.getCache(event.cacheName())
.evict(event.key());
}
catch (Exception x)
{
log.error("Callback error: {}", x.getMessage(), x);
}
//Return the event
return true;
});
return true;
}
},
EVT_CACHE_ENTRY_EVICTED,
EVT_CACHE_OBJECT_EXPIRED);
-
KJQ
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Confusion with Events
Thank you for the quick response - it is much appreciated. We are still
working through our Ignite integration, so far it has been nice, but it is
definitely a learning curve.
So, we have an environment that is deployed as services in a K8S cluster.
Each service uses Caffeine as the in-memory cache (i know Ignite has a
near-cache but cannot make that change now). Caffeine is also being tested
inside of Reactive Flux/Mono calls making it harder to change quickly. Each
service, a deployed pod, is also an Ignite "server" node as well. We use
Ignite, partitioned, as the primary cache (and some distributed compute with
Drools).
Because all of the nodes use Caffeine, and becoming just a near-cache when
Ignite is included, we would like Ignite to raise an "expired" event to all
the nodes and evict that item from Caffeine (before Ignite, Caffeine was
used as the only in-memory cache per service) - we want to cleanup the local
caches on all the nodes. Each Ignite cache configuration has an expiration
policy setup.
1) I tried using the `addCacheEntryListenerConfiguration` with each Ignite
cache thinking this was a better choice because (i thought) it was backed by
the continuous query and would not require me to explicitly use events.
But, it looked like that only fired on the node where the operation happened
(i.e. locally). Maybe I could broadcast the event from within this listener
to the other nodes?
2) My next attempt is using the "remoteListen()." If I understand you
correctly, I do not need a "local listener" but the "remote listener" should
broadcast a message when it is triggered?
*Couple of things I noticed in my test below:*
- If i take out the PUT it seems I never see any callback notifications
- Without the local event listener I do not see any expiration messages
(possibly because of where the data is being cached in the test - local vs.
remote node)
Basically, I would like to do this:
1) R/W to Ignite cache with an "expiration" policy
2) When Ignite decides to "expire" an item raise an event to all Ignite
nodes
3) From the event, evict the cache item locally.
This is what I have right now for testing:
ig.events(
ig.cluster().forServers())
.remoteListen(
new IgniteBiPredicate()
{
@Override
public boolean apply(UUID uuid, CacheEvent evt)
{
log.debug("Received local event {} {} {} // {} //
{} ",
evt.cacheName(),
evt.name(),
evt.key(),
evt.node().id(),
evt.eventNode().id());
return true; // Continue listening.
},
new IgnitePredicate()
{
@Override
public boolean apply(final CacheEvent evt)
{
log.debug("Received remote event {} {} {} / {} {} ",
evt.cacheName(),
evt.name(),
evt.key(),
evt.node().id(),
evt.eventNode().id());
//Broadcast the callback
ig.compute().broadcastAsync(new IgniteCallable()
{
@Override
public Object call() throws Exception
{
log.debug("Received callback {} {} {} / {} {} ",
evt.cacheName(),
evt.name(),
evt.key(),
evt.node().id(),
evt.eventNode().id());
return null;
}
});
return true;
}
},
EVT_CACHE_OBJECT_PUT,
EVT_CACHE_OBJECT_EXPIRED)
-
KJQ
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Confusion with Events
I have some questions regarding Cache Listeners/Events.
We have a system that used a lot of "Caffeine" based caches spread across
multiple services (in K8S). Basically "near-caches" (without a backing
store). We are now trying to fit Ignite behind those usages.
*What we are trying to do is when Ignite /expires/ an entry receive the
event on all the nodes and evict it in from Caffeine*.
Are one of these approaches below correct? And/or how can I accomplish this?
Is there a better/easier way?
1) I tried registering a CacheListener with each cache configuration but
that seemed to only fire where the cache event was fired:
config.addCacheEntryListenerConfiguration(new
IgniteExpiredListener<>(cacheManagerProvider));
2) I am experimenting with cache events as well like this below.
ig.events(
ig.cluster().forServers())
.remoteListen(
new IgniteBiPredicate()
{
@Override
public boolean apply(UUID uuid, CacheEvent evt)
{
log.debug("Received local event "
+ evt.name()
+ ", key="
+ evt.key()
+ ", at="
+ evt.node().consistentId().toString()
+ ", "
+
evt.eventNode().consistentId().toString() );
cm.getCache(evt.cacheName()).evict(evt.key());
return true; // Continue listening.
}
},
new IgnitePredicate()
{
@Override
public boolean apply(final CacheEvent evt)
{
log.debug("Received remote event "
+ evt.name()
+ ", key="
+ evt.key()
+ ", at="
+ evt.node().consistentId().toString()
+ ", "
+
evt.eventNode().consistentId().toString() );
return true;
}
},
EVTS_CACHE);
-
KJQ
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Question about Caching, Console, and PeerClassLoading
I have a question about how Caching, the Web Console, and Peer ClassLoading work... I have 2 nodes, both with PeerClassLoading enabled, and the web console started. From the caching side, it appears everything is working as it should (without peer class loading I seemed to get intermittent cannot marshal errors). *When I go to the WebConsole and "scan" I various results:* - I get all the items on all nodes - I get a Query error - I can only get the items on a specific node (leading me to believe it is not recognizing the classes to unmarshal on other nodes). - I get an occasional cannot read "field" error Most of the time, if I scan the specific node (for example see 2 out of 3 elements) then do a full scan I will see all of the items (3 of 3 elements). Is this expected behaviour for the WebConsole? I am just using it to spot check the entries. Shouldn't it always return all the results (especially since it does some of the time)? Also, i've seen that you should not use PeerClassLoading but for my purposes, a self-contained cluster of embedded Ignite instances (in SpringBoot) and one external instance, this seems to work well. What are the reasons for not using PeerClassLoading in such a scenario or is it ok? - KJQ -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: how to configure apache ignite as cache api and as spring cache provider
I dont know if this helps or not but we ran into a similar issue testing the distributed cache with Shiro authentication. Shiro was eagerly trying to create the cache but the Spring Ignite instance (inside the SpringCacheManager) had not yet been initialized (I believe it initializes after all the beans are configured). In our case, we wrapped the "cache" with a Provider<> so that we could lazily access it when needed after everything started up. - KJQ -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: WebConsole does not see cluster
So, I finally got this working. I ended up: - Copying the rest-http module into my main Ignite node (the only one who's IP i am using for discovery right now). - I shell into the running "ignite" instance and run the web agent from there (not outside the containers) Questions: - I see in "caches" my cache. Should I see my configurations in configurations as well or is that only for created ones? - Where do I need the agent and rest api? It seems like having it in the one node will suffice. Do I need to have both on every node? - KJQ -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: WebConsole does not see cluster
All right, im completely confused at this point. Trying to follow this thread: http://apache-ignite-users.70518.x6.nabble.com/Agent-not-able-to-connect-tt21714.html#a21745 <http://apache-ignite-users.70518.x6.nabble.com/Agent-not-able-to-connect-tt21714.html#a21745> *I see this in the console startup:* ignite-console_1 | 00:36:34 0|index| Start listening on 127.0.0.1:3000 I have, for now, exposed all of the ports "i think" so I can try the agent outside of the containers: ignite-console: image: apacheignite/web-console-standalone expose: ... ports: - 3000:3000 - 3001:80 server: ports: - 3080:3080 The server has the REST API, exposed on 3080, and I can access it with curl. *I then run the agent:* ./ignite-web-agent.sh --server-uri http://localhost:3001 --node-uri http://localhost:3080 *And get this:* Agent configuration: User's security tokens: 4BDV URI to Ignite node REST server: http://localhost:3080 URI to Ignite Console server : http://localhost:3001 Path to agent property file : default.properties Path to JDBC drivers folder : /Downloads/ignite-web-agent-2.6.0/jdbc-drivers Demo mode : enabled [2018-08-12 20:39:48,639][INFO ][main][AgentLauncher] Connecting to: http://localhost:3001 [2018-08-12 20:39:48,754][INFO ][EventThread][AgentLauncher] Connection established. [2018-08-12 20:39:48,806][INFO ][EventThread][AgentLauncher] Authentication success. [2018-08-12 20:39:48,880][INFO ][pool-1-thread-1][RestExecutor] Connected to cluster [url=http://localhost:3080] This is all running on one machine but different docker containers (from compose) - KJQ -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
WebConsole does not see cluster
Hello. I am trying to get the Dockerized WebConsole running but no matter what I do it does not recognize any clusters. I am running the Web Agent from one of my nodes with the REST (maven) dependencies. I can see the nodes coming up when I start my services and using the CLI I can inspect the cluster fine. Topology snapshot [ver=4, servers=4, clients=0, CPUs=16, offheap=6.2GB, heap=2.5GB] *Running the agent I see:* URI to Ignite node REST server: http://localhost:3080 URI to Ignite Console server : http://ignite-console [2018-08-12 22:23:08,341][INFO ][main][AgentLauncher] Connecting to: http://ignite-console [2018-08-12 22:23:08,441][INFO ][EventThread][AgentLauncher] Connection established. [2018-08-12 22:23:08,524][INFO ][EventThread][AgentLauncher] Authentication success. [2018-08-12 22:23:08,751][INFO ][pool-1-thread-1][RestExecutor] Connected to cluster [url=http://localhost:3080] I can curl the REST API using 'http://localhost:3080/ignite?cmd=version' from the console instance to the node running the REST module and get back the version. "response": "2.6.0" But, no matter what I seem to do the WebConsole does not recognize the cluster - what am I doing wrong here? - KJQ -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Ignite running on JDK10? (Maybe Solved)
Added the exports to the SpringBoot plugin seem to work for me in addition to having them in my compiler and surefire plugins...it at least starts up Ignite and I see the Ignite banner. org.springframework.boot spring-boot-maven-plugin --add-exports=java.base/jdk.internal.misc=ALL-UNNAMED --add-exports=java.base/sun.nio.ch=ALL-UNNAMED - KJQ -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Ignite running on JDK10?
As a note, I downgraded all of the Docker containers to use JDK 9 (9.0.4) and I still get the same problem running the SpringBoot 2 application. Running in my IDE a test case works perfectly fine. *Caused by: java.lang.RuntimeException: jdk.internal.misc.JavaNioAccess class is unavailable.* *Caused by: java.lang.IllegalAccessException: class org.apache.ignite.internal.util.GridUnsafe cannot access class jdk.internal.misc.SharedSecrets (in module java.base) because module java.base does not export jdk.internal.misc to unnamed module @78a89eea* - KJQ -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Ignite running on JDK10?
We are currently integrating Apache Ignite v2.6.0 into our platform. We upgraded to JDK9 just to make sure we were compliant and then upgraded to JDK10 try and preemptively catch anything planned for removal. I can run Ignite great in test cases but when I go to startup a SpringBoot 2 application that depends on the module which brings in the Ignite dependencies it gives me this error when initializing Ignite: *Caused by: java.lang.IllegalAccessException: class org.apache.ignite.internal.util.GridUnsafe cannot access class jdk.internal.misc.SharedSecrets (in module java.base) because module java.base does not export jdk.internal.misc to unnamed module* I know JDK10 is not supported yet but is there some temporary quick hack or fix I can apply to get past this? I've already tried adding this to Maven to see if it helps but it did not: --add-exports=java.base/jdk.internal.misc=ALL-UNNAMED --add-exports=java.base/sun.nio.ch=ALL-UNNAMED --add-exports=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED --add-exports=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED --add-modules=java.xml.bind BTW: I posted this same message earlier but I dont know if I was subscribed yet so apologies if it appears twice. ----- KJQ -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
