Re: Ignite equivalent of @GridUserResource
I will try that. Thanks, Paolo On Wed, Sep 30, 2015 at 2:39 AM, vkulichenko wrote: > Hi Paolo, > > You can use node local map instead [1]. It's a per-node singleton storage > that has standard ConcurrentMap API and can be used for any user resources > (e.g., JDBC connections, pools, etc). You can inject Ignite instance to > your > job using @IgniteInstanceResource annotation and acquire the map with > Ignite.cluster().nodeLocalMap() method. > > Let me know if it works for you. > > [1] > > https://ignite.apache.org/releases/1.3.0/javadoc/org/apache/ignite/IgniteCluster.html#nodeLocalMap() > > -Val > > > > -- > View this message in context: > http://apache-ignite-users.70518.x6.nabble.com/Ignite-equivalent-of-GridUserResource-tp1524p1527.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. >
How shutdown all nodes
Hi, Is there an API to gently shut-down all the nodes that made up an Ignite cluster? Cheers, Paolo
Re: How shutdown all nodes
The best solution I've found so far is defining a "poison" task like the
following one (Groovy code):
static class PoisonPill implements IgniteRunnable {
@IgniteInstanceResource
private transient Ignite ignite
@Override
void run() {
ignite.close()
}
}
and broadcasting it to all remote nodes:
def void shutdown() {
grid.cluster().forRemotes().ignite().compute().broadcast(new
PoisonPill())
}
That works but in the "master" I'm getting the following exception:
java.lang.IllegalStateException: Grid is in invalid state to perform this
operation. It either not started yet or has already being or have stopped
[gridName=nextflow, state=STOPPING]
at
org.apache.ignite.internal.GridKernalGatewayImpl.illegalState(GridKernalGatewayImpl.java:190)
~[ignite-core-1.4.0.jar:1.4.0]
at
org.apache.ignite.internal.GridKernalGatewayImpl.readLock(GridKernalGatewayImpl.java:90)
~[ignite-core-1.4.0.jar:1.4.0]
at
org.apache.ignite.internal.cluster.ClusterGroupAdapter.guard(ClusterGroupAdapter.java:170)
~[ignite-core-1.4.0.jar:1.4.0]
at
org.apache.ignite.internal.cluster.ClusterGroupAdapter.forPredicate(ClusterGroupAdapter.java:367)
~[ignite-core-1.4.0.jar:1.4.0]
at
org.apache.ignite.internal.cluster.ClusterGroupAdapter.forOthers(ClusterGroupAdapter.java:564)
~[ignite-core-1.4.0.jar:1.4.0]
at
org.apache.ignite.internal.cluster.ClusterGroupAdapter.forRemotes(ClusterGroupAdapter.java:536)
~[ignite-core-1.4.0.jar:1.4.0]
at org.apache.ignite.cluster.ClusterGroup$forRemotes$1.call(Unknown Source)
~[na:na]
at nextflow.executor.IgConnector.shutdown(IgConnector.groovy:155)
~[nxf-ignite-0.16.0-SNAPSHOT.jar:na]
Is there a better way to shut-down all cluster instances ?
Cheers,
Paolo
On Wed, Sep 30, 2015 at 4:03 PM, Paolo Di Tommaso wrote:
> Hi,
>
>
> Is there an API to gently shut-down all the nodes that made up an Ignite
> cluster?
>
>
> Cheers,
> Paolo
>
>
Re: How shutdown all nodes
Hello,
Please try
grid.compute(grid.cluster().forRemotes()).broadcast(new PoisonPill());
and
static class PoisonPill implements IgniteRunnable {
@IgniteInstanceResource
private transient Ignite ignite;
@Override
public void run() {
new Thread() {
@Override public void run() {
ignite.close();
}
}.start();
}
}
On Wed, Sep 30, 2015 at 5:45 PM, Paolo Di Tommaso wrote:
> The best solution I've found so far is defining a "poison" task like the
> following one (Groovy code):
>
> static class PoisonPill implements IgniteRunnable {
>
> @IgniteInstanceResource
> private transient Ignite ignite
>
> @Override
> void run() {
> ignite.close()
> }
> }
>
>
>
> and broadcasting it to all remote nodes:
>
> def void shutdown() {
> grid.cluster().forRemotes().ignite().compute().broadcast(new
> PoisonPill())
> }
>
>
>
> That works but in the "master" I'm getting the following exception:
>
> java.lang.IllegalStateException: Grid is in invalid state to perform this
> operation. It either not started yet or has already being or have stopped
> [gridName=nextflow, state=STOPPING]
> at
> org.apache.ignite.internal.GridKernalGatewayImpl.illegalState(GridKernalGatewayImpl.java:190)
> ~[ignite-core-1.4.0.jar:1.4.0]
> at
> org.apache.ignite.internal.GridKernalGatewayImpl.readLock(GridKernalGatewayImpl.java:90)
> ~[ignite-core-1.4.0.jar:1.4.0]
> at
> org.apache.ignite.internal.cluster.ClusterGroupAdapter.guard(ClusterGroupAdapter.java:170)
> ~[ignite-core-1.4.0.jar:1.4.0]
> at
> org.apache.ignite.internal.cluster.ClusterGroupAdapter.forPredicate(ClusterGroupAdapter.java:367)
> ~[ignite-core-1.4.0.jar:1.4.0]
> at
> org.apache.ignite.internal.cluster.ClusterGroupAdapter.forOthers(ClusterGroupAdapter.java:564)
> ~[ignite-core-1.4.0.jar:1.4.0]
> at
> org.apache.ignite.internal.cluster.ClusterGroupAdapter.forRemotes(ClusterGroupAdapter.java:536)
> ~[ignite-core-1.4.0.jar:1.4.0]
> at org.apache.ignite.cluster.ClusterGroup$forRemotes$1.call(Unknown
> Source) ~[na:na]
> at nextflow.executor.IgConnector.shutdown(IgConnector.groovy:155)
> ~[nxf-ignite-0.16.0-SNAPSHOT.jar:na]
>
>
>
> Is there a better way to shut-down all cluster instances ?
>
>
> Cheers,
> Paolo
>
>
>
>
> On Wed, Sep 30, 2015 at 4:03 PM, Paolo Di Tommaso <
> paolo.ditomm...@gmail.com> wrote:
>
>> Hi,
>>
>>
>> Is there an API to gently shut-down all the nodes that made up an Ignite
>> cluster?
>>
>>
>> Cheers,
>> Paolo
>>
>>
>
Re: How shutdown all nodes
Perfect !
Thanks a lot.
Cheers,
p
On Wed, Sep 30, 2015 at 5:39 PM, Anton Vinogradov
wrote:
> Hello,
>
> Please try
>
> grid.compute(grid.cluster().forRemotes()).broadcast(new PoisonPill());
>
> and
>
> static class PoisonPill implements IgniteRunnable {
> @IgniteInstanceResource
> private transient Ignite ignite;
>
> @Override
> public void run() {
> new Thread() {
> @Override public void run() {
> ignite.close();
> }
> }.start();
> }
> }
>
> On Wed, Sep 30, 2015 at 5:45 PM, Paolo Di Tommaso <
> paolo.ditomm...@gmail.com> wrote:
>
>> The best solution I've found so far is defining a "poison" task like the
>> following one (Groovy code):
>>
>> static class PoisonPill implements IgniteRunnable {
>>
>> @IgniteInstanceResource
>> private transient Ignite ignite
>>
>> @Override
>> void run() {
>> ignite.close()
>> }
>> }
>>
>>
>>
>> and broadcasting it to all remote nodes:
>>
>> def void shutdown() {
>> grid.cluster().forRemotes().ignite().compute().broadcast(new
>> PoisonPill())
>> }
>>
>>
>>
>> That works but in the "master" I'm getting the following exception:
>>
>> java.lang.IllegalStateException: Grid is in invalid state to perform this
>> operation. It either not started yet or has already being or have stopped
>> [gridName=nextflow, state=STOPPING]
>> at
>> org.apache.ignite.internal.GridKernalGatewayImpl.illegalState(GridKernalGatewayImpl.java:190)
>> ~[ignite-core-1.4.0.jar:1.4.0]
>> at
>> org.apache.ignite.internal.GridKernalGatewayImpl.readLock(GridKernalGatewayImpl.java:90)
>> ~[ignite-core-1.4.0.jar:1.4.0]
>> at
>> org.apache.ignite.internal.cluster.ClusterGroupAdapter.guard(ClusterGroupAdapter.java:170)
>> ~[ignite-core-1.4.0.jar:1.4.0]
>> at
>> org.apache.ignite.internal.cluster.ClusterGroupAdapter.forPredicate(ClusterGroupAdapter.java:367)
>> ~[ignite-core-1.4.0.jar:1.4.0]
>> at
>> org.apache.ignite.internal.cluster.ClusterGroupAdapter.forOthers(ClusterGroupAdapter.java:564)
>> ~[ignite-core-1.4.0.jar:1.4.0]
>> at
>> org.apache.ignite.internal.cluster.ClusterGroupAdapter.forRemotes(ClusterGroupAdapter.java:536)
>> ~[ignite-core-1.4.0.jar:1.4.0]
>> at org.apache.ignite.cluster.ClusterGroup$forRemotes$1.call(Unknown
>> Source) ~[na:na]
>> at nextflow.executor.IgConnector.shutdown(IgConnector.groovy:155)
>> ~[nxf-ignite-0.16.0-SNAPSHOT.jar:na]
>>
>>
>>
>> Is there a better way to shut-down all cluster instances ?
>>
>>
>> Cheers,
>> Paolo
>>
>>
>>
>>
>> On Wed, Sep 30, 2015 at 4:03 PM, Paolo Di Tommaso <
>> paolo.ditomm...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>>
>>> Is there an API to gently shut-down all the nodes that made up an Ignite
>>> cluster?
>>>
>>>
>>> Cheers,
>>> Paolo
>>>
>>>
>>
>
Changing node attributes at runtime
This was asked about a month ago, but the discussion ended up going a different direction. I have a use case involving targeting computation to nodes where the most natural answer _seems_ to be to be able to change node attributes at runtime. I'm aware that right now node attributes can't be changed at runtime; I'm curious if a.) there is a technical limitation why this couldn't be supported, and b.) if there's perhaps a better way for me to solve my problem. I have a cluster of nodes, which can each have a list of tags indicating whether a job should run on them. So I would like to be able to target jobs to only those services that have a particular tag. However, users can edit which tags apply to which nodes at runtime through a UI. I can't restart nodes when tags are edited. I'm flexible about how I store the tags -- I could store them in a cache or another central store, for example. So the only alternative I can think of to having attributes editable at runtime is to have a map of tags to cluster node IDs in a central location, and explicitly build a ClusterGroup from that when launching compute jobs. Does that sound reasonable? Is there a better way to handle this kind of thing? Thanks, Matt
Failed to wait for asynchronous operation permit (thread got interrupted)
During a regular get(...) operation on some of our caches we're occasionally receiving IgniteInterruptedExceptions complaining that Ignite "Failed to wait for asynchronous operation permit (thread got interrupted)" (full stack trace below). Can anybody shine some light on what exactly those exceptions mean and how we can further debug and fix the problem? Thanks! javax.cache.CacheException: class org.apache.ignite.IgniteInterruptedException: Failed to wait for asynchronous operation permit (thread got interrupted). at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1580) ~[ignite-core-1.3.3.jar:1.3.3] at org.apache.ignite.internal.processors.cache.IgniteCacheProxy.cacheException(IgniteCacheProxy.java:1689) ~[ignite-core-1.3.3.jar:1.3.3] at org.apache.ignite.internal.processors.cache.IgniteCacheProxy.get(IgniteCacheProxy.java:755) ~[ignite-core-1.3.3.jar:1.3.3] at com..xx.products.business.repositories.IgniteProductRepository.getById(IgniteProductRepository.java:68) ~[ref-data-models.jar:na] ... 27 common frames omitted Caused by: org.apache.ignite.IgniteInterruptedException: Failed to wait for asynchronous operation permit (thread got interrupted). at org.apache.ignite.internal.util.IgniteUtils$2.apply(IgniteUtils.java:559) ~[ignite-core-1.3.3.jar:1.3.3] at org.apache.ignite.internal.util.IgniteUtils$2.apply(IgniteUtils.java:557) ~[ignite-core-1.3.3.jar:1.3.3] ... 31 common frames omitted Caused by: java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302) ~[na:1.8.0_45] at java.util.concurrent.Semaphore.acquire(Semaphore.java:312) ~[na:1.8.0_45] at org.apache.ignite.internal.processors.cache.GridCacheAdapter.asyncOpAcquire(GridCacheAdapter.java:4149) ~[ignite-core-1.3.3.jar:1.3.3] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.asyncOp(GridDhtAtomicCache.java:555) ~[ignite-core-1.3.3.jar:1.3.3] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.getAllAsync(GridDhtAtomicCache.java:271) ~[ignite-core-1.3.3.jar:1.3.3] at org.apache.ignite.internal.processors.cache.GridCacheAdapter.getAllAsync(GridCacheAdapter.java:4429) ~[ignite-core-1.3.3.jar:1.3.3] at org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4376) ~[ignite-core-1.3.3.jar:1.3.3] at org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1448) ~[ignite-core-1.3.3.jar:1.3.3] at org.apache.ignite.internal.processors.cache.IgniteCacheProxy.get(IgniteCacheProxy.java:748) ~[ignite-core-1.3.3.jar:1.3.3] ... 28 common frames omitted -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Failed-to-wait-for-asynchronous-operation-permit-thread-got-interrupted-tp1536.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Changing node attributes at runtime
Hi Matt, Attributes can't be changed in runtime because they are exchanged in discovery when a node join topology. I think you should have a replicated cache with node ID to collection of tags mapping. You can create cluster groups that will dynamically assign computations to nodes based on the cache contents. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Changing-node-attributes-at-runtime-tp1535p1537.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Failed to wait for asynchronous operation permit (thread got interrupted)
Hi Mirko, >From what I see, the thread that was doing get() was interrupted. Is it your thread or you're calling cache from an Ignite job? -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Failed-to-wait-for-asynchronous-operation-permit-thread-got-interrupted-tp1536p1538.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
