RE: Continuous queries and duplicates

[Sobolewski, Krzysztof]

I will take a look, thanks!

But, upon further investigation, it appears that there is no isolation 
whatsoever between the initial query and the listener in ContinuousQuery. The 
initial query can pick up entries added after it started and which had been 
already sent to the local listener. This way the entry is reported twice, and 
hence the duplicates. This is regardless of the type of the initial query 
(ScanQuery or SqlQuery). This reduces the usefulness of ContinuousQuery a lot, 
because we have to find out a way to rule out these duplicates, which is 
difficult and can incur a significant overhead.

So my follow-up question is this: is it behaving as designed, or is there some 
mechanism to prevent these duplicates from happening?
-KS


From: Ilya Kasnacheev [mailto:ilya.kasnach...@gmail.com]
Sent: 30 listopada 2018 14:02
To: user@ignite.apache.org
Subject: Re: Continuous queries and duplicates

Hello!

There should be isolation in AI 2.7 as an experimental feature.

Regards,
--
Ilya Kasnacheev


пн, 26 нояб. 2018 г. в 12:53, Sobolewski, Krzysztof 
:
Thanks. This is a little disappointing. ScanQuery would probably work, but it’s 
not as efficient (can’t use indexes etc.). Are there any plans to enable 
isolation on SqlQuery?
-KS

From: Ilya Kasnacheev [mailto:ilya.kasnach...@gmail.com]
Sent: 26 listopada 2018 08:01
To: user@ignite.apache.org
Subject: Re: Continuous queries and duplicates

Hello!

SQL queries have no isolation currently so it is not possible to avoid the 
problem that you described. You could try switching to ScanQuery, see if it 
helps; or learning to deal with duplicates.

Regards,
--
Ilya Kasnacheev


пт, 23 нояб. 2018 г. в 19:25, Sobolewski, Krzysztof 
:
Hi,

I'm wanting to use a ContinuousQuery and there is a slight issue with how it 
transitions from the initial query to the notifications phase. It turns out 
that if there are additions to the cache happening while the continuous query 
runs, an entry may be reported twice - once by the initial query and once by 
the listener. This is confirmed by experiment, BTW :) The initial query in this 
case is an SqlQuery.

So my question is: is this intentional? Or is it a bug? Is there something I 
can do to mitigate this? Is this is an issue of isolation level?

Thanks a lot for any pointers :)
-KS



Your Personal Data: We may collect and process information about you that may 
be subject to data protection laws. For more information about how we use and 
disclose your personal data, how we protect your information, our legal basis 
to use your information, your rights and who you can contact, please refer to: 
www.gs.com/privacy-notices<http://www.gs.com/privacy-notices>



Your Personal Data: We may collect and process information about you that may 
be subject to data protection laws. For more information about how we use and 
disclose your personal data, how we protect your information, our legal basis 
to use your information, your rights and who you can contact, please refer to: 
www.gs.com/privacy-notices



Your Personal Data: We may collect and process information about you that may 
be subject to data protection laws. For more information about how we use and 
disclose your personal data, how we protect your information, our legal basis 
to use your information, your rights and who you can contact, please refer to: 
www.gs.com/privacy-notices<http://www.gs.com/privacy-notices>

RE: Continuous queries and duplicates

[Sobolewski, Krzysztof]

Thanks. This is a little disappointing. ScanQuery would probably work, but it’s 
not as efficient (can’t use indexes etc.). Are there any plans to enable 
isolation on SqlQuery?
-KS

From: Ilya Kasnacheev [mailto:ilya.kasnach...@gmail.com]
Sent: 26 listopada 2018 08:01
To: user@ignite.apache.org
Subject: Re: Continuous queries and duplicates

Hello!

SQL queries have no isolation currently so it is not possible to avoid the 
problem that you described. You could try switching to ScanQuery, see if it 
helps; or learning to deal with duplicates.

Regards,
--
Ilya Kasnacheev


пт, 23 нояб. 2018 г. в 19:25, Sobolewski, Krzysztof 
mailto:krzysztof.sobolew...@gs.com>>:
Hi,

I'm wanting to use a ContinuousQuery and there is a slight issue with how it 
transitions from the initial query to the notifications phase. It turns out 
that if there are additions to the cache happening while the continuous query 
runs, an entry may be reported twice - once by the initial query and once by 
the listener. This is confirmed by experiment, BTW :) The initial query in this 
case is an SqlQuery.

So my question is: is this intentional? Or is it a bug? Is there something I 
can do to mitigate this? Is this is an issue of isolation level?

Thanks a lot for any pointers :)
-KS



Your Personal Data: We may collect and process information about you that may 
be subject to data protection laws. For more information about how we use and 
disclose your personal data, how we protect your information, our legal basis 
to use your information, your rights and who you can contact, please refer to: 
www.gs.com/privacy-notices<http://www.gs.com/privacy-notices><http://www.gs.com/privacy-notices>



Your Personal Data: We may collect and process information about you that may 
be subject to data protection laws. For more information about how we use and 
disclose your personal data, how we protect your information, our legal basis 
to use your information, your rights and who you can contact, please refer to: 
www.gs.com/privacy-notices<http://www.gs.com/privacy-notices>

Continuous queries and duplicates

[Sobolewski, Krzysztof]

Hi,

I'm wanting to use a ContinuousQuery and there is a slight issue with how it 
transitions from the initial query to the notifications phase. It turns out 
that if there are additions to the cache happening while the continuous query 
runs, an entry may be reported twice - once by the initial query and once by 
the listener. This is confirmed by experiment, BTW :) The initial query in this 
case is an SqlQuery.

So my question is: is this intentional? Or is it a bug? Is there something I 
can do to mitigate this? Is this is an issue of isolation level?

Thanks a lot for any pointers :)
-KS



Your Personal Data: We may collect and process information about you that may 
be subject to data protection laws. For more information about how we use and 
disclose your personal data, how we protect your information, our legal basis 
to use your information, your rights and who you can contact, please refer to: 
www.gs.com/privacy-notices

Use custom Data Region or custom Cache for IgniteAtomicReference - Ignite 2.3

[Krzysztof Chmielewski]

Hi all,
I would like to ask, if there is a way to use custom DataRegionConfiguration
(other than default) for storing IgniteAtomicReferences.

I would like to use Ignite's 2.3 feature where I can turn on/off Persistent
per Cache (via DataRegionConfiguration). I would like to enable persistent
only for a few IgniteAtomiReferences.

If we could set data region via AtomicConfiguration as we do for caches, or
set different cache to store AtomicReference other than Ignite's system
cache, this would be possible. 

Unfortunately AtomicConfiguration does not have this feature from what I
see.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Compute async Job from Ignite Service and return IgniteFuture

[Krzysztof Chmielewski]

Hi, thanks for clarification.

My use case it that I have some logic that is hidden from client but client
can execute this logic by calling Ignite Service. 

Some of those operation may take a while, so I thought that it would be nice
to run it asynchronously and return Future object so the client would be
able to know the end result if he wants. 

Thanks
Krzysztof 



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Compute async Job from Ignite Service and return IgniteFuture

[Krzysztof Chmielewski]

Hi Team,
I have a question regarding Ignite Services and Compute async.

If we have a deployed IgniteService like this

public class MySercviceImpl implements Service, MyService {


@IgniteInstanceResource
  private Ignite ignite;
  
  @Override
  public IgniteFuture doStuff() {

IgniteFuture runAsync =
ignite.compute(ignite.cluster().forLocal()).runAsync(new IgniteRunnable() {
  
  private static final long serialVersionUID = 1L;

  @Override
  public void run() {
   System.out.println(Thread.currentThread() + " Hello");
   System.out.println("World");
  }
});
  
return runAsync;
  }
}

Is it correct to call this service's method from a client and waits on
returned future? Something like this:
MyService serviceProxy = ignite.services().serviceProxy("MyService",
MyService.class, false);
IgniteFuture future = serviceProxy.doStuff();
future.get();

I've noticed that my client stuck on  doStuff.get();

Thanks
Krzysztof





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite Service and node failure

[Krzysztof Chmielewski]

Thanks for a fast response :)

All is clear now. 

Regards,
Krzysztof



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Ignite Service and node failure

[Krzysztof Chmielewski]

Hi Team,
I would like to ask about a preferable approach regarding the Service
acquiring via Ignite's Service Proxy and a node failure. 

Lets assume we have a service deployed as a Cluster Singleton. Grid client
acquires the service instance via ignite.services().serviceProxy(...)

The questions are:
1. What would happen if between service fetching and service usage, node
that has this service deployed failed. Would this returned proxy be still
valid? Assuming that service reinitialized it self on another node in the
grid. 

2. Should we always acquire new service proxy (call
gnite.services().serviceProxy(...)) when we want to call the service's
method or can we get it only once, keep the reference, and reusing it. 

Thanks
Krzysztof






--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Use @ServiceResource with property placeholder

[Krzysztof Chmielewski]

Hi all,
is it possible to use @ServiceResource with service name from property
placeholder?

In my Ignite config.xml I'm already using values from properties file

Thanks
Krzysztof



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Unable to query system cache through Visor console

[Krzysztof Chmielewski]

Thanks for your response Denis. 
My uses case would be to list all Ignite semaphores and Ignite atomic
references used in my Ignite grid.
As fare as I know, those are stored in system caches.

Thanks
Krzysztof 



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Unable to query system cache through Visor console

[Krzysztof Chmielewski]

Hi, 
is there a way to query system caches like *ignite-sys-atomic-cache* via
Visor console?
If I'm trying to execute Visor command "cache -scan -c=@c2" where @2 is a
cache's id, I'm receiving this error:
*(wrn) : Failed to get cache because it is a system cache:
ignite-sys-atomic-cache@default-ds-group*


Im using Ignite 2.3.0

Thanks
Krzysztof



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

IgniteCache.localEntries(CachePeekMode... peekModes) result

[Krzysztof Chmielewski]

Hello All,

I have a question about IgniteCache's *localEntries(CachePeekMode
<https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/CachePeekMode.html>*
*... peekModes)* method result.
In our case we are using Ignite 2.1.0

We are broadcasting a closure to our ignite nodes to call
igniteCache.localEntires. We are using offHeap mode, so obviously all
entries are stored offHeap. But when we call cache.*localEntires *and we
get the Iterable
<http://docs.oracle.com/javase/7/docs/api/java/lang/Iterable.html?is-external=true>
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteCache.html>
,V
<https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteCache.html>
*>> *as a result,  does it men that, those entries were "transferred" to
heap?

I'm asking because in ScanQuery object we can set the pageSize to get only
a certain number of entries at once, and further pages are fetched
automatically.

Thanks,
Krzysztof

Re: Partitioning on a non-uniform cluster

[Krzysztof]

For the second option: would not it cause the over-utilisation of CPUs on the
nodes with two servers running?



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Partitioning-on-a-non-uniform-cluster-tp8940p9076.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Partitioning on a non-uniform cluster

[Krzysztof]

Hello,

Judging by the documentation and some discussions on this list, can you
confirm that Ignite cache does not take into account different memory
settings, i.e. if we have various nodes with 16GB and 32GB allocated for
cache, there would be no two times more partitions assigned to larger nodes?

In order to not to underutilize larger nodes or overfill smaller nodes we
would have to develop our own affinity strategy via AffinityFunction in
order to make it cache-size aware?

RendezvousAffinityFunction seems to be completely resource-blind?

Could you please clarify what would be the best way to achieve balanced
distribution cluster memory-wise?

Thanks
Krzysztof







--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Partitioning-on-a-non-uniform-cluster-tp8940.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite Cluster Communication with SSH Tunnels

[Krzysztof]

Hello,

We would appreciate confirmation that
GridDhtAssignmentFetchFuture::requestFromNextNode() makes impossible to have
a client communicating via NAT, even with AddressResolver - or a pointer
where we make a mistake in the code/logic.
Judging by the code, GridDhtAssignmentFetchFuture has no notion of
AddressResolver at all and will always try to contact returned nodes from
behind the NAT.. 

Is there any way to make it work? WE must make a decision if to use Ignite
in our project and this seems to be a blocker..

Best Regards



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cluster-Communication-with-SSH-Tunnels-tp273p8225.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite Cluster Communication with SSH Tunnels

[Krzysztof]

We would really like to use Ignite for our project 

(http://gaia.esac.esa.int/documentation/GDR1/Data_analysis/sec_cu7var/sec_cu7introduction.html
which is part of http://sci.esa.int/gaia/) but this is a blocker.

Do you think we could make client behind the NAT working?

Best regards,
Krzysztof





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cluster-Communication-with-SSH-Tunnels-tp273p8180.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite Cluster Communication with SSH Tunnels

[Krzysztof]

ationSpi.java:1951)

at
org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1304)

... 11 more

Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
connect to node (is node still alive?). Make sure that each ComputeTask and
cache Transaction has a timeout set in order to prevent parties from
waiting forever in case of network issues
[nodeId=8148bc4e-6348-4a7a-95ed-14e403b9c615

, addrs=[/192.168.168.5:47100, /127.0.0.1:47100, 0:0:0:0:0:0:0:1%lo:47100]]

at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2519)

at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioClient(TcpCommunicationSpi.java:2157)

at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2051)

at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:1985)

... 13 more

Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to
connect to address: /192.168.168.5:47100

at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2521)

... 16 more

Caused by: java.net.ConnectException: Connection refused

at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)

at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)

at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:111)

at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2380)

... 16 more

Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to
connect to address: /127.0.0.1:47100

at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2521)

... 16 more

Caused by: java.net.ConnectException: Connection refused

at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)

at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)

at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:111)

at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2380)

... 16 more

Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to
connect to address: 0:0:0:0:0:0:0:1%lo:47100

at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2521)

... 16 more

Caused by: java.net.UnknownHostException

at sun.nio.ch.Net.translateException(Net.java:155)

at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:127)

at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2380)

... 16 more

[01:22:09,700][WARN ][main][GridCachePartitionExchangeManager] Failed to
wait for initial partition map exchange. Possible reasons are:

  ^-- Transactions in deadlock.

  ^-- Long running transactions (ignore if this is the case).

...


I would appreciate clarifying this..
Krzysztof

On Fri, Oct 7, 2016 at 12:58 AM, vkulichenko [via Apache Ignite Users] <
ml-node+s70518n8134...@n6.nabble.com> wrote:

> You set 47099 as a communication port, but I don't see how it's mapped in
> the resolver. I think this is the reason.
>
> -Val
>
> --
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-
> Cluster-Communication-with-SSH-Tunnels-tp273p8134.html
> To unsubscribe from Ignite Cluster Communication with SSH Tunnels, click
> here
> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=273&code=eWF6dW5hQGdtYWlsLmNvbXwyNzN8NjE2ODY4NTk3>
> .
> NAML
> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cluster-Communication-with-SSH-Tunnels-tp273p8135.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite Cluster Communication with SSH Tunnels

[Krzysztof]

Thanks for the hint, but in the snippet I sent there's already:

// Override default discovery SPI.

cfg.setDiscoverySpi(spi);
cfg.setAddressResolver(basicResolver); // it does not change anything
(spi, commSpi have the resolve set too).

Or do you mean something else?

Cheers

On Wed, Oct 5, 2016 at 7:20 PM, vkulichenko [via Apache Ignite Users]
 wrote:
> Hi,
>
> Try to set the address resolver on the IgniteConfiguration instead of
> TcpDiscoverySpi. It looks like discovery works, but further communication
> doesn't.
>
> -Val
>
> 
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cluster-Communication-with-SSH-Tunnels-tp273p8106.html
> To unsubscribe from Ignite Cluster Communication with SSH Tunnels, click
> here.
> NAML




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cluster-Communication-with-SSH-Tunnels-tp273p8114.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite Cluster Communication with SSH Tunnels

[Krzysztof]

ogyVersion
> [topVer=276, min
>
> orTopVer=0]], policy=4]
>
> at 
> org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1309)
>
> at 
> org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1373)
>
> at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.send(GridCacheIoManager.java:841)
>
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtAssignmentFetchFuture.requestFromNextNode(GridDhtAssignmentFetchFuture.java:185)
>
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtAssignmentFetchFuture.init(GridDhtAssignmentFetchFuture.java:107)
>
> at 
> org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.fetchAffinityOnJoin(CacheAffinitySharedManager.java:953)
>
> at 
> org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onClientEvent(CacheAffinitySharedManager.java:639)
>
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onClientNodeEvent(GridDhtPartitionsExchangeFuture.java:619)
>
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:464)
>
> at 
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1447)
>
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
>
> at java.lang.Thread.run(Thread.java:745)
>
> Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to
> send message to remote node: TcpDiscoveryNode
> [id=337d89f0-c14a-4b5d-8d33-b995331dcf63, addrs=[0:0:0:0:0:0:0:1%lo,
> 127.0.0.1, 192.168.168.8], sockAddrs=[0:0:0:0:0:0:0:1%lo:48500,
> /127.0.0.1:48500, /192.168.168.8:48500], discPort=48
>
> 500, order=218, intOrder=110, lastExchangeTime=1475627164474,
> loc=false, ver=1.8.0#20161004-sha1:a370bad1, isClient=false]
>
> at 
> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2013)
>
> at 
> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:1951)
>
> at 
> org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1304)
>
> 
>
>
> What am I missing?
>
> In 1.8 I am unable to have any DEBUG messages with the config that
> worked with 1.7, both  log4j and ignite-log4j from 1.8 on the
> classpath.
>
> Cheers
> Krzysztof
>
> On Tue, Oct 4, 2016 at 11:27 PM, vkulichenko [via Apache Ignite Users]
>  wrote:
>> All nodes should run on the same version.
>>
>> What is exactly different in log4j behavior?
>>
>> -Val
>>
>> 
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cluster-Communication-with-SSH-Tunnels-tp273p8090.html
>> To unsubscribe from Ignite Cluster Communication with SSH Tunnels, click
>> here.
>> NAML




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cluster-Communication-with-SSH-Tunnels-tp273p8103.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite Cluster Communication with SSH Tunnels

[Krzysztof]

.fetchAffinityOnJoin(CacheAffinitySharedManager.java:953)

at 
org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onClientEvent(CacheAffinitySharedManager.java:639)

at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onClientNodeEvent(GridDhtPartitionsExchangeFuture.java:619)

at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:464)

at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1447)

at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)

at java.lang.Thread.run(Thread.java:745)

Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to
send message to remote node: TcpDiscoveryNode
[id=337d89f0-c14a-4b5d-8d33-b995331dcf63, addrs=[0:0:0:0:0:0:0:1%lo,
127.0.0.1, 192.168.168.8], sockAddrs=[0:0:0:0:0:0:0:1%lo:48500,
/127.0.0.1:48500, /192.168.168.8:48500], discPort=48

500, order=218, intOrder=110, lastExchangeTime=1475627164474,
loc=false, ver=1.8.0#20161004-sha1:a370bad1, isClient=false]

at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2013)

at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:1951)

at 
org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1304)




What am I missing?

In 1.8 I am unable to have any DEBUG messages with the config that
worked with 1.7, both  log4j and ignite-log4j from 1.8 on the
classpath.

Cheers
Krzysztof

On Tue, Oct 4, 2016 at 11:27 PM, vkulichenko [via Apache Ignite Users]
 wrote:
> All nodes should run on the same version.
>
> What is exactly different in log4j behavior?
>
> -Val
>
> 
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cluster-Communication-with-SSH-Tunnels-tp273p8090.html
> To unsubscribe from Ignite Cluster Communication with SSH Tunnels, click
> here.
> NAML




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cluster-Communication-with-SSH-Tunnels-tp273p8095.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite Cluster Communication with SSH Tunnels

[Krzysztof]

Thanks for the hint!
I was trying to use BasicAddressResolver from 1.7.1, but it was never
called for the list returned by the discovery.
I reckon the whole cluster must be the same version? With 1.8 on the
client only, even discovery does not work, just stays in the endless
loop.
I will convert the full cluster (not only client outside the cluster) into 1.8.

I noticed also some different behaviour with log4j module in 1.8 -
should it be configured differently?


for completeness trimmed sample code I am trying to run:


System.setProperty("IGNITE_QUIET", "false");

Ignition.setClientMode(true);

TcpDiscoverySpi spi = new TcpDiscoverySpi();

TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();


// Set initial IP addresses.

// Note that you can optionally specify a port or a port range.

ipFinder.setAddresses(Arrays.asList("localhost:48500")); // this is
pointing to one of the cluster nodes via the ssh tunnel, works in 1.7,
list is returned, not in 1.8 vs 1.7

spi.setIpFinder(ipFinder);

spi.setLocalAddress("localhost");

Map resolverAddresses = new HashMap();

for(int i=0; i< 55 ;i++){

 String orig = String.format("192.168.168.%d:47100", 1+i); / I would
expect this IPs returned
 String dest = String.format("localhost:%03d", 47101 + i);

resolverAddresses.put(orig, dest);

}

BasicAddressResolver basicResolver = null;

try {

basicResolver = new BasicAddressResolver(resolverAddresses);

} catch (UnknownHostException e) {

// TODO Auto-generated catch block

e.printStackTrace();

}



spi.setAddressResolver(basicResolver);

IgniteConfiguration cfg = new IgniteConfiguration();



// Override default discovery SPI.

cfg.setDiscoverySpi(spi);
cfg.setAddressResolver(basicResolver); // it does not change anything
(spi, commSpi have the resolve set too).

cfg.setFailureDetectionTimeout(15000);

// Explicitly configure TCP communication SPI by changing local port number for

// the nodes from the first cluster.

TcpCommunicationSpi commSpi=new TcpCommunicationSpi();


commSpi.setLocalPort(47099); //exposed via reverse channel -R:47099:...

commSpi.setLocalPortRange(1);

commSpi.setLocalAddress("localhost");

commSpi.setAddressResolver(basicResolver);

commSpi.setSharedMemoryPort(-1);

// Overriding communication SPI.

cfg.setCommunicationSpi(commSpi);

// logging

cfg.setGridLogger(new Log4JLogger("config/ignite-log4j.xml")); //has
effect in 1.7 but not in 1.8


// Start Ignite node.

Ignite ignite = Ignition.start(cfg);


CacheConfiguration cacheCfg = new CacheConfiguration("run_cache");

cacheCfg.setCacheMode(CacheMode.PARTITIONED);

cacheCfg.setBackups(0);


Any hints?

Cheers

On Tue, Oct 4, 2016 at 1:15 AM, vkulichenko [via Apache Ignite Users]
 wrote:
> Hi Krzysztof,
>
> This issue was fixed some time ago, so in 1.8 Ignite will publish public
> addresses in the IP finder. You can with the nightly build [1] in the
> meantime.
>
> [1] https://cwiki.apache.org/confluence/display/IGNITE/Nightly+Builds
>
> -Val
>
> 
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cluster-Communication-with-SSH-Tunnels-tp273p8066.html
> To unsubscribe from Ignite Cluster Communication with SSH Tunnels, click
> here.
> NAML




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cluster-Communication-with-SSH-Tunnels-tp273p8081.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite Cluster Communication with SSH Tunnels

[Krzysztof]

I have a simpler scenario in mind:
just one client connecting to the cluster of say 50 servers via ssh-tunnel.

I would be ready to define 50 tunnels forwarding to 50 servers on the
cluster, but it does not seem supported??

Discovery works via a single tunnel, static-based discovery of the
multicast-based cluster and AddressResolver, but TcpCommunicationSpi does
not, as the discovery returns list of internal addresses of the cluster
which are private of course, i.e. node from the list of nodes is defined as
such:

TcpDiscoveryNode [id=e8744d31-ce77-43d6-b3de-09dfa22516dc,
addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 192.168.168.22],
sockAddrs=[0:0:0:0:0:0:0:1%lo:48500, /127.0.0.1:48500,
/192.168.168.22:48500], discPort=48500, order=2, intOrder=2,
lastExchangeTime=1475454891191, loc=false, ver=1.7.0#19700101-sha1:,
isClient=false]

and 192.168.168.22 addrs is used. I would imagine that AddressResolver could
be used for TcpCommunicationSpi that would translate both
192.168.168.22:COMMS_PORT into my
localhost:SSH_TUNNELED_PORT_FOR_192.168.168.22.

Is it possible?

Am I missing something, the same problem would be with NATed/Firewalled
addresses always, except the localhost the router/firewall address would be
returned?

Thanks
Krzysztof




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cluster-Communication-with-SSH-Tunnels-tp273p8046.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.