question about igntie1.7 update

[Zhengqingzheng]

Hi there,
I have update my ignite server to the newest version 1.7.0. And I find some 
interest phenomenons.

1.   I notice that sometimes I cannot get into the ignite console via the 
ignitevisorcmd command. This happens when the cpu load is high(actually not 
very high ,for instance in my case when the cpu load is upto 12.5%, I cannot 
get into the ignite cmd console).  After the cpu load drop to 0.57%, I can 
login the console.

2.   I have two server nodes, but sometimes when I login the console and 
use "node" command to check the statics info, I find that only one server node 
is available. However If I use "ps"  command under ubuntu system, I am sure the 
other server thread is still running.  I seems that all the data is transferred 
to one server node, and dropped from the grid, but the actual ignite instance 
is still running in the ubuntu system without login the ignite server grid.

3.   One good aspect is that, the new version ignite can serialize and 
deserialize the java object better(under 1.5.0.fianal, when save an java object 
without jdbc mapping, it will be saved as hash map data, and returns hashmap 
rather than the java object. In the new version, I can get java object directly.

4.   Also I notice that the sql query performance is still not as fast as I 
expected (currently, I have a sql query which will return null result, it still 
cost about 17ms).

Any suggestions for 1 ,2, and 4?


Best regards,

Re: Near Cache

[javastuff....@gmail.com]

Thank you for quick response. Here are mode details and my view point of
usage -

>From java application we are starting Ignite instance using
"Ignition.start(configFilename)". 
We are keeping "clienMode" default. ( clientMode property not defined in XML
configuration and java code also not using Ignition.setClientMode() )
Typically we run 2 to 4 remote instances of application.
Total cache size (multiple caches) can grow beyond 10s of GB, so off-heap.
We have few caches with READ ONLY data and for those we would like to have
Near cache.

With Near cache configuration we are looking at below benefits -
1. Avoid Serialization/de-serialization for frequently used objects
2. Owner node for cached entry can be Node2, but Node 1 or all nodes are
frequently using it, so avoid network hop. (Node2 holds data in off-heap
partitioned cache, at the same time it is available locally with node1 or
all nodes in near cache after first access)
3. Want to keep limited entries in near cache to avoid huge heap size.

Thanks,
--Sambhav




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Near-Cache-tp7033p7035.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Near Cache

[vkulichenko]

Hi Sam,

How many nodes do you have? Is it a server node that is embedded in the
application? If yes, what is the reason for having a near cache?

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Near-Cache-tp7033p7034.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Near Cache

[javastuff....@gmail.com]

Hello,

I am new to Apache Ignite and exploring many cool features. My application
requires Ignite to run as embedded and use off-heap memory to cache data.
That means each application node is Server with off-heap storage capability. 

Can you please post example XML configuration for Near cache? 
I used below and caching seems to work, but not sure if it has correctly
configured it as Near cache. Tried MBEAN and visor console, but no hints.
Can you please comment on below configuration and way to ensure it is
configured correctly?









 
 
 




How can I configure off-heap 5GB and near cache for 1 entries?  I am
unable to configure eviction policy for near cache in XML.

Thanks a lot.
-Sam



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Near-Cache-tp7033.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cassandra - Ignite Exception

[Igor Rudyak]

Hi Kamal,

Do you have this problem on Ignite-Cassandra module side or on Cassandra
nodes side? If you have it on Ignite-Cassandra module side that could be
because of this:  https://issues.apache.org/jira/browse/CASSANDRA-8387. It
sometimes happen when you start from clean Cassandra cluster without any
keyspaces/tables to store your Ignite caches.

In this situation Ignite-Cassandra module tries to create new
keyspaces/tables in Cassandra and executes CQL statements like "create
table  if not exists". According to the Cassandra ticket above, when
statement like "create table" is executed simultaneously from multiple
clients (Ignite nodes in our case) it could cause such an error in the log.



Regards,
Igor Rudyak

On Fri, Aug 12, 2016 at 12:20 PM, vkulichenko  wrote:

> Hi Kamal,
>
> I would ask on the Cassandra forum about this. Can you do this? The
> exception looks very weird to me, because it complains about the mismatch,
> but the IDs are actually equal:
>
> org.apache.cassandra.exceptions.ConfigurationException: Column family ID
> mismatch (found b2b47bf0-5f94-11e6-867a-cb7bb92c; expected
> b2b1bcd0-5f94-11e6-867a-cb7bb92c)
>
> -Val
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Cassandra-Ignite-Exception-tp7013p7024.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Any plan for a book on Apache Ignite

[Igor Rudyak]

Looks cool!!!

Igor

On Fri, Aug 12, 2016 at 12:48 PM, Denis Magda  wrote:

> Cross-posting to the dev list.
>
> Wow, this is great. Are you the one of the authors?
>
> —
> Denis
>
> > On Aug 12, 2016, at 12:08 PM, srecon  wrote:
> >
> > One book is going to publish soon, you can check the table of contents
> here  Best Regards
> > View this message in context: Re: Any plan for a book on Apache Ignite <
> http://apache-ignite-users.70518.x6.nabble.com/Any-plan-
> for-a-book-on-Apache-Ignite-tp357p7022.html>
> > Sent from the Apache Ignite Users mailing list archive <
> http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>
>

Re: Computing grid from two VPS and localhost

[vkulichenko]

Hi Petr,

Please properly subscribe to the mailing list so that the community can
receive email notifications for your messages. To subscribe, send empty
email to user-subscr...@ignite.apache.org and follow simple instructions in
the reply.


petr.somek wrote
> I have this problem. I have two servers which has public IP and is fully
> accessible from internet. I have installed ignite 1.6 to both of them, ale
> configure them to join to together. Everything is working well, and both
> server work together in cloud. But now I'd like to join to this cloud my
> PC which is connected to internet but doesn't have public IP. So from my
> computer I can see both of servers, but server doesn't see my computer. Is
> it possible to have compute grid from this three machines?

You need to create an implementation of AddressResolver [1] and provide it
in the configuration via IgniteConfiguration.setAddressResolver property.
The resolver should map the private node address (the one behind NAT) to the
public address of the router. This public address will be published by a
node as one of his addresses, so that other nodes will be able to connect
through the router. The router itself should have properly configured port
forwarding.

[1]
https://ignite.apache.org/releases/1.7.0/javadoc/org/apache/ignite/configuration/AddressResolver.html

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Computing-grid-from-two-VPS-and-localhost-tp6920p7030.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Failed to wait for initial partition map exchange

[vkulichenko]

Hi Jason,

Can you try with 1.7? There were couple of serious fixes in communication
SPI implementation which could potentially cause these issues.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Failed-to-wait-for-initial-partition-map-exchange-tp6252p7029.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: About hot deployment

[vkulichenko]

Hi,

Please properly subscribe to the mailing list so that the community can
receive email notifications for your messages. To subscribe, send empty
email to user-subscr...@ignite.apache.org and follow simple instructions in
the reply.


xjg125 wrote
> I load a lot of business data into cache, this process takes a lot of
> time.
> I use functions such as ContinuousQuery, IgniteCompute,
> IgniteDataStreamer, IgniteServices, etc..
> 
> Now I need to update my business class frequently, because some of the
> code needs to run on the server, I have to restart the server node, but
> also had to reload the data.
*
> I want to know if I can update my class without loading the data again？
*
> Use OSGi or any other way?
> If you can, can you provide a document, or an example?
> 
> Thank you

Model classes are not required on server nodes at all, because all the data
in cache is stored in binary format and is never deserialized there unless
you explicitly ask for it. See [1] for more details. In a nutshell, the
binary format allows to lookup individual fields without deserializing the
whole object, this makes dynamic schema possible out of the box.

[1] https://apacheignite.readme.io/docs/binary-marshaller

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/About-hot-deployment-tp6902p7028.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Any plan for a book on Apache Ignite

[Denis Magda]

Cross-posting to the dev list.

Wow, this is great. Are you the one of the authors?

—
Denis

> On Aug 12, 2016, at 12:08 PM, srecon  wrote:
> 
> One book is going to publish soon, you can check the table of contents here  
> Best Regards 
> View this message in context: Re: Any plan for a book on Apache Ignite 
> 
> Sent from the Apache Ignite Users mailing list archive 
>  at Nabble.com.

Re: Embedded mode ignite on spark

[vkulichenko]

Hi,

We're missing something here... Can you go through the steps described in
[1] and let me know if get correct results there?

Also if you can create a small project on GitHub that I can use to reproduce
the issue, that would be great.

[1]
https://apacheignite-fs.readme.io/docs/testing-integration-with-spark-shell

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Embedded-mode-ignite-on-spark-tp6942p7026.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Fail to get the result by using use TextQuery

[vkulichenko]

Hi,

Please properly subscribe to the mailing list so that the community can
receive email notifications for your messages. To subscribe, send empty
email to user-subscr...@ignite.apache.org and follow simple instructions in
the reply.


scq wrote
> I can't get the result by using use TextQuery,  code example as follows:
> ...

Can you provide more details? What is the actual issue? Are there any
exceptions?

It would be also great if you create a project on GitHub that I can run and
reproduce the issue. Now it's not cleat what is happening at all, you didn't
even provide the query you're executing.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Fail-to-get-the-result-by-using-use-TextQuery-tp7005p7025.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cassandra - Ignite Exception

[vkulichenko]

Hi Kamal,

I would ask on the Cassandra forum about this. Can you do this? The
exception looks very weird to me, because it complains about the mismatch,
but the IDs are actually equal:

org.apache.cassandra.exceptions.ConfigurationException: Column family ID
mismatch (found b2b47bf0-5f94-11e6-867a-cb7bb92c; expected
b2b1bcd0-5f94-11e6-867a-cb7bb92c)

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Cassandra-Ignite-Exception-tp7013p7024.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: LOOK THORUGH THIS ERROR

[vkulichenko]

Is it possible that you already have another version of Hibernate in your
application?

Actually, I remember you having the same issue a while ago. Were you able to
resolve it?

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/LOOK-THORUGH-THIS-ERROR-tp6977p7023.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Any plan for a book on Apache Ignite

[srecon]

One book is going to publish soon, you can check the table of contents  here
  Best Regards  



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Any-plan-for-a-book-on-Apache-Ignite-tp357p7022.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite Compute - Rebalance Scenario

[vkulichenko]

Note that this was changed recently [1] and the change was not released in
1.7. Kamal, can you try the nightly build [2] and check if it works as you
expect there?

[1] https://issues.apache.org/jira/browse/IGNITE-2310
[2]
https://builds.apache.org/view/H-L/view/Ignite/job/Ignite-nightly/lastSuccessfulBuild/



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Compute-Rebalance-Scenario-tp7004p7021.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite Compute - Rebalance Scenario

[Vladislav Pyatkov]

Partitions, where a key stored, will nut be migrated during task executing.

By java doc[1]:

The data of the partition where affKey is stored will not be migrated
from the target node while the job is executed.

You not need to worried, data will be situated in the same node and will
migrate after.

[1]:
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteCompute.java

On Fri, Aug 12, 2016 at 9:31 AM, Kamal C  wrote:

> Hi all,
>
> I'm using IgniteCompute#affinityRun(cacheName, key, runnable) to
> execute the tasks where the data resides. If cache rebalance happens in the
> middle of task execution, then the task is not aware of the rebalance. Is
> there any provision available to interrupt the task ?
>
> I would like to throw `ComputeJobFailOverException` if interrupted.
>
> Regards,
> Kamal C
>

Re: Fail to join topology and repeat join process

[Vladislav Pyatkov]

Most likely, rebalancing, after node joined to cluster, entails garbage
collection.
You can see how to configure GC in this tutorial[1].
Can you provide log from joining node and GC log from there?

[1]: https://apacheignite.readme.io/docs/jvm-and-system-tuning

On Fri, Aug 12, 2016 at 4:53 AM, Jason  wrote:

> The attached is the config file.
>
> For server node, one node one machine. For the client node, one machine may
> have few ones.
>
> Just do a rough check, the error logs in all the server nodes are almost
> the
> same.
>
> default-config.xml
>  n7002/default-config.xml>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Fail-to-join-topology-and-repeat-join-
> process-tp6987p7002.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>



-- 
Vladislav Pyatkov

Re: IgniteDataStreamer can't wirte data to ignite cluster

[vdpyatkov]

Continue discussion here
http://apache-ignite-users.70518.x6.nabble.com/IgniteDataStreamer-can-t-wirte-data-to-ignite-cluster-tt7012.html



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/IgniteDataStreamer-can-t-wirte-data-to-ignite-cluster-tp7003p7018.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: LOOK THORUGH THIS ERROR

[Ravi Puri]

Still the same error as i matched ignite-core 1.5 ver with ignite-hibernate
1.5... but still it is not resolved.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/LOOK-THORUGH-THIS-ERROR-tp6977p7017.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IgniteDataStreamer can't wirte data to ignite cluster

[Vladislav Pyatkov]

Hello,

The issue with grid segmented: The reasons of this may be long garbage
collector pause, long pause of process of user layer due to process of
system kernal work, network problem (like break).

By steps:

1) Update ignite version, because the 1.5.0 is very old. 1.7.0 is already
available.

2) Collect logs work garbage collector. You can use the article [1].

3) Check your system, using utils top, dstat, ping

It the issue continue to repeat, provide logs from server and client,
garbage collector logs and output the utils at the time of the crash.

On Fri, Aug 12, 2016 at 5:08 AM, 波板仔 <373891...@qq.com> wrote:

> My ignite cluster is deploy on 2 machines. Each machine has 4 ignite
> process. like:
>
> $jps
> 30678 CommandLineStartup
> 30723 CommandLineStartup
> 52461 Jps
> 30698 CommandLineStartup
> 30681 CommandLineStartup
>
> When there is little data in the ignite cluster, I can use
> IgniteDataStreamer to write data into the ignite cluster. But if there have
> been more than 200G data in the ignite cluster, I can't use
> IgniteDataStreamer to write more data into the cluster. The memory and disk
> space are both enough. and the CPU load average is 2.09. The client log is :
>
>
> [17:04:00,407][INFO][main][IgniteKernal%grid_recmd] Daemon mode: off
> [17:04:00,407][INFO][main][IgniteKernal%grid_recmd] OS: Linux
> [17:04:00,407][INFO][main][IgniteKernal%grid_recmd] OS user: mqq
> [17:04:00,408][INFO][main][IgniteKernal%grid_recmd] Language runtime:
> Java Platform API Specification ver. 1.7
> [17:04:00,408][INFO][main][IgniteKernal%grid_recmd] VM information:
> Java(TM) SE Runtime Environment 1.7.0_72-b14 Oracle Corporation Java
> HotSpot(TM) 64-Bit Server VM 24.72-b04
> [17:04:00,409][INFO][main][IgniteKernal%grid_recmd] VM total memory:
> 2.9GB
> [17:04:00,409][INFO][main][IgniteKernal%grid_recmd] Remote Management
> [restart: off, REST: on, JMX (remote: off)]
> [17:04:00,409][INFO][main][IgniteKernal%grid_recmd]
> IGNITE_HOME=/data1/home/igni1.5
> [17:04:00,410][INFO][main][IgniteKernal%grid_recmd] VM arguments:
> [-Xms3g, -Xmx3g, -XX:-UseGCOverheadLimit, -XX:+AggressiveOpts,
> -XX:MaxPermSize=256m, -XX:+UseParNewGC, -XX:+UseConcMarkSweepGC,
> -XX:+UseTLAB, -DIGNITE_QUIET=false]
> [17:04:00,418][INFO][main][IgniteKernal%grid_recmd] Configured caches
> ['ignite-marshaller-sys-cache', 'ignite-sys-cache',
> 'ignite-atomics-sys-cache', 'cache_recmd']
> [17:04:00,422][INFO][main][IgniteKernal%grid_recmd] 3-rd party
> licenses can be found at: /data1/home/igni1.5/libs/licenses
> [17:04:00,518][INFO][main][IgniteKernal%grid_recmd] Non-loopback
> local IPs: 10.0.0.1
> [17:04:00,519][INFO][main][IgniteKernal%grid_recmd] Enabled local
> MACs: 9017ACC1492D
> [17:04:00,530][INFO][main][IgnitePluginProcessor] Configured plugins:
> [17:04:00,530][INFO][main][IgnitePluginProcessor]   ^-- None
> [17:04:00,530][INFO][main][IgnitePluginProcessor]
> [17:04:00,574][WARNING][main][TcpCommunicationSpi] Failure detection
> timeout will be ignored (one of SPI parameters has been set explicitly)
> [17:04:00,627][INFO][main][TcpCommunicationSpi] IPC shared memory
> server endpoint started [port=48104, tokDir=/data1/home/igni1.5/
> work/ipc/shmem/e052dc07-8992-4834-87fd-abf84c0f644a-14472]
> [17:04:00,628][INFO][main][TcpCommunicationSpi] Successfully bound
> shared memory communication to TCP port [port=48104, locHost=
> 0.0.0.0/0.0.0.0]
> [17:04:00,652][INFO][main][TcpCommunicationSpi] Successfully bound to
> TCP port [port=47106, locHost=0.0.0.0/0.0.0.0]
> [17:04:00,685][WARNING][main][NoopCheckpointSpi] Checkpoints are
> disabled (to enable configure any GridCheckpointSpi implementation)
> [17:04:00,749][WARNING][main][GridCollisionManager] Collision
> resolution is disabled (all jobs will be activated upon arrival).
> [17:04:00,755][WARNING][main][NoopSwapSpaceSpi] Swap space is
> disabled. To enable use FileSwapSpaceSpi.
> [17:04:00,757][INFO][main][IgniteKernal%grid_recmd] Security status
> [authentication=off, tls/ssl=off]
> [17:04:00,861][WARNING][main][GridCacheProcessor] Deployment mode for
> cache is not CONTINUOUS or SHARED (it is recommended that you change
> deployment mode and restart): ISOLATED
> [17:04:01,040][INFO][main][GridTcpRestProtocol] Command protocol
> successfully started [name=TCP binary, host=0.0.0.0/0.0.0.0, port=11217]
> 
> [17:04:01,138][SEVERE][tcp-client-disco-msg-worker-#4%grid_recmd%][TcpDiscoverySpi]
> Exception on joining: Connection refused
> java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.AbstractPlainSocketImpl.doConnect(
> AbstractPlainSocketImpl.java:339)
> at java.net.AbstractPlainSocketImpl.connectToAddress(
> AbstractPlainSocketImpl.java:200)
> at java.net.AbstractPlainSocketImpl.connect(
> AbstractPlainSocketImpl.java:182)
> at 

Re: Re:Re: Ignite for Spark on YARN Deployment

[Nikolai Tikhonov]

Hi,

Could you make sure that you have access from "driver" machine to machines
on which running YARN? Could you ping them?

On Fri, Aug 12, 2016 at 11:26 AM, percent620  wrote:

> scala> [14:52:04] New version is available at ignite.apache.org: 1.7.0
> 16/08/12 14:52:46 ERROR TcpDiscoverySpi: Failed to reconnect to cluster
> (consider increasing 'networkTimeout' configuration property)
> [networkTimeout=5000]
> 16/08/12 15:37:54 ERROR GridClockSyncProcessor: Failed to send time sync
> snapshot to remote node (did not leave grid?)
> [nodeId=20d8035e-abec-44ca-a5c0-7e3308984d83,
> msg=GridClockDeltaSnapshotMessage [snapVer=GridClockDeltaVersion [ver=34,
> topVer=13], deltas={3286d19e-72d5-4353-86d2-03ffdb6c4733=0,
> ac2c7723-fa93-49d5-92c3-1d815b6b178b=0,
> 7d7790d2-a67f-4d76-b36f-47dc08025594=0,
> 3cbcbe73-f29d-4051-952a-9ba7b80cf1c3=0,
> 20d8035e-abec-44ca-a5c0-7e3308984d83=0,
> f94defdc-ca9b-450b-91c0-6a6b26f5d553=0}], err=Failed to send message (node
> may have left the grid or TCP connection cannot be established due to
> firewall issues) [node=TcpDiscoveryNode
> [id=20d8035e-abec-44ca-a5c0-7e3308984d83, addrs=[XXX, 127.0.0.1],
> sockAddrs=[/Z:47500, /XXX:47500, /127.0.0.1:47500],
> discPort=47500,
> order=1, intOrder=1, lastExchangeTime=1470987075851, loc=false,
> ver=1.6.0#20160518-sha1:0b22c45b, isClient=false], topic=TOPIC_TIME_SYNC,
> msg=GridClockDeltaSnapshotMessage [snapVer=GridClockDeltaVersion [ver=34,
> topVer=13], deltas={3286d19e-72d5-4353-86d2-03ffdb6c4733=0,
> ac2c7723-fa93-49d5-92c3-1d815b6b178b=0,
> 7d7790d2-a67f-4d76-b36f-47dc08025594=0,
> 3cbcbe73-f29d-4051-952a-9ba7b80cf1c3=0,
> 20d8035e-abec-44ca-a5c0-7e3308984d83=0,
> f94defdc-ca9b-450b-91c0-6a6b26f5d553=0}], policy=2]]
> 16/08/12 15:38:02 ERROR TcpDiscoverySpi: Failed to reconnect to cluster
> (consider increasing 'networkTimeout' configuration property)
> [networkTimeout=5000]
> 16/08/12 15:50:02 ERROR TcpDiscoverySpi: Failed to reconnect to cluster
> (consider increasing 'networkTimeout' configuration property)
> [networkTimeout=5000]
> Exception in thread "ignite-update-notifier-timer" class
> org.apache.ignite.IgniteClientDisconnectedException: Client node
> disconnected: null
> at
> org.apache.ignite.internal.GridKernalGatewayImpl.readLock(
> GridKernalGatewayImpl.java:87)
> at
> org.apache.ignite.internal.cluster.ClusterGroupAdapter.
> guard(ClusterGroupAdapter.java:170)
> at
> org.apache.ignite.internal.cluster.ClusterGroupAdapter.
> nodes(ClusterGroupAdapter.java:288)
> at
> org.apache.ignite.internal.processors.cluster.ClusterProcessor$
> UpdateNotifierTimerTask.safeRun(ClusterProcessor.java:224)
> at org.apache.ignite.internal.util.GridTimerTask.run(
> GridTimerTask.java:34)
> at java.util.TimerThread.mainLoop(Timer.java:555)
> at java.util.TimerThread.run(Timer.java:505)
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Ignite-for-Spark-on-YARN-Deployment-tp6910p7010.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Re:Re: Ignite for Spark on YARN Deployment

[percent620]

scala> [14:52:04] New version is available at ignite.apache.org: 1.7.0
16/08/12 14:52:46 ERROR TcpDiscoverySpi: Failed to reconnect to cluster
(consider increasing 'networkTimeout' configuration property)
[networkTimeout=5000]
16/08/12 15:37:54 ERROR GridClockSyncProcessor: Failed to send time sync
snapshot to remote node (did not leave grid?)
[nodeId=20d8035e-abec-44ca-a5c0-7e3308984d83,
msg=GridClockDeltaSnapshotMessage [snapVer=GridClockDeltaVersion [ver=34,
topVer=13], deltas={3286d19e-72d5-4353-86d2-03ffdb6c4733=0,
ac2c7723-fa93-49d5-92c3-1d815b6b178b=0,
7d7790d2-a67f-4d76-b36f-47dc08025594=0,
3cbcbe73-f29d-4051-952a-9ba7b80cf1c3=0,
20d8035e-abec-44ca-a5c0-7e3308984d83=0,
f94defdc-ca9b-450b-91c0-6a6b26f5d553=0}], err=Failed to send message (node
may have left the grid or TCP connection cannot be established due to
firewall issues) [node=TcpDiscoveryNode
[id=20d8035e-abec-44ca-a5c0-7e3308984d83, addrs=[XXX, 127.0.0.1],
sockAddrs=[/Z:47500, /XXX:47500, /127.0.0.1:47500], discPort=47500,
order=1, intOrder=1, lastExchangeTime=1470987075851, loc=false,
ver=1.6.0#20160518-sha1:0b22c45b, isClient=false], topic=TOPIC_TIME_SYNC,
msg=GridClockDeltaSnapshotMessage [snapVer=GridClockDeltaVersion [ver=34,
topVer=13], deltas={3286d19e-72d5-4353-86d2-03ffdb6c4733=0,
ac2c7723-fa93-49d5-92c3-1d815b6b178b=0,
7d7790d2-a67f-4d76-b36f-47dc08025594=0,
3cbcbe73-f29d-4051-952a-9ba7b80cf1c3=0,
20d8035e-abec-44ca-a5c0-7e3308984d83=0,
f94defdc-ca9b-450b-91c0-6a6b26f5d553=0}], policy=2]]
16/08/12 15:38:02 ERROR TcpDiscoverySpi: Failed to reconnect to cluster
(consider increasing 'networkTimeout' configuration property)
[networkTimeout=5000]
16/08/12 15:50:02 ERROR TcpDiscoverySpi: Failed to reconnect to cluster
(consider increasing 'networkTimeout' configuration property)
[networkTimeout=5000]
Exception in thread "ignite-update-notifier-timer" class
org.apache.ignite.IgniteClientDisconnectedException: Client node
disconnected: null
at
org.apache.ignite.internal.GridKernalGatewayImpl.readLock(GridKernalGatewayImpl.java:87)
at
org.apache.ignite.internal.cluster.ClusterGroupAdapter.guard(ClusterGroupAdapter.java:170)
at
org.apache.ignite.internal.cluster.ClusterGroupAdapter.nodes(ClusterGroupAdapter.java:288)
at
org.apache.ignite.internal.processors.cluster.ClusterProcessor$UpdateNotifierTimerTask.safeRun(ClusterProcessor.java:224)
at 
org.apache.ignite.internal.util.GridTimerTask.run(GridTimerTask.java:34)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-for-Spark-on-YARN-Deployment-tp6910p7010.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Re:Re: Ignite for Spark on YARN Deployment

[percent620]

Get the following error message after a few minutes
scala> val ic = new IgniteContext[Integer,
Integer](sc,"/u01/yueyi/apache-ignite-fabric-1.6.0-bin/config/default-config.xml")
ic: org.apache.ignite.spark.IgniteContext[Integer,Integer] =
org.apache.ignite.spark.IgniteContext@26917c50

scala> [14:52:04] New version is available at ignite.apache.org: 1.7.0
16/08/12 14:52:46 ERROR TcpDiscoverySpi: Failed to reconnect to cluster
(consider increasing 'networkTimeout' configuration property)
[networkTimeout=5000]


What should i do next step on this issue? Can anyone tell me? thanks



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-for-Spark-on-YARN-Deployment-tp6910p7007.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Cassandra - Ignite Exception

[Kamal C]

Hi all,

I'm using Cassandra as Persistent store for the caches. Once in a
while, I'm getting the below exceptions in the cassandra
but the data (put / get) is fine. I've searched to find the root-cause but
unable to conclude it.

I'm using cassandra-3.7 version, JDK 1.8, centos- 6.5 in standalone mode.
(only one cassandra node)


INFO  07:24:53 Create new table:
org.apache.cassandra.config.CFMetaData@64b34445[cfId=b2b1bcd0-5f94-11e6-867a-cb7bb92c,ksName=fms,cfName=flapalarmcache,flags=[COMPOUND],params=TableParams{comment=,
read_repair_chance=0.0, dclocal_read_repair_chance=0.1,
bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000,
default_time_to_live=0, memtable_flush_period_in_ms=0,
min_index_interval=128, max_index_interval=2048,
speculative_retry=99PERCENTILE, caching={'keys' : 'ALL',
'rows_per_partition' : 'NONE'},
compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,
options={min_threshold=4, max_threshold=32}},
compression=org.apache.cassandra.schema.CompressionParams@baaacb37,
extensions={}},comparator=comparator(),partitionColumns=[[] |
[alarm]],partitionKeyColumns=[ColumnDefinition{name=key,
type=org.apache.cassandra.db.marshal.BytesType, kind=PARTITION_KEY,
position=0}],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.BytesType,columnMetadata=[ColumnDefinition{name=alarm,
type=org.apache.cassandra.db.marshal.BytesType, kind=REGULAR, position=-1},
ColumnDefinition{name=key, type=org.apache.cassandra.db.marshal.BytesType,
kind=PARTITION_KEY, position=0}],droppedColumns={},triggers=[],indexes=[]]
INFO  07:24:53 Create new table:
org.apache.cassandra.config.CFMetaData@641c948a[cfId=b2b47bf0-5f94-11e6-867a-cb7bb92c,ksName=fms,cfName=flapalarmcache,flags=[COMPOUND],params=TableParams{comment=,
read_repair_chance=0.0, dclocal_read_repair_chance=0.1,
bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000,
default_time_to_live=0, memtable_flush_period_in_ms=0,
min_index_interval=128, max_index_interval=2048,
speculative_retry=99PERCENTILE, caching={'keys' : 'ALL',
'rows_per_partition' : 'NONE'},
compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,
options={min_threshold=4, max_threshold=32}},
compression=org.apache.cassandra.schema.CompressionParams@baaacb37,
extensions={}},comparator=comparator(),partitionColumns=[[] |
[alarm]],partitionKeyColumns=[ColumnDefinition{name=key,
type=org.apache.cassandra.db.marshal.BytesType, kind=PARTITION_KEY,
position=0}],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.BytesType,columnMetadata=[ColumnDefinition{name=alarm,
type=org.apache.cassandra.db.marshal.BytesType, kind=REGULAR, position=-1},
ColumnDefinition{name=key, type=org.apache.cassandra.db.marshal.BytesType,
kind=PARTITION_KEY, position=0}],droppedColumns={},triggers=[],indexes=[]]
INFO  07:24:55 Initializing fms.flapalarmcache
ERROR 07:24:57 Exception in thread Thread[MigrationStage:1,5,main]
org.apache.cassandra.exceptions.ConfigurationException: Column family ID
mismatch (found b2b47bf0-5f94-11e6-867a-cb7bb92c; expected
b2b1bcd0-5f94-11e6-867a-cb7bb92c)
at
org.apache.cassandra.config.CFMetaData.validateCompatibility(CFMetaData.java:789)
~[apache-cassandra-3.7.jar:3.7]
at org.apache.cassandra.config.CFMetaData.apply(CFMetaData.java:749)
~[apache-cassandra-3.7.jar:3.7]
at org.apache.cassandra.config.Schema.updateTable(Schema.java:663)
~[apache-cassandra-3.7.jar:3.7]
at
org.apache.cassandra.schema.SchemaKeyspace.updateKeyspace(SchemaKeyspace.java:1347)
~[apache-cassandra-3.7.jar:3.7]
at
org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1303)
~[apache-cassandra-3.7.jar:3.7]
at
org.apache.cassandra.schema.SchemaKeyspace.mergeSchemaAndAnnounceVersion(SchemaKeyspace.java:1253)
~[apache-cassandra-3.7.jar:3.7]
at
org.apache.cassandra.service.MigrationManager$1.runMayThrow(MigrationManager.java:532)
~[apache-cassandra-3.7.jar:3.7]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-3.7.jar:3.7]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
ERROR 07:24:57 Unexpected error during query
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
org.apache.cassandra.exceptions.ConfigurationException: Column family ID
mismatch (found b2b47bf0-5f94-11e6-867a-cb7bb92c; expected
b2b1bcd0-5f94-11e6-867a-cb7bb92c)
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:384)
~[apache-cassandra-3.7.jar:3.7]
at

IgniteDataStreamer can't wirte data to ignite cluster

[??????]

My ignite cluster is deploy on 2 machines. Each machine has 4 ignite process. 
like:


$jps
30678 CommandLineStartup
30723 CommandLineStartup
52461 Jps
30698 CommandLineStartup
30681 CommandLineStartup


When there is little data in the ignite cluster, I can use IgniteDataStreamer 
to write data into the ignite cluster. But if there have been more than 200G 
data in the ignite cluster, I can't use IgniteDataStreamer to write more data 
into the cluster. The memory and disk space are both enough. and the CPU load 
average is 2.09. The client log is :




[17:04:00,407][INFO][main][IgniteKernal%grid_recmd] Daemon mode: off
[17:04:00,407][INFO][main][IgniteKernal%grid_recmd] OS: Linux
[17:04:00,407][INFO][main][IgniteKernal%grid_recmd] OS user: mqq
[17:04:00,408][INFO][main][IgniteKernal%grid_recmd] Language runtime: Java 
Platform API Specification ver. 1.7
[17:04:00,408][INFO][main][IgniteKernal%grid_recmd] VM information: 
Java(TM) SE Runtime Environment 1.7.0_72-b14 Oracle Corporation Java 
HotSpot(TM) 64-Bit Server VM 24.72-b04
[17:04:00,409][INFO][main][IgniteKernal%grid_recmd] VM total memory: 2.9GB
[17:04:00,409][INFO][main][IgniteKernal%grid_recmd] Remote Management 
[restart: off, REST: on, JMX (remote: off)]
[17:04:00,409][INFO][main][IgniteKernal%grid_recmd] 
IGNITE_HOME=/data1/home/igni1.5
[17:04:00,410][INFO][main][IgniteKernal%grid_recmd] VM arguments: [-Xms3g, 
-Xmx3g, -XX:-UseGCOverheadLimit, -XX:+AggressiveOpts, -XX:MaxPermSize=256m, 
-XX:+UseParNewGC, -XX:+UseConcMarkSweepGC, -XX:+UseTLAB, -DIGNITE_QUIET=false]
[17:04:00,418][INFO][main][IgniteKernal%grid_recmd] Configured caches 
['ignite-marshaller-sys-cache', 'ignite-sys-cache', 'ignite-atomics-sys-cache', 
'cache_recmd']
[17:04:00,422][INFO][main][IgniteKernal%grid_recmd] 3-rd party licenses can 
be found at: /data1/home/igni1.5/libs/licenses
[17:04:00,518][INFO][main][IgniteKernal%grid_recmd] Non-loopback local IPs: 
10.0.0.1
[17:04:00,519][INFO][main][IgniteKernal%grid_recmd] Enabled local MACs: 
9017ACC1492D
[17:04:00,530][INFO][main][IgnitePluginProcessor] Configured plugins:
[17:04:00,530][INFO][main][IgnitePluginProcessor]   ^-- None
[17:04:00,530][INFO][main][IgnitePluginProcessor] 
[17:04:00,574][WARNING][main][TcpCommunicationSpi] Failure detection 
timeout will be ignored (one of SPI parameters has been set explicitly)
[17:04:00,627][INFO][main][TcpCommunicationSpi] IPC shared memory server 
endpoint started [port=48104, 
tokDir=/data1/home/igni1.5/work/ipc/shmem/e052dc07-8992-4834-87fd-abf84c0f644a-14472]
[17:04:00,628][INFO][main][TcpCommunicationSpi] Successfully bound shared 
memory communication to TCP port [port=48104, locHost=0.0.0.0/0.0.0.0]
[17:04:00,652][INFO][main][TcpCommunicationSpi] Successfully bound to TCP 
port [port=47106, locHost=0.0.0.0/0.0.0.0]
[17:04:00,685][WARNING][main][NoopCheckpointSpi] Checkpoints are disabled 
(to enable configure any GridCheckpointSpi implementation)
[17:04:00,749][WARNING][main][GridCollisionManager] Collision resolution is 
disabled (all jobs will be activated upon arrival).
[17:04:00,755][WARNING][main][NoopSwapSpaceSpi] Swap space is disabled. To 
enable use FileSwapSpaceSpi.
[17:04:00,757][INFO][main][IgniteKernal%grid_recmd] Security status 
[authentication=off, tls/ssl=off]
[17:04:00,861][WARNING][main][GridCacheProcessor] Deployment mode for cache 
is not CONTINUOUS or SHARED (it is recommended that you change deployment mode 
and restart): ISOLATED
[17:04:01,040][INFO][main][GridTcpRestProtocol] Command protocol 
successfully started [name=TCP binary, host=0.0.0.0/0.0.0.0, port=11217]

[17:04:01,138][SEVERE][tcp-client-disco-msg-worker-#4%grid_recmd%][TcpDiscoverySpi]
 Exception on joining: Connection refused
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at 
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.openSocket(TcpDiscoverySpi.java:1266)
at 
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.openSocket(TcpDiscoverySpi.java:1241)
at 
org.apache.ignite.spi.discovery.tcp.ClientImpl.sendJoinRequest(ClientImpl.java:579)
at 
org.apache.ignite.spi.discovery.tcp.ClientImpl.joinTopology(ClientImpl.java:505)
at 
org.apache.ignite.spi.discovery.tcp.ClientImpl.access$900(ClientImpl.java:118)
at 
org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.tryJoin(ClientImpl.java:1508)
at 

Re: Understanding Grid Gain Multi-tenancy

[vkulichenko]

Hi,

This is Ignite community forum. For questions about GridGain enterprise
features please contact GridGain directly:
http://go.gridgain.com/Contact_Us.html

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Understanding-Grid-Gain-Multi-tenancy-tp6968p7000.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Embedded mode ignite on spark

[percent620]

I verified this code as below

val ic = new IgniteContext[Int, Int](sc, () => new
IgniteConfiguration(),false)
val sharedRDD = ic.fromCache("sharedBaselineCacheRDD")
val initalRDD = sc.parallelize(1 to 10,10).map(i => (i, i))
println("initalRDD.couner=/. " + initalRDD.count() +"\tparition=> " +
initalRDD.partitions.size)
sharedRDD.savePairs(initalRDD)
println("=>totalcounter" + sharedRDD.count + "\t paris => " +
sharedRDD.partitions.size)
println("=>" + sharedRDD.filter(_._2 > 5).count)

but this result is not what i need, 

define shared context start
define shared context end   
[Stage 1:=>   (28 + 2) /
30][09:29:21] New version is available at ignite.apache.org: 1.7.0
initalRDD.couner=/. 10  parition=> 10   
=>totalcounter4  paris => 1024  (need to be 1)  
  
=>1 
==


Can anyone help me on this issues? Thanks!!!



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Embedded-mode-ignite-on-spark-tp6942p7001.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: LOOK THORUGH THIS ERROR

[Dmitriy Setrakyan]

Val, do we have this documented?


> On Aug 11, 2016, at 10:18 AM, vkulichenko  
> wrote:
> 
> Ravi,
> 
> You have to use the same version of ignite-hibernate as the ignite-core (the
> latest is 1.7). The ignite-hibernate module is not deployed to Maven central
> anymore due to licensing restrictions, but you can use the repo provided by
> GridGain [1]. Another option is to build this module from sources by
> yourself.
> 
> [1] www.gridgainsystems.com/nexus/content/repositories/external
> 
> -Val
> 
> 
> 
> --
> View this message in context: 
> http://apache-ignite-users.70518.x6.nabble.com/LOOK-THORUGH-THIS-ERROR-tp6977p6995.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Embedded mode ignite on spark

[percent620]

thaks ,Alisher,

Can you also give more ideas about discard datas for ignite cache? thanks!



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Embedded-mode-ignite-on-spark-tp6942p6999.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite 1.7 doesn't play along with ignite-spatial 1.2-incubating

[WhiteAncient]

That was exactly the information I needed!

Thank you very much for your time to help me out!

/J



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-1-7-doesn-t-play-along-with-ignite-spatial-1-2-incubating-tp6950p7009.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cannot get the data in Tableau

[Vladimir Ozerov]

Hi Austin,

Most probably Igor Sapego will be able to assist you. As far as I know he
is unavailable during this week, but I hope he will be able to answer you
in the beginning of the next week.

Vladimir.

On Tue, Aug 9, 2016 at 2:34 PM, austin solomon 
wrote:

> Hi,
>
> I am using Apache ignite 1.7.0-SNAPSHOT version.
>
> My ignite cluster is running in CentOS server and I'm trying to connect to
> it through ODBC driver from Windows PC.
>
> My confguration is like below:
>
> 
> 
> 
>
> 
> 
> 
> 
> 
> 
>  value="FULL_SYNC"/>
>
> 
> 
> 
> 
>  value="java.lang.Long"/>
> 
> 
> 
>  value="java.lang.Long"/>
> 
> 
> 
> 
> 
>  value="java.lang.String"/>
>  value="java.lang.String"/>
>  value="java.lang.String"/>
>  value="java.lang.Integer"/>
> 
> 
> 
> 
> 
> 
>
> 
> 
> 
> 
>  value="FULL_SYNC"/>
>
> 
> 
> 
> 
>  value="java.lang.Long"/>
>  value="Organization"/>
> 
> 
>  value="java.lang.String"/>
> 
> 
> 
> 
> 
> 
> 
> 
>
> My problem here is, I could not able to get the data present in cache that
> i have created, In Tableau i can see only the columns.
>
> When i checked the cache using ignitevisorcmd.sh i got the following
> output:
> Time of the snapshot: 08/09/16, 16:52:46
> +===
> ==+
> |  Name(@)  |Mode | Nodes | Entries (Heap / Off heap) |
> Hits|  Misses   |   Reads   |  Writes   |
> +===
> ==+
> | Organization(@c0) | PARTITIONED | 1 | min: 2 (2 / 0)|
> min: 0| min: 0| min: 0| min: 0|
> |   | |   | avg: 2.00 (2.00 / 0.00)   |
> avg: 0.00 | avg: 0.00 | avg: 0.00 | avg: 0.00 |
> |   | |   | max: 2 (2 / 0)|
> max: 0| max: 0| max: 0| max: 0|
> +---+-+---+-
> --+---+---+---+---+
> | Person(@c1)   | PARTITIONED | 1 | min: 5 (5 / 0)|
> min: 0| min: 0| min: 0| min: 0|
> |   | |   | avg: 5.00 (5.00 / 0.00)   |
> avg: 0.00 | avg: 0.00 | avg: 0.00 | avg: 0.00 |
> |   | |   | max: 5 (5 / 0)|
> max: 0| max: 0| max: 0| max: 0|
> +---
> --+
>
> I have attached the screen shot.
>
> Can you please help me what I am missing here.
>
> Thanks
> Austin
>

Re: MR jobs NOT starting in HIVE over IGFS

[Vladimir Ozerov]

Hi,

Could you please subscribe to the user list so that we see your messages not
only on the forum, but in the mailing lists as well? Instructions can be
found here: http://ignite.apache.org/community/resources.html#mail-lists

As per your problem, we need additional information. In particular:
1) Please attach full Ignite XML configuration. You provided only some
beans, but we need to get the whole picture.
2) Please attach Ignite server logs. Chances that we will see some
exceptions or warnings there.

Vladimir.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/MR-jobs-NOT-starting-in-HIVE-over-IGFS-tp6870p7006.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Ignite Compute - Rebalance Scenario

[Kamal C]

Hi all,

I'm using IgniteCompute#affinityRun(cacheName, key, runnable) to
execute the tasks where the data resides. If cache rebalance happens in the
middle of task execution, then the task is not aware of the rebalance. Is
there any provision available to interrupt the task ?

I would like to throw `ComputeJobFailOverException` if interrupted.

Regards,
Kamal C