Re: ScanQuery predicate serialization

[route99]

Indeed it seems that I was looking in older version documentation.

Thank you for the quick reply.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Gemfire replacement to Apache Ignite | Guidance Require

[Nilesh]

Hello,

We are looking for Apache Ignite for Gemfire replacement. Current we are
facing following issues in Gemfire.

1. Disk store corruption.

2. Service suddenly stopped working.

3. Service takes more time to start and stop.

4. Data recovery time is more when node down in the cluster.

5. Wan replication is manual between cluster.

6. Session replication is not available.


To overcome these current issues faced in our organization, Is Apache Ignite
correct solution?
Will appreciate any replies, suggestions. 

Current gemfire setup of our organization.

1.Gemfire version8

2.Two clusters in two regions.

3. Each cluster has 4 locators and 8 cache servers



-
Thank you,
Nilesh
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

allow

[Nilesh Patil1]

Regards,
Nilesh
CoreServices|Morningstar|Mumbai
[GPTW India Signature]

Re: How to run a job every 5 seconds in Ignite

[Evgenii Zhuravlev]

Hi Lokesh,

I'd suggest to start Ignite service, which will guarantee failover-safety
for you: https://apacheignite.readme.io/docs/service-grid. Just choose
cluster-singleton to make sure that you will have 1 instance of Service in
cluster. Inside this service you can use ignite-scheduler:
https://apacheignite.readme.io/docs/cron-based-scheduling, it has cron API.

Evgenii

пн, 27 авг. 2018 г. в 9:16, Lokesh Sharma :

> I'm using Ignite with Spring Boot. Is there a way to run a job every 5
> seconds on exactly one node of the cluster (which could be any node)?
>
> Thank You
>

Re: Gemfire replacement to Apache Ignite | Guidance Require

[Evgeniy Ignatiev]

Hello, Nilesh.

You may need looking into GridGain distribution considering point 5, as 
Apache Ignite does not offer WAN replication (cross cluster replication 
I assume from you message) out of the box.


https://docs.gridgain.com/docs/data-center-replication

Best regards,
Evgeniy Ignatiev.


On 8/27/2018 1:26 PM, Nilesh wrote:

Hello,

We are looking for Apache Ignite for Gemfire replacement. Current we are
facing following issues in Gemfire.

1. Disk store corruption.

2. Service suddenly stopped working.

3. Service takes more time to start and stop.

4. Data recovery time is more when node down in the cluster.

5. Wan replication is manual between cluster.

6. Session replication is not available.


To overcome these current issues faced in our organization, Is Apache Ignite
correct solution?
Will appreciate any replies, suggestions.

Current gemfire setup of our organization.

1.Gemfire version8

2.Two clusters in two regions.

3. Each cluster has 4 locators and 8 cache servers



-
Thank you,
Nilesh
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Gemfire replacement to Apache Ignite | Guidance Require

[Evgeniy Ignatiev]

Sorry disregard my previous message, misread you points.


On 8/27/2018 1:26 PM, Nilesh wrote:

Hello,

We are looking for Apache Ignite for Gemfire replacement. Current we are
facing following issues in Gemfire.

1. Disk store corruption.

2. Service suddenly stopped working.

3. Service takes more time to start and stop.

4. Data recovery time is more when node down in the cluster.

5. Wan replication is manual between cluster.

6. Session replication is not available.


To overcome these current issues faced in our organization, Is Apache Ignite
correct solution?
Will appreciate any replies, suggestions.

Current gemfire setup of our organization.

1.Gemfire version8

2.Two clusters in two regions.

3. Each cluster has 4 locators and 8 cache servers



-
Thank you,
Nilesh
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Cross plat form put/get

[Igor Sapego]

GetHashCode() method is not used for hashing in Ignite C++ any
more, as well as hashCode() method in Java. You can still find it
in legacy code, but since Ignite 2.1 this method is not used and
you can safely remove it.

By default, hashing algorithm in Ignite is now operates on byte-array
which is obtained when object is serialized in the same way for all
platforms. So now you should not care much about it, as long as your
serialization/deserialization algorithms are implemented the same way
on all platforms.

Best Regards,
Igor


On Sat, Aug 25, 2018 at 1:18 AM wengyao04  wrote:

> Hi, we have c++ ignite client and java client to read/write to the same
> cache.
> Our key is a composite object, my cache is  IgniteCache
>
> ---c++ ---
> namespace ignite
> {
> namespace binary
> {
> struct Key
> {
> std::string id;
> bool flag;
> Key()
> : id() , flag(false)
> {
> }
>
> Key(const std::string& id, bool flag = false)
> : id(id)
> , flag(flag)
> {
> }
>
> Key(const  Key& rhs)
> : id(rhs.id)
> , flag(rhs.flag)
> {
> }
>
> bool operator<(const Key& rhs) const
> {
> return (id == rhs.id) ?
>   flag < rhs.flag : id < rhs.id;
> }
>
> bool operator==(const Key& rhs) const
> {
> return id == rhs.id && flag == rhs.flag;
> }
>
> bool operator!=(const Key& rhs) const
> {
> return !(*this == rhs);
> }
>
> ~Key()
> {
> }
>
> // METHODS
> bool isEmpty() const { return id.empty(); }
> };
> template<>
> struct BinaryType
> {
> static int32_t GetTypeId()
> {
> return GetBinaryStringHashCode("Key");
> }
>
> static void GetTypeName(native_std::string& dst)
> {
> dst = "Key";
> }
>
> static int32_t GetFieldId(const char* name)
> {
> return GetBinaryStringHashCode(name);
> }
>
> static bool IsNull(const Key& key)
> {
> return und.isEmpty();
> }
>
> static void GetNull(Key& key)
> {
> key = Key();
> }
>
> static void Write(BinaryWriter& writer, Key& key)
> {
> writer.WriteString("id", key.id);
> writer.WriteBool("flag", key.flag);
> }
>
> static void Read(BinaryReader& reader, Key& key)
> {
> key.id = reader.ReadString("id");
> key.flag = reader.ReadBool("flag");
> }
>
> int32_t GetHashCode(const Key& key) const
> {
> const int32_t prime = 31;
> int32_t boolHashCode = key.is_delayed ? 1231 : 1237;
> int32_t strHashCode = 0;
> for (int i = 0; i < (int)key.bbgid.size(); ++i)
> {
> strHashCode = prime * strHashCode + key.bbgid[i];
> }
> return strHashCode + prime * boolHashCode;
> }
> };
> }
> }
>
> Our java key is
> ---
> mport org.apache.ignite.binary.BinaryObjectException;
> import org.apache.ignite.binary.BinaryReader;
> import org.apache.ignite.binary.BinaryWriter;
> import org.apache.ignite.binary.Binarylizable;
>
> public class Key implements Binarylizable {
>
> public String id;
> public Boolean flag;
> private final static int prime = 31;
>
> public Key() {
> //no-op
> }
>
> public Key(String id) {
> this.id = id;
> this.flag = false;
> }
>
> public Key(String id, boolean flag) {
> this.id = id;
> this.flag = flag;
> }
>
> @Override
> public int hashCode() {
>
> int boolHashCode = flag ? 1231 : 1237;
> int strHashCode = 0;
>
> for (int i = 0; i < id.length(); i++) {
> strHashCode = prime * strHashCode + id.charAt(i);
> }
>
> return strHashCode + prime * boolHashCode;
> }
>
> @Override
> public boolean equals(Object o) {
> if (o == this) {
> return true;
> }
>
> if (!(o instanceof Key)) {
> return false;
> }
>
> Key cast = (Key) o;
> return this.id.equals(cast.id) && this.flag == cast.flag;
> }
>
> public void readBinary(BinaryReader reader) throws
> BinaryObjectException
> {
> id = reader.readString("id");
> flag = reader.readBoolean("flag");
> }
>
> public void writeBinary(BinaryWriter writer) throws
> BinaryObjectException {
> writer.writeString("id", id);
> writer.writeBoolean("flag", flag);
> }
>
> @Override public String toString() {
> return "Key = [" + id + " " + flag +  "]\n";
> }
> }
>
> I see people put the  GetHashCode in c++ BinaryType, some one use static,
> and others just put it as a const method. This GetHashCode implements hash
> code as java class Key. Where this GetHashCode is used ? Say if I write a
> record from c++ client, is this key the cache hash key ? Do I also need to
> implement hash in c++ class Key ? Is this GetHhashCode needed to

Re: ignte cluster hang with GridCachePartitionExchangeManager

[Ilya Kasnacheev]

Hello!

1. As far as my understanding goes, there's no such handling of OOM in
Apache Ignite that would guarantee not causing cluster crash. This means
you should be extra careful with that. This is since after OOM node doesn't
have a chance to quit gracefully. Maybe other nodes will be able to
eventually remove it, maybe not.

2. It's hard to say what happens here. Can you provide logs?

Regards,

-- 
Ilya Kasnacheev


пт, 24 авг. 2018 г. в 19:49, wangsan :

> Now my cluster topology is Node a,b,c,d  all with persistence enable and
> peerclassloader false. b c d have different class(cache b) from a.
> 1.When any node crash with oom(memory or stack) .all nodes hang with " -
> Still waiting for initial partition map exchange "
> 2.When a start first,  b,c,d start in multi threads concurrent.b,c,d hang
> with " - Still waiting for initial partition map exchange ".a hang with
> "Unable to await partitions release latch"
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Example for server client configuration in ignite

[Sriveena Mattaparthi]

Hi,

What is the recommended configuration for remote Server and one client node 
setup?
Can someone point to the relevant examples?

Thanks & Regards,
Sriveena

Re: Example for server client configuration in ignite

[Evgenii Zhuravlev]

Hi,

The default configuration will work with the remote server, because it uses
multicast. It will work if you want to check some examples, etc.

For production usage, I'd recommend using static Ip Finder instead of
multicast, here is the simplest configuration that will work with remote
server:
https://apacheignite.readme.io/docs/tcpip-discovery#section-static-ip-finder
There you will need just to replace address in IpFinder with the address of
your remote server. You don't need to place client address there.

Yo start client instead of the server node, you have to add clientMode=true
property in this configuration.

Evgenii


пн, 27 авг. 2018 г. в 13:47, Sriveena Mattaparthi <
sriveena.mattapar...@ekaplus.com>:

> Hi,
>
>
>
> What is the recommended configuration for remote Server and one client
> node setup?
>
> Can someone point to the relevant examples?
>
>
>
> Thanks & Regards,
>
> Sriveena
>
>
>

Re: Cache Configuration Templates

[Ilya Kasnacheev]

Hello!

Unfortunately, cache templates are not documented that good. AFAIK they
were mostly implemented to be able to reference to complex cache
configurations with CREATE TABLE.

As far as my understanding goes, caches from cacheConfigurations are
actually started when grid starts.

1) I think that only the first one will be used to create a cache. Even if
you join a node with distinct cacheConfigurations to the cluster, and it
already has some caches started, those will just be re-used (by name).
2) Yes, you can have a cacheConfiguration with name "prefix*", which will
lead to a) not starting this cache on grid start, and b) when you start a
cache "prefix1" it will use configuration from template. There's a test for
it named IgniteCacheConfigurationTemplateTest in code base.
3) Nothing will happen, it will return early if such cache already exists.
4) Yes.
5) Good question, I will have to check that. Still I won't rely on that and
just always have this configuration around.
6) See above about the '*'.

Regards,
-- 
Ilya Kasnacheev


сб, 25 авг. 2018 г. в 0:55, Dave Harvey :

> I found what I've read in this area confusing, and here is my current
> understanding.
>
> When creating an IgniteConfiguration in Java or XML, I can specify the
> property cacheConfiguration, which is an array of CacheConfigurations.
> This causes Ignite to preserve these configurations, but this will not
> cause Ignite to create a cache. If I call
> Ignite.getOrCreateCache(string), if there is an existing cache, I will get
> that, otherwise a new cache will be created using that configuration.
>
> It seems like creating a cache with a configuration will add to this list,
> because Ignite.configuration.getCacheConfiguration() returns all caches.
>
>
> I can later call Ignite.addCacheConfiguration(). This will add a template
> to that list.
>
> Questions:
> 1)  what happens if there are entries with duplicate names on
> IgniteConfiguration.setCacheConfiguration()   when this is used to create a
> grid?
> 2) There was mention in one e-mail talking about a convention where
> templates have "*" in their name?
> 3) What happens if addCacheConfiguration() tries to add a duplicate name?
> 4) Is a template simply a cache that not fully instantiated?
> 5) What about template persistence?   Are they persisted if they specify a
> region that is persistent?
> 6) My use case is that I want to create caches based some default for the
> cluster, so in Java I would like to construct the new configuration from
> the a template of a known name.   So far, I can only see that I can call
> Ignite.configuration.getCacheConfiguration() and then search the array for
> a matching name.   Is there a better way?
>
>
> *Disclaimer*
>
> The information contained in this communication from the sender is
> confidential. It is intended solely for use by the recipient and others
> authorized to receive it. If you are not the recipient, you are hereby
> notified that any disclosure, copying, distribution or taking action in
> relation of the contents of this information is strictly prohibited and may
> be unlawful.
>
> This email has been scanned for viruses and malware, and may have been
> automatically archived by *Mimecast Ltd*, an innovator in Software as a
> Service (SaaS) for business. Providing a *safer* and *more useful* place
> for your human generated data. Specializing in; Security, archiving and
> compliance. To find out more Click Here
> .
>

RE: Example for server client configuration in ignite

[Sriveena Mattaparthi]

Thanks Evgenii

Started remote server, using the default configuration given as part of the 
ignite disctribution using
$ bin\ignite.bat examples\config\example-ignite.xml

And client node started in java  as pointed below using

Ignition.setClientMode(true);

TcpDiscoverySpi spi = new TcpDiscoverySpi();
TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
ipFinder.setAddresses(Arrays.asList("1.2.3.4", "1.2.3.5:47500..47509"));
spi.setIpFinder(ipFinder);
IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setDiscoverySpi(spi);
// Start Ignite node.
Ignition.start(cfg);

My question is
1.Do we have to start Ignite again on client node as Ignition.start(cfg)? What 
does this imply?

2.Will the client node uses the JVM heap memory

Thanks & Regards,
Sriveena

From: Evgenii Zhuravlev [mailto:e.zhuravlev...@gmail.com]
Sent: Monday, August 27, 2018 4:52 PM
To: user@ignite.apache.org
Subject: Re: Example for server client configuration in ignite

Hi,

The default configuration will work with the remote server, because it uses 
multicast. It will work if you want to check some examples, etc.

For production usage, I'd recommend using static Ip Finder instead of 
multicast, here is the simplest configuration that will work with remote server:
https://apacheignite.readme.io/docs/tcpip-discovery#section-static-ip-finder
There you will need just to replace address in IpFinder with the address of 
your remote server. You don't need to place client address there.

Yo start client instead of the server node, you have to add clientMode=true 
property in this configuration.

Evgenii


пн, 27 авг. 2018 г. в 13:47, Sriveena Mattaparthi 
mailto:sriveena.mattapar...@ekaplus.com>>:
Hi,

What is the recommended configuration for remote Server and one client node 
setup?
Can someone point to the relevant examples?

Thanks & Regards,
Sriveena

Re: Cross plat form put/get

[wengyao04]

Thank you very much, I will remove GetHashCode and hashCode in c++ and java
client code and tried it again



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Network Segmentation

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: network segmentation

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Cluster segmentation

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Segmentation policy configuration

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: RESTART_JVM Segmentation Policy

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Segmentation policy configuration

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Segmentation policy.

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Network Segmentation configuarion

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite node not stopping after segmentation

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Easy way to determine a node has been segmented

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite Node failure - Node out of topology (SEGMENTED)

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Stopping local node according to configured segmentation policy.

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Stopping local node according to configured segmentation policy.

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Local node SEGMENTED. Data Lost on a Partitioned Cache with backups of 3

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Segmentation fault (JVM crash) while memory restoring on start with native persistance

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Behaviour of Apache Ignite instances in case of network isolation

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Two Ignite Clusters formed after network disturbance

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Local node failure starts cluster-wide procedure

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Cluster breaks down on network failure

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite cluster

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Local node seems to be disconnected from topology (failure detection timeout is reached)

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: split-brain problem and GridSegmentationProcessor

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Node fails to recover

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Timed out waiting for message delivery receipt

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: How many nodes within your ignite cluster

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: How to overcome short-time network problems?

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite cluster recovery after network partition

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: how to automatically exit JVM when the node stops?

[luqmanahmad]

See [1] for free network segmentation plugin

[1]  https://github.com/luqmanahmad/ignite-plugins
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

C++ affinity key set up

[wengyao04]

Hi, we tried to set id in Key as our affinity key. Our c++ Key is defined 
namespace ignite 
{ 
namespace binary 
{ 
struct Key 
{ 
std::string id; 
bool flag; 
Key() 
: id() , flag(false) 
{ 
} 

Key(const std::string& id, bool flag = false) 
: id(id) 
, flag(flag) 
{ 
} 

Key(const  Key& rhs) 
: id(rhs.id) 
, flag(rhs.flag) 
{ 
} 

bool operator<(const Key& rhs) const 
{ 
return (id == rhs.id) ? 
  flag < rhs.flag : id < rhs.id; 
} 

bool operator==(const Key& rhs) const 
{ 
return id == rhs.id && flag == rhs.flag; 
} 

bool operator!=(const Key& rhs) const 
{ 
return !(*this == rhs); 
} 

~Key() 
{ 
} 

// METHODS 
bool isEmpty() const { return id.empty(); } 
}; 
template<> 
struct BinaryType 
{ 
static int32_t GetTypeId() 
{ 
return GetBinaryStringHashCode("Key"); 
} 

static void GetTypeName(native_std::string& dst) 
{ 
dst = "Key"; 
} 

static int32_t GetFieldId(const char* name) 
{ 
return GetBinaryStringHashCode(name); 
} 

static bool IsNull(const Key& key) 
{ 
return und.isEmpty(); 
} 

static void GetNull(Key& key) 
{ 
key = Key(); 
} 

static void Write(BinaryWriter& writer, Key& key) 
{ 
writer.WriteString("id", key.id); 
writer.WriteBool("flag", key.flag); 
} 

static void Read(BinaryReader& reader, Key& key) 
{ 
key.id = reader.ReadString("id"); 
key.flag = reader.ReadBool("flag"); 
} 
}; 
} 
} 

The cache key configuration is

  


  


However, after we populated data we find some keys with same ids but
different flag does not reside on the same cache node. Do I missed anything
is the configuration ? Thanks




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Example for server client configuration in ignite

[Evgenii Zhuravlev]

>1.Do we have to start Ignite again on client node as Ignition.start(cfg)?
What does this imply?
Sorry, I didn't get it, what do you mean by "start again"?
>2.Will the client node uses the JVM heap memory
Ignite runs inside JVM, so yes, it will use JVM heap. Could you specify
your question?

Evgenii

пн, 27 авг. 2018 г. в 14:36, Sriveena Mattaparthi <
sriveena.mattapar...@ekaplus.com>:

> Thanks Evgenii
>
>
>
> Started remote server, using the default configuration given as part of
> the ignite disctribution using
>
> $ bin\ignite.bat examples\config\example-ignite.xml
>
>
>
> And client node started in java  as pointed below using
>
>
>
> Ignition.*setClientMode*(*true*);
>
>
>
> TcpDiscoverySpi spi = new TcpDiscoverySpi();
>
> TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
>
> ipFinder.setAddresses(Arrays.asList("1.2.3.4", "1.2.3.5:47500..47509"));
>
> spi.setIpFinder(ipFinder);
>
> IgniteConfiguration cfg = new IgniteConfiguration();
>
> cfg.setDiscoverySpi(spi);
>
> // Start Ignite node.
>
> Ignition.start(cfg);
>
>
>
> My question is
>
> 1.Do we have to start Ignite again on client node as Ignition.start(cfg)?
> What does this imply?
>
>
>
> 2.Will the client node uses the JVM heap memory
>
>
>
> Thanks & Regards,
>
> Sriveena
>
>
>
> *From:* Evgenii Zhuravlev [mailto:e.zhuravlev...@gmail.com]
> *Sent:* Monday, August 27, 2018 4:52 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: Example for server client configuration in ignite
>
>
>
> Hi,
>
>
>
> The default configuration will work with the remote server, because it
> uses multicast. It will work if you want to check some examples, etc.
>
>
>
> For production usage, I'd recommend using static Ip Finder instead of
> multicast, here is the simplest configuration that will work with remote
> server:
>
>
> https://apacheignite.readme.io/docs/tcpip-discovery#section-static-ip-finder
> 
>
> There you will need just to replace address in IpFinder with the
> address of your remote server. You don't need to place client address there.
>
>
>
> Yo start client instead of the server node, you have to add
> clientMode=true property in this configuration.
>
>
>
> Evgenii
>
>
>
>
>
> пн, 27 авг. 2018 г. в 13:47, Sriveena Mattaparthi <
> sriveena.mattapar...@ekaplus.com>:
>
> Hi,
>
>
>
> What is the recommended configuration for remote Server and one client
> node setup?
>
> Can someone point to the relevant examples?
>
>
>
> Thanks & Regards,
>
> Sriveena
>
>
>
>

Re: apache ignite cassandra persistentStore for enum fields

[michal23849]

Hi Ilya,

Following up to ENUMs - what should I put then as the JavaFieldType? 

I tried java.lang.Enum, java.lang.Integer and it is failing. What value
should be specified for PojoStore to correctly map the Enum to
DatabaseFieldName and Type?

Thank You,
Michal



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: apache ignite cassandra persistentStore for enum fields

[Ilya Kasnacheev]

Have you tried specifying the actual enum type? e.g. com.corp.my.Enum

Regards,
-- 
Ilya Kasnacheev


пн, 27 авг. 2018 г. в 17:41, michal23849 :

> Hi Ilya,
>
> Following up to ENUMs - what should I put then as the JavaFieldType?
>
> I tried java.lang.Enum, java.lang.Integer and it is failing. What value
> should be specified for PojoStore to correctly map the Enum to
> DatabaseFieldName and Type?
>
> Thank You,
> Michal
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Example for server client configuration in ignite

[Sriveena Mattaparthi]

Thanks Evgenii..

To reframe my question ..server is started on a separate host...client node is 
setup using java connecting to remote server as mentioned in the  below mail...

Does invoking ignition.start on client node means starting ignite once more?

Thanks & Regards,
Sriveena

From: Evgenii Zhuravlev 
Sent: 27 August 2018 19:54:16
To: user@ignite.apache.org
Subject: Re: Example for server client configuration in ignite


>1.Do we have to start Ignite again on client node as Ignition.start(cfg)? What 
>does this imply?
Sorry, I didn't get it, what do you mean by "start again"?
>2.Will the client node uses the JVM heap memory
Ignite runs inside JVM, so yes, it will use JVM heap. Could you specify your 
question?

Evgenii

пн, 27 авг. 2018 г. в 14:36, Sriveena Mattaparthi 
mailto:sriveena.mattapar...@ekaplus.com>>:

Thanks Evgenii



Started remote server, using the default configuration given as part of the 
ignite disctribution using

$ bin\ignite.bat examples\config\example-ignite.xml



And client node started in java  as pointed below using



Ignition.setClientMode(true);



TcpDiscoverySpi spi = new TcpDiscoverySpi();

TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();

ipFinder.setAddresses(Arrays.asList("1.2.3.4", "1.2.3.5:47500..47509"));

spi.setIpFinder(ipFinder);

IgniteConfiguration cfg = new IgniteConfiguration();

cfg.setDiscoverySpi(spi);

// Start Ignite node.

Ignition.start(cfg);



My question is

1.Do we have to start Ignite again on client node as Ignition.start(cfg)? What 
does this imply?



2.Will the client node uses the JVM heap memory



Thanks & Regards,

Sriveena



From: Evgenii Zhuravlev 
[mailto:e.zhuravlev...@gmail.com]
Sent: Monday, August 27, 2018 4:52 PM
To: user@ignite.apache.org
Subject: Re: Example for server client configuration in ignite



Hi,



The default configuration will work with the remote server, because it uses 
multicast. It will work if you want to check some examples, etc.



For production usage, I'd recommend using static Ip Finder instead of 
multicast, here is the simplest configuration that will work with remote server:

https://apacheignite.readme.io/docs/tcpip-discovery#section-static-ip-finder

There you will need just to replace address in IpFinder with the address of 
your remote server. You don't need to place client address there.



Yo start client instead of the server node, you have to add clientMode=true 
property in this configuration.



Evgenii





пн, 27 авг. 2018 г. в 13:47, Sriveena Mattaparthi 
mailto:sriveena.mattapar...@ekaplus.com>>:

Hi,



What is the recommended configuration for remote Server and one client node 
setup?

Can someone point to the relevant examples?



Thanks & Regards,

Sriveena



"Confidentiality Notice: The contents of this email message and any attachments 
are intended solely for the addressee(s) and may contain confidential and/or 
privileged information and may be legally protected from disclosure. If you are 
not the intended recipient of this message or their agent, or if this message 
has been addressed to you in error, please immediately alert the sender by 
reply email and then delete this message and any attachments. If you are not 
the intended recipient, you are hereby notified that any use, dissemination, 
copying, or storage of this message or its attachments is strictly prohibited."

Re: Example for server client configuration in ignite

[Evgenii Zhuravlev]

Yes, you start client node, not a server. I'd recommend you to read about
differences between client and server nodes:
https://apacheignite.readme.io/docs/clients-vs-servers

2018-08-27 19:06 GMT+03:00 Sriveena Mattaparthi <
sriveena.mattapar...@ekaplus.com>:

> Thanks Evgenii..
>
> To reframe my question ..server is started on a separate host...client
> node is setup using java connecting to remote server as mentioned in the
> below mail...
>
> Does invoking ignition.start on client node means starting ignite once
> more?
>
> Thanks & Regards,
> Sriveena
> --
> *From:* Evgenii Zhuravlev 
> *Sent:* 27 August 2018 19:54:16
>
> *To:* user@ignite.apache.org
> *Subject:* Re: Example for server client configuration in ignite
>
>
> >1.Do we have to start Ignite again on client node as Ignition.start(cfg)?
> What does this imply?
> Sorry, I didn't get it, what do you mean by "start again"?
> >2.Will the client node uses the JVM heap memory
> Ignite runs inside JVM, so yes, it will use JVM heap. Could you specify
> your question?
>
> Evgenii
>
> пн, 27 авг. 2018 г. в 14:36, Sriveena Mattaparthi <
> sriveena.mattapar...@ekaplus.com>:
>
> Thanks Evgenii
>
>
>
> Started remote server, using the default configuration given as part of
> the ignite disctribution using
>
> $ bin\ignite.bat examples\config\example-ignite.xml
>
>
>
> And client node started in java  as pointed below using
>
>
>
> Ignition.*setClientMode*(*true*);
>
>
>
> TcpDiscoverySpi spi = new TcpDiscoverySpi();
>
> TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
>
> ipFinder.setAddresses(Arrays.asList("1.2.3.4", "1.2.3.5:47500..47509"));
>
> spi.setIpFinder(ipFinder);
>
> IgniteConfiguration cfg = new IgniteConfiguration();
>
> cfg.setDiscoverySpi(spi);
>
> // Start Ignite node.
>
> Ignition.start(cfg);
>
>
>
> My question is
>
> 1.Do we have to start Ignite again on client node as Ignition.start(cfg)?
> What does this imply?
>
>
>
> 2.Will the client node uses the JVM heap memory
>
>
>
> Thanks & Regards,
>
> Sriveena
>
>
>
> *From:* Evgenii Zhuravlev [mailto:e.zhuravlev...@gmail.com]
> *Sent:* Monday, August 27, 2018 4:52 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: Example for server client configuration in ignite
>
>
>
> Hi,
>
>
>
> The default configuration will work with the remote server, because it
> uses multicast. It will work if you want to check some examples, etc.
>
>
>
> For production usage, I'd recommend using static Ip Finder instead of
> multicast, here is the simplest configuration that will work with remote
> server:
>
> https://apacheignite.readme.io/docs/tcpip-discovery#
> section-static-ip-finder
> 
>
> There you will need just to replace address in IpFinder with the
> address of your remote server. You don't need to place client address there.
>
>
>
> Yo start client instead of the server node, you have to add
> clientMode=true property in this configuration.
>
>
>
> Evgenii
>
>
>
>
>
> пн, 27 авг. 2018 г. в 13:47, Sriveena Mattaparthi <
> sriveena.mattapar...@ekaplus.com>:
>
> Hi,
>
>
>
> What is the recommended configuration for remote Server and one client
> node setup?
>
> Can someone point to the relevant examples?
>
>
>
> Thanks & Regards,
>
> Sriveena
>
>
>
> “Confidentiality Notice: The contents of this email message and any
> attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
>

Reaching Ignite's maximum throughput

[Kia Rahmani]

Hey all,

I have been playing with Ignite and studying how applications can be
correctly optimized for it for a while now. However, I am having trouble
witnessing *reasonable* transactional throughputs even in the baseline
setting and I am wondering maybe I'm doing something fundamentally wrong
(e.g. my current clients or workers' tasks are being serialized somewhere)

For example, in my experiments (code attached) on a t2.2xlarge EC2 instance
with 8 vCPU, consisting of a very simple read/write (counter increment)
transaction (SERIALIZABLE + PESSIMISTIC), I fail to achieve a throughput
above ~600 txns/sec with 16 concurrent Java threads (all connecting to the
same Ignite client node running on the same machine)

I would really appreciate it if you could help me understand the followings: 
Q1: Is the above number reasonable for Ignite transactional layer's maximum
throughput? and if not,
Q2: What am I doing wrong? 

Code Snippets:
https://gist.github.com/Kiarahmani/2a49a0c0a512c99bfd65c69cce257aab



Thanks a lot in advance!
Kia







--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Reaching Ignite's maximum throughput

[Ilya Kasnacheev]

Hello!

I imagine that serializable + pessimistic is not a very practical setting
from performance standpoint.

You want maximum guarantees there, so you will get the least performance,
and having a distributed system would not help you much in this case, if
not hinder.

Regards,
-- 
Ilya Kasnacheev


пн, 27 авг. 2018 г. в 19:24, Kia Rahmani :

> Hey all,
>
> I have been playing with Ignite and studying how applications can be
> correctly optimized for it for a while now. However, I am having trouble
> witnessing *reasonable* transactional throughputs even in the baseline
> setting and I am wondering maybe I'm doing something fundamentally wrong
> (e.g. my current clients or workers' tasks are being serialized somewhere)
>
> For example, in my experiments (code attached) on a t2.2xlarge EC2 instance
> with 8 vCPU, consisting of a very simple read/write (counter increment)
> transaction (SERIALIZABLE + PESSIMISTIC), I fail to achieve a throughput
> above ~600 txns/sec with 16 concurrent Java threads (all connecting to the
> same Ignite client node running on the same machine)
>
> I would really appreciate it if you could help me understand the
> followings:
> Q1: Is the above number reasonable for Ignite transactional layer's maximum
> throughput? and if not,
> Q2: What am I doing wrong?
>
> Code Snippets:
> https://gist.github.com/Kiarahmani/2a49a0c0a512c99bfd65c69cce257aab
>
>
>
> Thanks a lot in advance!
> Kia
>
>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Example for server client configuration in ignite

[Sriveena Mattaparthi]

Thanks Evgenii..

Now the question is code for client ignite start is deployed in an application 
server.
Any data join performed on the client node cache is utilizing jvm heap memory 
assigned to the  application resulting in outofmemory on the application server.

Is there a way to limit the client node memory utilization of application 
server heap memory or run client node on separate ibn?

Please note that persistence is enabled on the ignite configuration.


From: Evgenii Zhuravlev 
Sent: 27 August 2018 21:41:41
To: user@ignite.apache.org
Subject: Re: Example for server client configuration in ignite

Yes, you start client node, not a server. I'd recommend you to read about 
differences between client and server nodes: 
https://apacheignite.readme.io/docs/clients-vs-servers

2018-08-27 19:06 GMT+03:00 Sriveena Mattaparthi 
mailto:sriveena.mattapar...@ekaplus.com>>:
Thanks Evgenii..

To reframe my question ..server is started on a separate host...client node is 
setup using java connecting to remote server as mentioned in the  below mail...

Does invoking ignition.start on client node means starting ignite once more?

Thanks & Regards,
Sriveena

From: Evgenii Zhuravlev 
mailto:e.zhuravlev...@gmail.com>>
Sent: 27 August 2018 19:54:16

To: user@ignite.apache.org
Subject: Re: Example for server client configuration in ignite


>1.Do we have to start Ignite again on client node as Ignition.start(cfg)? What 
>does this imply?
Sorry, I didn't get it, what do you mean by "start again"?
>2.Will the client node uses the JVM heap memory
Ignite runs inside JVM, so yes, it will use JVM heap. Could you specify your 
question?

Evgenii

пн, 27 авг. 2018 г. в 14:36, Sriveena Mattaparthi 
mailto:sriveena.mattapar...@ekaplus.com>>:

Thanks Evgenii



Started remote server, using the default configuration given as part of the 
ignite disctribution using

$ bin\ignite.bat examples\config\example-ignite.xml



And client node started in java  as pointed below using



Ignition.setClientMode(true);



TcpDiscoverySpi spi = new TcpDiscoverySpi();

TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();

ipFinder.setAddresses(Arrays.asList("1.2.3.4", "1.2.3.5:47500..47509"));

spi.setIpFinder(ipFinder);

IgniteConfiguration cfg = new IgniteConfiguration();

cfg.setDiscoverySpi(spi);

// Start Ignite node.

Ignition.start(cfg);



My question is

1.Do we have to start Ignite again on client node as Ignition.start(cfg)? What 
does this imply?



2.Will the client node uses the JVM heap memory



Thanks & Regards,

Sriveena



From: Evgenii Zhuravlev 
[mailto:e.zhuravlev...@gmail.com]
Sent: Monday, August 27, 2018 4:52 PM
To: user@ignite.apache.org
Subject: Re: Example for server client configuration in ignite



Hi,



The default configuration will work with the remote server, because it uses 
multicast. It will work if you want to check some examples, etc.



For production usage, I'd recommend using static Ip Finder instead of 
multicast, here is the simplest configuration that will work with remote server:

https://apacheignite.readme.io/docs/tcpip-discovery#section-static-ip-finder

There you will need just to replace address in IpFinder with the address of 
your remote server. You don't need to place client address there.



Yo start client instead of the server node, you have to add clientMode=true 
property in this configuration.



Evgenii





пн, 27 авг. 2018 г. в 13:47, Sriveena Mattaparthi 
mailto:sriveena.mattapar...@ekaplus.com>>:

Hi,



What is the recommended configuration for remote Server and one client node 
setup?

Can someone point to the relevant examples?



Thanks & Regards,

Sriveena



“Confidentiality Notice: The contents of this email message and any attachments 
are intended solely for the addressee(s) and may contain confidential and/or 
privileged information and may be legally protected from disclosure. If you are 
not the intended recipient of this message or their agent, or if this message 
has been addressed to you in error, please immediately alert the sender by 
reply email and then delete this message and any attachments. If y

Re: Reaching Ignite's maximum throughput

[Kia Rahmani]

Thanks, Ilya for your message. 
I am aware of the cost of enforcing Serializability. What I am more
concerned about is if I am achieving Ignite's best or not? Specifically,
since I am not performing any replication and I have only one server right
now, I should be getting a performance somewhat close a centralized database
which I assume should be much much much more than 600 txns/sec 





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Affinity key in SQL execution

[vkulichenko]

Hi Prasad,

1. First plan is for map phase (executed on server side) and the second one
is for reduce phase (executed on client side). Merge scan means that it just
merges result sets from all participating server nodes. Sometime it can
contain additional reduce steps like final groupings, ordering, etc.

2. This sounds like index is not actually declared correctly and doesn't
exist. Most likely this is reason why Ignite doesn't use it automatically as
well. Please show your full indexing configuration.

3. Of course you can, it can participate in any indexes as any other field.
Individual index for affinity key field is created automatically.

4. Unfortunately, I don't think there is a way to check this except
debugging the code.

5. That depends on what is the foreign key. As a matter of fact, I'm a bit
confused by the query. Looks like the affinity key is not the one you join
on. That means that join is not collocated which is likely to cause
performance issues and/or incorrect results.

6. Can you give a particular query example?

-Val



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Affinity key in SQL execution

[Prasad Bhalerao]

Hi val

1. If the second execution plan is for reduce phase, should it come in
queries which includes affinity key in their where clause? Because there is
going to be no reduce phase in this case.

2. Is it mandatory to anotate affinity id field in my key class with
@QueryField?
Without this annotation I was getting error something like field not found?

3. E.g. if my all cache's data is colacated on same affinity key i.e.
subscriptionId.

Select * from cache1 c1 join cache2 c2 on ( c1.userId = c2.userId)
Where c1.affinityId = ? And c2.affinityId = ?

In above example, to execute the SQL only on single data node is it enough
to write only one condition i.e. "c1.affinityId = ?".

Or Do I have to write affinityId in where clause for every cache involved
in sql like "cach2.affinityId =?"

4. If the ignite automatically creates index on affinity key field then why
it is not showing in execution plan?

Thanks,
Prasad

On Tue, Aug 28, 2018, 1:48 AM vkulichenko 
wrote:

> Hi Prasad,
>
> 1. First plan is for map phase (executed on server side) and the second one
> is for reduce phase (executed on client side). Merge scan means that it
> just
> merges result sets from all participating server nodes. Sometime it can
> contain additional reduce steps like final groupings, ordering, etc.
>
> 2. This sounds like index is not actually declared correctly and doesn't
> exist. Most likely this is reason why Ignite doesn't use it automatically
> as
> well. Please show your full indexing configuration.
>
> 3. Of course you can, it can participate in any indexes as any other field.
> Individual index for affinity key field is created automatically.
>
> 4. Unfortunately, I don't think there is a way to check this except
> debugging the code.
>
> 5. That depends on what is the foreign key. As a matter of fact, I'm a bit
> confused by the query. Looks like the affinity key is not the one you join
> on. That means that join is not collocated which is likely to cause
> performance issues and/or incorrect results.
>
> 6. Can you give a particular query example?
>
> -Val
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: ignte cluster hang with GridCachePartitionExchangeManager

[wangsan wang]

About question 2, debug level like this：
I start a node，then b,c,d,e,f nodes in mulitithread. then close them all.
in the debugs log, A server latch created with participantsSize=5 but only
one countdown .then latch will be hang.

simple logs is:

>>> ++
> >>> Topology snapshot.
> >>> ++
> >>> Ignite instance name: test01
> >>> Number of server nodes: 5
> >>> Number of client nodes: 0
> >>> Topology version: 71
> >>> Local: 016F5C35-AC7D-4391-9142-1A7AEA1C3378, [
> 192.168.10.103/0:0:0:0:0:0:0:1, /127.0.0.1, /192.168.10.103], 21, Mac OS
> X x86_64 10.11.6, wangqingpeng, Java(TM) SE Runtime Environment
> 1.8.0_131-b11
> >>> Remote: 58973B59-DB04-4325-9966-D259A7AE3BBD, [
> 192.168.10.103/0:0:0:0:0:0:0:1, /127.0.0.1, /192.168.10.103], 66, Mac OS
> X x86_64 10.11.6, wangqingpeng, Java(TM) SE Runtime Environment
> 1.8.0_131-b11
> >>> Remote: A79A57F0-D3A4-4ECA-890B-0A043BB0D23F, [
> 192.168.10.103/0:0:0:0:0:0:0:1, /127.0.0.1, /192.168.10.103], 67, Mac OS
> X x86_64 10.11.6, wangqingpeng, Java(TM) SE Runtime Environment
> 1.8.0_131-b11
> >>> Remote: C261B07C-9495-4058-889B-BD484BE10477, [
> 192.168.10.103/0:0:0:0:0:0:0:1, /127.0.0.1, /192.168.10.103], 68, Mac OS
> X x86_64 10.11.6, wangqingpeng, Java(TM) SE Runtime Environment
> 1.8.0_131-b11
> >>> Remote: 230E516F-6C12-4391-B902-822AFC6F7BC4, [
> 192.168.10.103/0:0:0:0:0:0:0:1, /127.0.0.1, /192.168.10.103], 70, Mac OS
> X x86_64 10.11.6, wangqingpeng, Java(TM) SE Runtime Environment
> 1.8.0_131-b11
> >>> Total number of CPUs: 4
> >>> Total heap size: 3.6GB
> >>> Total offheap size: 6.4GB
> ..
> 23:48:44.629 [exchange-worker-#42%test01%] DEBUG
> o.a.i.i.p.c.d.d.p.l.ExchangeLatchManager  - Server latch is created
> [latch=IgniteBiTuple [val1=exchange, val2=AffinityTopologyVersion
> [topVer=71, minorTopVer=0]], participantsSize=5]
> ..
> 23:48:44.635 [exchange-worker-#42%test01%] DEBUG
> o.a.i.i.p.c.d.d.p.l.ExchangeLatchManager  - Count down +
> [latch=exchange-AffinityTopologyVersion [topVer=71, minorTopVer=0],
> remaining=4]
> ..
> 23:48:54.639 [exchange-worker-#42%test01%] WARN
> o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture  - Unable to await
> partitions release latch within timeout: ServerLatch [permits=4,
> pendingAcks=[230e516f-6c12-4391-b902-822afc6f7bc4,
> 58973b59-db04-4325-9966-d259a7ae3bbd, a79a57f0-d3a4-4eca-890b-0a043bb0d23f,
> c261b07c-9495-4058-889b-bd484be10477], super=CompletableLatch [id=exchange,
> topVer=AffinityTopologyVersion [topVer=71, minorTopVer=0]]]



full logs is:
see the attachment

tks


question2.log
Description: Binary data

unsubscribe

[Murthy Kakarlamudi]

ignite cluster management

[wangsan]

I am using ignite to manage my cluster.when node join I save an item in my
NodeCache, when node left I delete an item in my NodeCache with ignite event
listener. 
Use custom NodeCache but not ignite cluster topology,The reason is I want to
keep node status persistence when node left for some restart operations.
I have some problems
1. How to guarantee client node with same consistent(or other properties) do
not join the cluster twice like server node works.
2. My nodes topology will have server 100+ client 2000+. how to make the
cluster keep high stability.

tks



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: How much heap to allocate

[eugene miretsky]

Denis, thanks for the detailed response.

A few more follow up questions
1) Are indexs loaded into heap (when used)?
2) Are full pages loaded into heap, or only the matching records?
3) When the query needs more processing than the exisiting index
(non-indexed columns, groupBy, aggreag) where/how does it happen?
4) How is the query coordinator chosen? Is it the client node? How about
when using the web console?
5) What paralalism settings would your recomend, we were thinking to set
parallelJobsNumber  to 1  and task parallelism to number of cores * 2 -
this way we can make sure that each job gets al the heap memory instead of
all jobs fighting each other. Not sure if it makes sense, and it will also
prevent us from making real time transactional transactional queries.(we
are hoping to use ignite for both olap and simple real time queries)

Cheers,
Eugene


On Sat, Aug 25, 2018 at 3:25 AM Denis Magda  wrote:

> Hello Eugene,
>
> 1) In what format is data stored off heap?
>
>
> Data is always stored in the binary format let it be on-heap, off-heap or
> Ignite persistence.
> https://apacheignite.readme.io/docs/binary-marshaller
>
> 2) What happens when a SQL query is executed, in particular
>
>>
>>- How is H2 used? How is data loaded in H2? What if some of the  data
>>is on disk?
>>
>> H2 is used to build execution plans for SELECTs. H2 calls Ignite's B+Tree
> based indexing implementation to see which indexes are set. All the data
> and indexes are always stored in Ignite (off-heap + disk).
>
>>
>>- When is data loaded into heap, and how much? Is only the output of
>>H2 loaded, or everything?
>>
>> Queries results are stored in Java heap temporarily. Once the result set
> is read by your application, it will be garbage collected.
>
>>
>>- How is the reduce stage performed? Is it performed only on one node
>>(hence that node needs to load all the data into memory)
>>
>> Correct, the final result set is reduced on a query coordinator - your
> application that executed a SELECT.
>
> 3) What happens when Ingite runs out of memory during execution? Is data
>> evictied to disk (if persistence is enabled)?
>
>
> I guess you mean what happens if a result set doesn't fit in RAM during
> the execution, right? If so, then OOM will occur. We're working on an
> improvement that will offload the result set to disk to avoid OOM for all
> the scenarious:
> https://issues.apache.org/jira/browse/IGNITE-7526
>
>
>
>> 4) Based on the code, it looks like I need to set my data region size to
>> at most 50% of available memory (to avoid the warning), this seems a bit
>> wastefull.
>
>
> There is no such a requirement. I know many deployments use cases when one
> data region is given 20% of RAM, the other is given 40% and everything else
> is persisted to disk.
>
> 5) Do you have any general advice on benchmarking the memory requirpement?
>> So far I have not been able to find a way to check how much memory each
>> table takes on and off heap, and how much memory each query takes.
>
>
> We use Yardstick for performance benchmarking:
> https://apacheignite.readme.io/docs/perfomance-benchmarking
>
> --
> Denis
>
> On Fri, Aug 24, 2018 at 7:06 AM eugene miretsky 
> wrote:
>
>> Thanks!
>>
>> I am trying to understand when and how data is moved from off-heap to on
>> heap, particularly when using SQL.  I took a look at the wiki
>> 
>>  but
>> still have a few questions
>>
>> My understanding is that data is always store off-heap
>>
>> 1) In what format is data stored off heap?
>> 2) What happens when a SQL query is executed, in particular
>>
>>- How is H2 used? How is data loaded in H2? What if some of the  data
>>is on disk?
>>- When is data loaded into heap, and how much? Is only the output of
>>H2 loaded, or everything?
>>- How is the reduce stage performed? Is it performed only on one node
>>(hence that node needs to load all the data into memory)
>>
>> 3) What happens when Ingite runs out of memory during execution? Is data
>> evictied to disk (if persistence is enabled)?
>> 4) Based on the code, it looks like I need to set my data region size to
>> at most 50% of available memory (to avoid the warning), this seems a bit
>> wastefull.
>> 5) Do you have any general advice on benchmarking the memory
>> requirpement? So far I have not been able to find a way to check how much
>> memory each table takes on and off heap, and how much memory each query
>> takes.
>>
>> Cheers,
>> Eugene
>>
>> On Fri, Aug 24, 2018 at 8:06 AM, NSAmelchev  wrote:
>>
>>> Hi Eugene,
>>>
>>> Yes, it's a misprint as Dmitry wrote.
>>>
>>> Ignite print this warning if nodes on local machine require more than
>>> 80% of
>>> physical RAM.
>>>
>>> From code, you can see that total heap/offheap memory summing
>>> from nodes having the same mac address. This way calculates total memory
>>> used
>>> by the local machine.
>>>
>>> --

Re: How to run a job every 5 seconds in Ignite

[Lokesh Sharma]

This is what I was looking for. Many thanks!

On Mon, Aug 27, 2018 at 3:01 PM Evgenii Zhuravlev 
wrote:

> Hi Lokesh,
>
> I'd suggest to start Ignite service, which will guarantee failover-safety
> for you: https://apacheignite.readme.io/docs/service-grid. Just choose
> cluster-singleton to make sure that you will have 1 instance of Service in
> cluster. Inside this service you can use ignite-scheduler:
> https://apacheignite.readme.io/docs/cron-based-scheduling, it has
> cron API.
>
> Evgenii
>
> пн, 27 авг. 2018 г. в 9:16, Lokesh Sharma :
>
>> I'm using Ignite with Spring Boot. Is there a way to run a job every 5
>> seconds on exactly one node of the cluster (which could be any node)?
>>
>> Thank You
>>
>