Re: Ignite cluster will be jammed for long-running queries

[ght230]

Yes, some queries will be long running, they are unavoidable.

Query slow is acceptable, but the cluster jammed is a bit of trouble.

I will be grateful if IGNITE-4105 can be solved as soon as possible.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-cluster-will-be-jammed-for-long-running-queries-tp10210p10238.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Kafka as a Persistent Store

[pragmaticbigdata]

Can I configure Kafka as a persistent store for my apache ignite cache? If
so, I assume it could be configured in both write-through and write-behind
modes. Kindly share some insights on the configuration of such an
implementation. 

Thanks.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Kafka-as-a-Persistent-Store-tp10237.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Getting previous value in cache immediately after performing an unlock

[styriver]

Below is an example of a test case that simulates the problem we are
experiencing. 

After we put data to a cache and unlock it if we read the value immediately
after unlock
we randomly get the previous value prior to the update. We are utilizing
locks
to prevent data being overwritten between our web application and a possible
back end process
update. I simulated a multi-thread scenario that we would commonly see. We
tried transactions but
the failure rate was even higher. 

package com.xxx.cachemanager.service;

import java.util.concurrent.locks.Lock;

import org.apache.commons.lang.text.StrBuilder;
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteCache;
import org.apache.ignite.Ignition;

public class IgniteCacheLockingExample
{
   private static Object simpleMonitorLock = new Object();
   private PerformSimpleCacheValidationThreadWithLock
simpleCacheValidationWithLock = new
PerformSimpleCacheValidationThreadWithLock();

   public static void main (String[] args)
   {
  IgniteCacheLockingExample locking = new IgniteCacheLockingExample();
  locking.lockTestIgniteExample(true);
   }

   public IgniteCacheLockingExample ()
   {
  simpleCacheValidationWithLock.setDaemon(true);
  simpleCacheValidationWithLock.start();
   }

   public void lockTestIgniteExample (boolean multithread)
   {
  Ignite ignite = null;
  try {
 StrBuilder builder = new StrBuilder("Starting
lockTestIgniteExample(). Multithreading value \"" + multithread + "\"");

 Ignition.setClientMode(true);
 ignite = Ignition.start("testLock.xml");
 System.out.println("Starting lockTestIgniteExample().
Multithreading value \"" + multithread + "\"");

 IgniteCache testLockCache =
ignite.getOrCreateCache("TestLockCache");
 simpleCacheValidationWithLock.setCache(testLockCache);

 // initialize to zero
 testLockCache.removeAll();
 for (int i = 0; i < 2; i++) {
testLockCache.put(i, 0);
 }

 int valueCounter = 0;
 for (int i = 0; i < 2; i++) {
Integer value = null;
if (i != 0) {
   if (multithread) {
  // validate
  simpleCacheValidationWithLock.setIndex(i - 1);
 
simpleCacheValidationWithLock.setValueToValidate(valueCounter);
  synchronized (simpleMonitorLock) {
 simpleMonitorLock.notifyAll();
  }
   } else {
  // validate
  value = (Integer) testLockCache.get(i - 1);
  if (value.intValue() != valueCounter) {
 builder.appendln("Validation after update failed. Cache
Value \"" + value + "\" Test Value \"" + valueCounter + "\"");
  }
   }
}

// lock
Lock lock = testLockCache.lock(i);
lock.lock();

// get
value = testLockCache.get(i);

// update and put
valueCounter++;
testLockCache.put(i, valueCounter);

// unlock
lock.unlock();
 }
 System.out.println(builder.toString());
 System.out.println("Test Complete");
  } catch (Throwable throwable) {
 throwable.printStackTrace();
  } finally {
 Ignition.stop(ignite.name(), false);
  }
   }

   private class PerformSimpleCacheValidationThreadWithLock extends Thread
   {
  private int valueToValidate;
  private int index;
  private IgniteCache testCacheLock;

  public PerformSimpleCacheValidationThreadWithLock ()
  {
  }

  public void run ()
  {
 while (true)
synchronized (simpleMonitorLock) {
   try {
  simpleMonitorLock.wait();

  // We are simulating an independent process updating cache
outside of our
  // normal web flow. That is why we are locking here. It
will still fail if you
  // remove the lock below but it fails more frequently with
the lock present.
  Lock lock = testCacheLock.lock(index);
  lock.lock();

  // validate
  Integer value = (Integer) testCacheLock.get(index);
  if (value.intValue() != valueToValidate) {
 System.out.println("Validation after update failed.
Cache Value \"" + value + "\" Test Value \"" + valueToValidate + "\"");
  }

  lock.unlock();
   } catch (InterruptedException e) {
   }
}
  }

  public void setValueToValidate (int valueToValidate)
  {
 this.valueToValidate = valueToValidate;
  }

  public void setIndex (int index)
  {
 this.index = index;
  }

  public void setCache (IgniteCache testCacheLock)
  {
 this.testCacheLock = testCacheLock;
  }
   }
}

Re: OFFHEAP_TIERED and memory fragmentation

[vkulichenko]

I don't think I will be able to explain better than it's done in the ticket
:) Current approach is naive and not always effective for multiple reasons.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/OFFHEAP-TIERED-and-memory-fragmentation-tp10218p10235.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: OFFHEAP_TIERED and memory fragmentation

[Peter Schmitt]

Hi Val,

that means it depends a lot on the OS and if/how it handles virtual
memory-pages.
So why do you plan to change that?

Thanks,
Peter



2017-01-25 0:14 GMT+01:00 vkulichenko :

> Random means anywhere in empty memory space. When you create an entry, a
> new
> block is create for this value only. If you remove an entry, memory is
> released. Once removed, it can be used by any application including Ignite.
> Basically, the actual location where memory is allocated is defined by OS.
>
> Cache clear will free all the memory consumed by entries in this cache.
>
> -val
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/OFFHEAP-TIERED-and-memory-fragmentation-tp10218p10232.
> html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Connection problems

[vkulichenko]

Lukas,

Check that nodes can connect to each other (i.e. there are no network
issues, no firewall or ports are opened, etc.). Another possible reason is
GC - make sure that you have enough heap memory.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Connection-problems-tp10224p10233.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: OFFHEAP_TIERED and memory fragmentation

[vkulichenko]

Random means anywhere in empty memory space. When you create an entry, a new
block is create for this value only. If you remove an entry, memory is
released. Once removed, it can be used by any application including Ignite.
Basically, the actual location where memory is allocated is defined by OS.

Cache clear will free all the memory consumed by entries in this cache.

-val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/OFFHEAP-TIERED-and-memory-fragmentation-tp10218p10232.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Abandon of the non-throwing version of C++ client API

[Denis Magda]

Thanks for the explanation. Support your intention than, let’s make the 
exceptions-based approach default one and leave an ability to fallback to 
error-codes if required.

—
Denis

> On Jan 24, 2017, at 3:01 AM, Igor Sapego  wrote:
> 
> I've read through the old discussion. It seems that I was not yet
> a part of the community back there thats why I could not find it.
> 
> Denis, error code approach is more a C way, not C++. In most
> cases C++ developers avoid exceptions in embedded development,
> in legacy applications where it is hard to introduce exception safety,
> or/and if their compiler is old, as old compilers had significant
> overhead when used with exceptions and thus had exceptions
> disabled by default.
> 
> However, most applications now use modern compilers with
> zero-cost exceptions implementation and actively use standard
> library, which uses exceptions for error reporting anyway.
> 
> Best Regards,
> Igor
> 
> On Tue, Jan 24, 2017 at 10:49 AM, Vladimir Ozerov  > wrote:
> Originally I tried to cover situation when user disabled exceptions. In
> this case throwing something will lead to application crash. However, for
> now I believe that such use case is not very likely as Ignite is relatively
> high-level product. For this reason I would prefer to keep clean and
> compact API which will throw exceptions. But still have a workaround for
> users which do not want that. For instance, we may consult to some
> environment variable in runtime, or to some preprocessor flag in compile
> time, and expose additional static "GetLastError" method.
> 
> On Mon, Jan 23, 2017 at 11:35 PM, Denis Magda  > wrote:
> 
> > Guys,
> >
> > I found the initial discussion from the early times of our C++ client:
> > http://apache-ignite-developers.2346864.n4.nabble 
> > .
> > com/C-exception-handling-strategy-td778.html
> >
> > Vovan, that time you were on the side of error-code methods. Why have you
> > changed you mind proposing to make the throwing version default one? Any
> > new tendency in C++ community?
> >
> > —
> > Denis
> >
> > On Jan 23, 2017, at 2:56 AM, Vladimir Ozerov  > > wrote:
> >
> > +1 to Igor's idea. Ignite is relatively high-level product and we do not
> > expect ultra-optimized users who cannot allow exceptions to be enabled.
> > Macros should be a good workaround for them, though.
> >
> > On Sat, Jan 21, 2017 at 6:47 PM, Denis Magda  > > wrote:
> >
> >> Hi Igor,
> >>
> >> My C++ experience is based only on error code methods. This is why I
> >> thought that exceptions based approach is unrelated to C++ at all.
> >>
> >> I do remember we discussed all the pros and cons of these ways before.
> >> Could you find that old discussion and share it here? I'm on a mobile now,
> >> not easy to do on my own.
> >>
> >> Denis
> >>
> >>
> >> On Friday, January 20, 2017, Igor Sapego  >> > wrote:
> >>
> >>> Hi Igniters,
> >>>
> >>> I'm the guy who mostly contribute in C++ Ignite client and I
> >>> need your advice. Mostly I'd like to hear from our users and
> >>> those who are experienced in C++. Currently we have two
> >>> versions of most API methods - the throwing one and the
> >>> one that returns error through output argument. This was initially
> >>> done because we were not sure which way of error-reporting
> >>> is going to be preferred by our users.
> >>>
> >>> Now this approach bloats C++ API a lot and makes it harder to
> >>> maintain and optimize code. I propose like to abandon and deprecate
> >>> non-throwing version of API and only leave throwing version,
> >>> but first I want to hear from you guys - what do you think? Does
> >>> anyone use non-throwing version of the API? Maybe your toolchain
> >>> does not support exceptions or are you disabling them on purpose?
> >>>
> >>> For those who prefer disabling exceptions I propose to introduce
> >>> some macros like IGNITE_DISABLE_EXCEPTIONS and add
> >>> some thread-local error-storing mechanism like ignite::GetLastError().
> >>>
> >>> What do you guys think?
> >>>
> >>> Best Regards,
> >>> Igor
> >>>
> >>
> >
> >
> 

Re: OFFHEAP_TIERED and memory fragmentation

[Peter Schmitt]

Hi Val,

What does randomly mean in this context?
If one entry is removed, does Ignite reuse the space for new entries with
the same or smaller size?
Does Ignite organize it in small blocks? If yes, what happens if all
entries stored in a block were removed?
What does happen in case of a cache-clear?

Thanks,
Peter



2017-01-24 20:10 GMT+01:00 vkulichenko :

> Hi Peter,
>
> Current implementation rendomly allocates memory for each value when it's
> added, and frees memory when the entry is removed. Therefore, the
> segmentation is possible if you constantly remove entries and create new
> once. In case you usually *update* instead, especially with low throughput
> that you have, I would not expect any problems.
>
> -Val
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/OFFHEAP-TIERED-and-memory-fragmentation-tp10218p10227.
> html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

AW: AW: AW: AW: Rediscovery after Startup

[Lukas Lentner]

Thankx. That did it!




Lukas Lentner, B. Sc.
 
St.-Cajetan-Straße 13
81669 München
Deutschland
 
Fon: +49 / 89  / 71 67 44 96
Mobile:  +49 / 176 / 24 77 09 22
 
E-Mail:  kont...@lukaslentner.de
Website: www.LukasLentner.de

IBAN:DE41 7019  0002 1125 58
BIC: GENODEF1M01 (Münchner Bank)

> -Ursprüngliche Nachricht-
> Von: vkulichenko [mailto:valentin.kuliche...@gmail.com]
> Gesendet: Mittwoch, 11. Januar 2017 23:50
> An: user@ignite.apache.org
> Betreff: Re: AW: AW: AW: Rediscovery after Startup
> 
> I just noticed that you set ipFinder.setShared(true); which actually causing
> this behavior. If you remove this line, B will wait for A in your case, as I
> described before.
> 
> -Val
> 
> 
> 
> --
> View this message in context: http://apache-ignite-
> users.70518.x6.nabble.com/Rediscovery-after-Startup-tp10007p10043.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: AddressResolver in clients

[jimfcarroll]

Thanks.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/AddressResolver-in-clients-tp10223p10228.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: AddressResolver in clients

[vkulichenko]

Hi,

In current implementation connections can be established either way, so
servers must be able to connect to clients. Therefore client nodes should
also have address resolvers exposing their public addresses.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/AddressResolver-in-clients-tp10223p10226.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: OFFHEAP_TIERED and memory fragmentation

[vkulichenko]

Hi Peter,

Current implementation rendomly allocates memory for each value when it's
added, and frees memory when the entry is removed. Therefore, the
segmentation is possible if you constantly remove entries and create new
once. In case you usually *update* instead, especially with low throughput
that you have, I would not expect any problems.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/OFFHEAP-TIERED-and-memory-fragmentation-tp10218p10227.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite cluster will be jammed for long-running queries

[vkulichenko]

Are you expecting these to queries to be long running? If not, this ticket
will not help you in any way and you should work on resolving the root cause
of bad query performance. Check that the execution plan is correct and that
you have enough resources (memory and CPU) to execute all queries. You can
also try to scale out by adding more nodes.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-cluster-will-be-jammed-for-long-running-queries-tp10210p10225.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Connection problems

[Lukas Lentner]

Hi,

I have the problem that 2 nodes all the time loose their connection. What can I 
do to make it more durable?

My Log:

[ignite-update-notifier-timer] INFO 
org.apache.ignite.internal.processors.cluster.GridUpdateNotifier - Update 
status is not available.
[tcp-disco-msg-worker-#2%null%] WARN 
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi - Timed out waiting for 
message delivery receipt (most probably, the reason is in long GC pauses on 
remote node; consider tuning GC and increasing 'ackTimeout' configuration 
property). Will retry to send message with increased timeout. Current timeout: 
1.
[tcp-disco-msg-worker-#2%null%] WARN 
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi - Failed to send message to 
next node [msg=TcpDiscoveryStatusCheckMessage [creatorNode=TcpDiscoveryNode 
[id=663d9d58-49ad-4993-9495-2304120958c3, addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 
172.17.0.6], sockAddrs=[/0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500, 
86f7fe7944ed/172.17.0.6:47500], discPort=47500, order=2, intOrder=2, 
lastExchangeTime=1485283294557, loc=true, ver=1.7.0#20160801-sha1:383273e3, 
isClient=false], failedNodeId=null, status=0, super=TcpDiscoveryAbstractMessage 
[sndNodeId=null, id=18e26c1d951-663d9d58-49ad-4993-9495-2304120958c3, 
verifierNodeId=null, topVer=0, pendingIdx=0, failedNodes=null, 
isClient=false]], next=TcpDiscoveryNode 
[id=0392e735-de23-475d-bae3-dfcb2512d586, addrs=[127.0.0.1, 172.17.0.5], 
sockAddrs=[483aeb5c0a2b/172.17.0.5:47500, /127.0.0.1:47500], discPort=47500, 
order=1, intOrder=1, lastExchangeTime=1485283222882, loc=false, 
ver=1.7.0#20160801-sha1:383273e3, isClient=false], errMsg=Failed to send 
message to next node [msg=TcpDiscoveryStatusCheckMessage 
[creatorNode=TcpDiscoveryNode [id=663d9d58-49ad-4993-9495-2304120958c3, 
addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 172.17.0.6], 
sockAddrs=[/0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500, 
86f7fe7944ed/172.17.0.6:47500], discPort=47500, order=2, intOrder=2, 
lastExchangeTime=1485283294557, loc=true, ver=1.7.0#20160801-sha1:383273e3, 
isClient=false], failedNodeId=null, status=0, super=TcpDiscoveryAbstractMessage 
[sndNodeId=null, id=18e26c1d951-663d9d58-49ad-4993-9495-2304120958c3, 
verifierNodeId=null, topVer=0, pendingIdx=0, failedNodes=null, 
isClient=false]], next=ClusterNode [id=0392e735-de23-475d-bae3-dfcb2512d586, 
order=1, addr=[127.0.0.1, 172.17.0.5], daemon=false]]]
[tcp-disco-msg-worker-#2%null%] WARN 
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi - Local node has detected 
failed nodes and started cluster-wide procedure. To speed up failure detection 
please see 'Failure Detection' section under javadoc for 'TcpDiscoverySpi'
[disco-event-worker-#43%null%] WARN 
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager - Node 
FAILED: TcpDiscoveryNode [id=0392e735-de23-475d-bae3-dfcb2512d586, 
addrs=[127.0.0.1, 172.17.0.5], sockAddrs=[483aeb5c0a2b/172.17.0.5:47500, 
/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, 
lastExchangeTime=1485283222882, loc=false, ver=1.7.0#20160801-sha1:383273e3, 
isClient=false]
[disco-event-worker-#43%null%] INFO 
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager - Topology 
snapshot [ver=3, servers=1, clients=0, CPUs=1, heap=0.47GB]
[exchange-worker-#45%null%] INFO 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager - 
Skipping rebalancing (nothing scheduled) [top=AffinityTopologyVersion 
[topVer=3, minorTopVer=0], evt=NODE_FAILED, 
node=0392e735-de23-475d-bae3-dfcb2512d586]
[tcp-disco-msg-worker-#2%null%] WARN 
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi - Node is out of topology 
(probably, due to short-time network problems).
[disco-event-worker-#43%null%] WARN 
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager - Local node 
SEGMENTED: TcpDiscoveryNode [id=663d9d58-49ad-4993-9495-2304120958c3, 
addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 172.17.0.6], 
sockAddrs=[/0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500, 
86f7fe7944ed/172.17.0.6:47500], discPort=47500, order=2, intOrder=2, 
lastExchangeTime=1485283316748, loc=true, ver=1.7.0#20160801-sha1:383273e3, 
isClient=false]
[disco-event-worker-#43%null%] WARN 
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager - Stopping 
local node according to configured segmentation policy.
[Thread-4] INFO 
org.apache.ignite.internal.processors.rest.protocols.tcp.GridTcpRestProtocol - 
Command protocol successfully stopped: TCP binary
[Thread-4] INFO org.apache.ignite.internal.processors.cache.GridCacheProcessor 
- Stopped cache: ignite-marshaller-sys-cache
[Thread-4] INFO org.apache.ignite.internal.processors.cache.GridCacheProcessor 
- Stopped cache: ignite-sys-cache
[Thread-4] INFO org.apache.ignite.internal.processors.cache.GridCacheProcessor 
- Stopped cache: ignite-atomics-sys-cache
[Thread-4] INFO org.apache.ignite.internal.IgniteKernal -

>>> +-+
>>> Ignite ver. 1.7.0#2016

AddressResolver in clients

[jimfcarroll]

I managed to get a cluster of servers set up using docker in bridge mode by
supplying an AddressResolver for both discovery and communications. I only
did this in the servers. Intuitively it doesn't seem necessary to do this in
clients since I'm assuming connection initiation is done from the clients to
the servers, so as long as the clients get the correct addresses during the
discovery phase, they shouldn't need to resolve their own external address. 

However, it's not clear from the documentation on Clients vs Servers if this
is the case. I'm assuming cleints don't even listen on any ports. Can
someone clarify?

Thanks




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/AddressResolver-in-clients-tp10223.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Apache Ignite 1.7.0 Session caching issue with update session attributes

[Andrey Gura]

Use BinaryMarshaller or OptimizedMarshaller with require serializable
flag with false value.

On Tue, Jan 24, 2017 at 5:50 PM, Siva Annapareddy
 wrote:
> While trying Appache Ignite 1.7.0 for Web Session Caching for a Groovy
> Application, we seem to have hit issue with
> 'Failed to unmarshal object with optimized marshaller'
> or
> 'Failed to serialize object: ' with
> JdkMarshaller that points to 'Caused by: java.io.NotSerializableException:
> groovy.json.internal.LazyMap'
>
> Application is running in GGTS on Windows Platform and Apache Ignite 1.7.0
> two-node cluster running on RHEL 7
>
> Any clues on how to proceed?
>
>
>
> --
> View this message in context: 
> http://apache-ignite-users.70518.x6.nabble.com/Apache-Ignite-1-7-0-Session-caching-issue-with-update-session-attributes-tp10220.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: monitoring issue -Connection to Ignite Node is not established

[Alexey Kuznetsov]

Hi!

Are you sure that you copied /libs/optional/ignite-rest-http to /libs
folder?


On Tue, Jan 24, 2017 at 8:57 PM, minisoft_rm 
wrote:

> dear experts:
>
> I am using ignite 1.8 with agent 1.7.4
>
> after start the agent with default properties, I can import local db schema
> things into webconsole.
>
> but... the "monitoring clusters" function doesn't work. like below:
>
> in command line:
> [21:50:56,754][INFO ][EventThread][RestHandler] Failed connect to node and
> execute REST command
> [uri=http://localhost:8080/ignite?mtr=true&cmd=top&attr=true]
>
>
> in browser:
> Connection to Ignite Node is not established
>
> Failed connect to node and execute REST command.
>
>
> I surely enjoyed this monitoring feature in previous version.. but this new
> 1.8 with 1.7.4 doesn't work.
>
> please tell me what changes product team make? thanks~
>
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/monitoring-issue-Connection-to-Ignite-Node-is-not-
> established-tp10216.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>



-- 
Alexey Kuznetsov

Apache Ignite 1.7.0 Session caching issue with update session attributes

[Siva Annapareddy]

While trying Appache Ignite 1.7.0 for Web Session Caching for a Groovy
Application, we seem to have hit issue with 
'Failed to unmarshal object with optimized marshaller'  
or
'Failed to serialize object: ' with
JdkMarshaller that points to 'Caused by: java.io.NotSerializableException:
groovy.json.internal.LazyMap'

Application is running in GGTS on Windows Platform and Apache Ignite 1.7.0
two-node cluster running on RHEL 7

Any clues on how to proceed?



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Apache-Ignite-1-7-0-Session-caching-issue-with-update-session-attributes-tp10220.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Re[4]: Faceted search

[Andrey Mashenkov]

Hi Mikhail,

With 1-st approach
1. Underlying storage can be changed without rewiring its code to add
indexing support.
2. IndexSPI will updated in writeThrough(false) mode, however CacheStore
won't.

With 2-nd
1. You should bother about transaction handling (implementing
CacheStore.sessionEnd() method).
2. If you use Lucene as CacheStore in readTrough(true), you should store
whole entries in Lucene index to be able return entries via load\loadAll
methods. This will significantly increase index size.



On Mon, Jan 23, 2017 at 5:31 PM, Mikhail  wrote:

> Hi, Andrey.
>
>Thanks a lot. What are the current plans about updating the
> version of Lucene to the latest one? It is really straightforward, I spend
> about 3 hours to implement it.
>I see 2 approaches:
>1.) Implement IndexingSPI using ElasticSearch to store and
> get entities;
>2.) Store entities using ElasticSearch in CacheStore and
> query them without usage of Ignite.
>
>What are the advantages of the first option?
>
> --
> Best Regards,
> Mikhail
>
> Четверг, 19 января 2017, 16:26 +03:00 от Andrey Mashenkov <
> andrey.mashen...@gmail.com>:
>
> Hi Mikhail,
>
> You can try to implement your own IndexinSPI implementation and use SPI
> queries, but it seems to be non trivial task.
> Moreover Ignite already have Lucene 3.5.0 that is not supports Faceted
> search. So, you will need to resolve version conflicts while implementing
> IndexingSPI.
>
> On Thu, Jan 19, 2017 at 4:12 PM, Mikhail  wrote:
>
> Hi,
>
>  I need to implement something like this [1]. I think, I can't
> implement it using just one SQL query (count the facets manually on every
> query is not an option). Faceted search is brilliant feature of Lucene and
> its family (ElasticSearch, Solr, etc). So, in order to implement it there
> are two options:
>
> 1.) Use ElasticSearch (or other full text engine) as a Persistent Store.
> 2.) Use existing Ignite Lucene index for faceted search.
>
> I don't like the first option, because I persist my entities in database.
> Also I don't want to make ElasticSearch query for entities and then second
> Ignite query to get entities from caches. Therefore, it would be perfect if
> there were some mechanism in Ignite to override the "GridLuceneIndex" class
> and implement necessary changes. Is there some way to do it?
>
> [1] http://proiot.ru/blog/posts/2016/05/15/lucene-fasetnyi-poisk/
>
> --
> Best Regards,
> Mikhail
>
> Wednesday, 12 January 2017, 12:19 +03:00 от vkulichenko <
> valentin.kuliche...@gmail.com>:
> >
> >Mikhail,
> >
> >From what I here, you can simply use SQL for this task. Is there
> something in particular that doesn't work for you?
> >
> >-Val
>
> Hi,
>
>   For my big data project I need to implement a catalog facet
> search (like in modern online shops). I need to filter by criteria
> (boolean, enum, range) and:
>   1.) Show count of suitable items.
>   2.) Hide filters that can't be applied mutually with the
> specified filters.
>   3.) Show (maybe approximate) count of items for each filter,
> that can be specified.
>
>   All this I need to implement without retriving the whole
> list of items (opposite - it would be trivial).
>   Could you imagine how to implement it using Ignite?
>   I can create Persistent Store which will write to Solr or
> ElasticSearch, but I have already persist entities in database and use
> Ignite for SQL and Fulltext queries. So I would prefer not to have 2
> different platforms for indexing content.
>
> --
> Best Regards,
> Mikhail Fokanov
>
>
>
>
> --
> С уважением,
> Машенков Андрей Владимирович
> Тел. +7-921-932-61-82
>
> Best regards,
> Andrey V. Mashenkov
> Cerr: +7-921-932-61-82
>
>
>

OFFHEAP_TIERED and memory fragmentation

[Peter Schmitt]

Hello Ignite-Community!

I just found IGNITE-3477 and esp.

"Continuous put/remove operations with OFFHEAP_TIERED mode lead to
uncontrollable memory fragmentation"

How does Ignite handle OFFHEAP_TIERED caches currently (= how does it store
the data)?
I just tried to create some tests to get an impression of the
fragmentation, however, I couldn't see a huge issue.
In my case I've 30gb Offheap memory available and about 5 caches storing
about 3gb of data (after starting the application).
Every entry of the caches gets updated every few days the latest and some
several times a day.
However, I would like to avoid that cache-fragmentation hits the
application out of the blue and therefore I try to get an impression about
those internals.

Any hint is appreciated,
Peter

IGFS Questions

[pragmaticbigdata]

I have some questions of deploying IGFS as a cache layer given that ignite
could be deployed both as a key-value store and as a file system

1. How does IGFS behave when deployed in standalone mode? I wanted to
confirm that there is no durability in this mode. Assuming I persist a
parquet file on IGFS, if the cluster goes down I lose the file, right?
2. Do we get the ability to specify the fact that the file stored in IGFS
could be both partitioned (with backup nodes) or replicated?
3. IGFS can act as a cache layer over HDFS and local file system, can it act
as a caching layer over S3 store?
4. As with the key-value store, can I configure tiered storage in IGFS i.e.
given that IGFS is configured with local file system as the secondary store
and the ignite cluster of 3 server nodes configured with 5GB memory each,
would the data spill over to the local disk if I try to load a 25GB file
into IGFS? If so, what is the configuration needed?
5. Can I configure local SSD disks as the secondary store for IGFS?
6. I browsed through the documentation but I didn't find the capability of
pinning and unpinning files in IGFS. I am looking something similar to what 
alluxio

 
provides. Can it be implemented?
7. Could you elaborate a bit on how IGFS Proxy Mode works? What is its
recommended use case?
8. With DUAL_ASYNC (write-behind mode), does ignite have failover guarantees
which is lacking in the key-value store -
https://issues.apache.org/jira/browse/IGNITE-1897?

Thanks in advance.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/IGFS-Questions-tp10217.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

monitoring issue -Connection to Ignite Node is not established

[minisoft_rm]

dear experts:

I am using ignite 1.8 with agent 1.7.4

after start the agent with default properties, I can import local db schema
things into webconsole.

but... the "monitoring clusters" function doesn't work. like below:

in command line:
[21:50:56,754][INFO ][EventThread][RestHandler] Failed connect to node and
execute REST command
[uri=http://localhost:8080/ignite?mtr=true&cmd=top&attr=true]


in browser:
Connection to Ignite Node is not established

Failed connect to node and execute REST command.


I surely enjoyed this monitoring feature in previous version.. but this new
1.8 with 1.7.4 doesn't work.

please tell me what changes product team make? thanks~





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/monitoring-issue-Connection-to-Ignite-Node-is-not-established-tp10216.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Jetty configuration for adding new Handler

[Gaurav Bajaj]

Hello Andrey/Saurabh,

I could get it working with following additions in the jetty configuration :







/
Access-Control-Allow-Origin
*




/
Access-Control-Allow-Methods
GET,POST



  



Basically had to add "Access-Control-Allow-Methods" too.

Thanks,
Gaurav


On Mon, Jan 23, 2017 at 6:29 PM, GB  wrote:

> Hi Andry/Saurabh,
>
> I also have same issue as in original post.
> I tried exact steps as mentioned by Andry but still facing same issue.
> Header "Access-Control-Allow-Origin" is still not set in the response. Does
> anything else needed in Jetty Configuration other than what Andry
> mentioned?
>
> Thanks a lot.
>
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Jetty-configuration-for-adding-new-
> Handler-tp5841p10197.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Abandon of the non-throwing version of C++ client API

[Pavel Tupitsyn]

+1 for ditching non-throwing methods.

On Tue, Jan 24, 2017 at 2:01 PM, Igor Sapego  wrote:

> I've read through the old discussion. It seems that I was not yet
> a part of the community back there thats why I could not find it.
>
> Denis, error code approach is more a C way, not C++. In most
> cases C++ developers avoid exceptions in embedded development,
> in legacy applications where it is hard to introduce exception safety,
> or/and if their compiler is old, as old compilers had significant
> overhead when used with exceptions and thus had exceptions
> disabled by default.
>
> However, most applications now use modern compilers with
> zero-cost exceptions implementation and actively use standard
> library, which uses exceptions for error reporting anyway.
>
> Best Regards,
> Igor
>
> On Tue, Jan 24, 2017 at 10:49 AM, Vladimir Ozerov 
> wrote:
>
>> Originally I tried to cover situation when user disabled exceptions. In
>> this case throwing something will lead to application crash. However, for
>> now I believe that such use case is not very likely as Ignite is
>> relatively
>> high-level product. For this reason I would prefer to keep clean and
>> compact API which will throw exceptions. But still have a workaround for
>> users which do not want that. For instance, we may consult to some
>> environment variable in runtime, or to some preprocessor flag in compile
>> time, and expose additional static "GetLastError" method.
>>
>> On Mon, Jan 23, 2017 at 11:35 PM, Denis Magda  wrote:
>>
>> > Guys,
>> >
>> > I found the initial discussion from the early times of our C++ client:
>> > http://apache-ignite-developers.2346864.n4.nabble.
>> > com/C-exception-handling-strategy-td778.html
>> >
>> > Vovan, that time you were on the side of error-code methods. Why have
>> you
>> > changed you mind proposing to make the throwing version default one? Any
>> > new tendency in C++ community?
>> >
>> > —
>> > Denis
>> >
>> > On Jan 23, 2017, at 2:56 AM, Vladimir Ozerov 
>> wrote:
>> >
>> > +1 to Igor's idea. Ignite is relatively high-level product and we do not
>> > expect ultra-optimized users who cannot allow exceptions to be enabled.
>> > Macros should be a good workaround for them, though.
>> >
>> > On Sat, Jan 21, 2017 at 6:47 PM, Denis Magda 
>> wrote:
>> >
>> >> Hi Igor,
>> >>
>> >> My C++ experience is based only on error code methods. This is why I
>> >> thought that exceptions based approach is unrelated to C++ at all.
>> >>
>> >> I do remember we discussed all the pros and cons of these ways before.
>> >> Could you find that old discussion and share it here? I'm on a mobile
>> now,
>> >> not easy to do on my own.
>> >>
>> >> Denis
>> >>
>> >>
>> >> On Friday, January 20, 2017, Igor Sapego  wrote:
>> >>
>> >>> Hi Igniters,
>> >>>
>> >>> I'm the guy who mostly contribute in C++ Ignite client and I
>> >>> need your advice. Mostly I'd like to hear from our users and
>> >>> those who are experienced in C++. Currently we have two
>> >>> versions of most API methods - the throwing one and the
>> >>> one that returns error through output argument. This was initially
>> >>> done because we were not sure which way of error-reporting
>> >>> is going to be preferred by our users.
>> >>>
>> >>> Now this approach bloats C++ API a lot and makes it harder to
>> >>> maintain and optimize code. I propose like to abandon and deprecate
>> >>> non-throwing version of API and only leave throwing version,
>> >>> but first I want to hear from you guys - what do you think? Does
>> >>> anyone use non-throwing version of the API? Maybe your toolchain
>> >>> does not support exceptions or are you disabling them on purpose?
>> >>>
>> >>> For those who prefer disabling exceptions I propose to introduce
>> >>> some macros like IGNITE_DISABLE_EXCEPTIONS and add
>> >>> some thread-local error-storing mechanism like ignite::GetLastError().
>> >>>
>> >>> What do you guys think?
>> >>>
>> >>> Best Regards,
>> >>> Igor
>> >>>
>> >>
>> >
>> >
>>
>
>

Re: Abandon of the non-throwing version of C++ client API

[Igor Sapego]

I've read through the old discussion. It seems that I was not yet
a part of the community back there thats why I could not find it.

Denis, error code approach is more a C way, not C++. In most
cases C++ developers avoid exceptions in embedded development,
in legacy applications where it is hard to introduce exception safety,
or/and if their compiler is old, as old compilers had significant
overhead when used with exceptions and thus had exceptions
disabled by default.

However, most applications now use modern compilers with
zero-cost exceptions implementation and actively use standard
library, which uses exceptions for error reporting anyway.

Best Regards,
Igor

On Tue, Jan 24, 2017 at 10:49 AM, Vladimir Ozerov 
wrote:

> Originally I tried to cover situation when user disabled exceptions. In
> this case throwing something will lead to application crash. However, for
> now I believe that such use case is not very likely as Ignite is relatively
> high-level product. For this reason I would prefer to keep clean and
> compact API which will throw exceptions. But still have a workaround for
> users which do not want that. For instance, we may consult to some
> environment variable in runtime, or to some preprocessor flag in compile
> time, and expose additional static "GetLastError" method.
>
> On Mon, Jan 23, 2017 at 11:35 PM, Denis Magda  wrote:
>
> > Guys,
> >
> > I found the initial discussion from the early times of our C++ client:
> > http://apache-ignite-developers.2346864.n4.nabble.
> > com/C-exception-handling-strategy-td778.html
> >
> > Vovan, that time you were on the side of error-code methods. Why have you
> > changed you mind proposing to make the throwing version default one? Any
> > new tendency in C++ community?
> >
> > —
> > Denis
> >
> > On Jan 23, 2017, at 2:56 AM, Vladimir Ozerov 
> wrote:
> >
> > +1 to Igor's idea. Ignite is relatively high-level product and we do not
> > expect ultra-optimized users who cannot allow exceptions to be enabled.
> > Macros should be a good workaround for them, though.
> >
> > On Sat, Jan 21, 2017 at 6:47 PM, Denis Magda 
> wrote:
> >
> >> Hi Igor,
> >>
> >> My C++ experience is based only on error code methods. This is why I
> >> thought that exceptions based approach is unrelated to C++ at all.
> >>
> >> I do remember we discussed all the pros and cons of these ways before.
> >> Could you find that old discussion and share it here? I'm on a mobile
> now,
> >> not easy to do on my own.
> >>
> >> Denis
> >>
> >>
> >> On Friday, January 20, 2017, Igor Sapego  wrote:
> >>
> >>> Hi Igniters,
> >>>
> >>> I'm the guy who mostly contribute in C++ Ignite client and I
> >>> need your advice. Mostly I'd like to hear from our users and
> >>> those who are experienced in C++. Currently we have two
> >>> versions of most API methods - the throwing one and the
> >>> one that returns error through output argument. This was initially
> >>> done because we were not sure which way of error-reporting
> >>> is going to be preferred by our users.
> >>>
> >>> Now this approach bloats C++ API a lot and makes it harder to
> >>> maintain and optimize code. I propose like to abandon and deprecate
> >>> non-throwing version of API and only leave throwing version,
> >>> but first I want to hear from you guys - what do you think? Does
> >>> anyone use non-throwing version of the API? Maybe your toolchain
> >>> does not support exceptions or are you disabling them on purpose?
> >>>
> >>> For those who prefer disabling exceptions I propose to introduce
> >>> some macros like IGNITE_DISABLE_EXCEPTIONS and add
> >>> some thread-local error-storing mechanism like ignite::GetLastError().
> >>>
> >>> What do you guys think?
> >>>
> >>> Best Regards,
> >>> Igor
> >>>
> >>
> >
> >
>

Re: Timed out waiting for message delivery receipt

[Yakov Zhdanov]

Very good! Please keep us up to date.

--Yakov

2017-01-24 9:30 GMT+03:00 hitendrapratap <
hitendrapratapsingh.si...@target.com>:

> I has been resolved when increased "ignite.failure.detection.timeout" to
> 60sec from 10sec.
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Timed-out-waiting-for-message-delivery-
> receipt-tp10171p10209.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>