Ignite Service Unmarshal or Deserialize Error

[yonggu.lee]

After deploying an ignite service with my service implementation jar, an
annoying unmartial or deserialize error occurs.

My service interface is as follows.

  ResultTitle getTitle(String url, TitleCandidates titleCandidates);

If i change the class name of a service parameter (TitleCandidates ->
TitleCandidates_V1), then the unmarshal error disappears and everything runs
ok. But, I could not always change the class name when the jars is changed.

When the service function called via service proxy, the error message as
follows.

Failed to obtain remote job result policy for result from
ComputeTask.result(..) method (will fail the whole task): GridJobResultImpl
[job=C2 [c=ServiceProxyCallable [mtdName=getTitle,
svcName=TitleMakerService, ignite=null]], sib=GridJobSiblingImpl
[sesId=0f3f0699261-2df0837d-d5d6-4d45-ae32-16422c1d9704,
jobId=1f3f0699261-2df0837d-d5d6-4d45-ae32-16422c1d9704,
nodeId=10cdcacf-7768-41af-b277-df113d648e56, isJobDone=false],
jobCtx=GridJobContextImpl
[jobId=1f3f0699261-2df0837d-d5d6-4d45-ae32-16422c1d9704, timeoutObj=null,
attrs={}], node=TcpDiscoveryNode [id=10cdcacf-7768-41af-b277-df113d648e56,
addrs=[10.116.25.122, 10.244.8.0, 127.0.0.1, 172.17.0.1, 192.168.133.0],
sockAddrs=[/10.244.8.0:47500, /10.116.25.122:47500, /172.17.0.1:47500,
/192.168.133.0:47500, /127.0.0.1:47500], discPort=47500, order=5,
intOrder=5, lastExchangeTime=1522991697421, loc=false,
ver=2.3.0#20171220-sha1:8431829c, isClient=false], ex=class
o.a.i.IgniteException: Failed to deserialize object
[typeName=o.a.i.i.processors.closure.GridClosureProcessor$C2], hasRes=true,
isCancelled=false, isOccupied=true]
class org.apache.ignite.IgniteException: Remote job threw user exception
(override or implement ComputeTask.result(..) method if you would like to
have automatic failover for this exception).
at
org.apache.ignite.compute.ComputeTaskAdapter.result(ComputeTaskAdapter.java:101)
at
org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1047)
at
org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1040)
at
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6663)
at
org.apache.ignite.internal.processors.task.GridTaskWorker.result(GridTaskWorker.java:1040)
at
org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:858)
at
org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1066)
at
org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1301)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
at
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: class org.apache.ignite.IgniteException: Failed to deserialize
object
[typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2]
at
org.apache.ignite.internal.processors.job.GridJobWorker.initialize(GridJobWorker.java:457)
at
org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1109)
at
org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1913)
... 7 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
deserialize object
[typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2]
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9801)
at
org.apache.ignite.internal.processors.job.GridJobWorker.initialize(GridJobWorker.java:438)
... 9 more
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
deserialize object
[typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2]
at
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1762)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1714)
at
org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:310)
at
org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:99)
at
org.apache.ign

Ignite with hive as 3rdparty persistence

[Venkata Bhagavatula]

Hi,

We want to use hive as a 3rd party persistence for ignite. To achieve this
can you let us know how to take it forward. From the documentation, i
understand that we need to implement the CacheStore interface for achieving
this.

How can we make it generic enough irrespective of any schema?
We came to know that Hadoop accelerator is not to be used.

A quick response is highly appreciated.

Thanks n Regards,
Chalapathi.

Re: Cache access based on the user previleges

[vbm]

Hi Denis,

Thanks for the reply. 
Is this not needed. How does the isolation between different applications
happen ?

One thing that comes to my mind is, we can create different data regions and
put the cache needed for specific application in these data regions. Is this
the right approach ?


Regards,
Vishwas



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Kubernetes - Access Ignite Cluster Externally

[Roman Shtykh]

I have been playing with this for a while, and managed to get an external 
client node get into the topology, but it failed to communicate to the cluster. 
Some points:1. Used TcpDiscoveryKubernetesIpFinder for sever nodes
2. Exposed pods via NodePort on custom ports (higher than 3); more flexible 
than hostNetwork = True
3. Created an address resolver for finding and communication SPIs, so that pods 
addresses are conveyed to the external client.4. Used TcpDiscoveryVmIpFinder 
for the client to successfully join the topology.
Unfortunately, TcpCommunicationSPI fails to connect to the server nodes. The 
client has all addresses of the ClusterNode it attempts to connect to, 
including internal pods' IP, 127.0.0.1, 0:0:0:0:0:0:0:0 and external IP, but 
fails to reach the external IP from the list (see 
TcpCommunicationSPI.createTcpClient). Having ClusterNode expose only the 
address that was registered by the address resolver might fix it (haven't 
checked yet) -- anyway, I think the client should not be provided with and 
communicate via internal addresses iff external addresses are provided, should 
it?
-- Roman
 

On Thursday, April 5, 2018, 3:33:10 p.m. GMT+9, sid_human 
 wrote:  
 
 Hi 

I had recently come across this similar issue. I have multiple ignite server
pods up in a cluster and running in 3 - 4 nodes. At the same time, I have
ignite clients that can connect and work perfectly fine if they are running
as pods in these nodes. 

I tried to implement an external client connecting to the ignite server pods
cluster but no avail. I have followed through the readme.io pages of
kubernetes discovery. Here are the issues I had confronted:

1) Setting hostNetwork = True in the yaml configuration of ignite pods is a
network issue because running multiple containers on a node and each
container trying to host the port would send an error. So, I tried running
one container - one node. 
2) I have added a comment  IGNITE-4161
  with the exact client
configurations which seem to retrieve the server pods IP addresses. But no
connection is made.

I'm afraid no one has tried this before successfully and no development on
external clients are made as someone had posted in the same Jira- issue.

Thank you.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
  

Transaction progress, affinity, locks

[Ariel Tubaltsev]

I have 2 different caches C1, C2 on 3 nodes N1, N2, N3. I want to configure
it in the most possibly reliable way to avoid any inconsistency and data
loss, so both caches are REPLICATED, TRANSACTIONAL,  FULL_SYNC.

Most of my operations are transactions: PESSIMISTIC, SERIALIZABLE that touch
both caches, for example
tx {
  v= get(C1, k1);
  put(C2, v, x);
}

But there are also non-transactional reads from one of the caches.
All operations are performed from client (in client mode).

Some questions to better map the features:

* There will be only 2PC commit for transactions, no 1-phase commit, since I
have 3 nodes, REPLICATED?
* During the transaction, even for get, locks will be taken on backups
during the prepare phase, since it's PESSIMISTIC concurrency?
* readFromBackup - looks like it's not going to improve transactions
performance since transaction coordinator still going to always send it to
primary node and primary node will instruct backup to take locks?
* readFromBackup - still should help in non-transactional read, no? or it
always sent to primary node?
* affinity - will it help to configure it for REPLICATED caches, where
transaction touches all the node anyway?
* If there is no timeout for transaction and one of the backup nodes is
segmented out, transaction seems to be blocked for FailureDetectionTimeout
time (worst case). But is it supposed to resume and succeed after topology
is recalculated to include 2 nodes only?

Thank you
Ariel





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

TcpDiscoverSpi error connection refused

[Neeraj Vaidya]

Hi,
I have 2 virtualbox guest OS'es (CentOS7 64-bit) , each having firewalld 
stopped and disabled.
The nodes are axlrate-node-1 and axlrate-node-2. I have updated 
default-config.xml in both servers
My default-config.xml looks like below :
-default-config.xml @ 
axlrate-node-1-
http://www.springframework.org/schema/beans";
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
   xsi:schemaLocation="
   http://www.springframework.org/schema/beans
   http://www.springframework.org/schema/beans/spring-beans.xsd";>





  

  

  
  axlrate-node-1
  axlrate-node-2
  

  

  

  


---

-default-config.xml @ 
axlrate-node-2-
http://www.springframework.org/schema/beans";
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
   xsi:schemaLocation="
   http://www.springframework.org/schema/beans
   http://www.springframework.org/schema/beans/spring-beans.xsd";>





  

  

  
  axlrate-node-1
  axlrate-node-2
  

  

  

  


---


When I try to start ignite on axlrate-node-1, I see the following error at 
startup. 

---log 
on 
axlrate-node-1---
[09:14:41,747][SEVERE][main][TcpDiscoverySpi] Exception on direct send: 
Connection refused (Connection refused)
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at 
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.openSocket(TcpDiscoverySpi.java:1386)
at 
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.openSocket(TcpDiscoverySpi.java:1349)
at 
org.apache.ignite.spi.discovery.tcp.ServerImpl.sendMessageDirectly(ServerImpl.java:1169)
at 
org.apache.ignite.spi.discovery.tcp.ServerImpl.sendJoinRequestMessage(ServerImpl.java:1016)
at 
org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:860)
at 
org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:360)
at 
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:1846)
at 
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
at 
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:882)
at 
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1852)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1002)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1909)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1652)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1080)
at 
org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:998)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:884)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:783)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:653)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:622)
at org.apache.ignite.Ignition.start(Ignition.java:347)
at 
org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:302)
-

Then, when I start the axlrate-node-2, the nodes discover each other 
successfully.
However, when I stop axlrate-node-2, I see the following error in console of 
axlrate-node-1. 

Re: Cache access based on the user previleges

[Denis Mekhanikov]

Vishwas,

There is no such possibility in Ignite, as far as I know.
You will have to implement a security layer on top of Ignite yourself, if
you want to have such functionality.

Denis

чт, 5 апр. 2018 г. в 20:19, vbm :

> Hi,
>
> Is there any configuration with which we can restrict access to a cache
> based on user privileges ?
> In a production environment, how does access restriction to different cache
> are enforced ?
>
>
> Regards,
> Vishwas
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Ignite Sql Query Execution time is too long

[Denis Mekhanikov]

Siva,

The execution plan shows, that no indexes are used. If you create an index
on *EntityType* field, it should make the performance better.
Also if you make all data be stored in memory, then performance of select
queries will be better. Otherwise you will be limited by the performance of
your disk.

The information, that you provided is not enough to tell, why you get
the "Failed
to run map query remotely" exception.
Look through logs on other nodes and check, if they have any exceptions.

Denis

чт, 5 апр. 2018 г. в 21:05, begineer :

> Have you tried adding indexes to frequently searched fields ? Indexing can
> make searching the cache quite faster
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Failed to lock keys (all partition nodes left the grid)

[Olexandr K]

Here is client code which fails:

public void storeRefreshToken(UUID uid, UUID refreshTokenID) {
IgniteCache> cache =
getRefreshTokenCache();

try (Transaction tx = startTransaction()) {
LinkedList ids = cache.get(uid);  //getting error
here

On Thu, Apr 5, 2018 at 11:22 PM, Olexandr K 
wrote:

> Hi team,
>
> I'm getting strange exception when trying to save Key/Value pair.
> When I tested locally everything was fine.
> Now I deployed application to servers cluster and stuck with this error.
>
> What does it mean? My both Ignite server nodes are UP.
>
> Here is configuration for this cache:
>
> 
> 
> 
>  value="FULL_SYNC"/>
> 
> 
> 
>
> Here is my topology
>
> visor> top
> Hosts: 4
> +===
> +
> |  Int./Ext. IPs  |   Node ID8(@)| Node Type |
> OS| CPUs |  MACs   | CPU Load |
> +===
> +
> | 0:0:0:0:0:0:0:1 | 1: 35CD3AD1(@n2) | Client| Windows Server 2012 R2
> amd64 6.3 | 4| 00:00:00:00:00:00:00:E0 | 0.00 %   |
> | 10.2.0.225  |  |
> |  |  | 00:50:56:25:00:B9
> |  |
> | 127.0.0.1   |  |
> |  |  |
> |  |
> | 30.251.106.197  |  |
> |  |  |
> |  |
> +-+--+---+--
> +--+-+--+
> | 0:0:0:0:0:0:0:1 | 1: 230F83B8(@n3) | Client| Windows Server 2012 R2
> amd64 6.3 | 4| 00:00:00:00:00:00:00:E0 | 0.13 %   |
> | 10.2.0.250  |  |
> |  |  | 00:50:56:25:00:35
> |  |
> | 127.0.0.1   |  |
> |  |  |
> |  |
> | 30.251.106.11   |  |
> |  |  |
> |  |
> +-+--+---+--
> +--+-+--+
> | 0:0:0:0:0:0:0:1 | 1: DF9FD2A4(@n0) | Server| Windows Server 2012 R2
> amd64 6.3 | 4| 00:00:00:00:00:00:00:E0 | 0.00 %   |
> | 10.2.0.163  |  |
> |  |  | 00:50:56:25:00:B7
> |  |
> | 127.0.0.1   |  |
> |  |  |
> |  |
> | 30.251.106.199  |  |
> |  |  |
> |  |
> +-+--+---+--
> +--+-+--+
> | 0:0:0:0:0:0:0:1 | 1: 965AD7CD(@n1) | Server| Windows Server 2012 R2
> amd64 6.3 | 4| 00:00:00:00:00:00:00:E0 | 0.00 %   |
> | 10.2.0.252  |  |
> |  |  | 00:50:56:25:00:BB
> |  |
> | 127.0.0.1   |  |
> |  |  |
> |  |
> | 30.251.106.90   |  |
> |  |  |
> |  |
> +---
> +
>
> REQ_002 2018-04-05 23:08:05.259 [async-worker-2] ERROR
> com.xxx.backend.async.AsyncExecutor - Failed to lock keys (all partition
> nodes left the grid).
> org.apache.ignite.cache.CacheServerNotFoundException: Failed to lock keys
> (all partition nodes left the grid).
> at org.apache.ignite.internal.processors.cache.GridCacheUtils.
> convertToCacheException(GridCacheUtils.java:1282)
> ~[ignite-core-2.4.0.jar:2.4.0]
> at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.
> cacheException(IgniteCacheProxyImpl.java:1673)
> ~[ignite-core-2.4.0.jar:2.4.0]
> at org.apache.ignite.internal.processors.cache.
> IgniteCacheProxyImpl.get(IgniteCacheProxyImpl.java:852)
> ~[ignite-core-2.4.0.jar:2.4.0]
> at org.apache.ignite.internal.processors.cache.
> GatewayProtectedCacheProxy.get(GatewayProtectedCacheProxy.java:676)
> ~[ignite-core-2.4.0.jar:2.4.0]
> at 
> com.xxx.backend.cache.impl.DurableCacheIgnite.storeRefreshToken(DurableCacheIgnite.java:268)
> ~[lk.backend-1.0.0.jar:?]
> at 
> com.xxx.cache.impl.DurableCacheIgnite$$FastClassBySpringCGLIB$$fa2c0515.invoke()
> ~[lk.backend-1.0.0.jar:?]
> at 
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> ~[spring-core-5.0.4.RELEASE.jar:5.0.4.RELEASE]
> at org.springframework.aop.framework.CglibAopProxy$
> CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:747)
> ~

Failed to lock keys (all partition nodes left the grid)

[Olexandr K]

Hi team,

I'm getting strange exception when trying to save Key/Value pair.
When I tested locally everything was fine.
Now I deployed application to servers cluster and stuck with this error.

What does it mean? My both Ignite server nodes are UP.

Here is configuration for this cache:









Here is my topology

visor> top
Hosts: 4
+===+
|  Int./Ext. IPs  |   Node ID8(@)| Node Type |
OS| CPUs |  MACs   | CPU Load |
+===+
| 0:0:0:0:0:0:0:1 | 1: 35CD3AD1(@n2) | Client| Windows Server 2012 R2
amd64 6.3 | 4| 00:00:00:00:00:00:00:E0 | 0.00 %   |
| 10.2.0.225  |  |
|  |  | 00:50:56:25:00:B9
|  |
| 127.0.0.1   |  |
|  |  |
|  |
| 30.251.106.197  |  |
|  |  |
|  |
+-+--+---+--+--+-+--+
| 0:0:0:0:0:0:0:1 | 1: 230F83B8(@n3) | Client| Windows Server 2012 R2
amd64 6.3 | 4| 00:00:00:00:00:00:00:E0 | 0.13 %   |
| 10.2.0.250  |  |
|  |  | 00:50:56:25:00:35
|  |
| 127.0.0.1   |  |
|  |  |
|  |
| 30.251.106.11   |  |
|  |  |
|  |
+-+--+---+--+--+-+--+
| 0:0:0:0:0:0:0:1 | 1: DF9FD2A4(@n0) | Server| Windows Server 2012 R2
amd64 6.3 | 4| 00:00:00:00:00:00:00:E0 | 0.00 %   |
| 10.2.0.163  |  |
|  |  | 00:50:56:25:00:B7
|  |
| 127.0.0.1   |  |
|  |  |
|  |
| 30.251.106.199  |  |
|  |  |
|  |
+-+--+---+--+--+-+--+
| 0:0:0:0:0:0:0:1 | 1: 965AD7CD(@n1) | Server| Windows Server 2012 R2
amd64 6.3 | 4| 00:00:00:00:00:00:00:E0 | 0.00 %   |
| 10.2.0.252  |  |
|  |  | 00:50:56:25:00:BB
|  |
| 127.0.0.1   |  |
|  |  |
|  |
| 30.251.106.90   |  |
|  |  |
|  |
+---+

REQ_002 2018-04-05 23:08:05.259 [async-worker-2] ERROR
com.xxx.backend.async.AsyncExecutor - Failed to lock keys (all partition
nodes left the grid).
org.apache.ignite.cache.CacheServerNotFoundException: Failed to lock keys
(all partition nodes left the grid).
at
org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1282)
~[ignite-core-2.4.0.jar:2.4.0]
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:1673)
~[ignite-core-2.4.0.jar:2.4.0]
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.get(IgniteCacheProxyImpl.java:852)
~[ignite-core-2.4.0.jar:2.4.0]
at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.get(GatewayProtectedCacheProxy.java:676)
~[ignite-core-2.4.0.jar:2.4.0]
at
com.xxx.backend.cache.impl.DurableCacheIgnite.storeRefreshToken(DurableCacheIgnite.java:268)
~[lk.backend-1.0.0.jar:?]
at
com.xxx.cache.impl.DurableCacheIgnite$$FastClassBySpringCGLIB$$fa2c0515.invoke()
~[lk.backend-1.0.0.jar:?]
at
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
~[spring-core-5.0.4.RELEASE.jar:5.0.4.RELEASE]
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:747)
~[spring-aop-5.0.4.RELEASE.jar:5.0.4.RELEASE]
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
~[spring-aop-5.0.4.RELEASE.jar:5.0.4.RELEASE]
at
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
~[spring-aop-5.0.4.RELEASE.jar:5.0.4.RELEASE]
at
com.xxx.backend.logging.ComponentLogger.logTimeMethod(ComponentLogger.java:34)
~[lk.backend-1.0.0.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_162]
   

Re: Slow invoke call

[javastuff....@gmail.com]

Thanks Pavel for reply.

/"When you store entries into the off-heap, any update operation requires
copying value from the off-heap to the heap." /
I am updating using remote entry processor, does that also need to copy
value to calling heap? If yes then no benefit using entry processor here, I
can fetch and put, probably with a lock.

Why does invoking remote method is taking 50% of time? Is it because of
concurrency and entry processor will execute under a lock internally? 

Coping existing and incoming bytes, must be generating a lot for GC.

Is there any better way to deal with this usecase? right now it seems slower
than DB updates.

Thanks,
-Sam



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite Sql Query Execution time is too long

[begineer]

Have you tried adding indexes to frequently searched fields ? Indexing can
make searching the cache quite faster



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Lucene CorruptIndexException (checksum failed) on GridLuceneIndex - suggested patch

[Ilya Kasnacheev]

Hello Manu!

Can you also provide a test that will show that the problem exists, and
that the fix does something helpful? Preferably in the form of code, but
"steps to reproduce" may work too.

Other than that, I recommend you creating a pull request for
https://github.com/apache/ignite/
and a JIRA ticket with description of the problem in
https://issues.apache.org/jira/projects/IGNITE/issues
and link them together :)

This will increase the chance that this patch is merged into Ignite source.

For further discussion I recommend writing on Ignite developer list, and
not the User list we're on.

Regards,

-- 
Ilya Kasnacheev

2018-04-05 20:17 GMT+03:00 Manu :

> Hi,
>
> *GridLuceneOutputStream* has a bug on */copyBytes/* method and
> *GridLuceneInputStream* on */readBytes/* method for direct calls from
> GridLuceneOutputStream.
>
> On both methods internal GridLuceneOutputStream's CRC is not updated, so we
> get  /org.apache.lucene.index.CorruptIndexException: checksum failed
> (hardware problem?) [...]/ when the use of lucene index is intensive and
> lucene internally try to merge it.
>
> Suggested patch to fix CorruptIndexException on GridLuceneIndex
>  FIX-IGNITE-LUCENE-STREAM-CRC.patch>
>
> Hope it helps!!
>
> Bye!
>
> Manu
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Cache access based on the user previleges

[vbm]

Hi,

Is there any configuration with which we can restrict access to a cache
based on user privileges ?
In a production environment, how does access restriction to different cache
are enforced ?


Regards,
Vishwas




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Lucene CorruptIndexException (checksum failed) on GridLuceneIndex - suggested patch

[Manu]

Hi,

*GridLuceneOutputStream* has a bug on */copyBytes/* method and
*GridLuceneInputStream* on */readBytes/* method for direct calls from
GridLuceneOutputStream.

On both methods internal GridLuceneOutputStream's CRC is not updated, so we
get  /org.apache.lucene.index.CorruptIndexException: checksum failed
(hardware problem?) [...]/ when the use of lucene index is intensive and
lucene internally try to merge it.

Suggested patch to fix CorruptIndexException on GridLuceneIndex

  

Hope it helps!!

Bye!

Manu



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Ignite Sql Query Execution time is too long

[siva]

Hi,

We have 2 nodes(client&server) and using ignite native persistence.
In cache we have around 3lakh records.

We are querying from Client node and using cursor.getAll() to get the
resultset.

If we execute a query which needs to returns 10k records (ex:select * from
Entity Where EntityType='Ignite') its giving an exception
javax.cache.CacheException: Failed to run map query remotely. 


Another query will return less records "select organization from Entity
Where EntityType='Spark' "
its taking too much time to execute the query. queryExecution.PNG
  


How can we overcome this problem?





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Using Apache Ignite with Zookeeper

[Denis Mekhanikov]

You should open the port, specified as a *localPort* in *TcpDiscoverySpi *
configuration.
The same for communication SPI and other protocols, that are used in your
cluster, like JDBC, REST, etc.

The only difference in Zookeeper-based discovery is that Zookeeper is used
to find other nodes on the network.
So, only IP finder is different. Discovery protocol works the same way as
with *TcpDiscoveryVmIpFinder *or any other IP finder.

Denis

чт, 5 апр. 2018 г. в 14:09, Roman Shtykh :

> You can specify the port your zk is running on as shown in [1], and it
> should be opened
>
>
> https://apacheignite.readme.io/docs/cluster-config#zookeeper-based-discovery
>
> Roman
>
>
> On Thursday, April 5, 2018, 4:48:05 p.m. GMT+9, monalsinha <
> monalsi...@gmail.com> wrote:
>
>
> I am using Apche Ignite with zookeeper based discovery using curator
> framework.
> Do I need to open any port on my ignite node like TCP based discovery, in
> order for this to work?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Ignite Single Node vs Cluster of Nodes

[Denis Mekhanikov]

Hi!

If you add more nodes to your cluster, you'll get performance boost,
because queries
will be executed in a distributed manner.
But this boost will be noticeable only starting from some particular data
volume, because
distributed query execution is associated with some overhead.

You will also get higher durability, if you configure backups for
partitions.
In this case you will be protected from data loss, when some nodes are
failing.

Denis

чт, 5 апр. 2018 г. в 10:43, kotamrajuyashasvi :

> Hi
>
> I would like to know the added advantages of using Cluster of Nodes instead
> of a single node(single node is sufficient for my data volume) in my use
> case . All the operations are sql queries. And if I use a cluster of nodes,
> the result set of a query will always be present on a single node(due to
> affinitykey set. We cannot change the affinitykey as we also perform join
> queries. Even join queries result set also will be on a single node). Also
> if we did use cluster of nodes we will not be using more than 10 nodes.
> Will
> there be any advantage if we use cluster of nodes instead of a single node?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Upgrade from 2.1.0 to 2.4.0 resulting in error within transaction block

[Vladimir Ozerov]

I've just fixed possible root cause in master [1]. However, as exact use
case details is not known, may be it was something else.
Is it possible to provide more info on the use case: cache configuratioh,
model classes?

[1] https://issues.apache.org/jira/browse/IGNITE-8147

On Mon, Apr 2, 2018 at 12:21 PM, Yakov Zhdanov  wrote:

> Cross posting to dev.
>
> Vladimir Ozerov, can you please take a look at NPE from query processor
> (see below - GridQueryProcessor.typeByValue(GridQueryProcessor.java:1901)
> )?
>
> --Yakov
>
> 2018-03-29 0:19 GMT+03:00 smurphy :
>
>> Code works in Ignite 2.1.0. Upgrading to 2.4.0 produces the stack trace
>> below. The delete statement that is causing the error is:
>>
>> SqlFieldsQuery sqlQuery = new SqlFieldsQuery("delete from EngineFragment
>> where " + criteria());
>> fragmentCache.query(sqlQuery.setArgs(criteria.getArgs()));
>>
>> The code above is called from within a transactional block managed by a
>> PlatformTransactionManager which is in turn managed by Spring's
>> ChainedTransactionManager. If the @Transactional annotation is removed
>> from
>> the surrounding code, then the code works ok...
>>
>> 2018-03-28 15:50:05,748 WARN  [engine 127.0.0.1] progress_monitor_2
>> unknown
>> unknown {ProgressMonitorImpl.java:112} - Scan
>> [ec7af5e8-a773-40fd-9722-f81103de73dc] is unable to process!
>> javax.cache.CacheException: Failed to process key '247002'
>> at
>> org.apache.ignite.internal.processors.cache.IgniteCacheProxy
>> Impl.query(IgniteCacheProxyImpl.java:618)
>> at
>> org.apache.ignite.internal.processors.cache.IgniteCacheProxy
>> Impl.query(IgniteCacheProxyImpl.java:557)
>> at
>> org.apache.ignite.internal.processors.cache.GatewayProtected
>> CacheProxy.query(GatewayProtectedCacheProxy.java:382)
>> at
>> com.company.core.dao.ignite.IgniteFragmentDao.delete(IgniteF
>> ragmentDao.java:143)
>> at
>> com.company.core.dao.ignite.IgniteFragmentDao$$FastClassBySp
>> ringCGLIB$$c520aa1b.invoke()
>> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodPro
>> xy.java:204)
>> at
>> org.springframework.aop.framework.CglibAopProxy$CglibMethodI
>> nvocation.invokeJoinpoint(CglibAopProxy.java:720)
>> at
>> org.springframework.aop.framework.ReflectiveMethodInvocation
>> .proceed(ReflectiveMethodInvocation.java:157)
>> at
>> org.springframework.dao.support.PersistenceExceptionTranslat
>> ionInterceptor.invoke(PersistenceExceptionTranslationInterce
>> ptor.java:136)
>> at
>> org.springframework.aop.framework.ReflectiveMethodInvocation
>> .proceed(ReflectiveMethodInvocation.java:179)
>> at
>> org.springframework.aop.framework.CglibAopProxy$DynamicAdvis
>> edInterceptor.intercept(CglibAopProxy.java:655)
>> at
>> com.company.core.dao.ignite.IgniteFragmentDao$$EnhancerBySpr
>> ingCGLIB$$ce60f71c.delete()
>> at
>> com.company.core.core.service.impl.InternalScanServiceImpl.p
>> urgeScanFromGrid(InternalScanServiceImpl.java:455)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>> ssorImpl.java:62)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>> thodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>> org.springframework.aop.support.AopUtils.invokeJoinpointUsin
>> gReflection(AopUtils.java:302)
>> at
>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(
>> JdkDynamicAopProxy.java:202)
>> at com.sun.proxy.$Proxy417.purgeScanFromGrid(Unknown Source)
>> at com.company.core.core.async.tasks.PurgeTask.process(PurgeTas
>> k.java:85)
>> at sun.reflect.GeneratedMethodAccessor197.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>> thodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>> org.springframework.aop.support.AopUtils.invokeJoinpointUsin
>> gReflection(AopUtils.java:302)
>> at
>> org.springframework.aop.framework.ReflectiveMethodInvocation
>> .invokeJoinpoint(ReflectiveMethodInvocation.java:190)
>> at
>> org.springframework.aop.framework.ReflectiveMethodInvocation
>> .proceed(ReflectiveMethodInvocation.java:157)
>> at
>> org.springframework.transaction.interceptor.TransactionInter
>> ceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
>> at
>> org.springframework.transaction.interceptor.TransactionAspec
>> tSupport.invokeWithinTransaction(TransactionAspectSupport.java:281)
>> at
>> org.springframework.transaction.interceptor.TransactionInter
>> ceptor.invoke(TransactionInterceptor.java:96)
>> at
>> org.springframework.aop.framework.ReflectiveMethodInvocation
>> .proceed(ReflectiveMethodInvocation.java:179)
>> at
>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(
>> JdkDynamicAopProxy.

Re: Running heavy queries on Ignite cluster on backing store directly without impacting the cluster

[Andrey Mashenkov]

Hi,

>So are you saying, query execution will not have any impact on the cluster
>activities like GET/PUTs in general ??
Why? Queries and get\put share same resources. So, they will affect each
other.

>I was thinking, if we can run these queries directly on backing store, it
>may not have any impact on RAM from where we always do GETs, will that
work?
When you use CacheStore, you have some external DB and you can run query on
DB bypassing Ignite. It is clear.

1. What "backing store" do you mean when Ignite persistence is used?
2. How are you imagine query will be run on "backing store" directly?
3.Why do you think it will help you as anyway disk pressure will be
increased? Moreover, disk is usually a bottleneck for PUT operations and
for GET in some cases (load from disk or TTL update).

On Tue, Apr 3, 2018 at 9:27 AM, Naveen  wrote:

> Hi ANdrew
>
> There were cases, when I just run select * from table on SQLLINE
> unknowingly, we could see queries getting slowed down and OOM errors. Our
> dev machines not very high end ones.
>
> When we deliver this solution, production support guys can run queries to
> debug any data related issues, during which they may need to join tables,
> for every adhoc query we cant create indexes to speedup the query execution
> right ??
>
> So are you saying, query execution will not have any impact on the cluster
> activities like GET/PUTs in general ??
>
> I was thinking, if we can run these queries directly on backing store, it
> may not have any impact on RAM from where we always do GETs, will that
> work?
>
> Thanks
> Naveen
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov

Re: Using Apache Ignite with Zookeeper

[Roman Shtykh]

You can specify the port your zk is running on as shown in [1], and it should 
be opened

https://apacheignite.readme.io/docs/cluster-config#zookeeper-based-discovery
Roman
 

On Thursday, April 5, 2018, 4:48:05 p.m. GMT+9, monalsinha 
 wrote:  
 
 I am using Apche Ignite with zookeeper based discovery using curator
framework.
Do I need to open any port on my ignite node like TCP based discovery, in
order for this to work?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
  

Using Apache Ignite with Zookeeper

[monalsinha]

I am using Apche Ignite with zookeeper based discovery using curator
framework.
Do I need to open any port on my ignite node like TCP based discovery, in
order for this to work?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Ignite Single Node vs Cluster of Nodes

[kotamrajuyashasvi]

Hi

I would like to know the added advantages of using Cluster of Nodes instead
of a single node(single node is sufficient for my data volume) in my use
case . All the operations are sql queries. And if I use a cluster of nodes,
the result set of a query will always be present on a single node(due to
affinitykey set. We cannot change the affinitykey as we also perform join
queries. Even join queries result set also will be on a single node). Also
if we did use cluster of nodes we will not be using more than 10 nodes. Will
there be any advantage if we use cluster of nodes instead of a single node?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: ClusterTopologyServerNotFoundException

[Pavel Vinokurov]

Hi Venkat,

Could you setup *constistentId* (IgniteConfiguration#setConsistentId)  for
3 server nodes and add these ids to the baseline topology
using ignite.cluster().setBaselineTopology(nodes);

Please setup the baseline and activate the cluster as described in
https://apacheignite.readme.io/docs/cluster-activation.

Thanks,
Pavel

2018-04-01 15:58 GMT+03:00 kvenkatramtreddy :

> Hi,
>
> Thank you very much.
>
> We are receiving this error only when one node was running. Exception is
> gone as soon other nodes are started and joined the cluster.
>
> I have attached the single node where we received the error.
>
> exception.log
> 
>
> Thanks & Regards,
> Venkat
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 

Regards

Pavel Vinokurov