Confusing Binary Marshaller and EntryProcessor

[张鹏鹏]

First,I can't run the 'Modifying Binary Objects' example in section Binary
Marshaller.
I must copy the jar contain the Person Class to the libs fold belong the
Server node.
But that is against Binary Marshaller's advantages.In the Doc,it said I
don't need to copy key and value classes everywhere.Then I try to turn
 on PeerClassLoadingEnabled,It works.But When I change some code and
restart the client,the server side don't refresh.Do I misunderstand
something in the Doc?

I have another question about generics.I find out that withKeepBinary
return
IgniteCache,
so I can't write

IgniteCache cache = ignite.getOrCreateCache(cacheCfg);

cache.withKeepBinary().invoke(i,(entry, arguments)-> {

BinaryObjectBuilder bldr = entry.getValue().toBuilder();//can't compile
bldr.setField("name", "name");
entry.setValue(bldr.build());
return null;
});

entry.getValue() lost its type!

I must code this way:

IgniteCache cache = ignite.getOrCreateCache(cacheCfg);
IgniteCache binaryCache = cache.withKeepBinary();

binaryCache.invoke(...)

Re: Hibernate connection inspite of jdbc?

[Ravi Puri]

yes its pointing the correct folder. Still facing this issue. I think there
is problem with hibernate open sesssion which is not mentioned anywhere in
program?

Session hibSes = ses.attachment();


n mostly it throws error in this statement
 cache.loadCache(null, 1);

it should go to loadcache function of CacheHibernatePersonStore class but it
never hits the function. it automatically throws error.. please tell me what
is the statement about cache.loadcache()???





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Hibernate-connection-inspite-of-jdbc-tp3412p3679.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Hibernate loadcache error?

[Ravi Puri]

still its not working . i am facing the same issue again and again.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Hibernate-loadcache-error-tp3534p3680.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Help with Ignite Zeppelin Demo

[Murthy Kakarlamudi]

Hi,
I attended Turbocharge SQL on Ignite webinar earlier in the week and
wanted to get familiar with the examples. I downloaded the code from
https://github.com/vkulichenko/zeppelin-demo and tried to run Query.java
after loading the data. I am getting the below error running Query 2.
Also I did not understand the part where in Load.java why the cache name
was set to null while creating the cache to load Persons.

 try (IgniteDataStreamer, Person> personStreamer =
ignite.dataStreamer(null)) { - Line 67 in Load.java

14:51:49] Ignite node started OK (id=f2e7030f)
[14:51:49] Topology snapshot [ver=4, servers=1, clients=1, CPUs=4,
heap=3.5GB]
500
50
Query 1:
Time: 293
Time: 71
Time: 65
Query 2:
[14:51:50] Ignite node stopped OK [uptime=00:00:00:848]
Exception in thread "main" javax.cache.CacheException: class
org.apache.ignite.IgniteException: Failed to bind parameters: [qry=SELECT
o.name as Organization, avg(p.salary) as Salary FROM Person p,
"Organizations".Organization o WHERE p.orgId = o.id AND p.managerId is null
GROUP BY o.name ORDER BY p.salary LIMIT 100, params=[10]]
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:656)
at org.apache.ignite.zeppelin.Query.query(Query.java:58)
at org.apache.ignite.zeppelin.Query.main(Query.java:50)
Caused by: class org.apache.ignite.IgniteException: Failed to bind
parameters: [qry=SELECT o.name as Organization, avg(p.salary) as Salary
FROM Person p, "Organizations".Organization o WHERE p.orgId = o.id AND
p.managerId is null GROUP BY o.name ORDER BY p.salary LIMIT 100,
params=[10]]
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.queryTwoStep(GridQueryProcessor.java:806)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:647)
... 2 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to bind
parameters: [qry=SELECT o.name as Organization, avg(p.salary) as Salary
FROM Person p, "Organizations".Organization o WHERE p.orgId = o.id AND
p.managerId is null GROUP BY o.name ORDER BY p.salary LIMIT 100,
params=[10]]
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:1782)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.queryTwoStep(GridQueryProcessor.java:799)
... 3 more
Caused by: javax.cache.CacheException: Failed to bind parameters:
[qry=SELECT o.name as Organization, avg(p.salary) as Salary FROM Person p,
"Organizations".Organization o WHERE p.orgId = o.id AND p.managerId is null
GROUP BY o.name ORDER BY p.salary LIMIT 100, params=[10]]
at
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.queryTwoStep(IgniteH2Indexing.java:1050)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:801)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:799)
at
org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:1764)
... 4 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to bind
parameter [idx=1, obj=10, stmt=prep5: SELECT o.name as Organization,
avg(p.salary) as Salary FROM Person p, "Organizations".Organization o WHERE
p.orgId = o.id AND p.managerId is null GROUP BY o.name ORDER BY p.salary
LIMIT 100]
at
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.bindObject(IgniteH2Indexing.java:489)
at
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.bindParameters(IgniteH2Indexing.java:914)
at
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.queryTwoStep(IgniteH2Indexing.java:1047)
... 8 more
Caused by: org.h2.jdbc.JdbcSQLException: Invalid value "1" for parameter
"parameterIndex" [90008-175]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:332)
at org.h2.message.DbException.get(DbException.java:172)
at org.h2.message.DbException.getInvalidValueException(DbException.java:218)
at
org.h2.jdbc.JdbcPreparedStatement.setParameter(JdbcPreparedStatement.java:1338)
at
org.h2.jdbc.JdbcPreparedStatement.setObject(JdbcPreparedStatement.java:451)
at
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.bindObject(IgniteH2Indexing.java:486)
... 10 more

Thanks,
Murthy.

Ignite Cache - High Memory Overhead caused by GridCircularBuffer$Item instances

[abarbaro]

Hi!

We're seeing Ignite take a significant amount of memory on a per cache basis
(about 10 MB per cache), even when the caches have no entries at all, and
would like to understand how to reduce this footprint.

Before getting into the technical details, I would like to explain our use
case:  we're considering using Ignite's distributed cache for caching
entities in our J2EE application (basically beans and other data that does
not change very often).  This is kind of similar to Hibernate L2 Cache.  We
have approximately 200 different types of beans, and each bean type
typically has a few thousand instances.  One huge benefit of Ignite is that
we can use SQL queries against it, which have much lower latency than going
to the database layer.  So, we're thinking about this structure:
- One cache per bean type.
- The entries in a given cache will be of type Entry where the
key (of type Long) is the ID of the bean, and the value (of type T) is a
bean instance.
We expect the total heap footprint of all these caches together to be in the
neighborhood of 100MB or so.

The issue/problem at hand is that when we create 100 caches Ignite is
consuming over 1GB of heap, even without having any entries on the caches
themselves.  This is prohibitive for our intended use.  The bulk of the
memory is taken by org.apache.ignite.internal.util.GridCircularBuffer$Item.

I have attached the code we're using for this test, and the config file for
the Ignite server node so these conditions can be easily reproduced.  I'm
also attaching some pictures showing the memory allocation and the heap over
time.

We'd appreciate any suggestions and comments.  Thanks in advance.


CacheMemoryTest.java

  

default-config.xml
  


 


 





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cache-High-Memory-Overhead-caused-by-GridCircularBuffer-Item-instances-tp3682.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Hibernate loadcache error?

[vkulichenko]

Ravi,

Can you provide the exact steps that will reproduce the issue?

Also please check that if you set IGNITE_HOME environment variable or system
property. If you do have it, make sure that it points to the correct folder
or just remove it.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Hibernate-loadcache-error-tp3534p3683.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Help with Ignite Zeppelin Demo

[vkulichenko]

Hi Murthy,

The error fails because the argument is set, but the query doesn't expect
any. Actually, the Query class was created only for testing purposes and
apparently was broken at some point. This project is created to run all the
queries from Zeppelin to demonstrate its integration with Ignite.

'null' is the valid name for a cache. In case of SQL it's treated as the
default schema and is used in the demo for simplicity.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Help-with-Ignite-Zeppelin-Demo-tp3681p3684.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Help with Ignite Zeppelin Demo

[Murthy Kakarlamudi]

Ahh..Thanks for the response. Yeah, in looking closely I got the same
impression and was trying to run the code removing the Arguments.

In any case my application needs to query data that was stored in cache
using sql. What will be a good starting point to look at some sample code?
I downloaded ignite-examples from githud and started looking into those.

Also in going through the docs there was a mention that h2 console can be
used to query objects in cache. Not able to find much documentation around
how to bring up h2 console. Any pointers will be much appreciated.

Thanks,
Murthy.

On Fri, Mar 25, 2016 at 4:19 PM, vkulichenko 
wrote:

> Hi Murthy,
>
> The error fails because the argument is set, but the query doesn't expect
> any. Actually, the Query class was created only for testing purposes and
> apparently was broken at some point. This project is created to run all the
> queries from Zeppelin to demonstrate its integration with Ignite.
>
> 'null' is the valid name for a cache. In case of SQL it's treated as the
> default schema and is used in the demo for simplicity.
>
> -Val
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Help-with-Ignite-Zeppelin-Demo-tp3681p3684.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Ignite Cache - High Memory Overhead caused by GridCircularBuffer$Item instances

[vkulichenko]

Hi,

You can decrease the size of this buffer by setting the system property
(default value is 20): -DIGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE=5

Also you can decrease the number of partitions by adding this to the
CacheConfiguraiton:







Note that to guarantee even data distribution, number of partitions should
be at least order of magnitude higher than number of nodes in topology.

Please try to play with these parameters and let us know if it helps.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cache-High-Memory-Overhead-caused-by-GridCircularBuffer-Item-instances-tp3682p3686.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Ignite terminology

[alexGalushka]

Hello Ignite community!

I read through the Ignite docs a few times and was playing with Ignite for a
few weeks. 
But I still can't figure out the Ignite terminology...

What are definitions and relations of the following: grid, grid instance,
ignite cluster (cluster), ignite node (node)?
I feel like in the Ignite context grid and cluster are interchangeably
terms, but still not 100% sure about it.

How to make the nodes belong to the same cluster (/grid, if the term is
interchangeable)? Setting the same gridName seems to do it: 

Ignition.ignite("MY_GRID_NAME").cluster().nodes()... 

I see 2 nodes in the collection and with the correct node IDs.

But I'm confused by the .ignite(...) java doc, which says: "Note that caller
of this method should not assume that it will return the same instance every
time. Grid name defines what grid a particular grid instance (and
correspondingly its node) belongs to"

--Alexander.







--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-terminology-tp3687.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Confusing Binary Marshaller and EntryProcessor

[vkulichenko]

Hi,

You don't need the Person class to be deployed, but you need entry processor
implementation to be available on the server nodes. You can either
explicitly add it on classpath or use peer class loading.

As for the compilation error, you are correct and I just updated the docs.
You can do like this:

cache.withKeepBinary().invoke(...);

-Val 



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Confusing-Binary-Marshaller-and-EntryProcessor-tp3678p3688.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Help with Ignite Zeppelin Demo

[vkulichenko]

Murthy,

SQL query documentation [1] and query example [2] are the best starting
points for using Ignite query API.

To start the console simply set 'IGNITE_H2_DEBUG_CONSOLE' system property
and it will be automatically opened in the browser. E.g., if you start a
standalone node using ignite.sh, execute it like this:

ignite.sh -J-DIGNITE_H2_DEBUG_CONSOLE

-Val

[1] https://apacheignite.readme.io/docs/sql-queries
[2]
https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/datagrid/CacheQueryExample.java



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Help-with-Ignite-Zeppelin-Demo-tp3681p3689.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Help with Ignite Zeppelin Demo

[Murthy Kakarlamudi]

Great..thanks much. That really helped a lot.

Murthy.

On Fri, Mar 25, 2016 at 5:24 PM, vkulichenko 
wrote:

> Murthy,
>
> SQL query documentation [1] and query example [2] are the best starting
> points for using Ignite query API.
>
> To start the console simply set 'IGNITE_H2_DEBUG_CONSOLE' system property
> and it will be automatically opened in the browser. E.g., if you start a
> standalone node using ignite.sh, execute it like this:
>
> ignite.sh -J-DIGNITE_H2_DEBUG_CONSOLE
>
> -Val
>
> [1] https://apacheignite.readme.io/docs/sql-queries
> [2]
>
> https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/datagrid/CacheQueryExample.java
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Help-with-Ignite-Zeppelin-Demo-tp3681p3689.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Ignite Cache - High Memory Overhead caused by GridCircularBuffer$Item instances

[abarbaro]

Wow.  Outstanding.  Thanks so much for the reply.  It did bring about almost
an order of magnitude improvement!  I tweaked the settings a bit more so I
ended up using 1 (instead of 5) for the delete history.

The new setting for IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE brought down the
total heap to ~420MB (60% reduction from 1GB).  And after applying the
setting for affinity it came down to ~200MB, which is perfectly acceptable. 
I will do a bit more tuning but we're where we want to be in terms of heap
footprint already.

The final question then is, where can I find more documentation about what
these settings do and what they really control so we can understand the
potential side-effects/cons of using these non-default settings.

Again, thanks so much for your prompt reply and help!



 


 



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cache-High-Memory-Overhead-caused-by-GridCircularBuffer-Item-instances-tp3682p3691.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Confusing Binary Marshaller and EntryProcessor

[张鹏鹏]

Thank you for your reply!

Can you explain how the 'peer class loading' runs?I found it just run
once.How can I make other nodes reload the class?




2016-03-26 5:18 GMT+08:00 vkulichenko :

> Hi,
>
> You don't need the Person class to be deployed, but you need entry
> processor
> implementation to be available on the server nodes. You can either
> explicitly add it on classpath or use peer class loading.
>
> As for the compilation error, you are correct and I just updated the docs.
> You can do like this:
>
> cache.withKeepBinary().invoke(...);
>
> -Val
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Confusing-Binary-Marshaller-and-EntryProcessor-tp3678p3688.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>