Re: problem of using object as key in cache configurations
We already have such examples. For example, CacheClientBinaryPutGetExample. But I agree with Andrey that this message is very confusing. I will fix it shortly. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/problem-of-using-object-as-key-in-cache-configurations-tp4116p4153.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Schema Import Utility - Mismatch when loading data between Date and Timestamp
Thank you, planning a test session next week! -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Schema-Import-Utility-Mismatch-when-loading-data-between-Date-and-Timestamp-tp3790p4152.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: problem of using object as key in cache configurations
I think ignite should add example codes or document to tell the beginners. I did search the forum but no results. but the error log message did give me hint on where the problem might be occurred (in my case, the object serialization part). -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/problem-of-using-object-as-key-in-cache-configurations-tp4116p4151.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: problem of using object as key in cache configurations
Hello,A few teams at our company have independently run into this issue and ended up each wasting time figuring out how to fix it. And this is not the first time I see people asking this very same question here. I wonder if it is possible to improve diagnostics in Ignite code and log a more helpful message?ThanksAndrey _ From: kevin.zheng Sent: Wednesday, April 13, 2016 6:15 PM Subject: Re: problem of using object as key in cache configurations To: Finally, I found the solution: I need to use "cache.withKeepBinary()" to call the query method when the cache Key is an object type. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/problem-of-using-object-as-key-in-cache-configurations-tp4116p4146.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Schema Import Utility - Mismatch when loading data between Date and Timestamp
Hello. Issue with loading of Date data https://issues.apache.org/jira/browse/IGNITE-2936 is closed. It will be available at master branch or in next nightly build. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Schema-Import-Utility-Mismatch-when-loading-data-between-Date-and-Timestamp-tp3790p4149.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: org.apache.ignite.IgniteCheckedException: Failed to register query type: TypeDescriptor
Hello. Issue with error in index generation https://issues.apache.org/jira/browse/IGNITE-2856 is closed. It will be available at master branch or in next nightly build. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/org-apache-ignite-IgniteCheckedException-Failed-to-register-query-type-TypeDescriptor-tp3447p4148.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: re: querying, sql performance of apache ignite
Read-through works for key-based access (methods like get, remove, invoke, etc.), but not for SQL queries. That's because Ignite doesn't know what data it will need for the query prior to running it. But you can preload it manually. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/querying-sql-performance-of-apache-ignite-tp4135p4147.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: problem of using object as key in cache configurations
Finally, I found the solution: I need to use "cache.withKeepBinary()" to call the query method when the cache Key is an object type. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/problem-of-using-object-as-key-in-cache-configurations-tp4116p4146.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
re: querying, sql performance of apache ignite
Hi Val, If " SQL queries only data that is already in memory ", what is the purpose of setReadThrough(true) in the cache configuration settings? I thought the cache would load data automatically, if no exist in the cache. Regards, Kevin -邮件原件- 发件人: vkulichenko [mailto:valentin.kuliche...@gmail.com] 发送时间: 2016年4月14日 5:24 收件人: user@ignite.apache.org 主题: Re: querying, sql performance of apache ignite Hi, 1. This is correct, current implementation of SQL queries only data that is already in memory. This is enough for majority of use cases, because usually in-memory systems deal with operational data sets (i.e. subsets of "hot", most frequently accessed data) and can safely ignore the rest of the data. For example, you can maintain the data for the last several days and discard historical data. If you have a different case, let us know. 2. IgniteCache is a key-value storage, but it's up to you how to map objects to relational tables. This is fully defined by CacheStore interface [1] and you can always provide your own implementation. Also take a look at [2] for information about bulk loading the data. There are more efficient ways then using get() method. [1] https://apacheignite.readme.io/docs/persistent-store [2] https://apacheignite.readme.io/docs/data-loading -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/querying-sql-performance-of-apache-ignite-tp4135p4143.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: multiple value key
Hi, If you're querying value by key, it's better to use get() method instead of query. The key can be either a primitive, or any Java object. In the latter case don't forget to properly implement equals() and hashCode() methods. With queries you can do any other more complicated requests. Ignite uses standard SQL, so all your examples should work. See [1] for more details. [1] https://apacheignite.readme.io/docs/sql-queries -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/multiple-value-key-tp4138p4144.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: querying, sql performance of apache ignite
Hi, 1. This is correct, current implementation of SQL queries only data that is already in memory. This is enough for majority of use cases, because usually in-memory systems deal with operational data sets (i.e. subsets of "hot", most frequently accessed data) and can safely ignore the rest of the data. For example, you can maintain the data for the last several days and discard historical data. If you have a different case, let us know. 2. IgniteCache is a key-value storage, but it's up to you how to map objects to relational tables. This is fully defined by CacheStore interface [1] and you can always provide your own implementation. Also take a look at [2] for information about bulk loading the data. There are more efficient ways then using get() method. [1] https://apacheignite.readme.io/docs/persistent-store [2] https://apacheignite.readme.io/docs/data-loading -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/querying-sql-performance-of-apache-ignite-tp4135p4143.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: executing jar application with ignite server
Hi, You can use Compute Grid [1] to execute custom code on the server nodes. Is that what you're looking for? [1] https://apacheignite.readme.io/docs/compute-grid -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/executing-jar-application-with-ignite-server-tp4129p4142.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite: ERROR: Java classpath is empty
This thread duplicates http://stackoverflow.com/questions/36594113/ignite-error-java-classpath-is-empty -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-ERROR-Java-classpath-is-empty-tp4118p4140.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Unexpected heap usage by GridDhtPartitionTopologyImpl
Hi, You can find the list of commands to build the project here: https://github.com/apache/ignite/blob/master/DEVNOTES.txt Let us know if something doesn't work for you. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Unexpected-heap-usage-by-GridDhtPartitionTopologyImpl-tp3967p4141.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: What does mean caching database exactly ?
Data loading happens on the server (in case you're using CacheStore for this). But someone has to initiate the process by calling loadCache method. It can be done by any of the apps, I believe it's better to do it by one that starts first. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/What-does-mean-caching-database-exactly-tp4043p4139.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
multiple value key
Hello, Is it possible to use multiple value key ? For example, Timestamp, Integer, Integer ? I know that loading data maybe done using key - get method for example. But how to implement load with many value key ? Now, I simply write query: ... WHERE key = 42. However, for multiple key: WHERE key= 42 && timestamp ='2001-12...' && k2 = 21. What about situation when I give only two parameters, for example WHERE key= 42 && timestamp ='2001-12...' . Moreover, is it possible to get all rows with timestamp later than year 2000 ? -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/multiple-value-key-tp4138.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: how to check whether server and client connected?
Hello Ravi, Why are you not using Ignite client () node for access to cluster? Client node special created for using as gateway: doesn't sores data and doesn't compute job by default. On Wed, Apr 13, 2016 at 6:01 PM, Ravi kumar Puri wrote: > How do i connect to client with my server . Tell me something related to > port or host adrzs > On 13-Apr-2016 17:46, "Alexei Scherbakov" > wrote: > >> Hi, >> >> Every Ignite node works either in server or client mode. >> Server nodes actually contain data and perform grid tasks, while client >> nodes provide API for working with server nodes remotely. >> Refer to https://apacheignite.readme.io/docs/clients-vs-servers for >> details. >> >> Note: where is a notion of "near" cache, which may be useful on client >> nodes for speeding up access to frequently accessed data. >> Refer to https://apacheignite.readme.io/docs/near-caches for details. >> >> 2016-04-13 12:25 GMT+03:00 Ravi Puri : >> >>> where to configure the client and server to interact. >>> >>> Ignition.setClientMode(true); >>> >>> this has been set but how to access local and remote nodes and how to >>> start >>> caching the data remotely available? >>> >>> >>> >>> -- >>> View this message in context: >>> http://apache-ignite-users.70518.x6.nabble.com/how-to-check-whether-server-and-client-connected-tp4117.html >>> Sent from the Apache Ignite Users mailing list archive at Nabble.com. >>> >> >> >> >> -- >> >> Best regards, >> Alexei Scherbakov >> >
Re: How to load 2 tables in a cache
OK, UPDATE:
This method is working, but how do I add setArgs (I mean, I want to pass the
argument for orgname as "Ignite" i.e. to only print person names in Ignite
organization. How to do that??? It's giving error if I simply do as given on
the ignite website:
private static void sqlFieldsQueryWithJoin() {
IgniteCache, Person> cache =
Ignition.ignite().cache("PCache");
//String oname = "Barclays";
// Execute query to get names of all employees.
String sql =
"select concat(firstName, ' ', lastName), org.orgname " +
"from Person, \"" + ORG_CACHE + "\".Organization as org " +
"where Person.orgid = org.orgid";
QueryCursor> cursor = cache.query(new
SqlFieldsQuery(sql));
// In this particular case each row will have one element with full
name of an employees.
List> res = cursor.getAll();
// Print persons' names and organizations' names.
print("Names of all employees and organizations they belong to:",
res);
}
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/How-to-load-2-tables-in-a-cache-tp4026p4136.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
querying, sql performance of apache ignite
Hi, I consider some issues about cache in apache-ignite. 1. Firstly, from what you said me, we can query apache-ignite using SQL. However, then results will be only from cache RAM. So, what about situation when table from database doesn't fit into RAM. Then SELECT * on ignite returns only the rows that are loaded into cache. Am I ok ? 2. We always must set key-value. Now, key should be the same as in my database ? Then, before doing select sql we can use get method to load into cache needed rows. Am I ok ? -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/querying-sql-performance-of-apache-ignite-tp4135.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Data Grid with write through mode to HDFS with layer of IGFS
Vij, No, it doesn't. IGFS serves the very different purpose - it is Hadoop-compatible file system. It means that, for example, you can load data to IGFS and then query it using Hive. But native Ignite SQL is not applicable here. Vladimir. On Wed, Apr 13, 2016 at 3:55 PM, vijayendra bhati wrote: > Thanks Vladimir, > > I have not gone through complete documentation but if you could let me > know does IGFS provide SQL support like Ignite cache does ? > > Regards, > Vij > > > On Wednesday, April 13, 2016 5:54 PM, Vladimir Ozerov < > voze...@gridgain.com> wrote: > > > Hi Vijayendra, > > IGFS is designed to be a distributed file system which could cache data > from Hadoop file systems. It cannot be used as cache store by design. > Ignite doesn't have store implementation for HDFS, so you should implement > your own if needed. Particularly, you should implement > org.apache.ignite.cache.store.CacheStore interface. > > Vladimir. > > On Wed, Apr 13, 2016 at 2:38 PM, vijayendra bhati > wrote: > > Hi, > > Can some body please provide me any pointers regarding how I can use > Ignite Data Grid/ In Memory caching with write through/write behind mode > and writing to HDFS ? > > I know Ignite provides IGFS but its different from what I am looking for. > > The other way could be I can use IGFS as my In Memory store but is it the > right approach ? > > Regards, > Vijayendra Bhati > > > > >
Re: How to load 2 tables in a cache
Could this error be because I haven't done setIndexedType?: cfg1.setIndexedTypes(OrganizationKey.class, Organization.class); OR because I have same column name "orgid" in both the tables?? -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-load-2-tables-in-a-cache-tp4026p4133.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: how to check whether server and client connected?
How do i connect to client with my server . Tell me something related to port or host adrzs On 13-Apr-2016 17:46, "Alexei Scherbakov" wrote: > Hi, > > Every Ignite node works either in server or client mode. > Server nodes actually contain data and perform grid tasks, while client > nodes provide API for working with server nodes remotely. > Refer to https://apacheignite.readme.io/docs/clients-vs-servers for > details. > > Note: where is a notion of "near" cache, which may be useful on client > nodes for speeding up access to frequently accessed data. > Refer to https://apacheignite.readme.io/docs/near-caches for details. > > 2016-04-13 12:25 GMT+03:00 Ravi Puri : > >> where to configure the client and server to interact. >> >> Ignition.setClientMode(true); >> >> this has been set but how to access local and remote nodes and how to >> start >> caching the data remotely available? >> >> >> >> -- >> View this message in context: >> http://apache-ignite-users.70518.x6.nabble.com/how-to-check-whether-server-and-client-connected-tp4117.html >> Sent from the Apache Ignite Users mailing list archive at Nabble.com. >> > > > > -- > > Best regards, > Alexei Scherbakov >
Re: link cache to object class?
So i need to define jst d cache name in configuratn n nothng else as their attributes .how can i wil be able to relate to class .. even i dnr need any bean or class path/package in bean form to get name in ignite.xml configuration On 13-Apr-2016 17:36, "Denis Magda" wrote: > Give a cache a name that is related to an objects simple class name. > > As an example, you have a class “Person”. Create the cache with the same > name and get a reference to it using ignite.cache(“Person”) > > — > Denis > > > On Apr 13, 2016, at 1:35 PM, Ravi Puri > wrote: > > > > how can i link the cache created in configuration to the object class so > that > > i can decide which cache is for which class? > > > > > > > > -- > > View this message in context: > http://apache-ignite-users.70518.x6.nabble.com/link-cache-to-object-class-tp4119.html > > Sent from the Apache Ignite Users mailing list archive at Nabble.com. > >
Re: Host Arrays in C++ API
Sergi, Can you please take a look? Here is the query: SELECT sid_per_id, sid_mah_id, sid_itm_id, sid_prm_id, sid_cha_code, sid_service, sid_itm_dist FROM SHOP_ITEM_BITMAP_CACHE.ShopItemBitmap as bitmap JOIN PROD_HIER_CACHE.ProdHier as prod ON prod.GHD_LEV_6 = bitmap.SID_ITM_ID AND prod.GHD_HIE_ID = 0 AND prod.ghd_lev_2_desc = 'WINE' JOIN MRKT_POD_ACV_CACHE.MarketPodAcv as mkt ON mkt.CHLD_MRKT_KEY = bitmap.SID_MAH_ID and mkt.MRKT_KEY = 4378038 AND mkt.prd_id = 712 WHERE bitmap.SID_PER_ID = 712; Here is the explain output: [[SELECT DISTINCT SID_PER_ID AS __C0, SID_MAH_ID AS __C1, SID_ITM_ID AS __C2, SID_PRM_ID AS __C3, SID_CHA_CODE AS __C4, SID_SERVICE AS __C5, SID_ITM_DIST AS __C6 FROM PROD_HIER_CACHE.PRODHIER PROD /* PROD_HIER_CACHE."ghd_hie_id_asc_ghd_lev_2_desc_asc_idx": GHD_HIE_ID = 0 AND GHD_LEV_2_DESC = 'WINE' */ /* WHERE (PROD.GHD_HIE_ID = 0) AND (PROD.GHD_LEV_2_DESC = 'WINE') */ INNER JOIN SHOP_ITEM_BITMAP_CACHE.SHOPITEMBITMAP /* SHOP_ITEM_BITMAP_CACHE."sid_itm_id_asc_idx": SID_ITM_ID = PROD.GHD_LEV_6 */ ON 1=1 /* WHERE (SHOPITEMBITMAP.SID_PER_ID = 712) AND (PROD.GHD_LEV_6 = SHOPITEMBITMAP.SID_ITM_ID) */ INNER JOIN MARKET_POD_ACV_CACHE.MARKETPODACV MKT /* MARKET_POD_ACV_CACHE."mrkt_key_asc_prd_id_asc_chld_mrkt_key_asc_idx": PRD_ID = 712 AND MRKT_KEY = 4378038 AND CHLD_MRKT_KEY = SHOPITEMBITMAP.SID_MAH_ID */ ON 1=1 WHERE (SHOPITEMBITMAP.SID_PER_ID = 712) AND (((MKT.PRD_ID = 712) AND ((MKT.MRKT_KEY = 4378038) AND (MKT.CHLD_MRKT_KEY = SHOPITEMBITMAP.SID_MAH_ID))) AND ((PROD.GHD_LEV_6 = SHOPITEMBITMAP.SID_ITM_ID) AND ((PROD.GHD_HIE_ID = 0) AND (PROD.GHD_LEV_2_DESC = 'WINE'], [SELECT DISTINCT __C0 AS SID_PER_ID, __C1 AS SID_MAH_ID, __C2 AS SID_ITM_ID, __C3 AS SID_PRM_ID, __C4 AS SID_CHA_CODE, __C5 AS SID_SERVICE, __C6 AS SID_ITM_DIST FROM PUBLIC.__T0 /* SHOP_ITEM_BITMAP_CACHE."merge_scan" */]] The first thing I could see is that there is no index for "sid_per_id" and the second thing is that "merge_scan" is being used. What is your thoughts? Best Regards, Igor On Tue, Apr 12, 2016 at 2:11 PM, Igor Sapego wrote: > Arthi, > > How much time does it take? What is the speed of fetching in rows > per second? > > Guys, can someone who is more familiar with Ignite's SQL queries > take a look? > > Best Regards, > Igor > > On Mon, Apr 11, 2016 at 5:17 PM, arthi > wrote: > >> Hi Igor, >> >> the SQL query for getNext() takes a whole lot of time when the results >> returned is more than 900,000 both using Java and C++ API. I have tried a >> number of configurations with indexes, but just dont get to speed these >> up. >> >> thanks, >> Arthi >> >> >> >> -- >> View this message in context: >> http://apache-ignite-users.70518.x6.nabble.com/Host-Arrays-in-C-API-tp3707p4064.html >> Sent from the Apache Ignite Users mailing list archive at Nabble.com. >> > >
executing jar application with ignite server
Hello, Maybe, I don't understand something, but how to execute program on server node ? I mean something like spark-submit. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/executing-jar-application-with-ignite-server-tp4129.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: How to load 2 tables in a cache
CacheConfig class:
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.ignite.organization;
import java.sql.*;
import java.util.*;
import org.apache.ignite.cache.*;
import org.apache.ignite.cache.store.jdbc.*;
import org.apache.ignite.configuration.*;
/**
* CacheConfig definition.
*
* Code generated by Apache Ignite Schema Import utility: 04/11/2016.
*/
public class CacheConfig {
/**
* Create JDBC type for organization.
*
* @param cacheName Cache name.
* @return Configured JDBC type.
*/
private static JdbcType jdbcTypeOrganization(String cacheName) {
JdbcType jdbcType = new JdbcType();
jdbcType.setCacheName(cacheName);
jdbcType.setDatabaseSchema("ORG");
jdbcType.setDatabaseTable("organization");
jdbcType.setKeyType("org.apache.ignite.organization.OrganizationKey");
jdbcType.setValueType("org.apache.ignite.organization.Organization");
// Key fields for organization.
Collection keys = new ArrayList<>();
keys.add(new JdbcTypeField(Types.INTEGER, "orgId", int.class,
"orgid"));
jdbcType.setKeyFields(keys.toArray(new JdbcTypeField[keys.size()]));
// Value fields for organization.
Collection vals = new ArrayList<>();
vals.add(new JdbcTypeField(Types.INTEGER, "orgId", int.class,
"orgid"));
vals.add(new JdbcTypeField(Types.VARCHAR, "orgname", String.class,
"orgname"));
jdbcType.setValueFields(vals.toArray(new
JdbcTypeField[vals.size()]));
return jdbcType;
}
/**
* Create SQL Query descriptor for organization.
*
* @return Configured query entity.
*/
private static QueryEntity queryEntityOrganization() {
QueryEntity qryEntity = new QueryEntity();
qryEntity.setKeyType("org.apache.ignite.organization.OrganizationKey");
qryEntity.setValueType("org.apache.ignite.organization.Organization");
// Query fields for organization.
LinkedHashMap fields = new LinkedHashMap<>();
fields.put("orgid", "int");
fields.put("orgname", "String");
qryEntity.setFields(fields);
// Indexes for organization.
Collection idxs = new ArrayList<>();
idxs.add(new QueryIndex("orgId", true, "PRIMARY"));
qryEntity.setIndexes(idxs);
return qryEntity;
}
/**
* Create JDBC type for person.
*
* @param cacheName Cache name.
* @return Configured JDBC type.
*/
private static JdbcType jdbcTypePerson(String cacheName) {
JdbcType jdbcType = new JdbcType();
jdbcType.setCacheName(cacheName);
jdbcType.setDatabaseSchema("ORG");
jdbcType.setDatabaseTable("person");
jdbcType.setKeyType("org.apache.ignite.organization.PersonKey");
jdbcType.setValueType("org.apache.ignite.organization.Person");
// Key fields for person.
Collection keys = new ArrayList<>();
keys.add(new JdbcTypeField(Types.INTEGER, "id", int.class, "id"));
jdbcType.setKeyFields(keys.toArray(new JdbcTypeField[keys.size()]));
// Value fields for person.
Collection vals = new ArrayList<>();
vals.add(new JdbcTypeField(Types.INTEGER, "id", int.class, "id"));
vals.add(new JdbcTypeField(Types.VARCHAR, "first_name",
String.class, "firstName"));
vals.add(new JdbcTypeField(Types.VARCHAR, "last_name", String.class,
"lastName"));
vals.add(new JdbcTypeField(Types.INTEGER, "orgId", Integer.class,
"orgid"));
jdbcType.setValueFields(vals.toArray(new
JdbcTypeField[vals.size()]));
return jdbcType;
}
/**
* Create SQL Query descriptor for person.
*
* @return Configured query entity.
*/
private static QueryEntity queryEntityPerson() {
QueryEntity qryEntity = new QueryEntity();
qryEntity.setKeyType("org.apache.ignite.organization.PersonKey");
qryEntity.setValueType("org.apache.ignite.organization.Person");
// Query fields for person.
LinkedHashMap fields = new LinkedHashMap<>();
fields.put("id", "int");
fields.put("firstName", "String");
fields.put("lastName", "String");
f
Re: Ignite Data Grid with write through mode to HDFS with layer of IGFS
Thanks Vladimir, I have not gone through complete documentation but if you could let me know does IGFS provide SQL support like Ignite cache does ? Regards,Vij On Wednesday, April 13, 2016 5:54 PM, Vladimir Ozerov wrote: Hi Vijayendra, IGFS is designed to be a distributed file system which could cache data from Hadoop file systems. It cannot be used as cache store by design.Ignite doesn't have store implementation for HDFS, so you should implement your own if needed. Particularly, you should implement org.apache.ignite.cache.store.CacheStore interface. Vladimir. On Wed, Apr 13, 2016 at 2:38 PM, vijayendra bhati wrote: Hi, Can some body please provide me any pointers regarding how I can use Ignite Data Grid/ In Memory caching with write through/write behind mode and writing to HDFS ? I know Ignite provides IGFS but its different from what I am looking for. The other way could be I can use IGFS as my In Memory store but is it the right approach ? Regards,Vijayendra Bhati
Re: spark and ignite - possibilities
IGFS can works with thriftserver without HDFS? -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/spark-and-ignite-possibilities-tp4055p4126.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Data Grid with write through mode to HDFS with layer of IGFS
Hi Vijayendra, IGFS is designed to be a distributed file system which could cache data from Hadoop file systems. It cannot be used as cache store by design. Ignite doesn't have store implementation for HDFS, so you should implement your own if needed. Particularly, you should implement org.apache.ignite.cache.store.CacheStore interface. Vladimir. On Wed, Apr 13, 2016 at 2:38 PM, vijayendra bhati wrote: > Hi, > > Can some body please provide me any pointers regarding how I can use > Ignite Data Grid/ In Memory caching with write through/write behind mode > and writing to HDFS ? > > I know Ignite provides IGFS but its different from what I am looking for. > > The other way could be I can use IGFS as my In Memory store but is it the > right approach ? > > Regards, > Vijayendra Bhati >
Re: how to check whether server and client connected?
Hi, Every Ignite node works either in server or client mode. Server nodes actually contain data and perform grid tasks, while client nodes provide API for working with server nodes remotely. Refer to https://apacheignite.readme.io/docs/clients-vs-servers for details. Note: where is a notion of "near" cache, which may be useful on client nodes for speeding up access to frequently accessed data. Refer to https://apacheignite.readme.io/docs/near-caches for details. 2016-04-13 12:25 GMT+03:00 Ravi Puri : > where to configure the client and server to interact. > > Ignition.setClientMode(true); > > this has been set but how to access local and remote nodes and how to start > caching the data remotely available? > > > > -- > View this message in context: > http://apache-ignite-users.70518.x6.nabble.com/how-to-check-whether-server-and-client-connected-tp4117.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. > -- Best regards, Alexei Scherbakov
Re: link cache to object class?
Give a cache a name that is related to an objects simple class name. As an example, you have a class “Person”. Create the cache with the same name and get a reference to it using ignite.cache(“Person”) — Denis > On Apr 13, 2016, at 1:35 PM, Ravi Puri wrote: > > how can i link the cache created in configuration to the object class so that > i can decide which cache is for which class? > > > > -- > View this message in context: > http://apache-ignite-users.70518.x6.nabble.com/link-cache-to-object-class-tp4119.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Ignite Data Grid with write through mode to HDFS with layer of IGFS
Hi, Can some body please provide me any pointers regarding how I can use Ignite Data Grid/ In Memory caching with write through/write behind mode and writing to HDFS ? I know Ignite provides IGFS but its different from what I am looking for. The other way could be I can use IGFS as my In Memory store but is it the right approach ? Regards,Vijayendra Bhati
Re: Unexpected heap usage by GridDhtPartitionTopologyImpl
Hi, Currently we are using 1.5.0.final version, since we have an urgency with our timings , we decided to change the source of 1.5.0.final. But i cant find a developer documentation, can you please share the link. A short brief also can be helpful. Thanks. On Fri, Apr 8, 2016 at 2:56 PM, Tolga Kavukcu wrote: > Thanks for the response, we are following the issue. After its done we can > keep in touch. > > Regards. > > On Fri, Apr 8, 2016 at 12:56 PM, Yakov Zhdanov > wrote: > >> Guys, I like the idea of separate pool. But, please note that >> write-behind pool may be slower and may not be able to flush all cache >> updates to DB. We will have to force system threads to help with this. >> >> Tolga, I know that Nick is currently working on async cache callbacks and >> he will be introducing a new pool to system. I think you will be able to >> use it. Ticket for tracking is >> https://issues.apache.org/jira/browse/IGNITE-2004 >> >> For now you can start reviewing >> - org.apache.ignite.internal.processors.cache.store.GridCacheWriteBehindStore >> I think we will need to refactor it in the way we move flusher threads >> logic to execute in separate pool instead of dedicated threads we have now. >> >> --Yakov >> >> 2016-04-08 11:11 GMT+03:00 Tolga Kavukcu : >> >>> Hi Denis, >>> >>> Yes we don't need to have expiration policy, so setting >>> CacheConfiguraiton.setEagerTtl to false solved this problem. >>> >>> So we are testing the whole system, we also found out that using a cache >>> with writeBehind enabled causes a new thread creation for each one. So >>> if we think about future plans and possibilities, it's not a best practice >>> to have increasing number of threads within jvm. >>> >>> We wonder that if there is a option to use a thread pool for writeBehind >>> jobs. If there is not, we could implement it for the community. So if you >>> can guide us where to start, i would be glad :) >>> >>> Thanks. >>> >>> On Fri, Apr 8, 2016 at 1:54 AM, Denis Magda wrote: >>> Tolga, The cleanup threads "ttl-cleanup-workers" are used to eagerly remove expired cache entries. Expiration policy can be set either a cache wide in CacheConfiguration or can be used later with cache.withExpirePolicy(...) calls. I failed to reproduce your case. What I've done is started 30 caches and destroyed all of them later. Visual VM showed that all "ttl-cleanup-workers" were stopped successfully. What Ignite version do you use? In any case if you are not planing to use expiration policy you can set CacheConfiguraiton.setEagerTtl to false and the ttl workers Threads won't be created at all. Regards, Denis On 4/7/2016 3:43 PM, Tolga Kavukcu wrote: Hi Denis, IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE parameter seems like decreased heap usage. I will run longer tests to check heap behaviour. Also i need another help with thread's created by ignite. I found out that ignite creates a cleanup thread named "ttl-cleanup-worker" for each cache. But when cache is destroyed, clean up thread does not deleted. Instead it waits sleeping state at all. My first question is that , is it possible to decrease thread count with a configuration, like "thread pool with x threads" for all caches. Secondly, is "unremoved threads" are expected behaviour. Thanks. On Thu, Apr 7, 2016 at 2:40 PM, Denis Magda wrote: > Hi Tolga, > > GridDhtPartitionTopologyImpl is created per cache. If you destroy a > cache this object should be GCed. However you should use cache.destroy() > for that. > > Please also make sure that you make "live set" heap dumps only. Try to > perform GC explicitly before making the dump because a collector may clean > dead objects much later depending on its heuristics. > > -- > Denis > > On 4/7/2016 8:27 AM, Tolga Kavukcu wrote: > > Hi Denis, > > Thanks for the response. I will try IGNITE_ > ATOMIC_CACHE_DELETE_HISTORY_SIZE parameter. The screnshots taken from > eclipse memory analyser which opens and analyses heap dump. I understand > heap requirement for wrapping and indexing off-heap entry positions. But > also found out that instances of > *org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl > *is constantly increasing within jvm. > > > I also create and destroy so many small caches during the lifecycle, > do you think that it is possible to destroyed caches leaves a footprint in > heap. > > The previous scrreenshots was dominator tree view of memory analyser. > I attached again with headers. > > You can see that each of GridDhtPartitionTopologyImpl uses 20mb~ > heap. And there are 72 instances of GridDhtPartitionTopologyImpl living. > > Also i attached screenshots
Re: spark and ignite - possibilities
Hi Tomk, IGFS is essentially a caching layer which could be used on top of any Hadoop file system. If ThriftServer performs heave data processing, then it is definitely make sense to try IGFS. BTW, please properly subscribe to the user list (this way we will not have to manually approve your emails). All you need to do is send an email to “ user-subscr...@ignite.apache.org” and follow simple instructions in the reply. Vladimir. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/spark-and-ignite-possibilities-tp4055p4120.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
link cache to object class?
how can i link the cache created in configuration to the object class so that i can decide which cache is for which class? -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/link-cache-to-object-class-tp4119.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
how to check whether server and client connected?
where to configure the client and server to interact. Ignition.setClientMode(true); this has been set but how to access local and remote nodes and how to start caching the data remotely available? -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/how-to-check-whether-server-and-client-connected-tp4117.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
problem of using object as key in cache configurations
Dear all,I have a table which has no primary index (need to use two columns as composite index).at the beginning, I use only one column as key to load all the data into the cache. However, lots of records were ignored ( because they have the same key).Then I create an java object to store two column data as a key in cache configuration object. now I can load all the data into the cache. However, when I test the query function. I got exceptions info as follows?Exception in thread "main" class org.apache.ignite.binary.BinaryObjectException: Failed resolve class for ID: 524756802 at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:499) at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1443) at org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:537) at org.apache.ignite.internal.binary.BinaryObjectImpl.value(BinaryObjectImpl.java:117) at org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinary(CacheObjectContext.java:280) at org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinaries(CacheObjectContext.java:228) at org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinariesIfNeeded(CacheObjectContext.java:165) at org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinariesIfNeeded(CacheObjectContext.java:154) at org.apache.ignite.internal.processors.cache.GridCacheContext.unwrapBinariesIfNeeded(GridCacheContext.java:1737) at org.apache.ignite.internal.processors.query.GridQueryCacheObjectsIterator.next(GridQueryCacheObjectsIterator.java:64) at org.apache.ignite.internal.processors.query.GridQueryCacheObjectsIterator.next(GridQueryCacheObjectsIterator.java:29) at org.apache.ignite.internal.processors.cache.QueryCursorImpl.getAll(QueryCursorImpl.java:73) at org.apache.ignite.examples.datagrid.store.auto.CacheAutoStoreExample_new.simpleQuery(CacheAutoStoreExample_new.java:544) at org.apache.ignite.examples.datagrid.store.auto.CacheAutoStoreExample_new.composedIndexDemo(CacheAutoStoreExample_new.java:246) at org.apache.ignite.examples.datagrid.store.auto.CacheAutoStoreExample_new.main(CacheAutoStoreExample_new.java:213)Caused by: class org.apache.ignite.IgniteCheckedException: Failed to read class name from file [id=524756802, file=D:\ignite-master\work\marshaller\524756802.classname] at org.apache.ignite.internal.MarshallerContextImpl.className(MarshallerContextImpl.java:159) at org.apache.ignite.internal.MarshallerContextAdapter.getClass(MarshallerContextAdapter.java:174) at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:483) ... 14 moreCaused by: java.io.FileNotFoundException: D:\ignite-master\work\marshaller\524756802.classnameany idea what was wrong?best regards,Kevin -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/problem-of-using-object-as-key-in-cache-configurations-tp4116.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: What does mean caching database exactly ?
Yes, firstly I am going to loadCache contain of database. Then, I am going to write newly data (into cache and db - write throught). Loader app will be only read data from cache (not from db). Thanks to read-through it will see new data. Tell me please, what exactly I should say in .xml configure file ? only name for cache ? Writer and reader are reader, server is running by ./ignite.sh -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/What-does-mean-caching-database-exactly-tp4043p4115.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: visor keep trigger alert when condition no longer satisfied?
thanks Vasiliy!! -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/visor-keep-trigger-alert-when-condition-no-longer-satisfied-tp3772p4114.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
