Query execution is too long

[siva]

Hi,
I have .Net Client And Server App using Ignite V2.9.1.

I have cache Name COMMONFORMSSCHEMA and cache created using Model class.And
contains json data in ```Schema``` column.Even i have tried without select
```Schema``` Column in query taking more time.

here is cache details,query and model class details.
*Model class*

public class CommonFormsSchema
{
[QuerySqlField]
public string SchemaId { get; set; }
[QuerySqlField]
public string SchemaName { get; set; }
[QuerySqlField]
public string Schema { get; set; }
[QuerySqlField]
public string Parent { get; set; }
[QuerySqlField]
public int Version { get; set; }
[QuerySqlField]
public string Tenant { get; set; }
[QuerySqlField]
public string Application { get; set; }
[QuerySqlField]
public string PageType { get; set; }
[QuerySqlField]
public string Page { get; set; }
[QuerySqlField]
public string CategoryType { get; set; }
[QuerySqlField]
public bool favorite { get; set; }
[QuerySqlField]
public string filename { get; set; }
[QuerySqlField]
public bool isDeleted { get; set; }
[QuerySqlField]
public string CreatedBy { get; set; }
[QuerySqlField]
public string ModifiedBy { get; set; }
[QuerySqlField]
public DateTime _CreatedDateTime { get; set; }
public DateTime CreatedDateTime
{
get { return _CreatedDateTime; }
set
{
_CreatedDateTime = DateTime.SpecifyKind(value != null ?
value : DateTime.UtcNow, DateTimeKind.Utc);
}
}

[QuerySqlField]
public DateTime _ModifiedDateTime { get; set; }
public DateTime ModifiedDateTime
{
get { return _ModifiedDateTime; }
set
{
_ModifiedDateTime = DateTime.SpecifyKind(value != null ?
value : DateTime.UtcNow, DateTimeKind.Utc);
}
}
public CommonFormsSchema()
{
ModifiedDateTime = DateTime.SpecifyKind(DateTime.Now,
DateTimeKind.Utc);
CreatedDateTime = DateTime.SpecifyKind(DateTime.Now,
DateTimeKind.Utc);
}
}


*Cache details*

var primaryKeysIndex = new string[] { "SchemaId" };
var queryEntity = new QueryEntity(typeof(string),
typeof(CommonFormsSchema)) {
Indexes = new
List(primaryKeysIndex.Count() + 1)
{
new QueryIndex(true,0,primaryKeysIndex)
}
};
var queryList = new List { queryEntity };

var cacheName = "COMMONFORMSSCHEMA";
bool cacheMetrics = false;

var cacheCfg = new CacheConfiguration(cacheName)
{
Name = cacheName,
CacheStoreFactory = new
ConstructionCommonFormSchemaCacheStoreFactory(_logger, connectionString),
KeepBinaryInStore = false,  // Cache store works
with deserialized data.
ReadThrough = true,
WriteThrough = true,
QueryEntities = queryList,
DataRegionName = "IgniteDataRegion",
EvictionPolicy = new LruEvictionPolicy
{
MaxSize = 100
},
EnableStatistics = cacheMetrics
};

// Create cache with given name, if it does not exist.
ICache cache =
Ignite.GetOrCreateCache(cacheCfg);

*Query:*

SELECT SchemaId, SchemaName,SCHEMA ,Parent, Application, _ModifiedDateTime
as ModifiedDateTime, Version, PageType, favorite, 
CategoryType, Tenant, IsDeleted FROM "COMMONFORMSSCHEMA".CommonFormsSchema 
WHERE SchemaId in (SELECT DISTINCT SchemaId from
"COMMONFORMSSCHEMA".CommonFormsSchema WHERE Tenant = 'constructiontest'
UNION SELECT DISTINCT SchemaId FROM "COMMONFORMSSCHEMA".CommonFormsSchema
WHERE SchemaName NOT IN 
(SELECT DISTINCT SchemaName FROM "COMMONFORMSSCHEMA".CommonFormsSchema WHERE
Tenant = 'constructiontest') AND (Tenant = 'All' OR Tenant IS NULL))

*Ignite console Warnings:*


[10:20:21,728][WARNING][long-qry-#40%ServerNode%][LongRunningQueryManager]
Query execution is too long [duration=48294ms, type=MAP,
distributedJoin=false, enforceJoinOrder=false, lazy=false, schema=PUBLIC,
sql='SELECT
__Z0.SCHEMAID __C0_0,
__Z0.SCHEMANAME __C0_1,
__Z0.SCHEMA __C0_2,
__Z0.PARENT __C0_3,
__Z0.APPLICATION __C0_4,
__Z0._MODIFIEDDATETIME __C0_5,
__Z0.VERSION __C0_6,
__Z0.PAGETYPE __C0_7,
__Z0.FAVORITE __C0_8,
__Z0.CATEGORYTYPE __C0_9,
__Z0.TENANT __C0_10,
__Z0.ISDELETED __C0_11
FROM COMMONFORMSSCHEMA.COMMONFORMSSCHEMA __Z

Re: 2.8.1 : INFO org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi [] - Accepted incoming communication connection

[VeenaMithare]

Hi Ilya,

Please find the reproducer project attached. 

Steps to reproduce : 
1. Run the server.java 
2. Run the ListenerClient.java. Wait till it starts fully.
3. Stop and start the client again ( within the failuredetectiontimeout of
10 secs. )

You will see these logs on the console.

Another thing I noticed today is that , the logs seem to be connected to
this code in Server.java
ignite.active(true);
addPersistentCacheConfiguration(
ignite);

If I comment out the above lines and try the steps above, I dont get those
logs.

regards,
Veena. ClientConnectProject.zip

  







--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: not able to change keyspace name when Using 3rd Party Persistence (Cassandra) Together with Ignite Native Persistence

[Ilya Kasnacheev]

Hello!

Ignite will use existing cache configuration if cache is already present.
There's nothing you can do here. This includes the cache store.

However, the cache store may determine what to keep in its serialized form,
and what to re-read/infer every time it is started. So as I have said,
you may improve Cassandra cache store to track environment changes.

Regards,

-- 
Ilya Kasnacheev


вт, 5 янв. 2021 г. в 04:38, xmw45688 :

> Hi Ilya,
>
> Thanks for your guidance and happy new year!  Sorry for late catchup.
>
> You are right, Ignite Native Persistence does track the changes in class
> definition for "Ignite Native Persistence".  But the configuration to load
> data from Cache to Cassandra store is stored in xml configuration file.
> This configuration xml file is passed at runtime (see the example
> configuration below). When ignite server is stopped, the cache
> configuration
> is gone, not stored in the any place as far as I understand.  When the
> Ignite is restarted, the new xml is passed, there is no previous
> configuration for the old class definition in the xml configuration.
>
> My question - how/where does the ignite server get/read the old class
> definition if this class definition is provided in the most recent xml
> config file?
>
> So what I said is that the Cassandra Store implementation may not need to
> change.  It's the call that reads the xml configuration and stores the data
> in Cassandra Store via Ignite Cache.   If I remove Ignite Native
> configuration ("persistenceEnabled" value="false"), then Ignite server uses
> the xml configuration file passed at runtime.  The Ignite Cache has all new
> class definiton.
>
> I'd like your help to guide me make the changes to read the xml
> configuration at runtime instead of reading the previously cached
> configuration.
>
>
> 
> class="org.apache.ignite.cache.store.cassandra.persistence.KeyValuePersistenceSettings">
> 
> 
> 
> 
> 
>
>  class="org.apache.ignite.configuration.DataStorageConfiguration">
>
> 
>  class="org.apache.ignite.configuration.DataRegionConfiguration">
>  value="Default_Region"/>
> 
>
> 
>
>
>  value="true"/>
> 
> 
> 
>
> 
> 
> 
> 
> 
> 
> 
> 
> 
>  type="java.lang.Class">java.lang.String
>  type="java.lang.Class">com.procurant.catalog.entity.Uom
> 
> 
> 
>  class="org.apache.ignite.cache.store.cassandra.CassandraCacheStoreFactory">
>  ref="cassandra"/>
>  value="catalog_uom_persistence_settings">
> 
> 
> 
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Requesting mapping from grid failed issue with Ignite 2.9.0 and C# model with ICloneable interface but same was working with Ignite 2.8.1

[Pavel Tupitsyn]

Hi Charlin,

I've filed a bug [1]

However, it looks like [QuerySqlField] attributes are missing on
TestModel29WithICloneable properties.
To make your model queryable with SQL/LINQ, make sure to mark some or all
properties like this:

public class TestModel29WithICloneable : ICloneable, IBinarizable
{
[QuerySqlField]
public string TestField1 { get; set; }

[QuerySqlField]
public string TestField2 { get; set; }

[QuerySqlField]
public string TestField3 { get; set; }
...
}

This will get rid of the exception (which is still a bug, but for a weird
use case when no SQL fields are defined).

Thanks,
Pavel

[1] https://issues.apache.org/jira/browse/IGNITE-13957

On Tue, Jan 5, 2021 at 10:28 AM Ilya Kazakov  wrote:

> Hello, Charlin!
>
> Try to enable peer class loading:
> https://ignite.apache.org/docs/latest/code-deployment/peer-class-loading
>
> --
> Ilya Kazakov
>
> вт, 5 янв. 2021 г. в 14:52, Charlin S :
>
>> Hi,
>>
>> i'm getting exception on when a new record is added, which was working
>> till ignite 2.8.1
>> This issue seems to be with Ignite 2.9.0 and Ignite 2.9.1
>> The c# code is below:
>> void Main()
>> {
>> A a = new A();
>> }
>>
>> public class A
>> {
>> public A()
>> {
>>
>>  IgniteConfiguration igniteGridIg = new IgniteConfiguration();
>> igniteGridIg.AutoGenerateIgniteInstanceName = true;
>> igniteGridIg.IgniteHome =
>> @"D:\Software\apache-ignite-2.9.1-bin";
>> igniteGridIg.SpringConfigUrl =
>> Path.Combine(@"D:\IgniteConfig\",
>> "common_dynamiccache_client_config_2.9.1.xml");
>> igniteGridIg.ConsistentId =
>> Guid.NewGuid().ToString().ToUpper();
>>
>> IIgnite StaticGrid_Dev = Ignition.Start(igniteGridIg);
>> TestModel29WithICloneable model = new TestModel29WithICloneable();
>>
>> model.TestField1 = "11";
>> model.TestField2 = "22";
>>
>> ICache
>> TestModel29WithICloneableICache=null;
>>CacheConfiguration cgTest = new
>> CacheConfiguration("TestModel29WithICloneable", new
>> QueryEntity(typeof(string), typeof(TestModel29WithICloneable)));
>> cgTest.CopyOnRead = false;
>> cgTest.EagerTtl = true;
>> cgTest.Backups = 1;
>>
>> var cacheName = StaticGrid_Dev.GetOrCreateCache> object>(cgTest).WithExpiryPolicy(new ExpiryPolicy(
>>  TimeSpan.FromSeconds(3600),
>>  TimeSpan.FromSeconds(3600),
>>  TimeSpan.FromSeconds(3600)
>>  ));
>>
>> cacheName.Put("TestModel29WithICloneable:Test|0100010test2", model);
>>
>> //Console.Write(cacheName);
>> Ignition.StopAll(true);
>> }
>> }
>>
>>
>> Model class:
>> public class TestModel29WithICloneable : ICloneable, IBinarizable
>> {
>> public TestModel29WithICloneable Copy()
>> {
>> return (TestModel29WithICloneable)this.MemberwiseClone();
>> }
>>
>> public object Clone()
>> {
>> var clone = this.MemberwiseClone();
>> return clone;
>> }
>> public string TestField1 { get; set; }
>> public string TestField2 { get; set; }
>> public string TestField3 { get; set; }
>>
>> public void ReadBinary(IBinaryReader reader)
>> {
>> if (reader != null)
>> {
>> TestField1 = reader.ReadString("testfield1");
>> TestField2 = reader.ReadString("testfield2");
>> TestField3 = reader.ReadString("testfield3");
>> }
>> }
>>
>> public void WriteBinary(IBinaryWriter writer)
>> {
>> if (writer != null)
>> {
>> writer.WriteString("testfield1", TestField1);
>> writer.WriteString("testfield2", TestField2);
>> writer.WriteString("testfield3", TestField3);
>> }
>> }
>> }
>>
>> Result :
>> javax.cache.CacheException: class
>> org.apache.ignite.IgniteCheckedException: Requesting mapping from grid
>> failed for [platformId=0, typeId=1876507903]
>>   at
>> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1270)
>>   at
>> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2083)
>>   at
>> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1319)
>>   at
>> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:856)
>>   at
>> org.apache.ignite.internal.processors.platform.cache.PlatformCache.processInStreamOutLong(PlatformCache.java:839)
>>   at
>> org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67)
>> Caused by: class org.apache.ignite.IgniteCheckedException: Requesting
>> mapping from grid failed for [p