date:20220419

Re: Ignite query is taking long time even no data in that cache model

2022-04-19 Thread Surinder Mehra

this might help, scroll down to group indexes section:
https://ignite.apache.org/docs/latest/SQL/indexes

On Wed, Apr 20, 2022 at 10:59 AM Charlin S  wrote:

> Hi,
> why my query is having only one index field, it supposed to be two index
> column
>
> /* "TestModel".TESTMODEL_TESTFIELD3_ASC_IDX: TESTFIELD3 = 'EN' */
>
> Thanks & Regards,
> Charlin
>
>
> On Tue, 19 Apr 2022 at 19:34, Surinder Mehra  wrote:
>
>> I may be wrong but as an excercise, can you try group index on these
>> fields please and see if it makes any difference.
>> I would request Apache ignite Dev's to validate it.
>>
>> On Tue, 19 Apr 2022, 19:20 Charlin S,  wrote:
>>
>>> Hi,
>>> EXPLAIN SELECT
>>> TestField1,TestField2,TestField3,TestField4,TestField5
>>>  FROM
>>> TestModel
>>>  WHERE
>>> TestField2 = 'A02'
>>>  AND
>>> TestField3 = 'EN'
>>>
>>> resulview [0]: SELECT
>>> __Z0.TESTFIELD1 AS __C0_0,
>>> __Z0.TESTFIELD2 AS __C0_1,
>>> __Z0.TESTFIELD3 AS __C0_2,
>>> __Z0.TESTFIELD4 AS __C0_3,
>>> __Z0.TESTFIELD5 AS __C0_4
>>> FROM "TestModel".TESTMODEL __Z0
>>> /* "TestModel".TESTMODEL_TESTFIELD3_ASC_IDX: TESTFIELD3 = 'EN' */
>>> WHERE (__Z0.TESTFIELD2 = 'A02')
>>> AND (__Z0.TESTFIELD3 = 'EN')
>>>
>>> resultview[1]: SELECT
>>> __C0_0 AS TESTFIELD1,
>>> __C0_1 AS TESTFIELD2,
>>> __C0_2 AS TESTFIELD3,
>>> __C0_3 AS TESTFIELD4,
>>> __C0_4 AS TESTFIELD5
>>> FROM PUBLIC.__T0
>>> /* "TestModel"."merge_scan" */
>>>
>>> Thanks & Regards,
>>> Charlin
>>>
>>> On Tue, 19 Apr 2022 at 18:04, Surinder Mehra  wrote:
>>>
 Looks correct to me. Can you run explain plain for this query and see
 if it uses index.

 On Tue, 19 Apr 2022, 17:41 Charlin S,  wrote:

> Hi,
> My query details are
> fieldsQuery="SELECT
> TestField1,TestField2,TestField3,TestField4,TestField5
>  FROM
> TestModel
>  WHERE
> TestField2 = 'A02'
>  AND
> TestField2 = 'EN'"
>
> //cache model
>  public class TestModel : IBinarizable
> {
>
> [QuerySqlField(IsIndexed = true)]
> public string TestField1 { get; set; }
> [QuerySqlField(IsIndexed = true)]
> public string TestField2 { get; set; }
> [QuerySqlField(IsIndexed = true)]
> public string TestField3 { get; set; }
> [QuerySqlField]
> public string TestField4 { get; set; }
> [QuerySqlField]
> public decimal? TestField5 { get; set; }
>
> public void ReadBinary(IBinaryReader reader){//implementation}
> public void WriteBinary(IBinaryWriter writer){//implementation}
> }
> implementation
>
> SqlFieldsQuery fieldsQuery = new SqlFieldsQuery(query) { Timeout =
> TimeSpan.FromMilliseconds(1) };
> List list = new List();
> // public ICache IgniteCache { get; set; }
> IFieldsQueryCursor queryCursor =
> IgniteCache.Query(fieldsQuery);
>
> //our implementation
>  queryCursor.Dispose();
>
> Thanks,
> Charlin
>
>
>
> On Mon, 18 Apr 2022 at 13:35, Surinder Mehra 
> wrote:
>
>> Can you please show slow query console log output if it's using index
>> scan or full cache scan.
>> I ran into one scenario where index wasn't used and it ended up
>> scaning whole cache.
>> You can try this locally by using control centre and run explain query
>>
>> On Mon, 18 Apr 2022, 13:08 Charlin S,  wrote:
>>
>>> Hi Ignite team,
>>> We are using Ignite 2.10.0 with 4.6.2 and .Net 5 WebAPI and we have
>>> a 16-nodes(including 2 server nodes) Ignite cluster.
>>> We are facing slowness issues with some particular cache model query
>>> and other models query are fine.
>>>
>>> query type: SqlFieldsQuery
>>> Index: index created for where clause columns.
>>>
>>> Regards,
>>> Charlin
>>>
>>>

Re: BinaryObject Data Can Not Mapping To SQL-Data

2022-04-19 Thread vtchernyi

Hi Tianyue,IMHO fully compilable project is usefull for newbie, while short code snippets are not. You can start a single server cluster and debug code in your IDE, check some suggestions about how it works. Couple of years ago I found such a compilable project describing microservices written by @DenisMagda, it helped a lot. So I hope my post will be usefull.VladimirPSI know out there in China last name is written first, while here in Russia it is written last, so the names are Vladimir or Zhenya. Hope I am correct and your name is Tianyue5:12, 20 апреля 2022 г., y :Hi Stanilovsky,I don't know how to describe my problem to you, but I'm sure there is not an error and the data was successfully inserted but not mapped to SQL-data. Vladimir give me a link https://www.gridgain.com/resources/blog/how-fast-load-large-datasets-apache-ignite-using-key-value-api. I decided to take a look at this link first. Anyway, thanks for your advise and hope can help you in the future.Tianyue Hu,2022/4/20在 2022-04-19 14:50:51，"Zhenya Stanilovsky"  写道：hi !BinaryObjectBuilder oldBuilder = igniteClient.binary().builder(«com.inspur...PubPartitionKeys_1_7»); do you call: oldBuilder.build(); // after ? If so — what this mean ? «data is not mapped to sql» is it error in log or client side or smth ? thanks !Hi, I have had the same experience without sql, using KV API only. My cluster consists of several data nodes and self-written jar application that starts the client node. When started, client node executes mapreduce tasks for data load and processing. The workaround is as follows:1. create POJO on the client node;2. convert it to the binary object;3. on the data node, get binary object over the network and get its builder (obj.toBuilder());4. set some fields, build and put in the cache. The builder on the step 3 seems to be the same as the one on the cluent node. Hope that helps,Vladimir 13:06, 18 апреля 2022 г., y :Hi ,When using binary to insert data, I need to  get an exist BinaryObject/BinaryObjectBuilder  from the database, similar to the code below. 442062c6$3$1803c222cba$Coremail$hty1994712$163.comIf I create a BinaryObjectBuilder directly, inserting binary data does not map to table data. The following code will not throw error, but the data is not mapped to sql. If there is no data in my table at first, how can I insert data?3ecbd8f9$4$1803c222cba$Coremail$hty1994712$163.com   --Отправлено из мобильного приложения Яндекс Почты
 -- Отправлено из мобильного приложения Яндекс Почты

Re: Ignite query is taking long time even no data in that cache model

2022-04-19 Thread Charlin S

Hi,
why my query is having only one index field, it supposed to be two index
column

/* "TestModel".TESTMODEL_TESTFIELD3_ASC_IDX: TESTFIELD3 = 'EN' */

Thanks & Regards,
Charlin


On Tue, 19 Apr 2022 at 19:34, Surinder Mehra  wrote:

> I may be wrong but as an excercise, can you try group index on these
> fields please and see if it makes any difference.
> I would request Apache ignite Dev's to validate it.
>
> On Tue, 19 Apr 2022, 19:20 Charlin S,  wrote:
>
>> Hi,
>> EXPLAIN SELECT
>> TestField1,TestField2,TestField3,TestField4,TestField5
>>  FROM
>> TestModel
>>  WHERE
>> TestField2 = 'A02'
>>  AND
>> TestField3 = 'EN'
>>
>> resulview [0]: SELECT
>> __Z0.TESTFIELD1 AS __C0_0,
>> __Z0.TESTFIELD2 AS __C0_1,
>> __Z0.TESTFIELD3 AS __C0_2,
>> __Z0.TESTFIELD4 AS __C0_3,
>> __Z0.TESTFIELD5 AS __C0_4
>> FROM "TestModel".TESTMODEL __Z0
>> /* "TestModel".TESTMODEL_TESTFIELD3_ASC_IDX: TESTFIELD3 = 'EN' */
>> WHERE (__Z0.TESTFIELD2 = 'A02')
>> AND (__Z0.TESTFIELD3 = 'EN')
>>
>> resultview[1]: SELECT
>> __C0_0 AS TESTFIELD1,
>> __C0_1 AS TESTFIELD2,
>> __C0_2 AS TESTFIELD3,
>> __C0_3 AS TESTFIELD4,
>> __C0_4 AS TESTFIELD5
>> FROM PUBLIC.__T0
>> /* "TestModel"."merge_scan" */
>>
>> Thanks & Regards,
>> Charlin
>>
>> On Tue, 19 Apr 2022 at 18:04, Surinder Mehra  wrote:
>>
>>> Looks correct to me. Can you run explain plain for this query and see if
>>> it uses index.
>>>
>>> On Tue, 19 Apr 2022, 17:41 Charlin S,  wrote:
>>>
 Hi,
 My query details are
 fieldsQuery="SELECT
 TestField1,TestField2,TestField3,TestField4,TestField5
  FROM
 TestModel
  WHERE
 TestField2 = 'A02'
  AND
 TestField2 = 'EN'"

 //cache model
  public class TestModel : IBinarizable
 {

 [QuerySqlField(IsIndexed = true)]
 public string TestField1 { get; set; }
 [QuerySqlField(IsIndexed = true)]
 public string TestField2 { get; set; }
 [QuerySqlField(IsIndexed = true)]
 public string TestField3 { get; set; }
 [QuerySqlField]
 public string TestField4 { get; set; }
 [QuerySqlField]
 public decimal? TestField5 { get; set; }

 public void ReadBinary(IBinaryReader reader){//implementation}
 public void WriteBinary(IBinaryWriter writer){//implementation}
 }
 implementation

 SqlFieldsQuery fieldsQuery = new SqlFieldsQuery(query) { Timeout =
 TimeSpan.FromMilliseconds(1) };
 List list = new List();
 // public ICache IgniteCache { get; set; }
 IFieldsQueryCursor queryCursor =
 IgniteCache.Query(fieldsQuery);

 //our implementation
  queryCursor.Dispose();

 Thanks,
 Charlin



 On Mon, 18 Apr 2022 at 13:35, Surinder Mehra 
 wrote:

> Can you please show slow query console log output if it's using index
> scan or full cache scan.
> I ran into one scenario where index wasn't used and it ended up
> scaning whole cache.
> You can try this locally by using control centre and run explain query
>
> On Mon, 18 Apr 2022, 13:08 Charlin S,  wrote:
>
>> Hi Ignite team,
>> We are using Ignite 2.10.0 with 4.6.2 and .Net 5 WebAPI and we have a
>> 16-nodes(including 2 server nodes) Ignite cluster.
>> We are facing slowness issues with some particular cache model query
>> and other models query are fine.
>>
>> query type: SqlFieldsQuery
>> Index: index created for where clause columns.
>>
>> Regards,
>> Charlin
>>
>>

[Ignite 2.12.0] Long Executing Query

2022-04-19 Thread Hui, Steven

Hello Ignite Team,

Recently, we have upgraded the ignite version from 2.10.0 to 2.12.0 and found 
some queries keep executing and never return under ver. 2.12.0 but no such 
issue in ver. 2.10.0.

Below is the scenario to simulate this issue.

Could you please check why they keep running and never return?

Thanks!

Create Table DDL
CREATE TABLE IF NOT EXISTS TestRiskTemplate(
id VARCHAR(36) NOT NULL,
details VARCHAR NOT NULL,
PRIMARY KEY (id)
) WITH "cache_name=TestRiskTemplateCache, affinity_key=id, 
template=RiskDataTemplate";


RiskDataTemplate configuration















AVAILABILITY_ZONE








Failure Query
select * from TestRiskTemplate where id in (1, 3)
select * from TestRiskTemplate where id in (1) UNION ALL select * from 
TestRiskTemplate where id = 3
select * from TestRiskTemplate where id = 1 OR id = 3

Re: BinaryObject Data Can Not Mapping To SQL-Data

2022-04-19 Thread y

Hi Stanilovsky,

I don't know how to describe my problem to you, but I'm sure there is not an
error and the data was successfully inserted but not mapped to SQL-data.
Vladimir give me a link
https://www.gridgain.com/resources/blog/how-fast-load-large-datasets-apache-ignite-using-key-value-api.
I decided to take a look at this link first.

Anyway, thanks for your advise and hope can help you in the future.

Tianyue Hu,
2022/4/20

在 2022-04-19 14:50:51，"Zhenya Stanilovsky" 写道：

hi !
BinaryObjectBuilder oldBuilder =
igniteClient.binary().builder(«com.inspur...PubPartitionKeys_1_7»);

do you call:

oldBuilder.build(); // after ?

If so — what this mean ? «data is not mapped to sql» is it error in log or
client side or smth ?

thanks !

Hi,

I have had the same experience without sql, using KV API only. My cluster
consists of several data nodes and self-written jar application that starts the
client node. When started, client node executes mapreduce tasks for data load
and processing.

The workaround is as follows:
1. create POJO on the client node;
2. convert it to the binary object;
3. on the data node, get binary object over the network and get its builder
(obj.toBuilder());
4. set some fields, build and put in the cache.

The builder on the step 3 seems to be the same as the one on the cluent node.

Hope that helps,
Vladimir

13:06, 18 апреля 2022 г., y :
Hi ,
When using binary to insert data, I need to get an exist
BinaryObject/BinaryObjectBuilder from the database, similar to the code below.
442062c6$3$1803c222cba$Coremail$hty1994712$163.com

If I create a BinaryObjectBuilder directly, inserting binary data does not map
to table data. The following code will not throw error, but the data is not
mapped to sql. If there is no data in my table at first, how can I insert data?
3ecbd8f9$4$1803c222cba$Coremail$hty1994712$163.com

--
Отправлено из мобильного приложения Яндекс Почты

Re:Re: Re: BinaryObject Data Can Not Mapping To SQL-Data

2022-04-19 Thread y

Hello Vladimir,

Thanks for your reply, I will carefully refer to the link you sent to see how 
it is written differently from mine. If there is a problem, I will ask for help 
again.(hope not)
By the way, i am not Russiasn. I am a young developer from China and studied 
ignite for more than a year : ）


Tianyue Hu
2022/4/20




在 2022-04-19 17:12:39，"Vladimir Tchernyi"  写道：

Hello Huty,
please read my post [1]. The approach in that paper works successfully in 
production for more than one year and seems to be correct
[1] 
https://www.gridgain.com/resources/blog/how-fast-load-large-datasets-apache-ignite-using-key-value-api


Vladimir
telegram @vtchernyi


PS
hope I named you correct, the name is not widespread here in Russia


вт, 19 апр. 2022 г. в 09:46, y :

Hi Vladimir,
Thank you for your answer.  Emmm.Actually, most of my methods are the same 
as yours except for the following two points：
1、I didn't use ComputeTask. The data is sent to the server node through the 
thin client.

2、I didn't use the standard POJO. Key-type is the following code and value-type 
is an empty class. That means All columns are dynamically specified through 
BinaryObjectBuilder. 

public class PubPartionKeys_1_7 {
@AffinityKeyMapped
private String TBDATA_DX01;
private String TBDATA_DX02;
private String TBDATA_DX03;
private String TBDATA_DX04;
private String TBDATA_DX05;
private String TBDATA_DX06;
private String TBDATA_DX07;

public PubPartionKeys_1_7() {

}

// get/set method 
// .

}
I would be appreciate it very much if you attach your code back！ ：）

Huty，
2022/4/19




At 2022-04-19 12:40:20, vtcher...@gmail.com wrote:

Hi,


I have had the same experience without sql, using KV API only. My cluster 
consists of several data nodes and self-written jar application that starts the 
client node. When started, client node executes mapreduce tasks for data load 
and processing.


The workaround is as follows:
1. create POJO on the client node;
2. convert it to the binary object;
3. on the data node, get binary object over the network and get its builder 
(obj.toBuilder());
4. set some fields, build and put in the cache.


The builder on the step 3 seems to be the same as the one on the cluent node.


Hope that helps,
Vladimir


13:06, 18 апреля 2022 г., y :

Hi ,
When using binary to insert data, I need to  get an exist 
BinaryObject/BinaryObjectBuilder  from the database, similar to the code below. 
442062c6$3$1803c222cba$Coremail$hty1994712$163.com

If I create a BinaryObjectBuilder directly, inserting binary data does not map 
to table data. The following code will not throw error, but the data is not 
mapped to sql. If there is no data in my table at first, how can I insert data?
3ecbd8f9$4$1803c222cba$Coremail$hty1994712$163.com







 



--
Отправлено из мобильного приложения Яндекс Почты

Re: Ignite query is taking long time even no data in that cache model

2022-04-19 Thread Surinder Mehra

I may be wrong but as an excercise, can you try group index on these fields
please and see if it makes any difference.
I would request Apache ignite Dev's to validate it.

On Tue, 19 Apr 2022, 19:20 Charlin S,  wrote:

> Hi,
> EXPLAIN SELECT
> TestField1,TestField2,TestField3,TestField4,TestField5
>  FROM
> TestModel
>  WHERE
> TestField2 = 'A02'
>  AND
> TestField3 = 'EN'
>
> resulview [0]: SELECT
> __Z0.TESTFIELD1 AS __C0_0,
> __Z0.TESTFIELD2 AS __C0_1,
> __Z0.TESTFIELD3 AS __C0_2,
> __Z0.TESTFIELD4 AS __C0_3,
> __Z0.TESTFIELD5 AS __C0_4
> FROM "TestModel".TESTMODEL __Z0
> /* "TestModel".TESTMODEL_TESTFIELD3_ASC_IDX: TESTFIELD3 = 'EN' */
> WHERE (__Z0.TESTFIELD2 = 'A02')
> AND (__Z0.TESTFIELD3 = 'EN')
>
> resultview[1]: SELECT
> __C0_0 AS TESTFIELD1,
> __C0_1 AS TESTFIELD2,
> __C0_2 AS TESTFIELD3,
> __C0_3 AS TESTFIELD4,
> __C0_4 AS TESTFIELD5
> FROM PUBLIC.__T0
> /* "TestModel"."merge_scan" */
>
> Thanks & Regards,
> Charlin
>
> On Tue, 19 Apr 2022 at 18:04, Surinder Mehra  wrote:
>
>> Looks correct to me. Can you run explain plain for this query and see if
>> it uses index.
>>
>> On Tue, 19 Apr 2022, 17:41 Charlin S,  wrote:
>>
>>> Hi,
>>> My query details are
>>> fieldsQuery="SELECT
>>> TestField1,TestField2,TestField3,TestField4,TestField5
>>>  FROM
>>> TestModel
>>>  WHERE
>>> TestField2 = 'A02'
>>>  AND
>>> TestField2 = 'EN'"
>>>
>>> //cache model
>>>  public class TestModel : IBinarizable
>>> {
>>>
>>> [QuerySqlField(IsIndexed = true)]
>>> public string TestField1 { get; set; }
>>> [QuerySqlField(IsIndexed = true)]
>>> public string TestField2 { get; set; }
>>> [QuerySqlField(IsIndexed = true)]
>>> public string TestField3 { get; set; }
>>> [QuerySqlField]
>>> public string TestField4 { get; set; }
>>> [QuerySqlField]
>>> public decimal? TestField5 { get; set; }
>>>
>>> public void ReadBinary(IBinaryReader reader){//implementation}
>>> public void WriteBinary(IBinaryWriter writer){//implementation}
>>> }
>>> implementation
>>>
>>> SqlFieldsQuery fieldsQuery = new SqlFieldsQuery(query) { Timeout =
>>> TimeSpan.FromMilliseconds(1) };
>>> List list = new List();
>>> // public ICache IgniteCache { get; set; }
>>> IFieldsQueryCursor queryCursor =
>>> IgniteCache.Query(fieldsQuery);
>>>
>>> //our implementation
>>>  queryCursor.Dispose();
>>>
>>> Thanks,
>>> Charlin
>>>
>>>
>>>
>>> On Mon, 18 Apr 2022 at 13:35, Surinder Mehra  wrote:
>>>
 Can you please show slow query console log output if it's using index
 scan or full cache scan.
 I ran into one scenario where index wasn't used and it ended up scaning
 whole cache.
 You can try this locally by using control centre and run explain query

 On Mon, 18 Apr 2022, 13:08 Charlin S,  wrote:

> Hi Ignite team,
> We are using Ignite 2.10.0 with 4.6.2 and .Net 5 WebAPI and we have a
> 16-nodes(including 2 server nodes) Ignite cluster.
> We are facing slowness issues with some particular cache model query
> and other models query are fine.
>
> query type: SqlFieldsQuery
> Index: index created for where clause columns.
>
> Regards,
> Charlin
>
>

Re: Ignite query is taking long time even no data in that cache model

2022-04-19 Thread Charlin S

Hi,
EXPLAIN SELECT
TestField1,TestField2,TestField3,TestField4,TestField5
 FROM
TestModel
 WHERE
TestField2 = 'A02'
 AND
TestField3 = 'EN'

resulview [0]: SELECT
__Z0.TESTFIELD1 AS __C0_0,
__Z0.TESTFIELD2 AS __C0_1,
__Z0.TESTFIELD3 AS __C0_2,
__Z0.TESTFIELD4 AS __C0_3,
__Z0.TESTFIELD5 AS __C0_4
FROM "TestModel".TESTMODEL __Z0
/* "TestModel".TESTMODEL_TESTFIELD3_ASC_IDX: TESTFIELD3 = 'EN' */
WHERE (__Z0.TESTFIELD2 = 'A02')
AND (__Z0.TESTFIELD3 = 'EN')

resultview[1]: SELECT
__C0_0 AS TESTFIELD1,
__C0_1 AS TESTFIELD2,
__C0_2 AS TESTFIELD3,
__C0_3 AS TESTFIELD4,
__C0_4 AS TESTFIELD5
FROM PUBLIC.__T0
/* "TestModel"."merge_scan" */

Thanks & Regards,
Charlin

On Tue, 19 Apr 2022 at 18:04, Surinder Mehra  wrote:

> Looks correct to me. Can you run explain plain for this query and see if
> it uses index.
>
> On Tue, 19 Apr 2022, 17:41 Charlin S,  wrote:
>
>> Hi,
>> My query details are
>> fieldsQuery="SELECT
>> TestField1,TestField2,TestField3,TestField4,TestField5
>>  FROM
>> TestModel
>>  WHERE
>> TestField2 = 'A02'
>>  AND
>> TestField2 = 'EN'"
>>
>> //cache model
>>  public class TestModel : IBinarizable
>> {
>>
>> [QuerySqlField(IsIndexed = true)]
>> public string TestField1 { get; set; }
>> [QuerySqlField(IsIndexed = true)]
>> public string TestField2 { get; set; }
>> [QuerySqlField(IsIndexed = true)]
>> public string TestField3 { get; set; }
>> [QuerySqlField]
>> public string TestField4 { get; set; }
>> [QuerySqlField]
>> public decimal? TestField5 { get; set; }
>>
>> public void ReadBinary(IBinaryReader reader){//implementation}
>> public void WriteBinary(IBinaryWriter writer){//implementation}
>> }
>> implementation
>>
>> SqlFieldsQuery fieldsQuery = new SqlFieldsQuery(query) { Timeout =
>> TimeSpan.FromMilliseconds(1) };
>> List list = new List();
>> // public ICache IgniteCache { get; set; }
>> IFieldsQueryCursor queryCursor =
>> IgniteCache.Query(fieldsQuery);
>>
>> //our implementation
>>  queryCursor.Dispose();
>>
>> Thanks,
>> Charlin
>>
>>
>>
>> On Mon, 18 Apr 2022 at 13:35, Surinder Mehra  wrote:
>>
>>> Can you please show slow query console log output if it's using index
>>> scan or full cache scan.
>>> I ran into one scenario where index wasn't used and it ended up scaning
>>> whole cache.
>>> You can try this locally by using control centre and run explain query
>>>
>>> On Mon, 18 Apr 2022, 13:08 Charlin S,  wrote:
>>>
 Hi Ignite team,
 We are using Ignite 2.10.0 with 4.6.2 and .Net 5 WebAPI and we have a
 16-nodes(including 2 server nodes) Ignite cluster.
 We are facing slowness issues with some particular cache model query
 and other models query are fine.

 query type: SqlFieldsQuery
 Index: index created for where clause columns.

 Regards,
 Charlin

Re: Ignite query is taking long time even no data in that cache model

2022-04-19 Thread Surinder Mehra

Looks correct to me. Can you run explain plain for this query and see if it
uses index.

On Tue, 19 Apr 2022, 17:41 Charlin S,  wrote:

> Hi,
> My query details are
> fieldsQuery="SELECT
> TestField1,TestField2,TestField3,TestField4,TestField5
>  FROM
> TestModel
>  WHERE
> TestField2 = 'A02'
>  AND
> TestField2 = 'EN'"
>
> //cache model
>  public class TestModel : IBinarizable
> {
>
> [QuerySqlField(IsIndexed = true)]
> public string TestField1 { get; set; }
> [QuerySqlField(IsIndexed = true)]
> public string TestField2 { get; set; }
> [QuerySqlField(IsIndexed = true)]
> public string TestField3 { get; set; }
> [QuerySqlField]
> public string TestField4 { get; set; }
> [QuerySqlField]
> public decimal? TestField5 { get; set; }
>
> public void ReadBinary(IBinaryReader reader){//implementation}
> public void WriteBinary(IBinaryWriter writer){//implementation}
> }
> implementation
>
> SqlFieldsQuery fieldsQuery = new SqlFieldsQuery(query) { Timeout =
> TimeSpan.FromMilliseconds(1) };
> List list = new List();
> // public ICache IgniteCache { get; set; }
> IFieldsQueryCursor queryCursor =
> IgniteCache.Query(fieldsQuery);
>
> //our implementation
>  queryCursor.Dispose();
>
> Thanks,
> Charlin
>
>
>
> On Mon, 18 Apr 2022 at 13:35, Surinder Mehra  wrote:
>
>> Can you please show slow query console log output if it's using index
>> scan or full cache scan.
>> I ran into one scenario where index wasn't used and it ended up scaning
>> whole cache.
>> You can try this locally by using control centre and run explain query
>>
>> On Mon, 18 Apr 2022, 13:08 Charlin S,  wrote:
>>
>>> Hi Ignite team,
>>> We are using Ignite 2.10.0 with 4.6.2 and .Net 5 WebAPI and we have a
>>> 16-nodes(including 2 server nodes) Ignite cluster.
>>> We are facing slowness issues with some particular cache model query and
>>> other models query are fine.
>>>
>>> query type: SqlFieldsQuery
>>> Index: index created for where clause columns.
>>>
>>> Regards,
>>> Charlin
>>>
>>>

Re: Ignite query is taking long time even no data in that cache model

2022-04-19 Thread Charlin S

Hi,
My query details are
fieldsQuery="SELECT
TestField1,TestField2,TestField3,TestField4,TestField5
 FROM
TestModel
 WHERE
TestField2 = 'A02'
 AND
TestField2 = 'EN'"

//cache model
 public class TestModel : IBinarizable
{

[QuerySqlField(IsIndexed = true)]
public string TestField1 { get; set; }
[QuerySqlField(IsIndexed = true)]
public string TestField2 { get; set; }
[QuerySqlField(IsIndexed = true)]
public string TestField3 { get; set; }
[QuerySqlField]
public string TestField4 { get; set; }
[QuerySqlField]
public decimal? TestField5 { get; set; }

public void ReadBinary(IBinaryReader reader){//implementation}
public void WriteBinary(IBinaryWriter writer){//implementation}
}
implementation

SqlFieldsQuery fieldsQuery = new SqlFieldsQuery(query) { Timeout =
TimeSpan.FromMilliseconds(1) };
List list = new List();
// public ICache IgniteCache { get; set; }
IFieldsQueryCursor queryCursor =
IgniteCache.Query(fieldsQuery);

//our implementation
 queryCursor.Dispose();

Thanks,
Charlin



On Mon, 18 Apr 2022 at 13:35, Surinder Mehra  wrote:

> Can you please show slow query console log output if it's using index scan
> or full cache scan.
> I ran into one scenario where index wasn't used and it ended up scaning
> whole cache.
> You can try this locally by using control centre and run explain query
>
> On Mon, 18 Apr 2022, 13:08 Charlin S,  wrote:
>
>> Hi Ignite team,
>> We are using Ignite 2.10.0 with 4.6.2 and .Net 5 WebAPI and we have a
>> 16-nodes(including 2 server nodes) Ignite cluster.
>> We are facing slowness issues with some particular cache model query and
>> other models query are fine.
>>
>> query type: SqlFieldsQuery
>> Index: index created for where clause columns.
>>
>> Regards,
>> Charlin
>>
>>

Re: Re: BinaryObject Data Can Not Mapping To SQL-Data

2022-04-19 Thread Vladimir Tchernyi

Hello Huty,
please read my post [1]. The approach in that paper works successfully in
production for more than one year and seems to be correct
[1]
https://www.gridgain.com/resources/blog/how-fast-load-large-datasets-apache-ignite-using-key-value-api

Vladimir
telegram @vtchernyi

PS
hope I named you correct, the name is not widespread here in Russia

вт, 19 апр. 2022 г. в 09:46, y :

> Hi Vladimir,
> Thank you for your answer.  Emmm.Actually, most of my methods are the
> same as yours except for the following two points：
> 1、I didn't use ComputeTask. The data is sent to the server node through
> the* thin client.*
>
> 2、I didn't use the standard POJO. Key-type is the following code and
> value-type is an empty class. That means *All columns are dynamically
> specified through BinaryObjectBuilder. *
>
> public class PubPartionKeys_1_7 {
> @AffinityKeyMapped
> private String TBDATA_DX01;
> private String TBDATA_DX02;
> private String TBDATA_DX03;
> private String TBDATA_DX04;
> private String TBDATA_DX05;
> private String TBDATA_DX06;
> private String TBDATA_DX07;
>
> public PubPartionKeys_1_7() {
> }
>
> // get/set method
> // .
> }
>
> I would be appreciate it very much if you attach your code back！ ：）
>
> Huty，
> 2022/4/19
>
>
> At 2022-04-19 12:40:20, vtcher...@gmail.com wrote:
>
> Hi,
>
> I have had the same experience without sql, using KV API only. My cluster
> consists of several data nodes and self-written jar application that starts
> the client node. When started, client node executes mapreduce tasks for
> data load and processing.
>
> The workaround is as follows:
> 1. create POJO on the client node;
> 2. convert it to the binary object;
> 3. on the data node, get binary object over the network and get its
> builder (obj.toBuilder());
> 4. set some fields, build and put in the cache.
>
> The builder on the step 3 seems to be the same as the one on the cluent
> node.
>
> Hope that helps,
> Vladimir
>
> 13:06, 18 апреля 2022 г., y :
>
> Hi ,
> When using binary to insert data, I need to  get *an
> exist BinaryObject/BinaryObjectBuilder*  from the database, similar to
> the code below.
> 442062c6$3$1803c222cba$Coremail$hty1994712$163.com
>
> If I create a BinaryObjectBuilder directly, inserting binary data does not
> map to table data. The following code will not throw error, but the* data
> is not mapped to sql. *If there is *no data in my table at first*, how
> can I insert data?
> 3ecbd8f9$4$1803c222cba$Coremail$hty1994712$163.com
>
>
>
>
>
>
>
>
> --
> Отправлено из мобильного приложения Яндекс Почты
>
>
>
>
>

Re[2]: BinaryObject Data Can Not Mapping To SQL-Data

2022-04-19 Thread Zhenya Stanilovsky


hi !
BinaryObjectBuilder oldBuilder = 
igniteClient.binary().builder(«com.inspur...PubPartitionKeys_1_7»);
 
do you call:
 
oldBuilder.build(); // after ?
 
If so — what this mean ? «data is not mapped to sql» is it error in log or 
client side or smth ?
 
thanks !
>Hi,
> 
>I have had the same experience without sql, using KV API only. My cluster 
>consists of several data nodes and self-written jar application that starts 
>the client node. When started, client node executes mapreduce tasks for data 
>load and processing.
> 
>The workaround is as follows:
>1. create POJO on the client node;
>2. convert it to the binary object;
>3. on the data node, get binary object over the network and get its builder 
>(obj.toBuilder());
>4. set some fields, build and put in the cache.
> 
>The builder on the step 3 seems to be the same as the one on the cluent node.
> 
>Hope that helps,
>Vladimir
>  13:06, 18 апреля 2022 г., y < hty1994...@163.com >:
>>Hi ,
>>When using binary to insert data, I need to  get  an exist 
>>BinaryObject/BinaryObjectBuilder   from the database, similar to the code 
>>below. 
>>442062c6$3$1803c222cba$Coremail$hty1994712$163.com
>>
>>If I create a BinaryObjectBuilder directly, inserting binary data does not 
>>map to table data. The following code will not throw error, but the data is 
>>not mapped to sql.  If there is  no data in my table at first , how can I 
>>insert data?
>>3ecbd8f9$4$1803c222cba$Coremail$hty1994712$163.com
>>
>>   
>> 
>
>--
>Отправлено из мобильного приложения Яндекс Почты

Re: Ignite query is taking long time even no data in that cache model

Re: BinaryObject Data Can Not Mapping To SQL-Data

Re: Ignite query is taking long time even no data in that cache model

[Ignite 2.12.0] Long Executing Query

Re: BinaryObject Data Can Not Mapping To SQL-Data

Re:Re: Re: BinaryObject Data Can Not Mapping To SQL-Data

Re: Ignite query is taking long time even no data in that cache model

Re: Ignite query is taking long time even no data in that cache model

Re: Ignite query is taking long time even no data in that cache model

Re: Ignite query is taking long time even no data in that cache model

Re: Re: BinaryObject Data Can Not Mapping To SQL-Data

Re[2]: BinaryObject Data Can Not Mapping To SQL-Data

12 matches

Site Navigation

Mail list logo

Footer information