Re:Re: BinaryObject Data Can Not Mapping To SQL-Data

2022-04-20 Thread y 
Hi Vladimir
Yes, my first name is Tianyue... In fact, I work for an ERP company. ERP you 
know ,   "there are strict latency requirements for data processing and for the 
data-loading phase" in your words.  I has deployed Ignite  in a small, simple 
production environment. Now  I need to deploy another one on a more complex 
environment which has xx TB data. For some business reasons, the previous code 
is no more appropriate. The current plan  is ’Computetask + Binaryserializer‘, 
but this is also some problems and I will describe it in the future. Besides, I 
have tried to customize CacheStore, failed: (  
To be honest, since I have only graduated from University for three years and 
my technical level is so limited, exploring ignite often frustrates me. （Damn 
it, why is it wrong again ! why? 

Tianyue Hu，
2022/4/20

PS：Zhenya sounds really really like a Chinese name，maybe a Russian Chinese, 
just kidding .







在 2022-04-20 12:50:42，vtcher...@gmail.com 写道：

Hi Tianyue,


IMHO fully compilable project is usefull for newbie, while short code snippets 
are not. You can start a single server cluster and debug code in your IDE, 
check some suggestions about how it works. Couple of years ago I found such a 
compilable project describing microservices written by @DenisMagda, it helped a 
lot. So I hope my post will be usefull.



Vladimir
PS
I know out there in China last name is written first, while here in Russia it 
is written last, so the names are Vladimir or Zhenya. Hope I am correct and 
your name is Tianyue

5:12, 20 апреля 2022 г., y :





Hi Stanilovsky,

I don't know how to describe my problem to you, but I'm sure there is not an 
error and the data was successfully inserted but not mapped to SQL-data. 
Vladimir give me a link 
https://www.gridgain.com/resources/blog/how-fast-load-large-datasets-apache-ignite-using-key-value-api.
 I decided to take a look at this link first.

Anyway, thanks for your advise and hope can help you in the future.

Tianyue Hu,
2022/4/20













在 2022-04-19 14:50:51，"Zhenya Stanilovsky"  写道：

hi !
BinaryObjectBuilder oldBuilder = 
igniteClient.binary().builder(«com.inspur...PubPartitionKeys_1_7»);
 
do you call:
 
oldBuilder.build(); // after ?
 
If so — what this mean ? «data is not mapped to sql» is it error in log or 
client side or smth ?
 
thanks !


Hi,
 
I have had the same experience without sql, using KV API only. My cluster 
consists of several data nodes and self-written jar application that starts the 
client node. When started, client node executes mapreduce tasks for data load 
and processing.
 
The workaround is as follows:
1. create POJO on the client node;
2. convert it to the binary object;
3. on the data node, get binary object over the network and get its builder 
(obj.toBuilder());
4. set some fields, build and put in the cache.
 
The builder on the step 3 seems to be the same as the one on the cluent node.
 
Hope that helps,
Vladimir
 
13:06, 18 апреля 2022 г., y :
Hi ,
When using binary to insert data, I need to  get an exist 
BinaryObject/BinaryObjectBuilder  from the database, similar to the code below. 
442062c6$3$1803c222cba$Coremail$hty1994712$163.com

If I create a BinaryObjectBuilder directly, inserting binary data does not map 
to table data. The following code will not throw error, but the data is not 
mapped to sql. If there is no data in my table at first, how can I insert data?
3ecbd8f9$4$1803c222cba$Coremail$hty1994712$163.com

 
 

 



--
Отправлено из мобильного приложения Яндекс Почты
 
 
 
 




 



--
Отправлено из мобильного приложения Яндекс Почты

Re: BinaryObject Data Can Not Mapping To SQL-Data

2022-04-19 Thread vtchernyi 
Hi Tianyue,IMHO fully compilable project is usefull for newbie, while short code snippets are not. You can start a single server cluster and debug code in your IDE, check some suggestions about how it works. Couple of years ago I found such a compilable project describing microservices written by @DenisMagda, it helped a lot. So I hope my post will be usefull.VladimirPSI know out there in China last name is written first, while here in Russia it is written last, so the names are Vladimir or Zhenya. Hope I am correct and your name is Tianyue5:12, 20 апреля 2022 г., y :Hi Stanilovsky,I don't know how to describe my problem to you, but I'm sure there is not an error and the data was successfully inserted but not mapped to SQL-data. Vladimir give me a link https://www.gridgain.com/resources/blog/how-fast-load-large-datasets-apache-ignite-using-key-value-api. I decided to take a look at this link first. Anyway, thanks for your advise and hope can help you in the future.Tianyue Hu,2022/4/20在 2022-04-19 14:50:51，"Zhenya Stanilovsky"  写道：hi !BinaryObjectBuilder oldBuilder = igniteClient.binary().builder(«com.inspur...PubPartitionKeys_1_7»); do you call: oldBuilder.build(); // after ? If so — what this mean ? «data is not mapped to sql» is it error in log or client side or smth ? thanks !Hi, I have had the same experience without sql, using KV API only. My cluster consists of several data nodes and self-written jar application that starts the client node. When started, client node executes mapreduce tasks for data load and processing. The workaround is as follows:1. create POJO on the client node;2. convert it to the binary object;3. on the data node, get binary object over the network and get its builder (obj.toBuilder());4. set some fields, build and put in the cache. The builder on the step 3 seems to be the same as the one on the cluent node. Hope that helps,Vladimir 13:06, 18 апреля 2022 г., y :Hi ,When using binary to insert data, I need to  get an exist BinaryObject/BinaryObjectBuilder  from the database, similar to the code below. 442062c6$3$1803c222cba$Coremail$hty1994712$163.comIf I create a BinaryObjectBuilder directly, inserting binary data does not map to table data. The following code will not throw error, but the data is not mapped to sql. If there is no data in my table at first, how can I insert data?3ecbd8f9$4$1803c222cba$Coremail$hty1994712$163.com   --Отправлено из мобильного приложения Яндекс Почты
 -- Отправлено из мобильного приложения Яндекс Почты

Re: BinaryObject Data Can Not Mapping To SQL-Data

2022-04-19 Thread y 



Hi Stanilovsky,

I don't know how to describe my problem to you, but I'm sure there is not an 
error and the data was successfully inserted but not mapped to SQL-data. 
Vladimir give me a link 
https://www.gridgain.com/resources/blog/how-fast-load-large-datasets-apache-ignite-using-key-value-api.
 I decided to take a look at this link first.

Anyway, thanks for your advise and hope can help you in the future.

Tianyue Hu,
2022/4/20













在 2022-04-19 14:50:51，"Zhenya Stanilovsky"  写道：

hi !
BinaryObjectBuilder oldBuilder = 
igniteClient.binary().builder(«com.inspur...PubPartitionKeys_1_7»);
 
do you call:
 
oldBuilder.build(); // after ?
 
If so — what this mean ? «data is not mapped to sql» is it error in log or 
client side or smth ?
 
thanks !


Hi,
 
I have had the same experience without sql, using KV API only. My cluster 
consists of several data nodes and self-written jar application that starts the 
client node. When started, client node executes mapreduce tasks for data load 
and processing.
 
The workaround is as follows:
1. create POJO on the client node;
2. convert it to the binary object;
3. on the data node, get binary object over the network and get its builder 
(obj.toBuilder());
4. set some fields, build and put in the cache.
 
The builder on the step 3 seems to be the same as the one on the cluent node.
 
Hope that helps,
Vladimir
 
13:06, 18 апреля 2022 г., y :
Hi ,
When using binary to insert data, I need to  get an exist 
BinaryObject/BinaryObjectBuilder  from the database, similar to the code below. 
442062c6$3$1803c222cba$Coremail$hty1994712$163.com

If I create a BinaryObjectBuilder directly, inserting binary data does not map 
to table data. The following code will not throw error, but the data is not 
mapped to sql. If there is no data in my table at first, how can I insert data?
3ecbd8f9$4$1803c222cba$Coremail$hty1994712$163.com

 
 

 



--
Отправлено из мобильного приложения Яндекс Почты

Re:Re: Re: BinaryObject Data Can Not Mapping To SQL-Data

2022-04-19 Thread y 
Hello Vladimir,

Thanks for your reply, I will carefully refer to the link you sent to see how 
it is written differently from mine. If there is a problem, I will ask for help 
again.(hope not)
By the way, i am not Russiasn. I am a young developer from China and studied 
ignite for more than a year : ）


Tianyue Hu
2022/4/20




在 2022-04-19 17:12:39，"Vladimir Tchernyi"  写道：

Hello Huty,
please read my post [1]. The approach in that paper works successfully in 
production for more than one year and seems to be correct
[1] 
https://www.gridgain.com/resources/blog/how-fast-load-large-datasets-apache-ignite-using-key-value-api


Vladimir
telegram @vtchernyi


PS
hope I named you correct, the name is not widespread here in Russia


вт, 19 апр. 2022 г. в 09:46, y :

Hi Vladimir,
Thank you for your answer.  Emmm.Actually, most of my methods are the same 
as yours except for the following two points：
1、I didn't use ComputeTask. The data is sent to the server node through the 
thin client.

2、I didn't use the standard POJO. Key-type is the following code and value-type 
is an empty class. That means All columns are dynamically specified through 
BinaryObjectBuilder. 

public class PubPartionKeys_1_7 {
@AffinityKeyMapped
private String TBDATA_DX01;
private String TBDATA_DX02;
private String TBDATA_DX03;
private String TBDATA_DX04;
private String TBDATA_DX05;
private String TBDATA_DX06;
private String TBDATA_DX07;

public PubPartionKeys_1_7() {

}

// get/set method 
// .

}
I would be appreciate it very much if you attach your code back！ ：）

Huty，
2022/4/19




At 2022-04-19 12:40:20, vtcher...@gmail.com wrote:

Hi,


I have had the same experience without sql, using KV API only. My cluster 
consists of several data nodes and self-written jar application that starts the 
client node. When started, client node executes mapreduce tasks for data load 
and processing.


The workaround is as follows:
1. create POJO on the client node;
2. convert it to the binary object;
3. on the data node, get binary object over the network and get its builder 
(obj.toBuilder());
4. set some fields, build and put in the cache.


The builder on the step 3 seems to be the same as the one on the cluent node.


Hope that helps,
Vladimir


13:06, 18 апреля 2022 г., y :

Hi ,
When using binary to insert data, I need to  get an exist 
BinaryObject/BinaryObjectBuilder  from the database, similar to the code below. 
442062c6$3$1803c222cba$Coremail$hty1994712$163.com

If I create a BinaryObjectBuilder directly, inserting binary data does not map 
to table data. The following code will not throw error, but the data is not 
mapped to sql. If there is no data in my table at first, how can I insert data?
3ecbd8f9$4$1803c222cba$Coremail$hty1994712$163.com







 



--
Отправлено из мобильного приложения Яндекс Почты

Re: Re: BinaryObject Data Can Not Mapping To SQL-Data

2022-04-19 Thread Vladimir Tchernyi 
Hello Huty,
please read my post [1]. The approach in that paper works successfully in
production for more than one year and seems to be correct
[1]
https://www.gridgain.com/resources/blog/how-fast-load-large-datasets-apache-ignite-using-key-value-api

Vladimir
telegram @vtchernyi

PS
hope I named you correct, the name is not widespread here in Russia

вт, 19 апр. 2022 г. в 09:46, y :

> Hi Vladimir,
> Thank you for your answer.  Emmm.Actually, most of my methods are the
> same as yours except for the following two points：
> 1、I didn't use ComputeTask. The data is sent to the server node through
> the* thin client.*
>
> 2、I didn't use the standard POJO. Key-type is the following code and
> value-type is an empty class. That means *All columns are dynamically
> specified through BinaryObjectBuilder. *
>
> public class PubPartionKeys_1_7 {
> @AffinityKeyMapped
> private String TBDATA_DX01;
> private String TBDATA_DX02;
> private String TBDATA_DX03;
> private String TBDATA_DX04;
> private String TBDATA_DX05;
> private String TBDATA_DX06;
> private String TBDATA_DX07;
>
> public PubPartionKeys_1_7() {
> }
>
> // get/set method
> // .
> }
>
> I would be appreciate it very much if you attach your code back！ ：）
>
> Huty，
> 2022/4/19
>
>
> At 2022-04-19 12:40:20, vtcher...@gmail.com wrote:
>
> Hi,
>
> I have had the same experience without sql, using KV API only. My cluster
> consists of several data nodes and self-written jar application that starts
> the client node. When started, client node executes mapreduce tasks for
> data load and processing.
>
> The workaround is as follows:
> 1. create POJO on the client node;
> 2. convert it to the binary object;
> 3. on the data node, get binary object over the network and get its
> builder (obj.toBuilder());
> 4. set some fields, build and put in the cache.
>
> The builder on the step 3 seems to be the same as the one on the cluent
> node.
>
> Hope that helps,
> Vladimir
>
> 13:06, 18 апреля 2022 г., y :
>
> Hi ,
> When using binary to insert data, I need to  get *an
> exist BinaryObject/BinaryObjectBuilder*  from the database, similar to
> the code below.
> 442062c6$3$1803c222cba$Coremail$hty1994712$163.com
>
> If I create a BinaryObjectBuilder directly, inserting binary data does not
> map to table data. The following code will not throw error, but the* data
> is not mapped to sql. *If there is *no data in my table at first*, how
> can I insert data?
> 3ecbd8f9$4$1803c222cba$Coremail$hty1994712$163.com
>
>
>
>
>
>
>
>
> --
> Отправлено из мобильного приложения Яндекс Почты
>
>
>
>
>

Re[2]: BinaryObject Data Can Not Mapping To SQL-Data

2022-04-19 Thread Zhenya Stanilovsky 

hi !
BinaryObjectBuilder oldBuilder = 
igniteClient.binary().builder(«com.inspur...PubPartitionKeys_1_7»);
 
do you call:
 
oldBuilder.build(); // after ?
 
If so — what this mean ? «data is not mapped to sql» is it error in log or 
client side or smth ?
 
thanks !
>Hi,
> 
>I have had the same experience without sql, using KV API only. My cluster 
>consists of several data nodes and self-written jar application that starts 
>the client node. When started, client node executes mapreduce tasks for data 
>load and processing.
> 
>The workaround is as follows:
>1. create POJO on the client node;
>2. convert it to the binary object;
>3. on the data node, get binary object over the network and get its builder 
>(obj.toBuilder());
>4. set some fields, build and put in the cache.
> 
>The builder on the step 3 seems to be the same as the one on the cluent node.
> 
>Hope that helps,
>Vladimir
>  13:06, 18 апреля 2022 г., y < hty1994...@163.com >:
>>Hi ,
>>When using binary to insert data, I need to  get  an exist 
>>BinaryObject/BinaryObjectBuilder   from the database, similar to the code 
>>below. 
>>442062c6$3$1803c222cba$Coremail$hty1994712$163.com
>>
>>If I create a BinaryObjectBuilder directly, inserting binary data does not 
>>map to table data. The following code will not throw error, but the data is 
>>not mapped to sql.  If there is  no data in my table at first , how can I 
>>insert data?
>>3ecbd8f9$4$1803c222cba$Coremail$hty1994712$163.com
>>
>>   
>> 
>
>--
>Отправлено из мобильного приложения Яндекс Почты 
 
 
 
 

Re:Re: BinaryObject Data Can Not Mapping To SQL-Data

2022-04-19 Thread y 
Hi Vladimir,
Thank you for your answer.  Emmm.Actually, most of my methods are the same 
as yours except for the following two points：
1、I didn't use ComputeTask. The data is sent to the server node through the 
thin client.

2、I didn't use the standard POJO. Key-type is the following code and value-type 
is an empty class. That means All columns are dynamically specified through 
BinaryObjectBuilder. 

public class PubPartionKeys_1_7 {
@AffinityKeyMapped
private String TBDATA_DX01;
private String TBDATA_DX02;
private String TBDATA_DX03;
private String TBDATA_DX04;
private String TBDATA_DX05;
private String TBDATA_DX06;
private String TBDATA_DX07;

public PubPartionKeys_1_7() {

}

// get/set method 
// .

}
I would be appreciate it very much if you attach your code back！ ：）

Huty，
2022/4/19




At 2022-04-19 12:40:20, vtcher...@gmail.com wrote:

Hi,


I have had the same experience without sql, using KV API only. My cluster 
consists of several data nodes and self-written jar application that starts the 
client node. When started, client node executes mapreduce tasks for data load 
and processing.


The workaround is as follows:
1. create POJO on the client node;
2. convert it to the binary object;
3. on the data node, get binary object over the network and get its builder 
(obj.toBuilder());
4. set some fields, build and put in the cache.


The builder on the step 3 seems to be the same as the one on the cluent node.


Hope that helps,
Vladimir


13:06, 18 апреля 2022 г., y :

Hi ,
When using binary to insert data, I need to  get an exist 
BinaryObject/BinaryObjectBuilder  from the database, similar to the code below. 
442062c6$3$1803c222cba$Coremail$hty1994712$163.com

If I create a BinaryObjectBuilder directly, inserting binary data does not map 
to table data. The following code will not throw error, but the data is not 
mapped to sql. If there is no data in my table at first, how can I insert data?
3ecbd8f9$4$1803c222cba$Coremail$hty1994712$163.com







 



--
Отправлено из мобильного приложения Яндекс Почты

Re: BinaryObject Data Can Not Mapping To SQL-Data

2022-04-18 Thread vtchernyi 
Hi,I have had the same experience without sql, using KV API only. My cluster consists of several data nodes and self-written jar application that starts the client node. When started, client node executes mapreduce tasks for data load and processing.The workaround is as follows:1. create POJO on the client node;2. convert it to the binary object;3. on the data node, get binary object over the network and get its builder (obj.toBuilder());4. set some fields, build and put in the cache.The builder on the step 3 seems to be the same as the one on the cluent node.Hope that helps,Vladimir13:06, 18 апреля 2022 г., y :Hi ,When using binary to insert data, I need to  get an exist BinaryObject/BinaryObjectBuilder  from the database, similar to the code below. 442062c6$3$1803c222cba$Coremail$hty1994712$163.comIf I create a BinaryObjectBuilder directly, inserting binary data does not map to table data. The following code will not throw error, but the data is not mapped to sql. If there is no data in my table at first, how can I insert data?3ecbd8f9$4$1803c222cba$Coremail$hty1994712$163.com -- Отправлено из мобильного приложения Яндекс Почты

BinaryObject Data Can Not Mapping To SQL-Data

2022-04-18 Thread y 
Hi ,
When using binary to insert data, I need to  get an exist 
BinaryObject/BinaryObjectBuilder  from the database, similar to the code below. 


If I create a BinaryObjectBuilder directly, inserting binary data does not map 
to table data. The following code will not throw error, but the data is not 
mapped to sql. If there is no data in my table at first, how can I insert data?