Re: Deleting from kudu table issue

[Todd Lipcon]

Are you sure that the primary keys of the data that you are inserting are
actually unique? You will see this error if your dataset has duplicates for
that column.

Perhaps you can share "show create table odata_uat.od_policy_fee;"?

On Wed, Jul 11, 2018 at 8:16 PM, wangji...@ehuatai.com <
wangji...@ehuatai.com> wrote:

> Hi there !
> I'm facing a problem while using kudu with impala .I run a SQL script on
> impala which includes  these operations :
> 1.Delete from odata_uat.od_policy_fee ;
> 2.insert into odata_uat.od_policy_fee select *
> from odata_uat.od_policy_fee_his;
> When i'm doing the second step to insert data into the table which is
> supposed to be empty，a warning comes :
>
> WARNINGS: Key already present in Kudu table 'impala::odata_uat.od_policy_fee'.
> (1 of 538 similar)
>
> which means I lost 1 record from odata_uat.od_policy_fee_his .This
> happens occasionally  and I dunno what cause .
>
> environment： CDH 5.12.0kudu  1.4.0
>
> I have 1 kudu master with 16 cores and 128g RAM each, 4 TServers with 16 
> cores with 256g RAM each.
>
> how can I fix this or avoid it .
>
> Thanks !
>
> Best regards .
>
>
> --
>
> wang jiaxi
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Re: Deleting from kudu table issue

[wangji...@ehuatai.com]

Hi Todd:
I'm sure that my inserting dataset has no duplicate primary keys  .Cause the 
origin table has the same schema as odata_uat.od_policy_fee
and as is said  this error happens occasionally  ,when i redo the delete and 
insert operations ,the error usually won't appear again.

[cdh-dn4:21000] > show create table od_policy_fee ;
Query: show create table od_policy_fee
++
| result
 |
++
| CREATE TABLE odata_uat.od_policy_fee (
 |
|   fee_id STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION 
DEFAULT_COMPRESSION,   |
|   head_id STRING NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION, 
 |
|   branch_id STRING NULL ENCODING AUTO_ENCODING COMPRESSION 
DEFAULT_COMPRESSION,|

|   PRIMARY KEY (fee_id)
 |
| ) 
 |
| PARTITION BY HASH (fee_id) PARTITIONS 16  
 |
| STORED AS KUDU
 |
| TBLPROPERTIES ('kudu.master_addresses'='cdh-cn1') 
 |
++
thank you for your help .
best regards.


wangjiaxi