[jira] [Commented] (KUDU-3518) node error when impala query
[
https://issues.apache.org/jira/browse/KUDU-3518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775973#comment-17775973
]
Pain Sun commented on KUDU-3518:
[~zhangyifan27] Tested any tables whitch field is a primary key and stored as
an empty string and the quantity should be millions would appear this bug.I
tried many times including recreate tables.
> node error when impala query
>
>
> Key: KUDU-3518
> URL: https://issues.apache.org/jira/browse/KUDU-3518
> Project: Kudu
> Issue Type: Bug
> Components: tserver
>Affects Versions: 1.17.0
> Environment: centos7.9
>Reporter: Pain Sun
>Priority: Major
>
> Scan kudu with impala-4.3.0 ,there is a bug when reading a table with an
> empty string in primary key field.
> sql:
> select
> count(distinct thirdnick)
> from
> member.qyexternaluserdetailinfo_new
> where
> (
> mainshopnick = "xxx"
> and ownercorpid in ("xxx", "")
> and shoptype not in ("35", "56")
> and isDelete = 0
> and thirdnick != ""
> and thirdnick is not null
> );
>
> error:ERROR: Unable to open scanner for node with id '1' for Kudu table
> 'impala::member.qyexternaluserdetailinfo_new': Invalid argument: No such
> column: shopnick
>
> If update sql like this:
> select
> count(distinct thirdnick)
> from
> member.qyexternaluserdetailinfo_new
> where
> (
> mainshopnick = "xxx"
> and ownercorpid in ("xxx", "")
> and shopnick not in ('')
> and shoptype not in ("35", "56")
> and isDelete = 0
> and thirdnick != ""
> and thirdnick is not null
> );
> no error.
>
> this error appears in kudu-1.17.0 ,but kudu-1.16.0 is good.
>
> There is 100 items in this table ,28 items where empty string.
> table schema like this:
> ++---+-+-++--+---+---+-++
> | name | type | comment | primary_key | key_unique | nullable
> | default_value | encoding | compression | block_size |
> ++---+-+-++--+---+---+-++
> | mainshopnick | string | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | shopnick | string | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | ownercorpid | string | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | shoptype | string | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | clientid | string | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | thirdnick | string | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | id | bigint | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | receivermobile | string | | false | | true
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | thirdrealname | string | | false | | true
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | remark | string | | false | | true
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | createtime | timestamp | | false | | true
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | updatetime | timestamp | | false | | true
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | isdelete | int | | false | | true
> | 0 | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | buyernick | string | | false | | true
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> ++---+-+-++--+---+---+-++
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KUDU-3353) Add an immutable attribute on column schema
[ https://issues.apache.org/jira/browse/KUDU-3353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775857#comment-17775857 ] ASF subversion and git services commented on KUDU-3353: --- Commit 1b015d7ee9cc8af262f398961cf471f76bb225ea in kudu's branch refs/heads/master from Marton Greber [ https://gitbox.apache.org/repos/asf?p=kudu.git;h=1b015d7ee ] [Python] KUDU-3353 add support for UPSERT IGNORE This patch is a follow-up to commit: ec3a9f75b6924a70ecbf08e3805228ad9b92b9f0, it adds UPSERT IGNORE support to the Python client. Extended the already existing tests: * added write op metrics verification for immutable column tests, * extended immutable column tests with UPSERT IGNORE test, * addressed an UPSERT IGNORE TODO in the auto-incrementing column tests. Change-Id: I9112b96a5688287352307c05e60030a217154cbd Reviewed-on: http://gerrit.cloudera.org:8080/20527 Tested-by: Kudu Jenkins Reviewed-by: Yingchun Lai > Add an immutable attribute on column schema > --- > > Key: KUDU-3353 > URL: https://issues.apache.org/jira/browse/KUDU-3353 > Project: Kudu > Issue Type: New Feature > Components: api, server >Reporter: Yingchun Lai >Assignee: Yingchun Lai >Priority: Major > > h1. motivation > In some usage scenarios, Kudu table has a column with semantic of "create > time", which means it represent the create timestamp of the row. The other > columns have the similar semantic as before, for example, the user properties > like age, address, and etc. > Upstream and Kudu user doesn't know whether a row is exist or not, and every > cell data is the lastest ingested from, for example, event stream. > If without the "create time" column, Kudu user can use UPSERT operations to > write data to the table, every columns with data will overwrite the old data. > But if with the "create time" column, the cell data will be overwrote by the > following UPSERT ops, which is not what we expect. > To achive the goal, we have to read the column out to judge whether the > column is NULL or not, if it's NULL, we can fill the row with the cell, if > not NULL, we will drop it from the data before UPSERT, to avoid overwite > "create time". > It's expensive, is there a way to avoid a read from Kudu? > h1. Resolvation > We can implement column schema with semantic of "update if null". That means > cell data in changelist will update the base data if the latter is NULL, and > will ignore updates if it is not NULL. > So we can use Kudu similarly as before, but only defined the column as > "update if null" when create table or add column. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KUDU-3351) Add insert error count metrics in WriteResponsePB
[ https://issues.apache.org/jira/browse/KUDU-3351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775856#comment-17775856 ] ASF subversion and git services commented on KUDU-3351: --- Commit c02ad2fe699c7607d6fbe52d2c7e73ca2313d36e in kudu's branch refs/heads/master from Marton Greber [ https://gitbox.apache.org/repos/asf?p=kudu.git;h=c02ad2fe6 ] [Python] KUDU-3351 Add write op metrics This is a follow-up patch for commit: 0ddcaaabc97c85a4715ae79ff5604feb9b342779, adding per-session write op metrics to the Python client. In the test function "test_insert_and_mutate_rows" I added verification function calls, to check that the metrics are gathered properly. Only "upsert_ignore_errors" is left out, as UPSERT IGNORE is not yet supported by the Python client. I'm planning to address that in the next patch. Change-Id: Id76f8f0cb11ef5e4b9d06508a39492bc6b0109a9 Reviewed-on: http://gerrit.cloudera.org:8080/20526 Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin > Add insert error count metrics in WriteResponsePB > - > > Key: KUDU-3351 > URL: https://issues.apache.org/jira/browse/KUDU-3351 > Project: Kudu > Issue Type: Improvement >Reporter: Riza Suminto >Assignee: Riza Suminto >Priority: Major > Fix For: 1.17.0 > > > KUDU-1563 adds support for INSERT_IGNORE, UPDATE_IGNORE, and DELETE_IGNORE. > However, it is currently lack of per-session metrics about how many rows get > ignored vs modified. > In regular INSERT operation, we depends on per_row_errors field in > WriteResponsePB to count how many row operations are failed. > [https://github.com/apache/kudu/blob/0cb2b7c/src/kudu/tserver/tserver.proto#L175-L183] > > But with INSERT_IGNORE, this field will be empty. We should add relevant > metrics field to this WriteResponsePB. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KUDU-3518) node error when impala query
[
https://issues.apache.org/jira/browse/KUDU-3518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775760#comment-17775760
]
YifanZhang commented on KUDU-3518:
--
[~MadBeeDo] Does this issue only affect the specific table? Is it possible to
reproduce it again?
> node error when impala query
>
>
> Key: KUDU-3518
> URL: https://issues.apache.org/jira/browse/KUDU-3518
> Project: Kudu
> Issue Type: Bug
> Components: tserver
>Affects Versions: 1.17.0
> Environment: centos7.9
>Reporter: Pain Sun
>Priority: Major
>
> Scan kudu with impala-4.3.0 ,there is a bug when reading a table with an
> empty string in primary key field.
> sql:
> select
> count(distinct thirdnick)
> from
> member.qyexternaluserdetailinfo_new
> where
> (
> mainshopnick = "xxx"
> and ownercorpid in ("xxx", "")
> and shoptype not in ("35", "56")
> and isDelete = 0
> and thirdnick != ""
> and thirdnick is not null
> );
>
> error:ERROR: Unable to open scanner for node with id '1' for Kudu table
> 'impala::member.qyexternaluserdetailinfo_new': Invalid argument: No such
> column: shopnick
>
> If update sql like this:
> select
> count(distinct thirdnick)
> from
> member.qyexternaluserdetailinfo_new
> where
> (
> mainshopnick = "xxx"
> and ownercorpid in ("xxx", "")
> and shopnick not in ('')
> and shoptype not in ("35", "56")
> and isDelete = 0
> and thirdnick != ""
> and thirdnick is not null
> );
> no error.
>
> this error appears in kudu-1.17.0 ,but kudu-1.16.0 is good.
>
> There is 100 items in this table ,28 items where empty string.
> table schema like this:
> ++---+-+-++--+---+---+-++
> | name | type | comment | primary_key | key_unique | nullable
> | default_value | encoding | compression | block_size |
> ++---+-+-++--+---+---+-++
> | mainshopnick | string | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | shopnick | string | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | ownercorpid | string | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | shoptype | string | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | clientid | string | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | thirdnick | string | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | id | bigint | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | receivermobile | string | | false | | true
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | thirdrealname | string | | false | | true
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | remark | string | | false | | true
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | createtime | timestamp | | false | | true
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | updatetime | timestamp | | false | | true
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | isdelete | int | | false | | true
> | 0 | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | buyernick | string | | false | | true
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> ++---+-+-++--+---+---+-++
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
