[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543960#comment-15543960
]
Hadoop QA commented on PHOENIX-2783:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12831419/PHOENIX-2783-5.patch
against master branch at commit 0494e54de2a9ff7e1115678658fb092c95602930.
ATTACHMENT ID: 12831419
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
38 warning messages.
{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the master's current 0 warnings).
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/611//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/611//artifact/patchprocess/patchReleaseAuditWarnings.txt
Javadoc warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/611//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/611//console
This message is automatically generated.
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch,
> PHOENIX-2783-3.patch, PHOENIX-2783-4.patch, PHOENIX-2783-5.patch,
> PHOENIX-2783-INIT.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543937#comment-15543937
]
Sergey Soldatov commented on PHOENIX-2783:
--
[~jamestaylor] you are right. Just got older sqlline in classpath when tested
whether it's still reproducible or not :( Definitely PHOENIX-930 covers the
case. Closing it as duplicate.
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch,
> PHOENIX-2783-3.patch, PHOENIX-2783-4.patch, PHOENIX-2783-5.patch,
> PHOENIX-2783-INIT.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543798#comment-15543798
]
James Taylor commented on PHOENIX-2783:
---
Thanks, [~sergey.soldatov]. Just curious why doesn't the fix made for
PHOENIX-930 catch this (i.e. the {{columns.put(column.,column)}} call a few
lines down)?
{code}
+} else {
+if (!nonPkColumns.add(column)) {
+throw new ColumnAlreadyExistsException(schemaName,
tableName, column.getName().getString());
+}
}
if (columns.put(column, column) != null) {
throw new ColumnAlreadyExistsException(schemaName,
tableName, column.getName().getString());
{code}
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch,
> PHOENIX-2783-3.patch, PHOENIX-2783-4.patch, PHOENIX-2783-5.patch,
> PHOENIX-2783-INIT.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15261492#comment-15261492
]
James Taylor commented on PHOENIX-2783:
---
Ah, yes - I missed that, [~sergey.soldatov]. That's a bug - please file a
separate JIRA.
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch,
> PHOENIX-2783-3.patch, PHOENIX-2783-4.patch, PHOENIX-2783-INIT.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15261448#comment-15261448
]
Sergey Soldatov commented on PHOENIX-2783:
--
Possible I've missed something in the explanation, but select * is supposed to
show all columns, so * should be expanded to the {{$0, $1}} and in our case it
will be {{i, "0".i}}, but not {{i, i}}.
[~maryannxue] am I wrong?
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch,
> PHOENIX-2783-3.patch, PHOENIX-2783-4.patch, PHOENIX-2783-INIT.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15261358#comment-15261358
]
James Taylor commented on PHOENIX-2783:
---
[~maryannxue] did a good job of explaining the precedence that Phoenix uses
over on CALCITE-1208. The primary key column would take precedence in the
example you listed. Note that any non primary key column can be prefixed with
it's column family (which is 0 by default if not explicitly named), so this
should work as well:
{code}
select i, "0".i from tbl;
{code}
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch,
> PHOENIX-2783-3.patch, PHOENIX-2783-4.patch, PHOENIX-2783-INIT.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15261329#comment-15261329
]
Sergey Soldatov commented on PHOENIX-2783:
--
[~jamestaylor] Interesting. How {{SELECT i FROM tbl1}} is supposed to work in
this case? I understand that from HBase point of view it's a valid case, but
what about the SQL. And we already have quite unpredictable behavior in this
case:
{noformat}
0: jdbc:phoenix:localhost> create table tbl (id integer primary key, id
integer);
No rows affected (1.278 seconds)
0: jdbc:phoenix:localhost> upsert into tbl values (1,2);
1 row affected (0.041 seconds)
0: jdbc:phoenix:localhost> select * from tbl;
+-+-+
| ID | ID |
+-+-+
| 1 | 1 |
+-+-+
1 row selected (0.026 seconds)
{noformat}
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch,
> PHOENIX-2783-3.patch, PHOENIX-2783-4.patch, PHOENIX-2783-INIT.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15261310#comment-15261310
]
James Taylor commented on PHOENIX-2783:
---
Thanks for the patch, [~gsbiju] & [~sergey.soldatov]. This case is actually
allowed, as columns that are part of the primary key constraint may have the
same name as key value columns:
{quote}
2.2 Table create statement with duplicate columns of same data type
{noformat}
0: jdbc:phoenix:localhost:2181:/hbase> create table tbl1 (i integer not null
primary key, i integer);
Error: ERROR 502 (42702): Column reference ambiguous or duplicate names. In
create table columnName=TBL1.I (state=42702,code=502)
{noformat}
{code}
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch,
> PHOENIX-2783-3.patch, PHOENIX-2783-4.patch, PHOENIX-2783-INIT.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15259950#comment-15259950
]
Hadoop QA commented on PHOENIX-2783:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12800986/PHOENIX-2783-4.patch
against master branch at commit 5574811ddfe24a7f78ec875d3591439c32433f47.
ATTACHMENT ID: 12800986
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
26 warning messages.
{color:red}-1 release audit{color}. The applied patch generated 4 release
audit warnings (more than the master's current 0 warnings).
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 core tests{color}. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.UpsertValuesIT
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/310//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/310//artifact/patchprocess/patchReleaseAuditWarnings.txt
Javadoc warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/310//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/310//console
This message is automatically generated.
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch,
> PHOENIX-2783-3.patch, PHOENIX-2783-4.patch, PHOENIX-2783-INIT.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15235670#comment-15235670
]
Sergey Soldatov commented on PHOENIX-2783:
--
I'm not sure about the error reporting. What is the reason to change
COLUMN_EXIST_IN_DEF to AMBIGUOUS_COLUMN? We don't have any check for ambiguous
column, do we? Also no ASF License header in the new class. Related UT/IT
should be updated as well Such as {{QueryCompilerTest#testDuplicatePKColumn}}.
And there is a bunch of white space formatting errors in the patch.
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch,
> PHOENIX-2783-3.patch, PHOENIX-2783-INIT.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15213759#comment-15213759
]
Hadoop QA commented on PHOENIX-2783:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12795565/PHOENIX-2783-3.patch
against master branch at commit cd8e86ca7170876a30771fcc16c027f8dc8dd386.
ATTACHMENT ID: 12795565
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
20 warning messages.
{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the master's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+public CreateStatementErrorException(String message, String
schemaName, String tableName, String familyName, String columnName) {
+ throw new CreateStatementErrorException(message,
schemaName, tblName, familyName, columnName);
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.phoenix.compile.QueryCompilerTest
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages.testChangedStorageId(TestPendingCorruptDnMessages.java:102)
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/286//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/286//artifact/patchprocess/patchReleaseAuditWarnings.txt
Javadoc warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/286//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/286//console
This message is automatically generated.
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch,
> PHOENIX-2783-3.patch, PHOENIX-2783-INIT.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15213752#comment-15213752
]
Biju Nair commented on PHOENIX-2783:
Thanks [~jamestaylor], [~sergey.soldatov] for your comments. Based on the
comments, attached is a new patch for your review. The following are the some
sample exceptions generated for the three scenarios identified earlier.
1. Index create statement with duplicate columns
{noformat}
0: jdbc:phoenix:localhost:2181:/hbase> create table z (t1 varchar primary key,
t2 varchar, t3 varchar);
No rows affected (1.445 seconds)
0: jdbc:phoenix:localhost:2181:/hbase> create index idx on z (t2) include
(t1,t3,t3);
Error: ERROR 502 (42702): Column reference ambiguous or duplicate names. In
create index columnName=IDX.0.0:T3 (state=42702,code=502)
{noformat}
2.1 Table create statement with duplicate columns of different data types
{noformat}
0: jdbc:phoenix:localhost:2181:/hbase> create table tbl2 (i integer not null
primary key, i integer, i varchar);
Error: ERROR 502 (42702): Column reference ambiguous or duplicate names. In
create table columnName=TBL2.I (state=42702,code=502)
{noformat}
2.2 Table create statement with duplicate columns of same data type
{noformat}
0: jdbc:phoenix:localhost:2181:/hbase> create table tbl1 (i integer not null
primary key, i integer);
Error: ERROR 502 (42702): Column reference ambiguous or duplicate names. In
create table columnName=TBL1.I (state=42702,code=502)
{noformat}
3 Duplicate columns in create view statement
{noformat}
0: jdbc:phoenix:localhost:2181:/hbase> create view z_view (t1_v varchar, t1_v
varchar) as select * from z where t1 = 'TEST';
Error: ERROR 502 (42702): Column reference ambiguous or duplicate names. In
create view columnName=Z_VIEW.T1_V (state=42702,code=502)
{noformat}
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch,
> PHOENIX-2783-3.patch, PHOENIX-2783-INIT.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15212551#comment-15212551
]
Sergey Soldatov commented on PHOENIX-2783:
--
[~gsbiju] yep, sounds reasonable. I would only suggest to add a separator
between family and column names when you concatenate strings like I did during
performance testing. Since creating columns A.AB and AA.B is supposed to be a
valid case.
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch,
> PHOENIX-2783-INIT.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211358#comment-15211358
]
Hadoop QA commented on PHOENIX-2783:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12795344/PHOENIX-2783-INIT.patch
against master branch at commit cd8e86ca7170876a30771fcc16c027f8dc8dd386.
ATTACHMENT ID: 12795344
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
20 warning messages.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.phoenix.compile.QueryCompilerTest
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/285//testReport/
Javadoc warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/285//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/285//console
This message is automatically generated.
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch,
> PHOENIX-2783-INIT.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211249#comment-15211249
]
James Taylor commented on PHOENIX-2783:
---
Thanks for the thorough analysis, [~biju74]. I'm definitely in support of
getting this fixed up to prevent these inconsistencies. I'm happy to
participate in the review of your patch.
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch,
> PHOENIX-2783-INIT.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211238#comment-15211238
]
Biju Nair commented on PHOENIX-2783:
Hi [~sergey.soldatov], since the change is in the {{DDL}} code path and the
number of columns in a index will not be large, it may be worth trading the
small difference in performance identified in your experiment for a cleaner
code using {{HashSet}}. But looking further into the
[code|https://github.com/apache/phoenix/blob/dbc9ee9dfe9e168c45ad279f8478c59f0882240c/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java]
I am not sure whether the proposed changed in the correct one due to two
reasons.
1.
[createIndex|https://github.com/apache/phoenix/blob/dbc9ee9dfe9e168c45ad279f8478c59f0882240c/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java#L1117]
in turn gets converted into a call to
[createTable|https://github.com/apache/phoenix/blob/dbc9ee9dfe9e168c45ad279f8478c59f0882240c/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java#L1320]
which also includes logic similar to the one in the proposed fix to identify
duplicate columns which is duplication of code/logic which can make future
maintenance difficult.
2. There are other scenarios, where duplicate columns in DDLs generate
undesired results i.e. the scope of this issue is not confined only to index
creation. The following are two which I came across.
a) In table creation where two columns of same name but different types are
used in the DDL, the DDL generates an error in {{sqlline.py}}, but the table is
created and left in an unusable state similar to the index creation issue
reported in this jura ticket.
{noformat}
0: jdbc:phoenix:vm1:2181:/hbase> create table tbl2 (i integer not null primary
key, i integer, i varchar);
Error: ERROR 514 (42892): A duplicate column name was detected in the object
definition or ALTER TABLE statement. columnName=TBL2.I (state=42892,code=514)
{noformat}
b) Again in table creation if the DDL has two columns with same name and type,
the table creation goes through successfully and the table is usable. But the
expectation of users from SQL DB background, it is not the expected behavior.
{noformat}
0: jdbc:phoenix:vm1:2181:/hbase> create table tbl1 (i integer not null primary
key, i integer);
No rows affected (0.632 seconds)
0: jdbc:phoenix:vm1:2181:/hbase> upsert into tbl1 values (1, 2);
1 row affected (0.057 seconds)
0: jdbc:phoenix:vm1:2181:/hbase> select * from tbl1;
+++
| I | I |
+++
| 1 | 2 |
+++
1 row selected (0.084 seconds)
{noformat}
Since the issue impacts both the index and table creation, it may be better to
have the duplication checking logic at the start of
[createTableInternal|https://github.com/apache/phoenix/blob/dbc9ee9dfe9e168c45ad279f8478c59f0882240c/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java#L1502]
method. I have done a quick change to test it and at least it handles these
three scenarios. Will attach the change as a patch file which will help with
the conversation. It will be good to know the feedback from project members
like [~jamestaylor] who has more knowledge about the history of the code on
whether we are approaching it in the right direction.
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211063#comment-15211063
]
Biju Nair commented on PHOENIX-2783:
If anyone face the scenario detailed in the description of this issue i.e.
table not being usable due to duplicate columns used while creating an index,
the following may be way to recover. Please validate the process yourself
before using it.
Through Phoenix {{sqlline.py}} delete relevant {{SYSTEM.CATALOG}} entries
{noformat}
delete from SYSTEM.CATALOG where table_name = 'IDX';
delete from SYSTEM.CATALOG where COLUMN_FAMILY = 'IDX' and TABLE_NAME = 'X';
{{noformat}}
Through base shell, delete the index created with duplicate columns
{{noformat}}
$ hbase shell
hbase(main):002:0> disable 'IDX'
0 row(s) in 1.6090 seconds
hbase(main):006:0> drop 'IDX'
0 row(s) in 0.3120 seconds
{{noformat}}
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209304#comment-15209304
]
Sergey Soldatov commented on PHOENIX-2783:
--
Yep. That's the test I used:
{noformat}
@Benchmark
public void listMultimapTest() {
ListMultimap columnsByName = ArrayListMultimap.create
(NUM, 1);
for (Pair column : l) {
String familyName = column.getS1();
String columnName = column.getS2();
if (columnsByName.put(columnName, column)) {
int count = 0;
for (Pair dupColumn : columnsByName.get(columnName)) {
if (Objects.equal(familyName, dupColumn.getS1())) {
count++;
if (count > 1) {
System.out.println("Found duplicate");
break;
}
}
}
}
}
}
@Benchmark
public void hashSetTest() {
HashSet set = new HashSet(NUM);
for (Pair column : l) {
String familyName = column.getS1();
String columnName = column.getS2();
if(!set.add(familyName+"."+columnName)) {
System.out.println("Found duplicate");
break;
}
}
}
{noformat}
Values for pairs were UUID generated, so cycles were running for all values
without breaking for duplicates ( the worst case). I agree that using HashSet
looks cleaner. I have no objection to do it in that way. But the way, should I
refactor the check in PTableImpl in the same way?
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209236#comment-15209236
]
Biju Nair commented on PHOENIX-2783:
Thanks [~sergey.soldatov] for sharing the experiment results. Just to confirm,
the {{listMultimapTest}} was simulating the nested {{for loop}} as in the
patch. Correct? Agree about string concatenation but thought of the code being
a bit more cleaner.
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208939#comment-15208939
]
Sergey Soldatov commented on PHOENIX-2783:
--
Actually I was thinking about using HashSet, but I don't like string
concatenation. And I did some simple benchmarking for 10 entries
Benchmark Mode Cnt Score Error Units
MyBenchmark.hashSetTest avgt 15 21776823.378 ± 3728502.403 ns/op
MyBenchmark.listMultimapTest avgt 15 20121794.022 ± 2185643.518 ns/op
It seems that ListMultiMap is still better a bit.
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208775#comment-15208775
]
Biju Nair commented on PHOENIX-2783:
Hi [~sergey.soldatov], will it make the code better if we use {{hash}} with
{{familyName+columnName}} as the key with a dummy value of 0 so that the check
for the duplicate for the column in the index definition statement can be a
simple {{get}} and {{put}} if the key is not in the {{hash}}. Just a thought.
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201304#comment-15201304
]
Rajeshbabu Chintaguntla commented on PHOENIX-2783:
--
[~sergey.soldatov] it would be better to add the logic before creating the
index id in sequence table and just after collecting the columns in
MetaDataClient#createIndex.
{noformat}
// Don't re-allocate indexId on
ConcurrentTableMutationException,
// as there's no need to burn another sequence value.
if (allocateIndexId && indexId == null) {
{noformat}
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201276#comment-15201276
]
Hadoop QA commented on PHOENIX-2783:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12794152/PHOENIX-2783-1.patch
against master branch at commit cd8e86ca7170876a30771fcc16c027f8dc8dd386.
ATTACHMENT ID: 12794152
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
20 warning messages.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/278//testReport/
Javadoc warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/278//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/278//console
This message is automatically generated.
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201952#comment-15201952
]
Sergey Soldatov commented on PHOENIX-2783:
--
Updated the patch.
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2783) Creating secondary index with duplicated columns makes the catalog corrupted
[
https://issues.apache.org/jira/browse/PHOENIX-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15202298#comment-15202298
]
Hadoop QA commented on PHOENIX-2783:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12794235/PHOENIX-2783-2.patch
against master branch at commit cd8e86ca7170876a30771fcc16c027f8dc8dd386.
ATTACHMENT ID: 12794235
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
20 warning messages.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/281//testReport/
Javadoc warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/281//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/281//console
This message is automatically generated.
> Creating secondary index with duplicated columns makes the catalog corrupted
>
>
> Key: PHOENIX-2783
> URL: https://issues.apache.org/jira/browse/PHOENIX-2783
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: PHOENIX-2783-1.patch, PHOENIX-2783-2.patch
>
>
> Simple example
> {noformat}
> create table x (t1 varchar primary key, t2 varchar, t3 varchar);
> create index idx on x (t2) include (t1,t3,t3);
> {noformat}
> cause an exception that duplicated column was detected, but the client
> updates the catalog before throwing it and makes it unusable. All following
> attempt to use table x cause an exception ArrayIndexOutOfBounds. This problem
> was discussed on the user list recently.
> The cause of the problem is that check for duplicated columns happen in
> PTableImpl after MetaDataClient complete the server createTable.
> The simple way to fix is to add a similar check in MetaDataClient before
> createTable is called.
> Possible someone can suggest a more elegant way to fix it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
