[
https://issues.apache.org/jira/browse/PHOENIX-5055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
cuizhaohua updated PHOENIX-5055:
--------------------------------
Attachment: image-2019-07-15-17-21-03-956.png
> Split mutations batches probably affects correctness of index data
> ------------------------------------------------------------------
>
> Key: PHOENIX-5055
> URL: https://issues.apache.org/jira/browse/PHOENIX-5055
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 5.0.0, 4.14.1
> Reporter: jaanai
> Assignee: jaanai
> Priority: Critical
> Fix For: 5.1.0, 4.14.2, 5.0.1
>
> Attachments: ConcurrentTest.java,
> PHOENIX-5055-4.x-HBase-1.4-v2.patch, PHOENIX-5055-4.x-HBase-1.4-v3.patch,
> PHOENIX-5055-4.x-HBase-1.4-v4.patch, PHOENIX-5055-v4.x-HBase-1.4.patch,
> image-2019-07-15-17-21-03-956.png, image-2019-07-15-17-21-51-557.png
>
>
> In order to get more performance, we split the list of mutations into
> multiple batches in MutationSate. For one upsert SQL with some null values
> that will produce two type KeyValues(Put and DeleteColumn), These KeyValues
> should have the same timestamp so that keep on an atomic operation for
> corresponding the row key.
> [^ConcurrentTest.java] produced some random upsert/delete SQL and
> concurrently executed, some SQL snippets as follows:
> {code:java}
> 1149:UPSERT INTO ConcurrentReadWritTest(A,C,E,F,G) VALUES
> ('3826','2563','3052','3170','3767');
> 1864:UPSERT INTO ConcurrentReadWritTest(A,B,C,D,E,F,G) VALUES
> ('2563','4926','3526','678',null,null,'1617');
> 2332:UPSERT INTO ConcurrentReadWritTest(A,B,C,D,E,F,G) VALUES
> ('1052','2563','1120','2314','1456',null,null);
> 2846:UPSERT INTO ConcurrentReadWritTest(A,B,C,D,G) VALUES
> ('1922','146',null,'469','2563');
> 2847:DELETE FROM ConcurrentReadWritTest WHERE A = '2563’;
> {code}
> Found incorrect indexed data for the index tables by sqlline.
> !https://gw.alicdn.com/tfscom/TB1nSDqpxTpK1RjSZFGXXcHqFXa.png|width=665,height=400!
> Debugged the mutations of batches on the server side. the DeleteColumns and
> Puts were splitted into the different batches for the once upsert, the
> DeleteFaimly also was executed by another thread. due to DeleteColumns's
> timestamp is larger than DeleteFaimly under multiple threads.
> !https://gw.alicdn.com/tfscom/TB1frHmpCrqK1RjSZK9XXXyypXa.png|width=901,height=120!
>
> Running the following:
> {code:java}
> conn.createStatement().executeUpdate( "CREATE TABLE " + tableName + " (" + "A
> VARCHAR NOT NULL PRIMARY KEY," + "B VARCHAR," + "C VARCHAR," + "D VARCHAR)
> COLUMN_ENCODED_BYTES = 0");
> conn.createStatement().executeUpdate("CREATE INDEX " + indexName + " on " +
> tableName + " (C) INCLUDE(D)");
> conn.createStatement().executeUpdate("UPSERT INTO " + tableName + "(A,B,C,D)
> VALUES ('A2','B2','C2','D2')");
> conn.createStatement().executeUpdate("UPSERT INTO " + tableName + "(A,B,C,D)
> VALUES ('A3','B3', 'C3', null)");
> {code}
> dump IndexMemStore:
> {code:java}
> hbase.index.covered.data.IndexMemStore(117):
> Inserting:\x01A3/0:D/1542190446218/DeleteColumn/vlen=0/seqid=0/value=
> phoenix.hbase.index.covered.data.IndexMemStore(133): Current kv state:
> phoenix.hbase.index.covered.data.IndexMemStore(135): KV:
> \x01A3/0:B/1542190446167/Put/vlen=2/seqid=5/value=B3
> phoenix.hbase.index.covered.data.IndexMemStore(135): KV:
> \x01A3/0:C/1542190446167/Put/vlen=2/seqid=5/value=C3
> phoenix.hbase.index.covered.data.IndexMemStore(135): KV:
> \x01A3/0:D/1542190446218/DeleteColumn/vlen=0/seqid=0/value=
> phoenix.hbase.index.covered.data.IndexMemStore(135): KV:
> \x01A3/0:_0/1542190446167/Put/vlen=1/seqid=5/value=x
> phoenix.hbase.index.covered.data.IndexMemStore(137): ========== END MemStore
> Dump ==================
> {code}
>
> The DeleteColumn's timestamp larger than other mutations.
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
