[ https://issues.apache.org/jira/browse/PHOENIX-5055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas D'Silva closed PHOENIX-5055. ----------------------------------- Bulk closing jiras for the 4.14.2 relase. > Split mutations batches probably affects correctness of index data > ------------------------------------------------------------------ > > Key: PHOENIX-5055 > URL: https://issues.apache.org/jira/browse/PHOENIX-5055 > Project: Phoenix > Issue Type: Bug > Affects Versions: 5.0.0, 4.14.1 > Reporter: jaanai > Assignee: jaanai > Priority: Critical > Fix For: 5.1.0, 4.14.2, 5.0.1 > > Attachments: ConcurrentTest.java, > PHOENIX-5055-4.x-HBase-1.4-v2.patch, PHOENIX-5055-4.x-HBase-1.4-v3.patch, > PHOENIX-5055-4.x-HBase-1.4-v4.patch, PHOENIX-5055-v4.x-HBase-1.4.patch > > > In order to get more performance, we split the list of mutations into > multiple batches in MutationSate. For one upsert SQL with some null values > that will produce two type KeyValues(Put and DeleteColumn), These KeyValues > should have the same timestamp so that keep on an atomic operation for > corresponding the row key. > [^ConcurrentTest.java] produced some random upsert/delete SQL and > concurrently executed, some SQL snippets as follows: > {code:java} > 1149:UPSERT INTO ConcurrentReadWritTest(A,C,E,F,G) VALUES > ('3826','2563','3052','3170','3767'); > 1864:UPSERT INTO ConcurrentReadWritTest(A,B,C,D,E,F,G) VALUES > ('2563','4926','3526','678',null,null,'1617'); > 2332:UPSERT INTO ConcurrentReadWritTest(A,B,C,D,E,F,G) VALUES > ('1052','2563','1120','2314','1456',null,null); > 2846:UPSERT INTO ConcurrentReadWritTest(A,B,C,D,G) VALUES > ('1922','146',null,'469','2563'); > 2847:DELETE FROM ConcurrentReadWritTest WHERE A = '2563’; > {code} > Found incorrect indexed data for the index tables by sqlline. > !https://gw.alicdn.com/tfscom/TB1nSDqpxTpK1RjSZFGXXcHqFXa.png|width=665,height=400! > Debugged the mutations of batches on the server side. the DeleteColumns and > Puts were splitted into the different batches for the once upsert, the > DeleteFaimly also was executed by another thread. due to DeleteColumns's > timestamp is larger than DeleteFaimly under multiple threads. > !https://gw.alicdn.com/tfscom/TB1frHmpCrqK1RjSZK9XXXyypXa.png|width=901,height=120! > > Running the following: > {code:java} > conn.createStatement().executeUpdate( "CREATE TABLE " + tableName + " (" + "A > VARCHAR NOT NULL PRIMARY KEY," + "B VARCHAR," + "C VARCHAR," + "D VARCHAR) > COLUMN_ENCODED_BYTES = 0"); > conn.createStatement().executeUpdate("CREATE INDEX " + indexName + " on " + > tableName + " (C) INCLUDE(D)"); > conn.createStatement().executeUpdate("UPSERT INTO " + tableName + "(A,B,C,D) > VALUES ('A2','B2','C2','D2')"); > conn.createStatement().executeUpdate("UPSERT INTO " + tableName + "(A,B,C,D) > VALUES ('A3','B3', 'C3', null)"); > {code} > dump IndexMemStore: > {code:java} > hbase.index.covered.data.IndexMemStore(117): > Inserting:\x01A3/0:D/1542190446218/DeleteColumn/vlen=0/seqid=0/value= > phoenix.hbase.index.covered.data.IndexMemStore(133): Current kv state: > phoenix.hbase.index.covered.data.IndexMemStore(135): KV: > \x01A3/0:B/1542190446167/Put/vlen=2/seqid=5/value=B3 > phoenix.hbase.index.covered.data.IndexMemStore(135): KV: > \x01A3/0:C/1542190446167/Put/vlen=2/seqid=5/value=C3 > phoenix.hbase.index.covered.data.IndexMemStore(135): KV: > \x01A3/0:D/1542190446218/DeleteColumn/vlen=0/seqid=0/value= > phoenix.hbase.index.covered.data.IndexMemStore(135): KV: > \x01A3/0:_0/1542190446167/Put/vlen=1/seqid=5/value=x > phoenix.hbase.index.covered.data.IndexMemStore(137): ========== END MemStore > Dump ================== > {code} > > The DeleteColumn's timestamp larger than other mutations. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)