[ https://issues.apache.org/jira/browse/TRAFODION-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15926409#comment-15926409 ]
ASF GitHub Bot commented on TRAFODION-2535: ------------------------------------------- Github user asfgit closed the pull request at: https://github.com/apache/incubator-trafodion/pull/1008 > Create index succeeds but resulting index is empty, if dop > 32 > --------------------------------------------------------------- > > Key: TRAFODION-2535 > URL: https://issues.apache.org/jira/browse/TRAFODION-2535 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-exe > Affects Versions: any > Reporter: Suresh Subbiah > Assignee: Suresh Subbiah > Fix For: 2.2-incubating > > > If more than 32 HFiles are created for a single region of an index or table > during bulkload, then LOAD COMPLETE step will fail. However if snapshot > recovery is not enabled the error is silently ignored and LOAD (or CREATE > INDEX which uses LOAD) statement appears to succeed. But the table or index > will be empty leading to wrong results for subsequent queries. > To reproduce > create table test1 (a int not null primary key, b int not null) salt using 40 > partitions; > insert into test1 values > (1,1),(2,2),(3,3),(4,4),(5,5),(6,6),(7,7),(8,8),(9,9),(10,10) ; > upsert using load into test1 select a+10, b+10 from test1; > upsert using load into test1 select a+20, b+20 from test1; > upsert using load into test1 select a+40, b+40 from test1; > upsert using load into test1 select a+80, b+80 from test1; > upsert using load into test1 select a+160, b+160 from test1; > upsert using load into test1 select a+320, b+320 from test1; > upsert using load into test1 select a+640, b+640 from test1; > upsert using load into test1 select a+1280, b+1280 from test1; > upsert using load into test1 select a+2560, b+2560 from test1; > upsert using load into test1 select a+5120, b+5120 from test1; > upsert using load into test1 select a+10240, b+10240 from test1; > upsert using load into test1 select a+20480, b+20480 from test1; > upsert using load into test1 select a+40960, b+40960 from test1; > upsert using load into test1 select a+81920, b+81920 from test1; > exit ; > from HBase shell flush 'TRAFODION.SCH.TEST1' > -- new sqlci > cqd HBASE_MIN_BYTES_PER_ESP_PARTITION '10' ; > cqd parallel_num_esps '40' ; > create index ix1 on test1(b) ; > set parserflags 1 ; > select * from table(index_table ix1) ; > -- will have 0 rows when this problem is present. > in logs directory (cdl), trafodion.hdfs.log will have a similar message. > 2017-03-14 01:28:29,242 ,ERROR ,mapreduce.LoadIncrementalHFiles ,Trying to > load more than 32 hfiles to family #1 of region with start key > With the fix we get this message in sqlci and index is not created. > >>create index ix1 on test1(b) ; > *** ERROR[8448] Unable to access Hbase interface. Call to > ExpHbaseInterface::doBulkLoad returned error HBASE_DOBULK_LOAD_ERROR(-715). > Cause: java.io.IOException: Trying to load more than 32 hfiles to one family > of one region > org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:377) > org.trafodion.sql.HBulkLoadClient.doSnapshotNBulkLoad(HBulkLoadClient.java:381) > org.trafodion.sql.HBulkLoadClient.doBulkLoad(HBulkLoadClient.java:454). > *** ERROR[1081] Loading of index TRAFODION.SCH.IX1 failed unexpectedly. > *** ERROR[8811] Trying to close a statement that is either not in the open > state or has not reached EOF. > --- SQL operation failed with errors. > >>showddl test1 ; > CREATE TABLE TRAFODION.SCH.TEST1 > ( > A INT NO DEFAULT NOT NULL NOT DROPPABLE NOT > SERIALIZED > , B INT NO DEFAULT NOT NULL NOT DROPPABLE NOT > SERIALIZED > , PRIMARY KEY (A ASC) > ) > SALT USING 40 PARTITIONS > ATTRIBUTES ALIGNED FORMAT > ; > -- GRANT SELECT, INSERT, DELETE, UPDATE, REFERENCES ON TRAFODION.SCH.TEST1 TO > DB__ROOT WITH GRANT OPTION; > --- SQL operation complete. > >>exit ; -- This message was sent by Atlassian JIRA (v6.3.15#6346)