[ https://issues.apache.org/jira/browse/ACCUMULO-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708281#comment-14708281 ]
Josh Elser commented on ACCUMULO-3967: -------------------------------------- Also, just to confirm: the rfile does contain all of the entries. This is definitely an issue with bulk load. {noformat} % accumulo rfile-info /accumulo/tables/4/b-0000127/I0000128.rf Reading file: hdfs://localhost:8020/accumulo/tables/4/b-0000127/I0000128.rf Locality group : <DEFAULT> Start block : 0 Num blocks : 360 Index level 0 : 29,497 bytes 1 blocks First key : 00:0002275267839497F4F86F7142D0EC2F Family:Qualifier [] 9223372036854775807 false Last key : 23:FFFF7F3618B77EB07A51EE91381CE0BF Family:Qualifier [] 9223372036854775807 false Num entries : 1,000,000 Column families : [Family] {noformat} > bulk import loses records when loading pre-split table > ------------------------------------------------------ > > Key: ACCUMULO-3967 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3967 > Project: Accumulo > Issue Type: Bug > Components: client, tserver > Affects Versions: 1.7.0 > Environment: generic hadoop 2.6.0, zookeeper 3.4.6 on redhat 6.7 > 7 node cluster > Reporter: Edward Seidl > Priority: Blocker > Fix For: 1.7.1, 1.8.0 > > > I just noticed that some records I'm loading via importDirectory go missing. > After a lot of digging around trying to reproduce the problem, I discovered > that it occurs most frequently when loading a table that I have just recently > added splits to. In the tserver logs I'll see messages like > 20 16:25:36,805 [client.BulkImporter] INFO : Could not assign 1 map files to > tablet 1xw;18;17 because : Not Serving Tablet . Will retry ... > > or > 20 16:25:44,826 [tserver.TabletServer] INFO : files > [hdfs://xxxx:54310/accumulo/tables/1xw/b-00jnmxe/I00jnmxq.rf] not imported to > 1xw;03;02: tablet 1xw;03;02 is closed > these appear after messages about unloading tablets...it seems that tablets > are being redistributed at the same time as the bulk import is occuring. > Steps to reproduce > 1) I run a mapreduce job that produces random data in rfiles > 2) copy the rfiles to an import directory > 3) create table or deleterows -f > 4) addsplits > 5) importdirectory > I have also performed the above completely within the mapreduce job, with > similar results. The difference with the mapreduce job is that the time > between adding splits and the import directory is minutes rather than seconds. > my current test creates 1000000 records, and after the importdirectory > returns a count of rows will be anywhere from ~800000 to 1000000. > With my original workflow, I found that re-importing the same set of rfiles > three times would eventually get all rows loaded. -- This message was sent by Atlassian JIRA (v6.3.4#6332)