DomGarguilo commented on PR #5822: URL: https://github.com/apache/accumulo/pull/5822#issuecomment-3234266843
> > One thing that I noticed during developing the import test case was that if i tried to import multiple tables that had been exported with data into a single tablename, the test would fail with something like File F0000001.rf does not exist in import dir which is coming from [here](https://github.com/apache/accumulo/blob/501efad92bee10aad2910b0984752879d5e42322/server/manager/src/main/java/org/apache/accumulo/manager/tableOps/tableImport/PopulateMetadataTable.java#L159). I changed the test to import from tables exported with NO data and this exception is avoided and we get the TableExistsException as expected. > > For this scenario were multiple threads trying to import a table from the same source dir in dfs? There is code in the import fate operation that attempt to reserve the source dir in hdfs, maybe this is not working. Opened #5838 after looking into this a bit. The Utils.reserveHdfsDirectory method referenced in #5838 is called by the import table fate code. May need to open a follow in issue to investigate this for import table. I don't think it made a difference if it was the same or different source dirs. If you paste the following into the new IT in this PR you'll see "AccumuloException: File F0000001.rf does not exist in import dir" ```java @Test public void importTable() throws Exception { final int numTasks = 10; final int numIterations = 3; ExecutorService pool = Executors.newFixedThreadPool(numTasks); String[] targetTableNames = getUniqueNames(numIterations); var ntc = new NewTableConfiguration().withInitialTabletAvailability(TabletAvailability.HOSTED); for (String importTableName : targetTableNames) { // Create separate source tables and export directories for each thread List<String> exportDirs = new ArrayList<>(numTasks); for (int i = 0; i < numTasks; i++) { String sourceTableName = importTableName + "_export_source_" + i; client.tableOperations().create(sourceTableName, ntc); try (BatchWriter bw = client.createBatchWriter(sourceTableName)) { Mutation m = new Mutation("myRow"); m.put("cf1", "cq1", "v1"); bw.addMutation(m); } client.tableOperations().offline(sourceTableName); String exportDir = getCluster().getTemporaryPath() + "/export_" + sourceTableName; client.tableOperations().exportTable(sourceTableName, exportDir); exportDirs.add(exportDir); } int tableCountBefore = client.tableOperations().list().size(); // All threads attempt to import to the same target table name int successCount = runConcurrentTableOperation(pool, numTasks, (index) -> { client.tableOperations().importTable(importTableName, exportDirs.get(index)); return true; }); assertEquals(1, successCount, "Expected only one import operation to succeed"); assertTrue(client.tableOperations().exists(importTableName), "Expected import table " + importTableName + " to exist"); assertEquals(tableCountBefore + 1, client.tableOperations().list().size(), "Expected +1 table count for import operation"); } pool.shutdown(); } ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@accumulo.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
