keith-turner opened a new pull request, #297:
URL: https://github.com/apache/accumulo-testing/pull/297
Running bulk import test failed with the following error.
```
2025-08-29T00:44:43,285 [randomwalk.bulk.BulkMinusOne] ERROR:
java.lang.IllegalStateException: java.util.concurrent.ExecutionException:
java.lang.IllegalArgumentException: The file
hdfs://localhost:8020/tmp/bulk_ab3612b1-ef53-4cd5-8c28
-7dd75de47c87/part_0.rf attempted to import to 122 tablets. Max tablets
allowed set to 100
java.lang.IllegalStateException: java.util.concurrent.ExecutionException:
java.lang.IllegalArgumentException: The file
hdfs://localhost:8020/tmp/bulk_ab3612b1-ef53-4cd5-8c28-7dd75de47c87/part_0.rf
attempted to import to 122 tablets. Max tablets allowed set to 100
at
org.apache.accumulo.core.clientImpl.bulk.BulkImport.computeFileToTabletMappings(BulkImport.java:591)
~[accumulo-testing-shaded.jar:?]
at
org.apache.accumulo.core.clientImpl.bulk.BulkImport.computeMappingFromFiles(BulkImport.java:499)
~[accumulo-testing-shaded.jar:?]
at
org.apache.accumulo.core.clientImpl.bulk.BulkImport.load(BulkImport.java:155)
~[accumulo-testing-shaded.jar:?]
at
org.apache.accumulo.testing.randomwalk.bulk.BulkPlusOne.bulkLoadLots(BulkPlusOne.java:98)
~[accumulo-testing-shaded.jar:?]
at
org.apache.accumulo.testing.randomwalk.bulk.BulkMinusOne.runLater(BulkMinusOne.java:34)
~[accumulo-testing-shaded.jar:?]
at
org.apache.accumulo.testing.randomwalk.bulk.BulkTest.lambda$visit$0(BulkTest.java:33)
~[accumulo-testing-shaded.jar:?]
at
org.apache.accumulo.core.trace.TraceWrappedRunnable.run(TraceWrappedRunnable.java:52)
[accumulo-testing-shaded.jar:?]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at
org.apache.accumulo.core.trace.TraceWrappedRunnable.run(TraceWrappedRunnable.java:52)
[accumulo-testing-shaded.jar:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[?:?]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[?:?]
at
org.apache.accumulo.core.trace.TraceWrappedRunnable.run(TraceWrappedRunnable.java:52)
[accumulo-testing-shaded.jar:?]
at java.lang.Thread.run(Thread.java:840) [?:?]
Caused by: java.util.concurrent.ExecutionException:
java.lang.IllegalArgumentException: The file
hdfs://localhost:8020/tmp/bulk_ab3612b1-ef53-4cd5-8c28-7dd75de47c87/part_0.rf
attempted to import to 122 tablets. Max tablets allowed set
to 100
at
java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
~[?:?]
at
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?]
at
org.apache.accumulo.core.clientImpl.bulk.BulkImport.computeFileToTabletMappings(BulkImport.java:585)
~[accumulo-testing-shaded.jar:?]
... 13 more
Caused by: java.lang.IllegalArgumentException: The file
hdfs://localhost:8020/tmp/bulk_ab3612b1-ef53-4cd5-8c28-7dd75de47c87/part_0.rf
attempted to import to 122 tablets. Max tablets allowed set to 100
at
org.apache.accumulo.core.clientImpl.bulk.BulkImport.checkTabletCount(BulkImport.java:627)
~[accumulo-testing-shaded.jar:?]
at
org.apache.accumulo.core.clientImpl.bulk.BulkImport.lambda$computeFileToTabletMappings$7(BulkImport.java:563)
~[accumulo-testing-shaded.jar:?]
at
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
~[?:?]
... 5 more
```
To avoid this adjusted the tablet setting for the max number of files that
can go to a single tablet to 1000. Also fixed the following problems with the
test code that made this problem harder to debug.
* The test schedules lots of background task, but keeps running when they
fail. Made the test exit when it notices a background test failed. This
changes required making a map synchronized.
* Some log messages related to the test bulk import code logged a marker
count that correlates the bulk import w/ data in the table. This marker count
is really important for tracking down bugs w/ missing data, but only a subet of
log messages included it. Added the marker count to all log messages since
multiple threads log messages are interleaved in test output.
Since the test did not immediataly fail when the bulk import call failed, I
assumed all bulk imports were succesful and accumulo lost data. Looked down the
wrong path for a bit, hoping these changes help avoid that in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
