kevinjqliu commented on code in PR #13345:
URL: https://github.com/apache/iceberg/pull/13345#discussion_r2161873771
##########
core/src/test/java/org/apache/iceberg/jdbc/TestJdbcTableConcurrency.java:
##########
@@ -149,4 +190,1015 @@ public synchronized void testConcurrentConnections()
throws InterruptedException
assertThat(executorService.awaitTermination(3,
TimeUnit.MINUTES)).as("Timeout").isTrue();
assertThat(Iterables.size(icebergTable.snapshots())).isEqualTo(7);
}
+
+ @Test
+ public synchronized void testInitializeWithSlowConcurrentConnections()
+ throws InterruptedException, SQLException, ExecutionException,
ClassNotFoundException {
+ // number of threads and requests to attempt.
+ int parallelism = 2;
+ // verifies that multiple calls to initialize with slow responses will not
fail.
+ Map<String, String> properties = Maps.newHashMap();
+
+ properties.put(CatalogProperties.WAREHOUSE_LOCATION,
tableDir.getAbsolutePath());
+ String testingDB = "jdbc:slow:derby:memory:testDb;create=true";
+ new org.apache.derby.jdbc.EmbeddedDriver();
+ properties.put(CatalogProperties.URI, testingDB);
+ SlowDriver slowDriver = new SlowDriver(testingDB);
Review Comment:
Curious if we can test this without `SlowDriver`.
Can we simulate the same behavior by raising the parallelism?
For example, `testConcurrentConnections` uses 7 threads
##########
core/src/test/java/org/apache/iceberg/jdbc/TestJdbcTableConcurrency.java:
##########
@@ -149,4 +190,1015 @@ public synchronized void testConcurrentConnections()
throws InterruptedException
assertThat(executorService.awaitTermination(3,
TimeUnit.MINUTES)).as("Timeout").isTrue();
assertThat(Iterables.size(icebergTable.snapshots())).isEqualTo(7);
}
+
+ @Test
+ public synchronized void testInitializeWithSlowConcurrentConnections()
+ throws InterruptedException, SQLException, ExecutionException,
ClassNotFoundException {
+ // number of threads and requests to attempt.
+ int parallelism = 2;
+ // verifies that multiple calls to initialize with slow responses will not
fail.
+ Map<String, String> properties = Maps.newHashMap();
+
+ properties.put(CatalogProperties.WAREHOUSE_LOCATION,
tableDir.getAbsolutePath());
+ String testingDB = "jdbc:slow:derby:memory:testDb;create=true";
+ new org.apache.derby.jdbc.EmbeddedDriver();
+ properties.put(CatalogProperties.URI, testingDB);
+ SlowDriver slowDriver = new SlowDriver(testingDB);
+
+ Callable<JdbcCatalog> makeCatalog =
+ () -> {
+ JdbcCatalog catalog = new JdbcCatalog();
+ catalog.setConf(new Configuration());
+ catalog.initialize("jdbc", properties);
+ return catalog;
+ };
+
+ try {
+ DriverManager.registerDriver(slowDriver);
+ ExecutorService executorService =
+ MoreExecutors.getExitingExecutorService(
+ (ThreadPoolExecutor) Executors.newFixedThreadPool(parallelism));
+
+ List<Future<JdbcCatalog>> futures = Lists.newArrayList();
+ for (int i = 0; i < parallelism; i++) {
+ futures.add(executorService.submit(makeCatalog));
+ }
+ for (Future<JdbcCatalog> future : futures) {
+ future.get();
+ }
+ } finally {
+ DriverManager.deregisterDriver(slowDriver);
+ }
Review Comment:
nit: add a catalog operation here to verify that both catalog clients are
initialized and ready
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
