haizhou-zhao commented on code in PR #11093:
URL: https://github.com/apache/iceberg/pull/11093#discussion_r1852627137
##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestBaseWithCatalog.java:
##########
@@ -59,20 +87,29 @@ protected static Object[][] parameters() {
}
@BeforeAll
- public static void createWarehouse() throws IOException {
+ public static void setUpAll() throws IOException {
TestBaseWithCatalog.warehouse = File.createTempFile("warehouse", null);
assertThat(warehouse.delete()).isTrue();
+ restCatalog = REST_SERVER_EXTENSION.client();
Review Comment:
As discussed offline, the biggest issue right now to incorporate rest client
init into BeforeEach call are the fact that
1. REST server&client (and all their configurations) are initialized in
Extension, not in the test class itself
2. REST server (port) is dynamically assigned, but that information is
needed to initialize Spark session in a static context
For 1, the current work around is to give the extension class a getter
method so that the test class can obtain necessary information.
For 2, as an example, the existing test Hive Metastore server for spark
tests also has its port dynamically assigned. The existing approach is that the
test Hive Metastore will be initialized in the test class itself in its
BeforeAll call, so that the server hiveConf could be used to initialize Spark
session in later steps of the BeforeAll calls. Looking back at rest client, if
we don't refactor the Spark session to be initialized in BeforeEach stage, then
it's unlikely we can delay the init of rest client to BeforeEach stage.
Meanwhile, init Spark session at BeforeEach stage (every test has its own Spark
session spin up) would be quite expensive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
