[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #7941: [HUDI-5786] Add a new config to specific spark write rdd storage level

[via GitHub]

zhangyue19921010 commented on code in PR #7941:
URL: https://github.com/apache/hudi/pull/7941#discussion_r1107957756


##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##########
@@ -127,6 +127,12 @@ public class HoodieWriteConfig extends HoodieConfig {
       .noDefaultValue()
       .withDocumentation("Table name that will be used for registering with 
metastores like HMS. Needs to be same across runs.");
 
+  public static final ConfigProperty<String> SPARK_WRITE_STORAGE_LEVEL_VALUE = 
ConfigProperty
+       .key("hoodie.spark.write.storage.level")

Review Comment:
   nit: could we use `hoodie.write.tagged.record.storage.level`



##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##########
@@ -127,6 +127,12 @@ public class HoodieWriteConfig extends HoodieConfig {
       .noDefaultValue()
       .withDocumentation("Table name that will be used for registering with 
metastores like HMS. Needs to be same across runs.");
 
+  public static final ConfigProperty<String> SPARK_WRITE_STORAGE_LEVEL_VALUE = 
ConfigProperty
+       .key("hoodie.spark.write.storage.level")
+       .defaultValue("MEMORY_AND_DISK_SER")
+       .withDocumentation("Determine what level of persistence is used to 
cache write RDDs. "

Review Comment:
   is used to cache the tagged records



##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##########
@@ -127,6 +127,12 @@ public class HoodieWriteConfig extends HoodieConfig {
       .noDefaultValue()
       .withDocumentation("Table name that will be used for registering with 
metastores like HMS. Needs to be same across runs.");
 
+  public static final ConfigProperty<String> SPARK_WRITE_STORAGE_LEVEL_VALUE = 
ConfigProperty

Review Comment:
   TAGGED_RECORD_STORAGE_LEVEL_VALUE



##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##########
@@ -1069,6 +1075,10 @@ public String getWriteSchema() {
     return getSchema();
   }
 
+  public String getSparkWriteStorageLevel() {

Review Comment:
   getTaggedRecordStorageLevel()



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org