vinothchandar commented on a change in pull request #3153:
URL: https://github.com/apache/hudi/pull/3153#discussion_r662716233
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java
##########
@@ -194,6 +194,37 @@
public static final String EXTERNAL_RECORD_AND_SCHEMA_TRANSFORMATION =
AVRO_SCHEMA + ".externalTransformation";
public static final String DEFAULT_EXTERNAL_RECORD_AND_SCHEMA_TRANSFORMATION
= "false";
+ public static final String PRE_COMMIT_VALIDATORS =
"hoodie.precommit.validators";
+ private static final String DEFAULT_PRE_COMMIT_VALIDATORS = "";
+ public static final String VALIDATOR_TABLE_VARIABLE = "<TABLE_NAME>";
+
+ /**
+ * Spark SQL queries to run on table before committing new data to validate
state before and after commit.
+ * Multiple queries separated by ';' delimiter are supported.
+ * example: "select count(*) from \<TABLE_NAME\>"
+ * Note \<TABLE_NAME\> is replaced by table state before and after commit.
+ */
+ public static final String PRE_COMMIT_VALIDATORS_EQUALITY_SQL_QUERIES =
"hoodie.precommit.validators.equality.sql.queries";
Review comment:
please move all these configs to as ConfigProperty
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java
##########
@@ -194,6 +194,37 @@
public static final String EXTERNAL_RECORD_AND_SCHEMA_TRANSFORMATION =
AVRO_SCHEMA + ".externalTransformation";
public static final String DEFAULT_EXTERNAL_RECORD_AND_SCHEMA_TRANSFORMATION
= "false";
+ public static final String PRE_COMMIT_VALIDATORS =
"hoodie.precommit.validators";
Review comment:
can we create a new Config class for this, instead of overloading the
WriteConfig?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
