vandonr-amz commented on code in PR #32572:
URL: https://github.com/apache/airflow/pull/32572#discussion_r1263075204


##########
airflow/configuration.py:
##########
@@ -378,6 +379,27 @@ def validate(self):
         self._upgrade_postgres_metastore_conn()
         self.is_validated = True
 
+    def _validate_max_tis_per_query(self) -> None:
+        """
+        Check if config ``scheduler.max_tis_per_query`` is not greater than 
``core.parallelism``.
+        If not met, a warning message printed, to guide user for best practice.
+
+        More info: https://github.com/apache/airflow/pull/29602
+
+        """
+        # TODO: this type of checking can be generalized like the deprecate 
checking
+        max_tis_per_query = self.getint("scheduler", "max_tis_per_query")
+        parallelism = self.getint("core", "parallelism")
+
+        if max_tis_per_query > parallelism:
+            warnings.warn(
+                f"Configure 
`scheduler.max_tis_per_query`(value:{max_tis_per_query}) "
+                f"should NOT be greater than 
`core.parallelism`(value:{parallelism}). "
+                f"Now, SchedulerJob will take up to `core.parallelism` as the 
query batch "
+                f"size when enqueue TaskInstances.",

Review Comment:
   I rephrased to explain what that this "wrong" setting is going to be 
ignored, which is more relevant to the user.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to