pankajastro commented on code in PR #38423:
URL: https://github.com/apache/airflow/pull/38423#discussion_r1609678726


##########
airflow/providers/elasticsearch/log/es_task_handler.py:
##########
@@ -213,6 +216,21 @@ def format_url(host: str) -> str:
 
         return host
 
+    def _get_index_patterns(self, ti: TaskInstance | None) -> str:
+        """
+        Get index patterns by calling index_patterns_callable, if provided, or 
the configured index_patterns.
+
+        :param ti: A TaskInstance object or None.
+        """
+        if self.index_patterns_callable:
+            self.log.debug("Using index_patterns_callable: %s", 
self.index_patterns_callable)
+            module_path, index_pattern_function = 
self.index_patterns_callable.rsplit(".", 1)
+            module = importlib.import_module(module_path)
+            index_pattern_callable_obj = getattr(module, 
index_pattern_function)
+            return index_pattern_callable_obj(ti)

Review Comment:
   We have documented this in 
[provider.yaml](https://github.com/apache/airflow/pull/38423/files#diff-9c59fdec9c2afa5ff01b344180d154746a25ee13aa1cf4b748199ad51c998c7a)
 and docs will be generated for the same. Just TI needed at the moment



##########
airflow/providers/elasticsearch/log/es_task_handler.py:
##########
@@ -152,7 +153,8 @@ def __init__(
         offset_field: str = "offset",
         host: str = "http://localhost:9200";,
         frontend: str = "localhost:5601",
-        index_patterns: str | None = conf.get("elasticsearch", 
"index_patterns", fallback="_all"),
+        index_patterns: str = conf.get("elasticsearch", "index_patterns"),
+        index_patterns_callable: str = conf.get("elasticsearch", 
"index_patterns_callable", fallback=""),

Review Comment:
   historically if a param comes from the config file then it is not added in 
doc string you can see `index_patterns`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to