matthewblock commented on code in PR #27506:
URL: https://github.com/apache/airflow/pull/27506#discussion_r1014270814
##########
airflow/models/mappedoperator.py:
##########
@@ -620,13 +620,19 @@ def expand_mapped_task(self, run_id: str, *, session:
Session) -> tuple[Sequence
try:
total_length =
self._get_specified_expand_input().get_total_map_length(run_id, session=session)
except NotFullyPopulated as e:
- self.log.info(
- "Cannot expand %r for run %s; missing upstream values: %s",
- self,
- run_id,
- sorted(e.missing),
- )
- total_length = None
+ if self.dag and self.dag.partial:
+ # partial dags comes from the mini scheduler. It's
+ # likely that the upstream tasks are not yet done,
+ # so we ignore this exception.
+ total_length = None
+ else:
+ self.log.info(
Review Comment:
The fact that this was an `INFO` level log made it hard to track down - Do
you think it could be an `ERROR` or `WARNING`? i.e. Does this indicate that
something went wrong and requires the user's attention?
##########
airflow/models/mappedoperator.py:
##########
@@ -620,13 +620,19 @@ def expand_mapped_task(self, run_id: str, *, session:
Session) -> tuple[Sequence
try:
total_length =
self._get_specified_expand_input().get_total_map_length(run_id, session=session)
except NotFullyPopulated as e:
- self.log.info(
- "Cannot expand %r for run %s; missing upstream values: %s",
- self,
- run_id,
- sorted(e.missing),
- )
- total_length = None
+ if self.dag and self.dag.partial:
+ # partial dags comes from the mini scheduler. It's
+ # likely that the upstream tasks are not yet done,
+ # so we ignore this exception.
+ total_length = None
+ else:
+ self.log.info(
Review Comment:
The fact that this was an `INFO` level log made it hard for me to track down
- Do you think it could be an `ERROR` or `WARNING`? i.e. Does this indicate
that something went wrong and requires the user's attention?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
