vuonghoangbntt commented on code in PR #73433:
URL: https://github.com/apache/airflow/pull/73433#discussion_r4266473084
##########
airflow-core/tests/unit/dag_processing/test_manager.py:
##########
@@ -1087,6 +1087,61 @@ def
test_recently_modified_file_uses_versioned_stats_without_creating_duplicate_
assert known_file not in manager._file_stats
assert versioned_file in manager._file_stats
+ @conf_vars({("dag_processor", "file_parsing_sort_mode"): "alphabetical"})
+ def
test_prepare_file_queue_ignores_stale_callback_stats_for_recently_parsed_file(self):
+ """A stale callback stat must not mask a recent parse of the same
file."""
+ freezed_base_time = timezone.datetime(2020, 1, 5, 0, 0, 0)
+ known_file = _get_file_infos(["file_1.py"])[0]
+ versioned_file = _get_versioned_file_info("file_1.py")
+ known_files = {"testing": {known_file}}
+
+ manager = DagFileProcessorManager(max_runs=3)
+ # A callback pins bundle_version, so its entry is added after the scan
entry and
+ # then never refreshed -- resolving by insertion order would pick the
stale one.
+ manager._file_stats = {
+ known_file: DagFileStat(1, 0, freezed_base_time -
timedelta(seconds=5), 1.0, 1, 1),
+ versioned_file: DagFileStat(1, 0, freezed_base_time -
timedelta(hours=2), 1.0, 1, 1),
Review Comment:
Thank kaxil, you're right.
test_callback_only_parse_does_not_requeue_recently_parsed_file now runs a real
callback-only handle_parsing_result, so the None comes from
process_parse_results rather than from the test, in both insertion orders.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
