phi-friday opened a new issue, #41571:
URL: https://github.com/apache/airflow/issues/41571

   ### Apache Airflow version
   
   2.10.0rc1
   
   ### If "Other Airflow 2 version" selected, which one?
   
   2.10.0(not rc)
   
   ### What happened?
   
   I was expecting to be able to access the `DatasetEvent` of the currently 
triggered `DatasetEvent`, but it accesses the last created `Extra`.
   
   ### What you think should happen instead?
   
   Only `DatasetEvent` created before `logical_date` should be looked up.
   
   ### How to reproduce
   
   ```python
   from __future__ import annotations
   
   from collections.abc import Generator
   
   from airflow.datasets import Dataset, DatasetAlias
   from airflow.datasets.metadata import Metadata
   from airflow.decorators import dag, task
   from airflow.operators.bash import BashOperator
   from airflow.operators.trigger_dagrun import TriggerDagRunOperator
   from pendulum import datetime
   
   dataset_alias = DatasetAlias("my_dataset_alias")
   dataset = Dataset("my_dataset")
   
   
   @dag(start_date=datetime(2024, 1, 1), schedule=None, catchup=False)
   def dataset_producer() -> None:
       @task.python(outlets=[dataset_alias])
       def with_extra() -> Generator[Metadata, None, None]:
           import json
   
           from airflow.operators.python import get_current_context
   
           context = get_current_context()
           conf = dict(context["dag_run"].conf)
           conf = json.dumps(conf)
   
           yield Metadata(
               target="my_dataset",
               extra={"key": "value", "conf": conf},
               alias="my_dataset_alias",
           )
   
       with_extra()
   
   
   @dag(start_date=datetime(2024, 1, 1), schedule=None, catchup=False)
   def dataset_producer_twice() -> None:
       first = TriggerDagRunOperator(
           task_id="first",
           trigger_dag_id="dataset_producer",
           deferrable=False,
           wait_for_completion=False,
           conf={"task_id": "first"},
       )
       second = TriggerDagRunOperator(
           task_id="second",
           trigger_dag_id="dataset_producer",
           deferrable=False,
           wait_for_completion=False,
           conf={"task_id": "second"},
       )
       wait = BashOperator(task_id="wait", bash_command="sleep 5")
       _ = [first, wait] >> second
   
   
   @dag(start_date=datetime(2024, 1, 1), schedule=dataset_alias, catchup=False)
   def dataset_consumer() -> None:
       wait = BashOperator(task_id="wait", bash_command="sleep 10")
   
       @task.python(inlets=[dataset_alias])
       def consume_dataset() -> None:
           from pprint import pprint
   
           from airflow.datasets import DatasetAlias
           from airflow.operators.python import get_current_context
   
           context = get_current_context()
           inlet_events = context["inlet_events"]
           target_events = inlet_events[DatasetAlias("my_dataset_alias")]
           event = target_events[-1]
           pprint(event.extra)
   
       show = consume_dataset()
       _ = wait >> show
   
   
   dataset_producer()
   dataset_consumer()
   dataset_producer_twice()
   ```
   
   You can see the same logs in both `DagRun`s.
   ```[2024-08-19, 12:06:20 KST] {logging_mixin.py:190} INFO - {'conf': 
'{"task_id": "second"}', 'key': 'value'}```
   
   ### Operating System
   
   PRETTY_NAME="Debian GNU/Linux 12 (bookworm)" NAME="Debian GNU/Linux" 
VERSION_ID="12" VERSION="12 (bookworm)" VERSION_CODENAME=bookworm ID=debian 
HOME_URL="https://www.debian.org/"; SUPPORT_URL="https://www.debian.org/support"; 
BUG_REPORT_URL="https://bugs.debian.org/";
   
   ### Versions of Apache Airflow Providers
   
   apache-airflow-providers-celery==3.7.3
   apache-airflow-providers-common-compat==1.1.0
   apache-airflow-providers-common-io==1.4.0
   apache-airflow-providers-common-sql==1.15.0
   apache-airflow-providers-docker==3.12.3
   apache-airflow-providers-fab==1.2.2
   apache-airflow-providers-ftp==3.10.1
   apache-airflow-providers-http==4.12.0
   apache-airflow-providers-imap==3.6.1
   apache-airflow-providers-jdbc==4.4.0
   apache-airflow-providers-odbc==4.6.3
   apache-airflow-providers-postgres==5.11.3
   apache-airflow-providers-redis==3.7.1
   apache-airflow-providers-smtp==1.7.1
   apache-airflow-providers-sqlite==3.8.2
   
   ### Deployment
   
   Docker-Compose
   
   ### Deployment details
   
   _No response_
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to