jscheffl commented on issue #37810: URL: https://github.com/apache/airflow/issues/37810#issuecomment-2016560016
> This also opens the door for sending multiple things from one single function if we allow `yield Output(...)`. I can think of future extensions that the return value does not go to the XCom storage, but whatever is specified in `outlets` directly (without needing to explicitly write data in the function). There are a lot of opportunities. Like to have this option. Also thought of this. Had the idea of "future extension" as well with the primary intend to keep it simple first :-D > That said, I think implementing the context-based approach is still a good first step toward all this. Even with the more magical and convenient return-as-metadata syntax, using a context variable is still explicit and may be preferred by some. It is also easier to implement, and should be a good way to start things rolling without getting into a ton of syntax design but focus on the core feature here. So I’m going to start with that first. Looking forward to a PR. What do you think if maybe your any my proposal are both possible, mostly based on how the used needs it? In many cases the extra information might not be used at-all or with the existing mechanisms is totally fine. Something like: ``` @task(outlets=[Dataset("s3://my/data.json", extra_from_return=True)]) ``` or ``` @task(outlets=[Dataset("s3://my/data.json", extra_from_xcom="dataset_key")]) ``` ..whereas in your notation with the `yield` you explicitly push the dataset, I assume before task completion already. Instead of `uri` might be better to define the target Dataset directly? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org