Re: Support for Datasets with execution-time values for use cases requiring more fine-grained dataset specs

2023-07-14 Thread Jeff Payne
Hi Vikram, Thanks for the additional context. That sounds great. What's the best way for us to connect? Jeff Payne From: Vikram Koka Sent: Friday, July 14, 2023 8:00 AM To: jul...@astronomer.io.invalid Cc: dev@airflow.apache.org Subject: Re: Suppor

Re: Support for Datasets with execution-time values for use cases requiring more fine-grained dataset specs

2023-07-09 Thread Jeff Payne
. > > I think we need to first discuss the mechanism that allows Datasets to > support the new use cases in a holistic manner rather than patching each > individual use case. > > I don't know if this is AIP worthy. If you'd like to lead this I'm happy to > re

Support for Datasets with execution-time values for use cases requiring more fine-grained dataset specs

2023-07-05 Thread Jeff Payne
dCommentId=217385741#comment-217385741. I have not looked at the Dataset related code and am not sure what the complexity of something like this would be. I assume it wouldn't be trivial, given that the Datasets are currently static objects. Would love to hear some feedback. Thanks! Jeff Payne