eladkal commented on issue #50433: URL: https://github.com/apache/airflow/issues/50433#issuecomment-2869035962
I am not sure if this is right approch. I do understand what you are asking but I am not sure if Dag / task status is the way to do it. Either it's OK to run downstream or not. There can't be "OK but..." Can you please describe a use case where notifiers/on_x_callback/mertics collector is not covering? > It means that is your task is smart enough to detect that that data scrapped is good enough, but there are problems that should be taken care off, the only option currently is to implement a home made "visualization/notification" system, like sending an email to the inbox so that the person in charge can have a look. Otherwise, it's just binary: ok/nok This exists today with Notifiers. I think the missing component in the puzzle is allowing tasks to report status and take action based on these statuses. I thought about it mostly on failures (like do something different in retry if previous try had specific exception) but I guess it can be applied to success status as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
