Thanks Jarek, For the use cases I currently deal with:
1) Relatively unfeasible, we've spent a lot of time decoupling what logic we can but fundamentally these are highly coupled business processes and if we were to strip the parts out that need their own SLAs we would end up with a dozen disjoint DAGs with unclear purposes to replace one cohesive DAG that has a clear purpose. 2) This seems workable, it's a little clunky because you can't just attach the SLA as a property of the task and it would take up UI/graph space somewhere, but as best as I can think it would serve the use cases we have and it would be possible to customize for other use cases (e.g. use cases I've had users ask for in the past: SLA 'start time' off data_interval_en, or dag start time, task start time, or take different actions based on SLA breach, or have multiple levels of breaches e.g. "yellow", "red") Thanks for being receptive to these concerns. Damian -----Original Message----- From: Jarek Potiuk <[email protected]> Sent: Monday, March 20, 2023 4:57 PM To: [email protected] Subject: Re: [DISCUSS] Airflow - New SLA AIP Thanks Damian for the comments (similarly to Sung Yun I am super-happy to get feedback from someone who uses the current SLAs ! ). I have some wild questions though - not to challenge the current SLA usage, but to get genuine feedback on the options you might have if/when we deprecate task-level SLA. 1) How feasible would it be in your case to split your DAG into smaller DAGs and link them with "data-aware scheduling" https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/datasets.html and turn the "critical" part of the current DAG into a separate DAG with its own DAG SLA ? 2) Another option (likely simpler for you) -> how about having a separate (parallel to your critical task) Timed Trigger that could raise notifications when particular task is not complete when the trigger completes (see "Example Task-level SLA feature Substitute using DateTimeTrigger and CustomOperator" from the https://docs.google.com/document/d/1drNaYmAy6GqC4WGGn4MNt6VqbOwVNm7jPfmr5Pc52AU/edit#heading=h.mitwd5au1d6t. doc ? Maybe we could implement some built-in SLAOperator, in which you could set task_id and it could run the SLA notification (even better than currently because it could run at the time when SLA time passes rather than when the task completes ? I am just thinking of possible paths for people like you, could take when/if we deprecate current SLA. I would really see if any of those is a feasible option for you. The thing is that while it was not possible some time ago, with deferrable operators and dataset-driven scheduling, you could achieve much better "final" result - either by micro-pipelining your DAG and making it more modular, or by using deferrable "almost no resources used" task to signal the SLA miss much faster. Yes, it means a change (which means cost and effort of course), but maybe for the better? Would love to see if there are issues with one of those approaches other than genuine (and of course I understand it is important) cost of change? J. J. On Mon, Mar 20, 2023 at 8:04 PM Damian Shaw <[email protected]> wrote: > > Hi Sung, > > Thanks for the clarification, with that information in mind I think I can > more accurately summarize my concern with this AIP as it is currently > presented. > > Deprecating one feature "Task SLAs" and introducing a different feature "DAG > SLAs" seems like two fairly independent choices, not one. From my experience > of using Airflow in different contexts and companies DAG SLA uses cases would > not overlap enough with Task SLA use cases (which have been compelling enough > for some users such as myself to put up with the existing > brokenness/misfeatureness) and therefore if Task SLAs are deprecated it does > not help that DAG SLAs have been introduced. > > I appreciate that Task SLAs are taxing for Airflow devs and deprecating them > helps maintainability and support bandwidth, in my opinion this should be a > strong enough reason on it's own to take this action. > > Damian > > > -----Original Message----- > From: Sung Yun <[email protected]> > Sent: Monday, March 20, 2023 2:23 PM > To: [email protected] > Subject: Re: [DISCUSS] Airflow - New SLA AIP > > Hi Damian, > > It’s great to have your feedback! And I think it’s especially great that we > are gaining feedback from people who are using the current sla feature. To > answer your points: > > 1. I never made that assumption at all. I think the reason Ash suggested that > we put it up for votes and the reason we are marking this proposal for a > Major version update, is because we are aware that there may be current users > making use of the existing feature, no matter how broken it is. > > 2. That is absolutely correct. And that is the reason why I’m giving an > example of how people will be able to write up their own sla checking tasks > that will be more responsive than the existing misfeature. My proposal only > highlights all the problems that are in place, why I think that these > problems are going to be impossible to fix when we support task-level slas, > and why I am proposing a DAG-level SLA, given how much more correct, > lightweight and simpler it will be to implement in Airflow. > > I personally don’t have a strong opinion on whether we should be deleting the > existing misfeature, mostly out of concern from the pushback we might be > getting from any existing users. However, I can definitely empathize in the > dev community in wanting to deprecate the existing misfeature, because > keeping a misfeature is costly to the community. It’s costly to new users who > are wasting time figuring out why it’s not behaving as they expect. It’s > costly to contributors trying to figure out how they could fix it. And last > but not least, it’s costly to the maintainers who need to repeatedly respond > to questions asking why the feature is broken. > > > Sent from my iPhone > > > On Mar 20, 2023, at 1:42 PM, Damian Shaw <[email protected]> > > wrote: > > > > Thanks, I've caught up on the discussion and the word document, while I > > agree with a lot of the reasoning I have two pieces of feedback: > > > > 1. The document seems to assume that the current SLAs are > > sufficiently broken that no one is depending on them, this isn't the > > case, I am at least 1 user depending on the current set-up and have > > put efforts in to making sure there is better documentation around > > it's behavior: https://github.com/apache/airflow/pull/27111 > > > > 2. The most important use case for my set-up is that different tasks in the > > same DAG can have different SLAs, e.g. I need an SLA if 1 task does not > > complete 1 hour after data_interval_end, and a different SLA of 3 hours for > > a different task in the same DAG. As a DAG may contain many 1000s of tasks > > but only a small subset of be critical enough to warrant an SLA, I imagine > > this would be a widespread use case. Am I understanding it correctly that > > this use case will no longer be supported? Please correct me if I am > > misreading this document. > > > > Thanks, > > Damian > > > > > > > > -----Original Message----- > > From: Jarek Potiuk <[email protected]> > > Sent: Sunday, March 19, 2023 5:22 PM > > To: [email protected] > > Subject: Re: [DISCUSS] Airflow - New SLA AIP > > > > You can see all the discussions in > > https://lists.apache.org/[email protected]. The SLA > > discussion is in progress in the google docs for now (soon to be converted > > into cwiki AIP). > > > > J, > > > >> On Sun, Mar 19, 2023 at 10:15 PM Damian Shaw > >> <[email protected]> wrote: > >> > >> I'm missing the previous conversation / who this is replying to and I > >> don't see a new AIP yet, is this still in progress? > >> > >> I would definitely want to be able provide feedback on an AIP for SLAs, > >> while the current SLA system is limited I am making heavy usage of it. > >> > >> -----Original Message----- > >> From: Jarek Potiuk <[email protected]> > >> Sent: Sunday, March 19, 2023 4:29 PM > >> To: [email protected] > >> Subject: Re: [DISCUSS] Airflow - New SLA AIP > >> > >>> Jarek - thank you for the suggestions - those are great suggestions in > >>> enhancing the proposal... Regarding your first point - how does > >>> deprecating an existing feature work for Airflow? Do we need to give a > >>> grace period of a minor or major patch over which we continue to support > >>> the feature after marking it for deprecation? Or do we remove them right > >>> away on a major version / minor version update? > >> > >> We raise a deprecation warning. You can see a number of examples in a > >> number of places. We cannot remove them in minor. We have not yet > >> discussed the timeline for Airflow 3 - but that is the first time when > >> things can get removed. > >> > >>> Also, do you folks have any thoughts on my last point listed in 'Other > >>> Considerations' section? I've highlighted the section to make it easier > >>> to identify, and I think it's an edge case that would be worth thinking > >>> about. > >> > >> responded in the doc. > >> > >> ------------------------------------------------------------------- > >> -- To unsubscribe, e-mail: [email protected] > >> For additional commands, e-mail: [email protected] > >> > >> ________________________________ > >> Strike Technologies, LLC (“Strike†) is part of the GTS family of > >> companies. Strike is a technology solutions provider, and is not a broker > >> or dealer and does not transact any securities related business directly > >> whatsoever. This communication is the property of Strike and its > >> affiliates, and does not constitute an offer to sell or the solicitation > >> of an offer to buy any security in any jurisdiction. It is intended only > >> for the person to whom it is addressed and may contain information that is > >> privileged, confidential, or otherwise protected from disclosure. > >> Distribution or copying of this communication, or the information > >> contained herein, by anyone other than the intended recipient is > >> prohibited. If you have received this communication in error, please > >> immediately notify Strike at [email protected], and delete and > >> destroy any copies hereof. > >> ________________________________ > >> > >> CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any attachments > >> are intended solely for the addressee. This transmission is covered by the > >> Electronic Communications Privacy Act, 18 U.S.C ''2510-2521. The > >> information contained in this transmission is confidential in nature and > >> protected from further use or disclosure under U.S. Pub. L. 106-102, 113 > >> U.S. Stat. 1338 (1999), and may be subject to attorney-client or other > >> legal privilege. Your use or disclosure of this information for any > >> purpose other than that intended by its transmittal is strictly > >> prohibited, and may subject you to fines and/or penalties under federal > >> and state law. If you are not the intended recipient of this transmission, > >> please DESTROY ALL COPIES RECEIVED and confirm destruction to the sender > >> via return transmittal. > > > > -------------------------------------------------------------------- > > - To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > ________________________________ > > Strike Technologies, LLC (“Strike†) is part of the GTS family of > > companies. Strike is a technology solutions provider, and is not a broker > > or dealer and does not transact any securities related business directly > > whatsoever. This communication is the property of Strike and its > > affiliates, and does not constitute an offer to sell or the solicitation of > > an offer to buy any security in any jurisdiction. It is intended only for > > the person to whom it is addressed and may contain information that is > > privileged, confidential, or otherwise protected from disclosure. > > Distribution or copying of this communication, or the information contained > > herein, by anyone other than the intended recipient is prohibited. If you > > have received this communication in error, please immediately notify Strike > > at [email protected], and delete and destroy any copies hereof. > > ________________________________ > > > > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any attachments > > are intended solely for the addressee. This transmission is covered by the > > Electronic Communications Privacy Act, 18 U.S.C ''2510-2521. The > > information contained in this transmission is confidential in nature and > > protected from further use or disclosure under U.S. Pub. L. 106-102, 113 > > U.S. Stat. 1338 (1999), and may be subject to attorney-client or other > > legal privilege. Your use or disclosure of this information for any purpose > > other than that intended by its transmittal is strictly prohibited, and may > > subject you to fines and/or penalties under federal and state law. If you > > are not the intended recipient of this transmission, please DESTROY ALL > > COPIES RECEIVED and confirm destruction to the sender via return > > transmittal. > > > > B‹KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK > > KKCB• È [œÝXœØÜšX™K K[XZ[ > ˆ ]‹][œÝXœØÜšX™P Z\™› ݢ\ XÚ K›Ü™ÃB‘›Üˆ Y ] [Û˜[ ÛÛ[X[™ Ë K[XZ[ > ˆ ]‹Z [ Z\™› ݢ\ XÚ K›Ü™ÃB > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > ________________________________ > Strike Technologies, LLC (“Strike”) is part of the GTS family of companies. > Strike is a technology solutions provider, and is not a broker or dealer and > does not transact any securities related business directly whatsoever. This > communication is the property of Strike and its affiliates, and does not > constitute an offer to sell or the solicitation of an offer to buy any > security in any jurisdiction. It is intended only for the person to whom it > is addressed and may contain information that is privileged, confidential, or > otherwise protected from disclosure. Distribution or copying of this > communication, or the information contained herein, by anyone other than the > intended recipient is prohibited. If you have received this communication in > error, please immediately notify Strike at [email protected], and > delete and destroy any copies hereof. > ________________________________ > > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any attachments are > intended solely for the addressee. This transmission is covered by the > Electronic Communications Privacy Act, 18 U.S.C ''2510-2521. The information > contained in this transmission is confidential in nature and protected from > further use or disclosure under U.S. Pub. L. 106-102, 113 U.S. Stat. 1338 > (1999), and may be subject to attorney-client or other legal privilege. Your > use or disclosure of this information for any purpose other than that > intended by its transmittal is strictly prohibited, and may subject you to > fines and/or penalties under federal and state law. If you are not the > intended recipient of this transmission, please DESTROY ALL COPIES RECEIVED > and confirm destruction to the sender via return transmittal. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected] ________________________________ Strike Technologies, LLC (“Strike”) is part of the GTS family of companies. Strike is a technology solutions provider, and is not a broker or dealer and does not transact any securities related business directly whatsoever. This communication is the property of Strike and its affiliates, and does not constitute an offer to sell or the solicitation of an offer to buy any security in any jurisdiction. It is intended only for the person to whom it is addressed and may contain information that is privileged, confidential, or otherwise protected from disclosure. Distribution or copying of this communication, or the information contained herein, by anyone other than the intended recipient is prohibited. If you have received this communication in error, please immediately notify Strike at [email protected], and delete and destroy any copies hereof. ________________________________ CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any attachments are intended solely for the addressee. This transmission is covered by the Electronic Communications Privacy Act, 18 U.S.C ''2510-2521. The information contained in this transmission is confidential in nature and protected from further use or disclosure under U.S. Pub. L. 106-102, 113 U.S. Stat. 1338 (1999), and may be subject to attorney-client or other legal privilege. Your use or disclosure of this information for any purpose other than that intended by its transmittal is strictly prohibited, and may subject you to fines and/or penalties under federal and state law. If you are not the intended recipient of this transmission, please DESTROY ALL COPIES RECEIVED and confirm destruction to the sender via return transmittal.
