Re: [I] Airflow 3.0.x : Bring back from Airflow 2.x "Status of all previous DAG runs" column in DAGS main page [airflow]
Saadmadni84 commented on issue #50624: URL: https://github.com/apache/airflow/issues/50624#issuecomment-3708133034 Hi @pierrejeambrun, I’d like to work on this issue and propose a PR for the TI summaries at the DAG list level. Please let me know if that’s okay. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow 3.0.x : Bring back from Airflow 2.x "Status of all previous DAG runs" column in DAGS main page [airflow]
pierrejeambrun commented on issue #50624: URL: https://github.com/apache/airflow/issues/50624#issuecomment-3197454871 What is mentioned by @nathadfield isn't planned at the moment but most likely not very hard to implement and could be a nice addition (TI summaries at the DAG list level for the most recent dag run, or last 14 runs which are those shown in the 'card') I am not aware of anybody working on this at the moment, I have added the `good first issue` label to bring attention to it. I believe that with the recent addition to the grid endpoints we can retrieve that TI summaries information for a single dagrun and use it directly in the UI, we probably don't need any backend development. It'd be happy to review a PR that adds this. That would be great to have it for 3.1.0. On the other hand we are actively working on adding more filters to the DagRun, Dag, Task Instance, etc... tables for 3.1.0. With those, you should be able to work your way around that missing feature by hitting the TaskInstance table and applying your 'date range' + 'status' + 'dag ids' filters to achieve something similar. What information are you interested in with that TI summary at a DAG level? I'm asking this because depending on your use case I can think of a few reason why this could be useful directly on the Dashboard (home) page, like monitoring metrics for your cluster. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow 3.0.x : Bring back from Airflow 2.x "Status of all previous DAG runs" column in DAGS main page [airflow]
nathadfield commented on issue #50624: URL: https://github.com/apache/airflow/issues/50624#issuecomment-3183453034 @uranusjr At the very least, I would like to see Task State indicators for the most recent DagRun as part of 3.1.0 but I don't know if anyone is actually planning to do this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow 3.0.x : Bring back from Airflow 2.x "Status of all previous DAG runs" column in DAGS main page [airflow]
uranusjr commented on issue #50624: URL: https://github.com/apache/airflow/issues/50624#issuecomment-3183442039 What should we do here? Is it still on track for the 3.1.0 timeline, or should it be scheduled for a later release? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow 3.0.x : Bring back from Airflow 2.x "Status of all previous DAG runs" column in DAGS main page [airflow]
pierrejeambrun commented on issue #50624: URL: https://github.com/apache/airflow/issues/50624#issuecomment-3089843716 I think that allowing more advanced filtering options on TaskInstance table (start_date, end_date, dag_ids, run_ids, etc...) will fill that void. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow 3.0.x : Bring back from Airflow 2.x "Status of all previous DAG runs" column in DAGS main page [airflow]
bbovenzi commented on issue #50624: URL: https://github.com/apache/airflow/issues/50624#issuecomment-3078826879 Makes sense. We can certainly add more info here. and make it more visual too https://github.com/user-attachments/assets/f20ff4bf-bca7-4c28-8207-49f5eed4353b"; /> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow 3.0.x : Bring back from Airflow 2.x "Status of all previous DAG runs" column in DAGS main page [airflow]
nathadfield commented on issue #50624: URL: https://github.com/apache/airflow/issues/50624#issuecomment-3077336407 It's not just about that DAG Run status either. In #51653, I talk about the Task state indicators too and how in v2 these provided a quick way to assess the operational status. Basically, I still want to quickly see a task status summary from the most recent DagRun similar to that which was previously provided by `Recent Tasks`. This doesn't seem to present the same issues as the DagRun history so I hope that something can be done in this area. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow 3.0.x : Bring back from Airflow 2.x "Status of all previous DAG runs" column in DAGS main page [airflow]
bbovenzi commented on issue #50624: URL: https://github.com/apache/airflow/issues/50624#issuecomment-3075209714 Right now, the bar charts show the last 14. Personally, I am focused on other features for 3.1.0. But I will welcome discussion and a PR for a design that can show a better summary. Do we simply show more bars in the bar chart? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow 3.0.x : Bring back from Airflow 2.x "Status of all previous DAG runs" column in DAGS main page [airflow]
nicku33 commented on issue #50624: URL: https://github.com/apache/airflow/issues/50624#issuecomment-3066019604 All of history for dag runs was always a pain point for me. I think a reasonable default should be last 30 or 60 days for dags, that is configurable. Having the ability to filter via a dropdown is good but only if it resets to a default. I know it's hard to pick a default. Some people will want 7 days, some people 90 days. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow 3.0.x : Bring back from Airflow 2.x "Status of all previous DAG runs" column in DAGS main page [airflow]
nathadfield commented on issue #50624: URL: https://github.com/apache/airflow/issues/50624#issuecomment-2967373533 Sorry. Didn't mean to close. 🤦 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow 3.0.x : Bring back from Airflow 2.x "Status of all previous DAG runs" column in DAGS main page [airflow]
nathadfield closed issue #50624: Airflow 3.0.x : Bring back from Airflow 2.x "Status of all previous DAG runs" column in DAGS main page URL: https://github.com/apache/airflow/issues/50624 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow 3.0.x : Bring back from Airflow 2.x "Status of all previous DAG runs" column in DAGS main page [airflow]
nathadfield commented on issue #50624: URL: https://github.com/apache/airflow/issues/50624#issuecomment-2967368092 Throwing my input on to this as I logged the related above request. Personally, I really liked the `Recent Tasks` status circles as they would gave quick, at-a-glance run down of what the current state is. Also the dynamic nature of them conveys a sense of activity which I seems a bit lacking in v3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow 3.0.x : Bring back from Airflow 2.x "Status of all previous DAG runs" column in DAGS main page [airflow]
gbonazzoli commented on issue #50624: URL: https://github.com/apache/airflow/issues/50624#issuecomment-2891553944 Time range filtering of dags is a great idea ! And with this filtering active, the numbers calculated on "Status of filtered DAG runs" is not noisy anymore ! in my opinion with such a filter in place it will better the user experience in comparison to Airflow 2! If I can suggest, time range filter in Grafana is something to take inspiration at: https://github.com/user-attachments/assets/9115d6c3-6da2-42ba-820c-d4098b0ca45a"; /> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow 3.0.x : Bring back from Airflow 2.x "Status of all previous DAG runs" column in DAGS main page [airflow]
bbovenzi commented on issue #50624: URL: https://github.com/apache/airflow/issues/50624#issuecomment-2891100863 Yes, I would like to explore improving the UI and filtering functionality before adding new UI config. But something like "List all dags with activity in the last 24 hours" we could then show useful more useful information in the card list than today -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow 3.0.x : Bring back from Airflow 2.x "Status of all previous DAG runs" column in DAGS main page [airflow]
pierrejeambrun commented on issue #50624: URL: https://github.com/apache/airflow/issues/50624#issuecomment-2890927462 Maybe what we want is a way to filter failed task instances, over a temporal window + state + list of dag ids ? For instance this would allow you to find all the failed task for your given dags over the last 24 hours. We can also imagine a more complexe filtering options for the dag list. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow 3.0.x : Bring back from Airflow 2.x "Status of all previous DAG runs" column in DAGS main page [airflow]
pierrejeambrun commented on issue #50624: URL: https://github.com/apache/airflow/issues/50624#issuecomment-2890919044 I'm not in favor of adding more conf options for the UI. Ideally the UI should solve this problem by its own with an appropriate user interface, allowing to do what's needed to solve this use case. (define a column order or any other solution). The main reason is that I wouldn't want to see a multiplicity of such params `web_ui_dags_page` `web_ui_ti_page` `web_ui_asset_page` and so on. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow 3.0.x : Bring back from Airflow 2.x "Status of all previous DAG runs" column in DAGS main page [airflow]
gbonazzoli commented on issue #50624: URL: https://github.com/apache/airflow/issues/50624#issuecomment-2888157383 @bbovenzi I agree with you, sometimes the information carried by the coloured circles is irrelevant, but sometimes it is not. What about having a configuration directive in `airflow.cfg` to make parametric the order in which to display the blocks on the web UI ? Something like: `web_ui_dags_page = "#Dag #Schedule #DAG_Runs_Status #Next_Dag_Run #Last_Dag_Run #Tags"` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
