[jira] [Assigned] (AIRFLOW-5540) task with SparkSubmitOperator does not fail if the spark job it executes fails.
[ https://issues.apache.org/jira/browse/AIRFLOW-5540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adan Christian Rosales Ornelas reassigned AIRFLOW-5540: --- Assignee: Adan Christian Rosales Ornelas > task with SparkSubmitOperator does not fail if the spark job it executes > fails. > -- > > Key: AIRFLOW-5540 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5540 > Project: Apache Airflow > Issue Type: Wish > Components: operators >Affects Versions: 1.10.3 > Environment: RHEL 7.3 with default airflow installation. >Reporter: Juan M George >Assignee: Adan Christian Rosales Ornelas >Priority: Major > Attachments: airfow_issues > > > In my test Dag, I have a task that uses SparkSubmitOperator operator to > execute a spark job that reads from a table in a database and does some > processing and write into a file. In a scenario, where source table from > where I read data does not exist, the spark job fails but the task is > still be shown as executed successfully. I don't see any other way to > handle this business logic failure from the operator side. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (AIRFLOW-2024) Add DAG execution (duration) time information in the Web UI
[ https://issues.apache.org/jira/browse/AIRFLOW-2024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938977#comment-16938977 ] Adan Christian Rosales Ornelas edited comment on AIRFLOW-2024 at 9/26/19 9:53 PM: -- [~kaxilnaik] [~jghoman] As per attached screen shoots looks like this work is already done right? was (Author: adankro): [~kaxilnaik] As per attached screen shoots looks like this work is already done right? > Add DAG execution (duration) time information in the Web UI > --- > > Key: AIRFLOW-2024 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2024 > Project: Apache Airflow > Issue Type: New Feature > Components: ui >Reporter: Kaxil Naik >Assignee: Adan Christian Rosales Ornelas >Priority: Minor > Attachments: Screen Shot 2019-09-26 at 4.16.01 PM.png, Screen Shot > 2019-09-26 at 4.16.26 PM.png > > > It would be good to know the execution time or the duration of DAG run > information in the Web UI. > > Currently this needs to be calculated manually by looking at the start time > of the first task in the dag and the end time of the last task in the dag. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-2024) Add DAG execution (duration) time information in the Web UI
[ https://issues.apache.org/jira/browse/AIRFLOW-2024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938977#comment-16938977 ] Adan Christian Rosales Ornelas commented on AIRFLOW-2024: - [~kaxilnaik] As per attached screen shoots looks like this work is already done right? > Add DAG execution (duration) time information in the Web UI > --- > > Key: AIRFLOW-2024 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2024 > Project: Apache Airflow > Issue Type: New Feature > Components: ui >Reporter: Kaxil Naik >Assignee: Adan Christian Rosales Ornelas >Priority: Minor > Attachments: Screen Shot 2019-09-26 at 4.16.01 PM.png, Screen Shot > 2019-09-26 at 4.16.26 PM.png > > > It would be good to know the execution time or the duration of DAG run > information in the Web UI. > > Currently this needs to be calculated manually by looking at the start time > of the first task in the dag and the end time of the last task in the dag. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-2024) Add DAG execution (duration) time information in the Web UI
[ https://issues.apache.org/jira/browse/AIRFLOW-2024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adan Christian Rosales Ornelas updated AIRFLOW-2024: Attachment: Screen Shot 2019-09-26 at 4.16.26 PM.png Screen Shot 2019-09-26 at 4.16.01 PM.png > Add DAG execution (duration) time information in the Web UI > --- > > Key: AIRFLOW-2024 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2024 > Project: Apache Airflow > Issue Type: New Feature > Components: ui >Reporter: Kaxil Naik >Assignee: Adan Christian Rosales Ornelas >Priority: Minor > Attachments: Screen Shot 2019-09-26 at 4.16.01 PM.png, Screen Shot > 2019-09-26 at 4.16.26 PM.png > > > It would be good to know the execution time or the duration of DAG run > information in the Web UI. > > Currently this needs to be calculated manually by looking at the start time > of the first task in the dag and the end time of the last task in the dag. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (AIRFLOW-2024) Add DAG execution (duration) time information in the Web UI
[ https://issues.apache.org/jira/browse/AIRFLOW-2024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adan Christian Rosales Ornelas reassigned AIRFLOW-2024: --- Assignee: Adan Christian Rosales Ornelas > Add DAG execution (duration) time information in the Web UI > --- > > Key: AIRFLOW-2024 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2024 > Project: Apache Airflow > Issue Type: New Feature > Components: ui >Reporter: Kaxil Naik >Assignee: Adan Christian Rosales Ornelas >Priority: Minor > > It would be good to know the execution time or the duration of DAG run > information in the Web UI. > > Currently this needs to be calculated manually by looking at the start time > of the first task in the dag and the end time of the last task in the dag. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (AIRFLOW-5495) Remove unneeded parens in dataproc.py
[
https://issues.apache.org/jira/browse/AIRFLOW-5495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adan Christian Rosales Ornelas reassigned AIRFLOW-5495:
---
Assignee: Adan Christian Rosales Ornelas
> Remove unneeded parens in dataproc.py
> -
>
> Key: AIRFLOW-5495
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5495
> Project: Apache Airflow
> Issue Type: Bug
> Components: operators
>Affects Versions: 1.10.6
>Reporter: Jakob Homan
>Assignee: Adan Christian Rosales Ornelas
>Priority: Minor
> Labels: ccoss2019, newbie
>
> Note: This ticket's being created to facilitate a new contributor's workshop
> for Airflow. After the workshop has completed, I'll mark these all available
> for anyone that might like to take them on.
> The parens around {{self.custom_image_project_id}} don't need to be there; we
> should remove them.
> airflow/gcp/operators/dataproc.py:409
> {code:java}
> elif self.custom_image:
> project_id = self.custom_image_project_id if
> (self.custom_image_project_id) else self.project_id
> custom_image_url = 'https://www.googleapis.com/compute/beta/projects/' \
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
