nuclearpinguin commented on issue #6371: [AIRFLOW-5691] Rewrite Dataproc operators to use python library URL: https://github.com/apache/airflow/pull/6371#issuecomment-590757917 There's a JIRA: https://issues.apache.org/jira/browse/AIRFLOW-5691 No change was intentional. Also, this change was not backported to 1.10.x . So, from what I understand the submit job operator should: - check if job with provided id already exists - if yes then the existing job should be used - if no create new job Doesn't `request_id` work like that? > If the server receives two SubmitJobRequest requests with the same id, then the second request will be ignored and the first Job created and stored in the backend is returned. https://googleapis.dev/python/dataproc/latest/gapic/v1/api.html#google.cloud.dataproc_v1.JobControllerClient.submit_job @dossett @digger
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
