gfeldman commented on a change in pull request #6643: [AIRFLOW-6040] Fix 
KubernetesJobWatcher Read time out error
URL: https://github.com/apache/airflow/pull/6643#discussion_r350343567
 
 

 ##########
 File path: airflow/executors/kubernetes_executor.py
 ##########
 @@ -265,12 +265,12 @@ def run(self):
 
     def _run(self, kube_client, resource_version, worker_uuid, kube_config):
         self.log.info(
-            'Event: and now my watch begins starting at resource_version: %s',
-            resource_version
-        )
+            'Event: and now my watch begins starting at resource_version: %s, '
+            'worker_uuid: %s', resource_version, worker_uuid)
         watcher = watch.Watch()
 
-        kwargs = {'label_selector': 'airflow-worker={}'.format(worker_uuid)}
+        kwargs = {'label_selector': 'airflow-worker={}'.format(worker_uuid),
+                  'timeout_seconds': 50}
 
 Review comment:
   If `kube_client_request_args` is used the Kubernetes executor fails to kick 
off tasks and the scheduler throws this exception:
   ```
   [2019-11-25 18:02:53,397] {scheduler_job.py:1352} ERROR - Exception when 
executing execute_helper
   Traceback (most recent call last):
     File 
"/usr/local/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py", line 
1350, in _execute
       self._execute_helper()
     File 
"/usr/local/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py", line 
1439, in _execute_helper
       self.executor.heartbeat()
     File 
"/usr/local/lib/python3.7/site-packages/airflow/executors/base_executor.py", 
line 136, in heartbeat
       self.sync()
     File 
"/usr/local/lib/python3.7/site-packages/airflow/contrib/executors/kubernetes_executor.py",
 line 801, in sync
       self.kube_scheduler.run_next(task)
     File 
"/usr/local/lib/python3.7/site-packages/airflow/contrib/executors/kubernetes_executor.py",
 line 456, in run_next
       self.launcher.run_pod_async(pod, 
**self.kube_config.kube_client_request_args)
     File 
"/usr/local/lib/python3.7/site-packages/airflow/contrib/kubernetes/pod_launcher.py",
 line 62, in run_pod_async
       resp = self._client.create_namespaced_pod(body=req, 
namespace=pod.namespace, **kwargs)
     File 
"/usr/local/lib/python3.7/site-packages/kubernetes/client/apis/core_v1_api.py", 
line 6115, in create_namespaced_pod
       (data) = self.create_namespaced_pod_with_http_info(namespace, body, 
**kwargs)
     File 
"/usr/local/lib/python3.7/site-packages/kubernetes/client/apis/core_v1_api.py", 
line 6148, in create_namespaced_pod_with_http_info
       " to method create_namespaced_pod" % key
   TypeError: Got an unexpected keyword argument 'timeout_seconds' to method 
create_namespaced_pod
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[email protected]


With regards,
Apache Git Services

Reply via email to