karakanb commented on issue #33066:
URL: https://github.com/apache/airflow/issues/33066#issuecomment-1667591575
done that but doesn't seem like much changed:
```
$ cat
/home/airflow/.local/lib/python3.10/site-packages/airflow/providers/cncf/kubernetes/executors/kubernetes_executor_utils.py
| grep Unknown
self.log.exception("Unknown error in KubernetesJobWatcher.
Failing", exc_info=True)
```
here's what it says with that change:
```
[2023-08-07T10:13:19.098+0000] {kubernetes_executor.py:114} ERROR - Unknown
error in KubernetesJobWatcher. Failing
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.10/site-packages/urllib3/response.py", line
761, in _update_chunk_length
self.chunk_left = int(line, 16)
ValueError: invalid literal for int() with base 16: b''
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.10/site-packages/urllib3/response.py", line
444, in _error_catcher
yield
File
"/home/airflow/.local/lib/python3.10/site-packages/urllib3/response.py", line
828, in read_chunked
self._update_chunk_length()
File
"/home/airflow/.local/lib/python3.10/site-packages/urllib3/response.py", line
765, in _update_chunk_length
raise InvalidChunkLength(self, line)
urllib3.exceptions.InvalidChunkLength: InvalidChunkLength(got length b'', 0
bytes read)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/executors/kubernetes_executor.py",
line 105, in run
self.resource_version = self._run(
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/executors/kubernetes_executor.py",
line 161, in _run
for event in self._pod_events(kube_client=kube_client,
query_kwargs=kwargs):
File
"/home/airflow/.local/lib/python3.10/site-packages/kubernetes/watch/watch.py",
line 165, in stream
for line in iter_resp_lines(resp):
File
"/home/airflow/.local/lib/python3.10/site-packages/kubernetes/watch/watch.py",
line 56, in iter_resp_lines
for seg in resp.stream(amt=None, decode_content=False):
File
"/home/airflow/.local/lib/python3.10/site-packages/urllib3/response.py", line
624, in stream
for line in self.read_chunked(amt, decode_content=decode_content):
File
"/home/airflow/.local/lib/python3.10/site-packages/urllib3/response.py", line
816, in read_chunked
with self._error_catcher():
File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
self.gen.throw(typ, value, traceback)
File
"/home/airflow/.local/lib/python3.10/site-packages/urllib3/response.py", line
461, in _error_catcher
raise ProtocolError("Connection broken: %r" % e, e)
urllib3.exceptions.ProtocolError: ("Connection broken:
InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got
length b'', 0 bytes read))
Process KubernetesJobWatcher-3:
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.10/site-packages/urllib3/response.py", line
761, in _update_chunk_length
self.chunk_left = int(line, 16)
ValueError: invalid literal for int() with base 16: b''
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.10/site-packages/urllib3/response.py", line
444, in _error_catcher
yield
File
"/home/airflow/.local/lib/python3.10/site-packages/urllib3/response.py", line
828, in read_chunked
self._update_chunk_length()
File
"/home/airflow/.local/lib/python3.10/site-packages/urllib3/response.py", line
765, in _update_chunk_length
raise InvalidChunkLength(self, line)
urllib3.exceptions.InvalidChunkLength: InvalidChunkLength(got length b'', 0
bytes read)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/multiprocessing/process.py", line 314, in
_bootstrap
self.run()
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/executors/kubernetes_executor.py",
line 105, in run
self.resource_version = self._run(
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/executors/kubernetes_executor.py",
line 161, in _run
for event in self._pod_events(kube_client=kube_client,
query_kwargs=kwargs):
File
"/home/airflow/.local/lib/python3.10/site-packages/kubernetes/watch/watch.py",
line 165, in stream
for line in iter_resp_lines(resp):
File
"/home/airflow/.local/lib/python3.10/site-packages/kubernetes/watch/watch.py",
line 56, in iter_resp_lines
for seg in resp.stream(amt=None, decode_content=False):
File
"/home/airflow/.local/lib/python3.10/site-packages/urllib3/response.py", line
624, in stream
for line in self.read_chunked(amt, decode_content=decode_content):
File
"/home/airflow/.local/lib/python3.10/site-packages/urllib3/response.py", line
816, in read_chunked
with self._error_catcher():
File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
self.gen.throw(typ, value, traceback)
File
"/home/airflow/.local/lib/python3.10/site-packages/urllib3/response.py", line
461, in _error_catcher
raise ProtocolError("Connection broken: %r" % e, e)
urllib3.exceptions.ProtocolError: ("Connection broken:
InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got
length b'', 0 bytes read))
[2023-08-07T10:13:19.432+0000] {kubernetes_executor.py:335} ERROR - Error
while health checking kube watcher process for namespace airflow. Process died
for unknown reasons
```
it looks like the same thing to me, maybe I have done the wrong change
somewhere?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
