GitHub user bangbangDong closed a discussion: log encoding and reading in webserver
In the task, I used SubprocessHook and set output_encoding="gb18030" in run_command. Code: SubprocessHook().run_command(command, env=env, cwd=work_dir, output_encoding="gb18030") In this case, the webserver reports an error: The File "/ home/TTRS airflow_dir/venv/lib/python3.11 / site - packages/flask/app. Py", line 1823, in full_dispatch_request rv = self.dispatch_request() The File "/ home/TTRS airflow_dir/venv/lib/python3.11 / site - packages/flask/app. Py", line 1799, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) The File "/ home/TTRS airflow_dir/venv/lib/python3.11 / site - packages/connexion/decorators/decorator. Py", line 68, in wrapper response = function(request) The File "/ home/TTRS airflow_dir/venv/lib/python3.11 / site - packages/connexion/decorators/uri_parsing. Py", line 149, in wrapper response = function(request) The File "/ home/TTRS airflow_dir/venv/lib/python3.11 / site - packages/connexion/decorators/validation. Py", line 399, in wrapper return function(request) The File "/ home/TTRS airflow_dir/venv/lib/python3.11 / site - packages/connexion/decorators/response. Py", line 113, in wrapper return _wrapper(request, response) The File "/ home/TTRS airflow_dir/venv/lib/python3.11 / site - packages/connexion/decorators/response. Py", line 90, in _wrapper self.operation.api.get_connexion_response(response, self.mimetype) File "/ home/TTRS airflow_dir/venv/lib/python3.11 / site - packages/connexion/apis/abstract. Py", line 366, in get_connexion_response return cls._framework_to_connexion_response(response=response, mimetype=mimetype) The File "/ home/TTRS airflow_dir/venv/lib/python3.11 / site - packages/connexion/apis/flask_api. Py", line 165, in _framework_to_connexion_response body=response.get_data() if not response.direct_passthrough else None, The File "/ home/TTRS airflow_dir/venv/lib/python3.11 / site - packages/werkzeug/wrappers/response. Py", line 314, in get_data self._ensure_sequence() The File "/ home/TTRS airflow_dir/venv/lib/python3.11 / site - packages/werkzeug/wrappers/response. Py", line 376, in _ensure_sequence self.make_sequence() The File "/ home/TTRS airflow_dir/venv/lib/python3.11 / site - packages/werkzeug/wrappers/response. Py", line 391, in make_sequence self.response = list(self.iter_encoded()) The File "/ home/TTRS airflow_dir/venv/lib/python3.11 / site - packages/werkzeug/wrappers/response. Py", line, 50 in _iter_encoded for item in iterable: The File "/ home/TTRS airflow_dir/venv/lib/python3.11 / site - packages/airflow/utils/log/log_reader py", line 87, in read_log_stream logs, metadata = self.read_log_chunks(ti, current_try_number, metadata) The File "/ home/TTRS airflow_dir/venv/lib/python3.11 / site - packages/airflow/utils/log/log_reader py", line 64, in read_log_chunks logs, metadatas = self.log_handler.read(ti, try_number, metadata=metadata) The File "/ home/TTRS airflow_dir/venv/lib/python3.11 / site - packages/airflow/utils/log/file_task_handler py", line 491, in read log, out_metadata = self._read(task_instance, try_number_element, metadata) The File "/ home/TTRS airflow_dir/venv/lib/python3.11 / site - packages/airflow/utils/log/file_task_handler py", line 398, in _read local_messages, local_logs = self._read_from_local(worker_log_full_path) The File "/ home/TTRS airflow_dir/venv/lib/python3.11 / site - packages/airflow/utils/log/file_task_handler py", line 569, in _read_from_local logs = [file.read_text() for file in paths] The File "/ home/TTRS airflow_dir/venv/lib/python3.11 / site - packages/airflow/utils/log/file_task_handler py", line 569, in <listcomp> logs = [file.read_text() for file in paths] The File "/ opt/tsysmart/local/python/lib/python3.11 / pathlib py", line 1059, in read_text return f.read() UnicodeDecodeError: 'gb18030' codec can't decode byte 0xae in position 2978: illegal multibyte sequence GitHub link: https://github.com/apache/airflow/discussions/43431 ---- This is an automatically sent email for commits@airflow.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@airflow.apache.org