[
https://issues.apache.org/jira/browse/AIRFLOW-2363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16449156#comment-16449156
]
Kevin Yang commented on AIRFLOW-2363:
-------------------------------------
Hi [~hamlinkn], I've created a PR to fix it but I think I didn't tag you
correctly on github.... [https://github.com/apache/incubator-airflow/pull/3259]
Do you think it is possible for you to test it in your infra end to end? It
might take some extra work to have S3 set up on my side and I think you might
benefit from a faster merge.
Thank you!
> S3 remote logging appending tuple instead of str
> ------------------------------------------------
>
> Key: AIRFLOW-2363
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2363
> Project: Apache Airflow
> Issue Type: Bug
> Components: logging
> Reporter: Kyle Hamlin
> Assignee: Kevin Yang
> Priority: Major
> Fix For: 1.10.0
>
>
> A recent merge into master that added support for Elasticsearch logging seems
> to have broken S3 logging by returning a tuple instead of a string.
> [https://github.com/apache/incubator-airflow/commit/ec38ba9594395de04ec932481212a86fbe9ae107#diff-0442332ecbe42ebbf426911c68d8cd4aR128]
>
> following errors thrown:
>
> *Session NoneType error*
> Traceback (most recent call last):
> File
> "/usr/local/lib/python3.6/site-packages/airflow/utils/log/s3_task_handler.py",
> line 171, in s3_write
> encrypt=configuration.conf.getboolean('core', 'ENCRYPT_S3_LOGS'),
> File "/usr/local/lib/python3.6/site-packages/airflow/hooks/S3_hook.py",
> line 274, in load_string
> encrypt=encrypt)
> File "/usr/local/lib/python3.6/site-packages/airflow/hooks/S3_hook.py",
> line 313, in load_bytes
> client = self.get_conn()
> File "/usr/local/lib/python3.6/site-packages/airflow/hooks/S3_hook.py",
> line 34, in get_conn
> return self.get_client_type('s3')
> File
> "/usr/local/lib/python3.6/site-packages/airflow/contrib/hooks/aws_hook.py",
> line 151, in get_client_type
> session, endpoint_url = self._get_credentials(region_name)
> File
> "/usr/local/lib/python3.6/site-packages/airflow/contrib/hooks/aws_hook.py",
> line 97, in _get_credentials
> connection_object = self.get_connection(self.aws_conn_id)
> File "/usr/local/lib/python3.6/site-packages/airflow/hooks/base_hook.py",
> line 82, in get_connection
> conn = random.choice(cls.get_connections(conn_id))
> File "/usr/local/lib/python3.6/site-packages/airflow/hooks/base_hook.py",
> line 77, in get_connections
> conns = cls._get_connections_from_db(conn_id)
> File "/usr/local/lib/python3.6/site-packages/airflow/utils/db.py", line
> 72, in wrapper
> with create_session() as session:
> File "/usr/local/lib/python3.6/contextlib.py", line 81, in __enter__
> return next(self.gen)
> File "/usr/local/lib/python3.6/site-packages/airflow/utils/db.py", line
> 41, in create_session
> session = settings.Session()
> TypeError: 'NoneType' object is not callable
>
> *TypeError must be str not tuple*
> [2018-04-16 18:37:28,200] ERROR in app: Exception on
> /admin/airflow/get_logs_with_metadata [GET]
> Traceback (most recent call last):
> File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1982, in
> wsgi_app
> response = self.full_dispatch_request()
> File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1614, in
> full_dispatch_request
> rv = self.handle_user_exception(e)
> File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1517, in
> handle_user_exception
> reraise(exc_type, exc_value, tb)
> File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 33,
> in reraise
> raise value
> File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1612, in
> full_dispatch_request
> rv = self.dispatch_request()
> File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1598, in
> dispatch_request
> return self.view_functions[rule.endpoint](**req.view_args)
> File "/usr/local/lib/python3.6/site-packages/flask_admin/base.py", line
> 69, in inner
> return self._run_view(f, *args, **kwargs)
> File "/usr/local/lib/python3.6/site-packages/flask_admin/base.py", line
> 368, in _run_view
> return fn(self, *args, **kwargs)
> File "/usr/local/lib/python3.6/site-packages/flask_login.py", line 755, in
> decorated_view
> return func(*args, **kwargs)
> File "/usr/local/lib/python3.6/site-packages/airflow/www/utils.py", line
> 269, in wrapper
> return f(*args, **kwargs)
> File "/usr/local/lib/python3.6/site-packages/airflow/utils/db.py", line
> 74, in wrapper
> return func(*args, **kwargs)
> File "/usr/local/lib/python3.6/site-packages/airflow/www/views.py", line
> 770, in get_logs_with_metadata
> logs, metadatas = handler.read(ti, try_number, metadata=metadata)
> File
> "/usr/local/lib/python3.6/site-packages/airflow/utils/log/file_task_handler.py",
> line 165, in read
> logs[i] += log
> TypeError: must be str, not tuple
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
