[jira] [Commented] (AIRFLOW-2244) Exception importing Azure (wasb) log handler
[
https://issues.apache.org/jira/browse/AIRFLOW-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16414417#comment-16414417
]
ASF subversion and git services commented on AIRFLOW-2244:
--
Commit 1d8ed0925a0f0d229f71a3ade0386644d4dcff57 in incubator-airflow's branch
refs/heads/master from [~johnarnold]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=1d8ed09 ]
[AIRFLOW-2244] bugfix: remove legacy LongText code from models.py
Closes #3151 from johnarnold/settings_guard
> Exception importing Azure (wasb) log handler
>
>
> Key: AIRFLOW-2244
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2244
> Project: Apache Airflow
> Issue Type: Bug
> Components: core
>Reporter: John Arnold
>Priority: Major
> Fix For: 2.0.0
>
>
> Attempting to configure remote logging in azure storage, I'm getting import
> exceptions trying to load the wasb handler:
> Unable to load the config, contains a configuration error.
> Traceback (most recent call last):
> File "/usr/lib/python3.6/logging/config.py", line 382, in resolve
> found = getattr(found, frag)
> AttributeError: module 'airflow.utils.log' has no attribute
> 'wasb_task_handler'
> During handling of the above exception, another exception occurred:
> Traceback (most recent call last):
> File "/usr/lib/python3.6/logging/config.py", line 558, in configure
> handler = self.configure_handler(handlers[name])
> File "/usr/lib/python3.6/logging/config.py", line 708, in configure_handler
> klass = self.resolve(cname)
> File "/usr/lib/python3.6/logging/config.py", line 384, in resolve
> self.importer(used)
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/utils/log/wasb_task_handler.py",
> line 18, in
> from airflow.contrib.hooks.wasb_hook import WasbHook
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/contrib/hooks/wasb_hook.py",
> line 16, in
> from airflow.hooks.base_hook import BaseHook
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/hooks/base_hook.py",
> line 24, in
> from airflow.models import Connection
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/models.py",
> line 119, in
> if 'mysql' in settings.SQL_ALCHEMY_CONN:
> TypeError: argument of type 'NoneType' is not iterable
> During handling of the above exception, another exception occurred:
> Traceback (most recent call last):
> File "/var/lib/airflow/venv/bin/airflow", line 4, in
>
> __import__('pkg_resources').run_script('apache-airflow==1.10.0.dev0+incubating',
> 'airflow')
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/pkg_resources/__init__.py",
> line 658, in run_script
> self.require(requires)[0].run_script(script_name, ns)
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/pkg_resources/__init__.py",
> line 1438, in run_script
> exec(code, namespace, namespace)
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/EGG-INFO/scripts/airflow",
> line 16, in
> from airflow import configuration
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/__init__.py",
> line 31, in
> from airflow import settings
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/settings.py",
> line 198, in
> configure_logging()
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/logging_config.py",
> line 76, in configure_logging
> raise e
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/logging_config.py",
> line 71, in configure_logging
> dictConfig(logging_config)
> File "/usr/lib/python3.6/logging/config.py", line 795, in dictConfig
> dictConfigClass(config).configure()
> File "/usr/lib/python3.6/logging/config.py", line 566, in configure
> '%r: %s' % (name, e))
> ValueError: Unable to configure handler 'processor': argument of type
> 'NoneType' is not iterable
> (venv) johnar@netdev1-westus2:~/github/incubator-airflow$ airflow scheduler
> Unable to load the config, contains a configuration error.
> Traceback (most recent call last):
> File "/usr/lib/python3.6/logging/config.py", line 382, in resolve
> found = getattr(found, frag)
> AttributeError: module 'airflow.utils.log' has no attribute
> 'wasb_task_handler'
> During handling of the above exception, another exception occurred:
> Traceback (most rec
[jira] [Commented] (AIRFLOW-2244) Exception importing Azure (wasb) log handler
[
https://issues.apache.org/jira/browse/AIRFLOW-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16412325#comment-16412325
]
John Arnold commented on AIRFLOW-2244:
--
It doesn't appear that this LongText variable in models.py is even referenced
anywhere else -- can we just remove it?
> Exception importing Azure (wasb) log handler
>
>
> Key: AIRFLOW-2244
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2244
> Project: Apache Airflow
> Issue Type: Bug
> Components: core
>Reporter: John Arnold
>Priority: Major
>
> Attempting to configure remote logging in azure storage, I'm getting import
> exceptions trying to load the wasb handler:
> Unable to load the config, contains a configuration error.
> Traceback (most recent call last):
> File "/usr/lib/python3.6/logging/config.py", line 382, in resolve
> found = getattr(found, frag)
> AttributeError: module 'airflow.utils.log' has no attribute
> 'wasb_task_handler'
> During handling of the above exception, another exception occurred:
> Traceback (most recent call last):
> File "/usr/lib/python3.6/logging/config.py", line 558, in configure
> handler = self.configure_handler(handlers[name])
> File "/usr/lib/python3.6/logging/config.py", line 708, in configure_handler
> klass = self.resolve(cname)
> File "/usr/lib/python3.6/logging/config.py", line 384, in resolve
> self.importer(used)
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/utils/log/wasb_task_handler.py",
> line 18, in
> from airflow.contrib.hooks.wasb_hook import WasbHook
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/contrib/hooks/wasb_hook.py",
> line 16, in
> from airflow.hooks.base_hook import BaseHook
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/hooks/base_hook.py",
> line 24, in
> from airflow.models import Connection
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/models.py",
> line 119, in
> if 'mysql' in settings.SQL_ALCHEMY_CONN:
> TypeError: argument of type 'NoneType' is not iterable
> During handling of the above exception, another exception occurred:
> Traceback (most recent call last):
> File "/var/lib/airflow/venv/bin/airflow", line 4, in
>
> __import__('pkg_resources').run_script('apache-airflow==1.10.0.dev0+incubating',
> 'airflow')
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/pkg_resources/__init__.py",
> line 658, in run_script
> self.require(requires)[0].run_script(script_name, ns)
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/pkg_resources/__init__.py",
> line 1438, in run_script
> exec(code, namespace, namespace)
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/EGG-INFO/scripts/airflow",
> line 16, in
> from airflow import configuration
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/__init__.py",
> line 31, in
> from airflow import settings
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/settings.py",
> line 198, in
> configure_logging()
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/logging_config.py",
> line 76, in configure_logging
> raise e
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/logging_config.py",
> line 71, in configure_logging
> dictConfig(logging_config)
> File "/usr/lib/python3.6/logging/config.py", line 795, in dictConfig
> dictConfigClass(config).configure()
> File "/usr/lib/python3.6/logging/config.py", line 566, in configure
> '%r: %s' % (name, e))
> ValueError: Unable to configure handler 'processor': argument of type
> 'NoneType' is not iterable
> (venv) johnar@netdev1-westus2:~/github/incubator-airflow$ airflow scheduler
> Unable to load the config, contains a configuration error.
> Traceback (most recent call last):
> File "/usr/lib/python3.6/logging/config.py", line 382, in resolve
> found = getattr(found, frag)
> AttributeError: module 'airflow.utils.log' has no attribute
> 'wasb_task_handler'
> During handling of the above exception, another exception occurred:
> Traceback (most recent call last):
> File "/usr/lib/python3.6/logging/config.py", line 558, in configure
> handler = self.configure_handler(handlers[name])
> File "/usr/lib/python3.6/logging/config.py", line 708, in configure_handler
> klass = self.resolve(cname)
> File "/
[jira] [Commented] (AIRFLOW-2244) Exception importing Azure (wasb) log handler
[
https://issues.apache.org/jira/browse/AIRFLOW-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16410636#comment-16410636
]
John Arnold commented on AIRFLOW-2244:
--
in airflow/models.py:
if 'mysql' in settings.SQL_ALCHEMY_CONN:
LongText = LONGTEXT
else:
LongText = Text
This is not guarded at all, you can't iterate settings.SQL_ALCHEMY_CONN if it's
empty. You get a TypeError.
> Exception importing Azure (wasb) log handler
>
>
> Key: AIRFLOW-2244
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2244
> Project: Apache Airflow
> Issue Type: Bug
> Components: core
>Reporter: John Arnold
>Priority: Major
>
> Attempting to configure remote logging in azure storage, I'm getting import
> exceptions trying to load the wasb handler:
> Unable to load the config, contains a configuration error.
> Traceback (most recent call last):
> File "/usr/lib/python3.6/logging/config.py", line 382, in resolve
> found = getattr(found, frag)
> AttributeError: module 'airflow.utils.log' has no attribute
> 'wasb_task_handler'
> During handling of the above exception, another exception occurred:
> Traceback (most recent call last):
> File "/usr/lib/python3.6/logging/config.py", line 558, in configure
> handler = self.configure_handler(handlers[name])
> File "/usr/lib/python3.6/logging/config.py", line 708, in configure_handler
> klass = self.resolve(cname)
> File "/usr/lib/python3.6/logging/config.py", line 384, in resolve
> self.importer(used)
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/utils/log/wasb_task_handler.py",
> line 18, in
> from airflow.contrib.hooks.wasb_hook import WasbHook
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/contrib/hooks/wasb_hook.py",
> line 16, in
> from airflow.hooks.base_hook import BaseHook
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/hooks/base_hook.py",
> line 24, in
> from airflow.models import Connection
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/models.py",
> line 119, in
> if 'mysql' in settings.SQL_ALCHEMY_CONN:
> TypeError: argument of type 'NoneType' is not iterable
> During handling of the above exception, another exception occurred:
> Traceback (most recent call last):
> File "/var/lib/airflow/venv/bin/airflow", line 4, in
>
> __import__('pkg_resources').run_script('apache-airflow==1.10.0.dev0+incubating',
> 'airflow')
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/pkg_resources/__init__.py",
> line 658, in run_script
> self.require(requires)[0].run_script(script_name, ns)
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/pkg_resources/__init__.py",
> line 1438, in run_script
> exec(code, namespace, namespace)
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/EGG-INFO/scripts/airflow",
> line 16, in
> from airflow import configuration
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/__init__.py",
> line 31, in
> from airflow import settings
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/settings.py",
> line 198, in
> configure_logging()
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/logging_config.py",
> line 76, in configure_logging
> raise e
> File
> "/var/lib/airflow/venv/lib/python3.6/site-packages/apache_airflow-1.10.0.dev0+incubating-py3.6.egg/airflow/logging_config.py",
> line 71, in configure_logging
> dictConfig(logging_config)
> File "/usr/lib/python3.6/logging/config.py", line 795, in dictConfig
> dictConfigClass(config).configure()
> File "/usr/lib/python3.6/logging/config.py", line 566, in configure
> '%r: %s' % (name, e))
> ValueError: Unable to configure handler 'processor': argument of type
> 'NoneType' is not iterable
> (venv) johnar@netdev1-westus2:~/github/incubator-airflow$ airflow scheduler
> Unable to load the config, contains a configuration error.
> Traceback (most recent call last):
> File "/usr/lib/python3.6/logging/config.py", line 382, in resolve
> found = getattr(found, frag)
> AttributeError: module 'airflow.utils.log' has no attribute
> 'wasb_task_handler'
> During handling of the above exception, another exception occurred:
> Traceback (most recent call last):
> File "/usr/lib/python3.6/logging/config.py", line 558, in configure
> handler = self.configure_handler(handlers[name])
> File "/usr/lib/pytho
