[ https://issues.apache.org/jira/browse/AIRFLOW-2939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kaxil Naik resolved AIRFLOW-2939. --------------------------------- Resolution: Fixed Fix Version/s: 1.10.2 Resolved by https://github.com/apache/incubator-airflow/pull/4371 > `set` fails in case of `exisiting_files is None` and in case of `json.dumps` > ---------------------------------------------------------------------------- > > Key: AIRFLOW-2939 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2939 > Project: Apache Airflow > Issue Type: Bug > Components: operators > Affects Versions: 2.0.0 > Reporter: Kiyoshi Nomo > Assignee: Yohei Onishi > Priority: Major > Fix For: 1.10.2 > > > h1. Problems > h2. TypeError: 'NoneType' object is not iterable > [https://github.com/apache/incubator-airflow/blob/06b62c42b0b55ea55b86b130317594738d2f36a2/airflow/contrib/operators/gcs_to_s3.py#L91] > > {code:java} > >>> set(None) > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > TypeError: 'NoneType' object is not iterable > {code} > > h2. TypeError: set(['a']) is not JSON serializable > [https://github.com/apache/incubator-airflow/blob/b78c7fb8512f7a40f58b46530e9b3d5562fe84ea/airflow/models.py#L4483] > > {code:python} > >>> json.dumps(set(['a'])) > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/usr/local/opt/pyenv/versions/2.7.11/lib/python2.7/json/__init__.py", > line 244, in dumps > return _default_encoder.encode(obj) > File "/usr/local/opt/pyenv/versions/2.7.11/lib/python2.7/json/encoder.py", > line 207, in encode > chunks = self.iterencode(o, _one_shot=True) > File "/usr/local/opt/pyenv/versions/2.7.11/lib/python2.7/json/encoder.py", > line 270, in iterencode > return _iterencode(o, 0) > File "/usr/local/opt/pyenv/versions/2.7.11/lib/python2.7/json/encoder.py", > line 184, in default > raise TypeError(repr(o) + " is not JSON serializable") > TypeError: set(['a']) is not JSON serializable > {code} > > h1. Solution > * Check that the existing fils is not None. > * Convert it to the `set` and return it to the `list` after get to the > difference of files. > {code:python} > if existing_files is not None: > files = list(set(files) - set(existing_files)) > {code} > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)