[GitHub] [airflow] potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for newly added fields in BaseOperator
potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for newly added fields in BaseOperator URL: https://github.com/apache/airflow/pull/7162#discussion_r367413700 ## File path: tests/serialization/test_dag_serialization.py ## @@ -543,6 +543,68 @@ def test_dag_serialized_fields_with_schema(self): dag_params: set = set(dag_schema.keys()) - ignored_keys self.assertEqual(set(DAG.get_serialized_fields()), dag_params) +def test_no_new_fields_added_to_base_operator(self): +""" +This test verifies that there are no new fields added to BaseOperator. And reminds that +tests should be added for it. +""" +base_operator = BaseOperator(task_id="10") +fields = base_operator.__dict__ +self.assertEqual({'_dag': None, + '_downstream_task_ids': set(), + '_inlets': [], + '_log': base_operator.log, + '_outlets': [], + '_upstream_task_ids': set(), + 'depends_on_past': False, + 'do_xcom_push': True, + 'email': None, + 'email_on_failure': True, + 'email_on_retry': True, + 'end_date': None, + 'execution_timeout': None, + 'executor_config': {}, + 'inlets': [], + 'max_retry_delay': None, + 'on_execute_callback': None, + 'on_failure_callback': None, + 'on_retry_callback': None, + 'on_success_callback': None, + 'outlets': [], + 'owner': 'airflow', + 'params': {}, + 'pool': 'default_pool', + 'priority_weight': 1, + 'queue': 'default', + 'resources': None, + 'retries': 0, + 'retry_delay': timedelta(0, 300), + 'retry_exponential_backoff': False, + 'run_as_user': None, + 'sla': None, + 'start_date': None, + 'subdag': None, + 'task_concurrency': None, + 'task_id': '10', + 'trigger_rule': 'all_success', + 'wait_for_downstream': False, + 'weight_rule': 'downstream'}, fields, + """ +!!! + + ACTION NEEDED! PLEASE READ THIS CAREFULLY AND CORRECT TESTS CAREFULLY + + Some fields were added to the BaseOperator! Please add them to the list above and make sure that + you add support for DAG serialization - you should add the field to + `airflow/serialization/schema.json` - they should have correct type defined there. + + Note that we do not support versioning yet so you should only add optional fields. We do not support Review comment: ```suggestion Note that we do not support versioning yet so you should only add optional fields. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for newly added fields in BaseOperator
potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for newly added fields in BaseOperator URL: https://github.com/apache/airflow/pull/7162#discussion_r367413700 ## File path: tests/serialization/test_dag_serialization.py ## @@ -543,6 +543,68 @@ def test_dag_serialized_fields_with_schema(self): dag_params: set = set(dag_schema.keys()) - ignored_keys self.assertEqual(set(DAG.get_serialized_fields()), dag_params) +def test_no_new_fields_added_to_base_operator(self): +""" +This test verifies that there are no new fields added to BaseOperator. And reminds that +tests should be added for it. +""" +base_operator = BaseOperator(task_id="10") +fields = base_operator.__dict__ +self.assertEqual({'_dag': None, + '_downstream_task_ids': set(), + '_inlets': [], + '_log': base_operator.log, + '_outlets': [], + '_upstream_task_ids': set(), + 'depends_on_past': False, + 'do_xcom_push': True, + 'email': None, + 'email_on_failure': True, + 'email_on_retry': True, + 'end_date': None, + 'execution_timeout': None, + 'executor_config': {}, + 'inlets': [], + 'max_retry_delay': None, + 'on_execute_callback': None, + 'on_failure_callback': None, + 'on_retry_callback': None, + 'on_success_callback': None, + 'outlets': [], + 'owner': 'airflow', + 'params': {}, + 'pool': 'default_pool', + 'priority_weight': 1, + 'queue': 'default', + 'resources': None, + 'retries': 0, + 'retry_delay': timedelta(0, 300), + 'retry_exponential_backoff': False, + 'run_as_user': None, + 'sla': None, + 'start_date': None, + 'subdag': None, + 'task_concurrency': None, + 'task_id': '10', + 'trigger_rule': 'all_success', + 'wait_for_downstream': False, + 'weight_rule': 'downstream'}, fields, + """ +!!! + + ACTION NEEDED! PLEASE READ THIS CAREFULLY AND CORRECT TESTS CAREFULLY + + Some fields were added to the BaseOperator! Please add them to the list above and make sure that + you add support for DAG serialization - you should add the field to + `airflow/serialization/schema.json` - they should have correct type defined there. + + Note that we do not support versioning yet so you should only add optional fields. We do not support Review comment: ```suggestion Note that we do not support versioning yet so you should only add optional fields. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for newly added fields in BaseOperator
potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for newly added fields in BaseOperator URL: https://github.com/apache/airflow/pull/7162#discussion_r367413148 ## File path: tests/serialization/test_dag_serialization.py ## @@ -543,6 +543,68 @@ def test_dag_serialized_fields_with_schema(self): dag_params: set = set(dag_schema.keys()) - ignored_keys self.assertEqual(set(DAG.get_serialized_fields()), dag_params) +def test_no_new_fields_added_to_base_operator(self): +""" +This test verifies that there are no new fields added to BaseOperator. And reminds that +tests should be added for it. +""" +base_operator = BaseOperator(task_id="10") +fields = base_operator.__dict__ +self.assertEqual({'_dag': None, + '_downstream_task_ids': set(), + '_inlets': [], + '_log': base_operator.log, + '_outlets': [], + '_upstream_task_ids': set(), + 'depends_on_past': False, + 'do_xcom_push': True, + 'email': None, + 'email_on_failure': True, + 'email_on_retry': True, + 'end_date': None, + 'execution_timeout': None, + 'executor_config': {}, + 'inlets': [], + 'max_retry_delay': None, + 'on_execute_callback': None, + 'on_failure_callback': None, + 'on_retry_callback': None, + 'on_success_callback': None, + 'outlets': [], + 'owner': 'airflow', + 'params': {}, + 'pool': 'default_pool', + 'priority_weight': 1, + 'queue': 'default', + 'resources': None, + 'retries': 0, + 'retry_delay': timedelta(0, 300), + 'retry_exponential_backoff': False, + 'run_as_user': None, + 'sla': None, + 'start_date': None, + 'subdag': None, + 'task_concurrency': None, + 'task_id': '10', + 'trigger_rule': 'all_success', + 'wait_for_downstream': False, + 'weight_rule': 'downstream'}, fields, + """ +!!! + + ACTION NEEDED! PLEASE READ THIS CAREFULLY AND CORRECT TESTS CAREFULLY + + Some fields were added to the BaseOperator! Please add them to the list above and make sure that + you add support for DAG serialization - you should add the field to + `airflow/serialization/schema.json` - they should have correct type defined there. + + Note that we do not support versioning yet so you should only add optional fields. We do not support + versioning yet so you should make sure all fields added to the BaseOperator should be optional. Review comment: Ah yeah :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for newly added fields in BaseOperator
potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for newly added fields in BaseOperator URL: https://github.com/apache/airflow/pull/7162#discussion_r367100569 ## File path: tests/serialization/test_dag_serialization.py ## @@ -543,6 +543,66 @@ def test_dag_serialized_fields_with_schema(self): dag_params: set = set(dag_schema.keys()) - ignored_keys self.assertEqual(set(DAG.get_serialized_fields()), dag_params) +def test_no_new_fields_added_to_base_operator(self): +""" +This test verifies that there are no new fields added to BaseOperator. And reminds that +tests should be added for it. +""" +base_operator = BaseOperator(task_id="10") +fields = base_operator.__dict__ +self.assertEqual({'_dag': None, + '_downstream_task_ids': set(), + '_inlets': [], + '_log': base_operator.log, + '_outlets': [], + '_upstream_task_ids': set(), + 'depends_on_past': False, + 'do_xcom_push': True, + 'email': None, + 'email_on_failure': True, + 'email_on_retry': True, + 'end_date': None, + 'execution_timeout': None, + 'executor_config': {}, + 'inlets': [], + 'max_retry_delay': None, + 'on_execute_callback': None, + 'on_failure_callback': None, + 'on_retry_callback': None, + 'on_success_callback': None, + 'outlets': [], + 'owner': 'airflow', + 'params': {}, + 'pool': 'default_pool', + 'priority_weight': 1, + 'queue': 'default', + 'resources': None, + 'retries': 0, + 'retry_delay': timedelta(0, 300), + 'retry_exponential_backoff': False, + 'run_as_user': None, + 'sla': None, + 'start_date': None, + 'subdag': None, + 'task_concurrency': None, + 'task_id': '10', + 'trigger_rule': 'all_success', + 'wait_for_downstream': False, + 'weight_rule': 'downstream'}, fields, + """ +!!! + + ACTION NEEDED! PLEASE READ THIS CAREFULLY AND CORRECT TESTS CAREFULLY + + Some fields were added to the BaseOperator! Please add them to the list above and make sure that + you add support for DAG serialization - you should add the field to + `airflow/serialization/schema.json` and add it in `serialized_simple_dag_ground_truth` above Review comment: Similar rules as for protobuf. I will modify the message. We cannot test a lot of that fully automatically yet but we can at least provide the right message. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for newly added fields in BaseOperator
potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for newly added fields in BaseOperator URL: https://github.com/apache/airflow/pull/7162#discussion_r366367086 ## File path: tests/serialization/test_dag_serialization.py ## @@ -543,6 +543,66 @@ def test_dag_serialized_fields_with_schema(self): dag_params: set = set(dag_schema.keys()) - ignored_keys self.assertEqual(set(DAG.get_serialized_fields()), dag_params) +def test_no_new_fields_added_to_base_operator(self): +""" +This test verifies that there are no new fields added to BaseOperator. And reminds that +tests should be added for it. +""" +base_operator = BaseOperator(task_id="10") +fields = base_operator.__dict__ +self.assertEqual({'_dag': None, + '_downstream_task_ids': set(), + '_inlets': [], + '_log': base_operator.log, + '_outlets': [], + '_upstream_task_ids': set(), + 'depends_on_past': False, + 'do_xcom_push': True, + 'email': None, + 'email_on_failure': True, + 'email_on_retry': True, + 'end_date': None, + 'execution_timeout': None, + 'executor_config': {}, + 'inlets': [], + 'max_retry_delay': None, + 'on_execute_callback': None, + 'on_failure_callback': None, + 'on_retry_callback': None, + 'on_success_callback': None, + 'outlets': [], + 'owner': 'airflow', + 'params': {}, + 'pool': 'default_pool', + 'priority_weight': 1, + 'queue': 'default', + 'resources': None, + 'retries': 0, + 'retry_delay': timedelta(0, 300), + 'retry_exponential_backoff': False, + 'run_as_user': None, + 'sla': None, + 'start_date': None, + 'subdag': None, + 'task_concurrency': None, + 'task_id': '10', + 'trigger_rule': 'all_success', + 'wait_for_downstream': False, + 'weight_rule': 'downstream'}, fields, + """ +!!! + + ACTION NEEDED! PLEASE READ THIS CAREFULLY AND CORRECT TESTS CAREFULLY + + Some fields were added to the BaseOperator! Please add them to the list above and make sure that + you add support for DAG serialization - you should add the field to + `airflow/serialization/schema.json` and add it in `serialized_simple_dag_ground_truth` above Review comment: I think then we need to decide what to do in this case and tell @lokeshlal what to do in this case: https://github.com/apache/airflow/pull/7162 (and update this description). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services