Repository: incubator-airflow Updated Branches: refs/heads/master d69fb31dd -> 62768bc08
[AIRFLOW-399] - Remove dags/testdruid.py Remove the current example in dags/testdruid.py. It requires the installation of an extraneous library (Hive) and executes a query against table that no one has. In its place, I am creating a simple DAG that only depends on DummyOperator and standard Python packages. Dear Airflow Maintainers, Please accept this PR that addresses the following issues: - https://issues.apache.org/jira/browse/AIRFLOW-399 Testing Done: Manual. aoen criccomini artwr jlowin bolkedebruin Closes #1709 from r39132/master Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/62768bc0 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/62768bc0 Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/62768bc0 Branch: refs/heads/master Commit: 62768bc087e95bf4506a8ec24d4291d7dfa7abaf Parents: d69fb31 Author: Siddharth Anand <siddharthan...@yahoo.com> Authored: Mon Aug 8 09:23:49 2016 -0700 Committer: Siddharth Anand <siddharthan...@yahoo.com> Committed: Mon Aug 8 09:23:49 2016 -0700 ---------------------------------------------------------------------- dags/test_dag.py | 37 +++++++++++++++++++++++++++++++++++++ dags/testdruid.py | 33 --------------------------------- 2 files changed, 37 insertions(+), 33 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/62768bc0/dags/test_dag.py ---------------------------------------------------------------------- diff --git a/dags/test_dag.py b/dags/test_dag.py new file mode 100644 index 0000000..9c506b2 --- /dev/null +++ b/dags/test_dag.py @@ -0,0 +1,37 @@ +# -*- coding: utf-8 -*- +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from airflow import DAG +from airflow.operators.dummy_operator import DummyOperator +from datetime import datetime + +now = datetime.now() +now_to_the_hour = now.replace(hour=now.time().hour-3 , minute=0, second=0, microsecond=0) +START_DATE = now_to_the_hour +DAG_NAME = 'test_dag_v1' + +default_args = { + 'owner': 'airflow', + 'depends_on_past': True, + 'start_date': START_DATE, +} +dag = DAG(DAG_NAME, schedule_interval='*/10 * * * *', default_args=default_args) + +run_this_1 = DummyOperator(task_id='run_this_1', dag=dag) +run_this_2 = DummyOperator(task_id='run_this_2', dag=dag) +run_this_2.set_upstream(run_this_1) +run_this_3 = DummyOperator(task_id='run_this_3', dag=dag) +run_this_3.set_upstream(run_this_2) + + http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/62768bc0/dags/testdruid.py ---------------------------------------------------------------------- diff --git a/dags/testdruid.py b/dags/testdruid.py deleted file mode 100644 index 2af8f51..0000000 --- a/dags/testdruid.py +++ /dev/null @@ -1,33 +0,0 @@ -# -*- coding: utf-8 -*- -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -from airflow.operators.hive_to_druid import HiveToDruidTransfer -from airflow import DAG -from datetime import datetime - -args = { - 'owner': 'qi_wang', - 'start_date': datetime(2015, 4, 4), -} - -dag = DAG("test_druid", default_args=args) - - -HiveToDruidTransfer(task_id="load_dummy_test", - sql="select * from qi.druid_test_dataset_w_platform_1 \ - limit 10;", - druid_datasource="airflow_test", - ts_dim="ds", - dag=dag - )
