[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r382581685 ## File path: airflow/providers/salesforce/example_dags/example_tableau_refresh_workbook.py ## @@ -0,0 +1,59 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" +This is an example dag that performs a refresh operation on a Tableau Workbook aka Extract. Since this is an +asynchronous operation we don't know when the operation actually finishes. That's why we have a second task +that checks exactly that. So that you can perform further operations after the extract has been refreshed. +""" +from datetime import timedelta + +from airflow import DAG +from airflow.providers.salesforce.operators.tableau_refresh_workbook import TableauRefreshWorkbookOperator +from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor +from airflow.utils.dates import days_ago + +DEFAULT_ARGS = { +'owner': 'airflow', +'depends_on_past': False, +'start_date': days_ago(2), +'email': ['airf...@example.com'], +'email_on_failure': False, +'email_on_retry': False +} + +with DAG( +dag_id='example_tableau_refresh_workbook', +default_args=DEFAULT_ARGS, +dagrun_timeout=timedelta(hours=2), +schedule_interval=None, +tags=['example'], +) as dag: +task_refresh_workbook = TableauRefreshWorkbookOperator( +site_id='my_site', +workbook_name='MyWorkbook', +task_id='refresh_tableau_workbook', +dag=dag +) +task_check_job_status = TableauJobStatusSensor( Review comment: Seems to work :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r382099957 ## File path: airflow/providers/salesforce/example_dags/example_tableau_refresh_workbook.py ## @@ -0,0 +1,59 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" +This is an example dag that performs a refresh operation on a Tableau Workbook aka Extract. Since this is an +asynchronous operation we don't know when the operation actually finishes. That's why we have a second task +that checks exactly that. So that you can perform further operations after the extract has been refreshed. +""" +from datetime import timedelta + +from airflow import DAG +from airflow.providers.salesforce.operators.tableau_refresh_workbook import TableauRefreshWorkbookOperator +from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor +from airflow.utils.dates import days_ago + +DEFAULT_ARGS = { +'owner': 'airflow', +'depends_on_past': False, +'start_date': days_ago(2), +'email': ['airf...@example.com'], +'email_on_failure': False, +'email_on_retry': False +} + +with DAG( +dag_id='example_tableau_refresh_workbook', +default_args=DEFAULT_ARGS, +dagrun_timeout=timedelta(hours=2), +schedule_interval=None, +tags=['example'], +) as dag: +task_refresh_workbook = TableauRefreshWorkbookOperator( +site_id='my_site', +workbook_name='MyWorkbook', +task_id='refresh_tableau_workbook', +dag=dag +) +task_check_job_status = TableauJobStatusSensor( Review comment: I need to fully test this behaviour first. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r382099699 ## File path: airflow/providers/salesforce/example_dags/example_tableau_refresh_workbook.py ## @@ -0,0 +1,59 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" +This is an example dag that performs a refresh operation on a Tableau Workbook aka Extract. Since this is an +asynchronous operation we don't know when the operation actually finishes. That's why we have a second task +that checks exactly that. So that you can perform further operations after the extract has been refreshed. +""" +from datetime import timedelta + +from airflow import DAG +from airflow.providers.salesforce.operators.tableau_refresh_workbook import TableauRefreshWorkbookOperator +from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor +from airflow.utils.dates import days_ago + +DEFAULT_ARGS = { +'owner': 'airflow', +'depends_on_past': False, +'start_date': days_ago(2), +'email': ['airf...@example.com'], +'email_on_failure': False, +'email_on_retry': False +} + +with DAG( +dag_id='example_tableau_refresh_workbook', +default_args=DEFAULT_ARGS, +dagrun_timeout=timedelta(hours=2), +schedule_interval=None, +tags=['example'], +) as dag: +task_refresh_workbook = TableauRefreshWorkbookOperator( +site_id='my_site', +workbook_name='MyWorkbook', +task_id='refresh_tableau_workbook', +dag=dag +) +task_check_job_status = TableauJobStatusSensor( Review comment: I am using the Sensor right know with the default mode `poke` means it will `sleep` :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r382025336 ## File path: airflow/providers/salesforce/example_dags/example_tableau_refresh_workbook.py ## @@ -0,0 +1,59 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" +This is an example dag that performs a refresh operation on a Tableau Workbook aka Extract. Since this is an +asynchronous operation we don't know when the operation actually finishes. That's why we have a second task +that checks exactly that. So that you can perform further operations after the extract has been refreshed. +""" +from datetime import timedelta + +from airflow import DAG +from airflow.providers.salesforce.operators.tableau_refresh_workbook import TableauRefreshWorkbookOperator +from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor +from airflow.utils.dates import days_ago + +DEFAULT_ARGS = { +'owner': 'airflow', +'depends_on_past': False, +'start_date': days_ago(2), +'email': ['airf...@example.com'], +'email_on_failure': False, +'email_on_retry': False +} + +with DAG( +dag_id='example_tableau_refresh_workbook', +default_args=DEFAULT_ARGS, +dagrun_timeout=timedelta(hours=2), +schedule_interval=None, +tags=['example'], +) as dag: +task_refresh_workbook = TableauRefreshWorkbookOperator( Review comment: I forgot to update this example. If we now have `block=True` by default means it will wait until the extract has been refreshed, we don't need the second task. Or I add `block=False` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r382025336 ## File path: airflow/providers/salesforce/example_dags/example_tableau_refresh_workbook.py ## @@ -0,0 +1,59 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" +This is an example dag that performs a refresh operation on a Tableau Workbook aka Extract. Since this is an +asynchronous operation we don't know when the operation actually finishes. That's why we have a second task +that checks exactly that. So that you can perform further operations after the extract has been refreshed. +""" +from datetime import timedelta + +from airflow import DAG +from airflow.providers.salesforce.operators.tableau_refresh_workbook import TableauRefreshWorkbookOperator +from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor +from airflow.utils.dates import days_ago + +DEFAULT_ARGS = { +'owner': 'airflow', +'depends_on_past': False, +'start_date': days_ago(2), +'email': ['airf...@example.com'], +'email_on_failure': False, +'email_on_retry': False +} + +with DAG( +dag_id='example_tableau_refresh_workbook', +default_args=DEFAULT_ARGS, +dagrun_timeout=timedelta(hours=2), +schedule_interval=None, +tags=['example'], +) as dag: +task_refresh_workbook = TableauRefreshWorkbookOperator( Review comment: I forgot to update this example. If we now have `blocking=True` by default means it will wait until the extract has been refreshed, we don't need the second task. Or I add `blocking=False` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381921320 ## File path: airflow/providers/salesforce/example_dags/example_tableau_refresh_workbook.py ## @@ -0,0 +1,59 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" +This is an example dag that performs a refresh operation on a Tableau Workbook aka Extract. Since this is an +asynchronous operation we don't know when the operation actually finishes. That's why we have a second task +that checks exactly that. So that you can perform further operations after the extract has been refreshed. +""" +from datetime import timedelta + +from airflow import DAG +from airflow.providers.salesforce.operators.tableau_refresh_workbook import TableauRefreshWorkbookOperator +from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor +from airflow.utils.dates import days_ago + +DEFAULT_ARGS = { +'owner': 'airflow', +'depends_on_past': False, +'start_date': days_ago(2), +'email': ['airf...@example.com'], +'email_on_failure': False, +'email_on_retry': False +} + +with DAG( +dag_id='example_tableau_refresh_workbook', +default_args=DEFAULT_ARGS, +dagrun_timeout=timedelta(hours=2), +schedule_interval=None, +tags=['example'], +) as dag: +task_refresh_workbook = TableauRefreshWorkbookOperator( +site_id='my_site', +workbook_name='MyWorkbook', +task_id='refresh_tableau_workbook', +dag=dag +) +task_check_job_status = TableauJobStatusSensor( Review comment: @Fokko it really sounds reasonable to me, but I tried to implement it and I don't think it works out nicely... :/ I don't like the two connections one for the refresh operation and one for the sensor task. If I use the code from the sensor directly - it is duplicated code... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381921320 ## File path: airflow/providers/salesforce/example_dags/example_tableau_refresh_workbook.py ## @@ -0,0 +1,59 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" +This is an example dag that performs a refresh operation on a Tableau Workbook aka Extract. Since this is an +asynchronous operation we don't know when the operation actually finishes. That's why we have a second task +that checks exactly that. So that you can perform further operations after the extract has been refreshed. +""" +from datetime import timedelta + +from airflow import DAG +from airflow.providers.salesforce.operators.tableau_refresh_workbook import TableauRefreshWorkbookOperator +from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor +from airflow.utils.dates import days_ago + +DEFAULT_ARGS = { +'owner': 'airflow', +'depends_on_past': False, +'start_date': days_ago(2), +'email': ['airf...@example.com'], +'email_on_failure': False, +'email_on_retry': False +} + +with DAG( +dag_id='example_tableau_refresh_workbook', +default_args=DEFAULT_ARGS, +dagrun_timeout=timedelta(hours=2), +schedule_interval=None, +tags=['example'], +) as dag: +task_refresh_workbook = TableauRefreshWorkbookOperator( +site_id='my_site', +workbook_name='MyWorkbook', +task_id='refresh_tableau_workbook', +dag=dag +) +task_check_job_status = TableauJobStatusSensor( Review comment: @Fokko it really sounds reasonable to me, but I tried to implement it and I don't think it works out nicely... :/ Here is what I have right now: ```python with TableauHook(self.site_id, self.tableau_conn_id) as tableau_hook: workbook = self._get_workbook_by_name(tableau_hook) job_id = self._refresh_workbook(tableau_hook, workbook.id) if self.blocking: from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor TableauJobStatusSensor( job_id=job_id, site_id=self.site_id, allowed_states=[TableauJobFinishCode.SUCCESS], tableau_conn_id=self.tableau_conn_id, task_id='wait_until_succeeded', dag=None ).poke(context={}) return job_id ``` **EDIT:** I don't like the two connections one for the refresh operation and one for the sensor task. If I use the code from the sensor directly - it is duplicated code... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381929302 ## File path: airflow/providers/salesforce/example_dags/example_tableau_refresh_workbook.py ## @@ -0,0 +1,59 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" +This is an example dag that performs a refresh operation on a Tableau Workbook aka Extract. Since this is an +asynchronous operation we don't know when the operation actually finishes. That's why we have a second task +that checks exactly that. So that you can perform further operations after the extract has been refreshed. +""" +from datetime import timedelta + +from airflow import DAG +from airflow.providers.salesforce.operators.tableau_refresh_workbook import TableauRefreshWorkbookOperator +from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor +from airflow.utils.dates import days_ago + +DEFAULT_ARGS = { +'owner': 'airflow', +'depends_on_past': False, +'start_date': days_ago(2), +'email': ['airf...@example.com'], +'email_on_failure': False, +'email_on_retry': False +} + +with DAG( +dag_id='example_tableau_refresh_workbook', +default_args=DEFAULT_ARGS, +dagrun_timeout=timedelta(hours=2), +schedule_interval=None, +tags=['example'], +) as dag: +task_refresh_workbook = TableauRefreshWorkbookOperator( +site_id='my_site', +workbook_name='MyWorkbook', +task_id='refresh_tableau_workbook', +dag=dag +) +task_check_job_status = TableauJobStatusSensor( Review comment: And do I wait until it has succeeded or until it has finished? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381921320 ## File path: airflow/providers/salesforce/example_dags/example_tableau_refresh_workbook.py ## @@ -0,0 +1,59 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" +This is an example dag that performs a refresh operation on a Tableau Workbook aka Extract. Since this is an +asynchronous operation we don't know when the operation actually finishes. That's why we have a second task +that checks exactly that. So that you can perform further operations after the extract has been refreshed. +""" +from datetime import timedelta + +from airflow import DAG +from airflow.providers.salesforce.operators.tableau_refresh_workbook import TableauRefreshWorkbookOperator +from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor +from airflow.utils.dates import days_ago + +DEFAULT_ARGS = { +'owner': 'airflow', +'depends_on_past': False, +'start_date': days_ago(2), +'email': ['airf...@example.com'], +'email_on_failure': False, +'email_on_retry': False +} + +with DAG( +dag_id='example_tableau_refresh_workbook', +default_args=DEFAULT_ARGS, +dagrun_timeout=timedelta(hours=2), +schedule_interval=None, +tags=['example'], +) as dag: +task_refresh_workbook = TableauRefreshWorkbookOperator( +site_id='my_site', +workbook_name='MyWorkbook', +task_id='refresh_tableau_workbook', +dag=dag +) +task_check_job_status = TableauJobStatusSensor( Review comment: @Fokko it really sounds reasonable to me, but I tried to implement it and I don't think it works out nicely... :/ Here is what I have right now: ```python with TableauHook(self.site_id, self.tableau_conn_id) as tableau_hook: workbook = self._get_workbook_by_name(tableau_hook) job_id = self._refresh_workbook(tableau_hook, workbook.id) if self.blocking: from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor TableauJobStatusSensor(job_id, self.site_id, self.tableau_conn_id).poke(context) return job_id ``` EDIT: I don't like the two connections one for the refresh operation and one for the sensor task. If I use the code from the sensor directly - it is duplicated code... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381921320 ## File path: airflow/providers/salesforce/example_dags/example_tableau_refresh_workbook.py ## @@ -0,0 +1,59 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" +This is an example dag that performs a refresh operation on a Tableau Workbook aka Extract. Since this is an +asynchronous operation we don't know when the operation actually finishes. That's why we have a second task +that checks exactly that. So that you can perform further operations after the extract has been refreshed. +""" +from datetime import timedelta + +from airflow import DAG +from airflow.providers.salesforce.operators.tableau_refresh_workbook import TableauRefreshWorkbookOperator +from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor +from airflow.utils.dates import days_ago + +DEFAULT_ARGS = { +'owner': 'airflow', +'depends_on_past': False, +'start_date': days_ago(2), +'email': ['airf...@example.com'], +'email_on_failure': False, +'email_on_retry': False +} + +with DAG( +dag_id='example_tableau_refresh_workbook', +default_args=DEFAULT_ARGS, +dagrun_timeout=timedelta(hours=2), +schedule_interval=None, +tags=['example'], +) as dag: +task_refresh_workbook = TableauRefreshWorkbookOperator( +site_id='my_site', +workbook_name='MyWorkbook', +task_id='refresh_tableau_workbook', +dag=dag +) +task_check_job_status = TableauJobStatusSensor( Review comment: @Fokko it really sounds reasonable to me, but I tried to implement it and I don't think it works out nicely... :/ Here is what I have right now: ```python with TableauHook(self.site_id, self.tableau_conn_id) as tableau_hook: workbook = self._get_workbook_by_name(tableau_hook) job_id = self._refresh_workbook(tableau_hook, workbook.id) if self.blocking: from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor TableauJobStatusSensor(job_id, self.site_id, self.tableau_conn_id).poke(context) return job_id ``` **EDIT:** I don't like the two connections one for the refresh operation and one for the sensor task. If I use the code from the sensor directly - it is duplicated code... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381921320 ## File path: airflow/providers/salesforce/example_dags/example_tableau_refresh_workbook.py ## @@ -0,0 +1,59 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" +This is an example dag that performs a refresh operation on a Tableau Workbook aka Extract. Since this is an +asynchronous operation we don't know when the operation actually finishes. That's why we have a second task +that checks exactly that. So that you can perform further operations after the extract has been refreshed. +""" +from datetime import timedelta + +from airflow import DAG +from airflow.providers.salesforce.operators.tableau_refresh_workbook import TableauRefreshWorkbookOperator +from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor +from airflow.utils.dates import days_ago + +DEFAULT_ARGS = { +'owner': 'airflow', +'depends_on_past': False, +'start_date': days_ago(2), +'email': ['airf...@example.com'], +'email_on_failure': False, +'email_on_retry': False +} + +with DAG( +dag_id='example_tableau_refresh_workbook', +default_args=DEFAULT_ARGS, +dagrun_timeout=timedelta(hours=2), +schedule_interval=None, +tags=['example'], +) as dag: +task_refresh_workbook = TableauRefreshWorkbookOperator( +site_id='my_site', +workbook_name='MyWorkbook', +task_id='refresh_tableau_workbook', +dag=dag +) +task_check_job_status = TableauJobStatusSensor( Review comment: @Fokko it really sounds reasonable to me, but I tried to implement it and I don't think it works out nicely... :/ Here is what I have right now: ```python with TableauHook(self.site_id, self.tableau_conn_id) as tableau_hook: workbook = self._get_workbook_by_name(tableau_hook) job_id = self._refresh_workbook(tableau_hook, workbook.id) if self.blocking: from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor TableauJobStatusSensor(job_id, self.site_id, self.tableau_conn_id).poke(context) return job_id ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381393733 ## File path: setup.py ## @@ -321,6 +321,7 @@ def write_version(filename: str = os.path.join(*["airflow", "git_version"])): ] salesforce = [ 'simple-salesforce>=0.72', +'tableauserverclient' Review comment: Yes, will do that. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381206745 ## File path: setup.py ## @@ -321,6 +321,7 @@ def write_version(filename: str = os.path.join(*["airflow", "git_version"])): ] salesforce = [ Review comment: If I put it into a separate group, does it mean it also belongs to the tableau provider? No, right? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381203640 ## File path: airflow/providers/salesforce/example_dags/example_tableau_refresh_workbook.py ## @@ -0,0 +1,59 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" +This is an example dag that performs a refresh operation on a Tableau Workbook aka Extract. Since this is an +asynchronous operation we don't know when the operation actually finishes. That's why we have a second task +that checks exactly that. So that you can perform further operations after the extract has been refreshed. +""" +from datetime import timedelta + +from airflow import DAG +from airflow.providers.salesforce.operators.tableau_refresh_workbook import TableauRefreshWorkbookOperator +from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor +from airflow.utils.dates import days_ago + +DEFAULT_ARGS = { +'owner': 'airflow', +'depends_on_past': False, +'start_date': days_ago(2), +'email': ['airf...@example.com'], +'email_on_failure': False, +'email_on_retry': False +} + +with DAG( +dag_id='example_tableau_refresh_workbook', +default_args=DEFAULT_ARGS, +dagrun_timeout=timedelta(hours=2), +schedule_interval=None, +tags=['example'], +) as dag: +task_refresh_workbook = TableauRefreshWorkbookOperator( +site_id='my_site', +workbook_name='MyWorkbook', +task_id='refresh_tableau_workbook', +dag=dag +) +task_check_job_status = TableauJobStatusSensor( Review comment: I could make this configurable so that after the refresh operation has been kicked off It will poke until it succeeds and then the task status will be success. But this would mean that you can have this bad case where the refresh operation has been kicked off successfully but somehow the poke fails due to a unusual error. When you then clear this task (because it failed) you will kick off another extract refresh. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381189484 ## File path: airflow/providers/salesforce/example_dags/example_tableau_refresh_workbook.py ## @@ -0,0 +1,59 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" +This is an example dag that performs a refresh operation on a Tableau Workbook aka Extract. Since this is an +asynchronous operation we don't know when the operation actually finishes. That's why we have a second task +that checks exactly that. So that you can perform further operations after the extract has been refreshed. +""" +from datetime import timedelta + +from airflow import DAG +from airflow.providers.salesforce.operators.tableau_refresh_workbook import TableauRefreshWorkbookOperator +from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor +from airflow.utils.dates import days_ago + +DEFAULT_ARGS = { +'owner': 'airflow', +'depends_on_past': False, +'start_date': days_ago(2), +'email': ['airf...@example.com'], +'email_on_failure': False, +'email_on_retry': False +} + +with DAG( +dag_id='example_tableau_refresh_workbook', +default_args=DEFAULT_ARGS, +dagrun_timeout=timedelta(hours=2), +schedule_interval=None, +tags=['example'], +) as dag: +task_refresh_workbook = TableauRefreshWorkbookOperator( +site_id='my_site', +workbook_name='MyWorkbook', +task_id='refresh_tableau_workbook', +dag=dag +) +task_check_job_status = TableauJobStatusSensor( Review comment: > I'd like to have the original Operator to be blocking (maybe make this configurable) So you want it to `sleep` ? An extract refresh can take very long. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381189484 ## File path: airflow/providers/salesforce/example_dags/example_tableau_refresh_workbook.py ## @@ -0,0 +1,59 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" +This is an example dag that performs a refresh operation on a Tableau Workbook aka Extract. Since this is an +asynchronous operation we don't know when the operation actually finishes. That's why we have a second task +that checks exactly that. So that you can perform further operations after the extract has been refreshed. +""" +from datetime import timedelta + +from airflow import DAG +from airflow.providers.salesforce.operators.tableau_refresh_workbook import TableauRefreshWorkbookOperator +from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor +from airflow.utils.dates import days_ago + +DEFAULT_ARGS = { +'owner': 'airflow', +'depends_on_past': False, +'start_date': days_ago(2), +'email': ['airf...@example.com'], +'email_on_failure': False, +'email_on_retry': False +} + +with DAG( +dag_id='example_tableau_refresh_workbook', +default_args=DEFAULT_ARGS, +dagrun_timeout=timedelta(hours=2), +schedule_interval=None, +tags=['example'], +) as dag: +task_refresh_workbook = TableauRefreshWorkbookOperator( +site_id='my_site', +workbook_name='MyWorkbook', +task_id='refresh_tableau_workbook', +dag=dag +) +task_check_job_status = TableauJobStatusSensor( Review comment: > I'd like to have the original Operator to be blocking (maybe make this configurable) So you want it to `sleep` ? An extract refresh can last very long. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381189484 ## File path: airflow/providers/salesforce/example_dags/example_tableau_refresh_workbook.py ## @@ -0,0 +1,59 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" +This is an example dag that performs a refresh operation on a Tableau Workbook aka Extract. Since this is an +asynchronous operation we don't know when the operation actually finishes. That's why we have a second task +that checks exactly that. So that you can perform further operations after the extract has been refreshed. +""" +from datetime import timedelta + +from airflow import DAG +from airflow.providers.salesforce.operators.tableau_refresh_workbook import TableauRefreshWorkbookOperator +from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor +from airflow.utils.dates import days_ago + +DEFAULT_ARGS = { +'owner': 'airflow', +'depends_on_past': False, +'start_date': days_ago(2), +'email': ['airf...@example.com'], +'email_on_failure': False, +'email_on_retry': False +} + +with DAG( +dag_id='example_tableau_refresh_workbook', +default_args=DEFAULT_ARGS, +dagrun_timeout=timedelta(hours=2), +schedule_interval=None, +tags=['example'], +) as dag: +task_refresh_workbook = TableauRefreshWorkbookOperator( +site_id='my_site', +workbook_name='MyWorkbook', +task_id='refresh_tableau_workbook', +dag=dag +) +task_check_job_status = TableauJobStatusSensor( Review comment: > I'd like to have the original Operator to be blocking (maybe make this configurable) So you want it to `sleep` ? An extract refresh can take very long. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381189484 ## File path: airflow/providers/salesforce/example_dags/example_tableau_refresh_workbook.py ## @@ -0,0 +1,59 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" +This is an example dag that performs a refresh operation on a Tableau Workbook aka Extract. Since this is an +asynchronous operation we don't know when the operation actually finishes. That's why we have a second task +that checks exactly that. So that you can perform further operations after the extract has been refreshed. +""" +from datetime import timedelta + +from airflow import DAG +from airflow.providers.salesforce.operators.tableau_refresh_workbook import TableauRefreshWorkbookOperator +from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor +from airflow.utils.dates import days_ago + +DEFAULT_ARGS = { +'owner': 'airflow', +'depends_on_past': False, +'start_date': days_ago(2), +'email': ['airf...@example.com'], +'email_on_failure': False, +'email_on_retry': False +} + +with DAG( +dag_id='example_tableau_refresh_workbook', +default_args=DEFAULT_ARGS, +dagrun_timeout=timedelta(hours=2), +schedule_interval=None, +tags=['example'], +) as dag: +task_refresh_workbook = TableauRefreshWorkbookOperator( +site_id='my_site', +workbook_name='MyWorkbook', +task_id='refresh_tableau_workbook', +dag=dag +) +task_check_job_status = TableauJobStatusSensor( Review comment: > I'd like to have the original Operator to be blocking (maybe make this configurable) So you want it to `sleep` ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381188357 ## File path: airflow/providers/salesforce/example_dags/example_tableau_refresh_workbook.py ## @@ -0,0 +1,59 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" +This is an example dag that performs a refresh operation on a Tableau Workbook aka Extract. Since this is an +asynchronous operation we don't know when the operation actually finishes. That's why we have a second task +that checks exactly that. So that you can perform further operations after the extract has been refreshed. +""" +from datetime import timedelta + +from airflow import DAG +from airflow.providers.salesforce.operators.tableau_refresh_workbook import TableauRefreshWorkbookOperator +from airflow.providers.salesforce.sensors.tableau_job_status import TableauJobStatusSensor +from airflow.utils.dates import days_ago + +DEFAULT_ARGS = { +'owner': 'airflow', +'depends_on_past': False, +'start_date': days_ago(2), +'email': ['airf...@example.com'], +'email_on_failure': False, +'email_on_retry': False +} + +with DAG( +dag_id='example_tableau_refresh_workbook', +default_args=DEFAULT_ARGS, +dagrun_timeout=timedelta(hours=2), +schedule_interval=None, +tags=['example'], +) as dag: +task_refresh_workbook = TableauRefreshWorkbookOperator( +site_id='my_site', +workbook_name='MyWorkbook', +task_id='refresh_tableau_workbook', +dag=dag +) +task_check_job_status = TableauJobStatusSensor( Review comment: > instead of always using an Operator/Sensor pair. That's where I am using subdags :) ❤️ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381186349 ## File path: airflow/providers/salesforce/operators/tableau_refresh_workbook.py ## @@ -0,0 +1,77 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +from typing import Optional + +from airflow import AirflowException +from airflow.models import BaseOperator +from airflow.providers.salesforce.hooks.tableau import TableauHook +from airflow.utils.decorators import apply_defaults + + +class TableauRefreshWorkbookOperator(BaseOperator): +""" +Refreshes a Tableau Workbook/Extract + +.. seealso:: https://tableau.github.io/server-client-python/docs/api-ref#workbooks + +:param workbook_name: The name of the workbook to refresh. +:type workbook_name: str +:param site_id: The id of the site where the workbook belongs to. +:type site_id: Optional[str] +:param tableau_conn_id: The Tableau Connection id containing the credentials +to authenticate to the Tableau Server. +:type tableau_conn_id: str +""" + +@apply_defaults +def __init__(self, + workbook_name: str, + *args, + site_id: Optional[str] = None, + tableau_conn_id: str = 'tableau_default', + **kwargs): +super().__init__(*args, **kwargs) +self.workbook_name = workbook_name +self.site_id = site_id +self.tableau_conn_id = tableau_conn_id + +def execute(self, context): +""" +Executes the Tableau Extract Refresh and pushes the job id to xcom. + +:param context: The task context during execution. +:type context: dict +:return: the id of the job that executes the extract refresh +:rtype: str +""" +with TableauHook(self.site_id, self.tableau_conn_id) as tableau_hook: +workbook = self._get_workbook_by_name(tableau_hook) + +return self._refresh_workbook(tableau_hook, workbook.id) + +def _get_workbook_by_name(self, tableau_hook): Review comment: Yes Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381186397 ## File path: airflow/providers/salesforce/operators/tableau_refresh_workbook.py ## @@ -0,0 +1,77 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +from typing import Optional + +from airflow import AirflowException +from airflow.models import BaseOperator +from airflow.providers.salesforce.hooks.tableau import TableauHook +from airflow.utils.decorators import apply_defaults + + +class TableauRefreshWorkbookOperator(BaseOperator): +""" +Refreshes a Tableau Workbook/Extract + +.. seealso:: https://tableau.github.io/server-client-python/docs/api-ref#workbooks + +:param workbook_name: The name of the workbook to refresh. +:type workbook_name: str +:param site_id: The id of the site where the workbook belongs to. +:type site_id: Optional[str] +:param tableau_conn_id: The Tableau Connection id containing the credentials +to authenticate to the Tableau Server. +:type tableau_conn_id: str +""" + +@apply_defaults +def __init__(self, + workbook_name: str, + *args, + site_id: Optional[str] = None, + tableau_conn_id: str = 'tableau_default', + **kwargs): +super().__init__(*args, **kwargs) +self.workbook_name = workbook_name +self.site_id = site_id +self.tableau_conn_id = tableau_conn_id + +def execute(self, context): Review comment: Yes Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381186194 ## File path: airflow/providers/salesforce/operators/tableau_refresh_workbook.py ## @@ -0,0 +1,77 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +from typing import Optional + +from airflow import AirflowException +from airflow.models import BaseOperator +from airflow.providers.salesforce.hooks.tableau import TableauHook +from airflow.utils.decorators import apply_defaults + + +class TableauRefreshWorkbookOperator(BaseOperator): +""" +Refreshes a Tableau Workbook/Extract + +.. seealso:: https://tableau.github.io/server-client-python/docs/api-ref#workbooks + +:param workbook_name: The name of the workbook to refresh. +:type workbook_name: str +:param site_id: The id of the site where the workbook belongs to. +:type site_id: Optional[str] +:param tableau_conn_id: The Tableau Connection id containing the credentials +to authenticate to the Tableau Server. +:type tableau_conn_id: str +""" + +@apply_defaults +def __init__(self, + workbook_name: str, + *args, + site_id: Optional[str] = None, + tableau_conn_id: str = 'tableau_default', + **kwargs): +super().__init__(*args, **kwargs) +self.workbook_name = workbook_name +self.site_id = site_id +self.tableau_conn_id = tableau_conn_id + +def execute(self, context): +""" +Executes the Tableau Extract Refresh and pushes the job id to xcom. + +:param context: The task context during execution. +:type context: dict +:return: the id of the job that executes the extract refresh +:rtype: str +""" +with TableauHook(self.site_id, self.tableau_conn_id) as tableau_hook: +workbook = self._get_workbook_by_name(tableau_hook) + +return self._refresh_workbook(tableau_hook, workbook.id) + +def _get_workbook_by_name(self, tableau_hook): +for workbook in tableau_hook.get_all(resource_name='workbooks'): +if workbook.name == self.workbook_name: +self.log.info('Found matching workbook with id %s', workbook.id) +return workbook + +raise AirflowException(f'Workbook {self.workbook_name} not found!') + +def _refresh_workbook(self, tableau_hook, workbook_id): Review comment: Yes Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381186131 ## File path: airflow/providers/salesforce/sensors/tableau_job_status.py ## @@ -0,0 +1,63 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +from typing import Optional + +from airflow.providers.salesforce.hooks.tableau import TableauHook +from airflow.sensors.base_sensor_operator import BaseSensorOperator +from airflow.utils.decorators import apply_defaults + + +class TableauJobStatusSensor(BaseSensorOperator): +""" +Watches the status of a Tableau Server Job. + +.. seealso:: https://tableau.github.io/server-client-python/docs/api-ref#jobs + +:param job_id: The job to watch. +:type job_id: str +:param site_id: The id of the site where the workbook belongs to. +:type site_id: Optional[str] +:param tableau_conn_id: The Tableau Connection id containing the credentials +to authenticate to the Tableau Server. +:type tableau_conn_id: str +""" + +template_fields = ('job_id',) + +@apply_defaults +def __init__(self, + job_id: str, + *args, + site_id: Optional[str] = None, + tableau_conn_id: str = 'tableau_default', + **kwargs): +super().__init__(*args, **kwargs) +self.tableau_conn_id = tableau_conn_id +self.job_id = job_id +self.site_id = site_id + +def poke(self, context): Review comment: Yes Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381185866 ## File path: airflow/providers/salesforce/sensors/tableau_job_status.py ## @@ -0,0 +1,63 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +from typing import Optional + +from airflow.providers.salesforce.hooks.tableau import TableauHook +from airflow.sensors.base_sensor_operator import BaseSensorOperator +from airflow.utils.decorators import apply_defaults + + +class TableauJobStatusSensor(BaseSensorOperator): +""" +Watches the status of a Tableau Server Job. + +.. seealso:: https://tableau.github.io/server-client-python/docs/api-ref#jobs + +:param job_id: The job to watch. +:type job_id: str +:param site_id: The id of the site where the workbook belongs to. +:type site_id: Optional[str] +:param tableau_conn_id: The Tableau Connection id containing the credentials +to authenticate to the Tableau Server. +:type tableau_conn_id: str +""" + +template_fields = ('job_id',) + +@apply_defaults +def __init__(self, + job_id: str, + *args, Review comment: Will do. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381185407 ## File path: airflow/providers/salesforce/sensors/tableau_job_status.py ## @@ -0,0 +1,63 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +from typing import Optional + +from airflow.providers.salesforce.hooks.tableau import TableauHook +from airflow.sensors.base_sensor_operator import BaseSensorOperator +from airflow.utils.decorators import apply_defaults + + +class TableauJobStatusSensor(BaseSensorOperator): +""" +Watches the status of a Tableau Server Job. + +.. seealso:: https://tableau.github.io/server-client-python/docs/api-ref#jobs + +:param job_id: The job to watch. +:type job_id: str +:param site_id: The id of the site where the workbook belongs to. +:type site_id: Optional[str] +:param tableau_conn_id: The Tableau Connection id containing the credentials +to authenticate to the Tableau Server. +:type tableau_conn_id: str +""" + +template_fields = ('job_id',) + +@apply_defaults +def __init__(self, + job_id: str, + *args, + site_id: Optional[str] = None, + tableau_conn_id: str = 'tableau_default', + **kwargs): +super().__init__(*args, **kwargs) +self.tableau_conn_id = tableau_conn_id +self.job_id = job_id +self.site_id = site_id + +def poke(self, context): +""" +Pokes for a job to successfully finish. + +:param context: The task context during execution. +:type context: dict +:return: True if it succeeded and False if not. +:rtype: bool +""" +with TableauHook(self.site_id, self.tableau_conn_id) as tableau_hook: +return tableau_hook.server.jobs.get(self.job_id).finish_code != -1 Review comment: > Can we make a constant out of the -1? To avoid magic numbers in the code. Good Point. I will do that. > What are the options in terms of return codes here? Can we check if the notebook successfully refreshed, or maybe if it threw an error while refreshing? Yes I need to check that! Thanks. _The finishCode indicates the status of the job: 0 for success, 1 for error, or 2 for cancelled._ - REST API docs (not python client docs) That is really weird, because I got a `-1` for a failed Job. I definitly need to check that again! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381179958 ## File path: setup.py ## @@ -321,6 +321,7 @@ def write_version(filename: str = os.path.join(*["airflow", "git_version"])): ] salesforce = [ Review comment: Okay. I am fine with this :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r381179743 ## File path: setup.py ## @@ -321,6 +321,7 @@ def write_version(filename: str = os.path.join(*["airflow", "git_version"])): ] salesforce = [ 'simple-salesforce>=0.72', +'tableauserverclient' Review comment: I think we could (and maybe should) since the current version v0.9 seems to be near a v1 release which could be a release with breaking changes (I don't know). I couldn't find anything related to release schedule / plan nor versioning of releases. Actually because you are bringing this up I am wondering if we should create general rules for pinning versions of dependencies. **This of course should be discussed in the dev list.** It is a very complicated topic. I think there are several points that could be relevant: * knowing the releases and versioning of the dependency * having one or more users in the community which have responsibilities in the project the dependency belongs to. (like committer status for apache projects, etc.) * ... If we have an issue due to a new release: Maybe we should somewhere write down our dependencies and keep track of the issues we have towards non-pinning a dependency. So that we can better understand their releases. Every dependency has their own semantic versioning of releases. Maybe we should add a list of requirements when we want to add a non-pinned dependency?! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r380186136 ## File path: airflow/providers/salesforce/operators/tableau_refresh_workbook.py ## @@ -0,0 +1,74 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from airflow import AirflowException +from airflow.models import BaseOperator +from airflow.providers.salesforce.hooks.tableau import TableauHook +from airflow.utils.decorators import apply_defaults + + +class TableauRefreshWorkbookOperator(BaseOperator): +""" +Refreshes a Tableau Workbook/Extract + +:param workbook_name: The name of the workbook to refresh. +:type workbook_name: str +:param site_id: The id of the site where the workbook belongs to. +:type site_id: str +:param tableau_conn_id: The Tableau Connection id containing the credentials +to authenticate to the Tableau Server. +:type tableau_conn_id: str +""" + +@apply_defaults +def __init__(self, + workbook_name, + *args, + site_id='', + tableau_conn_id='tableau_default', + **kwargs): +super().__init__(*args, **kwargs) +self.workbook_name = workbook_name +self.site_id = site_id +self.tableau_conn_id = tableau_conn_id + +def execute(self, context): +""" +Executes the Tableau Extract Refresh and pushes the job id to xcom. + +:param context: The task context during execution. +:type context: dict +:return: the id of the job that executes the extract refresh +:rtype: str +""" +with TableauHook(self.site_id, self.tableau_conn_id) as tableau_hook: +workbook = self._get_workbook_by_name(tableau_hook) + +return self._refresh_workbook(tableau_hook, workbook.id) + +def _get_workbook_by_name(self, tableau_hook): +for workbook in tableau_hook.get_all(resource_name='workbooks'): +if workbook.name == self.workbook_name: +self.log.info('Found matching workbook with id %s', workbook.id) +return workbook + +raise AirflowException(f'Workbook {self.workbook_name} not found!') + +def _refresh_workbook(self, tableau_hook, workbook_id): +job = tableau_hook.server.workbooks.refresh(workbook_id) Review comment: You actually pass an ID of a workbook not the name to the Tableau Server. I am iterating through workbooks by names and I am returning the workbook id of the workbook that matches. -> which means it will not find `Calls/Agents` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r380186136 ## File path: airflow/providers/salesforce/operators/tableau_refresh_workbook.py ## @@ -0,0 +1,74 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from airflow import AirflowException +from airflow.models import BaseOperator +from airflow.providers.salesforce.hooks.tableau import TableauHook +from airflow.utils.decorators import apply_defaults + + +class TableauRefreshWorkbookOperator(BaseOperator): +""" +Refreshes a Tableau Workbook/Extract + +:param workbook_name: The name of the workbook to refresh. +:type workbook_name: str +:param site_id: The id of the site where the workbook belongs to. +:type site_id: str +:param tableau_conn_id: The Tableau Connection id containing the credentials +to authenticate to the Tableau Server. +:type tableau_conn_id: str +""" + +@apply_defaults +def __init__(self, + workbook_name, + *args, + site_id='', + tableau_conn_id='tableau_default', + **kwargs): +super().__init__(*args, **kwargs) +self.workbook_name = workbook_name +self.site_id = site_id +self.tableau_conn_id = tableau_conn_id + +def execute(self, context): +""" +Executes the Tableau Extract Refresh and pushes the job id to xcom. + +:param context: The task context during execution. +:type context: dict +:return: the id of the job that executes the extract refresh +:rtype: str +""" +with TableauHook(self.site_id, self.tableau_conn_id) as tableau_hook: +workbook = self._get_workbook_by_name(tableau_hook) + +return self._refresh_workbook(tableau_hook, workbook.id) + +def _get_workbook_by_name(self, tableau_hook): +for workbook in tableau_hook.get_all(resource_name='workbooks'): +if workbook.name == self.workbook_name: +self.log.info('Found matching workbook with id %s', workbook.id) +return workbook + +raise AirflowException(f'Workbook {self.workbook_name} not found!') + +def _refresh_workbook(self, tableau_hook, workbook_id): +job = tableau_hook.server.workbooks.refresh(workbook_id) Review comment: You actually pass an ID of a workbook not the name. I am iterating through workbooks by names and I am returning the workbook id of the workbook that matches. -> which means it will not find `Calls/Agents` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r380186136 ## File path: airflow/providers/salesforce/operators/tableau_refresh_workbook.py ## @@ -0,0 +1,74 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from airflow import AirflowException +from airflow.models import BaseOperator +from airflow.providers.salesforce.hooks.tableau import TableauHook +from airflow.utils.decorators import apply_defaults + + +class TableauRefreshWorkbookOperator(BaseOperator): +""" +Refreshes a Tableau Workbook/Extract + +:param workbook_name: The name of the workbook to refresh. +:type workbook_name: str +:param site_id: The id of the site where the workbook belongs to. +:type site_id: str +:param tableau_conn_id: The Tableau Connection id containing the credentials +to authenticate to the Tableau Server. +:type tableau_conn_id: str +""" + +@apply_defaults +def __init__(self, + workbook_name, + *args, + site_id='', + tableau_conn_id='tableau_default', + **kwargs): +super().__init__(*args, **kwargs) +self.workbook_name = workbook_name +self.site_id = site_id +self.tableau_conn_id = tableau_conn_id + +def execute(self, context): +""" +Executes the Tableau Extract Refresh and pushes the job id to xcom. + +:param context: The task context during execution. +:type context: dict +:return: the id of the job that executes the extract refresh +:rtype: str +""" +with TableauHook(self.site_id, self.tableau_conn_id) as tableau_hook: +workbook = self._get_workbook_by_name(tableau_hook) + +return self._refresh_workbook(tableau_hook, workbook.id) + +def _get_workbook_by_name(self, tableau_hook): +for workbook in tableau_hook.get_all(resource_name='workbooks'): +if workbook.name == self.workbook_name: +self.log.info('Found matching workbook with id %s', workbook.id) +return workbook + +raise AirflowException(f'Workbook {self.workbook_name} not found!') + +def _refresh_workbook(self, tableau_hook, workbook_id): +job = tableau_hook.server.workbooks.refresh(workbook_id) Review comment: You actually pass an ID of a workbook not the name. I am iterating through workbooks by names and I am returning the workbook id of the workbook that matches. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r380184478 ## File path: airflow/providers/salesforce/operators/tableau_refresh_workbook.py ## @@ -0,0 +1,74 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from airflow import AirflowException +from airflow.models import BaseOperator +from airflow.providers.salesforce.hooks.tableau import TableauHook +from airflow.utils.decorators import apply_defaults + + +class TableauRefreshWorkbookOperator(BaseOperator): +""" +Refreshes a Tableau Workbook/Extract + +:param workbook_name: The name of the workbook to refresh. +:type workbook_name: str +:param site_id: The id of the site where the workbook belongs to. +:type site_id: str +:param tableau_conn_id: The Tableau Connection id containing the credentials +to authenticate to the Tableau Server. +:type tableau_conn_id: str +""" + +@apply_defaults +def __init__(self, + workbook_name, + *args, + site_id='', + tableau_conn_id='tableau_default', + **kwargs): +super().__init__(*args, **kwargs) +self.workbook_name = workbook_name +self.site_id = site_id +self.tableau_conn_id = tableau_conn_id + +def execute(self, context): +""" +Executes the Tableau Extract Refresh and pushes the job id to xcom. + +:param context: The task context during execution. +:type context: dict +:return: the id of the job that executes the extract refresh +:rtype: str +""" +with TableauHook(self.site_id, self.tableau_conn_id) as tableau_hook: +workbook = self._get_workbook_by_name(tableau_hook) + +return self._refresh_workbook(tableau_hook, workbook.id) + +def _get_workbook_by_name(self, tableau_hook): +for workbook in tableau_hook.get_all(resource_name='workbooks'): +if workbook.name == self.workbook_name: +self.log.info('Found matching workbook with id %s', workbook.id) +return workbook + +raise AirflowException(f'Workbook {self.workbook_name} not found!') + +def _refresh_workbook(self, tableau_hook, workbook_id): +job = tableau_hook.server.workbooks.refresh(workbook_id) Review comment: > if I pass Calls would it refresh both Department & Agents ? Yes. > What if I pass Calls/Agents ? I think it fails, because that's not a Workbook - it is a View in a Workbook, right?! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r380183636 ## File path: airflow/providers/salesforce/operators/tableau_refresh_workbook.py ## @@ -0,0 +1,74 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from airflow import AirflowException +from airflow.models import BaseOperator +from airflow.providers.salesforce.hooks.tableau import TableauHook +from airflow.utils.decorators import apply_defaults + + +class TableauRefreshWorkbookOperator(BaseOperator): +""" +Refreshes a Tableau Workbook/Extract + +:param workbook_name: The name of the workbook to refresh. +:type workbook_name: str +:param site_id: The id of the site where the workbook belongs to. +:type site_id: str +:param tableau_conn_id: The Tableau Connection id containing the credentials +to authenticate to the Tableau Server. +:type tableau_conn_id: str +""" + +@apply_defaults +def __init__(self, + workbook_name, + *args, + site_id='', + tableau_conn_id='tableau_default', + **kwargs): +super().__init__(*args, **kwargs) +self.workbook_name = workbook_name +self.site_id = site_id +self.tableau_conn_id = tableau_conn_id + +def execute(self, context): +""" +Executes the Tableau Extract Refresh and pushes the job id to xcom. + +:param context: The task context during execution. +:type context: dict +:return: the id of the job that executes the extract refresh +:rtype: str +""" +with TableauHook(self.site_id, self.tableau_conn_id) as tableau_hook: +workbook = self._get_workbook_by_name(tableau_hook) + +return self._refresh_workbook(tableau_hook, workbook.id) + +def _get_workbook_by_name(self, tableau_hook): +for workbook in tableau_hook.get_all(resource_name='workbooks'): +if workbook.name == self.workbook_name: +self.log.info('Found matching workbook with id %s', workbook.id) +return workbook + +raise AirflowException(f'Workbook {self.workbook_name} not found!') + +def _refresh_workbook(self, tableau_hook, workbook_id): +job = tableau_hook.server.workbooks.refresh(workbook_id) Review comment: > Refreshes the specified workbook. > > This method refreshes the specified workbook, with no need to associate the workbook refresh with a scheduled task. This method is the equivalent of selecting a workbook using the Tableau Server UI, and then selecting Refresh Extracts from the menu (also known as a "manual refresh"). > from https://help.tableau.com/current/api/rest_api/en-us/REST/rest_api_ref_workbooksviews.htm#update_workbook_now This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r379058038 ## File path: airflow/providers/salesforce/operators/tableau_refresh_workbook.py ## @@ -0,0 +1,74 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from airflow import AirflowException +from airflow.models import BaseOperator +from airflow.providers.salesforce.hooks.tableau import TableauHook +from airflow.utils.decorators import apply_defaults + + +class TableauRefreshWorkbookOperator(BaseOperator): +""" +Refreshes a Tableau Workbook/Extract + +:param workbook_name: The name of the workbook to refresh. +:type workbook_name: str +:param site_id: The id of the site where the workbook belongs to. +:type site_id: str +:param tableau_conn_id: The Tableau Connection id containing the credentials +to authenticate to the Tableau Server. +:type tableau_conn_id: str +""" + +@apply_defaults +def __init__(self, + workbook_name, + *args, + site_id='', + tableau_conn_id='tableau_default', + **kwargs): +super().__init__(*args, **kwargs) +self.workbook_name = workbook_name +self.site_id = site_id +self.tableau_conn_id = tableau_conn_id + +def execute(self, context): +""" +Executes the Tableau Extract Refresh and pushes the job id to xcom. + +:param context: The task context during execution. +:type context: dict +:return: the id of the job that executes the extract refresh +:rtype: str +""" +with TableauHook(self.site_id, self.tableau_conn_id) as tableau_hook: +workbook = self._get_workbook_by_name(tableau_hook) + +return self._refresh_workbook(tableau_hook, workbook.id) + +def _get_workbook_by_name(self, tableau_hook): +for workbook in tableau_hook.get_all(resource_name='workbooks'): +if workbook.name == self.workbook_name: +self.log.info('Found matching workbook with id %s', workbook.id) +return workbook + +raise AirflowException(f'Workbook {self.workbook_name} not found!') + +def _refresh_workbook(self, tableau_hook, workbook_id): +job = tableau_hook.server.workbooks.refresh(workbook_id) Review comment: I am gonna check that. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r379057502 ## File path: airflow/providers/salesforce/operators/tableau_refresh_workbook.py ## @@ -0,0 +1,74 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from airflow import AirflowException +from airflow.models import BaseOperator +from airflow.providers.salesforce.hooks.tableau import TableauHook +from airflow.utils.decorators import apply_defaults + + +class TableauRefreshWorkbookOperator(BaseOperator): +""" +Refreshes a Tableau Workbook/Extract + +:param workbook_name: The name of the workbook to refresh. +:type workbook_name: str +:param site_id: The id of the site where the workbook belongs to. +:type site_id: str +:param tableau_conn_id: The Tableau Connection id containing the credentials +to authenticate to the Tableau Server. +:type tableau_conn_id: str +""" + +@apply_defaults +def __init__(self, + workbook_name, + *args, + site_id='', + tableau_conn_id='tableau_default', + **kwargs): +super().__init__(*args, **kwargs) +self.workbook_name = workbook_name +self.site_id = site_id Review comment: Good point. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r379056816 ## File path: airflow/providers/salesforce/hooks/tableau.py ## @@ -0,0 +1,61 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from tableauserverclient import Pager, Server, TableauAuth + +from airflow.hooks.base_hook import BaseHook + + +class TableauHook(BaseHook): +""" +Connects to the Tableau Server Instance and allows to communicate with it. + +:param site_id: The id of the site where the workbook belongs to. +:type site_id: str +:param tableau_conn_id: The Tableau Connection id containing the credentials +to authenticate to the Tableau Server. +:type tableau_conn_id: str +""" + +def __init__(self, site_id='', tableau_conn_id='tableau_default'): Review comment: Actually I think I will remove the default. WDYT? ```suggestion def __init__(self, site_id='', tableau_conn_id): ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r379056816 ## File path: airflow/providers/salesforce/hooks/tableau.py ## @@ -0,0 +1,61 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from tableauserverclient import Pager, Server, TableauAuth + +from airflow.hooks.base_hook import BaseHook + + +class TableauHook(BaseHook): +""" +Connects to the Tableau Server Instance and allows to communicate with it. + +:param site_id: The id of the site where the workbook belongs to. +:type site_id: str +:param tableau_conn_id: The Tableau Connection id containing the credentials +to authenticate to the Tableau Server. +:type tableau_conn_id: str +""" + +def __init__(self, site_id='', tableau_conn_id='tableau_default'): Review comment: Actually I think I will remove the default. WDYT? ```suggestion def __init__(self, site_id='', tableau_conn_id'): ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r379056518 ## File path: airflow/providers/salesforce/operators/tableau_refresh_workbook.py ## @@ -0,0 +1,74 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from airflow import AirflowException +from airflow.models import BaseOperator +from airflow.providers.salesforce.hooks.tableau import TableauHook +from airflow.utils.decorators import apply_defaults + + +class TableauRefreshWorkbookOperator(BaseOperator): +""" +Refreshes a Tableau Workbook/Extract + +:param workbook_name: The name of the workbook to refresh. +:type workbook_name: str +:param site_id: The id of the site where the workbook belongs to. +:type site_id: str +:param tableau_conn_id: The Tableau Connection id containing the credentials +to authenticate to the Tableau Server. +:type tableau_conn_id: str +""" + +@apply_defaults +def __init__(self, + workbook_name, + *args, + site_id='', Review comment: An empty string is the default value in the library. See https://tableau.github.io/server-client-python/docs/api-ref#authentication This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r379056816 ## File path: airflow/providers/salesforce/hooks/tableau.py ## @@ -0,0 +1,61 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from tableauserverclient import Pager, Server, TableauAuth + +from airflow.hooks.base_hook import BaseHook + + +class TableauHook(BaseHook): +""" +Connects to the Tableau Server Instance and allows to communicate with it. + +:param site_id: The id of the site where the workbook belongs to. +:type site_id: str +:param tableau_conn_id: The Tableau Connection id containing the credentials +to authenticate to the Tableau Server. +:type tableau_conn_id: str +""" + +def __init__(self, site_id='', tableau_conn_id='tableau_default'): Review comment: Actually I think I will remove the default ```suggestion def __init__(self, site_id='', tableau_conn_id'): ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services