[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-06 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r388868149
 
 

 ##
 File path: airflow/providers/google/marketing_platform/hooks/analytics.py
 ##
 @@ -0,0 +1,87 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+
+from googleapiclient.discovery import Resource, build
+
+from airflow.providers.google.cloud.hooks.base import CloudBaseHook
+
+
+class GoogleAnalyticsHook(CloudBaseHook):
+"""
+Hook for Google Analytics 360.
+"""
+
+def __init__(
+self,
+api_version: str = "v3",
+gcp_connection_id: str = "google cloud default",
+*args,
+**kwargs
+):
+super().__init__(*args, **kwargs)
+self.api_version = api_version
+self.gcp_connection_is = gcp_connection_id
+self._conn = None
+
+def get_conn(self) -> Resource:
+"""
+Retrieves connection to Google Analytics 360.
+"""
+if not self._conn:
+http_authorized = self._authorize()
+self._conn = build(
+"analytics",
+self.api_version,
+http=http_authorized,
+cache_discovery=False,
+)
+return self._conn
+
+# pylint: disable=unused-argument
+def list_accounts(self, max_results: int, start_index: int) -> list:
+"""
+Lists accounts list from Google Analytics 360.
+
+:param max_results: The maximum number of accounts to include in this 
response.
+:type: max_result: int
+:param start_index: An index of the first account to retrieve
+Use this parameter as a pagination mechanism along with the 
max-results parameter.
 
 Review comment:
   Thank you :)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-07 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r389258263
 
 

 ##
 File path: airflow/providers/google/marketing_platform/hooks/analytics.py
 ##
 @@ -0,0 +1,88 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+
+from googleapiclient.discovery import Resource, build
+
+from airflow.providers.google.cloud.hooks.base import CloudBaseHook
+
+
+class GoogleAnalyticsHook(CloudBaseHook):
+"""
+Hook for Google Analytics 360.
+"""
+
+def __init__(
+self,
+api_version: str = "v3",
+gcp_connection_id: str = "google cloud default",
+*args,
+**kwargs
+):
+super().__init__(*args, **kwargs)
+self.api_version = api_version
+self.gcp_connection_is = gcp_connection_id
+self._conn = None
+
+def get_conn(self) -> Resource:
+"""
+Retrieves connection to Google Analytics 360.
+"""
+if not self._conn:
+http_authorized = self._authorize()
+self._conn = build(
+"analytics",
+self.api_version,
+http=http_authorized,
+cache_discovery=False,
+)
+return self._conn
+
+# pylint: disable=unused-argument
 
 Review comment:
   I had to add this option because of pylint.
   This methods `def list_accounts(self, max_results: int, start_index: int)` 
has two args, but none of them is used in method itself. Args are used in 
operator. 
   AFAIK Analytics API method supports args: max_results and start_index but I 
cannot pass them into this request 
`analytics.management().accounts().list().execute()`. I have tried but it 
failed. 
   I am open for the suggestions if you have any. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-07 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r389258274
 
 

 ##
 File path: airflow/providers/google/marketing_platform/hooks/analytics.py
 ##
 @@ -0,0 +1,88 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+
+from googleapiclient.discovery import Resource, build
+
+from airflow.providers.google.cloud.hooks.base import CloudBaseHook
+
+
+class GoogleAnalyticsHook(CloudBaseHook):
+"""
+Hook for Google Analytics 360.
+"""
+
+def __init__(
+self,
+api_version: str = "v3",
+gcp_connection_id: str = "google cloud default",
+*args,
+**kwargs
+):
+super().__init__(*args, **kwargs)
+self.api_version = api_version
+self.gcp_connection_is = gcp_connection_id
+self._conn = None
+
+def get_conn(self) -> Resource:
+"""
+Retrieves connection to Google Analytics 360.
+"""
+if not self._conn:
+http_authorized = self._authorize()
+self._conn = build(
+"analytics",
+self.api_version,
+http=http_authorized,
+cache_discovery=False,
+)
+return self._conn
+
+# pylint: disable=unused-argument
+def list_accounts(self, max_results: int, start_index: int) -> list:
 
 Review comment:
   Nice :) I will do it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-07 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r389258274
 
 

 ##
 File path: airflow/providers/google/marketing_platform/hooks/analytics.py
 ##
 @@ -0,0 +1,88 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+
+from googleapiclient.discovery import Resource, build
+
+from airflow.providers.google.cloud.hooks.base import CloudBaseHook
+
+
+class GoogleAnalyticsHook(CloudBaseHook):
+"""
+Hook for Google Analytics 360.
+"""
+
+def __init__(
+self,
+api_version: str = "v3",
+gcp_connection_id: str = "google cloud default",
+*args,
+**kwargs
+):
+super().__init__(*args, **kwargs)
+self.api_version = api_version
+self.gcp_connection_is = gcp_connection_id
+self._conn = None
+
+def get_conn(self) -> Resource:
+"""
+Retrieves connection to Google Analytics 360.
+"""
+if not self._conn:
+http_authorized = self._authorize()
+self._conn = build(
+"analytics",
+self.api_version,
+http=http_authorized,
+cache_discovery=False,
+)
+return self._conn
+
+# pylint: disable=unused-argument
+def list_accounts(self, max_results: int, start_index: int) -> list:
 
 Review comment:
   Why this way? 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-07 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r389258274
 
 

 ##
 File path: airflow/providers/google/marketing_platform/hooks/analytics.py
 ##
 @@ -0,0 +1,88 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+
+from googleapiclient.discovery import Resource, build
+
+from airflow.providers.google.cloud.hooks.base import CloudBaseHook
+
+
+class GoogleAnalyticsHook(CloudBaseHook):
+"""
+Hook for Google Analytics 360.
+"""
+
+def __init__(
+self,
+api_version: str = "v3",
+gcp_connection_id: str = "google cloud default",
+*args,
+**kwargs
+):
+super().__init__(*args, **kwargs)
+self.api_version = api_version
+self.gcp_connection_is = gcp_connection_id
+self._conn = None
+
+def get_conn(self) -> Resource:
+"""
+Retrieves connection to Google Analytics 360.
+"""
+if not self._conn:
+http_authorized = self._authorize()
+self._conn = build(
+"analytics",
+self.api_version,
+http=http_authorized,
+cache_discovery=False,
+)
+return self._conn
+
+# pylint: disable=unused-argument
+def list_accounts(self, max_results: int, start_index: int) -> list:
 
 Review comment:
   Why this way? I see, it is more explicit. What other benefits are?

   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-07 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r389258760
 
 

 ##
 File path: airflow/providers/google/marketing_platform/hooks/analytics.py
 ##
 @@ -0,0 +1,88 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+
+from googleapiclient.discovery import Resource, build
+
+from airflow.providers.google.cloud.hooks.base import CloudBaseHook
+
+
+class GoogleAnalyticsHook(CloudBaseHook):
+"""
+Hook for Google Analytics 360.
+"""
+
+def __init__(
+self,
+api_version: str = "v3",
+gcp_connection_id: str = "google cloud default",
+*args,
+**kwargs
+):
+super().__init__(*args, **kwargs)
+self.api_version = api_version
+self.gcp_connection_is = gcp_connection_id
+self._conn = None
+
+def get_conn(self) -> Resource:
+"""
+Retrieves connection to Google Analytics 360.
+"""
+if not self._conn:
+http_authorized = self._authorize()
+self._conn = build(
+"analytics",
+self.api_version,
+http=http_authorized,
+cache_discovery=False,
+)
+return self._conn
+
+# pylint: disable=unused-argument
+def list_accounts(self, max_results: int, start_index: int) -> list:
+"""
+Lists accounts list from Google Analytics 360.
+
+:param max_results: The maximum number of accounts to include in this 
response.
+:type: max_result: int
+:param start_index: An index of the first account to retrieve
+Use this parameter as a pagination mechanism along with the 
max-results parameter.
+:type: start_index: int
+"""
+
+self.log.info("Retrieving accounts list")
+
+# pylint: disable=no-member
+accounts = (
+self.get_conn()
+.management()
+.accounts()
+.list()
+.execute(num_retries=self.num_retries)
+)
+return self._list_emails_from_response(accounts)
+
+@staticmethod
+def _list_emails_from_response(accounts) -> list:
+"""
+:param accounts: GET request to Analytics service
+:type accounts: list
+"""
+
+result = [value for key, value in accounts.items() if key == 
"username"]
+return result
 
 Review comment:
   That's nit, but I like it. :)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-07 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r389305670
 
 

 ##
 File path: airflow/providers/google/marketing_platform/hooks/analytics.py
 ##
 @@ -0,0 +1,88 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+
+from googleapiclient.discovery import Resource, build
+
+from airflow.providers.google.cloud.hooks.base import CloudBaseHook
+
+
+class GoogleAnalyticsHook(CloudBaseHook):
+"""
+Hook for Google Analytics 360.
+"""
+
+def __init__(
+self,
+api_version: str = "v3",
+gcp_connection_id: str = "google cloud default",
+*args,
+**kwargs
+):
+super().__init__(*args, **kwargs)
+self.api_version = api_version
+self.gcp_connection_is = gcp_connection_id
+self._conn = None
+
+def get_conn(self) -> Resource:
+"""
+Retrieves connection to Google Analytics 360.
+"""
+if not self._conn:
+http_authorized = self._authorize()
+self._conn = build(
+"analytics",
+self.api_version,
+http=http_authorized,
+cache_discovery=False,
+)
+return self._conn
+
+# pylint: disable=unused-argument
+def list_accounts(self, max_results: int, start_index: int) -> list:
+"""
+Lists accounts list from Google Analytics 360.
+
+:param max_results: The maximum number of accounts to include in this 
response.
+:type: max_result: int
+:param start_index: An index of the first account to retrieve
+Use this parameter as a pagination mechanism along with the 
max-results parameter.
+:type: start_index: int
+"""
+
+self.log.info("Retrieving accounts list")
+
+# pylint: disable=no-member
+accounts = (
+self.get_conn()
+.management()
+.accounts()
+.list()
+.execute(num_retries=self.num_retries)
+)
+return self._list_emails_from_response(accounts)
+
+@staticmethod
+def _list_emails_from_response(accounts) -> list:
 
 Review comment:
   Done.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-07 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r389305653
 
 

 ##
 File path: airflow/providers/google/marketing_platform/hooks/analytics.py
 ##
 @@ -0,0 +1,88 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+
+from googleapiclient.discovery import Resource, build
+
+from airflow.providers.google.cloud.hooks.base import CloudBaseHook
+
+
+class GoogleAnalyticsHook(CloudBaseHook):
+"""
+Hook for Google Analytics 360.
+"""
+
+def __init__(
+self,
+api_version: str = "v3",
+gcp_connection_id: str = "google cloud default",
+*args,
+**kwargs
+):
+super().__init__(*args, **kwargs)
+self.api_version = api_version
+self.gcp_connection_is = gcp_connection_id
+self._conn = None
+
+def get_conn(self) -> Resource:
+"""
+Retrieves connection to Google Analytics 360.
+"""
+if not self._conn:
+http_authorized = self._authorize()
+self._conn = build(
+"analytics",
+self.api_version,
+http=http_authorized,
+cache_discovery=False,
+)
+return self._conn
+
+# pylint: disable=unused-argument
+def list_accounts(self, max_results: int, start_index: int) -> list:
 
 Review comment:
   Got it. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-07 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r389305801
 
 

 ##
 File path: airflow/providers/google/marketing_platform/hooks/analytics.py
 ##
 @@ -0,0 +1,88 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+
+from googleapiclient.discovery import Resource, build
+
+from airflow.providers.google.cloud.hooks.base import CloudBaseHook
+
+
+class GoogleAnalyticsHook(CloudBaseHook):
+"""
+Hook for Google Analytics 360.
+"""
+
+def __init__(
+self,
+api_version: str = "v3",
+gcp_connection_id: str = "google cloud default",
+*args,
+**kwargs
+):
+super().__init__(*args, **kwargs)
+self.api_version = api_version
+self.gcp_connection_is = gcp_connection_id
+self._conn = None
+
+def get_conn(self) -> Resource:
+"""
+Retrieves connection to Google Analytics 360.
+"""
+if not self._conn:
+http_authorized = self._authorize()
+self._conn = build(
+"analytics",
+self.api_version,
+http=http_authorized,
+cache_discovery=False,
+)
+return self._conn
+
+# pylint: disable=unused-argument
 
 Review comment:
   I agree, it is confusing. I will remove these args. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-11 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r390964634
 
 

 ##
 File path: airflow/providers/google/marketing_platform/operators/analytics.py
 ##
 @@ -0,0 +1,67 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+This module contains Google Analytics 360 operators.
+"""
+
+from airflow.models import BaseOperator
+from airflow.providers.google.marketing_platform.hooks.analytics import 
GoogleAnalyticsHook
+from airflow.utils.decorators import apply_defaults
+
+
+class GoogleAnalyticsListAccountsOperator(BaseOperator):
+"""
+Lists all accounts to which the user has access.
+
+.. seealso::
+Check official API docs:
+
https://developers.google.com/analytics/devguides/config/mgmt/v3/mgmtReference/management/accounts/list
+
+.. seealso::
+For more information on how to use this operator, take a look at the 
guide:
+:ref:`howto/operator:GoogleAnalyticsListAccountsOperator`
+
+:param max_results: The maximum number of accounts to include in this 
response.
+:type: max_results: int
+:param: start_index: An index of the first account to retrieve
+Use this parameter as a pagination mechanism along with the 
max-results parameter.
+:type: start_index: int
+:param api_version: The version of the api that will be requested for 
example 'v3'.
+:type api_version: str
+:param gcp_conn_id: The connection ID to use when fetching connection info.
+:type gcp_conn_id: str
+"""
+
+template_fields = ("api_version", "gcp_connection_id",)
+
+@apply_defaults
+def __init__(self,
+ api_version: str = "v3",
+ gcp_connection_id: str = "google cloud default",
+ *args,
+ **kwargs):
+super().__init__(*args, **kwargs)
+
+self.api_version = api_version
+self.gcp_connection_id = gcp_connection_id
+
+def execute(self, context):
+hook = GoogleAnalyticsHook(api_version=self.api_version,
+   gcp_connection_id=self.gcp_connection_id)
+result = hook.list_accounts()
 
 Review comment:
   I put logging into hook. `self.log.info("Retrieving accounts list...")`
   Should it be in the operator? What's the best practice?  
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-11 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r390965342
 
 

 ##
 File path: airflow/providers/google/marketing_platform/operators/analytics.py
 ##
 @@ -0,0 +1,67 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+This module contains Google Analytics 360 operators.
+"""
+
+from airflow.models import BaseOperator
+from airflow.providers.google.marketing_platform.hooks.analytics import 
GoogleAnalyticsHook
+from airflow.utils.decorators import apply_defaults
+
+
+class GoogleAnalyticsListAccountsOperator(BaseOperator):
+"""
+Lists all accounts to which the user has access.
+
+.. seealso::
+Check official API docs:
+
https://developers.google.com/analytics/devguides/config/mgmt/v3/mgmtReference/management/accounts/list
+
+.. seealso::
+For more information on how to use this operator, take a look at the 
guide:
+:ref:`howto/operator:GoogleAnalyticsListAccountsOperator`
+
+:param max_results: The maximum number of accounts to include in this 
response.
+:type: max_results: int
+:param: start_index: An index of the first account to retrieve
+Use this parameter as a pagination mechanism along with the 
max-results parameter.
+:type: start_index: int
 
 Review comment:
   It already fixed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-11 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r390967147
 
 

 ##
 File path: airflow/providers/google/marketing_platform/hooks/analytics.py
 ##
 @@ -0,0 +1,77 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from typing import Any, Dict, List
+
+from googleapiclient.discovery import Resource, build
+
+from airflow.providers.google.cloud.hooks.base import CloudBaseHook
+
+
+class GoogleAnalyticsHook(CloudBaseHook):
+"""
+Hook for Google Analytics 360.
+"""
+
+def __init__(
+self,
+api_version: str = "v3",
+gcp_connection_id: str = "google cloud default",
+*args,
+**kwargs
+):
+super().__init__(*args, **kwargs)
+self.api_version = api_version
+self.gcp_connection_is = gcp_connection_id
+self._conn = None
+
+def get_conn(self) -> Resource:
+"""
+Retrieves connection to Google Analytics 360.
+"""
+if not self._conn:
+http_authorized = self._authorize()
+self._conn = build(
+"analytics",
+self.api_version,
+http=http_authorized,
+cache_discovery=False,
+)
+return self._conn
+
+def list_accounts(self) -> List[Dict[str, Any]]:
+"""
+Lists accounts list from Google Analytics 360.
+
+More information about API:
+
http://googleapis.github.io/google-api-python-client/docs/dyn/analytics_v3.management.accounts.html
+
+"""
+
+self.log.info("Retrieving accounts list...")
+
+# pylint: disable=no-member
+result = []  # type: List[Dict]
+conn = self.get_conn()
+# return self._list_emails_from_response(accounts)
+while True:
+request = 
conn.management().accounts().list(start_index=len(result) + 1)
+response = request.execute(num_retries=self.num_retries)
+result.extend(response.get('items', []))
+if response["totalResults"] >= len(result):
 
 Review comment:
   Yes, below this is a `break`. 
   I changed it to `<=` in the latest commit.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-11 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r390969317
 
 

 ##
 File path: airflow/providers/google/marketing_platform/hooks/analytics.py
 ##
 @@ -0,0 +1,77 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from typing import Any, Dict, List
+
+from googleapiclient.discovery import Resource, build
+
+from airflow.providers.google.cloud.hooks.base import CloudBaseHook
+
+
+class GoogleAnalyticsHook(CloudBaseHook):
+"""
+Hook for Google Analytics 360.
+"""
+
+def __init__(
+self,
+api_version: str = "v3",
+gcp_connection_id: str = "google cloud default",
+*args,
+**kwargs
+):
+super().__init__(*args, **kwargs)
+self.api_version = api_version
+self.gcp_connection_is = gcp_connection_id
+self._conn = None
+
+def get_conn(self) -> Resource:
+"""
+Retrieves connection to Google Analytics 360.
+"""
+if not self._conn:
+http_authorized = self._authorize()
+self._conn = build(
+"analytics",
+self.api_version,
+http=http_authorized,
+cache_discovery=False,
+)
+return self._conn
+
+def list_accounts(self) -> List[Dict[str, Any]]:
+"""
+Lists accounts list from Google Analytics 360.
+
+More information about API:
+
http://googleapis.github.io/google-api-python-client/docs/dyn/analytics_v3.management.accounts.html
+
+"""
+
+self.log.info("Retrieving accounts list...")
+
+# pylint: disable=no-member
 
 Review comment:
   Since then little refactor has been made and few things are different right 
now, sorry. :/


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-12 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r391459941
 
 

 ##
 File path: airflow/providers/google/marketing_platform/hooks/analytics.py
 ##
 @@ -0,0 +1,77 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from typing import Any, Dict, List
+
+from googleapiclient.discovery import Resource, build
+
+from airflow.providers.google.cloud.hooks.base import CloudBaseHook
+
+
+class GoogleAnalyticsHook(CloudBaseHook):
+"""
+Hook for Google Analytics 360.
 
 Review comment:
   Updated with the latest commit.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-12 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r391460324
 
 

 ##
 File path: airflow/providers/google/marketing_platform/hooks/analytics.py
 ##
 @@ -0,0 +1,77 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from typing import Any, Dict, List
+
+from googleapiclient.discovery import Resource, build
+
+from airflow.providers.google.cloud.hooks.base import CloudBaseHook
+
+
+class GoogleAnalyticsHook(CloudBaseHook):
+"""
+Hook for Google Analytics 360.
+"""
+
+def __init__(
+self,
+api_version: str = "v3",
+gcp_connection_id: str = "google cloud default",
+*args,
+**kwargs
+):
+super().__init__(*args, **kwargs)
+self.api_version = api_version
+self.gcp_connection_is = gcp_connection_id
+self._conn = None
+
+def get_conn(self) -> Resource:
+"""
+Retrieves connection to Google Analytics 360.
+"""
+if not self._conn:
+http_authorized = self._authorize()
+self._conn = build(
+"analytics",
+self.api_version,
+http=http_authorized,
+cache_discovery=False,
+)
+return self._conn
+
+def list_accounts(self) -> List[Dict[str, Any]]:
+"""
+Lists accounts list from Google Analytics 360.
+
+More information about API:
+
http://googleapis.github.io/google-api-python-client/docs/dyn/analytics_v3.management.accounts.html
+
+"""
+
+self.log.info("Retrieving accounts list...")
+
+# pylint: disable=no-member
+result = []  # type: List[Dict]
+conn = self.get_conn()
+# return self._list_emails_from_response(accounts)
+while True:
+request = 
conn.management().accounts().list(start_index=len(result) + 1)
+response = request.execute(num_retries=self.num_retries)
+result.extend(response.get('items', []))
+if response["totalResults"] >= len(result):
 
 Review comment:
   One more fix no is `==`.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-12 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r391752126
 
 

 ##
 File path: airflow/providers/google/marketing_platform/operators/analytics.py
 ##
 @@ -0,0 +1,67 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+This module contains Google Analytics 360 operators.
+"""
+
+from airflow.models import BaseOperator
+from airflow.providers.google.marketing_platform.hooks.analytics import 
GoogleAnalyticsHook
+from airflow.utils.decorators import apply_defaults
+
+
+class GoogleAnalyticsListAccountsOperator(BaseOperator):
+"""
+Lists all accounts to which the user has access.
+
+.. seealso::
+Check official API docs:
+
https://developers.google.com/analytics/devguides/config/mgmt/v3/mgmtReference/management/accounts/list
+
+.. seealso::
+For more information on how to use this operator, take a look at the 
guide:
+:ref:`howto/operator:GoogleAnalyticsListAccountsOperator`
+
+:param max_results: The maximum number of accounts to include in this 
response.
+:type: max_results: int
+:param: start_index: An index of the first account to retrieve
+Use this parameter as a pagination mechanism along with the 
max-results parameter.
+:type: start_index: int
+:param api_version: The version of the api that will be requested for 
example 'v3'.
+:type api_version: str
+:param gcp_conn_id: The connection ID to use when fetching connection info.
+:type gcp_conn_id: str
+"""
+
+template_fields = ("api_version", "gcp_connection_id",)
+
+@apply_defaults
+def __init__(self,
+ api_version: str = "v3",
+ gcp_connection_id: str = "google cloud default",
+ *args,
+ **kwargs):
+super().__init__(*args, **kwargs)
+
+self.api_version = api_version
+self.gcp_connection_id = gcp_connection_id
+
+def execute(self, context):
+hook = GoogleAnalyticsHook(api_version=self.api_version,
+   gcp_connection_id=self.gcp_connection_id)
+result = hook.list_accounts()
 
 Review comment:
   Perfect. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-12 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r391755573
 
 

 ##
 File path: airflow/providers/google/marketing_platform/hooks/analytics.py
 ##
 @@ -0,0 +1,77 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from typing import Any, Dict, List
+
+from googleapiclient.discovery import Resource, build
+
+from airflow.providers.google.cloud.hooks.base import CloudBaseHook
+
+
+class GoogleAnalyticsHook(CloudBaseHook):
+"""
+Hook for Google Analytics 360.
 
 Review comment:
   I looked into others hooks, none of them have a link to the API doc. I will 
leave it as it is. The operator have link to API doc. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-12 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r391768447
 
 

 ##
 File path: airflow/providers/google/marketing_platform/hooks/analytics.py
 ##
 @@ -0,0 +1,80 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from typing import Any, Dict, List
+
+from googleapiclient.discovery import Resource, build
+
+from airflow.providers.google.cloud.hooks.base import CloudBaseHook
+
+
+class GoogleAnalyticsHook(CloudBaseHook):
+"""
+Hook for Google Analytics 360.
+"""
+
+def __init__(
+self,
+api_version: str = "v3",
+gcp_connection_id: str = "google cloud default",
+*args,
+**kwargs
+):
+super().__init__(*args, **kwargs)
+self.api_version = api_version
+self.gcp_connection_is = gcp_connection_id
+self._conn = None
+
+def get_conn(self) -> Resource:
+"""
+Retrieves connection to Google Analytics 360.
+"""
+if not self._conn:
+http_authorized = self._authorize()
+self._conn = build(
+"analytics",
+self.api_version,
+http=http_authorized,
+cache_discovery=False,
+)
+return self._conn
+
+def list_accounts(self) -> List[Dict[str, Any]]:
+"""
+Lists accounts list from Google Analytics 360.
+
+More information about API for python client:
+
http://googleapis.github.io/google-api-python-client/docs/dyn/analytics_v3.management.accounts.html
+
+"""
+
+self.log.info("Retrieving accounts list...")
+result = []  # type: List[Dict]
+conn = self.get_conn()
+# pylint: disable=no-member
+accounts = conn.management().accounts()
 
 Review comment:
   It works when the `# pylint: disable=no-member` is above the code. But 
sometimes it doesn't.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-12 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r391792290
 
 

 ##
 File path: airflow/providers/google/marketing_platform/hooks/analytics.py
 ##
 @@ -0,0 +1,80 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from typing import Any, Dict, List
+
+from googleapiclient.discovery import Resource, build
+
+from airflow.providers.google.cloud.hooks.base import CloudBaseHook
+
+
+class GoogleAnalyticsHook(CloudBaseHook):
+"""
+Hook for Google Analytics 360.
+"""
+
+def __init__(
+self,
+api_version: str = "v3",
+gcp_connection_id: str = "google cloud default",
+*args,
+**kwargs
+):
+super().__init__(*args, **kwargs)
+self.api_version = api_version
+self.gcp_connection_is = gcp_connection_id
+self._conn = None
+
+def get_conn(self) -> Resource:
+"""
+Retrieves connection to Google Analytics 360.
+"""
+if not self._conn:
+http_authorized = self._authorize()
+self._conn = build(
+"analytics",
+self.api_version,
+http=http_authorized,
+cache_discovery=False,
+)
+return self._conn
+
+def list_accounts(self) -> List[Dict[str, Any]]:
+"""
+Lists accounts list from Google Analytics 360.
+
+More information about API for python client:
+
http://googleapis.github.io/google-api-python-client/docs/dyn/analytics_v3.management.accounts.html
+
+"""
+
+self.log.info("Retrieving accounts list...")
+result = []  # type: List[Dict]
+conn = self.get_conn()
+# pylint: disable=no-member
+accounts = conn.management().accounts()
+while True:
+# start index has value 1
+request = accounts.list(start_index=len(result) + 1)
+response = request.execute(num_retries=self.num_retries)
+result.extend(response.get('items', []))
+# result is the number of fetched accounts from Analytics
+# when all accounts will be add to the result
+# the loop will be break
+if response["totalResults"] == len(result):
 
 Review comment:
   We discussed it offline. `<=` condition will be safer then `==`.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] Add Google Analytics 360 Accounts Retrieve Operator

2020-03-13 Thread GitBox
michalslowikowski00 commented on a change in pull request #7630: [AIRFLOW-6724] 
Add Google Analytics 360 Accounts Retrieve Operator
URL: https://github.com/apache/airflow/pull/7630#discussion_r392187313
 
 

 ##
 File path: airflow/providers/google/marketing_platform/hooks/analytics.py
 ##
 @@ -0,0 +1,80 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from typing import Any, Dict, List
+
+from googleapiclient.discovery import Resource, build
+
+from airflow.providers.google.cloud.hooks.base import CloudBaseHook
+
+
+class GoogleAnalyticsHook(CloudBaseHook):
+"""
+Hook for Google Analytics 360.
+"""
+
+def __init__(
+self,
+api_version: str = "v3",
+gcp_connection_id: str = "google cloud default",
+*args,
+**kwargs
+):
+super().__init__(*args, **kwargs)
+self.api_version = api_version
+self.gcp_connection_is = gcp_connection_id
+self._conn = None
+
+def get_conn(self) -> Resource:
+"""
+Retrieves connection to Google Analytics 360.
+"""
+if not self._conn:
+http_authorized = self._authorize()
+self._conn = build(
+"analytics",
+self.api_version,
+http=http_authorized,
+cache_discovery=False,
+)
+return self._conn
+
+def list_accounts(self) -> List[Dict[str, Any]]:
+"""
+Lists accounts list from Google Analytics 360.
+
+More information about API for python client:
+
http://googleapis.github.io/google-api-python-client/docs/dyn/analytics_v3.management.accounts.html
+
+"""
+
+self.log.info("Retrieving accounts list...")
+result = []  # type: List[Dict]
+conn = self.get_conn()
+# pylint: disable=no-member
+accounts = conn.management().accounts()
 
 Review comment:
   Fixed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services