[ https://issues.apache.org/jira/browse/BEAM-6169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kenneth Knowles updated BEAM-6169: ---------------------------------- Component/s: (was: gcp-quota) sdk-py-core > Dataflow runner: rateLimit exceeded from BigQuery API in start_bundle > --------------------------------------------------------------------- > > Key: BEAM-6169 > URL: https://issues.apache.org/jira/browse/BEAM-6169 > Project: Beam > Issue Type: Bug > Components: sdk-py-core > Affects Versions: 2.8.0 > Environment: Python SDK 2.8, Dataflow runner > Reporter: Thomas Pilewicz > Assignee: Kenneth Knowles > Priority: Minor > > When many machines, or too big machines are used, BigQuery returns rateLimit > exceeded errors in response to the GET requests from > BigQueryWriteFn.start_bundle. > Unsure if this causes performance issues, or is just noise. (Produced with a > pipeline reading strings from Pub/Sub that are ~1000 characters long, and > writing directly to BigQuery) > {{... File > "/usr/local/lib/python2.7/dist-packages/apache_beam/io/gcp/bigquery.py", line > 1087, in get_or_create_table found_table = self._get_table(project_id, > dataset_id, table_id) File > "/usr/local/lib/python2.7/dist-packages/apache_beam/utils/retry.py", line > 184, in wrapper return fun(*args, **kwargs) File > "/usr/local/lib/python2.7/dist-packages/apache_beam/io/gcp/bigquery.py", line > 925, in _get_table response = self.client.tables.Get(request) File > "/usr/local/lib/python2.7/dist-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", > line 611, in Get config, request, global_params=global_params) File > "/usr/local/lib/python2.7/dist-packages/apitools/base/py/base_api.py", line > 722, in _RunMethod return self.ProcessHttpResponse(method_config, > http_response, request) File > "/usr/local/lib/python2.7/dist-packages/apitools/base/py/base_api.py", line > 728, in ProcessHttpResponse self.__ProcessHttpResponse(method_config, > http_response, request)) File > "/usr/local/lib/python2.7/dist-packages/apitools/base/py/base_api.py", line > 599, in __ProcessHttpResponse http_response, method_config=method_config, > request=request) HttpForbiddenError: HttpError accessing > <[https://www.googleapis.com/bigquery/v2/projects/<xxxxxx>/datasets/|https://www.googleapis.com/bigquery/v2/projects/carrefour-170216/datasets/realtime_dashboard/tables/realtime_errors?alt=json] > > [<xxxxxx>|https://www.googleapis.com/bigquery/v2/projects/carrefour-170216/datasets/realtime_dashboard/tables/realtime_errors?alt=json] > > [/tables/|https://www.googleapis.com/bigquery/v2/projects/carrefour-170216/datasets/realtime_dashboard/tables/realtime_errors?alt=json] > > [<xxxxxx>|https://www.googleapis.com/bigquery/v2/projects/carrefour-170216/datasets/realtime_dashboard/tables/realtime_errors?alt=json] > > [?alt=json|https://www.googleapis.com/bigquery/v2/projects/carrefour-170216/datasets/realtime_dashboard/tables/realtime_errors?alt=json]>: > response: < > {'status': '403', 'content-length': '577', 'x-xss-protection': '1; > mode=block', 'x-content-type-options': 'nosniff', 'transfer-encoding': > 'chunked', 'expires': 'Sun, 25 Nov 2018 14:36:24 GMT', 'vary': 'Origin, > X-Origin', 'server': 'GSE', '-content-encoding': 'gzip', 'cache-control': > 'private, max-age=0', 'date': 'Sun, 25 Nov 2018 14:36:24 GMT', > 'x-frame-options': 'SAMEORIGIN', 'content-type': 'application/json; > charset=UTF-8'} > >, content <{ "error": { "errors": [ > { "domain": "global", "reason": "rateLimitExceeded", "message": "Exceeded > rate limits: Your user_method exceeded quota for api requests per user per > method. For more information, see > https://cloud.google.com/bigquery/troubleshooting-errors", "locationType": > "other", "location": "helix_api.method_request" } > ], "code": 403, "message": "Exceeded rate limits: Your user_method exceeded > quota for api requests per user per method. For more information, see > [https://cloud.google.com/bigquery/troubleshooting-errors]"}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)