[ https://issues.apache.org/jira/browse/BEAM-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17037207#comment-17037207 ]
Yifan Zou commented on BEAM-9319: --------------------------------- Looked into our pubsub topics. Seems none of them has ttl applied. I can help to delete stale topics which start with "integ-test-PubsubJsonIT" at this moment. And we do need to consider to have the tests release the resource usage after completion. Or, having a tool to cleanup the unused resources periodically. > ResourceExhausted: topics-per-project > ------------------------------------- > > Key: BEAM-9319 > URL: https://issues.apache.org/jira/browse/BEAM-9319 > Project: Beam > Issue Type: Bug > Components: test-failures, testing > Reporter: Ahmet Altay > Assignee: Yifan Zou > Priority: Major > > Tests are failing due to quota issues. Do we need to clean up topics after > tests or set a shorter TTL? > Log: https://builds.apache.org/job/beam_PreCommit_Python_Commit/11178/ > Error: > 08:24:40 > ====================================================================== > 08:24:40 ERROR: test_streaming_wordcount_it > (apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT) > 08:24:40 > ---------------------------------------------------------------------- > 08:24:40 Traceback (most recent call last): > 08:24:40 File > "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/apache_beam/examples/streaming_wordcount_it_test.py", > line 58, in setUp > 08:24:40 self.pub_client.topic_path(self.project, INPUT_TOPIC + > self.uuid)) > 08:24:40 File > "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/cloud/pubsub_v1/_gapic.py", > line 40, in <lambda> > 08:24:40 fx = lambda self, *a, **kw: wrapped_fx(self.api, *a, **kw) # > noqa > 08:24:40 File > "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/cloud/pubsub_v1/gapic/publisher_client.py", > line 332, in create_topic > 08:24:40 request, retry=retry, timeout=timeout, metadata=metadata > 08:24:40 File > "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/api_core/gapic_v1/method.py", > line 143, in __call__ > 08:24:40 return wrapped_func(*args, **kwargs) > 08:24:40 File > "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/api_core/retry.py", > line 286, in retry_wrapped_func > 08:24:40 on_error=on_error, > 08:24:40 File > "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/api_core/retry.py", > line 184, in retry_target > 08:24:40 return target() > 08:24:40 File > "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/api_core/timeout.py", > line 214, in func_with_timeout > 08:24:40 return func(*args, **kwargs) > 08:24:40 File > "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/api_core/grpc_helpers.py", > line 59, in error_remapped_callable > 08:24:40 six.raise_from(exceptions.from_grpc_error(exc), exc) > 08:24:40 File > "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/six.py", > line 738, in raise_from > 08:24:40 raise value > 08:24:40 ResourceExhausted: 429 Your project has exceeded a limit: > (type="topics-per-project", current=10000, maximum=10000). -- This message was sent by Atlassian Jira (v8.3.4#803005)