Thank you for quick response Andrew. I'll cleanup these. I'll keep the bug open and assign it to @Kenneth Knowles <k...@google.com> who's working on SQK for follow up: we need a way to automatically cleanup topics.
Current suggestions: 1. Make SQL cleanup created topics 2. Cleanup topics created by SQL in tests 3. Increase quota so that topics have enough time to be cleaned up automatically. Regards, --Mikhail Have feedback <http://go/migryz-feedback>? On Wed, Feb 6, 2019 at 8:05 AM Andrew Pilloud <apill...@google.com> wrote: > SQL doesn't cleanup pubsub subscriptions. Feel free to delete those. > > Andrew > > On Wed, Feb 6, 2019, 8:01 AM Mikhail Gryzykhin <mig...@google.com> wrote: > >> +Kenneth Knowles <k...@google.com> you're working on SQL recently, so >> might provide some info. >> >> I see a lot of topics of format >> rojects/apache-beam-testing/topics/integ-test-PubsubJsonIT-testSQLLimit-2018-08-27-16-55-44-342-start-7392789257486934721 >> <https://pantheon.corp.google.com/cloudpubsub/topics/integ-test-PubsubJsonIT-testSQLLimit-2018-08-27-16-55-44-342-start-7392789257486934721?project=apache-beam-testing>. >> Seems we do not cleanup properly. >> >> Is it safe to cleanup topics with this name? >> >> --Mikhail >> >> Have feedback <http://go/migryz-feedback>? >> >> >> On Wed, Feb 6, 2019 at 7:46 AM Mikhail Gryzykhin <mig...@google.com> >> wrote: >> >>> Minor UPD: >>> As expected it fails most of our test jobs, since we use Pub/Subs in >>> many tests. >>> >>> --Mikhail >>> >>> Have feedback <http://go/migryz-feedback>? >>> >>> >>> On Wed, Feb 6, 2019 at 7:38 AM Mikhail Gryzykhin <mig...@google.com> >>> wrote: >>> >>>> Hi everyone, >>>> >>>> Our python pipelines failed with limit exceeded error >>>> <https://builds.apache.org/job/beam_PostCommit_Python_Verify/7313/console> >>>> : >>>> >>>> ResourceExhausted: 429 Your project has exceeded a limit: >>>> (type="topics-per-project", current=10000, maximum=10000). >>>> >>>> >>>> Does anyone know if there were new tests that use topics added recently? >>>> >>>> I tried to see list of topics, but UI fails >>>> <https://pantheon.corp.google.com/cloudpubsub/topicList?project=apache-beam-testing> >>>> to load. Will see if I can use APIs to investigate. >>>> >>>> If anyone has good insight, please, pick up BEAM-6610 >>>> <https://issues.apache.org/jira/browse/BEAM-6610>. >>>> >>>> --Mikhail >>>> >>>> Have feedback <http://go/migryz-feedback>? >>>> >>>
