Hello Sumit, There is no requirement to delete old jobs, though you can archive completed jobs via a recently released feature [1]. There is no limit to the number of old jobs that are accumulated. Further, the UI will show all running Dataflow jobs and all jobs run in the last 30 days [2].
Hope this helps! [1] https://cloud.google.com/dataflow/docs/guides/stopping-a-pipeline#archive [2] https://cloud.google.com/dataflow/docs/guides/monitoring-overview Svetak Sundhar Data Engineer s <nellywil...@google.com>vetaksund...@google.com On Tue, Jan 2, 2024 at 6:11 AM Sumit Desai via user <user@beam.apache.org> wrote: > Hi all, > > I have integrated Google Dataflow with my application. After every run, > there is one job created under Dataflow as shown in the screenshot. > > Do I need to delete the old jobs? Is there any restriction on the number > of old jobs that can get accumulated here? > > Thanks & Regards, > Sumit Desai >
