You should set max num workers to set an upper bound on how many will be used in your job. It will not fail the pipeline when that limit is reached.
On Fri, May 8, 2020 at 10:03 AM OrielResearch Eila Arich-Landkof < [email protected]> wrote: > Hi all, > > I am hitting the compute engine limit. I need to escalate that request > with Google support to increase this quota. > Could you please advise what is the best way to manage this? see attached > the exact quota screenshot. > > Does limiting the num_of_workers will allow the job to proceed with a > limited number of workers or fail the job when reaching the workers number? > > or is there a different way to manage this limit? > > Thanks, > -- > Eila > <http://www.orielresearch.com> > Meetup <https://www.meetup.com/Deep-Learning-In-Production/> >
