Could someone please help me to understand better.. On Thu, Oct 17, 2019 at 7:41 PM Chetan Khatri <chetan.opensou...@gmail.com> wrote:
> Hi Users, > > I do submit *X* number of jobs with Airflow to Yarn as a part of workflow > for *Y *customer. I could potentially run workflow for customer *Z *but I > need to check that how much resources are available over the cluster so > jobs for next customer should start. > > Could you please tell what is the best way to handle this. Currently, I am > just checking availableMB > 100 then trigger next Airflow DAG over Yarn. > > GET http://rm-http-address:port/ws/v1/cluster/metrics > > Thanks. > >
