[jira] [Updated] (YARN-6871) Add additional deSelects params in getAppReport
[ https://issues.apache.org/jira/browse/YARN-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanuj Nayak updated YARN-6871: -- Attachment: YARN-6871.008.patch > Add additional deSelects params in getAppReport > --- > > Key: YARN-6871 > URL: https://issues.apache.org/jira/browse/YARN-6871 > Project: Hadoop YARN > Issue Type: New Feature > Components: resourcemanager, router >Reporter: Giovanni Matteo Fumarola >Assignee: Tanuj Nayak > Attachments: YARN-6871.002.patch, YARN-6871.003.patch, > YARN-6871.004.patch, YARN-6871.005.patch, YARN-6871.006.patch, > YARN-6871.007.patch, YARN-6871.008.patch, YARN-6871.proto.patch > > > This jira tracks the effort to add additional deSelect params to the > GetAppReport to make it lighter and faster. > With the current one we are facing a scalability issues. > E.g. with ~500 applications running the AppReport can reach up to 300MB in > size due to the {{ResourceRequest}} in the {{AppInfo}}. > Yarn RM will return the new result faster and it will use less compute cycles > to create the report and it will improve the YARN RM and Client's > performances. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6871) Add additional deSelects params in getAppReport
[ https://issues.apache.org/jira/browse/YARN-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanuj Nayak updated YARN-6871: -- Attachment: YARN-6871.007.patch > Add additional deSelects params in getAppReport > --- > > Key: YARN-6871 > URL: https://issues.apache.org/jira/browse/YARN-6871 > Project: Hadoop YARN > Issue Type: New Feature > Components: resourcemanager, router >Reporter: Giovanni Matteo Fumarola >Assignee: Tanuj Nayak > Attachments: YARN-6871.002.patch, YARN-6871.003.patch, > YARN-6871.004.patch, YARN-6871.005.patch, YARN-6871.006.patch, > YARN-6871.007.patch, YARN-6871.proto.patch > > > This jira tracks the effort to add additional deSelect params to the > GetAppReport to make it lighter and faster. > With the current one we are facing a scalability issues. > E.g. with ~500 applications running the AppReport can reach up to 300MB in > size due to the {{ResourceRequest}} in the {{AppInfo}}. > Yarn RM will return the new result faster and it will use less compute cycles > to create the report and it will improve the YARN RM and Client's > performances. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6871) Add additional deSelects params in getAppReport
[ https://issues.apache.org/jira/browse/YARN-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanuj Nayak updated YARN-6871: -- Attachment: YARN-6871.006.patch > Add additional deSelects params in getAppReport > --- > > Key: YARN-6871 > URL: https://issues.apache.org/jira/browse/YARN-6871 > Project: Hadoop YARN > Issue Type: New Feature > Components: resourcemanager, router >Reporter: Giovanni Matteo Fumarola >Assignee: Tanuj Nayak > Attachments: YARN-6871.002.patch, YARN-6871.003.patch, > YARN-6871.004.patch, YARN-6871.005.patch, YARN-6871.006.patch, > YARN-6871.proto.patch > > > This jira tracks the effort to add additional deSelect params to the > GetAppReport to make it lighter and faster. > With the current one we are facing a scalability issues. > E.g. with ~500 applications running the AppReport can reach up to 300MB in > size due to the {{ResourceRequest}} in the {{AppInfo}}. > Yarn RM will return the new result faster and it will use less compute cycles > to create the report and it will improve the YARN RM and Client's > performances. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6871) Add additional deSelects params in getAppReport
[ https://issues.apache.org/jira/browse/YARN-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanuj Nayak updated YARN-6871: -- Attachment: YARN-6871.005.patch > Add additional deSelects params in getAppReport > --- > > Key: YARN-6871 > URL: https://issues.apache.org/jira/browse/YARN-6871 > Project: Hadoop YARN > Issue Type: New Feature > Components: resourcemanager, router >Reporter: Giovanni Matteo Fumarola >Assignee: Tanuj Nayak > Attachments: YARN-6871.002.patch, YARN-6871.003.patch, > YARN-6871.004.patch, YARN-6871.005.patch, YARN-6871.proto.patch > > > This jira tracks the effort to add additional deSelect params to the > GetAppReport to make it lighter and faster. > With the current one we are facing a scalability issues. > E.g. with ~500 applications running the AppReport can reach up to 300MB in > size due to the {{ResourceRequest}} in the {{AppInfo}}. > Yarn RM will return the new result faster and it will use less compute cycles > to create the report and it will improve the YARN RM and Client's > performances. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6871) Add additional deSelects params in getAppReport
[ https://issues.apache.org/jira/browse/YARN-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanuj Nayak updated YARN-6871: -- Attachment: YARN-6871.004.patch > Add additional deSelects params in getAppReport > --- > > Key: YARN-6871 > URL: https://issues.apache.org/jira/browse/YARN-6871 > Project: Hadoop YARN > Issue Type: New Feature > Components: resourcemanager, router >Reporter: Giovanni Matteo Fumarola >Assignee: Tanuj Nayak > Attachments: YARN-6871.002.patch, YARN-6871.003.patch, > YARN-6871.004.patch, YARN-6871.proto.patch > > > This jira tracks the effort to add additional deSelect params to the > GetAppReport to make it lighter and faster. > With the current one we are facing a scalability issues. > E.g. with ~500 applications running the AppReport can reach up to 300MB in > size due to the {{ResourceRequest}} in the {{AppInfo}}. > Yarn RM will return the new result faster and it will use less compute cycles > to create the report and it will improve the YARN RM and Client's > performances. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6871) Add additional deSelects params in getAppReport
[ https://issues.apache.org/jira/browse/YARN-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanuj Nayak updated YARN-6871: -- Attachment: YARN-6871.003.patch > Add additional deSelects params in getAppReport > --- > > Key: YARN-6871 > URL: https://issues.apache.org/jira/browse/YARN-6871 > Project: Hadoop YARN > Issue Type: New Feature > Components: resourcemanager, router >Reporter: Giovanni Matteo Fumarola >Assignee: Tanuj Nayak > Attachments: YARN-6871.002.patch, YARN-6871.003.patch, > YARN-6871.proto.patch > > > This jira tracks the effort to add additional deSelect params to the > GetAppReport to make it lighter and faster. > With the current one we are facing a scalability issues. > E.g. with ~500 applications running the AppReport can reach up to 300MB in > size due to the {{ResourceRequest}} in the {{AppInfo}}. > Yarn RM will return the new result faster and it will use less compute cycles > to create the report and it will improve the YARN RM and Client's > performances. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6871) Add additional deSelects params in getAppReport
[ https://issues.apache.org/jira/browse/YARN-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanuj Nayak updated YARN-6871: -- Attachment: YARN-6871.002.patch Added changes from [~giovanni.fumarola]'s and [~sunilg]'s comments. > Add additional deSelects params in getAppReport > --- > > Key: YARN-6871 > URL: https://issues.apache.org/jira/browse/YARN-6871 > Project: Hadoop YARN > Issue Type: New Feature > Components: resourcemanager, router >Reporter: Giovanni Matteo Fumarola >Assignee: Tanuj Nayak > Attachments: YARN-6871.002.patch, YARN-6871.proto.patch > > > This jira tracks the effort to add additional deSelect params to the > GetAppReport to make it lighter and faster. > With the current one we are facing a scalability issues. > E.g. with ~500 applications running the AppReport can reach up to 300MB in > size due to the {{ResourceRequest}} in the {{AppInfo}}. > Yarn RM will return the new result faster and it will use less compute cycles > to create the report and it will improve the YARN RM and Client's > performances. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6871) Add additional deSelects params in getAppReport
[ https://issues.apache.org/jira/browse/YARN-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanuj Nayak updated YARN-6871: -- Attachment: YARN-6871.proto.patch Uploaded proto. From an offline discussion with @Giovanni Matteo Fumarola, we decided to start with 4 params: appNodeLabelExpression, amNodeLabelExpression, resourceInfo and timeouts. > Add additional deSelects params in getAppReport > --- > > Key: YARN-6871 > URL: https://issues.apache.org/jira/browse/YARN-6871 > Project: Hadoop YARN > Issue Type: New Feature > Components: resourcemanager, router >Reporter: Giovanni Matteo Fumarola >Assignee: Tanuj Nayak > Attachments: YARN-6871.proto.patch > > > This jira tracks the effort to add additional deSelect params to the > GetAppReport to make it lighter and faster. > With the current one we are facing a scalability issues. > E.g. with ~500 applications running the AppReport can reach up to 300MB in > size due to the {{ResourceRequest}} in the {{AppInfo}}. > Yarn RM will return the new result faster and it will use less compute cycles > to create the report and it will improve the YARN RM and Client's > performances. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6871) Add additional deSelects params in getAppReport
[ https://issues.apache.org/jira/browse/YARN-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni Matteo Fumarola updated YARN-6871: --- Description: This jira tracks the effort to add additional deSelect params to the GetAppReport to make it lighter and faster. With the current one we are facing a scalability issues. E.g. with ~500 applications running the AppReport can reach up to 300MB in size due to the {{ResourceRequest}} in the {{AppInfo}}. Yarn RM will return the new result faster and it will use less compute cycles to create the report and it will improve the YARN RM and Client's performances. was: This jira tracks the effort to create a new REST API similar to the current GetAppReport but lighter and faster. With the current one we are facing a scalability issues. E.g. with ~500 applications running the AppReport can reach up to 300MB in size due to the {{ResourceRequest}} in the {{AppInfo}}. The new method returns for each application the following and essential information: * The application id; * The application name; * The application state according to the ResourceManager - valid values are members of the YarnApplicationState enum: NEW, NEW_SAVING, SUBMITTED, ACCEPTED, RUNNING, FINISHED, FAILED, KILLED; * The final status of the application if finished - reported by the application itself - valid values are: UNDEFINED, SUCCEEDED, FAILED, KILLED; * The web URL that can be used to track the application; * Detailed diagnostics information; * The URL of the application master container logs; * The nodes http address of the application master; * The progress of the application as a percent. Yarn RM will return the new result faster and it will use less compute cycles to create the report and it will improve the YARN RM and Client's performances. > Add additional deSelects params in getAppReport > --- > > Key: YARN-6871 > URL: https://issues.apache.org/jira/browse/YARN-6871 > Project: Hadoop YARN > Issue Type: New Feature > Components: resourcemanager, router >Reporter: Giovanni Matteo Fumarola > > This jira tracks the effort to add additional deSelect params to the > GetAppReport to make it lighter and faster. > With the current one we are facing a scalability issues. > E.g. with ~500 applications running the AppReport can reach up to 300MB in > size due to the {{ResourceRequest}} in the {{AppInfo}}. > Yarn RM will return the new result faster and it will use less compute cycles > to create the report and it will improve the YARN RM and Client's > performances. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6871) Add additional deSelects params in getAppReport
[ https://issues.apache.org/jira/browse/YARN-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni Matteo Fumarola updated YARN-6871: --- Summary: Add additional deSelects params in getAppReport (was: Add a new REST API GetAppReport call lighter and faster than the current one) > Add additional deSelects params in getAppReport > --- > > Key: YARN-6871 > URL: https://issues.apache.org/jira/browse/YARN-6871 > Project: Hadoop YARN > Issue Type: New Feature > Components: resourcemanager, router >Reporter: Giovanni Matteo Fumarola > > This jira tracks the effort to create a new REST API similar to the current > GetAppReport but lighter and faster. > With the current one we are facing a scalability issues. > E.g. with ~500 applications running the AppReport can reach up to 300MB in > size due to the {{ResourceRequest}} in the {{AppInfo}}. > The new method returns for each application the following and essential > information: > * The application id; > * The application name; > * The application state according to the ResourceManager - valid values are > members of the YarnApplicationState enum: NEW, NEW_SAVING, SUBMITTED, > ACCEPTED, RUNNING, FINISHED, FAILED, KILLED; > * The final status of the application if finished - reported by the > application itself - valid values are: UNDEFINED, SUCCEEDED, FAILED, KILLED; > * The web URL that can be used to track the application; > * Detailed diagnostics information; > * The URL of the application master container logs; > * The nodes http address of the application master; > * The progress of the application as a percent. > Yarn RM will return the new result faster and it will use less compute cycles > to create the report and it will improve the YARN RM and Client's > performances. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org