brumi1024 commented on code in PR #8765:
URL: https://github.com/apache/hadoop/pull/8765#discussion_r4206887664
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui/src/main/webapp/src/components/elements/DiagnosticsDialog.tsx:
##########
@@ -99,8 +120,33 @@ export function DiagnosticsDialog() {
description: 'Node metadata returned by /nodes.',
data: nodes,
},
+ {
+ id: 'bulkActivities',
+ label: 'Scheduler Bulk Activities',
+ description: 'Live response from /scheduler/bulk-activities.',
+ live: {
+ countLabel: 'Bulk activities count',
+ maxCount: MAX_DIAGNOSTIC_BULK_ACTIVITIES_COUNT,
+ fetch: (count) => apiClient.getBulkSchedulerActivities(count),
+ },
+ },
+ {
+ id: 'rmJstack',
+ label: 'ResourceManager JStack',
+ description: 'Live thread dump from /jstack (ResourceManager JVM).',
+ live: {
+ countLabel: 'ResourceManager jstack count',
+ maxCount: MAX_DIAGNOSTIC_RM_JSTACK_COUNT,
+ fetch: (count) => apiClient.getResourceManagerJstack(count),
+ },
Review Comment:
> This PR depends on the following PR for getting the RM JStacks(by default,
it is false and has to be turned on):
Will these be shown if the RM JStacks are turned off?
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui/src/main/webapp/src/components/elements/DiagnosticsDialog.tsx:
##########
@@ -123,7 +169,9 @@ export function DiagnosticsDialog() {
for (const option of datasetOptions) {
if (selectedDatasets.includes(option.id)) {
- (payload.datasets as Record<string, unknown>)[option.id] = option.data;
+ (payload.datasets as Record<string, unknown>)[option.id] = option.live
+ ? await option.live.fetch(Number(counts[option.id]))
Review Comment:
getBulkActivities calls ActivitiesManager.recordAndGetBulkActivitiesInfo,
which busy-waits (Thread.sleep(1) loop) until activitiesCount node-update
cycles have been recorded. Is there a chance this runs past the client's
default 30s timeout? If yes what happens?
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui/src/main/webapp/src/components/elements/DiagnosticsDialog.tsx:
##########
@@ -123,7 +169,9 @@ export function DiagnosticsDialog() {
for (const option of datasetOptions) {
if (selectedDatasets.includes(option.id)) {
- (payload.datasets as Record<string, unknown>)[option.id] = option.data;
+ (payload.datasets as Record<string, unknown>)[option.id] = option.live
Review Comment:
What about running these calls in parallel? Promise.all / Promise.allSettled.
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui/src/main/webapp/src/lib/api/YarnApiClient.ts:
##########
@@ -34,9 +34,22 @@ import type {
VersionResponse,
YarnConfigResponse,
ValidationResponse,
+ BulkActivitiesResponse,
} from '~/types';
import { HTTP_AUTH_PROPERTY, READ_ONLY_PROPERTY } from '~/config';
+/** Matches RMWebServices DEFAULT_ACTIVITIES_COUNT. */
+export const DEFAULT_DIAGNOSTIC_BULK_ACTIVITIES_COUNT = 10;
+
+/** Matches RMWebServices MAX_ACTIVITIES_COUNT. */
+export const MAX_DIAGNOSTIC_BULK_ACTIVITIES_COUNT = 500;
+
+/** Default jstack iterations for RM diagnostics export. */
+export const DEFAULT_DIAGNOSTIC_RM_JSTACK_COUNT = 2;
+
+/** Upper bound for RM jstack iterations in the diagnostics UI. */
+export const MAX_DIAGNOSTIC_RM_JSTACK_COUNT = 10;
Review Comment:
Are these configurable? If yes these could be collected from the RM to avoid
mismatch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
