[
https://issues.apache.org/jira/browse/EAGLE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15923454#comment-15923454
]
ASF GitHub Bot commented on EAGLE-946:
--------------------------------------
Github user wujinhu commented on a diff in the pull request:
https://github.com/apache/eagle/pull/869#discussion_r105816819
--- Diff:
eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourcefetch/RMResourceFetcher.java
---
@@ -112,40 +110,77 @@ private String getMRFinishedJobURL(String
lastFinishedTime) {
+ lastFinishedTime + "&" + Constants.ANONYMOUS_PARAMETER;
}
- private List<AppInfo> doFetchRunningApplicationsList(String urlString,
Constants.CompressionType compressionType) throws Exception {
- List<AppInfo> result = new ArrayList<>(0);
- InputStream is = null;
+ private String getAccepedAppURL() {
+ String baseUrl =
URLUtil.removeTrailingSlash(selector.getSelectedUrl());
+ return String.format("%s/%s?state=ACCEPTED&%s", baseUrl,
Constants.V2_APPS_URL, Constants.ANONYMOUS_PARAMETER);
+ }
+
+ private List<AppInfo> doFetchRunningApplicationsList(Constants.JobType
jobType,
+
Constants.CompressionType compressionType,
+ Object...
parameter) throws Exception {
+ Map<String, AppInfo> result = new HashMap();
+ List<AppInfo> apps = new ArrayList<>();
try {
- checkUrl();
- LOG.info("Going to call yarn api to fetch running application
list: " + urlString);
- is = InputStreamUtils.getInputStream(urlString, null,
compressionType);
- final AppsWrapper appWrapper = OBJ_MAPPER.readValue(is,
AppsWrapper.class);
- if (appWrapper != null && appWrapper.getApps() != null &&
appWrapper.getApps().getApp() != null) {
- result = appWrapper.getApps().getApp();
+ selector.checkUrl();
+ String urlString = getRunningJobURL(jobType, null, null);
+ if (parameter.length == 0) {
+ return doFetchApplicationsList(urlString, compressionType);
}
- return result;
- } finally {
- if (is != null) {
- try {
- is.close();
- } catch (Exception e) {
- LOG.warn("{}", e);
- }
+
+ int requests = (int) parameter[0];
+ if (requests <= 1) {
+ return doFetchApplicationsList(urlString, compressionType);
}
+
+ long interval = DateTimeUtil.ONEHOUR * 6 / (requests - 1);
+ long currentTime = System.currentTimeMillis();
+ long earliestTime = currentTime - DateTimeUtil.ONEHOUR * 6;
--- End diff --
Should we make this configurable? I am not sure whether the current number
of running jobs that started 6 hours ago is still large?
> Refactor MRRunningJobApp & HadoopQueueApp
> ------------------------------------------
>
> Key: EAGLE-946
> URL: https://issues.apache.org/jira/browse/EAGLE-946
> Project: Eagle
> Issue Type: Improvement
> Affects Versions: v0.5.0
> Reporter: Zhao, Qingwen
> Assignee: Zhao, Qingwen
>
> Requirements
> 1. REST apis to the remote cluster should be called only once.
> 2. For each request, the fetch running apps should be limits
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
