[GitHub] eagle issue #869: EAGLE-946: Refactor MRRunningJobApp & HadoopQueueApp
Github user wujinhu commented on the issue: https://github.com/apache/eagle/pull/869 mark ---
[GitHub] eagle issue #963: Eagle-1046 Eagle supports policies import to a new site fr...
Github user wujinhu commented on the issue: https://github.com/apache/eagle/pull/963 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle issue #909: EAGLE-999: JobConfigSerDeser fails to serialize/deserializ...
Github user wujinhu commented on the issue: https://github.com/apache/eagle/pull/909 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle issue #902: [EAGLE-985] Upgrade hbase dependency to 1.2.5
Github user wujinhu commented on the issue: https://github.com/apache/eagle/pull/902 Should we have multiple version support for HBase(Storm/Kafka etc) so that users can still use old versions if they cannot switch to new version quickly? I think it will be better if we have some abstractions first instead of doing like this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle issue #900: EAGLE-989: Fix a bug in Resource Manager HA checker
Github user wujinhu commented on the issue: https://github.com/apache/eagle/pull/900 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle pull request #869: EAGLE-946: Refactor MRRunningJobApp & HadoopQueueAp...
Github user wujinhu commented on a diff in the pull request:
https://github.com/apache/eagle/pull/869#discussion_r105816819
--- Diff:
eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourcefetch/RMResourceFetcher.java
---
@@ -112,40 +110,77 @@ private String getMRFinishedJobURL(String
lastFinishedTime) {
+ lastFinishedTime + "&" + Constants.ANONYMOUS_PARAMETER;
}
-private List doFetchRunningApplicationsList(String urlString,
Constants.CompressionType compressionType) throws Exception {
-List result = new ArrayList<>(0);
-InputStream is = null;
+private String getAccepedAppURL() {
+String baseUrl =
URLUtil.removeTrailingSlash(selector.getSelectedUrl());
+return String.format("%s/%s?state=ACCEPTED&%s", baseUrl,
Constants.V2_APPS_URL, Constants.ANONYMOUS_PARAMETER);
+}
+
+private List doFetchRunningApplicationsList(Constants.JobType
jobType,
+
Constants.CompressionType compressionType,
+ Object...
parameter) throws Exception {
+Map result = new HashMap();
+List apps = new ArrayList<>();
try {
-checkUrl();
-LOG.info("Going to call yarn api to fetch running application
list: " + urlString);
-is = InputStreamUtils.getInputStream(urlString, null,
compressionType);
-final AppsWrapper appWrapper = OBJ_MAPPER.readValue(is,
AppsWrapper.class);
-if (appWrapper != null && appWrapper.getApps() != null &&
appWrapper.getApps().getApp() != null) {
-result = appWrapper.getApps().getApp();
+selector.checkUrl();
+String urlString = getRunningJobURL(jobType, null, null);
+if (parameter.length == 0) {
+return doFetchApplicationsList(urlString, compressionType);
}
-return result;
-} finally {
-if (is != null) {
-try {
-is.close();
-} catch (Exception e) {
-LOG.warn("{}", e);
-}
+
+int requests = (int) parameter[0];
+if (requests <= 1) {
+return doFetchApplicationsList(urlString, compressionType);
}
+
+long interval = DateTimeUtil.ONEHOUR * 6 / (requests - 1);
+long currentTime = System.currentTimeMillis();
+long earliestTime = currentTime - DateTimeUtil.ONEHOUR * 6;
--- End diff --
Should we make this configurable? I am not sure whether the current number
of running jobs that started 6 hours ago is still large?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] eagle pull request #863: [MINOR] fix sla job timezone issue
GitHub user wujinhu opened a pull request: https://github.com/apache/eagle/pull/863 [MINOR] fix sla job timezone issue You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-851 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/863.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #863 commit 6d3d47e52a8baf7208c3c7fff69eb36e95d1a3a0 Author: wujinhu Date: 2017-03-09T08:47:40Z fix sla job timezone issue --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle pull request #858: [MINOR] optimize sla publisher
GitHub user wujinhu opened a pull request: https://github.com/apache/eagle/pull/858 [MINOR] optimize sla publisher You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-850 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/858.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #858 commit ee8e8b0b3ab922721fe033146d7e035d097a45e6 Author: wujinhu Date: 2017-03-07T10:25:20Z optimize sla publisher --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle pull request #857: [MINOR] fix null pointer exception for sla jobs
GitHub user wujinhu opened a pull request: https://github.com/apache/eagle/pull/857 [MINOR] fix null pointer exception for sla jobs You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-849 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/857.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #857 commit 11a8e862151f4fd491bd2c916ca7a6a1ad8cb324 Author: wujinhu Date: 2017-03-07T05:20:16Z fix null pointer exception for sla commit 1b35f1cdeb89ee632ff63d7663cdb665d91813b3 Author: wujinhu Date: 2017-03-07T05:36:21Z fix null pointer exception for sla --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle pull request #856: [MINOR] fix null pointer exception for sla
Github user wujinhu closed the pull request at: https://github.com/apache/eagle/pull/856 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle pull request #856: [MINOR] fix null pointer exception for sla
GitHub user wujinhu opened a pull request: https://github.com/apache/eagle/pull/856 [MINOR] fix null pointer exception for sla You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-849 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/856.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #856 commit 11a8e862151f4fd491bd2c916ca7a6a1ad8cb324 Author: wujinhu Date: 2017-03-07T05:20:16Z fix null pointer exception for sla --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle pull request #854: [EAGLE-935] add jdbc storage support for sla job me...
GitHub user wujinhu opened a pull request: https://github.com/apache/eagle/pull/854 [EAGLE-935] add jdbc storage support for sla job meta You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-935 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/854.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #854 commit 82a0db78e9689ca99e0f1e8ec276f2fbd4615a30 Author: wujinhu Date: 2017-03-03T02:34:57Z add jdbc storage support for sla job meta --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle issue #854: [EAGLE-935] add jdbc storage support for sla job meta
Github user wujinhu commented on the issue: https://github.com/apache/eagle/pull/854 Continue to update --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle issue #850: EAGLE-941 reset timezone of time parse
Github user wujinhu commented on the issue: https://github.com/apache/eagle/pull/850 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (EAGLE-935) send alerts if finished job duration exceeds average duration
wujinhu created EAGLE-935: - Summary: send alerts if finished job duration exceeds average duration Key: EAGLE-935 URL: https://issues.apache.org/jira/browse/EAGLE-935 Project: Eagle Issue Type: New Feature Reporter: wujinhu Assignee: wujinhu -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] eagle pull request #846: [MINOR] add string:listSize siddhi extension
GitHub user wujinhu opened a pull request: https://github.com/apache/eagle/pull/846 [MINOR] add string:listSize siddhi extension You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-848 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/846.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #846 commit ba772317c33a9e672d493217eca3ac9028974d5f Author: wujinhu Date: 2017-02-28T06:27:07Z add string:listSize siddhi extension --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle pull request #844: [MINOR] support sequence in policy definition
GitHub user wujinhu opened a pull request: https://github.com/apache/eagle/pull/844 [MINOR] support sequence in policy definition You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-847 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/844.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #844 commit 95cc99b76941916a21930ba26d8f12639175054d Author: wujinhu Date: 2017-02-27T09:38:49Z support sequence in policy definition --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle issue #841: EAGLE-926: Alert engine fails to support more than one str...
Github user wujinhu commented on the issue: https://github.com/apache/eagle/pull/841 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle pull request #839: [EAGLE-927] support for hadoop jmx resource info
GitHub user wujinhu reopened a pull request: https://github.com/apache/eagle/pull/839 [EAGLE-927] support for hadoop jmx resource info You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-927 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/839.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #839 commit ddcd59616109caa7833af2d3e442cb9ad778ca28 Author: wujinhu Date: 2017-02-23T05:40:44Z support for hadoop jmx resource info --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle pull request #839: [EAGLE-927] support for hadoop jmx resource info
Github user wujinhu closed the pull request at: https://github.com/apache/eagle/pull/839 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle pull request #839: [EAGLE-927] support for hadoop jmx resource info
GitHub user wujinhu opened a pull request: https://github.com/apache/eagle/pull/839 [EAGLE-927] support for hadoop jmx resource info You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-927 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/839.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #839 commit ddcd59616109caa7833af2d3e442cb9ad778ca28 Author: wujinhu Date: 2017-02-23T05:40:44Z support for hadoop jmx resource info --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (EAGLE-927) add jmx resource info to Hadoop JMX Monitor
wujinhu created EAGLE-927: - Summary: add jmx resource info to Hadoop JMX Monitor Key: EAGLE-927 URL: https://issues.apache.org/jira/browse/EAGLE-927 Project: Eagle Issue Type: Improvement Components: App::Hadoop JMX Monitor Affects Versions: v0.5.0 Reporter: wujinhu Assignee: wujinhu Fix For: v0.5.0 We need add an stream(HADOOP_JMX_RESOURCE_STREAM) to stand for jmx resource like corrupt files and live nodes and so on. For example, when we found missing block number via HADOOP_JMX_METRIC_STREAM by define a policy, we can find missing block id via HADOOP_JMX_RESOURCE_STREAM -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (EAGLE-920) mr failed job trouble shooting
wujinhu created EAGLE-920:
-
Summary: mr failed job trouble shooting
Key: EAGLE-920
URL: https://issues.apache.org/jira/browse/EAGLE-920
Project: Eagle
Issue Type: Improvement
Components: App::Job Performance Monitor
Affects Versions: v0.5.0
Reporter: wujinhu
Assignee: wujinhu
Fix For: v0.5.0
We will follow below steps when we find a failed mr job.
1. get error category distribution of the job via api
query=TaskAttemptErrorCategoryService[@site="sandbox" and
@jobId="job_1486726244016_162594"]<@errorCategory>{count}
2. get error category - error message mapping and failed task attempts list
query=JobErrorMappingService[@site="sandbox" and
@jobId="job_1486726244016_162594" and
@errorCategory="java.lang.RuntimeException"]
3. dive into one task attempt
query=TaskAttemptExecutionService[@site="sandbox" and
@taskAttemptId="attempt_1486726244016_162594_m_002451_1"]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[GitHub] eagle pull request #832: [MINOR] fix job error mapping flush bug
GitHub user wujinhu opened a pull request: https://github.com/apache/eagle/pull/832 [MINOR] fix job error mapping flush bug You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-845 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/832.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #832 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle pull request #830: [MINOR] fix mr history null pointer bug
GitHub user wujinhu opened a pull request: https://github.com/apache/eagle/pull/830 [MINOR] fix mr history null pointer bug You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-845 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/830.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #830 commit 206d0fe56180718196a60066ac7e8c65e0ebc218 Author: wujinhu Date: 2017-02-21T09:01:07Z add Job error category and error message mapping commit b2122e656841f35fdc165acc8aeeff981ae7c4c8 Author: wujinhu Date: 2017-02-21T15:20:31Z fix mr history null pointer bug --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle pull request #829: [MINOR] add Job error category and error message ma...
GitHub user wujinhu opened a pull request: https://github.com/apache/eagle/pull/829 [MINOR] add Job error category and error message mapping You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-845 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/829.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #829 commit 206d0fe56180718196a60066ac7e8c65e0ebc218 Author: wujinhu Date: 2017-02-21T09:01:07Z add Job error category and error message mapping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle pull request #828: [EAGLE-917] add Job error category and error messag...
Github user wujinhu closed the pull request at: https://github.com/apache/eagle/pull/828 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle pull request #828: [EAGLE-917] add Job error category and error messag...
GitHub user wujinhu opened a pull request: https://github.com/apache/eagle/pull/828 [EAGLE-917] add Job error category and error message mapping You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-917 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/828.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #828 commit 84ef5b70ea3b5de307738c546f433ad8793ba5b4 Author: wujinhu Date: 2017-02-21T06:51:14Z add Job error category and error message mapping commit f9afafe7810d49e5338a9f8de76315c801f6fe24 Author: wujinhu Date: 2017-02-21T07:08:40Z add Job error category and error message mapping commit fbee685dc1394629f71a4bc785864a7d5b127332 Author: wujinhu Date: 2017-02-21T07:34:09Z add Job error category and error message mapping commit e68842c24c10e93229e64af918296e4825b420bd Author: wujinhu Date: 2017-02-21T08:57:07Z add Job error category and error message mapping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Closed] (EAGLE-840) Task Failing Nodes List and bad node detection
[ https://issues.apache.org/jira/browse/EAGLE-840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wujinhu closed EAGLE-840. - Resolution: Resolved > Task Failing Nodes List and bad node detection > -- > > Key: EAGLE-840 > URL: https://issues.apache.org/jira/browse/EAGLE-840 > Project: Eagle > Issue Type: Improvement > Components: App::Job Performance Monitor >Affects Versions: v0.5.0 >Reporter: wujinhu >Assignee: wujinhu > Fix For: v0.5.0 > > > add Task Failing Nodes List and bad node detection -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (EAGLE-917) add Job error category and error message mapping
[ https://issues.apache.org/jira/browse/EAGLE-917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wujinhu closed EAGLE-917. - Resolution: Resolved > add Job error category and error message mapping > > > Key: EAGLE-917 > URL: https://issues.apache.org/jira/browse/EAGLE-917 > Project: Eagle > Issue Type: Improvement > Reporter: wujinhu > Assignee: wujinhu > > In order to query error category and error message quickly, we need to add > this mapping to eagle storage although task attempt already has this > information -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] eagle pull request #827: [EAGLE-917] add Job error category and error messag...
GitHub user wujinhu opened a pull request: https://github.com/apache/eagle/pull/827 [EAGLE-917] add Job error category and error message mapping You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-917 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/827.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #827 commit 84ef5b70ea3b5de307738c546f433ad8793ba5b4 Author: wujinhu Date: 2017-02-21T06:51:14Z add Job error category and error message mapping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (EAGLE-917) add Job error category and error message mapping
wujinhu created EAGLE-917: - Summary: add Job error category and error message mapping Key: EAGLE-917 URL: https://issues.apache.org/jira/browse/EAGLE-917 Project: Eagle Issue Type: Improvement Reporter: wujinhu Assignee: wujinhu In order to query error category and error message quickly, we need to add this mapping to eagle storage although task attempt already has this information -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (EAGLE-904) Refactor un-meaningful health alert message
[
https://issues.apache.org/jira/browse/EAGLE-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wujinhu closed EAGLE-904.
-
Resolution: Resolved
> Refactor un-meaningful health alert message
> ---
>
> Key: EAGLE-904
> URL: https://issues.apache.org/jira/browse/EAGLE-904
> Project: Eagle
> Issue Type: Improvement
> Components: Core::App Engine
>Affects Versions: v0.5.0
> Environment: PRODUCTION
>Reporter: Hao Chen
>Assignee: wujinhu
> Fix For: v0.5.0
>
>
> {code}
> An exception was caught when fetch application current process time:
> java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 at
> java.util.LinkedList.checkElementIndex(LinkedList.java:555) at
> java.util.LinkedList.get(LinkedList.java:476) at
> org.apache.eagle.jpm.mr.history.MRHistoryJobApplicationHealthCheck.check(MRHistoryJobApplicationHealthCheck.java:75)
> at com.codahale.metrics.health.HealthCheck.execute(HealthCheck.java:172) at
> org.apache.eagle.app.service.impl.ApplicationHealthCheckServiceImpl.runOneIteration(ApplicationHealthCheckServiceImpl.java:180)
> at
> com.google.common.util.concurrent.AbstractScheduledService$1$1.run(AbstractScheduledService.java:172)
> at com.google.common.util.concurrent.Callables$3.run(Callables.java:93) at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at
> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Closed] (EAGLE-912) make application health check message more readable
[ https://issues.apache.org/jira/browse/EAGLE-912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wujinhu closed EAGLE-912. - Resolution: Resolved > make application health check message more readable > --- > > Key: EAGLE-912 > URL: https://issues.apache.org/jira/browse/EAGLE-912 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 > Reporter: wujinhu >Assignee: wujinhu > Fix For: v0.5.0 > > > If an application delayed a long time, health check scheduler will send > message like this: > An exception was caught when fetch application current process time: > java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 at > java.util.LinkedList.checkElementIndex(LinkedList.java:555) at > java.util.LinkedList.get(LinkedList.java:476) at > org.apache.eagle.jpm.mr.history.MRHistoryJobApplicationHealthCheck.check(MRHistoryJobApplicationHealthCheck.java:75) > at com.codahale.metrics.health.HealthCheck.execute(HealthCheck.java:172) at > org.apache.eagle.app.service.impl.ApplicationHealthCheckServiceImpl.runOneIteration(ApplicationHealthCheckServiceImpl.java:180) > at > com.google.common.util.concurrent.AbstractScheduledService$1$1.run(AbstractScheduledService.java:172) > at com.google.common.util.concurrent.Callables$3.run(Callables.java:93) at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > This is unreadable -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] eagle pull request #820: [EAGLE-912] make application health check message m...
Github user wujinhu closed the pull request at: https://github.com/apache/eagle/pull/820 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle pull request #820: [EAGLE-912] make application health check message m...
GitHub user wujinhu reopened a pull request: https://github.com/apache/eagle/pull/820 [EAGLE-912] make application health check message more readable You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-912 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/820.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #820 commit fd1ced58428b77ce0f382f13996a465fea338abd Author: wujinhu Date: 2017-02-20T04:17:57Z optimize health check message --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle pull request #820: [EAGLE-912] make application health check message m...
GitHub user wujinhu opened a pull request: https://github.com/apache/eagle/pull/820 [EAGLE-912] make application health check message more readable You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-912 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/820.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #820 commit fd1ced58428b77ce0f382f13996a465fea338abd Author: wujinhu Date: 2017-02-20T04:17:57Z optimize health check message --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (EAGLE-912) make application health check message more readable
wujinhu created EAGLE-912: - Summary: make application health check message more readable Key: EAGLE-912 URL: https://issues.apache.org/jira/browse/EAGLE-912 Project: Eagle Issue Type: Improvement Affects Versions: v0.5.0 Reporter: wujinhu Assignee: wujinhu Fix For: v0.5.0 If an application delayed a long time, health check scheduler will send message like this: An exception was caught when fetch application current process time: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 at java.util.LinkedList.checkElementIndex(LinkedList.java:555) at java.util.LinkedList.get(LinkedList.java:476) at org.apache.eagle.jpm.mr.history.MRHistoryJobApplicationHealthCheck.check(MRHistoryJobApplicationHealthCheck.java:75) at com.codahale.metrics.health.HealthCheck.execute(HealthCheck.java:172) at org.apache.eagle.app.service.impl.ApplicationHealthCheckServiceImpl.runOneIteration(ApplicationHealthCheckServiceImpl.java:180) at com.google.common.util.concurrent.AbstractScheduledService$1$1.run(AbstractScheduledService.java:172) at com.google.common.util.concurrent.Callables$3.run(Callables.java:93) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) This is unreadable -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] eagle pull request #809: [MINOR] remove useless code and fix fetch running j...
GitHub user wujinhu opened a pull request: https://github.com/apache/eagle/pull/809 [MINOR] remove useless code and fix fetch running job config time out You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-844 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/809.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #809 commit 8d23f8c247ba963aba206d87281ecc627a606b5a Author: wujinhu Date: 2017-02-16T07:00:51Z remove useless code and fix fetch running job config time out --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle pull request #807: [MINOR] add missing task error category
GitHub user wujinhu opened a pull request: https://github.com/apache/eagle/pull/807 [MINOR] add missing task error category You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-843 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/807.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #807 commit 8d5c53fa36836da1b7fe577eae03b773dabed1f9 Author: wujinhu Date: 2017-02-16T05:45:25Z add missing task error category --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle issue #796: [EAGLE-888] Application submitted to Storm is always shown...
Github user wujinhu commented on the issue: https://github.com/apache/eagle/pull/796 @DadanielZ I think you can add application-debug.conf to eagle-server/src/test/resources/ if you run Class ServerDebug. Pls have a try. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle issue #796: [EAGLE-888] Application submitted to Storm is always shown...
Github user wujinhu commented on the issue: https://github.com/apache/eagle/pull/796 IMO, "server config" has "siteId = testSite", "appId = HBaseAuditLog" may be wrong. It should not contains these. I mean wrong application.conf has been loaded probably. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle issue #796: [EAGLE-888] Application submitted to Storm is always shown...
Github user wujinhu commented on the issue: https://github.com/apache/eagle/pull/796 @DadanielZ There are a lot of application.conf files in eagle. I think this is the root cause. Besides, I think server config has higher priority than generated config(context) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle issue #796: [EAGLE-888] Application submitted to Storm is always shown...
Github user wujinhu commented on the issue: https://github.com/apache/eagle/pull/796 @DadanielZ Do you find this problem in your dev env? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] eagle pull request #768: [MINOR] optimize mr running job list api
GitHub user wujinhu opened a pull request: https://github.com/apache/eagle/pull/768 [MINOR] optimize mr running job list api You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-842 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/768.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #768 commit f3d1917cfeb74cecf4b67cfaa6ab0f1d0e5ba97e Author: wujinhu Date: 2017-01-09T13:04:44Z optimize mr running job list api --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (EAGLE-850) fetch node usage from name node/resource manager jmx
wujinhu created EAGLE-850:
-
Summary: fetch node usage from name node/resource manager jmx
Key: EAGLE-850
URL: https://issues.apache.org/jira/browse/EAGLE-850
Project: Eagle
Issue Type: Improvement
Components: Hadoop JMX Monitor
Affects Versions: v0.5.0
Reporter: wujinhu
Assignee: Michael Wu
Fix For: v0.5.0
fetch NodeUsage from jmx. The data in jmx looks like
NodeUsage:
"{"nodeUsage":{"min":"0.30%","median":"87.04%","max":"98.92%","stdDev":"31.67%"}}"
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] eagle pull request #762: [EAGLE-797] add job analysis
GitHub user wujinhu opened a pull request: https://github.com/apache/eagle/pull/762 [EAGLE-797] add job analysis for continuous development You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/eagle EAGLE-797 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/eagle/pull/762.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #762 commit 2f6faca2c2fd0a5017c7e2386ac94debf9deda64 Author: wujinhu Date: 2016-12-30T03:23:53Z add jpm analysis commit 568be20b6258ac5d58e71b11ae420fa1399fd8c0 Author: wujinhu Date: 2016-12-30T03:29:23Z Merge branch 'master' of https://github.com/apache/eagle into EAGLE-797 commit e314e30d8eed8d1dc9426e7743b23d843a288629 Author: wujinhu Date: 2017-01-03T02:27:43Z Merge branch 'master' of https://github.com/apache/eagle into EAGLE-797 commit 9965135672489ade3ad8e9e504a98ee4106d7823 Author: wujinhu Date: 2017-01-03T08:58:25Z add jpm analysis --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
