[jira] [Commented] (SLIDER-584) CommandEnvironmentIT failing on HA cluster - assertion failures
[ https://issues.apache.org/jira/browse/SLIDER-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14191689#comment-14191689 ] Steve Loughran commented on SLIDER-584: --- possible race condition of some sort, at least that is what it looks like. See if this fixes it before I can confirm CommandEnvironmentIT failing on HA cluster - assertion failures --- Key: SLIDER-584 URL: https://issues.apache.org/jira/browse/SLIDER-584 Project: Slider Issue Type: Sub-task Components: test Affects Versions: Slider 0.60 Environment: HA yarn cluster Reporter: Steve Loughran Assignee: Steve Loughran Fix For: Slider 0.60 The CommandEnvironmentIT test is failing, assertions that slider.confdir and slider libdir are in stdout are failing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-570) funtests to grab last (failing) app report on a launch failuret
[ https://issues.apache.org/jira/browse/SLIDER-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14191846#comment-14191846 ] ASF subversion and git services commented on SLIDER-570: Commit 1f98074f16e7840d11727d9bcf1efb5a7858595e in incubator-slider's branch refs/heads/develop from [~ste...@apache.org] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=1f98074 ] SLIDER-570: AgentClusterLifecycleIT - lookup/appId driven on thaw operations funtests to grab last (failing) app report on a launch failuret --- Key: SLIDER-570 URL: https://issues.apache.org/jira/browse/SLIDER-570 Project: Slider Issue Type: Sub-task Components: test Affects Versions: Slider 0.60 Reporter: Steve Loughran Assignee: Steve Loughran Fix For: Slider 0.60 when an AM doesn't start in a functional test the log message is simply that the app didn't come up in the time limit. We can do better: on a launch failure # enum the app records for that app instance name # grab the latest one # print it to the junit log # include it in the failure exception thrown This will make launch failure causes more obvious, and enable more meaningful bug reports to be filed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-570) funtests to grab last (failing) app report on a launch failuret
[ https://issues.apache.org/jira/browse/SLIDER-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14191845#comment-14191845 ] ASF subversion and git services commented on SLIDER-570: Commit e1023eadf896866ce6c63f06e47bf835adc8576c in incubator-slider's branch refs/heads/develop from [~ste...@apache.org] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=e1023ea ] SLIDER-570: tuning of tests, agent failure tests working (i.e. regression fixed) funtests to grab last (failing) app report on a launch failuret --- Key: SLIDER-570 URL: https://issues.apache.org/jira/browse/SLIDER-570 Project: Slider Issue Type: Sub-task Components: test Affects Versions: Slider 0.60 Reporter: Steve Loughran Assignee: Steve Loughran Fix For: Slider 0.60 when an AM doesn't start in a functional test the log message is simply that the app didn't come up in the time limit. We can do better: on a launch failure # enum the app records for that app instance name # grab the latest one # print it to the junit log # include it in the failure exception thrown This will make launch failure causes more obvious, and enable more meaningful bug reports to be filed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-570) funtests to grab last (failing) app report on a launch failuret
[ https://issues.apache.org/jira/browse/SLIDER-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14191843#comment-14191843 ] ASF subversion and git services commented on SLIDER-570: Commit e7df654f29e414b822a93738321940dc8f164114 in incubator-slider's branch refs/heads/develop from [~ste...@apache.org] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=e7df654 ] SLIDER-570 migrate tests to app-id based startup cycle funtests to grab last (failing) app report on a launch failuret --- Key: SLIDER-570 URL: https://issues.apache.org/jira/browse/SLIDER-570 Project: Slider Issue Type: Sub-task Components: test Affects Versions: Slider 0.60 Reporter: Steve Loughran Assignee: Steve Loughran Fix For: Slider 0.60 when an AM doesn't start in a functional test the log message is simply that the app didn't come up in the time limit. We can do better: on a launch failure # enum the app records for that app instance name # grab the latest one # print it to the junit log # include it in the failure exception thrown This will make launch failure causes more obvious, and enable more meaningful bug reports to be filed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLIDER-570) funtests to grab last (failing) app report on a launch failuret
[ https://issues.apache.org/jira/browse/SLIDER-570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved SLIDER-570. --- Resolution: Fixed funtests to grab last (failing) app report on a launch failuret --- Key: SLIDER-570 URL: https://issues.apache.org/jira/browse/SLIDER-570 Project: Slider Issue Type: Sub-task Components: test Affects Versions: Slider 0.60 Reporter: Steve Loughran Assignee: Steve Loughran Fix For: Slider 0.60 when an AM doesn't start in a functional test the log message is simply that the app didn't come up in the time limit. We can do better: on a launch failure # enum the app records for that app instance name # grab the latest one # print it to the junit log # include it in the failure exception thrown This will make launch failure causes more obvious, and enable more meaningful bug reports to be filed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-579) Add slider lookup operations --out args to get the app reports of launched apps
[
https://issues.apache.org/jira/browse/SLIDER-579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated SLIDER-579:
--
Summary: Add slider lookup operations --out args to get the app reports
of launched apps (was: Add CLI operations/--out args to get the app reports of
launched apps)
Add slider lookup operations --out args to get the app reports of
launched apps
---
Key: SLIDER-579
URL: https://issues.apache.org/jira/browse/SLIDER-579
Project: Slider
Issue Type: Sub-task
Components: client, test
Reporter: Steve Loughran
Assignee: Steve Loughran
Fix For: Slider 0.60
add a way for functional tests to get the app Id and app report of a launched
app -reliably.
Currently the exists/list operations are approximate as they are looking for
an instance rather than the specific one launched.
Proposed
# add a JSON-serialized form of an app report
# add a {{--out}} operation on the create/start operations to save this to a
file (Before entering any --wait action)
# add a {{lookup --id appid --out file}} action to look up any YARN app
and generate the report.
Test runs can then
* get the app id of the launched app
* retrieve parse the app report
* use that in failure messages
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (SLIDER-591) regression: ListCommandIT broken with changed exit codes
[
https://issues.apache.org/jira/browse/SLIDER-591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved SLIDER-591.
---
Resolution: Fixed
regression: ListCommandIT broken with changed exit codes
Key: SLIDER-591
URL: https://issues.apache.org/jira/browse/SLIDER-591
Project: Slider
Issue Type: Sub-task
Components: build
Affects Versions: Slider 0.60
Reporter: Steve Loughran
Assignee: Steve Loughran
Fix For: Slider 0.60
SLIDER-306 changed the exit codes of the list command; as a result the
{{ListCommandIT}} test is failing
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (SLIDER-579) Add slider lookup operations --out args to get the app reports of launched apps
[
https://issues.apache.org/jira/browse/SLIDER-579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved SLIDER-579.
---
Resolution: Fixed
Add slider lookup operations --out args to get the app reports of
launched apps
---
Key: SLIDER-579
URL: https://issues.apache.org/jira/browse/SLIDER-579
Project: Slider
Issue Type: Sub-task
Components: client, test
Reporter: Steve Loughran
Assignee: Steve Loughran
Fix For: Slider 0.60
add a way for functional tests to get the app Id and app report of a launched
app -reliably.
Currently the exists/list operations are approximate as they are looking for
an instance rather than the specific one launched.
Proposed
# add a JSON-serialized form of an app report
# add a {{--out}} operation on the create/start operations to save this to a
file (Before entering any --wait action)
# add a {{lookup --id appid --out file}} action to look up any YARN app
and generate the report.
Test runs can then
* get the app id of the launched app
* retrieve parse the app report
* use that in failure messages
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-588) AppsThroughAgentIT test failed with 'Application registry is not accessible'
[
https://issues.apache.org/jira/browse/SLIDER-588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192080#comment-14192080
]
ASF subversion and git services commented on SLIDER-588:
Commit a9e81c0b6ba05bc45d5c135c503238c12ea4e715 in incubator-slider's branch
refs/heads/develop from [~ste...@apache.org]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=a9e81c0 ]
SLIDER-588 AppsThroughAgentIT test failed with 'Application registry is not
accessible' ... more diagnostics now
AppsThroughAgentIT test failed with 'Application registry is not accessible'
Key: SLIDER-588
URL: https://issues.apache.org/jira/browse/SLIDER-588
Project: Slider
Issue Type: Bug
Affects Versions: Slider 0.60
Reporter: Yesha Vora
Assignee: Steve Loughran
Fix For: Slider 0.60
AppsThroughAgentIT.testCreateFlex failed with 'Application registry is not
accessible' error
{code}
AppsThroughAgentIT.testCreateFlex:81-CommandTestBase.ensureRegistryCallSucceeds:690-CommandTestBase.repeatUntilTrue:783-Assert.fail:88
Application registry is not accessible, failing test.
{code}
{code:title=testCreateFlex failure}
testcase name=testCreateFlex
classname=org.apache.slider.funtest.lifecycle.AppsThroughAgentIT
time=387.795
failure message=Application registry is not accessible, failing test.
type=java.lang.AssertionError
java.lang.AssertionError: Application registry is not accessible, failing
test. at org.junit.Assert.fail(Assert.java:88) at
org.apache.slider.funtest.framework.CommandTestBase.repeatUntilTrue(CommandTestBase.groovy:783)
at
org.apache.slider.funtest.framework.CommandTestBase.ensureRegistryCallSucceeds(CommandTestBase.groovy:690)
at
org.apache.slider.funtest.lifecycle.AppsThroughAgentIT.testCreateFlex(AppsThroughAgentIT.groovy:81)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606) at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
/failure
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (SLIDER-588) AppsThroughAgentIT test failed with 'Application registry is not accessible'
[
https://issues.apache.org/jira/browse/SLIDER-588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran reassigned SLIDER-588:
-
Assignee: Sumit Mohanty (was: Steve Loughran)
AppsThroughAgentIT test failed with 'Application registry is not accessible'
Key: SLIDER-588
URL: https://issues.apache.org/jira/browse/SLIDER-588
Project: Slider
Issue Type: Bug
Affects Versions: Slider 0.60
Reporter: Yesha Vora
Assignee: Sumit Mohanty
Fix For: Slider 0.60
AppsThroughAgentIT.testCreateFlex failed with 'Application registry is not
accessible' error
{code}
AppsThroughAgentIT.testCreateFlex:81-CommandTestBase.ensureRegistryCallSucceeds:690-CommandTestBase.repeatUntilTrue:783-Assert.fail:88
Application registry is not accessible, failing test.
{code}
{code:title=testCreateFlex failure}
testcase name=testCreateFlex
classname=org.apache.slider.funtest.lifecycle.AppsThroughAgentIT
time=387.795
failure message=Application registry is not accessible, failing test.
type=java.lang.AssertionError
java.lang.AssertionError: Application registry is not accessible, failing
test. at org.junit.Assert.fail(Assert.java:88) at
org.apache.slider.funtest.framework.CommandTestBase.repeatUntilTrue(CommandTestBase.groovy:783)
at
org.apache.slider.funtest.framework.CommandTestBase.ensureRegistryCallSucceeds(CommandTestBase.groovy:690)
at
org.apache.slider.funtest.lifecycle.AppsThroughAgentIT.testCreateFlex(AppsThroughAgentIT.groovy:81)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606) at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
/failure
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-577) Need a warning message when no service keytabs are localized
[ https://issues.apache.org/jira/browse/SLIDER-577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SLIDER-577: -- Fix Version/s: Slider 0.60 Need a warning message when no service keytabs are localized Key: SLIDER-577 URL: https://issues.apache.org/jira/browse/SLIDER-577 Project: Slider Issue Type: Bug Affects Versions: Slider 0.60 Reporter: Jonathan Maron Assignee: Jonathan Maron Fix For: Slider 0.60 there should be a warning message in the AM log indicating that no service keytabs have been localized If leveraging HDFS installed keytabs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (SLIDER-468) Stopped application status should say Stopped
[
https://issues.apache.org/jira/browse/SLIDER-468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran reassigned SLIDER-468:
-
Assignee: Steve Loughran
Stopped application status should say Stopped
-
Key: SLIDER-468
URL: https://issues.apache.org/jira/browse/SLIDER-468
Project: Slider
Issue Type: Bug
Components: client
Affects Versions: Slider 0.50
Reporter: Sumit Mohanty
Assignee: Steve Loughran
Labels: feedback
Fix For: Slider 0.60
For a stopped application.
{noformat}
slider exists hbase1
...
...
2014-10-01 00:58:06,785 [main] INFO impl.TimelineClientImpl - Timeline
service address: http://c6401.ambari.apache.org:8188/ws/v1/timeline/
2014-10-01 00:58:06,967 [main] INFO client.RMProxy - Connecting to
ResourceManager at c6401.ambari.apache.org/192.168.64.101:8050
2014-10-01 00:58:07,723 [main] WARN shortcircuit.DomainSocketFactory - The
short-circuit local reads feature cannot be used because libhadoop cannot be
loaded.
2014-10-01 00:58:07,846 [main] INFO client.SliderClient - Cluster hbase1
exists
2014-10-01 00:58:07,847 [main] INFO util.ExitUtil - Exiting with status 0
{noformat}
{noformat}
slider status hbase1
...
2014-10-01 00:57:56,039 [main] INFO impl.TimelineClientImpl - Timeline
service address: http://c6401.ambari.apache.org:8188/ws/v1/timeline/
2014-10-01 00:57:56,271 [main] INFO client.RMProxy - Connecting to
ResourceManager at c6401.ambari.apache.org/192.168.64.101:8050
2014-10-01 00:57:56,924 [main] WARN shortcircuit.DomainSocketFactory - The
short-circuit local reads feature cannot be used because libhadoop cannot be
loaded.
2014-10-01 00:57:57,240 [main] ERROR main.ServiceLauncher - Application not
running: application_1412109423959_0006 state=FINISHED
2014-10-01 00:57:57,243 [main] INFO util.ExitUtil - Exiting with status 71
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (SLIDER-592) Slider command indicate not enough resource when that happens during application creation
thomas liu created SLIDER-592: - Summary: Slider command indicate not enough resource when that happens during application creation Key: SLIDER-592 URL: https://issues.apache.org/jira/browse/SLIDER-592 Project: Slider Issue Type: Improvement Affects Versions: Slider 2.0.0 Reporter: thomas liu Fix For: Slider 2.0.0 When submitting an application to a YARN cluster and if there is not enough resource requested by the application, the application stuck in the state of ACCEPTED. Maybe we should put it in a state explicitly indicate lack of resource; or maybe we should return a different value from create to indicate this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SLIDER-593) registry/REST interactions fail with 302 Found errors if HTTPS is enabled for RM
Jonathan Maron created SLIDER-593: - Summary: registry/REST interactions fail with 302 Found errors if HTTPS is enabled for RM Key: SLIDER-593 URL: https://issues.apache.org/jira/browse/SLIDER-593 Project: Slider Issue Type: Bug Reporter: Jonathan Maron The java HTTP support does not redirect between protocols (HTTP and HTTPS). So in situations where HTTPS is enabled for the RM (yarn.http.policy), the redirections from AM to RM fail with a 302 Found error. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-593) registry/REST interactions fail with 302 Found errors if HTTPS is enabled for RM
[ https://issues.apache.org/jira/browse/SLIDER-593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192178#comment-14192178 ] Jonathan Maron commented on SLIDER-593: --- It looks like we can enable cross protocol redirects, but a redirect to an HTTPS host is going to require trust store configuration of the slider client (import of the RM certificate into the client side trust store). registry/REST interactions fail with 302 Found errors if HTTPS is enabled for RM -- Key: SLIDER-593 URL: https://issues.apache.org/jira/browse/SLIDER-593 Project: Slider Issue Type: Bug Reporter: Jonathan Maron The java HTTP support does not redirect between protocols (HTTP and HTTPS). So in situations where HTTPS is enabled for the RM (yarn.http.policy), the redirections from AM to RM fail with a 302 Found error. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SLIDER-594) Add a sleep before container restart as ports may not be released from the last activation
Sumit Mohanty created SLIDER-594: Summary: Add a sleep before container restart as ports may not be released from the last activation Key: SLIDER-594 URL: https://issues.apache.org/jira/browse/SLIDER-594 Project: Slider Issue Type: Bug Components: agent-provider Affects Versions: Slider 0.50 Reporter: Sumit Mohanty Assignee: Sumit Mohanty Priority: Critical Fix For: Slider 0.60 This is critical for applications that do not use dynamic port and applications using labels do not use dynamic ports. A configurable delay should be added to allow for scenarios where component instances get killed rather than stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SLIDER-595) Default app paster container size should be 1024m
Sumit Mohanty created SLIDER-595:
Summary: Default app paster container size should be 1024m
Key: SLIDER-595
URL: https://issues.apache.org/jira/browse/SLIDER-595
Project: Slider
Issue Type: Bug
Components: appmaster
Affects Versions: Slider 0.50
Reporter: Sumit Mohanty
Assignee: Steve Loughran
Fix For: Slider 0.60
This is from a HBase application run.
{noformat}
2014-10-29 16:10:11,185 INFO monitor.ContainersMonitorImpl
(ContainersMonitorImpl.java:run(408)) - Memory usage of ProcessTree 29820 for
container-id container_1414597212088_0002_01_01: 78.8 MB of 256 MB physical
memory used; 1.1 GB of 537.6 MB virtual memory used
2014-10-29 16:10:14,213 INFO monitor.ContainersMonitorImpl
(ContainersMonitorImpl.java:run(408)) - Memory usage of ProcessTree 29820 for
container-id container_1414597212088_0002_01_01: 138.7 MB of 256 MB
physical memory used; 1.1 GB of 537.6 MB virtual memory used
2014-10-29 16:10:17,233 INFO monitor.ContainersMonitorImpl
(ContainersMonitorImpl.java:run(408)) - Memory usage of ProcessTree 29820 for
container-id container_1414597212088_0002_01_01: 182.3 MB of 256 MB
physical memory used; 1.2 GB of 537.6 MB virtual memory used
2014-10-29 16:10:20,276 INFO monitor.ContainersMonitorImpl
(ContainersMonitorImpl.java:run(408)) - Memory usage of ProcessTree 29820 for
container-id container_1414597212088_0002_01_01: 209.4 MB of 256 MB
physical memory used; 1.2 GB of 537.6 MB virtual memory used
2014-10-29 16:10:23,294 INFO monitor.ContainersMonitorImpl
(ContainersMonitorImpl.java:run(408)) - Memory usage of ProcessTree 29820 for
container-id container_1414597212088_0002_01_01: 212.8 MB of 256 MB
physical memory used; 1.2 GB of 537.6 MB virtual memory used
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-593) registry/REST interactions fail with 302 Found errors if HTTPS is enabled for RM
[ https://issues.apache.org/jira/browse/SLIDER-593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-593: - Sprint: Slider October #2 registry/REST interactions fail with 302 Found errors if HTTPS is enabled for RM -- Key: SLIDER-593 URL: https://issues.apache.org/jira/browse/SLIDER-593 Project: Slider Issue Type: Bug Reporter: Jonathan Maron Fix For: Slider 0.60 The java HTTP support does not redirect between protocols (HTTP and HTTPS). So in situations where HTTPS is enabled for the RM (yarn.http.policy), the redirections from AM to RM fail with a 302 Found error. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-594) Add a sleep before container restart as ports may not be released from the last activation
[ https://issues.apache.org/jira/browse/SLIDER-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-594: - Sprint: Slider October #2 Add a sleep before container restart as ports may not be released from the last activation -- Key: SLIDER-594 URL: https://issues.apache.org/jira/browse/SLIDER-594 Project: Slider Issue Type: Bug Components: agent-provider Affects Versions: Slider 0.50 Reporter: Sumit Mohanty Assignee: Sumit Mohanty Priority: Critical Fix For: Slider 0.60 This is critical for applications that do not use dynamic port and applications using labels do not use dynamic ports. A configurable delay should be added to allow for scenarios where component instances get killed rather than stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (SLIDER-593) registry/REST interactions fail with 302 Found errors if HTTPS is enabled for RM
[ https://issues.apache.org/jira/browse/SLIDER-593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Maron reassigned SLIDER-593: - Assignee: Jonathan Maron registry/REST interactions fail with 302 Found errors if HTTPS is enabled for RM -- Key: SLIDER-593 URL: https://issues.apache.org/jira/browse/SLIDER-593 Project: Slider Issue Type: Bug Reporter: Jonathan Maron Assignee: Jonathan Maron Fix For: Slider 0.60 The java HTTP support does not redirect between protocols (HTTP and HTTPS). So in situations where HTTPS is enabled for the RM (yarn.http.policy), the redirections from AM to RM fail with a 302 Found error. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-596) Auto replace ${USER_NAME} token with the user that is running the AppMaster process
[
https://issues.apache.org/jira/browse/SLIDER-596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty updated SLIDER-596:
-
Sprint: Slider October #2
Auto replace ${USER_NAME} token with the user that is running the AppMaster
process
---
Key: SLIDER-596
URL: https://issues.apache.org/jira/browse/SLIDER-596
Project: Slider
Issue Type: Bug
Components: appmaster
Affects Versions: Slider 0.50
Reporter: Sumit Mohanty
Assignee: Jonathan Maron
Fix For: Slider 0.60
The application packages are using ${USER_NAME} as the token for user name.
AppMaster should query the running user and replace this token.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (SLIDER-596) Auto replace ${USER_NAME} token with the user that is running the AppMaster process
Sumit Mohanty created SLIDER-596:
Summary: Auto replace ${USER_NAME} token with the user that is
running the AppMaster process
Key: SLIDER-596
URL: https://issues.apache.org/jira/browse/SLIDER-596
Project: Slider
Issue Type: Bug
Components: appmaster
Affects Versions: Slider 0.50
Reporter: Sumit Mohanty
Assignee: Jonathan Maron
Fix For: Slider 0.60
The application packages are using ${USER_NAME} as the token for user name.
AppMaster should query the running user and replace this token.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-593) registry/REST interactions fail with 302 Found errors if HTTPS is enabled for RM
[ https://issues.apache.org/jira/browse/SLIDER-593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Maron updated SLIDER-593: -- Due Date: 2/Nov/15 registry/REST interactions fail with 302 Found errors if HTTPS is enabled for RM -- Key: SLIDER-593 URL: https://issues.apache.org/jira/browse/SLIDER-593 Project: Slider Issue Type: Bug Reporter: Jonathan Maron Assignee: Jonathan Maron Fix For: Slider 0.60 Attachments: SLIDER-593.001.patch The java HTTP support does not redirect between protocols (HTTP and HTTPS). So in situations where HTTPS is enabled for the RM (yarn.http.policy), the redirections from AM to RM fail with a 302 Found error. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-596) Auto replace ${USER_NAME} token with the user that is running the AppMaster process
[
https://issues.apache.org/jira/browse/SLIDER-596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Maron updated SLIDER-596:
--
Due Date: 3/Nov/14
Auto replace ${USER_NAME} token with the user that is running the AppMaster
process
---
Key: SLIDER-596
URL: https://issues.apache.org/jira/browse/SLIDER-596
Project: Slider
Issue Type: Bug
Components: appmaster
Affects Versions: Slider 0.50
Reporter: Sumit Mohanty
Assignee: Jonathan Maron
Fix For: Slider 0.60
The application packages are using ${USER_NAME} as the token for user name.
AppMaster should query the running user and replace this token.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (SLIDER-596) Auto replace ${USER_NAME} token with the user that is running the AppMaster process
[
https://issues.apache.org/jira/browse/SLIDER-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192338#comment-14192338
]
Jonathan Maron edited comment on SLIDER-596 at 10/31/14 7:46 PM:
-
That is correct. So:
1) Any sense which is more pervasive? Is Accumulo or any other app package
already making use of ${USER}?
2) Would it be difficult to steer the Slider view to leverage ${USER}
instead of ${USER_NAME}?
was (Author: jmaron):
That is correct. So:
1) Any sense which is more pervasive? Is Accumulo or any other app package
already making use of ${USER}?
2) Would it be difficult to steer the Slider view to leverage ${USER} instead
of ${USER_NAME}?
Auto replace ${USER_NAME} token with the user that is running the AppMaster
process
---
Key: SLIDER-596
URL: https://issues.apache.org/jira/browse/SLIDER-596
Project: Slider
Issue Type: Bug
Components: appmaster
Affects Versions: Slider 0.50
Reporter: Sumit Mohanty
Assignee: Jonathan Maron
Fix For: Slider 0.60
The application packages are using ${USER_NAME} as the token for user name.
AppMaster should query the running user and replace this token.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-574) document managing port ranges for slider containers
[ https://issues.apache.org/jira/browse/SLIDER-574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Maron updated SLIDER-574: -- Due Date: 9/Nov/14 document managing port ranges for slider containers --- Key: SLIDER-574 URL: https://issues.apache.org/jira/browse/SLIDER-574 Project: Slider Issue Type: Documentation Components: appmaster Affects Versions: Slider 0.60 Reporter: Jonathan Maron Assignee: Jonathan Maron Fix For: Slider 0.60 Need to document these settings in an existing document or new configuration document -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-596) Auto replace ${USER_NAME} token with the user that is running the AppMaster process
[
https://issues.apache.org/jira/browse/SLIDER-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192378#comment-14192378
]
Billie Rinaldi commented on SLIDER-596:
---
Using this handy command I can see that $USER appears in several hbase-env.xml,
hadoop-env.sh, and mapred-env.sh files. These are not slider or accumulo
configs, so I don't know anything about these usages. The only other usages
are where the substitution takes place, and in accumulo, hbase, and storm
appConfig files. hbase and storm appConfigs are both using USER_NAME.
Accumulo's default appConfig uses USER, and its secure appConfig was forced to
use both USER and USER_NAME, to match the secure appConfigs of the other apps.
{noformat}
find . -type f -exec egrep -HE [$][{]?USER {} \;
{noformat}
I don't care which we choose, and it shouldn't be difficult to switch entirely
to either one (although presumably the env files should remain $USER
regardless). I'm just glad we'll be settling on one of them, instead of
continuing to use both.
Auto replace ${USER_NAME} token with the user that is running the AppMaster
process
---
Key: SLIDER-596
URL: https://issues.apache.org/jira/browse/SLIDER-596
Project: Slider
Issue Type: Bug
Components: appmaster
Affects Versions: Slider 0.50
Reporter: Sumit Mohanty
Assignee: Jonathan Maron
Fix For: Slider 0.60
The application packages are using ${USER_NAME} as the token for user name.
AppMaster should query the running user and replace this token.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-596) Auto replace ${USER_NAME} token with the user that is running the AppMaster process
[
https://issues.apache.org/jira/browse/SLIDER-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192427#comment-14192427
]
Jonathan Maron commented on SLIDER-596:
---
Not necessarily in the near term... ;)
I think we're going to accept both in the short term. I'm going to file a JIRA
for the Slider 2.0 timeframe to reconcile them to one.
Auto replace ${USER_NAME} token with the user that is running the AppMaster
process
---
Key: SLIDER-596
URL: https://issues.apache.org/jira/browse/SLIDER-596
Project: Slider
Issue Type: Bug
Components: appmaster
Affects Versions: Slider 0.50
Reporter: Sumit Mohanty
Assignee: Jonathan Maron
Fix For: Slider 0.60
The application packages are using ${USER_NAME} as the token for user name.
AppMaster should query the running user and replace this token.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-596) Auto replace ${USER_NAME} token with the user that is running the AppMaster process
[
https://issues.apache.org/jira/browse/SLIDER-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192458#comment-14192458
]
Billie Rinaldi commented on SLIDER-596:
---
Accepting both sounds fine to me. I guess my real problem with the current
situation was having one be auto-substituted and the other not. If they are
both treated the same, I don't care if they continue to coexist ad infinitum.
Auto replace ${USER_NAME} token with the user that is running the AppMaster
process
---
Key: SLIDER-596
URL: https://issues.apache.org/jira/browse/SLIDER-596
Project: Slider
Issue Type: Bug
Components: appmaster
Affects Versions: Slider 0.50
Reporter: Sumit Mohanty
Assignee: Jonathan Maron
Fix For: Slider 0.60
The application packages are using ${USER_NAME} as the token for user name.
AppMaster should query the running user and replace this token.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-587) Query: how well are dynamic role placement histories maintained?
[ https://issues.apache.org/jira/browse/SLIDER-587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192514#comment-14192514 ] ASF subversion and git services commented on SLIDER-587: Commit 34f78ada7282a781dc6e440d9f4e4fd38d455f8d in incubator-slider's branch refs/heads/develop from [~ste...@apache.org] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=34f78ad ] SLIDER-587 test for role history on dynamic roles Query: how well are dynamic role placement histories maintained? Key: SLIDER-587 URL: https://issues.apache.org/jira/browse/SLIDER-587 Project: Slider Issue Type: Task Components: appmaster Reporter: Steve Loughran Assignee: Steve Loughran Fix For: Slider 0.60 Review the code to make sure that dynamic roles get their placement history maintained. Maybe even add some tests to validate this -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-587) Query: how well are dynamic role placement histories maintained?
[ https://issues.apache.org/jira/browse/SLIDER-587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192515#comment-14192515 ] ASF subversion and git services commented on SLIDER-587: Commit 0b06d631598097ea4d867252a2864d3ce374b768 in incubator-slider's branch refs/heads/develop from [~ste...@apache.org] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=0b06d63 ] SLIDER-587 test for dynamic role replacement passing Query: how well are dynamic role placement histories maintained? Key: SLIDER-587 URL: https://issues.apache.org/jira/browse/SLIDER-587 Project: Slider Issue Type: Task Components: appmaster Reporter: Steve Loughran Assignee: Steve Loughran Fix For: Slider 0.60 Review the code to make sure that dynamic roles get their placement history maintained. Maybe even add some tests to validate this -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-587) Query: how well are dynamic role placement histories maintained?
[ https://issues.apache.org/jira/browse/SLIDER-587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192513#comment-14192513 ] ASF subversion and git services commented on SLIDER-587: Commit 22b4b5e72aa458f4e7b5cf4741ee3304057d8ed2 in incubator-slider's branch refs/heads/develop from [~ste...@apache.org] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=22b4b5e ] SLIDER-587: building test for dynamic role placement history Query: how well are dynamic role placement histories maintained? Key: SLIDER-587 URL: https://issues.apache.org/jira/browse/SLIDER-587 Project: Slider Issue Type: Task Components: appmaster Reporter: Steve Loughran Assignee: Steve Loughran Fix For: Slider 0.60 Review the code to make sure that dynamic roles get their placement history maintained. Maybe even add some tests to validate this -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-587) Query: how well are dynamic role placement histories maintained?
[ https://issues.apache.org/jira/browse/SLIDER-587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192516#comment-14192516 ] ASF subversion and git services commented on SLIDER-587: Commit f83ce7571aeafcf782d6462ef779479c1f08924c in incubator-slider's branch refs/heads/develop from [~ste...@apache.org] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=f83ce75 ] Merge branch 'feature/SLIDER-587-dynamic_role_placement' into develop Query: how well are dynamic role placement histories maintained? Key: SLIDER-587 URL: https://issues.apache.org/jira/browse/SLIDER-587 Project: Slider Issue Type: Task Components: appmaster Reporter: Steve Loughran Assignee: Steve Loughran Fix For: Slider 0.60 Review the code to make sure that dynamic roles get their placement history maintained. Maybe even add some tests to validate this -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-588) AppsThroughAgentIT test failed with 'Application registry is not accessible'
[
https://issues.apache.org/jira/browse/SLIDER-588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty updated SLIDER-588:
-
Assignee: Gour Saha (was: Sumit Mohanty)
AppsThroughAgentIT test failed with 'Application registry is not accessible'
Key: SLIDER-588
URL: https://issues.apache.org/jira/browse/SLIDER-588
Project: Slider
Issue Type: Bug
Affects Versions: Slider 0.60
Reporter: Yesha Vora
Assignee: Gour Saha
Fix For: Slider 0.60
AppsThroughAgentIT.testCreateFlex failed with 'Application registry is not
accessible' error
{code}
AppsThroughAgentIT.testCreateFlex:81-CommandTestBase.ensureRegistryCallSucceeds:690-CommandTestBase.repeatUntilTrue:783-Assert.fail:88
Application registry is not accessible, failing test.
{code}
{code:title=testCreateFlex failure}
testcase name=testCreateFlex
classname=org.apache.slider.funtest.lifecycle.AppsThroughAgentIT
time=387.795
failure message=Application registry is not accessible, failing test.
type=java.lang.AssertionError
java.lang.AssertionError: Application registry is not accessible, failing
test. at org.junit.Assert.fail(Assert.java:88) at
org.apache.slider.funtest.framework.CommandTestBase.repeatUntilTrue(CommandTestBase.groovy:783)
at
org.apache.slider.funtest.framework.CommandTestBase.ensureRegistryCallSucceeds(CommandTestBase.groovy:690)
at
org.apache.slider.funtest.lifecycle.AppsThroughAgentIT.testCreateFlex(AppsThroughAgentIT.groovy:81)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606) at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
/failure
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (SLIDER-526) Create a python wrapper and replace the shell script for slider
[ https://issues.apache.org/jira/browse/SLIDER-526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty resolved SLIDER-526. -- Resolution: Fixed Create a python wrapper and replace the shell script for slider --- Key: SLIDER-526 URL: https://issues.apache.org/jira/browse/SLIDER-526 Project: Slider Issue Type: Bug Components: client Affects Versions: Slider 0.50 Reporter: Sumit Mohanty Assignee: thomas liu Fix For: Slider 0.60 Attachments: slider526.patch, slider526.patch, slider526.patch, slider526.patch, slider526.patch The goal is to have a single python script that can work on Windows and Linux and also be able to call commands directly e..g slider list without having to say python slider list. Of course python slider list should also work. Lets try to make name change on a linux environment and see what issues, if any, we run into. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLIDER-587) Query: how well are dynamic role placement histories maintained?
[ https://issues.apache.org/jira/browse/SLIDER-587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved SLIDER-587. --- Resolution: Fixed answer: not perfectly. furthermore, For all containers, short-lived failures weren't marking the nodes as free Query: how well are dynamic role placement histories maintained? Key: SLIDER-587 URL: https://issues.apache.org/jira/browse/SLIDER-587 Project: Slider Issue Type: Task Components: appmaster Reporter: Steve Loughran Assignee: Steve Loughran Fix For: Slider 0.60 Review the code to make sure that dynamic roles get their placement history maintained. Maybe even add some tests to validate this -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-570) funtests to grab last (failing) app report on a launch failuret
[ https://issues.apache.org/jira/browse/SLIDER-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192539#comment-14192539 ] ASF subversion and git services commented on SLIDER-570: Commit 1635866 from [~ste...@apache.org] in branch 'site/trunk' [ https://svn.apache.org/r1635866 ] SLIDER-570 chaos monkey coverage funtests to grab last (failing) app report on a launch failuret --- Key: SLIDER-570 URL: https://issues.apache.org/jira/browse/SLIDER-570 Project: Slider Issue Type: Sub-task Components: test Affects Versions: Slider 0.60 Reporter: Steve Loughran Assignee: Steve Loughran Fix For: Slider 0.60 when an AM doesn't start in a functional test the log message is simply that the app didn't come up in the time limit. We can do better: on a launch failure # enum the app records for that app instance name # grab the latest one # print it to the junit log # include it in the failure exception thrown This will make launch failure causes more obvious, and enable more meaningful bug reports to be filed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLIDER-596) Auto replace ${USER_NAME} token with the user that is running the AppMaster process
[
https://issues.apache.org/jira/browse/SLIDER-596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Maron resolved SLIDER-596.
---
Resolution: Fixed
Auto replace ${USER_NAME} token with the user that is running the AppMaster
process
---
Key: SLIDER-596
URL: https://issues.apache.org/jira/browse/SLIDER-596
Project: Slider
Issue Type: Bug
Components: appmaster
Affects Versions: Slider 0.50
Reporter: Sumit Mohanty
Assignee: Jonathan Maron
Fix For: Slider 0.60
The application packages are using ${USER_NAME} as the token for user name.
AppMaster should query the running user and replace this token.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-596) Auto replace ${USER_NAME} token with the user that is running the AppMaster process
[
https://issues.apache.org/jira/browse/SLIDER-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192722#comment-14192722
]
ASF subversion and git services commented on SLIDER-596:
Commit 9c2080baa788f7b476e33e53dc5f5cf50490d458 in incubator-slider's branch
refs/heads/develop from [~jmaron]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=9c2080b ]
SLIDER-596 add processing for USER_NAME token to slider client
Auto replace ${USER_NAME} token with the user that is running the AppMaster
process
---
Key: SLIDER-596
URL: https://issues.apache.org/jira/browse/SLIDER-596
Project: Slider
Issue Type: Bug
Components: appmaster
Affects Versions: Slider 0.50
Reporter: Sumit Mohanty
Assignee: Jonathan Maron
Fix For: Slider 0.60
The application packages are using ${USER_NAME} as the token for user name.
AppMaster should query the running user and replace this token.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Slider-develop - Build # 402 - Still Failing
The Apache Jenkins build system has built Slider-develop (build #402) Status: Still Failing Check console output at https://builds.apache.org/job/Slider-develop/402/ to view the results.
[jira] [Resolved] (SLIDER-559) Ensure Slider generated log files are readable by NM log aggregators
[ https://issues.apache.org/jira/browse/SLIDER-559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty resolved SLIDER-559. -- Resolution: Fixed Ensure Slider generated log files are readable by NM log aggregators Key: SLIDER-559 URL: https://issues.apache.org/jira/browse/SLIDER-559 Project: Slider Issue Type: Bug Components: agent Affects Versions: Slider 0.50 Reporter: Sumit Mohanty Assignee: Sumit Mohanty Fix For: Slider 0.60 Attachments: SLIDER-559.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SLIDER-597) Evaluate all Slider log files for PII data and strip them off as needed
Sumit Mohanty created SLIDER-597: Summary: Evaluate all Slider log files for PII data and strip them off as needed Key: SLIDER-597 URL: https://issues.apache.org/jira/browse/SLIDER-597 Project: Slider Issue Type: Bug Components: agent, app-package, appmaster Affects Versions: Slider 0.50 Reporter: Sumit Mohanty Fix For: Slider 2.0.0 All Slider log files should be evaluated for PII data. Candidate files are command*json, slider-agent.log, output-*.txt. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-595) Default app master container size should be 1024m
[
https://issues.apache.org/jira/browse/SLIDER-595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty updated SLIDER-595:
-
Summary: Default app master container size should be 1024m (was: Default
app paster container size should be 1024m)
Default app master container size should be 1024m
-
Key: SLIDER-595
URL: https://issues.apache.org/jira/browse/SLIDER-595
Project: Slider
Issue Type: Bug
Components: appmaster
Affects Versions: Slider 0.50
Reporter: Sumit Mohanty
Assignee: Steve Loughran
Fix For: Slider 0.60
This is from a HBase application run.
{noformat}
2014-10-29 16:10:11,185 INFO monitor.ContainersMonitorImpl
(ContainersMonitorImpl.java:run(408)) - Memory usage of ProcessTree 29820 for
container-id container_1414597212088_0002_01_01: 78.8 MB of 256 MB
physical memory used; 1.1 GB of 537.6 MB virtual memory used
2014-10-29 16:10:14,213 INFO monitor.ContainersMonitorImpl
(ContainersMonitorImpl.java:run(408)) - Memory usage of ProcessTree 29820 for
container-id container_1414597212088_0002_01_01: 138.7 MB of 256 MB
physical memory used; 1.1 GB of 537.6 MB virtual memory used
2014-10-29 16:10:17,233 INFO monitor.ContainersMonitorImpl
(ContainersMonitorImpl.java:run(408)) - Memory usage of ProcessTree 29820 for
container-id container_1414597212088_0002_01_01: 182.3 MB of 256 MB
physical memory used; 1.2 GB of 537.6 MB virtual memory used
2014-10-29 16:10:20,276 INFO monitor.ContainersMonitorImpl
(ContainersMonitorImpl.java:run(408)) - Memory usage of ProcessTree 29820 for
container-id container_1414597212088_0002_01_01: 209.4 MB of 256 MB
physical memory used; 1.2 GB of 537.6 MB virtual memory used
2014-10-29 16:10:23,294 INFO monitor.ContainersMonitorImpl
(ContainersMonitorImpl.java:run(408)) - Memory usage of ProcessTree 29820 for
container-id container_1414597212088_0002_01_01: 212.8 MB of 256 MB
physical memory used; 1.2 GB of 537.6 MB virtual memory used
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-595) Default app master container size should be 1024m
[
https://issues.apache.org/jira/browse/SLIDER-595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192909#comment-14192909
]
Sumit Mohanty commented on SLIDER-595:
--
The alternative is to have this value be upped to 512m by default and up the
value in the app packages to be 1024m. 512 because on my machine where I run
smaller applications (~2/3 container) the vmem footprint seem to be ~850m.
Default app master container size should be 1024m
-
Key: SLIDER-595
URL: https://issues.apache.org/jira/browse/SLIDER-595
Project: Slider
Issue Type: Bug
Components: appmaster
Affects Versions: Slider 0.50
Reporter: Sumit Mohanty
Assignee: Steve Loughran
Fix For: Slider 0.60
This is from a HBase application run.
{noformat}
2014-10-29 16:10:11,185 INFO monitor.ContainersMonitorImpl
(ContainersMonitorImpl.java:run(408)) - Memory usage of ProcessTree 29820 for
container-id container_1414597212088_0002_01_01: 78.8 MB of 256 MB
physical memory used; 1.1 GB of 537.6 MB virtual memory used
2014-10-29 16:10:14,213 INFO monitor.ContainersMonitorImpl
(ContainersMonitorImpl.java:run(408)) - Memory usage of ProcessTree 29820 for
container-id container_1414597212088_0002_01_01: 138.7 MB of 256 MB
physical memory used; 1.1 GB of 537.6 MB virtual memory used
2014-10-29 16:10:17,233 INFO monitor.ContainersMonitorImpl
(ContainersMonitorImpl.java:run(408)) - Memory usage of ProcessTree 29820 for
container-id container_1414597212088_0002_01_01: 182.3 MB of 256 MB
physical memory used; 1.2 GB of 537.6 MB virtual memory used
2014-10-29 16:10:20,276 INFO monitor.ContainersMonitorImpl
(ContainersMonitorImpl.java:run(408)) - Memory usage of ProcessTree 29820 for
container-id container_1414597212088_0002_01_01: 209.4 MB of 256 MB
physical memory used; 1.2 GB of 537.6 MB virtual memory used
2014-10-29 16:10:23,294 INFO monitor.ContainersMonitorImpl
(ContainersMonitorImpl.java:run(408)) - Memory usage of ProcessTree 29820 for
container-id container_1414597212088_0002_01_01: 212.8 MB of 256 MB
physical memory used; 1.2 GB of 537.6 MB virtual memory used
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (SLIDER-598) HBase application package for Windows need to be tested and validated
Sumit Mohanty created SLIDER-598: Summary: HBase application package for Windows need to be tested and validated Key: SLIDER-598 URL: https://issues.apache.org/jira/browse/SLIDER-598 Project: Slider Issue Type: Bug Components: app-package Affects Versions: Slider 0.50 Reporter: Sumit Mohanty Assignee: Sumit Mohanty Fix For: Slider 0.60 Update hbase app package configurations and metadata. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-598) HBase application package for Windows need to be tested and validated
[ https://issues.apache.org/jira/browse/SLIDER-598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-598: - Sprint: Slider October #2 HBase application package for Windows need to be tested and validated - Key: SLIDER-598 URL: https://issues.apache.org/jira/browse/SLIDER-598 Project: Slider Issue Type: Bug Components: app-package Affects Versions: Slider 0.50 Reporter: Sumit Mohanty Assignee: Sumit Mohanty Fix For: Slider 0.60 Update hbase app package configurations and metadata. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-598) HBase application package for Windows need to be tested and validated
[ https://issues.apache.org/jira/browse/SLIDER-598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192912#comment-14192912 ] ASF subversion and git services commented on SLIDER-598: Commit 48d6bd2d62716b77210506bd07bd9754b26658ce in incubator-slider's branch refs/heads/develop from [~sumitmohanty] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=48d6bd2 ] SLIDER-598. HBase application package for Windows HBase application package for Windows need to be tested and validated - Key: SLIDER-598 URL: https://issues.apache.org/jira/browse/SLIDER-598 Project: Slider Issue Type: Bug Components: app-package Affects Versions: Slider 0.50 Reporter: Sumit Mohanty Assignee: Sumit Mohanty Fix For: Slider 0.60 Update hbase app package configurations and metadata. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Slider-develop - Build # 403 - Still Failing
The Apache Jenkins build system has built Slider-develop (build #403) Status: Still Failing Check console output at https://builds.apache.org/job/Slider-develop/403/ to view the results.
Slider-develop - Build # 404 - Fixed
The Apache Jenkins build system has built Slider-develop (build #404) Status: Fixed Check console output at https://builds.apache.org/job/Slider-develop/404/ to view the results.
[jira] [Updated] (SLIDER-599) When application is created as user hdfs need to call destroy twice to delete the hdfs folder for the app
[
https://issues.apache.org/jira/browse/SLIDER-599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty updated SLIDER-599:
-
Sprint: Slider October #2
When application is created as user hdfs need to call destroy twice to delete
the hdfs folder for the app
-
Key: SLIDER-599
URL: https://issues.apache.org/jira/browse/SLIDER-599
Project: Slider
Issue Type: Bug
Components: client
Affects Versions: Slider 0.50
Reporter: Sumit Mohanty
Assignee: Steve Loughran
Fix For: Slider 0.60
It was also reported by another user. This is not a critical issue as it is
not expected that application be created as user hdfs.
Assigning to check if there is any other issue hiding behind this symptom.
{noformat}
[hdfs@c6403 bin]$ ./slider destroy cl1
2014-11-01 04:34:06,112 [main] INFO impl.TimelineClientImpl - Timeline
service address: http://c6403.ambari.apache.org:8188/ws/v1/timeline/
2014-11-01 04:34:07,161 [main] WARN shortcircuit.DomainSocketFactory - The
short-circuit local reads feature cannot be used because libhadoop cannot be
loaded.
2014-11-01 04:34:07,172 [main] INFO client.RMProxy - Connecting to
ResourceManager at c6403.ambari.apache.org/192.168.64.103:8050
2014-11-01 04:34:07,516 [main] INFO zk.BlockingZKWatcher - waiting for ZK
event
2014-11-01 04:34:07,568 [main-EventThread] INFO zk.BlockingZKWatcher - ZK
binding callback received
2014-11-01 04:34:07,572 [main] INFO client.SliderClient - Deleting zookeeper
path /services/slider/users/hdfs/cl1
2014-11-01 04:34:07,852 [main] INFO imps.CuratorFrameworkImpl - Starting
2014-11-01 04:34:07,942 [main-EventThread] INFO state.ConnectionStateManager
- State change: CONNECTED
2014-11-01 04:34:07,943 [ConnectionStateManager-0] WARN
state.ConnectionStateManager - There are no ConnectionStateListeners
registered.
2014-11-01 04:34:08,969 [main] INFO client.SliderClient - Destroyed cluster
cl1
2014-11-01 04:34:08,977 [main] INFO util.ExitUtil - Exiting with status 0
{noformat}
{noformat}
[hdfs@c6403 bin]$ ./slider create cl1 --template
/usr/work/hbase/appConfig.json --resources /usr/work/hbase/resources.json
2014-11-01 04:35:12,816 [main] INFO impl.TimelineClientImpl - Timeline
service address: http://c6403.ambari.apache.org:8188/ws/v1/timeline/
2014-11-01 04:35:13,561 [main] WARN shortcircuit.DomainSocketFactory - The
short-circuit local reads feature cannot be used because libhadoop cannot be
loaded.
2014-11-01 04:35:13,568 [main] INFO client.RMProxy - Connecting to
ResourceManager at c6403.ambari.apache.org/192.168.64.103:8050
2014-11-01 04:35:14,028 [main] INFO zk.BlockingZKWatcher - waiting for ZK
event
2014-11-01 04:35:14,052 [main-EventThread] INFO zk.BlockingZKWatcher - ZK
binding callback received
2014-11-01 04:35:14,063 [main] INFO agent.AgentClientProvider - Validating
app definition
.slider/package/HBASE/slider-hbase-app-package-0.98.4.2.2.0.0-1623-hadoop2.zip
2014-11-01 04:35:14,064 [main] INFO agent.AgentUtils - Reading metainfo at
.slider/package/HBASE/slider-hbase-app-package-0.98.4.2.2.0.0-1623-hadoop2.zip
2014-11-01 04:35:14,299 [main] INFO tools.SliderUtils - Reading metainfo.xml
of size 6909
2014-11-01 04:35:14,447 [main] ERROR tools.CoreFileSystem - Dir
hdfs://c6403.ambari.apache.org:8020/user/hdfs/.slider/cluster/cl1 exists:
hdfs://c6403.ambari.apache.org:8020/user/hdfs/.slider/cluster/cl1/database 0
2014-11-01 04:35:14,448 [main] ERROR main.ServiceLauncher - Application
Instance dir already exists:
hdfs://c6403.ambari.apache.org:8020/user/hdfs/.slider/cluster/cl1
2014-11-01 04:35:14,450 [main] INFO util.ExitUtil - Exiting with status 75
{noformat}
{noformat}
[hdfs@c6403 bin]$ hdfs dfs -ls /user/hdfs/.slider/cluster
Found 1 items
drwxr-xr-x - hdfs hdfs 0 2014-11-01 04:34
/user/hdfs/.slider/cluster/cl1
{noformat}
{noformat}
[hdfs@c6403 bin]$ ./slider destroy cl1
2014-11-01 04:37:25,003 [main] INFO impl.TimelineClientImpl - Timeline
service address: http://c6403.ambari.apache.org:8188/ws/v1/timeline/
2014-11-01 04:37:25,682 [main] WARN shortcircuit.DomainSocketFactory - The
short-circuit local reads feature cannot be used because libhadoop cannot be
loaded.
2014-11-01 04:37:25,692 [main] INFO client.RMProxy - Connecting to
ResourceManager at c6403.ambari.apache.org/192.168.64.103:8050
2014-11-01 04:37:25,965 [main] INFO zk.BlockingZKWatcher - waiting for ZK
event
2014-11-01 04:37:25,989 [main-EventThread] INFO zk.BlockingZKWatcher - ZK
binding callback received
2014-11-01 04:37:25,993 [main] INFO client.SliderClient - Deleting zookeeper
path /services/slider/users/hdfs/cl1
2014-11-01 04:37:26,037 [main] INFO
[jira] [Created] (SLIDER-599) When application is created as user hdfs need to call destroy twice to delete the hdfs folder for the app
Sumit Mohanty created SLIDER-599:
Summary: When application is created as user hdfs need to call
destroy twice to delete the hdfs folder for the app
Key: SLIDER-599
URL: https://issues.apache.org/jira/browse/SLIDER-599
Project: Slider
Issue Type: Bug
Components: client
Affects Versions: Slider 0.50
Reporter: Sumit Mohanty
Assignee: Steve Loughran
Fix For: Slider 0.60
It was also reported by another user. This is not a critical issue as it is not
expected that application be created as user hdfs.
Assigning to check if there is any other issue hiding behind this symptom.
{noformat}
[hdfs@c6403 bin]$ ./slider destroy cl1
2014-11-01 04:34:06,112 [main] INFO impl.TimelineClientImpl - Timeline service
address: http://c6403.ambari.apache.org:8188/ws/v1/timeline/
2014-11-01 04:34:07,161 [main] WARN shortcircuit.DomainSocketFactory - The
short-circuit local reads feature cannot be used because libhadoop cannot be
loaded.
2014-11-01 04:34:07,172 [main] INFO client.RMProxy - Connecting to
ResourceManager at c6403.ambari.apache.org/192.168.64.103:8050
2014-11-01 04:34:07,516 [main] INFO zk.BlockingZKWatcher - waiting for ZK event
2014-11-01 04:34:07,568 [main-EventThread] INFO zk.BlockingZKWatcher - ZK
binding callback received
2014-11-01 04:34:07,572 [main] INFO client.SliderClient - Deleting zookeeper
path /services/slider/users/hdfs/cl1
2014-11-01 04:34:07,852 [main] INFO imps.CuratorFrameworkImpl - Starting
2014-11-01 04:34:07,942 [main-EventThread] INFO state.ConnectionStateManager -
State change: CONNECTED
2014-11-01 04:34:07,943 [ConnectionStateManager-0] WARN
state.ConnectionStateManager - There are no ConnectionStateListeners registered.
2014-11-01 04:34:08,969 [main] INFO client.SliderClient - Destroyed cluster cl1
2014-11-01 04:34:08,977 [main] INFO util.ExitUtil - Exiting with status 0
{noformat}
{noformat}
[hdfs@c6403 bin]$ ./slider create cl1 --template /usr/work/hbase/appConfig.json
--resources /usr/work/hbase/resources.json
2014-11-01 04:35:12,816 [main] INFO impl.TimelineClientImpl - Timeline service
address: http://c6403.ambari.apache.org:8188/ws/v1/timeline/
2014-11-01 04:35:13,561 [main] WARN shortcircuit.DomainSocketFactory - The
short-circuit local reads feature cannot be used because libhadoop cannot be
loaded.
2014-11-01 04:35:13,568 [main] INFO client.RMProxy - Connecting to
ResourceManager at c6403.ambari.apache.org/192.168.64.103:8050
2014-11-01 04:35:14,028 [main] INFO zk.BlockingZKWatcher - waiting for ZK event
2014-11-01 04:35:14,052 [main-EventThread] INFO zk.BlockingZKWatcher - ZK
binding callback received
2014-11-01 04:35:14,063 [main] INFO agent.AgentClientProvider - Validating app
definition
.slider/package/HBASE/slider-hbase-app-package-0.98.4.2.2.0.0-1623-hadoop2.zip
2014-11-01 04:35:14,064 [main] INFO agent.AgentUtils - Reading metainfo at
.slider/package/HBASE/slider-hbase-app-package-0.98.4.2.2.0.0-1623-hadoop2.zip
2014-11-01 04:35:14,299 [main] INFO tools.SliderUtils - Reading metainfo.xml
of size 6909
2014-11-01 04:35:14,447 [main] ERROR tools.CoreFileSystem - Dir
hdfs://c6403.ambari.apache.org:8020/user/hdfs/.slider/cluster/cl1 exists:
hdfs://c6403.ambari.apache.org:8020/user/hdfs/.slider/cluster/cl1/database0
2014-11-01 04:35:14,448 [main] ERROR main.ServiceLauncher - Application
Instance dir already exists:
hdfs://c6403.ambari.apache.org:8020/user/hdfs/.slider/cluster/cl1
2014-11-01 04:35:14,450 [main] INFO util.ExitUtil - Exiting with status 75
{noformat}
{noformat}
[hdfs@c6403 bin]$ hdfs dfs -ls /user/hdfs/.slider/cluster
Found 1 items
drwxr-xr-x - hdfs hdfs 0 2014-11-01 04:34
/user/hdfs/.slider/cluster/cl1
{noformat}
{noformat}
[hdfs@c6403 bin]$ ./slider destroy cl1
2014-11-01 04:37:25,003 [main] INFO impl.TimelineClientImpl - Timeline service
address: http://c6403.ambari.apache.org:8188/ws/v1/timeline/
2014-11-01 04:37:25,682 [main] WARN shortcircuit.DomainSocketFactory - The
short-circuit local reads feature cannot be used because libhadoop cannot be
loaded.
2014-11-01 04:37:25,692 [main] INFO client.RMProxy - Connecting to
ResourceManager at c6403.ambari.apache.org/192.168.64.103:8050
2014-11-01 04:37:25,965 [main] INFO zk.BlockingZKWatcher - waiting for ZK event
2014-11-01 04:37:25,989 [main-EventThread] INFO zk.BlockingZKWatcher - ZK
binding callback received
2014-11-01 04:37:25,993 [main] INFO client.SliderClient - Deleting zookeeper
path /services/slider/users/hdfs/cl1
2014-11-01 04:37:26,037 [main] INFO imps.CuratorFrameworkImpl - Starting
2014-11-01 04:37:26,099 [main-EventThread] INFO state.ConnectionStateManager -
State change: CONNECTED
2014-11-01 04:37:26,100 [ConnectionStateManager-0] WARN
state.ConnectionStateManager - There are no ConnectionStateListeners registered.
2014-11-01 04:37:27,107 [main] INFO
[jira] [Resolved] (SLIDER-205) hbase package to work on windows
[ https://issues.apache.org/jira/browse/SLIDER-205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty resolved SLIDER-205. -- Resolution: Fixed Fix Version/s: Slider 0.60 hbase package to work on windows Key: SLIDER-205 URL: https://issues.apache.org/jira/browse/SLIDER-205 Project: Slider Issue Type: Sub-task Components: client, core, test Environment: windows Reporter: Steve Loughran Fix For: Slider 0.60 hbase package enhanced to work with / tested on windows -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLIDER-420) slider funtests not working on windows
[ https://issues.apache.org/jira/browse/SLIDER-420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty resolved SLIDER-420. -- Resolution: Fixed Fix Version/s: Slider 0.60 slider funtests not working on windows -- Key: SLIDER-420 URL: https://issues.apache.org/jira/browse/SLIDER-420 Project: Slider Issue Type: Sub-task Components: test, windows Affects Versions: Slider 0.60 Reporter: Steve Loughran Assignee: Steve Loughran Fix For: Slider 0.60 Original Estimate: 3h Remaining Estimate: 3h -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLIDER-401) slider-agent tests not running on windows
[ https://issues.apache.org/jira/browse/SLIDER-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty resolved SLIDER-401. -- Resolution: Fixed Fix Version/s: Slider 0.60 slider-agent tests not running on windows - Key: SLIDER-401 URL: https://issues.apache.org/jira/browse/SLIDER-401 Project: Slider Issue Type: Sub-task Components: agent, test Affects Versions: Slider 0.60 Reporter: Steve Loughran Assignee: Sumit Mohanty Fix For: Slider 0.60 Original Estimate: 0.5h Time Spent: 1h Remaining Estimate: 2h tests aren't starting ... launch scripts by the look of things -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-598) HBase application package for Windows need to be tested and validated
[ https://issues.apache.org/jira/browse/SLIDER-598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192962#comment-14192962 ] ASF subversion and git services commented on SLIDER-598: Commit a07eaf0a08a6f18ea7672403e369def3080fab89 in incubator-slider's branch refs/heads/develop from [~sumitmohanty] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=a07eaf0 ] SLIDER-598. HBase application package for Windows HBase application package for Windows need to be tested and validated - Key: SLIDER-598 URL: https://issues.apache.org/jira/browse/SLIDER-598 Project: Slider Issue Type: Bug Components: app-package Affects Versions: Slider 0.50 Reporter: Sumit Mohanty Assignee: Sumit Mohanty Fix For: Slider 0.60 Update hbase app package configurations and metadata. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
