[jira] [Commented] (SLIDER-582) Registry test may fail if ~ expands to a path with upper case
[
https://issues.apache.org/jira/browse/SLIDER-582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14189845#comment-14189845
]
Steve Loughran commented on SLIDER-582:
---
This should be addressed by YARN-2677, but I'll add some checks
> Registry test may fail if ~ expands to a path with upper case
> -
>
> Key: SLIDER-582
> URL: https://issues.apache.org/jira/browse/SLIDER-582
> Project: Slider
> Issue Type: Bug
> Components: registry
>Affects Versions: Slider 0.50
>Reporter: Sumit Mohanty
>Assignee: Steve Loughran
> Fix For: Slider 0.60
>
>
> {noformat}
> 2014-10-29 22:51:10,667 [JUnit] INFO framework.SliderShell (?:call(?)) -
> /Users/smohanty/enlistments/incubator-slider/slider-assembly/target/slider-0.51.0-incubating-SNAPSHOT-all/slider-0.51.0-incubating-SNAPSHOT/bin/slider
> resolve --list --path \~
> 2014-10-29 22:51:10,667 [JUnit] DEBUG framework.SliderShell (?:call(?)) -
> export
> SLIDER_CONF_DIR=/Users/smohanty/enlistments/incubator-slider/src/test/clusters/remote/slider;
> /Users/smohanty/enlistments/incubator-slider/slider-assembly/target/slider-0.51.0-incubating-SNAPSHOT-all/slider-0.51.0-incubating-SNAPSHOT/bin/slider
> resolve --list --path \~
> 2014-10-29 22:51:12,765 [JUnit] ERROR framework.SliderShell (?:call(?)) - 56
> =>/Users/smohanty/enlistments/incubator-slider/slider-assembly/target/slider-0.51.0-incubating-SNAPSHOT-all/slider-0.51.0-incubating-SNAPSHOT/bin/slider
> resolve --list --path \~
> 2014-10-29 22:51:12,765 [JUnit] ERROR framework.SliderShell
> (NativeMethodAccessorImpl.java:invoke0(?)) - return code = 56
> 2014-10-29 22:51:12,765 [JUnit] INFO framework.SliderShell
> (NativeMethodAccessorImpl.java:invoke0(?)) -
> {noformat}
> {noformat}
> 2014-10-29 22:51:12,652 [main] ERROR main.ServiceLauncher
> (ServiceLauncher.java:error(344)) - Exception: `/Users/smohanty': Invalid
> Path element "Users"
> org.apache.hadoop.registry.client.exceptions.InvalidPathnameException:
> `/Users/smohanty': Invalid Path element "Users"
> at
> org.apache.hadoop.registry.client.binding.RegistryPathUtils.validateElementsAsDNS(RegistryPathUtils.java:78)
> at
> org.apache.hadoop.registry.client.impl.zk.RegistryOperationsService.validatePath(RegistryOperationsService.java:91)
> at
> org.apache.hadoop.registry.client.impl.zk.RegistryOperationsService.list(RegistryOperationsService.java:145)
> at
> org.apache.hadoop.registry.client.binding.RegistryUtils.statChildren(RegistryUtils.java:178)
> at
> org.apache.slider.client.SliderClient.actionResolve(SliderClient.java:2475)
> at org.apache.slider.client.SliderClient.exec(SliderClient.java:398)
> at
> org.apache.slider.client.SliderClient.runService(SliderClient.java:341)
> at
> org.apache.slider.core.main.ServiceLauncher.launchService(ServiceLauncher.java:188)
> at
> org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(ServiceLauncher.java:473)
> at
> org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(ServiceLauncher.java:403)
> at
> org.apache.slider.core.main.ServiceLauncher.serviceMain(ServiceLauncher.java:628)
> at org.apache.slider.Slider.main(Slider.java:49)
> 2014-10-29 22:51:12,655 [main] INFO util.ExitUtil
> (ExitUtil.java:terminate(124)) - Exiting with status 56
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (SLIDER-583) Slider-HA:ClusterConnectivityIT failed due to connection refused
Gour Saha created SLIDER-583: Summary: Slider-HA:ClusterConnectivityIT failed due to connection refused Key: SLIDER-583 URL: https://issues.apache.org/jira/browse/SLIDER-583 Project: Slider Issue Type: Bug Reporter: Gour Saha The following 2 tests are failing in HA setup - org.apache.slider.funtest.basic.ClusterConnectivityIT-testRMBinding org.apache.slider.funtest.basic.ClusterConnectivityIT-testRMTelnet due to java.net.ConnectException: Connection refused -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (SLIDER-583) Slider-HA:ClusterConnectivityIT failed due to connection refused
[ https://issues.apache.org/jira/browse/SLIDER-583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha reassigned SLIDER-583: Assignee: Gour Saha > Slider-HA:ClusterConnectivityIT failed due to connection refused > > > Key: SLIDER-583 > URL: https://issues.apache.org/jira/browse/SLIDER-583 > Project: Slider > Issue Type: Bug >Reporter: Gour Saha >Assignee: Gour Saha > > The following 2 tests are failing in HA setup - > org.apache.slider.funtest.basic.ClusterConnectivityIT-testRMBinding > org.apache.slider.funtest.basic.ClusterConnectivityIT-testRMTelnet > due to java.net.ConnectException: Connection refused -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-583) Slider-HA:ClusterConnectivityIT failed due to connection refused
[ https://issues.apache.org/jira/browse/SLIDER-583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14189860#comment-14189860 ] ASF subversion and git services commented on SLIDER-583: Commit 20b071e87eda37625d17b53e7eafc40ef9c30363 in incubator-slider's branch refs/heads/develop from [~gsaha] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=20b071e ] SLIDER-583 Slider-HA:ClusterConnectivityIT failed due to connection refused > Slider-HA:ClusterConnectivityIT failed due to connection refused > > > Key: SLIDER-583 > URL: https://issues.apache.org/jira/browse/SLIDER-583 > Project: Slider > Issue Type: Bug >Reporter: Gour Saha >Assignee: Gour Saha > > The following 2 tests are failing in HA setup - > org.apache.slider.funtest.basic.ClusterConnectivityIT-testRMBinding > org.apache.slider.funtest.basic.ClusterConnectivityIT-testRMTelnet > due to java.net.ConnectException: Connection refused -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLIDER-583) Slider-HA:ClusterConnectivityIT failed due to connection refused
[ https://issues.apache.org/jira/browse/SLIDER-583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha resolved SLIDER-583. -- Resolution: Fixed > Slider-HA:ClusterConnectivityIT failed due to connection refused > > > Key: SLIDER-583 > URL: https://issues.apache.org/jira/browse/SLIDER-583 > Project: Slider > Issue Type: Bug >Reporter: Gour Saha >Assignee: Gour Saha > > The following 2 tests are failing in HA setup - > org.apache.slider.funtest.basic.ClusterConnectivityIT-testRMBinding > org.apache.slider.funtest.basic.ClusterConnectivityIT-testRMTelnet > due to java.net.ConnectException: Connection refused -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SLIDER-584) CommandEnvironmentIT failing on HA cluster - assertion failures
Steve Loughran created SLIDER-584: - Summary: CommandEnvironmentIT failing on HA cluster - assertion failures Key: SLIDER-584 URL: https://issues.apache.org/jira/browse/SLIDER-584 Project: Slider Issue Type: Sub-task Components: test Affects Versions: Slider 0.60 Environment: HA yarn cluster Reporter: Steve Loughran Assignee: Steve Loughran The CommandEnvironmentIT test is failing, assertions that slider.confdir and slider libdir are in stdout are failing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-584) CommandEnvironmentIT failing on HA cluster - assertion failures
[
https://issues.apache.org/jira/browse/SLIDER-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14189953#comment-14189953
]
Steve Loughran commented on SLIDER-584:
---
Test output
{code}
--
Test set: org.apache.slider.funtest.commands.CommandEnvironmentIT
---
Tests run: 6, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 34.876 sec <<<
FAILURE! - in org.apache.slider.funtest.commands.CommandEnvironmentIT
testLibdirPython(org.apache.slider.funtest.commands.CommandEnvironmentIT) Time
elapsed: 5.482 sec <<< FAILURE!
Assertion failed:
assert shell.outputContains(lookThisUp)
| | |
| false slider.libdir
0
=>/grid/0/hadoopqe/artifacts/fun-tests/slider-0.51.0.2.2.0.0/slider-assembly/target/slider-0.51.0.2.2.0.0-1162-all/slider-0.51.0.2.2.0.0-1162/bin/slider.py
diagnostics --client --verbose
at
org.codehaus.groovy.runtime.InvokerHelper.assertFailed(InvokerHelper.java:398)
at
org.codehaus.groovy.runtime.ScriptBytecodeAdapter.assertFailed(ScriptBytecodeAdapter.java:646)
at
org.apache.slider.funtest.framework.CommandTestBase.assertOutputContains(CommandTestBase.groovy:498)
at
org.apache.slider.funtest.framework.CommandTestBase.assertOutputContains(CommandTestBase.groovy)
at
org.apache.slider.funtest.commands.CommandEnvironmentIT.testLibdirPython(CommandEnvironmentIT.groovy:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
testConfdirPython(org.apache.slider.funtest.commands.CommandEnvironmentIT)
Time elapsed: 5.195 sec <<< FAILURE!
{code}
> CommandEnvironmentIT failing on HA cluster - assertion failures
> ---
>
> Key: SLIDER-584
> URL: https://issues.apache.org/jira/browse/SLIDER-584
> Project: Slider
> Issue Type: Sub-task
> Components: test
>Affects Versions: Slider 0.60
> Environment: HA yarn cluster
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Fix For: Slider 0.60
>
>
> The CommandEnvironmentIT test is failing, assertions that slider.confdir and
> slider libdir are in stdout are failing
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-584) CommandEnvironmentIT failing on HA cluster - assertion failures
[
https://issues.apache.org/jira/browse/SLIDER-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14189956#comment-14189956
]
Steve Loughran commented on SLIDER-584:
---
output as printed
{code}
/
2014-10-29 22:50:53,301 [main] DEBUG main.ServiceLauncher
(ServiceLauncher.java:serviceMain(622)) - "org.apache.slider.Slider"
"diagnostics" "--client" "--verbose"
2014-10-29 22:50:53,419 [main] DEBUG utility.LaunchedWorkflowCompositeService
(LaunchedWorkflowCompositeService.java:bindArgs(80)) - Binding 3 Arguments:
2014-10-29 22:50:53,419 [main] DEBUG utility.LaunchedWorkflowCompositeService
(LaunchedWorkflowCompositeService.java:bindArgs(86)) - "diagnostics" "--client"
"--verbose"
2014-10-29 22:50:53,659 [main] DEBUG params.CommonArgs
(CommonArgs.java:validate(241)) - action=diagnostics
2014-10-29 22:50:54,370 [main] DEBUG tools.ConfigHelper
(ConfigHelper.java:loadFromResource(511)) - loaded resources from
file:/grid/0/hadoopqe/artifacts/fun-tests/slider-conf/slider-client.xml
2014-10-29 22:50:56,176 [main] INFO impl.TimelineClientImpl
(TimelineClientImpl.java:serviceInit(284)) - Timeline service address:
http://ip-172-31-45-192.ec2.internal:8188/ws/v1/timeline/
2014-10-29 22:50:56,667 [main] WARN hdfs.DFSClient
(DFSClient.java:(624)) - dfs.client.test.drop.namenode.response.number is
set to 1, this hacked client will proactively drop responses
2014-10-29 22:50:57,373 [main] WARN shortcircuit.DomainSocketFactory
(DomainSocketFactory.java:(116)) - The short-circuit local reads feature
cannot be used because libhadoop cannot be loaded.
2014-10-29 22:50:57,557 [main] INFO client.SliderClient
(SliderVersionInfo.java:loadAndPrintVersionInfo(95)) - Slider Core-0.51 Built
against commit# 8f7593d863 on Java 1.7.0_67 by hrt_qa
2014-10-29 22:50:57,558 [main] INFO client.SliderClient
(SliderVersionInfo.java:loadAndPrintVersionInfo(96)) - Compiled against Hadoop
2.6.0
2014-10-29 22:50:57,560 [main] INFO client.SliderClient
(SliderVersionInfo.java:loadAndPrintVersionInfo(98)) - Hadoop runtime version
(no branch) with source checksum 5910844e41d553e4b21e14c3cc68c84 and build date
2014-10-29T03:38Z
The slider command path:
/grid/0/hadoopqe/artifacts/fun-tests/slider-0.51/slider-assembly/target/slider-0.51-all/slider-0.51/lib/slider-core-0.51.jar
The slider-client.xml used by current running command path:
file:/grid/0/hadoopqe/artifacts/fun-tests/slider-conf/slider-client.xml
The version of the JDK invoking the current running slider command: 1.7.0_67;
The path to it is: /usr/jdk64/jdk1.7.0_67/jre
Hadoop Cluster is insecure
Environment variables:
CVS_RSH=ssh
G_BROKEN_FILENAMES=1
HADOOP_CONF_DIR=/etc/hadoop/conf
HISTSIZE=1000
HOME=/home/hrt_qa
HOSTNAME=ip-172-31-45-190
JAVA_HOME=/usr/hadoop-jdk1.6.0_31
JAVA_OPTS= -Duser.timezone=UTC
LANG=en_US.UTF-8
LESSOPEN=|/usr/bin/lesspipe.sh %s
LOGNAME=hrt_qa
M2_HOME=/grid/0/hadoopqe/tools/apache-maven-3.0.4
MAIL=/var/spool/mail/root
NLSPATH=/usr/dt/lib/nls/msg/%L/%N.cat
PATH=/sbin:/bin:/usr/sbin:/usr/bin
PWD=/grid/0/hadoopqe/artifacts/fun-tests/slider-0.51/slider-funtest
PYTHONPATH=:/usr/lib/python2.6/site-packages
QTDIR=/usr/lib64/qt-3.3
SHELL=/bin/bash
SHLVL=4
SLIDER_CONF_DIR=/grid/0/hadoopqe/artifacts/fun-tests/slider-conf
SLIDER_JVM_OPTS=-DtestpropertySetInFuntest=TestPropertyValue
SUDO_COMMAND=/bin/su hrt_qa
SUDO_GID=0
SUDO_UID=0
SUDO_USER=root
TERM=xterm-256color
USER=hrt_qa
USERNAME=root
XFILESEARCHPATH=/usr/dt/app-defaults/%L/Dt
_=/grid/0/hadoopqe/artifacts/fun-tests/slider-0.51/slider-assembly/target/slider-0.51-all/slider-0.51/bin/slider.py
JVM Properties
awt.toolkit=sun.awt.X11.XToolkit
file.encoding=UTF-8
file.encoding.pkg=sun.io
file.separator=/
java.awt.graphicsenv=sun.awt.X11GraphicsEnvironment
java.awt.printerjob=sun.print.PSPrinterJob
]]>
{code}
> CommandEnvironmentIT failing on HA cluster - assertion failures
> ---
>
> Key: SLIDER-584
> URL: https://issues.apache.org/jira/browse/SLIDER-584
> Project: Slider
> Issue Type: Sub-task
> Components: test
>Affects Versions: Slider 0.60
> Environment: HA yarn cluster
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Fix For: Slider 0.60
>
>
> The CommandEnvironmentIT test is failing, assertions that slider.confdir and
> slider libdir are in stdout are failing
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-584) CommandEnvironmentIT failing on HA cluster - assertion failures
[
https://issues.apache.org/jira/browse/SLIDER-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14189957#comment-14189957
]
Steve Loughran commented on SLIDER-584:
---
Notable that stdout stops early —more specifically it stops before
{{java.class.path}} is output.
Maybe the string is too long for the groovy readline logic to handle
> CommandEnvironmentIT failing on HA cluster - assertion failures
> ---
>
> Key: SLIDER-584
> URL: https://issues.apache.org/jira/browse/SLIDER-584
> Project: Slider
> Issue Type: Sub-task
> Components: test
>Affects Versions: Slider 0.60
> Environment: HA yarn cluster
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Fix For: Slider 0.60
>
>
> The CommandEnvironmentIT test is failing, assertions that slider.confdir and
> slider libdir are in stdout are failing
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-584) CommandEnvironmentIT failing on HA cluster - assertion failures
[ https://issues.apache.org/jira/browse/SLIDER-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14189982#comment-14189982 ] Steve Loughran commented on SLIDER-584: --- Notable that it is the .py scripts that are failing. Is there something that's unique to them? > CommandEnvironmentIT failing on HA cluster - assertion failures > --- > > Key: SLIDER-584 > URL: https://issues.apache.org/jira/browse/SLIDER-584 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.60 > Environment: HA yarn cluster >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.60 > > > The CommandEnvironmentIT test is failing, assertions that slider.confdir and > slider libdir are in stdout are failing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-584) CommandEnvironmentIT failing on HA cluster - assertion failures
[
https://issues.apache.org/jira/browse/SLIDER-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14189984#comment-14189984
]
Steve Loughran commented on SLIDER-584:
---
SLIDER-86 reworked stdout/stderr forwarding for the python scripts so they'd be
consistent across windows and unix. Now they spin waiting for lines and
printing them
{code}
while not finished:
(line, done) = read(src, line)
info(line)
line = ""
{code}
It could just be some concurrency logic here: the line isn't completely parsed
before the {{finished}} bit is set, so the handler should add a teardown
sequence that just does a read & print until the end of the file is hit, at
which time the thread finishes.
One caveat: windows does behave differently here.
> CommandEnvironmentIT failing on HA cluster - assertion failures
> ---
>
> Key: SLIDER-584
> URL: https://issues.apache.org/jira/browse/SLIDER-584
> Project: Slider
> Issue Type: Sub-task
> Components: test
>Affects Versions: Slider 0.60
> Environment: HA yarn cluster
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Fix For: Slider 0.60
>
>
> The CommandEnvironmentIT test is failing, assertions that slider.confdir and
> slider libdir are in stdout are failing
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-584) CommandEnvironmentIT failing on HA cluster - assertion failures
[ https://issues.apache.org/jira/browse/SLIDER-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1419#comment-1419 ] ASF subversion and git services commented on SLIDER-584: Commit 035dbe7fec92baf97f97b45038f0882ff3306a44 in incubator-slider's branch refs/heads/feature/SLIDER-584_CommandEnvironmentIT from [~ste...@apache.org] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=035dbe7 ] SLIDER-584 ensure that all of stdout/stderr gets through the test run > CommandEnvironmentIT failing on HA cluster - assertion failures > --- > > Key: SLIDER-584 > URL: https://issues.apache.org/jira/browse/SLIDER-584 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.60 > Environment: HA yarn cluster >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.60 > > > The CommandEnvironmentIT test is failing, assertions that slider.confdir and > slider libdir are in stdout are failing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-584) CommandEnvironmentIT failing on HA cluster - assertion failures
[ https://issues.apache.org/jira/browse/SLIDER-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190036#comment-14190036 ] ASF subversion and git services commented on SLIDER-584: Commit 514d87e6d2592efb7b8d0bbf869e0d0085015def in incubator-slider's branch refs/heads/develop from [~ste...@apache.org] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=514d87e ] Merge branch 'feature/SLIDER-584_CommandEnvironmentIT' into develop > CommandEnvironmentIT failing on HA cluster - assertion failures > --- > > Key: SLIDER-584 > URL: https://issues.apache.org/jira/browse/SLIDER-584 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.60 > Environment: HA yarn cluster >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.60 > > > The CommandEnvironmentIT test is failing, assertions that slider.confdir and > slider libdir are in stdout are failing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-584) CommandEnvironmentIT failing on HA cluster - assertion failures
[ https://issues.apache.org/jira/browse/SLIDER-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190034#comment-14190034 ] ASF subversion and git services commented on SLIDER-584: Commit 035dbe7fec92baf97f97b45038f0882ff3306a44 in incubator-slider's branch refs/heads/develop from [~ste...@apache.org] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=035dbe7 ] SLIDER-584 ensure that all of stdout/stderr gets through the test run > CommandEnvironmentIT failing on HA cluster - assertion failures > --- > > Key: SLIDER-584 > URL: https://issues.apache.org/jira/browse/SLIDER-584 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.60 > Environment: HA yarn cluster >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.60 > > > The CommandEnvironmentIT test is failing, assertions that slider.confdir and > slider libdir are in stdout are failing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-584) CommandEnvironmentIT failing on HA cluster - assertion failures
[ https://issues.apache.org/jira/browse/SLIDER-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190035#comment-14190035 ] ASF subversion and git services commented on SLIDER-584: Commit 2826027e260b9d278769eb22a87d37238eb4419a in incubator-slider's branch refs/heads/develop from [~ste...@apache.org] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=2826027 ] SLIDER-584 flush output after every line and at the end of the run > CommandEnvironmentIT failing on HA cluster - assertion failures > --- > > Key: SLIDER-584 > URL: https://issues.apache.org/jira/browse/SLIDER-584 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.60 > Environment: HA yarn cluster >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.60 > > > The CommandEnvironmentIT test is failing, assertions that slider.confdir and > slider libdir are in stdout are failing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-584) CommandEnvironmentIT failing on HA cluster - assertion failures
[ https://issues.apache.org/jira/browse/SLIDER-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190040#comment-14190040 ] Steve Loughran commented on SLIDER-584: --- changed how python code prints output and has a teardown phase to stream through all final bytes from the spawned process. Also cleanly split stderr and stdout output into the respective channels. Manually tested on windows as well as manual + CommandEnvironmentIT on unix to verify processes complete, print their output > CommandEnvironmentIT failing on HA cluster - assertion failures > --- > > Key: SLIDER-584 > URL: https://issues.apache.org/jira/browse/SLIDER-584 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.60 > Environment: HA yarn cluster >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.60 > > > The CommandEnvironmentIT test is failing, assertions that slider.confdir and > slider libdir are in stdout are failing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-581) AgentClusterLifecycleIT test failing
[
https://issues.apache.org/jira/browse/SLIDER-581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190052#comment-14190052
]
Steve Loughran commented on SLIDER-581:
---
local exec() should be called in both places now; I've changed how they grab
stdout/stderr in response to SLIDER-584. I don't think that is where the
problem lies, but I'm just being thorough
> AgentClusterLifecycleIT test failing
>
>
> Key: SLIDER-581
> URL: https://issues.apache.org/jira/browse/SLIDER-581
> Project: Slider
> Issue Type: Sub-task
> Components: test
>Affects Versions: Slider 0.60
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Fix For: Slider 0.60
>
> Attachments: SLIDER-581.trial.patch
>
>
> Test {{AgentClusterLifecycleIT}} ; either destroy or thaw is succeeding when
> it should fail with a cluster in use message
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-584) CommandEnvironmentIT failing on HA cluster - assertion failures
[ https://issues.apache.org/jira/browse/SLIDER-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190091#comment-14190091 ] ASF subversion and git services commented on SLIDER-584: Commit 2826027e260b9d278769eb22a87d37238eb4419a in incubator-slider's branch refs/heads/feature/SLIDER-584_CommandEnvironmentIT from [~ste...@apache.org] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=2826027 ] SLIDER-584 flush output after every line and at the end of the run > CommandEnvironmentIT failing on HA cluster - assertion failures > --- > > Key: SLIDER-584 > URL: https://issues.apache.org/jira/browse/SLIDER-584 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.60 > Environment: HA yarn cluster >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.60 > > > The CommandEnvironmentIT test is failing, assertions that slider.confdir and > slider libdir are in stdout are failing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-584) CommandEnvironmentIT failing on HA cluster - assertion failures
[ https://issues.apache.org/jira/browse/SLIDER-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190092#comment-14190092 ] ASF subversion and git services commented on SLIDER-584: Commit 05cf898288af14da01a0481c2ad5d69c6fcb19dd in incubator-slider's branch refs/heads/develop from [~ste...@apache.org] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=05cf898 ] SLIDER-584 and convert to utf8 before printing > CommandEnvironmentIT failing on HA cluster - assertion failures > --- > > Key: SLIDER-584 > URL: https://issues.apache.org/jira/browse/SLIDER-584 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.60 > Environment: HA yarn cluster >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.60 > > > The CommandEnvironmentIT test is failing, assertions that slider.confdir and > slider libdir are in stdout are failing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-581) AgentClusterLifecycleIT test failing on windows
[
https://issues.apache.org/jira/browse/SLIDER-581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated SLIDER-581:
--
Environment: Windows
Sprint: (was: Slider July #1, Slider August #1, Slider August #2,
Slider September #1, Slider October #1, Slider October #2)
Summary: AgentClusterLifecycleIT test failing on windows (was:
AgentClusterLifecycleIT test failing)
> AgentClusterLifecycleIT test failing on windows
> ---
>
> Key: SLIDER-581
> URL: https://issues.apache.org/jira/browse/SLIDER-581
> Project: Slider
> Issue Type: Sub-task
> Components: test
>Affects Versions: Slider 0.60
> Environment: Windows
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Fix For: Slider 0.60
>
> Attachments: SLIDER-581.trial.patch
>
>
> Test {{AgentClusterLifecycleIT}} ; either destroy or thaw is succeeding when
> it should fail with a cluster in use message
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (SLIDER-585) Localize SSL certs for apps
Billie Rinaldi created SLIDER-585: - Summary: Localize SSL certs for apps Key: SLIDER-585 URL: https://issues.apache.org/jira/browse/SLIDER-585 Project: Slider Issue Type: Improvement Components: security Reporter: Billie Rinaldi Assignee: Jonathan Maron Fix For: Slider 2.0.0 See discussion on SLIDER-580. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-580) Install SSL certs
[ https://issues.apache.org/jira/browse/SLIDER-580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190104#comment-14190104 ] Billie Rinaldi commented on SLIDER-580: --- Created SLIDER-585. > Install SSL certs > - > > Key: SLIDER-580 > URL: https://issues.apache.org/jira/browse/SLIDER-580 > Project: Slider > Issue Type: Improvement >Reporter: Billie Rinaldi >Assignee: Jonathan Maron > > In addition to keytabs, it would be useful to be able to install SSL certs > for localization. We could simply add jks files as a type of file understood > by install-keytab. Although this does lead to the question of whether we'd > want to support installing arbitrary resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-581) AgentClusterLifecycleIT test failing on windows
[
https://issues.apache.org/jira/browse/SLIDER-581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190112#comment-14190112
]
ASF subversion and git services commented on SLIDER-581:
Commit 2d953fff1f3d65dc19e4d0fb41ea9fd394de00d5 in incubator-slider's branch
refs/heads/feature/SLIDER-581_funtest_return_values_on_windows from
[~ste...@apache.org]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=2d953ff ]
SLIDER-581 ensure that exit codes come back on funtests on windows
> AgentClusterLifecycleIT test failing on windows
> ---
>
> Key: SLIDER-581
> URL: https://issues.apache.org/jira/browse/SLIDER-581
> Project: Slider
> Issue Type: Sub-task
> Components: test
>Affects Versions: Slider 0.60
> Environment: Windows
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Fix For: Slider 0.60
>
> Attachments: SLIDER-581.trial.patch
>
>
> Test {{AgentClusterLifecycleIT}} ; either destroy or thaw is succeeding when
> it should fail with a cluster in use message
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-581) AgentClusterLifecycleIT test failing on windows
[
https://issues.apache.org/jira/browse/SLIDER-581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190136#comment-14190136
]
ASF subversion and git services commented on SLIDER-581:
Commit ef516d9cd704beb84fda08df4b510ca58c906c40 in incubator-slider's branch
refs/heads/feature/SLIDER-581_funtest_return_values_on_windows from
[~ste...@apache.org]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=ef516d9 ]
SLIDER-581 moving where the exit codes are set up/using non superclass exec
> AgentClusterLifecycleIT test failing on windows
> ---
>
> Key: SLIDER-581
> URL: https://issues.apache.org/jira/browse/SLIDER-581
> Project: Slider
> Issue Type: Sub-task
> Components: test
>Affects Versions: Slider 0.60
> Environment: Windows
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Fix For: Slider 0.60
>
> Attachments: SLIDER-581.trial.patch
>
>
> Test {{AgentClusterLifecycleIT}} ; either destroy or thaw is succeeding when
> it should fail with a cluster in use message
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-581) AgentClusterLifecycleIT test failing on windows
[
https://issues.apache.org/jira/browse/SLIDER-581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190143#comment-14190143
]
ASF subversion and git services commented on SLIDER-581:
Commit 518d7508c239692db60722fb2cbe9805551b66f0 in incubator-slider's branch
refs/heads/develop from [~ste...@apache.org]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=518d750 ]
SLIDER-581 moving where the exit codes are set up/using non superclass exec
> AgentClusterLifecycleIT test failing on windows
> ---
>
> Key: SLIDER-581
> URL: https://issues.apache.org/jira/browse/SLIDER-581
> Project: Slider
> Issue Type: Sub-task
> Components: test
>Affects Versions: Slider 0.60
> Environment: Windows
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Fix For: Slider 0.60
>
> Attachments: SLIDER-581.trial.patch
>
>
> Test {{AgentClusterLifecycleIT}} ; either destroy or thaw is succeeding when
> it should fail with a cluster in use message
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-581) AgentClusterLifecycleIT test failing on windows
[
https://issues.apache.org/jira/browse/SLIDER-581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190145#comment-14190145
]
ASF subversion and git services commented on SLIDER-581:
Commit a19c50525e922214d0a52b1d99e9e90be51f7645 in incubator-slider's branch
refs/heads/develop from [~ste...@apache.org]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=a19c505 ]
Merge branch 'feature/SLIDER-581_funtest_return_values_on_windows' into develop
> AgentClusterLifecycleIT test failing on windows
> ---
>
> Key: SLIDER-581
> URL: https://issues.apache.org/jira/browse/SLIDER-581
> Project: Slider
> Issue Type: Sub-task
> Components: test
>Affects Versions: Slider 0.60
> Environment: Windows
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Fix For: Slider 0.60
>
> Attachments: SLIDER-581.trial.patch
>
>
> Test {{AgentClusterLifecycleIT}} ; either destroy or thaw is succeeding when
> it should fail with a cluster in use message
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-581) AgentClusterLifecycleIT test failing on windows
[
https://issues.apache.org/jira/browse/SLIDER-581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190144#comment-14190144
]
ASF subversion and git services commented on SLIDER-581:
Commit ef516d9cd704beb84fda08df4b510ca58c906c40 in incubator-slider's branch
refs/heads/develop from [~ste...@apache.org]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=ef516d9 ]
SLIDER-581 moving where the exit codes are set up/using non superclass exec
> AgentClusterLifecycleIT test failing on windows
> ---
>
> Key: SLIDER-581
> URL: https://issues.apache.org/jira/browse/SLIDER-581
> Project: Slider
> Issue Type: Sub-task
> Components: test
>Affects Versions: Slider 0.60
> Environment: Windows
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Fix For: Slider 0.60
>
> Attachments: SLIDER-581.trial.patch
>
>
> Test {{AgentClusterLifecycleIT}} ; either destroy or thaw is succeeding when
> it should fail with a cluster in use message
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-581) AgentClusterLifecycleIT test failing on windows
[
https://issues.apache.org/jira/browse/SLIDER-581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190142#comment-14190142
]
ASF subversion and git services commented on SLIDER-581:
Commit 2d953fff1f3d65dc19e4d0fb41ea9fd394de00d5 in incubator-slider's branch
refs/heads/develop from [~ste...@apache.org]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=2d953ff ]
SLIDER-581 ensure that exit codes come back on funtests on windows
> AgentClusterLifecycleIT test failing on windows
> ---
>
> Key: SLIDER-581
> URL: https://issues.apache.org/jira/browse/SLIDER-581
> Project: Slider
> Issue Type: Sub-task
> Components: test
>Affects Versions: Slider 0.60
> Environment: Windows
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Fix For: Slider 0.60
>
> Attachments: SLIDER-581.trial.patch
>
>
> Test {{AgentClusterLifecycleIT}} ; either destroy or thaw is succeeding when
> it should fail with a cluster in use message
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (SLIDER-581) AgentClusterLifecycleIT test failing on windows
[
https://issues.apache.org/jira/browse/SLIDER-581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved SLIDER-581.
---
Resolution: Fixed
took Sumit's code and moved where the {{exit %EXITCODE%}} command was set up;
now it is done with the rest of the command sequence in
{{SliderShell.execute()}}
The (new) test {{CommandExitCodesIT}} shows that exit codes are getting back to
the fun tests;
> AgentClusterLifecycleIT test failing on windows
> ---
>
> Key: SLIDER-581
> URL: https://issues.apache.org/jira/browse/SLIDER-581
> Project: Slider
> Issue Type: Sub-task
> Components: test
>Affects Versions: Slider 0.60
> Environment: Windows
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Fix For: Slider 0.60
>
> Attachments: SLIDER-581.trial.patch
>
>
> Test {{AgentClusterLifecycleIT}} ; either destroy or thaw is succeeding when
> it should fail with a cluster in use message
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-581) AgentClusterLifecycleIT test failing on windows
[
https://issues.apache.org/jira/browse/SLIDER-581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190135#comment-14190135
]
ASF subversion and git services commented on SLIDER-581:
Commit 518d7508c239692db60722fb2cbe9805551b66f0 in incubator-slider's branch
refs/heads/feature/SLIDER-581_funtest_return_values_on_windows from
[~ste...@apache.org]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=518d750 ]
SLIDER-581 moving where the exit codes are set up/using non superclass exec
> AgentClusterLifecycleIT test failing on windows
> ---
>
> Key: SLIDER-581
> URL: https://issues.apache.org/jira/browse/SLIDER-581
> Project: Slider
> Issue Type: Sub-task
> Components: test
>Affects Versions: Slider 0.60
> Environment: Windows
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Fix For: Slider 0.60
>
> Attachments: SLIDER-581.trial.patch
>
>
> Test {{AgentClusterLifecycleIT}} ; either destroy or thaw is succeeding when
> it should fail with a cluster in use message
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (SLIDER-572) add startup delay for the chaos monkey
[ https://issues.apache.org/jira/browse/SLIDER-572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved SLIDER-572. --- Resolution: Fixed > add startup delay for the chaos monkey > -- > > Key: SLIDER-572 > URL: https://issues.apache.org/jira/browse/SLIDER-572 > Project: Slider > Issue Type: Sub-task > Components: appmaster, test >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.60 > > > for functional tests we need to be able to turn the monkey on or off; this > would let us set up a test cluster into a functional state and then trigger > the chaos sequence. > Currently enabling AM killing in chaos monkey can cause tests to fail as the > test runner thinks the cluster startup failed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-582) Registry test may fail if ~ expands to a path with upper case
[
https://issues.apache.org/jira/browse/SLIDER-582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190164#comment-14190164
]
Steve Loughran commented on SLIDER-582:
---
On any branch with YARN-2677 patch 1 this would not surface: the pattern check
doesn't happen. But the names were coming in as upper or mixed case. YARN-2677
patch 002 converts all names to lower case when building the path (and it
doesn't check against the standard DNS naming pattern)
> Registry test may fail if ~ expands to a path with upper case
> -
>
> Key: SLIDER-582
> URL: https://issues.apache.org/jira/browse/SLIDER-582
> Project: Slider
> Issue Type: Bug
> Components: registry
>Affects Versions: Slider 0.50
>Reporter: Sumit Mohanty
>Assignee: Steve Loughran
> Fix For: Slider 0.60
>
>
> {noformat}
> 2014-10-29 22:51:10,667 [JUnit] INFO framework.SliderShell (?:call(?)) -
> /Users/smohanty/enlistments/incubator-slider/slider-assembly/target/slider-0.51.0-incubating-SNAPSHOT-all/slider-0.51.0-incubating-SNAPSHOT/bin/slider
> resolve --list --path \~
> 2014-10-29 22:51:10,667 [JUnit] DEBUG framework.SliderShell (?:call(?)) -
> export
> SLIDER_CONF_DIR=/Users/smohanty/enlistments/incubator-slider/src/test/clusters/remote/slider;
> /Users/smohanty/enlistments/incubator-slider/slider-assembly/target/slider-0.51.0-incubating-SNAPSHOT-all/slider-0.51.0-incubating-SNAPSHOT/bin/slider
> resolve --list --path \~
> 2014-10-29 22:51:12,765 [JUnit] ERROR framework.SliderShell (?:call(?)) - 56
> =>/Users/smohanty/enlistments/incubator-slider/slider-assembly/target/slider-0.51.0-incubating-SNAPSHOT-all/slider-0.51.0-incubating-SNAPSHOT/bin/slider
> resolve --list --path \~
> 2014-10-29 22:51:12,765 [JUnit] ERROR framework.SliderShell
> (NativeMethodAccessorImpl.java:invoke0(?)) - return code = 56
> 2014-10-29 22:51:12,765 [JUnit] INFO framework.SliderShell
> (NativeMethodAccessorImpl.java:invoke0(?)) -
> {noformat}
> {noformat}
> 2014-10-29 22:51:12,652 [main] ERROR main.ServiceLauncher
> (ServiceLauncher.java:error(344)) - Exception: `/Users/smohanty': Invalid
> Path element "Users"
> org.apache.hadoop.registry.client.exceptions.InvalidPathnameException:
> `/Users/smohanty': Invalid Path element "Users"
> at
> org.apache.hadoop.registry.client.binding.RegistryPathUtils.validateElementsAsDNS(RegistryPathUtils.java:78)
> at
> org.apache.hadoop.registry.client.impl.zk.RegistryOperationsService.validatePath(RegistryOperationsService.java:91)
> at
> org.apache.hadoop.registry.client.impl.zk.RegistryOperationsService.list(RegistryOperationsService.java:145)
> at
> org.apache.hadoop.registry.client.binding.RegistryUtils.statChildren(RegistryUtils.java:178)
> at
> org.apache.slider.client.SliderClient.actionResolve(SliderClient.java:2475)
> at org.apache.slider.client.SliderClient.exec(SliderClient.java:398)
> at
> org.apache.slider.client.SliderClient.runService(SliderClient.java:341)
> at
> org.apache.slider.core.main.ServiceLauncher.launchService(ServiceLauncher.java:188)
> at
> org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(ServiceLauncher.java:473)
> at
> org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(ServiceLauncher.java:403)
> at
> org.apache.slider.core.main.ServiceLauncher.serviceMain(ServiceLauncher.java:628)
> at org.apache.slider.Slider.main(Slider.java:49)
> 2014-10-29 22:51:12,655 [main] INFO util.ExitUtil
> (ExitUtil.java:terminate(124)) - Exiting with status 56
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (SLIDER-586) reject resource configurations with negative node counts
Steve Loughran created SLIDER-586: - Summary: reject resource configurations with negative node counts Key: SLIDER-586 URL: https://issues.apache.org/jira/browse/SLIDER-586 Project: Slider Issue Type: Bug Components: appmaster Reporter: Steve Loughran Assignee: Steve Loughran it should be an error for any configuration with negative node counts to be accepted. this check must go in server side so that changes to the resources.json file always get validated -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-586) reject resource configurations with negative node counts
[ https://issues.apache.org/jira/browse/SLIDER-586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SLIDER-586: -- Sprint: Slider October #2 > reject resource configurations with negative node counts > > > Key: SLIDER-586 > URL: https://issues.apache.org/jira/browse/SLIDER-586 > Project: Slider > Issue Type: Bug > Components: appmaster >Reporter: Steve Loughran >Assignee: Steve Loughran > Original Estimate: 0.5h > Remaining Estimate: 0.5h > > it should be an error for any configuration with negative node counts to be > accepted. > this check must go in server side so that changes to the resources.json file > always get validated -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-586) reject resource configurations with negative node counts
[ https://issues.apache.org/jira/browse/SLIDER-586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190243#comment-14190243 ] Steve Loughran commented on SLIDER-586: --- this was already checked on the client side when building or flexing a cluster. But it was not checked in the AM. Now it is, with mock tests to verify that an invalid desired count triggers a failure > reject resource configurations with negative node counts > > > Key: SLIDER-586 > URL: https://issues.apache.org/jira/browse/SLIDER-586 > Project: Slider > Issue Type: Bug > Components: appmaster >Reporter: Steve Loughran >Assignee: Steve Loughran > Original Estimate: 0.5h > Remaining Estimate: 0.5h > > it should be an error for any configuration with negative node counts to be > accepted. > this check must go in server side so that changes to the resources.json file > always get validated -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-587) Query: how well are dynamic role placement histories maintained?
[ https://issues.apache.org/jira/browse/SLIDER-587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SLIDER-587: -- Sprint: Slider October #2 > Query: how well are dynamic role placement histories maintained? > > > Key: SLIDER-587 > URL: https://issues.apache.org/jira/browse/SLIDER-587 > Project: Slider > Issue Type: Task > Components: appmaster >Reporter: Steve Loughran >Assignee: Steve Loughran > > Review the code to make sure that dynamic roles get their placement history > maintained. Maybe even add some tests to validate this -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLIDER-586) reject resource configurations with negative node counts
[ https://issues.apache.org/jira/browse/SLIDER-586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved SLIDER-586. --- Resolution: Fixed > reject resource configurations with negative node counts > > > Key: SLIDER-586 > URL: https://issues.apache.org/jira/browse/SLIDER-586 > Project: Slider > Issue Type: Bug > Components: appmaster >Reporter: Steve Loughran >Assignee: Steve Loughran > Original Estimate: 0.5h > Remaining Estimate: 0.5h > > it should be an error for any configuration with negative node counts to be > accepted. > this check must go in server side so that changes to the resources.json file > always get validated -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SLIDER-587) Query: how well are dynamic role placement histories maintained?
Steve Loughran created SLIDER-587: - Summary: Query: how well are dynamic role placement histories maintained? Key: SLIDER-587 URL: https://issues.apache.org/jira/browse/SLIDER-587 Project: Slider Issue Type: Task Components: appmaster Reporter: Steve Loughran Assignee: Steve Loughran Review the code to make sure that dynamic roles get their placement history maintained. Maybe even add some tests to validate this -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-586) reject resource configurations with negative node counts
[ https://issues.apache.org/jira/browse/SLIDER-586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190244#comment-14190244 ] ASF subversion and git services commented on SLIDER-586: Commit fbca62d883c7b1bd8327ca44d06e7e7e5b8f284b in incubator-slider's branch refs/heads/develop from [~ste...@apache.org] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=fbca62d ] SLIDER-586: negative node count checks on AM > reject resource configurations with negative node counts > > > Key: SLIDER-586 > URL: https://issues.apache.org/jira/browse/SLIDER-586 > Project: Slider > Issue Type: Bug > Components: appmaster >Reporter: Steve Loughran >Assignee: Steve Loughran > Original Estimate: 0.5h > Remaining Estimate: 0.5h > > it should be an error for any configuration with negative node counts to be > accepted. > this check must go in server side so that changes to the resources.json file > always get validated -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SLIDER-588) AppsThroughAgentIT test failed with 'Application registry is not accessible'
Yesha Vora created SLIDER-588:
-
Summary: AppsThroughAgentIT test failed with 'Application registry
is not accessible'
Key: SLIDER-588
URL: https://issues.apache.org/jira/browse/SLIDER-588
Project: Slider
Issue Type: Bug
Affects Versions: Slider 0.60
Reporter: Yesha Vora
Fix For: Slider 0.60
AppsThroughAgentIT.testCreateFlex failed with 'Application registry is not
accessible' error
{code}
AppsThroughAgentIT.testCreateFlex:81->CommandTestBase.ensureRegistryCallSucceeds:690->CommandTestBase.repeatUntilTrue:783->Assert.fail:88
Application registry is not accessible, failing test.
{code}
{code:title=testCreateFlex failure}
java.lang.AssertionError: Application registry is not accessible, failing test.
at org.junit.Assert.fail(Assert.java:88) at
org.apache.slider.funtest.framework.CommandTestBase.repeatUntilTrue(CommandTestBase.groovy:783)
at
org.apache.slider.funtest.framework.CommandTestBase.ensureRegistryCallSucceeds(CommandTestBase.groovy:690)
at
org.apache.slider.funtest.lifecycle.AppsThroughAgentIT.testCreateFlex(AppsThroughAgentIT.groovy:81)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606) at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-588) AppsThroughAgentIT test failed with 'Application registry is not accessible'
[
https://issues.apache.org/jira/browse/SLIDER-588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty updated SLIDER-588:
-
Sprint: Slider October #2
> AppsThroughAgentIT test failed with 'Application registry is not accessible'
>
>
> Key: SLIDER-588
> URL: https://issues.apache.org/jira/browse/SLIDER-588
> Project: Slider
> Issue Type: Bug
>Affects Versions: Slider 0.60
>Reporter: Yesha Vora
> Fix For: Slider 0.60
>
>
> AppsThroughAgentIT.testCreateFlex failed with 'Application registry is not
> accessible' error
> {code}
> AppsThroughAgentIT.testCreateFlex:81->CommandTestBase.ensureRegistryCallSucceeds:690->CommandTestBase.repeatUntilTrue:783->Assert.fail:88
> Application registry is not accessible, failing test.
> {code}
> {code:title=testCreateFlex failure}
> classname="org.apache.slider.funtest.lifecycle.AppsThroughAgentIT"
> time="387.795">
> type="java.lang.AssertionError">
> java.lang.AssertionError: Application registry is not accessible, failing
> test. at org.junit.Assert.fail(Assert.java:88) at
> org.apache.slider.funtest.framework.CommandTestBase.repeatUntilTrue(CommandTestBase.groovy:783)
> at
> org.apache.slider.funtest.framework.CommandTestBase.ensureRegistryCallSucceeds(CommandTestBase.groovy:690)
> at
> org.apache.slider.funtest.lifecycle.AppsThroughAgentIT.testCreateFlex(AppsThroughAgentIT.groovy:81)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-580) Install SSL certs
[ https://issues.apache.org/jira/browse/SLIDER-580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-580: - Fix Version/s: Slider 0.60 > Install SSL certs > - > > Key: SLIDER-580 > URL: https://issues.apache.org/jira/browse/SLIDER-580 > Project: Slider > Issue Type: Improvement >Reporter: Billie Rinaldi >Assignee: Jonathan Maron > Fix For: Slider 0.60 > > > In addition to keytabs, it would be useful to be able to install SSL certs > for localization. We could simply add jks files as a type of file understood > by install-keytab. Although this does lead to the question of whether we'd > want to support installing arbitrary resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-574) document managing port ranges for slider containers
[ https://issues.apache.org/jira/browse/SLIDER-574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-574: - Issue Type: Documentation (was: Bug) > document managing port ranges for slider containers > --- > > Key: SLIDER-574 > URL: https://issues.apache.org/jira/browse/SLIDER-574 > Project: Slider > Issue Type: Documentation > Components: appmaster >Affects Versions: Slider 0.60 >Reporter: Jonathan Maron >Assignee: Jonathan Maron > Fix For: Slider 0.60 > > > Need to document these settings in an existing document or new configuration > document -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-587) Query: how well are dynamic role placement histories maintained?
[ https://issues.apache.org/jira/browse/SLIDER-587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-587: - Fix Version/s: Slider 0.60 > Query: how well are dynamic role placement histories maintained? > > > Key: SLIDER-587 > URL: https://issues.apache.org/jira/browse/SLIDER-587 > Project: Slider > Issue Type: Task > Components: appmaster >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.60 > > > Review the code to make sure that dynamic roles get their placement history > maintained. Maybe even add some tests to validate this -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-586) reject resource configurations with negative node counts
[ https://issues.apache.org/jira/browse/SLIDER-586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-586: - Fix Version/s: Slider 0.60 > reject resource configurations with negative node counts > > > Key: SLIDER-586 > URL: https://issues.apache.org/jira/browse/SLIDER-586 > Project: Slider > Issue Type: Bug > Components: appmaster >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.60 > > Original Estimate: 0.5h > Remaining Estimate: 0.5h > > it should be an error for any configuration with negative node counts to be > accepted. > this check must go in server side so that changes to the resources.json file > always get validated -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: problems when I use hadoop2.6
not seen that before: something went wrong with the HDFS binding. At a guess, I'd say there's some JAR conflict, maybe conflicting versions of hadoop JARS on the classpath or something similar. On 29 October 2014 15:24, Rui Zhang wrote: > Hi, > > I am using the hadoop2.6 I compiled myself. When I run exists command, it > has this error. What's the problem? I have included all the libraries in > the slider_client.xml. > > ~/Slider_Vertica/Linux64/slider-0.51.0/bin/slider exists slider_test > Exception: org.apache.hadoop.ipc.RPC.getProtocolProxy(Ljava/lang/ > Class;JLjava/net/InetSocketAddress;Lorg/apache/hadoop/security/ > UserGroupInformation;Lorg/apache/hadoop/conf/Configuration;Ljavax/net/ > SocketFactory;ILorg/apache/hadoop/io/retry/RetryPolicy; > Ljava/util/concurrent/atomic/AtomicBoolean;)Lorg/apache/ > hadoop/ipc/ProtocolProxy; > 2014-10-29 11:09:28,601 [main] ERROR main.ServiceLauncher - Exception: > org.apache.hadoop.ipc.RPC.getProtocolProxy(Ljava/lang/Class;JLjava/net/ > InetSocketAddress;Lorg/apache/hadoop/security/UserGroupInformation;Lorg/ > apache/hadoop/conf/Configuration;Ljavax/net/SocketFactory;ILorg/apache/ > hadoop/io/retry/RetryPolicy;Ljava/util/concurrent/atomic/ > AtomicBoolean;)Lorg/apache/hadoop/ipc/ProtocolProxy; > java.lang.NoSuchMethodError: org.apache.hadoop.ipc.RPC. > getProtocolProxy(Ljava/lang/Class;JLjava/net/ > InetSocketAddress;Lorg/apache/hadoop/security/UserGroupInformation;Lorg/ > apache/hadoop/conf/Configuration;Ljavax/net/SocketFactory;ILorg/apache/ > hadoop/io/retry/RetryPolicy;Ljava/util/concurrent/atomic/ > AtomicBoolean;)Lorg/apache/hadoop/ipc/ProtocolProxy; > at org.apache.hadoop.hdfs.NameNodeProxies. > createNNProxyWithClientProtocol(NameNodeProxies.java:420) > at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy( > NameNodeProxies.java:316) > at org.apache.hadoop.hdfs.NameNodeProxies.createProxy( > NameNodeProxies.java:178) > at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:658) > at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:592) > at org.apache.hadoop.hdfs.DistributedFileSystem.initialize( > DistributedFileSystem.java:148) > at org.apache.hadoop.fs.FileSystem.createFileSystem( > FileSystem.java:2397) > at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89) > at org.apache.hadoop.fs.FileSystem$Cache.getInternal( > FileSystem.java:2431) > at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2413) > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368) > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:167) > at org.apache.slider.common.tools.CoreFileSystem.( > CoreFileSystem.java:79) > at org.apache.slider.common.tools.SliderFileSystem.( > SliderFileSystem.java:38) > at org.apache.slider.client.SliderClient.initHadoopBinding( > SliderClient.java:436) > at org.apache.slider.client.SliderClient.serviceInit( > SliderClient.java:246) > at org.apache.hadoop.service.AbstractService.init( > AbstractService.java:163) > at org.apache.slider.core.main.ServiceLauncher.launchService( > ServiceLauncher.java:182) > at org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly( > ServiceLauncher.java:473) > at org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit( > ServiceLauncher.java:403) > at org.apache.slider.core.main.ServiceLauncher.serviceMain( > ServiceLauncher.java:628) > at org.apache.slider.Slider.main(Slider.java:49) > 2014-10-29 11:09:28,604 [main] INFO util.ExitUtil - Exiting with status 56 > > Thanks, > Rui > > -- > Rui Zhang > Software engineer Intern > Vertica, an HP Company > rzh...@vertica.com > > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (SLIDER-584) CommandEnvironmentIT failing on HA cluster - assertion failures
[ https://issues.apache.org/jira/browse/SLIDER-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190641#comment-14190641 ] ASF subversion and git services commented on SLIDER-584: Commit e7713527ea61c9da7ce2d20cee431abfe2288989 in incubator-slider's branch refs/heads/feature/SLIDER-584-python-stdout from [~ste...@apache.org] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=e771352 ] SLIDER-584 don't drop the rest of the line before the closedown loop kicks in > CommandEnvironmentIT failing on HA cluster - assertion failures > --- > > Key: SLIDER-584 > URL: https://issues.apache.org/jira/browse/SLIDER-584 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.60 > Environment: HA yarn cluster >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.60 > > > The CommandEnvironmentIT test is failing, assertions that slider.confdir and > slider libdir are in stdout are failing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: problems when I use hadoop2.6
It's the conflicting jars problem. It's already fixed. Thanks. On 10/30/2014 02:52 PM, Steve Loughran wrote: not seen that before: something went wrong with the HDFS binding. At a guess, I'd say there's some JAR conflict, maybe conflicting versions of hadoop JARS on the classpath or something similar. On 29 October 2014 15:24, Rui Zhang wrote: Hi, I am using the hadoop2.6 I compiled myself. When I run exists command, it has this error. What's the problem? I have included all the libraries in the slider_client.xml. ~/Slider_Vertica/Linux64/slider-0.51.0/bin/slider exists slider_test Exception: org.apache.hadoop.ipc.RPC.getProtocolProxy(Ljava/lang/ Class;JLjava/net/InetSocketAddress;Lorg/apache/hadoop/security/ UserGroupInformation;Lorg/apache/hadoop/conf/Configuration;Ljavax/net/ SocketFactory;ILorg/apache/hadoop/io/retry/RetryPolicy; Ljava/util/concurrent/atomic/AtomicBoolean;)Lorg/apache/ hadoop/ipc/ProtocolProxy; 2014-10-29 11:09:28,601 [main] ERROR main.ServiceLauncher - Exception: org.apache.hadoop.ipc.RPC.getProtocolProxy(Ljava/lang/Class;JLjava/net/ InetSocketAddress;Lorg/apache/hadoop/security/UserGroupInformation;Lorg/ apache/hadoop/conf/Configuration;Ljavax/net/SocketFactory;ILorg/apache/ hadoop/io/retry/RetryPolicy;Ljava/util/concurrent/atomic/ AtomicBoolean;)Lorg/apache/hadoop/ipc/ProtocolProxy; java.lang.NoSuchMethodError: org.apache.hadoop.ipc.RPC. getProtocolProxy(Ljava/lang/Class;JLjava/net/ InetSocketAddress;Lorg/apache/hadoop/security/UserGroupInformation;Lorg/ apache/hadoop/conf/Configuration;Ljavax/net/SocketFactory;ILorg/apache/ hadoop/io/retry/RetryPolicy;Ljava/util/concurrent/atomic/ AtomicBoolean;)Lorg/apache/hadoop/ipc/ProtocolProxy; at org.apache.hadoop.hdfs.NameNodeProxies. createNNProxyWithClientProtocol(NameNodeProxies.java:420) at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy( NameNodeProxies.java:316) at org.apache.hadoop.hdfs.NameNodeProxies.createProxy( NameNodeProxies.java:178) at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:658) at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:592) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize( DistributedFileSystem.java:148) at org.apache.hadoop.fs.FileSystem.createFileSystem( FileSystem.java:2397) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89) at org.apache.hadoop.fs.FileSystem$Cache.getInternal( FileSystem.java:2431) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2413) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:167) at org.apache.slider.common.tools.CoreFileSystem.( CoreFileSystem.java:79) at org.apache.slider.common.tools.SliderFileSystem.( SliderFileSystem.java:38) at org.apache.slider.client.SliderClient.initHadoopBinding( SliderClient.java:436) at org.apache.slider.client.SliderClient.serviceInit( SliderClient.java:246) at org.apache.hadoop.service.AbstractService.init( AbstractService.java:163) at org.apache.slider.core.main.ServiceLauncher.launchService( ServiceLauncher.java:182) at org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly( ServiceLauncher.java:473) at org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit( ServiceLauncher.java:403) at org.apache.slider.core.main.ServiceLauncher.serviceMain( ServiceLauncher.java:628) at org.apache.slider.Slider.main(Slider.java:49) 2014-10-29 11:09:28,604 [main] INFO util.ExitUtil - Exiting with status 56 Thanks, Rui -- Rui Zhang Software engineer Intern Vertica, an HP Company rzh...@vertica.com -- Rui Zhang Software engineer Intern Vertica, an HP Company rzh...@vertica.com
[jira] [Commented] (SLIDER-584) CommandEnvironmentIT failing on HA cluster - assertion failures
[ https://issues.apache.org/jira/browse/SLIDER-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190676#comment-14190676 ] Gour Saha commented on SLIDER-584: -- Just wondering why they passed in non-HA. Does HA dump more text to stdout? > CommandEnvironmentIT failing on HA cluster - assertion failures > --- > > Key: SLIDER-584 > URL: https://issues.apache.org/jira/browse/SLIDER-584 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.60 > Environment: HA yarn cluster >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.60 > > > The CommandEnvironmentIT test is failing, assertions that slider.confdir and > slider libdir are in stdout are failing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-584) CommandEnvironmentIT failing on HA cluster - assertion failures
[ https://issues.apache.org/jira/browse/SLIDER-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190748#comment-14190748 ] ASF subversion and git services commented on SLIDER-584: Commit e7713527ea61c9da7ce2d20cee431abfe2288989 in incubator-slider's branch refs/heads/develop from [~ste...@apache.org] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=e771352 ] SLIDER-584 don't drop the rest of the line before the closedown loop kicks in > CommandEnvironmentIT failing on HA cluster - assertion failures > --- > > Key: SLIDER-584 > URL: https://issues.apache.org/jira/browse/SLIDER-584 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.60 > Environment: HA yarn cluster >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.60 > > > The CommandEnvironmentIT test is failing, assertions that slider.confdir and > slider libdir are in stdout are failing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SLIDER-589) No plain text handler for calls to AppMaster UI for registry
Sumit Mohanty created SLIDER-589:
Summary: No plain text handler for calls to AppMaster UI for
registry
Key: SLIDER-589
URL: https://issues.apache.org/jira/browse/SLIDER-589
Project: Slider
Issue Type: Bug
Components: appmaster, client
Affects Versions: Slider 0.50
Reporter: Sumit Mohanty
Assignee: Steve Loughran
Fix For: Slider 0.60
This is a HA setup and "slider registry" command was issued where the response
was a plain/text.
{noformat}
2014-10-30 19:29:16,976 [main] DEBUG retrieve.RegistryRetriever
(RegistryRetriever.java:getConfigurations(155)) - GET
http://ip-172-31-45-191.ec2.internal:37538/ws/v1/slider/publisher/slider
Oct 30, 2014 7:29:17 PM com.sun.jersey.api.client.ClientResponse getEntity
SEVERE: A message body reader for Java class
org.apache.slider.core.registry.docstore.PublishedConfigSet, and Java type
class org.apache.slider.core.registry.docstore.PublishedConfigSet, and MIME
media type text/plain; charset=UTF-8 was not found
Oct 30, 2014 7:29:17 PM com.sun.jersey.api.client.ClientResponse getEntity
SEVERE: The registered message body readers compatible with the MIME media type
are:
text/plain; charset=UTF-8 ->
com.sun.jersey.core.impl.provider.entity.StringProvider
com.sun.jersey.core.impl.provider.entity.ReaderProvider
*/* ->
com.sun.jersey.core.impl.provider.entity.FormProvider
com.sun.jersey.json.impl.provider.entity.JSONJAXBElementProvider$General
com.sun.jersey.json.impl.provider.entity.JSONArrayProvider$General
com.sun.jersey.json.impl.provider.entity.JSONObjectProvider$General
com.sun.jersey.core.impl.provider.entity.StringProvider
com.sun.jersey.core.impl.provider.entity.ByteArrayProvider
com.sun.jersey.core.impl.provider.entity.FileProvider
com.sun.jersey.core.impl.provider.entity.InputStreamProvider
com.sun.jersey.core.impl.provider.entity.DataSourceProvider
com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$General
com.sun.jersey.core.impl.provider.entity.ReaderProvider
com.sun.jersey.core.impl.provider.entity.DocumentProvider
com.sun.jersey.core.impl.provider.entity.SourceProvider$StreamSourceReader
com.sun.jersey.core.impl.provider.entity.SourceProvider$SAXSourceReader
com.sun.jersey.core.impl.provider.entity.SourceProvider$DOMSourceReader
com.sun.jersey.json.impl.provider.entity.JSONRootElementProvider$General
com.sun.jersey.json.impl.provider.entity.JSONListElementProvider$General
com.sun.jersey.json.impl.provider.entity.JacksonProviderProxy
com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$General
com.sun.jersey.core.impl.provider.entity.XMLListElementProvider$General
com.sun.jersey.core.impl.provider.entity.XMLRootObjectProvider$General
com.sun.jersey.core.impl.provider.entity.EntityHolderReader
Exception: A message body reader for Java class
org.apache.slider.core.registry.docstore.PublishedConfigSet, and Java type
class org.apache.slider.core.registry.docstore.PublishedConfigSet, and MIME
media type text/plain; charset=UTF-8 was not found
2014-10-30 19:29:17,032 [main] ERROR main.ServiceLauncher
(ServiceLauncher.java:error(344)) - Exception: A message body reader for Java
class org.apache.slider.core.registry.docstore.PublishedConfigSet, and Java
type class org.apache.slider.core.registry.docstore.PublishedConfigSet, and
MIME media type text/plain; charset=UTF-8 was not found
com.sun.jersey.api.client.ClientHandlerException: A message body reader for
Java class org.apache.slider.core.registry.docstore.PublishedConfigSet, and
Java type class org.apache.slider.core.registry.docstore.PublishedConfigSet,
and MIME media type text/plain; charset=UTF-8 was not found
at
com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:549)
at
com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:506)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:674)
at com.sun.jersey.api.client.WebResource.get(WebResource.java:191)
at
org.apache.slider.core.registry.retrieve.RegistryRetriever.getConfigurations(RegistryRetriever.java:156)
at
org.apache.slider.client.SliderClient.actionRegistryListConfigsYarn(SliderClient.java:2973)
at
org.apache.slider.client.SliderClient.actionRegistry(SliderClient.java:2548)
at org.apache.slider.client.SliderClient.exec(SliderClient.java:396)
at
org.apache.slider.client.SliderClient.runService(SliderClient.java:341)
at
org.apache.slider.core.main.ServiceLauncher.launchService(ServiceLauncher.java:188)
at
org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(ServiceLauncher.java:473)
at
org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(ServiceLauncher.java:403)
at
org.apache.slider.core.main.ServiceLauncher.serviceMain(ServiceLaun
Why command script is written in python?
May I ask why do you choose python to write the command script. And is there any other language? I find difficult to debug because each time if I find an issue I have to change file package upload to hdfs, rerun and I'm not very familiar with python Best Siyuan
Re: Why command script is written in python?
Actually, you can debug it after deploying it once. This will require setting some YARN variables to ensure containers are not released even if the application fails. The agent logs report back the commands being invoked and the parameters used to invoke them. Let me try to write that up on the wiki. *I will need a day or so.* The choice of python was mostly a reflection of going for scripting language that works on Windows and linux. As such it is possible to use any scripting language as long as the agent code is modified to handle various script types. Of course, today the only supported/tested one is python. On Thu, Oct 30, 2014 at 1:55 PM, hsy...@gmail.com wrote: > May I ask why do you choose python to write the command script. And is > there any other language? I find difficult to debug because each time if I > find an issue I have to change file package upload to hdfs, rerun and I'm > not very familiar with python > > > Best > Siyuan > -- thanks Sumit
How to inject my own web service handler into app master?
Hi, I noticed slider app master has an embedded web service server. Is there a way to inject my own web service call into that server? Another question how to you communication between containers(components) and AM(through agent code)? Is there a way to customize the communication/data between container and AM? Is there a detail document about the agent API? Best, Siyuan
Re: Why command script is written in python?
Thanks Sumit, The only concern about script language is you don't know the compile error until runtime. And in fact it's not arbitrary script it depends on the library in slider for example /python/resource_management/core/resources/system.py Best, Siyuan On Thu, Oct 30, 2014 at 2:24 PM, Sumit Mohanty wrote: > Actually, you can debug it after deploying it once. This will require > setting some YARN variables to ensure containers are not released even if > the application fails. The agent logs report back the commands being > invoked and the parameters used to invoke them. Let me try to write that up > on the wiki. *I will need a day or so.* > > The choice of python was mostly a reflection of going for scripting > language that works on Windows and linux. As such it is possible to use any > scripting language as long as the agent code is modified to handle various > script types. Of course, today the only supported/tested one is python. > > On Thu, Oct 30, 2014 at 1:55 PM, hsy...@gmail.com > wrote: > > > May I ask why do you choose python to write the command script. And is > > there any other language? I find difficult to debug because each time > if I > > find an issue I have to change file package upload to hdfs, rerun and I'm > > not very familiar with python > > > > > > Best > > Siyuan > > > > > > -- > thanks > Sumit >
Re: Why command script is written in python?
Agree. In fact, we need to also document how Slider scripts can be debugged during development. If you do not mind can you open a JIRA at https://issues.apache.org/jira/browse/SLIDER describing the requirements of "debugging" during development time and post development. I will use that JIRA to drive the issue. -Sumit On Thu, Oct 30, 2014 at 2:40 PM, hsy...@gmail.com wrote: > Thanks Sumit, > > The only concern about script language is you don't know the compile error > until runtime. And in fact it's not arbitrary script it depends on the > library in slider for example > /python/resource_management/core/resources/system.py > > Best, > Siyuan > > On Thu, Oct 30, 2014 at 2:24 PM, Sumit Mohanty > wrote: > > > Actually, you can debug it after deploying it once. This will require > > setting some YARN variables to ensure containers are not released even if > > the application fails. The agent logs report back the commands being > > invoked and the parameters used to invoke them. Let me try to write that > up > > on the wiki. *I will need a day or so.* > > > > The choice of python was mostly a reflection of going for scripting > > language that works on Windows and linux. As such it is possible to use > any > > scripting language as long as the agent code is modified to handle > various > > script types. Of course, today the only supported/tested one is python. > > > > On Thu, Oct 30, 2014 at 1:55 PM, hsy...@gmail.com > > wrote: > > > > > May I ask why do you choose python to write the command script. And is > > > there any other language? I find difficult to debug because each time > > if I > > > find an issue I have to change file package upload to hdfs, rerun and > I'm > > > not very familiar with python > > > > > > > > > Best > > > Siyuan > > > > > > > > > > > -- > > thanks > > Sumit > > > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Updated] (SLIDER-526) Create a python wrapper and replace the shell script for slider
[ https://issues.apache.org/jira/browse/SLIDER-526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-526: - Summary: Create a python wrapper and replace the shell script for slider (was: Replace slider shell script with slider.py and rename it slider) > Create a python wrapper and replace the shell script for slider > --- > > Key: SLIDER-526 > URL: https://issues.apache.org/jira/browse/SLIDER-526 > Project: Slider > Issue Type: Bug > Components: client >Affects Versions: Slider 0.50 >Reporter: Sumit Mohanty >Assignee: thomas liu > Fix For: Slider 0.60 > > Attachments: slider526.patch, slider526.patch, slider526.patch, > slider526.patch, slider526.patch > > > The goal is to have a single python script that can work on Windows and Linux > and also be able to call commands directly "e..g slider list" without having > to say "python slider list". Of course "python slider list" should also work. > Lets try to make name change on a linux environment and see what issues, if > any, we run into. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-337) Agent error when trying to deploy Accumulo
[
https://issues.apache.org/jira/browse/SLIDER-337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190918#comment-14190918
]
Siyuan Hua commented on SLIDER-337:
---
I may be wrong, but the right solution should be configurable. I'm using
linuxmint. It should belongs to AptProvider. So user should be able to set the
provider if the system cannot recognize
> Agent error when trying to deploy Accumulo
> ---
>
> Key: SLIDER-337
> URL: https://issues.apache.org/jira/browse/SLIDER-337
> Project: Slider
> Issue Type: Bug
> Components: agent-provider
> Environment: Gentoo Linux, develop branch (035ebbd),
> Hadoop-2.6.0-SNAPSHOT (r1619002)
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: Slider 0.60
>
>
> Tried to start an Accumulo cluster:
> {noformat}
> slider create accumulo --image
> hdfs://localhost:8020/slider/agent/slider-agent.tar.gz --template
> /home/elserj/slider/app-packages/accumulo/target/apache-slider-accumulo-1.6.0-app-package-0.40/appConfig.json
> --resources
> /home/elserj/slider/app-packages/accumulo/target/apache-slider-accumulo-1.6.0-app-package-0.40/resources.json
> {noformat}
> SliderAM came up, but no Accumulo procs got started. Looking at the AM
> webapp, the Accumulo master container kept cycling. Dug around in the
> NM/container logs dir, and found the following stack:
> {noformat}
> Traceback (most recent call last):
> File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/container_1408503547578_0002_01_21/app/definition/package/scripts/accumulo_master.py",
> line 24, in
> AccumuloScript('master').execute()
> File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/container_1408503547578_0002_01_21/infra/agent/slider-agent/resource_management/libraries/script/script.py",
> line 114, in execute
> method(env)
> File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/filecache/19/apache-slider-accumulo-1.6.0-app-package-0.40.zip/package/scripts/accumulo_script.py",
> line 82, in start
> self.configure(env) # for security
> File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/filecache/19/apache-slider-accumulo-1.6.0-app-package-0.40.zip/package/scripts/accumulo_script.py",
> line 76, in configure
> setup_conf_dir(name=self.component)
> File
> ".../tmp/nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/filecache/19/apache-slider-accumulo-1.6.0-app-package-0.40.zip/package/scripts/accumulo_configuration.py",
> line 31, in setup_conf_dir
> recursive = True
> File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/container_1408503547578_0002_01_21/infra/agent/slider-agent/resource_management/core/base.py",
> line 148, in __init__
> self.env.run()
> File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/container_1408503547578_0002_01_21/infra/agent/slider-agent/resource_management/core/environment.py",
> line 149, in run
> self.run_action(resource, action)
> File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/container_1408503547578_0002_01_21/infra/agent/slider-agent/resource_management/core/environment.py",
> line 109, in run_action
> resource.provider)
> File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/container_1408503547578_0002_01_21/infra/agent/slider-agent/resource_management/core/providers/__init__.py",
> line 72, in find_provider
> if resource in provider[env.system.os_family]:
> KeyError: 'gentoo base system'
> {noformat}
> I'm rusty on my Python, but I believe this means that in
> {{slider-agent/src/main/python/resource_management/core/providers/__init__.py}}
> {noformat}
> if resource in provider[env.system.os_family]:
> {noformat}
> should be
> {noformat}
> if env.system.os_family in provider:
> if resource in provider[env.system.os_family]:
> {noformat}
> to avoid the Exception when the key doesn't exist (alternatively try/except,
> too), and get down to the {{default}} case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-337) Agent error when trying to deploy Accumulo
[
https://issues.apache.org/jira/browse/SLIDER-337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190925#comment-14190925
]
Josh Elser commented on SLIDER-337:
---
[~hsy541], I think I'd agree with you. This was very much a work around to make
sure that we don't flat-out fail. I'm not familiar enough with how the library
providers are used, but you should feel free to open another issue to make them
configurable. We can have discussion there :)
> Agent error when trying to deploy Accumulo
> ---
>
> Key: SLIDER-337
> URL: https://issues.apache.org/jira/browse/SLIDER-337
> Project: Slider
> Issue Type: Bug
> Components: agent-provider
> Environment: Gentoo Linux, develop branch (035ebbd),
> Hadoop-2.6.0-SNAPSHOT (r1619002)
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: Slider 0.60
>
>
> Tried to start an Accumulo cluster:
> {noformat}
> slider create accumulo --image
> hdfs://localhost:8020/slider/agent/slider-agent.tar.gz --template
> /home/elserj/slider/app-packages/accumulo/target/apache-slider-accumulo-1.6.0-app-package-0.40/appConfig.json
> --resources
> /home/elserj/slider/app-packages/accumulo/target/apache-slider-accumulo-1.6.0-app-package-0.40/resources.json
> {noformat}
> SliderAM came up, but no Accumulo procs got started. Looking at the AM
> webapp, the Accumulo master container kept cycling. Dug around in the
> NM/container logs dir, and found the following stack:
> {noformat}
> Traceback (most recent call last):
> File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/container_1408503547578_0002_01_21/app/definition/package/scripts/accumulo_master.py",
> line 24, in
> AccumuloScript('master').execute()
> File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/container_1408503547578_0002_01_21/infra/agent/slider-agent/resource_management/libraries/script/script.py",
> line 114, in execute
> method(env)
> File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/filecache/19/apache-slider-accumulo-1.6.0-app-package-0.40.zip/package/scripts/accumulo_script.py",
> line 82, in start
> self.configure(env) # for security
> File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/filecache/19/apache-slider-accumulo-1.6.0-app-package-0.40.zip/package/scripts/accumulo_script.py",
> line 76, in configure
> setup_conf_dir(name=self.component)
> File
> ".../tmp/nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/filecache/19/apache-slider-accumulo-1.6.0-app-package-0.40.zip/package/scripts/accumulo_configuration.py",
> line 31, in setup_conf_dir
> recursive = True
> File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/container_1408503547578_0002_01_21/infra/agent/slider-agent/resource_management/core/base.py",
> line 148, in __init__
> self.env.run()
> File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/container_1408503547578_0002_01_21/infra/agent/slider-agent/resource_management/core/environment.py",
> line 149, in run
> self.run_action(resource, action)
> File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/container_1408503547578_0002_01_21/infra/agent/slider-agent/resource_management/core/environment.py",
> line 109, in run_action
> resource.provider)
> File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/container_1408503547578_0002_01_21/infra/agent/slider-agent/resource_management/core/providers/__init__.py",
> line 72, in find_provider
> if resource in provider[env.system.os_family]:
> KeyError: 'gentoo base system'
> {noformat}
> I'm rusty on my Python, but I believe this means that in
> {{slider-agent/src/main/python/resource_management/core/providers/__init__.py}}
> {noformat}
> if resource in provider[env.system.os_family]:
> {noformat}
> should be
> {noformat}
> if env.system.os_family in provider:
> if resource in provider[env.system.os_family]:
> {noformat}
> to avoid the Exception when the key doesn't exist (alternatively try/except,
> too), and get down to the {{default}} case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (SLIDER-590) Need some convinient debug ability of command script during/post development time
Siyuan Hua created SLIDER-590: - Summary: Need some convinient debug ability of command script during/post development time Key: SLIDER-590 URL: https://issues.apache.org/jira/browse/SLIDER-590 Project: Slider Issue Type: Bug Reporter: Siyuan Hua Nowadays, whatever errors I made in command script to launch the component, I have to modify the python code repackage and redeployed to hdfs and relaunch the application. It's quite inconvenient. I hope there is a way to run, test and debug the command script right after they are installed on target machine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Why command script is written in python?
Logged a ticket here, https://issues.apache.org/jira/browse/SLIDER-590 feel free to rephrase if anything I said is wrong. Thanks! Best On Thu, Oct 30, 2014 at 2:45 PM, Sumit Mohanty wrote: > Agree. In fact, we need to also document how Slider scripts can be debugged > during development. > > If you do not mind can you open a JIRA at > https://issues.apache.org/jira/browse/SLIDER describing the requirements > of > "debugging" during development time and post development. I will use that > JIRA to drive the issue. > > -Sumit > > On Thu, Oct 30, 2014 at 2:40 PM, hsy...@gmail.com > wrote: > > > Thanks Sumit, > > > > The only concern about script language is you don't know the compile > error > > until runtime. And in fact it's not arbitrary script it depends on the > > library in slider for example > > /python/resource_management/core/resources/system.py > > > > Best, > > Siyuan > > > > On Thu, Oct 30, 2014 at 2:24 PM, Sumit Mohanty > > wrote: > > > > > Actually, you can debug it after deploying it once. This will require > > > setting some YARN variables to ensure containers are not released even > if > > > the application fails. The agent logs report back the commands being > > > invoked and the parameters used to invoke them. Let me try to write > that > > up > > > on the wiki. *I will need a day or so.* > > > > > > The choice of python was mostly a reflection of going for scripting > > > language that works on Windows and linux. As such it is possible to use > > any > > > scripting language as long as the agent code is modified to handle > > various > > > script types. Of course, today the only supported/tested one is python. > > > > > > On Thu, Oct 30, 2014 at 1:55 PM, hsy...@gmail.com > > > wrote: > > > > > > > May I ask why do you choose python to write the command script. And > is > > > > there any other language? I find difficult to debug because each > time > > > if I > > > > find an issue I have to change file package upload to hdfs, rerun and > > I'm > > > > not very familiar with python > > > > > > > > > > > > Best > > > > Siyuan > > > > > > > > > > > > > > > > -- > > > thanks > > > Sumit > > > > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
Re: [jira] [Commented] (SLIDER-337) Agent error when trying to deploy Accumulo
Yes, lets open a JIRA. I also need to understand what it means to make it
configurable.
On Thu, Oct 30, 2014 at 3:04 PM, Josh Elser (JIRA) wrote:
>
> [
> https://issues.apache.org/jira/browse/SLIDER-337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190925#comment-14190925
> ]
>
> Josh Elser commented on SLIDER-337:
> ---
>
> [~hsy541], I think I'd agree with you. This was very much a work around to
> make sure that we don't flat-out fail. I'm not familiar enough with how the
> library providers are used, but you should feel free to open another issue
> to make them configurable. We can have discussion there :)
>
> > Agent error when trying to deploy Accumulo
> > ---
> >
> > Key: SLIDER-337
> > URL: https://issues.apache.org/jira/browse/SLIDER-337
> > Project: Slider
> > Issue Type: Bug
> > Components: agent-provider
> > Environment: Gentoo Linux, develop branch (035ebbd),
> Hadoop-2.6.0-SNAPSHOT (r1619002)
> >Reporter: Josh Elser
> >Assignee: Josh Elser
> > Fix For: Slider 0.60
> >
> >
> > Tried to start an Accumulo cluster:
> > {noformat}
> > slider create accumulo --image
> hdfs://localhost:8020/slider/agent/slider-agent.tar.gz --template
> /home/elserj/slider/app-packages/accumulo/target/apache-slider-accumulo-1.6.0-app-package-0.40/appConfig.json
> --resources
> /home/elserj/slider/app-packages/accumulo/target/apache-slider-accumulo-1.6.0-app-package-0.40/resources.json
> > {noformat}
> > SliderAM came up, but no Accumulo procs got started. Looking at the AM
> webapp, the Accumulo master container kept cycling. Dug around in the
> NM/container logs dir, and found the following stack:
> > {noformat}
> > Traceback (most recent call last):
> > File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/container_1408503547578_0002_01_21/app/definition/package/scripts/accumulo_master.py",
> line 24, in
> > AccumuloScript('master').execute()
> > File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/container_1408503547578_0002_01_21/infra/agent/slider-agent/resource_management/libraries/script/script.py",
> line 114, in execute
> > method(env)
> > File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/filecache/19/apache-slider-accumulo-1.6.0-app-package-0.40.zip/package/scripts/accumulo_script.py",
> line 82, in start
> > self.configure(env) # for security
> > File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/filecache/19/apache-slider-accumulo-1.6.0-app-package-0.40.zip/package/scripts/accumulo_script.py",
> line 76, in configure
> > setup_conf_dir(name=self.component)
> > File
> ".../tmp/nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/filecache/19/apache-slider-accumulo-1.6.0-app-package-0.40.zip/package/scripts/accumulo_configuration.py",
> line 31, in setup_conf_dir
> > recursive = True
> > File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/container_1408503547578_0002_01_21/infra/agent/slider-agent/resource_management/core/base.py",
> line 148, in __init__
> > self.env.run()
> > File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/container_1408503547578_0002_01_21/infra/agent/slider-agent/resource_management/core/environment.py",
> line 149, in run
> > self.run_action(resource, action)
> > File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/container_1408503547578_0002_01_21/infra/agent/slider-agent/resource_management/core/environment.py",
> line 109, in run_action
> > resource.provider)
> > File
> ".../nm-local-dir/usercache/elserj/appcache/application_1408503547578_0002/container_1408503547578_0002_01_21/infra/agent/slider-agent/resource_management/core/providers/__init__.py",
> line 72, in find_provider
> > if resource in provider[env.system.os_family]:
> > KeyError: 'gentoo base system'
> > {noformat}
> > I'm rusty on my Python, but I believe this means that in
> {{slider-agent/src/main/python/resource_management/core/providers/__init__.py}}
> > {noformat}
> > if resource in provider[env.system.os_family]:
> > {noformat}
> > should be
> > {noformat}
> > if env.system.os_family in provider:
> > if resource in provider[env.system.os_family]:
> > {noformat}
> > to avoid the Exception when the key doesn't exist (alternatively
> try/except, too), and get down to the {{default}} case.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicabl
[jira] [Updated] (SLIDER-590) Need some convinient debug ability of command script during/post development time
[ https://issues.apache.org/jira/browse/SLIDER-590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-590: - Fix Version/s: Slider 2.0.0 > Need some convinient debug ability of command script during/post development > time > -- > > Key: SLIDER-590 > URL: https://issues.apache.org/jira/browse/SLIDER-590 > Project: Slider > Issue Type: Bug >Reporter: Siyuan Hua > Fix For: Slider 2.0.0 > > > Nowadays, whatever errors I made in command script to launch the component, I > have to modify the python code repackage and redeployed to hdfs and relaunch > the application. It's quite inconvenient. I hope there is a way to run, test > and debug the command script right after they are installed on target > machine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (SLIDER-590) Need some convinient debug ability of command script during/post development time
[ https://issues.apache.org/jira/browse/SLIDER-590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty reassigned SLIDER-590: Assignee: Sumit Mohanty > Need some convinient debug ability of command script during/post development > time > -- > > Key: SLIDER-590 > URL: https://issues.apache.org/jira/browse/SLIDER-590 > Project: Slider > Issue Type: Bug >Reporter: Siyuan Hua >Assignee: Sumit Mohanty > Fix For: Slider 2.0.0 > > > Nowadays, whatever errors I made in command script to launch the component, I > have to modify the python code repackage and redeployed to hdfs and relaunch > the application. It's quite inconvenient. I hope there is a way to run, test > and debug the command script right after they are installed on target > machine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-590) Need some convinient debug ability of command script during/post development time
[ https://issues.apache.org/jira/browse/SLIDER-590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190938#comment-14190938 ] Sumit Mohanty commented on SLIDER-590: -- There are two requirements covered by this JIRA * While developing command scripts how can the application developer test their scripts without having to create a full application package. * After a reasonably stable application package is developed how can the command scripts be debugged without going through the full re-package, create, debug cycle > Need some convinient debug ability of command script during/post development > time > -- > > Key: SLIDER-590 > URL: https://issues.apache.org/jira/browse/SLIDER-590 > Project: Slider > Issue Type: Bug >Reporter: Siyuan Hua >Assignee: Sumit Mohanty > Fix For: Slider 2.0.0 > > > Nowadays, whatever errors I made in command script to launch the component, I > have to modify the python code repackage and redeployed to hdfs and relaunch > the application. It's quite inconvenient. I hope there is a way to run, test > and debug the command script right after they are installed on target > machine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-590) Need some convenient debugability of command script during/post development time
[ https://issues.apache.org/jira/browse/SLIDER-590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SLIDER-590: -- Summary: Need some convenient debugability of command script during/post development time (was: Need some convinient debug ability of command script during/post development time) > Need some convenient debugability of command script during/post development > time > > > Key: SLIDER-590 > URL: https://issues.apache.org/jira/browse/SLIDER-590 > Project: Slider > Issue Type: Bug >Reporter: Siyuan Hua >Assignee: Sumit Mohanty > Fix For: Slider 2.0.0 > > > Nowadays, whatever errors I made in command script to launch the component, I > have to modify the python code repackage and redeployed to hdfs and relaunch > the application. It's quite inconvenient. I hope there is a way to run, test > and debug the command script right after they are installed on target > machine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-572) add startup delay for the chaos monkey
[ https://issues.apache.org/jira/browse/SLIDER-572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190966#comment-14190966 ] ASF subversion and git services commented on SLIDER-572: Commit 1635632 from [~ste...@apache.org] in branch 'site/trunk' [ https://svn.apache.org/r1635632 ] SLIDER-572 add startup delay for the chaos monkey > add startup delay for the chaos monkey > -- > > Key: SLIDER-572 > URL: https://issues.apache.org/jira/browse/SLIDER-572 > Project: Slider > Issue Type: Sub-task > Components: appmaster, test >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.60 > > > for functional tests we need to be able to turn the monkey on or off; this > would let us set up a test cluster into a functional state and then trigger > the chaos sequence. > Currently enabling AM killing in chaos monkey can cause tests to fail as the > test runner thinks the cluster startup failed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLIDER-582) Registry test may fail if ~ expands to a path with upper case
[
https://issues.apache.org/jira/browse/SLIDER-582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved SLIDER-582.
---
Resolution: Fixed
> Registry test may fail if ~ expands to a path with upper case
> -
>
> Key: SLIDER-582
> URL: https://issues.apache.org/jira/browse/SLIDER-582
> Project: Slider
> Issue Type: Bug
> Components: registry
>Affects Versions: Slider 0.50
>Reporter: Sumit Mohanty
>Assignee: Steve Loughran
> Fix For: Slider 0.60
>
>
> {noformat}
> 2014-10-29 22:51:10,667 [JUnit] INFO framework.SliderShell (?:call(?)) -
> /Users/smohanty/enlistments/incubator-slider/slider-assembly/target/slider-0.51.0-incubating-SNAPSHOT-all/slider-0.51.0-incubating-SNAPSHOT/bin/slider
> resolve --list --path \~
> 2014-10-29 22:51:10,667 [JUnit] DEBUG framework.SliderShell (?:call(?)) -
> export
> SLIDER_CONF_DIR=/Users/smohanty/enlistments/incubator-slider/src/test/clusters/remote/slider;
> /Users/smohanty/enlistments/incubator-slider/slider-assembly/target/slider-0.51.0-incubating-SNAPSHOT-all/slider-0.51.0-incubating-SNAPSHOT/bin/slider
> resolve --list --path \~
> 2014-10-29 22:51:12,765 [JUnit] ERROR framework.SliderShell (?:call(?)) - 56
> =>/Users/smohanty/enlistments/incubator-slider/slider-assembly/target/slider-0.51.0-incubating-SNAPSHOT-all/slider-0.51.0-incubating-SNAPSHOT/bin/slider
> resolve --list --path \~
> 2014-10-29 22:51:12,765 [JUnit] ERROR framework.SliderShell
> (NativeMethodAccessorImpl.java:invoke0(?)) - return code = 56
> 2014-10-29 22:51:12,765 [JUnit] INFO framework.SliderShell
> (NativeMethodAccessorImpl.java:invoke0(?)) -
> {noformat}
> {noformat}
> 2014-10-29 22:51:12,652 [main] ERROR main.ServiceLauncher
> (ServiceLauncher.java:error(344)) - Exception: `/Users/smohanty': Invalid
> Path element "Users"
> org.apache.hadoop.registry.client.exceptions.InvalidPathnameException:
> `/Users/smohanty': Invalid Path element "Users"
> at
> org.apache.hadoop.registry.client.binding.RegistryPathUtils.validateElementsAsDNS(RegistryPathUtils.java:78)
> at
> org.apache.hadoop.registry.client.impl.zk.RegistryOperationsService.validatePath(RegistryOperationsService.java:91)
> at
> org.apache.hadoop.registry.client.impl.zk.RegistryOperationsService.list(RegistryOperationsService.java:145)
> at
> org.apache.hadoop.registry.client.binding.RegistryUtils.statChildren(RegistryUtils.java:178)
> at
> org.apache.slider.client.SliderClient.actionResolve(SliderClient.java:2475)
> at org.apache.slider.client.SliderClient.exec(SliderClient.java:398)
> at
> org.apache.slider.client.SliderClient.runService(SliderClient.java:341)
> at
> org.apache.slider.core.main.ServiceLauncher.launchService(ServiceLauncher.java:188)
> at
> org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(ServiceLauncher.java:473)
> at
> org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(ServiceLauncher.java:403)
> at
> org.apache.slider.core.main.ServiceLauncher.serviceMain(ServiceLauncher.java:628)
> at org.apache.slider.Slider.main(Slider.java:49)
> 2014-10-29 22:51:12,655 [main] INFO util.ExitUtil
> (ExitUtil.java:terminate(124)) - Exiting with status 56
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (SLIDER-573) AgentIT test to include GET of / of web UI
[ https://issues.apache.org/jira/browse/SLIDER-573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved SLIDER-573. --- Resolution: Fixed > AgentIT test to include GET of / of web UI > -- > > Key: SLIDER-573 > URL: https://issues.apache.org/jira/browse/SLIDER-573 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.60 >Reporter: Steve Loughran >Assignee: Sumit Mohanty > Fix For: Slider 0.60 > > > Extend the AgentLifecycleIT test with a check for the web UI page being > retrievable. > needs to determine the AM UI (registry resolve) then a get -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SLIDER-591) regression: ListCommandIT broken with changed exit codes
Steve Loughran created SLIDER-591:
-
Summary: regression: ListCommandIT broken with changed exit codes
Key: SLIDER-591
URL: https://issues.apache.org/jira/browse/SLIDER-591
Project: Slider
Issue Type: Sub-task
Affects Versions: Slider 0.60
Reporter: Steve Loughran
Assignee: Steve Loughran
SLIDER-306 changed the exit codes of the list command; as a result the
{{ListCommandIT}} test is failing
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-591) regression: ListCommandIT broken with changed exit codes
[
https://issues.apache.org/jira/browse/SLIDER-591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191004#comment-14191004
]
ASF subversion and git services commented on SLIDER-591:
Commit fdcc5c2cfaa60d4a80506dc9e141e824a5fdc1b2 in incubator-slider's branch
refs/heads/develop from [~ste...@apache.org]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=fdcc5c2 ]
SLIDER-591 regression: ListCommandIT broken with changed exit codes
> regression: ListCommandIT broken with changed exit codes
>
>
> Key: SLIDER-591
> URL: https://issues.apache.org/jira/browse/SLIDER-591
> Project: Slider
> Issue Type: Sub-task
> Components: build
>Affects Versions: Slider 0.60
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Fix For: Slider 0.60
>
>
> SLIDER-306 changed the exit codes of the list command; as a result the
> {{ListCommandIT}} test is failing
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-263) AM should not persist keystore password
[ https://issues.apache.org/jira/browse/SLIDER-263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Maron updated SLIDER-263: -- Summary: AM should not persist keystore password (was: leverage CredentialProvider for password and possibly passphrase retrieval for agent SSL communication) > AM should not persist keystore password > --- > > Key: SLIDER-263 > URL: https://issues.apache.org/jira/browse/SLIDER-263 > Project: Slider > Issue Type: Bug >Reporter: Jonathan Maron >Assignee: Jonathan Maron > Fix For: Slider 0.60 > > > Much like the accumulo approach in SLIDER-254, look at using the credential > provider in the context of establishing SSL communication between the App > Master and agents in spawned containers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-526) Create a python wrapper and replace the shell script for slider
[ https://issues.apache.org/jira/browse/SLIDER-526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191034#comment-14191034 ] ASF subversion and git services commented on SLIDER-526: Commit d03bf28e84b155332f4b78ff3ff412c9982d2ef4 in incubator-slider's branch refs/heads/develop from [~sumitmohanty] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=d03bf28 ] SLIDER-526. Create a python wrapper and replace the shell script for slider (Thomas Liu via smohanty) > Create a python wrapper and replace the shell script for slider > --- > > Key: SLIDER-526 > URL: https://issues.apache.org/jira/browse/SLIDER-526 > Project: Slider > Issue Type: Bug > Components: client >Affects Versions: Slider 0.50 >Reporter: Sumit Mohanty >Assignee: thomas liu > Fix For: Slider 0.60 > > Attachments: slider526.patch, slider526.patch, slider526.patch, > slider526.patch, slider526.patch > > > The goal is to have a single python script that can work on Windows and Linux > and also be able to call commands directly "e..g slider list" without having > to say "python slider list". Of course "python slider list" should also work. > Lets try to make name change on a linux environment and see what issues, if > any, we run into. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-263) AM should not persist keystore password
[ https://issues.apache.org/jira/browse/SLIDER-263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191059#comment-14191059 ] Jonathan Maron commented on SLIDER-263: --- It turns out that persisting the password to a file does prove useful for testing (it allows the use of the generated file to be leveraged as keystore by the AM and truststore by a test client). Therefore, the approach taken is that the AM does not persist the password. However, if a password file is found it will use it to read the password. That way a test client can generate a keystore and associated password that will be utilized by both the AM and the test client. But this will really only work in a situation where the test client and AM are running on the same host (unit test). > AM should not persist keystore password > --- > > Key: SLIDER-263 > URL: https://issues.apache.org/jira/browse/SLIDER-263 > Project: Slider > Issue Type: Bug >Reporter: Jonathan Maron >Assignee: Jonathan Maron > Fix For: Slider 0.60 > > > Much like the accumulo approach in SLIDER-254, look at using the credential > provider in the context of establishing SSL communication between the App > Master and agents in spawned containers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
2 questions
Hi all, I have 2 questions, 1. Is there a way to extend the REST API(add some customized api)? 2. Is there a way to add/remove component/component instance at the runtime? Thanks! Best, Siyuan
[jira] [Commented] (SLIDER-263) AM should not persist keystore password
[ https://issues.apache.org/jira/browse/SLIDER-263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191156#comment-14191156 ] ASF subversion and git services commented on SLIDER-263: Commit 1a071e31dd2f2c4486bcbdfe2f02065efe636545 in incubator-slider's branch refs/heads/develop from [~jmaron] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=1a071e3 ] SLIDER-263 AM no longer persists keystore password > AM should not persist keystore password > --- > > Key: SLIDER-263 > URL: https://issues.apache.org/jira/browse/SLIDER-263 > Project: Slider > Issue Type: Bug >Reporter: Jonathan Maron >Assignee: Jonathan Maron > Fix For: Slider 0.60 > > > Much like the accumulo approach in SLIDER-254, look at using the credential > provider in the context of establishing SSL communication between the App > Master and agents in spawned containers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLIDER-263) AM should no longer persist keystore password
[ https://issues.apache.org/jira/browse/SLIDER-263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Maron resolved SLIDER-263. --- Resolution: Fixed > AM should no longer persist keystore password > - > > Key: SLIDER-263 > URL: https://issues.apache.org/jira/browse/SLIDER-263 > Project: Slider > Issue Type: Bug >Reporter: Jonathan Maron >Assignee: Jonathan Maron > Fix For: Slider 0.60 > > > Much like the accumulo approach in SLIDER-254, look at using the credential > provider in the context of establishing SSL communication between the App > Master and agents in spawned containers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-263) AM should no longer persist keystore password
[ https://issues.apache.org/jira/browse/SLIDER-263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Maron updated SLIDER-263: -- Summary: AM should no longer persist keystore password (was: AM should not persist keystore password) > AM should no longer persist keystore password > - > > Key: SLIDER-263 > URL: https://issues.apache.org/jira/browse/SLIDER-263 > Project: Slider > Issue Type: Bug >Reporter: Jonathan Maron >Assignee: Jonathan Maron > Fix For: Slider 0.60 > > > Much like the accumulo approach in SLIDER-254, look at using the credential > provider in the context of establishing SSL communication between the App > Master and agents in spawned containers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLIDER-574) document managing port ranges for slider containers
[ https://issues.apache.org/jira/browse/SLIDER-574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Maron resolved SLIDER-574. --- Resolution: Fixed > document managing port ranges for slider containers > --- > > Key: SLIDER-574 > URL: https://issues.apache.org/jira/browse/SLIDER-574 > Project: Slider > Issue Type: Documentation > Components: appmaster >Affects Versions: Slider 0.60 >Reporter: Jonathan Maron >Assignee: Jonathan Maron > Fix For: Slider 0.60 > > > Need to document these settings in an existing document or new configuration > document -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-574) document managing port ranges for slider containers
[ https://issues.apache.org/jira/browse/SLIDER-574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191172#comment-14191172 ] ASF subversion and git services commented on SLIDER-574: Commit e23e680fe8e329410126ff4a83ad004ce91b528a in incubator-slider's branch refs/heads/develop from [~jmaron] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=e23e680 ] SLIDER-574 print log message when no service keytabs are localized > document managing port ranges for slider containers > --- > > Key: SLIDER-574 > URL: https://issues.apache.org/jira/browse/SLIDER-574 > Project: Slider > Issue Type: Documentation > Components: appmaster >Affects Versions: Slider 0.60 >Reporter: Jonathan Maron >Assignee: Jonathan Maron > Fix For: Slider 0.60 > > > Need to document these settings in an existing document or new configuration > document -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (SLIDER-574) document managing port ranges for slider containers
[ https://issues.apache.org/jira/browse/SLIDER-574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191172#comment-14191172 ] Jonathan Maron edited comment on SLIDER-574 at 10/31/14 1:34 AM: - Removed erroneous commit message was (Author: jira-bot): Commit e23e680fe8e329410126ff4a83ad004ce91b528a in incubator-slider's branch refs/heads/develop from [~jmaron] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=e23e680 ] SLIDER-574 print log message when no service keytabs are localized > document managing port ranges for slider containers > --- > > Key: SLIDER-574 > URL: https://issues.apache.org/jira/browse/SLIDER-574 > Project: Slider > Issue Type: Documentation > Components: appmaster >Affects Versions: Slider 0.60 >Reporter: Jonathan Maron >Assignee: Jonathan Maron > Fix For: Slider 0.60 > > > Need to document these settings in an existing document or new configuration > document -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (SLIDER-574) document managing port ranges for slider containers
[ https://issues.apache.org/jira/browse/SLIDER-574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191172#comment-14191172 ] Jonathan Maron edited comment on SLIDER-574 at 10/31/14 1:34 AM: - Commit e23e680fe8e329410126ff4a83ad004ce91b528a in incubator-slider's branch refs/heads/develop from [~jmaron] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=e23e680 ] SLIDER-574 print log message when no service keytabs are localized was (Author: jira-bot): Commit e23e680fe8e329410126ff4a83ad004ce91b528a in incubator-slider's branch refs/heads/develop from [~jmaron] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=e23e680 ] SLIDER-574 print log message when no service keytabs are localized > document managing port ranges for slider containers > --- > > Key: SLIDER-574 > URL: https://issues.apache.org/jira/browse/SLIDER-574 > Project: Slider > Issue Type: Documentation > Components: appmaster >Affects Versions: Slider 0.60 >Reporter: Jonathan Maron >Assignee: Jonathan Maron > Fix For: Slider 0.60 > > > Need to document these settings in an existing document or new configuration > document -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (SLIDER-574) document managing port ranges for slider containers
[ https://issues.apache.org/jira/browse/SLIDER-574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Maron reopened SLIDER-574: --- > document managing port ranges for slider containers > --- > > Key: SLIDER-574 > URL: https://issues.apache.org/jira/browse/SLIDER-574 > Project: Slider > Issue Type: Documentation > Components: appmaster >Affects Versions: Slider 0.60 >Reporter: Jonathan Maron >Assignee: Jonathan Maron > Fix For: Slider 0.60 > > > Need to document these settings in an existing document or new configuration > document -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-577) Need a warning message when no service keytabs are localized
[ https://issues.apache.org/jira/browse/SLIDER-577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191175#comment-14191175 ] Jonathan Maron commented on SLIDER-577: --- Commit e23e680fe8e329410126ff4a83ad004ce91b528a in incubator-slider's branch refs/heads/develop from [~jmaron] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=e23e680 ] SLIDER-577 print log message when no service keytabs are localized > Need a warning message when no service keytabs are localized > > > Key: SLIDER-577 > URL: https://issues.apache.org/jira/browse/SLIDER-577 > Project: Slider > Issue Type: Bug >Affects Versions: Slider 0.60 >Reporter: Jonathan Maron >Assignee: Jonathan Maron > > there should be a warning message in the AM log indicating that no service > keytabs have been localized If leveraging HDFS installed keytabs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLIDER-577) Need a warning message when no service keytabs are localized
[ https://issues.apache.org/jira/browse/SLIDER-577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Maron resolved SLIDER-577. --- Resolution: Fixed > Need a warning message when no service keytabs are localized > > > Key: SLIDER-577 > URL: https://issues.apache.org/jira/browse/SLIDER-577 > Project: Slider > Issue Type: Bug >Affects Versions: Slider 0.60 >Reporter: Jonathan Maron >Assignee: Jonathan Maron > > there should be a warning message in the AM log indicating that no service > keytabs have been localized If leveraging HDFS installed keytabs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: git commit: SLIDER-574 print log message when no service keytabs are localized
Mistakenly associated with SLIDER-574. Should be associated with SLIDER-577.
On Oct 30, 2014, at 9:31 PM, jma...@apache.org wrote:
> Repository: incubator-slider
> Updated Branches:
> refs/heads/develop 1a071e31d -> e23e680fe
>
>
> SLIDER-574 print log message when no service keytabs are localized
>
>
> Project: http://git-wip-us.apache.org/repos/asf/incubator-slider/repo
> Commit:
> http://git-wip-us.apache.org/repos/asf/incubator-slider/commit/e23e680f
> Tree: http://git-wip-us.apache.org/repos/asf/incubator-slider/tree/e23e680f
> Diff: http://git-wip-us.apache.org/repos/asf/incubator-slider/diff/e23e680f
>
> Branch: refs/heads/develop
> Commit: e23e680fe8e329410126ff4a83ad004ce91b528a
> Parents: 1a071e3
> Author: Jon Maron
> Authored: Thu Oct 30 21:31:41 2014 -0400
> Committer: Jon Maron
> Committed: Thu Oct 30 21:31:41 2014 -0400
>
> --
> .../apache/slider/providers/agent/AgentProviderService.java | 9 +
> 1 file changed, 9 insertions(+)
> --
>
>
> http://git-wip-us.apache.org/repos/asf/incubator-slider/blob/e23e680f/slider-core/src/main/java/org/apache/slider/providers/agent/AgentProviderService.java
> --
> diff --git
> a/slider-core/src/main/java/org/apache/slider/providers/agent/AgentProviderService.java
>
> b/slider-core/src/main/java/org/apache/slider/providers/agent/AgentProviderService.java
> index 934e5f0..4720ced 100644
> ---
> a/slider-core/src/main/java/org/apache/slider/providers/agent/AgentProviderService.java
> +++
> b/slider-core/src/main/java/org/apache/slider/providers/agent/AgentProviderService.java
> @@ -521,9 +521,11 @@ public class AgentProviderService extends
> AbstractProviderService implements
> getClusterName());
> FileStatus[] keytabs =
> fileSystem.getFileSystem().listStatus(keytabDirPath);
> LocalResource keytabRes;
> + boolean serviceKeytabsDeployed = false;
> for (FileStatus keytab : keytabs) {
> if (!amKeytabName.equals(keytab.getPath().getName())
> && keytab.getPath().getName().endsWith(".keytab")) {
> + serviceKeytabsDeployed = true;
> log.info("Localizing keytab {}", keytab.getPath().getName());
> keytabRes = fileSystem.createAmResource(keytab.getPath(),
> LocalResourceType.FILE);
> @@ -532,6 +534,13 @@ public class AgentProviderService extends
> AbstractProviderService implements
> keytabRes);
> }
> }
> + if (!serviceKeytabsDeployed) {
> +log.warn("No service keytabs for the application have been
> localized. "
> + + "If the application requires keytabs for secure
> operation, "
> + + "please ensure that the required keytabs have been
> uploaded "
> + + "to the folder designated by the property {}: {}",
> + SliderXmlConfKeys.KEY_HDFS_KEYTAB_DIR, keytabDirPath);
> + }
> }
> }
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
