[jira] [Assigned] (SLIDER-466) Slider should allow specification of JAVA_HOME as a token that should get automatically replaced
[ https://issues.apache.org/jira/browse/SLIDER-466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty reassigned SLIDER-466: Assignee: (was: Sumit Mohanty) > Slider should allow specification of JAVA_HOME as a token that should get > automatically replaced > > > Key: SLIDER-466 > URL: https://issues.apache.org/jira/browse/SLIDER-466 > Project: Slider > Issue Type: Bug > Components: app-package >Affects Versions: Slider 0.50 >Reporter: Sumit Mohanty > Fix For: Slider 2.0.0 > > > Slider app packages should allow specification of JAVA_HOME as a config > property that can be replaced by AppMaster (AgentProviderService) or the > Agent while creating the component instances. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SLIDER-402) Add support for general purpose cross referencing of named properties
[
https://issues.apache.org/jira/browse/SLIDER-402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty reassigned SLIDER-402:
Assignee: (was: Sumit Mohanty)
> Add support for general purpose cross referencing of named properties
> -
>
> Key: SLIDER-402
> URL: https://issues.apache.org/jira/browse/SLIDER-402
> Project: Slider
> Issue Type: Bug
> Components: agent-provider, app-package
>Affects Versions: Slider 0.50
>Reporter: Sumit Mohanty
> Fix For: Slider 2.0.0
>
>
> We need to make sure that whatever syntax we have for cross referencing
> continues to work even as we look at adding more sources of configuration
> data.
> *examples*: sysprops, late binding information from the registry to link to
> other components.
> We may need to do something more typed, eg
> {attr:property}
> or even maybe {@property}, {@/global/property} to indicate attributes. And
> maybe even cross reference internal/resources
> size:"{@//resources/components/regions/yarn.component.instances}"
> Other references we could do with another syntax in the {}. Example
> {env:Path}, {sysprop:java.home}
> We'll need to allow for the { to be escaped somehow.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (SLIDER-341) Add a window based failure count for auto-start to limit indefinite attempt
[ https://issues.apache.org/jira/browse/SLIDER-341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty reassigned SLIDER-341: Assignee: (was: Sumit Mohanty) > Add a window based failure count for auto-start to limit indefinite attempt > --- > > Key: SLIDER-341 > URL: https://issues.apache.org/jira/browse/SLIDER-341 > Project: Slider > Issue Type: Bug > Components: agent, app-package >Affects Versions: Slider 0.50 >Reporter: Sumit Mohanty > Fix For: Slider 2.0.0 > > > The agent's auto-restart feature for component instance should be gated by a > window based upper limit. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SLIDER-261) Improvements to port allocation support
[ https://issues.apache.org/jira/browse/SLIDER-261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty reassigned SLIDER-261: Assignee: (was: Sumit Mohanty) > Improvements to port allocation support > --- > > Key: SLIDER-261 > URL: https://issues.apache.org/jira/browse/SLIDER-261 > Project: Slider > Issue Type: New Feature > Components: agent, agent-provider >Affects Versions: Slider 0.50 >Reporter: Sumit Mohanty > Fix For: Slider 2.0.0 > > > Applications like Storm require dynamic port allocation. However, the > allocated port may not get immediately used - e.g. till a Topology is > deployed. In general, there is always a small window between Slider > allocating a port and the application actually using it. > The ideal solution is some form of port reservation system that is OS backed. > But in absence of such a capability Slider should ensure that allocated port > is reserved for the application instance till the application is active > independent of whether the application actually uses it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SLIDER-259) Ensure resource management library supports all operations on Windows
[ https://issues.apache.org/jira/browse/SLIDER-259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty reassigned SLIDER-259: Assignee: (was: Sumit Mohanty) > Ensure resource management library supports all operations on Windows > - > > Key: SLIDER-259 > URL: https://issues.apache.org/jira/browse/SLIDER-259 > Project: Slider > Issue Type: Sub-task > Components: agent, app-package >Affects Versions: Slider 0.50 >Reporter: Sumit Mohanty > > The platform specific operations for an application package are available in > the resource_management library. The library needs Windows specific providers > for operations that are platform specific - e.g. modify file/folder > permission, execute scripts, handle tarball, etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SLIDER-186) Provide a mechanism for a provider to know if an application is being created for the first time
[ https://issues.apache.org/jira/browse/SLIDER-186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty reassigned SLIDER-186: Assignee: (was: Sumit Mohanty) > Provide a mechanism for a provider to know if an application is being created > for the first time > > > Key: SLIDER-186 > URL: https://issues.apache.org/jira/browse/SLIDER-186 > Project: Slider > Issue Type: Improvement > Components: agent-provider, appmaster >Affects Versions: Slider 0.40 >Reporter: Sumit Mohanty > Fix For: Slider 2.0.0 > > > If its known to the application that the instance is being created for the > first time (as opposed to thawed) then the application can perform some one > time clean-up or setup if needed. > AppMaster can provide the flag to a provider. The agent provider can make the > flag available to the app package during INSTALL command and the application > package can use the flag as appropriate. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SLIDER-127) Create Slider App spec/pkg for Flume
[ https://issues.apache.org/jira/browse/SLIDER-127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty reassigned SLIDER-127: Assignee: (was: Sumit Mohanty) > Create Slider App spec/pkg for Flume > > > Key: SLIDER-127 > URL: https://issues.apache.org/jira/browse/SLIDER-127 > Project: Slider > Issue Type: Bug > Components: appspec >Affects Versions: Slider 0.40 >Reporter: Sumit Mohanty > Fix For: Slider 2.0.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SLIDER-237) Create a Slider app package for HDP log collector
[ https://issues.apache.org/jira/browse/SLIDER-237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty reassigned SLIDER-237: Assignee: (was: Sumit Mohanty) > Create a Slider app package for HDP log collector > -- > > Key: SLIDER-237 > URL: https://issues.apache.org/jira/browse/SLIDER-237 > Project: Slider > Issue Type: Bug > Components: app-package >Affects Versions: Slider 0.50 >Reporter: Sumit Mohanty >Priority: Minor > Fix For: Slider 2.0.0 > > > The log collector app can run on each host with log folder locations that are > provided as configuration. The log collector can collect logs and move to a > central location or a shared storage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SLIDER-564) handle --zkpath argument in a secure cluster
[
https://issues.apache.org/jira/browse/SLIDER-564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty reassigned SLIDER-564:
Assignee: (was: Sumit Mohanty)
> handle --zkpath argument in a secure cluster
>
>
> Key: SLIDER-564
> URL: https://issues.apache.org/jira/browse/SLIDER-564
> Project: Slider
> Issue Type: Task
> Components: client
>Affects Versions: Slider 0.50
>Reporter: Steve Loughran
> Fix For: Slider 2.0.0
>
>
> SLIDER-134 added a default node for an application instance.
> Looking at the {{createZookeeperNode()}} class, it does nothing if the
> cluster is secure. Which means that any expectation that slider is going to
> set up a default path will be false in this situation.
> We need to decide the policy here and then implement it. As setting up secure
> ZK paths is very hard (more precisely, knowing what permissions to pass down
> is very hard), it is simplest to delegate to the user/ZK cluster admins. In
> which case the client should fail if a {{--zkpath}} attribute is passed in at
> build time
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (SLIDER-666) Default configure method for apps
[ https://issues.apache.org/jira/browse/SLIDER-666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty reassigned SLIDER-666: Assignee: (was: Sumit Mohanty) > Default configure method for apps > - > > Key: SLIDER-666 > URL: https://issues.apache.org/jira/browse/SLIDER-666 > Project: Slider > Issue Type: Improvement > Components: agent >Reporter: Billie Rinaldi > Fix For: Slider 2.0.0 > > > We have these configFiles specified in app package metainfo with file names > and types. It would be nice if there were a default configure method for > apps that would create all the configFiles in a specified directory, or in > some default directory that could be referenced. One snag I see is for files > like accumulo-env.sh and hbase-env.sh. They both have xml config files in > metainfo, and the "content" field in the xml has the actual contents of the > .sh file. Maybe this could be a special config file type that just pulls out > that content field from the xml and writes it to an .sh file. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SLIDER-480) [FUN TEST] Add a fun test to ensure that containers are restarted on nodes with correct labels
[ https://issues.apache.org/jira/browse/SLIDER-480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty reassigned SLIDER-480: Assignee: (was: Sumit Mohanty) > [FUN TEST] Add a fun test to ensure that containers are restarted on nodes > with correct labels > -- > > Key: SLIDER-480 > URL: https://issues.apache.org/jira/browse/SLIDER-480 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.50 >Reporter: Sumit Mohanty > > fun-test, AppsThroughAgentQueueAndLabelsIT, tests apps against the user of > labels and queues. This should be modified to cover the error scenarios where > containers go down and come up but on nodes with correct labels. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SLIDER-788) Add support for package in HDFS for install client command
[ https://issues.apache.org/jira/browse/SLIDER-788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty reassigned SLIDER-788: Assignee: (was: Sumit Mohanty) > Add support for package in HDFS for install client command > -- > > Key: SLIDER-788 > URL: https://issues.apache.org/jira/browse/SLIDER-788 > Project: Slider > Issue Type: Bug > Components: client >Affects Versions: Slider 0.70 >Reporter: Sumit Mohanty > Fix For: Slider 2.0.0 > > > Add support for application package to be read from HDFS. > Related to SLIDER-763. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SLIDER-703) Slider extensibility
[ https://issues.apache.org/jira/browse/SLIDER-703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty reassigned SLIDER-703: Assignee: (was: Sumit Mohanty) > Slider extensibility > > > Key: SLIDER-703 > URL: https://issues.apache.org/jira/browse/SLIDER-703 > Project: Slider > Issue Type: Task > Components: app-package, appmaster >Affects Versions: Slider 0.70 >Reporter: Sumit Mohanty > Fix For: Slider 2.0.0 > > > Slider does not yet provide functional extensibility for applications where > the application can extend the behavior of the AppMaster and/or the Agent to > provide some custom behavior. Such extension is by definition localized to > application instance and thus will not affect other application instances. > Such extension are an easy way to add plugins into Agent and AppMaster which > would otherwise have needed creation of component types. > Possible examples are: > * Have agent report back some application specific data through the heartbeat > * Have AppMaster show some application specific data through the REST end > point > * The above REST endpoint could be read-only or it can have limited PUT/POST > capabilities > * AppMaster may allow plugins that perform application instance specific > flexing, container allocation that are influenced by current application > state, etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SLIDER-774) In both appConfig and resources inputs, global and metadata sections, and slider-appmaster component section should not be required
[
https://issues.apache.org/jira/browse/SLIDER-774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty reassigned SLIDER-774:
Assignee: (was: Sumit Mohanty)
> In both appConfig and resources inputs, global and metadata sections, and
> slider-appmaster component section should not be required
> ---
>
> Key: SLIDER-774
> URL: https://issues.apache.org/jira/browse/SLIDER-774
> Project: Slider
> Issue Type: Bug
> Components: app-package, client
>Affects Versions: Slider 0.61
>Reporter: Sumit Mohanty
> Fix For: Slider 2.0.0
>
>
> The following sections should not be required in appConfig.json or
> resources.json. If they are not specified then an empty section of a section
> with default values are assumed.
> {noformat}
> "components": {
> "slider-appmaster": {
> "jvm.heapsize": "256M"
> }
> }
> {noformat}
> {noformat}
> "metadata": {
> }
> {noformat}
> {noformat}
> "global": {
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (SLIDER-590) Need some convenient debugability of command script during/post development time
[ https://issues.apache.org/jira/browse/SLIDER-590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty reassigned SLIDER-590: Assignee: (was: Sumit Mohanty) > Need some convenient debugability of command script during/post development > time > > > Key: SLIDER-590 > URL: https://issues.apache.org/jira/browse/SLIDER-590 > Project: Slider > Issue Type: Bug >Reporter: Siyuan Hua > Fix For: Slider 2.0.0 > > > Nowadays, whatever errors I made in command script to launch the component, I > have to modify the python code repackage and redeployed to hdfs and relaunch > the application. It's quite inconvenient. I hope there is a way to run, test > and debug the command script right after they are installed on target > machine. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SLIDER-701) Support alerts for Slider Apps
[ https://issues.apache.org/jira/browse/SLIDER-701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty reassigned SLIDER-701: Assignee: (was: Sumit Mohanty) > Support alerts for Slider Apps > -- > > Key: SLIDER-701 > URL: https://issues.apache.org/jira/browse/SLIDER-701 > Project: Slider > Issue Type: Task > Components: agent, app-package, appmaster >Affects Versions: Slider 0.70 >Reporter: Sumit Mohanty > Fix For: Slider 2.0.0 > > > Traditional deployment of apps typically include alerts configured for > alerting systems such as Nagios. This includes configuring the alerting > system to check various data points such as live port, jmx data, etc. For a > slider app similar configurations may be defined while being aware of the > fact that the application components may move during the life time of the > application. Additionally, YARN/Slider provides several status information > (e.g. live component instance count) that can be used for alerts. > This task covers investigation into various alerting infrastructure and > providing recommendation or solution for specific alerting infrastructures > for Slider apps. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SLIDER-789) Add support for install client based on app name
[
https://issues.apache.org/jira/browse/SLIDER-789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty reassigned SLIDER-789:
Assignee: (was: Sumit Mohanty)
> Add support for install client based on app name
>
>
> Key: SLIDER-789
> URL: https://issues.apache.org/jira/browse/SLIDER-789
> Project: Slider
> Issue Type: Bug
> Components: client
>Reporter: Sumit Mohanty
> Fix For: Slider 2.0.0
>
>
> client install can accept an application name as a parameter and
> automatically extract the application package associated with the application
> name and use that. When application name is specified, it can be made
> available as a parameter to the install script {{app_name}} to use.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (SLIDER-109) Detect and report application liveness
[ https://issues.apache.org/jira/browse/SLIDER-109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty reassigned SLIDER-109: Assignee: (was: Sumit Mohanty) > Detect and report application liveness > -- > > Key: SLIDER-109 > URL: https://issues.apache.org/jira/browse/SLIDER-109 > Project: Slider > Issue Type: Improvement > Components: agent-provider, appmaster >Affects Versions: Slider 0.40 >Reporter: Sumit Mohanty > Fix For: Slider 2.0.0 > > > Yarn Application state is different than the application state as perceived > by Yarn. Such as: > * When Yarn app state says RUNNING, the application deployed by Slider may > actually be in the process of starting, and not yet ready for clients. > * When Yarn app state says RUNNING, the application may in fact be unhealthy > as in the component instances have gone down and waiting to come back up > Application should be allowed to define its state (for its admin and clients) > that is different than the application state as reported by Yarn. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SLIDER-688) Zero touch install support
[ https://issues.apache.org/jira/browse/SLIDER-688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-688: - Sprint: Slider Feb #1, Slider April #1 (was: Slider Feb #1, Slider April #1, Slider September #2) > Zero touch install support > -- > > Key: SLIDER-688 > URL: https://issues.apache.org/jira/browse/SLIDER-688 > Project: Slider > Issue Type: Improvement >Affects Versions: Slider 0.60 >Reporter: Thomas Weise >Assignee: Sumit Mohanty > > Currently the user needs to specify environment variable such as > HADOOP_CONF_DIR and JAVA_HOME. Typlically the environment has Hadoop > installed and distros have already provided the dependencies. User should not > have to configure this for Slider. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (SLIDER-688) Zero touch install support
[ https://issues.apache.org/jira/browse/SLIDER-688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty closed SLIDER-688. Resolution: Won't Fix > Zero touch install support > -- > > Key: SLIDER-688 > URL: https://issues.apache.org/jira/browse/SLIDER-688 > Project: Slider > Issue Type: Improvement >Affects Versions: Slider 0.60 >Reporter: Thomas Weise >Assignee: Sumit Mohanty > > Currently the user needs to specify environment variable such as > HADOOP_CONF_DIR and JAVA_HOME. Typlically the environment has Hadoop > installed and distros have already provided the dependencies. User should not > have to configure this for Slider. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SLIDER-1055) hbase-daemon executed by slider is excepted from nodemanager container monitoring
[
https://issues.apache.org/jira/browse/SLIDER-1055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094323#comment-15094323
]
Sumit Mohanty commented on SLIDER-1055:
---
I have noticed it before and thought it was some config options in YARN that is
missing. Its possible for agents to start the process while maintaining the
same process group - not sure if that will remedy the situation.
> hbase-daemon executed by slider is excepted from nodemanager container
> monitoring
> -
>
> Key: SLIDER-1055
> URL: https://issues.apache.org/jira/browse/SLIDER-1055
> Project: Slider
> Issue Type: Bug
> Components: application/hbase
>Affects Versions: Slider 0.81
>Reporter: kyungwan nam
>
> here is nodemanager log of a host where a HBASE_REGIONSERVER component is
> running
> {code}
> 2016-01-12 14:11:49,237 DEBUG monitor.ContainersMonitorImpl
> (ContainersMonitorImpl.java:run(361)) - Current ProcessTree list : [ 9801 ]
> 2016-01-12 14:11:49,237 DEBUG monitor.ContainersMonitorImpl
> (ContainersMonitorImpl.java:run(436)) - Constructing ProcessTree for : PID =
> 9801 ContainerId = container_e07_1451897008090_0009_01_03
> 2016-01-12 14:11:49,262 DEBUG util.ProcfsBasedProcessTree
> (ProcfsBasedProcessTree.java:updateProcessTree(274)) - [ 9801 9806 ]
> 2016-01-12 14:11:49,262 INFO monitor.ContainersMonitorImpl
> (ContainersMonitorImpl.java:run(458)) - Memory usage of ProcessTree 9801 for
> container-id container_e07_1451897008090_0009_01_03: 14.2 MB of 1 GB
> physical memory used; 517.1 MB of 2.1 GB virtual memory used
> {code}
> used memory for the container is lower than i expected.
> because pids ( 9801 9806 ) are slider-agent process. regionserver process was
> excepted from monitoring.
> here is the result of "ps axjf"
> {code}
> 9798 9801 9801 9801 ? -1 Ss 500 0:00 \_ /bin/bash -c
> python ./infra/agent/slider-agent/agent/main.py --label
> container_e07_1451897008090_0009_01_03___HBASE_REGIONSERVER --zk-quorum
> 9801 9806 9801 9801 ? -1 Sl 500 0:01 \_ python
> ./infra/agent/slider-agent/agent/main.py --label
> container_e07_1451897008090_0009_01_03___HBASE_REGIONSERVER --zk-quorum
> 1 9979 9801 9801 ? -1 S 500 0:00 bash
> /volume/nodemanager/usercache/yarn/appcache/application_1451897008090_0009/container_e07_1451897008090_0009_01_03/app/install/hbase-0.98.13-hadoop2/bin/hbase-daemon.sh
> --config
> /volume/nodemanager/usercache/yarn/appcache/application_1451897008090_0009/container_e07_1451897008090_0009_01_03/app/install/hbase-0.98.13-hadoop2/conf
> foreground_start regionserver
> 9979 9994 9801 9801 ? -1 Sl 500 0:10 \_
> /package/jdk-1.7.0_45/bin/java -Dproc_regionserver
> -XX:OnOutOfMemoryError=kill -9 %p -Xmx1000m -XX:+UseConcMarkSweepGC
> -XX:ErrorFile=/var/logs/application_1451897008090_0009/container_e07_1451897008090_0009_01_03/hs_err_pid%p.log
> -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps
> -Xloggc:/var/logs/application_1451897008090_0009/container_e07_1451897008090_0009_01_03/gc.log-201601121408
> -Xmn200m -XX:CMSInitiatingOccupancyFraction=70 -Xms1024m -Xmx1024m
> -Dhbase.log.dir=/var/logs/application_1451897008090_0009/container_e07_1451897008090_0009_01_03
> -Dhbase.log.file=hbase-yarn-regionserver.log
> -Dhbase.home.dir=/volume/nodemanager/usercache/yarn/appcache/application_1451897008090_0009/container_e07_1451897008090_0009_01_03/app/install/hbase-0.98.13-hadoop2/bin/..
> -Dhbase.id.str=yarn -Dhbase.root.logger=INFO,RFA
> -Djava.library.path=/package/hadoop-yarn-2.7.1-arch-centos6-x86_64/lib/native
> -Dhbase.security.logger=INFO,RFAS
> org.apache.hadoop.hbase.regionserver.HRegionServer start
> {code}
> when i use the ProcfsBasedProcessTree (default)
> process-tree is determined by relationship between parent and child process.
> so, daemonized process (ppid=1) can’t be included in process-tree.
> I don't know it can be fixed in slider.
> does it need to implement another ResourceCalculatorProcessTree to replace
> the ProcfsBasedProcessTree?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-1041) Discrepancy between /ws/v1/slider/publisher/exports and /ws/v1/slider/publisher/exports/{export_name}
[
https://issues.apache.org/jira/browse/SLIDER-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074616#comment-15074616
]
Sumit Mohanty commented on SLIDER-1041:
---
[~elserj] I do not remember specifics but it could be to get a summary and then
details if you go one level deeper. Also, I remember trying to get to a more
formatted export and possibly creating an alternate endpoint for those exports.
Its been a while :-)
> Discrepancy between /ws/v1/slider/publisher/exports and
> /ws/v1/slider/publisher/exports/{export_name}
> -
>
> Key: SLIDER-1041
> URL: https://issues.apache.org/jira/browse/SLIDER-1041
> Project: Slider
> Issue Type: Bug
> Components: Web & REST
>Affects Versions: Slider 0.90.2
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: Slider 0.91
>
>
> In working on SLIDER-809, I noticed that I could see the map of all exports
> listed in /ws/v1/slider/publisher/exports, but this endpoint did not actually
> contain the value from the {{ExportEntry}}. If I request some
> /ws/v1/slider/publisher/exports/, then I can see the value:
> For example:
> {noformat}
> {"exports":{"servers":{"description":"Servers","updated":1451440844425,"updatedTime":"Tue
> Dec 29 21:00:44 EST
> 2015","entries":{},"empty":true},"quicklinks":{"description":"QuickLinks","updated":1451440844425,"updatedTime":"Tue
> Dec 29 21:00:44 EST
> 2015","entries":{},"empty":true},"container_log_dirs":{"description":"container_log_dirs","updated":1451440840223,"updatedTime":"Tue
> Dec 29 21:00:40 EST
> 2015","entries":{},"empty":true},"container_work_dirs":{"description":"container_work_dirs","updated":1451440840223,"updatedTime":"Tue
> Dec 29 21:00:40 EST 2015","entries":{},"empty":true}}}
> {noformat}
> And if I fetch the quicklinks:
> {noformat}
> {"description":"QuickLinks","updated":1451440844425,"updatedTime":"Tue Dec 29
> 21:00:44 EST
> 2015","entries":{"HTTP":[{"value":"http://hw10447.local:60283","level":"application","updatedTime":"Tue
> Dec 29 21:00:44 EST 2015"}]},"empty":false}
> {noformat}
> Suddenly, entries is not empty. These seems to be inconsistent since both
> REST endpoints are backed by the same data structure.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (SLIDER-763) Add install client command to slider
[ https://issues.apache.org/jira/browse/SLIDER-763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty resolved SLIDER-763. -- Resolution: Fixed Add install client command to slider -- Key: SLIDER-763 URL: https://issues.apache.org/jira/browse/SLIDER-763 Project: Slider Issue Type: Improvement Reporter: Billie Rinaldi Assignee: Sumit Mohanty Fix For: Slider 2.0.0 Attachments: SLIDER-763-add-appname.1.patch, SliderAppClientInstallation.pdf It should be possible to mimic the agent's environment and run the INSTALL command locally for a component to obtain a client installation of an existing app. It may require the app to have a CLIENT category of component. If we had this, each app would no longer need a special script for doing client setup. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-861) Failure counts should be clarified on AppMaster UI
[ https://issues.apache.org/jira/browse/SLIDER-861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-861: - Assignee: Gour Saha Failure counts should be clarified on AppMaster UI -- Key: SLIDER-861 URL: https://issues.apache.org/jira/browse/SLIDER-861 Project: Slider Issue Type: Bug Affects Versions: Slider 0.70 Reporter: Ted Yu Assignee: Gour Saha Sumit and I were looking at why Slider Phoenix was not running. The Slider App Master UI showed 1 failed HBASE_REGIONSERVER instance under Failed column. All the other failure count was 0. However, from the values of container Ids, it was clear that the total number of failures was higher. This is related to windowed failure tracking. We may add new column(s) showing the total number of failures so that the user has better sense of the reliability of the Slider instance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SLIDER-826) Client install related unit tests are failing on Windows
Sumit Mohanty created SLIDER-826: Summary: Client install related unit tests are failing on Windows Key: SLIDER-826 URL: https://issues.apache.org/jira/browse/SLIDER-826 Project: Slider Issue Type: Bug Components: client Affects Versions: Slider 0.70 Reporter: Sumit Mohanty Assignee: Sumit Mohanty Priority: Critical Fix For: Slider 0.80 The following tests are failing: * org.apache.slider.providers.agent.TestAgentClientProvider2.testGetCommandJson 0.38 sec29 * org.apache.slider.providers.agent.TestAgentClientProvider2.testRunCommand -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-773) Add co-processor support for app packages
[ https://issues.apache.org/jira/browse/SLIDER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-773: - Assignee: thomas liu (was: Sumit Mohanty) Add co-processor support for app packages - Key: SLIDER-773 URL: https://issues.apache.org/jira/browse/SLIDER-773 Project: Slider Issue Type: Bug Components: app-package, client Affects Versions: Slider 0.60 Reporter: Sumit Mohanty Assignee: thomas liu Priority: Critical Fix For: Slider 0.80 Attachments: Co-processorSupport.pdf It is typical for applications to allow plugins/co-processors that are essentially a set of additional jar files in the classpath and optionally a set of config files or config changes. Current, slider app packages can handle additional config changes/entries very well. Additional configs files can be added as well but it is not easy if the config files include parameters that need to be resolved by the agent. This requires app package changes. Dropping additional jar files into the class path is not easy and requires app package changes. It is not efficient to modify the app package to support such plugins. App packaging and create command should be modified such that the user can dynamically specify additional jars, config files, configs etc. Specific scenarios are modifying HBase to add support for Phoenix or Ranger. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-756) Create accumulo client script
[
https://issues.apache.org/jira/browse/SLIDER-756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty updated SLIDER-756:
-
Sprint: Slider Feb #1 (was: Slider Jan #2)
Create accumulo client script
-
Key: SLIDER-756
URL: https://issues.apache.org/jira/browse/SLIDER-756
Project: Slider
Issue Type: Improvement
Components: app-package
Reporter: Billie Rinaldi
Assignee: Billie Rinaldi
Fix For: Slider 0.70
Attachments: SLIDER-756.1.patch
The idea behind this script is to make it easier to run the accumulo script
(particularly the accumulo shell) against an accumulo instance running on
slider. The basic idea is to have:
{noformat}
accumulo-slider --app appName accumuloCommand
{noformat}
and a few extra commands like:
{noformat}
accumulo-slider --app appName quicklinks
accumulo-slider --app appName proxies
accumulo-slider --app appName --appconf confDir getconf
accumulo-slider --app appName install installDir
{noformat}
To run accumulo commands, the script requires ACCUMULO_HOME and SLIDER_HOME
to be set in the environment.
For the getconf command, you can download the conf to a specified directory,
and in subsequent commands to accumulo-slider you could use the following to
avoid having to download the conf each time:
{noformat}
accumulo-slider --appconf confDir command
{noformat}
For the install command, you specify an install dir, and it will pull down
the app package, untar the accumulo tarball, and configure it. (I have
concerns that this may be a brittle process.) Once this install dir is made,
you could use it as a normal accumulo installation, using the accumulo script
directly instead of the accumulo-slider wrapper script. The install dir
could also be used as an ACCUMULO_HOME for future calls to accumulo-slider
for other app instances.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-629) Slider's count of failure threshold may not be accurate or it could be a logging issue
[
https://issues.apache.org/jira/browse/SLIDER-629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty updated SLIDER-629:
-
Sprint: Slider December #1, Slider Jan #2, Slider Feb #1 (was: Slider
December #1, Slider Jan #2)
Slider's count of failure threshold may not be accurate or it could be a
logging issue
--
Key: SLIDER-629
URL: https://issues.apache.org/jira/browse/SLIDER-629
Project: Slider
Issue Type: Bug
Components: appmaster
Affects Versions: Slider 0.50
Reporter: Sumit Mohanty
Assignee: Jonathan Maron
Fix For: Slider 0.70
One of the long running HBase tests failed with the following error:
{noformat}
2014-11-08 01:07:26,407 [AmExecutor-008] ERROR appmaster.SliderAppMaster -
Cluster teardown triggered
org.apache.slider.core.exceptions.TriggerClusterTeardownException: Unstable
Application Instance : - failed with component H BASE_REGIONSERVER
failing 8 times (0 in startup); threshold is 5 - last failure: Failure
container_1415341585168_0005_01_08 on host onprem-slider23:
http://onprem-slider21:19888/jobhistory/logs/onprem-slider23:45454/contai
ner_1415341585168_0005_01_08/ctx/hadoop^M
{noformat}
However, there were total of 9 REGION_SERVERs created.
{noformat}
2014-11-07 16:00:35,346 [AMRM Callback Handler Thread] INFO state.AppState -
Assigning role HBASE_REGIONSERVER to container
container_1415341585168_0005_01_02, on onprem-slider25:45454,
2014-11-07 16:00:35,347 [AMRM Callback Handler Thread] INFO state.AppState -
Assigning role HBASE_REGIONSERVER to container
container_1415341585168_0005_01_05, on onprem-slider24:45454,
2014-11-07 16:00:35,347 [AMRM Callback Handler Thread] INFO state.AppState -
Assigning role HBASE_REGIONSERVER to container
container_1415341585168_0005_01_07, on onprem-slider22:45454,
2014-11-07 16:00:35,347 [AMRM Callback Handler Thread] INFO state.AppState -
Assigning role HBASE_REGIONSERVER to container
container_1415341585168_0005_01_08, on onprem-slider23:45454,
2014-11-07 23:51:20,040 [AMRM Callback Handler Thread] INFO state.AppState -
Assigning role HBASE_REGIONSERVER to container
container_1415341585168_0005_01_09, on onprem-slider22:45454,
2014-11-07 23:58:44,810 [AMRM Callback Handler Thread] INFO state.AppState -
Assigning role HBASE_REGIONSERVER to container
container_1415341585168_0005_01_13, on onprem-slider24:45454,
2014-11-08 00:12:17,804 [AMRM Callback Handler Thread] INFO state.AppState -
Assigning role HBASE_REGIONSERVER to container
container_1415341585168_0005_01_15, on onprem-slider22:45454,
2014-11-08 00:15:57,373 [AMRM Callback Handler Thread] INFO state.AppState -
Assigning role HBASE_REGIONSERVER to container
container_1415341585168_0005_01_18, on onprem-slider25:45454,
2014-11-08 01:06:36,771 [AMRM Callback Handler Thread] INFO state.AppState -
Assigning role HBASE_REGIONSERVER to container
container_1415341585168_0005_01_20, on onprem-slider25:45454,
{noformat}
As the ask was for 4 but 9 were created, obviously there are 5 failures.
Perhaps its a logging issue. Can we also print the Window - e.g. 5 failures
in X minutes or hours.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-774) In both appConfig and resources inputs, global and metadata sections, and slider-appmaster component section should not be required
[
https://issues.apache.org/jira/browse/SLIDER-774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty updated SLIDER-774:
-
Sprint: Slider Mar #1
In both appConfig and resources inputs, global and metadata sections, and
slider-appmaster component section should not be required
---
Key: SLIDER-774
URL: https://issues.apache.org/jira/browse/SLIDER-774
Project: Slider
Issue Type: Bug
Components: app-package, client
Affects Versions: Slider 0.61
Reporter: Sumit Mohanty
Assignee: Sumit Mohanty
Fix For: Slider 0.80
The following sections should not be required in appConfig.json or
resources.json. If they are not specified then an empty section of a section
with default values are assumed.
{noformat}
components: {
slider-appmaster: {
jvm.heapsize: 256M
}
}
{noformat}
{noformat}
metadata: {
}
{noformat}
{noformat}
global: {
}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-788) Add support for package in HDFS for install client command
[ https://issues.apache.org/jira/browse/SLIDER-788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-788: - Sprint: Slider Mar #1 Add support for package in HDFS for install client command -- Key: SLIDER-788 URL: https://issues.apache.org/jira/browse/SLIDER-788 Project: Slider Issue Type: Bug Components: client Affects Versions: Slider 0.70 Reporter: Sumit Mohanty Assignee: Sumit Mohanty Fix For: Slider 0.80 Add support for application package to be read from HDFS. Related to SLIDER-763. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-779) Move to later version of Groovy
[ https://issues.apache.org/jira/browse/SLIDER-779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-779: - Sprint: Slider Feb #1 (was: Slider Jan #2) Move to later version of Groovy --- Key: SLIDER-779 URL: https://issues.apache.org/jira/browse/SLIDER-779 Project: Slider Issue Type: Improvement Components: build, test Affects Versions: Slider 0.70 Reporter: Steve Loughran Assignee: Steve Loughran Fix For: Slider 0.80 Original Estimate: 1h Remaining Estimate: 1h We've been on groovy 2.2 for a while; some compilation problems have shown its time to move on. move to groovy 2.3.7 so that runtime and compiler version are in sync. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-208) Ongoing: keep Jenkins builds working
[ https://issues.apache.org/jira/browse/SLIDER-208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-208: - Sprint: Slider July #1, Slider August #1, Slider August #2, Slider September #1, Slider October #1, Slider October #2, Slider November #2, Slider Feb #1 (was: Slider July #1, Slider August #1, Slider August #2, Slider September #1, Slider October #1, Slider October #2, Slider November #2, Slider Jan #2) Ongoing: keep Jenkins builds working Key: SLIDER-208 URL: https://issues.apache.org/jira/browse/SLIDER-208 Project: Slider Issue Type: Task Components: build Affects Versions: Slider 0.40 Reporter: Steve Loughran Fix For: Slider 0.70 Uber-JIRA that never gets finished: keep Jenkins working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-151) Implement full slider API in REST and switch client to it
[ https://issues.apache.org/jira/browse/SLIDER-151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-151: - Sprint: Slider December #1, Slider Feb #1 (was: Slider December #1, Slider Jan #2) Implement full slider API in REST and switch client to it - Key: SLIDER-151 URL: https://issues.apache.org/jira/browse/SLIDER-151 Project: Slider Issue Type: New Feature Components: client, Web REST Affects Versions: Slider 0.60 Reporter: Steve Loughran Assignee: Steve Loughran Attachments: 2014-12-10-slider-REST.pptx, 2014-12-11-slider-REST.pdf, SliderRESTAPIv3-001.pdf -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-789) Add support for install client based on app name
[
https://issues.apache.org/jira/browse/SLIDER-789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty updated SLIDER-789:
-
Sprint: Slider Mar #1
Add support for install client based on app name
Key: SLIDER-789
URL: https://issues.apache.org/jira/browse/SLIDER-789
Project: Slider
Issue Type: Bug
Components: client
Reporter: Sumit Mohanty
Assignee: Sumit Mohanty
Fix For: Slider 0.80
client install can accept an application name as a parameter and
automatically extract the application package associated with the application
name and use that. When application name is specified, it can be made
available as a parameter to the install script {{app_name}} to use.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-773) Add co-processor support for app packages
[ https://issues.apache.org/jira/browse/SLIDER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-773: - Sprint: Slider Mar #1 Add co-processor support for app packages - Key: SLIDER-773 URL: https://issues.apache.org/jira/browse/SLIDER-773 Project: Slider Issue Type: Bug Components: app-package, client Affects Versions: Slider 0.60 Reporter: Sumit Mohanty Assignee: Sumit Mohanty Priority: Critical Fix For: Slider 0.80 Attachments: Co-processorSupport.pdf It is typical for applications to allow plugins/co-processors that are essentially a set of additional jar files in the classpath and optionally a set of config files or config changes. Current, slider app packages can handle additional config changes/entries very well. Additional configs files can be added as well but it is not easy if the config files include parameters that need to be resolved by the agent. This requires app package changes. Dropping additional jar files into the class path is not easy and requires app package changes. It is not efficient to modify the app package to support such plugins. App packaging and create command should be modified such that the user can dynamically specify additional jars, config files, configs etc. Specific scenarios are modifying HBase to add support for Phoenix or Ranger. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-226) Provide mechanism for rolling upgrade
[ https://issues.apache.org/jira/browse/SLIDER-226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-226: - Assignee: Gour Saha Provide mechanism for rolling upgrade - Key: SLIDER-226 URL: https://issues.apache.org/jira/browse/SLIDER-226 Project: Slider Issue Type: Improvement Reporter: Ted Yu Assignee: Gour Saha Fix For: Slider 0.80 update action paves the road for supporting rolling upgrade on Slider deployed cluster. User can specify a different image for a running cluster with N instances for role R. The rolling upgrade would restart each of the N instances while maintaining the invariant that there're at least N-1 active instances for role R. After restart, new image would be effective. This capability would, e.g., allow HBase to perform compatibility check across releases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-763) Add install client command to slider
[ https://issues.apache.org/jira/browse/SLIDER-763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-763: - Sprint: Slider Feb #1 Add install client command to slider -- Key: SLIDER-763 URL: https://issues.apache.org/jira/browse/SLIDER-763 Project: Slider Issue Type: Improvement Reporter: Billie Rinaldi Assignee: Sumit Mohanty Fix For: Slider 2.0.0 Attachments: SliderAppClientInstallation.pdf It should be possible to mimic the agent's environment and run the INSTALL command locally for a component to obtain a client installation of an existing app. It may require the app to have a CLIENT category of component. If we had this, each app would no longer need a special script for doing client setup. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-226) Provide mechanism for rolling upgrade
[ https://issues.apache.org/jira/browse/SLIDER-226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-226: - Fix Version/s: (was: Slider 2.0.0) Slider 0.80 Provide mechanism for rolling upgrade - Key: SLIDER-226 URL: https://issues.apache.org/jira/browse/SLIDER-226 Project: Slider Issue Type: Improvement Reporter: Ted Yu Assignee: Gour Saha Fix For: Slider 0.80 update action paves the road for supporting rolling upgrade on Slider deployed cluster. User can specify a different image for a running cluster with N instances for role R. The rolling upgrade would restart each of the N instances while maintaining the invariant that there're at least N-1 active instances for role R. After restart, new image would be effective. This capability would, e.g., allow HBase to perform compatibility check across releases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-663) Make it easy to develop and deploy application packages that are essentially shell commands
[ https://issues.apache.org/jira/browse/SLIDER-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-663: - Sprint: Slider Feb #1 Make it easy to develop and deploy application packages that are essentially shell commands --- Key: SLIDER-663 URL: https://issues.apache.org/jira/browse/SLIDER-663 Project: Slider Issue Type: Task Components: agent-provider, app-package Affects Versions: Slider 0.60 Reporter: Sumit Mohanty Assignee: Sumit Mohanty Priority: Critical Attachments: PackagingSimplificationandCreateEnhancements.pdf Slider app packages require several artifacts for completeness such as a metainfo.xml, a python script to read config and a python script for life cycle commands, a tarball, etc. A simple application can be modeled as a shell command or as a java application that just needs a jar and some system properties. So the application requirement can be summarized as: * a jar * a command * a small set of properties While it is possible to model these as an application package (e.g. memcached) it is also evident that there are a lot of common patterns that need to be duplicated. Slider should provide a way to pass these parameters in the create call itself rather than having to create a full application package. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-763) Add install client command to slider
[ https://issues.apache.org/jira/browse/SLIDER-763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-763: - Attachment: (was: SliderAppClientInstallation.pdf) Add install client command to slider -- Key: SLIDER-763 URL: https://issues.apache.org/jira/browse/SLIDER-763 Project: Slider Issue Type: Improvement Reporter: Billie Rinaldi Assignee: Sumit Mohanty Fix For: Slider 2.0.0 Attachments: SliderAppClientInstallation.pdf It should be possible to mimic the agent's environment and run the INSTALL command locally for a component to obtain a client installation of an existing app. It may require the app to have a CLIENT category of component. If we had this, each app would no longer need a special script for doing client setup. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-763) Add install client command to slider
[ https://issues.apache.org/jira/browse/SLIDER-763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-763: - Attachment: SliderAppClientInstallation.pdf Add install client command to slider -- Key: SLIDER-763 URL: https://issues.apache.org/jira/browse/SLIDER-763 Project: Slider Issue Type: Improvement Reporter: Billie Rinaldi Assignee: Sumit Mohanty Fix For: Slider 2.0.0 Attachments: SliderAppClientInstallation.pdf It should be possible to mimic the agent's environment and run the INSTALL command locally for a component to obtain a client installation of an existing app. It may require the app to have a CLIENT category of component. If we had this, each app would no longer need a special script for doing client setup. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SLIDER-789) Add support for install client based on app name
Sumit Mohanty created SLIDER-789:
Summary: Add support for install client based on app name
Key: SLIDER-789
URL: https://issues.apache.org/jira/browse/SLIDER-789
Project: Slider
Issue Type: Bug
Components: client
Reporter: Sumit Mohanty
Assignee: Sumit Mohanty
Fix For: Slider 0.80
client install can accept an application name as a parameter and automatically
extract the application package associated with the application name and use
that. When application name is specified, it can be made available as a
parameter to the install script {{app_name}} to use.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-763) Add install client command to slider
[ https://issues.apache.org/jira/browse/SLIDER-763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-763: - Attachment: SliderAppClientInstallation.pdf Add install client command to slider -- Key: SLIDER-763 URL: https://issues.apache.org/jira/browse/SLIDER-763 Project: Slider Issue Type: Improvement Reporter: Billie Rinaldi Assignee: Sumit Mohanty Fix For: Slider 2.0.0 Attachments: SliderAppClientInstallation.pdf It should be possible to mimic the agent's environment and run the INSTALL command locally for a component to obtain a client installation of an existing app. It may require the app to have a CLIENT category of component. If we had this, each app would no longer need a special script for doing client setup. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-773) Add co-processor support for app packages
[ https://issues.apache.org/jira/browse/SLIDER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-773: - Attachment: (was: Co-processorSupport.pdf) Add co-processor support for app packages - Key: SLIDER-773 URL: https://issues.apache.org/jira/browse/SLIDER-773 Project: Slider Issue Type: Bug Components: app-package, client Affects Versions: Slider 0.60 Reporter: Sumit Mohanty Assignee: Sumit Mohanty Priority: Critical Fix For: Slider 0.80 Attachments: Co-processorSupport.pdf It is typical for applications to allow plugins/co-processors that are essentially a set of additional jar files in the classpath and optionally a set of config files or config changes. Current, slider app packages can handle additional config changes/entries very well. Additional configs files can be added as well but it is not easy if the config files include parameters that need to be resolved by the agent. This requires app package changes. Dropping additional jar files into the class path is not easy and requires app package changes. It is not efficient to modify the app package to support such plugins. App packaging and create command should be modified such that the user can dynamically specify additional jars, config files, configs etc. Specific scenarios are modifying HBase to add support for Phoenix or Ranger. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-773) Add co-processor support for app packages
[ https://issues.apache.org/jira/browse/SLIDER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-773: - Attachment: Co-processorSupport.pdf Add co-processor support for app packages - Key: SLIDER-773 URL: https://issues.apache.org/jira/browse/SLIDER-773 Project: Slider Issue Type: Bug Components: app-package, client Affects Versions: Slider 0.60 Reporter: Sumit Mohanty Assignee: Sumit Mohanty Priority: Critical Fix For: Slider 0.80 Attachments: Co-processorSupport.pdf It is typical for applications to allow plugins/co-processors that are essentially a set of additional jar files in the classpath and optionally a set of config files or config changes. Current, slider app packages can handle additional config changes/entries very well. Additional configs files can be added as well but it is not easy if the config files include parameters that need to be resolved by the agent. This requires app package changes. Dropping additional jar files into the class path is not easy and requires app package changes. It is not efficient to modify the app package to support such plugins. App packaging and create command should be modified such that the user can dynamically specify additional jars, config files, configs etc. Specific scenarios are modifying HBase to add support for Phoenix or Ranger. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-763) Add install client command to slider
[ https://issues.apache.org/jira/browse/SLIDER-763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-763: - Assignee: Sumit Mohanty Add install client command to slider -- Key: SLIDER-763 URL: https://issues.apache.org/jira/browse/SLIDER-763 Project: Slider Issue Type: Improvement Reporter: Billie Rinaldi Assignee: Sumit Mohanty Fix For: Slider 2.0.0 It should be possible to mimic the agent's environment and run the INSTALL command locally for a component to obtain a client installation of an existing app. It may require the app to have a CLIENT category of component. If we had this, each app would no longer need a special script for doing client setup. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-780) Support for Docker based application packaging in Slider
[ https://issues.apache.org/jira/browse/SLIDER-780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-780: - Assignee: thomas liu Support for Docker based application packaging in Slider Key: SLIDER-780 URL: https://issues.apache.org/jira/browse/SLIDER-780 Project: Slider Issue Type: Task Reporter: thomas liu Assignee: thomas liu Fix For: Slider 0.80 Enable Slider to deploy an application defined as Docker image, monitor its running status, fetching exported configs, and maintain its lifecycle. A quick summary can be found here: https://docs.google.com/a/hortonworks.com/document/d/1Yi1i2Mr8URr-iZ_wcUFGybMeX1VKr2XFhjnAPcC96Gc/edit# -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-780) Support for Docker based application packaging in Slider
[ https://issues.apache.org/jira/browse/SLIDER-780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-780: - Description: Enable Slider to deploy an application defined as Docker image, monitor its running status, fetching exported configs, and maintain its lifecycle. was: Enable Slider to deploy an application defined as Docker image, monitor its running status, fetching exported configs, and maintain its lifecycle. A quick summary can be found here: https://docs.google.com/a/hortonworks.com/document/d/1Yi1i2Mr8URr-iZ_wcUFGybMeX1VKr2XFhjnAPcC96Gc/edit# Support for Docker based application packaging in Slider Key: SLIDER-780 URL: https://issues.apache.org/jira/browse/SLIDER-780 Project: Slider Issue Type: Task Reporter: thomas liu Assignee: thomas liu Fix For: Slider 0.80 Enable Slider to deploy an application defined as Docker image, monitor its running status, fetching exported configs, and maintain its lifecycle. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-706) application installation failure does not result in clear failure indication
[
https://issues.apache.org/jira/browse/SLIDER-706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14309408#comment-14309408
]
Sumit Mohanty commented on SLIDER-706:
--
SLIDER-723 fixed the issue where INSTALL failure was not handled properly. Now
the container goes down after INSTALL failure.
application installation failure does not result in clear failure indication
Key: SLIDER-706
URL: https://issues.apache.org/jira/browse/SLIDER-706
Project: Slider
Issue Type: Bug
Affects Versions: Slider 0.60
Environment: Windows
Reporter: Jonathan Maron
Fix For: Slider 0.80
Though an HBase deployment on a windows secure cluster results in a failure
to install, the application master and application remain running with no
attempts to restart the failed container etc. The app master log does seem
to indicate the failure:
2014-12-09 09:07:11,548 [973031640@qtp-618337492-7] DEBUG
agent.AgentProviderService (AgentProviderService.java:handleHeartBeat(649)) -
Handling heartbeat: HeartBeat{responseId=6, timestamp=1418144831565,
hostname='container_1418069434921_0004_01_03___HBASE_REGIONSERVER',
reports=[CommandReport{role='HBASE_REGIONSERVER', actionId='10-1',
status='FAILED', exitCode=1, clusterName='hbase-test',
serviceName='hbase-test', taskId=10, roleCommand=INSTALL,
configurationTags=null}], componentStatus=[],
nodeStatus=HostStatus{status=HEALTHY, cause='NONE'}}
2014-12-09 09:07:11,549 [973031640@qtp-618337492-7] INFO
agent.AgentProviderService (AgentProviderService.java:handleHeartBeat(698)) -
Component operation. Status: FAILED
However, the heartbeats from the given container continue:
2014-12-09 09:07:21,561 [973031640@qtp-618337492-7] DEBUG
agent.AgentProviderService (AgentProviderService.java:handleHeartBeat(649)) -
Handling heartbeat: HeartBeat{responseId=7, timestamp=1418144841601,
hostname='container_1418069434921_0004_01_03___HBASE_REGIONSERVER',
reports=[], componentStatus=[], nodeStatus=HostStatus{status=HEALTHY,
cause='NONE'}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-706) application installation failure does not result in clear failure indication
[
https://issues.apache.org/jira/browse/SLIDER-706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty updated SLIDER-706:
-
Fix Version/s: Slider 0.80
application installation failure does not result in clear failure indication
Key: SLIDER-706
URL: https://issues.apache.org/jira/browse/SLIDER-706
Project: Slider
Issue Type: Bug
Affects Versions: Slider 0.60
Environment: Windows
Reporter: Jonathan Maron
Fix For: Slider 0.80
Though an HBase deployment on a windows secure cluster results in a failure
to install, the application master and application remain running with no
attempts to restart the failed container etc. The app master log does seem
to indicate the failure:
2014-12-09 09:07:11,548 [973031640@qtp-618337492-7] DEBUG
agent.AgentProviderService (AgentProviderService.java:handleHeartBeat(649)) -
Handling heartbeat: HeartBeat{responseId=6, timestamp=1418144831565,
hostname='container_1418069434921_0004_01_03___HBASE_REGIONSERVER',
reports=[CommandReport{role='HBASE_REGIONSERVER', actionId='10-1',
status='FAILED', exitCode=1, clusterName='hbase-test',
serviceName='hbase-test', taskId=10, roleCommand=INSTALL,
configurationTags=null}], componentStatus=[],
nodeStatus=HostStatus{status=HEALTHY, cause='NONE'}}
2014-12-09 09:07:11,549 [973031640@qtp-618337492-7] INFO
agent.AgentProviderService (AgentProviderService.java:handleHeartBeat(698)) -
Component operation. Status: FAILED
However, the heartbeats from the given container continue:
2014-12-09 09:07:21,561 [973031640@qtp-618337492-7] DEBUG
agent.AgentProviderService (AgentProviderService.java:handleHeartBeat(649)) -
Handling heartbeat: HeartBeat{responseId=7, timestamp=1418144841601,
hostname='container_1418069434921_0004_01_03___HBASE_REGIONSERVER',
reports=[], componentStatus=[], nodeStatus=HostStatus{status=HEALTHY,
cause='NONE'}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (SLIDER-706) application installation failure does not result in clear failure indication
[
https://issues.apache.org/jira/browse/SLIDER-706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty resolved SLIDER-706.
--
Resolution: Fixed
Fix Version/s: (was: Slider 0.80)
Slider 0.70
application installation failure does not result in clear failure indication
Key: SLIDER-706
URL: https://issues.apache.org/jira/browse/SLIDER-706
Project: Slider
Issue Type: Bug
Affects Versions: Slider 0.60
Environment: Windows
Reporter: Jonathan Maron
Fix For: Slider 0.70
Though an HBase deployment on a windows secure cluster results in a failure
to install, the application master and application remain running with no
attempts to restart the failed container etc. The app master log does seem
to indicate the failure:
2014-12-09 09:07:11,548 [973031640@qtp-618337492-7] DEBUG
agent.AgentProviderService (AgentProviderService.java:handleHeartBeat(649)) -
Handling heartbeat: HeartBeat{responseId=6, timestamp=1418144831565,
hostname='container_1418069434921_0004_01_03___HBASE_REGIONSERVER',
reports=[CommandReport{role='HBASE_REGIONSERVER', actionId='10-1',
status='FAILED', exitCode=1, clusterName='hbase-test',
serviceName='hbase-test', taskId=10, roleCommand=INSTALL,
configurationTags=null}], componentStatus=[],
nodeStatus=HostStatus{status=HEALTHY, cause='NONE'}}
2014-12-09 09:07:11,549 [973031640@qtp-618337492-7] INFO
agent.AgentProviderService (AgentProviderService.java:handleHeartBeat(698)) -
Component operation. Status: FAILED
However, the heartbeats from the given container continue:
2014-12-09 09:07:21,561 [973031640@qtp-618337492-7] DEBUG
agent.AgentProviderService (AgentProviderService.java:handleHeartBeat(649)) -
Handling heartbeat: HeartBeat{responseId=7, timestamp=1418144841601,
hostname='container_1418069434921_0004_01_03___HBASE_REGIONSERVER',
reports=[], componentStatus=[], nodeStatus=HostStatus{status=HEALTHY,
cause='NONE'}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-663) Make it easy to develop and deploy application packages that are essentially shell commands
[ https://issues.apache.org/jira/browse/SLIDER-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-663: - Attachment: PackagingSimplificationandCreateEnhancements.pdf Make it easy to develop and deploy application packages that are essentially shell commands --- Key: SLIDER-663 URL: https://issues.apache.org/jira/browse/SLIDER-663 Project: Slider Issue Type: Task Components: agent-provider, app-package Affects Versions: Slider 0.60 Reporter: Sumit Mohanty Assignee: Sumit Mohanty Priority: Critical Attachments: PackagingSimplificationandCreateEnhancements.pdf Slider app packages require several artifacts for completeness such as a metainfo.xml, a python script to read config and a python script for life cycle commands, a tarball, etc. A simple application can be modeled as a shell command or as a java application that just needs a jar and some system properties. So the application requirement can be summarized as: * a jar * a command * a small set of properties While it is possible to model these as an application package (e.g. memcached) it is also evident that there are a lot of common patterns that need to be duplicated. Slider should provide a way to pass these parameters in the create call itself rather than having to create a full application package. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SLIDER-777) Provide slider dependencies as a self contained versioned tarball
Sumit Mohanty created SLIDER-777: Summary: Provide slider dependencies as a self contained versioned tarball Key: SLIDER-777 URL: https://issues.apache.org/jira/browse/SLIDER-777 Project: Slider Issue Type: Bug Components: build, client Affects Versions: Slider 0.70 Reporter: Sumit Mohanty Fix For: Slider 0.80 Currently, slider client picks up all the dependencies from the lib folder. This causes an issue for code that embed Slider client where either they need to specify a folder which has the all the required (but no more) jar files or their own lib directory which contains jar files of the embedding application as well. Slider should explore (similar to Tez or MR) the possibility of packaging all dependencies into a tarball that can be stored in a version aware fashion in HDFS or local FS. A Slider client can choose to use a specific version when creating the an instance of an application. Now, it is possible for a client of certain version to create an app using another version of the tarball. _This requires further investigation in terms of how easy to debug the issues if there is no compatibility between versions._ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-777) Provide slider dependencies as a self contained versioned tarball
[ https://issues.apache.org/jira/browse/SLIDER-777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-777: - Description: Currently, slider client picks up all the dependencies from the lib folder. This causes an issue for code that embed Slider client where either they need to specify a folder which has the all the required (but no more) jar files or their own lib directory which contains jar files of the embedding application as well. Slider should explore (similar to Tez or MR) the possibility of packaging all dependencies into a tarball that can be stored in a version aware fashion in HDFS or local FS. A Slider client can choose to use a specific version when creating the an instance of an application. Now, it is possible for a client of certain version to create an app using another version of the tarball. _This requires further investigation in terms of how easy to debug the issues if there is no compatibility between versions of the client and the tarball._ was: Currently, slider client picks up all the dependencies from the lib folder. This causes an issue for code that embed Slider client where either they need to specify a folder which has the all the required (but no more) jar files or their own lib directory which contains jar files of the embedding application as well. Slider should explore (similar to Tez or MR) the possibility of packaging all dependencies into a tarball that can be stored in a version aware fashion in HDFS or local FS. A Slider client can choose to use a specific version when creating the an instance of an application. Now, it is possible for a client of certain version to create an app using another version of the tarball. _This requires further investigation in terms of how easy to debug the issues if there is no compatibility between versions._ Provide slider dependencies as a self contained versioned tarball - Key: SLIDER-777 URL: https://issues.apache.org/jira/browse/SLIDER-777 Project: Slider Issue Type: Bug Components: build, client Affects Versions: Slider 0.70 Reporter: Sumit Mohanty Fix For: Slider 0.80 Currently, slider client picks up all the dependencies from the lib folder. This causes an issue for code that embed Slider client where either they need to specify a folder which has the all the required (but no more) jar files or their own lib directory which contains jar files of the embedding application as well. Slider should explore (similar to Tez or MR) the possibility of packaging all dependencies into a tarball that can be stored in a version aware fashion in HDFS or local FS. A Slider client can choose to use a specific version when creating the an instance of an application. Now, it is possible for a client of certain version to create an app using another version of the tarball. _This requires further investigation in terms of how easy to debug the issues if there is no compatibility between versions of the client and the tarball._ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-774) In both appConfig and resources inputs, global and metadata sections, and slider-appmaster component section should not be required
[
https://issues.apache.org/jira/browse/SLIDER-774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty updated SLIDER-774:
-
Fix Version/s: (was: Slider 0.70)
Slider 0.80
In both appConfig and resources inputs, global and metadata sections, and
slider-appmaster component section should not be required
---
Key: SLIDER-774
URL: https://issues.apache.org/jira/browse/SLIDER-774
Project: Slider
Issue Type: Bug
Components: app-package, client
Affects Versions: Slider 0.61
Reporter: Sumit Mohanty
Assignee: Sumit Mohanty
Fix For: Slider 0.80
The following sections should not be required in appConfig.json or
resources.json. If they are not specified then an empty section of a section
with default values are assumed.
{noformat}
components: {
slider-appmaster: {
jvm.heapsize: 256M
}
}
{noformat}
{noformat}
metadata: {
}
{noformat}
{noformat}
global: {
}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (SLIDER-776) additional_cp is not necessary for memcached
[ https://issues.apache.org/jira/browse/SLIDER-776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty resolved SLIDER-776. -- Resolution: Fixed additional_cp is not necessary for memcached Key: SLIDER-776 URL: https://issues.apache.org/jira/browse/SLIDER-776 Project: Slider Issue Type: Improvement Affects Versions: Slider 0.60 Reporter: Yang Hao Assignee: Yang Hao Fix For: Slider 0.60 Attachments: SLIDER-776.patch Original Estimate: 2h Remaining Estimate: 2h When deploying memcached, the additional_cp is not necessary, but if not configured, there will be an error. So a clever way is to change the logic of additional_cp to avoid this error -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-776) additional_cp is not necessary for memcached
[ https://issues.apache.org/jira/browse/SLIDER-776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14306231#comment-14306231 ] Sumit Mohanty commented on SLIDER-776: -- Good catch, LGTM. I will commit the patch. additional_cp is not necessary for memcached Key: SLIDER-776 URL: https://issues.apache.org/jira/browse/SLIDER-776 Project: Slider Issue Type: Improvement Affects Versions: Slider 0.60 Reporter: Yang Hao Fix For: Slider 0.60 Attachments: SLIDER-776.patch Original Estimate: 2h Remaining Estimate: 2h When deploying memcached, the additional_cp is not necessary, but if not configured, there will be an error. So a clever way is to change the logic of additional_cp to avoid this error -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-776) additional_cp is not necessary for memcached
[ https://issues.apache.org/jira/browse/SLIDER-776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-776: - Assignee: Yang Hao additional_cp is not necessary for memcached Key: SLIDER-776 URL: https://issues.apache.org/jira/browse/SLIDER-776 Project: Slider Issue Type: Improvement Affects Versions: Slider 0.60 Reporter: Yang Hao Assignee: Yang Hao Fix For: Slider 0.60 Attachments: SLIDER-776.patch Original Estimate: 2h Remaining Estimate: 2h When deploying memcached, the additional_cp is not necessary, but if not configured, there will be an error. So a clever way is to change the logic of additional_cp to avoid this error -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SLIDER-774) In both appConfig and resources inputs, global and metadata sections, and slider-appmaster component section should not be required
Sumit Mohanty created SLIDER-774:
Summary: In both appConfig and resources inputs, global and
metadata sections, and slider-appmaster component section should not be required
Key: SLIDER-774
URL: https://issues.apache.org/jira/browse/SLIDER-774
Project: Slider
Issue Type: Bug
Components: app-package, client
Affects Versions: Slider 0.61
Reporter: Sumit Mohanty
Assignee: Sumit Mohanty
Fix For: Slider 0.70
The following sections should not be required in appConfig.json or
resources.json. If they are not specified then an empty section of a section
with default values are assumed.
{noformat}
components: {
slider-appmaster: {
jvm.heapsize: 256M
}
}
{noformat}
{noformat}
metadata: {
}
{noformat}
{noformat}
global: {
}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (SLIDER-743) Include node failure history when choosing placement hints
[ https://issues.apache.org/jira/browse/SLIDER-743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty resolved SLIDER-743. -- Resolution: Fixed Assignee: Sumit Mohanty (was: Steve Loughran) Include node failure history when choosing placement hints -- Key: SLIDER-743 URL: https://issues.apache.org/jira/browse/SLIDER-743 Project: Slider Issue Type: Improvement Components: appmaster Affects Versions: Slider 0.60 Reporter: Steve Loughran Assignee: Sumit Mohanty Fix For: Slider 0.70 Attachments: SLIDER-743.patch Use the (collected) history on per-node failures to tune placement requests, so that even if there's a history of components running there, if its been failing recently then it's not asked for # Add a recent failures field to the node entries, alongside the total failures counter # the scheduled failure count resetter will set that field to zero,alongside the component failures # When Slider has to request a new container, if a node has a recent failure count above some threshold, it will not be requested, even if it is a past location of that component. When the placement policy is STRICT, node failure history will not be taken into account -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLIDER-641) multiple slider-agent tarballs in lib dir
[ https://issues.apache.org/jira/browse/SLIDER-641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty resolved SLIDER-641. -- Resolution: Fixed multiple slider-agent tarballs in lib dir - Key: SLIDER-641 URL: https://issues.apache.org/jira/browse/SLIDER-641 Project: Slider Issue Type: Bug Components: build Affects Versions: Slider 0.60 Reporter: Steve Loughran Assignee: Sumit Mohanty Fix For: Slider 0.70 full builds create two versions of the agent tarball, one with a version number, one without -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-743) Include node failure history when choosing placement hints
[ https://issues.apache.org/jira/browse/SLIDER-743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-743: - Attachment: SLIDER-743.patch Include node failure history when choosing placement hints -- Key: SLIDER-743 URL: https://issues.apache.org/jira/browse/SLIDER-743 Project: Slider Issue Type: Improvement Components: appmaster Affects Versions: Slider 0.60 Reporter: Steve Loughran Assignee: Steve Loughran Fix For: Slider 0.70 Attachments: SLIDER-743.patch Use the (collected) history on per-node failures to tune placement requests, so that even if there's a history of components running there, if its been failing recently then it's not asked for # Add a recent failures field to the node entries, alongside the total failures counter # the scheduled failure count resetter will set that field to zero,alongside the component failures # When Slider has to request a new container, if a node has a recent failure count above some threshold, it will not be requested, even if it is a past location of that component. When the placement policy is STRICT, node failure history will not be taken into account -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-767) rename apache-slider-command-logger to command-logger-app-package
[ https://issues.apache.org/jira/browse/SLIDER-767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-767: - Sprint: Slider Jan #2 (was: Slider Jan #1) rename apache-slider-command-logger to command-logger-app-package --- Key: SLIDER-767 URL: https://issues.apache.org/jira/browse/SLIDER-767 Project: Slider Issue Type: Improvement Components: build Affects Versions: Slider 0.60 Reporter: Steve Loughran Assignee: Steve Loughran Fix For: Slider 0.61 Original Estimate: 0.25h Remaining Estimate: 0.25h to avoid some confusion in package names. rename apache-slider-command-logger to command-logger-app-package -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-208) Ongoing: keep Jenkins builds working
[ https://issues.apache.org/jira/browse/SLIDER-208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-208: - Sprint: Slider July #1, Slider August #1, Slider August #2, Slider September #1, Slider October #1, Slider October #2, Slider November #2, Slider Jan #2 (was: Slider July #1, Slider August #1, Slider August #2, Slider September #1, Slider October #1, Slider October #2, Slider November #2, Slider Jan #1) Ongoing: keep Jenkins builds working Key: SLIDER-208 URL: https://issues.apache.org/jira/browse/SLIDER-208 Project: Slider Issue Type: Task Components: build Affects Versions: Slider 0.40 Reporter: Steve Loughran Fix For: Slider 0.70 Uber-JIRA that never gets finished: keep Jenkins working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-745) release slider 0.61 with mvn artifacts
[ https://issues.apache.org/jira/browse/SLIDER-745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-745: - Sprint: Slider Jan #2 (was: Slider Jan #1) release slider 0.61 with mvn artifacts -- Key: SLIDER-745 URL: https://issues.apache.org/jira/browse/SLIDER-745 Project: Slider Issue Type: Task Components: build Affects Versions: Slider 0.60 Reporter: Steve Loughran Assignee: Steve Loughran Fix For: Slider 0.61 Attachments: selective-release-of-modules.patch Original Estimate: 8h Time Spent: 61h Remaining Estimate: 0h # Backport SLIDER-671 changes to slider-0.60 branch # Release slider 0.61 with mvn artifacts published This will be purely 0.60 with the build changes; no other fixes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLIDER-723) Memcached component launch fail does not propagate
[ https://issues.apache.org/jira/browse/SLIDER-723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty resolved SLIDER-723. -- Resolution: Fixed Memcached component launch fail does not propagate -- Key: SLIDER-723 URL: https://issues.apache.org/jira/browse/SLIDER-723 Project: Slider Issue Type: Improvement Affects Versions: Slider 0.60 Reporter: Yang Hao Assignee: Sumit Mohanty Fix For: Slider 0.70 When am applies a container for the component ,and the container starts, but the component may not launch successfully. So a friendly way should be out. Like just kill the container when the component don't launch successfully. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLIDER-755) AgentProvider doesn't raise and exception when the agent tar isn't there
[
https://issues.apache.org/jira/browse/SLIDER-755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty resolved SLIDER-755.
--
Resolution: Fixed
AgentProvider doesn't raise and exception when the agent tar isn't there
Key: SLIDER-755
URL: https://issues.apache.org/jira/browse/SLIDER-755
Project: Slider
Issue Type: Bug
Components: agent-provider
Affects Versions: Slider 0.60
Reporter: Steve Loughran
Assignee: Sumit Mohanty
Fix For: Slider 0.70
Agent provider line 325 says
{code}
// TODO: throw exception when agent tarball is not available
{code}
all it does is log an error
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (SLIDER-740) support configurating HBASE_OPTS
[ https://issues.apache.org/jira/browse/SLIDER-740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty resolved SLIDER-740. -- Resolution: Fixed support configurating HBASE_OPTS Key: SLIDER-740 URL: https://issues.apache.org/jira/browse/SLIDER-740 Project: Slider Issue Type: Improvement Components: app-package Affects Versions: Slider 0.60 Reporter: Yang Hao Assignee: Yang Hao Fix For: Slider 0.70 Attachments: SLIDER-740.patch, SLIDER-740.v2.patch It would be very helpful that we can add some additional jvm options to the HBASE cluster. An alternative way is to configure HBASE_OPTS. Adding this patch, user can make it by adding site.hbase-env.hbase_opts in appConf.json, such as site.hbase-env.hbase_opts: -Djava.security.krb5.conf=/home/work/app/yarn/lgtst-xiaomi2/nodemanager/krb5.conf , -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-641) multiple slider-agent tarballs in lib dir
[ https://issues.apache.org/jira/browse/SLIDER-641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-641: - Fix Version/s: Slider 0.70 multiple slider-agent tarballs in lib dir - Key: SLIDER-641 URL: https://issues.apache.org/jira/browse/SLIDER-641 Project: Slider Issue Type: Bug Components: build Affects Versions: Slider 0.60 Reporter: Steve Loughran Assignee: Sumit Mohanty Fix For: Slider 0.70 full builds create two versions of the agent tarball, one with a version number, one without -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (SLIDER-641) multiple slider-agent tarballs in lib dir
[ https://issues.apache.org/jira/browse/SLIDER-641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty reassigned SLIDER-641: Assignee: Sumit Mohanty (was: Steve Loughran) multiple slider-agent tarballs in lib dir - Key: SLIDER-641 URL: https://issues.apache.org/jira/browse/SLIDER-641 Project: Slider Issue Type: Bug Components: build Affects Versions: Slider 0.60 Reporter: Steve Loughran Assignee: Sumit Mohanty Fix For: Slider 0.70 full builds create two versions of the agent tarball, one with a version number, one without -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-740) support configurating HBASE_OPTS
[ https://issues.apache.org/jira/browse/SLIDER-740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14288598#comment-14288598 ] Sumit Mohanty commented on SLIDER-740: -- LGTM, I will commit it. Thanks. support configurating HBASE_OPTS Key: SLIDER-740 URL: https://issues.apache.org/jira/browse/SLIDER-740 Project: Slider Issue Type: Improvement Components: app-package Affects Versions: Slider 0.60 Reporter: Yang Hao Assignee: Yang Hao Fix For: Slider 0.70 Attachments: SLIDER-740.patch, SLIDER-740.v2.patch It would be very helpful that we can add some additional jvm options to the HBASE cluster. An alternative way is to configure HBASE_OPTS. Adding this patch, user can make it by adding site.hbase-env.hbase_opts in appConf.json, such as site.hbase-env.hbase_opts: -Djava.security.krb5.conf=/home/work/app/yarn/lgtst-xiaomi2/nodemanager/krb5.conf , -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-740) support configurating HBASE_OPTS
[
https://issues.apache.org/jira/browse/SLIDER-740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286110#comment-14286110
]
Sumit Mohanty commented on SLIDER-740:
--
We need to modify the patch a bit - mostly due to the nuances of the python
resource management library that Slider uses.
Instead of
{noformat}
+hbase_opts = config['configurations']['hbase-env']['hbase_opts']
+if is_empty(hbase_opts):
+ hbase_opts = ''
{noformat}
you can use
{code}
hbase_opts = default('/configurations/hbase-env/hbase_opts', '')
{code}
support configurating HBASE_OPTS
Key: SLIDER-740
URL: https://issues.apache.org/jira/browse/SLIDER-740
Project: Slider
Issue Type: Improvement
Components: app-package
Affects Versions: Slider 0.60
Reporter: Yang Hao
Assignee: Sumit Mohanty
Fix For: Slider 0.70
Attachments: SLIDER-740.patch
It would be very helpful that we can add some additional jvm options to the
HBASE cluster. An alternative way is to configure HBASE_OPTS. Adding this
patch, user can make it by adding site.hbase-env.hbase_opts in
appConf.json, such as
site.hbase-env.hbase_opts:
-Djava.security.krb5.conf=/home/work/app/yarn/lgtst-xiaomi2/nodemanager/krb5.conf
,
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-641) multiple slider-agent tarballs in lib dir
[ https://issues.apache.org/jira/browse/SLIDER-641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286472#comment-14286472 ] Sumit Mohanty commented on SLIDER-641: -- The one is the lib/ is automatically picked up when creating the application package. The one in the agent/ is not being used as of now. multiple slider-agent tarballs in lib dir - Key: SLIDER-641 URL: https://issues.apache.org/jira/browse/SLIDER-641 Project: Slider Issue Type: Bug Components: build Affects Versions: Slider 0.60 Reporter: Steve Loughran Assignee: Steve Loughran full builds create two versions of the agent tarball, one with a version number, one without -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-756) Create accumulo client script
[
https://issues.apache.org/jira/browse/SLIDER-756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty updated SLIDER-756:
-
Sprint: Slider Jan #2
Create accumulo client script
-
Key: SLIDER-756
URL: https://issues.apache.org/jira/browse/SLIDER-756
Project: Slider
Issue Type: Improvement
Components: app-package
Reporter: Billie Rinaldi
Assignee: Billie Rinaldi
Fix For: Slider 0.70
Attachments: SLIDER-756.1.patch
The idea behind this script is to make it easier to run the accumulo script
(particularly the accumulo shell) against an accumulo instance running on
slider. The basic idea is to have:
{noformat}
accumulo-slider --app appName accumuloCommand
{noformat}
and a few extra commands like:
{noformat}
accumulo-slider --app appName quicklinks
accumulo-slider --app appName proxies
accumulo-slider --app appName --appconf confDir getconf
accumulo-slider --app appName install installDir
{noformat}
To run accumulo commands, the script requires ACCUMULO_HOME and SLIDER_HOME
to be set in the environment.
For the getconf command, you can download the conf to a specified directory,
and in subsequent commands to accumulo-slider you could use the following to
avoid having to download the conf each time:
{noformat}
accumulo-slider --appconf confDir command
{noformat}
For the install command, you specify an install dir, and it will pull down
the app package, untar the accumulo tarball, and configure it. (I have
concerns that this may be a brittle process.) Once this install dir is made,
you could use it as a normal accumulo installation, using the accumulo script
directly instead of the accumulo-slider wrapper script. The install dir
could also be used as an ACCUMULO_HOME for future calls to accumulo-slider
for other app instances.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-740) support configurating HBASE_OPTS
[ https://issues.apache.org/jira/browse/SLIDER-740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-740: - Fix Version/s: Slider 0.70 support configurating HBASE_OPTS Key: SLIDER-740 URL: https://issues.apache.org/jira/browse/SLIDER-740 Project: Slider Issue Type: Improvement Components: app-package Affects Versions: Slider 0.60 Reporter: Yang Hao Assignee: Sumit Mohanty Fix For: Slider 0.70 Attachments: SLIDER-740.patch It would be very helpful that we can add some additional jvm options to the HBASE cluster. An alternative way is to configure HBASE_OPTS. Adding this patch, user can make it by adding site.hbase-env.hbase_opts in appConf.json, such as site.hbase-env.hbase_opts: -Djava.security.krb5.conf=/home/work/app/yarn/lgtst-xiaomi2/nodemanager/krb5.conf , -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-755) AgentProvider doesn't raise and exception when the agent tar isn't there
[
https://issues.apache.org/jira/browse/SLIDER-755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty updated SLIDER-755:
-
Sprint: Slider Jan #2
AgentProvider doesn't raise and exception when the agent tar isn't there
Key: SLIDER-755
URL: https://issues.apache.org/jira/browse/SLIDER-755
Project: Slider
Issue Type: Bug
Components: agent-provider
Affects Versions: Slider 0.60
Reporter: Steve Loughran
Assignee: Sumit Mohanty
Fix For: Slider 0.70
Agent provider line 325 says
{code}
// TODO: throw exception when agent tarball is not available
{code}
all it does is log an error
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-740) support configurating HBASE_OPTS
[ https://issues.apache.org/jira/browse/SLIDER-740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-740: - Assignee: Sumit Mohanty support configurating HBASE_OPTS Key: SLIDER-740 URL: https://issues.apache.org/jira/browse/SLIDER-740 Project: Slider Issue Type: Improvement Components: app-package Affects Versions: Slider 0.60 Reporter: Yang Hao Assignee: Sumit Mohanty Attachments: SLIDER-740.patch It would be very helpful that we can add some additional jvm options to the HBASE cluster. An alternative way is to configure HBASE_OPTS. Adding this patch, user can make it by adding site.hbase-env.hbase_opts in appConf.json, such as site.hbase-env.hbase_opts: -Djava.security.krb5.conf=/home/work/app/yarn/lgtst-xiaomi2/nodemanager/krb5.conf , -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-740) support configurating HBASE_OPTS
[ https://issues.apache.org/jira/browse/SLIDER-740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-740: - Sprint: Slider Jan #2 support configurating HBASE_OPTS Key: SLIDER-740 URL: https://issues.apache.org/jira/browse/SLIDER-740 Project: Slider Issue Type: Improvement Components: app-package Affects Versions: Slider 0.60 Reporter: Yang Hao Attachments: SLIDER-740.patch It would be very helpful that we can add some additional jvm options to the HBASE cluster. An alternative way is to configure HBASE_OPTS. Adding this patch, user can make it by adding site.hbase-env.hbase_opts in appConf.json, such as site.hbase-env.hbase_opts: -Djava.security.krb5.conf=/home/work/app/yarn/lgtst-xiaomi2/nodemanager/krb5.conf , -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-723) Memcached component launch fail does not propagate
[ https://issues.apache.org/jira/browse/SLIDER-723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-723: - Fix Version/s: Slider 0.70 Memcached component launch fail does not propagate -- Key: SLIDER-723 URL: https://issues.apache.org/jira/browse/SLIDER-723 Project: Slider Issue Type: Improvement Affects Versions: Slider 0.60 Reporter: Yang Hao Assignee: Sumit Mohanty Fix For: Slider 0.70 When am applies a container for the component ,and the container starts, but the component may not launch successfully. So a friendly way should be out. Like just kill the container when the component don't launch successfully. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-617) review conf/ directories in slider-assembly
[ https://issues.apache.org/jira/browse/SLIDER-617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-617: - Fix Version/s: (was: Slider 0.70) Slider 2.0.0 review conf/ directories in slider-assembly --- Key: SLIDER-617 URL: https://issues.apache.org/jira/browse/SLIDER-617 Project: Slider Issue Type: Test Reporter: Steve Loughran Fix For: Slider 2.0.0 have a look at the assembly/src/conf dirs and make sure we are happy with them -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-760) Release Slider 0.70
[ https://issues.apache.org/jira/browse/SLIDER-760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-760: - Sprint: Slider Feb #1 (was: Slider Jan #2) Release Slider 0.70 --- Key: SLIDER-760 URL: https://issues.apache.org/jira/browse/SLIDER-760 Project: Slider Issue Type: Brainstorming Components: build Affects Versions: Slider 0.70 Reporter: Steve Loughran Assignee: Gour Saha Fix For: Slider 0.70 Original Estimate: 24h Remaining Estimate: 24h -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-688) Zero touch install support
[ https://issues.apache.org/jira/browse/SLIDER-688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-688: - Sprint: Slider Jan #2 (was: Slider Jan #1) Zero touch install support -- Key: SLIDER-688 URL: https://issues.apache.org/jira/browse/SLIDER-688 Project: Slider Issue Type: Improvement Affects Versions: Slider 0.60 Reporter: Thomas Weise Assignee: Sumit Mohanty Currently the user needs to specify environment variable such as HADOOP_CONF_DIR and JAVA_HOME. Typlically the environment has Hadoop installed and distros have already provided the dependencies. User should not have to configure this for Slider. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-723) Memcached component launch fail does not propagate
[ https://issues.apache.org/jira/browse/SLIDER-723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-723: - Sprint: Slider Jan #2 (was: Slider Jan #1) Memcached component launch fail does not propagate -- Key: SLIDER-723 URL: https://issues.apache.org/jira/browse/SLIDER-723 Project: Slider Issue Type: Improvement Affects Versions: Slider 0.60 Reporter: Yang Hao Assignee: Sumit Mohanty When am applies a container for the component ,and the container starts, but the component may not launch successfully. So a friendly way should be out. Like just kill the container when the component don't launch successfully. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-685) Better propagation of config options from client to deployed service
[
https://issues.apache.org/jira/browse/SLIDER-685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty updated SLIDER-685:
-
Fix Version/s: (was: Slider 0.70)
Slider 2.0.0
Better propagation of config options from client to deployed service
Key: SLIDER-685
URL: https://issues.apache.org/jira/browse/SLIDER-685
Project: Slider
Issue Type: Improvement
Components: appmaster, client
Affects Versions: Slider 0.60
Reporter: Steve Loughran
Fix For: Slider 2.0.0
We are only propagating a few specific options from the slider client to the
AM: FS binding, non-HA RM binding, registry binding.
This makes it hard to set server-side properties such as metrics options
I propose that we propagate all options from slider-client.xml, leaving all
that are pulled in indirectly from the -site.xml files alone. Also: include
all properties set on the command line with {{-D key=value}}. As these are
picked up before the client is instantiated, we'll need to rummage into the
Configuration and filter values based on their provenance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-743) Include node failure history when choosing placement hints
[ https://issues.apache.org/jira/browse/SLIDER-743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-743: - Assignee: Sumit Mohanty Include node failure history when choosing placement hints -- Key: SLIDER-743 URL: https://issues.apache.org/jira/browse/SLIDER-743 Project: Slider Issue Type: Improvement Components: appmaster Affects Versions: Slider 0.60 Reporter: Steve Loughran Assignee: Sumit Mohanty Fix For: Slider 0.70 Use the (collected) history on per-node failures to tune placement requests, so that even if there's a history of components running there, if its been failing recently then it's not asked for # Add a recent failures field to the node entries, alongside the total failures counter # the scheduled failure count resetter will set that field to zero,alongside the component failures # When Slider has to request a new container, if a node has a recent failure count above some threshold, it will not be requested, even if it is a past location of that component. When the placement policy is STRICT, node failure history will not be taken into account -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-703) Slider extensibility
[ https://issues.apache.org/jira/browse/SLIDER-703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-703: - Fix Version/s: (was: Slider 0.70) Slider 2.0.0 Slider extensibility Key: SLIDER-703 URL: https://issues.apache.org/jira/browse/SLIDER-703 Project: Slider Issue Type: Task Components: app-package, appmaster Affects Versions: Slider 0.70 Reporter: Sumit Mohanty Assignee: Sumit Mohanty Fix For: Slider 2.0.0 Slider does not yet provide functional extensibility for applications where the application can extend the behavior of the AppMaster and/or the Agent to provide some custom behavior. Such extension is by definition localized to application instance and thus will not affect other application instances. Such extension are an easy way to add plugins into Agent and AppMaster which would otherwise have needed creation of component types. Possible examples are: * Have agent report back some application specific data through the heartbeat * Have AppMaster show some application specific data through the REST end point * The above REST endpoint could be read-only or it can have limited PUT/POST capabilities * AppMaster may allow plugins that perform application instance specific flexing, container allocation that are influenced by current application state, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-723) Memcached component launch fail does not propagate
[ https://issues.apache.org/jira/browse/SLIDER-723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14282123#comment-14282123 ] Sumit Mohanty commented on SLIDER-723: -- Yes, that's what I am targeting. Memcached component launch fail does not propagate -- Key: SLIDER-723 URL: https://issues.apache.org/jira/browse/SLIDER-723 Project: Slider Issue Type: Improvement Affects Versions: Slider 0.60 Reporter: Yang Hao Assignee: Sumit Mohanty When am applies a container for the component ,and the container starts, but the component may not launch successfully. So a friendly way should be out. Like just kill the container when the component don't launch successfully. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-755) AgentProvider doesn't raise and exception when the agent tar isn't there
[
https://issues.apache.org/jira/browse/SLIDER-755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty updated SLIDER-755:
-
Fix Version/s: Slider 0.70
AgentProvider doesn't raise and exception when the agent tar isn't there
Key: SLIDER-755
URL: https://issues.apache.org/jira/browse/SLIDER-755
Project: Slider
Issue Type: Bug
Components: agent-provider
Affects Versions: Slider 0.60
Reporter: Steve Loughran
Assignee: Sumit Mohanty
Fix For: Slider 0.70
Agent provider line 325 says
{code}
// TODO: throw exception when agent tarball is not available
{code}
all it does is log an error
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (SLIDER-755) AgentProvider doesn't raise and exception when the agent tar isn't there
[
https://issues.apache.org/jira/browse/SLIDER-755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty reassigned SLIDER-755:
Assignee: Sumit Mohanty
AgentProvider doesn't raise and exception when the agent tar isn't there
Key: SLIDER-755
URL: https://issues.apache.org/jira/browse/SLIDER-755
Project: Slider
Issue Type: Bug
Components: agent-provider
Affects Versions: Slider 0.60
Reporter: Steve Loughran
Assignee: Sumit Mohanty
Fix For: Slider 0.70
Agent provider line 325 says
{code}
// TODO: throw exception when agent tarball is not available
{code}
all it does is log an error
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-736) build hbase mvn command is not intuitive
[
https://issues.apache.org/jira/browse/SLIDER-736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty updated SLIDER-736:
-
Priority: Minor (was: Major)
build hbase mvn command is not intuitive
Key: SLIDER-736
URL: https://issues.apache.org/jira/browse/SLIDER-736
Project: Slider
Issue Type: Bug
Components: application/hbase
Reporter: Yang Hao
Priority: Minor
Labels: 0.60
when I execute the command refers to app-packages/hbase/README.md ,the target
file is not slider-hbase-app-package-${pkg.version}.zip but
slider-hbase-app-package-0.60.0-incubating.zip
mvn clean package -Phbase-app-package -Dpkg.version=0.98.9-hadoop2
-Dpkg.name=hbase-0.98.9-hadoop2-bin.tar.gz -Prelease-2.6
-Dpkg.src=/home/yang/ -DskipTests
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-736) build hbase mvn command is not intuitive
[
https://issues.apache.org/jira/browse/SLIDER-736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14272183#comment-14272183
]
Sumit Mohanty commented on SLIDER-736:
--
{{pkg.version}} is not named appropriately. In develop branch (for next
release) it is changed to {{hbase.version}}. Its the version of the tarball
that is included in the Slider app package. The app pkg version is picked up
from the version of the Slider build.
build hbase mvn command is not intuitive
Key: SLIDER-736
URL: https://issues.apache.org/jira/browse/SLIDER-736
Project: Slider
Issue Type: Bug
Components: application/hbase
Reporter: Yang Hao
Labels: 0.60
when I execute the command refers to app-packages/hbase/README.md ,the target
file is not slider-hbase-app-package-${pkg.version}.zip but
slider-hbase-app-package-0.60.0-incubating.zip
mvn clean package -Phbase-app-package -Dpkg.version=0.98.9-hadoop2
-Dpkg.name=hbase-0.98.9-hadoop2-bin.tar.gz -Prelease-2.6
-Dpkg.src=/home/yang/ -DskipTests
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-736) build hbase mvn command is not intuitive
[
https://issues.apache.org/jira/browse/SLIDER-736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumit Mohanty updated SLIDER-736:
-
Summary: build hbase mvn command is not intuitive (was: build hbase)
build hbase mvn command is not intuitive
Key: SLIDER-736
URL: https://issues.apache.org/jira/browse/SLIDER-736
Project: Slider
Issue Type: Bug
Components: application/hbase
Reporter: Yang Hao
Labels: 0.60
when I execute the command refers to app-packages/hbase/README.md ,the target
file is not slider-hbase-app-package-${pkg.version}.zip but
slider-hbase-app-package-0.60.0-incubating.zip
mvn clean package -Phbase-app-package -Dpkg.version=0.98.9-hadoop2
-Dpkg.name=hbase-0.98.9-hadoop2-bin.tar.gz -Prelease-2.6
-Dpkg.src=/home/yang/ -DskipTests
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLIDER-723) Memcached component launch fail does not propagate
[ https://issues.apache.org/jira/browse/SLIDER-723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270334#comment-14270334 ] Sumit Mohanty commented on SLIDER-723: -- Reproduced the scenario: * Container fails at the INSTALL step * The agent's logic to monitor process health only starts after the START is issued * Agent reports back failure to AM and AM issues 2 more retries (default behavior) In the end the agent remains running as it does not have to logic to go down in case of install failures. Its a BUG. Agent needs to go down if INSTALL fails. Memcached component launch fail does not propagate -- Key: SLIDER-723 URL: https://issues.apache.org/jira/browse/SLIDER-723 Project: Slider Issue Type: Improvement Affects Versions: Slider 0.60 Reporter: Yang Hao When am applies a container for the component ,and the container starts, but the component may not launch successfully. So a friendly way should be out. Like just kill the container when the component don't launch successfully. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-723) Memcached component launch fail does not propagate
[ https://issues.apache.org/jira/browse/SLIDER-723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-723: - Assignee: Sumit Mohanty Memcached component launch fail does not propagate -- Key: SLIDER-723 URL: https://issues.apache.org/jira/browse/SLIDER-723 Project: Slider Issue Type: Improvement Affects Versions: Slider 0.60 Reporter: Yang Hao Assignee: Sumit Mohanty When am applies a container for the component ,and the container starts, but the component may not launch successfully. So a friendly way should be out. Like just kill the container when the component don't launch successfully. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-726) Different instances of the same component should run on different nodes (anti-affinity)
[ https://issues.apache.org/jira/browse/SLIDER-726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-726: - Summary: Different instances of the same component should run on different nodes (anti-affinity) (was: To run on different nodes) Different instances of the same component should run on different nodes (anti-affinity) --- Key: SLIDER-726 URL: https://issues.apache.org/jira/browse/SLIDER-726 Project: Slider Issue Type: New Feature Components: appmaster, client Affects Versions: Slider 0.60 Reporter: Yang Hao To let component run on different will be very useful, for these reasons 1. many distributed services want to or should be run on different nodes 2. slider try to let yarn be a common resource OS If this feature is add, slider will be more popular -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLIDER-398) memcached package is broken in master
[ https://issues.apache.org/jira/browse/SLIDER-398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty resolved SLIDER-398. -- Resolution: Fixed We can resolve this as 0.60 had the fix as well (I forgot to cherry-pick and instead committed twice). master was updated out of band as it was the reference branch for trying out slider and a broken memcached package was creating problems for the folks trying it out. memcached package is broken in master - Key: SLIDER-398 URL: https://issues.apache.org/jira/browse/SLIDER-398 Project: Slider Issue Type: Bug Affects Versions: Slider 0.50 Reporter: Sumit Mohanty Assignee: Sumit Mohanty -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLIDER-566) Default ZK node created by Slider does not have any ACL
[ https://issues.apache.org/jira/browse/SLIDER-566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty resolved SLIDER-566. -- Resolution: Not a Problem Apps are free to add ACL to sub-nodes created under the default path. This is what HBase does as well. Default ZK node created by Slider does not have any ACL --- Key: SLIDER-566 URL: https://issues.apache.org/jira/browse/SLIDER-566 Project: Slider Issue Type: Documentation Components: app-package Affects Versions: Slider 0.50 Reporter: Sumit Mohanty Assignee: Sumit Mohanty Fix For: Slider 2.0.0 Document that the default ZK node created by Slider when applications expect a default node is not associated with any ACL. The application is responsible for adding its own sub-nodes and adding ACLs as needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-688) Zero touch install support
[ https://issues.apache.org/jira/browse/SLIDER-688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-688: - Sprint: Slider Jan #1 Zero touch install support -- Key: SLIDER-688 URL: https://issues.apache.org/jira/browse/SLIDER-688 Project: Slider Issue Type: Improvement Affects Versions: Slider 0.60 Reporter: Thomas Weise Assignee: Sumit Mohanty Currently the user needs to specify environment variable such as HADOOP_CONF_DIR and JAVA_HOME. Typlically the environment has Hadoop installed and distros have already provided the dependencies. User should not have to configure this for Slider. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-723) Memcached component launch fail does not propagate
[ https://issues.apache.org/jira/browse/SLIDER-723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Mohanty updated SLIDER-723: - Sprint: Slider Jan #1 Memcached component launch fail does not propagate -- Key: SLIDER-723 URL: https://issues.apache.org/jira/browse/SLIDER-723 Project: Slider Issue Type: Improvement Affects Versions: Slider 0.60 Reporter: Yang Hao Assignee: Sumit Mohanty When am applies a container for the component ,and the container starts, but the component may not launch successfully. So a friendly way should be out. Like just kill the container when the component don't launch successfully. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SLIDER-701) Support alerts for Slider Apps
Sumit Mohanty created SLIDER-701: Summary: Support alerts for Slider Apps Key: SLIDER-701 URL: https://issues.apache.org/jira/browse/SLIDER-701 Project: Slider Issue Type: Task Components: agent, app-package, appmaster Affects Versions: Slider 0.70 Reporter: Sumit Mohanty Assignee: Sumit Mohanty Fix For: Slider 0.70 Traditional deployment of apps typically include alerts configured for alerting systems such as Nagios. This includes configuring the alerting system to check various data points such as live port, jmx data, etc. For a slider app similar configurations may be defined while being aware of the fact that the application components may move during the life time of the application. Additionally, YARN/Slider provides several status information (e.g. live component instance count) that can be used for alerts. This task covers investigation into various alerting infrastructure and providing recommendation or solution for specific alerting infrastructures for Slider apps. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
