[VOTE] Apache Slider Incubating Release 0.70.0-incubating
Hello, This is a call for a vote on Apache Slider Incubating 0.70.0-incubating release. This is a source+binary release. The issues fixed in this release are listed at at: https://issues.apache.org/jira/browse/SLIDER/fixforversion/12327847 (or the shortened URL http://s.apache.org/AnM) Artifacts at https://repository.apache.org/content/repositories/orgapacheslider-1004/org/apache/slider Git source tag: https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;a=tag;h=refs/tags/slider-0.70.0-incubating PGP keys at http://pgp.mit.edu/pks/lookup?op=vindex&search=gourks...@apache.org Build instructions at: http://slider.incubator.apache.org/developing/building.html Vote will be open for 72 hours [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) To start, here's my vote: +1 -Gour
[jira] [Updated] (SLIDER-760) Release Slider 0.70
[ https://issues.apache.org/jira/browse/SLIDER-760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-760: - Fix Version/s: (was: Slider 0.80) Slider 0.70 > Release Slider 0.70 > --- > > Key: SLIDER-760 > URL: https://issues.apache.org/jira/browse/SLIDER-760 > Project: Slider > Issue Type: Brainstorming > Components: build >Affects Versions: Slider 0.70 >Reporter: Steve Loughran >Assignee: Gour Saha > Fix For: Slider 0.70 > > Attachments: selective-release-of-modules.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > Slider version to be released is 0.70.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-208) Ongoing: keep Jenkins builds working
[ https://issues.apache.org/jira/browse/SLIDER-208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-208: - Fix Version/s: (was: Slider 0.70) Slider 0.80 > Ongoing: keep Jenkins builds working > > > Key: SLIDER-208 > URL: https://issues.apache.org/jira/browse/SLIDER-208 > Project: Slider > Issue Type: Task > Components: build >Affects Versions: Slider 0.40 >Reporter: Steve Loughran > Fix For: Slider 0.80 > > > Uber-JIRA that never gets finished: keep Jenkins working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-681) [FUN TEST] Validate that Slider App remains active if NodeManager is killed on the host where AppMaster is running
[ https://issues.apache.org/jira/browse/SLIDER-681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-681: - Fix Version/s: (was: Slider 0.70) Slider 0.80 > [FUN TEST] Validate that Slider App remains active if NodeManager is killed > on the host where AppMaster is running > -- > > Key: SLIDER-681 > URL: https://issues.apache.org/jira/browse/SLIDER-681 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.60 >Reporter: Sumit Mohanty >Assignee: Gour Saha > Fix For: Slider 0.80 > > > NodeManager may go down on the host where AppMaster is running. > The test should verify that the application remains active. If AppMaster is > restarted on another node then the original AppMaster should gracefully go > down and control should get transferred to the new AppMaster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-682) [FUN TEST] Verify App destroy and cleanup
[ https://issues.apache.org/jira/browse/SLIDER-682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-682: - Fix Version/s: (was: Slider 0.70) Slider 0.80 > [FUN TEST] Verify App destroy and cleanup > - > > Key: SLIDER-682 > URL: https://issues.apache.org/jira/browse/SLIDER-682 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.60 >Reporter: Sumit Mohanty >Assignee: Sumit Mohanty > Fix For: Slider 0.80 > > > Verify that destroy command cleans all default resources created for the > application as well as does not destroy any non-default resources that app is > using. The default resources created for the application are: > * Yarn application > * ZK node > * HDFS data dir -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-697) Handle certificate/keystore expiry for Agent/AM SSL communication
[ https://issues.apache.org/jira/browse/SLIDER-697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-697: - Fix Version/s: (was: Slider 0.70) Slider 0.80 > Handle certificate/keystore expiry for Agent/AM SSL communication > - > > Key: SLIDER-697 > URL: https://issues.apache.org/jira/browse/SLIDER-697 > Project: Slider > Issue Type: Sub-task > Components: agent, appmaster, security >Reporter: Jonathan Maron >Assignee: Jonathan Maron > Fix For: Slider 0.80 > > > Look at the possibility of being able to distribute updated certificates in > the case of a long-running application that run beyond the certificate expiry > period (server certificate for one way SSL, agent and AM for two way) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-760) Release Slider 0.70
[ https://issues.apache.org/jira/browse/SLIDER-760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-760: - Fix Version/s: (was: Slider 0.70) Slider 0.80 > Release Slider 0.70 > --- > > Key: SLIDER-760 > URL: https://issues.apache.org/jira/browse/SLIDER-760 > Project: Slider > Issue Type: Brainstorming > Components: build >Affects Versions: Slider 0.70 >Reporter: Steve Loughran >Assignee: Gour Saha > Fix For: Slider 0.80 > > Attachments: selective-release-of-modules.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > Slider version to be released is 0.70.0-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-749) jenkins on windows failing -disk?
[ https://issues.apache.org/jira/browse/SLIDER-749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-749: - Fix Version/s: (was: Slider 0.70) Slider 0.80 > jenkins on windows failing -disk? > - > > Key: SLIDER-749 > URL: https://issues.apache.org/jira/browse/SLIDER-749 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.70 > Environment: Windows >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.80 > > Original Estimate: 1h > Time Spent: 2h > Remaining Estimate: 1h > > A windows-based jenkins server is failing, even though the tests pass for me. > it looks FS related. > # improve reporting on problems (i.e. make sure all the assertions are > meaningful) > # add specific tests to look at tmp dir writes & reads -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-686) [FUN TEST]: flexing
[ https://issues.apache.org/jira/browse/SLIDER-686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-686: - Fix Version/s: (was: Slider 0.70) Slider 0.80 > [FUN TEST]: flexing > --- > > Key: SLIDER-686 > URL: https://issues.apache.org/jira/browse/SLIDER-686 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.60 >Reporter: Steve Loughran > Fix For: Slider 0.80 > > > Test flexing in a functional cluster > # down > # up > # no-op > # down one role, up another > # adding a new role entirely > # labelled cluster > # request more instances than can be granted, flex down, verify the > outstanding request is the instance released > # flex value staying constant over AM restart -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-695) Explore mechanisms for establishing doAs identity
[ https://issues.apache.org/jira/browse/SLIDER-695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-695: - Fix Version/s: (was: Slider 0.70) Slider 0.80 > Explore mechanisms for establishing doAs identity > - > > Key: SLIDER-695 > URL: https://issues.apache.org/jira/browse/SLIDER-695 > Project: Slider > Issue Type: Sub-task > Components: appmaster, security, Web & REST >Reporter: Jonathan Maron >Assignee: Jonathan Maron > Fix For: Slider 0.80 > > > When know invokes a slider rest api it will be doing so on behalf on an end > user. There is a need to establish that "doAs" context for the invocation of > Slider, so mechanisms such as those used by oozie (e.g. > hadoop.proxyuser.oozie.* properties) need to be examined and evaluated for > use in slider. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-629) Slider's count of failure threshold may not be accurate or it could be a logging issue
[
https://issues.apache.org/jira/browse/SLIDER-629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gour Saha updated SLIDER-629:
-
Fix Version/s: (was: Slider 0.70)
Slider 0.80
> Slider's count of failure threshold may not be accurate or it could be a
> logging issue
> --
>
> Key: SLIDER-629
> URL: https://issues.apache.org/jira/browse/SLIDER-629
> Project: Slider
> Issue Type: Bug
> Components: appmaster
>Affects Versions: Slider 0.50
>Reporter: Sumit Mohanty
>Assignee: Jonathan Maron
> Fix For: Slider 0.80
>
>
> One of the long running HBase tests failed with the following error:
> {noformat}
> 2014-11-08 01:07:26,407 [AmExecutor-008] ERROR appmaster.SliderAppMaster -
> Cluster teardown triggered
> org.apache.slider.core.exceptions.TriggerClusterTeardownException: Unstable
> Application Instance : - failed with component H BASE_REGIONSERVER
> failing 8 times (0 in startup); threshold is 5 - last failure: Failure
> container_1415341585168_0005_01_08 on host onprem-slider23:
> http://onprem-slider21:19888/jobhistory/logs/onprem-slider23:45454/contai
> ner_1415341585168_0005_01_08/ctx/hadoop^M
> {noformat}
> However, there were total of "9" REGION_SERVERs created.
> {noformat}
> 2014-11-07 16:00:35,346 [AMRM Callback Handler Thread] INFO state.AppState -
> Assigning role HBASE_REGIONSERVER to container
> container_1415341585168_0005_01_02, on onprem-slider25:45454,
> 2014-11-07 16:00:35,347 [AMRM Callback Handler Thread] INFO state.AppState -
> Assigning role HBASE_REGIONSERVER to container
> container_1415341585168_0005_01_05, on onprem-slider24:45454,
> 2014-11-07 16:00:35,347 [AMRM Callback Handler Thread] INFO state.AppState -
> Assigning role HBASE_REGIONSERVER to container
> container_1415341585168_0005_01_07, on onprem-slider22:45454,
> 2014-11-07 16:00:35,347 [AMRM Callback Handler Thread] INFO state.AppState -
> Assigning role HBASE_REGIONSERVER to container
> container_1415341585168_0005_01_08, on onprem-slider23:45454,
> 2014-11-07 23:51:20,040 [AMRM Callback Handler Thread] INFO state.AppState -
> Assigning role HBASE_REGIONSERVER to container
> container_1415341585168_0005_01_09, on onprem-slider22:45454,
> 2014-11-07 23:58:44,810 [AMRM Callback Handler Thread] INFO state.AppState -
> Assigning role HBASE_REGIONSERVER to container
> container_1415341585168_0005_01_13, on onprem-slider24:45454,
> 2014-11-08 00:12:17,804 [AMRM Callback Handler Thread] INFO state.AppState -
> Assigning role HBASE_REGIONSERVER to container
> container_1415341585168_0005_01_15, on onprem-slider22:45454,
> 2014-11-08 00:15:57,373 [AMRM Callback Handler Thread] INFO state.AppState -
> Assigning role HBASE_REGIONSERVER to container
> container_1415341585168_0005_01_18, on onprem-slider25:45454,
> 2014-11-08 01:06:36,771 [AMRM Callback Handler Thread] INFO state.AppState -
> Assigning role HBASE_REGIONSERVER to container
> container_1415341585168_0005_01_20, on onprem-slider25:45454,
> {noformat}
> As the ask was for 4 but 9 were created, obviously there are 5 failures.
> Perhaps its a logging issue. Can we also print the Window - e.g. 5 failures
> in X minutes or hours.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-632) intermittent assertion failure of TestHeartbeatMonitor.testRegularHeartbeat
[
https://issues.apache.org/jira/browse/SLIDER-632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gour Saha updated SLIDER-632:
-
Fix Version/s: (was: Slider 0.70)
Slider 0.80
> intermittent assertion failure of TestHeartbeatMonitor.testRegularHeartbeat
> ---
>
> Key: SLIDER-632
> URL: https://issues.apache.org/jira/browse/SLIDER-632
> Project: Slider
> Issue Type: Sub-task
> Components: build
> Environment: mac with a VM doing lots of work in the background
>Reporter: Steve Loughran
>Assignee: Gour Saha
>Priority: Minor
> Fix For: Slider 0.80
>
>
> I got a failure of {{TestHeartbeatMonitortestRegularHeartbeat}} while doing a
> full core test. I did not see it when I ran the single test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-693) Need to enable support for spnego/kerberos for AM Rest resources
[ https://issues.apache.org/jira/browse/SLIDER-693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-693: - Fix Version/s: (was: Slider 0.70) Slider 0.80 > Need to enable support for spnego/kerberos for AM Rest resources > > > Key: SLIDER-693 > URL: https://issues.apache.org/jira/browse/SLIDER-693 > Project: Slider > Issue Type: Sub-task > Components: appmaster, security >Reporter: Jonathan Maron >Assignee: Jonathan Maron > Fix For: Slider 0.80 > > > Need to add the spnego support API calls to the AM WebApp to enable the > kerberos configuration for the AM exposed REST endpoint (Not agent facing > server). This is required to support trust proxy configuration for knox > integration. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-712) Implement REST app setup resources
[ https://issues.apache.org/jira/browse/SLIDER-712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-712: - Fix Version/s: (was: Slider 0.70) Slider 0.80 > Implement REST app setup resources > -- > > Key: SLIDER-712 > URL: https://issues.apache.org/jira/browse/SLIDER-712 > Project: Slider > Issue Type: Sub-task > Components: Web & REST >Affects Versions: Slider 0.70 >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.80 > > Original Estimate: 40h > Remaining Estimate: 40h > > Add the resources to set up the AM, that is set app config/resources, > possibly combined. > This will be complex behind the endpoints, and need to be matched by client > operations for tests and future migration. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-717) Migrate slider client to slider REST API
[ https://issues.apache.org/jira/browse/SLIDER-717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-717: - Fix Version/s: (was: Slider 0.70) Slider 0.80 > Migrate slider client to slider REST API > > > Key: SLIDER-717 > URL: https://issues.apache.org/jira/browse/SLIDER-717 > Project: Slider > Issue Type: Sub-task > Components: Web & REST >Affects Versions: Slider 0.70 >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.80 > > Original Estimate: 32h > Remaining Estimate: 32h > > Assuming client-side REST operations have been implemented, move the client > to issuing REST calls to the AM. > # YARN-2031 is a pre-requisite > # This will not be compatible with Hadoop 2.6 due to the YARN proxy > This *may* involve significant changes to the client, hence estimate of 4d. > Hopefully it will not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-684) [FUN TEST] Verify expectations when a NodeManager is killed
[ https://issues.apache.org/jira/browse/SLIDER-684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-684: - Fix Version/s: (was: Slider 0.70) Slider 0.80 > [FUN TEST] Verify expectations when a NodeManager is killed > --- > > Key: SLIDER-684 > URL: https://issues.apache.org/jira/browse/SLIDER-684 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.60 >Reporter: Sumit Mohanty > Fix For: Slider 0.80 > > > When NodeManager is killed, container on that host should get killed (after > the time interval for YARN to wait for NM restart) and the container should > get replaced by instances on other hosts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-718) Slider client to support remote file operations via WebHDFS
[ https://issues.apache.org/jira/browse/SLIDER-718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-718: - Fix Version/s: (was: Slider 0.70) Slider 0.80 > Slider client to support remote file operations via WebHDFS > --- > > Key: SLIDER-718 > URL: https://issues.apache.org/jira/browse/SLIDER-718 > Project: Slider > Issue Type: Sub-task > Components: client >Affects Versions: Slider 0.70 >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.80 > > Original Estimate: 16h > Remaining Estimate: 16h > > If slider is to work long-haul, it will need to work (client-side) with the > WebHDFS HDFS API -yet potentially set up paths/configs & yarn resources to > bond to the hdfs:// URLs in the destination cluster. > The issue here is not webhdfs directly (simple testing), but the possible > difference between client-side URLs and in-cluster paths. > Whatever other YARN apps (will) do here, slider must copy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-699) AM HTTPS support
[ https://issues.apache.org/jira/browse/SLIDER-699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-699: - Fix Version/s: (was: Slider 0.70) Slider 0.80 > AM HTTPS support > > > Key: SLIDER-699 > URL: https://issues.apache.org/jira/browse/SLIDER-699 > Project: Slider > Issue Type: Sub-task > Components: appmaster, security, Web & REST >Reporter: Jonathan Maron >Assignee: Jonathan Maron > Fix For: Slider 0.80 > > > Examine the feasibility of providing direct HTTPS support for the AM REST/Web > endpoint. This would require support for AM specific keystores, availability > of AM certificate to associated yarn RM etc/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-698) AM keytab creation and distribution in Ambari managed cluster
[ https://issues.apache.org/jira/browse/SLIDER-698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-698: - Fix Version/s: (was: Slider 0.70) Slider 0.80 > AM keytab creation and distribution in Ambari managed cluster > - > > Key: SLIDER-698 > URL: https://issues.apache.org/jira/browse/SLIDER-698 > Project: Slider > Issue Type: Sub-task > Components: appmaster, security >Reporter: Jonathan Maron > Fix For: Slider 0.80 > > > Look the integration requirements (if any) for leveraging keytab facilities > in ambari to support AM keytab creation and distribution. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-722) Front page to list YARN resource requirements of each component
[ https://issues.apache.org/jira/browse/SLIDER-722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-722: - Fix Version/s: (was: Slider 0.70) Slider 0.80 > Front page to list YARN resource requirements of each component > --- > > Key: SLIDER-722 > URL: https://issues.apache.org/jira/browse/SLIDER-722 > Project: Slider > Issue Type: Improvement > Components: Web & REST >Affects Versions: Slider 0.60 >Reporter: Steve Loughran >Priority: Minor > Fix For: Slider 0.80 > > > It will help debugging why requests are outstanding if the front page view > listed > all the yarn RAM, cpu and label requirements for each component. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-687) [FUN TEST] Failure windows
[ https://issues.apache.org/jira/browse/SLIDER-687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-687: - Fix Version/s: (was: Slider 0.70) Slider 0.80 > [FUN TEST] Failure windows > -- > > Key: SLIDER-687 > URL: https://issues.apache.org/jira/browse/SLIDER-687 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.60 >Reporter: Steve Loughran > Fix For: Slider 0.80 > > > Test AM and YARN failure windows > set them to be very short, then > # verify that failure rates within the window trigger app failure > # verify that a failure rate slower than the window size does not trigger app > failure > We know that a restarted AM resets the component window; there's no need to > test for that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-714) Expose REST view of containers/component instances with DELETE operation
[
https://issues.apache.org/jira/browse/SLIDER-714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gour Saha updated SLIDER-714:
-
Fix Version/s: (was: Slider 0.70)
Slider 0.80
> Expose REST view of containers/component instances with DELETE operation
>
>
> Key: SLIDER-714
> URL: https://issues.apache.org/jira/browse/SLIDER-714
> Project: Slider
> Issue Type: Sub-task
> Components: Web & REST
>Affects Versions: Slider 0.70
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Fix For: Slider 0.80
>
> Original Estimate: 16h
> Remaining Estimate: 16h
>
> Provide a view of the containers for each resource, including state, maybe
> config. Support DELETE to kill a container; this will not change the cluster
> size, merely decommission that container.
> Will need decommissioning support in AM: list of decommissions in progress,
> plus tests.
> the current {{kill-container}} command does not decommission, it simply
> kills, and is for testing purposes —the AM receives the reports and treats
> them as failures
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-716) Offer REST upgrade operation
[ https://issues.apache.org/jira/browse/SLIDER-716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-716: - Fix Version/s: (was: Slider 0.70) Slider 0.80 > Offer REST upgrade operation > > > Key: SLIDER-716 > URL: https://issues.apache.org/jira/browse/SLIDER-716 > Project: Slider > Issue Type: Sub-task > Components: Web & REST >Affects Versions: Slider 0.70 >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: Slider 0.80 > > Original Estimate: 8h > Remaining Estimate: 8h > > Provide a REST operation to trigger an upgrade. > Depends on the upgrade operation being implemented. > Assumes upgrade tests have already been written; this operation switches them > from IPC to REST, & rerunning tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLIDER-552) remove root registry path setup in AM
[
https://issues.apache.org/jira/browse/SLIDER-552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gour Saha updated SLIDER-552:
-
Fix Version/s: (was: Slider 0.70)
Slider 0.80
> remove root registry path setup in AM
> -
>
> Key: SLIDER-552
> URL: https://issues.apache.org/jira/browse/SLIDER-552
> Project: Slider
> Issue Type: Sub-task
> Components: registry
>Affects Versions: Slider 0.60
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Critical
> Fix For: Slider 0.80
>
>
> once YARN-2571 is in, downgrade the {{RegistryOperations}} instance
> instantied in the slider AM from a {{RegistryRMOperationsService}} —which
> creates the root paths on startup— to a normal client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLIDER-683) [FUN TEST] Verify that restarted app preserve data as well as republish
[ https://issues.apache.org/jira/browse/SLIDER-683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated SLIDER-683: - Fix Version/s: (was: Slider 0.70) Slider 0.80 > [FUN TEST] Verify that restarted app preserve data as well as republish > --- > > Key: SLIDER-683 > URL: https://issues.apache.org/jira/browse/SLIDER-683 > Project: Slider > Issue Type: Sub-task > Components: test >Affects Versions: Slider 0.60 >Reporter: Sumit Mohanty > Fix For: Slider 0.80 > > > Add a functional test that verifies that an application, when restarted, > preserves data/config as well as republishes all information that it had > published before. > A variations of this test will be to restart > * AppMaster > * Key components (e.g. HBase Master) > and ensure that data is preserved as well as data are republished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLIDER-802) Need mechanism for client truststore and keystore retrieval
[ https://issues.apache.org/jira/browse/SLIDER-802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Maron resolved SLIDER-802. --- Resolution: Fixed > Need mechanism for client truststore and keystore retrieval > --- > > Key: SLIDER-802 > URL: https://issues.apache.org/jira/browse/SLIDER-802 > Project: Slider > Issue Type: Bug > Components: client, security >Reporter: Jonathan Maron >Assignee: Jonathan Maron > > For applications that take advantage of the certificate store generation > feature, there may be a need for the client's to obtain truststores and, > optionally, keystores to successfully communicate via SSL to the server-side > application components. > One possible approach is a CLI option: > slider ssl-support --keystore --subj > Some key points: > 1) Could use the localhost from where the client is invoked to create the > subject DN (CN=) if no subject is specified > 2) For clients to access these stores they will need access to store > passwords. I believe it's be most appropriate to leverage the Credential > Provider facility. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-802) Need mechanism for client truststore and keystore retrieval
[ https://issues.apache.org/jira/browse/SLIDER-802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351330#comment-14351330 ] ASF subversion and git services commented on SLIDER-802: Commit c1b83467cebc90b4e258d963ac8c5d90b85f3b1c in incubator-slider's branch refs/heads/feature/SLIDER-802-cert-sign from [~jmaron] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=c1b8346 ] SLIDER-802 add client keystore and trustore retrieval commands > Need mechanism for client truststore and keystore retrieval > --- > > Key: SLIDER-802 > URL: https://issues.apache.org/jira/browse/SLIDER-802 > Project: Slider > Issue Type: Bug > Components: client, security >Reporter: Jonathan Maron >Assignee: Jonathan Maron > > For applications that take advantage of the certificate store generation > feature, there may be a need for the client's to obtain truststores and, > optionally, keystores to successfully communicate via SSL to the server-side > application components. > One possible approach is a CLI option: > slider ssl-support --keystore --subj > Some key points: > 1) Could use the localhost from where the client is invoked to create the > subject DN (CN=) if no subject is specified > 2) For clients to access these stores they will need access to store > passwords. I believe it's be most appropriate to leverage the Credential > Provider facility. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-802) Need mechanism for client truststore and keystore retrieval
[ https://issues.apache.org/jira/browse/SLIDER-802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351328#comment-14351328 ] ASF subversion and git services commented on SLIDER-802: Commit c1b83467cebc90b4e258d963ac8c5d90b85f3b1c in incubator-slider's branch refs/heads/develop from [~jmaron] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=c1b8346 ] SLIDER-802 add client keystore and trustore retrieval commands > Need mechanism for client truststore and keystore retrieval > --- > > Key: SLIDER-802 > URL: https://issues.apache.org/jira/browse/SLIDER-802 > Project: Slider > Issue Type: Bug > Components: client, security >Reporter: Jonathan Maron >Assignee: Jonathan Maron > > For applications that take advantage of the certificate store generation > feature, there may be a need for the client's to obtain truststores and, > optionally, keystores to successfully communicate via SSL to the server-side > application components. > One possible approach is a CLI option: > slider ssl-support --keystore --subj > Some key points: > 1) Could use the localhost from where the client is invoked to create the > subject DN (CN=) if no subject is specified > 2) For clients to access these stores they will need access to store > passwords. I believe it's be most appropriate to leverage the Credential > Provider facility. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-802) Need mechanism for client truststore and keystore retrieval
[ https://issues.apache.org/jira/browse/SLIDER-802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351327#comment-14351327 ] ASF subversion and git services commented on SLIDER-802: Commit b18daa35d105e76d9739f5aaab035c828872fa24 in incubator-slider's branch refs/heads/develop from [~ste...@apache.org] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=b18daa3 ] SLIDER-802 stub protobuf call and proxy operations to sign certificates > Need mechanism for client truststore and keystore retrieval > --- > > Key: SLIDER-802 > URL: https://issues.apache.org/jira/browse/SLIDER-802 > Project: Slider > Issue Type: Bug > Components: client, security >Reporter: Jonathan Maron >Assignee: Jonathan Maron > > For applications that take advantage of the certificate store generation > feature, there may be a need for the client's to obtain truststores and, > optionally, keystores to successfully communicate via SSL to the server-side > application components. > One possible approach is a CLI option: > slider ssl-support --keystore --subj > Some key points: > 1) Could use the localhost from where the client is invoked to create the > subject DN (CN=) if no subject is specified > 2) For clients to access these stores they will need access to store > passwords. I believe it's be most appropriate to leverage the Credential > Provider facility. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-802) Need mechanism for client truststore and keystore retrieval
[ https://issues.apache.org/jira/browse/SLIDER-802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351329#comment-14351329 ] ASF subversion and git services commented on SLIDER-802: Commit bf293b9df7dd06ee80a6c8876d0463da38e61a8f in incubator-slider's branch refs/heads/develop from [~jmaron] [ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=bf293b9 ] Merge branch 'feature/SLIDER-802-cert-sign' into develop > Need mechanism for client truststore and keystore retrieval > --- > > Key: SLIDER-802 > URL: https://issues.apache.org/jira/browse/SLIDER-802 > Project: Slider > Issue Type: Bug > Components: client, security >Reporter: Jonathan Maron >Assignee: Jonathan Maron > > For applications that take advantage of the certificate store generation > feature, there may be a need for the client's to obtain truststores and, > optionally, keystores to successfully communicate via SSL to the server-side > application components. > One possible approach is a CLI option: > slider ssl-support --keystore --subj > Some key points: > 1) Could use the localhost from where the client is invoked to create the > subject DN (CN=) if no subject is specified > 2) For clients to access these stores they will need access to store > passwords. I believe it's be most appropriate to leverage the Credential > Provider facility. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLIDER-799) AM to decide when to relax placement policy from specific host to rack/cluster
[ https://issues.apache.org/jira/browse/SLIDER-799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14350635#comment-14350635 ] Steve Loughran commented on SLIDER-799: --- If we are escalating, we could also consider # having a rack-local escalation before going cluster wide. This reduces the cost of fetching blocks from the previous host (assuming it is up & has all the blocks local). # maybe even falling back to other labels/queues. This is trickier and could lead to cluster admins dealing with support problems like 'why is my hbase master not running on a node of a given label?'. I think I'd rather have the component request unsatisfied and let those admins add new nodes to the label set explicitly. > AM to decide when to relax placement policy from specific host to rack/cluster > -- > > Key: SLIDER-799 > URL: https://issues.apache.org/jira/browse/SLIDER-799 > Project: Slider > Issue Type: Improvement > Components: appmaster >Affects Versions: Slider 0.70 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Critical > Fix For: Slider 0.80 > > Original Estimate: 24h > Remaining Estimate: 24h > > If Slider asks for relaxed affinity, YARN only gives it ~1 second for free > capacity to appear on a node before it falls back to non-local assignment. > While this is OK for analytics throughput, it's suboptimal for placement of > code such as HBase region servers. > AM needs to take charge of the placement and decide for itself when to > convert from placed to relaxed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
