lakshmi-manasa-g opened a new pull request #1445:
URL: https://github.com/apache/samza/pull/1445


   **Feature:** Main feature is Cluster based Job coordinator (aka AM) high 
availability (HA) (TODO: sep/doc how?). The feature ensures that the new AM can 
establish connection with already running containers to avoid restarting all 
running containers when AM dies. This PR enables capturing of the physical 
execution environment container id (ex: yarn container id "container_123_123") 
mapping to Samza logical processor id (ex: "0"). In future PRs, this mapping 
will be used by the new AM.
   
   **Changes:**
   1. Introduce new Coordinator Stream Message and manager to read/write this 
message
   2. Container upon launch will write to c-stream, its logical and physical id
   3. Job Coordinator(AM) upon launch will read the mapping of all containers 
from c-stream.
   
   **Tests:** 
   1. added unit test for new manager
   2. working on tests for other classes as they have no coverage for relevant 
code.
   
   **API changes:**
   1. New c-stream message 
   
   **Usage instructions:** None
   
   **Upgrade instructions:** Backwards compatible. N/A


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Reply via email to