... Key for storing the ContainerPlacementRequestMessage & ContainerPlacementResponseMessage in Metastore is chosen to be UUID + "." + messageType(ContainerPlacementResponseMessage or ContainerPlacementRequestMessage). Value will be payload container ContainerPlacementRequestMessage & ContainerPlacementResponseMessage. Messages are written and read to the Metastore through the MetadataStore abstraction. Since the metastore is eventually consistent, duplicate messages are required to be handled by ContainerPlacementService. ContainerPlacementRequestMessage:
Key |
Value |
Field Description |
Field Type |
"UUID.subType" |
uuid |
Unique identifier of a response message |
Required |
|
processorId |
Logical processor id 0,1,2 of the container |
Required |
|
deploymentId |
Unique identifier for a deployment |
Required |
|
subType |
Type of message here: ContainerPlacementResponseMessage |
Required |
|
destinationHost |
Destination host where the container is desired to be moved |
Required |
|
statusCode |
Status of the current action |
Required |
|
responseMessage |
Response message in conjunction to status |
Required |
|
timestamp |
The timestamp of the response message |
Required |
|
requestExpiry |
Eequest expiry which acts as a timeout for any resource request to cluster resource manager |
Optional |
Sample KV
Key |
Value |
[1,"samza-place-container-v1","f068175b-c9b6-4f34-982b-ecb5619f21de.ContainerPlacementRequestMessage"] |
{"processorId":"1","deploymentId":"app-atttempt-001","subType":"ContainerPlacementRequestMessage","uuid":"f068175b-c9b6-4f34-982b-ecb5619f21de","destinationHost":"ANY_HOST","statusCode":"CREATED","timestamp":1578693870484} |
ContainerPlacementResponseMessage:
Key |
Value |
Field Description |
Field Type |
"UUID.subType" |
uuid |
Unique identifier of a response message |
Required |
|
processorId |
Logical processor id 0,1,2 of the container |
Required |
|
deploymentId |
Unique identifier for a deployment |
Required |
|
subType |
Type of message here: ContainerPlacementResponseMessage |
Required |
|
destinationHost |
Destination host where the container is desired to be moved |
Required |
|
statusCode |
Status of the current action |
Required |
|
responseMessage |
Response message in conjunction to status |
Required |
|
timestamp |
The timestamp of the response message |
Required |
|
requestExpiry |
Eequest expiry which acts as a timeout for any resource request to cluster resource manager |
Optional |
Sample KV
Key |
Value |
[1,"samza-place-container-v1","88b0d30c-d518-4307-9e8e-c8529eb30f04.ContainerPlacementResponseMessage"] |
{"processorId":"1","deploymentId":"app-atttempt-001","subType":"ContainerPlacementResponseMessage","responseMessage":"Request is accepted","uuid":"88b0d30c-d518-4307-9e8e-c8529eb30f04","destinationHost":"ANY_HOST","statusCode":"ACCEPTED","timestamp":1578694070875} |
GC policy for stale messages in metastore
- One way to delete stale ContainerPlacementMessages is to delete request / responses from the previous incarnation of the job in the metastore on job restarts
- Once the request is complete, ContainerPlacementService can issue an async delete to the metastore
- Request / response message can be externally cleaned by a tool
Part 2. Container Placement Service ...
- Remove the HostAwareContainerAllocator & ContainerAllocator, simplify Container Allocator as a simple lightweight entity allocating requests to available resources (PR1, PR2)
- Introduce ContainerManager which acts as a brain for validating and issuing any actions on containers in the Job Coordinator for both active & Standby containers. (PR)
- Transfer state & validation of container launch & expired request handling from ContainerAllocator to ContainerManager
- Transfer state & validation for callback handler (ContainerProcessManager) to ContainerManager
- Encapsulates logic and state related to container placement actions like move, restarts for active & standby container in ContainerManager (PR-1, TDB)
- It is ContainerManager’s duty to validate any ContainerPlacementRequestMessages & also invalidate messages from the previous deployment incarnation
- It is ContainerManager’s duty to write ContainerPlacementResponseMessages to Metastore for external control to query the status of the request
- ContainerPlacementMetadata is a metadata holder for container actions (ControlActionMetadata) for ex request_id, current status, requested resources etc
Image RemovedImage Added 2.1 Container Move 2.1.1 Stateless Container Move & Stateful Container Move (without Standby) ... |