[jira] [Updated] (YARN-1915) ClientToAMTokenMasterKey should be provided to AM at launch time
[ https://issues.apache.org/jira/browse/YARN-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-1915: - Priority: Blocker (was: Critical) Per offline discussion with [~vinodkv] marking this as a blocker for 2.6. > ClientToAMTokenMasterKey should be provided to AM at launch time > > > Key: YARN-1915 > URL: https://issues.apache.org/jira/browse/YARN-1915 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.2.0 >Reporter: Hitesh Shah >Assignee: Jason Lowe >Priority: Blocker > Attachments: YARN-1915.patch, YARN-1915v2.patch, YARN-1915v3.patch > > > Currently, the AM receives the key as part of registration. This introduces a > race where a client can connect to the AM when the AM has not received the > key. > Current Flow: > 1) AM needs to start the client listening service in order to get host:port > and send it to the RM as part of registration > 2) RM gets the port info in register() and transitions the app to RUNNING. > Responds back with client secret to AM. > 3) User asks RM for client token. Gets it and pings the AM. AM hasn't > received client secret from RM and so RPC itself rejects the request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-1915) ClientToAMTokenMasterKey should be provided to AM at launch time
[ https://issues.apache.org/jira/browse/YARN-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-1915: - Attachment: YARN-1915v3.patch Refreshed patch to latest trunk. [~vinodkv] could you comment? I fully agree with Hitesh that the current patch is a stop-gap at best. However there's some confusion as to how the client token master key should be sent to the RM (e.g.: via container credentials, via the current method, etc.). The original env variable approach apparently is problematic on Windows per YARN-610. If we won't have time to develop the best fix for 2.6 then I'd like to see something like this patch put in to improve things in the interim. > ClientToAMTokenMasterKey should be provided to AM at launch time > > > Key: YARN-1915 > URL: https://issues.apache.org/jira/browse/YARN-1915 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.2.0 >Reporter: Hitesh Shah >Assignee: Jason Lowe >Priority: Critical > Attachments: YARN-1915.patch, YARN-1915v2.patch, YARN-1915v3.patch > > > Currently, the AM receives the key as part of registration. This introduces a > race where a client can connect to the AM when the AM has not received the > key. > Current Flow: > 1) AM needs to start the client listening service in order to get host:port > and send it to the RM as part of registration > 2) RM gets the port info in register() and transitions the app to RUNNING. > Responds back with client secret to AM. > 3) User asks RM for client token. Gets it and pings the AM. AM hasn't > received client secret from RM and so RPC itself rejects the request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-1915) ClientToAMTokenMasterKey should be provided to AM at launch time
[ https://issues.apache.org/jira/browse/YARN-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-1915: - Attachment: YARN-1915v2.patch Fixed findbug warning. > ClientToAMTokenMasterKey should be provided to AM at launch time > > > Key: YARN-1915 > URL: https://issues.apache.org/jira/browse/YARN-1915 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.2.0 >Reporter: Hitesh Shah >Assignee: Jason Lowe >Priority: Critical > Attachments: YARN-1915.patch, YARN-1915v2.patch > > > Currently, the AM receives the key as part of registration. This introduces a > race where a client can connect to the AM when the AM has not received the > key. > Current Flow: > 1) AM needs to start the client listening service in order to get host:port > and send it to the RM as part of registration > 2) RM gets the port info in register() and transitions the app to RUNNING. > Responds back with client secret to AM. > 3) User asks RM for client token. Gets it and pings the AM. AM hasn't > received client secret from RM and so RPC itself rejects the request. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-1915) ClientToAMTokenMasterKey should be provided to AM at launch time
[ https://issues.apache.org/jira/browse/YARN-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-1915: - Attachment: YARN-1915.patch We're starting to see this as well in our rollout of 2.x. Attaching a patch that works around the issue by having the AM secret manager wait around for a bit before trying to validate a token if the master key isn't set yet. Another approach we could try is to have the RM not advertise to clients where the AM is (i.e.: hide the host, port, and tracking URL) until the RM has seen at least one heartbeat after the AM registered. The approach in this patch was easy to implement and probably just as effective in practice. > ClientToAMTokenMasterKey should be provided to AM at launch time > > > Key: YARN-1915 > URL: https://issues.apache.org/jira/browse/YARN-1915 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.2.0 >Reporter: Hitesh Shah >Priority: Critical > Attachments: YARN-1915.patch > > > Currently, the AM receives the key as part of registration. This introduces a > race where a client can connect to the AM when the AM has not received the > key. > Current Flow: > 1) AM needs to start the client listening service in order to get host:port > and send it to the RM as part of registration > 2) RM gets the port info in register() and transitions the app to RUNNING. > Responds back with client secret to AM. > 3) User asks RM for client token. Gets it and pings the AM. AM hasn't > received client secret from RM and so RPC itself rejects the request. -- This message was sent by Atlassian JIRA (v6.2#6252)
