Bug#954363: cloud-init fails to obtain an IMDS API token on Amazon EC2
Control: tags -1 + upstream
> 2020-03-20 18:25:10,332 - url_helper.py[DEBUG]: [0/1] open
> 'http://169.254.169.254/latest/api/token' with {'url':
> 'http://169.254.169.254/latest/api/token', 'allow_redirects': True, 'method':
> 'PUT', 'timeout': 1.0, 'headers': {'User-Agent': 'Cloud-Init/20.1',
> 'X-aws-ec2-metadata-token-ttl-seconds': 'REDACTED'}} configuration
It seems that the "redaction" of the
X-aws-ec2-metadata-token-ttl-seconds header value happens before the
actual request is made, so where the IMDS server expects a TTL in
seconds, cloud-init actually passes it the literal string "REDACTED".
Unsurprisingly, this fails.
I've verified this by avoiding the redacting, via the attached patch.
This isn't an ideal solution, as it avoids all redacting. The intent of
cloud-init's behavior is to avoid storing IMDS API tokens in the logs,
which is sensible, and is broken by my change.
noah
diff --git a/debian/patches/no-redact-imds-headers.patch b/debian/patches/no-redact-imds-headers.patch
new file mode 100644
index ..26195c02
--- /dev/null
+++ b/debian/patches/no-redact-imds-headers.patch
@@ -0,0 +1,25 @@
Index: cloud-init/cloudinit/sources/DataSourceEc2.py
===
--- cloud-init.orig/cloudinit/sources/DataSourceEc2.py
+++ cloud-init/cloudinit/sources/DataSourceEc2.py
@@ -32,7 +32,7 @@ API_TOKEN_ROUTE = 'latest/api/token'
AWS_TOKEN_TTL_SECONDS = '21600'
AWS_TOKEN_PUT_HEADER = 'X-aws-ec2-metadata-token'
AWS_TOKEN_REQ_HEADER = AWS_TOKEN_PUT_HEADER + '-ttl-seconds'
-AWS_TOKEN_REDACT = [AWS_TOKEN_PUT_HEADER, AWS_TOKEN_REQ_HEADER]
+AWS_TOKEN_REDACT = []
class CloudNames(object):
Index: cloud-init/tests/unittests/test_datasource/test_ec2.py
===
--- cloud-init.orig/tests/unittests/test_datasource/test_ec2.py
+++ cloud-init/tests/unittests/test_datasource/test_ec2.py
@@ -479,6 +479,7 @@ class TestEc2(test_helpers.HttprettyTest
def test_aws_token_redacted(self):
"""Verify that aws tokens are redacted when logged."""
+self.skipTest('skipping for now...')
ds = self._setup_ds(
platform_data=self.valid_platform_data,
sys_cfg={'datasource': {'Ec2': {'strict_id': False}}},
Bug#954363: cloud-init fails to obtain an IMDS API token on Amazon EC2
Package: cloud-init
Version: 20.1-1
Severity: important
Cloud-init 20.1 attempts to obtain an API token for use with Amazon EC2
instance metadata service (IMDS). On EC2, this operation should always
succeed, whether using IMDSv1 or v2, and cloud-init will always access
IMDS in v2 mode. However, this fails on EC2:
2020-03-20 18:25:10,331 - DataSourceEc2.py[DEBUG]: Fetching Ec2 IMDSv2 API Token
2020-03-20 18:25:10,332 - url_helper.py[DEBUG]: [0/1] open
'http://169.254.169.254/latest/api/token' with {'url':
'http://169.254.169.254/latest/api/token', 'allow_redirects': True, 'method':
'PUT', 'timeout': 1.0, 'headers': {'User-Agent': 'Cloud-Init/20.1',
'X-aws-ec2-metadata-token-ttl-seconds': 'REDACTED'}} configuration
2020-03-20 18:25:10,336 - url_helper.py[DEBUG]: Read from
http://169.254.169.254/latest/api/token (400, 0b) after 1 attempts
2020-03-20 18:25:10,336 - DataSourceEc2.py[WARNING]: Calling
'http://169.254.169.254/latest/api/token' failed [0/1s]: empty response [400]
2020-03-20 18:25:10,344 - url_helper.py[DEBUG]: Please wait 1 seconds while we
wait to try again
With 20.1, cloud-init will fall back to using IMDSv1 in this case, but
this behavior will change in future versions, which will always use v2
mode (it is backwards-compatible with v1), and only use v1 mode for
compatibility with non-AWS services providing IMDS-compatible metadata
endpoints.
-- System Information:
Debian Release: bullseye/sid
APT prefers unstable
APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)
Kernel: Linux 5.4.0-4-cloud-amd64 (SMP w/2 CPU cores)
Locale: LANG=C.UTF-8, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE=C.UTF-8
(charmap=UTF-8)
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages cloud-init depends on:
ii fdisk 2.34-0.1
ii gdisk 1.0.5-1
ii ifupdown0.8.35+b1
ii locales 2.30-2
ii lsb-base11.1.0
ii lsb-release 11.1.0
ii net-tools 1.60+git20180626.aebd88e-1
ii procps 2:3.3.16-4
ii python3 3.8.2-1
ii python3-configobj 5.0.6-3
ii python3-jinja2 2.10.1-2
ii python3-jsonpatch 1.23-3
ii python3-jsonschema 3.0.2-4
ii python3-oauthlib3.1.0-1
ii python3-requests2.22.0-2
ii python3-six 1.14.0-2
ii python3-yaml5.3.1-1
ii util-linux 2.34-0.1
Versions of packages cloud-init recommends:
ii cloud-guest-utils 0.31-1
pn eatmydata
ii sudo 1.8.31-1
Versions of packages cloud-init suggests:
pn btrfs-progs
ii e2fsprogs1.45.5-2
pn xfsprogs
-- debconf information:
* cloud-init/datasources:Ec2
