Taragolis commented on issue #26571:
URL: https://github.com/apache/airflow/issues/26571#issuecomment-1257221373
@dwreeves very detail explanation 👍 💯 🥇
My thought about `ast.literal_eval` and `full_url_mode` argument.
As I understand release after October 11, 2022 would `major` so it might
allow breaking changes in provider in this case we actually might drop support
of weird `ast.literal_eval` if user want keep use dict representation like
`{'region_name': 'us-east-1'}` the only option do not upgrade provider or
downgrade to previous version (it might mention in documentation). For user who
actually wanted fix their values in AWS SB we might prepare some snippets which
help to fix values in SB.
For `full_url_mode` I have two thoughts.
**First**. We can get rid of this argument (just do something
`kwargs.pop("full_url_mode", None)` and if value is set just inform users that
this argument has no affect anymore). In fact detects automatically is it json
or string would allow store for different connections in different formats:
JSON and URL-encoded
It could be archived by different ways, it might be something like this
sample
```python
import json
def validate(v: str) -> str:
if not v:
return v
v = v.strip()
try:
parsed_value = json.loads(v)
except json.JSONDecodeError as ex:
if v.startswith("{") or "://" not in v:
raise ValueError(f"Not valid JSON-Object or URL-encoded string.
Original error:\n * {ex}")
else:
if not isinstance(parsed_value, dict):
raise TypeError(f"Expected JSON-object, got {type(v).__name__}.")
return v
values_from_sb = [
'{"conn_type": "aws"}', # JSON Object - Valid
'aws://', # String, URL encoded - Valid
'"aws://"', # JSON String, Valid JSON object, Invalid Secret Backend
"{'conn_type': 'aws'}", # repr of dict, not failed during parsing
however it failed on url-unparse
'{"conn_type": "aws"', # Invalid JSON object, missing `}`
'\t\t\t\t\t{"conn_type": "aws"}\n', # Some whitespaces
]
for value in values_from_sb:
try:
print(f"{value!r} is valid value for SB: {validate(value)!r}")
except Exception as ex:
print(f"Unable to parse: {value!r}. Original error:\n - {ex}")
```
**Second**. It also might be a good idea to add ability to store as JSON
not only in `SecretsManagerBackend` but also in
`SystemsManagerParameterStoreBackend`. As result might be possible to create
some base class for AWS Secret Backends
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
