Variable is either all encrypted as a single blob or all plain.
I think changing to use a single EMR connection type and require the
key/secret/role info to be in that connection makes the most sense.
-a
> On 15 Apr 2019, at 12:03, Daniel Mateus Pires wrote:
>
> In our company we use EMR
In our company we use EMR based operators a lot and it's always been
confusing for new users to find the different kinds of EMR clusters as
"Connections".
Not sure you could just remove the aws_conn_id, because the emr_conn_id
doesn't define which AWS account, which region, which profile to use
Or we should remove the aws_conn_id from the Emr* (hook and op) rather than
passing in two connection types.
Anyone have a though as to which way to go?
> On 15 Apr 2019, at 11:51, Ash Berlin-Taylor wrote:
>
> We have an EMR connection type, but the operator actually uses this as a
> config
We have an EMR connection type, but the operator actually uses this as a config
value, and the actual credentials come form the default aws_conn_id:
def __init__(
self,
aws_conn_id='s3_default',
emr_conn_id='emr_default',
