It can get more confusing because airflow allow to create two connection with same conn_id but different conn_type https://issues.apache.org/jira/browse/AIRFLOW-2784
Sent with ProtonMail Secure Email. ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Sunday, April 14, 2019 12:22 AM, Maxime Beauchemin <maximebeauche...@gmail.com> wrote: > People may rely on this feature for [poor man's] load balancing though, I > forgot what the exact use case was but used this at Airbnb at some point. > > Maybe the solution is to make the UI/UX/log output much more clear around > this. Making the CLI log more clear should be really easy to do, web server > might be a little more complicated but nothing too complicated. > > Max > > On Fri, Apr 12, 2019 at 7:51 AM James Meickle > jmeic...@quantopian.com.invalid wrote: > > > Airflow fetches connections by name, but doesn't enforce unique names. My > > team got bit by this, since it's very unexpected behavior for most types of > > data entry. The reason for this behavior is explained in the docs: > > "Many connections with the same conn_id can be defined and when that is the > > case, and when the hooks uses the get_connection method from BaseHook, > > Airflow will choose one connection randomly, allowing for some basic load > > balancing and fault tolerance when used in conjunction with retries." > > I think this is very non-intuitive UX. If we even want to support this > > feature within Airflow - and I don't think that is a given - it would make > > much more sense to require a unique (conn_id, conn_type) but allow storing > > multiple related records. This wouldn't be a huge data modeling change, but > > would require changing the web UI to appear as a form with subforms.
