People may rely on this feature for [poor man's] load balancing though, I forgot what the exact use case was but used this at Airbnb at some point.
Maybe the solution is to make the UI/UX/log output much more clear around this. Making the CLI log more clear should be really easy to do, web server might be a little more complicated but nothing too complicated. Max On Fri, Apr 12, 2019 at 7:51 AM James Meickle <jmeic...@quantopian.com.invalid> wrote: > Airflow fetches connections by name, but doesn't enforce unique names. My > team got bit by this, since it's very unexpected behavior for most types of > data entry. The reason for this behavior is explained in the docs: > > "Many connections with the same conn_id can be defined and when that is the > case, and when the hooks uses the get_connection method from BaseHook, > Airflow will choose one connection randomly, allowing for some basic load > balancing and fault tolerance when used in conjunction with retries." > > I think this is very non-intuitive UX. If we even want to support this > feature within Airflow - and I don't think that is a given - it would make > much more sense to require a unique (conn_id, conn_type) but allow storing > multiple related records. This wouldn't be a huge data modeling change, but > would require changing the web UI to appear as a form with subforms. >
