I have been building a plugin with hooks for Redshift and Teradata, and
initially was also confused as to why my I didn't see Redshift and Teradata
appear as "Connection Types" in the Connections admin web interface. I
then realized, as Andrew Phillips pointed out here, that the list of
connection types for the UI is hardcoded, so it is not aware of any hooks
you add via the Plugin interface.
As a workaround, I found that it was perfectly fine to create my
connections in the Web UI as the "MySQL" connection type and then fill in
the host/login/password I needed for connecting to my specific Redshift and
Teradata instances. My custom hooks were then able to read those
host/login/password attributes without any issue - regardless of the fact
that I had set the connection type to "MySQL". So, it seems that the
"connection type" that one chooses ("MySQL", "Postgres", etc) is mostly
irrelevant, since the same fields (host/login/password) are collected by
the UI regardless of the "connection type" (with the exception of the JDBC
and Google Cloud connection types, which show specifically labelled subset
of fields).. Shin, can you try that and see if it works as well in your
case?
More generally, would it make sense to create an enhancement to the plugin
architecture that allows the developer of a new hook to also specify the
name of a new 'connection type' along with what fields would be required
for that connection type? The web UI would then take its existing
hardcoded list and append these plugged-in connection types and fields to
display...
-Rob
On Fri, Sep 23, 2016 at 5:31 PM, Maxime Beauchemin <
maximebeauche...@gmail.com> wrote:
> False, your hook will still read from the centralized connection store,
> even if it lives outside the Airflow package.
>
> Max
>
> On Fri, Sep 23, 2016 at 2:05 PM, Shin Tai <shin.n...@gmail.com> wrote:
>
> > Ok - thanks. Sounds like the sort of thing Airflow isn't meant to do
> right
> > now because it wouldn't be possible to configure the connector with
> > usernames, passwords etc
> >
> > I may just have to fork for the time being.
> >
> > Thanks again.
> >
> > On 23 September 2016 at 17:40, Andrew Phillips <andr...@apache.org>
> wrote:
> >
> > > 6. Went to UI/Connections but couldn't find my connector/hook
> > >>
> > >
> > > The experts on the list will hopefully set me right I'm my
> understanding
> > > is incorrect, but from what I can see the list of hook types available
> > *in
> > > the UI* is fixed:
> > >
> > > https://github.com/apache/incubator-airflow/blob/f360414774f
> > > 1ecb2e0f4e53ebdd623c7435f9a78/airflow/www/views.py#L2424
> > >
> > > Regards
> > >
> > > ap
> > >
> >
>
