Re: Custom (mongodb) hook as a plugin

2016-10-03 Thread Jakob Homan 
On 23 September 2016 at 15:06, Rob Goretsky  wrote:
> More generally, would it make sense to create an enhancement to the plugin
> architecture that allows the developer of a new hook to also specify the
> name of a new 'connection type' along with what fields would be required
> for that connection type?  The web UI would then take its existing
> hardcoded list and append these plugged-in connection types and fields to
> display...

Yeah, I filed a ticket a while ago to do so
(https://issues.apache.org/jira/browse/AIRFLOW-235)

Additionally, it would be good if the connections were not hardcoded
so that it were easier to add new connections directly (perhaps
through the config file)

-Jakob

Re: Custom (mongodb) hook as a plugin

2016-09-23 Thread Maxime Beauchemin 
False, your hook will still read from the centralized connection store,
even if it lives outside the Airflow package.

Max

On Fri, Sep 23, 2016 at 2:05 PM, Shin Tai  wrote:

> Ok - thanks. Sounds like the sort of thing Airflow isn't meant to do right
> now because it wouldn't be possible to configure the connector with
> usernames, passwords etc
>
> I may just have to fork for the time being.
>
> Thanks again.
>
> On 23 September 2016 at 17:40, Andrew Phillips  wrote:
>
> > 6. Went to UI/Connections but couldn't find my connector/hook
> >>
> >
> > The experts on the list will hopefully set me right I'm my understanding
> > is incorrect, but from what I can see the list of hook types available
> *in
> > the UI* is fixed:
> >
> > https://github.com/apache/incubator-airflow/blob/f360414774f
> > 1ecb2e0f4e53ebdd623c7435f9a78/airflow/www/views.py#L2424
> >
> > Regards
> >
> > ap
> >
>

Re: Custom (mongodb) hook as a plugin

2016-09-23 Thread Shin Tai 
Ok - thanks. Sounds like the sort of thing Airflow isn't meant to do right
now because it wouldn't be possible to configure the connector with
usernames, passwords etc

I may just have to fork for the time being.

Thanks again.

On 23 September 2016 at 17:40, Andrew Phillips  wrote:

> 6. Went to UI/Connections but couldn't find my connector/hook
>>
>
> The experts on the list will hopefully set me right I'm my understanding
> is incorrect, but from what I can see the list of hook types available *in
> the UI* is fixed:
>
> https://github.com/apache/incubator-airflow/blob/f360414774f
> 1ecb2e0f4e53ebdd623c7435f9a78/airflow/www/views.py#L2424
>
> Regards
>
> ap
>

Re: Custom (mongodb) hook as a plugin

2016-09-23 Thread Andrew Phillips 

6. Went to UI/Connections but couldn't find my connector/hook


The experts on the list will hopefully set me right I'm my understanding 
is incorrect, but from what I can see the list of hook types available 
*in the UI* is fixed:


https://github.com/apache/incubator-airflow/blob/f360414774f1ecb2e0f4e53ebdd623c7435f9a78/airflow/www/views.py#L2424

Regards

ap