Re: Airflow Developers Meeting - 08/03 Notes
ASF provides per-projects blog infrastructure: http://www.apache.org/dev/project-blogs -Jakob On 23 August 2016 at 15:07, Maxime Beauchemin wrote: > I'm planning on working on a minimal apache release this week. If someone > can point to the commit or PRs I can add them to `branch-1.7.2-apache` and > start moving towards the release. > > Max > > On Tue, Aug 23, 2016 at 11:13 AM, Alex Van Boxel wrote: > >> All stuff is in master (I'm running pre-production on master), if it needs >> to be cherry-picked to a branch I can always test it. I would be grate if >> it's in the next release so I can start blogging about it. >> >> On Tue, Aug 23, 2016 at 8:05 PM Bolke de Bruin wrote: >> >> > (Sorry for the late response, I was on holiday) >> > >> > I think the G* operators just need to be cherry picked. This will make us >> > deviate slightly from the >> > previous release, but makes sure we don’t have to ‘fix’ history >> afterwards. >> > >> > Anyone against this? >> > >> > - B. >> > >> > > Op 8 aug. 2016, om 14:21 heeft Alex Van Boxel het >> > volgende geschreven: >> > > >> > > Sorry, Bolke. What needs to be done for Google Cloud operators/hooks? >> > > >> > > If you bring me up-to-speed I can do this. I'm currently working an a >> CI >> > > setup for the Google stuff). >> > > >> > > On Sat, Aug 6, 2016 at 3:56 PM Bolke de Bruin >> wrote: >> > > >> > >> I have indeed a branch with cherry picked commits. This branch is >> > >> available in the apache repo. The branch is available as >> > >> “branch-1.7.2-apache”. What is left to do is to decide weather we do >> > >> upgrade the google cloud operators/hooks as part of this release as >> the >> > >> licenses were only added after these changes came in. I prefer this >> > >> approach as we stay away from custom commits as much as possible then. >> > >> >> > >> - Bolke >> > >> >> > >>> Op 5 aug. 2016, om 00:21 heeft Gurer Kiratli < >> gurer.kira...@airbnb.com >> > .INVALID> >> > >> het volgende geschreven: >> > >>> >> > >>> Agenda >> > >>> >> > >>> >> > >>> - >> > >>> >> > >>> Committers sync-up: progress and plans >> > >>> - >> > >>> >> > >>> Max to do a recap since last release >> > >>> - >> > >>> >> > >>> Airbnb >> > >>> - >> > >>> >> > >>> Anyone else? >> > >>> - >> > >>> >> > >>> Cooperation Best Practices >> > >>> - >> > >>> >> > >>> Solicit for feedback for Impersonation Design Review >> > >>> - >> > >>> >> > >>> Release Schedule, Management >> > >>> - >> > >>> >> > >>> Roadmap discussion >> > >>> >> > >>> >> > >>> Attendees: Jeremiah Lowin, Chris Riccomini, Andrew Phillips(?), >> Maxime >> > >>> Beauchemin, Paul Yang, Dan Davydov, Xuanji Li, George Ke, Arthur >> > Wiedmer, >> > >>> Gurer Kiratli >> > >>> >> > >>> Notes >> > >>> >> > >>> >> > >>> - >> > >>> >> > >>> Recap for the overall project >> > >>> - >> > >>> >> > >>> [Max] >> > >>> >> > >> https://gist.github.com/mistercrunch/5460483ec764e2a1cb816c6b1d6ad5a3 >> > >>> - >> > >>> >> > >>> Airbnb Recap >> > >>> - >> > >>> >> > >>> [Paul] Airbnb is continuing work on features related to migrating >> > >>> more jobs onto Airflow - DB connection scalability, >> impersonation, >> > >>> refreshing web UI, DAG git versioning, task resource isolation >> > >>> - >> > >>> >> > >>> Other Recaps >> > >>> - >> > >>> >> > >>> [Chris] Improvements to Import functionality. Joy did some work >> > >>> around CLIs and mark success in collaboration with Bolke. >> > >>> - >> > >>> >> > >>> [Jeremiah] Came up with the Merge tool. Focusing on the work >> around >> > >>> configuration. >> > >>> - >> > >>> >> > >>> Meeting cadence >> > >>> - >> > >>> >> > >>> [Max] We can have monthly meetings. Let’s start with monthly we >> can >> > >>> change the cadence. We should invite all contributors. We should >> > >>> specify in >> > >>> the invite that everyone is welcome. Currently posting this on >> > >>> the mailing >> > >>> list and wiki. >> > >>> - >> > >>> >> > >>> *Action Item* Gurer to schedule monthly meetings next one at >> > >> Airbnb. >> > >>> Airbnb is welcoming folks to come onsite >> > >>> - >> > >>> >> > >>> Blog/Documentation >> > >>> - >> > >>> >> > >>> [Max, Chris, Jeremiah] Having a blog would be useful. We can >> > >>> structure our thoughts. >> > >>> - >> > >>> >> > >>> [Andrew] Blog should have better language with screenshots and >> all. >> > >>> This raises the level of effort bar and this might mean that >> > >>> nobody really >> > >>> does it. >> > >>> - >> > >>> >> > >>> [Max] Maybe use Medium? We can try it out. >> > >>> - >> > >>> >> > >>> [Max] Haven’t looked at the documentation for a long time. I >> don’t >> > >>> really know about its quality. >> > >>> - >> > >>> >> > >>> [Arthur] Looking at the email list responses, seems like the >> > >>> documentation is missing certain elements. >> > >>> - >> > >>> >> > >>>
Re: Airflow Developers Meeting - 08/03 Notes
I'm planning on working on a minimal apache release this week. If someone can point to the commit or PRs I can add them to `branch-1.7.2-apache` and start moving towards the release. Max On Tue, Aug 23, 2016 at 11:13 AM, Alex Van Boxel wrote: > All stuff is in master (I'm running pre-production on master), if it needs > to be cherry-picked to a branch I can always test it. I would be grate if > it's in the next release so I can start blogging about it. > > On Tue, Aug 23, 2016 at 8:05 PM Bolke de Bruin wrote: > > > (Sorry for the late response, I was on holiday) > > > > I think the G* operators just need to be cherry picked. This will make us > > deviate slightly from the > > previous release, but makes sure we don’t have to ‘fix’ history > afterwards. > > > > Anyone against this? > > > > - B. > > > > > Op 8 aug. 2016, om 14:21 heeft Alex Van Boxel het > > volgende geschreven: > > > > > > Sorry, Bolke. What needs to be done for Google Cloud operators/hooks? > > > > > > If you bring me up-to-speed I can do this. I'm currently working an a > CI > > > setup for the Google stuff). > > > > > > On Sat, Aug 6, 2016 at 3:56 PM Bolke de Bruin > wrote: > > > > > >> I have indeed a branch with cherry picked commits. This branch is > > >> available in the apache repo. The branch is available as > > >> “branch-1.7.2-apache”. What is left to do is to decide weather we do > > >> upgrade the google cloud operators/hooks as part of this release as > the > > >> licenses were only added after these changes came in. I prefer this > > >> approach as we stay away from custom commits as much as possible then. > > >> > > >> - Bolke > > >> > > >>> Op 5 aug. 2016, om 00:21 heeft Gurer Kiratli < > gurer.kira...@airbnb.com > > .INVALID> > > >> het volgende geschreven: > > >>> > > >>> Agenda > > >>> > > >>> > > >>> - > > >>> > > >>> Committers sync-up: progress and plans > > >>> - > > >>> > > >>> Max to do a recap since last release > > >>> - > > >>> > > >>> Airbnb > > >>> - > > >>> > > >>> Anyone else? > > >>> - > > >>> > > >>> Cooperation Best Practices > > >>> - > > >>> > > >>> Solicit for feedback for Impersonation Design Review > > >>> - > > >>> > > >>> Release Schedule, Management > > >>> - > > >>> > > >>> Roadmap discussion > > >>> > > >>> > > >>> Attendees: Jeremiah Lowin, Chris Riccomini, Andrew Phillips(?), > Maxime > > >>> Beauchemin, Paul Yang, Dan Davydov, Xuanji Li, George Ke, Arthur > > Wiedmer, > > >>> Gurer Kiratli > > >>> > > >>> Notes > > >>> > > >>> > > >>> - > > >>> > > >>> Recap for the overall project > > >>> - > > >>> > > >>> [Max] > > >>> > > >> https://gist.github.com/mistercrunch/5460483ec764e2a1cb816c6b1d6ad5a3 > > >>> - > > >>> > > >>> Airbnb Recap > > >>> - > > >>> > > >>> [Paul] Airbnb is continuing work on features related to migrating > > >>> more jobs onto Airflow - DB connection scalability, > impersonation, > > >>> refreshing web UI, DAG git versioning, task resource isolation > > >>> - > > >>> > > >>> Other Recaps > > >>> - > > >>> > > >>> [Chris] Improvements to Import functionality. Joy did some work > > >>> around CLIs and mark success in collaboration with Bolke. > > >>> - > > >>> > > >>> [Jeremiah] Came up with the Merge tool. Focusing on the work > around > > >>> configuration. > > >>> - > > >>> > > >>> Meeting cadence > > >>> - > > >>> > > >>> [Max] We can have monthly meetings. Let’s start with monthly we > can > > >>> change the cadence. We should invite all contributors. We should > > >>> specify in > > >>> the invite that everyone is welcome. Currently posting this on > > >>> the mailing > > >>> list and wiki. > > >>> - > > >>> > > >>> *Action Item* Gurer to schedule monthly meetings next one at > > >> Airbnb. > > >>> Airbnb is welcoming folks to come onsite > > >>> - > > >>> > > >>> Blog/Documentation > > >>> - > > >>> > > >>> [Max, Chris, Jeremiah] Having a blog would be useful. We can > > >>> structure our thoughts. > > >>> - > > >>> > > >>> [Andrew] Blog should have better language with screenshots and > all. > > >>> This raises the level of effort bar and this might mean that > > >>> nobody really > > >>> does it. > > >>> - > > >>> > > >>> [Max] Maybe use Medium? We can try it out. > > >>> - > > >>> > > >>> [Max] Haven’t looked at the documentation for a long time. I > don’t > > >>> really know about its quality. > > >>> - > > >>> > > >>> [Arthur] Looking at the email list responses, seems like the > > >>> documentation is missing certain elements. > > >>> - > > >>> > > >>> [Max] Maybe we can interview people who just started and identify > > >>> gaps. > > >>> - > > >>> > > >>> *Action Item* [Jeremiah] We can also ask the mailing list about > > >> this. > > >>> He will send this out. > > >>> - > > >>> > > >>> Cooperation guidelines > > >>> - > > >>> > > >>> [Max,
Re: why is this alwasys running? Should be once an hour
even @hourly is not working. Fires every cycle. wow
On Wed, Aug 24, 2016 at 5:09 AM, David Montgomery wrote:
> I updated the dag. In thje UI I see 0 * * * * in the schedule field
>
>
>
> default_args = {
> 'owner': 'airflow',
> 'depends_on_past': False,
> "start_date": datetime.now(),
> 'email': ['t...@gmail.com'],
> 'email_on_failure': True,
> 'email_on_retry': True,
> 'retries': 1,
> 'retry_delay': timedelta(minutes=5)
> }
>
>
>
> dag = DAG('first_test', schedule_interval="0 * * * *",
> default_args=default_args)
>
> node_0 = PythonOperator(
> task_id='isnewdata',
> provide_context=False,
> python_callable=checkfornewdata,
> dag=dag)
>
>
> node_0_1 = PythonOperator(
> task_id='fetchdata',
> provide_context=False,
> python_callable=fetchdata,
> dag=dag)
>
> node_0_1_2 = PythonOperator(
> task_id='uploadtoes',
> provide_context=False,
> python_callable= uploadtoes,
> dag=dag)
>
>
why is this alwasys running? Should be once an hour
I updated the dag. In thje UI I see 0 * * * * in the schedule field
default_args = {
'owner': 'airflow',
'depends_on_past': False,
"start_date": datetime.now(),
'email': ['t...@gmail.com'],
'email_on_failure': True,
'email_on_retry': True,
'retries': 1,
'retry_delay': timedelta(minutes=5)
}
dag = DAG('first_test', schedule_interval="0 * * * *",
default_args=default_args)
node_0 = PythonOperator(
task_id='isnewdata',
provide_context=False,
python_callable=checkfornewdata,
dag=dag)
node_0_1 = PythonOperator(
task_id='fetchdata',
provide_context=False,
python_callable=fetchdata,
dag=dag)
node_0_1_2 = PythonOperator(
task_id='uploadtoes',
provide_context=False,
python_callable= uploadtoes,
dag=dag)
Re: Airflow Developers Meeting - 08/03 Notes
All stuff is in master (I'm running pre-production on master), if it needs to be cherry-picked to a branch I can always test it. I would be grate if it's in the next release so I can start blogging about it. On Tue, Aug 23, 2016 at 8:05 PM Bolke de Bruin wrote: > (Sorry for the late response, I was on holiday) > > I think the G* operators just need to be cherry picked. This will make us > deviate slightly from the > previous release, but makes sure we don’t have to ‘fix’ history afterwards. > > Anyone against this? > > - B. > > > Op 8 aug. 2016, om 14:21 heeft Alex Van Boxel het > volgende geschreven: > > > > Sorry, Bolke. What needs to be done for Google Cloud operators/hooks? > > > > If you bring me up-to-speed I can do this. I'm currently working an a CI > > setup for the Google stuff). > > > > On Sat, Aug 6, 2016 at 3:56 PM Bolke de Bruin wrote: > > > >> I have indeed a branch with cherry picked commits. This branch is > >> available in the apache repo. The branch is available as > >> “branch-1.7.2-apache”. What is left to do is to decide weather we do > >> upgrade the google cloud operators/hooks as part of this release as the > >> licenses were only added after these changes came in. I prefer this > >> approach as we stay away from custom commits as much as possible then. > >> > >> - Bolke > >> > >>> Op 5 aug. 2016, om 00:21 heeft Gurer Kiratli .INVALID> > >> het volgende geschreven: > >>> > >>> Agenda > >>> > >>> > >>> - > >>> > >>> Committers sync-up: progress and plans > >>> - > >>> > >>> Max to do a recap since last release > >>> - > >>> > >>> Airbnb > >>> - > >>> > >>> Anyone else? > >>> - > >>> > >>> Cooperation Best Practices > >>> - > >>> > >>> Solicit for feedback for Impersonation Design Review > >>> - > >>> > >>> Release Schedule, Management > >>> - > >>> > >>> Roadmap discussion > >>> > >>> > >>> Attendees: Jeremiah Lowin, Chris Riccomini, Andrew Phillips(?), Maxime > >>> Beauchemin, Paul Yang, Dan Davydov, Xuanji Li, George Ke, Arthur > Wiedmer, > >>> Gurer Kiratli > >>> > >>> Notes > >>> > >>> > >>> - > >>> > >>> Recap for the overall project > >>> - > >>> > >>> [Max] > >>> > >> https://gist.github.com/mistercrunch/5460483ec764e2a1cb816c6b1d6ad5a3 > >>> - > >>> > >>> Airbnb Recap > >>> - > >>> > >>> [Paul] Airbnb is continuing work on features related to migrating > >>> more jobs onto Airflow - DB connection scalability, impersonation, > >>> refreshing web UI, DAG git versioning, task resource isolation > >>> - > >>> > >>> Other Recaps > >>> - > >>> > >>> [Chris] Improvements to Import functionality. Joy did some work > >>> around CLIs and mark success in collaboration with Bolke. > >>> - > >>> > >>> [Jeremiah] Came up with the Merge tool. Focusing on the work around > >>> configuration. > >>> - > >>> > >>> Meeting cadence > >>> - > >>> > >>> [Max] We can have monthly meetings. Let’s start with monthly we can > >>> change the cadence. We should invite all contributors. We should > >>> specify in > >>> the invite that everyone is welcome. Currently posting this on > >>> the mailing > >>> list and wiki. > >>> - > >>> > >>> *Action Item* Gurer to schedule monthly meetings next one at > >> Airbnb. > >>> Airbnb is welcoming folks to come onsite > >>> - > >>> > >>> Blog/Documentation > >>> - > >>> > >>> [Max, Chris, Jeremiah] Having a blog would be useful. We can > >>> structure our thoughts. > >>> - > >>> > >>> [Andrew] Blog should have better language with screenshots and all. > >>> This raises the level of effort bar and this might mean that > >>> nobody really > >>> does it. > >>> - > >>> > >>> [Max] Maybe use Medium? We can try it out. > >>> - > >>> > >>> [Max] Haven’t looked at the documentation for a long time. I don’t > >>> really know about its quality. > >>> - > >>> > >>> [Arthur] Looking at the email list responses, seems like the > >>> documentation is missing certain elements. > >>> - > >>> > >>> [Max] Maybe we can interview people who just started and identify > >>> gaps. > >>> - > >>> > >>> *Action Item* [Jeremiah] We can also ask the mailing list about > >> this. > >>> He will send this out. > >>> - > >>> > >>> Cooperation guidelines > >>> - > >>> > >>> [Max, Arthur] > >>> > >> > https://cwiki.apache.org/confluence/display/AIRFLOW/Community+Guidelines > >>> - > >>> > >>> [Chris] Let’s link the guidelines to contributor md . This should > be > >>> discoverable > >>> - > >>> > >>> Release > >>> - > >>> > >>> [Chris, Max] Bolke might have branch with High charts and Licensing > >>> cherry picked. We can use this as a dry run. We can test the > >> process. > >>> - > >>> > >>> [Max] Big release is probably around September. > >>> - > >>> > >>> [Andrew] If we want to have a true practice, we need to be > >> diligently > >>>
Re: Airflow Developers Meeting - 08/03 Notes
(Sorry for the late response, I was on holiday) I think the G* operators just need to be cherry picked. This will make us deviate slightly from the previous release, but makes sure we don’t have to ‘fix’ history afterwards. Anyone against this? - B. > Op 8 aug. 2016, om 14:21 heeft Alex Van Boxel het volgende > geschreven: > > Sorry, Bolke. What needs to be done for Google Cloud operators/hooks? > > If you bring me up-to-speed I can do this. I'm currently working an a CI > setup for the Google stuff). > > On Sat, Aug 6, 2016 at 3:56 PM Bolke de Bruin wrote: > >> I have indeed a branch with cherry picked commits. This branch is >> available in the apache repo. The branch is available as >> “branch-1.7.2-apache”. What is left to do is to decide weather we do >> upgrade the google cloud operators/hooks as part of this release as the >> licenses were only added after these changes came in. I prefer this >> approach as we stay away from custom commits as much as possible then. >> >> - Bolke >> >>> Op 5 aug. 2016, om 00:21 heeft Gurer Kiratli >>> >> het volgende geschreven: >>> >>> Agenda >>> >>> >>> - >>> >>> Committers sync-up: progress and plans >>> - >>> >>> Max to do a recap since last release >>> - >>> >>> Airbnb >>> - >>> >>> Anyone else? >>> - >>> >>> Cooperation Best Practices >>> - >>> >>> Solicit for feedback for Impersonation Design Review >>> - >>> >>> Release Schedule, Management >>> - >>> >>> Roadmap discussion >>> >>> >>> Attendees: Jeremiah Lowin, Chris Riccomini, Andrew Phillips(?), Maxime >>> Beauchemin, Paul Yang, Dan Davydov, Xuanji Li, George Ke, Arthur Wiedmer, >>> Gurer Kiratli >>> >>> Notes >>> >>> >>> - >>> >>> Recap for the overall project >>> - >>> >>> [Max] >>> >> https://gist.github.com/mistercrunch/5460483ec764e2a1cb816c6b1d6ad5a3 >>> - >>> >>> Airbnb Recap >>> - >>> >>> [Paul] Airbnb is continuing work on features related to migrating >>> more jobs onto Airflow - DB connection scalability, impersonation, >>> refreshing web UI, DAG git versioning, task resource isolation >>> - >>> >>> Other Recaps >>> - >>> >>> [Chris] Improvements to Import functionality. Joy did some work >>> around CLIs and mark success in collaboration with Bolke. >>> - >>> >>> [Jeremiah] Came up with the Merge tool. Focusing on the work around >>> configuration. >>> - >>> >>> Meeting cadence >>> - >>> >>> [Max] We can have monthly meetings. Let’s start with monthly we can >>> change the cadence. We should invite all contributors. We should >>> specify in >>> the invite that everyone is welcome. Currently posting this on >>> the mailing >>> list and wiki. >>> - >>> >>> *Action Item* Gurer to schedule monthly meetings next one at >> Airbnb. >>> Airbnb is welcoming folks to come onsite >>> - >>> >>> Blog/Documentation >>> - >>> >>> [Max, Chris, Jeremiah] Having a blog would be useful. We can >>> structure our thoughts. >>> - >>> >>> [Andrew] Blog should have better language with screenshots and all. >>> This raises the level of effort bar and this might mean that >>> nobody really >>> does it. >>> - >>> >>> [Max] Maybe use Medium? We can try it out. >>> - >>> >>> [Max] Haven’t looked at the documentation for a long time. I don’t >>> really know about its quality. >>> - >>> >>> [Arthur] Looking at the email list responses, seems like the >>> documentation is missing certain elements. >>> - >>> >>> [Max] Maybe we can interview people who just started and identify >>> gaps. >>> - >>> >>> *Action Item* [Jeremiah] We can also ask the mailing list about >> this. >>> He will send this out. >>> - >>> >>> Cooperation guidelines >>> - >>> >>> [Max, Arthur] >>> >> https://cwiki.apache.org/confluence/display/AIRFLOW/Community+Guidelines >>> - >>> >>> [Chris] Let’s link the guidelines to contributor md . This should be >>> discoverable >>> - >>> >>> Release >>> - >>> >>> [Chris, Max] Bolke might have branch with High charts and Licensing >>> cherry picked. We can use this as a dry run. We can test the >> process. >>> - >>> >>> [Max] Big release is probably around September. >>> - >>> >>> [Andrew] If we want to have a true practice, we need to be >> diligently >>> following Apache guidelines. >>> - >>> >>> [Max] We might want to revive the email thread about how we should >> do >>> this. >>> - >>> >>> [Chris] It would be great for someone who hasn’t done this before to >>> do this. >>> - >>> >>> [Andrew] Building a script for this is smart otherwise the manual >>> work is painful from experience. >>> >> https://cwiki.apache.org/confluence/display/JCLOUDS/Releasing+jclouds >>> >>> >>> https://cwiki.apache.org/confluence/display/JCLOUDS/Validate+a+Release >>> >>> - >>>
Re: airflow supervisord scripts do not work
Hi David, > But I want to use supervisor. And below is the terrible error I get We've been using supervisord successfully to start and manage Airflow processes for a while now. Below is our /etc/supervisord.conf file. The only quirk with our setup is that we use Python 3.5 to run Airflow, but Supervisor is only officially supported on Python 2 so we use pyenv to install 2.7.11 just to run Supervisor. We modify /usr/local/bin/supervisord and supervisorctl files to start with: #!/root/.pyenv/versions/2.7.11/bin/python Hope this helps, -Joe ; Configuration for Airflow webserver and scheduler in Supervisor [program:webserver] command=/usr/local/bin/airflow webserver stopsignal=QUIT stopasgroup=true user=root stdout_logfile=/var/log/airflow/webserver-stdout.log stderr_logfile=/var/log/airflow/webserver-stderr.log environment=HOME="/root",AIRFLOW_HOME="/usr/local/airflow",TMPDIR="/tmp/airflow" pidfile = /usr/local/airflow/airflow-webserver.pid [program:scheduler] command=/usr/local/bin/airflow scheduler -n 5 stopsignal=QUIT stopasgroup=true killasgroup=true user=root stdout_logfile=/var/log/airflow/scheduler-stdout.log stderr_logfile=/var/log/airflow/scheduler-stderr.log environment=HOME="/root",AIRFLOW_HOME="/usr/local/airflow",TMPDIR="/tmp/airflow" autorestart=true [program:worker] command=/usr/local/bin/airflow worker stopsignal=QUIT stopasgroup=true killasgroup=true user=root stdout_logfile=/var/log/airflow/worker-stdout.log stderr_logfile=/var/log/airflow/worker-stderr.log environment=HOME="/root",AIRFLOW_HOME="/usr/local/airflow",TMPDIR="/tmp/airflow" autorestart=true [program:flower] command=/usr/local/bin/celery flower --broker=redis://localhost:6379/0 --basic_auth=airflow:%(ENV_FLOWER_PASSWORD)s %(ENV_FLOWER_CERTFILE_OPTION)s %(ENV_FLOWER_KEYFILE_OPTION)s stopsignal=QUIT stopasgroup=true killasgroup=true user=root stdout_logfile=/var/log/airflow/flower-stdout.log stderr_logfile=/var/log/airflow/flower-stderr.log environment=HOME="/root",AIRFLOW_HOME="/usr/local/airflow",TMPDIR="/tmp/airflow" autorestart=true [supervisord] logfile = /tmp/supervisord.log logfile_maxbytes = 50MB logfile_backups=10 loglevel = info pidfile = /tmp/supervisord.pid [unix_http_server] file=/tmp/supervisor.sock [supervisorctl] serverurl = unix:///tmp/supervisor.sock prompt = airflow [rpcinterface:supervisor] supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
Re: Long pending PR
Hi Sid, Can you merge this now? Thanks, Sumit On Thu, Aug 11, 2016 at 11:43 AM, siddharth anand wrote: > Looking at it now! > > > -s > > On Wed, Aug 10, 2016 at 10:59 PM, Sumit Maheshwari > > wrote: > > > Hello guys, > > > > There has been a PR long pending for review, can some one please take a > > look on it and merge it? > > > > https://github.com/apache/incubator-airflow/pull/1682/files > > > > There is a very small change in core files and all changes are in contrib > > section only, which too are mainly for fixing pep8 warnings only. > > > > > > Thanks, > > Sumit > > >
Re: Private channel being used for discussion on gitter
Unfortunately chat rooms cannot be made public/private after creation. See https://github.com/gitterHQ/gitter/issues/676 I have created #airflow and #airflow-dev on freenode.net (IRC). I have requested logging with botbot.me (https://botbot.me), although that may take a while to come through. Links to the logs will updated once available. If you are behind a corporate firewall or don’t have an irc client you can try this: Dev: https://kiwiirc.com/client/irc.freenode.net/#airflow-dev Users: https://kiwiirc.com/client/irc.freenode.net/#airflow Otherwise do the usual. Cheers Bolke. > Op 23 aug. 2016, om 04:37 heeft siddharth anand het > volgende geschreven: > > Sure, we can make it public for posterity if that will alleviate concerns > about openness. I don't think it will provide any value to do so because > it will create 2 channels for the same group and will then be super hard > for anyone to follow a conversation. As it is, with JIRA emails, PR emails, > and dev list emails, I find it impossible to keep up. I'm completely unable > to keep up with most gitter conversations as it is. > > I'd vote to just delete that channel. Gitter, AFAIK, does not support the > option to provide a read only archive of a dead (unjoinable) channel, so I > feel closing the channel via filing an infra request is the best route. > > Ironically, the committers' channel above was created prior to the Apache > Incubation move to discuss the move to Apache and to coordinate the work > required. Once Airflow moved to the Apache Incubator and the Git repo's > ownership transitioned to the ASF Infra team, we (the members of the > channel) lost the ability to delete this channel and let it continue its > life out of laziness. > > -s > > On Mon, Aug 22, 2016 at 7:08 PM, Bolke de Bruin wrote: > >> So we ask infra to make it public? >> >> Sent from my iPhone >> >>> On 23 aug. 2016, at 04:02, siddharth anand wrote: >>> >>> No one on this thread owns that channel, so all we can do is leave it. >> The >>> owner is ASF Infra. >>> >>> -s >>> On Mon, Aug 22, 2016 at 2:07 PM, Jakob Homan wrote: Gitter is fine and the public channel I linked to in the original email is fine. It's functionally no different than IRC. The channel just has to be open for any one rather than invite only. -Jakob On 22 August 2016 at 13:10, Maxime Beauchemin wrote: > I think Gitter must have just transferred along as we moved the repo. >> I'm > personally open to using IRC. > > Max > >> On Mon, Aug 22, 2016 at 12:50 PM, Bolke de Bruin > wrote: > >> Also who is the owner of this room? It seems created by Apache so they >> (INFRA?) can set it to public? Logs will be public then by default. >> >> Wasn't there some discussion on a more relaxed requirement of this >> happening on ASF infrastructure? >> >> Bolke. >> >> Sent from my iPhone >> >>> On 22 aug. 2016, at 20:29, Jakob Homan wrote: >>> >>> Hey all- >>> Apparently there is a private channel being used for project >>> discussion on gitter (the archive would be here: >>> https://gitter.im/apache/incubator-airflow/Airflow_ >> committers/archives/all, >>> similar to the public archive of the regular gitter channel: >>> https://gitter.im/apache/incubator-airflow/archives/all), titled >>> "Airflow_committers." >>> >>> As has been discussed before, all non-PMC level ASF communication >>> *has* to be in a public accessible forum (preferably that is archived >>> to an ASF list). Even if well intentioned, these channels are >>> exclusionary and absolutely verboten per ASF. >>> >>> We need to open or shut that channel down and make sure we note this >>> on the next IPMC report... >>> >>> -Jakob >>
