Re: logging config

Hi Luke, The work in that PR was done by me and one of my colleagues. Unfortunately, we don't have time at the moment to be working on this further, but I'd be happy to answer any questions about what we've done there if it would help you pick this up and move it forwards. I agree that havin

Re: Is anybody using CeleryExecutor + redis-sentinel message broker?

Hi Desiree, It sounds to me like you'll probably need to add an additional configuration option to pass to Celery. In the file airflow/executors/celery_executor.py you can see all of the config options that are passed into Airflow in the CeleryConfig class. I've currently got a PR open for a c

Re: Airflow issue

Hi Franck We also hit this issue and rolled back our Celery version. We later decided to run Airflow on Python 3 and discovered that librabbitmq isn't compatible with Python 3 either and used the "amqp" package instead (version 2.1.2). We never got around to moving back up to a more recent ver

Re: Unit Test Failures

onger and are tied to > > python 3. > > > > Can someone have a look at its please? I am a bit preoccupied with > getting > > the release out. > > > > Cheers > > Bolke > > > >> On 8 Feb 2017, at 13:18, Miller, Robin < > > robin.mil...@a

Unit Test Failures

Hi All, On a couple of my open PRs, I've been having trouble getting the units tests to run successfully, not because there's a test failure, but because Travis is reporting that the unit test log has become too large: " The log length has exceeded the limit of 4 MB (this usually means that

Re: Airflow Logging Updates

patch branch, but maybe for the next release. I don't know what the other people think. On Wed, Feb 1, 2017 at 11:58 AM Miller, Robin < robin.mil...@affiliate.oliverwyman.com> wrote: > Hi All, > > > A while ago we took over this issue: > https://issues.apache.org/jira

Airflow Logging Updates

Hi All, A while ago we took over this issue: https://issues.apache.org/jira/browse/AIRFLOW-409, which revolved around avoiding making changes to the root python logging configuration so as to avoid any accidental side effects that this could produce in other python applications. The approac

Re: Airflow 1.8.0 alpha 2

Hi Chris, I think I ran into this issue when setting up LDAP Auth in our environment (we're using very close to master as we needed some of the newer features/bugfixes). The problem turned out to be that the search was finding no results, so the line: groups_list = [regex.search(i).group(1)

Re: AIRFLOW-699 - dags can't be triggered at the same second

Hi Olivier, Someone correct me if I'm wrong, but my understanding is as follows: At the moment the only way to identify a task_instance as for a specific dag_run is by comparing the dag_id and execution_date. As such, having two dag_runs with the same execution_date would make it impossible t

Re: Issue with latest versions of Celery & Kombu

Hi Nadeem, We are using Celery with RabbitMQ. The upgrade to Celery 4.0 last week did cause RabbitMQ trouble. The web interface (for RabbitMQ) started giving us errors pointing to the use of non-utf8 characters in message queues (or queue names, it wasn't clear), which RabbitMQ does not suppo