Re: celery problem, AttributeError: async
Will the branch name be the same (v1-10-test) after you rebase? Will you change the version number in that branch to 1.10: ( see https://github.com/apache/incubator-airflow/pull/3389 )? I am heavily testing this branch right now, and want to deploy it to my prod system this Friday. Thanks. -- Craig On 2018/05/21 17:58:12, Bolke de Bruin wrote: > We will rebranch 1.10 from master. Sorry, I have been too busy with normal > life to be able to follow up on the release of 1.10. > > B.
Re: celery problem, AttributeError: async
We will rebranch 1.10 from master. Sorry, I have been too busy with normal life to be able to follow up on the release of 1.10. B. > On 21 May 2018, at 19:54, Craig Rodrigues wrote: > > Kaxil, > > Thanks for merging this into master. > What is the procedure to get this into v1-10-test branch? > I am heavily testing that branch right now, and want to deploy that branch to > my prod system > this Friday. > > -- > Craig > > On 2018/05/21 14:52:33, Craig Rodrigues wrote: >> I have submitted: >> >> https://github.com/apache/incubator-airflow/pull/3388 >> >> -- >> Craig >
Re: celery problem, AttributeError: async
Kaxil, Thanks for merging this into master. What is the procedure to get this into v1-10-test branch? I am heavily testing that branch right now, and want to deploy that branch to my prod system this Friday. -- Craig On 2018/05/21 14:52:33, Craig Rodrigues wrote: > I have submitted: > > https://github.com/apache/incubator-airflow/pull/3388 > > -- > Craig
Re: celery problem, AttributeError: async
I have submitted:
https://github.com/apache/incubator-airflow/pull/3388
--
Craig
On Mon, May 21, 2018 at 7:00 AM Naik Kaxil wrote:
> Thanks. Please do that.
>
> On 21/05/2018, 14:59, "Craig Rodrigues" wrote:
>
> celery 4.1.1 was just released last night which has all the async
> problems fixed:
>
> https://github.com/celery/celery/commits/v4.1.1
>
> I'll test this out, and then submit a PR to bump airflow's celery
> version to 4.1.1
>
> --
> Craig
>
> On 2018/05/21 07:20:50, Craig Rodrigues wrote:
> > Hi,
> >
> > I used a requiremens.txt file with these three lines:
> >
> > git+
> https://github.com/apache/incubator-airflow@v1-10-test#egg=apache-airflow[celery,crypto,emr,hive,hdfs,ldap,mysql,postgres,redis,slack,s3]
> > celery>=4.2.0rc3
> > kombu>=4.2.0
> >
> >
> > I did
> >
> > pip install -r requirements.txt
> >
> > When I started my worker, I got:
> >
> > [2018-05-21 06:39:23,980] {__init__.py:48} INFO - Using executor
> CeleryExecutor
> > Traceback (most recent call last):
> > File "/bin/airflow", line 32, in
> > args.func(args)
> > File "/usr/lib/python2.7/site-packages/airflow/utils/cli.py", line
> 74, in wrapper
> > return f(*args, **kwargs)
> > File "/usr/lib/python2.7/site-packages/airflow/bin/cli.py", line
> 959, in worker
> > worker.run(**options)
> > File "/usr/lib/python2.7/site-packages/celery/bin/worker.py", line
> 257, in run
> > **kwargs)
> > File "/usr/lib/python2.7/site-packages/celery/worker/worker.py",
> line 101, in __init__
> > self.setup_instance(**self.prepare_args(**kwargs))
> > File "/usr/lib/python2.7/site-packages/celery/worker/worker.py",
> line 124, in setup_instance
> > self.should_use_eventloop() if use_eventloop is None
> > File "/usr/lib/python2.7/site-packages/celery/worker/worker.py",
> line 243, in should_use_eve
> > self._conninfo.transport.implements.async and
> > File "/usr/lib/python2.7/site-packages/kombu/transport/base.py",
> line 125, in __getattr__
> > raise AttributeError(key)
> > AttributeError: async
> >
> >
> >
> > I reported this as a bug in celery here:
> >
> > https://github.com/celery/celery/issues/4747
> >
> > It turns out that this is a known problem. It looks like in kombu,
> they
> > renamed a bunch of uses of "async" to "asynchronous", to not clash
> with the new
> > 'async' keyword in Python 3.7.
> >
> > In celery, they are catching up and doing the same rename.
> > Almost at the same time that I reported this bug, the issue was
> fixed in the master branch of celery:
> >
> >
> https://github.com/celery/celery/commit/c8ef7ad60b72a194654c58beb04a1d65cd0435ad
> >
> > Hopefully the versions of kombu and celery will stabilize so that it
> all works properly with airflow.
> >
> > --
> > Craig
> >
> >
> >
>
>
>
>
> Kaxil Naik
>
> Data Reply
> 2nd Floor, Nova South
> 160 Victoria Street, Westminster
> London SW1E 5LB - UK
> phone: +44 (0)20 7730 6000
> k.n...@reply.com
> www.reply.com
>
Re: celery problem, AttributeError: async
Thanks. Please do that.
On 21/05/2018, 14:59, "Craig Rodrigues" wrote:
celery 4.1.1 was just released last night which has all the async problems
fixed:
https://github.com/celery/celery/commits/v4.1.1
I'll test this out, and then submit a PR to bump airflow's celery version
to 4.1.1
--
Craig
On 2018/05/21 07:20:50, Craig Rodrigues wrote:
> Hi,
>
> I used a requiremens.txt file with these three lines:
>
>
git+https://github.com/apache/incubator-airflow@v1-10-test#egg=apache-airflow[celery,crypto,emr,hive,hdfs,ldap,mysql,postgres,redis,slack,s3]
> celery>=4.2.0rc3
> kombu>=4.2.0
>
>
> I did
>
> pip install -r requirements.txt
>
> When I started my worker, I got:
>
> [2018-05-21 06:39:23,980] {__init__.py:48} INFO - Using executor
CeleryExecutor
> Traceback (most recent call last):
> File "/bin/airflow", line 32, in
> args.func(args)
> File "/usr/lib/python2.7/site-packages/airflow/utils/cli.py", line 74,
in wrapper
> return f(*args, **kwargs)
> File "/usr/lib/python2.7/site-packages/airflow/bin/cli.py", line 959,
in worker
> worker.run(**options)
> File "/usr/lib/python2.7/site-packages/celery/bin/worker.py", line 257,
in run
> **kwargs)
> File "/usr/lib/python2.7/site-packages/celery/worker/worker.py", line
101, in __init__
> self.setup_instance(**self.prepare_args(**kwargs))
> File "/usr/lib/python2.7/site-packages/celery/worker/worker.py", line
124, in setup_instance
> self.should_use_eventloop() if use_eventloop is None
> File "/usr/lib/python2.7/site-packages/celery/worker/worker.py", line
243, in should_use_eve
> self._conninfo.transport.implements.async and
> File "/usr/lib/python2.7/site-packages/kombu/transport/base.py", line
125, in __getattr__
> raise AttributeError(key)
> AttributeError: async
>
>
>
> I reported this as a bug in celery here:
>
> https://github.com/celery/celery/issues/4747
>
> It turns out that this is a known problem. It looks like in kombu, they
> renamed a bunch of uses of "async" to "asynchronous", to not clash with
the new
> 'async' keyword in Python 3.7.
>
> In celery, they are catching up and doing the same rename.
> Almost at the same time that I reported this bug, the issue was fixed in
the master branch of celery:
>
>
https://github.com/celery/celery/commit/c8ef7ad60b72a194654c58beb04a1d65cd0435ad
>
> Hopefully the versions of kombu and celery will stabilize so that it all
works properly with airflow.
>
> --
> Craig
>
>
>
Kaxil Naik
Data Reply
2nd Floor, Nova South
160 Victoria Street, Westminster
London SW1E 5LB - UK
phone: +44 (0)20 7730 6000
k.n...@reply.com
www.reply.com
Re: celery problem, AttributeError: async
celery 4.1.1 was just released last night which has all the async problems
fixed:
https://github.com/celery/celery/commits/v4.1.1
I'll test this out, and then submit a PR to bump airflow's celery version to
4.1.1
--
Craig
On 2018/05/21 07:20:50, Craig Rodrigues wrote:
> Hi,
>
> I used a requiremens.txt file with these three lines:
>
> git+https://github.com/apache/incubator-airflow@v1-10-test#egg=apache-airflow[celery,crypto,emr,hive,hdfs,ldap,mysql,postgres,redis,slack,s3]
> celery>=4.2.0rc3
> kombu>=4.2.0
>
>
> I did
>
> pip install -r requirements.txt
>
> When I started my worker, I got:
>
> [2018-05-21 06:39:23,980] {__init__.py:48} INFO - Using executor
> CeleryExecutor
> Traceback (most recent call last):
> File "/bin/airflow", line 32, in
> args.func(args)
> File "/usr/lib/python2.7/site-packages/airflow/utils/cli.py", line 74, in
> wrapper
> return f(*args, **kwargs)
> File "/usr/lib/python2.7/site-packages/airflow/bin/cli.py", line 959, in
> worker
> worker.run(**options)
> File "/usr/lib/python2.7/site-packages/celery/bin/worker.py", line 257, in
> run
> **kwargs)
> File "/usr/lib/python2.7/site-packages/celery/worker/worker.py", line 101,
> in __init__
> self.setup_instance(**self.prepare_args(**kwargs))
> File "/usr/lib/python2.7/site-packages/celery/worker/worker.py", line 124,
> in setup_instance
> self.should_use_eventloop() if use_eventloop is None
> File "/usr/lib/python2.7/site-packages/celery/worker/worker.py", line 243,
> in should_use_eve
> self._conninfo.transport.implements.async and
> File "/usr/lib/python2.7/site-packages/kombu/transport/base.py", line 125,
> in __getattr__
> raise AttributeError(key)
> AttributeError: async
>
>
>
> I reported this as a bug in celery here:
>
> https://github.com/celery/celery/issues/4747
>
> It turns out that this is a known problem. It looks like in kombu, they
> renamed a bunch of uses of "async" to "asynchronous", to not clash with the
> new
> 'async' keyword in Python 3.7.
>
> In celery, they are catching up and doing the same rename.
> Almost at the same time that I reported this bug, the issue was fixed in the
> master branch of celery:
>
> https://github.com/celery/celery/commit/c8ef7ad60b72a194654c58beb04a1d65cd0435ad
>
> Hopefully the versions of kombu and celery will stabilize so that it all
> works properly with airflow.
>
> --
> Craig
>
>
>
