[GitHub] [airflow] jsmodic commented on issue #8480: Celery autoscaling overrides normal worker_concurrency setting

2020-05-03 Thread GitBox
jsmodic commented on issue #8480: URL: https://github.com/apache/airflow/issues/8480#issuecomment-623192154 I can't speak to how Celery's various versions work, but as far as I tested it should be enough to remove 'autoscale' from the dictionary to be unpacked into Celery if autoscaling is

[GitHub] [airflow] jsmodic commented on issue #8480: Celery autoscaling overrides normal worker_concurrency setting

2020-04-22 Thread GitBox
jsmodic commented on issue #8480: URL: https://github.com/apache/airflow/issues/8480#issuecomment-618135462 My airflow.cfg (and celery_config.py) doesn't reference autoscaling at all (since I haven't updated them really since 1.10.2 or so, before this feature landed in airflow). It's inter

[GitHub] [airflow] jsmodic commented on issue #8480: Celery autoscaling overrides normal worker_concurrency setting

2020-04-21 Thread GitBox
jsmodic commented on issue #8480: URL: https://github.com/apache/airflow/issues/8480#issuecomment-617282541 @turbaszek what celery version are you using? On 1.10.9 it starts up like this for me on Celery 4.3.0 with no autoscaling configured anywhere. ` [config] app: air

[GitHub] [airflow] jsmodic commented on issue #8480: Celery autoscaling overrides normal worker_concurrency setting

2020-04-20 Thread GitBox
jsmodic commented on issue #8480: URL: https://github.com/apache/airflow/issues/8480#issuecomment-616892817 Autoscaling isn't supposed to be supported in Celery 4.x, so I wouldn't be surprised to see it behaving erratically anyways. I just *was* surprised to see it always defaulting to 12