Re: [External] Re: scheduler logging

2019-04-23 Thread Frank Maritato
ok, yes, looks like 1.10.3 resolves this. Thanks for the help! On Tue, Apr 23, 2019 at 2:32 PM Kaxil Naik wrote: > Yes, please update us if 1.10.3 resolves that for you or if you face > same problem. > > On Tue, Apr 23, 2019 at 10:30 PM Frank Maritato > wrote: > > > Ok, I'll update to 1.10.3 a

Re: [External] Re: scheduler logging

2019-04-23 Thread Kaxil Naik
Yes, please update us if 1.10.3 resolves that for you or if you face same problem. On Tue, Apr 23, 2019 at 10:30 PM Frank Maritato wrote: > Ok, I'll update to 1.10.3 and see if I see the same behavior. > > On Tue, Apr 23, 2019 at 2:28 PM Kaxil Naik wrote: > > > It works for me i.e in *1.10.3 *i

Re: [External] Re: scheduler logging

2019-04-23 Thread Frank Maritato
Ok, I'll update to 1.10.3 and see if I see the same behavior. On Tue, Apr 23, 2019 at 2:28 PM Kaxil Naik wrote: > It works for me i.e in *1.10.3 *if I changed the "logging_level = > WARN" in *airflow.cfg > *file. I no longer get *info* logs in logs/schedule or in the UI. > > > > On Tue, Apr 23,

Re: [External] Re: scheduler logging

2019-04-23 Thread Kaxil Naik
It works for me i.e in *1.10.3 *if I changed the "logging_level = WARN" in *airflow.cfg *file. I no longer get *info* logs in logs/schedule or in the UI. On Tue, Apr 23, 2019 at 10:08 PM Kaxil Naik wrote: > On having a closer look, only cli.py was using the LOGGING_LEVEL from > settings.py >

Re: [External] Re: scheduler logging

2019-04-23 Thread Frank Maritato
Yes, I've tried this. I created config/log_config.py with the 1.10.0 contents of that file and with airflow.cfg:core.logging_level = ERROR and I still get the INFO log messages when I run: airflow scheduler -D On Tue, Apr 23, 2019 at 2:09 PM Kaxil Naik wrote: > On having a closer look, only cl

Re: [External] Re: scheduler logging

2019-04-23 Thread Kaxil Naik
On having a closer look, only cli.py was using the LOGGING_LEVEL from settings.py The logger is configured at https://github.com/apache/airflow/blob/master/airflow/config_templates/airflow_local_settings.py Check the following note in UPDATING.md: https://github.com/apache/airflow/blob/85899b3ae

Re: scheduler logging

2019-04-23 Thread Daniel Standish
Re: > Can you confirm if that log message is actually affecting behaviour, or > does it just look bad? > -ash I have not noticed any issues. My guess is it just looks bad. I am not sure which process it's coming from. On Tue, Apr 23, 2019 at 12:13 PM Ash Berlin-Taylor wrote: > Can you conf

Re: [External] Re: scheduler logging

2019-04-23 Thread Frank Maritato
Thank you!! On Tue, Apr 23, 2019 at 1:54 PM Kaxil Naik wrote: > @Frank - It is still the case. I will fix this in master and should be > fixed in 1.10.4 > > On Tue, Apr 23, 2019 at 9:38 PM Frank Maritato > wrote: > > > sorry, it's line 93, not 35. > > > > On Tue, Apr 23, 2019 at 1:34 PM Frank M

Re: [External] Re: scheduler logging

2019-04-23 Thread Kaxil Naik
@Frank - It is still the case. I will fix this in master and should be fixed in 1.10.4 On Tue, Apr 23, 2019 at 9:38 PM Frank Maritato wrote: > sorry, it's line 93, not 35. > > On Tue, Apr 23, 2019 at 1:34 PM Frank Maritato > wrote: > > > Ok, I figured out why this is happening. settings.py:35 i

Re: [External] Re: scheduler logging

2019-04-23 Thread Frank Maritato
sorry, it's line 93, not 35. On Tue, Apr 23, 2019 at 1:34 PM Frank Maritato wrote: > Ok, I figured out why this is happening. settings.py:35 is hard coding the > logging level to INFO. I hacked it locally to: > > LOGGING_LEVEL = conf.get('core', 'logging_level') > > and now all the INFO logs are

Re: [External] Re: scheduler logging

2019-04-23 Thread Frank Maritato
Ok, I figured out why this is happening. settings.py:35 is hard coding the logging level to INFO. I hacked it locally to: LOGGING_LEVEL = conf.get('core', 'logging_level') and now all the INFO logs are gone. Is this fixed in a later version of airflow? On Tue, Apr 23, 2019 at 1:09 PM Frank Mari

Re: [External] Re: scheduler logging

2019-04-23 Thread Frank Maritato
Those two issues don't seem to be present in 1.10.0. I only see one message for "db connection invalidated" and I don't see anything about harvesting. I have tried using a custom logger to see if that effects the output to airflow-scheduler.{log|out} but so far I haven't had any luck. The logs tha

Re: scheduler logging

2019-04-23 Thread Ash Berlin-Taylor
Can you confirm if that log message is actually affecting behaviour, or does it just look bad? -ash On 23 April 2019 19:38:57 BST, Daniel Standish wrote: >I noticed two issues in 1.10.2, one of which has been resolved, but I >am >not sure if they were present in 1.10.0, or if they are the messa

Re: scheduler logging

2019-04-23 Thread Daniel Standish
I noticed two issues in 1.10.2, one of which has been resolved, but I am not sure if they were present in 1.10.0, or if they are the messages that are bothering you. 1. "Harvesting DAG parsing results" was printed every 2 seconds or something. This was resolved in commit [AIRFLOW-3911] Change Harv

Re: scheduler logging

2019-04-23 Thread Bolke de Bruin
It's probably better to create a custom logging.conf and use that instead. B. Op di 23 apr. 2019 18:13 schreef Frank Maritato : > No one else has this issue? Or no one has a solution? > > > On Wed, Apr 17, 2019 at 5:49 PM Frank Maritato > wrote: > > > Hi All, > > > > We are running airflow 1.10

Re: scheduler logging

2019-04-23 Thread Frank Maritato
No one else has this issue? Or no one has a solution? On Wed, Apr 17, 2019 at 5:49 PM Frank Maritato wrote: > Hi All, > > We are running airflow 1.10.0 and I'm wondering how I can turn off or turn > down the logging for the scheduler? I tried setting logging_level=WARN in > airflow.cfg and rest

scheduler logging

2019-04-17 Thread Frank Maritato
Hi All, We are running airflow 1.10.0 and I'm wondering how I can turn off or turn down the logging for the scheduler? I tried setting logging_level=WARN in airflow.cfg and restarting the process but I'm still seeing a ton of info logging to .out and .log. Thanks! -- Frank Maritato