stephenatgithub opened a new issue, #37987:
URL: https://github.com/apache/airflow/issues/37987

   ### Apache Airflow version
   
   Other Airflow 2 version (please specify below)
   
   ### If "Other Airflow 2 version" selected, which one?
   
   2.6.3
   
   ### What happened?
   
   We have a DAG scheduled to run on 2024-03-08 08:00 HKT. However, it is not 
run as expected.
   
   Checked linux log folder, 2024-03-08 folder is not found at 
/airflow/logs/dag_id=<second DAG>/run_id=scheduled__2024-03-08T00:00:00+00:00
   
   We are using postgres and checked database table dag_run, the 2024-03-08 
record is not found.
   
   So there is no error message that we can google it.
   
   This DAG run fine for a long time before and the last run in dashboard shows 
that yesterday, but not today.
   
![image](https://github.com/apache/airflow/assets/101494292/2272ba2e-e707-4c05-a061-827e9b8c59d7)
   
   This is the DAG code
   ```
   from airflow.decorators import dag, task
   from datetime import datetime
   from airflow.timetables.trigger import CronTriggerTimetable
   
   @dag(
       start_date=datetime(2023,9,12),
       schedule=CronTriggerTimetable("0 8 * * 1-5", timezone="Asia/Hong_Kong")
   )
   def xxx():
   
       @task
       def mssql_task():
           from airflow.models import Variable
           xxx_Variable = Variable.get("xxx", deserialize_json=True)
   
           from airflow.providers.microsoft.mssql.hooks.mssql import MsSqlHook
   
           mssql_hook = MsSqlHook(mssql_conn_id=xxx_Variable["mssql_conn_id"])
           mssql_result = mssql_hook.get_records(sql=xxx_Variable["sql"])
   
       mssql_task()
   
   xxx()
   ```
   
   for scheduler, we are using LocalExecutor with all setup
   
![image](https://github.com/apache/airflow/assets/101494292/d294305e-d188-45cf-b1d9-e7853eb18bd2)
   
![image](https://github.com/apache/airflow/assets/101494292/1e0df086-ac72-49ac-9f9f-00c2cbe4f30e)
   
   
   
   
   
   ### What you think should happen instead?
   
   We tried to check log file but no today folder is created.
   We tried to check table dag_run, no today record is created.
   We tried to search github issue history, but no error message can be looked 
at.
   
   Wild guess...
   It is not webserver problem.
   It is not triggerer problem as we did not use it.
   It is not DAG file problem as we did not change the code for 5 months.
   It is not metadata DB problem as no record is created.
   It is not parsing problem in scheduler as this DAG run as expected for long 
time.
   It is not executing problem in scheduler as this DAG does not even run.
   
   It may be scheduling problem in scheduler scheduler_job_runner.py
   
   However, another DAG running at the same time on 2024-03-08 08:00 HKT worked 
fine. There is no relation between them.
   
![image](https://github.com/apache/airflow/assets/101494292/3bb3be89-c9c1-4b90-8e9b-3a80b0c79298)
   
   
   
   
   
   
   
   ### How to reproduce
   
   1. create 2 separate different DAG running at the same time
   2. the first and second DAG run as expected for a long time.
   3. Suddenly, one of them did not run while another DAG keep running fine.
   
   ### Operating System
   
   Ubuntu 22.04.2 LTS
   
   ### Versions of Apache Airflow Providers
   
   Package Name Version Description
   
   apache-airflow-providers-common-sql  1.5.2   Common SQL Provider<br>
   apache-airflow-providers-ftp 3.4.2   File Transfer Protocol (FTP)<br>
   apache-airflow-providers-http        4.4.2   Hypertext Transfer Protocol 
(HTTP)<br>
   apache-airflow-providers-imap        3.2.2   Internet Message Access 
Protocol (IMAP)<br>
   apache-airflow-providers-microsoft-mssql     3.4.1   Microsoft SQL Server 
(MSSQL)<br>
   apache-airflow-providers-sftp        4.3.1   SSH File Transfer Protocol 
(SFTP)<br>
   apache-airflow-providers-smtp        1.2.0   Simple Mail Transfer Protocol 
(SMTP)<br>
   apache-airflow-providers-sqlite      3.4.2   SQLite<br>
   apache-airflow-providers-ssh 3.7.1   Secure Shell (SSH)<br>
   apache-airflow-providers-telegram    4.1.1   Telegram<br>
   
   ### Deployment
   
   Virtualenv installation
   
   ### Deployment details
   
   pip install 
"apache-airflow[statsd,ssh,smtp,http,ftp,sftp,common.sql,microsoft.mssql,telegram]==${AIRFLOW_VERSION}"
 --constraint "${CONSTRAINT_URL}"
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to