Sounds like an interesting application of airflow which may not be the best use-case for it.
How are you triggering these dag runs? airflow trigger_dag? If so, you can find the logs for your specific RUN_ID in the ui without grepping through all logs first, but with hundreds of dag runs, it will still take a bit of searching. If the unique ID is something known internally to your application, then it's not something that Airflow knows about, so grep is your only recourse. On Wed, May 3, 2017 at 9:05 AM, EKC (Erik Cederstrand) <[email protected]> wrote: > Hi all, > > > I have created a DAG, 'my_dag', that will always be triggered manually, > and will always receive a unique ID from the command-line, specifying the > entity the DAG must process. The DAG may be triggered hundreds of times a > day. > > > In the Airflow UI, the DAG will show up everywhere as 'my_dag', which is > not very relevant to me. A single DAG run may run for several days, so I > want to easily look up specific DAG runs using the ID, to check the status > of that DAG run. But the ID only shows up in the log file, so to find the > status of a specific DAG run, I need to open every DAG run and check the > log file. That is very cumbersome when there are hundreds of simultaneous > DAG runs. > > > What are my options if I want to improve this situation? > > > Kind regards, > > Erik >
