While I understand the benefits of the "clean" "REST" API where resources
are identified by a nice hierarchy, I do not think it is necessary to
follow it for this kind of queries.I think the "resources" based approach
indeed breaks down when you want to make more complex queries and want to
do it
I would recommend option III.
IMHO, hierarchy based endpoints are for CURD operations on a single
entity. The flat endpoint, like /taskinstance, is for read-only
multi-facet search queries.
For example:
* to create a resource Foo: "POST /api/parent/{parent_id}/foo"
* to update a resource Foo: "P
I strongly agree with reducing Airflows complexity and the plan to serialize
DAGs by default makes a lot of sense. So wouldn't it make sense to also
serialize the schedule of a DAG?
e.g. Provide a table of execution dates that the scheduler can keep upcoming
ones cached and periodically check
I strongly agree with Ash, I also think we should strive to decrease the
complexity of core Airflow components and not offer
customization/extensibility especially in the form of plugins where it is
not needed to make Airflow more robust and easier to reason about (less
testing configuration). I th
> 2) Many times, especially once a DAG is running, in the UI I see the warning
> "The scheduler does not appear to be running.”
The other thing that was an issue: `SCHEDULER_RUNS` in the environment was set
to some value `> -1`. This was causing "The scheduler does not appear to be
running.” wa
Hello,
Just to remind everyone: we are talking about what's coming in Apache
Airflow 2.0 this Wednesday at the online NYC Meetup:
https://www.meetup.com/NYC-Apache-Airflow-Meetup/events/270483933/.
This will be a joint presentation by Ash, Kaxil, Tomek, Daniel, Kamil, and
myself.
This Wednesday:
Hello,
I apologize for my very negative behavior. I misunderstood the rules.
I updated the specification based on the discussion. I hope that now I
have not missed any suggestions that I had to make. I tried to make
every change in a separate commit, so you can review the changes one
by one.
Here
> Ash, you had mentioned something about some plans that were in conflict
> with the above hack could you maybe share a thought or two about what
> you were thinking?
>
The main thing here is around scheduler HA, and I want the scheduler to
be able to make all scheduling decisions without nee
