javatarz commented on issue #31875:
URL: https://github.com/apache/airflow/issues/31875#issuecomment-1590306133

   > create a concatenation scripts
   
   This is precisely what I meant in my original post when I said "The 
individual building blocks for this change are already available. The 
(design/philosophical) choice is between making Airflow easier to use and 
keeping airflow minimalistic i.e. provide the building blocks for users to 
build on". The blocks are available and it's fairly trivial to write a script 
to do this. Doing so would mean it becomes a small script that many Airflow 
consumers have to write. It's simple enough to build but would be better off as 
a capability in Airflow (despite it being a deceptively large change to make 
sure Airflow is consistent in it's interfaces).
   
   I'm happy writing up a script and pushing it on the thread if someone's in 
need of one. It'll be a script that you will have to maintain over a period of 
time. Our team already has a couple of such things. Every airflow upgrade feels 
more painful than it should be because we spend half our time just getting 
these utilities to work fine. One such example is what we've had to do for unit 
tests (which I'm hoping, some day, will become part of airflow's core offering 
too). But that's a discussion for another ticket on another day :) I seem to 
have bit off an AIP level change that I'd like to see through so baby steps.. :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to