Re: [DISCUSS] Removing Pickling from Airflow 2.0

Thank You for the thoughts. I will create a VOTE thread at some point today / early tomorrow. Regards, Kaxil On Mon, Sep 21, 2020 at 6:11 PM Jarek Potiuk wrote: > Happy to kill pickling for 2.0. While starting to review the HA, I see > that more and more we rely on Serialization and there are s

Re: [DISCUSS] Removing Pickling from Airflow 2.0

Happy to kill pickling for 2.0. While starting to review the HA, I see that more and more we rely on Serialization and there are some rather weird-looking left-overs in Ash's change that are only there because of pickling. I think we already know that Serialization becomes a first-class citizen in

Re: [DISCUSS] Removing Pickling from Airflow 2.0

Are there any use-cases that REQUIRE pickle? Do we have any sense of what % of the Airflow community depends on Pickle? I’m all for killing it if possible but I want to make sure we’re not setting up a major hurdle for migration. via Newton Mail [https://cloudmagic.com/k/d/mailapp?ct=dx&cv=10.0

Re: [DISCUSS] Removing Pickling from Airflow 2.0

I'm getting bad flashbacks of fighting with pickles early on in the history of the project. I've learned since then to stay away. Almost all solutions that involve pickles are bad solutions. Beyond but related to the security implication are the issues of pickle entanglement, not really knowing wha

[DISCUSS] Removing Pickling from Airflow 2.0

Hi all, We briefly discussed how pickling is currently used in Airflow codebase and whether or not we should remove it for 2.0 in the Airflow 2.0 Dev call this Monday. Currently, AFAIK only *CeleryExecutor* supports pickling (code