hussein-awala commented on code in PR #38531:
URL: https://github.com/apache/airflow/pull/38531#discussion_r1545512710
##########
airflow/models/dagpickle.py:
##########
@@ -42,7 +42,7 @@ class DagPickle(Base):
"""
id = Column(Integer, primary_key=True)
- pickle = Column(PickleType(pickler=dill))
+ pickle = Column(PickleType(pickler=cloudpickle))
Review Comment:
I think the changes in the models needs a migration script to keep the DB
rows parsable, I'm trying to do something similar
[here](https://github.com/apache/airflow/pull/38358).
##########
airflow/models/taskinstance.py:
##########
@@ -1287,7 +1287,7 @@ class TaskInstance(Base, LoggingMixin):
queued_dttm = Column(UtcDateTime)
queued_by_job_id = Column(Integer)
pid = Column(Integer)
- executor_config = Column(ExecutorConfigType(pickler=dill))
+ executor_config = Column(ExecutorConfigType(pickler=cloudpickle))
Review Comment:
same here
##########
airflow/operators/python.py:
##########
@@ -416,8 +418,16 @@ def __init__(
**kwargs,
)
self.string_args = string_args or []
- self.use_dill = use_dill
- self.pickling_library = dill if self.use_dill else pickle
+ if use_dill:
+ warnings.warn(
+ "The 'use_dill' parameter is deprecated and will be removed
after 01.10.2024. Please use "
+ "'use_cloudpickle' instead. ",
+ AirflowProviderDeprecationWarning,
+ stacklevel=2,
+ )
Review Comment:
In case we decide to deprecate the argument (not sure after reading @potiuk
[comment](https://github.com/apache/airflow/pull/38531/files#r1540962108) which
makes sense), it would be better to update the default value of `use_dill` to
`NOTSET` and check if it is set to `True` or `False` in case it is set
explicitly to `False`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
