loftiest commented on PR #18273: URL: https://github.com/apache/dolphinscheduler/pull/18273#issuecomment-4495883561
> We do not recommend using `localparams` to store file paths, as this can easily lead to issues such as file renaming and may be affected by `varpool`. Instead, we should add a new switch at the task level to indicate whether the current task’s output should be uploaded, and include a parameter to specify whether to use the output from a particular upstream task. Thanks for the suggestion! I agree that storing file paths in localParams and coupling with varPool can be fragile, especially with renaming. A dedicated task-level switch would indeed be a more robust long-term solution. However, the goal of this PR is to restore the file parameter transfer feature, rather than redesign it. The existing FILE parameter mechanism has been used and documented for several versions. Changing it now would be a breaking change and should go through a separate improvement proposal. I think your idea is great as a new feature/improvement. Could we track that in a new issue? I'd be happy to help design and implement it. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
