[DISCUSSION] Which pagination to use in the v2-API of DS

nation: Choosing the Best Approach]( https://medium.com/@maryam-bit/offset-vs-cursor-based-pagination-choosing-the-best-approach-2e93702a118b ) Best Regards, Rick Cheng

Re: [Discussion] Move DolphinScheduler Api V2 Forward

Thanks Yifan for bringing this up. This is meaningful and long-term work, and I'm also willing to participate and contribute. Best Regards, Rick Cheng Jay Chung 于2023年10月31日周二 09:49写道： > +1 for this and thanks to Yifan for bringing this up. > we should continue API refactoring > >

[DISCUSS][Feature] Organize workflows and tasks with directories

, Rick Cheng

[DISCUSS][Feature][Parameter] Add project-level parameters

location parameter ${loc} I gave some design in this issue. If you are interested, any comments or suggestions are welcome. Best Regards, Rick Cheng

Re: [DISCUSS][Feature][Remote Logging] Add support for writing task logs to remote storage

retention on OSS]( https://help.aliyun.com/document_detail/326319.html) Best Regards, Rick Cheng Rick Cheng 于2022年12月8日周四 10:31写道： > Hi community, > > Here are some discussions on the weekly meeting about this feature: > > **Q1: In the k8s environment, users can choose to mount per

Re: [DISCUSS][Feature][Remote Logging] Add support for writing task logs to remote storage

: This is a good idea and it can simplify the design of remote logging, I'll look into it. Related issue: https://github.com/apache/dolphinscheduler/issues/13017 Thanks again for all the suggestions at the weekly meeting, please correct me if I'm wrong. Best Regards, Rick Cheng Rick Cheng 于2022年11月28日周一 13

[DISCUSS][Feature][Remote Logging] Add support for writing task logs to remote storage

the remote log if the local log file does not exist. We could discuss this further. ## Log retention strategy For example, the maximum capacity of remote storage can be set, and old logs can be deleted by rolling. # Sub-tasks WIP Any comments or suggestions are welcome. Best Regards, Rick Cheng

[DISCUSS] Fix the errors when starting 2 times with dolphinscheduler-daemon.sh

the pid). And I personally prefer to check the pid file because it's a way used by a lot of programs on linux. Any comments and suggestions are welcome! Related PR: https://github.com/apache/dolphinscheduler/pull/12118 Best Regards, Rick Cheng

Re: [DISCUSS] Improved way to collect yarn job's appIds

Hi, Aaron Wang I agree that the current way of getting the yarn application id from the log is not elegant. Just for discussion, there is another way to get yarn application id as below: We can put some unique tags on tasks submitted from DS to yarn. E.g., for spark tasks, we can add the

[DISCUSS] Mask the sensitive data (e.g., password) in the task log output

Hi, all There are 2 possible locations that the log may contain the sensitive data (e.g., password). Taking the sqoop task as an example: 1. The log output in the `buildCommand()` in `SqoopTask`. 2. The log output in the `createCommandFileIfNotExists(String execCommand, String commandFile)` in