Hi all,
I am here to propose Apache Dolphinscheduler version 3.2.0 plan to
release in July 2023 or August 2023.
Actually, I already created a new branch named `3.2.0-prepare` for
it[1], including all features merged to branch dev after we release
3.1.0(date Sep 28, 2022), including many interesting functional
additions and code base refactoring.
Version 3.2.0 is WIP in testing and bug fixing now, anyone interested
to Dolphinscheduler or want to contribute to the latest version is
welcome to test it. You can checkout the 3.2.0-prepare[1] branch and
start the standalone server, or build tarball for cluster mode test
based on your wish. In this version, we need to test the core function
of dolphinscheduler like CURD of project, workflow, task and execute
them. we allow need to test the resource center carefully because it
changes a lot in this version.
You can report dolpinscheduler issue[2] if you find any bugs. We have
a specific version for this incoming version name `3.2.0-prepare` in
the bugs reported template.
And feel free to submit PR to fix bugs, whatever is reported by others or you
Some of the functional features list for 3.2.0:
* User friendly
* UI only: More hyperlinks jump between resource, adjustable column width
* Default tenant: Out of box run workflow by default tenant
* Run or rerun task in task instance
* Human readable json export result
* Cloud native
* Kubernetes Real-time log and custom label support
* Support KEDA autoscaler worker
* Support Terraform deploy to AWS
* Multi arch for zookeper and postgresql(all services are supported now)
* Resource center
* Refactor resource center, we use provider API to list and manage
resource now
* Allow resource re-upload now
* Datasource
* New datasource:
snowflake、databend、kyuubi、oceanbase、dameng、azuresql、starrocks、AWS
Athena
* Support access key for redshift datasource
* Tasks
* New tasks: azure datafactory、apache
linkis、Kubeflow、datasync、remote-shell、AWS Database Migration、Java Task
* Improve for exists: Add presto for datax, parameter passing for
python and http, sql task multiple sql delimiter, dependent depend
itself
* Introduce task cache for MLops or other long-run tasks
* Parameter
* Add new rule of build-in function
* Introduce file type parameter
* Startup parameters have the highest priority
* Logs: Introduce remote logs mechanism and support Google Cloud
Storage、S3、OSS up to now
* API:
* Introduce some new restful API(experimental), including
taskInstance、workflow state、workflowInstance、workflow and
schedule、task relation
* Can get task instance id when triggering workflow via API
* Registry: Introduce etcd, jdbc supported
[1]: https://github.com/apache/dolphinscheduler/tree/3.2.0-prepare
[2]: https://github.com/apache/dolphinscheduler/issues/new/choose
--
Best Wish
—Jay Chung
