soulmem opened a new issue, #17495: URL: https://github.com/apache/dolphinscheduler/issues/17495
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues. ### What happened A clean install fails until two manual changes are made: 1.Missing storage plug-in JARs The storage JARs (dolphinscheduler-storage-s3-*.jar, dolphinscheduler-storage-hdfs-*.jar, etc.) exist in /opt/dolphinscheduler/libs inside the official images, but are NOT present in /opt/dolphinscheduler/worker-server/libs or /opt/dolphinscheduler/api-server/libs. → I had to copy them into those sub-folders and re-build custom images; otherwise Worker and API pods crash with No qualifying bean of type 'org.apache.dolphinscheduler.plugin.storage.api.StorageOperator'. 2.Wrong ConfigMap mount path statefulset-dolphinscheduler-worker.yaml and deployment-dolphinscheduler-api.yaml mount config-volume at /opt/dolphinscheduler/conf/common.properties. However, the start-scripts inside worker-server and api-server actually load the file from /opt/dolphinscheduler/worker-server/conf/common.properties and /opt/dolphinscheduler/api-server/conf/common.properties respectively. → I had to patch both manifests and move the mount path, otherwise storage-type is ignored and the same “StorageOperator not found” error appears. After these two work-arounds the cluster starts and workflows run correctly. ### What you expected to happen May I ask, although everything is working now, could these manual changes I made introduce any hidden risks or future operational problems? ### How to reproduce This is my values.yaml: alert: affinity: {} annotations: {} customizedConfig: {} enableCustomizedConfig: false enabled: true env: JAVA_OPTS: '-Xms512m -Xmx512m -Xmn256m' livenessProbe: enabled: true failureThreshold: '3' initialDelaySeconds: '30' periodSeconds: '30' successThreshold: '1' timeoutSeconds: '5' nodeSelector: {} persistentVolumeClaim: accessModes: - ReadWriteOnce enabled: true storage: 20Gi storageClassName: vsphere-csi-sc readinessProbe: enabled: true failureThreshold: '3' initialDelaySeconds: '30' periodSeconds: '30' successThreshold: '1' timeoutSeconds: '5' replicas: 1 resources: {} service: annotations: {} serviceMonitor: annotations: {} enabled: false interval: 15s labels: {} path: /actuator/prometheus strategy: rollingUpdate: maxSurge: 25% maxUnavailable: 25% type: RollingUpdate tolerations: [] api: affinity: {} annotations: {} customizedConfig: {} enableCustomizedConfig: false enabled: true env: JAVA_OPTS: '-Xms512m -Xmx512m -Xmn256m' livenessProbe: enabled: true failureThreshold: '3' initialDelaySeconds: '30' periodSeconds: '30' successThreshold: '1' timeoutSeconds: '5' nodeSelector: {} persistentVolumeClaim: accessModes: - ReadWriteOnce enabled: true storage: 20Gi storageClassName: vsphere-csi-sc readinessProbe: enabled: true failureThreshold: '3' initialDelaySeconds: '30' periodSeconds: '30' successThreshold: '1' timeoutSeconds: '5' replicas: '1' resources: {} service: annotations: {} clusterIP: '' externalIPs: [] externalName: '' loadBalancerIP: '' nodePort: '' pythonNodePort: '' serviceMonitor: annotations: {} enabled: false interval: 15s labels: {} path: /dolphinscheduler/actuator/prometheus type: ClusterIP strategy: rollingUpdate: maxSurge: 25% maxUnavailable: 25% type: RollingUpdate taskTypeFilter: enabled: false task: {} tolerations: [] common: configmap: DATAX_LAUNCHER: /opt/soft/datax/bin/datax.py DATA_BASEDIR_PATH: /tmp/dolphinscheduler DOLPHINSCHEDULER_OPTS: '' FLINK_HOME: /opt/soft/flink HADOOP_CONF_DIR: /opt/soft/hadoop/etc/hadoop HADOOP_HOME: /opt/soft/hadoop HIVE_HOME: /opt/soft/hive JAVA_HOME: /opt/java/openjdk PYTHON_LAUNCHER: /usr/bin/python3 RESOURCE_UPLOAD_PATH: /dolphinscheduler SPARK_HOME: /opt/soft/spark RESOURCE_STORAGE_TYPE: S3 fsFileResourcePersistence: accessModes: - ReadWriteMany enabled: true storage: 20Gi storageClassName: vsphere-csi-sc-rwx sharedStoragePersistence: accessModes: - ReadWriteMany enabled: true mountPath: /opt/soft storage: 20Gi storageClassName: vsphere-csi-sc-rwx conf: auto: false common: appId.collect: log aws.credentials.provider.type: AWSStaticCredentialsProvider aws.s3.access.key.id: dolphinscheduler aws.s3.access.key.secret: [email protected] aws.s3.bucket.name: dolphinscheduler aws.s3.endpoint: https://oss.cduestc.cn:443 aws.s3.region: ca-central-1 conda.path: /opt/anaconda3/etc/profile.d/conda.sh data.basedir.path: /tmp/dolphinscheduler datasource.encryption.enable: false datasource.encryption.salt: '!@#$%^&*' development.state: false hadoop.security.authentication.startup.state: false java.security.krb5.conf.path: /opt/krb5.conf kerberos.expire.time: 2 login.user.keytab.path: /opt/hdfs.headless.keytab login.user.keytab.username: [email protected] ml.mlflow.preset_repository: https://github.com/apache/dolphinscheduler-mlflow ml.mlflow.preset_repository_version: main resource.aws.credentials.provider.type: AWSStaticCredentialsProvider resource.aws.s3.access.key.id: dolphinscheduler resource.aws.s3.access.key.secret: [email protected] resource.aws.s3.bucket.name: dolphinscheduler resource.aws.s3.endpoint: https://oss.cduestc.cn:443 resource.aws.s3.region: ca-central-1 resource.alibaba.cloud.access.key.id: <your-access-key-id> resource.alibaba.cloud.access.key.secret: <your-access-key-secret> resource.alibaba.cloud.oss.bucket.name: dolphinscheduler resource.alibaba.cloud.oss.endpoint: https://oss-cn-hangzhou.aliyuncs.com resource.alibaba.cloud.region: cn-hangzhou resource.azure.client.id: minioadmin resource.azure.client.secret: minioadmin resource.azure.subId: minioadmin resource.azure.tenant.id: minioadmin resource.hdfs.fs.defaultFS: hdfs://mycluster:8020 resource.hdfs.root.user: hdfs resource.manager.httpaddress.port: 8088 resource.storage.type: S3 resource.storage.upload.base.path: /dolphinscheduler shell.kill.wait.timeout: 10 sudo.enable: true support.hive.oneSession: false task.resource.limit.state: false yarn.application.status.address: http://ds1:%s/ws/v1/cluster/apps/%s yarn.job.history.status.address: http://ds1:19888/ws/v1/history/mapreduce/jobs/%s yarn.resourcemanager.ha.rm.ids: 192.168.xx.xx,192.168.xx.xx datasource: profile: postgresql externalDatabase: database: dolphinscheduler driverClassName: org.postgresql.Driver enabled: false host: localhost params: characterEncoding=utf8 password: root port: '5432' type: postgresql username: root externalRegistry: registryPluginName: zookeeper registryServers: zookeeper-headless.zookeeper.svc.cluster.local:2181 image: alert: dolphinscheduler-alert-server api: dolphinscheduler-api master: dolphinscheduler-master pullPolicy: IfNotPresent pullSecret: cduestc-harbor registry: myharbor/library tag: 3.3.1 tools: dolphinscheduler-tools worker: dolphinscheduler-worker ingress: annotations: {} enabled: true host: xxx.mydomain.cn path: /dolphinscheduler tls: enabled: false secretName: dolphinscheduler-tls initImage: busybox: busybox:1.30.1 pullPolicy: IfNotPresent master: affinity: {} annotations: {} customizedConfig: {} enableCustomizedConfig: false enabled: true env: JAVA_OPTS: '-Xms1g -Xmx1g -Xmn512m' MASTER_DISPATCH_TASK_NUM: '3' MASTER_EXEC_TASK_NUM: '20' MASTER_EXEC_THREADS: '100' MASTER_FAILOVER_INTERVAL: 10m MASTER_HEARTBEAT_ERROR_THRESHOLD: '5' MASTER_HOST_SELECTOR: LowerWeight MASTER_KILL_APPLICATION_WHEN_HANDLE_FAILOVER: 'true' MASTER_MAX_HEARTBEAT_INTERVAL: 10s MASTER_SERVER_LOAD_PROTECTION_ENABLED: false MASTER_SERVER_LOAD_PROTECTION_MAX_DISK_USAGE_PERCENTAGE_THRESHOLDS: 0.7 MASTER_SERVER_LOAD_PROTECTION_MAX_JVM_CPU_USAGE_PERCENTAGE_THRESHOLDS: 0.7 MASTER_SERVER_LOAD_PROTECTION_MAX_SYSTEM_CPU_USAGE_PERCENTAGE_THRESHOLDS: 0.7 MASTER_SERVER_LOAD_PROTECTION_MAX_SYSTEM_MEMORY_USAGE_PERCENTAGE_THRESHOLDS: 0.7 MASTER_STATE_WHEEL_INTERVAL: 5s MASTER_TASK_COMMIT_INTERVAL: 1s MASTER_TASK_COMMIT_RETRYTIMES: '5' livenessProbe: enabled: true failureThreshold: '3' initialDelaySeconds: '30' periodSeconds: '30' successThreshold: '1' timeoutSeconds: '5' nodeSelector: {} persistentVolumeClaim: accessModes: - ReadWriteOnce enabled: true storage: 20Gi storageClassName: vsphere-csi-sc podManagementPolicy: Parallel readinessProbe: enabled: true failureThreshold: '3' initialDelaySeconds: '30' periodSeconds: '30' successThreshold: '1' timeoutSeconds: '5' replicas: '3' resources: {} service: annotations: {} serviceMonitor: annotations: {} enabled: false interval: 15s labels: {} path: /actuator/prometheus tolerations: [] updateStrategy: type: RollingUpdate minio: auth: rootPassword: [email protected] rootUser: minioadmin defaultBuckets: dolphinscheduler enabled: false persistence: enabled: false mysql: auth: database: dolphinscheduler params: characterEncoding=utf8 password: ds username: ds driverClassName: com.mysql.cj.jdbc.Driver enabled: false primary: persistence: enabled: false size: 20Gi storageClass: vsphere-csi-sc postgresql: driverClassName: org.postgresql.Driver enabled: true params: characterEncoding=utf8 persistence: enabled: true size: 100Gi storageClass: vsphere-csi-sc postgresqlDatabase: dolphinscheduler postgresqlPassword: [email protected] postgresqlUsername: root registryEtcd: authority: '' enabled: false endpoints: '' namespace: dolphinscheduler passWord: '' ssl: certFile: etcd-certs/ca.crt enabled: false keyCertChainFile: etcd-certs/client.crt keyFile: etcd-certs/client.pem user: '' registryJdbc: enabled: false hikariConfig: driverClassName: com.mysql.cj.jdbc.Driver enabled: false jdbcurl: jdbc:mysql:// password: '' username: '' termExpireTimes: 3 termRefreshInterval: 2s security: authentication: ldap: basedn: dc=example,dc=com password: password ssl: enable: false jksbase64content: '' truststore: /opt/ldapkeystore.jks truststorepassword: '' urls: ldap://ldap.forumsys.com:389/ user: admin: read-only-admin emailattribute: mail identityattribute: uid notexistaction: CREATE username: cn=read-only-admin,dc=example,dc=com type: PASSWORD timezone: Asia/Shanghai worker: affinity: {} annotations: {} customizedConfig: {} enableCustomizedConfig: false enabled: true env: WORKER_EXEC_THREADS: '100' WORKER_HOST_WEIGHT: '100' WORKER_MAX_HEARTBEAT_INTERVAL: 10s WORKER_SERVER_LOAD_PROTECTION_ENABLED: false WORKER_SERVER_LOAD_PROTECTION_MAX_DISK_USAGE_PERCENTAGE_THRESHOLDS: 0.7 WORKER_SERVER_LOAD_PROTECTION_MAX_JVM_CPU_USAGE_PERCENTAGE_THRESHOLDS: 0.7 WORKER_SERVER_LOAD_PROTECTION_MAX_SYSTEM_CPU_USAGE_PERCENTAGE_THRESHOLDS: 0.7 WORKER_SERVER_LOAD_PROTECTION_MAX_SYSTEM_MEMORY_USAGE_PERCENTAGE_THRESHOLDS: 0.7 WORKER_TENANT_CONFIG_AUTO_CREATE_TENANT_ENABLED: true WORKER_TENANT_CONFIG_DEFAULT_TENANT_ENABLED: false keda: advanced: {} cooldownPeriod: 30 enabled: false maxReplicaCount: 3 minReplicaCount: 0 namespaceLabels: {} pollingInterval: 5 livenessProbe: enabled: true failureThreshold: '3' initialDelaySeconds: '30' periodSeconds: '30' successThreshold: '1' timeoutSeconds: '5' nodeSelector: {} persistentVolumeClaim: dataPersistentVolume: accessModes: - ReadWriteOnce enabled: true storage: 20Gi storageClassName: vsphere-csi-sc enabled: false logsPersistentVolume: accessModes: - ReadWriteOnce enabled: true storage: 20Gi storageClassName: vsphere-csi-sc podManagementPolicy: Parallel readinessProbe: enabled: true failureThreshold: '3' initialDelaySeconds: '30' periodSeconds: '30' successThreshold: '1' timeoutSeconds: '5' replicas: '3' resources: {} service: annotations: {} serviceMonitor: annotations: {} enabled: false interval: 15s labels: {} path: /actuator/prometheus tolerations: [] updateStrategy: type: RollingUpdate extraVolumeMounts: - name: worker-cm mountPath: /opt/dolphinscheduler/worker-server/conf/common.properties subPath: common.properties extraVolumes: - name: worker-cm configMap: name: ds-configs zookeeper: enabled: false fourlwCommandsWhitelist: srvr,ruok,wchs,cons persistence: enabled: false size: 20Gi storageClass: vsphere-csi-sc service: port: 2181 global: cattle: systemProjectId: p-np6fz ### Anything else _No response_ ### Version dev ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
