This is an automated email from the ASF dual-hosted git repository.
dhanak pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-kie-kogito-docs.git
The following commit(s) were added to refs/heads/main by this push:
new fe738703f kie-kogito-docs-716: [sonataflow-operator] Improvements on
the DB migration documentation section (#717)
fe738703f is described below
commit fe738703f0314609b76d6c2938178d949e8c3013
Author: Walter Medvedeo <[email protected]>
AuthorDate: Wed Apr 30 07:43:46 2025 +0200
kie-kogito-docs-716: [sonataflow-operator] Improvements on the DB migration
documentation section (#717)
* kie-kogito-docs-716: [sonataflow-operator] Improvements on the DB
migration documentation section
* Apply suggestions from code review
Co-authored-by: Ricardo Zanini
<[email protected]>
---------
Co-authored-by: Ricardo Zanini
<[email protected]>
---
.../pages/cloud/operator/supporting-services.adoc | 162 +++++++++------------
1 file changed, 71 insertions(+), 91 deletions(-)
diff --git
a/serverlessworkflow/modules/ROOT/pages/cloud/operator/supporting-services.adoc
b/serverlessworkflow/modules/ROOT/pages/cloud/operator/supporting-services.adoc
index 2f7510dab..1260bdf17 100644
---
a/serverlessworkflow/modules/ROOT/pages/cloud/operator/supporting-services.adoc
+++
b/serverlessworkflow/modules/ROOT/pages/cloud/operator/supporting-services.adoc
@@ -98,32 +98,44 @@ And thus, it's only recommended for development and testing
purposes.
The ephemeral deployment of a service requires no additional configurations
than the shown, <<supporting-services-configuration, here>>.
-[#postgresql-persistence-configuration]
-=== PostgreSQL persistence configuration
+[#database-migration-configuration]
+=== Database migration configuration
-The PostgreSQL persistence of a service is supported by a PostgreSQL server
instance that you must previously install on the cluster.
-The administration of that instance is totally independent of the
{operator_name} scope, and to connect a supporting service with it, you must
only configure the correct database connection parameters.
+The term database migration refers to either initializing a given Data Index
or Jobs Service database to the respective schema, or applying data or schema
updates as new versions are released.
+
+The database migration strategy must be configured individually for each
supporting service, by using the optional
`dataIndex.persistence.dbMigrationStrategy` and
`jobService.persistence.dbMigrationStrategy` fields respectively.
-==== {product_name} Operator Database Migrator capabilities
-The {product_name} Operator offers out of the box database migration features,
which help in configuring the databases for use by Data Index and/or Jobs
Service.
+You can use any of the following database migration strategies: `job`,
`service`, and `none`. When not configured, the default value is `service`.
+
+[NOTE]
+====
+Currently, database migration is only supported when using the PostgreSQL
persistence configuration.
+====
-The term database migration refers to either initializing a given Data Index
or Jobs Service database to the schema needed by them or applying data or
schema updates as new versions are released.
+==== Job based database migration strategy
-{product_name} Operator offers jobs based database migration capabilities
along with two other modes: service based database migration and none.
+When you configure the `job` based strategy, the {product_name} Operator
decouples the database migration by using a dedicated Kubernetes Job that
manages the entire migration process.
-The {product_name} Operator offered database migration modes are described as
follows.
+That Job executes before the supporting services deployment, and thus, ensures
that a given service is started only if the corresponding migration was
successful.
-.Jobs based database migration
-In this mode, the {product_name} Operator creates a Kubernetes Job and uses
the database migrator component to migrate configured databases. You would
typically use this mode when running in non-development scenarios such as
Production environments, where there could be an existing database for use by
Data Index and/or Jobs Service.
+You would typically use this strategy in production environments.
-.Service based database migration
-In this mode, the {product_name} Operator does not create a Kubernetes Job and
hence does not use the database migrator component to migrate configured
databases. In this mode, thus the Data Index or Jobs Service are responsible to
migrate the configured databases. You would typically use this mode when
running in development scenarios such as local environments, where you are
testing scenarios with clean state everytime you deploy and run.
+==== Service based database migration strategy
-.None mode
-In this mode, the {product_name} Operator makes no migration attempt.
+When you configure the `service` based strategy, the database migration is
managed by each supporting service and is executed as part of the service's
startup sequence.
+In worst-case scenarios, a service could start with failures.
+==== None migration strategy
-=== Overview of PostgreSQL persistence configuration
+When you configure the `none` strategy, no Operator nor service migration
attempt is executed.
+
+You would typically use this strategy in environments where a database
administrator executes all the database migrations.
+
+[#postgresql-persistence-configuration]
+=== PostgreSQL persistence configuration
+
+The PostgreSQL persistence of a service is supported by a PostgreSQL server
instance that you must have previously installed on the cluster.
+The administration of that instance is totally independent of the
{operator_name} scope, and to connect a supporting service with it, you must
only configure the correct database connection parameters.
The following `SonataFlowPlatform` CR fragment shows the configuration options
that you must use:
@@ -140,7 +152,6 @@ spec:
dataIndex:
enabled: true
persistence:
- # job=job based db migration, service=service based db migration,
none=no db migration
dbMigrationStrategy: job <1>
postgresql:
serviceRef:
@@ -156,11 +167,12 @@ spec:
jobService:
enabled: true
persistence:
+ dbMigrationStrategy: job
postgresql:
# Specific database configuration for the Job Service
# might be included here.
----
-<1> (Optional) Database migration approach to be used by {product_name}
Operator. Defaults to a value of `service`.
+<1> (Optional) Database migration strategy to use,
<<database-migration-configuration, see>>. Defaults to `service`.
<2> Name of the Kubernetes Service to connect with the PostgreSQL database
server.
<3> (Optional) Kubernetes namespace containing the PostgreSQL Service.
Defaults to the `SonataFlowPlatform's` local namespace.
<4> Name of the PostgreSQL database to store the supporting service data.
@@ -184,97 +196,65 @@ To create the secrets for the example above you can use a
command like this:
kubectl create secret generic postgres-secrets-example
--from-literal=POSTGRESQL_USER=<user>
--from-literal=POSTGRESQL_PASSWORD=<password> -n postgres-example-namespace
----
-[#common-persistence-scenarios]
-=== Persistence examples with most common scenarios
-==== Specify database information at persistence level instead of service level
-You will use SonataFlowPlatform CR's `.spec.persistence.postgresql` to specify
the database to be used by the Data Index and Jobs Service. Then, specify the
database migration mode to use under
`.spec.services.dataIndex.persistence.dbMigrationStrategy` or
`.spec.services.jobService.persistence.dbMigrationStrategy` with a value such
as job, service, or none.
-In this mode the Data Index and Jobs Service will make use of one common
database.
+[#common-persistence-configuration]
+=== Platform-scoped PostgreSQL persistence configuration
+
+To configure a common PostgreSQL service and database for the supporting
services, you must use the field `spec.persistence.postgresql` in the
`SonataFlowPlatform` CR.
+
+This information signals the {operator_name} to configure the supporting
services to connect to that database.
+
+Additionally, workflows deployed in that namespace, with the `preview` or
`gitops` profile, that don’t provide a custom persistence configuration, will
connect to that database too,
xref:cloud/operator/using-persistence.adoc#configuring-persistence-using-the-sonataflowplatform-cr[see].
+
+And, similarly to the workflow's persistence, the following rules apply:
+
+* If a supporting service has a configured persistence, for example, the field
`services.dataIndex.persistence.postgresql` is configured. That configuration
will apply.
+
+* If a supporting service has no configured persistence, for example, if the
`services.dataIndex.persistence.postgresql` field is not set, the persistence
configuration will be taken from the current platform.
+
+* Additionally, if a supporting service requires a particular database
migration strategy, you must use the respective
`dataIndex.persistence.dbMigrationStrategy` and
`jobService.persistence.dbMigrationStrategy` fields,
<<#database-migration-configuration, see>>.
-[source, yaml]
+The following `SonataFlowPlatform` CR fragment shows the configuration options
that you must use:
+
+.SonataFlowPlatform CR persistence configuration example
+[source,yaml]
----
apiVersion: sonataflow.org/v1alpha08
kind: SonataFlowPlatform
metadata:
- name: sonataflow-platform
+ name: sonataflow-platform-example
+ namespace: example-namespace
spec:
- build:
- config:
- strategyOptions:
- KanikoBuildCacheEnabled: "true"
persistence:
postgresql:
- secretRef:
- name: postgres-secrets
- userKey: POSTGRES_USER
- passwordKey: POSTGRES_PASSWORD
serviceRef:
- name: postgres
- databaseName: sonataflow
- services:
- dataIndex:
- enabled: true
- persistence:
- # job=job based db migration, service=service based db migration,
none=no db migration
- dbMigrationStrategy: job
- jobService:
- enabled: true
- persistence:
- # job=job based db migration, service=service based db migration,
none=no db migration
- dbMigrationStrategy: job
-----
-
-==== Specify database information at service level
-You will use SonataFlowPlatform CR's `.spec.services.dataIndex` or
`.spec.services.jobService.persistence` to specify database information as
shown in the example. As earlier, the `dbMigrationStrategy` can be used with a
value such as job, service, or none.
-
-[source, yaml]
-----
-apiVersion: sonataflow.org/v1alpha08
-kind: SonataFlowPlatform
-metadata:
- name: sonataflow-platform
-spec:
- build:
- config:
- strategyOptions:
- KanikoBuildCacheEnabled: "true"
- services:
+ name: postgres-example <1>
+ namespace: postgres-example-namespace <2>
+ databaseName: example-database <3>
+ port: 1234 <4>
+ secretRef:
+ name: postgres-secrets-example <5>
+ userKey: POSTGRESQL_USER <6>
+ passwordKey: POSTGRESQL_PASSWORD <7>
dataIndex:
enabled: true
persistence:
- # job=job based db migration, service=service based db migration,
none=no db migration
- dbMigrationStrategy: job
- postgresql:
- jdbcUrl:
jdbc:postgresql://postgres:5432/sonataflow?currentSchema=data-index-service
- secretRef:
- name: postgres-secrets
- userKey: POSTGRES_USER
- passwordKey: POSTGRES_PASSWORD
+ dbMigrationStrategy: job <8>
jobService:
enabled: true
persistence:
- # job=job based db migration, service=service based db migration,
none=no db migration
- dbMigrationStrategy: job
- postgresql:
- jdbcUrl:
jdbc:postgresql://postgres:5432/sonataflow?currentSchema=jobs-service
- secretRef:
- name: postgres-secrets
- userKey: POSTGRES_USER
- passwordKey: POSTGRES_PASSWORD
+ dbMigrationStrategy: service <9>
----
-
-[#common-persistence-configuration]
-=== Common PostgreSQL persistence configuration
-
-To configure a common PostgreSQL service instance for all the supporting
services you must read,
xref:cloud/operator/using-persistence.adoc#configuring-persistence-using-the-sonataflowplatform-cr[Configuring
the persistence using the SonataFlowPlatform CR].
-
-In that case, the {operator_name} will automatically connect any of the
supporting services with that common server configured in the field
`spec.persistence`. And, similarly to the workflow's persistence, the following
precedence rules apply:
-
-* If a supporting service has a configured persistence, for example, the field
`services.dataIndex.persistence` is configured. That configuration will apply.
-
-* If a supporting service has no configured persistence, for example, the
field `services.dataIndex.persistence` is not set at all, the persistence
configuration will be taken from the current platform.
-
+<1> Name of the Kubernetes Service to connect with the PostgreSQL database
server.
+<2> (Optional) Kubernetes namespace containing the PostgreSQL Service.
Defaults to the `SonataFlowPlatform's` local namespace.
+<3> Name of the PostgreSQL database to store the supporting services and
workflows data.
+<4> (Optional) Port number to connect with the PostgreSQL Service. Defaults to
5432.
+<5> Name of the link:{k8n_secrets_url}[Kubernetes Secret] containing the
username and password to connect with the database.
+<6> Name of the link:{k8n_secrets_url}[Kubernetes Secret] `key` containing the
username to connect with the database.
+<7> Name of the link:{k8n_secrets_url}[Kubernetes Secret] `key` containing the
password to connect with the database.
+<8> (Optional) Database migration strategy to use for the Data Index,
<<database-migration-configuration, see>>. Defaults to `service`.
+<9> (Optional) Database migration strategy to use for the Job Service Index,
<<database-migration-configuration, see>>. Defaults to `service`. While no
usual, the supporting services could use distinct database migration strategies.
[NOTE]
====
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
