This is an automated email from the ASF dual-hosted git repository.
hez pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-devlake-website.git
The following commit(s) were added to refs/heads/main by this push:
new 66ec39bf8f docs(cicd): update docs (#421)
66ec39bf8f is described below
commit 66ec39bf8f6031284d59318ed32b2be069aabeb3
Author: Warren Chen <[email protected]>
AuthorDate: Wed Feb 8 11:41:07 2023 +0800
docs(cicd): update docs (#421)
* docs(cicd): update docs
* fix for review
---
docs/DataModels/DevLakeDomainLayerSchema.md | 61 +++++++++++++--------
docs/UserManuals/ConfigUI/GitHub.md | 7 ++-
docs/UserManuals/ConfigUI/Jenkins.md | 14 +++--
static/img/ConfigUI/github-action-job.png | Bin 0 -> 132479 bytes
static/img/ConfigUI/github-action-run.png | Bin 0 -> 112303 bytes
static/img/ConfigUI/jenkins-job-build-stage.png | Bin 0 -> 244394 bytes
.../DataModels/DevLakeDomainLayerSchema.md | 57 ++++++++++++-------
.../version-v0.15/UserManuals/ConfigUI/GitHub.md | 6 +-
.../version-v0.15/UserManuals/ConfigUI/Jenkins.md | 15 +++--
9 files changed, 107 insertions(+), 53 deletions(-)
diff --git a/docs/DataModels/DevLakeDomainLayerSchema.md
b/docs/DataModels/DevLakeDomainLayerSchema.md
index 519d54e94e..78c0539ec8 100644
--- a/docs/DataModels/DevLakeDomainLayerSchema.md
+++ b/docs/DataModels/DevLakeDomainLayerSchema.md
@@ -151,14 +151,14 @@ This table shows the work logged under issues. Usually,
an issue has multiple wo
A `board` is an issue list or a collection of issues. It's the abstraction of
a Jira board, a Jira project, a [GitHub issue
list](https://github.com/apache/incubator-devlake/issues) or a GitLab issue
list. This table can be used to filter issues by the boards they belong to.
-| **field** | **type** | **length** | **description**
| **key** |
-| :------------- | :------- | :--------- |
:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| :------ |
-| `id` | varchar | 255 | A board's `id` is composed of "<
plugin >:< Entity >:< PK0 >[:PK1]..." <ul><li>For a Github repo's issue list,
the board id is like "< github >:< GithubRepos >:< GithubRepoId >". Eg.
"github:GithubRepo:384111310"</li> <li>For a Jira Board, the id is like the
board id is like "< jira >:< JiraSourceId >< JiraBoards >:< JiraBoardsId >".
Eg. "jira:1:JiraBoards:12"</li></ul> | PK |
-| `name` | varchar | 255 | The name of the board. Note: the
board name of a Github project 'apache/incubator-devlake' is
'apache/incubator-devlake', representing the [default issue
list](https://github.com/apache/incubator-devlake/issues).
| |
-| `description` | varchar | 255 | The description of the board.
| |
-| `url` | varchar | 255 | The url of the board. Eg.
https://github.com/apache/incubator-devlake
| |
-| `created_date` | datetime | 3 | Board creation time
| |
-| `type` | varchar | 255 | Identify scrum and non-scrum board
| |
+| **field** | **type** | **length** | **description**
| **key** |
+| :------------- | :------- | :---------
|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
:------ |
+| `id` | varchar | 255 | A board's `id` is composed of "<
plugin >:< Entity >:< PK0 >[:PK1]..." <ul><li>For a Github repo's issue list,
the board id is like "< github >:< GithubRepos >:< ConnectionId >:<
GithubRepoId >". Eg. "github:GithubRepo:384111310"</li> <li>For a Jira Board,
the id is like the board id is like "< jira >:< JiraSourceId >< JiraBoards >:<
ConnectionId >:< JiraBoardsId >". Eg. "jira:1:JiraBoards:1:12"</li></ul> | PK
|
+| `name` | varchar | 255 | The name of the board. Note: the
board name of a Github project 'apache/incubator-devlake' is
'apache/incubator-devlake', representing the [default issue
list](https://github.com/apache/incubator-devlake/issues).
| |
+| `description` | varchar | 255 | The description of the board.
| |
+| `url` | varchar | 255 | The url of the board. Eg.
https://github.com/apache/incubator-devlake
| |
+| `created_date` | datetime | 3 | Board creation time
| |
+| `type` | varchar | 255 | Identify scrum and non-scrum board
| |
#### board_issues
@@ -213,18 +213,18 @@ This table shows the relation between sprints and issues
that have been added to
Information about GitHub or Gitlab repositories. A repository is always owned
by a user.
-| **field** | **type** | **length** | **description**
| **key** |
-| :------------- | :------- | :--------- |
:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| :------------- |
-| `id` | varchar | 255 | A repo's `id` is composed of "<
plugin >:< Entity >:< PK0 >[:PK1]..."<br/>For example, a Github repo's id is
like "< github >:< GithubRepos >< GithubRepoId >". Eg.
'github:GithubRepos:384111310' | PK |
-| `name` | varchar | 255 | The name of repo.
| |
-| `description` | varchar | 255 | The description of repo.
| |
-| `url` | varchar | 255 | The url of repo. Eg.
https://github.com/apache/incubator-devlake
| |
-| `owner_id` | varchar | 255 | The id of the owner of repo
| FK_accounts.id |
-| `language` | varchar | 255 | The major language of repo. Eg. The
language for apache/incubator-devlake is 'Go'
| |
-| `forked_from` | varchar | 255 | Empty unless the repo is a fork in
which case it contains the `id` of the repo the repo is forked from.
| |
-| `deleted` | tinyint | 255 | 0: repo is active 1: repo has been
deleted
| |
-| `created_date` | datetime | 3 | Repo creation date
| |
-| `updated_date` | datetime | 3 | Last full update was done for this
repo
| |
+| **field** | **type** | **length** | **description**
| **key** |
+| :------------- | :------- | :---------
|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
:------------- |
+| `id` | varchar | 255 | A repo's `id` is composed of "<
plugin >:< Entity >:< PK0 >[:PK1]..."<br/>For example, a Github repo's id is
like "< github >:< GithubRepos >:< ConnectionId >:< GithubRepoId >". Eg.
'github:GithubRepos:1:384111310' | PK |
+| `name` | varchar | 255 | The name of repo.
| |
+| `description` | varchar | 255 | The description of repo.
| |
+| `url` | varchar | 255 | The url of repo. Eg.
https://github.com/apache/incubator-devlake
| |
+| `owner_id` | varchar | 255 | The id of the owner of repo
| FK_accounts.id |
+| `language` | varchar | 255 | The major language of repo. Eg. The
language for apache/incubator-devlake is 'Go'
| |
+| `forked_from` | varchar | 255 | Empty unless the repo is a fork in
which case it contains the `id` of the repo the repo is forked from.
| |
+| `deleted` | tinyint | 255 | 0: repo is active 1: repo has been
deleted
| |
+| `created_date` | datetime | 3 | Repo creation date
| |
+| `updated_date` | datetime | 3 | Last full update was done for this
repo
| |
#### repo_languages(WIP)
@@ -435,6 +435,23 @@ Events of pull requests.
### Domain 4 - CI/CD(WIP)
+#### cicd_scopes
+
+Information about Jenkins Job, GitHub Action or Gitlab CI.
+- For GitHub: a GitHub repo is converted to a cicd_scope
+- For Jenkins: a GitLab project is converted to a cicd_scope
+- For GitLab: a Jenkins job is converted to a cicd_scope
+
+| **field** | **type** | **length** | **description**
| **key** |
+| :------------- | :------- | :---------
|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
:------------- |
+| `id` | varchar | 255 | A cicd_scope's `id` is composed of
"< plugin >:< Entity >:< PK0 >[:PK1]..."<br/>For example, a Github cicd_scope's
id is like "< github >:< GithubRepos >:< ConnectionId >:< GithubRepoId >". Eg.
'github:GithubRepos:1:384111310' | PK |
+| `name` | varchar | 255 | The name of cicd_scope.
| |
+| `description` | varchar | 255 | The description of cicd_scope.
| |
+| `url` | varchar | 255 | The url of cicd_scope. Eg.
https://github.com/apache/incubator-devlake or
https://jenkins.xxx.cn/view/PROD/job/OPS_releasev2/
|
|
+| `created_date` | datetime | 3 | cicd_scope creation date
| |
+| `updated_date` | datetime | 3 | Date of the last data collection
for this cicd_scope
| |
+
+
#### cicd_pipelines
A cicd_pipeline is a series of builds that have connections or a standalone
build.
@@ -481,7 +498,7 @@ A cicd_task is a single job of ci/cd.
### Project Metric Entities
-#### project_pr_metrics
+#### project_pr_metrics
| **field** | **type** | **length** | **description**
| **key** |
| :-------- | :--------
|:-----------|:---------------------------------------------------------------------------------------|
:-------- |
@@ -649,7 +666,7 @@ import
"github.com/apache/incubator-devlake/models/domainlayer/domaininfo"
domaininfo := domaininfo.GetDomainTablesInfo()
for _, table := range domaininfo {
- // do something
+// do something
}
```
diff --git a/docs/UserManuals/ConfigUI/GitHub.md
b/docs/UserManuals/ConfigUI/GitHub.md
index 4cb53afc1d..040ed52d49 100644
--- a/docs/UserManuals/ConfigUI/GitHub.md
+++ b/docs/UserManuals/ConfigUI/GitHub.md
@@ -120,7 +120,12 @@ If you're using GitHub Action to conduct `deployments`,
please select "Detect De
- Deployment: A GitHub Action job with a name that matches the given regEx
will be considered as a deployment.
- Production: A GitHub Action job with a name that matches the given regEx
will be considered a job in the production environment.
-By the above two fields, DevLake can identify a production deployment among
massive CI jobs.
+A GitHub workflow run has many jobs. Each GitHub workflow run is converted to
a
+cicd_pipeline in the domain layer and each GitHub Action job is converted to a
cicd_task in the domain layer.
+
+
+
+The deployment and production regex is always applied to the records in the
cicd_tasks table.
You can also select "Not using Jobs in GitHub Action as Deployments" if you're
not using GitHub action to conduct deployments.
diff --git a/docs/UserManuals/ConfigUI/Jenkins.md
b/docs/UserManuals/ConfigUI/Jenkins.md
index 41ba2d6e51..999a9e04c9 100644
--- a/docs/UserManuals/ConfigUI/Jenkins.md
+++ b/docs/UserManuals/ConfigUI/Jenkins.md
@@ -54,12 +54,18 @@ Jenkins only supports `CI/CD` domain entities, transformed
from Jenkins builds a
This set of configurations is used for calculating [DORA metrics](../DORA.md).
-If you're using Jenkins builds to conduct `deployments`, please select "Detect
Deployment from Jenkins Builds", and input the RegEx in the following fields:
+If you'd like to define `deployments` with Jenkins, please select "Detect
Deployment from Jenkins Builds", and provide the following regexes
-- Deployment: A Jenkins build with a name that matches the given regEx will be
considered as a deployment.
-- Production: A Jenkins build with a name that matches the given regEx will be
considered a build in the production environment.
+- Deployment: Jenkins stages whose names match the regex will be registered as
deployments (if a Jenkins build has no stage, its job name will be used to
match the regex)
+- Production: Jenkins stages whose names match the regex will be assigned
environment 'PRODUCTION' (if a Jenkins build has no stage, its job name will be
used to match the regex)
-By the above two fields, DevLake can identify a production deployment among
massive CI jobs.
+This is how it works behind the scene:
+
+- If a Jenkins build has stages, it's converted to a cicd_pipeline and its
stages are converted to cicd_tasks in DevLake's domain layer schema.
+- If a Jenkins build has no stage, it's converted to both a cicd_pipeline and
a cicd_task whose names are the build's jobName.
+
+After the conversion, the two regexes are applied to the records in the
cicd_tasks table.
+
You can also select "Not using Jenkins builds as Deployments" if you're not
using Jenkins to conduct deployments.
diff --git a/static/img/ConfigUI/github-action-job.png
b/static/img/ConfigUI/github-action-job.png
new file mode 100644
index 0000000000..de6847e39f
Binary files /dev/null and b/static/img/ConfigUI/github-action-job.png differ
diff --git a/static/img/ConfigUI/github-action-run.png
b/static/img/ConfigUI/github-action-run.png
new file mode 100644
index 0000000000..a7ee280c1d
Binary files /dev/null and b/static/img/ConfigUI/github-action-run.png differ
diff --git a/static/img/ConfigUI/jenkins-job-build-stage.png
b/static/img/ConfigUI/jenkins-job-build-stage.png
new file mode 100644
index 0000000000..0f5879b277
Binary files /dev/null and b/static/img/ConfigUI/jenkins-job-build-stage.png
differ
diff --git
a/versioned_docs/version-v0.15/DataModels/DevLakeDomainLayerSchema.md
b/versioned_docs/version-v0.15/DataModels/DevLakeDomainLayerSchema.md
index 519d54e94e..5450c74868 100644
--- a/versioned_docs/version-v0.15/DataModels/DevLakeDomainLayerSchema.md
+++ b/versioned_docs/version-v0.15/DataModels/DevLakeDomainLayerSchema.md
@@ -151,14 +151,14 @@ This table shows the work logged under issues. Usually,
an issue has multiple wo
A `board` is an issue list or a collection of issues. It's the abstraction of
a Jira board, a Jira project, a [GitHub issue
list](https://github.com/apache/incubator-devlake/issues) or a GitLab issue
list. This table can be used to filter issues by the boards they belong to.
-| **field** | **type** | **length** | **description**
| **key** |
-| :------------- | :------- | :--------- |
:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| :------ |
-| `id` | varchar | 255 | A board's `id` is composed of "<
plugin >:< Entity >:< PK0 >[:PK1]..." <ul><li>For a Github repo's issue list,
the board id is like "< github >:< GithubRepos >:< GithubRepoId >". Eg.
"github:GithubRepo:384111310"</li> <li>For a Jira Board, the id is like the
board id is like "< jira >:< JiraSourceId >< JiraBoards >:< JiraBoardsId >".
Eg. "jira:1:JiraBoards:12"</li></ul> | PK |
-| `name` | varchar | 255 | The name of the board. Note: the
board name of a Github project 'apache/incubator-devlake' is
'apache/incubator-devlake', representing the [default issue
list](https://github.com/apache/incubator-devlake/issues).
| |
-| `description` | varchar | 255 | The description of the board.
| |
-| `url` | varchar | 255 | The url of the board. Eg.
https://github.com/apache/incubator-devlake
| |
-| `created_date` | datetime | 3 | Board creation time
| |
-| `type` | varchar | 255 | Identify scrum and non-scrum board
| |
+| **field** | **type** | **length** | **description**
| **key** |
+| :------------- | :------- | :---------
|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
:------ |
+| `id` | varchar | 255 | A board's `id` is composed of "<
plugin >:< Entity >:< PK0 >[:PK1]..." <ul><li>For a Github repo's issue list,
the board id is like "< github >:< GithubRepos >:< ConnectionId >:<
GithubRepoId >". Eg. "github:GithubRepo:384111310"</li> <li>For a Jira Board,
the id is like the board id is like "< jira >:< JiraSourceId >< JiraBoards >:<
ConnectionId >:< JiraBoardsId >". Eg. "jira:1:JiraBoards:1:12"</li></ul> | PK
|
+| `name` | varchar | 255 | The name of the board. Note: the
board name of a Github project 'apache/incubator-devlake' is
'apache/incubator-devlake', representing the [default issue
list](https://github.com/apache/incubator-devlake/issues).
| |
+| `description` | varchar | 255 | The description of the board.
| |
+| `url` | varchar | 255 | The url of the board. Eg.
https://github.com/apache/incubator-devlake
| |
+| `created_date` | datetime | 3 | Board creation time
| |
+| `type` | varchar | 255 | Identify scrum and non-scrum board
| |
#### board_issues
@@ -213,18 +213,18 @@ This table shows the relation between sprints and issues
that have been added to
Information about GitHub or Gitlab repositories. A repository is always owned
by a user.
-| **field** | **type** | **length** | **description**
| **key** |
-| :------------- | :------- | :--------- |
:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| :------------- |
-| `id` | varchar | 255 | A repo's `id` is composed of "<
plugin >:< Entity >:< PK0 >[:PK1]..."<br/>For example, a Github repo's id is
like "< github >:< GithubRepos >< GithubRepoId >". Eg.
'github:GithubRepos:384111310' | PK |
-| `name` | varchar | 255 | The name of repo.
| |
-| `description` | varchar | 255 | The description of repo.
| |
-| `url` | varchar | 255 | The url of repo. Eg.
https://github.com/apache/incubator-devlake
| |
-| `owner_id` | varchar | 255 | The id of the owner of repo
| FK_accounts.id |
-| `language` | varchar | 255 | The major language of repo. Eg. The
language for apache/incubator-devlake is 'Go'
| |
-| `forked_from` | varchar | 255 | Empty unless the repo is a fork in
which case it contains the `id` of the repo the repo is forked from.
| |
-| `deleted` | tinyint | 255 | 0: repo is active 1: repo has been
deleted
| |
-| `created_date` | datetime | 3 | Repo creation date
| |
-| `updated_date` | datetime | 3 | Last full update was done for this
repo
| |
+| **field** | **type** | **length** | **description**
| **key** |
+| :------------- | :------- | :---------
|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
:------------- |
+| `id` | varchar | 255 | A repo's `id` is composed of "<
plugin >:< Entity >:< PK0 >[:PK1]..."<br/>For example, a Github repo's id is
like "< github >:< GithubRepos >:< ConnectionId >:< GithubRepoId >". Eg.
'github:GithubRepos:1:384111310' | PK |
+| `name` | varchar | 255 | The name of repo.
| |
+| `description` | varchar | 255 | The description of repo.
| |
+| `url` | varchar | 255 | The url of repo. Eg.
https://github.com/apache/incubator-devlake
| |
+| `owner_id` | varchar | 255 | The id of the owner of repo
| FK_accounts.id |
+| `language` | varchar | 255 | The major language of repo. Eg. The
language for apache/incubator-devlake is 'Go'
| |
+| `forked_from` | varchar | 255 | Empty unless the repo is a fork in
which case it contains the `id` of the repo the repo is forked from.
| |
+| `deleted` | tinyint | 255 | 0: repo is active 1: repo has been
deleted
| |
+| `created_date` | datetime | 3 | Repo creation date
| |
+| `updated_date` | datetime | 3 | Last full update was done for this
repo
| |
#### repo_languages(WIP)
@@ -435,6 +435,23 @@ Events of pull requests.
### Domain 4 - CI/CD(WIP)
+#### cicd_scopes
+
+Information about Jenkins Job, GitHub Action or Gitlab CI.
+- For GitHub: a GitHub repo is converted to a cicd_scope
+- For Jenkins: a GitLab project is converted to a cicd_scope
+- For GitLab: a Jenkins job is converted to a cicd_scope
+
+| **field** | **type** | **length** | **description**
| **key** |
+| :------------- | :------- | :---------
|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
:------------- |
+| `id` | varchar | 255 | A cicd_scope's `id` is composed of
"< plugin >:< Entity >:< PK0 >[:PK1]..."<br/>For example, a Github cicd_scope's
id is like "< github >:< GithubRepos >:< ConnectionId >:< GithubRepoId >". Eg.
'github:GithubRepos:1:384111310' | PK |
+| `name` | varchar | 255 | The name of cicd_scope.
| |
+| `description` | varchar | 255 | The description of cicd_scope.
| |
+| `url` | varchar | 255 | The url of cicd_scope. Eg.
https://github.com/apache/incubator-devlake or
https://jenkins.xxx.cn/view/PROD/job/OPS_releasev2/
|
|
+| `created_date` | datetime | 3 | cicd_scope creation date
| |
+| `updated_date` | datetime | 3 | Date of the last data collection
for this cicd_scope
| |
+
+
#### cicd_pipelines
A cicd_pipeline is a series of builds that have connections or a standalone
build.
diff --git a/versioned_docs/version-v0.15/UserManuals/ConfigUI/GitHub.md
b/versioned_docs/version-v0.15/UserManuals/ConfigUI/GitHub.md
index 4cb53afc1d..c80152260f 100644
--- a/versioned_docs/version-v0.15/UserManuals/ConfigUI/GitHub.md
+++ b/versioned_docs/version-v0.15/UserManuals/ConfigUI/GitHub.md
@@ -120,7 +120,11 @@ If you're using GitHub Action to conduct `deployments`,
please select "Detect De
- Deployment: A GitHub Action job with a name that matches the given regEx
will be considered as a deployment.
- Production: A GitHub Action job with a name that matches the given regEx
will be considered a job in the production environment.
-By the above two fields, DevLake can identify a production deployment among
massive CI jobs.
+A GitHub workflow run has many jobs. Each GitHub workflow run is converted to
a cicd_pipeline in the domain layer and each GitHub Action job is converted to
a cicd_task in the domain layer.
+
+
+
+The deployment and production regex is always applied to the records in the
cicd_tasks table.
You can also select "Not using Jobs in GitHub Action as Deployments" if you're
not using GitHub action to conduct deployments.
diff --git a/versioned_docs/version-v0.15/UserManuals/ConfigUI/Jenkins.md
b/versioned_docs/version-v0.15/UserManuals/ConfigUI/Jenkins.md
index 41ba2d6e51..260348e53e 100644
--- a/versioned_docs/version-v0.15/UserManuals/ConfigUI/Jenkins.md
+++ b/versioned_docs/version-v0.15/UserManuals/ConfigUI/Jenkins.md
@@ -51,15 +51,20 @@ Jenkins only supports `CI/CD` domain entities, transformed
from Jenkins builds a
- CI/CD: Jenkins builds, stages, etc.
### Step 3 - Adding Transformation Rules (Optional)
-
This set of configurations is used for calculating [DORA metrics](../DORA.md).
-If you're using Jenkins builds to conduct `deployments`, please select "Detect
Deployment from Jenkins Builds", and input the RegEx in the following fields:
+If you'd like to define `deployments` with Jenkins, please select "Detect
Deployment from Jenkins Builds", and provide the following regexes
+
+- Deployment: Jenkins stages whose names match the regex will be registered as
deployments (if a Jenkins build has no stage, its job name will be used to
match the regex)
+- Production: Jenkins stages whose names match the regex will be assigned
environment 'PRODUCTION' (if a Jenkins build has no stage, its job name will be
used to match the regex)
+
+This is how it works behind the scene:
-- Deployment: A Jenkins build with a name that matches the given regEx will be
considered as a deployment.
-- Production: A Jenkins build with a name that matches the given regEx will be
considered a build in the production environment.
+- If a Jenkins build has stages, it's converted to a cicd_pipeline and its
stages are converted to cicd_tasks in DevLake's domain layer schema.
+- If a Jenkins build has no stage, it's converted to both a cicd_pipeline and
a cicd_task whose names are the build's jobName.
-By the above two fields, DevLake can identify a production deployment among
massive CI jobs.
+After the conversion, the two regexes are applied to the records in the
cicd_tasks table.
+
You can also select "Not using Jenkins builds as Deployments" if you're not
using Jenkins to conduct deployments.
