This is an automated email from the ASF dual-hosted git repository.
klesh pushed a change to branch kw-5519-remoteapi-dshelper
in repository https://gitbox.apache.org/repos/asf/incubator-devlake.git
from df4a07510 refactor: wip Dynamic Model
add 6f57a15a9 fix: git clone failing on azuredevops (#7356)
add d571e97c7 Support human readable project name (#7357)
add 2c5a08d38 fix: CLT ignores the first deployment (#7362)
add c7b6336ba fix: plugin github config error (#7365)
add ca89492a9 feat: add remote repository support for azure devops (#7364)
add d89234cb2 feat: onboard support dora (#7372)
add 2f3b5367b fix: add jira jql checker debug log (#7376)
add 95100c518 feat: enable dora when onboarding progress (#7373)
add 8f8d637ce feat: add quick link to connection in project select
connectons (#7374)
add c1eebe8f0 Reference zentao instead of gitlab (#7382)
add 2814cf1df fix: should not throwing error on empty repo (#7383)
add afe49e109 feat: subtask state manager (#7384)
add 696ed6e8c feat: jira issue and changelog extractors support
incremental mode (#7387)
add 6ab1d296c fix: customize plugin can not update old data (#7389)
add f51e718f1 fix: expand click area for add new connection (#7395)
add cd081d979 fix: something for onboard (#7397)
add c48e27892 fix: reduce the entities collected by default (#7366)
add 9542510e3 fix: make FinishTime optional for unfinished builds (#7401)
add 804eb2058 docs(main README): Fixes broken links for getting started on
development. Rearranges existing getting started section to reduce duplication
and make it logical for new users to get started. (#7305)
add ea2cdbcc5 fix: compatible with unset configurations (#7402)
add f1ead30cc fix(zentao): update storyID field in tasks reponse, make it
compatiable with oss 18.11 (#7403)
add ebbf84073 fix: unnecessary encode name for project (#7405)
add d7cb635c6 feat: jira issue / changelog converter supports incremental
mode (#7394)
add c890de735 feat: simplify scope config configuration (#7408)
add c3a5b740e feat: support document title (#7410)
add cb146f8cc fix: allow selecting FreeStyleProject in jenkins scopes UI
(#7411)
add 24abaeffd feat: support get related projects by scope config (#7414)
add d0c0842c3 fix: onboard cannot exit (#7416)
add c667a5ad4 fix: unable to create scope config for circleci (#7417)
add ee7a29307 fix: bitbucket server search not working (#7418)
add 8a6bbe2b8 adding description and ranges for the dora dashboard (#7378)
add 2164b9d7b feat: add some tips when updating scope config (#7422)
add 217a36ba9 fix: noShallowClone was overwritten (#7430)
add da9930e05 feat: get projects by scope config feature add scope name
(#7431)
add 9edd876e0 fix cors (#7433)
add bd670d705 fix: when DORA's project is all or multiple ,the drill-down
is incorrect (#7432)
add a1abf4bf3 fix: gitextractor fail to clone (#7434)
add 6f3fe00a1 fix: file_path column too long (#7437)
add 6d5a387cb fix: unable to get project detail when name ends with /
(#7438)
add 7da743a02 feat: add duplicate operate for scope config (#7439)
add 1b1291f61 fix: jenkins plugin some scope data was not returned (#7441)
add 653456fee fix: some bugs for scope config (#7443)
add d2319a4fb fix(fix boolean query that support both postgres and mysql
in get subtask info): boolean data type on column must compared with boolean
values too, not with integer for consistency (#7442)
add 4b89aa064 fix: jenkins jobs are mislabeled as production deployments
when they are skipped. (#7444)
add 145bba092 fix: nginx api proxy path should not be decoded (#7445)
add bee0bce1b fix: typo error in connection detail (#7428)
add a61f43e0f feat: add new component icon-button (#7429)
add cd5777a22 feat: add blueprintId for projects by scope config (#7447)
add 3973c95ed feat: optional security enhancement (#7451)
add 2cc08c7ec fix: some bugs for scope config (#7452)
add 5aac71130 fix: error condition for not related project data scope
(#7454)
add d7807f9e6 fix: error id for scope config changed (#7456)
add 0d032b3d6 fix: incorrect LOC of the first commit in a given time range
(#7458)
add 26a0c8367 fix: post project with blueprint (#7449)
add 1b8a1877d fix: missing required value for field display_title (#7460)
add 44ccba374 fix: optimize error messages when there are duplicate scope
configurations. (#7461)
add f7f808742 fix: remove auto create blueprint when creating project
(#7462)
new b2b13ca90 Merge remote-tracking branch 'origin/main' into
kw-5519-remoteapi-dshelper
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
README.md | 43 +++-
backend/Makefile | 2 +-
backend/core/config/config_viper.go | 1 +
.../common/{string_float64.go => string_int64.go} | 43 ++--
...s_to_plan.go => 20240424_add_subtask_states.go} | 55 ++--
backend/core/models/migrationscripts/register.go | 1 +
backend/core/models/project.go | 15 ++
backend/core/models/subtask_state.go | 40 +++
backend/core/utils/json.go | 9 +
backend/helpers/e2ehelper/data_flow_tester.go | 5 +-
backend/helpers/pluginhelper/api/api_client.go | 66 ++++-
.../helpers/pluginhelper/api/api_client_test.go | 65 +++++
backend/helpers/pluginhelper/api/api_extractor.go | 2 +-
...{api_extractor.go => api_extractor_stateful.go} | 94 ++++---
.../helpers/pluginhelper/api/batch_save_divider.go | 42 +--
.../pluginhelper/api/data_convertor_stateful.go | 129 ++++++++++
backend/helpers/pluginhelper/api/ds_helper.go | 2 +-
.../pluginhelper/api/ds_scope_config_api_helper.go | 11 +
.../helpers/pluginhelper/api/graphql_collector.go | 2 +-
backend/helpers/pluginhelper/api/pipeline_plan.go | 24 ++
.../pluginhelper/api/subtask_state_manager.go | 155 +++++++++++
...nager_test.go => subtask_state_manager_test.go} | 91 ++++---
.../helpers/srvhelper/any_model_service_helper.go | 2 +-
.../srvhelper/any_scope_config_service_helper.go | 72 ++++++
backend/impls/context/default_subtask_context.go | 6 +-
backend/impls/dalgorm/dalgorm.go | 14 +
.../azuredevops_go/api/azuredevops/client.go | 282 +++++++++++++++++++++
.../client_test.go} | 6 +-
.../azuredevops_go/api/azuredevops/models.go | 132 ++++++++++
.../api/{ => azuredevops}/testdata/test.txt | 0
.../plugins/azuredevops_go/api/blueprint_v200.go | 62 ++++-
.../azuredevops_go/api/blueprint_v200_test.go | 75 +++++-
.../plugins/azuredevops_go/api/connection_api.go | 9 +-
backend/plugins/azuredevops_go/api/remote_data.go | 65 -----
.../plugins/azuredevops_go/api/remote_helper.go | 223 +++++++++++-----
.../plugins/azuredevops_go/api/scope_config_api.go | 14 +
backend/plugins/azuredevops_go/api/vs_client.go | 137 ----------
backend/plugins/azuredevops_go/e2e/build_test.go | 1 +
.../raw_tables/_raw_azuredevops_go_api_builds.csv | 5 +-
.../_raw_azuredevops_go_api_timeline_records.csv | 6 +-
.../_tool_azuredevops_go_builds.csv | 5 +-
.../_tool_azuredevops_go_timeline_records.csv | 4 +
.../e2e/snapshot_tables/cicd_pipeline_commits.csv | 5 +-
.../e2e/snapshot_tables/cicd_pipelines.csv | 5 +-
.../e2e/snapshot_tables/cicd_tasks.csv | 4 +
backend/plugins/azuredevops_go/impl/impl.go | 6 +-
backend/plugins/azuredevops_go/models/base.go | 5 +
backend/plugins/azuredevops_go/models/build.go | 16 +-
.../20240413_add_remote_repo_support.go} | 47 ++--
.../models/migrationscripts/register.go | 1 +
backend/plugins/azuredevops_go/models/repo.go | 13 +-
.../azuredevops_go/tasks/ci_cd_build_collector.go | 12 +-
.../azuredevops_go/tasks/ci_cd_build_converter.go | 7 +-
.../azuredevops_go/tasks/ci_cd_build_extractor.go | 6 +-
backend/plugins/azuredevops_go/tasks/task_data.go | 4 +-
backend/plugins/bamboo/api/scope_config_api.go | 14 +
backend/plugins/bamboo/impl/impl.go | 3 +
backend/plugins/bitbucket/api/scope_config_api.go | 14 +
backend/plugins/bitbucket/impl/impl.go | 3 +
backend/plugins/bitbucket_server/api/remote_api.go | 29 +--
.../bitbucket_server/api/scope_config_api.go | 14 +
backend/plugins/bitbucket_server/impl/impl.go | 3 +
backend/plugins/circleci/api/connection_api.go | 6 +-
backend/plugins/circleci/api/scope_config_api.go | 14 +
backend/plugins/circleci/impl/impl.go | 5 +-
.../dora/e2e/change_lead_time/commits_diffs.csv | 13 +-
.../e2e/change_lead_time/project_pr_metrics.csv | 1 +
.../e2e/change_lead_time/pull_request_comments.csv | 27 +-
.../e2e/change_lead_time/pull_request_commits.csv | 27 +-
.../dora/e2e/change_lead_time/pull_requests.csv | 15 +-
.../dora/tasks/change_lead_time_calculator.go | 1 +
backend/plugins/gitextractor/gitextractor.go | 2 +-
backend/plugins/gitextractor/impl/impl.go | 2 +-
.../plugins/gitextractor/parser/clone_gitcli.go | 62 +++--
backend/plugins/gitextractor/parser/taskdata.go | 29 ++-
backend/plugins/gitextractor/tasks/repo_cloner.go | 3 +
backend/plugins/github/api/scope_config_api.go | 14 +
backend/plugins/github/impl/impl.go | 3 +
backend/plugins/gitlab/api/scope_config_api.go | 14 +
backend/plugins/gitlab/impl/impl.go | 3 +
backend/plugins/jenkins/api/remote_api.go | 13 +-
backend/plugins/jenkins/api/scope_config_api.go | 14 +
backend/plugins/jenkins/impl/impl.go | 3 +
.../plugins/jenkins/tasks/build_cicd_convertor.go | 4 +
backend/plugins/jenkins/tasks/stage_convertor.go | 4 +
backend/plugins/jira/api/scope_config_api.go | 14 +
backend/plugins/jira/impl/impl.go | 3 +
.../jira/tasks/board_filter_begin_collector.go | 3 +
.../jira/tasks/board_filter_end_collector.go | 3 +-
.../jira/tasks/issue_changelog_collector.go | 2 +-
.../jira/tasks/issue_changelog_convertor.go | 76 +++---
.../jira/tasks/issue_changelog_extractor.go | 15 +-
backend/plugins/jira/tasks/issue_convertor.go | 61 ++---
backend/plugins/jira/tasks/issue_extractor.go | 24 +-
...go => 20240508_modify_commit_character_type.go} | 18 +-
.../sonarqube/models/migrationscripts/register.go | 1 +
backend/plugins/tapd/api/scope_config_api.go | 14 +
backend/plugins/tapd/impl/impl.go | 3 +
.../plugins/tapd/tasks/bug_changelog_collector.go | 2 +-
.../plugins/tapd/tasks/bug_changelog_converter.go | 2 +-
.../plugins/tapd/tasks/bug_changelog_extractor.go | 3 +-
backend/plugins/trello/api/scope_config_api.go | 14 +
backend/plugins/trello/impl/impl.go | 3 +
backend/plugins/zentao/api/scope_api.go | 8 +-
backend/plugins/zentao/api/scope_config_api.go | 14 +
backend/plugins/zentao/impl/impl.go | 3 +
backend/plugins/zentao/models/task.go | 2 +-
.../zentao/tasks/bug_repo_commits_collector.go | 2 +-
.../zentao/tasks/bug_repo_commits_convertor.go | 2 +-
.../zentao/tasks/bug_repo_commits_extractor.go | 2 +-
.../zentao/tasks/story_repo_commits_collector.go | 2 +-
.../zentao/tasks/story_repo_commits_convertor.go | 5 +-
.../zentao/tasks/story_repo_commits_extractor.go | 3 +-
.../zentao/tasks/task_repo_commits_collector.go | 2 +-
.../zentao/tasks/task_repo_commits_convertor.go | 5 +-
.../zentao/tasks/task_repo_commits_extractor.go | 3 +-
.../python/plugins/azuredevops/azuredevops/main.py | 2 +-
.../plugins/azuredevops/azuredevops/models.py | 4 +-
.../python/pydevlake/pydevlake/pipeline_tasks.py | 5 +-
backend/server/api/api.go | 37 +--
backend/server/api/project/project.go | 4 +-
backend/server/api/router.go | 4 +-
backend/server/services/project.go | 32 +++
.../server/services/remote/plugin/default_api.go | 3 +
.../services/remote/plugin/scope_config_api.go | 4 +
backend/server/services/task.go | 9 +-
backend/test/e2e/manual/azuredevops/models.go | 1 +
backend/test/e2e/remote/helper.go | 7 +-
backend/test/helper/api.go | 37 ++-
backend/test/helper/models.go | 1 +
config-ui/env.example | 1 +
config-ui/index.html | 2 +-
config-ui/nginx.conf | 6 +-
config-ui/package.json | 4 +-
config-ui/src/api/blueprint/index.ts | 9 +-
config-ui/src/api/project/index.ts | 5 +-
config-ui/src/api/scope-config/index.ts | 5 +
.../register/webhook => api/scope-config}/types.ts | 8 +-
config-ui/src/app/store.ts | 3 +-
.../{tip-layout => action/icon-button}/index.tsx | 21 +-
config-ui/src/components/action/index.ts | 1 +
config-ui/src/features/index.ts | 1 -
config-ui/src/features/tips/index.ts | 19 --
config-ui/src/features/tips/slice.ts | 58 -----
config-ui/src/plugins/components/index.ts | 2 +
.../components/plugin-name}/index.tsx | 40 +--
.../plugins/components/scope-config-form/index.tsx | 16 +-
.../components/scope-config-select/index.tsx | 31 +--
.../src/plugins/components/scope-config/index.tsx | 204 +++++++++++++++
config-ui/src/plugins/register/github/config.tsx | 1 -
.../routes/blueprint/connection-detail/index.tsx | 138 +++++-----
.../routes/blueprint/connection-detail/table.tsx | 102 ++++++++
.../blueprint/detail/blueprint-detail-page.tsx | 8 +
.../components/add-connection-dialog/index.tsx | 50 ++--
config-ui/src/routes/blueprint/home/index.tsx | 15 +-
config-ui/src/routes/connection/connection.tsx | 153 +++++++----
config-ui/src/routes/layout/layout.tsx | 88 ++-----
config-ui/src/routes/layout/styled.ts | 34 ---
config-ui/src/routes/layout/tips-transition.css | 38 ---
config-ui/src/routes/onboard/components/card.tsx | 6 +-
config-ui/src/routes/onboard/index.tsx | 8 +-
config-ui/src/routes/onboard/step-0.tsx | 40 ++-
config-ui/src/routes/onboard/step-3.tsx | 20 +-
config-ui/src/routes/pipeline/components/table.tsx | 10 +-
config-ui/src/routes/pipeline/components/task.tsx | 3 +
config-ui/src/routes/project/detail/index.tsx | 8 +
config-ui/src/routes/project/home/index.tsx | 29 +--
config-ui/src/vite-env.d.ts | 1 +
config-ui/yarn.lock | 73 +++---
env.example | 12 +-
grafana/dashboards/DORA.json | 27 +-
grafana/dashboards/EngineeringOverview.json | 2 +-
172 files changed, 2978 insertions(+), 1295 deletions(-)
copy backend/core/models/common/{string_float64.go => string_int64.go} (62%)
copy backend/core/models/migrationscripts/{20220622_rename_tasks_to_plan.go =>
20240424_add_subtask_states.go} (51%)
create mode 100644 backend/core/models/subtask_state.go
create mode 100644 backend/helpers/pluginhelper/api/api_client_test.go
copy backend/helpers/pluginhelper/api/{api_extractor.go =>
api_extractor_stateful.go} (55%)
create mode 100644 backend/helpers/pluginhelper/api/data_convertor_stateful.go
create mode 100644 backend/helpers/pluginhelper/api/subtask_state_manager.go
copy backend/helpers/pluginhelper/api/{collector_state_manager_test.go =>
subtask_state_manager_test.go} (61%)
create mode 100644 backend/plugins/azuredevops_go/api/azuredevops/client.go
rename backend/plugins/azuredevops_go/api/{vs_client_test.go =>
azuredevops/client_test.go} (97%)
create mode 100644 backend/plugins/azuredevops_go/api/azuredevops/models.go
rename backend/plugins/azuredevops_go/api/{ => azuredevops}/testdata/test.txt
(100%)
delete mode 100644 backend/plugins/azuredevops_go/api/remote_data.go
delete mode 100644 backend/plugins/azuredevops_go/api/vs_client.go
copy backend/plugins/{jira/models/migrationscripts/20230129_add_multi_auth.go
=> azuredevops_go/models/migrationscripts/20240413_add_remote_repo_support.go}
(51%)
copy
backend/plugins/sonarqube/models/migrationscripts/{20240325_modify_commit_character_type.go
=> 20240508_modify_commit_character_type.go} (68%)
copy config-ui/src/{plugins/register/webhook => api/scope-config}/types.ts
(90%)
copy config-ui/src/components/{tip-layout => action/icon-button}/index.tsx
(66%)
delete mode 100644 config-ui/src/features/tips/index.ts
delete mode 100644 config-ui/src/features/tips/slice.ts
copy config-ui/src/{components/action/copy-text =>
plugins/components/plugin-name}/index.tsx (62%)
create mode 100644 config-ui/src/plugins/components/scope-config/index.tsx
create mode 100644 config-ui/src/routes/blueprint/connection-detail/table.tsx
delete mode 100644 config-ui/src/routes/layout/styled.ts
delete mode 100644 config-ui/src/routes/layout/tips-transition.css
