dosubot[bot] commented on issue #7877:
URL:
https://github.com/apache/incubator-devlake/issues/7877#issuecomment-2285008271
<!-- Answer -->
You can add the `customize` plugin configuration to an existing project by
updating the blueprint associated with that project. Alternatively, you can
create a new blueprint if you prefer to keep the configurations separate.
### Adding to an Existing Project
To add the `customize` plugin configuration to an existing project, you need
to update the blueprint associated with that project. You can do this by
modifying the existing blueprint's JSON configuration to include the new
transformation rules.
### Creating a New Blueprint
If you prefer to create a new blueprint, you can follow the steps provided
earlier to define the transformation rules and trigger the data extraction.
### Getting the Connection ID and Project ID
To find the correct `ConnectionId` and `ProjectId` for your configuration,
you need to refer to the `rawDataParams` field in the JSON configuration. The
`rawDataParams` should include the `ConnectionId` and any other relevant
identifiers such as `ProjectId`.
Here is an example JSON configuration for triggering data extraction:
```json
[
[
{
"plugin": "customize",
"options": {
"transformationRules": [
{
"table": "commit_files",
"rawDataTable": "_raw_gitlab_api_commit_files",
"rawDataParams": "{\"ConnectionId\":1,\"ProjectId\":123}",
"mapping": {
"x_file_path": "file_path",
"x_commit_sha": "commit_sha",
"x_change_type": "change_type"
}
}
]
}
}
]
]
```
In this example, the `rawDataParams` field contains
`{"ConnectionId":1,"ProjectId":123}`. Therefore, the `ConnectionId` is `1` and
the `ProjectId` is `123`.
### Triggering Data Extraction via API
You can trigger the data extraction by making a POST request to the
`/pipelines` endpoint with the JSON configuration. Here is an example `curl`
command:
```shell
curl 'http://localhost:8080/pipelines' \
--header 'Content-Type: application/json' \
--data-raw '
{
"name": "extract commit files",
"plan": [
[
{
"plugin": "customize",
"options": {
"transformationRules": [
{
"table": "commit_files",
"rawDataTable": "_raw_gitlab_api_commit_files",
"rawDataParams":
"{\"ConnectionId\":1,\"ProjectId\":123}",
"mapping": {
"x_file_path": "file_path",
"x_commit_sha": "commit_sha",
"x_change_type": "change_type"
}
}
]
}
}
]
]
}
'
```
### Managing Connections and Blueprints Programmatically
You can use the DevLake API to manage connections and blueprints
programmatically. For example, you can list all connections for a specific
plugin using the `ListConnections` method, or create a new connection using the
`CreateConnection` method. Here is a Go example from the DevLake client:
```go
// ListConnections lists all connections for a specific plugin
func (d *DevlakeClient) ListConnections(pluginName string) []*Connection {
d.testCtx.Helper()
all := sendHttpRequest[[]*Connection](d.testCtx, d.timeout, &testContext{
client: d,
printPayload: true,
inlineJson: false,
}, http.MethodGet, fmt.Sprintf("%s/plugins/%s/connections", d.Endpoint,
pluginName), nil, nil)
return all
}
// CreateConnection creates a new connection for a specific plugin
func (d *DevlakeClient) CreateConnection(pluginName string, connection any)
*Connection {
d.testCtx.Helper()
created := sendHttpRequest[Connection](d.testCtx, d.timeout,
&testContext{
client: d,
printPayload: true,
inlineJson: false,
}, http.MethodPost, fmt.Sprintf("%s/plugins/%s/connections", d.Endpoint,
pluginName), nil, connection)
return &created
}
```
These methods can help you programmatically manage your connections and
retrieve the necessary `ConnectionId` and `ProjectId` for your configurations
<sup>[[1]](https://github.com/apache/incubator-devlake-website/blob/main/versioned_docs/version-v1.0/Plugins/customize.md)</sup>.
<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
