kkc-tonywu commented on issue #8187:
URL:
https://github.com/apache/incubator-devlake/issues/8187#issuecomment-2505197433
It has been tested, and the issue of only collecting 100 records has been
fixed.
However, based on the content of the PR, it still seems to be limited to 100
pages of data:
```
//incubator-devlake/backend/plugins/gitlab/tasks
/deployment_collector.go
apiCollector.InitCollector(helper.ApiCollectorArgs{
RawDataSubTaskArgs: *rawDataSubTaskArgs,
ApiClient: data.ApiClient,
PageSize: 100, << here
UrlTemplate: "projects/{{ .Params.ProjectId
}}/deployments",
...
```
```
// backend/helpers/pluginhelper/api/api_collector.go
reqDataCopy := RequestData{
Pager: &Pager{
Page: i + 1,
Size: collector.args.PageSize, << here
Skip: collector.args.PageSize * (i),
},
Input: reqData.Input,
InputJSON: reqData.InputJSON,
}
```
For example:
I have set up to collect data for one year on DevLake, and the total number
of records will exceed 10,000. It seems that only 10,000 records will be
collected?
Is this scenario correct? My assumption is that regardless of the time range
setting, I must be able to collect all the data.
I'm not sure if I understand this correctly. Please let me know if there is
any issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
