ShivamSharma43 opened a new issue, #9318: URL: https://github.com/apache/devlake/issues/9318
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/devlake/issues?q=is%3Aissue) and found no similar issues. ### What happened Running a blueprint with **one** Tempo team scope adds **every** team of the Tempo instance as a data scope of the connection, and resets the scope config of the team scopes. The `_tool_tempo_teams` table is both the scope table (`TempoTeam` is the plugin's `ToolLayerScope`) and the output of the `extract_teams` subtask. In `backend/plugins/tempo/tasks`: - `collect-teams` calls `GET /4/teams` without a team filter, so it collects all teams of the instance, with connection-wide raw params `{"ConnectionId":1,"TeamId":0}` (the team id of the scope isn't set, unlike the worklog collector). - `extract_teams` turns every collected team into a `TempoTeam` with `ScopeConfigId` 0. Because the params are connection-wide, the extractor's batch save first deletes every `_tool_tempo_teams` row with those params (i.e. every team scope extracted before, for any blueprint of the connection) and re-inserts all teams. Result after one pipeline run: 1. Teams the user never selected show up under the connection's data scopes (`GET /plugins/tempo/connections/1/scopes`). 2. The `scope_config_id` of the selected team scopes is overwritten with 0, so the scope config assignment is lost on every run. 3. A team scope the user deletes from the connection comes back on the next run of any other team scope. Reproduced on `main` (84e3707) with the plugin's e2e helper on PostgreSQL: with only team 71 in `_tool_tempo_teams` (`scope_config_id = 3`) and the raw rows of `e2e/raw_tables/_raw_tempo_api_teams.csv` (teams 17 and 71), running `ExtractTeams` with `{ConnectionId: 1, TeamId: 71}` leaves: ``` team_id=17 name="Abstract BE Team" scope_config_id=0 params={"ConnectionId":1,"TeamId":0} team_id=71 name="SmartShopper (RTL+WHS)" scope_config_id=0 params={"ConnectionId":1,"TeamId":0} ``` Also, `collect-teams` is a stateful collector that sends `updatedFrom` on incremental runs, but `GET /4/teams` has no such parameter (the Tempo OpenAPI spec, `https://apidocs.tempo.io/tempo-openapi.yaml`, lists `offset`, `limit`, `teamMembers`, `name` and `teamIds`). So an incremental run fetches every team again and appends it to the raw table instead of replacing it. ### What do you expect to happen A team scope's pipeline only collects and extracts its own team. The other team scopes of the connection, and teams that were never selected, are left alone, and the team scope keeps its scope config. ### How to reproduce 1. Create a Tempo connection to an instance with several teams. 2. Add one team (e.g. id 71) as a data scope, assign it a scope config, and add it to a blueprint. 3. Run the blueprint. 4. `SELECT team_id, scope_config_id, _raw_data_params FROM _tool_tempo_teams WHERE connection_id = 1` lists every team of the instance, all with `scope_config_id = 0`; the connection's data scope list in config-ui shows all of them. ### Anything else Proposed fix (I have a PR ready): - Scope the team raw data and the extracted rows per team (`{ConnectionId, TeamId}`, like `TempoTeam.ScopeParams()` and the worklog collector), so extracting one team scope only replaces its own row. - `collect-teams`: request only the scope's team with `teamIds`, drop the unsupported `updatedFrom` and use a plain (non-incremental) collector so each run refreshes the raw rows. - `extract_teams`: skip teams other than the scope's team and keep the existing `scope_config_id`. - Unit tests and an e2e test. Related: #9284 (team summary longer than 255 characters). Since every team of the instance is extracted today, one long summary on any team can break `extract_teams` for every team scope of the connection. ### Version main ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
