ciaramulligan opened a new issue, #8834: URL: https://github.com/apache/incubator-devlake/issues/8834
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar issues. ### What happened Jira status/resolution changelog items are collected and stored in `_tool_jira_issue_changelog_items` but a significant percentage are never promoted to the domain-level `issue_changelogs` table. This happens silently — no errors, no warnings. The affected changelogs are from the same sync run, same board, same connection as the ones that ARE converted. The converter processed approximately 79% and silently skipped approximately 21% from the same sync run. Across multiple Jira projects on the same instance, the percentage of resolved issues with no status changelogs in the domain table ranges from 4% to 99%. For a specific issue, the tool layer has the data: ```sql ``` SELECT ci.field, ci.from_string, ci.to_string FROM _tool_jira_issue_changelog_items ci JOIN _tool_jira_issue_changelogs cl ON ci.connection_id = cl.connection_id AND ci.changelog_id = cl.changelog_id WHERE cl.issue_id = <ISSUE_ID> AND cl.connection_id = <CONN_ID> AND ci.field = 'status'; ``` -- Returns: e.g. New → In Progress, In Progress → Closed But the domain layer is empty: ``` SELECT field_name, original_from_value, original_to_value FROM issue_changelogs WHERE issue_id = 'jira:JiraIssue:<CONN_ID>:<ISSUE_ID>' AND field_name = 'status'; ``` -- Returns: (empty) Jira UI (History tab) confirms the status transitions exist. What we've ruled out: - Not board scoping — all affected issues exist in _tool_jira_board_issues for the same board as the working issues. - Not incremental since filter — _devlake_collector_latest_state shows time_after far in the past, meaning all changelogs should be converted. - Not a collection gap — 64% of the affected issues have tool-level changelogs that were synced on the same date as the working ones. - Not a specific field — the affected issues have changelog items for status, resolution, labels, Story Points, and other fields. None were converted. Related issues: #7911, #7826, #8196, #7394 ### What do you expect to happen Expected behavior: All changelog items in _tool_jira_issue_changelog_items that have a matching _tool_jira_board_issues entry should be converted to issue_changelogs. The converter should not silently skip records — if a changelog item cannot be converted, it should log a warning. Actual behavior: The converter silently skips a subset of changelogs. No error is logged. The skipped items remain in the tool layer but never appear in the domain layer. ### How to reproduce 1. Configure a Jira Cloud connection with a board 2. Run a full sync (not incremental) 3. Compare counts: -- Tool layer count ``` SELECT COUNT(*) FROM _tool_jira_issue_changelog_items ci JOIN _tool_jira_issue_changelogs cl ON ci.connection_id = cl.connection_id AND ci.changelog_id = cl.changelog_id WHERE cl.connection_id = <CONN_ID> AND ci.field = 'status'; ``` -- Domain layer count ``` SELECT COUNT(*) FROM issue_changelogs WHERE issue_id LIKE 'jira:JiraIssue:<CONN_ID>:%' AND field_name = 'status'; ``` 4. If domain count is significantly lower than tool count, the bug is present. 5. Verify affected issues have status transitions in Jira UI (History tab). ### Anything else Related issues - #7911 — field_id empty in changelog items (fixed, PR #7912) - #7826 — Issues disappearing from dataset (raw has data, domain doesn't — same symptom pattern) - #8196 — Request to use Jira Projects instead of Boards - #7394 — Incremental mode for changelog converter ### Version Latest as of March 2026. Managed instance — exact version/commit SHA not accessible. ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
