dosubot[bot] commented on issue #8838: URL: https://github.com/apache/incubator-devlake/issues/8838#issuecomment-4258234333
<!-- Greeting --> Hi @techdecline! I'm [Dosu](https://go.dosu.dev/dosubot) and I’m helping the incubator-devlake team. <!-- Answer --> Thanks for the detailed report. I dug into the code and can confirm the root cause and suggest where a fix should go. **Root cause:** The [`ParseRawMessageFromRecords`](https://github.com/apache/incubator-devlake/blob/2c15ad880dce830c6512fa58039527cbf4745b8b/backend/plugins/azuredevops_go/tasks/shared.go#L76-L85) response parser calls [`UnmarshalResponse`](https://github.com/apache/incubator-devlake/blob/2c15ad880dce830c6512fa58039527cbf4745b8b/backend/helpers/pluginhelper/api/api_client.go#L416-L435), which returns a hard error on both empty responses (`ErrEmptyResponse`) and malformed/non-JSON bodies. These errors propagate up through the `ApiCollector` and abort the entire subtask. The plugin already has a pattern for gracefully skipping problematic builds — [`ignoreDeletedBuilds`](https://github.com/apache/incubator-devlake/blob/2c15ad880dce830c6512fa58039527cbf4745b8b/backend/plugins/azuredevops_go/tasks/shared.go#L148-L153) uses `api.ErrIgnoreAndContinue` in the `AfterResponse` handler to skip 404s. However, when a YAML-broken build returns a 200 with an empty or non-JSON timeline body, the `AfterResponse` check passes (it's not a 404), and the error only surfaces later during response parsing where there's no ignore/skip mechanism. **Suggested fix approach:** There are a couple of options: 1. **Expand `AfterResponse`** — Check for empty or non-JSON response bodies in `ignoreDeletedBuilds` (or a new wrapper) and return `api.ErrIgnoreAndContinue` for those cases too. This is the minimal change. 2. **Make `ParseRawMessageFromRecords` tolerant** — Return an empty `[]json.RawMessage` instead of an error when the response is empty or cannot be parsed as JSON, with a warning log. This treats missing timelines as "no records" rather than a fatal error. Also worth noting: the collector query on [line 52](https://github.com/apache/incubator-devlake/blob/2c15ad880dce830c6512fa58039527cbf4745b8b/backend/plugins/azuredevops_go/tasks/ci_cd_timeline_records_collector.go#L52) already filters out `result != "failed"` builds, but builds broken by YAML errors may still have a non-`failed` result (e.g., `none`) depending on how Azure DevOps categorizes them, so they slip through. Since you've indicated you're willing to submit a PR — option 1 is probably the cleanest and most consistent with the existing codebase patterns. You could also consider adding a filter for builds with `result = "none"` at the query level if those are consistently the problematic ones. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot?utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=brand-link).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback?utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-great_response#message_id=e9f43039-8d96-4f36-8cd2-c1222bee624e&feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback?utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-irrelevant_answer#message_id=e9f43039-8d96-4f36-8cd2-c1222bee624e&feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback?utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-incorrect_sources#message_id=e9f43039-8d96-4f36-8cd2-c1222bee624e&feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback?utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-too_verbose#message_id=e9f43039-8d96-4f36-8cd2-c1222bee624e&feedback_type=too_verbose) | [Hallucination](https:// app.dosu.dev/response-feedback?utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-hallucination#message_id=e9f43039-8d96-4f36-8cd2-c1222bee624e&feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback?utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-bug_report#message_id=e9f43039-8d96-4f36-8cd2-c1222bee624e&feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback?utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-other#message_id=e9f43039-8d96-4f36-8cd2-c1222bee624e&feedback_type=other)</sup> [](https://app.dosu.dev/b4e8e847-d479-4541-83a8-d88d83fea5c9/ask?utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=chat-badge) [](https://cursor.com/link/prompt?text=Thanks%20for%20the%20detailed%20report.%20I%20dug%20into%20the%20code%20and%20can%20confirm%20the%20root%20cause%20and%20suggest%20where%20a%20fix%20should%20go.%0A%0A%2A%2ARoot%20cause%3A%2A%2A%20The%20%5B%60ParseRawMessageFromRecords%60%5D%28https%3A//github.com/apache/incubator-devlake/blob/2c15ad880dce830c6512fa58039527cbf4745b8b/backend/plugins/azuredevops_go/tasks/shared.go%23L76-L85%29%20response%20parser%20calls%20%5B%60UnmarshalResponse%60%5D%28https%3A//github.com/apache/incubator-devlake/blob/2c15ad880dce830c6512fa58039527cbf4745b8b/backend/helpers/pluginhelper/api/api_client.go%23L416-L435%29%2C%20which%20returns%20a%20hard%20error%20on%20both%20empty%20response s%20%28%60ErrEmptyResponse%60%29%20and%20malformed/non-JSON%20bodies.%20These%20errors%20propagate%20up%20through%20the%20%60ApiCollector%60%20and%20abort%20the%20entire%20subtask.%0A%0AThe%20plugin%20already%20has%20a%20pattern%20for%20gracefully%20skipping%20problematic%20builds%20%E2%80%94%20%5B%60ignoreDeletedBuilds%60%5D%28https%3A//github.com/apache/incubator-devlake/blob/2c15ad880dce830c6512fa58039527cbf4745b8b/backend/plugins/azuredevops_go/tasks/shared.go%23L148-L153%29%20uses%20%60api.ErrIgnoreAndContinue%60%20in%20the%20%60AfterResponse%60%20handler%20to%20skip%20404s.%20However%2C%20when%20a%20YAML-broken%20build%20returns%20a%20200%20with%20an%20empty%20or%20non-JSON%20timeline%20body%2C%20the%20%60AfterResponse%60%20check%20passes%20%28it%27s%20not%20a%20404%29%2C%20and%20the%20error%20only%20surfaces%20later%20during%20response%20parsing%20where%20there%27s%20no%20ignore/skip%20mechanism.%0A%0A%2A%2ASuggested%20fix%20approach%3A%2A%2A%20There%20are%20a%20couple%20of%2 0options%3A%0A%0A1.%20%2A%2AExpand%20%60AfterResponse%60%2A%2A%20%E2%80%94%20Check%20for%20empty%20or%20non-JSON%20response%20bodies%20in%20%60ignoreDeletedBuilds%60%20%28or%20a%20new%20wrapper%29%20and%20return%20%60api.ErrIgnoreAndContinue%60%20for%20those%20cases%20too.%20This%20is%20the%20minimal%20change.%0A%0A2.%20%2A%2AMake%20%60ParseRawMessageFromRecords%60%20tolerant%2A%2A%20%E2%80%94%20Return%20an%20empty%20%60%5B%5Djson.RawMessage%60%20instead%20of%20an%20error%20when%20the%20response%20is%20empty%20or%20cannot%20be%20parsed%20as%20JSON%2C%20with%20a%20warning%20log.%20This%20treats%20missing%20timelines%20as%20%22no%20records%22%20rather%20than%20a%20fatal%20error.%0A%0AAlso%20worth%20noting%3A%20the%20collector%20query%20on%20%5Bline%2052%5D%28https%3A//github.com/apache/incubator-devlake/blob/2c15ad880dce830c6512fa58039527cbf4745b8b/backend/plugins/azuredevops_go/tasks/ci_cd_timeline_records_collector.go%23L52%29%20already%20filters%20out%20%60result%20%21%3D%20%22fail ed%22%60%20builds%2C%20but%20builds%20broken%20by%20YAML%20errors%20may%20still%20have%20a%20non-%60failed%60%20result%20%28e.g.%2C%20%60none%60%29%20depending%20on%20how%20Azure%20DevOps%20categorizes%20them%2C%20so%20they%20slip%20through.%0A%0ASince%20you%27ve%20indicated%20you%27re%20willing%20to%20submit%20a%20PR%20%E2%80%94%20option%201%20is%20probably%20the%20cleanest%20and%20most%20consistent%20with%20the%20existing%20codebase%20patterns.%20You%20could%20also%20consider%20adding%20a%20filter%20for%20builds%20with%20%60result%20%3D%20%22none%22%60%20at%20the%20query%20level%20if%20those%20are%20consistently%20the%20problematic%20ones.) [](https://go.dosu.dev/discord-bot?utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=join-discord) [](https://twitter.com/intent/tweet?text=%40dosu_ai%20 helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/incubator-devlake/issues/8838) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
