GitHub user dosubot[bot] added a comment to the discussion: What is the meaning of setting custom value for Time Range in "Set Sync Policy"
<!-- Answer --> Setting a custom Time Range in "Set Sync Policy" defines the earliest date from which DevLake will collect data during syncs. This Time Range acts as the starting point for the initial historical data collection window. After the first successful sync, DevLake switches to incremental syncs, using the last successful sync time as the new lower bound for future data collection, not the original Time Range start date. So, the Time Range is mainly relevant for the first sync or if you reset the sync policy to an earlier date [source](https://github.com/apache/incubator-devlake/blob/2c15ad880dce830c6512fa58039527cbf4745b8b/backend/helpers/pluginhelper/api/collector_state_manager.go#L29-L119). For most data sources (like PRs), the Time Range filter is applied to the creation time. This means if you set the period to 3 months, PRs created more than 3 months ago but merged now will not be collected by the main collector, because their creation date is outside the window [source](https://github.com/apache/incubator-devlake/blob/2c15ad880dce830c6512fa58039527cbf4745b8b/backend/plugins/azuredevops_go/tasks/pr_collector.go#L65-L74). However, DevLake has a secondary mechanism: if incremental sync is enabled and the plugin supports it, there is a secondary collector that can pick up updates to "unfinished" or recently updated records (like open PRs), even if they were created before the Time Range. This helps capture late merges or updates, but only for records that are still open or have recent activity [source](https://github.com/apache/incubator-devlake/blob/2c15ad880dce830c6512fa58039527cbf4745b8b/backend/helpers/pluginhelper/api/api_collector_stateful.go#L30-L269). Here's a simplified diagram of the sync logic: ```mermaid graph TD A["Set Time Range (Start Date)"] --> B["First Sync: Collect data created after Start Date"] B --> C["Incremental Syncs: Use last successful sync time as new lower bound"] C --> D["Secondary Collector: Picks up updates to old records if supported"] ``` If you want to ensure you capture late updates to old PRs, consider setting a wider initial Time Range or check if your plugin supports the secondary collector for unfinished/updated records. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/8a10e72f-d666-48d9-86c4-55f58123fbed?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/8a10e72f-d666-48d9-86c4-55f58123fbed?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/8a10e72f-d666-48d9-86c4-55f58123fbed?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/8a10e72f-d666-48d9-86c4-55f58123fbed?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/8a10e72f-d666-48d9-86c4-55f58123fbed?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/8a10e72f-d666-48d9-86c4-55f58123fbed?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/8a10e72f-d666-48d9-86c4-55f58123fbed?feedback_type=other)</sup> [](https://app.dosu.dev/b4e8e847-d479-4541-83a8-d88d83fea5c9/ask?utm_source=github) [](https://cursor.com/link/prompt?text=Setting%20a%20custom%20Time%20Range%20in%20%22Set%20Sync%20Policy%22%20defines%20the%20earliest%20date%20from%20which%20DevLake%20will%20collect%20data%20during%20syncs.%20This%20Time%20Range%20acts%20as%20the%20starting%20point%20for%20the%20initial%20historical%20data%20collection%20window.%20After%20the%20first%20successful%20sync%2C%20DevLake%20switches%20to%20incremental%20syncs%2C%20using%20the%20last%20successful%20sync%20time%20as%20the%20new%20lower%20bound%20for%20future%20data%20collection%2C%20not%20the%20original%20Time%20Range%20start%20date.%20So%2C%20the%20Time%20Range%20is%20mainly%20relevant%20for%20the%20first%20sync%20or%20if%20you%20reset%20the%20sync%20policy%20to%20an%20earlier%20date%20%5Bsource%5D%28https%3A//github.com/apache /incubator-devlake/blob/2c15ad880dce830c6512fa58039527cbf4745b8b/backend/helpers/pluginhelper/api/collector_state_manager.go%23L29-L119%29.%0A%0AFor%20most%20data%20sources%20%28like%20PRs%29%2C%20the%20Time%20Range%20filter%20is%20applied%20to%20the%20creation%20time.%20This%20means%20if%20you%20set%20the%20period%20to%203%20months%2C%20PRs%20created%20more%20than%203%20months%20ago%20but%20merged%20now%20will%20not%20be%20collected%20by%20the%20main%20collector%2C%20because%20their%20creation%20date%20is%20outside%20the%20window%20%5Bsource%5D%28https%3A//github.com/apache/incubator-devlake/blob/2c15ad880dce830c6512fa58039527cbf4745b8b/backend/plugins/azuredevops_go/tasks/pr_collector.go%23L65-L74%29.%0A%0AHowever%2C%20DevLake%20has%20a%20secondary%20mechanism%3A%20if%20incremental%20sync%20is%20enabled%20and%20the%20plugin%20supports%20it%2C%20there%20is%20a%20secondary%20collector%20that%20can%20pick%20up%20updates%20to%20%22unfinished%22%20or%20recently%20updated%20records%20%2 8like%20open%20PRs%29%2C%20even%20if%20they%20were%20created%20before%20the%20Time%20Range.%20This%20helps%20capture%20late%20merges%20or%20updates%2C%20but%20only%20for%20records%20that%20are%20still%20open%20or%20have%20recent%20activity%20%5Bsource%5D%28https%3A//github.com/apache/incubator-devlake/blob/2c15ad880dce830c6512fa58039527cbf4745b8b/backend/helpers/pluginhelper/api/api_collector_stateful.go%23L30-L269%29.%0A%0AHere%27s%20a%20simplified%20diagram%20of%20the%20sync%20logic%3A%0A%0A%60%60%60mermaid%0Agraph%20TD%0A%20%20%20%20A%5B%22Set%20Time%20Range%20%28Start%20Date%29%22%5D%20--%3E%20B%5B%22First%20Sync%3A%20Collect%20data%20created%20after%20Start%20Date%22%5D%0A%20%20%20%20B%20--%3E%20C%5B%22Incremental%20Syncs%3A%20Use%20last%20successful%20sync%20time%20as%20new%20lower%20bound%22%5D%0A%20%20%20%20C%20--%3E%20D%5B%22Secondary%20Collector%3A%20Picks%20up%20updates%20to%20old%20records%20if%20supported%22%5D%0A%60%60%60%0A%0AIf%20you%20want%20to%20ensure%20you%20capt ure%20late%20updates%20to%20old%20PRs%2C%20consider%20setting%20a%20wider%20initial%20Time%20Range%20or%20check%20if%20your%20plugin%20supports%20the%20secondary%20collector%20for%20unfinished/updated%20records.) [](https://go.dosu.dev/discord-bot) [](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/incubator-devlake/discussions/8663) GitHub link: https://github.com/apache/incubator-devlake/discussions/8663#discussioncomment-15276175 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
