Hi, all,
A user reported a missing MR would cause the whole pipeline failed.
check https://github.com/apache/incubator-devlake/issues/5385 for detail.
It is not the first time we run into such a problem because we
believed MR being deleted is not a Norm, thus, we basically assume they
will last forever in the data source.
However, this "Abnormal Phenomenon" has happened for multiple times,
and we treated them as a set of independent bugs. I think it is time to
address the problem further down to the root.
I think we should ignore all 404 errors for all
secondary-data-collectors (collectors that rely on other entity, e.g.
detail-collector, changelogs-collector , comments-collector, etc)
Feel free to share your thoughts.
Best
Klesh Wong
