desusaiteja commented on issue #58943: URL: https://github.com/apache/airflow/issues/58943#issuecomment-4444623129
Hello - I looked at the code and planning to work on this. But, would like to know if my approach sounds good. My proposed approach is to add source-key validation on partition mappers before downstream mapping is recorded, with a backward-compatible default on the base mapper. For temporal mappers, the validation would reject keys that are parseable but not canonical for the declared input format, which seems to be the failure mode described here. I also plan to add regression coverage at the scheduler/asset-manager level so invalid source keys are logged and skipped instead of creating incorrect APDR state. If that direction sounds right, I can open a PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
