wjddn279 commented on issue #63532: URL: https://github.com/apache/airflow/issues/63532#issuecomment-4067111384
Yes, I ran a test with 10 million rows. Since it's difficult to exactly reproduce the migration environment, it would be hard to pinpoint the root cause of the error. (I'm attaching the test code I used for reference. [test_migration.py](https://github.com/user-attachments/files/26022469/test_migration.py)) I also tested with the changes from this [PR](https://github.com/apache/airflow/pull/63628) applied, and it worked fine. The performance was around 500 seconds for migrating 10 million records. (My implementation shows a similar level of performance.) Since this PR's approach works well across many cases, it seems like the safer option. If there's a need to go with my approach instead, I'm happy to set up a local migration environment and do a deeper dive. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
