William Lo created GOBBLIN-2165:
-----------------------------------
Summary: Fix a bug where post-commit steps are not run in partial
commit mode in Gobblin distcp
Key: GOBBLIN-2165
URL: https://issues.apache.org/jira/browse/GOBBLIN-2165
Project: Apache Gobblin
Issue Type: Bug
Reporter: William Lo
When Gobblin is using partial commit semantics, it is possible for files to be
missing on the destination after the job is completed (in line with the
semantic where any number of files may be missing, as long as one file is
successfully published it can still be considered successful).
Currently, Gobblin will raise a FileNotFoundException when this file is being
published if its performs a metadata sync, which prevents the job from running
through its postpublish steps. We still want the postpublish steps to go
through as they may contain important steps for the job to be considered at
least partially successful (permission sync, table registration).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
