[
https://issues.apache.org/jira/browse/GOBBLIN-1972?focusedWorklogId=894391&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-894391
]
ASF GitHub Bot logged work on GOBBLIN-1972:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 06/Dec/23 21:58
Start Date: 06/Dec/23 21:58
Worklog Time Spent: 10m
Work Description: codecov-commenter commented on PR #3844:
URL: https://github.com/apache/gobblin/pull/3844#issuecomment-1843755843
##
[Codecov](https://app.codecov.io/gh/apache/gobblin/pull/3844?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
Report
All modified and coverable lines are covered by tests :white_check_mark:
> Comparison is base
[(`bc456b9`)](https://app.codecov.io/gh/apache/gobblin/commit/bc456b96166db375f2ec4a2066f67f96a023d861?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
47.97% compared to head
[(`c3b8f28`)](https://app.codecov.io/gh/apache/gobblin/pull/3844?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
44.52%.
> Report is 1 commits behind head on master.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #3844 +/- ##
============================================
- Coverage 47.97% 44.52% -3.46%
+ Complexity 8868 2999 -5869
============================================
Files 1743 671 -1072
Lines 67439 26315 -41124
Branches 7299 2682 -4617
============================================
- Hits 32356 11717 -20639
+ Misses 32268 13588 -18680
+ Partials 2815 1010 -1805
```
</details>
[:umbrella: View full report in Codecov by
Sentry](https://app.codecov.io/gh/apache/gobblin/pull/3844?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it
here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
Issue Time Tracking
-------------------
Worklog Id: (was: 894391)
Time Spent: 20m (was: 10m)
> Fix Iceberg-Distcp, which now erroneously shows the job finishing as
> "SUCCESSFUL", despite the commit step failing with an exception.
> -------------------------------------------------------------------------------------------------------------------------------------
>
> Key: GOBBLIN-1972
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1972
> Project: Apache Gobblin
> Issue Type: Bug
> Components: gobblin-core
> Reporter: Kip Kohn
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Find and fix what appears to be a regression in `CopyDataPublisher` or
> `SafeDatasetCommit`...
> Here, the same Iceberg-Distcp `CommitStep` fails
> (`org.apache.gobblin.data.management.copy.iceberg.IcebergRegisterStep`), but
> with apache gobblin build `0.18.0-dev-493` it correctly shows "FAILED" yet
> with `0.18.0-dev-515` it claims "SUCCEEDED".
> BOTH display the same error:
> {code:java}
> ERROR [CopyDataPublisher] [Azkaban] Failed to publish tracking_db.MyEvent
> [then the exception stacktrace for the commit step, followed by...]
> ERROR [SafeDatasetCommit] [Azkaban] Failed to commit dataset
> - java.io.IOException: Not all datasets published successfully
> - at
> org.apache.gobblin.data.management.copy.publisher.CopyDataPublisher.publishData(CopyDataPublisher.java:160)
> - at
> org.apache.gobblin.publisher.DataPublisher.publish(DataPublisher.java:79)
> - at
> org.apache.gobblin.runtime.SafeDatasetCommit.commitDataset(SafeDatasetCommit.java:260)
> - at
> org.apache.gobblin.runtime.SafeDatasetCommit.call(SafeDatasetCommit.java:168)
> [...]{code}
> but only "FAILED" continues with:
> {code:java}
> WARN [SafeDatasetCommit] [Azkaban] At least one task in
> CopyEntity.DatasetAndPartition(dataset=CopyableDatasetMetadata(datasetURN=tracking_db.MyEvent),
> partition=tracking_db.MyEvent) did not get committed successfully. Setting
> dataset state to FAILED. java.io.IOException: Not all datasets published
> successfully
> - at
> org.apache.gobblin.data.management.copy.publisher.CopyDataPublisher.publishData(CopyDataPublisher.java:160)
> - at
> org.apache.gobblin.publisher.DataPublisher.publish(DataPublisher.java:79)
> - at
> org.apache.gobblin.runtime.SafeDatasetCommit.commitDataset(SafeDatasetCommit.java:260)
> - at
> org.apache.gobblin.runtime.SafeDatasetCommit.call(SafeDatasetCommit.java:168)
> [...]
> INFO [AbstractJobLauncher] [Azkaban] Due to task failure, will report that no
> records or bytes were written for
> CopyEntity.DatasetAndPartition(dataset=CopyableDatasetMetadata(datasetURN=tracking_db.MyEvent),
> partition=tracking_db.MyEvent){code}
> by contrast, "SUCCEEDED" announces:
> {code:java}
> INFO [SafeDatasetCommit] [Azkaban] Persisting dataset state for dataset
> CopyEntity.DatasetAndPartition(dataset=CopyableDatasetMetadata(datasetURN=tracking_db.MyEvent),
> partition=tracking_db.MyEvent)
> [...]
> INFO [AbstractJobLauncher] [Azkaban] DatasetMetrics for
> 'CopyEntity.DatasetAndPartition(dataset=CopyableDatasetMetadata(datasetURN=tracking_db.MyEvent),
> partition=tracking_db.MyEvent)' - (records: 0; bytes: 0) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
