[jira] [Created] (GOBBLIN-1003) hive data node to support adl and abfs uri for Gobblin Service
pawan teja created GOBBLIN-1003: --- Summary: hive data node to support adl and abfs uri for Gobblin Service Key: GOBBLIN-1003 URL: https://issues.apache.org/jira/browse/GOBBLIN-1003 Project: Apache Gobblin Issue Type: Improvement Reporter: pawan teja -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (GOBBLIN-1001) Implement TimePartitionGlobFinder
[ https://issues.apache.org/jira/browse/GOBBLIN-1001?focusedWorklogId=357564&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-357564 ] ASF GitHub Bot logged work on GOBBLIN-1001: --- Author: ASF GitHub Bot Created on: 11/Dec/19 01:53 Start Date: 11/Dec/19 01:53 Worklog Time Spent: 10m Work Description: autumnust commented on issue #2846: [GOBBLIN-1001] Implement TimePartitionGlobFinder URL: https://github.com/apache/incubator-gobblin/pull/2846#issuecomment-564341258 > @autumnust It does use, compositely use instead of inheriting, an existing GlobFinder, which is `DefaultFileSystemGlobFinder`. OK Gotcha. If `enableEmptyPartition` is not enabled, it does nothing more than just setting a different pattern using `DefaultFileSystemGlobFinder`. If it does, it somehow creates empty dataset for yesterday's partitions. This doesn't seem to be a generic finder to me but serving a very specific purpose. Would you consider doing it as a separated extension for GlobFinder internally instead of open-source? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 357564) Time Spent: 40m (was: 0.5h) > Implement TimePartitionGlobFinder > - > > Key: GOBBLIN-1001 > URL: https://issues.apache.org/jira/browse/GOBBLIN-1001 > Project: Apache Gobblin > Issue Type: Task >Reporter: Zhixiong Chen >Assignee: Zhixiong Chen >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [incubator-gobblin] autumnust commented on issue #2846: [GOBBLIN-1001] Implement TimePartitionGlobFinder
autumnust commented on issue #2846: [GOBBLIN-1001] Implement TimePartitionGlobFinder URL: https://github.com/apache/incubator-gobblin/pull/2846#issuecomment-564341258 > @autumnust It does use, compositely use instead of inheriting, an existing GlobFinder, which is `DefaultFileSystemGlobFinder`. OK Gotcha. If `enableEmptyPartition` is not enabled, it does nothing more than just setting a different pattern using `DefaultFileSystemGlobFinder`. If it does, it somehow creates empty dataset for yesterday's partitions. This doesn't seem to be a generic finder to me but serving a very specific purpose. Would you consider doing it as a separated extension for GlobFinder internally instead of open-source? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Work logged] (GOBBLIN-1001) Implement TimePartitionGlobFinder
[ https://issues.apache.org/jira/browse/GOBBLIN-1001?focusedWorklogId=357554&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-357554 ] ASF GitHub Bot logged work on GOBBLIN-1001: --- Author: ASF GitHub Bot Created on: 11/Dec/19 01:16 Start Date: 11/Dec/19 01:16 Worklog Time Spent: 10m Work Description: zxcware commented on issue #2846: [GOBBLIN-1001] Implement TimePartitionGlobFinder URL: https://github.com/apache/incubator-gobblin/pull/2846#issuecomment-564333132 @autumnust It does use, compositely use instead of inheriting, an existing GlobFinder, which is `DefaultFileSystemGlobFinder`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 357554) Time Spent: 0.5h (was: 20m) > Implement TimePartitionGlobFinder > - > > Key: GOBBLIN-1001 > URL: https://issues.apache.org/jira/browse/GOBBLIN-1001 > Project: Apache Gobblin > Issue Type: Task >Reporter: Zhixiong Chen >Assignee: Zhixiong Chen >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [incubator-gobblin] zxcware commented on issue #2846: [GOBBLIN-1001] Implement TimePartitionGlobFinder
zxcware commented on issue #2846: [GOBBLIN-1001] Implement TimePartitionGlobFinder URL: https://github.com/apache/incubator-gobblin/pull/2846#issuecomment-564333132 @autumnust It does use, compositely use instead of inheriting, an existing GlobFinder, which is `DefaultFileSystemGlobFinder`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (GOBBLIN-972) Make DEFAULT_NUM_THREADS in DagManager public
[ https://issues.apache.org/jira/browse/GOBBLIN-972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Guo resolved GOBBLIN-972. -- Resolution: Fixed > Make DEFAULT_NUM_THREADS in DagManager public > -- > > Key: GOBBLIN-972 > URL: https://issues.apache.org/jira/browse/GOBBLIN-972 > Project: Apache Gobblin > Issue Type: New Feature >Reporter: Chen Guo >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > Make DEFAULT_NUM_THREADS in DagManager public such that it can be referenced > in other places. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (GOBBLIN-1002) Set state id when deserializing state from Gobblin state store
[ https://issues.apache.org/jira/browse/GOBBLIN-1002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudarshan Vasudevan closed GOBBLIN-1002. > Set state id when deserializing state from Gobblin state store > -- > > Key: GOBBLIN-1002 > URL: https://issues.apache.org/jira/browse/GOBBLIN-1002 > Project: Apache Gobblin > Issue Type: Improvement > Components: gobblin-core >Affects Versions: 0.15.0 >Reporter: Sudarshan Vasudevan >Assignee: Abhishek Tiwari >Priority: Major > Fix For: 0.15.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Current behavior returns the deserialized state without setting the state id > member field. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (GOBBLIN-1002) Set state id when deserializing state from Gobblin state store
[ https://issues.apache.org/jira/browse/GOBBLIN-1002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudarshan Vasudevan resolved GOBBLIN-1002. -- Resolution: Fixed > Set state id when deserializing state from Gobblin state store > -- > > Key: GOBBLIN-1002 > URL: https://issues.apache.org/jira/browse/GOBBLIN-1002 > Project: Apache Gobblin > Issue Type: Improvement > Components: gobblin-core >Affects Versions: 0.15.0 >Reporter: Sudarshan Vasudevan >Assignee: Abhishek Tiwari >Priority: Major > Fix For: 0.15.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Current behavior returns the deserialized state without setting the state id > member field. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [incubator-gobblin] asfgit closed pull request #2847: GOBBLIN-1002: Set state id when deserializing state from Gobblin stat…
asfgit closed pull request #2847: GOBBLIN-1002: Set state id when deserializing state from Gobblin stat… URL: https://github.com/apache/incubator-gobblin/pull/2847 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Work logged] (GOBBLIN-1002) Set state id when deserializing state from Gobblin state store
[ https://issues.apache.org/jira/browse/GOBBLIN-1002?focusedWorklogId=357529&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-357529 ] ASF GitHub Bot logged work on GOBBLIN-1002: --- Author: ASF GitHub Bot Created on: 10/Dec/19 23:56 Start Date: 10/Dec/19 23:56 Worklog Time Spent: 10m Work Description: asfgit commented on pull request #2847: GOBBLIN-1002: Set state id when deserializing state from Gobblin stat… URL: https://github.com/apache/incubator-gobblin/pull/2847 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 357529) Time Spent: 50m (was: 40m) > Set state id when deserializing state from Gobblin state store > -- > > Key: GOBBLIN-1002 > URL: https://issues.apache.org/jira/browse/GOBBLIN-1002 > Project: Apache Gobblin > Issue Type: Improvement > Components: gobblin-core >Affects Versions: 0.15.0 >Reporter: Sudarshan Vasudevan >Assignee: Abhishek Tiwari >Priority: Major > Fix For: 0.15.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Current behavior returns the deserialized state without setting the state id > member field. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (GOBBLIN-972) Make DEFAULT_NUM_THREADS in DagManager public
[ https://issues.apache.org/jira/browse/GOBBLIN-972?focusedWorklogId=357528&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-357528 ] ASF GitHub Bot logged work on GOBBLIN-972: -- Author: ASF GitHub Bot Created on: 10/Dec/19 23:54 Start Date: 10/Dec/19 23:54 Worklog Time Spent: 10m Work Description: asfgit commented on pull request #2848: [GOBBLIN-972] Make DEFAULT_NUM_THREADS in DagManager public URL: https://github.com/apache/incubator-gobblin/pull/2848 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 357528) Time Spent: 40m (was: 0.5h) > Make DEFAULT_NUM_THREADS in DagManager public > -- > > Key: GOBBLIN-972 > URL: https://issues.apache.org/jira/browse/GOBBLIN-972 > Project: Apache Gobblin > Issue Type: New Feature >Reporter: Chen Guo >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > Make DEFAULT_NUM_THREADS in DagManager public such that it can be referenced > in other places. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [incubator-gobblin] asfgit closed pull request #2848: [GOBBLIN-972] Make DEFAULT_NUM_THREADS in DagManager public
asfgit closed pull request #2848: [GOBBLIN-972] Make DEFAULT_NUM_THREADS in DagManager public URL: https://github.com/apache/incubator-gobblin/pull/2848 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Work logged] (GOBBLIN-972) Make DEFAULT_NUM_THREADS in DagManager public
[ https://issues.apache.org/jira/browse/GOBBLIN-972?focusedWorklogId=357513&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-357513 ] ASF GitHub Bot logged work on GOBBLIN-972: -- Author: ASF GitHub Bot Created on: 10/Dec/19 23:42 Start Date: 10/Dec/19 23:42 Worklog Time Spent: 10m Work Description: codecov-io commented on issue #2848: [GOBBLIN-972] Make DEFAULT_NUM_THREADS in DagManager public URL: https://github.com/apache/incubator-gobblin/pull/2848#issuecomment-564308805 # [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=h1) Report > Merging [#2848](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-gobblin/commit/0469447f0732bf61a39d38af45e8c84715f87853?src=pr&el=desc) will **increase** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master#2848 +/- ## + Coverage 45.62% 45.62% +<.01% - Complexity 8988 8991 +3 Files 1904 1904 Lines 7134971349 Branches 7876 7876 + Hits 3255132554 +3 Misses3579535795 + Partials 3003 3000 -3 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...blin/service/modules/orchestration/DagManager.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9vcmNoZXN0cmF0aW9uL0RhZ01hbmFnZXIuamF2YQ==) | `79.36% <100%> (ø)` | `12 <0> (ø)` | :arrow_down: | | [.../org/apache/gobblin/cluster/GobblinTaskRunner.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpblRhc2tSdW5uZXIuamF2YQ==) | `63.42% <0%> (-1.39%)` | `27% <0%> (-1%)` | | | [...lin/util/filesystem/FileSystemInstrumentation.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvZmlsZXN5c3RlbS9GaWxlU3lzdGVtSW5zdHJ1bWVudGF0aW9uLmphdmE=) | `92.85% <0%> (ø)` | `4% <0%> (+1%)` | :arrow_up: | | [.../apache/gobblin/runtime/api/JobExecutionState.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvYXBpL0pvYkV4ZWN1dGlvblN0YXRlLmphdmE=) | `80.37% <0%> (+0.93%)` | `24% <0%> (ø)` | :arrow_down: | | [...lin/elasticsearch/writer/FutureCallbackHolder.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tZWxhc3RpY3NlYXJjaC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9lbGFzdGljc2VhcmNoL3dyaXRlci9GdXR1cmVDYWxsYmFja0hvbGRlci5qYXZh) | `62.85% <0%> (+1.42%)` | `4% <0%> (ø)` | :arrow_down: | | [.../org/apache/gobblin/metrics/RootMetricContext.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRyaWNzLWxpYnMvZ29iYmxpbi1tZXRyaWNzLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0cmljcy9Sb290TWV0cmljQ29udGV4dC5qYXZh) | `79.68% <0%> (+1.56%)` | `16% <0%> (+1%)` | :arrow_up: | | [...lin/restli/throttling/ZookeeperLeaderElection.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1yZXN0bGkvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2UvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2Utc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3Jlc3RsaS90aHJvdHRsaW5nL1pvb2tlZXBlckxlYWRlckVsZWN0aW9uLmphdmE=) | `72.22% <0%> (+2.22%)` | `13% <0%> (ø)` | :arrow_down: | | [...a/org/apache/gobblin/util/limiter/NoopLimiter.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvbGltaXRlci9Ob29wTGltaXRlci5qYXZh) | `60% <0%> (+20%)` | `3% <0%> (+1%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impa
[GitHub] [incubator-gobblin] codecov-io edited a comment on issue #2848: [GOBBLIN-972] Make DEFAULT_NUM_THREADS in DagManager public
codecov-io edited a comment on issue #2848: [GOBBLIN-972] Make DEFAULT_NUM_THREADS in DagManager public URL: https://github.com/apache/incubator-gobblin/pull/2848#issuecomment-564308805 # [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=h1) Report > Merging [#2848](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-gobblin/commit/0469447f0732bf61a39d38af45e8c84715f87853?src=pr&el=desc) will **increase** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master#2848 +/- ## + Coverage 45.62% 45.62% +<.01% - Complexity 8988 8991 +3 Files 1904 1904 Lines 7134971349 Branches 7876 7876 + Hits 3255132554 +3 Misses3579535795 + Partials 3003 3000 -3 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...blin/service/modules/orchestration/DagManager.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9vcmNoZXN0cmF0aW9uL0RhZ01hbmFnZXIuamF2YQ==) | `79.36% <100%> (ø)` | `12 <0> (ø)` | :arrow_down: | | [.../org/apache/gobblin/cluster/GobblinTaskRunner.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpblRhc2tSdW5uZXIuamF2YQ==) | `63.42% <0%> (-1.39%)` | `27% <0%> (-1%)` | | | [...lin/util/filesystem/FileSystemInstrumentation.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvZmlsZXN5c3RlbS9GaWxlU3lzdGVtSW5zdHJ1bWVudGF0aW9uLmphdmE=) | `92.85% <0%> (ø)` | `4% <0%> (+1%)` | :arrow_up: | | [.../apache/gobblin/runtime/api/JobExecutionState.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvYXBpL0pvYkV4ZWN1dGlvblN0YXRlLmphdmE=) | `80.37% <0%> (+0.93%)` | `24% <0%> (ø)` | :arrow_down: | | [...lin/elasticsearch/writer/FutureCallbackHolder.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tZWxhc3RpY3NlYXJjaC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9lbGFzdGljc2VhcmNoL3dyaXRlci9GdXR1cmVDYWxsYmFja0hvbGRlci5qYXZh) | `62.85% <0%> (+1.42%)` | `4% <0%> (ø)` | :arrow_down: | | [.../org/apache/gobblin/metrics/RootMetricContext.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRyaWNzLWxpYnMvZ29iYmxpbi1tZXRyaWNzLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0cmljcy9Sb290TWV0cmljQ29udGV4dC5qYXZh) | `79.68% <0%> (+1.56%)` | `16% <0%> (+1%)` | :arrow_up: | | [...lin/restli/throttling/ZookeeperLeaderElection.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1yZXN0bGkvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2UvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2Utc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3Jlc3RsaS90aHJvdHRsaW5nL1pvb2tlZXBlckxlYWRlckVsZWN0aW9uLmphdmE=) | `72.22% <0%> (+2.22%)` | `13% <0%> (ø)` | :arrow_down: | | [...a/org/apache/gobblin/util/limiter/NoopLimiter.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvbGltaXRlci9Ob29wTGltaXRlci5qYXZh) | `60% <0%> (+20%)` | `3% <0%> (+1%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=footer). Last update [0469447...351ea4a](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an aut
[jira] [Work logged] (GOBBLIN-972) Make DEFAULT_NUM_THREADS in DagManager public
[ https://issues.apache.org/jira/browse/GOBBLIN-972?focusedWorklogId=357512&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-357512 ] ASF GitHub Bot logged work on GOBBLIN-972: -- Author: ASF GitHub Bot Created on: 10/Dec/19 23:36 Start Date: 10/Dec/19 23:36 Worklog Time Spent: 10m Work Description: codecov-io commented on issue #2848: [GOBBLIN-972] Make DEFAULT_NUM_THREADS in DagManager public URL: https://github.com/apache/incubator-gobblin/pull/2848#issuecomment-564308805 # [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=h1) Report > Merging [#2848](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-gobblin/commit/0469447f0732bf61a39d38af45e8c84715f87853?src=pr&el=desc) will **decrease** coverage by `41.47%`. > The diff coverage is `0%`. [](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #2848 +/- ## - Coverage 45.62% 4.15% -41.48% + Complexity 8988 748 -8240 Files 19041904 Lines 71349 71349 Branches 78767876 - Hits 325512961-29590 - Misses35795 68070+32275 + Partials 3003 318 -2685 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...blin/service/modules/orchestration/DagManager.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9vcmNoZXN0cmF0aW9uL0RhZ01hbmFnZXIuamF2YQ==) | `0% <0%> (-79.37%)` | `0 <0> (-12)` | | | [...n/converter/AvroStringFieldDecryptorConverter.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tY3J5cHRvLXByb3ZpZGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvbnZlcnRlci9BdnJvU3RyaW5nRmllbGREZWNyeXB0b3JDb252ZXJ0ZXIuamF2YQ==) | `0% <0%> (-100%)` | `0% <0%> (-2%)` | | | [...he/gobblin/cluster/TaskRunnerSuiteThreadModel.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvVGFza1J1bm5lclN1aXRlVGhyZWFkTW9kZWwuamF2YQ==) | `0% <0%> (-100%)` | `0% <0%> (-5%)` | | | [...n/mapreduce/avro/AvroKeyCompactorOutputFormat.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1jb21wYWN0aW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvbXBhY3Rpb24vbWFwcmVkdWNlL2F2cm8vQXZyb0tleUNvbXBhY3Rvck91dHB1dEZvcm1hdC5qYXZh) | `0% <0%> (-100%)` | `0% <0%> (-3%)` | | | [...apache/gobblin/fork/CopyNotSupportedException.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZm9yay9Db3B5Tm90U3VwcG9ydGVkRXhjZXB0aW9uLmphdmE=) | `0% <0%> (-100%)` | `0% <0%> (-1%)` | | | [.../gobblin/kafka/writer/KafkaWriterCommonConfig.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4ta2Fma2EtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2thZmthL3dyaXRlci9LYWZrYVdyaXRlckNvbW1vbkNvbmZpZy5qYXZh) | `0% <0%> (-100%)` | `0% <0%> (-7%)` | | | [...ker/task/TaskLevelPolicyCheckerBuilderFactory.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3F1YWxpdHljaGVja2VyL3Rhc2svVGFza0xldmVsUG9saWN5Q2hlY2tlckJ1aWxkZXJGYWN0b3J5LmphdmE=) | `0% <0%> (-100%)` | `0% <0%> (-2%)` | | | [...bblin/data/management/copy/AllEqualComparator.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvQWxsRXF1YWxDb21wYXJhdG9yLmphdmE=) | `0% <0%> (-100%)` | `0% <0%> (-2%)` | | | [...blin/converter/string/ObjectToStringConverter.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvbnZlcnRlci9zdHJpbmcvT2JqZWN0VG9TdHJpbmdDb252ZXJ0ZXIuamF2YQ==) | `0% <0%> (-100%)` | `0% <0%> (-3%)` | | | [...rquet/JsonIntermediateToParquetG
[GitHub] [incubator-gobblin] codecov-io commented on issue #2848: [GOBBLIN-972] Make DEFAULT_NUM_THREADS in DagManager public
codecov-io commented on issue #2848: [GOBBLIN-972] Make DEFAULT_NUM_THREADS in DagManager public URL: https://github.com/apache/incubator-gobblin/pull/2848#issuecomment-564308805 # [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=h1) Report > Merging [#2848](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-gobblin/commit/0469447f0732bf61a39d38af45e8c84715f87853?src=pr&el=desc) will **decrease** coverage by `41.47%`. > The diff coverage is `0%`. [](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #2848 +/- ## - Coverage 45.62% 4.15% -41.48% + Complexity 8988 748 -8240 Files 19041904 Lines 71349 71349 Branches 78767876 - Hits 325512961-29590 - Misses35795 68070+32275 + Partials 3003 318 -2685 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2848?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...blin/service/modules/orchestration/DagManager.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9vcmNoZXN0cmF0aW9uL0RhZ01hbmFnZXIuamF2YQ==) | `0% <0%> (-79.37%)` | `0 <0> (-12)` | | | [...n/converter/AvroStringFieldDecryptorConverter.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tY3J5cHRvLXByb3ZpZGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvbnZlcnRlci9BdnJvU3RyaW5nRmllbGREZWNyeXB0b3JDb252ZXJ0ZXIuamF2YQ==) | `0% <0%> (-100%)` | `0% <0%> (-2%)` | | | [...he/gobblin/cluster/TaskRunnerSuiteThreadModel.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvVGFza1J1bm5lclN1aXRlVGhyZWFkTW9kZWwuamF2YQ==) | `0% <0%> (-100%)` | `0% <0%> (-5%)` | | | [...n/mapreduce/avro/AvroKeyCompactorOutputFormat.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1jb21wYWN0aW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvbXBhY3Rpb24vbWFwcmVkdWNlL2F2cm8vQXZyb0tleUNvbXBhY3Rvck91dHB1dEZvcm1hdC5qYXZh) | `0% <0%> (-100%)` | `0% <0%> (-3%)` | | | [...apache/gobblin/fork/CopyNotSupportedException.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZm9yay9Db3B5Tm90U3VwcG9ydGVkRXhjZXB0aW9uLmphdmE=) | `0% <0%> (-100%)` | `0% <0%> (-1%)` | | | [.../gobblin/kafka/writer/KafkaWriterCommonConfig.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4ta2Fma2EtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2thZmthL3dyaXRlci9LYWZrYVdyaXRlckNvbW1vbkNvbmZpZy5qYXZh) | `0% <0%> (-100%)` | `0% <0%> (-7%)` | | | [...ker/task/TaskLevelPolicyCheckerBuilderFactory.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3F1YWxpdHljaGVja2VyL3Rhc2svVGFza0xldmVsUG9saWN5Q2hlY2tlckJ1aWxkZXJGYWN0b3J5LmphdmE=) | `0% <0%> (-100%)` | `0% <0%> (-2%)` | | | [...bblin/data/management/copy/AllEqualComparator.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvQWxsRXF1YWxDb21wYXJhdG9yLmphdmE=) | `0% <0%> (-100%)` | `0% <0%> (-2%)` | | | [...blin/converter/string/ObjectToStringConverter.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvbnZlcnRlci9zdHJpbmcvT2JqZWN0VG9TdHJpbmdDb252ZXJ0ZXIuamF2YQ==) | `0% <0%> (-100%)` | `0% <0%> (-3%)` | | | [...rquet/JsonIntermediateToParquetGroupConverter.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tcGFycXVldC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9jb252ZXJ0ZXIvcGFycXVldC9Kc29uSW50ZXJtZWRpYXRlVG9QYXJxdWV0R3JvdXBDb252ZXJ0ZXIuamF2YQ==) | `0% <0%> (-100%)` | `0% <0%> (-3%)` | | | ... and [1099 more](https://codecov.io/gh/apache/incubator-gobblin/pull/2848/diff?src=pr&el=tree-more) | |
[GitHub] [incubator-gobblin] codecov-io edited a comment on issue #2847: GOBBLIN-1002: Set state id when deserializing state from Gobblin stat…
codecov-io edited a comment on issue #2847: GOBBLIN-1002: Set state id when deserializing state from Gobblin stat… URL: https://github.com/apache/incubator-gobblin/pull/2847#issuecomment-564304442 # [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=h1) Report > Merging [#2847](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-gobblin/commit/0469447f0732bf61a39d38af45e8c84715f87853?src=pr&el=desc) will **increase** coverage by `<.01%`. > The diff coverage is `28.57%`. [](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master#2847 +/- ## + Coverage 45.62% 45.62% +<.01% Complexity 8988 8988 Files 1904 1904 Lines 7134971355 +6 Branches 7876 7876 + Hits 3255132554 +3 - Misses3579535799 +4 + Partials 3003 3002 -1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../org/apache/gobblin/metastore/MysqlStateStore.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRhc3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0YXN0b3JlL015c3FsU3RhdGVTdG9yZS5qYXZh) | `8.19% <0%> (-0.07%)` | `2 <0> (ø)` | | | [.../gobblin/runtime/CheckpointableWatermarkState.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvQ2hlY2twb2ludGFibGVXYXRlcm1hcmtTdGF0ZS5qYXZh) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...ava/org/apache/gobblin/metastore/ZkStateStore.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4taGVsaXgvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0YXN0b3JlL1prU3RhdGVTdG9yZS5qYXZh) | `63.06% <100%> (+0.33%)` | `23 <0> (ø)` | :arrow_down: | | [...ava/org/apache/gobblin/metastore/FsStateStore.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRhc3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0YXN0b3JlL0ZzU3RhdGVTdG9yZS5qYXZh) | `36.05% <50%> (+0.19%)` | `14 <0> (ø)` | :arrow_down: | | [...e/gobblin/runtime/locks/ZookeeperBasedJobLock.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvbG9ja3MvWm9va2VlcGVyQmFzZWRKb2JMb2NrLmphdmE=) | `63.33% <0%> (-1.12%)` | `15% <0%> (-1%)` | | | [.../org/apache/gobblin/cluster/GobblinTaskRunner.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpblRhc2tSdW5uZXIuamF2YQ==) | `65.27% <0%> (+0.46%)` | `28% <0%> (ø)` | :arrow_down: | | [.../org/apache/gobblin/metrics/RootMetricContext.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRyaWNzLWxpYnMvZ29iYmxpbi1tZXRyaWNzLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0cmljcy9Sb290TWV0cmljQ29udGV4dC5qYXZh) | `79.68% <0%> (+1.56%)` | `16% <0%> (+1%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=footer). Last update [0469447...595c7a8](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Work logged] (GOBBLIN-1002) Set state id when deserializing state from Gobblin state store
[ https://issues.apache.org/jira/browse/GOBBLIN-1002?focusedWorklogId=357510&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-357510 ] ASF GitHub Bot logged work on GOBBLIN-1002: --- Author: ASF GitHub Bot Created on: 10/Dec/19 23:25 Start Date: 10/Dec/19 23:25 Worklog Time Spent: 10m Work Description: codecov-io commented on issue #2847: GOBBLIN-1002: Set state id when deserializing state from Gobblin stat… URL: https://github.com/apache/incubator-gobblin/pull/2847#issuecomment-564304442 # [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=h1) Report > Merging [#2847](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-gobblin/commit/0469447f0732bf61a39d38af45e8c84715f87853?src=pr&el=desc) will **increase** coverage by `<.01%`. > The diff coverage is `28.57%`. [](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master#2847 +/- ## + Coverage 45.62% 45.62% +<.01% Complexity 8988 8988 Files 1904 1904 Lines 7134971355 +6 Branches 7876 7876 + Hits 3255132554 +3 - Misses3579535799 +4 + Partials 3003 3002 -1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../org/apache/gobblin/metastore/MysqlStateStore.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRhc3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0YXN0b3JlL015c3FsU3RhdGVTdG9yZS5qYXZh) | `8.19% <0%> (-0.07%)` | `2 <0> (ø)` | | | [.../gobblin/runtime/CheckpointableWatermarkState.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvQ2hlY2twb2ludGFibGVXYXRlcm1hcmtTdGF0ZS5qYXZh) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...ava/org/apache/gobblin/metastore/ZkStateStore.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4taGVsaXgvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0YXN0b3JlL1prU3RhdGVTdG9yZS5qYXZh) | `63.06% <100%> (+0.33%)` | `23 <0> (ø)` | :arrow_down: | | [...ava/org/apache/gobblin/metastore/FsStateStore.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRhc3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0YXN0b3JlL0ZzU3RhdGVTdG9yZS5qYXZh) | `36.05% <50%> (+0.19%)` | `14 <0> (ø)` | :arrow_down: | | [...e/gobblin/runtime/locks/ZookeeperBasedJobLock.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvbG9ja3MvWm9va2VlcGVyQmFzZWRKb2JMb2NrLmphdmE=) | `63.33% <0%> (-1.12%)` | `15% <0%> (-1%)` | | | [.../org/apache/gobblin/cluster/GobblinTaskRunner.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpblRhc2tSdW5uZXIuamF2YQ==) | `65.27% <0%> (+0.46%)` | `28% <0%> (ø)` | :arrow_down: | | [.../org/apache/gobblin/metrics/RootMetricContext.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRyaWNzLWxpYnMvZ29iYmxpbi1tZXRyaWNzLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0cmljcy9Sb290TWV0cmljQ29udGV4dC5qYXZh) | `79.68% <0%> (+1.56%)` | `16% <0%> (+1%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=footer). Last update [0469447...595c7a8](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Servi
[GitHub] [incubator-gobblin] codecov-io commented on issue #2847: GOBBLIN-1002: Set state id when deserializing state from Gobblin stat…
codecov-io commented on issue #2847: GOBBLIN-1002: Set state id when deserializing state from Gobblin stat… URL: https://github.com/apache/incubator-gobblin/pull/2847#issuecomment-564304442 # [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=h1) Report > Merging [#2847](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-gobblin/commit/0469447f0732bf61a39d38af45e8c84715f87853?src=pr&el=desc) will **decrease** coverage by `41.47%`. > The diff coverage is `14.28%`. [](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #2847 +/- ## - Coverage 45.62% 4.14% -41.48% + Complexity 8988 747 -8241 Files 19041904 Lines 71349 71355+6 Branches 78767876 - Hits 325512961-29590 - Misses35795 68075+32280 + Partials 3003 319 -2684 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../org/apache/gobblin/metastore/MysqlStateStore.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRhc3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0YXN0b3JlL015c3FsU3RhdGVTdG9yZS5qYXZh) | `0% <0%> (-8.27%)` | `0 <0> (-2)` | | | [.../gobblin/runtime/CheckpointableWatermarkState.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvQ2hlY2twb2ludGFibGVXYXRlcm1hcmtTdGF0ZS5qYXZh) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...ava/org/apache/gobblin/metastore/FsStateStore.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRhc3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0YXN0b3JlL0ZzU3RhdGVTdG9yZS5qYXZh) | `0% <0%> (-35.87%)` | `0 <0> (-14)` | | | [...ava/org/apache/gobblin/metastore/ZkStateStore.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4taGVsaXgvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0YXN0b3JlL1prU3RhdGVTdG9yZS5qYXZh) | `63.06% <100%> (+0.33%)` | `23 <0> (ø)` | :arrow_down: | | [...n/converter/AvroStringFieldDecryptorConverter.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tY3J5cHRvLXByb3ZpZGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvbnZlcnRlci9BdnJvU3RyaW5nRmllbGREZWNyeXB0b3JDb252ZXJ0ZXIuamF2YQ==) | `0% <0%> (-100%)` | `0% <0%> (-2%)` | | | [...he/gobblin/cluster/TaskRunnerSuiteThreadModel.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvVGFza1J1bm5lclN1aXRlVGhyZWFkTW9kZWwuamF2YQ==) | `0% <0%> (-100%)` | `0% <0%> (-5%)` | | | [...n/mapreduce/avro/AvroKeyCompactorOutputFormat.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1jb21wYWN0aW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvbXBhY3Rpb24vbWFwcmVkdWNlL2F2cm8vQXZyb0tleUNvbXBhY3Rvck91dHB1dEZvcm1hdC5qYXZh) | `0% <0%> (-100%)` | `0% <0%> (-3%)` | | | [...apache/gobblin/fork/CopyNotSupportedException.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZm9yay9Db3B5Tm90U3VwcG9ydGVkRXhjZXB0aW9uLmphdmE=) | `0% <0%> (-100%)` | `0% <0%> (-1%)` | | | [.../gobblin/kafka/writer/KafkaWriterCommonConfig.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4ta2Fma2EtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2thZmthL3dyaXRlci9LYWZrYVdyaXRlckNvbW1vbkNvbmZpZy5qYXZh) | `0% <0%> (-100%)` | `0% <0%> (-7%)` | | | [...ker/task/TaskLevelPolicyCheckerBuilderFactory.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3F1YWxpdHljaGVja2VyL3Rhc2svVGFza0xldmVsUG9saWN5Q2hlY2tlckJ1aWxkZXJGYWN0b3J5LmphdmE=) | `0% <0%> (-100%)` | `0% <0%> (-2%)` | | | ... and [1103 more](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https:
[jira] [Work logged] (GOBBLIN-1002) Set state id when deserializing state from Gobblin state store
[ https://issues.apache.org/jira/browse/GOBBLIN-1002?focusedWorklogId=357506&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-357506 ] ASF GitHub Bot logged work on GOBBLIN-1002: --- Author: ASF GitHub Bot Created on: 10/Dec/19 23:19 Start Date: 10/Dec/19 23:19 Worklog Time Spent: 10m Work Description: codecov-io commented on issue #2847: GOBBLIN-1002: Set state id when deserializing state from Gobblin stat… URL: https://github.com/apache/incubator-gobblin/pull/2847#issuecomment-564304442 # [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=h1) Report > Merging [#2847](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-gobblin/commit/0469447f0732bf61a39d38af45e8c84715f87853?src=pr&el=desc) will **decrease** coverage by `41.47%`. > The diff coverage is `14.28%`. [](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #2847 +/- ## - Coverage 45.62% 4.14% -41.48% + Complexity 8988 747 -8241 Files 19041904 Lines 71349 71355+6 Branches 78767876 - Hits 325512961-29590 - Misses35795 68075+32280 + Partials 3003 319 -2684 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2847?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../org/apache/gobblin/metastore/MysqlStateStore.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRhc3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0YXN0b3JlL015c3FsU3RhdGVTdG9yZS5qYXZh) | `0% <0%> (-8.27%)` | `0 <0> (-2)` | | | [.../gobblin/runtime/CheckpointableWatermarkState.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvQ2hlY2twb2ludGFibGVXYXRlcm1hcmtTdGF0ZS5qYXZh) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...ava/org/apache/gobblin/metastore/FsStateStore.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRhc3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0YXN0b3JlL0ZzU3RhdGVTdG9yZS5qYXZh) | `0% <0%> (-35.87%)` | `0 <0> (-14)` | | | [...ava/org/apache/gobblin/metastore/ZkStateStore.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4taGVsaXgvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0YXN0b3JlL1prU3RhdGVTdG9yZS5qYXZh) | `63.06% <100%> (+0.33%)` | `23 <0> (ø)` | :arrow_down: | | [...n/converter/AvroStringFieldDecryptorConverter.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tY3J5cHRvLXByb3ZpZGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvbnZlcnRlci9BdnJvU3RyaW5nRmllbGREZWNyeXB0b3JDb252ZXJ0ZXIuamF2YQ==) | `0% <0%> (-100%)` | `0% <0%> (-2%)` | | | [...he/gobblin/cluster/TaskRunnerSuiteThreadModel.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvVGFza1J1bm5lclN1aXRlVGhyZWFkTW9kZWwuamF2YQ==) | `0% <0%> (-100%)` | `0% <0%> (-5%)` | | | [...n/mapreduce/avro/AvroKeyCompactorOutputFormat.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1jb21wYWN0aW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvbXBhY3Rpb24vbWFwcmVkdWNlL2F2cm8vQXZyb0tleUNvbXBhY3Rvck91dHB1dEZvcm1hdC5qYXZh) | `0% <0%> (-100%)` | `0% <0%> (-3%)` | | | [...apache/gobblin/fork/CopyNotSupportedException.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZm9yay9Db3B5Tm90U3VwcG9ydGVkRXhjZXB0aW9uLmphdmE=) | `0% <0%> (-100%)` | `0% <0%> (-1%)` | | | [.../gobblin/kafka/writer/KafkaWriterCommonConfig.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2847/diff?src=pr&el=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4ta2Fma2EtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2thZmthL3dyaXRlci9LYWZrYVdyaXRlckNvbW1vbkNvbmZpZy5qYXZh) | `0% <0%> (-100%)` | `0% <0%> (-7%)` | | | [...ker/task/TaskLevelPolicyCheckerBuilderFactory.java](https://codecov.io/gh/
[jira] [Resolved] (GOBBLIN-998) ExecutionStatus should be reset to PENDING before a job retries
[ https://issues.apache.org/jira/browse/GOBBLIN-998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Guo resolved GOBBLIN-998. -- Resolution: Fixed > ExecutionStatus should be reset to PENDING before a job retries > --- > > Key: GOBBLIN-998 > URL: https://issues.apache.org/jira/browse/GOBBLIN-998 > Project: Apache Gobblin > Issue Type: Bug >Reporter: Chen Guo >Priority: Critical > Time Spent: 50m > Remaining Estimate: 0h > > In the modifyStateIfRetryRequired of KafkaJobStatusMonitor, when the state is > Failed and currentAttempts < maxAttempts, the ExecutionStatus is set to > Running. > However, due to the checkin from > GOBBLIN-974([https://github.com/apache/incubator-gobblin/blob/9f50a2563cc257039da44018663b6b9e119fb499/gobblin-service/src/main/java/org/apache/gobblin/service/monitoring/KafkaJobStatusMonitor.java#L159]), > the currentAttempts update from a lower-order event(like Orchestrated) > cannot be consumed to update the jobState file. Thus it will cause infinite > retries in DagManagerThread for failed jobs when it poolAndAdvanceDag. > > The solution is to update ExecutionStatus to PENDING instead of Running. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (GOBBLIN-972) Make DEFAULT_NUM_THREADS in DagManager public
[
https://issues.apache.org/jira/browse/GOBBLIN-972?focusedWorklogId=357486&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-357486
]
ASF GitHub Bot logged work on GOBBLIN-972:
--
Author: ASF GitHub Bot
Created on: 10/Dec/19 22:40
Start Date: 10/Dec/19 22:40
Worklog Time Spent: 10m
Work Description: enjoyear commented on pull request #2848: [GOBBLIN-972]
Make DEFAULT_NUM_THREADS in DagManager public
URL: https://github.com/apache/incubator-gobblin/pull/2848
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I
have checked off all the steps below!
### JIRA
- [X] My PR addresses the following [Gobblin
JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references
them in the PR title. For example, "[GOBBLIN-972] My Gobblin PR"
- https://issues.apache.org/jira/browse/GOBBLIN-972
### Description
- [X] Here are some details about my PR, including screenshots (if
applicable):
### Tests
- [X My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [X] My commits all reference JIRA issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not "adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 357486)
Remaining Estimate: 0h
Time Spent: 10m
> Make DEFAULT_NUM_THREADS in DagManager public
> --
>
> Key: GOBBLIN-972
> URL: https://issues.apache.org/jira/browse/GOBBLIN-972
> Project: Apache Gobblin
> Issue Type: New Feature
>Reporter: Chen Guo
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Make DEFAULT_NUM_THREADS in DagManager public such that it can be referenced
> in other places.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [incubator-gobblin] enjoyear opened a new pull request #2848: [GOBBLIN-972] Make DEFAULT_NUM_THREADS in DagManager public
enjoyear opened a new pull request #2848: [GOBBLIN-972] Make
DEFAULT_NUM_THREADS in DagManager public
URL: https://github.com/apache/incubator-gobblin/pull/2848
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I
have checked off all the steps below!
### JIRA
- [X] My PR addresses the following [Gobblin
JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references
them in the PR title. For example, "[GOBBLIN-972] My Gobblin PR"
- https://issues.apache.org/jira/browse/GOBBLIN-972
### Description
- [X] Here are some details about my PR, including screenshots (if
applicable):
### Tests
- [X My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [X] My commits all reference JIRA issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not "adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Work logged] (GOBBLIN-1002) Set state id when deserializing state from Gobblin state store
[ https://issues.apache.org/jira/browse/GOBBLIN-1002?focusedWorklogId=357479&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-357479 ] ASF GitHub Bot logged work on GOBBLIN-1002: --- Author: ASF GitHub Bot Created on: 10/Dec/19 22:32 Start Date: 10/Dec/19 22:32 Worklog Time Spent: 10m Work Description: sv2000 commented on issue #2847: GOBBLIN-1002: Set state id when deserializing state from Gobblin stat… URL: https://github.com/apache/incubator-gobblin/pull/2847#issuecomment-564290025 @htran1 please review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 357479) Time Spent: 20m (was: 10m) > Set state id when deserializing state from Gobblin state store > -- > > Key: GOBBLIN-1002 > URL: https://issues.apache.org/jira/browse/GOBBLIN-1002 > Project: Apache Gobblin > Issue Type: Improvement > Components: gobblin-core >Affects Versions: 0.15.0 >Reporter: Sudarshan Vasudevan >Assignee: Abhishek Tiwari >Priority: Major > Fix For: 0.15.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Current behavior returns the deserialized state without setting the state id > member field. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [incubator-gobblin] sv2000 commented on issue #2847: GOBBLIN-1002: Set state id when deserializing state from Gobblin stat…
sv2000 commented on issue #2847: GOBBLIN-1002: Set state id when deserializing state from Gobblin stat… URL: https://github.com/apache/incubator-gobblin/pull/2847#issuecomment-564290025 @htran1 please review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-gobblin] sv2000 opened a new pull request #2847: GOBBLIN-1002: Set state id when deserializing state from Gobblin stat…
sv2000 opened a new pull request #2847: GOBBLIN-1002: Set state id when
deserializing state from Gobblin stat…
URL: https://github.com/apache/incubator-gobblin/pull/2847
…e store
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I
have checked off all the steps below!
### JIRA
- [x] My PR addresses the following [Gobblin
JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references
them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
- https://issues.apache.org/jira/browse/GOBBLIN-1002
### Description
- [x] Here are some details about my PR, including screenshots (if
applicable):
Current behavior returns the deserialized state without setting the state id
member field.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Enhanced unit tests for all state store implementations to validate the
returned state id.
### Commits
- [x] My commits all reference JIRA issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not "adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Work logged] (GOBBLIN-1002) Set state id when deserializing state from Gobblin state store
[
https://issues.apache.org/jira/browse/GOBBLIN-1002?focusedWorklogId=357473&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-357473
]
ASF GitHub Bot logged work on GOBBLIN-1002:
---
Author: ASF GitHub Bot
Created on: 10/Dec/19 22:27
Start Date: 10/Dec/19 22:27
Worklog Time Spent: 10m
Work Description: sv2000 commented on pull request #2847: GOBBLIN-1002:
Set state id when deserializing state from Gobblin stat…
URL: https://github.com/apache/incubator-gobblin/pull/2847
…e store
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I
have checked off all the steps below!
### JIRA
- [x] My PR addresses the following [Gobblin
JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references
them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
- https://issues.apache.org/jira/browse/GOBBLIN-1002
### Description
- [x] Here are some details about my PR, including screenshots (if
applicable):
Current behavior returns the deserialized state without setting the state id
member field.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Enhanced unit tests for all state store implementations to validate the
returned state id.
### Commits
- [x] My commits all reference JIRA issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not "adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 357473)
Remaining Estimate: 0h
Time Spent: 10m
> Set state id when deserializing state from Gobblin state store
> --
>
> Key: GOBBLIN-1002
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1002
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-core
>Affects Versions: 0.15.0
>Reporter: Sudarshan Vasudevan
>Assignee: Abhishek Tiwari
>Priority: Major
> Fix For: 0.15.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Current behavior returns the deserialized state without setting the state id
> member field.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (GOBBLIN-1002) Set state id when deserializing state from Gobblin state store
Sudarshan Vasudevan created GOBBLIN-1002: Summary: Set state id when deserializing state from Gobblin state store Key: GOBBLIN-1002 URL: https://issues.apache.org/jira/browse/GOBBLIN-1002 Project: Apache Gobblin Issue Type: Improvement Components: gobblin-core Affects Versions: 0.15.0 Reporter: Sudarshan Vasudevan Assignee: Abhishek Tiwari Fix For: 0.15.0 Current behavior returns the deserialized state without setting the state id member field. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (GOBBLIN-1000) Add min and max LogAppendTime to tracking events emitted from Gobblin Kafka Extractor
[ https://issues.apache.org/jira/browse/GOBBLIN-1000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudarshan Vasudevan resolved GOBBLIN-1000. -- Resolution: Fixed > Add min and max LogAppendTime to tracking events emitted from Gobblin Kafka > Extractor > - > > Key: GOBBLIN-1000 > URL: https://issues.apache.org/jira/browse/GOBBLIN-1000 > Project: Apache Gobblin > Issue Type: Improvement > Components: gobblin-kafka >Affects Versions: 0.15.0 >Reporter: Sudarshan Vasudevan >Assignee: Shirshanka Das >Priority: Major > Fix For: 0.15.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Add min and max logAppendTimes to the tracking events to better track sla > misses. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (GOBBLIN-1000) Add min and max LogAppendTime to tracking events emitted from Gobblin Kafka Extractor
[ https://issues.apache.org/jira/browse/GOBBLIN-1000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudarshan Vasudevan closed GOBBLIN-1000. > Add min and max LogAppendTime to tracking events emitted from Gobblin Kafka > Extractor > - > > Key: GOBBLIN-1000 > URL: https://issues.apache.org/jira/browse/GOBBLIN-1000 > Project: Apache Gobblin > Issue Type: Improvement > Components: gobblin-kafka >Affects Versions: 0.15.0 >Reporter: Sudarshan Vasudevan >Assignee: Shirshanka Das >Priority: Major > Fix For: 0.15.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Add min and max logAppendTimes to the tracking events to better track sla > misses. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (GOBBLIN-1001) Implement TimePartitionGlobFinder
[ https://issues.apache.org/jira/browse/GOBBLIN-1001?focusedWorklogId=357340&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-357340 ] ASF GitHub Bot logged work on GOBBLIN-1001: --- Author: ASF GitHub Bot Created on: 10/Dec/19 19:11 Start Date: 10/Dec/19 19:11 Worklog Time Spent: 10m Work Description: codecov-io commented on issue #2846: [GOBBLIN-1001] Implement TimePartitionGlobFinder URL: https://github.com/apache/incubator-gobblin/pull/2846#issuecomment-564186234 # [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2846?src=pr&el=h1) Report > Merging [#2846](https://codecov.io/gh/apache/incubator-gobblin/pull/2846?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-gobblin/commit/0a1debc30734ec57d2be7351270894756a6c18af?src=pr&el=desc) will **increase** coverage by `0.05%`. > The diff coverage is `96.15%`. [](https://codecov.io/gh/apache/incubator-gobblin/pull/2846?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master#2846 +/- ## + Coverage 45.59% 45.65% +0.05% - Complexity 8984 9002 +18 Files 1904 1907 +3 Lines 7134771396 +49 Branches 7876 7878 +2 + Hits 3253432593 +59 + Misses3580635802 -4 + Partials 3007 3001 -6 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2846?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...anagement/dataset/DefaultFileSystemGlobFinder.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2846/diff?src=pr&el=tree#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2RhdGFzZXQvRGVmYXVsdEZpbGVTeXN0ZW1HbG9iRmluZGVyLmphdmE=) | `100% <100%> (+100%)` | `2 <1> (+2)` | :arrow_up: | | [...ata/management/dataset/EmptyFileSystemDataset.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2846/diff?src=pr&el=tree#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2RhdGFzZXQvRW1wdHlGaWxlU3lzdGVtRGF0YXNldC5qYXZh) | `100% <100%> (ø)` | `1 <1> (?)` | | | [...ta/management/dataset/SimpleFileSystemDataset.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2846/diff?src=pr&el=tree#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2RhdGFzZXQvU2ltcGxlRmlsZVN5c3RlbURhdGFzZXQuamF2YQ==) | `80% <80%> (ø)` | `2 <2> (?)` | | | [...ta/management/dataset/TimePartitionGlobFinder.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2846/diff?src=pr&el=tree#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2RhdGFzZXQvVGltZVBhcnRpdGlvbkdsb2JGaW5kZXIuamF2YQ==) | `97.72% <97.72%> (ø)` | `12 <12> (?)` | | | [...main/java/org/apache/gobblin/yarn/YarnService.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2846/diff?src=pr&el=tree#diff-Z29iYmxpbi15YXJuL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3lhcm4vWWFyblNlcnZpY2UuamF2YQ==) | `14.84% <0%> (-0.85%)` | `3% <0%> (-1%)` | | | [.../org/apache/gobblin/cluster/GobblinTaskRunner.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2846/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpblRhc2tSdW5uZXIuamF2YQ==) | `64.81% <0%> (ø)` | `27% <0%> (-1%)` | :arrow_down: | | [...lin/elasticsearch/writer/FutureCallbackHolder.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2846/diff?src=pr&el=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tZWxhc3RpY3NlYXJjaC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9lbGFzdGljc2VhcmNoL3dyaXRlci9GdXR1cmVDYWxsYmFja0hvbGRlci5qYXZh) | `62.85% <0%> (+1.42%)` | `4% <0%> (ø)` | :arrow_down: | | ... and [6 more](https://codecov.io/gh/apache/incubator-gobblin/pull/2846/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2846?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2846?src=pr&el=footer). Last update [0a1debc...06403c7](https://codecov.io/gh/apache/incubator-gobblin/pull/2846?src=pr&el=lastupdated). Read the [comment docs](
[GitHub] [incubator-gobblin] codecov-io commented on issue #2846: [GOBBLIN-1001] Implement TimePartitionGlobFinder
codecov-io commented on issue #2846: [GOBBLIN-1001] Implement TimePartitionGlobFinder URL: https://github.com/apache/incubator-gobblin/pull/2846#issuecomment-564186234 # [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2846?src=pr&el=h1) Report > Merging [#2846](https://codecov.io/gh/apache/incubator-gobblin/pull/2846?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-gobblin/commit/0a1debc30734ec57d2be7351270894756a6c18af?src=pr&el=desc) will **increase** coverage by `0.05%`. > The diff coverage is `96.15%`. [](https://codecov.io/gh/apache/incubator-gobblin/pull/2846?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master#2846 +/- ## + Coverage 45.59% 45.65% +0.05% - Complexity 8984 9002 +18 Files 1904 1907 +3 Lines 7134771396 +49 Branches 7876 7878 +2 + Hits 3253432593 +59 + Misses3580635802 -4 + Partials 3007 3001 -6 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2846?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...anagement/dataset/DefaultFileSystemGlobFinder.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2846/diff?src=pr&el=tree#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2RhdGFzZXQvRGVmYXVsdEZpbGVTeXN0ZW1HbG9iRmluZGVyLmphdmE=) | `100% <100%> (+100%)` | `2 <1> (+2)` | :arrow_up: | | [...ata/management/dataset/EmptyFileSystemDataset.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2846/diff?src=pr&el=tree#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2RhdGFzZXQvRW1wdHlGaWxlU3lzdGVtRGF0YXNldC5qYXZh) | `100% <100%> (ø)` | `1 <1> (?)` | | | [...ta/management/dataset/SimpleFileSystemDataset.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2846/diff?src=pr&el=tree#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2RhdGFzZXQvU2ltcGxlRmlsZVN5c3RlbURhdGFzZXQuamF2YQ==) | `80% <80%> (ø)` | `2 <2> (?)` | | | [...ta/management/dataset/TimePartitionGlobFinder.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2846/diff?src=pr&el=tree#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2RhdGFzZXQvVGltZVBhcnRpdGlvbkdsb2JGaW5kZXIuamF2YQ==) | `97.72% <97.72%> (ø)` | `12 <12> (?)` | | | [...main/java/org/apache/gobblin/yarn/YarnService.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2846/diff?src=pr&el=tree#diff-Z29iYmxpbi15YXJuL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3lhcm4vWWFyblNlcnZpY2UuamF2YQ==) | `14.84% <0%> (-0.85%)` | `3% <0%> (-1%)` | | | [.../org/apache/gobblin/cluster/GobblinTaskRunner.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2846/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpblRhc2tSdW5uZXIuamF2YQ==) | `64.81% <0%> (ø)` | `27% <0%> (-1%)` | :arrow_down: | | [...lin/elasticsearch/writer/FutureCallbackHolder.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2846/diff?src=pr&el=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tZWxhc3RpY3NlYXJjaC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9lbGFzdGljc2VhcmNoL3dyaXRlci9GdXR1cmVDYWxsYmFja0hvbGRlci5qYXZh) | `62.85% <0%> (+1.42%)` | `4% <0%> (ø)` | :arrow_down: | | ... and [6 more](https://codecov.io/gh/apache/incubator-gobblin/pull/2846/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2846?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2846?src=pr&el=footer). Last update [0a1debc...06403c7](https://codecov.io/gh/apache/incubator-gobblin/pull/2846?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Work logged] (GOBBLIN-998) ExecutionStatus should be reset to PENDING before a job retries
[ https://issues.apache.org/jira/browse/GOBBLIN-998?focusedWorklogId=357336&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-357336 ] ASF GitHub Bot logged work on GOBBLIN-998: -- Author: ASF GitHub Bot Created on: 10/Dec/19 18:58 Start Date: 10/Dec/19 18:58 Worklog Time Spent: 10m Work Description: asfgit commented on pull request #2843: [GOBBLIN-998] ExecutionStatus should be reset to PENDING before a job retries URL: https://github.com/apache/incubator-gobblin/pull/2843 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 357336) Time Spent: 50m (was: 40m) > ExecutionStatus should be reset to PENDING before a job retries > --- > > Key: GOBBLIN-998 > URL: https://issues.apache.org/jira/browse/GOBBLIN-998 > Project: Apache Gobblin > Issue Type: Bug >Reporter: Chen Guo >Priority: Critical > Time Spent: 50m > Remaining Estimate: 0h > > In the modifyStateIfRetryRequired of KafkaJobStatusMonitor, when the state is > Failed and currentAttempts < maxAttempts, the ExecutionStatus is set to > Running. > However, due to the checkin from > GOBBLIN-974([https://github.com/apache/incubator-gobblin/blob/9f50a2563cc257039da44018663b6b9e119fb499/gobblin-service/src/main/java/org/apache/gobblin/service/monitoring/KafkaJobStatusMonitor.java#L159]), > the currentAttempts update from a lower-order event(like Orchestrated) > cannot be consumed to update the jobState file. Thus it will cause infinite > retries in DagManagerThread for failed jobs when it poolAndAdvanceDag. > > The solution is to update ExecutionStatus to PENDING instead of Running. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [incubator-gobblin] asfgit closed pull request #2843: [GOBBLIN-998] ExecutionStatus should be reset to PENDING before a job retries
asfgit closed pull request #2843: [GOBBLIN-998] ExecutionStatus should be reset to PENDING before a job retries URL: https://github.com/apache/incubator-gobblin/pull/2843 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Work logged] (GOBBLIN-1001) Implement TimePartitionGlobFinder
[
https://issues.apache.org/jira/browse/GOBBLIN-1001?focusedWorklogId=357315&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-357315
]
ASF GitHub Bot logged work on GOBBLIN-1001:
---
Author: ASF GitHub Bot
Created on: 10/Dec/19 18:21
Start Date: 10/Dec/19 18:21
Worklog Time Spent: 10m
Work Description: zxcware commented on pull request #2846: [GOBBLIN-1001]
Implement TimePartitionGlobFinder
URL: https://github.com/apache/incubator-gobblin/pull/2846
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I
have checked off all the steps below!
### JIRA
- [ ] My PR addresses the following [Gobblin
JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references
them in the PR title.
- https://issues.apache.org/jira/browse/GOBBLIN-1001
### Description
- [x] Here are some details about my PR:
- A `TimePartitionGlobFinder` has the capability to create empty file
system dataset if a partition has empty data
### Tests
- [x] My PR adds the following unit tests:
- `TimePartitionGlobFinder.testDayPartition` covers an empty day partition
dataset is created when `TimePartitionGlobFinder.enableEmptyPartition` is true.
### Commits
- [x] My commits all reference JIRA issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not "adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 357315)
Remaining Estimate: 0h
Time Spent: 10m
> Implement TimePartitionGlobFinder
> -
>
> Key: GOBBLIN-1001
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1001
> Project: Apache Gobblin
> Issue Type: Task
>Reporter: Zhixiong Chen
>Assignee: Zhixiong Chen
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [incubator-gobblin] zxcware opened a new pull request #2846: [GOBBLIN-1001] Implement TimePartitionGlobFinder
zxcware opened a new pull request #2846: [GOBBLIN-1001] Implement
TimePartitionGlobFinder
URL: https://github.com/apache/incubator-gobblin/pull/2846
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I
have checked off all the steps below!
### JIRA
- [ ] My PR addresses the following [Gobblin
JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references
them in the PR title.
- https://issues.apache.org/jira/browse/GOBBLIN-1001
### Description
- [x] Here are some details about my PR:
- A `TimePartitionGlobFinder` has the capability to create empty file
system dataset if a partition has empty data
### Tests
- [x] My PR adds the following unit tests:
- `TimePartitionGlobFinder.testDayPartition` covers an empty day partition
dataset is created when `TimePartitionGlobFinder.enableEmptyPartition` is true.
### Commits
- [x] My commits all reference JIRA issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not "adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (GOBBLIN-1001) Implement TimePartitionGlobFinder
Zhixiong Chen created GOBBLIN-1001: -- Summary: Implement TimePartitionGlobFinder Key: GOBBLIN-1001 URL: https://issues.apache.org/jira/browse/GOBBLIN-1001 Project: Apache Gobblin Issue Type: Task Reporter: Zhixiong Chen Assignee: Zhixiong Chen -- This message was sent by Atlassian Jira (v8.3.4#803005)
