Dave Hagman created HUDI-6339:
-
Summary: Ability to Disable Partition Deletes during Clean
Key: HUDI-6339
URL: https://issues.apache.org/jira/browse/HUDI-6339
Project: Apache Hudi
Issue Type:
[
https://issues.apache.org/jira/browse/HUDI-6339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Hagman updated HUDI-6339:
--
Description:
We recently experienced a large data loss in one of our largest Hudi tables. We
observed
[
https://issues.apache.org/jira/browse/HUDI-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496163#comment-17496163
]
Dave Hagman edited comment on HUDI-1307 at 2/22/22, 3:40 PM:
-
[~vinoth]
[
https://issues.apache.org/jira/browse/HUDI-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496163#comment-17496163
]
Dave Hagman commented on HUDI-1307:
---
[~vinoth] [~309637554] [~xushiyan] I'd like to +1 what [~vho]
[
https://issues.apache.org/jira/browse/HUDI-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17482553#comment-17482553
]
Dave Hagman commented on HUDI-2173:
---
[~shivnarayan] I did not yet get a chance to run tests on that but
[
https://issues.apache.org/jira/browse/HUDI-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446559#comment-17446559
]
Dave Hagman commented on HUDI-2173:
---
Will do!
> Enhancing DynamoDB based LockProvider
>
[
https://issues.apache.org/jira/browse/HUDI-2549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Hagman closed HUDI-2549.
-
Resolution: Duplicate
> Exceptions when using second writer into Hudi table managed by DeltaStreamer
>
Dave Hagman created HUDI-2579:
-
Summary: Deltastreamer checkpoint metadata is not merged from
previous commit instant
Key: HUDI-2579
URL: https://issues.apache.org/jira/browse/HUDI-2579
Project: Apache
[
https://issues.apache.org/jira/browse/HUDI-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430050#comment-17430050
]
Dave Hagman commented on HUDI-2559:
---
Testing approach 1 should be very easy given the way my branch is
[
https://issues.apache.org/jira/browse/HUDI-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430048#comment-17430048
]
Dave Hagman edited comment on HUDI-2559 at 10/18/21, 2:40 PM:
--
I have been
[
https://issues.apache.org/jira/browse/HUDI-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430048#comment-17430048
]
Dave Hagman commented on HUDI-2559:
---
I have been extensively testing approach #2 and so far it has
[
https://issues.apache.org/jira/browse/HUDI-2549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17428798#comment-17428798
]
Dave Hagman commented on HUDI-2549:
---
OK so you only ran 1 iteration of each (1 commit from each)? This
[
https://issues.apache.org/jira/browse/HUDI-2549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17428499#comment-17428499
]
Dave Hagman commented on HUDI-2549:
---
While continuing to test, I found that the
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Hagman closed HUDI-2275.
-
Fix Version/s: (was: 0.10.0)
0.9.0
Resolution: Fixed
>
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427809#comment-17427809
]
Dave Hagman commented on HUDI-2275:
---
We are experiencing new issues when migrating to version 0.9 so I
[
https://issues.apache.org/jira/browse/HUDI-2549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427807#comment-17427807
]
Dave Hagman commented on HUDI-2549:
---
In order to try and validate my hypothesis about race conditions I
Dave Hagman created HUDI-2549:
-
Summary: Exceptions when using second writer into Hudi table
managed by DeltaStreamer
Key: HUDI-2549
URL: https://issues.apache.org/jira/browse/HUDI-2549
Project: Apache
[
https://issues.apache.org/jira/browse/HUDI-2549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Hagman updated HUDI-2549:
--
Description:
When running the DeltaStreamer along with a second spark datasource writer
(with
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425773#comment-17425773
]
Dave Hagman commented on HUDI-2275:
---
Also I am not specifying a value for _hoodie.metadata.enable_ so
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425756#comment-17425756
]
Dave Hagman commented on HUDI-2275:
---
[~shivnarayan] I received the same error (above) even when running
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425741#comment-17425741
]
Dave Hagman commented on HUDI-2275:
---
I did not try that patch. I will do that and report back. FYI with
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17424154#comment-17424154
]
Dave Hagman edited comment on HUDI-2275 at 10/4/21, 8:09 PM:
-
[~vinoth] I'd
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17424154#comment-17424154
]
Dave Hagman commented on HUDI-2275:
---
[~vinoth] I'd argue that this is still a blocker as it completely
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398090#comment-17398090
]
Dave Hagman edited comment on HUDI-2275 at 8/12/21, 2:46 PM:
-
Continuing to
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398090#comment-17398090
]
Dave Hagman commented on HUDI-2275:
---
Continuing to investigate this and found something potentially
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17397979#comment-17397979
]
Dave Hagman commented on HUDI-2275:
---
I have run into the same issue even with the prefixes config set on
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17397582#comment-17397582
]
Dave Hagman commented on HUDI-2275:
---
Perfect tyvm. I will try this now!
> HoodieDeltaStreamerException
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17397454#comment-17397454
]
Dave Hagman edited comment on HUDI-2275 at 8/11/21, 4:19 PM:
-
[~vinoth] just
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17397454#comment-17397454
]
Dave Hagman commented on HUDI-2275:
---
[~vinoth] just so I understand, this configuration should be set
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17397328#comment-17397328
]
Dave Hagman commented on HUDI-2275:
---
I will try that this morning and report back.
>
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17396830#comment-17396830
]
Dave Hagman edited comment on HUDI-2275 at 8/10/21, 6:25 PM:
-
[~vinoth] OK
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17396830#comment-17396830
]
Dave Hagman commented on HUDI-2275:
---
[~vinoth] OK that at least ties together what I was missing around
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394190#comment-17394190
]
Dave Hagman commented on HUDI-2275:
---
Comment in Slack from Shiv Narayan:
{code:java}
here is my
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Hagman updated HUDI-2275:
--
Description:
I am trying to utilize [Optimistic Concurrency
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Hagman updated HUDI-2275:
--
Description:
I am trying to utilize [Optimistic Concurrency
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Hagman updated HUDI-2275:
--
Description:
I am trying to utilize [Optimistic Concurrency
[
https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Hagman updated HUDI-2275:
--
Description:
I am trying to utilize [Optimistic Concurrency
Dave Hagman created HUDI-2275:
-
Summary: HoodieDeltaStreamerException when using OCC and a second
concurrent writer
Key: HUDI-2275
URL: https://issues.apache.org/jira/browse/HUDI-2275
Project: Apache
[
https://issues.apache.org/jira/browse/HUDI-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Hagman reassigned HUDI-2230:
-
Assignee: Dave Hagman
> "Task not serializable" exception due to non-serializable Codahale
[
https://issues.apache.org/jira/browse/HUDI-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Hagman updated HUDI-2230:
--
Description:
Steps to reproduce:
* Enable graphite metrics via props file. Example:
{noformat}
[
https://issues.apache.org/jira/browse/HUDI-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Hagman updated HUDI-2230:
--
Description:
Steps to reproduce:
* Enable graphite metrics via props file. Example:
{noformat}
Dave Hagman created HUDI-2230:
-
Summary: "Task not serializable" exception due to non-serializable
Codahale Timers
Key: HUDI-2230
URL: https://issues.apache.org/jira/browse/HUDI-2230
Project: Apache Hudi
[
https://issues.apache.org/jira/browse/HUDI-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382962#comment-17382962
]
Dave Hagman edited comment on HUDI-2173 at 7/19/21, 2:58 AM:
-
Very basic
[
https://issues.apache.org/jira/browse/HUDI-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382962#comment-17382962
]
Dave Hagman edited comment on HUDI-2173 at 7/19/21, 2:57 AM:
-
Very basic
[
https://issues.apache.org/jira/browse/HUDI-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382962#comment-17382962
]
Dave Hagman commented on HUDI-2173:
---
Very basic pseudocode for an optimistic locking implementation:
[
https://issues.apache.org/jira/browse/HUDI-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382961#comment-17382961
]
Dave Hagman edited comment on HUDI-2173 at 7/19/21, 2:42 AM:
-
I have started
[
https://issues.apache.org/jira/browse/HUDI-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382961#comment-17382961
]
Dave Hagman commented on HUDI-2173:
---
I have started to look into the various options we have when
[
https://issues.apache.org/jira/browse/HUDI-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Hagman updated HUDI-2173:
--
Status: In Progress (was: Open)
> Implement a DynamoDB based LockProvider
>
48 matches
Mail list logo
