vinothchandar commented on pull request #2485:
URL: https://github.com/apache/hudi/pull/2485#issuecomment-766593559
cc @garyli1019 mind taking a first pass at this PR? :)
This is an automated message from the Apache Git Serv
vinothchandar commented on issue #1829:
URL: https://github.com/apache/hudi/issues/1829#issuecomment-766590769
@rubenssoto for some code paths, it will be. if you turn on
`hoodie.metadata.enable=true` on the writing, you should see improvements. Hive
queries should see improvement, SparkSQ
pengzhiwei2018 commented on pull request #1880:
URL: https://github.com/apache/hudi/pull/1880#issuecomment-766562247
> Hello,
>
> Hudi will have nice features like clustering and clustering probably will
rewrite a lot of data, so is it possible this rewrites without new data doesn't
git-raj commented on issue #2284:
URL: https://github.com/apache/hudi/issues/2284#issuecomment-766523668
using AWS Glue pySpark and Hudi and S3 as data store: i'm trying to do the
traditional SCD Type 2 where old record gets updated with the insert datetime
on 'effective to' field, 'isActi
codecov-io edited a comment on pull request #2485:
URL: https://github.com/apache/hudi/pull/2485#issuecomment-766519181
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2485?src=pr&el=h1) Report
> Merging
[#2485](https://codecov.io/gh/apache/hudi/pull/2485?src=pr&el=desc) (91cf083)
in
codecov-io edited a comment on pull request #2485:
URL: https://github.com/apache/hudi/pull/2485#issuecomment-766519181
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2485?src=pr&el=h1) Report
> Merging
[#2485](https://codecov.io/gh/apache/hudi/pull/2485?src=pr&el=desc) (fa0056e)
in
codecov-io edited a comment on pull request #2485:
URL: https://github.com/apache/hudi/pull/2485#issuecomment-766519181
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2485?src=pr&el=h1) Report
> Merging
[#2485](https://codecov.io/gh/apache/hudi/pull/2485?src=pr&el=desc) (fa0056e)
in
codecov-io edited a comment on pull request #2485:
URL: https://github.com/apache/hudi/pull/2485#issuecomment-766519181
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2485?src=pr&el=h1) Report
> Merging
[#2485](https://codecov.io/gh/apache/hudi/pull/2485?src=pr&el=desc) (5d3ec8d)
in
codecov-io commented on pull request #2485:
URL: https://github.com/apache/hudi/pull/2485#issuecomment-766519181
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2485?src=pr&el=h1) Report
> Merging
[#2485](https://codecov.io/gh/apache/hudi/pull/2485?src=pr&el=desc) (91cf083)
into
[ma
[
https://issues.apache.org/jira/browse/HUDI-1109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1109:
-
Labels: pull-request-available (was: )
> Support Spark Structured Streaming read from Hudi table
pengzhiwei2018 opened a new pull request #2485:
URL: https://github.com/apache/hudi/pull/2485
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a
pull request.*
## What is the purpose of
rubenssoto edited a comment on issue #1829:
URL: https://github.com/apache/hudi/issues/1829#issuecomment-766496187
@umehrot2 @bvaradar
Do you know if this problem will be solved in 0.7.0? I'm querying some big
datasets with more than 500 partitions and I had the same problem.
rubenssoto edited a comment on issue #1829:
URL: https://github.com/apache/hudi/issues/1829#issuecomment-766496187
@umehrot2 @bvaradar
Do you know if this problem will be solved in 0.7.0? I'm querying some big
datasets with more than 500 partitions and I had the same problem.
rubenssoto commented on issue #1829:
URL: https://github.com/apache/hudi/issues/1829#issuecomment-766496187
@umehrot2 @bvaradar
Do you know if this problem will be solved in 0.7.0? I'm querying some big
datasets with more than 500 partitions and I had the same problem.
Thank
codecov-io edited a comment on pull request #2382:
URL: https://github.com/apache/hudi/pull/2382#issuecomment-751367927
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2382?src=pr&el=h1) Report
> Merging
[#2382](https://codecov.io/gh/apache/hudi/pull/2382?src=pr&el=desc) (498109c)
in
zherenyu831 commented on issue #2285:
URL: https://github.com/apache/hudi/issues/2285#issuecomment-766482729
@bvaradar
Hi Bavaradar, it will be little difficult to replicate the problem, since it
only happens on huge amount of data.
vinothchandar commented on issue #2285:
URL: https://github.com/apache/hudi/issues/2285#issuecomment-766450275
cc @garyli1019 as well
This is an automated message from the Apache Git Service.
To respond to the message, please
nsivabalan commented on issue #2063:
URL: https://github.com/apache/hudi/issues/2063#issuecomment-766449860
@cadl : did you get a chance to try out the setting? We plan to close out
this issue due to inactivity in a weeks time. But feel free to reopen to create
a new ticket if you find any
nsivabalan commented on issue #2066:
URL: https://github.com/apache/hudi/issues/2066#issuecomment-766449665
@KarthickAN : did you get a chance to try out the suggestion from Balaji.
please do update the issue w/ any updates. If the issue is resolved, feel free
to close it out.
-
vinothchandar commented on issue #2330:
URL: https://github.com/apache/hudi/issues/2330#issuecomment-766441408
we can close this out
This is an automated message from the Apache Git Service.
To respond to the message, please
vinothchandar closed issue #2330:
URL: https://github.com/apache/hudi/issues/2330
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to th
[
https://issues.apache.org/jira/browse/HUDI-1549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271000#comment-17271000
]
Vinoth Chandar commented on HUDI-1549:
--
[~shivnarayan] the label `user-sup` is wrong?
[
https://issues.apache.org/jira/browse/HUDI-1549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Chandar updated HUDI-1549:
-
Labels: user-sup user-support-issues (was: user-sup)
> Programmatic way to fetch earliest commit
nsivabalan edited a comment on issue #2100:
URL: https://github.com/apache/hudi/issues/2100#issuecomment-766440534
@n3nash @bhasudha : sorry the thread is bit long, so couldn't gauge
correctly. I see some workarounds have been proposed and it worked. But do we
need to fixes in Hudi in gene
nsivabalan commented on issue #2100:
URL: https://github.com/apache/hudi/issues/2100#issuecomment-766440534
@n3nash @bhasudha : sorry the thread is bit long. I see some workarounds
have been proposed and it worked. But do we need to fixes in Hudi in general?
if yes, can you file a jira and
nsivabalan edited a comment on issue #2121:
URL: https://github.com/apache/hudi/issues/2121#issuecomment-766439932
@liujinhui1994 : Sorry about the delay. We already have an [example in our
HoodieTestDatagenerator](https://github.com/apache/hudi/blob/c4afd179c1983a382b8a5197d800b0f5dba254de
nsivabalan commented on issue #2121:
URL: https://github.com/apache/hudi/issues/2121#issuecomment-766439932
@liujinhui1994 : We already have an [example in our
HoodieTestDatagenerator](https://github.com/apache/hudi/blob/c4afd179c1983a382b8a5197d800b0f5dba254de/hudi-common/src/test/java/org
nsivabalan commented on issue #2123:
URL: https://github.com/apache/hudi/issues/2123#issuecomment-766439219
@satishkotha : when you get a chance, can you please follow up on this.
This is an automated message from the Apache
nsivabalan commented on issue #2135:
URL: https://github.com/apache/hudi/issues/2135#issuecomment-766439085
@andaag : I have created a Hudi ticket for this. Feel free to update the
desc of the ticket with more details
https://issues.apache.org/jira/browse/HUDI-1549
sivabalan narayanan created HUDI-1549:
-
Summary: Programmatic way to fetch earliest commit retained
Key: HUDI-1549
URL: https://issues.apache.org/jira/browse/HUDI-1549
Project: Apache Hudi
[
https://issues.apache.org/jira/browse/HUDI-1549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-1549:
--
Labels: user-sup (was: )
> Programmatic way to fetch earliest commit retained
> --
[
https://issues.apache.org/jira/browse/HUDI-1549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-1549:
--
Fix Version/s: 0.8.0
> Programmatic way to fetch earliest commit retained
> ---
nsivabalan commented on issue #2178:
URL: https://github.com/apache/hudi/issues/2178#issuecomment-766438221
@KarthickAN : hope you got a chance to go through our [blog on indexes in
Hudi](https://hudi.apache.org/blog/hudi-indexing-mechanisms/). Wrt this gh
issue, please do let us know if y
nsivabalan commented on issue #2204:
URL: https://github.com/apache/hudi/issues/2204#issuecomment-766437535
@BalaMahesh : Would you mind updating the ticket. We will close this out in
a weeks time if there are no activity. But feel free to re-open or create a new
ticket if you have more qu
nsivabalan commented on issue #2284:
URL: https://github.com/apache/hudi/issues/2284#issuecomment-766436747
@sanket-khedikar : can you please respond if the suggested approaches work
for you. or you still need more enhancements from Hudi? If it's solved, would
appreciate if you can close t
nsivabalan edited a comment on issue #2285:
URL: https://github.com/apache/hudi/issues/2285#issuecomment-766436364
@zherenyu831 : can you please respond with any updates on your end.
@n3nash : can you please take a look when you have time. If you were able to
narrow down the issue, plea
nsivabalan commented on issue #2285:
URL: https://github.com/apache/hudi/issues/2285#issuecomment-766436364
@zherenyu831 : can you please respond with any updates on your end.
@n3nash : can you take a look when you have time.
---
nsivabalan commented on issue #2323:
URL: https://github.com/apache/hudi/issues/2323#issuecomment-766435871
@Kirkuz: Do you have any updates in this regard. Can you please respond or
let us know if you have more questions.
[
https://issues.apache.org/jira/browse/HUDI-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-1460:
--
Labels: user-support-issues (was: )
> Time Travel (querying the historical versions of
nsivabalan commented on issue #2329:
URL: https://github.com/apache/hudi/issues/2329#issuecomment-766435383
https://issues.apache.org/jira/browse/HUDI-1460
This is an automated message from the Apache Git Service.
To resp
nsivabalan commented on issue #2330:
URL: https://github.com/apache/hudi/issues/2330#issuecomment-766433970
@vinothchandar @borislitvak : since we have a tracking jira, do you think
we can close this? or is there anything pending to be resolved or discussed.
sivabalan narayanan created HUDI-1548:
-
Summary: Fix documentation around schema evolution
Key: HUDI-1548
URL: https://issues.apache.org/jira/browse/HUDI-1548
Project: Apache Hudi
Issue
[
https://issues.apache.org/jira/browse/HUDI-1548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-1548:
--
Labels: user-support-issues (was: )
> Fix documentation around schema evolution
>
nsivabalan commented on issue #2331:
URL: https://github.com/apache/hudi/issues/2331#issuecomment-766432877
@prashantwason : In lieu of this ticket, do you think we can update our
documentation wrt schema evolution. If you don't mind can you take it up and
fix our documentation. https://i
[
https://issues.apache.org/jira/browse/HUDI-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-1539:
--
Affects Version/s: 0.8.0
> Bug in HoodieCombineRealtimeRecordReader returns wrong result
[
https://issues.apache.org/jira/browse/HUDI-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-1539:
--
Labels: user-support-issues (was: )
> Bug in HoodieCombineRealtimeRecordReader returns
nsivabalan commented on issue #2367:
URL: https://github.com/apache/hudi/issues/2367#issuecomment-766431687
Sure. sorry about the delay. will get to this in a day or two.
This is an automated message from the Apache Git Serv
nsivabalan commented on issue #2399:
URL: https://github.com/apache/hudi/issues/2399#issuecomment-766431496
@afeldman1 : can you respond when you can.
This is an automated message from the Apache Git Service.
To respond to t
nsivabalan closed issue #2429:
URL: https://github.com/apache/hudi/issues/2429
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the s
nsivabalan commented on issue #2429:
URL: https://github.com/apache/hudi/issues/2429#issuecomment-766428773
@vinothchandar : closing this for now. feel free to re-open if you see more
issues.
This is an automated message fr
[
https://issues.apache.org/jira/browse/HUDI-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-1528:
--
Labels: pull-request-available user-support-issues (was:
pull-request-available)
> hud
[
https://issues.apache.org/jira/browse/HUDI-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-1547:
--
Labels: user-support-issues (was: )
> CI intermittent failure:
> TestJsonStringToHoodi
sivabalan narayanan created HUDI-1547:
-
Summary: CI intermittent failure:
TestJsonStringToHoodieRecordMapFunction.testMapFunction
Key: HUDI-1547
URL: https://issues.apache.org/jira/browse/HUDI-1547
nsivabalan commented on issue #2467:
URL: https://github.com/apache/hudi/issues/2467#issuecomment-766427684
Have created a tracking jira https://issues.apache.org/jira/browse/HUDI-1547
This is an automated message from th
nsivabalan commented on issue #2480:
URL: https://github.com/apache/hudi/issues/2480#issuecomment-766427153
Sure, will take it up.
Closing it as we have a tracking jira.
https://issues.apache.org/jira/browse/HUDI-1546
---
nsivabalan closed issue #2480:
URL: https://github.com/apache/hudi/issues/2480
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the s
sivabalan narayanan created HUDI-1546:
-
Summary: Fix hive sync tool path in website documentation
Key: HUDI-1546
URL: https://issues.apache.org/jira/browse/HUDI-1546
Project: Apache Hudi
[
https://issues.apache.org/jira/browse/HUDI-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-1546:
--
Labels: user-support-issues (was: )
> Fix hive sync tool path in website documentation
[
https://issues.apache.org/jira/browse/HUDI-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-1070:
--
Labels: (was: user-support-issues)
> Direct write from spark to Parquet when doing Ups
[
https://issues.apache.org/jira/browse/HUDI-1290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17270969#comment-17270969
]
sivabalan narayanan commented on HUDI-1290:
---
Does this quality to be labeled as
[
https://issues.apache.org/jira/browse/HUDI-1505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-1505:
--
Labels: (was: user-support-issues)
> Allow pluggable option to write error records to
[
https://issues.apache.org/jira/browse/HUDI-849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17270966#comment-17270966
]
sivabalan narayanan commented on HUDI-849:
--
[~vbalaji]: this is more of a todo thi
[
https://issues.apache.org/jira/browse/HUDI-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17270965#comment-17270965
]
sivabalan narayanan commented on HUDI-1278:
---
[~vbalaji]: Can you clarify the req
This is an automated email from the ASF dual-hosted git repository.
xushiyan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new c4afd17 [HUDI-1476] Introduce unit test infra f
xushiyan merged pull request #2478:
URL: https://github.com/apache/hudi/pull/2478
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to th
rubenssoto opened a new issue #2484:
URL: https://github.com/apache/hudi/issues/2484
Hello,
I want to start using Hudi on my datalake, so I'm running some performance
tests comparing current processing time with and without Hudi. We have a lot of
tables in our datalake so we are pro
vinothchandar merged pull request #2481:
URL: https://github.com/apache/hudi/pull/2481
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
This is an automated email from the ASF dual-hosted git repository.
vinoth pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new 81836f0 Removing spring repos from pom (#2481)
81
vinothchandar commented on issue #2479:
URL: https://github.com/apache/hudi/issues/2479#issuecomment-766369989
Great. No thank you for catching :). eventually as m2 caches are lost, I
think build would have failed. may be month or so from now :). Will merge the
fix
-
rubenssoto closed issue #2479:
URL: https://github.com/apache/hudi/issues/2479
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the s
rubenssoto commented on issue #2479:
URL: https://github.com/apache/hudi/issues/2479#issuecomment-766352035
Thank you so much @vinothchandar !
It worked, I will close this issue.
This is an automated me
This is an automated email from the ASF dual-hosted git repository.
leesf pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git.
from e302c6b [HUDI-1453] Fix NPE using HoodieFlinkStreamer to etl data
from kafka to hudi (#2474)
add 84df263 [MINOR]
leesf merged pull request #2477:
URL: https://github.com/apache/hudi/pull/2477
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the s
codecov-io edited a comment on pull request #2478:
URL: https://github.com/apache/hudi/pull/2478#issuecomment-765971598
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2478?src=pr&el=h1) Report
> Merging
[#2478](https://codecov.io/gh/apache/hudi/pull/2478?src=pr&el=desc) (0a53aa4)
in
codecov-io edited a comment on pull request #2478:
URL: https://github.com/apache/hudi/pull/2478#issuecomment-765971598
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitH
codecov-io edited a comment on pull request #2483:
URL: https://github.com/apache/hudi/pull/2483#issuecomment-766306624
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2483?src=pr&el=h1) Report
> Merging
[#2483](https://codecov.io/gh/apache/hudi/pull/2483?src=pr&el=desc) (ba566c2)
in
codecov-io edited a comment on pull request #2483:
URL: https://github.com/apache/hudi/pull/2483#issuecomment-766306624
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitH
codecov-io edited a comment on pull request #2478:
URL: https://github.com/apache/hudi/pull/2478#issuecomment-765971598
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2478?src=pr&el=h1) Report
> Merging
[#2478](https://codecov.io/gh/apache/hudi/pull/2478?src=pr&el=desc) (0a53aa4)
in
codecov-io edited a comment on pull request #2483:
URL: https://github.com/apache/hudi/pull/2483#issuecomment-766306624
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2483?src=pr&el=h1) Report
> Merging
[#2483](https://codecov.io/gh/apache/hudi/pull/2483?src=pr&el=desc) (ba566c2)
in
codecov-io edited a comment on pull request #2478:
URL: https://github.com/apache/hudi/pull/2478#issuecomment-765971598
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2478?src=pr&el=h1) Report
> Merging
[#2478](https://codecov.io/gh/apache/hudi/pull/2478?src=pr&el=desc) (0a53aa4)
in
shenh062326 commented on a change in pull request #2478:
URL: https://github.com/apache/hudi/pull/2478#discussion_r563262860
##
File path:
hudi-client/hudi-java-client/src/test/java/org/apache/hudi/testutils/HoodieJavaClientTestUtils.java
##
@@ -0,0 +1,313 @@
+/*
+ * Licensed
shenh062326 commented on a change in pull request #2478:
URL: https://github.com/apache/hudi/pull/2478#discussion_r563262820
##
File path:
hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieTestUtils.java
##
@@ -107,6 +111,28 @@ public static HoodieTableMetaClien
codecov-io edited a comment on pull request #2483:
URL: https://github.com/apache/hudi/pull/2483#issuecomment-766306624
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2483?src=pr&el=h1) Report
> Merging
[#2483](https://codecov.io/gh/apache/hudi/pull/2483?src=pr&el=desc) (ba566c2)
in
vinothchandar commented on issue #2479:
URL: https://github.com/apache/hudi/issues/2479#issuecomment-766312107
Hi @rubenssoto #2481 , could you give this a shot? if it works, we can just
land that PR
This is an automated mes
codecov-io edited a comment on pull request #2483:
URL: https://github.com/apache/hudi/pull/2483#issuecomment-766306624
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2483?src=pr&el=h1) Report
> Merging
[#2483](https://codecov.io/gh/apache/hudi/pull/2483?src=pr&el=desc) (b38)
in
85 matches
Mail list logo
