danny0405 commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-834118649
> Looking at the usage of `createEmbeddedTimelineService()`, there are
broadly two.
>
> * AbstractHoodieClient can start (during intiialization) and stop (during
close())
danny0405 commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-834117293
> AbstractHoodieClient
They are not singleton, as i said, each writer task takes a
AbstractWriteClient, and there may be multiple write tasks in one process.
I agree
vinothchandar commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-834116492
>This change is introducing another instance which will double the overhead.
What benefits do you see by doing this ?
Would it though? I mean, its just an object refere
ssdong commented on pull request #2784:
URL: https://github.com/apache/hudi/pull/2784#issuecomment-834115800
@vinothchandar No problem! Thank you for your precious time and let me know
if you have a second thought on this fix. :)
--
This is an automated message from the Apache Git Servi
[
https://issues.apache.org/jira/browse/HUDI-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1872:
-
Labels: pull-request-available (was: )
> Move HoodieFlinkStreamer into hudi-utilities module
> --
MyLanPangzi opened a new pull request #2922:
URL: https://github.com/apache/hudi/pull/2922
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a
pull request.*
## What is the purpose of the
yanghua commented on issue #2883:
URL: https://github.com/apache/hudi/issues/2883#issuecomment-834111741
> @yanghua @leesf @dannyhchen : can one of you folks take this up
Yes, you can change or rewrite your `HoodieFlinkStreamer` so that you can
replace the source or do more things.
dirksan28 edited a comment on pull request #2012:
URL: https://github.com/apache/hudi/pull/2012#issuecomment-834091378
I've been observing this issue for several months now. From my point of view
a working schema evolution is highly fundamental for the meaningful use of
hudi. I am amazed t
codecov-commenter edited a comment on pull request #2920:
URL: https://github.com/apache/hudi/pull/2920#issuecomment-834095568
#
[Codecov](https://codecov.io/gh/apache/hudi/pull/2920?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The
dirksan28 edited a comment on pull request #2012:
URL: https://github.com/apache/hudi/pull/2012#issuecomment-834091378
I've been observing this issue for several months now. From my point of view
working schema evolution is highly fundamental for the meaningful use of hudi.
I am amazed tha
danny0405 commented on pull request #2904:
URL: https://github.com/apache/hudi/pull/2904#issuecomment-834095894
The test would be removed in HUDI-1878
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to g
codecov-commenter commented on pull request #2920:
URL: https://github.com/apache/hudi/pull/2920#issuecomment-834095568
#
[Codecov](https://codecov.io/gh/apache/hudi/pull/2920?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache
[
https://issues.apache.org/jira/browse/HUDI-1880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1880:
-
Labels: pull-request-available (was: )
> Support streaming read with compaction and cleaning
> --
danny0405 opened a new pull request #2921:
URL: https://github.com/apache/hudi/pull/2921
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a
pull request.*
## What is the purpose of the p
[
https://issues.apache.org/jira/browse/HUDI-1880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen updated HUDI-1880:
-
Summary: Support streaming read with compaction and cleaning (was: Support
flink streaming read with comp
Danny Chen created HUDI-1880:
Summary: Support flink streaming read with compaction and cleaning
Key: HUDI-1880
URL: https://issues.apache.org/jira/browse/HUDI-1880
Project: Apache Hudi
Issue Typ
[
https://issues.apache.org/jira/browse/HUDI-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1878:
-
Labels: pull-request-available (was: )
> Add max memory option for flink writer task
> --
danny0405 opened a new pull request #2920:
URL: https://github.com/apache/hudi/pull/2920
Also removes the rate limiter because it has the similar functionality.
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributi
dirksan28 commented on pull request #2012:
URL: https://github.com/apache/hudi/pull/2012#issuecomment-834091378
I've been observing this issue for several months now. From my point of view
that is a big problem for the meaningful use of hudi. I am amazed that this is
not classified as a mi
vinothchandar commented on pull request #2012:
URL: https://github.com/apache/hudi/pull/2012#issuecomment-834087051
Actually pretty bummed to hear that. @nsivabalan @n3nash can we do something
here to prioritize this? Or at least source more help from dev list/slack?
--
This is an autom
sbernauer edited a comment on pull request #2012:
URL: https://github.com/apache/hudi/pull/2012#issuecomment-834085407
Hi together,
we sadly can't do schema evolution for 10 months now
(https://github.com/apache/hudi/issues/1845) and have to rely on ugly
workarounds.
Many thanks
sbernauer commented on pull request #2012:
URL: https://github.com/apache/hudi/pull/2012#issuecomment-834085407
Hi together,
we sadly can't do schema evolution for 10 months now
(https://github.com/apache/hudi/issues/1845) and have to rely on ugly
workarounds.
Many thanks for wor
danny0405 commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-834084750
Hi @vinothchandar Can you take a look, this PR is small and we need this in
emergency.
--
This is an automated message from the Apache Git Service.
To respond to the message, p
tandonraghav opened a new issue #2919:
URL: https://github.com/apache/hudi/issues/2919
I am facing issue in Schema Evolution. While adding a new field to the Spark
DF, it is giving exception if there are previous Log files/Records which do not
have that field.
I can see *type* is re
[
https://issues.apache.org/jira/browse/HUDI-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17340578#comment-17340578
]
Vinoth Chandar commented on HUDI-431:
-
[~szhou] Not sure how read-ahead works with file
vinothchandar commented on pull request #2719:
URL: https://github.com/apache/hudi/pull/2719#issuecomment-834075609
any reason why you closed this @MyLanPangzi ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use th
nsivabalan commented on a change in pull request #2915:
URL: https://github.com/apache/hudi/pull/2915#discussion_r627927954
##
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/JdbcSource.java
##
@@ -0,0 +1,326 @@
+/*
+ * Licensed to the Apache Software
[
https://issues.apache.org/jira/browse/HUDI-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17340566#comment-17340566
]
Simon Zhou commented on HUDI-431:
-
I should have re-framed my question as: what is the adva
[
https://issues.apache.org/jira/browse/HUDI-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17340557#comment-17340557
]
Udit Mehrotra commented on HUDI-1879:
-
[~vinoth] [~pzw2018] Noticed a couple of blocki
[
https://issues.apache.org/jira/browse/HUDI-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Udit Mehrotra updated HUDI-1879:
Description:
*Read as DataSource Tables* and *HoodieFileIndex* implementation that went in
[https:/
[
https://issues.apache.org/jira/browse/HUDI-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Udit Mehrotra updated HUDI-1879:
Summary: Spark DataSource tables/HoodieFileIndex issues for Merge On Read
(was: HoodieFileIndex iss
[
https://issues.apache.org/jira/browse/HUDI-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Udit Mehrotra updated HUDI-1879:
Description:
*HoodieFileIndex* implementation that went in
[https://github.com/apache/hudi/pull/265
Udit Mehrotra created HUDI-1879:
---
Summary: HoodieFileIndex issues for Merge On Read
Key: HUDI-1879
URL: https://issues.apache.org/jira/browse/HUDI-1879
Project: Apache Hudi
Issue Type: Bug
Danny Chen created HUDI-1878:
Summary: Add max memory option for flink writer task
Key: HUDI-1878
URL: https://issues.apache.org/jira/browse/HUDI-1878
Project: Apache Hudi
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/HUDI-1821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vinoyang closed HUDI-1821.
--
Resolution: Done
0a5863939b8da149f6dab3ec0fdb86d77baa9739
> Remove legacy code for Flink writer
> -
This is an automated email from the ASF dual-hosted git repository.
vinoyang pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git.
from 0284cde [HUDI-1876] wiring in Hadoop Conf with AvroSchemaConverters
instantiation (#2914)
add 0a58639 [HUDI-1
yanghua merged pull request #2868:
URL: https://github.com/apache/hudi/pull/2868
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please co
yanghua commented on pull request #2868:
URL: https://github.com/apache/hudi/pull/2868#issuecomment-834023836
> > > see https://issues.apache.org/jira/browse/HUDI-1872
> >
> >
> > @MyLanPangzi Are you interested in this ticket? To continue refactoring.
>
> yes, I have alrea
liijiankang closed issue #2917:
URL: https://github.com/apache/hudi/issues/2917
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please con
MyLanPangzi closed pull request #2719:
URL: https://github.com/apache/hudi/pull/2719
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, pleas
nsivabalan commented on pull request #2915:
URL: https://github.com/apache/hudi/pull/2915#issuecomment-833988217
Can we add tests for these scenarios
1. no matching rows with incremental fetch. lets say everything is caught
up.
2. records in source table range from 0 to 100. you set
nsivabalan commented on a change in pull request #2915:
URL: https://github.com/apache/hudi/pull/2915#discussion_r627846007
##
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/JdbcSource.java
##
@@ -0,0 +1,326 @@
+/*
+ * Licensed to the Apache Software
codecov-commenter commented on pull request #2918:
URL: https://github.com/apache/hudi/pull/2918#issuecomment-833971983
#
[Codecov](https://codecov.io/gh/apache/hudi/pull/2918?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache
satishkotha opened a new pull request #2918:
URL: https://github.com/apache/hudi/pull/2918
## What is the purpose of the pull request
Add support for reusing fileId in clustering execution strategy. This is
strategy specific. Default is still to create new files
## Brief chang
[
https://issues.apache.org/jira/browse/HUDI-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
satish updated HUDI-1877:
-
Description: We want to support records staying in same fileId after
clustering (For example, sort each file OR re
[
https://issues.apache.org/jira/browse/HUDI-1833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
satish resolved HUDI-1833.
--
Resolution: Fixed
> pending cluster rollback when there are greater commits
> --
[
https://issues.apache.org/jira/browse/HUDI-1833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
satish closed HUDI-1833.
> pending cluster rollback when there are greater commits
> ---
>
>
satish created HUDI-1877:
Summary: clustering support for external index
Key: HUDI-1877
URL: https://issues.apache.org/jira/browse/HUDI-1877
Project: Apache Hudi
Issue Type: Sub-task
Repo
[
https://issues.apache.org/jira/browse/HUDI-874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17340409#comment-17340409
]
Udit Mehrotra commented on HUDI-874:
[~balajiit] can you share some quick/easy reproduc
vinothchandar commented on pull request #2915:
URL: https://github.com/apache/hudi/pull/2915#issuecomment-833746933
Great suggestions @leesf ! We can file followups for those.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
vburenin commented on a change in pull request #2714:
URL: https://github.com/apache/hudi/pull/2714#discussion_r606851124
##
File path:
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/util/HiveSchemaUtil.java
##
@@ -109,7 +109,7 @@ public static SchemaDifference
g
vinothchandar commented on pull request #2714:
URL: https://github.com/apache/hudi/pull/2714#issuecomment-833683614
>it would be changed to debug if there are some stats to indicate the time
cost. I think it is an important stat when profiling.
Typically, we turn on logs with perf pr
vinothchandar commented on pull request #2714:
URL: https://github.com/apache/hudi/pull/2714#issuecomment-833682238
This test is just flaky
```
[INFO]
[ERROR] Failures:
[ERROR]
TestHoodieMultiTableDeltaStreamer.testMultiTableExecutionWithKafkaSource:168
expected: but w
mpouttu removed a comment on pull request #2907:
URL: https://github.com/apache/hudi/pull/2907#issuecomment-833616754
Hello, I am looking at the test failures and they don't have to do with my
changes. The only possible side effect from my change would be that some detail
logging that is a
codecov-commenter edited a comment on pull request #2907:
URL: https://github.com/apache/hudi/pull/2907#issuecomment-830598873
#
[Codecov](https://codecov.io/gh/apache/hudi/pull/2907?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The
mpouttu commented on pull request #2907:
URL: https://github.com/apache/hudi/pull/2907#issuecomment-833616754
Hello, I am looking at the test failures and they don't have to do with my
changes. The only possible side effect from my change would be that some detail
logging that is already i
vinothchandar commented on a change in pull request #2809:
URL: https://github.com/apache/hudi/pull/2809#discussion_r627514500
##
File path:
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieHiveUtils.java
##
@@ -122,19 +135,44 @@ public static Path getNthParent(
vinothchandar commented on pull request #2784:
URL: https://github.com/apache/hudi/pull/2784#issuecomment-833601155
Thanks for this @ssdong and also the great explanation and detailed comments
along the way. I am on it. This is a bit deep, been trying to set aside an hour
or so. :)
--
leesf commented on a change in pull request #2915:
URL: https://github.com/apache/hudi/pull/2915#discussion_r627418429
##
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/JdbcSource.java
##
@@ -0,0 +1,326 @@
+/*
+ * Licensed to the Apache Software Foun
leesf commented on a change in pull request #2915:
URL: https://github.com/apache/hudi/pull/2915#discussion_r627417386
##
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/JdbcSource.java
##
@@ -0,0 +1,326 @@
+/*
+ * Licensed to the Apache Software Foun
leesf commented on a change in pull request #2915:
URL: https://github.com/apache/hudi/pull/2915#discussion_r627417046
##
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/JdbcSource.java
##
@@ -0,0 +1,326 @@
+/*
+ * Licensed to the Apache Software Foun
leesf commented on a change in pull request #2915:
URL: https://github.com/apache/hudi/pull/2915#discussion_r627412205
##
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/SqlQueryBuilder.java
##
@@ -0,0 +1,160 @@
+/*
+ * Licensed to the Apache Software Foundat
leesf commented on a change in pull request #2915:
URL: https://github.com/apache/hudi/pull/2915#discussion_r627411500
##
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/SqlQueryBuilder.java
##
@@ -0,0 +1,160 @@
+/*
+ * Licensed to the Apache Software Foundat
MyLanPangzi commented on pull request #2868:
URL: https://github.com/apache/hudi/pull/2868#issuecomment-833493033
> > see https://issues.apache.org/jira/browse/HUDI-1872
>
> @MyLanPangzi Are you interested in this ticket? To continue refactoring.
yes, I have already made part o
yanghua commented on pull request #2868:
URL: https://github.com/apache/hudi/pull/2868#issuecomment-833459923
> see https://issues.apache.org/jira/browse/HUDI-1872
@MyLanPangzi Are you interested in this ticket? To continue refactoring.
--
This is an automated message from the Apac
liijiankang commented on issue #2917:
URL: https://github.com/apache/hudi/issues/2917#issuecomment-833349514
--
This is an automated message from the Apache Git Service.
To res
MyLanPangzi commented on pull request #2904:
URL: https://github.com/apache/hudi/pull/2904#issuecomment-833342296
@danny0405 hi, the
org.apache.hudi.sink.transform.TestRowDataToHoodieFunction#testRateLimit
cannot pass, could you help me inspect the ut ?
--
This is an automated message
liijiankang opened a new issue #2917:
URL: https://github.com/apache/hudi/issues/2917
**Describe the problem you faced**
Kafka data written to the hudi MOR table throws an exception
**Expected behavior**
Kafka data is written to the hudi COW table without exception .Hope it can
MyLanPangzi commented on pull request #2719:
URL: https://github.com/apache/hudi/pull/2719#issuecomment-82159
i test with hive 2.3.8 and add follow shell. i got a runtime exception seems
abnormal.
HIVE_VERSION=$(hive --version|head -n 1|awk '{print $2}')
if [[ $HIVE_VERSIO
codecov-commenter edited a comment on pull request #2904:
URL: https://github.com/apache/hudi/pull/2904#issuecomment-830024729
#
[Codecov](https://codecov.io/gh/apache/hudi/pull/2904?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The
yanghua commented on a change in pull request #2898:
URL: https://github.com/apache/hudi/pull/2898#discussion_r627167070
##
File path:
hudi-flink/src/main/java/org/apache/hudi/sink/utils/HiveSyncContext.java
##
@@ -51,11 +51,10 @@ public HiveSyncTool hiveSyncTool() {
retu
hudi-bot edited a comment on pull request #2904:
URL: https://github.com/apache/hudi/pull/2904#issuecomment-830836315
## CI report:
* b43dc8db3e25949c77e9bb50d0122bdca9c43073 Travis:
[FAILURE](https://travis-ci.com/github/apachehudi-ci/hudi-branch-ci/builds/225042351)
codecov-commenter edited a comment on pull request #2904:
URL: https://github.com/apache/hudi/pull/2904#issuecomment-830024729
#
[Codecov](https://codecov.io/gh/apache/hudi/pull/2904?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The
73 matches
Mail list logo
