liujinhui1994 commented on a change in pull request #2438:
URL: https://github.com/apache/hudi/pull/2438#discussion_r594877281
##
File path:
hudi-integ-test/src/main/java/org/apache/hudi/integ/testsuite/HoodieDeltaStreamerWrapper.java
##
@@ -65,7 +65,7 @@ public void
[
https://issues.apache.org/jira/browse/HUDI-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302235#comment-17302235
]
Vinoth Govindarajan commented on HUDI-1695:
---
PR has been merged.
> Deltastreamer
nsivabalan commented on a change in pull request #2438:
URL: https://github.com/apache/hudi/pull/2438#discussion_r594471195
##
File path:
hudi-integ-test/src/main/java/org/apache/hudi/integ/testsuite/HoodieDeltaStreamerWrapper.java
##
@@ -65,7 +65,7 @@ public void
nsivabalan commented on a change in pull request #2438:
URL: https://github.com/apache/hudi/pull/2438#discussion_r594471195
##
File path:
hudi-integ-test/src/main/java/org/apache/hudi/integ/testsuite/HoodieDeltaStreamerWrapper.java
##
@@ -65,7 +65,7 @@ public void
This is an automated email from the ASF dual-hosted git repository.
vinoyang pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git.
from 3b36cb8 [HUDI-1552] Improve performance of key lookups from base file
in Metadata Table. (#2494)
add 16864ae
yanghua merged pull request #2679:
URL: https://github.com/apache/hudi/pull/2679
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
xiarixiaoyao commented on pull request #2673:
URL: https://github.com/apache/hudi/pull/2673#issuecomment-799905602
@nsivabalan yes， since the problem of company information security, i
cannot paste screenshot of test result and dump.
before fix
env: (executor 4 core 8G)*50
pengzhiwei2018 commented on issue #2656:
URL: https://github.com/apache/hudi/issues/2656#issuecomment-799888195
Hi @shivabansal1046 , no need to add a extra column, just set
`DataSourceWriteOptions#KEYGENERATOR_CLASS_OPT_KEY` to
`classOf[UuidKeyGenKeyGenerator]` which generate a uuid
[
https://issues.apache.org/jira/browse/HUDI-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xianghu Wang updated HUDI-1526:
---
Fix Version/s: 0.8.0
> Translate the spark api partitionBy to
>
[
https://issues.apache.org/jira/browse/HUDI-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xianghu Wang resolved HUDI-1526.
Resolution: Resolved
Resolved via master branch : 26da4f546275e8ab6496537743efe73510cb723d
>
satishkotha commented on a change in pull request #2678:
URL: https://github.com/apache/hudi/pull/2678#discussion_r594753875
##
File path:
hudi-cli/src/main/java/org/apache/hudi/cli/commands/CommitsCommand.java
##
@@ -431,4 +442,20 @@ public String syncCommits(@CliOption(key
codecov-io commented on pull request #2679:
URL: https://github.com/apache/hudi/pull/2679#issuecomment-799814201
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2679?src=pr=h1) Report
> Merging
[#2679](https://codecov.io/gh/apache/hudi/pull/2679?src=pr=desc) (818439b)
into
[
https://issues.apache.org/jira/browse/HUDI-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1695:
-
Labels: beginner pull-request-available (was: beginner)
> Deltastreamer HoodieIncrSource
vingov opened a new pull request #2679:
URL: https://github.com/apache/hudi/pull/2679
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a
pull request.*
## What is the purpose of the
[
https://issues.apache.org/jira/browse/HUDI-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Govindarajan updated HUDI-1695:
--
Summary: Deltastreamer HoodieIncrSource exception error messaging is
incorrect (was:
Vinoth Govindarajan created HUDI-1695:
-
Summary: Deltastream HoodieIncrSource exception error messaging is
incorrect
Key: HUDI-1695
URL: https://issues.apache.org/jira/browse/HUDI-1695
Project:
prashantwason commented on pull request #2494:
URL: https://github.com/apache/hudi/pull/2494#issuecomment-799740584
Looks good @vinothchandar
This is an automated message from the Apache Git Service.
To respond to the
prashantwason commented on a change in pull request #2494:
URL: https://github.com/apache/hudi/pull/2494#discussion_r594671546
##
File path:
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java
##
@@ -147,82 +150,91 @@ private void initIfNeeded()
vinothchandar merged pull request #2494:
URL: https://github.com/apache/hudi/pull/2494
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
This is an automated email from the ASF dual-hosted git repository.
vinoth pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git.
from 76bf2cc [HUDI-1692] Bounded source for stream writer (#2674)
add 3b36cb8 [HUDI-1552] Improve performance of key
vinothchandar commented on a change in pull request #2494:
URL: https://github.com/apache/hudi/pull/2494#discussion_r594656644
##
File path:
hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieHFileReader.java
##
@@ -232,12 +246,13 @@ public long getTotalRecords() {
codecov-io edited a comment on pull request #2494:
URL: https://github.com/apache/hudi/pull/2494#issuecomment-767956391
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2494?src=pr=h1) Report
> Merging
[#2494](https://codecov.io/gh/apache/hudi/pull/2494?src=pr=desc) (59b919a)
into
nsivabalan edited a comment on pull request #2438:
URL: https://github.com/apache/hudi/pull/2438#issuecomment-799571386
Thanks for your contribution. this is going to be useful to the community.
Few high level questions.
1. Why not we leverage DeltaSreamerConfig.checkpoint to pass in
nsivabalan commented on pull request #2438:
URL: https://github.com/apache/hudi/pull/2438#issuecomment-799571386
Few high level questions.
1. Why not we leverage DeltaSreamerConfig.checkpoint to pass in a timestamp
for Kafka source? Or do we expect the format of this config to be
jsbali opened a new pull request #2678:
URL: https://github.com/apache/hudi/pull/2678
…ow_write_stats, commit showfiles
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a
pull request.*
jsbali opened a new pull request #2677:
URL: https://github.com/apache/hudi/pull/2677
…eted clean and rollback actions.
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a
pull request.*
This is an automated email from the ASF dual-hosted git repository.
vinoth pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 565dac6 Travis CI build asf-site
565dac6 is
shivabansal1046 commented on issue #2656:
URL: https://github.com/apache/hudi/issues/2656#issuecomment-799528507
Hi pengzhiwei2018,
Are you suggesting to add extra column which is generated key?
Is this workaround or this is how it should be?
vburenin commented on pull request #2619:
URL: https://github.com/apache/hudi/pull/2619#issuecomment-799525731
@nsivabalan I am very strapped by time. I will be able get back to it only
next Q. The overall change is trivial. If you could continue, it would be
great. As soon as this one is
liujinhui1994 commented on pull request #2438:
URL: https://github.com/apache/hudi/pull/2438#issuecomment-799512747
no problem
-- Original --
From: Sivabalan Narayanan ***@***.***
Date: Mon,Mar 15,2021 11:28 PM
To: apache/hudi
pengzhiwei2018 commented on issue #2656:
URL: https://github.com/apache/hudi/issues/2656#issuecomment-799511860
Hi @shivabansal1046 , currently hudi would do the insert operator when you
have specified a rowkey for insert. You can use `UuidKeyGenKeyGenerator` as the
row key to walk around
nsivabalan commented on pull request #2438:
URL: https://github.com/apache/hudi/pull/2438#issuecomment-799511127
hey folks. may I know what's the status of this PR. I see this could benefit
others in the community as well. Do you think we can take it across the finish
line by this
nsivabalan commented on pull request #2619:
URL: https://github.com/apache/hudi/pull/2619#issuecomment-799507450
@vburenin : Did you get a chance to work on this PR. We would like to have
this in before our next release. If you are strapped for time, let me know. I
will try to squeeze in
This is an automated email from the ASF dual-hosted git repository.
sivabalan pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 601f54f [HUDI-1563] Adding hudi file
nsivabalan merged pull request #2612:
URL: https://github.com/apache/hudi/pull/2612
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
[
https://issues.apache.org/jira/browse/HUDI-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-1688:
--
Labels: pull-request-available sev:critical user-support-issues (was:
nsivabalan edited a comment on pull request #2673:
URL: https://github.com/apache/hudi/pull/2673#issuecomment-799497945
@xiarixiaoyao : thanks for your contribution. Were you able to test out the
fix in your env. That subsequent writes don't incur OOMs.
nsivabalan commented on pull request #2673:
URL: https://github.com/apache/hudi/pull/2673#issuecomment-799497945
@xiarixiaoyao : Were you able to test out the fix in your env. That
subsequent writes don't incur OOMs.
This
nsivabalan commented on pull request #2666:
URL: https://github.com/apache/hudi/pull/2666#issuecomment-799494981
@liujinhui1994 : Thanks for the contribution. There are 2 to 3 PRs with
similar goal. Did you get happen to check out existing ones before putting this
up?
shivabansal1046 commented on issue #2656:
URL: https://github.com/apache/hudi/issues/2656#issuecomment-799469658
Hi,
Below are the configs I am using
.write
.format("org.apache.hudi").
options(getQuickstartWriteConfigs).
option(TABLE_TYPE_OPT_KEY,
wosow opened a new issue #2676:
URL: https://github.com/apache/hudi/issues/2676
**Environment Description**
* Hudi version : 0.7.0/0.6.0
* Spark version : 2.4.4
* Hive version :2.3.1
* Hadoop version : 2.7.5
* Storage (HDFS/S3/GCS..) : HDFS
*
[
https://issues.apache.org/jira/browse/HUDI-1692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vinoyang closed HUDI-1692.
--
Resolution: Done
76bf2cc790edc0be10dfa2454c42687c38e7e5fc
> Bounded source for stream writer
>
This is an automated email from the ASF dual-hosted git repository.
vinoyang pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git.
from fc6c5f4 [HUDI-1684] Tweak hudi-flink-bundle module pom and reorganize
the pacakges for hudi-flink module (#2669)
yanghua merged pull request #2674:
URL: https://github.com/apache/hudi/pull/2674
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
codecov-io commented on pull request #2650:
URL: https://github.com/apache/hudi/pull/2650#issuecomment-799292031
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2650?src=pr=h1) Report
> Merging
[#2650](https://codecov.io/gh/apache/hudi/pull/2650?src=pr=desc) (1a5cb70)
into
codecov-io edited a comment on pull request #2674:
URL: https://github.com/apache/hudi/pull/2674#issuecomment-799291092
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2674?src=pr=h1) Report
> Merging
[#2674](https://codecov.io/gh/apache/hudi/pull/2674?src=pr=desc) (8419357)
into
codecov-io commented on pull request #2674:
URL: https://github.com/apache/hudi/pull/2674#issuecomment-799291092
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2674?src=pr=h1) Report
> Merging
[#2674](https://codecov.io/gh/apache/hudi/pull/2674?src=pr=desc) (8419357)
into
Sebastian Bernauer created HUDI-1694:
Summary: Preparation for Avro update
Key: HUDI-1694
URL: https://issues.apache.org/jira/browse/HUDI-1694
Project: Apache Hudi
Issue Type: Task
aditiwari01 opened a new issue #2675:
URL: https://github.com/apache/hudi/issues/2675
As per HUDI confluence
(https://cwiki.apache.org/confluence/display/HUDI/FAQ#FAQ-What'sHudi'sschemaevolutionstory),
as long as schema is backward compatible, hudi will support seamless
read/writes.
[
https://issues.apache.org/jira/browse/HUDI-1692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1692:
-
Labels: pull-request-available (was: )
> Bounded source for stream writer
>
danny0405 opened a new pull request #2674:
URL: https://github.com/apache/hudi/pull/2674
Supports bounded source such as VALUES for stream mode writer.
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html
sbernauer commented on a change in pull request #2650:
URL: https://github.com/apache/hudi/pull/2650#discussion_r594155813
##
File path:
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/index/simple/FlinkHoodieSimpleIndex.java
##
@@ -135,8 +133,8 @@ public boolean
[
https://issues.apache.org/jira/browse/HUDI-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen updated HUDI-1693:
-
Summary: Add document about HUDI Flink integration (was: Bounded source
for stream writer)
> Add
Danny Chen created HUDI-1693:
Summary: Bounded source for stream writer
Key: HUDI-1693
URL: https://issues.apache.org/jira/browse/HUDI-1693
Project: Apache Hudi
Issue Type: Sub-task
Danny Chen created HUDI-1692:
Summary: Bounded source for stream writer
Key: HUDI-1692
URL: https://issues.apache.org/jira/browse/HUDI-1692
Project: Apache Hudi
Issue Type: Sub-task
xiarixiaoyao edited a comment on pull request #2673:
URL: https://github.com/apache/hudi/pull/2673#issuecomment-799059684
cc @garyli1019 @nsivabalan , could you help me review this pr, thanks
This is an automated message
Sugamber commented on issue #2637:
URL: https://github.com/apache/hudi/issues/2637#issuecomment-799211234
@nsivabalan Yes,
I have added the jar file in both driver and executor class path.
`spark-submit --jars
[
https://issues.apache.org/jira/browse/HUDI-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vinoyang closed HUDI-1684.
--
Resolution: Done
fc6c5f4285098d18cd7f6e81785f59e68a3b6862
> Tweak hudi-flink-bundle module pom and re-organize
[
https://issues.apache.org/jira/browse/HUDI-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vinoyang reassigned HUDI-1684:
--
Assignee: Danny Chen
> Tweak hudi-flink-bundle module pom and re-organize the pacakges for
>
This is an automated email from the ASF dual-hosted git repository.
vinoyang pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git.
from e93c6a5 [HUDI-1496] Fixing input stream detection of GCS FileSystem
(#2500)
add fc6c5f4 [HUDI-1684] Tweak
yanghua merged pull request #2669:
URL: https://github.com/apache/hudi/pull/2669
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
shenbinglife closed issue #2652:
URL: https://github.com/apache/hudi/issues/2652
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
shenbinglife commented on issue #2652:
URL: https://github.com/apache/hudi/issues/2652#issuecomment-799201945
Thanks a lot
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
liujinhui1994 commented on pull request #2666:
URL: https://github.com/apache/hudi/pull/2666#issuecomment-799193666
@n3nash @satishkotha @yanghua
This is an automated message from the Apache Git Service.
To respond to the
liujinhui1994 commented on pull request #2666:
URL: https://github.com/apache/hudi/pull/2666#issuecomment-799193044
The content in this pr https://github.com/apache/hudi/pull/1929 comment is
resolved here, please review
n3nash commented on pull request #2374:
URL: https://github.com/apache/hudi/pull/2374#issuecomment-799186819
@vinothchandar Build succeeds locally and should pass on jenkins (will check
tomorrow morning), ready for review.
codecov-io edited a comment on pull request #2673:
URL: https://github.com/apache/hudi/pull/2673#issuecomment-799061373
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2673?src=pr=h1) Report
> Merging
[#2673](https://codecov.io/gh/apache/hudi/pull/2673?src=pr=desc) (2ecca39)
into
codecov-io edited a comment on pull request #2673:
URL: https://github.com/apache/hudi/pull/2673#issuecomment-799061373
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2673?src=pr=h1) Report
> Merging
[#2673](https://codecov.io/gh/apache/hudi/pull/2673?src=pr=desc) (2ecca39)
into
liujinhui1994 commented on pull request #1929:
URL: https://github.com/apache/hudi/pull/1929#issuecomment-799175366
@nsivabalan This PR has been modified based on the comments，
https://github.com/apache/hudi/pull/2666
Please review
codecov-io edited a comment on pull request #2669:
URL: https://github.com/apache/hudi/pull/2669#issuecomment-797515929
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2669?src=pr=h1) Report
> Merging
[#2669](https://codecov.io/gh/apache/hudi/pull/2669?src=pr=desc) (50d4722)
into
vinoyang created HUDI-1691:
--
Summary: Enrich HDFS data importer
Key: HUDI-1691
URL: https://issues.apache.org/jira/browse/HUDI-1691
Project: Apache Hudi
Issue Type: Improvement
danny0405 commented on a change in pull request #2669:
URL: https://github.com/apache/hudi/pull/2669#discussion_r594086761
##
File path:
hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java
##
@@ -16,14 +16,14 @@
* limitations under the License.
*/
danny0405 commented on a change in pull request #2669:
URL: https://github.com/apache/hudi/pull/2669#discussion_r594085950
##
File path:
hudi-flink/src/main/java/org/apache/hudi/streamer/HoodieFlinkStreamerV2.java
##
@@ -19,10 +19,10 @@
package org.apache.hudi.streamer;
yanghua commented on a change in pull request #2669:
URL: https://github.com/apache/hudi/pull/2669#discussion_r594077871
##
File path:
hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java
##
@@ -16,14 +16,14 @@
* limitations under the License.
*/
codecov-io edited a comment on pull request #2374:
URL: https://github.com/apache/hudi/pull/2374#issuecomment-750782300
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2374?src=pr=h1) Report
> Merging
[#2374](https://codecov.io/gh/apache/hudi/pull/2374?src=pr=desc) (d477189)
into
75 matches
Mail list logo
