[GitHub] [incubator-hudi] zhedoubushishi commented on a change in pull request #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema

zhedoubushishi commented on a change in pull request #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema URL: https://github.com/apache/incubator-hudi/pull/1406#discussion_r395456252 ## File path: hudi-spark/src/main/scala/org/apache/hudi/AvroConversionHelp

[GitHub] [incubator-hudi] BalaMahesh commented on issue #1423: [SUPPORT] I am trying to run hudi as mentioned in the getting started guide first time and I am facing this issue dyld: lazy symbol bindi

BalaMahesh commented on issue #1423: [SUPPORT] I am trying to run hudi as mentioned in the getting started guide first time and I am facing this issue dyld: lazy symbol binding failed: Symbol not found: chkstk_darwin. I am using MacOS Sierra 10.12.6 . Can anyone help me with this issue ! Tha

[GitHub] [incubator-hudi] BalaMahesh closed issue #1423: [SUPPORT] I am trying to run hudi as mentioned in the getting started guide first time and I am facing this issue dyld: lazy symbol binding fai

BalaMahesh closed issue #1423: [SUPPORT] I am trying to run hudi as mentioned in the getting started guide first time and I am facing this issue dyld: lazy symbol binding failed: Symbol not found: chkstk_darwin. I am using MacOS Sierra 10.12.6 . Can anyone help me with this issue ! Thanks in

[GitHub] [incubator-hudi] codecov-io edited a comment on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema

codecov-io edited a comment on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema URL: https://github.com/apache/incubator-hudi/pull/1406#issuecomment-599159357 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1406?src=pr&el=h1) Report > Mer

[GitHub] [incubator-hudi] codecov-io edited a comment on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema

codecov-io edited a comment on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema URL: https://github.com/apache/incubator-hudi/pull/1406#issuecomment-599159357 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1406?src=pr&el=h1) Report > Mer

[jira] [Updated] (HUDI-725) Remove or rewrite init log in the constructor of DeltaSync

[ https://issues.apache.org/jira/browse/HUDI-725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-725: - Summary: Remove or rewrite init log in the constructor of DeltaSync (was: Remove or rewrite init log in

[GitHub] [incubator-hudi] BalaMahesh opened a new issue #1423: [SUPPORT] I am trying to run hudi as mentioned in the getting started guide first time and I am facing this issue dyld: lazy symbol bindi

BalaMahesh opened a new issue #1423: [SUPPORT] I am trying to run hudi as mentioned in the getting started guide first time and I am facing this issue dyld: lazy symbol binding failed: Symbol not found: chkstk_darwin URL: https://github.com/apache/incubator-hudi/issues/1423 **_Tips bef

[jira] [Updated] (HUDI-725) Remove or rewrite init log in DeltaSync

[ https://issues.apache.org/jira/browse/HUDI-725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-725: - Description: When initializing HoodieDeltaStreamer, DeltaSyncService and DeltaSync are initialized in turn

[jira] [Updated] (HUDI-726) Delete unused method in HoodieDeltaStreamer

[ https://issues.apache.org/jira/browse/HUDI-726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-726: - Description: It seems that this method 'org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer#getDel

[jira] [Updated] (HUDI-726) Delete unused method in HoodieDeltaStreamer

[ https://issues.apache.org/jira/browse/HUDI-726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-726: - Description: It seems that this method 'org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer#getDel

[jira] [Commented] (HUDI-726) Delete unused method in HoodieDeltaStreamer

[ https://issues.apache.org/jira/browse/HUDI-726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17063105#comment-17063105 ] wangxianghu commented on HUDI-726: -- [~vinoth] what do you think ? > Delete unused method

[jira] [Commented] (HUDI-725) Remove or rewrite init log in DeltaSync

[ https://issues.apache.org/jira/browse/HUDI-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17063104#comment-17063104 ] wangxianghu commented on HUDI-725: -- [~vinoth] what do you think ? > Remove or rewrite in

[jira] [Created] (HUDI-726) Delete unused method in HoodieDeltaStreamer

wangxianghu created HUDI-726: Summary: Delete unused method in HoodieDeltaStreamer Key: HUDI-726 URL: https://issues.apache.org/jira/browse/HUDI-726 Project: Apache Hudi (incubating) Issue Type:

[GitHub] [incubator-hudi] ffcchi commented on a change in pull request #1421: [HUDI-724] Parallelize getSmallFiles for partitions

ffcchi commented on a change in pull request #1421: [HUDI-724] Parallelize getSmallFiles for partitions URL: https://github.com/apache/incubator-hudi/pull/1421#discussion_r395433296 ## File path: hudi-client/src/main/java/org/apache/hudi/table/HoodieCopyOnWriteTable.java #

[GitHub] [incubator-hudi] ffcchi commented on a change in pull request #1421: [HUDI-724] Parallelize getSmallFiles for partitions

ffcchi commented on a change in pull request #1421: [HUDI-724] Parallelize getSmallFiles for partitions URL: https://github.com/apache/incubator-hudi/pull/1421#discussion_r395433104 ## File path: hudi-client/src/main/java/org/apache/hudi/table/HoodieCopyOnWriteTable.java #

[GitHub] [incubator-hudi] ffcchi commented on a change in pull request #1421: [HUDI-724] Parallelize getSmallFiles for partitions

ffcchi commented on a change in pull request #1421: [HUDI-724] Parallelize getSmallFiles for partitions URL: https://github.com/apache/incubator-hudi/pull/1421#discussion_r395432973 ## File path: hudi-client/src/main/java/org/apache/hudi/table/HoodieMergeOnReadTable.java #

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #222

See Changes: -- [...truncated 2.37 KB...] /home/jenkins/tools/maven/apache-maven-3.5.4/conf: logging settings.xml toolchains.xml /home/jenkins/tools/maven/apache-maven-3.5.

[jira] [Updated] (HUDI-725) Remove or rewrite init log in DeltaSync

[ https://issues.apache.org/jira/browse/HUDI-725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-725: - Description: When initializing HoodieDeltaStreamer, DeltaSyncService and DeltaSync are initialized in turn

[jira] [Created] (HUDI-725) Remove or rewrite init log in DeltaSync

wangxianghu created HUDI-725: Summary: Remove or rewrite init log in DeltaSync Key: HUDI-725 URL: https://issues.apache.org/jira/browse/HUDI-725 Project: Apache Hudi (incubating) Issue Type: Imp

[GitHub] [incubator-hudi] codecov-io commented on issue #1422: [HUDI-400]check upgrade from old plan to new plan for compaction

codecov-io commented on issue #1422: [HUDI-400]check upgrade from old plan to new plan for compaction URL: https://github.com/apache/incubator-hudi/pull/1422#issuecomment-601500333 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1422?src=pr&el=h1) Report > Merging [#1422]

[GitHub] [incubator-hudi] leesf commented on issue #1165: [HUDI-76] Add CSV Source support for Hudi Delta Streamer

leesf commented on issue #1165: [HUDI-76] Add CSV Source support for Hudi Delta Streamer URL: https://github.com/apache/incubator-hudi/pull/1165#issuecomment-601498123 > I feel it can go on the contributing guide.. Code reviews are also contributing :) .. either way is fine by me.. Draft s

[incubator-hudi] branch asf-site updated: [HUDI-653] Add JMX Report Config to Doc (#1370)

This is an automated email from the ASF dual-hosted git repository. leesf pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 434e9c5 [HUDI-653] Add JMX Report Co

[GitHub] [incubator-hudi] leesf merged pull request #1370: [HUDI-653] Add JMX Report Config to Doc

leesf merged pull request #1370: [HUDI-653] Add JMX Report Config to Doc URL: https://github.com/apache/incubator-hudi/pull/1370 This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Commented] (HUDI-724) Parallelize GetSmallFiles For Partitions

[ https://issues.apache.org/jira/browse/HUDI-724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17063038#comment-17063038 ] Vinoth Chandar commented on HUDI-724: - Seems legit... I have not seen this with HDFS at

[GitHub] [incubator-hudi] vinothchandar commented on issue #1165: [HUDI-76] Add CSV Source support for Hudi Delta Streamer

vinothchandar commented on issue #1165: [HUDI-76] Add CSV Source support for Hudi Delta Streamer URL: https://github.com/apache/incubator-hudi/pull/1165#issuecomment-601497167 I feel it can go on the contributing guide.. Code reviews are also contributing :) .. either way is fine by me.. D

[GitHub] [incubator-hudi] vinothchandar commented on issue #1421: [HUDI-724] Parallelize getSmallFiles for partitions

vinothchandar commented on issue #1421: [HUDI-724] Parallelize getSmallFiles for partitions URL: https://github.com/apache/incubator-hudi/pull/1421#issuecomment-601496939 @bvaradar to make final pass and sign off This is an

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1421: [HUDI-724] Parallelize getSmallFiles for partitions

vinothchandar commented on a change in pull request #1421: [HUDI-724] Parallelize getSmallFiles for partitions URL: https://github.com/apache/incubator-hudi/pull/1421#discussion_r395410649 ## File path: hudi-client/src/main/java/org/apache/hudi/table/HoodieCopyOnWriteTable.java ##

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1421: [HUDI-724] Parallelize getSmallFiles for partitions

vinothchandar commented on a change in pull request #1421: [HUDI-724] Parallelize getSmallFiles for partitions URL: https://github.com/apache/incubator-hudi/pull/1421#discussion_r395410562 ## File path: hudi-client/src/main/java/org/apache/hudi/table/HoodieCopyOnWriteTable.java ##

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1421: [HUDI-724] Parallelize getSmallFiles for partitions

vinothchandar commented on a change in pull request #1421: [HUDI-724] Parallelize getSmallFiles for partitions URL: https://github.com/apache/incubator-hudi/pull/1421#discussion_r395410993 ## File path: hudi-client/src/main/java/org/apache/hudi/table/HoodieMergeOnReadTable.java ##

[jira] [Updated] (HUDI-400) Add more checks to TestCompactionUtils#testUpgradeDowngrade

[ https://issues.apache.org/jira/browse/HUDI-400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jerry updated HUDI-400: --- Status: In Progress (was: Open) > Add more checks to TestCompactionUtils#testUpgradeDowngrade > --

[GitHub] [incubator-hudi] zhaomin1423 opened a new pull request #1422: [HUDI-400]check upgrade from old plan to new plan for compaction

zhaomin1423 opened a new pull request #1422: [HUDI-400]check upgrade from old plan to new plan for compaction URL: https://github.com/apache/incubator-hudi/pull/1422 What is the purpose of the pull request Add more test for compaction plan upgrade Brief change log check upgrade

[GitHub] [incubator-hudi] zhaomin1423 closed pull request #1419: [HUDI-400]check upgrade from old plan to new plan for compaction

zhaomin1423 closed pull request #1419: [HUDI-400]check upgrade from old plan to new plan for compaction URL: https://github.com/apache/incubator-hudi/pull/1419 This is an automated message from the Apache Git Service. To res

[jira] [Commented] (HUDI-724) Parallelize GetSmallFiles For Partitions

[ https://issues.apache.org/jira/browse/HUDI-724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17063033#comment-17063033 ] Udit Mehrotra commented on HUDI-724: Thanks Feichi for putting this out ! [~vinoth] [~v

[GitHub] [incubator-hudi] ffcchi opened a new pull request #1421: [HUDI-724] Parallelize getSmallFiles for partitions

ffcchi opened a new pull request #1421: [HUDI-724] Parallelize getSmallFiles for partitions URL: https://github.com/apache/incubator-hudi/pull/1421 ## What is the purpose of the pull request *parallelizing the operation of getting small files for partitions when constructing the

[GitHub] [incubator-hudi] leesf commented on issue #1165: [HUDI-76] Add CSV Source support for Hudi Delta Streamer

leesf commented on issue #1165: [HUDI-76] Add CSV Source support for Hudi Delta Streamer URL: https://github.com/apache/incubator-hudi/pull/1165#issuecomment-601488498 > @leesf this has happened enough times now, that we probably need a Code Review guide as well? wdyt Agree, I would

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile

nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile URL: https://github.com/apache/incubator-hudi/pull/1176#discussion_r39540 ## File path: hudi-utilities/src/test/java/org/apache/hudi/

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile

nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile URL: https://github.com/apache/incubator-hudi/pull/1176#discussion_r395401242 ## File path: hudi-utilities/src/test/java/org/apache/hudi/

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile

nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile URL: https://github.com/apache/incubator-hudi/pull/1176#discussion_r395401013 ## File path: hudi-utilities/src/main/java/org/apache/hudi/

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile

nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile URL: https://github.com/apache/incubator-hudi/pull/1176#discussion_r395400942 ## File path: hudi-utilities/src/main/java/org/apache/hudi/

[GitHub] [incubator-hudi] lamber-ken commented on issue #1377: [HUDI-663] Fix HoodieDeltaStreamer offset not handled correctly

lamber-ken commented on issue #1377: [HUDI-663] Fix HoodieDeltaStreamer offset not handled correctly URL: https://github.com/apache/incubator-hudi/pull/1377#issuecomment-601486061 @garyli1019 thanks very much for your detail comment.

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile

nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile URL: https://github.com/apache/incubator-hudi/pull/1176#discussion_r395399661 ## File path: hudi-utilities/src/main/java/org/apache/hudi/

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile

nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile URL: https://github.com/apache/incubator-hudi/pull/1176#discussion_r395399488 ## File path: hudi-utilities/src/main/java/org/apache/hudi/

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile

nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile URL: https://github.com/apache/incubator-hudi/pull/1176#discussion_r395398271 ## File path: hudi-utilities/src/main/java/org/apache/hudi/

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile

nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile URL: https://github.com/apache/incubator-hudi/pull/1176#discussion_r395395831 ## File path: hudi-utilities/src/main/java/org/apache/hudi/

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile

nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile URL: https://github.com/apache/incubator-hudi/pull/1176#discussion_r395395084 ## File path: hudi-utilities/src/main/java/org/apache/hudi/

[GitHub] [incubator-hudi] satishkotha commented on a change in pull request #1368: [HUDI-650] Modify handleUpdate path to validate partitionPath

satishkotha commented on a change in pull request #1368: [HUDI-650] Modify handleUpdate path to validate partitionPath URL: https://github.com/apache/incubator-hudi/pull/1368#discussion_r395389042 ## File path: hudi-client/src/main/java/org/apache/hudi/io/HoodieMergeHandle.java ###

[GitHub] [incubator-hudi] satishkotha commented on a change in pull request #1368: [HUDI-650] Modify handleUpdate path to validate partitionPath

satishkotha commented on a change in pull request #1368: [HUDI-650] Modify handleUpdate path to validate partitionPath URL: https://github.com/apache/incubator-hudi/pull/1368#discussion_r395389012 ## File path: hudi-client/src/main/java/org/apache/hudi/io/HoodieAppendHandle.java ##

[GitHub] [incubator-hudi] satishkotha commented on issue #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction

satishkotha commented on issue #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction URL: https://github.com/apache/incubator-hudi/pull/1396#issuecomment-601473026 > @satishkotha : Some minor comments. Will approve once you reply/address them. Let's also wait fo

[GitHub] [incubator-hudi] satishkotha commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction

satishkotha commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction URL: https://github.com/apache/incubator-hudi/pull/1396#discussion_r395386191 ## File path: hudi-client/src/test/java/org/apache/hudi/common/HoodieMer

[GitHub] [incubator-hudi] satishkotha commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction

satishkotha commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction URL: https://github.com/apache/incubator-hudi/pull/1396#discussion_r395386043 ## File path: hudi-common/src/test/java/org/apache/hudi/common/table/str

[GitHub] [incubator-hudi] satishkotha commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction

satishkotha commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction URL: https://github.com/apache/incubator-hudi/pull/1396#discussion_r395386006 ## File path: hudi-client/src/test/java/org/apache/hudi/table/TestMergeO

[GitHub] [incubator-hudi] vinothchandar commented on issue #1362: [WIP]HUDI-644 Implement checkpoint generator helper tool

vinothchandar commented on issue #1362: [WIP]HUDI-644 Implement checkpoint generator helper tool URL: https://github.com/apache/incubator-hudi/pull/1362#issuecomment-601469123 No. thank you.. This kind of stuff, gives me energy to keep pushing more :) -

[jira] [Commented] (HUDI-648) Implement error log/table for Datasource/DeltaStreamer/WriteClient/Compaction writes

[ https://issues.apache.org/jira/browse/HUDI-648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17063001#comment-17063001 ] Vinoth Chandar commented on HUDI-648: - [~liujinhui] Actually, skipping should be suppor

[jira] [Updated] (HUDI-724) Parallelize GetSmallFiles For Partitions

[ https://issues.apache.org/jira/browse/HUDI-724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feichi Feng updated HUDI-724: - Description: When writing data, a gap was observed between spark stages. By tracking down where the time w

[GitHub] [incubator-hudi] garyli1019 closed pull request #1362: [WIP]HUDI-644 Implement checkpoint generator helper tool

garyli1019 closed pull request #1362: [WIP]HUDI-644 Implement checkpoint generator helper tool URL: https://github.com/apache/incubator-hudi/pull/1362 This is an automated message from the Apache Git Service. To respond to t

[GitHub] [incubator-hudi] garyli1019 commented on issue #1362: [WIP]HUDI-644 Implement checkpoint generator helper tool

garyli1019 commented on issue #1362: [WIP]HUDI-644 Implement checkpoint generator helper tool URL: https://github.com/apache/incubator-hudi/pull/1362#issuecomment-601468645 ok, I will make a separate PR for the tool. Thanks everyone who participated in this long discussion...

[GitHub] [incubator-hudi] vinothchandar commented on issue #1362: [WIP]HUDI-644 Implement checkpoint generator helper tool

vinothchandar commented on issue #1362: [WIP]HUDI-644 Implement checkpoint generator helper tool URL: https://github.com/apache/incubator-hudi/pull/1362#issuecomment-601468133 > Do you see any other use case the reverse search would be useful? No. not at the moment.. We can close th

[jira] [Created] (HUDI-724) Parallelize GetSmallFiles For Partitions

Feichi Feng created HUDI-724: Summary: Parallelize GetSmallFiles For Partitions Key: HUDI-724 URL: https://issues.apache.org/jira/browse/HUDI-724 Project: Apache Hudi (incubating) Issue Type: Imp

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1416: [HUDI-717] Fixed usage of HiveDriver for DDL statements for Hive 2.x

bvaradar commented on a change in pull request #1416: [HUDI-717] Fixed usage of HiveDriver for DDL statements for Hive 2.x URL: https://github.com/apache/incubator-hudi/pull/1416#discussion_r395337083 ## File path: hudi-hive-sync/src/main/java/org/apache/hudi/hive/HoodieHiveClient.

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1416: [HUDI-717] Fixed usage of HiveDriver for DDL statements for Hive 2.x

bvaradar commented on a change in pull request #1416: [HUDI-717] Fixed usage of HiveDriver for DDL statements for Hive 2.x URL: https://github.com/apache/incubator-hudi/pull/1416#discussion_r395341832 ## File path: hudi-hive-sync/src/test/java/org/apache/hudi/hive/TestHiveSyncTool.

[jira] [Commented] (HUDI-686) Implement BloomIndexV2 that does not depend on memory caching

[ https://issues.apache.org/jira/browse/HUDI-686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17062945#comment-17062945 ] Vinoth Chandar commented on HUDI-686: - [~vbalaji] [~shivnarayan] Please review this inf

[jira] [Commented] (HUDI-686) Implement BloomIndexV2 that does not depend on memory caching

[ https://issues.apache.org/jira/browse/HUDI-686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17062940#comment-17062940 ] Vinoth Chandar commented on HUDI-686: - Timing the individual stages  Roughly, here is

[GitHub] [incubator-hudi] garyli1019 commented on issue #1362: [WIP]HUDI-644 Implement checkpoint generator helper tool

garyli1019 commented on issue #1362: [WIP]HUDI-644 Implement checkpoint generator helper tool URL: https://github.com/apache/incubator-hudi/pull/1362#issuecomment-601416005 > Given that, do we still need the ability to search for the checkpoints in reverse time order? Maybe not any

[GitHub] [incubator-hudi] garyli1019 commented on issue #1377: [HUDI-663] Fix HoodieDeltaStreamer offset not handled correctly

garyli1019 commented on issue #1377: [HUDI-663] Fix HoodieDeltaStreamer offset not handled correctly URL: https://github.com/apache/incubator-hudi/pull/1377#issuecomment-601412133 @vinothchandar I thought the empty checkpoint was created by a bug before, but if the empty checkpoint is inte

[GitHub] [incubator-hudi] prashantwason commented on a change in pull request #1416: [HUDI-717] Fixed usage of HiveDriver for DDL statements for Hive 2.x

prashantwason commented on a change in pull request #1416: [HUDI-717] Fixed usage of HiveDriver for DDL statements for Hive 2.x URL: https://github.com/apache/incubator-hudi/pull/1416#discussion_r395308902 ## File path: hudi-hive-sync/src/main/java/org/apache/hudi/hive/HoodieHiveCl

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1368: [HUDI-650] Modify handleUpdate path to validate partitionPath

bvaradar commented on a change in pull request #1368: [HUDI-650] Modify handleUpdate path to validate partitionPath URL: https://github.com/apache/incubator-hudi/pull/1368#discussion_r395228837 ## File path: hudi-client/src/main/java/org/apache/hudi/io/HoodieAppendHandle.java #

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1368: [HUDI-650] Modify handleUpdate path to validate partitionPath

bvaradar commented on a change in pull request #1368: [HUDI-650] Modify handleUpdate path to validate partitionPath URL: https://github.com/apache/incubator-hudi/pull/1368#discussion_r395229769 ## File path: hudi-client/src/main/java/org/apache/hudi/io/HoodieMergeHandle.java ##

[GitHub] [incubator-hudi] vinothchandar commented on issue #1362: [WIP]HUDI-644 Implement checkpoint generator helper tool

vinothchandar commented on issue #1362: [WIP]HUDI-644 Implement checkpoint generator helper tool URL: https://github.com/apache/incubator-hudi/pull/1362#issuecomment-601321422 Given that, do we still need the ability to search for the checkpoints in reverse time order? tbh I don't see a va

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction

bvaradar commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction URL: https://github.com/apache/incubator-hudi/pull/1396#discussion_r395187395 ## File path: hudi-client/src/test/java/org/apache/hudi/table/TestMergeOnRe

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction

bvaradar commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction URL: https://github.com/apache/incubator-hudi/pull/1396#discussion_r395181942 ## File path: hudi-client/src/test/java/org/apache/hudi/common/HoodieMergeO

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction

bvaradar commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction URL: https://github.com/apache/incubator-hudi/pull/1396#discussion_r395185815 ## File path: hudi-common/src/test/java/org/apache/hudi/common/table/string

[jira] [Commented] (HUDI-686) Implement BloomIndexV2 that does not depend on memory caching

[ https://issues.apache.org/jira/browse/HUDI-686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17062781#comment-17062781 ] Vinoth Chandar commented on HUDI-686: - Running a local microbenchmark, I actually found

[jira] [Updated] (HUDI-686) Implement BloomIndexV2 that does not depend on memory caching

[ https://issues.apache.org/jira/browse/HUDI-686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-686: Attachment: image-2020-03-19-10-17-43-048.png > Implement BloomIndexV2 that does not depend on memory

[jira] [Updated] (HUDI-686) Implement BloomIndexV2 that does not depend on memory caching

[ https://issues.apache.org/jira/browse/HUDI-686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-686: Attachment: Screen Shot 2020-03-19 at 10.15.10 AM.png > Implement BloomIndexV2 that does not depend o

[jira] [Updated] (HUDI-686) Implement BloomIndexV2 that does not depend on memory caching

[ https://issues.apache.org/jira/browse/HUDI-686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-686: Attachment: Screen Shot 2020-03-19 at 10.15.10 AM.png > Implement BloomIndexV2 that does not depend o

[jira] [Updated] (HUDI-686) Implement BloomIndexV2 that does not depend on memory caching

[ https://issues.apache.org/jira/browse/HUDI-686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-686: Attachment: Screen Shot 2020-03-19 at 10.15.10 AM.png > Implement BloomIndexV2 that does not depend o

[GitHub] [incubator-hudi] codecov-io edited a comment on issue #1419: [HUDI-400]check upgrade from old plan to new plan for compaction

codecov-io edited a comment on issue #1419: [HUDI-400]check upgrade from old plan to new plan for compaction URL: https://github.com/apache/incubator-hudi/pull/1419#issuecomment-601246921 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1419?src=pr&el=h1) Report > Merging

[jira] [Commented] (HUDI-686) Implement BloomIndexV2 that does not depend on memory caching

[ https://issues.apache.org/jira/browse/HUDI-686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17062771#comment-17062771 ] Vinoth Chandar commented on HUDI-686: - candidates can be as big as N * size of HoodieRe

[GitHub] [incubator-hudi] vinothchandar commented on issue #1377: [HUDI-663] Fix HoodieDeltaStreamer offset not handled correctly

vinothchandar commented on issue #1377: [HUDI-663] Fix HoodieDeltaStreamer offset not handled correctly URL: https://github.com/apache/incubator-hudi/pull/1377#issuecomment-601303031 >https://github.com/apache/incubator-hudi/blob/master/hudi-utilities/src/main/java/org/apache/hudi/utilitie

[jira] [Commented] (HUDI-686) Implement BloomIndexV2 that does not depend on memory caching

[ https://issues.apache.org/jira/browse/HUDI-686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17062761#comment-17062761 ] lamber-ken commented on HUDI-686: - [~vinoth] thanks for bring up this new idea. here are so

[GitHub] [incubator-hudi] vinothchandar commented on issue #1165: [HUDI-76] Add CSV Source support for Hudi Delta Streamer

vinothchandar commented on issue #1165: [HUDI-76] Add CSV Source support for Hudi Delta Streamer URL: https://github.com/apache/incubator-hudi/pull/1165#issuecomment-601285910 @leesf this has happened enough times now, that we probably need a Code Review guide as well? wdyt -

[GitHub] [incubator-hudi] vinothchandar commented on issue #1417: [HUDI-720] NOTICE file needs to add more content based on the NOTICE files of the ASF projects that hudi bundles

vinothchandar commented on issue #1417: [HUDI-720] NOTICE file needs to add more content based on the NOTICE files of the ASF projects that hudi bundles URL: https://github.com/apache/incubator-hudi/pull/1417#issuecomment-601283565 @yanghua LGTM . lets roll the dice again

[GitHub] [incubator-hudi] vinothchandar commented on issue #1409: [HUDI-714]Add javadoc and comments to hudi write method link

vinothchandar commented on issue #1409: [HUDI-714]Add javadoc and comments to hudi write method link URL: https://github.com/apache/incubator-hudi/pull/1409#issuecomment-601280309 @nsivabalan could you please review this This

[GitHub] [incubator-hudi] deabreu opened a new issue #1420: Broken Maven dependencies.

deabreu opened a new issue #1420: Broken Maven dependencies. URL: https://github.com/apache/incubator-hudi/issues/1420 The following artifacts are missing from https://packages.confluent.io/maven org.apache.hudi:hudi-client::0.6.0-SNAPSHOT org.apache.hudi:hudi-common::0.6.0-SNAPSHOT

[GitHub] [incubator-hudi] bvaradar commented on issue #1400: optimization debian package manager tweaks

bvaradar commented on issue #1400: optimization debian package manager tweaks URL: https://github.com/apache/incubator-hudi/pull/1400#issuecomment-601277565 @Rajpratik71 : Just pinging to see if you are planning to work on this PR. ---

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1416: [HUDI-717] Fixed usage of HiveDriver for DDL statements for Hive 2.x

bvaradar commented on a change in pull request #1416: [HUDI-717] Fixed usage of HiveDriver for DDL statements for Hive 2.x URL: https://github.com/apache/incubator-hudi/pull/1416#discussion_r395155031 ## File path: hudi-hive-sync/src/main/java/org/apache/hudi/hive/HoodieHiveClient.

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile

nsivabalan commented on a change in pull request #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile URL: https://github.com/apache/incubator-hudi/pull/1176#discussion_r395120821 ## File path: hudi-utilities/src/main/java/org/apache/hudi/

[GitHub] [incubator-hudi] codecov-io edited a comment on issue #1419: [HUDI-400]check upgrade from old plan to new plan for compaction

codecov-io edited a comment on issue #1419: [HUDI-400]check upgrade from old plan to new plan for compaction URL: https://github.com/apache/incubator-hudi/pull/1419#issuecomment-601246921 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1419?src=pr&el=h1) Report > Merging

[GitHub] [incubator-hudi] codecov-io commented on issue #1419: [HUDI-400]check upgrade from old plan to new plan for compaction

codecov-io commented on issue #1419: [HUDI-400]check upgrade from old plan to new plan for compaction URL: https://github.com/apache/incubator-hudi/pull/1419#issuecomment-601246921 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1419?src=pr&el=h1) Report > Merging [#1419]

[GitHub] [incubator-hudi] zhaomin1423 opened a new pull request #1419: [HUDI-400]check upgrade from old plan to new plan for compaction

zhaomin1423 opened a new pull request #1419: [HUDI-400]check upgrade from old plan to new plan for compaction URL: https://github.com/apache/incubator-hudi/pull/1419 ## What is the purpose of the pull request Add more test for compaction plan upgrade ## Brief change log

[jira] [Updated] (HUDI-400) Add more checks to TestCompactionUtils#testUpgradeDowngrade

[ https://issues.apache.org/jira/browse/HUDI-400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-400: Labels: pull-request-available (was: ) > Add more checks to TestCompactionUtils#testUpgradeDowngrade

[GitHub] [incubator-hudi] nsivabalan commented on issue #1165: [HUDI-76] Add CSV Source support for Hudi Delta Streamer

nsivabalan commented on issue #1165: [HUDI-76] Add CSV Source support for Hudi Delta Streamer URL: https://github.com/apache/incubator-hudi/pull/1165#issuecomment-601219527 got it, sure. This is an automated message from the

[GitHub] [incubator-hudi] leesf commented on issue #1165: [HUDI-76] Add CSV Source support for Hudi Delta Streamer

leesf commented on issue #1165: [HUDI-76] Add CSV Source support for Hudi Delta Streamer URL: https://github.com/apache/incubator-hudi/pull/1165#issuecomment-601204501 > @leesf : Thanks. I got the permission now. You are welcome and a nice shot. just one minor tip, please merge(squas

[GitHub] [incubator-hudi] nsivabalan commented on issue #1165: [HUDI-76] Add CSV Source support for Hudi Delta Streamer

nsivabalan commented on issue #1165: [HUDI-76] Add CSV Source support for Hudi Delta Streamer URL: https://github.com/apache/incubator-hudi/pull/1165#issuecomment-601195118 @leesf : I got the permission now. This is an autom

[incubator-hudi] branch master updated: [HUDI-76] Add CSV Source support for Hudi Delta Streamer

This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new cf765df [HUDI-76] Add CSV Source sup

[GitHub] [incubator-hudi] nsivabalan merged pull request #1165: [HUDI-76] Add CSV Source support for Hudi Delta Streamer

nsivabalan merged pull request #1165: [HUDI-76] Add CSV Source support for Hudi Delta Streamer URL: https://github.com/apache/incubator-hudi/pull/1165 This is an automated message from the Apache Git Service. To respond to t

[GitHub] [incubator-hudi] nsivabalan edited a comment on issue #1165: [HUDI-76] Add CSV Source support for Hudi Delta Streamer

nsivabalan edited a comment on issue #1165: [HUDI-76] Add CSV Source support for Hudi Delta Streamer URL: https://github.com/apache/incubator-hudi/pull/1165#issuecomment-601195118 @leesf : Thanks. I got the permission now. T

[GitHub] [incubator-hudi] XuQianJin-Stars commented on issue #1370: [HUDI-653] Add JMX Report Config to Doc

XuQianJin-Stars commented on issue #1370: [HUDI-653] Add JMX Report Config to Doc URL: https://github.com/apache/incubator-hudi/pull/1370#issuecomment-601152532 > @XuQianJin-Stars Would you please only update the docs under _docs and please not update the docs under 0.5.0/0.5.1. Thanks.

[GitHub] [incubator-hudi] codecov-io commented on issue #1418: [HUDI-678] Make config package spark free

codecov-io commented on issue #1418: [HUDI-678] Make config package spark free URL: https://github.com/apache/incubator-hudi/pull/1418#issuecomment-601151728 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1418?src=pr&el=h1) Report > Merging [#1418](https://codecov.io/gh/a

[GitHub] [incubator-hudi] codecov-io edited a comment on issue #1418: [HUDI-678] Make config package spark free

codecov-io edited a comment on issue #1418: [HUDI-678] Make config package spark free URL: https://github.com/apache/incubator-hudi/pull/1418#issuecomment-601151728 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1418?src=pr&el=h1) Report > Merging [#1418](https://codecov

  1   2   >