[GitHub] [incubator-hudi] hddong commented on a change in pull request #1511: [HUDI-789]Adjust logic of upsert in HDFSParquetImporter

hddong commented on a change in pull request #1511: [HUDI-789]Adjust logic of upsert in HDFSParquetImporter URL: https://github.com/apache/incubator-hudi/pull/1511#discussion_r410005068 ## File path:

[GitHub] [incubator-hudi] EdwinGuo opened a new pull request #1526: Hudi 783

EdwinGuo opened a new pull request #1526: Hudi 783 URL: https://github.com/apache/incubator-hudi/pull/1526 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is

[jira] [Commented] (HUDI-773) Hudi On Azure Data Lake Storage V2

[ https://issues.apache.org/jira/browse/HUDI-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085409#comment-17085409 ] Yanjia Gary Li commented on HUDI-773: - Hello [~sasikumar.venkat], thanks for sharing! I am able to

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #250

See Changes: -- [...truncated 2.36 KB...] /home/jenkins/tools/maven/apache-maven-3.5.4/conf: logging settings.xml toolchains.xml

[jira] [Comment Edited] (HUDI-773) Hudi On Azure Data Lake Storage V2

[ https://issues.apache.org/jira/browse/HUDI-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085380#comment-17085380 ] Sasikumar Venkatesh edited comment on HUDI-773 at 4/17/20, 3:38 AM: Hi

[jira] [Commented] (HUDI-773) Hudi On Azure Data Lake Storage V2

[ https://issues.apache.org/jira/browse/HUDI-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085380#comment-17085380 ] Sasikumar Venkatesh commented on HUDI-773: -- Hi [~vinoth] and [~garyli1019] I am experimenting

[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource

xushiyan commented on a change in pull request #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource URL: https://github.com/apache/incubator-hudi/pull/1500#discussion_r409931981 ## File path: hudi-spark/src/test/java/DataSourceUtilsTest.java

[jira] [Commented] (HUDI-716) Exception: Not an Avro data file when running HoodieCleanClient.runClean

[ https://issues.apache.org/jira/browse/HUDI-716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085345#comment-17085345 ] lamber-ken commented on HUDI-716: - [https://github.com/apache/incubator-hudi/pull/1432] > Exception: Not

[GitHub] [incubator-hudi] lamber-ken commented on issue #1525: Fixing param parsing. It is array of array

lamber-ken commented on issue #1525: Fixing param parsing. It is array of array URL: https://github.com/apache/incubator-hudi/pull/1525#issuecomment-614948239 hi @afilipchik, thanks for your contribution, If the changes are minor, then [MINOR] prefix can be added to Pull Request title

[jira] [Commented] (HUDI-783) Add official python support to create hudi datasets using pyspark

[ https://issues.apache.org/jira/browse/HUDI-783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085299#comment-17085299 ] Vinoth Chandar commented on HUDI-783: - I think we can skip spark-packages.org for now. the repo is

[GitHub] [incubator-hudi] vinothchandar commented on issue #1524: Adding a way to post process schema after it is fetched

vinothchandar commented on issue #1524: Adding a way to post process schema after it is fetched URL: https://github.com/apache/incubator-hudi/pull/1524#issuecomment-614931414 Please create a JIRA for this work, tagged against 0.6.0 release

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1518: [HUDI-723] Register avro schema if infered from SQL transformation

vinothchandar commented on a change in pull request #1518: [HUDI-723] Register avro schema if infered from SQL transformation URL: https://github.com/apache/incubator-hudi/pull/1518#discussion_r409840574 ## File path:

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1518: [HUDI-723] Register avro schema if infered from SQL transformation

vinothchandar commented on a change in pull request #1518: [HUDI-723] Register avro schema if infered from SQL transformation URL: https://github.com/apache/incubator-hudi/pull/1518#discussion_r409887222 ## File path:

[GitHub] [incubator-hudi] kwondw commented on a change in pull request #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource

kwondw commented on a change in pull request #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource URL: https://github.com/apache/incubator-hudi/pull/1500#discussion_r409868821 ## File path:

[jira] [Commented] (HUDI-716) Exception: Not an Avro data file when running HoodieCleanClient.runClean

[ https://issues.apache.org/jira/browse/HUDI-716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085284#comment-17085284 ] Alexander Filipchik commented on HUDI-716: -- [~lamber-ken] what class? Don't see a fix in the

[GitHub] [incubator-hudi] kwondw commented on a change in pull request #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource

kwondw commented on a change in pull request #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource URL: https://github.com/apache/incubator-hudi/pull/1500#discussion_r409868673 ## File path: hudi-spark/src/main/java/org/apache/hudi/DataSourceUtils.java

[GitHub] [incubator-hudi] afilipchik opened a new pull request #1525: Fixing param parsing. It is array of array

afilipchik opened a new pull request #1525: Fixing param parsing. It is array of array URL: https://github.com/apache/incubator-hudi/pull/1525 ## What is the purpose of the pull request Addition of transformers broke deltastreamer. Current parsing logic ends up passing Array>

[jira] [Commented] (HUDI-783) Add official python support to create hudi datasets using pyspark

[ https://issues.apache.org/jira/browse/HUDI-783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085253#comment-17085253 ] Vinoth Govindarajan commented on HUDI-783: -- Thanks, [~vinoth]! We don't need to write any wrapper

[incubator-hudi] branch master updated (acdc4a8 -> acb1ada)

This is an automated email from the ASF dual-hosted git repository. vinoth pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git. from acdc4a8 [HUDI-798] Migrate to Mockito Jupiter for JUnit 5 (#1521) add acb1ada [HUDI-799] Use

[jira] [Updated] (HUDI-799) DeltaStreamer must use appropriate FS when loading configs

[ https://issues.apache.org/jira/browse/HUDI-799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-799: Labels: pull-request-available (was: ) > DeltaStreamer must use appropriate FS when loading configs

[GitHub] [incubator-hudi] vinothchandar merged pull request #1517: [HUDI-799] Use appropriate FS when loading configs

vinothchandar merged pull request #1517: [HUDI-799] Use appropriate FS when loading configs URL: https://github.com/apache/incubator-hudi/pull/1517 This is an automated message from the Apache Git Service. To respond to the

[jira] [Updated] (HUDI-799) DeltaStreamer must use appropriate FS when loading configs

[ https://issues.apache.org/jira/browse/HUDI-799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-799: Fix Version/s: 0.6.0 > DeltaStreamer must use appropriate FS when loading configs >

[jira] [Created] (HUDI-799) DeltaStreamer must use appropriate FS when loading configs

Vinoth Chandar created HUDI-799: --- Summary: DeltaStreamer must use appropriate FS when loading configs Key: HUDI-799 URL: https://issues.apache.org/jira/browse/HUDI-799 Project: Apache Hudi (incubating)

[GitHub] [incubator-hudi] vinothchandar commented on issue #1513: [HUDI-793] Adding proper default to hudi metadata fields and proper handling to rewrite routine

vinothchandar commented on issue #1513: [HUDI-793] Adding proper default to hudi metadata fields and proper handling to rewrite routine URL: https://github.com/apache/incubator-hudi/pull/1513#issuecomment-614885624 @pratyakshsharma I will take another pass once you and @afilipchik are on

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1469: [HUDI-686] Implement BloomIndexV2 that does not depend on memory caching

vinothchandar commented on a change in pull request #1469: [HUDI-686] Implement BloomIndexV2 that does not depend on memory caching URL: https://github.com/apache/incubator-hudi/pull/1469#discussion_r409828449 ## File path:

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1469: [HUDI-686] Implement BloomIndexV2 that does not depend on memory caching

vinothchandar commented on a change in pull request #1469: [HUDI-686] Implement BloomIndexV2 that does not depend on memory caching URL: https://github.com/apache/incubator-hudi/pull/1469#discussion_r409827627 ## File path:

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1469: [HUDI-686] Implement BloomIndexV2 that does not depend on memory caching

vinothchandar commented on a change in pull request #1469: [HUDI-686] Implement BloomIndexV2 that does not depend on memory caching URL: https://github.com/apache/incubator-hudi/pull/1469#discussion_r409759610 ## File path:

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1469: [HUDI-686] Implement BloomIndexV2 that does not depend on memory caching

vinothchandar commented on a change in pull request #1469: [HUDI-686] Implement BloomIndexV2 that does not depend on memory caching URL: https://github.com/apache/incubator-hudi/pull/1469#discussion_r409827146 ## File path:

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1469: [HUDI-686] Implement BloomIndexV2 that does not depend on memory caching

vinothchandar commented on a change in pull request #1469: [HUDI-686] Implement BloomIndexV2 that does not depend on memory caching URL: https://github.com/apache/incubator-hudi/pull/1469#discussion_r409827011 ## File path:

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1469: [HUDI-686] Implement BloomIndexV2 that does not depend on memory caching

vinothchandar commented on a change in pull request #1469: [HUDI-686] Implement BloomIndexV2 that does not depend on memory caching URL: https://github.com/apache/incubator-hudi/pull/1469#discussion_r409828710 ## File path:

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1469: [HUDI-686] Implement BloomIndexV2 that does not depend on memory caching

vinothchandar commented on a change in pull request #1469: [HUDI-686] Implement BloomIndexV2 that does not depend on memory caching URL: https://github.com/apache/incubator-hudi/pull/1469#discussion_r409735330 ## File path:

[GitHub] [incubator-hudi] afilipchik opened a new pull request #1524: Adding a way to post process schema after it is fetched

afilipchik opened a new pull request #1524: Adding a way to post process schema after it is fetched URL: https://github.com/apache/incubator-hudi/pull/1524 ## What is the purpose of the pull request This PR adds extension point to SchemaProvider. Sometimes it is needed to

[GitHub] [incubator-hudi] nsivabalan commented on issue #1402: [WIP][HUDI-407] Adding Simple Index

nsivabalan commented on issue #1402: [WIP][HUDI-407] Adding Simple Index URL: https://github.com/apache/incubator-hudi/pull/1402#issuecomment-614864302 > @nsivabalan confirming the next steps for this patch... getting the global indexing as well supported (probably need to support the

[incubator-hudi] branch asf-site updated: Travis CI build asf-site

This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new abb90a58 Travis CI build asf-site

[incubator-hudi] branch asf-site updated: [MINOR] Fix broken README link (#1523)

This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new a3715c9 [MINOR] Fix broken README

[GitHub] [incubator-hudi] vinothchandar merged pull request #1523: [MINOR] Fix broken README link

vinothchandar merged pull request #1523: [MINOR] Fix broken README link URL: https://github.com/apache/incubator-hudi/pull/1523 This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Resolved] (HUDI-741) Fix Hoodie's schema evolution checks

[ https://issues.apache.org/jira/browse/HUDI-741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason resolved HUDI-741. - Resolution: Fixed > Fix Hoodie's schema evolution checks > >

[jira] [Resolved] (HUDI-49) Handle non-backwards compatible or smaller schema appended to the log file during updates #337

[ https://issues.apache.org/jira/browse/HUDI-49?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason resolved HUDI-49. Resolution: Fixed > Handle non-backwards compatible or smaller schema appended to the log file >

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource

vinothchandar commented on a change in pull request #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource URL: https://github.com/apache/incubator-hudi/pull/1500#discussion_r409751909 ## File path:

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource

vinothchandar commented on a change in pull request #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource URL: https://github.com/apache/incubator-hudi/pull/1500#discussion_r409751557 ## File path: hudi-spark/src/test/java/DataSourceUtilsTest.java

[GitHub] [incubator-hudi] vinothchandar commented on issue #1151: [WIP][HUDI-476] Add hudi-examples module

vinothchandar commented on issue #1151: [WIP][HUDI-476] Add hudi-examples module URL: https://github.com/apache/incubator-hudi/pull/1151#issuecomment-614795517 >we can just delete the 3 XxxDeltaStreamerExample and replace them with a run_examples.sh and some config files? yes..

[GitHub] [incubator-hudi] vinothchandar commented on issue #1402: [WIP][HUDI-407] Adding Simple Index

vinothchandar commented on issue #1402: [WIP][HUDI-407] Adding Simple Index URL: https://github.com/apache/incubator-hudi/pull/1402#issuecomment-614794767 @nsivabalan confirming the next steps for this patch... getting the global indexing as well supported (probably need to support the

[GitHub] [incubator-hudi] lamber-ken opened a new pull request #1523: [MINOR] Fix broken README link

lamber-ken opened a new pull request #1523: [MINOR] Fix broken README link URL: https://github.com/apache/incubator-hudi/pull/1523 ## What is the purpose of the pull request Fix broken doc link. Change `https://github.com/apache/incubator-hudi/blob/asf-site` to

[jira] [Comment Edited] (HUDI-791) Replace null by Option in Delta Streamer

[ https://issues.apache.org/jira/browse/HUDI-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17084932#comment-17084932 ] Zili Chen edited comment on HUDI-791 at 4/16/20, 2:46 PM: -- I've taken a look at

[jira] [Commented] (HUDI-791) Replace null by Option in Delta Streamer

[ https://issues.apache.org/jira/browse/HUDI-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17084932#comment-17084932 ] Zili Chen commented on HUDI-791: I've taken a look at the {{null}}s. It seems most as field or parameter.

[GitHub] [incubator-hudi] yanghua commented on a change in pull request #1511: [HUDI-789]Adjust logic of upsert in HDFSParquetImporter

yanghua commented on a change in pull request #1511: [HUDI-789]Adjust logic of upsert in HDFSParquetImporter URL: https://github.com/apache/incubator-hudi/pull/1511#discussion_r409395668 ## File path:

[GitHub] [incubator-hudi] yanghua commented on a change in pull request #1511: [HUDI-789]Adjust logic of upsert in HDFSParquetImporter

yanghua commented on a change in pull request #1511: [HUDI-789]Adjust logic of upsert in HDFSParquetImporter URL: https://github.com/apache/incubator-hudi/pull/1511#discussion_r409396156 ## File path:

[GitHub] [incubator-hudi] yanghua commented on a change in pull request #1511: [HUDI-789]Adjust logic of upsert in HDFSParquetImporter

yanghua commented on a change in pull request #1511: [HUDI-789]Adjust logic of upsert in HDFSParquetImporter URL: https://github.com/apache/incubator-hudi/pull/1511#discussion_r409399060 ## File path:

[GitHub] [incubator-hudi] yanghua commented on a change in pull request #1511: [HUDI-789]Adjust logic of upsert in HDFSParquetImporter

yanghua commented on a change in pull request #1511: [HUDI-789]Adjust logic of upsert in HDFSParquetImporter URL: https://github.com/apache/incubator-hudi/pull/1511#discussion_r409389425 ## File path:

[GitHub] [incubator-hudi] yanghua commented on a change in pull request #1511: [HUDI-789]Adjust logic of upsert in HDFSParquetImporter

yanghua commented on a change in pull request #1511: [HUDI-789]Adjust logic of upsert in HDFSParquetImporter URL: https://github.com/apache/incubator-hudi/pull/1511#discussion_r409393206 ## File path:

[GitHub] [incubator-hudi] yanghua commented on a change in pull request #1511: [HUDI-789]Adjust logic of upsert in HDFSParquetImporter

yanghua commented on a change in pull request #1511: [HUDI-789]Adjust logic of upsert in HDFSParquetImporter URL: https://github.com/apache/incubator-hudi/pull/1511#discussion_r409396839 ## File path:

[jira] [Closed] (HUDI-798) Migrate Mockito to work with Junit 5

[ https://issues.apache.org/jira/browse/HUDI-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang closed HUDI-798. - Resolution: Done Done via master branch: acdc4a8d004394590f9e5ffcc703ea23624e66d9 > Migrate Mockito to work with

[incubator-hudi] branch master updated: [HUDI-798] Migrate to Mockito Jupiter for JUnit 5 (#1521)

This is an automated email from the ASF dual-hosted git repository. vinoyang pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new acdc4a8 [HUDI-798] Migrate to

[GitHub] [incubator-hudi] yanghua merged pull request #1521: [HUDI-798] Migrate to Mockito Jupiter for JUnit 5

yanghua merged pull request #1521: [HUDI-798] Migrate to Mockito Jupiter for JUnit 5 URL: https://github.com/apache/incubator-hudi/pull/1521 This is an automated message from the Apache Git Service. To respond to the

[jira] [Updated] (HUDI-702) Add unit test for HoodieLogFileCommand

[ https://issues.apache.org/jira/browse/HUDI-702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-702: Labels: pull-request-available (was: ) > Add unit test for HoodieLogFileCommand >

[GitHub] [incubator-hudi] hddong opened a new pull request #1522: [HUDI-702]Add test for HoodieLogFileCommand

hddong opened a new pull request #1522: [HUDI-702]Add test for HoodieLogFileCommand URL: https://github.com/apache/incubator-hudi/pull/1522 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a

[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1521: [HUDI-798] Migrate to Mockito Jupiter for JUnit 5

xushiyan commented on a change in pull request #1521: [HUDI-798] Migrate to Mockito Jupiter for JUnit 5 URL: https://github.com/apache/incubator-hudi/pull/1521#discussion_r409315688 ## File path:

[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1521: [HUDI-798] Migrate to Mockito Jupiter for JUnit 5

xushiyan commented on a change in pull request #1521: [HUDI-798] Migrate to Mockito Jupiter for JUnit 5 URL: https://github.com/apache/incubator-hudi/pull/1521#discussion_r409317776 ## File path:

[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1521: [HUDI-798] Migrate to Mockito Jupiter for JUnit 5

xushiyan commented on a change in pull request #1521: [HUDI-798] Migrate to Mockito Jupiter for JUnit 5 URL: https://github.com/apache/incubator-hudi/pull/1521#discussion_r409314065 ## File path: hudi-client/src/test/java/org/apache/hudi/index/TestHbaseIndex.java ## @@

[incubator-hudi] branch hudi_test_suite_refactor updated (37f91cb -> 7379b8d)

This is an automated email from the ASF dual-hosted git repository. nagarwal pushed a change to branch hudi_test_suite_refactor in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git. omit 37f91cb more fixes add 7379b8d more fixes This update added new revisions after

[incubator-hudi] branch master updated: [HUDI-741] Added checks to validate Hoodie's schema evolution.

This is an automated email from the ASF dual-hosted git repository. nagarwal pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new 19d29ac [HUDI-741] Added checks to

[GitHub] [incubator-hudi] n3nash merged pull request #1457: [HUDI-741] Added checks to validate Hoodie's schema evolution.

n3nash merged pull request #1457: [HUDI-741] Added checks to validate Hoodie's schema evolution. URL: https://github.com/apache/incubator-hudi/pull/1457 This is an automated message from the Apache Git Service. To respond

[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource

xushiyan commented on a change in pull request #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource URL: https://github.com/apache/incubator-hudi/pull/1500#discussion_r409313000 ## File path: hudi-spark/src/test/java/DataSourceUtilsTest.java

[GitHub] [incubator-hudi] xushiyan opened a new pull request #1521: [HUDI-798] Migrate to Mockito Jupiter for JUnit 5

xushiyan opened a new pull request #1521: [HUDI-798] Migrate to Mockito Jupiter for JUnit 5 URL: https://github.com/apache/incubator-hudi/pull/1521 - Remove old mockito dependency - Add mockito-junit-jupiter ## Committer checklist - [ ] Has a corresponding JIRA in PR

[jira] [Updated] (HUDI-798) Migrate Mockito to work with Junit 5

[ https://issues.apache.org/jira/browse/HUDI-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-798: Labels: pull-request-available (was: ) > Migrate Mockito to work with Junit 5 >