[GitHub] [hudi] kpurella opened a new issue #2062: [SUPPORT] Duplicates in _ro and _rt table for MOR Table type

2020-09-01 Thread GitBox
kpurella opened a new issue #2062: URL: https://github.com/apache/hudi/issues/2062 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://cwiki.apache.org/confluence/display/HUDI/FAQ)? yes - Join the mailing list to engage in conversations and get faster

[GitHub] [hudi] sathyaprakashg commented on pull request #2012: HUDI-1129 Deltastreamer Add support for schema evolution

2020-09-01 Thread GitBox
sathyaprakashg commented on pull request #2012: URL: https://github.com/apache/hudi/pull/2012#issuecomment-685308504 @n3nash I am working on fixing build issue and will have that fix pushed soon. I would like to point out that with this new approach, we are stroing writer schema part of pa

[GitHub] [hudi] n3nash commented on pull request #2012: HUDI-1129 Deltastreamer Add support for schema evolution

2020-09-01 Thread GitBox
n3nash commented on pull request #2012: URL: https://github.com/apache/hudi/pull/2012#issuecomment-685269655 @sathyaprakashg Looks good to me. Can you please see why the build is failing ? This is an automated message from t

[GitHub] [hudi] vinothchandar commented on pull request #1827: [HUDI-1089] Refactor hudi-client to support multi-engine

2020-09-01 Thread GitBox
vinothchandar commented on pull request #1827: URL: https://github.com/apache/hudi/pull/1827#issuecomment-685234513 @wangxianghu sounds good. Thanks for this monumental effort! Will start in the earliest. can we get the CI to pass ?

[GitHub] [hudi] vinothchandar opened a new pull request #2061: [DOCS] Adding coding guidelines

2020-09-01 Thread GitBox
vinothchandar opened a new pull request #2061: URL: https://github.com/apache/hudi/pull/2061 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of t

[GitHub] [hudi] hj2016 commented on a change in pull request #1978: [HUDI-1184] Fix the support of hbase index partition path change

2020-09-01 Thread GitBox
hj2016 commented on a change in pull request #1978: URL: https://github.com/apache/hudi/pull/1978#discussion_r481536312 ## File path: hudi-client/src/main/java/org/apache/hudi/index/hbase/HBaseIndex.java ## @@ -213,36 +215,61 @@ private boolean checkIfValidCommit(HoodieTableMet

[jira] [Commented] (HUDI-1207) Add kafka implementation of write commit callback to Spark datasources

2020-09-01 Thread wangxianghu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17188897#comment-17188897 ] wangxianghu commented on HUDI-1207: --- [~vinoth], thanks for the reply. Your view make se

[jira] [Updated] (HUDI-1207) Add kafka implementation of write commit callback to Spark datasources

2020-09-01 Thread wangxianghu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-1207: -- Summary: Add kafka implementation of write commit callback to Spark datasources (was: Add kafka impleme

[jira] [Updated] (HUDI-1207) Add kafka implementation of write commit callback to hudi-spark module

2020-09-01 Thread wangxianghu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-1207: -- Summary: Add kafka implementation of write commit callback to hudi-spark module (was: Move kafka implem

[jira] [Created] (HUDI-1266) Add e2e integration tests for replace and insert-overwrite

2020-09-01 Thread satish (Jira)
satish created HUDI-1266: Summary: Add e2e integration tests for replace and insert-overwrite Key: HUDI-1266 URL: https://issues.apache.org/jira/browse/HUDI-1266 Project: Apache Hudi Issue Type: Sub-

[jira] [Updated] (HUDI-619) Investigate and implement mechanism to have hive/presto/sparksql queries avoid stitching and return null values for hoodie columns

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-619: Parent: (was: HUDI-1265) Issue Type: Improvement (was: Sub-task) > Investigate a

[jira] [Assigned] (HUDI-619) Investigate and implement mechanism to have hive/presto/sparksql queries avoid stitching and return null values for hoodie columns

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-619: --- Assignee: (was: Balaji Varadarajan) > Investigate and implement mechanism to have

[jira] [Commented] (HUDI-619) Investigate and implement mechanism to have hive/presto/sparksql queries avoid stitching and return null values for hoodie columns

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17188799#comment-17188799 ] Balaji Varadarajan commented on HUDI-619: - We will be focussed on consolidated meta

[jira] [Updated] (HUDI-619) Investigate and implement mechanism to have hive/presto/sparksql queries avoid stitching and return null values for hoodie columns

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-619: Status: Open (was: New) > Investigate and implement mechanism to have hive/presto/sparksql q

[jira] [Assigned] (HUDI-1001) Add implementation to translate source partition paths when doing metadata bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-1001: Assignee: (was: Balaji Varadarajan) > Add implementation to translate source pa

[jira] [Updated] (HUDI-1001) Add implementation to translate source partition paths when doing metadata bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1001: - Parent: (was: HUDI-1265) Issue Type: New Feature (was: Sub-task) > Add implem

[jira] [Commented] (HUDI-1001) Add implementation to translate source partition paths when doing metadata bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17188797#comment-17188797 ] Balaji Varadarajan commented on HUDI-1001: -- THere is no concrete requirement for

[jira] [Commented] (HUDI-1060) Create plugin for bootstrapping iceberg, delta and hudi tables

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17188794#comment-17188794 ] Balaji Varadarajan commented on HUDI-1060: -- This is not high priority and will be

[jira] [Updated] (HUDI-1060) Create plugin for bootstrapping iceberg, delta and hudi tables

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1060: - Parent: (was: HUDI-1265) Issue Type: Improvement (was: Sub-task) > Create plu

[jira] [Assigned] (HUDI-1265) Followup Tasks for Bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-1265: Assignee: Udit Mehrotra > Followup Tasks for Bootstrap > --

[jira] [Resolved] (HUDI-242) [RFC-12] Support Efficient bootstrap of large parquet datasets to Hudi

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan resolved HUDI-242. - Resolution: Fixed > [RFC-12] Support Efficient bootstrap of large parquet datasets to Hudi

[jira] [Reopened] (HUDI-242) [RFC-12] Support Efficient bootstrap of large parquet datasets to Hudi

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reopened HUDI-242: - > [RFC-12] Support Efficient bootstrap of large parquet datasets to Hudi >

[jira] [Closed] (HUDI-242) [RFC-12] Support Efficient bootstrap of large parquet datasets to Hudi

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan closed HUDI-242. --- > [RFC-12] Support Efficient bootstrap of large parquet datasets to Hudi >

[jira] [Resolved] (HUDI-899) Add a knob to change partition-path style while performing metadata bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan resolved HUDI-899. - Resolution: Fixed > Add a knob to change partition-path style while performing metadata boo

[jira] [Reopened] (HUDI-899) Add a knob to change partition-path style while performing metadata bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reopened HUDI-899: - > Add a knob to change partition-path style while performing metadata bootstrap > -

[jira] [Updated] (HUDI-242) [RFC-12] Support Efficient bootstrap of large parquet datasets to Hudi

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-242: Status: Closed (was: Patch Available) > [RFC-12] Support Efficient bootstrap of large parque

[jira] [Updated] (HUDI-899) Add a knob to change partition-path style while performing metadata bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-899: Status: Closed (was: Patch Available) > Add a knob to change partition-path style while perf

[jira] [Closed] (HUDI-899) Add a knob to change partition-path style while performing metadata bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan closed HUDI-899. --- > Add a knob to change partition-path style while performing metadata bootstrap > -

[jira] [Reopened] (HUDI-428) Web documentation for explaining how to bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reopened HUDI-428: - > Web documentation for explaining how to bootstrap >

[jira] [Closed] (HUDI-428) Web documentation for explaining how to bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan closed HUDI-428. --- > Web documentation for explaining how to bootstrap >

[jira] [Updated] (HUDI-428) Web documentation for explaining how to bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-428: Status: Closed (was: Patch Available) > Web documentation for explaining how to bootstrap >

[jira] [Assigned] (HUDI-428) Web documentation for explaining how to bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-428: --- Assignee: Balaji Varadarajan > Web documentation for explaining how to bootstrap > --

[jira] [Resolved] (HUDI-428) Web documentation for explaining how to bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan resolved HUDI-428. - Resolution: Fixed > Web documentation for explaining how to bootstrap > --

[jira] [Updated] (HUDI-1060) Create plugin for bootstrapping iceberg, delta and hudi tables

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1060: - Parent: HUDI-1265 (was: HUDI-242) > Create plugin for bootstrapping iceberg, delta and hu

[jira] [Updated] (HUDI-955) Test MOR : Presto Read Optimized Query with metadata bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-955: Parent: HUDI-1265 (was: HUDI-242) > Test MOR : Presto Read Optimized Query with metadata boo

[jira] [Updated] (HUDI-992) For hive-style partitioned source data, partition columns synced with Hive will always have String type

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-992: Parent: HUDI-1265 (was: HUDI-242) > For hive-style partitioned source data, partition column

[jira] [Updated] (HUDI-915) Partition Columns missing in files upserted after Metadata Bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-915: Parent: HUDI-1265 (was: HUDI-242) > Partition Columns missing in files upserted after Metada

[jira] [Updated] (HUDI-619) Investigate and implement mechanism to have hive/presto/sparksql queries avoid stitching and return null values for hoodie columns

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-619: Parent: HUDI-1265 (was: HUDI-242) > Investigate and implement mechanism to have hive/presto/

[jira] [Updated] (HUDI-1142) Complete remaining code review comments/follow ups

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1142: - Parent: HUDI-1265 (was: HUDI-242) > Complete remaining code review comments/follow ups >

[jira] [Updated] (HUDI-954) Test COW : Presto Read Optimized Query with metadata bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-954: Parent: HUDI-1265 (was: HUDI-242) > Test COW : Presto Read Optimized Query with metadata boo

[jira] [Updated] (HUDI-956) Test COW : Presto Realtime Query with metadata bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-956: Parent: HUDI-1265 (was: HUDI-242) > Test COW : Presto Realtime Query with metadata bootstrap

[jira] [Updated] (HUDI-1157) Optimization whether to query Bootstrapped table using HoodieBootstrapRelation vs Sparks Parquet datasource

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1157: - Parent: HUDI-1265 (was: HUDI-242) > Optimization whether to query Bootstrapped table usin

[jira] [Updated] (HUDI-1021) [Bug] Unable to update bootstrapped table using rows from the written bootstrapped table

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1021: - Parent: HUDI-1265 (was: HUDI-242) > [Bug] Unable to update bootstrapped table using rows

[jira] [Updated] (HUDI-621) Presto Integration for supporting Bootstrapped table

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-621: Parent: HUDI-1265 (was: HUDI-242) > Presto Integration for supporting Bootstrapped table > -

[jira] [Updated] (HUDI-1001) Add implementation to translate source partition paths when doing metadata bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1001: - Parent: HUDI-1265 (was: HUDI-242) > Add implementation to translate source partition path

[jira] [Updated] (HUDI-1265) Followup Tasks for Bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1265: - Status: Open (was: New) > Followup Tasks for Bootstrap > > >

[jira] [Updated] (HUDI-1265) Followup Tasks for Bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1265: - Status: New (was: Open) > Followup Tasks for Bootstrap > > >

[jira] [Created] (HUDI-1265) Followup Tasks for Bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
Balaji Varadarajan created HUDI-1265: Summary: Followup Tasks for Bootstrap Key: HUDI-1265 URL: https://issues.apache.org/jira/browse/HUDI-1265 Project: Apache Hudi Issue Type: Improvemen

[jira] [Updated] (HUDI-1265) Followup Tasks for Bootstrap

2020-09-01 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1265: - Status: Open (was: New) > Followup Tasks for Bootstrap > > >

[hudi] branch asf-site updated: Travis CI build asf-site

2020-09-01 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new e109837 Travis CI build asf-site e109837 is d

[hudi] 02/02: [BLOG] Async Compaction Deployment Models

2020-09-01 Thread vbalaji
This is an automated email from the ASF dual-hosted git repository. vbalaji pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git commit 61305423b5b87b18ffd556e67f49ce8804a89dc8 Author: Balaji Varadarajan AuthorDate: Sat Aug 22 03:19:49 2020 -0700 [BL

[hudi] 01/02: [BLOG] Efficient Migration of large Parquet tables

2020-09-01 Thread vbalaji
This is an automated email from the ASF dual-hosted git repository. vbalaji pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git commit 69783ae794a3045b5ae642213907d4254026706c Author: Balaji Varadarajan AuthorDate: Thu Aug 20 03:19:34 2020 -0700 [BL

[GitHub] [hudi] bvaradar merged pull request #1996: [BLOG] Async Compaction and Efficient Migration of large Parquet tables

2020-09-01 Thread GitBox
bvaradar merged pull request #1996: URL: https://github.com/apache/hudi/pull/1996 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[hudi] branch asf-site updated (b9f5826 -> 6130542)

2020-09-01 Thread vbalaji
This is an automated email from the ASF dual-hosted git repository. vbalaji pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git. from b9f5826 Travis CI build asf-site new 69783ae [BLOG] Efficient Migration of large Parquet tables new 6130

[jira] [Assigned] (HUDI-1263) DeltaStreamer changes to support insert overwrite and replace

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish reassigned HUDI-1263: Assignee: satish > DeltaStreamer changes to support insert overwrite and replace >

[jira] [Assigned] (HUDI-1261) CLI tools update to support REPLACE and insert overwrite

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish reassigned HUDI-1261: Assignee: satish > CLI tools update to support REPLACE and insert overwrite > -

[jira] [Assigned] (HUDI-1262) Documentation Update for Insert Overwrite

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish reassigned HUDI-1262: Assignee: satish > Documentation Update for Insert Overwrite > - >

[jira] [Assigned] (HUDI-1260) Reader changes to supportinsert overwrite

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish reassigned HUDI-1260: Assignee: satish > Reader changes to supportinsert overwrite > - >

[jira] [Created] (HUDI-1264) incremental read support with replace

2020-09-01 Thread satish (Jira)
satish created HUDI-1264: Summary: incremental read support with replace Key: HUDI-1264 URL: https://issues.apache.org/jira/browse/HUDI-1264 Project: Apache Hudi Issue Type: Sub-task Repo

[jira] [Created] (HUDI-1263) DeltaStreamer changes to support insert overwrite and replace

2020-09-01 Thread satish (Jira)
satish created HUDI-1263: Summary: DeltaStreamer changes to support insert overwrite and replace Key: HUDI-1263 URL: https://issues.apache.org/jira/browse/HUDI-1263 Project: Apache Hudi Issue Type:

[jira] [Created] (HUDI-1262) Documentation Update for Insert Overwrite

2020-09-01 Thread satish (Jira)
satish created HUDI-1262: Summary: Documentation Update for Insert Overwrite Key: HUDI-1262 URL: https://issues.apache.org/jira/browse/HUDI-1262 Project: Apache Hudi Issue Type: Sub-task

[jira] [Created] (HUDI-1261) CLI tools update to support REPLACE and insert overwrite

2020-09-01 Thread satish (Jira)
satish created HUDI-1261: Summary: CLI tools update to support REPLACE and insert overwrite Key: HUDI-1261 URL: https://issues.apache.org/jira/browse/HUDI-1261 Project: Apache Hudi Issue Type: Sub-ta

[jira] [Updated] (HUDI-1260) Reader changes to supportinsert overwrite

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-1260: - Description: Same as HUDI-1072, but creating subtask for insert overwrite (was: HUDI-1072) > Reader changes to s

[jira] [Updated] (HUDI-868) Insert Overwrite API

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-868: Issue Type: New Feature (was: Improvement) > Insert Overwrite API > > > Key: HU

[jira] [Created] (HUDI-1260) Reader changes to supportinsert overwrite

2020-09-01 Thread satish (Jira)
satish created HUDI-1260: Summary: Reader changes to supportinsert overwrite Key: HUDI-1260 URL: https://issues.apache.org/jira/browse/HUDI-1260 Project: Apache Hudi Issue Type: Sub-task

[jira] [Assigned] (HUDI-868) Insert Overwrite API

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish reassigned HUDI-868: --- Assignee: satish > Insert Overwrite API > > > Key: HUDI-868 >

[jira] [Updated] (HUDI-1228) create a utility to query extra metadata

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-1228: - Status: Patch Available (was: In Progress) > create a utility to query extra metadata > -

[jira] [Updated] (HUDI-1228) create a utility to query extra metadata

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-1228: - Status: In Progress (was: Open) > create a utility to query extra metadata >

[jira] [Updated] (HUDI-1228) create a utility to query extra metadata

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-1228: - Status: Open (was: New) > create a utility to query extra metadata > > >

[jira] [Updated] (HUDI-1228) create a utility to query extra metadata

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-1228: - Status: Closed (was: Patch Available) > create a utility to query extra metadata > --

[jira] [Updated] (HUDI-1191) create incremental meta client abstraction to query modified partitions

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-1191: - Status: In Progress (was: Open) > create incremental meta client abstraction to query modified partitions > -

[jira] [Updated] (HUDI-1191) create incremental meta client abstraction to query modified partitions

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-1191: - Status: Closed (was: Patch Available) > create incremental meta client abstraction to query modified partitions >

[jira] [Updated] (HUDI-1226) ComplexKeyGenerator doesnt work for non partitioned tables

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-1226: - Status: In Progress (was: Open) > ComplexKeyGenerator doesnt work for non partitioned tables > --

[jira] [Updated] (HUDI-1226) ComplexKeyGenerator doesnt work for non partitioned tables

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-1226: - Status: Open (was: New) > ComplexKeyGenerator doesnt work for non partitioned tables > --

[jira] [Updated] (HUDI-1191) create incremental meta client abstraction to query modified partitions

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-1191: - Status: Open (was: New) > create incremental meta client abstraction to query modified partitions > -

[jira] [Updated] (HUDI-1191) create incremental meta client abstraction to query modified partitions

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-1191: - Status: In Progress (was: Open) > create incremental meta client abstraction to query modified partitions > -

[jira] [Updated] (HUDI-1191) create incremental meta client abstraction to query modified partitions

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-1191: - Status: Patch Available (was: In Progress) > create incremental meta client abstraction to query modified partiti

[jira] [Updated] (HUDI-1226) ComplexKeyGenerator doesnt work for non partitioned tables

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-1226: - Status: Closed (was: Patch Available) > ComplexKeyGenerator doesnt work for non partitioned tables >

[jira] [Updated] (HUDI-1226) ComplexKeyGenerator doesnt work for non partitioned tables

2020-09-01 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-1226: - Status: Patch Available (was: In Progress) > ComplexKeyGenerator doesnt work for non partitioned tables > ---

[GitHub] [hudi] satishkotha closed pull request #1859: [WIP] [HUDI-1072] Use replace metadata file to filter excluded files in views

2020-09-01 Thread GitBox
satishkotha closed pull request #1859: URL: https://github.com/apache/hudi/pull/1859 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] satishkotha commented on pull request #1859: [WIP] [HUDI-1072] Use replace metadata file to filter excluded files in views

2020-09-01 Thread GitBox
satishkotha commented on pull request #1859: URL: https://github.com/apache/hudi/pull/1859#issuecomment-685071442 Moved to #2048 This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hudi] satishkotha commented on a change in pull request #2048: [HUDI-1072][WIP] Introduce REPLACE top level action

2020-09-01 Thread GitBox
satishkotha commented on a change in pull request #2048: URL: https://github.com/apache/hudi/pull/2048#discussion_r481365985 ## File path: hudi-client/src/main/java/org/apache/hudi/table/HoodieTimelineArchiveLog.java ## @@ -301,6 +304,44 @@ private void deleteAnyLeftOverMarker

[GitHub] [hudi] satishkotha commented on a change in pull request #2048: [HUDI-1072][WIP] Introduce REPLACE top level action

2020-09-01 Thread GitBox
satishkotha commented on a change in pull request #2048: URL: https://github.com/apache/hudi/pull/2048#discussion_r481365742 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/view/SpillableMapBasedFileSystemView.java ## @@ -77,6 +79,13 @@ public SpillableMap

[GitHub] [hudi] satishkotha commented on a change in pull request #2048: [HUDI-1072][WIP] Introduce REPLACE top level action

2020-09-01 Thread GitBox
satishkotha commented on a change in pull request #2048: URL: https://github.com/apache/hudi/pull/2048#discussion_r481364995 ## File path: hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieCommonTestHarness.java ## @@ -104,4 +104,8 @@ protected SyncableFileSyste

[GitHub] [hudi] satishkotha commented on a change in pull request #2048: [HUDI-1072][WIP] Introduce REPLACE top level action

2020-09-01 Thread GitBox
satishkotha commented on a change in pull request #2048: URL: https://github.com/apache/hudi/pull/2048#discussion_r481364462 ## File path: hudi-common/src/main/java/org/apache/hudi/common/model/HoodieReplaceStat.java ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software

[GitHub] [hudi] satishkotha commented on a change in pull request #2048: [HUDI-1072][WIP] Introduce REPLACE top level action

2020-09-01 Thread GitBox
satishkotha commented on a change in pull request #2048: URL: https://github.com/apache/hudi/pull/2048#discussion_r481364171 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java ## @@ -173,29 +180,59 @@ protected void refre

[GitHub] [hudi] n3nash commented on pull request #1704: [HUDI-115] Enhance OverwriteWithLatestAvroPayload to also respect ordering value of record in storage

2020-09-01 Thread GitBox
n3nash commented on pull request #1704: URL: https://github.com/apache/hudi/pull/1704#issuecomment-685048439 @bhasudha let me know once you've addressed @nsivabalan comment and the build is passing, can merge this. This is a

[GitHub] [hudi] wangxianghu commented on pull request #1827: [HUDI-1089] Refactor hudi-client to support multi-engine

2020-09-01 Thread GitBox
wangxianghu commented on pull request #1827: URL: https://github.com/apache/hudi/pull/1827#issuecomment-685031856 Hi, @vinothchandar @yanghua @leesf @n3nash Sorry for the delay, This PR is ready for review now :) I have tested this PR by executing the insert, update, query, delete demo

[GitHub] [hudi] WTa-hash edited a comment on issue #2057: [SUPPORT] AWSDmsAvroPayload not processing Deletes correctly + IOException when reading log file

2020-09-01 Thread GitBox
WTa-hash edited a comment on issue #2057: URL: https://github.com/apache/hudi/issues/2057#issuecomment-685015564 For the 0.6.0 issue with error: java.lang.NoSuchMethodError: org.apache.spark.sql.execution.datasources.PartitionedFile.(Lorg/apache/spark/sql/catalyst/InternalRow;Ljava/lang/Str

[GitHub] [hudi] WTa-hash commented on issue #2057: [SUPPORT] AWSDmsAvroPayload not processing Deletes correctly + IOException when reading log file

2020-09-01 Thread GitBox
WTa-hash commented on issue #2057: URL: https://github.com/apache/hudi/issues/2057#issuecomment-685015564 For the 0.6.0 issue with error: java.lang.NoSuchMethodError: org.apache.spark.sql.execution.datasources.PartitionedFile.(Lorg/apache/spark/sql/catalyst/InternalRow;Ljava/lang/String;JJ[

[hudi] branch master updated: [HUDI-993] Let delete API use "hoodie.delete.shuffle.parallelism" (#1703)

2020-09-01 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 8d19ebf [HUDI-993] Let delete API use "hoodie

[GitHub] [hudi] nsivabalan merged pull request #1703: [HUDI-993] Let delete API use "hoodie.delete.shuffle.parallelism"

2020-09-01 Thread GitBox
nsivabalan merged pull request #1703: URL: https://github.com/apache/hudi/pull/1703 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] nsivabalan commented on a change in pull request #1978: [HUDI-1184] Fix the support of hbase index partition path change

2020-09-01 Thread GitBox
nsivabalan commented on a change in pull request #1978: URL: https://github.com/apache/hudi/pull/1978#discussion_r481292427 ## File path: hudi-client/src/test/java/org/apache/hudi/index/hbase/TestHBaseIndex.java ## @@ -156,6 +160,46 @@ public void testSimpleTagLocationAndUpdat

[GitHub] [hudi] bvaradar commented on pull request #1996: [BLOG] Async Compaction and Efficient Migration of large Parquet tables

2020-09-01 Thread GitBox
bvaradar commented on pull request #1996: URL: https://github.com/apache/hudi/pull/1996#issuecomment-684987414 @vinothchandar : Yes, I will go ahead and merge after resolving conflicts This is an automated message from the Ap

[GitHub] [hudi] xushiyan commented on a change in pull request #2060: [DOCS] Update community page

2020-09-01 Thread GitBox
xushiyan commented on a change in pull request #2060: URL: https://github.com/apache/hudi/pull/2060#discussion_r481249646 ## File path: docs/_pages/community.cn.md ## @@ -50,26 +50,27 @@ Committers are chosen by a majority vote of the Apache Hudi [PMC](https://www.ap - Great

[GitHub] [hudi] xushiyan opened a new pull request #2060: [DOCS] Update community page

2020-09-01 Thread GitBox
xushiyan opened a new pull request #2060: URL: https://github.com/apache/hudi/pull/2060 - Update committers list - Update cn page with latest en page content ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributi

[GitHub] [hudi] WTa-hash removed a comment on issue #2057: [SUPPORT] AWSDmsAvroPayload not processing Deletes correctly + IOException when reading log file

2020-09-01 Thread GitBox
WTa-hash removed a comment on issue #2057: URL: https://github.com/apache/hudi/issues/2057#issuecomment-684936471 For java.lang.NoSuchMethodError: org.apache.spark.sql.execution.datasources.PartitionedFile.(Lorg/apache/spark/sql/catalyst/InternalRow;Ljava/lang/String;JJ[Ljava/lang/String;)V

[GitHub] [hudi] WTa-hash commented on issue #2057: [SUPPORT] AWSDmsAvroPayload not processing Deletes correctly + IOException when reading log file

2020-09-01 Thread GitBox
WTa-hash commented on issue #2057: URL: https://github.com/apache/hudi/issues/2057#issuecomment-684936471 For java.lang.NoSuchMethodError: org.apache.spark.sql.execution.datasources.PartitionedFile.(Lorg/apache/spark/sql/catalyst/InternalRow;Ljava/lang/String;JJ[Ljava/lang/String;)V issue:

[GitHub] [hudi] vinothchandar commented on pull request #2058: [HUDI-1259] Cache some framework binaries to speed up the progress of building docker image in local env

2020-09-01 Thread GitBox
vinothchandar commented on pull request #2058: URL: https://github.com/apache/hudi/pull/2058#issuecomment-684914893 @bvaradar knows the best about this. can you please help review when the PR is ready This is an automated m

[jira] [Commented] (HUDI-1200) CustomKeyGenerator does not work,java.lang.NullPointerException

2020-09-01 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17188525#comment-17188525 ] Vinoth Chandar commented on HUDI-1200: -- [~liujinhui] I assume you unblocked yourself

  1   2   >