[jira] [Commented] (HUDI-2259) [SQL]Support referencing subquery with column aliases by table alias in merge into

2021-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398452#comment-17398452 ] ASF GitHub Bot commented on HUDI-2259: -- dongkelun commented on pull request #3380: URL:

[GitHub] [hudi] dongkelun commented on pull request #3380: [HUDI-2259]Support referencing subquery with column aliases by table alias in me…

2021-08-12 Thread GitBox
dongkelun commented on pull request #3380: URL: https://github.com/apache/hudi/pull/3380#issuecomment-898210263 > > @pengzhiwei2018 Hi,When I test Spark3, I find that Spark SQL for Hoodie with Spark3 uses the source code of Spark, but columns aliases in Merge Into is not supported in

[jira] [Commented] (HUDI-2279) Support column name matching for insert * and update set * in merge into when sourceTable's columns contains all targetTable's columns

2021-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398448#comment-17398448 ] ASF GitHub Bot commented on HUDI-2279: -- dongkelun commented on pull request #3415: URL:

[GitHub] [hudi] dongkelun commented on pull request #3415: [HUDI-2279]Support column name matching for insert * and update set *

2021-08-12 Thread GitBox
dongkelun commented on pull request #3415: URL: https://github.com/apache/hudi/pull/3415#issuecomment-898207251 > Hi @dongkelun , Thanks for the contribution for this. Overall LGTM except some minor optimize. And also you can run the test case in spark3 by the follow command: > > >

[jira] [Commented] (HUDI-2303) TestMereIntoLogOnlyTable with metadata enabled surfaces likely bug

2021-08-12 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398447#comment-17398447 ] Prashant Wason commented on HUDI-2303: -- So this is an issue with HoodieMetadataFileSystemView not

[jira] [Commented] (HUDI-2303) TestMereIntoLogOnlyTable with metadata enabled surfaces likely bug

2021-08-12 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398445#comment-17398445 ] Prashant Wason commented on HUDI-2303: -- The following diff fixes this issue:   {{}} diff --git

[GitHub] [hudi] nsivabalan commented on issue #2919: [SUPPORT] Schema Evolution Failing Spark+Hudi- Adding New fields

2021-08-12 Thread GitBox
nsivabalan commented on issue #2919: URL: https://github.com/apache/hudi/issues/2919#issuecomment-898181212 @tandonraghav : we had a [fix](https://github.com/apache/hudi/pull/3137) sometime back wrt schema evolution and latest master should work. Can you try out w/ latest master and let

[jira] [Commented] (HUDI-2303) TestMereIntoLogOnlyTable with metadata enabled surfaces likely bug

2021-08-12 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398431#comment-17398431 ] Prashant Wason commented on HUDI-2303: -- Confirmed: When the dataset is being compacted, the latest

[jira] [Commented] (HUDI-2119) Syncing of rollbacks to metadata table does not work in all cases

2021-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398430#comment-17398430 ] ASF GitHub Bot commented on HUDI-2119: -- hudi-bot edited a comment on pull request #3210: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3210: [HUDI-2119] Ensure the rolled-back instance was previously synced to the Metadata Table when syncing a Rollback Instant.

2021-08-12 Thread GitBox
hudi-bot edited a comment on pull request #3210: URL: https://github.com/apache/hudi/pull/3210#issuecomment-872541566 ## CI report: * 96c73ba8ed99f51f31efdac0ab60d6b95bc782ed Azure:

[jira] [Assigned] (HUDI-2305) Fix marker-based rollback in 0.9.0

2021-08-12 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-2305: --- Assignee: Ethan Guo > Fix marker-based rollback in 0.9.0 > -- > >

[jira] [Created] (HUDI-2305) Fix marker-based rollback in 0.9.0

2021-08-12 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-2305: --- Summary: Fix marker-based rollback in 0.9.0 Key: HUDI-2305 URL: https://issues.apache.org/jira/browse/HUDI-2305 Project: Apache Hudi Issue Type: Bug

[jira] [Updated] (HUDI-2304) Flip some config options for Flink

2021-08-12 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-2304: - Description: {{index.state.ttl}} default value changes from 1.5D to 0D, which means storing the index

[jira] [Created] (HUDI-2304) Flip some config options for Flink

2021-08-12 Thread Danny Chen (Jira)
Danny Chen created HUDI-2304: Summary: Flip some config options for Flink Key: HUDI-2304 URL: https://issues.apache.org/jira/browse/HUDI-2304 Project: Apache Hudi Issue Type: Improvement

[jira] [Commented] (HUDI-1897) Implement DeltaStreamer Source for AWS S3

2021-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398422#comment-17398422 ] ASF GitHub Bot commented on HUDI-1897: -- vinothchandar commented on a change in pull request #3433:

[GitHub] [hudi] vinothchandar commented on a change in pull request #3433: [HUDI-1897] Deltastreamer source for AWS S3

2021-08-12 Thread GitBox
vinothchandar commented on a change in pull request #3433: URL: https://github.com/apache/hudi/pull/3433#discussion_r688220583 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/CloudObjectsHoodieIncrSource.java ## @@ -0,0 +1,129 @@ +/* + * Licensed

[GitHub] [hudi] vinothchandar merged pull request #3464: [MINOR] Deprecate older configs

2021-08-12 Thread GitBox
vinothchandar merged pull request #3464: URL: https://github.com/apache/hudi/pull/3464 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HUDI-2119) Syncing of rollbacks to metadata table does not work in all cases

2021-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398418#comment-17398418 ] ASF GitHub Bot commented on HUDI-2119: -- hudi-bot edited a comment on pull request #3210: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3210: [HUDI-2119] Ensure the rolled-back instance was previously synced to the Metadata Table when syncing a Rollback Instant.

2021-08-12 Thread GitBox
hudi-bot edited a comment on pull request #3210: URL: https://github.com/apache/hudi/pull/3210#issuecomment-872541566 ## CI report: * a9dcb727c23272b2c8b74647b467f413b9e83f5d Azure:

[jira] [Commented] (HUDI-2119) Syncing of rollbacks to metadata table does not work in all cases

2021-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398417#comment-17398417 ] ASF GitHub Bot commented on HUDI-2119: -- hudi-bot edited a comment on pull request #3210: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3210: [HUDI-2119] Ensure the rolled-back instance was previously synced to the Metadata Table when syncing a Rollback Instant.

2021-08-12 Thread GitBox
hudi-bot edited a comment on pull request #3210: URL: https://github.com/apache/hudi/pull/3210#issuecomment-872541566 ## CI report: * a9dcb727c23272b2c8b74647b467f413b9e83f5d Azure:

[jira] [Commented] (HUDI-1897) Implement DeltaStreamer Source for AWS S3

2021-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398408#comment-17398408 ] ASF GitHub Bot commented on HUDI-1897: -- codope commented on a change in pull request #3433: URL:

[GitHub] [hudi] codope commented on a change in pull request #3433: [HUDI-1897] Deltastreamer source for AWS S3

2021-08-12 Thread GitBox
codope commented on a change in pull request #3433: URL: https://github.com/apache/hudi/pull/3433#discussion_r688207653 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/CloudObjectsMetaSource.java ## @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache

[jira] [Commented] (HUDI-1897) Implement DeltaStreamer Source for AWS S3

2021-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398407#comment-17398407 ] ASF GitHub Bot commented on HUDI-1897: -- codope commented on a change in pull request #3433: URL:

[jira] [Commented] (HUDI-1897) Implement DeltaStreamer Source for AWS S3

2021-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398406#comment-17398406 ] ASF GitHub Bot commented on HUDI-1897: -- codope commented on a change in pull request #3433: URL:

[GitHub] [hudi] codope commented on a change in pull request #3433: [HUDI-1897] Deltastreamer source for AWS S3

2021-08-12 Thread GitBox
codope commented on a change in pull request #3433: URL: https://github.com/apache/hudi/pull/3433#discussion_r688206771 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/CloudObjectsSelector.java ## @@ -0,0 +1,285 @@ +/* + * Licensed to the

[GitHub] [hudi] codope commented on a change in pull request #3433: [HUDI-1897] Deltastreamer source for AWS S3

2021-08-12 Thread GitBox
codope commented on a change in pull request #3433: URL: https://github.com/apache/hudi/pull/3433#discussion_r688206707 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/CloudObjectsMetaSelector.java ## @@ -0,0 +1,208 @@ +/* + * Licensed to

[jira] [Assigned] (HUDI-2299) The log format DELETE block lose the info orderingVal

2021-08-12 Thread Zheng yunhong (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng yunhong reassigned HUDI-2299: --- Assignee: (was: Zheng yunhong) > The log format DELETE block lose the info orderingVal >

[jira] [Assigned] (HUDI-2299) The log format DELETE block lose the info orderingVal

2021-08-12 Thread Zheng yunhong (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng yunhong reassigned HUDI-2299: --- Assignee: Zheng yunhong > The log format DELETE block lose the info orderingVal >

[jira] [Commented] (HUDI-1897) Implement DeltaStreamer Source for AWS S3

2021-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398394#comment-17398394 ] ASF GitHub Bot commented on HUDI-1897: -- codope commented on a change in pull request #3433: URL:

[GitHub] [hudi] codope commented on a change in pull request #3433: [HUDI-1897] Deltastreamer source for AWS S3

2021-08-12 Thread GitBox
codope commented on a change in pull request #3433: URL: https://github.com/apache/hudi/pull/3433#discussion_r688202916 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/CloudObjectsMetaSelector.java ## @@ -0,0 +1,208 @@ +/* + * Licensed to

[jira] [Commented] (HUDI-2259) [SQL]Support referencing subquery with column aliases by table alias in merge into

2021-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398389#comment-17398389 ] ASF GitHub Bot commented on HUDI-2259: -- pengzhiwei2018 commented on pull request #3380: URL:

[GitHub] [hudi] pengzhiwei2018 commented on pull request #3380: [HUDI-2259]Support referencing subquery with column aliases by table alias in me…

2021-08-12 Thread GitBox
pengzhiwei2018 commented on pull request #3380: URL: https://github.com/apache/hudi/pull/3380#issuecomment-898115748 > @pengzhiwei2018 Hi,When I test Spark3, I find that Spark SQL for Hoodie with Spark3 uses the source code of Spark, but columns aliases in Merge Into is not supported in

[jira] [Commented] (HUDI-1264) incremental read support with replace

2021-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398388#comment-17398388 ] ASF GitHub Bot commented on HUDI-1264: -- lw309637554 closed pull request #2199: URL:

[jira] [Commented] (HUDI-1264) incremental read support with replace

2021-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398387#comment-17398387 ] ASF GitHub Bot commented on HUDI-1264: -- lw309637554 commented on pull request #2199: URL:

[GitHub] [hudi] lw309637554 closed pull request #2199: [HUDI-1264][WIP] spark incremental read support with replace

2021-08-12 Thread GitBox
lw309637554 closed pull request #2199: URL: https://github.com/apache/hudi/pull/2199 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] lw309637554 commented on pull request #2199: [HUDI-1264][WIP] spark incremental read support with replace

2021-08-12 Thread GitBox
lw309637554 commented on pull request #2199: URL: https://github.com/apache/hudi/pull/2199#issuecomment-898113298 > @lw309637554 Is this PR still valid given that #3139 is merged now? @codope hello , i think i can close this one -- This is an automated message from the Apache Git

[jira] [Updated] (HUDI-2299) The log format DELETE block lose the info orderingVal

2021-08-12 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-2299: - Description: The append handle now always write data block first then delete block, and the delete block

[jira] [Updated] (HUDI-2299) The log format DELETE block lose the info orderingVal

2021-08-12 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-2299: - Description: The append handle now always write data block first then delete block, and the delete block

[jira] [Resolved] (HUDI-2250) [SQL] Bulk insert support for tables w/ primary key

2021-08-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan resolved HUDI-2250. --- Fix Version/s: (was: 0.10.0) 0.9.0 Resolution: Fixed >

[jira] [Assigned] (HUDI-2250) [SQL] Bulk insert support for tables w/ primary key

2021-08-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-2250: - Assignee: pengzhiwei > [SQL] Bulk insert support for tables w/ primary key >

[jira] [Updated] (HUDI-2250) [SQL] Bulk insert support for tables w/ primary key

2021-08-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2250: -- Status: In Progress (was: Open) > [SQL] Bulk insert support for tables w/ primary key

[jira] [Commented] (HUDI-1292) [Umbrella] RFC-15 : File Listing and Query Planning Optimizations

2021-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398383#comment-17398383 ] ASF GitHub Bot commented on HUDI-1292: -- danny0405 commented on pull request #3427: URL:

[GitHub] [hudi] danny0405 commented on pull request #3427: [HUDI-1292] Created a config to enable/disable syncing of metadata table.

2021-08-12 Thread GitBox
danny0405 commented on pull request #3427: URL: https://github.com/apache/hudi/pull/3427#issuecomment-898101210 +1 to @leesf , these two config options confuses us a lot, not to say the user. -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Commented] (HUDI-2119) Syncing of rollbacks to metadata table does not work in all cases

2021-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398372#comment-17398372 ] ASF GitHub Bot commented on HUDI-2119: -- hudi-bot edited a comment on pull request #3210: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3210: [HUDI-2119] Ensure the rolled-back instance was previously synced to the Metadata Table when syncing a Rollback Instant.

2021-08-12 Thread GitBox
hudi-bot edited a comment on pull request #3210: URL: https://github.com/apache/hudi/pull/3210#issuecomment-872541566 ## CI report: * a9dcb727c23272b2c8b74647b467f413b9e83f5d Azure:

[jira] [Updated] (HUDI-1507) Hive sync having issues w/ Clustering

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1507: Fix Version/s: 0.8.0 > Hive sync having issues w/ Clustering >

[jira] [Updated] (HUDI-1901) Not an Avro data file during archive

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1901: Fix Version/s: 0.9.0 > Not an Avro data file during archive >

[jira] [Commented] (HUDI-2119) Syncing of rollbacks to metadata table does not work in all cases

2021-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398361#comment-17398361 ] ASF GitHub Bot commented on HUDI-2119: -- hudi-bot edited a comment on pull request #3210: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3210: [HUDI-2119] Ensure the rolled-back instance was previously synced to the Metadata Table when syncing a Rollback Instant.

2021-08-12 Thread GitBox
hudi-bot edited a comment on pull request #3210: URL: https://github.com/apache/hudi/pull/3210#issuecomment-872541566 ## CI report: * 23f261c5ebd97be2b7bd5cc9bb9c536c8866da57 Azure:

[jira] [Updated] (HUDI-1764) Add support for Hudi CLI tools to schedule and run clustering

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1764: Fix Version/s: 0.9.0 > Add support for Hudi CLI tools to schedule and run clustering >

[jira] [Updated] (HUDI-2004) Move KafkaOffsetGen.CheckpointUtils test cases to independent class and improve coverage

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-2004: Fix Version/s: 0.9.0 > Move KafkaOffsetGen.CheckpointUtils test cases to independent class and >

[jira] [Commented] (HUDI-2119) Syncing of rollbacks to metadata table does not work in all cases

2021-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398360#comment-17398360 ] ASF GitHub Bot commented on HUDI-2119: -- hudi-bot edited a comment on pull request #3210: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3210: [HUDI-2119] Ensure the rolled-back instance was previously synced to the Metadata Table when syncing a Rollback Instant.

2021-08-12 Thread GitBox
hudi-bot edited a comment on pull request #3210: URL: https://github.com/apache/hudi/pull/3210#issuecomment-872541566 ## CI report: * 23f261c5ebd97be2b7bd5cc9bb9c536c8866da57 Azure:

[jira] [Updated] (HUDI-2159) Supporting Clustering and Metadata Table together

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-2159: Fix Version/s: (was: 0.9.0) 0.10.0 > Supporting Clustering and Metadata

[jira] [Updated] (HUDI-1290) Implement Debezium avro source for Delta Streamer

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1290: Fix Version/s: (was: 0.9.0) 0.10.0 > Implement Debezium avro source for

[jira] [Updated] (HUDI-1309) Listing Metadata unreadable in S3 as the log block is deemed corrupted

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1309: Fix Version/s: (was: 0.9.0) 0.10.0 > Listing Metadata unreadable in S3 as

[jira] [Updated] (HUDI-1537) Move validation of file listings to something that happens before each write

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1537: Fix Version/s: (was: 0.9.0) 0.10.0 > Move validation of file listings to

[jira] [Updated] (HUDI-1500) Support incrementally reading clustering commit via Spark Datasource/DeltaStreamer

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1500: Fix Version/s: (was: 0.9.0) 0.10.0 > Support incrementally reading

[jira] [Updated] (HUDI-1492) Handle DeltaWriteStat correctly for storage schemes that support appends

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1492: Fix Version/s: (was: 0.9.0) 0.10.0 > Handle DeltaWriteStat correctly for

[jira] [Updated] (HUDI-1542) Fix Flaky test : TestHoodieMetadata#testSync

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1542: Fix Version/s: (was: 0.9.0) 0.10.0 > Fix Flaky test :

[jira] [Updated] (HUDI-1706) Test flakiness w/ multiwriter test

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1706: Fix Version/s: (was: 0.9.0) 0.10.0 > Test flakiness w/ multiwriter test >

[jira] [Updated] (HUDI-1839) FSUtils getAllPartitions broken by NotSerializableException: org.apache.hadoop.fs.Path

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1839: Fix Version/s: (was: 0.9.0) 0.10.0 > FSUtils getAllPartitions broken by

[jira] [Updated] (HUDI-1912) Presto defaults to GenericHiveRecordCursor for all Hudi tables

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1912: Status: In Progress (was: Open) > Presto defaults to GenericHiveRecordCursor for all Hudi tables >

[jira] [Updated] (HUDI-1912) Presto defaults to GenericHiveRecordCursor for all Hudi tables

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1912: Fix Version/s: (was: 0.9.0) 0.7.0 > Presto defaults to

[jira] [Updated] (HUDI-1937) When clustering fail, generating unfinished replacecommit timeline.

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1937: Fix Version/s: (was: 0.9.0) 0.10.0 > When clustering fail, generating

[jira] [Updated] (HUDI-1958) [Umbrella] Follow up items from 1 pass over GH issues

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1958: Fix Version/s: (was: 0.9.0) 0.10.0 > [Umbrella] Follow up items from 1 pass

[jira] [Updated] (HUDI-1970) Performance testing/certification of key SQL DMLs

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1970: Fix Version/s: (was: 0.9.0) 0.10.0 > Performance testing/certification of

[jira] [Resolved] (HUDI-2063) [SQL] Add Doc For Spark Sql Integrates With Hudi

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra resolved HUDI-2063. - Resolution: Fixed > [SQL] Add Doc For Spark Sql Integrates With Hudi >

[jira] [Updated] (HUDI-2063) [SQL] Add Doc For Spark Sql Integrates With Hudi

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-2063: Status: In Progress (was: Open) > [SQL] Add Doc For Spark Sql Integrates With Hudi >

[jira] [Updated] (HUDI-2119) Syncing of rollbacks to metadata table does not work in all cases

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-2119: Status: In Progress (was: Open) > Syncing of rollbacks to metadata table does not work in all

[jira] [Updated] (HUDI-2120) Update docs about schema in flink sql configuration

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-2120: Status: In Progress (was: Open) > Update docs about schema in flink sql configuration >

[jira] [Updated] (HUDI-2250) [SQL] Bulk insert support for tables w/ primary key

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-2250: Fix Version/s: (was: 0.9.0) 0.10.0 > [SQL] Bulk insert support for tables w/

[jira] [Updated] (HUDI-1239) [UMBRELLA] Config clean up

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1239: Fix Version/s: (was: 0.9.0) 0.10.0 > [UMBRELLA] Config clean up >

[jira] [Updated] (HUDI-1237) [UMBRELLA] Checkstyle, formatting, warnings, spotless

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1237: Fix Version/s: (was: 0.9.0) 0.10.0 > [UMBRELLA] Checkstyle, formatting,

[jira] [Updated] (HUDI-1236) [UMBRELLA] Long running test suite

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1236: Fix Version/s: (was: 0.9.0) 0.10.0 > [UMBRELLA] Long running test suite >

[jira] [Updated] (HUDI-1015) Audit all getAllPartitionPaths() calls and keep em out of fast path

2021-08-12 Thread Udit Mehrotra (Jira)

[jira] [Updated] (HUDI-73) Support vanilla Avro Kafka Source in HoodieDeltaStreamer

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-73: -- Fix Version/s: (was: 0.9.0) 0.10.0 > Support vanilla Avro Kafka Source in

[jira] [Updated] (HUDI-304) Bring back spotless plugin

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-304: --- Fix Version/s: (was: 0.9.0) 0.10.0 > Bring back spotless plugin >

[jira] [Updated] (HUDI-1038) Adding perf benchmark using jmh to Hudi

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1038: Fix Version/s: (was: 0.9.0) 0.10.0 > Adding perf benchmark using jmh to Hudi

[jira] [Updated] (HUDI-1120) Support spotless for scala

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1120: Fix Version/s: (was: 0.9.0) 0.10.0 > Support spotless for scala >

[jira] [Updated] (HUDI-1063) Save in Google Cloud Storage not working

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1063: Fix Version/s: (was: 0.9.0) 0.10.0 > Save in Google Cloud Storage not

[jira] [Updated] (HUDI-1975) Upgrade java-prometheus-client from 3.1.2 to 4.x

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1975: Fix Version/s: (was: 0.9.0) 0.10.0 > Upgrade java-prometheus-client from

[jira] [Updated] (HUDI-1976) Upgrade hive, jackson, log4j, hadoop to remove vulnerability

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1976: Fix Version/s: (was: 0.9.0) 0.10.0 > Upgrade hive, jackson, log4j, hadoop to

[jira] [Updated] (HUDI-1856) Upstream changes made in PrestoDB to eliminate file listing to Trino

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1856: Fix Version/s: (was: 0.9.0) 0.10.0 > Upstream changes made in PrestoDB to

[jira] [Updated] (HUDI-1401) Presto use of Metadata Table for file listings

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1401: Priority: Major (was: Blocker) > Presto use of Metadata Table for file listings >

[jira] [Updated] (HUDI-1401) Presto use of Metadata Table for file listings

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1401: Fix Version/s: (was: 0.9.0) 0.7.0 > Presto use of Metadata Table for file

[jira] [Updated] (HUDI-1363) Provide Option to drop columns after they are used to generate partition or record keys

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1363: Labels: pull-request-available release-blocker (was: pull-request-available) > Provide Option to

[jira] [Updated] (HUDI-1353) Incremental timeline support for pending clustering operations

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1353: Fix Version/s: (was: 0.9.0) 0.10.0 > Incremental timeline support for

[jira] [Updated] (HUDI-1264) incremental read support with replace

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1264: Fix Version/s: (was: 0.9.0) 0.10.0 > incremental read support with replace >

[jira] [Updated] (HUDI-1251) [UMBRELLA] Migrate CI to azure

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1251: Fix Version/s: (was: 0.9.0) 0.10.0 > [UMBRELLA] Migrate CI to azure >

[jira] [Updated] (HUDI-651) Incremental Query on Hive via Spark SQL does not return expected results

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-651: --- Fix Version/s: (was: 0.9.0) 0.10.0 > Incremental Query on Hive via Spark SQL

[jira] [Updated] (HUDI-2229) Refact hoodie stream creator in flink

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-2229: Fix Version/s: (was: 0.9.0) 0.10.0 > Refact hoodie stream creator in flink >

[jira] [Assigned] (HUDI-1363) Provide Option to drop columns after they are used to generate partition or record keys

2021-08-12 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-1363: Assignee: Sagar Sumit (was: liwei) > Provide Option to drop columns after they are used

[jira] [Updated] (HUDI-1658) [UMBRELLA] Spark Sql Support For Hudi

2021-08-12 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1658: Fix Version/s: (was: 0.9.0) 0.10.0 > [UMBRELLA] Spark Sql Support For Hudi >

[jira] [Commented] (HUDI-2119) Syncing of rollbacks to metadata table does not work in all cases

2021-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398351#comment-17398351 ] ASF GitHub Bot commented on HUDI-2119: -- vinothchandar commented on a change in pull request #3210:

[GitHub] [hudi] vinothchandar commented on a change in pull request #3210: [HUDI-2119] Ensure the rolled-back instance was previously synced to the Metadata Table when syncing a Rollback Instant.

2021-08-12 Thread GitBox
vinothchandar commented on a change in pull request #3210: URL: https://github.com/apache/hudi/pull/3210#discussion_r688154275 ## File path: hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/client/functional/TestHoodieBackedMetadata.java ## @@ -480,6 +482,131 @@

[jira] [Commented] (HUDI-2277) Let HoodieDeltaStreamer reading ORC files using ORCDFSSource

2021-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398350#comment-17398350 ] ASF GitHub Bot commented on HUDI-2277: -- zhangyue19921010 commented on pull request #3413: URL:

[GitHub] [hudi] zhangyue19921010 commented on pull request #3413: [HUDI-2277] HoodieDeltaStreamer reading ORC files directly using ORCDFSSource

2021-08-12 Thread GitBox
zhangyue19921010 commented on pull request #3413: URL: https://github.com/apache/hudi/pull/3413#issuecomment-898039061 Hi @vinothchandar Thanks a lot for your review. Now Travis and Azure are all passed. PTAL :) -- This is an automated message from the Apache Git Service. To respond to

[jira] [Commented] (HUDI-2303) TestMereIntoLogOnlyTable with metadata enabled surfaces likely bug

2021-08-12 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398348#comment-17398348 ] Prashant Wason commented on HUDI-2303: -- Am looking into this. > TestMereIntoLogOnlyTable with

  1   2   3   4   >