[GitHub] [hudi] ycjunhua commented on issue #3704: [SUPPORT]flink hudi api use batch insert data happen error

2021-09-22 Thread GitBox
ycjunhua commented on issue #3704: URL: https://github.com/apache/hudi/issues/3704#issuecomment-925466895 10:34:06,021 INFO org.apache.flink.runtime.taskmanager.Task - Filter -> Map (1/8)#0 (921bd3faa6059f2011630dc6c7cf6205) switched from RUNNING to FINISHED.

[GitHub] [hudi] ycjunhua commented on issue #3704: [SUPPORT]flink hudi api use batch insert data happen error

2021-09-22 Thread GitBox
ycjunhua commented on issue #3704: URL: https://github.com/apache/hudi/issues/3704#issuecomment-925466818 10:34:05,441 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - hoodie_stream_write (3/4) (ff5491c88ffc17a816e22c29b6da47be) switched from DEPLOYING to RUNNING.

[GitHub] [hudi] ycjunhua commented on issue #3704: [SUPPORT]flink hudi api use batch insert data happen error

2021-09-22 Thread GitBox
ycjunhua commented on issue #3704: URL: https://github.com/apache/hudi/issues/3704#issuecomment-925466739 10:34:04,886 INFO io.javalin.Javalin - Starting Javalin ... 10:34:05,041 INFO io.javalin.Javalin

[GitHub] [hudi] ycjunhua commented on issue #3704: [SUPPORT]flink hudi api use batch insert data happen error

2021-09-22 Thread GitBox
ycjunhua commented on issue #3704: URL: https://github.com/apache/hudi/issues/3704#issuecomment-925466571 0:33:59,268 INFO org.apache.hudi.table.HoodieTableFactory - Table option [hoodie.datasource.write.keygenerator.class] is reset to

[GitHub] [hudi] hudi-bot edited a comment on pull request #3590: [HUDI-2285][HUDI-2476] Metadata table synchronous design. Rebased and Squashed from pull/3426

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3590: URL: https://github.com/apache/hudi/pull/3590#issuecomment-912237120 ## CI report: * aefac7ec2f2e40bdf3ad4365ea6aa825803a439d UNKNOWN * af27976ba07afb02305c699aeb7a75a122013b62 Azure:

[GitHub] [hudi] danny0405 commented on issue #3704: [SUPPORT]flink hudi api use batch insert data happen error

2021-09-22 Thread GitBox
danny0405 commented on issue #3704: URL: https://github.com/apache/hudi/issues/3704#issuecomment-925460959 Can you show the log of JobManager ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] danny0405 commented on issue #3704: [SUPPORT]flink hudi api use batch insert data happen error

2021-09-22 Thread GitBox
danny0405 commented on issue #3704: URL: https://github.com/apache/hudi/issues/3704#issuecomment-925460819 No successful commits. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot edited a comment on pull request #3590: [HUDI-2285][HUDI-2476] Metadata table synchronous design. Rebased and Squashed from pull/3426

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3590: URL: https://github.com/apache/hudi/pull/3590#issuecomment-912237120 ## CI report: * aefac7ec2f2e40bdf3ad4365ea6aa825803a439d UNKNOWN * 9c0123c0f27f990d009b323bab75b76ceecf3dab Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3590: [HUDI-2285][HUDI-2476] Metadata table synchronous design. Rebased and Squashed from pull/3426

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3590: URL: https://github.com/apache/hudi/pull/3590#issuecomment-912237120 ## CI report: * aefac7ec2f2e40bdf3ad4365ea6aa825803a439d UNKNOWN * 9c0123c0f27f990d009b323bab75b76ceecf3dab Azure:

[GitHub] [hudi] ycjunhua commented on issue #3704: [SUPPORT]flink hudi api use batch insert data happen error

2021-09-22 Thread GitBox
ycjunhua commented on issue #3704: URL: https://github.com/apache/hudi/issues/3704#issuecomment-925434266 ![2 LSXTO~ 5SSJ5F@9D9MNC](https://user-images.githubusercontent.com/8231999/134440547-d57a0133-bf68-4cc6-8bef-2cc2a8fc0ce2.png)

[jira] [Updated] (HUDI-2457) Fix hudi website for current and 090 for spark bundle versions.

2021-09-22 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2457: -- Fix Version/s: 0.10.0 > Fix hudi website for current and 090 for spark bundle versions.

[jira] [Updated] (HUDI-2472) Tests failure follow up when metadata is enabled by default

2021-09-22 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2472: -- Description: We plan to enable metadata by default. but there are some tests that fail

[jira] [Updated] (HUDI-2457) Fix hudi website for current and 090 for spark bundle versions.

2021-09-22 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2457: -- Status: In Progress (was: Open) > Fix hudi website for current and 090 for spark

[jira] [Resolved] (HUDI-2457) Fix hudi website for current and 090 for spark bundle versions.

2021-09-22 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan resolved HUDI-2457. --- Resolution: Fixed > Fix hudi website for current and 090 for spark bundle versions.

[GitHub] [hudi] alexeykudinkin commented on issue #2848: [SUPPORT] maven | jdk.tools:jdk.tools:1.7

2021-09-22 Thread GitBox
alexeykudinkin commented on issue #2848: URL: https://github.com/apache/hudi/issues/2848#issuecomment-925397543 For those that might be hitting the same issue: You need to make sure that in IDEA you select appropriate JDK (1.8) for your imported project -- This is an automated

[hudi] branch asf-site updated: [HUDI-2416] Move content from cwiki to website (FAQ movement) (#3496)

2021-09-22 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 317e284 [HUDI-2416] Move content from cwiki

[GitHub] [hudi] vinothchandar merged pull request #3496: [HUDI-2416] Move content from cwiki to website (FAQ movement)

2021-09-22 Thread GitBox
vinothchandar merged pull request #3496: URL: https://github.com/apache/hudi/pull/3496 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] vinothchandar commented on pull request #3330: [HUDI-2101][RFC-28]support z-order for hudi

2021-09-22 Thread GitBox
vinothchandar commented on pull request #3330: URL: https://github.com/apache/hudi/pull/3330#issuecomment-925371081 @xiarixiaoyao yeah. but given we have started this way. its okay for now. I ll take another pass at this and land. Will make small changes if any. -- This is an

[GitHub] [hudi] vinothchandar commented on pull request #3496: [HUDI-2416] Move content from cwiki to website (FAQ movement)

2021-09-22 Thread GitBox
vinothchandar commented on pull request #3496: URL: https://github.com/apache/hudi/pull/3496#issuecomment-925367776 Thanks for this @pratyakshsharma ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] vinothchandar commented on a change in pull request #3496: [HUDI-2416] Move content from cwiki to website (FAQ movement)

2021-09-22 Thread GitBox
vinothchandar commented on a change in pull request #3496: URL: https://github.com/apache/hudi/pull/3496#discussion_r714347494 ## File path: website/learn/faq.md ## @@ -0,0 +1,440 @@ +--- +title: FAQs +keywords: [hudi, writing, reading] +last_modified_at:

[GitHub] [hudi] ZeMirella commented on issue #3699: [SUPPORT] Job hanging on toRdd at HoodieSparkUtils

2021-09-22 Thread GitBox
ZeMirella commented on issue #3699: URL: https://github.com/apache/hudi/issues/3699#issuecomment-925219008 Hi, thanks for you reply **Which line of code from HoodieSparkUtils was ran here?** The jobs hangs before even start, it hangs when it start to list files and tries to read s3

[GitHub] [hudi] hudi-bot edited a comment on pull request #3696: [WIP][HUDI-2439] Refactor commit actions in hudi-client module

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3696: URL: https://github.com/apache/hudi/pull/3696#issuecomment-924126593 ## CI report: * 828bde6425c90468c28b57c3740f352a024d3e8d Azure:

[GitHub] [hudi] xushiyan commented on issue #3555: [SUPPORT] support show/drop partitions tablename sql

2021-09-22 Thread GitBox
xushiyan commented on issue #3555: URL: https://github.com/apache/hudi/issues/3555#issuecomment-925169525 @melin looks like you already have a version of drop partition implemented. Do you mind opening up a PR for it? Filed the drop partition ticket

[jira] [Updated] (HUDI-2456) Support show partitions SQL

2021-09-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2456: - Parent: HUDI-1658 Issue Type: Sub-task (was: Improvement) > Support show partitions SQL >

[jira] [Updated] (HUDI-2482) Support drop partitions SQL

2021-09-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2482: - Parent: HUDI-1658 Issue Type: Sub-task (was: Improvement) > Support drop partitions SQL >

[jira] [Assigned] (HUDI-2482) Support drop partitions SQL

2021-09-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-2482: Assignee: (was: Yann Byron) > Support drop partitions SQL > --- > >

[jira] [Updated] (HUDI-2482) Support drop partitions SQL

2021-09-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2482: - Description: (was: Spark SQL support the following syntax to show hudi tabls's partitions.

[jira] [Created] (HUDI-2482) Support drop partitions SQL

2021-09-22 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-2482: Summary: Support drop partitions SQL Key: HUDI-2482 URL: https://issues.apache.org/jira/browse/HUDI-2482 Project: Apache Hudi Issue Type: Improvement

[GitHub] [hudi] hudi-bot edited a comment on pull request #3695: [HUDI-2395] Metadata tests rewrite

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3695: URL: https://github.com/apache/hudi/pull/3695#issuecomment-923802661 ## CI report: * cd19e96d2729eb3e30204d6830737fc68a6810d2 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3696: [WIP][HUDI-2439] Refactor commit actions in hudi-client module

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3696: URL: https://github.com/apache/hudi/pull/3696#issuecomment-924126593 ## CI report: * 1c37ce18b451091cdcd751af679be6833d713c68 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3696: [WIP][HUDI-2439] Refactor commit actions in hudi-client module

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3696: URL: https://github.com/apache/hudi/pull/3696#issuecomment-924126593 ## CI report: * 1c37ce18b451091cdcd751af679be6833d713c68 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3695: [HUDI-2395] Metadata tests rewrite

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3695: URL: https://github.com/apache/hudi/pull/3695#issuecomment-923802661 ## CI report: * 936bb99c18312a348f0b82273778b45e95e319ae Azure:

[GitHub] [hudi] xushiyan commented on issue #3555: [SUPPORT] support show/drop partitions tablename sql

2021-09-22 Thread GitBox
xushiyan commented on issue #3555: URL: https://github.com/apache/hudi/issues/3555#issuecomment-925115797 PR opened by @YannByron https://github.com/apache/hudi/pull/3693 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] YannByron edited a comment on issue #3555: [SUPPORT] support show/drop partitions tablename sql

2021-09-22 Thread GitBox
YannByron edited a comment on issue #3555: URL: https://github.com/apache/hudi/issues/3555#issuecomment-922441246 jira to show partitions: https://issues.apache.org/jira/browse/HUDI-2456 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hudi] vinothchandar commented on issue #2688: [SUPPORT] Sync to Hive using Metastore

2021-09-22 Thread GitBox
vinothchandar commented on issue #2688: URL: https://github.com/apache/hudi/issues/2688#issuecomment-925113296 cc @codope and I actuallty starting to look at this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hudi] xushiyan closed issue #3555: [SUPPORT] support show/drop partitions tablename sql

2021-09-22 Thread GitBox
xushiyan closed issue #3555: URL: https://github.com/apache/hudi/issues/3555 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] vinothchandar commented on issue #2688: [SUPPORT] Sync to Hive using Metastore

2021-09-22 Thread GitBox
vinothchandar commented on issue #2688: URL: https://github.com/apache/hudi/issues/2688#issuecomment-925110472 We are still looking into it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] xushiyan commented on issue #3606: [SUPPORT]Upgrade the parquet 1.12 version to support zstd

2021-09-22 Thread GitBox
xushiyan commented on issue #3606: URL: https://github.com/apache/hudi/issues/3606#issuecomment-925088697 upgrading to parquet 1.12 also has advantage in getting some parquet fixes. it is now limited to the version due to Spark dependency. There is a plan to upgrade with newer Spark and

[GitHub] [hudi] xushiyan closed issue #3606: [SUPPORT]Upgrade the parquet 1.12 version to support zstd

2021-09-22 Thread GitBox
xushiyan closed issue #3606: URL: https://github.com/apache/hudi/issues/3606 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3695: [HUDI-2395] Metadata tests rewrite

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3695: URL: https://github.com/apache/hudi/pull/3695#issuecomment-923802661 ## CI report: * 936bb99c18312a348f0b82273778b45e95e319ae Azure:

[GitHub] [hudi] spyzzz commented on issue #2175: [SUPPORT] HUDI MOR/COW tuning with spark structured streaming

2021-09-22 Thread GitBox
spyzzz commented on issue #2175: URL: https://github.com/apache/hudi/issues/2175#issuecomment-925079262 Hello @parisni , Yes , I havent worked on this poc since a while but i was still using a stream per table because I had really specific treament on each. And its really not easy

[GitHub] [hudi] hudi-bot edited a comment on pull request #3695: [HUDI-2395] Metadata tests rewrite

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3695: URL: https://github.com/apache/hudi/pull/3695#issuecomment-923802661 ## CI report: * 936bb99c18312a348f0b82273778b45e95e319ae Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3695: [HUDI-2395] Metadata tests rewrite

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3695: URL: https://github.com/apache/hudi/pull/3695#issuecomment-923802661 ## CI report: * a4f54123f85678a6438ae448765c43fe183f23e0 Azure:

[GitHub] [hudi] rubenssoto commented on issue #3685: [FEATURE REQUEST] HUDI Add metadata for Presto CBO

2021-09-22 Thread GitBox
rubenssoto commented on issue #3685: URL: https://github.com/apache/hudi/issues/3685#issuecomment-925050531 I think it wouldn't have the same effect, because presto uses some hive statistics to do some optimizations for example the right table order on join. -- This is an automated

[GitHub] [hudi] xushiyan closed issue #3669: [SUPPORT] SQL stmt managed table empty in athena

2021-09-22 Thread GitBox
xushiyan closed issue #3669: URL: https://github.com/apache/hudi/issues/3669 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] xushiyan commented on issue #3669: [SUPPORT] SQL stmt managed table empty in athena

2021-09-22 Thread GitBox
xushiyan commented on issue #3669: URL: https://github.com/apache/hudi/issues/3669#issuecomment-925033102 Looks like an issue with Athena reader. Please report this with more details to AWS via AWS support case. cc @umehrot2 for your visibility -- This is an automated message from

[jira] [Updated] (HUDI-2452) spark on hudi metadata key length < 0

2021-09-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2452: - Labels: sev:critical (was: pull-request-available sev:critical) > spark on hudi metadata key length < 0

[jira] [Updated] (HUDI-2452) spark on hudi metadata key length < 0

2021-09-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2452: - Labels: pull-request-available sev:critical (was: sev:critical) > spark on hudi metadata key

[GitHub] [hudi] xushiyan closed issue #3688: [SUPPORT]HUDI-2452 spark on hudi metadata key length < 0

2021-09-22 Thread GitBox
xushiyan closed issue #3688: URL: https://github.com/apache/hudi/issues/3688 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] xushiyan commented on issue #3688: [SUPPORT]HUDI-2452 spark on hudi metadata key length < 0

2021-09-22 Thread GitBox
xushiyan commented on issue #3688: URL: https://github.com/apache/hudi/issues/3688#issuecomment-925027373 @xuzifu666 let's follow up on JIRA https://issues.apache.org/jira/browse/HUDI-2452 -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hudi] xushiyan commented on issue #3685: [FEATURE REQUEST] HUDI Add metadata for Presto CBO

2021-09-22 Thread GitBox
xushiyan commented on issue #3685: URL: https://github.com/apache/hudi/issues/3685#issuecomment-925018735 Probably better to enhance metadata table instead of sync to HMS. It can be some future work following this https://issues.apache.org/jira/browse/HUDI-1401 -- This is an

[GitHub] [hudi] xushiyan commented on issue #3672: [SUPPORT] Spark-SQL version 3.1.2 get exception while insert record into table

2021-09-22 Thread GitBox
xushiyan commented on issue #3672: URL: https://github.com/apache/hudi/issues/3672#issuecomment-925007563 The issue has been tracked in https://issues.apache.org/jira/browse/HUDI-1869 -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hudi] xushiyan closed issue #3672: [SUPPORT] Spark-SQL version 3.1.2 get exception while insert record into table

2021-09-22 Thread GitBox
xushiyan closed issue #3672: URL: https://github.com/apache/hudi/issues/3672 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] xushiyan commented on issue #3609: [SUPPORT] SQL stmt broken with spark 3.1.x

2021-09-22 Thread GitBox
xushiyan commented on issue #3609: URL: https://github.com/apache/hudi/issues/3609#issuecomment-925007395 The issue has been tracked in https://issues.apache.org/jira/browse/HUDI-1869 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] xushiyan closed issue #3609: [SUPPORT] SQL stmt broken with spark 3.1.x

2021-09-22 Thread GitBox
xushiyan closed issue #3609: URL: https://github.com/apache/hudi/issues/3609 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] xushiyan edited a comment on issue #3673: [SUPPORT] insert into get hang using spark-sql

2021-09-22 Thread GitBox
xushiyan edited a comment on issue #3673: URL: https://github.com/apache/hudi/issues/3673#issuecomment-925003284 @wuleistarrocks please provide more details, spark UI, job setup, table configs etc -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hudi] xushiyan edited a comment on issue #3673: [SUPPORT] insert into get hang using spark-sql

2021-09-22 Thread GitBox
xushiyan edited a comment on issue #3673: URL: https://github.com/apache/hudi/issues/3673#issuecomment-925003284 @wuleistarrocks please provide more details, spark UI, spark configs, env setup, table configs etc -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] xushiyan commented on issue #3673: [SUPPORT] insert into get hang using spark-sql

2021-09-22 Thread GitBox
xushiyan commented on issue #3673: URL: https://github.com/apache/hudi/issues/3673#issuecomment-925003284 @wuleistarrocks please provide more details, spark UI, job setup, data sizes, etc -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hudi] parisni commented on issue #2175: [SUPPORT] HUDI MOR/COW tuning with spark structured streaming

2021-09-22 Thread GitBox
parisni commented on issue #2175: URL: https://github.com/apache/hudi/issues/2175#issuecomment-924992418 @spyzzz you finally choosed to run one spark streaming per table instead of grouping all topics ? ``` for (table <- tables ) {

[GitHub] [hudi] rubenssoto commented on issue #3685: [FEATURE REQUEST] HUDI Add metadata for Presto CBO

2021-09-22 Thread GitBox
rubenssoto commented on issue #3685: URL: https://github.com/apache/hudi/issues/3685#issuecomment-924992319 @xushiyan yeah, thats it :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] Rap70r commented on issue #3697: [SUPPORT] Performance Tuning: How to speed up stages?

2021-09-22 Thread GitBox
Rap70r commented on issue #3697: URL: https://github.com/apache/hudi/issues/3697#issuecomment-924937292 Hello @xushiyan, Thank you for getting back to me. Just a clarification that above data size (1714 Megabytes, 1.4 million records) is the usual incremental data size we expect on each

[GitHub] [hudi] minihippo commented on pull request #3576: [HUDI-2383] Clean the marker files after compaction

2021-09-22 Thread GitBox
minihippo commented on pull request #3576: URL: https://github.com/apache/hudi/pull/3576#issuecomment-924921093 > @minihippo : Can you rebase and let me know once CI succeeds. I can land it once ready. Done. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [hudi] hudi-bot edited a comment on pull request #3695: [HUDI-2395] Metadata tests rewrite

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3695: URL: https://github.com/apache/hudi/pull/3695#issuecomment-923802661 ## CI report: * a4f54123f85678a6438ae448765c43fe183f23e0 Azure:

[GitHub] [hudi] JoshuaZhuCN commented on issue #3676: MOR table rolls out new parquet files at 10MB for new inserts - even though max file size set as 128MB

2021-09-22 Thread GitBox
JoshuaZhuCN commented on issue #3676: URL: https://github.com/apache/hudi/issues/3676#issuecomment-924884557 I've encountered the same problem, which I've temporarily resolved by setting the value of the parameter COPY_ON_WRITE_TABLE_RECORD_SIZE_ESTIMATE for each commit, and it doesn't

[GitHub] [hudi] danny0405 commented on issue #3704: [SUPPORT]flink hudi api use batch insert data happen error

2021-09-22 Thread GitBox
danny0405 commented on issue #3704: URL: https://github.com/apache/hudi/issues/3704#issuecomment-924883536 It seems that you do not write your data successfully, can you show your files in .hoodie directory. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] hudi-bot edited a comment on pull request #3695: [HUDI-2395] Metadata tests rewrite

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3695: URL: https://github.com/apache/hudi/pull/3695#issuecomment-923802661 ## CI report: * 1ade67a5e9001e0241290967412f8b6f3f165727 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3695: [HUDI-2395] Metadata tests rewrite

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3695: URL: https://github.com/apache/hudi/pull/3695#issuecomment-923802661 ## CI report: * 1ade67a5e9001e0241290967412f8b6f3f165727 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3203: [HUDI-2086] Redo the logical of mor_incremental_view for hive

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-872092745 ## CI report: * 42735e78f24bd3a5f622026627d6ff8b95708705 Azure:

[GitHub] [hudi] nsivabalan commented on a change in pull request #3695: [HUDI-2395] Metadata tests rewrite

2021-09-22 Thread GitBox
nsivabalan commented on a change in pull request #3695: URL: https://github.com/apache/hudi/pull/3695#discussion_r713842735 ## File path: hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/testutils/HoodieClientTestHarness.java ## @@ -418,4 +445,177 @@ public

[GitHub] [hudi] hudi-bot edited a comment on pull request #3578: [HUDI-2385] Make parquet dictionary encoding configurable

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3578: URL: https://github.com/apache/hudi/pull/3578#issuecomment-909972703 ## CI report: * 21a3795ba56ff13e0e7fb841ff0a01a070229125 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3576: [HUDI-2383] Clean the marker files after compaction

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3576: URL: https://github.com/apache/hudi/pull/3576#issuecomment-909856214 ## CI report: * 9dc2b99120cfb08d0973bfb7d2c67f544b2de2a9 Azure:

[GitHub] [hudi] xiarixiaoyao commented on pull request #3668: [RFC-33] [HUDI-2429][WIP] Full schema evolution

2021-09-22 Thread GitBox
xiarixiaoyao commented on pull request #3668: URL: https://github.com/apache/hudi/pull/3668#issuecomment-924829257 @bvaradar . thanks for your review。 I will try to solve these problems。 There is a little question, do we need to add all adaptations to the spark engine on this pr。 if

[GitHub] [hudi] hudi-bot edited a comment on pull request #3203: [HUDI-2086] Redo the logical of mor_incremental_view for hive

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-872092745 ## CI report: * 792ae1d9884eac5eb7004e4edb617ea6e86d79b7 Azure:

[GitHub] [hudi] xiarixiaoyao commented on pull request #3203: [HUDI-2086] Redo the logical of mor_incremental_view for hive

2021-09-22 Thread GitBox
xiarixiaoyao commented on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-924818277 @danny0405 already rebase this pr. pls tell me what problems did you encounter when you query global index table through Hive, i will test that problem with this pr,

[GitHub] [hudi] hudi-bot edited a comment on pull request #3203: [HUDI-2086] Redo the logical of mor_incremental_view for hive

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-872092745 ## CI report: * 792ae1d9884eac5eb7004e4edb617ea6e86d79b7 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3695: [HUDI-2395] Metadata tests rewrite

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3695: URL: https://github.com/apache/hudi/pull/3695#issuecomment-923802661 ## CI report: * 1ade67a5e9001e0241290967412f8b6f3f165727 Azure:

[GitHub] [hudi] bvaradar commented on a change in pull request #3668: [RFC-33] [HUDI-2429][WIP] Full schema evolution

2021-09-22 Thread GitBox
bvaradar commented on a change in pull request #3668: URL: https://github.com/apache/hudi/pull/3668#discussion_r713621765 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/TableSchemaResolver.java ## @@ -493,4 +496,83 @@ public static MessageType

[GitHub] [hudi] borasy commented on issue #3572: compatble version of hudi, hive and hadoop

2021-09-22 Thread GitBox
borasy commented on issue #3572: URL: https://github.com/apache/hudi/issues/3572#issuecomment-924802949 i also cannot sync hudi table to hive. - hudi 0.9 hive 3.1.2 spark 3.0.3 -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hudi] hudi-bot edited a comment on pull request #3674: [WIP][HUDI-2440] Add dependency change diff script for dependency governace

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3674: URL: https://github.com/apache/hudi/pull/3674#issuecomment-920690239 ## CI report: * e16725afd2d76d509abb7415166bfe4468ee6e58 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3578: [HUDI-2385] Make parquet dictionary encoding configurable

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3578: URL: https://github.com/apache/hudi/pull/3578#issuecomment-909972703 ## CI report: * e10bf61baab82c2402594cb43a2f6b4c384a3724 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3578: [HUDI-2385] Make parquet dictionary encoding configurable

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3578: URL: https://github.com/apache/hudi/pull/3578#issuecomment-909972703 ## CI report: * e10bf61baab82c2402594cb43a2f6b4c384a3724 Azure:

[GitHub] [hudi] minihippo commented on a change in pull request #3578: [HUDI-2385] Make parquet dictionary encoding configurable

2021-09-22 Thread GitBox
minihippo commented on a change in pull request #3578: URL: https://github.com/apache/hudi/pull/3578#discussion_r713798427 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/storage/HoodieFileWriterFactory.java ## @@ -71,7 +71,7 @@

[GitHub] [hudi] hudi-bot edited a comment on pull request #3576: [HUDI-2383] Clean the marker files after compaction

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3576: URL: https://github.com/apache/hudi/pull/3576#issuecomment-909856214 ## CI report: * a331089fa60e6df0ce180ca5dc0730e9d464e38f Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3576: [HUDI-2383] Clean the marker files after compaction

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3576: URL: https://github.com/apache/hudi/pull/3576#issuecomment-909856214 ## CI report: * a331089fa60e6df0ce180ca5dc0730e9d464e38f Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3695: [HUDI-2395] Metadata tests rewrite

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3695: URL: https://github.com/apache/hudi/pull/3695#issuecomment-923802661 ## CI report: * e2599109f909ffc35e107165f6f7393eceda8b61 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3674: [WIP][HUDI-2440] Add dependency change diff script for dependency governace

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3674: URL: https://github.com/apache/hudi/pull/3674#issuecomment-920690239 ## CI report: * a844e075069d260f22f1a3244708560bebf0128b Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3674: [WIP][HUDI-2440] Add dependency change diff script for dependency governace

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3674: URL: https://github.com/apache/hudi/pull/3674#issuecomment-920690239 ## CI report: * a844e075069d260f22f1a3244708560bebf0128b Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3623: [WIP][HUDI-2409] Using HBase shaded jars in Hudi presto bundle

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3623: URL: https://github.com/apache/hudi/pull/3623#issuecomment-915056982 ## CI report: * 72fc50ea33d6267ebdc9a0ecd81cb4df3c833814 Azure:

[GitHub] [hudi] JoshuaZhuCN edited a comment on issue #3647: [SUPPORT] Failed to read parquet file during upsert

2021-09-22 Thread GitBox
JoshuaZhuCN edited a comment on issue #3647: URL: https://github.com/apache/hudi/issues/3647#issuecomment-924755424 > This is because the Spark bulk insert use Spark's parquet writer which has its own decimal encode, how do you set up the parameter writelegacyformat ? @danny0405

[GitHub] [hudi] JoshuaZhuCN commented on issue #3647: [SUPPORT] Failed to read parquet file during upsert

2021-09-22 Thread GitBox
JoshuaZhuCN commented on issue #3647: URL: https://github.com/apache/hudi/issues/3647#issuecomment-924755424 > This is because the Spark bulk insert use Spark's parquet writer which has its own decimal encode, how do you set up the parameter writelegacyformat ? This parameter I set

[GitHub] [hudi] ycjunhua opened a new issue #3704: [SUPPORT]flink hudi api use batch insert data happen error

2021-09-22 Thread GitBox
ycjunhua opened a new issue #3704: URL: https://github.com/apache/hudi/issues/3704 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://cwiki.apache.org/confluence/display/HUDI/FAQ)? yes - Join the mailing list to engage in conversations and get faster

[GitHub] [hudi] hudi-bot edited a comment on pull request #3695: [HUDI-2395] Metadata tests rewrite

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3695: URL: https://github.com/apache/hudi/pull/3695#issuecomment-923802661 ## CI report: * e2599109f909ffc35e107165f6f7393eceda8b61 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3695: [HUDI-2395] Metadata tests rewrite

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3695: URL: https://github.com/apache/hudi/pull/3695#issuecomment-923802661 ## CI report: * f6e867edfee5d5f0777ab406ead51e25cd1315ce Azure:

[GitHub] [hudi] danny0405 commented on issue #3262: [SUPPORT] No successful commits under path

2021-09-22 Thread GitBox
danny0405 commented on issue #3262: URL: https://github.com/apache/hudi/issues/3262#issuecomment-924734251 > Hi~How can I use checkpoint in Flink SQL client? Take a look at this document: https://www.yuque.com/docs/share/01c98494-a980-414c-9c45-152023bf3c17?# -- This is an

[GitHub] [hudi] hudi-bot edited a comment on pull request #3623: [WIP][HUDI-2409] Using HBase shaded jars in Hudi presto bundle

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3623: URL: https://github.com/apache/hudi/pull/3623#issuecomment-915056982 ## CI report: * a34260e3bc4c2344005feefe4c7672b9589569af Azure:

[GitHub] [hudi] borasy commented on issue #2913: [SUPPORT] Hudi + Hive Metastore Sync

2021-09-22 Thread GitBox
borasy commented on issue #2913: URL: https://github.com/apache/hudi/issues/2913#issuecomment-924731093 Hi @Akshay2Agarwal , so in the end, it's not possible to use hivemetastore and we have to use hive2 jdbc? - hudi 0.9 spark 3.0.3 -- This is an automated message from

[GitHub] [hudi] codope commented on a change in pull request #3695: [HUDI-2395] Metadata tests rewrite

2021-09-22 Thread GitBox
codope commented on a change in pull request #3695: URL: https://github.com/apache/hudi/pull/3695#discussion_r713741677 ## File path: hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieTestTable.java ## @@ -214,6 +308,40 @@ public HoodieTestTable

[GitHub] [hudi] hudi-bot edited a comment on pull request #3703: [HUDI-2480] FileSlice after pending compaction-requested instant-time…

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3703: URL: https://github.com/apache/hudi/pull/3703#issuecomment-924695744 ## CI report: * a7f6880207d27eb58b074b4c9ae009ce17592f9e Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3623: [WIP][HUDI-2409] Using HBase shaded jars in Hudi presto bundle

2021-09-22 Thread GitBox
hudi-bot edited a comment on pull request #3623: URL: https://github.com/apache/hudi/pull/3623#issuecomment-915056982 ## CI report: * a34260e3bc4c2344005feefe4c7672b9589569af Azure:

[jira] [Created] (HUDI-2481) Fix Restore and RollbackMetadata in HoodieTestTable

2021-09-22 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-2481: - Summary: Fix Restore and RollbackMetadata in HoodieTestTable Key: HUDI-2481 URL: https://issues.apache.org/jira/browse/HUDI-2481 Project: Apache Hudi Issue Type:

  1   2   >