[jira] [Commented] (HUDI-1090) presto读取hudi数据错位

2020-07-14 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157818#comment-17157818 ] Hong Shen commented on HUDI-1090: - [~710514878] Please describe the issue in English. > presto读取hudi数据错位

[jira] [Comment Edited] (HUDI-1082) Bug in deciding the upsert/insert buckets

2020-07-14 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157341#comment-17157341 ] Hong Shen edited comment on HUDI-1082 at 7/15/20, 1:12 AM: --- It seems does not

[jira] [Comment Edited] (HUDI-1082) Bug in deciding the upsert/insert buckets

2020-07-14 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157341#comment-17157341 ] Hong Shen edited comment on HUDI-1082 at 7/15/20, 1:08 AM: --- It seems does not

[jira] [Comment Edited] (HUDI-1082) Bug in deciding the upsert/insert buckets

2020-07-14 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157341#comment-17157341 ] Hong Shen edited comment on HUDI-1082 at 7/14/20, 12:53 PM: It seems does not

[jira] [Comment Edited] (HUDI-1082) Bug in deciding the upsert/insert buckets

2020-07-14 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157341#comment-17157341 ] Hong Shen edited comment on HUDI-1082 at 7/14/20, 12:52 PM: It seems does not

[jira] [Commented] (HUDI-1082) Bug in deciding the upsert/insert buckets

2020-07-14 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157341#comment-17157341 ] Hong Shen commented on HUDI-1082: - It seems does not matter, we just need to ensure the distribution of

[jira] [Commented] (HUDI-1082) Bug in deciding the upsert/insert buckets

2020-07-14 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157284#comment-17157284 ] Hong Shen commented on HUDI-1082: - OK, I will add testcase and fix it. > Bug in deciding the

[jira] [Commented] (HUDI-1082) Bug in deciding the upsert/insert buckets

2020-07-12 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17156259#comment-17156259 ] Hong Shen commented on HUDI-1082: - I can work on this if you have not start. > Bug in deciding the

[jira] [Assigned] (HUDI-1058) Make delete marker configurable

2020-06-30 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen reassigned HUDI-1058: --- Assignee: Hong Shen > Make delete marker configurable > --- > >

[jira] [Commented] (HUDI-1058) Make delete marker configurable

2020-06-29 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147826#comment-17147826 ] Hong Shen commented on HUDI-1058: - If you haven't started yet, I can take this. > Make delete marker

[jira] [Commented] (HUDI-1057) optional int32 is not a group

2020-06-28 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147506#comment-17147506 ] Hong Shen commented on HUDI-1057: - Did you use hivemeta? Whether the int32 type is defined in hivemeta, or

[jira] [Commented] (HUDI-983) Add Metrics section to asf-site

2020-06-27 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146989#comment-17146989 ] Hong Shen commented on HUDI-983: [~rxu] I have pull a request in [https://github.com/apache/hudi/pull/1769] 

[jira] [Commented] (HUDI-983) Add Metrics section to asf-site

2020-06-26 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146094#comment-17146094 ] Hong Shen commented on HUDI-983: Sounds good. I will move deployment#metrics to the new metrics section. >

[jira] [Commented] (HUDI-983) Add Metrics section to asf-site

2020-06-25 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17145958#comment-17145958 ] Hong Shen commented on HUDI-983: [~rxu] Currently, there are doc for metrics in two places, at

[jira] [Assigned] (HUDI-1046) Support updates during clustering in CoW mode

2020-06-24 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen reassigned HUDI-1046: --- Assignee: Hong Shen > Support updates during clustering in CoW mode >

[jira] [Assigned] (HUDI-1004) Support update metrics in HoodieDeltaStreamer

2020-06-10 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen reassigned HUDI-1004: --- Assignee: Hong Shen > Support update metrics in HoodieDeltaStreamer >

[jira] [Commented] (HUDI-1004) Support update metrics in HoodieDeltaStreamer

2020-06-10 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17132815#comment-17132815 ] Hong Shen commented on HUDI-1004: - I will add a PR to suppport update metrics this weekend. > Support

[jira] [Updated] (HUDI-1004) Support update metrics in HoodieDeltaStreamer

2020-06-10 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-1004: Description: When I run HoodieDeltaStreamer with hoodie.metrics.on=true and

[jira] [Updated] (HUDI-1004) Support update metrics in HoodieDeltaStreamer

2020-06-10 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-1004: Summary: Support update metrics in HoodieDeltaStreamer (was: Failed to send metrics in

[jira] [Resolved] (HUDI-1005) NPE in HoodieWriteClient.clean

2020-06-09 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen resolved HUDI-1005. - Resolution: Fixed > NPE in HoodieWriteClient.clean > --- > >

[jira] [Updated] (HUDI-1005) NPE in HoodieWriteClient.clean

2020-06-09 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-1005: Status: Open (was: New) > NPE in HoodieWriteClient.clean > --- > >

[jira] [Updated] (HUDI-1016) [Minor] Code optimization

2020-06-09 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-1016: Status: Open (was: New) > [Minor] Code optimization > - > > Key:

[jira] [Resolved] (HUDI-1016) [Minor] Code optimization

2020-06-09 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen resolved HUDI-1016. - Resolution: Fixed > [Minor] Code optimization > - > > Key:

[jira] [Updated] (HUDI-1004) Failed to send metrics in HoodieDeltaStreamer

2020-06-09 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-1004: Summary: Failed to send metrics in HoodieDeltaStreamer (was: Failed to send metrics when

[jira] [Assigned] (HUDI-1016) [Minor] Code optimization

2020-06-08 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen reassigned HUDI-1016: --- Assignee: Hong Shen > [Minor] Code optimization > - > > Key:

[jira] [Created] (HUDI-1016) [Minor] Code optimization

2020-06-08 Thread Hong Shen (Jira)
Hong Shen created HUDI-1016: --- Summary: [Minor] Code optimization Key: HUDI-1016 URL: https://issues.apache.org/jira/browse/HUDI-1016 Project: Apache Hudi Issue Type: Improvement

[jira] [Updated] (HUDI-1002) Ignore case when setting incremental mode in hive query

2020-06-08 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-1002: Status: Open (was: New) > Ignore case when setting incremental mode in hive query >

[jira] [Resolved] (HUDI-1002) Ignore case when setting incremental mode in hive query

2020-06-08 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen resolved HUDI-1002. - Resolution: Fixed > Ignore case when setting incremental mode in hive query >

[jira] [Commented] (HUDI-1002) Ignore case when setting incremental mode in hive query

2020-06-08 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17127926#comment-17127926 ] Hong Shen commented on HUDI-1002: - [~xleesf] I'm going to work on this. :D > Ignore case when setting

[jira] [Assigned] (HUDI-1002) Ignore case when setting incremental mode in hive query

2020-06-08 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen reassigned HUDI-1002: --- Assignee: Hong Shen > Ignore case when setting incremental mode in hive query >

[jira] [Created] (HUDI-1005) NPE in HoodieWriteClient.clean

2020-06-07 Thread Hong Shen (Jira)
Hong Shen created HUDI-1005: --- Summary: NPE in HoodieWriteClient.clean Key: HUDI-1005 URL: https://issues.apache.org/jira/browse/HUDI-1005 Project: Apache Hudi Issue Type: Bug

[jira] [Updated] (HUDI-1004) Failed to send metrics when HoodieDeltaStreamer

2020-06-07 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-1004: Description: When I run HoodieDeltaStreamer with hoodie.metrics.on=true and

[jira] [Created] (HUDI-1004) Failed to send metrics when HoodieDeltaStreamer

2020-06-07 Thread Hong Shen (Jira)
Hong Shen created HUDI-1004: --- Summary: Failed to send metrics when HoodieDeltaStreamer Key: HUDI-1004 URL: https://issues.apache.org/jira/browse/HUDI-1004 Project: Apache Hudi Issue Type: Sub-task

[jira] [Assigned] (HUDI-983) Add Metrics section to asf-site

2020-06-03 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen reassigned HUDI-983: -- Assignee: Hong Shen > Add Metrics section to asf-site > --- > >

[jira] [Commented] (HUDI-983) Add Metrics section to asf-site

2020-06-03 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17125009#comment-17125009 ] Hong Shen commented on HUDI-983: [~rxu] I am interested in it, I will add a pull request in this weekend.

[jira] [Commented] (HUDI-957) Umbrella ticket for sequencing common tasks required to progress/unblock RFC-08, RFC-15 & RFC-19

2020-06-03 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124926#comment-17124926 ] Hong Shen commented on HUDI-957: [~nishith29] We also want to do this, please @ me if need. > Umbrella

[jira] [Assigned] (HUDI-908) Consistent test data generation with data type coverage

2020-05-29 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen reassigned HUDI-908: -- Assignee: Hong Shen > Consistent test data generation with data type coverage >

[jira] [Commented] (HUDI-908) Consistent test data generation with data type coverage

2020-05-29 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119297#comment-17119297 ] Hong Shen commented on HUDI-908: [~vinoth] [~shivnarayan] I am interested in it, I will add a pull request

[jira] [Created] (HUDI-903) Add hudi java client that do not depend on spark

2020-05-16 Thread Hong Shen (Jira)
Hong Shen created HUDI-903: -- Summary: Add hudi java client that do not depend on spark Key: HUDI-903 URL: https://issues.apache.org/jira/browse/HUDI-903 Project: Apache Hudi (incubating) Issue

[jira] [Resolved] (HUDI-866) Some classes that rely on JavaSparkContext to obtain Configuration can make them Spark-free.

2020-05-12 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen resolved HUDI-866. Resolution: Fixed Resolve by https://issues.apache.org/jira/browse/HUDI-870

[jira] [Updated] (HUDI-866) Some classes that rely on JavaSparkContext to obtain Configuration can make them Spark-free.

2020-05-12 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-866: --- Status: Open (was: New) > Some classes that rely on JavaSparkContext to obtain Configuration can make > them

[jira] [Resolved] (HUDI-881) Replace part of Spark context by hadoop configuration in AbstractHoodieClient

2020-05-12 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen resolved HUDI-881. Resolution: Fixed > Replace part of Spark context by hadoop configuration in AbstractHoodieClient >

[jira] [Resolved] (HUDI-886) Replace jsc.hadoopConfiguration by hadoop configuration in hudi-client testcase

2020-05-12 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen resolved HUDI-886. Resolution: Fixed > Replace jsc.hadoopConfiguration by hadoop configuration in hudi-client > testcase >

[jira] [Updated] (HUDI-881) Replace part of Spark context by hadoop configuration in AbstractHoodieClient

2020-05-12 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-881: --- Fix Version/s: 0.6.0 > Replace part of Spark context by hadoop configuration in AbstractHoodieClient >

[jira] [Commented] (HUDI-886) Replace jsc.hadoopConfiguration by hadoop configuration in hudi-client testcase

2020-05-12 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105212#comment-17105212 ] Hong Shen commented on HUDI-886: https://github.com/apache/incubator-hudi/pull/1621 > Replace

[jira] [Commented] (HUDI-881) Replace part of Spark context by hadoop configuration in AbstractHoodieClient

2020-05-12 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105213#comment-17105213 ] Hong Shen commented on HUDI-881: https://github.com/apache/incubator-hudi/pull/1620 > Replace part of

[jira] [Updated] (HUDI-886) Replace jsc.hadoopConfiguration by hadoop configuration in hudi-client testcase

2020-05-12 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-886: --- Status: In Progress (was: Open) > Replace jsc.hadoopConfiguration by hadoop configuration in hudi-client >

[jira] [Created] (HUDI-886) Replace jsc.hadoopConfiguration by hadoop configuration in hudi-client testcase

2020-05-12 Thread Hong Shen (Jira)
Hong Shen created HUDI-886: -- Summary: Replace jsc.hadoopConfiguration by hadoop configuration in hudi-client testcase Key: HUDI-886 URL: https://issues.apache.org/jira/browse/HUDI-886 Project: Apache Hudi

[jira] [Assigned] (HUDI-886) Replace jsc.hadoopConfiguration by hadoop configuration in hudi-client testcase

2020-05-12 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen reassigned HUDI-886: -- Assignee: Hong Shen > Replace jsc.hadoopConfiguration by hadoop configuration in hudi-client >

[jira] [Updated] (HUDI-886) Replace jsc.hadoopConfiguration by hadoop configuration in hudi-client testcase

2020-05-12 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-886: --- Status: Open (was: New) > Replace jsc.hadoopConfiguration by hadoop configuration in hudi-client > testcase

[jira] [Resolved] (HUDI-880) Replace part of spark context by hadoop configuration in HoodieTable

2020-05-12 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen resolved HUDI-880. Resolution: Fixed > Replace part of spark context by hadoop configuration in HoodieTable >

[jira] [Updated] (HUDI-881) Replace part of Spark context by hadoop configuration in AbstractHoodieClient

2020-05-12 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-881: --- Status: In Progress (was: Open) > Replace part of Spark context by hadoop configuration in

[jira] [Updated] (HUDI-881) Replace part of Spark context by hadoop configuration in AbstractHoodieClient

2020-05-12 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-881: --- Status: Open (was: New) > Replace part of Spark context by hadoop configuration in AbstractHoodieClient >

[jira] [Updated] (HUDI-880) Replace part of spark context by hadoop configuration in HoodieTable

2020-05-11 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-880: --- Status: In Progress (was: Open) > Replace part of spark context by hadoop configuration in HoodieTable >

[jira] [Updated] (HUDI-880) Replace part of spark context by hadoop configuration in HoodieTable

2020-05-11 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-880: --- Status: Open (was: New) > Replace part of spark context by hadoop configuration in HoodieTable >

[jira] [Updated] (HUDI-870) Remove spark context in ClientUtils and HoodieIndex

2020-05-11 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-870: --- Status: In Progress (was: Open) > Remove spark context in ClientUtils and HoodieIndex >

[jira] [Resolved] (HUDI-870) Remove spark context in ClientUtils and HoodieIndex

2020-05-11 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen resolved HUDI-870. Resolution: Fixed > Remove spark context in ClientUtils and HoodieIndex >

[jira] [Updated] (HUDI-870) Remove spark context in ClientUtils and HoodieIndex

2020-05-11 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-870: --- Status: Open (was: New) > Remove spark context in ClientUtils and HoodieIndex >

[jira] [Created] (HUDI-881) Replace part of Spark context by hadoop configuration in AbstractHoodieClient

2020-05-11 Thread Hong Shen (Jira)
Hong Shen created HUDI-881: -- Summary: Replace part of Spark context by hadoop configuration in AbstractHoodieClient Key: HUDI-881 URL: https://issues.apache.org/jira/browse/HUDI-881 Project: Apache Hudi

[jira] [Assigned] (HUDI-880) Replace part of spark context by hadoop configuration in HoodieTable

2020-05-11 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen reassigned HUDI-880: -- Assignee: Hong Shen > Replace part of spark context by hadoop configuration in HoodieTable >

[jira] [Created] (HUDI-880) Replace part of spark context by hadoop configuration in HoodieTable

2020-05-11 Thread Hong Shen (Jira)
Hong Shen created HUDI-880: -- Summary: Replace part of spark context by hadoop configuration in HoodieTable Key: HUDI-880 URL: https://issues.apache.org/jira/browse/HUDI-880 Project: Apache Hudi (incubating)

[jira] [Comment Edited] (HUDI-866) Some classes that rely on JavaSparkContext to obtain Configuration can make them Spark-free.

2020-05-09 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17103195#comment-17103195 ] Hong Shen edited comment on HUDI-866 at 5/9/20, 2:38 PM: - I sorted out the classes

[jira] [Comment Edited] (HUDI-866) Some classes that rely on JavaSparkContext to obtain Configuration can make them Spark-free.

2020-05-09 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17103195#comment-17103195 ] Hong Shen edited comment on HUDI-866 at 5/9/20, 8:36 AM: - I sorted out the classes

[jira] [Assigned] (HUDI-870) Remove spark context in ClientUtils and HoodieIndex

2020-05-09 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen reassigned HUDI-870: -- Assignee: Hong Shen > Remove spark context in ClientUtils and HoodieIndex >

[jira] [Updated] (HUDI-870) Remove spark context in ClientUtils and HoodieIndex

2020-05-09 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-870: --- Summary: Remove spark context in ClientUtils and HoodieIndex (was: Replace spark context directly with hadoop

[jira] [Comment Edited] (HUDI-866) Some classes that rely on JavaSparkContext to obtain Configuration can make them Spark-free.

2020-05-09 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17103195#comment-17103195 ] Hong Shen edited comment on HUDI-866 at 5/9/20, 8:27 AM: - I sorted out the classes

[jira] [Created] (HUDI-870) Replace spark context directly with hadoop configuration in ClientUtils and HoodieTimelineArchiveLog

2020-05-09 Thread Hong Shen (Jira)
Hong Shen created HUDI-870: -- Summary: Replace spark context directly with hadoop configuration in ClientUtils and HoodieTimelineArchiveLog Key: HUDI-870 URL: https://issues.apache.org/jira/browse/HUDI-870

[jira] [Commented] (HUDI-866) Some classes that rely on JavaSparkContext to obtain Configuration can make them Spark-free.

2020-05-09 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17103195#comment-17103195 ] Hong Shen commented on HUDI-866: I sorted out the classes in hudi-client that can replace spark context

[jira] [Commented] (HUDI-866) Some classes that rely on JavaSparkContext to obtain Configuration can make them Spark-free.

2020-05-08 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17102692#comment-17102692 ] Hong Shen commented on HUDI-866: [~yanghua]  agree, I will split the pull request.  > Some classes that

[jira] [Updated] (HUDI-866) Some classes that rely on JavaSparkContext to obtain Configuration can make them Spark-free.

2020-05-07 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-866: --- Summary: Some classes that rely on JavaSparkContext to obtain Configuration can make them Spark-free. (was:

[jira] [Updated] (HUDI-866) Replace Spark context directly with Hadoop configuration object in some scenarios

2020-05-07 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-866: --- Parent: HUDI-538 Issue Type: Sub-task (was: Task) > Replace Spark context directly with Hadoop

[jira] [Created] (HUDI-866) Replace Spark context directly with Hadoop configuration object in some scenarios

2020-05-07 Thread Hong Shen (Jira)
Hong Shen created HUDI-866: -- Summary: Replace Spark context directly with Hadoop configuration object in some scenarios Key: HUDI-866 URL: https://issues.apache.org/jira/browse/HUDI-866 Project: Apache Hudi

[jira] [Commented] (HUDI-538) Restructuring hudi client module for multi engine support

2020-04-30 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096380#comment-17096380 ] Hong Shen commented on HUDI-538: [~yanghua] please @ me if need.    > Restructuring hudi client module for

[jira] [Updated] (HUDI-782) Add support for aliyun OSS

2020-04-11 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-782: --- Issue Type: New Feature (was: Bug) > Add support for aliyun OSS > -- > >

[jira] [Updated] (HUDI-657) It's lots of ClientUtils.createMetaClient when write to hudi

2020-03-04 Thread Hong Shen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated HUDI-657: --- Description: When I write data to hudi, I find it's lots of loading table properties log, and each take 

[jira] [Created] (HUDI-657) It's lots of ClientUtils.createMetaClient when write to hudi

2020-03-04 Thread Hong Shen (Jira)
Hong Shen created HUDI-657: -- Summary: It's lots of ClientUtils.createMetaClient when write to hudi Key: HUDI-657 URL: https://issues.apache.org/jira/browse/HUDI-657 Project: Apache Hudi (incubating)