[GitHub] [hudi] garyli1019 commented on pull request #2008: [MINOR] fix get classname for hive sync

2020-08-30 Thread GitBox
garyli1019 commented on pull request #2008: URL: https://github.com/apache/hudi/pull/2008#issuecomment-683517896 Hi @cdmikechen , would you please review this PR? If this looks good then we can merge. This is an automated me

[jira] [Commented] (HUDI-487) Unit tests for hudi-cli

2020-08-30 Thread hong dongdong (Jira)
[ https://issues.apache.org/jira/browse/HUDI-487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17187369#comment-17187369 ] hong dongdong commented on HUDI-487: [~shivnarayan]: yes, recently, I have been entangl

[GitHub] [hudi] wangxianghu commented on pull request #2054: [HUDI-1252] Remove unused class NoOpBulkInsertPartitioner in DataSour…

2020-08-30 Thread GitBox
wangxianghu commented on pull request #2054: URL: https://github.com/apache/hudi/pull/2054#issuecomment-683503914 > Hi @wangxianghu the removal looks good. Good catch! > > While you're here, could you also clean up other unused methods in this class? namely > > ``` > org.ap

[jira] [Updated] (HUDI-1252) Remove unused code in DataSourceTestUtils

2020-08-30 Thread wangxianghu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-1252: -- Summary: Remove unused code in DataSourceTestUtils (was: Remove unused class NoOpBulkInsertPartitioner

[jira] [Closed] (HUDI-1253) Merge two TestHoodieSnapshotExporter to one class

2020-08-30 Thread wangxianghu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu closed HUDI-1253. - Resolution: Invalid > Merge two TestHoodieSnapshotExporter to one class >

[jira] [Updated] (HUDI-1253) Merge two TestHoodieSnapshotExporter to one class

2020-08-30 Thread wangxianghu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-1253: -- Status: Open (was: New) > Merge two TestHoodieSnapshotExporter to one class > -

[GitHub] [hudi] wangxianghu closed pull request #2055: [HUDI-1253] Merge two TestHoodieSnapshotExporter to one class

2020-08-30 Thread GitBox
wangxianghu closed pull request #2055: URL: https://github.com/apache/hudi/pull/2055 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] wangxianghu commented on pull request #2055: [HUDI-1253] Merge two TestHoodieSnapshotExporter to one class

2020-08-30 Thread GitBox
wangxianghu commented on pull request #2055: URL: https://github.com/apache/hudi/pull/2055#issuecomment-683501153 > Hi @wangxianghu thanks for the changes. The split was intended for distinguishing between unit test and functional tests. Please see this ticket https://issues.apache.org/jir

[jira] [Updated] (HUDI-945) Cleanup spillable map files eagerly as part of close

2020-08-30 Thread Sreeram Ramji (Jira)
[ https://issues.apache.org/jira/browse/HUDI-945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreeram Ramji updated HUDI-945: --- Status: In Progress (was: Open) > Cleanup spillable map files eagerly as part of close > -

[jira] [Updated] (HUDI-1153) Spark DataSource and Streaming Write must fail when operation type is misconfigured

2020-08-30 Thread Sreeram Ramji (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreeram Ramji updated HUDI-1153: Status: Patch Available (was: In Progress) > Spark DataSource and Streaming Write must fail when op

[GitHub] [hudi] zherenyu831 edited a comment on issue #2041: [SUPPORT] EMR 6.0 hudi-spark-bundle 0.6.0 is not able to query data, two problems

2020-08-30 Thread GitBox
zherenyu831 edited a comment on issue #2041: URL: https://github.com/apache/hudi/issues/2041#issuecomment-683495395 @bvaradar >Are you using version of spark built with scala 2.12 ? Yes I think AWS built spark-sql_2.12-2.4.4 by themself, some interfaces are different from

[GitHub] [hudi] zherenyu831 commented on issue #2041: [SUPPORT] EMR 6.0 hudi-spark-bundle 0.6.0 is not able to query data, two problems

2020-08-30 Thread GitBox
zherenyu831 commented on issue #2041: URL: https://github.com/apache/hudi/issues/2041#issuecomment-683495395 @bvaradar >Are you using version of spark built with scala 2.12 ? Yes I think AWS built spark-sql_2.12-2.4.4 by themself, some interfaces are different from oss version

[GitHub] [hudi] zherenyu831 edited a comment on issue #2041: [SUPPORT] EMR 6.0 hudi-spark-bundle 0.6.0 is not able to query data, two problems

2020-08-30 Thread GitBox
zherenyu831 edited a comment on issue #2041: URL: https://github.com/apache/hudi/issues/2041#issuecomment-682467983 @bvaradar I got the reason We are using EMR 6.0, in the spark-sql_2.12.2.4.4 The interface of org.apache.spark.sql.execution.datasources.PartitionedFile is differen

[GitHub] [hudi] xushiyan commented on pull request #2054: [HUDI-1252] Remove unused class NoOpBulkInsertPartitioner in DataSour…

2020-08-30 Thread GitBox
xushiyan commented on pull request #2054: URL: https://github.com/apache/hudi/pull/2054#issuecomment-683448216 Hi @wangxianghu the removal looks good. Good catch! While you're here, could you also clean up other unused methods in this class? namely ``` org.apache.hudi.testutils

[GitHub] [hudi] xushiyan edited a comment on pull request #2055: [HUDI-1253] Merge two TestHoodieSnapshotExporter to one class

2020-08-30 Thread GitBox
xushiyan edited a comment on pull request #2055: URL: https://github.com/apache/hudi/pull/2055#issuecomment-683446451 Hi @wangxianghu thanks for the changes. The split was intended for distinguishing between unit test and functional tests. Please see this ticket https://issues.apache.org/j

[GitHub] [hudi] xushiyan commented on pull request #2055: [HUDI-1253] Merge two TestHoodieSnapshotExporter to one class

2020-08-30 Thread GitBox
xushiyan commented on pull request #2055: URL: https://github.com/apache/hudi/pull/2055#issuecomment-683446451 Hi @wangxianghu thanks for the changes. The split was intended for distinguishing between unit test and functional tests. Please see this ticket https://issues.apache.org/jira/bro

[jira] [Updated] (HUDI-1254) TypedProperties can not get values by initializing an existing properties

2020-08-30 Thread cdmikechen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cdmikechen updated HUDI-1254: - Description: If I create a test to new a TypedProperties by a Properties that exists like blow: {code:jav

[jira] [Created] (HUDI-1254) TypedProperties can not get values by initializing an existing properties

2020-08-30 Thread cdmikechen (Jira)
cdmikechen created HUDI-1254: Summary: TypedProperties can not get values by initializing an existing properties Key: HUDI-1254 URL: https://issues.apache.org/jira/browse/HUDI-1254 Project: Apache Hudi

[jira] [Updated] (HUDI-1253) Merge two TestHoodieSnapshotExporter to one class

2020-08-30 Thread wangxianghu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-1253: -- Summary: Merge two TestHoodieSnapshotExporter to one class (was: Merge two TestHoodieSnapshotExporter i

[jira] [Updated] (HUDI-1253) Merge two TestHoodieSnapshotExporter to one class

2020-08-30 Thread wangxianghu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-1253: -- Description: Currently, there are two classes named `TestHoodieSnapshotExporter`, we'd better merge the

[GitHub] [hudi] wangxianghu commented on pull request #2054: [HUDI-1252] Remove unused class NoOpBulkInsertPartitioner in DataSour…

2020-08-30 Thread GitBox
wangxianghu commented on pull request #2054: URL: https://github.com/apache/hudi/pull/2054#issuecomment-683406638 @yanghua @xushiyan please take a look when free This is an automated message from the Apache Git Service. To

[GitHub] [hudi] wangxianghu commented on pull request #2055: [HUDI-1253] Merge two TestHoodieSnapshotExporter in hudi-utilities

2020-08-30 Thread GitBox
wangxianghu commented on pull request #2055: URL: https://github.com/apache/hudi/pull/2055#issuecomment-683406706 @yanghua @xushiyan please take a look when free This is an automated message from the Apache Git Service. T

[jira] [Updated] (HUDI-1253) Merge two TestHoodieSnapshotExporter in hudi-utilities

2020-08-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1253: - Labels: pull-request-available (was: ) > Merge two TestHoodieSnapshotExporter in hudi-utilities >

[GitHub] [hudi] wangxianghu opened a new pull request #2055: [HUDI-1253] Merge two TestHoodieSnapshotExporter in hudi-utilities

2020-08-30 Thread GitBox
wangxianghu opened a new pull request #2055: URL: https://github.com/apache/hudi/pull/2055 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the

[jira] [Updated] (HUDI-1253) Merge two TestHoodieSnapshotExporter in hudi-utilities

2020-08-30 Thread wangxianghu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-1253: -- Summary: Merge two TestHoodieSnapshotExporter in hudi-utilities (was: Rename TestHoodieSnapshotExporter

[jira] [Updated] (HUDI-1253) Rename TestHoodieSnapshotExporter in hudi-utilities

2020-08-30 Thread wangxianghu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianghu updated HUDI-1253: -- Description: Currently, there are two classes named `TestHoodieSnapshotExporter`, we'd better rename on

[jira] [Created] (HUDI-1253) Rename TestHoodieSnapshotExporter in hudi-utilities

2020-08-30 Thread wangxianghu (Jira)
wangxianghu created HUDI-1253: - Summary: Rename TestHoodieSnapshotExporter in hudi-utilities Key: HUDI-1253 URL: https://issues.apache.org/jira/browse/HUDI-1253 Project: Apache Hudi Issue Type: I

[jira] [Updated] (HUDI-1252) Remove unused class NoOpBulkInsertPartitioner in DataSourceTestUtils

2020-08-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1252: - Labels: pull-request-available (was: ) > Remove unused class NoOpBulkInsertPartitioner in DataSou

[GitHub] [hudi] wangxianghu opened a new pull request #2054: [HUDI-1252] Remove unused class NoOpBulkInsertPartitioner in DataSour…

2020-08-30 Thread GitBox
wangxianghu opened a new pull request #2054: URL: https://github.com/apache/hudi/pull/2054 …ceTestUtils ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What i

[jira] [Created] (HUDI-1252) Remove unused class NoOpBulkInsertPartitioner in DataSourceTestUtils

2020-08-30 Thread wangxianghu (Jira)
wangxianghu created HUDI-1252: - Summary: Remove unused class NoOpBulkInsertPartitioner in DataSourceTestUtils Key: HUDI-1252 URL: https://issues.apache.org/jira/browse/HUDI-1252 Project: Apache Hudi

[jira] [Commented] (HUDI-195) Bump jackson-databind to prevent deserialization loophole

2020-08-30 Thread vinoyang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17187151#comment-17187151 ] vinoyang commented on HUDI-195: --- [~shivnarayan] Will verify the version. > Bump jackson-data