[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1521: [HUDI-798] Migrate to Mockito Jupiter for JUnit 5

2020-04-15 Thread GitBox
xushiyan commented on a change in pull request #1521: [HUDI-798] Migrate to Mockito Jupiter for JUnit 5 URL: https://github.com/apache/incubator-hudi/pull/1521#discussion_r409314065 ## File path: hudi-client/src/test/java/org/apache/hudi/index/TestHbaseIndex.java ## @@ -70

[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1521: [HUDI-798] Migrate to Mockito Jupiter for JUnit 5

2020-04-15 Thread GitBox
xushiyan commented on a change in pull request #1521: [HUDI-798] Migrate to Mockito Jupiter for JUnit 5 URL: https://github.com/apache/incubator-hudi/pull/1521#discussion_r409315688 ## File path: hudi-hadoop-mr/src/test/java/org/apache/hudi/hadoop/realtime/TestHoodieRealtimeFileSpl

[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1521: [HUDI-798] Migrate to Mockito Jupiter for JUnit 5

2020-04-15 Thread GitBox
xushiyan commented on a change in pull request #1521: [HUDI-798] Migrate to Mockito Jupiter for JUnit 5 URL: https://github.com/apache/incubator-hudi/pull/1521#discussion_r409317776 ## File path: hudi-hadoop-mr/src/test/java/org/apache/hudi/hadoop/realtime/TestHoodieRealtimeFileSpl

[incubator-hudi] branch hudi_test_suite_refactor updated (37f91cb -> 7379b8d)

2020-04-15 Thread nagarwal
This is an automated email from the ASF dual-hosted git repository. nagarwal pushed a change to branch hudi_test_suite_refactor in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git. omit 37f91cb more fixes add 7379b8d more fixes This update added new revisions after u

[incubator-hudi] branch master updated: [HUDI-741] Added checks to validate Hoodie's schema evolution.

2020-04-15 Thread nagarwal
This is an automated email from the ASF dual-hosted git repository. nagarwal pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new 19d29ac [HUDI-741] Added checks to va

[GitHub] [incubator-hudi] n3nash merged pull request #1457: [HUDI-741] Added checks to validate Hoodie's schema evolution.

2020-04-15 Thread GitBox
n3nash merged pull request #1457: [HUDI-741] Added checks to validate Hoodie's schema evolution. URL: https://github.com/apache/incubator-hudi/pull/1457 This is an automated message from the Apache Git Service. To respond to

[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource

2020-04-15 Thread GitBox
xushiyan commented on a change in pull request #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource URL: https://github.com/apache/incubator-hudi/pull/1500#discussion_r409313000 ## File path: hudi-spark/src/test/java/DataSourceUtilsTest.java #

[GitHub] [incubator-hudi] xushiyan opened a new pull request #1521: [HUDI-798] Migrate to Mockito Jupiter for JUnit 5

2020-04-15 Thread GitBox
xushiyan opened a new pull request #1521: [HUDI-798] Migrate to Mockito Jupiter for JUnit 5 URL: https://github.com/apache/incubator-hudi/pull/1521 - Remove old mockito dependency - Add mockito-junit-jupiter ## Committer checklist - [ ] Has a corresponding JIRA in PR title

[jira] [Updated] (HUDI-798) Migrate Mockito to work with Junit 5

2020-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-798: Labels: pull-request-available (was: ) > Migrate Mockito to work with Junit 5 >

[jira] [Commented] (HUDI-69) Support realtime view in Spark datasource #136

2020-04-15 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-69?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084562#comment-17084562 ] Vinoth Chandar commented on HUDI-69: Switching to v2 for writing is tricky... V2 datasou

[jira] [Commented] (HUDI-69) Support realtime view in Spark datasource #136

2020-04-15 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-69?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084560#comment-17084560 ] Vinoth Chandar commented on HUDI-69: . I was wondering if we can just wrap the FileForma

[jira] [Updated] (HUDI-798) Migrate Mockito to work with Junit 5

2020-04-15 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-798: Status: In Progress (was: Open) > Migrate Mockito to work with Junit 5 > ---

[jira] [Updated] (HUDI-798) Migrate Mockito to work with Junit 5

2020-04-15 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-798: Status: Open (was: New) > Migrate Mockito to work with Junit 5 > > >

[jira] [Created] (HUDI-798) Migrate Mockito to work with Junit 5

2020-04-15 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-798: --- Summary: Migrate Mockito to work with Junit 5 Key: HUDI-798 URL: https://issues.apache.org/jira/browse/HUDI-798 Project: Apache Hudi (incubating) Issue Type: Test

[jira] [Assigned] (HUDI-765) Implement OrcReaderIterator

2020-04-15 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li reassigned HUDI-765: --- Assignee: Yanjia Gary Li > Implement OrcReaderIterator > --- > >

[jira] [Assigned] (HUDI-765) Implement OrcReaderIterator

2020-04-15 Thread lamber-ken (Jira)
[ https://issues.apache.org/jira/browse/HUDI-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lamber-ken reassigned HUDI-765: --- Assignee: (was: lamber-ken) > Implement OrcReaderIterator > --- > >

[GitHub] [incubator-hudi] lamber-ken commented on issue #1498: Migrating parquet table to hudi issue [SUPPORT]

2020-04-15 Thread GitBox
lamber-ken commented on issue #1498: Migrating parquet table to hudi issue [SUPPORT] URL: https://github.com/apache/incubator-hudi/issues/1498#issuecomment-614400699 hi @vinothchandar, already in our slack channel. ![image](https://user-images.githubusercontent.com/20113411/79412692-aec

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #249

2020-04-15 Thread Apache Jenkins Server
See Changes: -- [...truncated 2.37 KB...] /home/jenkins/tools/maven/apache-maven-3.5.4/conf: logging settings.xml toolchains.xml /home/jenkins/tools/maven/apache-maven-3.5.

[GitHub] [incubator-hudi] bvaradar commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

2020-04-15 Thread GitBox
bvaradar commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-614381927 @vinothchandar This fix should reflect only in inflight commit metadata and rollback works in the way you mentioned. As

[jira] [Closed] (HUDI-409) Replace Log Magic header with a secure hash to avoid clashes with data

2020-04-15 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar closed HUDI-409. --- Fix Version/s: (was: 0.6.0) 0.5.2 Resolution: Fixed > Replace Log Magic h

[jira] [Commented] (HUDI-409) Replace Log Magic header with a secure hash to avoid clashes with data

2020-04-15 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084495#comment-17084495 ] Vinoth Chandar commented on HUDI-409: - [~msramachandran] I was not aware of this conver

[GitHub] [incubator-hudi] vinothchandar commented on issue #1498: Migrating parquet table to hudi issue [SUPPORT]

2020-04-15 Thread GitBox
vinothchandar commented on issue #1498: Migrating parquet table to hudi issue [SUPPORT] URL: https://github.com/apache/incubator-hudi/issues/1498#issuecomment-614378189 @ahmed-elfar do you mind joining our slack channel and we can chat 1-1 to setup a time.. --

[GitHub] [incubator-hudi] vinothchandar commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

2020-04-15 Thread GitBox
vinothchandar commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-614376368 @n3nash can you actually help review this.. does this pose any issues for rollback today? (IIUC for rollback of ins

[GitHub] [incubator-hudi] liujianhuiouc commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

2020-04-15 Thread GitBox
liujianhuiouc commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-614374608 @lamber-ken please help to merge This is an automa

[GitHub] [incubator-hudi] lamber-ken commented on issue #1224: [HUDI-397] Normalize log print statement

2020-04-15 Thread GitBox
lamber-ken commented on issue #1224: [HUDI-397] Normalize log print statement URL: https://github.com/apache/incubator-hudi/pull/1224#issuecomment-614372674 hi @wangxianghu, you can use `git rebase` operation and try again Thi

[GitHub] [incubator-hudi] lamber-ken commented on issue #1488: [SUPPORT] Hudi table has only five rows when record key is binary

2020-04-15 Thread GitBox
lamber-ken commented on issue #1488: [SUPPORT] Hudi table has only five rows when record key is binary URL: https://github.com/apache/incubator-hudi/issues/1488#issuecomment-614370054 hi @jvaesteves, are you still facing this problem? : ) ---

[GitHub] [incubator-hudi] lamber-ken commented on issue #1517: Use appropriate FS when loading configs

2020-04-15 Thread GitBox
lamber-ken commented on issue #1517: Use appropriate FS when loading configs URL: https://github.com/apache/incubator-hudi/pull/1517#issuecomment-614368160 hi @afilipchik, thanks for your contribution, we need add `[MINOR]` to title for minor changes. e.g `[MINOR] Use appropriate FS

[GitHub] [incubator-hudi] lamber-ken commented on issue #1491: [SUPPORT] OutOfMemoryError during upsert 53M records

2020-04-15 Thread GitBox
lamber-ken commented on issue #1491: [SUPPORT] OutOfMemoryError during upsert 53M records URL: https://github.com/apache/incubator-hudi/issues/1491#issuecomment-614366584 hi @tverdokhlebd, are you still facing this problem? Or can you ask your colleagues to try again?

[GitHub] [incubator-hudi] lamber-ken commented on issue #1520: [HUDI-797] Small performance improvement for rewriting records.

2020-04-15 Thread GitBox
lamber-ken commented on issue #1520: [HUDI-797] Small performance improvement for rewriting records. URL: https://github.com/apache/incubator-hudi/pull/1520#issuecomment-614365314 > @lamber-ken could you please review this OK : ) -

[GitHub] [incubator-hudi] lamber-ken commented on issue #1519: [HUDI-777] Update Deltastreamer param description for --target-table

2020-04-15 Thread GitBox
lamber-ken commented on issue #1519: [HUDI-777] Update Deltastreamer param description for --target-table URL: https://github.com/apache/incubator-hudi/pull/1519#issuecomment-614364497 Welcome @iftachsc ๐Ÿ‘ This is an automate

[incubator-hudi] branch hudi_test_suite_refactor updated (1689d01 -> 37f91cb)

2020-04-15 Thread nagarwal
This is an automated email from the ASF dual-hosted git repository. nagarwal pushed a change to branch hudi_test_suite_refactor in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git. discard 1689d01 Merge branch 'master' into hudi_test_suite_refactor discard a096128 more fixes

[incubator-hudi] branch hudi_test_suite_refactor updated (a096128 -> 1689d01)

2020-04-15 Thread nagarwal
This is an automated email from the ASF dual-hosted git repository. nagarwal pushed a change to branch hudi_test_suite_refactor in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git. from a096128 more fixes add c0f96e0 [HUDI-687] Stop incremental reader on RO table when

[jira] [Resolved] (HUDI-780) Make JUnit 5 co-exist in the project

2020-04-15 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu resolved HUDI-780. - Resolution: Done > Make JUnit 5 co-exist in the project > > >

[jira] [Updated] (HUDI-780) Make JUnit 5 co-exist in the project

2020-04-15 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-780: Status: Closed (was: Patch Available) > Make JUnit 5 co-exist in the project > -

[jira] [Reopened] (HUDI-780) Make JUnit 5 co-exist in the project

2020-04-15 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reopened HUDI-780: - > Make JUnit 5 co-exist in the project > > > Key: HUDI-780

[GitHub] [incubator-hudi] afilipchik commented on a change in pull request #1515: [HUDI-795] Ignoring missing aux folder

2020-04-15 Thread GitBox
afilipchik commented on a change in pull request #1515: [HUDI-795] Ignoring missing aux folder URL: https://github.com/apache/incubator-hudi/pull/1515#discussion_r409208032 ## File path: hudi-client/src/main/java/org/apache/hudi/table/HoodieCommitArchiveLog.java ## @@ -21

[GitHub] [incubator-hudi] afilipchik commented on a change in pull request #1517: Use appropriate FS when loading configs

2020-04-15 Thread GitBox
afilipchik commented on a change in pull request #1517: Use appropriate FS when loading configs URL: https://github.com/apache/incubator-hudi/pull/1517#discussion_r409206422 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java

[GitHub] [incubator-hudi] afilipchik commented on a change in pull request #1517: Use appropriate FS when loading configs

2020-04-15 Thread GitBox
afilipchik commented on a change in pull request #1517: Use appropriate FS when loading configs URL: https://github.com/apache/incubator-hudi/pull/1517#discussion_r409205392 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java

[GitHub] [incubator-hudi] codecov-io edited a comment on issue #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource

2020-04-15 Thread GitBox
codecov-io edited a comment on issue #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource URL: https://github.com/apache/incubator-hudi/pull/1500#issuecomment-613236143 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1500?src=pr&el=h1) Report

[GitHub] [incubator-hudi] codecov-io edited a comment on issue #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource

2020-04-15 Thread GitBox
codecov-io edited a comment on issue #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource URL: https://github.com/apache/incubator-hudi/pull/1500#issuecomment-613236143 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1500?src=pr&el=h1) Report

[GitHub] [incubator-hudi] kwondw commented on a change in pull request #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource

2020-04-15 Thread GitBox
kwondw commented on a change in pull request #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource URL: https://github.com/apache/incubator-hudi/pull/1500#discussion_r409198506 ## File path: hudi-spark/src/test/java/DataSourceUtilsTest.java ##

[GitHub] [incubator-hudi] codecov-io commented on issue #1457: [HUDI-741] Added checks to validate Hoodie's schema evolution.

2020-04-15 Thread GitBox
codecov-io commented on issue #1457: [HUDI-741] Added checks to validate Hoodie's schema evolution. URL: https://github.com/apache/incubator-hudi/pull/1457#issuecomment-614333053 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1457?src=pr&el=h1) Report > Merging [#1457](h

[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource

2020-04-15 Thread GitBox
xushiyan commented on a change in pull request #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource URL: https://github.com/apache/incubator-hudi/pull/1500#discussion_r409173783 ## File path: hudi-spark/src/test/java/DataSourceUtilsTest.java #

[GitHub] [incubator-hudi] vinothchandar commented on issue #1512: [HUDI-763] Add hoodie.table.base.file.format option to hoodie.properties file

2020-04-15 Thread GitBox
vinothchandar commented on issue #1512: [HUDI-763] Add hoodie.table.base.file.format option to hoodie.properties file URL: https://github.com/apache/incubator-hudi/pull/1512#issuecomment-614308722 Fair enough.. Hypothetically, we could support a world where base files across partitions are

[incubator-hudi] branch hudi_test_suite_refactor updated (5803025 -> a096128)

2020-04-15 Thread nagarwal
This is an automated email from the ASF dual-hosted git repository. nagarwal pushed a change to branch hudi_test_suite_refactor in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git. discard 5803025 more fixes add a096128 more fixes This update added new revisions after u

[GitHub] [incubator-hudi] vinothchandar commented on issue #1520: [HUDI-797] Small performance improvement for rewriting records.

2020-04-15 Thread GitBox
vinothchandar commented on issue #1520: [HUDI-797] Small performance improvement for rewriting records. URL: https://github.com/apache/incubator-hudi/pull/1520#issuecomment-614301101 @lamber-ken could you please review this

[GitHub] [incubator-hudi] prashantwason commented on a change in pull request #1476: [HUDI-757] Added hudi-cli command to export metadata of Instants.

2020-04-15 Thread GitBox
prashantwason commented on a change in pull request #1476: [HUDI-757] Added hudi-cli command to export metadata of Instants. URL: https://github.com/apache/incubator-hudi/pull/1476#discussion_r409158527 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/ExportCommand

[GitHub] [incubator-hudi] prashantwason commented on a change in pull request #1476: [HUDI-757] Added hudi-cli command to export metadata of Instants.

2020-04-15 Thread GitBox
prashantwason commented on a change in pull request #1476: [HUDI-757] Added hudi-cli command to export metadata of Instants. URL: https://github.com/apache/incubator-hudi/pull/1476#discussion_r409158467 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/ExportCommand

[incubator-hudi] branch master updated (d65efe6 -> 9ca710c)

2020-04-15 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git. from d65efe6 [HUDI-780] Migrate test cases to Junit 5 (#1504) add 9ca710c [HUDI-777] Updated description fo

[GitHub] [incubator-hudi] vinothchandar merged pull request #1519: [HUDI-777] Update Deltastreamer param description for --target-table

2020-04-15 Thread GitBox
vinothchandar merged pull request #1519: [HUDI-777] Update Deltastreamer param description for --target-table URL: https://github.com/apache/incubator-hudi/pull/1519 This is an automated message from the Apache Git Service.

[jira] [Commented] (HUDI-409) Replace Log Magic header with a secure hash to avoid clashes with data

2020-04-15 Thread Ramachandran M S (Jira)
[ https://issues.apache.org/jira/browse/HUDI-409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084405#comment-17084405 ] Ramachandran M S commented on HUDI-409: --- [~vinoth]ย can you please give more details (

[jira] [Updated] (HUDI-797) Improve performance of rewriting AVRO records in HoodieAvroUtils::rewriteRecord

2020-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-797: Labels: pull-request-available (was: ) > Improve performance of rewriting AVRO records in > HoodieA

[GitHub] [incubator-hudi] prashantwason opened a new pull request #1520: [HUDI-797] Small performance improvement for rewriting records.

2020-04-15 Thread GitBox
prashantwason opened a new pull request #1520: [HUDI-797] Small performance improvement for rewriting records. URL: https://github.com/apache/incubator-hudi/pull/1520 When adding HUDI metadata field to schema, the position of incoming schema fields is retained. This allows us to lookup fie

[jira] [Commented] (HUDI-797) Improve performance of rewriting AVRO records in HoodieAvroUtils::rewriteRecord

2020-04-15 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084382#comment-17084382 ] Prashant Wason commented on HUDI-797: - My testing with two schemas (large / small) whil

[jira] [Commented] (HUDI-797) Improve performance of rewriting AVRO records in HoodieAvroUtils::rewriteRecord

2020-04-15 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084380#comment-17084380 ] Prashant Wason commented on HUDI-797: - Conversion step is uses a HashMap to find the fi

[jira] [Commented] (HUDI-797) Improve performance of rewriting AVRO records in HoodieAvroUtils::rewriteRecord

2020-04-15 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084378#comment-17084378 ] Prashant Wason commented on HUDI-797: - HoodieAvroUtils::rewriteRecord function has two

[jira] [Updated] (HUDI-797) Improve performance of rewriting AVRO records in HoodieAvroUtils::rewriteRecord

2020-04-15 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-797: Status: Open (was: New) > Improve performance of rewriting AVRO records in > HoodieAvroUtils::rewri

[jira] [Assigned] (HUDI-797) Improve performance of rewriting AVRO records in HoodieAvroUtils::rewriteRecord

2020-04-15 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason reassigned HUDI-797: --- Assignee: Prashant Wason > Improve performance of rewriting AVRO records in > HoodieAvroUtils

[jira] [Updated] (HUDI-797) Improve performance of rewriting AVRO records in HoodieAvroUtils::rewriteRecord

2020-04-15 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-797: Status: In Progress (was: Open) > Improve performance of rewriting AVRO records in > HoodieAvroUtil

[jira] [Created] (HUDI-797) Improve performance of rewriting AVRO records in HoodieAvroUtils::rewriteRecord

2020-04-15 Thread Prashant Wason (Jira)
Prashant Wason created HUDI-797: --- Summary: Improve performance of rewriting AVRO records in HoodieAvroUtils::rewriteRecord Key: HUDI-797 URL: https://issues.apache.org/jira/browse/HUDI-797 Project: Apac

[jira] [Commented] (HUDI-741) Fix Hoodie's schema evolution checks

2020-04-15 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084372#comment-17084372 ] Prashant Wason commented on HUDI-741: - Sure. Will work on the blog post soon. > Fix Ho

[GitHub] [incubator-hudi] lamber-ken commented on a change in pull request #1476: [HUDI-757] Added hudi-cli command to export metadata of Instants.

2020-04-15 Thread GitBox
lamber-ken commented on a change in pull request #1476: [HUDI-757] Added hudi-cli command to export metadata of Instants. URL: https://github.com/apache/incubator-hudi/pull/1476#discussion_r409110579 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/ExportCommand.ja

[GitHub] [incubator-hudi] lamber-ken commented on a change in pull request #1476: [HUDI-757] Added hudi-cli command to export metadata of Instants.

2020-04-15 Thread GitBox
lamber-ken commented on a change in pull request #1476: [HUDI-757] Added hudi-cli command to export metadata of Instants. URL: https://github.com/apache/incubator-hudi/pull/1476#discussion_r409110372 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/ExportCommand.ja

[GitHub] [incubator-hudi] lamber-ken commented on a change in pull request #1476: [HUDI-757] Added hudi-cli command to export metadata of Instants.

2020-04-15 Thread GitBox
lamber-ken commented on a change in pull request #1476: [HUDI-757] Added hudi-cli command to export metadata of Instants. URL: https://github.com/apache/incubator-hudi/pull/1476#discussion_r409110372 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/ExportCommand.ja

[GitHub] [incubator-hudi] lamber-ken edited a comment on issue #1512: [HUDI-763] Add hoodie.table.base.file.format option to hoodie.properties file

2020-04-15 Thread GitBox
lamber-ken edited a comment on issue #1512: [HUDI-763] Add hoodie.table.base.file.format option to hoodie.properties file URL: https://github.com/apache/incubator-hudi/pull/1512#issuecomment-614255323 hi @vinothchandar @pratyakshsharma, thanks very much for reviewing this. The base

[GitHub] [incubator-hudi] lamber-ken commented on issue #1512: [HUDI-763] Add hoodie.table.base.file.format option to hoodie.properties file

2020-04-15 Thread GitBox
lamber-ken commented on issue #1512: [HUDI-763] Add hoodie.table.base.file.format option to hoodie.properties file URL: https://github.com/apache/incubator-hudi/pull/1512#issuecomment-614255323 hi @vinothchandar @pratyakshsharma, thanks very much for reviewing this. The base file fo

[GitHub] [incubator-hudi] prashantwason commented on a change in pull request #1457: [HUDI-741] Added checks to validate Hoodie's schema evolution.

2020-04-15 Thread GitBox
prashantwason commented on a change in pull request #1457: [HUDI-741] Added checks to validate Hoodie's schema evolution. URL: https://github.com/apache/incubator-hudi/pull/1457#discussion_r409104802 ## File path: hudi-common/src/main/java/org/apache/hudi/common/util/SchemaCompatib

[GitHub] [incubator-hudi] prashantwason commented on a change in pull request #1457: [HUDI-741] Added checks to validate Hoodie's schema evolution.

2020-04-15 Thread GitBox
prashantwason commented on a change in pull request #1457: [HUDI-741] Added checks to validate Hoodie's schema evolution. URL: https://github.com/apache/incubator-hudi/pull/1457#discussion_r409104956 ## File path: hudi-common/src/main/java/org/apache/hudi/common/util/SchemaUtil.jav

[GitHub] [incubator-hudi] prashantwason commented on issue #1457: [HUDI-741] Added checks to validate Hoodie's schema evolution.

2020-04-15 Thread GitBox
prashantwason commented on issue #1457: [HUDI-741] Added checks to validate Hoodie's schema evolution. URL: https://github.com/apache/incubator-hudi/pull/1457#issuecomment-614253179 I have reworked the schema compatibility check code to remove copying the entire avro.SchemaCompatibility cl

[GitHub] [incubator-hudi] hmatu commented on issue #1432: [HUDI-716] Exception: Not an Avro data file when running HoodieCleanClient.runClean

2020-04-15 Thread GitBox
hmatu commented on issue #1432: [HUDI-716] Exception: Not an Avro data file when running HoodieCleanClient.runClean URL: https://github.com/apache/incubator-hudi/pull/1432#issuecomment-614241591 @lamber-ken thanks for the changes, I have been running into the same issue and worked! ๐Ÿ‘ ---

[GitHub] [incubator-hudi] vinothchandar merged pull request #1504: [HUDI-780] Migrate test cases to Junit 5

2020-04-15 Thread GitBox
vinothchandar merged pull request #1504: [HUDI-780] Migrate test cases to Junit 5 URL: https://github.com/apache/incubator-hudi/pull/1504 This is an automated message from the Apache Git Service. To respond to the message, p

[incubator-hudi] branch master updated: [HUDI-780] Migrate test cases to Junit 5 (#1504)

2020-04-15 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new d65efe6 [HUDI-780] Migrate test cases t

[jira] [Updated] (HUDI-777) Update Deltastreamer param description for --target-table

2020-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-777: Labels: docs newbie pull-request-available (was: docs newbie) > Update Deltastreamer param descripti

[GitHub] [incubator-hudi] iftachsc opened a new pull request #1519: [HUDI-777] Update Deltastreamer param description for --target-table

2020-04-15 Thread GitBox
iftachsc opened a new pull request #1519: [HUDI-777] Update Deltastreamer param description for --target-table URL: https://github.com/apache/incubator-hudi/pull/1519 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributi

[incubator-hudi] branch hudi_test_suite_refactor updated (c7bf608 -> 5803025)

2020-04-15 Thread nagarwal
This is an automated email from the ASF dual-hosted git repository. nagarwal pushed a change to branch hudi_test_suite_refactor in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git. discard c7bf608 more fixes add 5803025 more fixes This update added new revisions after u

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1515: [HUDI-795] Ignoring missing aux folde

2020-04-15 Thread GitBox
pratyakshsharma commented on a change in pull request #1515: [HUDI-795] Ignoring missing aux folde URL: https://github.com/apache/incubator-hudi/pull/1515#discussion_r408978903 ## File path: hudi-client/src/main/java/org/apache/hudi/table/HoodieCommitArchiveLog.java ## @@

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1517: Use appropriate FS when loading configs

2020-04-15 Thread GitBox
pratyakshsharma commented on a change in pull request #1517: Use appropriate FS when loading configs URL: https://github.com/apache/incubator-hudi/pull/1517#discussion_r408964846 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.

[GitHub] [incubator-hudi] vinothchandar commented on issue #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource

2020-04-15 Thread GitBox
vinothchandar commented on issue #1500: [HUDI-772] Make UserDefinedBulkInsertPartitioner configurable for DataSource URL: https://github.com/apache/incubator-hudi/pull/1500#issuecomment-614125874 could you please rebase against latest master and get this to a state where only the necessary

[GitHub] [incubator-hudi] ahmed-elfar edited a comment on issue #1498: Migrating parquet table to hudi issue [SUPPORT]

2020-04-15 Thread GitBox
ahmed-elfar edited a comment on issue #1498: Migrating parquet table to hudi issue [SUPPORT] URL: https://github.com/apache/incubator-hudi/issues/1498#issuecomment-613563197 @vinothchandar I apologies for the delayed response, and thanks again for your help and detailed answers. > I