[GitHub] [hudi] cxzl25 commented on pull request #1674: [HUDI-973] RemoteHoodieTableFileSystemView supports non-partitioned table queries

2020-05-27 Thread GitBox
cxzl25 commented on pull request #1674: URL: https://github.com/apache/hudi/pull/1674#issuecomment-635106053 Enable timeline-server by default(https://github.com/apache/hudi/pull/1634), may use the previous version to write non-partitioned table operations, throw an exception, and switch

[GitHub] [hudi] cxzl25 opened a new pull request #1674: [HUDI-973] RemoteHoodieTableFileSystemView supports non-partitioned table queries

2020-05-27 Thread GitBox
cxzl25 opened a new pull request #1674: URL: https://github.com/apache/hudi/pull/1674 ## What is the purpose of the pull request When hoodie.embed.timeline.server = true, the written table is a non-partitioned table, will get an exception. ```java

[jira] [Updated] (HUDI-973) RemoteHoodieTableFileSystemView supports non-partitioned table queries

2020-05-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-973: Labels: pull-request-available (was: ) > RemoteHoodieTableFileSystemView supports non-partitioned

[jira] [Created] (HUDI-973) RemoteHoodieTableFileSystemView supports non-partitioned table queries

2020-05-27 Thread dzcxzl (Jira)
dzcxzl created HUDI-973: --- Summary: RemoteHoodieTableFileSystemView supports non-partitioned table queries Key: HUDI-973 URL: https://issues.apache.org/jira/browse/HUDI-973 Project: Apache Hudi Issue

[jira] [Updated] (HUDI-972) Update hudi logo

2020-05-27 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-972: - Status: Open (was: New) > Update hudi logo > > > Key:

[jira] [Closed] (HUDI-972) Update hudi logo

2020-05-27 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-972. Resolution: Invalid > Update hudi logo > > > Key: HUDI-972

[jira] [Commented] (HUDI-972) Update hudi logo

2020-05-27 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118305#comment-17118305 ] sivabalan narayanan commented on HUDI-972: -- ;) my bad. I did refresh many times before filing one.

[jira] [Commented] (HUDI-972) Update hudi logo

2020-05-27 Thread leesf (Jira)
[ https://issues.apache.org/jira/browse/HUDI-972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118297#comment-17118297 ] leesf commented on HUDI-972: hi [~shivnarayan] the logo has been updated. you need refresh the website. >

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #291

2020-05-27 Thread Apache Jenkins Server
See Changes: -- [...truncated 2.41 KB...] toolchains.xml /home/jenkins/tools/maven/apache-maven-3.5.4/conf/logging: simplelogger.properties

[jira] [Updated] (HUDI-972) Update hudi logo

2020-05-27 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-972: - Attachment: Screen Shot 2020-05-28 at 12.10.12 AM.png > Update hudi logo >

[jira] [Created] (HUDI-972) Update hudi logo

2020-05-27 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-972: Summary: Update hudi logo Key: HUDI-972 URL: https://issues.apache.org/jira/browse/HUDI-972 Project: Apache Hudi Issue Type: Sub-task

[GitHub] [hudi] WilliamWhispell commented on issue #998: Incremental view not implemented yet, for merge-on-read datasets

2020-05-27 Thread GitBox
WilliamWhispell commented on issue #998: URL: https://github.com/apache/hudi/issues/998#issuecomment-635069879 @bhasudha - all the examples on https://hudi.apache.org/docs/quick-start-guide.html#setup-spark-shell for PIT queries are for reading from disk. Does documentation exist for

[jira] [Created] (HUDI-971) Fix HFileBootstrapIndexReader.getIndexedPartitions() returns unclean partition name

2020-05-27 Thread Wenning Ding (Jira)
Wenning Ding created HUDI-971: - Summary: Fix HFileBootstrapIndexReader.getIndexedPartitions() returns unclean partition name Key: HUDI-971 URL: https://issues.apache.org/jira/browse/HUDI-971 Project:

[hudi] 09/40: [HUDI-850] Avoid unnecessary listings in incremental cleaning mode (#1576)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit ea48ca9b62a2f216460a0795ef9d519481510cd7 Author: Balaji Varadarajan AuthorDate: Fri May 1 21:37:21 2020 -0700

[hudi] 02/40: [HUDI-652] Decouple HoodieReadClient and AbstractHoodieClient to break the inheritance chain (#1372)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit dd7952ec9de7f9858ded98b49923e8978fd1df78 Author: vinoyang AuthorDate: Sat Mar 7 01:59:35 2020 +0800

[hudi] 27/40: [HUDI-902] Avoid exception when getSchemaProvider (#1584)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 5fbb5a76f9ded25e9b6bd15e2db40e1a7c7a0f4d Author: Raymond Xu <2701446+xushi...@users.noreply.github.com>

[hudi] 19/40: [HUDI-742] Fix Java Math Exception (#1466)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 2f4ee0a4af3c088f825decd77c5429c822b50ad6 Author: Edwin Guo AuthorDate: Tue Mar 31 00:56:20 2020 -0400

[hudi] branch release-0.5.3 created (now 54bbe32)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git. at 54bbe32 [HUDI-938] Removing incubating/incubator from project (#1658) This branch includes the following

[hudi] 10/40: [HUDI-724] Parallelize getSmallFiles for partitions (#1421)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 96b2359fda4d5b55e51e8f347228fe2f7cf6b24f Author: ffcchi AuthorDate: Mon Mar 30 01:14:38 2020 -0600

[hudi] 30/40: [HUDI-820] cleaner repair command should only inspect clean metadata files (#1542)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 4095de4d45f174b45dd1310a6a99c4fecfb2466a Author: Balaji Varadarajan AuthorDate: Sun May 10 18:25:54 2020 -0700

[hudi] 05/40: [HUDI - 738] Add validation to DeltaStreamer to fail fast when filterDupes is enabled on UPSERT mode. (#1505)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 2112d0ada59cb12b3be6eb7d407ba48b494c505f Author: Bhavani Sudha Saktheeswaran AuthorDate: Fri Apr 10 08:58:55

[hudi] 28/40: [HUDI-789]Adjust logic of upsert in HDFSParquetImporter (#1511)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 8324f6ecb030227beca87d92031cefb08c21ae6e Author: hongdd AuthorDate: Tue Apr 21 14:21:30 2020 +0800

[hudi] 36/40: Fixing test failure in TestHoodieClientOnCopyOnWriteStorage

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 07345bacb0b494158760fa9f4aa92a4980f1f085 Author: Sivabalan Narayanan AuthorDate: Mon May 25 08:28:33 2020 -0400

[hudi] 26/40: [HUDI-616] Fixed parquet files getting created on local FS (#1434)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 77d7500d3bbcedda81fa182908984e658d6689e5 Author: Pratyaksh Sharma AuthorDate: Sun Mar 22 19:49:47 2020 +0530

[hudi] 03/40: [HUDI-681]Remove embeddedTimelineService from HoodieReadClient (#1388)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 1d16b151b9079c4ad9bfb08bfa6114c3ad585e0f Author: hongdd AuthorDate: Mon Mar 9 18:31:04 2020 +0800

[hudi] 14/40: Add changes for presto mor queries (#1578)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit d476ac541b00b6a00dd690c4a7bfa3fc6037e7ab Author: bschell AuthorDate: Mon May 4 11:27:14 2020 -0700 Add

[hudi] 16/40: [HUDI-716] Exception: Not an Avro data file when running HoodieCleanClient.runClean (#1432)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 86698b516001fc30427f3f15a56ec5289d17a9c1 Author: lamber-ken AuthorDate: Mon Mar 30 13:19:17 2020 -0500

[hudi] 24/40: [HUDI-852] adding check for table name for Append Save mode (#1580)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 288bc4dd809521af896d0538ec2ae87bc034df7b Author: AakashPradeep AuthorDate: Sun May 3 23:09:17 2020 -0700

[hudi] 32/40: [HUDI-895] Remove unnecessary listing .hoodie folder when using timeline server (#1636)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 0fb02e7adf1b84d48c8983e9a4cd7eb8b91e Author: Balaji Varadarajan AuthorDate: Sun May 17 18:18:53 2020 -0700

[hudi] 25/40: [MINOR] fixed building IndexFileFilter with a wrong condition in HoodieGlobalBloomIndex class (#1537)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit be205eda53ab3210aea455a377aaf7610b8e35e1 Author: Carm <154939...@qq.com> AuthorDate: Sun May 10 09:45:07 2020

[hudi] 04/40: [HUDI-629]: Replace Guava's Hashing with an equivalent in NumericUtils.java (#1350)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit cebce61a3496f8788f5b96d6aedce0a8394fc325 Author: Suneel Marthi AuthorDate: Fri Mar 13 20:28:05 2020 -0400

[hudi] 39/40: [HUDI-926] Removing DISCLAIMER from the repo (#1657)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 3a776f6eb2793b31ca4a76fbbe59e205b83ab75b Author: leesf AuthorDate: Sun May 24 18:27:08 2020 +0800

[hudi] 37/40: Fixing test failures for TestRepairsCommand

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 0a77a3bbaba64d9b5930429d465b5f4071d494f8 Author: Sivabalan Narayanan AuthorDate: Mon May 25 09:54:02 2020 -0400

[hudi] 18/40: [MINOR] Update DOAP with 0.5.2 Release (#1448)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 45efa695c15f64294e84506a8e8cb19aba1af7df Author: Suneel Marthi AuthorDate: Wed Mar 25 23:37:32 2020 -0400

[hudi] 01/40: Moving to 0.5.3-SNAPSHOT for bug-fix release over 0.5.2

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit f6c87b56c1f79697428eafde8d167dbcfe93b282 Author: Bhavani Sudha Saktheeswaran AuthorDate: Thu May 7 09:49:23

[hudi] 35/40: [HUDI-846][HUDI-848] Enable Incremental cleaning and embedded timeline-server by default (#1634)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit b69bb18fdd3d5ab825d1f59024b10204890968ff Author: Balaji Varadarajan AuthorDate: Wed May 20 05:29:43 2020 -0700

[hudi] 40/40: [HUDI-938] Removing incubating/incubator from project (#1658)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 54bbe32d741733e405b241fd95d1dc2dfefea144 Author: leesf AuthorDate: Sun May 24 18:28:13 2020 +0800

[hudi] 08/40: [HUDI-656][Performance] Return a dummy Spark relation after writing the DataFrame (#1394)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 9a2b3e3c93f670c79adb3d5cba69cdd3f551005a Author: Udit Mehrotra AuthorDate: Wed Mar 11 20:27:46 2020 -0700

[hudi] 06/40: [HUDI-799] Use appropriate FS when loading configs (#1517)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit e3f7659b3470a16f2d46ab75df576a7f00b432cd Author: Alexander Filipchik AuthorDate: Thu Apr 16 13:49:39 2020 -0700

[hudi] 17/40: [HUDI-400] Check upgrade from old plan to new plan for compaction (#1422)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 2e5f6011575cc9fb6adbae822718ea08ac00c4be Author: Zhiyuan Zhao <49054376+zhaomin1...@users.noreply.github.com>

[hudi] 11/40: [HUDI-607] Fix to allow creation/syncing of Hive tables partitioned by Date type columns (#1330)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 0050e00afdc728793f9928cbb594e657c7219c41 Author: Udit Mehrotra AuthorDate: Sun Mar 1 10:42:58 2020 -0800

[hudi] 15/40: [HUDI-782] Add support of Aliyun object storage service. (#1506)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit cc4aa45ae28bbd4135d1c65f38a2dcd5de749418 Author: Shen Hong AuthorDate: Sun Apr 12 10:06:30 2020 +0800

[hudi] 13/40: [HUDI-539] Make ROPathFilter conf member serializable (#1415)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit bd7060186f7165f01aa8508c3fd8c3c81b012a2f Author: vinoth chandar AuthorDate: Tue Mar 17 12:52:48 2020 -0700

[hudi] 33/40: [HUDI-858] Allow multiple operations to be executed within a single commit (#1633)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 555ff1aea4d7cc5947e890866b2a384b5409fb3a Author: Balaji Varadarajan AuthorDate: Mon May 18 19:27:24 2020 -0700

[hudi] 34/40: [HUDI-863] get decimal properties from derived spark DataType (#1596)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 4192cc65d02640f53db78922a1783103e7a46cfb Author: rolandjohann AuthorDate: Mon May 18 13:28:27 2020 +0200

[hudi] 31/40: HUDI-528 Handle empty commit in incremental pulling (#1612)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 7c766c33ddc5171743181767d1c6ce8d237ab5db Author: Gary Li AuthorDate: Thu May 14 22:55:25 2020 -0700

[hudi] 29/40: [HUDI-889] Writer supports useJdbc configuration when hive synchronization is enabled (#1627)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit da043af11e731b3e53d998c3db3a66f13fd59026 Author: cxzl25 AuthorDate: Thu May 14 00:20:13 2020 +0800

[hudi] 38/40: [MINOR] Remove incubating from README

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 498c86b2b8cdd640e9bf00e4a224f7c17ebef618 Author: vinoth chandar AuthorDate: Sat May 23 14:51:58 2020 -0700

[hudi] 22/40: [HUDI-795] Handle auto-deleted empty aux folder (#1515)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 500b3907a5057b3945115132beb97ef6cae37687 Author: Alexander Filipchik AuthorDate: Wed Apr 22 09:47:32 2020 -0700

[hudi] 07/40: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema (#1406)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 18453a456db0f5be0b935e5a04d8c0cdb3958936 Author: wenningd AuthorDate: Mon Mar 30 15:52:15 2020 -0700

[hudi] 20/40: [HUDI-717] Fixed usage of HiveDriver for DDL statements. (#1416)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit e80dd0414b1842fd1128fb7d6262b43f8426e048 Author: Prashant Wason AuthorDate: Fri Apr 3 16:23:05 2020 -0700

[hudi] 12/40: Add constructor to HoodieROTablePathFilter (#1413)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit d535381e3f98008cd00161304b7a4e59644f5da0 Author: bschell AuthorDate: Mon Mar 16 15:19:16 2020 -0700 Add

[hudi] 21/40: [HUDI-727]: Copy default values of fields if not present when rewriting incoming record with new schema (#1427)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit f9c3b078966fad1843554853b6179c04cae1d7f8 Author: Pratyaksh Sharma AuthorDate: Mon Apr 13 06:25:26 2020 +0530

[hudi] 23/40: [MINOR]: Fix cli docs for DeltaStreamer (#1547)

2020-05-27 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.5.3 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 0dcc20458d36fdb6d052a62ca52a77fa0c31745f Author: dengziming AuthorDate: Thu Apr 23 02:37:17 2020 +0800

[GitHub] [hudi] bvaradar commented on issue #1670: Error opening Hive split: Unknown converted type TIMESTAMP_MICROS

2020-05-27 Thread GitBox
bvaradar commented on issue #1670: URL: https://github.com/apache/hudi/issues/1670#issuecomment-635007877 @creactiviti : I think this is coming from spark. For ParquetDFSSource, Hudi uses spark.read().parquet() to get schema. Can you rewrite the same data again as plain parquet

[GitHub] [hudi] bvaradar commented on pull request #1664: HUDI-942 Increase default value number of delta commits for inline compaction

2020-05-27 Thread GitBox
bvaradar commented on pull request #1664: URL: https://github.com/apache/hudi/pull/1664#issuecomment-634986054 @vinothchandar : IIRC, the config was set to 1 to let new users have same out of box experience for Spark DataSource read (COW vs MOR).

[GitHub] [hudi] codecov-commenter edited a comment on pull request #1469: [HUDI-686] Implement BloomIndexV2 that does not depend on memory caching

2020-05-27 Thread GitBox
codecov-commenter edited a comment on pull request #1469: URL: https://github.com/apache/hudi/pull/1469#issuecomment-634970887 # [Codecov](https://codecov.io/gh/apache/hudi/pull/1469?src=pr=h1) Report > Merging [#1469](https://codecov.io/gh/apache/hudi/pull/1469?src=pr=desc) into

[GitHub] [hudi] vinothchandar commented on pull request #1664: HUDI-942 Increase default value number of delta commits for inline compaction

2020-05-27 Thread GitBox
vinothchandar commented on pull request #1664: URL: https://github.com/apache/hudi/pull/1664#issuecomment-634973785 @sathyaprakashg I am with you on this.. Just want to understand from @bvaradar why the default was made to 1..

[GitHub] [hudi] vinothchandar commented on pull request #1151: [HUDI-476] Add hudi-examples module

2020-05-27 Thread GitBox
vinothchandar commented on pull request #1151: URL: https://github.com/apache/hudi/pull/1151#issuecomment-634973383 Thank you @dengziming !! for this really great contribution.. We will keep improving this This is an

[GitHub] [hudi] codecov-commenter commented on pull request #1469: [HUDI-686] Implement BloomIndexV2 that does not depend on memory caching

2020-05-27 Thread GitBox
codecov-commenter commented on pull request #1469: URL: https://github.com/apache/hudi/pull/1469#issuecomment-634970887 # [Codecov](https://codecov.io/gh/apache/hudi/pull/1469?src=pr=h1) Report > Merging [#1469](https://codecov.io/gh/apache/hudi/pull/1469?src=pr=desc) into

[jira] [Resolved] (HUDI-773) Hudi On Azure Data Lake Storage V2

2020-05-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li resolved HUDI-773. - Resolution: Fixed Azure info was added to the docs. > Hudi On Azure Data Lake Storage V2 >

[jira] [Closed] (HUDI-773) Hudi On Azure Data Lake Storage V2

2020-05-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li closed HUDI-773. --- > Hudi On Azure Data Lake Storage V2 > -- > > Key:

[jira] [Resolved] (HUDI-804) Add Azure Support to Hudi Doc

2020-05-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li resolved HUDI-804. - Resolution: Fixed > Add Azure Support to Hudi Doc > - > >

[jira] [Closed] (HUDI-804) Add Azure Support to Hudi Doc

2020-05-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li closed HUDI-804. --- > Add Azure Support to Hudi Doc > - > > Key: HUDI-804 >

[jira] [Resolved] (HUDI-805) Verify which types of Azure storage support Hudi

2020-05-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li resolved HUDI-805. - Resolution: Fixed Azure Data Lake Storage Gen 2 and Azure Blob Storage support Hudi. > Verify

[jira] [Closed] (HUDI-805) Verify which types of Azure storage support Hudi

2020-05-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li closed HUDI-805. --- > Verify which types of Azure storage support Hudi > > >

[jira] [Updated] (HUDI-805) Verify which types of Azure storage support Hudi

2020-05-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-805: Status: Open (was: New) > Verify which types of Azure storage support Hudi >

[jira] [Updated] (HUDI-805) Verify which types of Azure storage support Hudi

2020-05-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-805: Status: In Progress (was: Open) > Verify which types of Azure storage support Hudi >

[hudi] branch asf-site updated: Travis CI build asf-site

2020-05-27 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 255db1b Travis CI build asf-site 255db1b is

[GitHub] [hudi] vinothchandar merged pull request #1673: [MINOR] Remove kyligence from powered_by page

2020-05-27 Thread GitBox
vinothchandar merged pull request #1673: URL: https://github.com/apache/hudi/pull/1673 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[hudi] branch asf-site updated: [MINOR] Remove kyligence from powered_by page (#1673)

2020-05-27 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 777f882 [MINOR] Remove kyligence from

[GitHub] [hudi] lamber-ken opened a new pull request #1673: [MINOR] Remove kyligence from powered_by page

2020-05-27 Thread GitBox
lamber-ken opened a new pull request #1673: URL: https://github.com/apache/hudi/pull/1673 ## What is the purpose of the pull request it's better to be conservative here, add it back if they are actually using it. : ) ## Brief change log - Remove kyligence from

[GitHub] [hudi] lamber-ken commented on pull request #1671: [HUDI-861]Add Github and Twitter Widget on Hudi's official website

2020-05-27 Thread GitBox
lamber-ken commented on pull request #1671: URL: https://github.com/apache/hudi/pull/1671#issuecomment-634867514 Hi @yanghua @hddong, maybe it's better to keep homepage clear. For example, https://bookkeeper.apache.org/

[GitHub] [hudi] lamber-ken commented on pull request #1671: [HUDI-861]Add Github and Twitter Widget on Hudi's official website

2020-05-27 Thread GitBox
lamber-ken commented on pull request #1671: URL: https://github.com/apache/hudi/pull/1671#issuecomment-634861772 Thanks for taking this, sync to https://lamber-ken.github.io

[GitHub] [hudi] xushiyan commented on pull request #1672: [HUDI-836] [BLOG] Monitor Hudi metrics with Datadog

2020-05-27 Thread GitBox
xushiyan commented on pull request #1672: URL: https://github.com/apache/hudi/pull/1672#issuecomment-634855005 @lamber-ken Seems like it'll be more appropriate if we incorporate all supported metric reporter configs in the new "Metrics" section. How about posting this as a blog first?

[GitHub] [hudi] xushiyan commented on pull request #1672: [HUDI-836] [BLOG] Monitor Hudi metrics with Datadog

2020-05-27 Thread GitBox
xushiyan commented on pull request #1672: URL: https://github.com/apache/hudi/pull/1672#issuecomment-634846869 @lamber-ken Sounds good... let me see if I need to reword some part accordingly.. This is an automated message

[GitHub] [hudi] lamber-ken commented on pull request #1672: [HUDI-836] [BLOG] Monitor Hudi metrics with Datadog

2020-05-27 Thread GitBox
lamber-ken commented on pull request #1672: URL: https://github.com/apache/hudi/pull/1672#issuecomment-634841513 hi @xushiyan, it's nice to add it here. ![image](https://user-images.githubusercontent.com/20113411/83055820-f5e97100-a086-11ea-9ea3-52b342aca9d4.png)

[GitHub] [hudi] lamber-ken merged pull request #1151: [HUDI-476] Add hudi-examples module

2020-05-27 Thread GitBox
lamber-ken merged pull request #1151: URL: https://github.com/apache/hudi/pull/1151 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[hudi] branch master updated: [HUDI-476]: Add hudi-examples module (#1151)

2020-05-27 Thread lamberken
This is an automated email from the ASF dual-hosted git repository. lamberken pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new bde7a70 [HUDI-476]: Add hudi-examples module

[GitHub] [hudi] prashanthpdesai edited a comment on issue #1653: [SUPPORT]: Hudi Deltastreamer OffsetoutofRange Exception reading from Kafka topic (12 partitions)

2020-05-27 Thread GitBox
prashanthpdesai edited a comment on issue #1653: URL: https://github.com/apache/hudi/issues/1653#issuecomment-634825141 @bhasudha: The offset issue actually occurred due our internal topic clean up, and last written checkpoint was not able to find the offset value in topic . the issue has

[GitHub] [hudi] prashanthpdesai commented on issue #1653: [SUPPORT]: Hudi Deltastreamer OffsetoutofRange Exception reading from Kafka topic (12 partitions)

2020-05-27 Thread GitBox
prashanthpdesai commented on issue #1653: URL: https://github.com/apache/hudi/issues/1653#issuecomment-634825141 @bhasudha: The offset issue actually occurred due our internal topic clean up, and last written checkpoint was not able to find that in topic so we are able to resolve the

[GitHub] [hudi] garyli1019 commented on a change in pull request #1652: [HUDI-918] Fix kafkaOffsetGen can not read kafka data bug

2020-05-27 Thread GitBox
garyli1019 commented on a change in pull request #1652: URL: https://github.com/apache/hudi/pull/1652#discussion_r431267636 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/KafkaOffsetGen.java ## @@ -207,6 +208,11 @@ public

[GitHub] [hudi] xushiyan commented on pull request #1672: [HUDI-836] [BLOG] Monitor Hudi metrics with Datadog

2020-05-27 Thread GitBox
xushiyan commented on pull request #1672: URL: https://github.com/apache/hudi/pull/1672#issuecomment-634758031 @vinothchandar @yanghua FYA. Thanks. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] xushiyan opened a new pull request #1672: [HUDI-836] [BLOG] Monitor Hudi metrics with Datadog

2020-05-27 Thread GitBox
xushiyan opened a new pull request #1672: URL: https://github.com/apache/hudi/pull/1672 ## Committer checklist - [ ] Has a corresponding JIRA in PR title & commit - [ ] Commit message is descriptive of the change - [ ] CI is green - [ ] Necessary doc

[GitHub] [hudi] codecov-commenter edited a comment on pull request #1151: [HUDI-476] Add hudi-examples module

2020-05-27 Thread GitBox
codecov-commenter edited a comment on pull request #1151: URL: https://github.com/apache/hudi/pull/1151#issuecomment-634017621 # [Codecov](https://codecov.io/gh/apache/hudi/pull/1151?src=pr=h1) Report > Merging [#1151](https://codecov.io/gh/apache/hudi/pull/1151?src=pr=desc) into

[jira] [Updated] (HUDI-305) Presto MOR "_rt" queries only reads base parquet file

2020-05-27 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-305: - Fix Version/s: (was: 0.5.3) 0.6.0 > Presto MOR "_rt" queries only

[jira] [Updated] (HUDI-907) Test Presto mor query support changes in HDFS Env

2020-05-27 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-907: - Fix Version/s: (was: 0.5.3) 0.6.0 > Test Presto mor query support

[hudi] branch asf-site updated: Travis CI build asf-site

2020-05-27 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new d2d133a Travis CI build asf-site d2d133a is

[GitHub] [hudi] leesf merged pull request #1668: [HUDI-804] Add Azure support to doc

2020-05-27 Thread GitBox
leesf merged pull request #1668: URL: https://github.com/apache/hudi/pull/1668 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[hudi] branch asf-site updated: [HUDI-804] Add Azure support to doc (#1668)

2020-05-27 Thread leesf
This is an automated email from the ASF dual-hosted git repository. leesf pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new b4f9ddc [HUDI-804] Add Azure support to doc

[GitHub] [hudi] wangxianghu commented on a change in pull request #1652: [HUDI-918] Fix kafkaOffsetGen can not read kafka data bug

2020-05-27 Thread GitBox
wangxianghu commented on a change in pull request #1652: URL: https://github.com/apache/hudi/pull/1652#discussion_r431174795 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/KafkaOffsetGen.java ## @@ -207,6 +208,11 @@ public

[GitHub] [hudi] wangxianghu commented on a change in pull request #1652: [HUDI-918] Fix kafkaOffsetGen can not read kafka data bug

2020-05-27 Thread GitBox
wangxianghu commented on a change in pull request #1652: URL: https://github.com/apache/hudi/pull/1652#discussion_r431174795 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/KafkaOffsetGen.java ## @@ -207,6 +208,11 @@ public

[GitHub] [hudi] codecov-commenter edited a comment on pull request #1151: [HUDI-476] Add hudi-examples module

2020-05-27 Thread GitBox
codecov-commenter edited a comment on pull request #1151: URL: https://github.com/apache/hudi/pull/1151#issuecomment-634017621 # [Codecov](https://codecov.io/gh/apache/hudi/pull/1151?src=pr=h1) Report > Merging [#1151](https://codecov.io/gh/apache/hudi/pull/1151?src=pr=desc) into

[GitHub] [hudi] lamber-ken commented on pull request #1151: [HUDI-476] Add hudi-examples module

2020-05-27 Thread GitBox
lamber-ken commented on pull request #1151: URL: https://github.com/apache/hudi/pull/1151#issuecomment-634637408 @vinothchandar I test these examples locally and yarn-cluster mode, worked fine, will merging. This is an

[GitHub] [hudi] lamber-ken commented on pull request #1469: [HUDI-686] Implement BloomIndexV2 that does not depend on memory caching

2020-05-27 Thread GitBox
lamber-ken commented on pull request #1469: URL: https://github.com/apache/hudi/pull/1469#issuecomment-634566313 Fixing conflicts, wait This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hudi] lamber-ken commented on pull request #1668: [HUDI-804] Add Azure support to doc

2020-05-27 Thread GitBox
lamber-ken commented on pull request #1668: URL: https://github.com/apache/hudi/pull/1668#issuecomment-634519545 thanks @garyli1019, take a final pass @leesf This is an automated message from the Apache Git Service. To

[jira] [Closed] (HUDI-811) Restructure test packages

2020-05-27 Thread vinoyang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang closed HUDI-811. - Resolution: Done Done via master branch: 03f136361a5fed594855992ab10bee8bb5060c5b > Restructure test packages >

[GitHub] [hudi] yanghua merged pull request #1644: [HUDI-811] Restructure test packages in hudi-common

2020-05-27 Thread GitBox
yanghua merged pull request #1644: URL: https://github.com/apache/hudi/pull/1644 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[hudi] branch master updated: [HUDI-811] Restructure test packages in hudi-common (#1644)

2020-05-27 Thread vinoyang
This is an automated email from the ASF dual-hosted git repository. vinoyang pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 03f1363 [HUDI-811] Restructure test packages

[GitHub] [hudi] yanghua commented on a change in pull request #1644: [HUDI-811] Restructure test packages in hudi-common

2020-05-27 Thread GitBox
yanghua commented on a change in pull request #1644: URL: https://github.com/apache/hudi/pull/1644#discussion_r430943683 ## File path: hudi-common/src/test/java/org/apache/hudi/common/fs/inline/TestInLineFileSystemHFileInLining.java ## @@ -40,18 +42,18 @@ import

  1   2   >