[GitHub] [incubator-hudi] cdmikechen commented on issue #774: Matching question of the version in Spark and Hive2

2019-07-30 Thread GitBox
cdmikechen commented on issue #774: Matching question of the version in Spark and Hive2 URL: https://github.com/apache/incubator-hudi/issues/774#issuecomment-516722052 Another things. I found that Hive2 use log4j2 and spark use log4j. If I commit a spark task like ```bash spark-subm

[GitHub] [incubator-hudi] cdmikechen edited a comment on issue #774: Matching question of the version in Spark and Hive2

2019-07-30 Thread GitBox
cdmikechen edited a comment on issue #774: Matching question of the version in Spark and Hive2 URL: https://github.com/apache/incubator-hudi/issues/774#issuecomment-516719772 @vinothchandar I think we should try to use Spark's basic functions as the standard. It means we use spark-hive l

[GitHub] [incubator-hudi] cdmikechen commented on issue #774: Matching question of the version in Spark and Hive2

2019-07-30 Thread GitBox
cdmikechen commented on issue #774: Matching question of the version in Spark and Hive2 URL: https://github.com/apache/incubator-hudi/issues/774#issuecomment-516719772 I think we should try to use Spark's basic functions as the standard. It means we use spark-hive libs when building, but

[GitHub] [incubator-hudi] arw357 commented on issue #590: Failed to APPEND_FILE /HUDI_MOR/2019/03/01/.abca886a-e213-4199-b571-475919d34fe5_20190301123358.log.1 for DFSClient_NONMAPREDUCE_-263902482_1

2019-07-30 Thread GitBox
arw357 commented on issue #590: Failed to APPEND_FILE /HUDI_MOR/2019/03/01/.abca886a-e213-4199-b571-475919d34fe5_20190301123358.log.1 for DFSClient_NONMAPREDUCE_-263902482_1 on 172.29.0.10 because lease recovery is in progress URL: https://github.com/apache/incubator-hudi/issues/590#issuecomment

[GitHub] [incubator-hudi] eisig commented on a change in pull request #805: fix DeltaStreamer writeConfig

2019-07-30 Thread GitBox
eisig commented on a change in pull request #805: fix DeltaStreamer writeConfig URL: https://github.com/apache/incubator-hudi/pull/805#discussion_r309046638 ## File path: hoodie-utilities/src/main/java/com/uber/hoodie/utilities/deltastreamer/DeltaSync.java ## @@ -453,14 +4

[GitHub] [incubator-hudi] eisig commented on a change in pull request #805: fix DeltaStreamer writeConfig

2019-07-30 Thread GitBox
eisig commented on a change in pull request #805: fix DeltaStreamer writeConfig URL: https://github.com/apache/incubator-hudi/pull/805#discussion_r309046638 ## File path: hoodie-utilities/src/main/java/com/uber/hoodie/utilities/deltastreamer/DeltaSync.java ## @@ -453,14 +4

[incubator-hudi] branch master updated: Generate Source Jars for bundle packages (#810)

2019-07-30 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new 6e0ff3a Generate Source Jars for bundle

[GitHub] [incubator-hudi] vinothchandar merged pull request #810: HUDI-176: Generate Source Jars for bundle packages

2019-07-30 Thread GitBox
vinothchandar merged pull request #810: HUDI-176: Generate Source Jars for bundle packages URL: https://github.com/apache/incubator-hudi/pull/810 This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [incubator-hudi] vinothchandar commented on issue #809: HUDI-177 : Adding support for hive registration using metastore along with JDBC

2019-07-30 Thread GitBox
vinothchandar commented on issue #809: HUDI-177 : Adding support for hive registration using metastore along with JDBC URL: https://github.com/apache/incubator-hudi/pull/809#issuecomment-516653416 @n3nash also, can you comment on what dependencies we can shed if we simply supported the one

[GitHub] [incubator-hudi] vinothchandar commented on issue #809: HUDI-177 : Adding support for hive registration using metastore along with JDBC

2019-07-30 Thread GitBox
vinothchandar commented on issue #809: HUDI-177 : Adding support for hive registration using metastore along with JDBC URL: https://github.com/apache/incubator-hudi/pull/809#issuecomment-516652593 this is HUDI-180 no? can you please add context into why we are doing and the tradeoffs with

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #805: fix DeltaStreamer writeConfig

2019-07-30 Thread GitBox
vinothchandar commented on a change in pull request #805: fix DeltaStreamer writeConfig URL: https://github.com/apache/incubator-hudi/pull/805#discussion_r308999210 ## File path: hoodie-utilities/src/main/java/com/uber/hoodie/utilities/deltastreamer/DeltaSync.java ## @@ -

[GitHub] [incubator-hudi] bvaradar merged pull request #799: HUDI-92 : Making deltastreamer with DistributedTestSource also run lo…

2019-07-30 Thread GitBox
bvaradar merged pull request #799: HUDI-92 : Making deltastreamer with DistributedTestSource also run lo… URL: https://github.com/apache/incubator-hudi/pull/799 This is an automated message from the Apache Git Service. To re

[incubator-hudi] branch master updated: HUDI-92 : Making deltastreamer with DistributedTestSource also run locally

2019-07-30 Thread vbalaji
This is an automated email from the ASF dual-hosted git repository. vbalaji pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new e20b77b HUDI-92 : Making deltastreamer

[GitHub] [incubator-hudi] n3nash opened a new pull request #811: Adding HoodieCombineHiveInputFormat for COW tables

2019-07-30 Thread GitBox
n3nash opened a new pull request #811: Adding HoodieCombineHiveInputFormat for COW tables URL: https://github.com/apache/incubator-hudi/pull/811 @vinothchandar @bvaradar I'm opening this PR since we want to ensure we have all the existing classes present in OSS when we rename the packages.

[GitHub] [incubator-hudi] vinothchandar commented on issue #799: HUDI-92 : Making deltastreamer with DistributedTestSource also run lo…

2019-07-30 Thread GitBox
vinothchandar commented on issue #799: HUDI-92 : Making deltastreamer with DistributedTestSource also run lo… URL: https://github.com/apache/incubator-hudi/pull/799#issuecomment-516612426 @bvaradar can you please review this and merge ---

[GitHub] [incubator-hudi] bvaradar opened a new pull request #810: HUDI-176: Generate Source Jars for bundle packages

2019-07-30 Thread GitBox
bvaradar opened a new pull request #810: HUDI-176: Generate Source Jars for bundle packages URL: https://github.com/apache/incubator-hudi/pull/810 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [incubator-hudi] n3nash opened a new pull request #809: Adding support for hive registration using metastore along with JDBC

2019-07-30 Thread GitBox
n3nash opened a new pull request #809: Adding support for hive registration using metastore along with JDBC URL: https://github.com/apache/incubator-hudi/pull/809 @bvaradar @vinothchandar PTAL. Documentation to follow. Th

[GitHub] [incubator-hudi] vinothchandar commented on issue #789: Demo : Unexpected result in some queries

2019-07-30 Thread GitBox
vinothchandar commented on issue #789: Demo : Unexpected result in some queries URL: https://github.com/apache/incubator-hudi/issues/789#issuecomment-516522986 Weird that it is intermittent. @bhasudha lets meet and take a stab at this sometime.. this also blocks #751 and related efforts ,

[GitHub] [incubator-hudi] n3nash commented on issue #801: How to customize schema

2019-07-30 Thread GitBox
n3nash commented on issue #801: How to customize schema URL: https://github.com/apache/incubator-hudi/issues/801#issuecomment-516512834 There is an effort to upgrade Hudi to using Spark 2.4.x. here is the PR : https://github.com/apache/incubator-hudi/pull/638. You should be able to use 2.4

[GitHub] [incubator-hudi] n3nash commented on issue #590: Failed to APPEND_FILE /HUDI_MOR/2019/03/01/.abca886a-e213-4199-b571-475919d34fe5_20190301123358.log.1 for DFSClient_NONMAPREDUCE_-263902482_1

2019-07-30 Thread GitBox
n3nash commented on issue #590: Failed to APPEND_FILE /HUDI_MOR/2019/03/01/.abca886a-e213-4199-b571-475919d34fe5_20190301123358.log.1 for DFSClient_NONMAPREDUCE_-263902482_1 on 172.29.0.10 because lease recovery is in progress URL: https://github.com/apache/incubator-hudi/issues/590#issuecomment

[GitHub] [incubator-hudi] v3nkatesh commented on a change in pull request #802: Cache RDD to avoid recomputing data ingestion and hbase puts actions

2019-07-30 Thread GitBox
v3nkatesh commented on a change in pull request #802: Cache RDD to avoid recomputing data ingestion and hbase puts actions URL: https://github.com/apache/incubator-hudi/pull/802#discussion_r308790009 ## File path: hoodie-client/src/test/java/com/uber/hoodie/index/TestHbaseIndex.jav

[GitHub] [incubator-hudi] arw357 edited a comment on issue #590: Failed to APPEND_FILE /HUDI_MOR/2019/03/01/.abca886a-e213-4199-b571-475919d34fe5_20190301123358.log.1 for DFSClient_NONMAPREDUCE_-26390

2019-07-30 Thread GitBox
arw357 edited a comment on issue #590: Failed to APPEND_FILE /HUDI_MOR/2019/03/01/.abca886a-e213-4199-b571-475919d34fe5_20190301123358.log.1 for DFSClient_NONMAPREDUCE_-263902482_1 on 172.29.0.10 because lease recovery is in progress URL: https://github.com/apache/incubator-hudi/issues/590#issue

[GitHub] [incubator-hudi] arw357 edited a comment on issue #590: Failed to APPEND_FILE /HUDI_MOR/2019/03/01/.abca886a-e213-4199-b571-475919d34fe5_20190301123358.log.1 for DFSClient_NONMAPREDUCE_-26390

2019-07-30 Thread GitBox
arw357 edited a comment on issue #590: Failed to APPEND_FILE /HUDI_MOR/2019/03/01/.abca886a-e213-4199-b571-475919d34fe5_20190301123358.log.1 for DFSClient_NONMAPREDUCE_-263902482_1 on 172.29.0.10 because lease recovery is in progress URL: https://github.com/apache/incubator-hudi/issues/590#issue

[GitHub] [incubator-hudi] arw357 edited a comment on issue #590: Failed to APPEND_FILE /HUDI_MOR/2019/03/01/.abca886a-e213-4199-b571-475919d34fe5_20190301123358.log.1 for DFSClient_NONMAPREDUCE_-26390

2019-07-30 Thread GitBox
arw357 edited a comment on issue #590: Failed to APPEND_FILE /HUDI_MOR/2019/03/01/.abca886a-e213-4199-b571-475919d34fe5_20190301123358.log.1 for DFSClient_NONMAPREDUCE_-263902482_1 on 172.29.0.10 because lease recovery is in progress URL: https://github.com/apache/incubator-hudi/issues/590#issue

[GitHub] [incubator-hudi] arw357 commented on issue #590: Failed to APPEND_FILE /HUDI_MOR/2019/03/01/.abca886a-e213-4199-b571-475919d34fe5_20190301123358.log.1 for DFSClient_NONMAPREDUCE_-263902482_1

2019-07-30 Thread GitBox
arw357 commented on issue #590: Failed to APPEND_FILE /HUDI_MOR/2019/03/01/.abca886a-e213-4199-b571-475919d34fe5_20190301123358.log.1 for DFSClient_NONMAPREDUCE_-263902482_1 on 172.29.0.10 because lease recovery is in progress URL: https://github.com/apache/incubator-hudi/issues/590#issuecomment

[GitHub] [incubator-hudi] vinothchandar merged pull request #808: [HUDI-181] Fix the Bold markdown grammar issue of README file

2019-07-30 Thread GitBox
vinothchandar merged pull request #808: [HUDI-181] Fix the Bold markdown grammar issue of README file URL: https://github.com/apache/incubator-hudi/pull/808 This is an automated message from the Apache Git Service. To respon

[incubator-hudi] branch master updated: [HUDI-181] Fix the Bold markdown grammar issue of README file (#808)

2019-07-30 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new 68464c7 [HUDI-181] Fix the Bold markdow