[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-24 Thread rajeshbalamohan
Github user rajeshbalamohan commented on the pull request: https://github.com/apache/spark/pull/12319#issuecomment-214132553 Thanks @liancheng , @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12319 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-22 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12319#issuecomment-213669972 Thanks - merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12319#issuecomment-213662852 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12319#issuecomment-213662854 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12319#issuecomment-213662817 **[Test build #56770 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56770/consoleFull)** for PR 12319 at commit [`d6bc52d`](https://g

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12319#issuecomment-213662708 **[Test build #2859 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2859/consoleFull)** for PR 12319 at commit [`d6bc52d`](https://

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12319#issuecomment-213659831 **[Test build #2859 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2859/consoleFull)** for PR 12319 at commit [`d6bc52d`](https://g

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12319#issuecomment-213659743 **[Test build #56770 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56770/consoleFull)** for PR 12319 at commit [`d6bc52d`](https://gi

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-22 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/12319#issuecomment-213659267 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-22 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/12319#issuecomment-213658964 add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-22 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/12319#issuecomment-213658842 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-20 Thread rajeshbalamohan
Github user rajeshbalamohan commented on the pull request: https://github.com/apache/spark/pull/12319#issuecomment-212766510 Thanks for the review @liancheng Latest commit addresses the review comments. Changes are as follows - Moved OrcRecordReader changes to SparkOrcNewRecord

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-20 Thread rajeshbalamohan
Github user rajeshbalamohan commented on the pull request: https://github.com/apache/spark/pull/12319#issuecomment-212393980 Thanks for the review @liancheng . Should i create separate PR for OrcRecordReader in https://github.com/pwendell/hive/tree/release-1.2.1-spark providing refere

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-20 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/12319#issuecomment-212323419 @rajeshbalamohan Sorry for the late review and thanks for working on this! My major concern is that the newly added `OrcRecordReader` should be live in spark-hive rat

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-20 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/12319#discussion_r60368033 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala --- @@ -337,10 +340,12 @@ private[orc] case class OrcTableScan(

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-20 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/12319#discussion_r60367904 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala --- @@ -160,19 +160,22 @@ private[sql] class DefaultSource

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-20 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/12319#discussion_r60365888 --- Diff: sql/core/pom.xml --- @@ -128,6 +128,10 @@ xbean-asm5-shaded test + + org.spark-project.hive +

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-20 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/12319#discussion_r60365709 --- Diff: sql/core/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcRecordReader.java --- @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-20 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/12319#discussion_r60365527 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala --- @@ -160,19 +160,22 @@ private[sql] class DefaultSource

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-20 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/12319#discussion_r60365406 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala --- @@ -160,19 +160,22 @@ private[sql] class DefaultSource

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-12 Thread rajeshbalamohan
Github user rajeshbalamohan commented on a diff in the pull request: https://github.com/apache/spark/pull/12319#discussion_r59328799 --- Diff: sql/core/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcRecordReader.java --- @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache So

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-11 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12319#issuecomment-208725255 cc @liancheng for review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-11 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12319#discussion_r59325255 --- Diff: sql/core/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcRecordReader.java --- @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-11 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12319#issuecomment-208724505 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-14551][SQL] Reduce number of NameNode c...

2016-04-11 Thread rajeshbalamohan
Github user rajeshbalamohan commented on the pull request: https://github.com/apache/spark/pull/12319#issuecomment-208694717 Sure @rxin. makes sense. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does no