[GitHub] spark issue #14659: [SPARK-16757] Set up Spark caller context to HDFS

2016-08-22 Thread Sherry302
Github user Sherry302 commented on the issue: https://github.com/apache/spark/pull/14659 Thanks a lot for adding me as “contributor” in Hadoop :) @steveloughran @cnauroth --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #14692: [SPARK-17115] [SQL] decrease the threshold when split ex...

2016-08-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14692 thanks, merging to master/2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #14659: [SPARK-16757] Set up Spark caller context to HDFS

2016-08-22 Thread Sherry302
Github user Sherry302 commented on the issue: https://github.com/apache/spark/pull/14659 Hi, @steveloughran Thanks a lot for the comments. In the audit log, if users set some configuration in spark-defaults.conf like `spark.eventLog.dir hdfs://localhost:9000/spark-history`,

[GitHub] spark pull request #14744: [SPARKR][SPARKSUBMIT] Allow to set sparkr shell c...

2016-08-22 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/14744#discussion_r75633858 --- Diff: docs/configuration.md --- @@ -1752,6 +1752,15 @@ showDF(properties, numRows = 200, truncate = FALSE) Executable for executing R scripts

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix an issue in QuantileDiscretizer

2016-08-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14747 No, my question is whether it remains sorted after `.distinct`. We'd need that, but I'm not sure it's guaranteed. It's easy to add `.sorted` to guarantee it. --- If your project is set up for it,

[GitHub] spark pull request #14738: [SPARK-17090][MINOR][ML]Add expert param support ...

2016-08-22 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/14738#discussion_r75633532 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/shared/sharedParams.scala --- @@ -397,13 +397,13 @@ private[ml] trait HasAggregationDepth

[GitHub] spark issue #12889: [SPARK-15113][PySpark][ML] Add missing num features num ...

2016-08-22 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/12889 jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #12889: [SPARK-15113][PySpark][ML] Add missing num featur...

2016-08-22 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/12889#discussion_r75633544 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -788,6 +788,8 @@ class

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix an issue in QuantileDiscretizer

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14747 **[Test build #64184 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64184/consoleFull)** for PR 14747 at commit

[GitHub] spark pull request #14155: [SPARK-16498][SQL] move hive hack for data source...

2016-08-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14155#discussion_r75633327 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -200,22 +375,77 @@ private[spark] class

[GitHub] spark pull request #14738: [SPARK-17090][MINOR][ML]Add expert param support ...

2016-08-22 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/14738#discussion_r75633356 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/shared/SharedParamsCodeGen.scala --- @@ -167,11 +173,11 @@ private[shared] object

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix an issue in QuantileDiscretizer

2016-08-22 Thread VinceShieh
Github user VinceShieh commented on the issue: https://github.com/apache/spark/pull/14747 yes, the output from approxQuantile is a sorted array. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #14747: [SPARK-17086][MLLIB] Fix an issue in QuantileDiscretizer

2016-08-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14747 Jenkins add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14747: [SPARK-17086][MLLIB] Fix an issue in QuantileDiscretizer

2016-08-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14747 Hm, does `.distinct` guarantee that the ordering is preserved? It looks like it generally happens to. I guess we could add `.sorted` to make sure; it wouldn't be expensive, especially if it's

[GitHub] spark issue #14747: [SPARK-17086][MLLIB] Fix an issue in QuantileDiscretizer

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14747 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14683: [SPARK-16968]Document additional options in jdbc ...

2016-08-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14683 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #11956: [SPARK-14098][SQL] Generate Java code that gets a float/...

2016-08-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/11956 @davies Would it be possible to share your opinions regarding these design questions among us? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #14723: [SQL][WIP][Test] Supports object-based aggregatio...

2016-08-22 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14723#discussion_r75632667 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/interfaces.scala --- @@ -389,3 +389,126 @@ abstract class

[GitHub] spark issue #13704: [SPARK-15985][SQL] Eliminate redundant cast from an arra...

2016-08-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13704 @liancheng Could you please review this? cc: @cloud-fan , we are waiting for @liancheng 's review for a long time. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #14700: [SPARK-17127]Make unaligned access in unsafe avai...

2016-08-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14700 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14738: [SPARK-17090][MINOR][ML]Add expert param support to Shar...

2016-08-22 Thread hqzizania
Github user hqzizania commented on the issue: https://github.com/apache/spark/pull/14738 It is linked to SPARK-17090 now. @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14747: [SPARK-17086] Fix an issue in QuantileDiscretizer

2016-08-22 Thread VinceShieh
GitHub user VinceShieh opened a pull request: https://github.com/apache/spark/pull/14747 [SPARK-17086] Fix an issue in QuantileDiscretizer ## What changes were proposed in this pull request? In cases when QuantileDiscretizerSuite is called upon a numeric array with

[GitHub] spark issue #14683: [SPARK-16968]Document additional options in jdbc Writer

2016-08-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14683 Merged to master as well --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14700: [SPARK-17127]Make unaligned access in unsafe available f...

2016-08-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14700 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #14625: [SPARK-17045] [SQL] Build/move Join-related test cases i...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14625 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64181/ Test FAILed. ---

[GitHub] spark issue #14625: [SPARK-17045] [SQL] Build/move Join-related test cases i...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14625 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14738: [MINOR][ML]Add expert param support to SharedParamsCodeG...

2016-08-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14738 OK, let's link this to the original JIRA then. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14625: [SPARK-17045] [SQL] Build/move Join-related test cases i...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14625 **[Test build #64181 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64181/consoleFull)** for PR 14625 at commit

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-22 Thread winningsix
Github user winningsix commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r75631172 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoConf.scala --- @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #14659: [SPARK-16757] Set up Spark caller context to HDFS

2016-08-22 Thread Sherry302
Github user Sherry302 commented on the issue: https://github.com/apache/spark/pull/14659 Hi, @cnauroth Thank you very much for the review and suggestion. I have removed the spaces in the value of the caller context, and prepended "Spark" instead (refer to the commit

[GitHub] spark issue #14537: [SPARK-16948][SQL] Querying empty partitioned orc tables...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14537 **[Test build #64183 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64183/consoleFull)** for PR 14537 at commit

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-08-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/13775 ping @yhuai Any chance you can review this? Thank you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #14388: [SPARK-16362][SQL] Support ArrayType and StructType in v...

2016-08-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14388 @maver1ck Any results about the test? Thank you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14642: [SPARK-17056][Core] Fix a wrong assert regarding unroll ...

2016-08-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14642 ping @cloud-fan @JoshRosen since you have modified this file, do you think this change make sense? Thank you. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #14729: [SPARK-17167] [SQL] Issue Exceptions when Analyze Table ...

2016-08-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14729 @hvanhovell Not related to this PR. But I would like to ask that do we need to support temporary tables in `ANALYZE TABLE`? Because temporary tables (and views) are actually represented by logical

[GitHub] spark issue #14746: [SPARK-17180] [SQL] Fix View Resolution Order in ALTER V...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14746 **[Test build #64182 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64182/consoleFull)** for PR 14746 at commit

[GitHub] spark pull request #14746: [SPARK-17180] [SQL] Fix View Resolution Order in ...

2016-08-22 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/14746 [SPARK-17180] [SQL] Fix View Resolution Order in ALTER VIEW AS SELECT ### What changes were proposed in this pull request? In the current master branch, when users do not specify the

[GitHub] spark issue #14625: [SPARK-17045] [SQL] Build/move Join-related test cases i...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14625 **[Test build #64181 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64181/consoleFull)** for PR 14625 at commit

[GitHub] spark issue #14745: [SPARK-16896][SQL] Handle duplicated field names in head...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14745 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14745: [SPARK-16896][SQL] Handle duplicated field names in head...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14745 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64178/ Test PASSed. ---

[GitHub] spark issue #14521: [SPARK-16935] [SQL] Verification of Function-related Ext...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14521 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64177/ Test PASSed. ---

[GitHub] spark issue #14745: [SPARK-16896][SQL] Handle duplicated field names in head...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14745 **[Test build #64178 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64178/consoleFull)** for PR 14745 at commit

[GitHub] spark issue #14521: [SPARK-16935] [SQL] Verification of Function-related Ext...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14521 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14521: [SPARK-16935] [SQL] Verification of Function-related Ext...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14521 **[Test build #64177 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64177/consoleFull)** for PR 14521 at commit

[GitHub] spark issue #14531: [SPARK-16943] [SPARK-16942] [SQL] Fix multiple bugs in C...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14531 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14531: [SPARK-16943] [SPARK-16942] [SQL] Fix multiple bugs in C...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14531 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64176/ Test PASSed. ---

[GitHub] spark issue #14531: [SPARK-16943] [SPARK-16942] [SQL] Fix multiple bugs in C...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14531 **[Test build #64176 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64176/consoleFull)** for PR 14531 at commit

[GitHub] spark issue #14572: [SPARK-16552] [FOLLOW-UP] [SQL] Store the Inferred Schem...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14572 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14723: [SQL][WIP][Test] Supports object-based aggregation funct...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14723 **[Test build #64180 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64180/consoleFull)** for PR 14723 at commit

[GitHub] spark issue #14723: [SQL][WIP][Test] Supports object-based aggregation funct...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14723 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14723: [SQL][WIP][Test] Supports object-based aggregation funct...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14723 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64180/ Test FAILed. ---

[GitHub] spark issue #14572: [SPARK-16552] [FOLLOW-UP] [SQL] Store the Inferred Schem...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14572 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64175/ Test PASSed. ---

[GitHub] spark issue #14572: [SPARK-16552] [FOLLOW-UP] [SQL] Store the Inferred Schem...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14572 **[Test build #64175 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64175/consoleFull)** for PR 14572 at commit

[GitHub] spark issue #14723: [SQL][WIP][Test] Supports object-based aggregation funct...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14723 **[Test build #64180 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64180/consoleFull)** for PR 14723 at commit

[GitHub] spark issue #14723: [SQL][WIP][Test] Supports object-based aggregation funct...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14723 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14723: [SQL][WIP][Test] Supports object-based aggregation funct...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14723 **[Test build #64179 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64179/consoleFull)** for PR 14723 at commit

[GitHub] spark issue #14723: [SQL][WIP][Test] Supports object-based aggregation funct...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14723 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64179/ Test FAILed. ---

[GitHub] spark issue #14723: [SQL][WIP][Test] Supports object-based aggregation funct...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14723 **[Test build #64179 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64179/consoleFull)** for PR 14723 at commit

[GitHub] spark pull request #14723: [SQL][WIP][Test] Supports object-based aggregatio...

2016-08-22 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/14723#discussion_r75622311 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/SortBasedAggregationIterator.scala --- @@ -90,6 +98,21 @@ class

[GitHub] spark pull request #14723: [SQL][WIP][Test] Supports object-based aggregatio...

2016-08-22 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/14723#discussion_r75622250 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/interfaces.scala --- @@ -389,3 +389,89 @@ abstract class

[GitHub] spark pull request #14723: [SQL][WIP][Test] Supports object-based aggregatio...

2016-08-22 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/14723#discussion_r75622002 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/AggregateWithObjectAggregateBufferSuite.scala --- @@ -0,0 +1,156 @@ +/* + * Licensed to the

<    3   4   5   6   7   8