[GitHub] spark pull request #17062: [SPARK-17495] [SQL] Support date, timestamp and i...

2017-02-27 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/17062#discussion_r103357588 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/HashExpressionsSuite.scala --- @@ -169,6 +171,96 @@ class

[GitHub] spark pull request #17062: [SPARK-17495] [SQL] Support date, timestamp and i...

2017-02-27 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/17062#discussion_r103300592 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/HashExpressionsSuite.scala --- @@ -169,6 +171,96 @@ class

[GitHub] spark pull request #16715: [Spark-18080][ML][PYTHON] Python API & Examples f...

2017-02-27 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/16715#discussion_r103357342 --- Diff: python/pyspark/ml/feature.py --- @@ -120,6 +122,196 @@ def getThreshold(self): return self.getOrDefault(self.threshold)

[GitHub] spark pull request #17062: [SPARK-17495] [SQL] Support date, timestamp and i...

2017-02-27 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/17062#discussion_r103281696 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/HashExpressionsSuite.scala --- @@ -169,6 +171,96 @@ class

[GitHub] spark pull request #17062: [SPARK-17495] [SQL] Support date, timestamp and i...

2017-02-27 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/17062#discussion_r103300013 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/HashExpressionsSuite.scala --- @@ -169,6 +171,96 @@ class

[GitHub] spark pull request #17062: [SPARK-17495] [SQL] Support date, timestamp and i...

2017-02-27 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/17062#discussion_r103357472 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/HashExpressionsSuite.scala --- @@ -169,6 +171,96 @@ class

[GitHub] spark pull request #17062: [SPARK-17495] [SQL] Support date, timestamp and i...

2017-02-27 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/17062#discussion_r103300293 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/HashExpressionsSuite.scala --- @@ -169,6 +171,96 @@ class

[GitHub] spark pull request #17079: [SPARK-19748][SQL]refresh function has a wrong or...

2017-02-27 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/17079#discussion_r103357633 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileIndexSuite.scala --- @@ -178,6 +178,34 @@ class FileIndexSuite extends

[GitHub] spark issue #16774: [SPARK-19357][ML] Adding parallel model evaluation in ML...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16774 **[Test build #73545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73545/testReport)** for PR 16774 at commit

[GitHub] spark issue #17080: [SPARK-19739][CORE] propagate S3 session token to cluser

2017-02-27 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/17080 @steveloughran IMHO, there is no need to use `org.apache.hadoop.fs.s3a.Constants` and `com.amazonaws.SDKGlobalConfiguration`, otherwise we will import `hadoop-aws` and `aws-java-sdk-core` into

[GitHub] spark issue #17079: [SPARK-19748][SQL]refresh function has a wrong order to ...

2017-02-27 Thread windpiger
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/17079 there is no related test case for InMemoryFileIndex with FileStatusCache. When I do this [PR](https://github.com/apache/spark/pull/17081), and add a fileStatusCache in DataSource, I found this

[GitHub] spark issue #17012: [SPARK-19677][SS] Renaming a file atop an existing one s...

2017-02-27 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/17012 What is the proposed semantics from this PR now ? - If file exists, ignore. - If file does not exist, try to rename - if fails, throw exception. Is this right ? If yes, the PR

[GitHub] spark issue #17070: [SPARK-19721][SS] Good error message for version mismatc...

2017-02-27 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/17070 /cc @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/17090#discussion_r103351299 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -248,18 +248,18 @@ class ALSModel private[ml] ( @Since("1.3.0")

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/17090#discussion_r103352432 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -285,6 +285,43 @@ class ALSModel private[ml] (

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/17090#discussion_r103350750 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -285,6 +285,43 @@ class ALSModel private[ml] (

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/17090#discussion_r103353799 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/TopByKeyAggregator.scala --- @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/17090#discussion_r103354132 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -285,6 +286,55 @@ class ALSModel private[ml] (

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/17090#discussion_r103353184 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -285,6 +285,43 @@ class ALSModel private[ml] (

[GitHub] spark pull request #14273: [SPARK-9140] [ML] Replace TimeTracker by MultiSto...

2017-02-27 Thread MechCoder
Github user MechCoder closed the pull request at: https://github.com/apache/spark/pull/14273 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #16959: [SPARK-19631][CORE] OutputCommitCoordinator shoul...

2017-02-27 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16959#discussion_r103354382 --- Diff: core/src/main/scala/org/apache/spark/scheduler/OutputCommitCoordinator.scala --- @@ -111,13 +115,13 @@ private[spark] class

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r103354299 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -40,19 +42,179 @@ abstract class

[GitHub] spark issue #17090: [Spark-19535][ML] RecommendForAllUsers RecommendForAllIt...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17090 **[Test build #73543 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73543/testReport)** for PR 17090 at commit

[GitHub] spark issue #16959: [SPARK-19631][CORE] OutputCommitCoordinator should not a...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16959 **[Test build #73544 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73544/testReport)** for PR 16959 at commit

[GitHub] spark issue #17052: [SPARK-19690][SS] Join a streaming DataFrame with a batc...

2017-02-27 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/17052 working on unit test failure --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r103353840 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -2512,3 +2522,67 @@ object ResolveCreateNamedStruct

[GitHub] spark pull request #17068: [SPARK-19709][SQL] Read empty file with CSV data ...

2017-02-27 Thread wojtek-szymanski
Github user wojtek-szymanski commented on a diff in the pull request: https://github.com/apache/spark/pull/17068#discussion_r103353729 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVInferSchema.scala --- @@ -40,7 +41,19 @@ private[csv] object

[GitHub] spark issue #16826: [SPARK-19540][SQL] Add ability to clone SparkSession whe...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16826 **[Test build #73542 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73542/testReport)** for PR 16826 at commit

[GitHub] spark issue #17091: [SPARK-19757][CORE] Executor with task scheduled could b...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17091 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103352266 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -17,43 +17,60 @@ package

[GitHub] spark pull request #12762: [SPARK-14891][ML] Add schema validation for ALS

2017-02-27 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/12762#discussion_r103352114 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -242,16 +263,19 @@ class ALSModel private[ml] ( }

[GitHub] spark issue #17078: [SPARK-19746][ML] Faster indexing for logistic aggregato...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17078 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17078: [SPARK-19746][ML] Faster indexing for logistic aggregato...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17078 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73537/ Test PASSed. ---

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103307491 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionState.scala --- @@ -146,4 +107,153 @@ private[hive] class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103338672 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SessionStateSuite.scala --- @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103305709 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -90,110 +208,29 @@ private[sql] class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r10330 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SessionStateSuite.scala --- @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103337066 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala --- @@ -136,6 +139,26 @@ private[sql] class SharedState(val

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103336475 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -65,22 +82,118 @@ private[sql] class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103295794 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -1196,4 +1198,28 @@ class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103347559 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/internal/CatalogSuite.scala --- @@ -493,6 +493,28 @@ class CatalogSuite } }

[GitHub] spark issue #17078: [SPARK-19746][ML] Faster indexing for logistic aggregato...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17078 **[Test build #73537 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73537/testReport)** for PR 17078 at commit

[GitHub] spark pull request #17064: [SPARK-19736][SQL] refreshByPath should clear all...

2017-02-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17064#discussion_r103351494 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -168,15 +168,16 @@ class CacheManager extends Logging {

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103331408 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionState.scala --- @@ -17,89 +17,50 @@ package org.apache.spark.sql.hive

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103336696 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -65,22 +82,118 @@ private[sql] class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103302329 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -65,22 +82,118 @@ private[sql] class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103295639 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -1178,4 +1181,36 @@ class SessionCatalog(

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103307776 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionState.scala --- @@ -146,4 +107,153 @@ private[hive] class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103328320 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSessionStateSuite.scala --- @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103303272 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -90,110 +203,37 @@ private[sql] class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103295692 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -1178,4 +1181,36 @@ class SessionCatalog(

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103306212 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala --- @@ -136,6 +139,26 @@ private[sql] class SharedState(val

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103307299 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala --- @@ -212,3 +247,31 @@ private[sql] class HiveSessionCatalog(

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103329699 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/test/TestHive.scala --- @@ -144,11 +145,37 @@ private[hive] class TestHiveSparkSession(

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103298123 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/ExperimentalMethods.scala --- @@ -46,4 +46,10 @@ class ExperimentalMethods private[sql]() {

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103297916 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -1196,4 +1198,28 @@ class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103307383 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionState.scala --- @@ -146,4 +107,153 @@ private[hive] class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103305545 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -90,110 +203,37 @@ private[sql] class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103307420 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionState.scala --- @@ -146,4 +107,153 @@ private[hive] class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103336676 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -65,22 +82,118 @@ private[sql] class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103298622 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -17,43 +17,60 @@ package

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103308243 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSessionCatalogSuite.scala --- @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103308317 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSessionStateSuite.scala --- @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #17078: [SPARK-19746][ML] Faster indexing for logistic aggregato...

2017-02-27 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17078 Thanks. Merged into master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread sueann
Github user sueann commented on a diff in the pull request: https://github.com/apache/spark/pull/17090#discussion_r103350775 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -248,18 +248,18 @@ class ALSModel private[ml] ( @Since("1.3.0")

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread sueann
Github user sueann commented on a diff in the pull request: https://github.com/apache/spark/pull/17090#discussion_r103350410 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/TopByKeyAggregator.scala --- @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #17091: [SPARK-19757][CORE] Executor with task scheduled ...

2017-02-27 Thread jxiang
GitHub user jxiang opened a pull request: https://github.com/apache/spark/pull/17091 [SPARK-19757][CORE] Executor with task scheduled could be killed due to idleness ## What changes were proposed in this pull request? In makeOffers, put in one synchronization block to check if

[GitHub] spark issue #17079: [SPARK-19748][SQL]refresh function has a wrong order to ...

2017-02-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17079 good catch! Can you show a real example that fails because of this bug? I'm wondering why the existing unit tests didn't expose this bug... --- If your project is set up for it, you can reply to

[GitHub] spark pull request #17078: [SPARK-19746][ML] Faster indexing for logistic ag...

2017-02-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17078 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #17079: [SPARK-19748][SQL]refresh function has a wrong or...

2017-02-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17079#discussion_r103350023 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileIndexSuite.scala --- @@ -178,6 +178,34 @@ class FileIndexSuite extends

[GitHub] spark issue #17090: [Spark-19535][ML] RecommendForAllUsers RecommendForAllIt...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17090 **[Test build #73540 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73540/testReport)** for PR 17090 at commit

[GitHub] spark issue #17089: [SPARK-19756][SQL] drop the table cache after inserting ...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17089 **[Test build #73541 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73541/testReport)** for PR 17089 at commit

[GitHub] spark pull request #16959: [SPARK-19631][CORE] OutputCommitCoordinator shoul...

2017-02-27 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16959#discussion_r103348105 --- Diff: core/src/main/scala/org/apache/spark/scheduler/OutputCommitCoordinator.scala --- @@ -48,25 +48,29 @@ private[spark] class

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/17090#discussion_r103349590 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -248,18 +248,18 @@ class ALSModel private[ml] ( @Since("1.3.0")

[GitHub] spark pull request #17079: [SPARK-19748][SQL]refresh function has a wrong or...

2017-02-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17079#discussion_r103349639 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileIndexSuite.scala --- @@ -178,6 +178,34 @@ class FileIndexSuite extends

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/17090#discussion_r103349429 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/TopByKeyAggregator.scala --- @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #17064: [SPARK-19736][SQL] refreshByPath should clear all...

2017-02-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17064#discussion_r103349345 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -168,15 +168,16 @@ class CacheManager extends Logging {

[GitHub] spark issue #17075: [SPARK-19727][SQL] Fix for round function that modifies ...

2017-02-27 Thread wojtek-szymanski
Github user wojtek-szymanski commented on the issue: https://github.com/apache/spark/pull/17075 I have just started refactoring of `changePrecission` in order to make it immutable. My idea was to change the signature from: `def changePrecision(precision: Int, scale: Int,

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread sueann
Github user sueann commented on a diff in the pull request: https://github.com/apache/spark/pull/17090#discussion_r103349139 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/TopByKeyAggregator.scala --- @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread sueann
GitHub user sueann opened a pull request: https://github.com/apache/spark/pull/17090 [Spark-19535][ML] RecommendForAllUsers RecommendForAllItems for ALS on Dataframe ## What changes were proposed in this pull request? This is a simple implementation of

[GitHub] spark issue #17089: [SPARK-19756][SQL] drop the table cache after inserting ...

2017-02-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17089 cc @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread sueann
Github user sueann commented on a diff in the pull request: https://github.com/apache/spark/pull/17090#discussion_r103349080 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/TopByKeyAggregator.scala --- @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #17089: [SPARK-19756][SQL] drop the table cache after ins...

2017-02-27 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/17089 [SPARK-19756][SQL] drop the table cache after inserting into a data source table ## What changes were proposed in this pull request? When we inserting into a table, we should uncache it

[GitHub] spark issue #17039: [SPARK-19710][SQL][TESTS] Fix ordering of rows in query ...

2017-02-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17039 @hvanhovell Is that possible the SQL queries are used to verify the behavior of ORDER BY? Do you think we should explicitly leave a comment to say SQLQueryTestSuite will not be used for this

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-02-27 Thread thunterdb
Github user thunterdb commented on the issue: https://github.com/apache/spark/pull/15770 Note that any of these formats would cause trouble for a graph with high centrality (lady gaga in the twitter graph). That being said, I do not have a strong opinion as to which option we pick,

[GitHub] spark pull request #17064: [SPARK-19736][SQL] refreshByPath should clear all...

2017-02-27 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17064#discussion_r103348444 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -168,15 +168,16 @@ class CacheManager extends Logging {

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r103347540 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -2332,6 +2337,11 @@ class Analyzer(

[GitHub] spark issue #14273: [SPARK-9140] [ML] Replace TimeTracker by MultiStopwatch

2017-02-27 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/14273 OK apologies @MechCoder for the delay. I guess we can close this issue, and someone else can open up a PR based on yours. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #17012: [SPARK-19677][SS] Renaming a file atop an existing one s...

2017-02-27 Thread hejix
Github user hejix commented on the issue: https://github.com/apache/spark/pull/17012 Just some feedback that I did some initial regression testing with this pull request on a full YARN (v2.7.3) 4 node cluster on GCP and it appears to have fixed the two issues we had- our structured

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r103346762 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1110,31 +1184,24 @@ class Analyzer( }

[GitHub] spark issue #14273: [SPARK-9140] [ML] Replace TimeTracker by MultiStopwatch

2017-02-27 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/14273 @jkbradley I do not think @MechCoder is working on Spark for the time being. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #16959: [SPARK-19631][CORE] OutputCommitCoordinator should not a...

2017-02-27 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16959 Looks ok to me, but let me ping some others @squito @kayousterhout --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #17081: [SPARK-18726][SQL][FOLLOW-UP]resolveRelation for FileFor...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17081 **[Test build #73539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73539/testReport)** for PR 17081 at commit

[GitHub] spark issue #17079: [SPARK-19748][SQL]refresh function has a wrong order to ...

2017-02-27 Thread windpiger
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/17079 cc @cloud-fan @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16965: [SPARK-18450][ML] Scala API Change for LSH AND-am...

2017-02-27 Thread Yunni
Github user Yunni closed the pull request at: https://github.com/apache/spark/pull/16965 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16965: [SPARK-18450][ML] Scala API Change for LSH AND-amplifica...

2017-02-27 Thread Yunni
Github user Yunni commented on the issue: https://github.com/apache/spark/pull/16965 Looks like the rebase is making it even worse. I will reopen a PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #17078: [SPARK-19746][ML] Faster indexing for logistic aggregato...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17078 **[Test build #73537 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73537/testReport)** for PR 17078 at commit

[GitHub] spark issue #16965: [SPARK-18450][ML] Scala API Change for LSH AND-amplifica...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16965 **[Test build #73538 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73538/testReport)** for PR 16965 at commit

[GitHub] spark pull request #17078: [SPARK-19746][ML] Faster indexing for logistic ag...

2017-02-27 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/17078#discussion_r103343872 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -1431,7 +1431,12 @@ private class LogisticAggregator(

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r103340921 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -40,19 +42,179 @@ abstract class PlanExpression[T

<    1   2   3   4   5   6   7   >