[GitHub] spark issue #14476: [SPARK-16867][SQL] createTable and alterTable in Externa...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14476 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14476: [SPARK-16867][SQL] createTable and alterTable in Externa...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14476 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63212/ Test FAILed. ---

[GitHub] spark issue #14476: [SPARK-16867][SQL] createTable and alterTable in Externa...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14476 **[Test build #63212 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63212/consoleFull)** for PR 14476 at commit

[GitHub] spark issue #14470: [SPARK-16863][ML] ProbabilisticClassifier.fit check thre...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14470 **[Test build #63213 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63213/consoleFull)** for PR 14470 at commit

[GitHub] spark pull request #14474: [SPARK-16853][SQL] fixes encoder error in DataSet...

2016-08-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14474#discussion_r73465751 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -184,6 +184,17 @@ class DatasetSuite extends QueryTest with

[GitHub] spark issue #14467: [SPARK-16861][PYSPARK][CORE] Refactor PySpark accumulato...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14467 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14467: [SPARK-16861][PYSPARK][CORE] Refactor PySpark accumulato...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14467 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63205/ Test FAILed. ---

[GitHub] spark issue #14467: [SPARK-16861][PYSPARK][CORE] Refactor PySpark accumulato...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14467 **[Test build #63205 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63205/consoleFull)** for PR 14467 at commit

[GitHub] spark issue #14476: [SPARK-16867][SQL] createTable and alterTable in Externa...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14476 **[Test build #63212 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63212/consoleFull)** for PR 14476 at commit

[GitHub] spark pull request #14470: [SPARK-16863][ML] ProbabilisticClassifier.fit che...

2016-08-03 Thread zhengruifeng
Github user zhengruifeng commented on a diff in the pull request: https://github.com/apache/spark/pull/14470#discussion_r73465384 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/DecisionTreeClassifier.scala --- @@ -84,6 +84,13 @@ class DecisionTreeClassifier

[GitHub] spark pull request #14470: [SPARK-16863][ML] ProbabilisticClassifier.fit che...

2016-08-03 Thread zhengruifeng
Github user zhengruifeng commented on a diff in the pull request: https://github.com/apache/spark/pull/14470#discussion_r73465400 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/NaiveBayes.scala --- @@ -101,6 +101,14 @@ class NaiveBayes @Since("1.5.0") (

[GitHub] spark issue #14486: [SQL][SPARK-16888] Implements eval method for expression...

2016-08-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14486 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #14486: [SQL][SPARK-16888] Implements eval method for exp...

2016-08-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14486 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #14482: [SPARK-16879][SQL] unify logical plans for CREATE...

2016-08-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14482#discussion_r73465089 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ddl.scala --- @@ -19,50 +19,25 @@ package

[GitHub] spark issue #14478: [SPARK-16875][SQL] Add args checking for DataSet randomS...

2016-08-03 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/14478 @srowen Right. I just add those checking for RDD. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #14482: [SPARK-16879][SQL] unify logical plans for CREATE...

2016-08-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14482#discussion_r73464594 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala --- @@ -154,6 +274,21 @@ private[sql] case class

[GitHub] spark issue #14482: [SPARK-16879][SQL] unify logical plans for CREATE TABLE ...

2016-08-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14482 cc @gatorsmile , IIRC, you have some PRs about error handling. After this PR, we can have a centre place for basic error handling, is it good enough for all the error cases you found? --- If

[GitHub] spark pull request #14480: [MINOR][SQL] Fix minor formatting issue of SortAg...

2016-08-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14480 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14482: [SPARK-16879][SQL] unify logical plans for CREATE TABLE ...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14482 **[Test build #63211 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63211/consoleFull)** for PR 14482 at commit

[GitHub] spark pull request #14482: [SPARK-16879][SQL] unify logical plans for CREATE...

2016-08-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14482#discussion_r73464308 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala --- @@ -206,22 +206,22 @@ private[sql] case class

[GitHub] spark pull request #14482: [SPARK-16879][SQL] unify logical plans for CREATE...

2016-08-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14482#discussion_r73464228 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala --- @@ -62,6 +66,122 @@ private[sql] class

[GitHub] spark issue #14480: [MINOR][SQL] Fix minor formatting issue of SortAggregate...

2016-08-03 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/14480 Thanks, merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #14482: [SPARK-16879][SQL] unify logical plans for CREATE...

2016-08-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14482#discussion_r73464090 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -420,45 +420,40 @@ private[sql] abstract class

[GitHub] spark issue #14482: [SPARK-16879][SQL] unify logical plans for CREATE TABLE ...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14482 **[Test build #63209 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63209/consoleFull)** for PR 14482 at commit

[GitHub] spark issue #14478: [SPARK-16875][SQL] Add args checking for DataSet randomS...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14478 **[Test build #63210 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63210/consoleFull)** for PR 14478 at commit

[GitHub] spark issue #14472: [SPARK-16866][SQL] Infrastructure for file-based SQL end...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14472 **[Test build #3201 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3201/consoleFull)** for PR 14472 at commit

[GitHub] spark issue #14492: [SPARK-16887] Add SPARK_DIST_CLASSPATH to LAUNCH_CLASSPA...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14492 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14492: [SPARK-16887] Add SPARK_DIST_CLASSPATH to LAUNCH_CLASSPA...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14492 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63203/ Test PASSed. ---

[GitHub] spark issue #14492: [SPARK-16887] Add SPARK_DIST_CLASSPATH to LAUNCH_CLASSPA...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14492 **[Test build #63203 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63203/consoleFull)** for PR 14492 at commit

[GitHub] spark issue #14492: [SPARK-16887] Add SPARK_DIST_CLASSPATH to LAUNCH_CLASSPA...

2016-08-03 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14492 More specifically, the Spark distribution has the jars needed by the launcher in `$SPARK_HOME/jars`, so basically this is extra code in Spark to support non-standard distributions. --- If your

[GitHub] spark issue #12135: [SPARK-14352][SQL] approxQuantile should support multi c...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12135 **[Test build #63208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63208/consoleFull)** for PR 12135 at commit

[GitHub] spark issue #14492: [SPARK-16887] Add SPARK_DIST_CLASSPATH to LAUNCH_CLASSPA...

2016-08-03 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14492 But what's the goal of that? If there's nothing in `$SPARK_HOME/jars`, why not create a symlink instead to the location where the jars are? The change itself doesn't really cause any

[GitHub] spark pull request #12135: [SPARK-14352][SQL] approxQuantile should support ...

2016-08-03 Thread zhengruifeng
Github user zhengruifeng commented on a diff in the pull request: https://github.com/apache/spark/pull/12135#discussion_r73462027 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1181,18 +1181,33 @@ def approxQuantile(self, col, probabilities, relativeError):

[GitHub] spark pull request #14482: [SPARK-16879][SQL] unify logical plans for CREATE...

2016-08-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14482#discussion_r73461786 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala --- @@ -233,12 +233,11 @@ private[sql] case class

[GitHub] spark issue #14490: [SPARK-16877][BUILD] Add rules for preventing to use Jav...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14490 **[Test build #63207 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63207/consoleFull)** for PR 14490 at commit

[GitHub] spark pull request #14482: [SPARK-16879][SQL] unify logical plans for CREATE...

2016-08-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14482#discussion_r73461643 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala --- @@ -233,12 +233,11 @@ private[sql] case class

[GitHub] spark issue #14490: [SPARK-16877][BUILD] Add rules for preventing to use Jav...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14490 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63201/ Test FAILed. ---

[GitHub] spark issue #14490: [SPARK-16877][BUILD] Add rules for preventing to use Jav...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14490 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14482: [SPARK-16879][SQL] unify logical plans for CREATE...

2016-08-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14482#discussion_r73461508 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala --- @@ -206,22 +206,22 @@ private[sql] case class

[GitHub] spark issue #14490: [SPARK-16877][BUILD] Add rules for preventing to use Jav...

2016-08-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14490 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14490: [SPARK-16877][BUILD] Add rules for preventing to use Jav...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14490 **[Test build #63201 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63201/consoleFull)** for PR 14490 at commit

[GitHub] spark pull request #14490: [SPARK-16877][BUILD] Add rules for preventing to ...

2016-08-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/14490#discussion_r73460838 --- Diff: scalastyle-config.xml --- @@ -250,6 +250,14 @@ This file is divided into 3 sections: Omit braces in case clauses. +

[GitHub] spark issue #14492: [SPARK-16887] Add SPARK_DIST_CLASSPATH to LAUNCH_CLASSPA...

2016-08-03 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/14492 Sure. This change is for putting Spark jars in a different dir than the default dir in `spark/assembly` or `spark/jars`. So, in this case, the main class is not in `SPARK_JARS_DIR`. --- If your

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63200/ Test PASSed. ---

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14065 **[Test build #63200 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63200/consoleFull)** for PR 14065 at commit

[GitHub] spark issue #14474: [SPARK-16853][SQL] fixes encoder error in DataSet typed ...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14474 **[Test build #63206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63206/consoleFull)** for PR 14474 at commit

[GitHub] spark issue #14467: [SPARK-16861][PYSPARK][CORE] Refactor PySpark accumulato...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14467 **[Test build #63205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63205/consoleFull)** for PR 14467 at commit

[GitHub] spark issue #14474: [SPARK-16853][SQL] fixes encoder error in DataSet typed ...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14474 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63204/ Test FAILed. ---

[GitHub] spark issue #14474: [SPARK-16853][SQL] fixes encoder error in DataSet typed ...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14474 **[Test build #63204 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63204/consoleFull)** for PR 14474 at commit

[GitHub] spark issue #14474: [SPARK-16853][SQL] fixes encoder error in DataSet typed ...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14474 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14473: [SPARK-16495] [MLlib]Add ADMM optimizer in mllib package

2016-08-03 Thread ZunwenYou
Github user ZunwenYou commented on the issue: https://github.com/apache/spark/pull/14473 @MLnick please have a look at this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution by dedupli...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14452 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63202/ Test FAILed. ---

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution by dedupli...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14452 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14490: [SPARK-16877][BUILD] Add rules for preventing to ...

2016-08-03 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/14490#discussion_r73459086 --- Diff: scalastyle-config.xml --- @@ -250,6 +250,14 @@ This file is divided into 3 sections: Omit braces in case clauses. +

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution by dedupli...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14452 **[Test build #63202 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63202/consoleFull)** for PR 14452 at commit

[GitHub] spark issue #14428: [SPARK-16810] Refactor registerSinks with multiple const...

2016-08-03 Thread lovexi
Github user lovexi commented on the issue: https://github.com/apache/spark/pull/14428 I think this one is ready for reviews. :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14490: [SPARK-16877][BUILD] Add rules for preventing to ...

2016-08-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/14490#discussion_r73458908 --- Diff: scalastyle-config.xml --- @@ -250,6 +250,14 @@ This file is divided into 3 sections: Omit braces in case clauses. +

[GitHub] spark issue #14156: [SPARK-16499][ML][MLLib] improve ApplyInPlace function i...

2016-08-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14156 That's the question indeed. I'm not sure because the function that's supplied could be anything. I don't see how it could automatically be converted to a vectorized operation automatically. --- If

[GitHub] spark issue #14156: [SPARK-16499][ML][MLLib] improve ApplyInPlace function i...

2016-08-03 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/14156 yeah, currently it seems to make a little overhead (do a copy), but I think it will take advantage of breeze optimization, in the future, e.g, SIMD instructions or something ? --- If your

[GitHub] spark pull request #14488: [SPARK-16826][SQL] Switch to java.net.URI for par...

2016-08-03 Thread sylvinus
Github user sylvinus commented on a diff in the pull request: https://github.com/apache/spark/pull/14488#discussion_r73458498 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -749,25 +749,44 @@ case class

[GitHub] spark pull request #14490: [SPARK-16877][BUILD] Add rules for preventing to ...

2016-08-03 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14490#discussion_r73458483 --- Diff: scalastyle-config.xml --- @@ -250,6 +250,14 @@ This file is divided into 3 sections: Omit braces in case clauses. +

[GitHub] spark issue #14474: [SPARK-16853][SQL] fixes encoder error in DataSet typed ...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14474 **[Test build #63204 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63204/consoleFull)** for PR 14474 at commit

[GitHub] spark issue #14492: [SPARK-16887] Add SPARK_DIST_CLASSPATH to LAUNCH_CLASSPA...

2016-08-03 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14492 @yhuai can you explain the use case you're trying to cover here with an example? `LAUNCH_CLASSPATH` is the classpath of the launcher process (the process that creates the command line to

[GitHub] spark issue #14445: [SPARK-16320] [SQL] Fix performance regression for parqu...

2016-08-03 Thread clockfly
Github user clockfly commented on the issue: https://github.com/apache/spark/pull/14445 @rxin maybe we can still use this Jira Id by @maver1ck's comment? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #14156: [SPARK-16499][ML][MLLib] improve ApplyInPlace function i...

2016-08-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14156 I see, this copies x to y then modifies y in place. OK. Is that more efficient? it seems like extra work, but does the transform method make up for it? just seeing if this has actually been observed

[GitHub] spark pull request #14476: [SPARK-16867][SQL] createTable and alterTable in ...

2016-08-03 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14476#discussion_r73458295 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalog.scala --- @@ -82,7 +82,7 @@ abstract class ExternalCatalog {

[GitHub] spark pull request #14482: [SPARK-16879][SQL] unify logical plans for CREATE...

2016-08-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14482#discussion_r73458290 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -367,15 +368,16 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark pull request #14490: [SPARK-16877][BUILD] Add rules for preventing to ...

2016-08-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/14490#discussion_r73458250 --- Diff: scalastyle-config.xml --- @@ -250,6 +250,14 @@ This file is divided into 3 sections: Omit braces in case clauses. +

[GitHub] spark pull request #14490: [SPARK-16877][BUILD] Add rules for preventing to ...

2016-08-03 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14490#discussion_r73458156 --- Diff: scalastyle-config.xml --- @@ -250,6 +250,14 @@ This file is divided into 3 sections: Omit braces in case clauses. +

[GitHub] spark issue #14156: [SPARK-16499][ML][MLLib] improve ApplyInPlace function i...

2016-08-03 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/14156 @srowen The := operator in BDM is simply copy one BDM to another, and it is widely used in breeze source, e.g, we can check DenseMatrix.copy function in Breeze: it first use

[GitHub] spark issue #14477: [SPARK-16870][docs]Summary:add "spark.sql.broadcastTimeo...

2016-08-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14477 Any objection to documenting it @liancheng ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14488: [SPARK-16826][SQL] Switch to java.net.URI for par...

2016-08-03 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14488#discussion_r73457928 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -749,25 +749,44 @@ case class

[GitHub] spark pull request #14488: [SPARK-16826][SQL] Switch to java.net.URI for par...

2016-08-03 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14488#discussion_r73457856 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -749,25 +749,44 @@ case class

[GitHub] spark pull request #14482: [SPARK-16879][SQL] unify logical plans for CREATE...

2016-08-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14482#discussion_r73457639 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -61,6 +64,38 @@ trait CheckAnalysis extends

[GitHub] spark pull request #14482: [SPARK-16879][SQL] unify logical plans for CREATE...

2016-08-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14482#discussion_r73457371 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -349,6 +384,27 @@ trait CheckAnalysis extends

[GitHub] spark issue #14488: [SPARK-16826][SQL] Switch to java.net.URI for parse_url(...

2016-08-03 Thread sylvinus
Github user sylvinus commented on the issue: https://github.com/apache/spark/pull/14488 rebase done! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #14491: [SPARK-16886] [EXAMPLES][SQL] structured streaming netwo...

2016-08-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14491 OK, doesn't need a JIRA. Looks like there are similar occurrences in the Python code, and streaming docs. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #13738: [SPARK-11227][CORE] UnknownHostException can be thrown w...

2016-08-03 Thread soldiershen
Github user soldiershen commented on the issue: https://github.com/apache/spark/pull/13738 @sarutak got it. I add hdfs conf file to specific the host .Thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #14156: [SPARK-16499][ML][MLLib] improve ApplyInPlace function i...

2016-08-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14156 Is there reasonable evidence this speeds things up? just want to make sure this does not make it slower. Help me understand the := operator? I don't recognize how it's helping compute y as a

[GitHub] spark issue #14488: [SPARK-16826][SQL] Switch to java.net.URI for parse_url(...

2016-08-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14488 Description looks good. You can use `git rebase -i HEAD~4` or similar to `drop` the extra commit here. Pending that and tests passing, looks good. --- If your project is set up for it, you can

[GitHub] spark issue #14492: [SPARK-16887] Add SPARK_DIST_CLASSPATH to LAUNCH_CLASSPA...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14492 **[Test build #63203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63203/consoleFull)** for PR 14492 at commit

[GitHub] spark pull request #14492: [SPARK-16887] Add SPARK_DIST_CLASSPATH to LAUNCH_...

2016-08-03 Thread yhuai
GitHub user yhuai opened a pull request: https://github.com/apache/spark/pull/14492 [SPARK-16887] Add SPARK_DIST_CLASSPATH to LAUNCH_CLASSPATH ## What changes were proposed in this pull request? To deploy Spark, it can be pretty convenient to put all jars (spark jars, hadoop

[GitHub] spark issue #14490: [SPARK-16877][BUILD] Add rules for preventing to use Jav...

2016-08-03 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14490 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #14472: [SPARK-16866][SQL] Infrastructure for file-based SQL end...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14472 **[Test build #3201 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3201/consoleFull)** for PR 14472 at commit

[GitHub] spark issue #14491: [SPARK-16886] [EXAMPLES][SQL] structured streaming netwo...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14491 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14472: [SPARK-16866][SQL] Infrastructure for file-based SQL end...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14472 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63197/ Test FAILed. ---

[GitHub] spark issue #14472: [SPARK-16866][SQL] Infrastructure for file-based SQL end...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14472 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14472: [SPARK-16866][SQL] Infrastructure for file-based SQL end...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14472 **[Test build #63197 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63197/consoleFull)** for PR 14472 at commit

[GitHub] spark pull request #14491: [SPARK-16886] [EXAMPLES][SQL] structured streamin...

2016-08-03 Thread ganeshchand
GitHub user ganeshchand opened a pull request: https://github.com/apache/spark/pull/14491 [SPARK-16886] [EXAMPLES][SQL] structured streaming network word count examples … ## What changes were proposed in this pull request? Fixed a minor code comment typo by replacing

[GitHub] spark issue #14490: [SPARK-16877][BUILD] Add rules for preventing to use Jav...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14490 **[Test build #63201 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63201/consoleFull)** for PR 14490 at commit

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution by dedupli...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14452 **[Test build #63202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63202/consoleFull)** for PR 14452 at commit

[GitHub] spark pull request #12913: [SPARK-928][CORE] Add support for Unsafe-based se...

2016-08-03 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/12913#discussion_r73454617 --- Diff: core/src/test/scala/org/apache/spark/serializer/KryoSerializerSuite.scala --- @@ -399,6 +399,14 @@ class KryoSerializerSuite extends

[GitHub] spark issue #14490: [SPARK-16877][BUILD] Add rules for preventing to use Jav...

2016-08-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14490 cc @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #14490: [SPARK-16877][BUILD] Add rules for preventing to ...

2016-08-03 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/14490 [SPARK-16877][BUILD] Add rules for preventing to use Java annotations (Deprecated and Override) ## What changes were proposed in this pull request? This PR adds both rules for

[GitHub] spark issue #14488: [SPARK-16826][SQL] Switch to java.net.URI for parse_url(...

2016-08-03 Thread sylvinus
Github user sylvinus commented on the issue: https://github.com/apache/spark/pull/14488 @rxin is that better? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14279: [SPARK-16216][SQL] Write Timestamp and Date in ISO 8601 ...

2016-08-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14279 Yes, as shown in https://github.com/apache/spark/pull/14279#issuecomment-236469454 (but we should still manually give the schema as inferring `DateType` and `TimestampType` is not

[GitHub] spark issue #13891: [SPARK-6685][MLLIB]Use DSYRK to compute AtA in ALS

2016-08-03 Thread hqzizania
Github user hqzizania commented on the issue: https://github.com/apache/spark/pull/13891 cc @mengxr @yanboliang Was this patch Okay? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14479: [SPARK-16873] [Core] Fix SpillReader NPE when spi...

2016-08-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14479 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14479: [SPARK-16873] [Core] Fix SpillReader NPE when spillFile ...

2016-08-03 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14479 LGTM - merging in master/2.0/1.6. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

  1   2   3   4   5   6   >