[GitHub] spark pull request: [CORE][MINOR][DOC] Removing incorrect scaladoc

2016-05-28 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/13384 [CORE][MINOR][DOC] Removing incorrect scaladoc ## What changes were proposed in this pull request? I don't think the method will ever throw an exception so removing a false comment

[GitHub] spark pull request: [CORE][DOC][MINOR] typos + links

2016-05-28 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/13383 [CORE][DOC][MINOR] typos + links ## What changes were proposed in this pull request? A very tiny change to javadoc (which I don't mind if gets merged with a bigger change). I've

[GitHub] spark pull request: [SPARK-15647] [SQL] Fix Boundary Cases in Opti...

2016-05-31 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/13392#discussion_r65136606 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -937,8 +937,12 @@ object SimplifyConditionals

[GitHub] spark pull request: [SPARK-15662][SQL] Add since annotation for cl...

2016-05-31 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/13406#discussion_r65140165 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/catalog/interface.scala --- @@ -25,6 +25,14 @@ import

[GitHub] spark pull request: [SPARK-15662][SQL] Add since annotation for cl...

2016-05-31 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/13406#discussion_r65140246 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/catalog/interface.scala --- @@ -41,6 +49,16 @@ class Database

[GitHub] spark pull request: [SPARK-15662][SQL] Add since annotation for cl...

2016-05-31 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/13406#discussion_r65140296 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/catalog/interface.scala --- @@ -83,9 +112,19 @@ class Column( } -// TODO

[GitHub] spark pull request: [CORE][SQL][MINOR] Scaladoc fixes + string int...

2016-05-27 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/13329#issuecomment-222140528 The issue with `nonEmpty` is that you could easily miss the negation (and that's why Scala offers `nonEmpty`). I don't think it's the final solution

[GitHub] spark pull request: [CORE][SQL][MINOR] Scaladoc fixes + string int...

2016-05-27 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/13329#discussion_r64900321 --- Diff: yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnScheduler.scala --- @@ -31,9 +31,8 @@ private[spark] class YarnScheduler(sc

[GitHub] spark pull request: [CORE][SQL][MINOR] Scaladoc fixes + string int...

2016-05-26 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/13329 [CORE][SQL][MINOR] Scaladoc fixes + string interpolation ## What changes were proposed in this pull request? Scaladoc fixes + string interpolation for logging ## How

[GitHub] spark pull request: [CORE][SQL][MINOR] Scaladoc fixes + string int...

2016-05-27 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/13329#issuecomment-42718 How am I supposed to read this? Do you want me to...forget about the changes? All of them or just some? Which one would you accept since @srowen said: "

[GitHub] spark pull request: [CORE][SQL][MINOR] Scaladoc fixes + string int...

2016-05-27 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/13329#issuecomment-70008 Thanks @rxin and @srowen for your help and patience! I'll close the pull request. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [CORE][SQL][MINOR] Scaladoc fixes + string int...

2016-05-27 Thread jaceklaskowski
Github user jaceklaskowski closed the pull request at: https://github.com/apache/spark/pull/13329 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request: [CORE][DOC][MINOR] Remove incorrect scaladoc

2016-06-01 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/13432 [CORE][DOC][MINOR] Remove incorrect scaladoc ## What changes were proposed in this pull request? It removes a sentence about `SparkListener` being internal and may change

[GitHub] spark pull request: [SPARK-15647] [SQL] Fix Boundary Cases in Opti...

2016-05-29 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/13392#discussion_r65008333 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -937,7 +937,8 @@ object SimplifyConditionals

[GitHub] spark pull request: [STREAMING][MINOR] Typo fixes

2016-01-11 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/10698 [STREAMING][MINOR] Typo fixes You can merge this pull request into a Git repository by running: $ git pull https://github.com/jaceklaskowski/spark streaming-kafka-typo-fixes

[GitHub] spark pull request #13909: [SPARK-16213][SQL] Reduce runtime overhead of a p...

2016-06-26 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/13909#discussion_r68506564 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -51,27 +51,52 @@ case class

[GitHub] spark pull request #13909: [SPARK-16213][SQL] Reduce runtime overhead of a p...

2016-06-26 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/13909#discussion_r68506515 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -51,27 +51,52 @@ case class

[GitHub] spark pull request #13909: [SPARK-16213][SQL] Reduce runtime overhead of a p...

2016-06-26 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/13909#discussion_r68506637 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameComplexTypeSuite.scala --- @@ -26,6 +26,20 @@ import

[GitHub] spark pull request #13912: [SPARK-16216][SQL] CSV data source does not write...

2016-06-26 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/13912#discussion_r68506371 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVRelation.scala --- @@ -195,18 +202,50 @@ private[sql] class

[GitHub] spark pull request #13912: [SPARK-16216][SQL] CSV data source does not write...

2016-06-26 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/13912#discussion_r68506352 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVRelation.scala --- @@ -195,18 +202,50 @@ private[sql] class

[GitHub] spark pull request #13909: [SPARK-16213][SQL] Reduce runtime overhead of a p...

2016-06-26 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/13909#discussion_r68506607 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameComplexTypeSuite.scala --- @@ -26,6 +26,20 @@ import

[GitHub] spark pull request: Fix for [SPARK-12854][SQL] Implement complex t...

2016-01-28 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/10946#issuecomment-176115262 I'd appreciate having it merged since the current master is broken without the fix - checked early morning today. --- If your project is set up for it, you can

[GitHub] spark pull request: Fix for [SPARK-12854][SQL] Implement complex t...

2016-02-01 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/10946#issuecomment-177873394 @JoshRosen @rxin a friendly reminder to merge the change (or close it if irrelevant) --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [STREAMING][MINOR] Scaladoc + logs

2016-01-22 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/10878 [STREAMING][MINOR] Scaladoc + logs Found while doing code review You can merge this pull request into a Git repository by running: $ git pull https://github.com/jaceklaskowski/spark

[GitHub] spark pull request: Fix for [SPARK-12854][SQL] Implement complex t...

2016-01-27 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/10946 Fix for [SPARK-12854][SQL] Implement complex types support in Columna… …rBatch Fixes build for Scala 2.11. You can merge this pull request into a Git repository by running

[GitHub] spark pull request: [STREAMING][DOCS] Fixes and code improvements ...

2016-02-18 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/11201#issuecomment-186008514 @srowen Mind having a look? I'd appreciate. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [STREAMING][DOCS] Fixes and code improvements ...

2016-02-14 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/11201 [STREAMING][DOCS] Fixes and code improvements for checkpointing You can merge this pull request into a Git repository by running: $ git pull https://github.com/jaceklaskowski/spark

[GitHub] spark pull request: [SPARK-13825][CORE] Upgrade to Scala 2.11.8

2016-03-15 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/11681#issuecomment-196929219 @srowen Mind triggering a test? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13825][CORE] Upgrade to Scala 2.11.8

2016-03-12 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/11681 [SPARK-13825][CORE] Upgrade to Scala 2.11.8 ## What changes were proposed in this pull request? Upgrade to 2.11.8 (from the current 2.11.7) ## How was this patch tested

[GitHub] spark pull request: [SPARK-13776][WebUI]Limit the max number of ac...

2016-03-11 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/11615#discussion_r55883199 --- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala --- @@ -270,9 +270,25 @@ private[spark] object JettyUtils extends Logging

[GitHub] spark pull request: [MINOR] Typo fixes

2016-03-18 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/11802#discussion_r56738903 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala --- @@ -277,7 +277,7 @@ abstract class DStream[T: ClassTag

[GitHub] spark pull request: [MINOR] Typo fixes

2016-03-18 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/11802#issuecomment-198593089 Thanks @srowen @thunterdb for review! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13776][WebUI]Limit the max number of ac...

2016-03-11 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/11615#discussion_r55868740 --- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala --- @@ -270,9 +270,25 @@ private[spark] object JettyUtils extends Logging

[GitHub] spark pull request: [SPARK-13776][WebUI]Limit the max number of ac...

2016-03-11 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/11615#discussion_r55869059 --- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala --- @@ -270,9 +270,25 @@ private[spark] object JettyUtils extends Logging

[GitHub] spark pull request: [SPARK-14500] [ML] Accept Dataset[_] instead o...

2016-04-09 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12274#discussion_r59124592 --- Diff: mllib/src/main/scala/org/apache/spark/ml/Predictor.scala --- @@ -171,18 +171,18 @@ abstract class PredictionModel[FeaturesType, M

[GitHub] spark pull request: [SPARK-14500] [ML] Accept Dataset[_] instead o...

2016-04-09 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12274#discussion_r59124613 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/impl/GLMRegressionModel.scala --- @@ -23,7 +23,7 @@ import

[GitHub] spark pull request: [SPARK-14500] [ML] Accept Dataset[_] instead o...

2016-04-09 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12274#discussion_r59124562 --- Diff: mllib/src/main/scala/org/apache/spark/ml/Pipeline.scala --- @@ -124,7 +124,7 @@ class Pipeline @Since("1.4.0") ( * @ret

[GitHub] spark pull request: [SPARK-14500] [ML] Accept Dataset[_] instead o...

2016-04-09 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12274#discussion_r59124622 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/model/DecisionTreeModel.scala --- @@ -32,7 +32,7 @@ import

[GitHub] spark pull request: [HOTFIX][SPARK-14402] Fix ExpressionDescriptio...

2016-04-05 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/12192 [HOTFIX][SPARK-14402] Fix ExpressionDescription annotation ## What changes were proposed in this pull request? Fix for the error introduced in https://github.com/apache/spark

[GitHub] spark pull request: [HOTFIX][SPARK-14402] Fix ExpressionDescriptio...

2016-04-05 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/12192#issuecomment-206095697 Is this really the patch to cause the issue or is this more an interim hiccup? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [HOTFIX][SPARK-14402] Fix ExpressionDescriptio...

2016-04-07 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/12192#issuecomment-207175670 Please please merge it as soon as possible as I'm suffering from not having it in master every time I do the build :( --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-14288][SQL] Memory Sink for streaming

2016-04-11 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12119#discussion_r59296064 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -275,23 +277,64 @@ final class DataFrameWriter private[sql](df

[GitHub] spark pull request: [MINOR] Typo fixes

2016-03-19 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/11802 [MINOR] Typo fixes ## What changes were proposed in this pull request? Typo fixes. No functional changes. ## How was this patch tested? Built the sources and ran

[GitHub] spark pull request: [MINOR] Typo fixes

2016-03-25 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/11802#discussion_r57455819 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala --- @@ -246,9 +246,7 @@ class StreamingContext private[streaming

[GitHub] spark pull request: [SPARK-11535][ML] handling empty string in Str...

2016-03-08 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/11575#discussion_r55345145 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/StringIndexerSuite.scala --- @@ -178,6 +178,20 @@ class StringIndexerSuite

[GitHub] spark pull request: [SPARK-12458][SQL] Add ExpressionDescription t...

2016-03-08 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/10428#discussion_r55347052 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -126,6 +144,13 @@ case class Hour

[GitHub] spark pull request: [SPARK-13727] [SQL] SparkConf.contains does no...

2016-03-08 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/11568#discussion_r55406003 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -351,7 +351,16 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable

[GitHub] spark pull request: [SPARK-13727] [SQL] SparkConf.contains does no...

2016-03-08 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/11568#discussion_r55408473 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -351,7 +351,16 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable

[GitHub] spark pull request: [MINOR][ML] Use Datasets (to improve internal ...

2016-03-23 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/11915 [MINOR][ML] Use Datasets (to improve internal implementation) ## What changes were proposed in this pull request? Change the current implementation to use Datasets (not the &quo

[GitHub] spark pull request: [SPARK-14889][Spark Core] scala.MatchError: NO...

2016-04-25 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12666#discussion_r60991107 --- Diff: core/src/main/scala/org/apache/spark/scheduler/Pool.scala --- @@ -55,6 +55,8 @@ private[spark] class Pool( new

[GitHub] spark pull request: [DOCS][MINOR] Accumulators

2016-04-22 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/12569#issuecomment-213275339 @HyukjinKwon In that case I'd ask for the alternative as I currently have no idea how to make it clearer (it wasn't me to say "the title is not clear"

[GitHub] spark pull request: [DOCS][MINOR] Screenshot + minor fixes to impr...

2016-04-22 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/12569#issuecomment-213387430 Title has changed to incorporate @HyukjinKwon's suggestion. Please review @srowen. I intentionally have not included "in the programming guide" due

[GitHub] spark pull request: [MINOR][DOCS] Minor typo fixes

2016-04-22 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12469#discussion_r60725372 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala --- @@ -80,10 +80,8 @@ class HiveContext private[hive]( protected

[GitHub] spark pull request: [MINOR][DOCS] Minor typo fixes

2016-04-22 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12469#discussion_r60726392 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala --- @@ -80,10 +80,8 @@ class HiveContext private[hive]( protected

[GitHub] spark pull request: [MINOR][DOCS] Minor typo fixes

2016-04-22 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12469#discussion_r60727194 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala --- @@ -80,10 +80,8 @@ class HiveContext private[hive]( protected

[GitHub] spark pull request: [MINOR][DOCS] Minor typo fixes

2016-04-21 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/12469#issuecomment-213118039 I fixed the issue with the two failing tests due to the error message having been changed. Jenkins retest this please :) --- If your project

[GitHub] spark pull request: [DOCS][MINOR] Screenshot + minor fixes to impr...

2016-04-23 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/12569#issuecomment-213819194 Could this PR have a bit of @srowen's attention? :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [MINOR][DOCS] Minor typo fixes

2016-04-26 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/12469#issuecomment-214638031 @srowen Reverted the line removal and rebased with master. Mind merging it to the repo? Thanks. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-15425][SQL] Disallow cartesian joins by...

2016-05-19 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/13209#discussion_r63987980 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -348,6 +348,11 @@ object SQLConf { .booleanConf

[GitHub] spark pull request: [SPARK-15428][SQL] Disable multiple streaming ...

2016-05-19 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/13210#discussion_r63987667 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala --- @@ -55,10 +55,19 @@ object

[GitHub] spark pull request: [SPARK-15428][SQL] Disable multiple streaming ...

2016-05-19 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/13210#discussion_r63987729 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala --- @@ -55,10 +55,19 @@ object

[GitHub] spark pull request: [SPARK-15428][SQL] Disable multiple streaming ...

2016-05-19 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/13210#discussion_r63987696 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala --- @@ -55,10 +55,19 @@ object

[GitHub] spark pull request: [SPARK-15458][SQL][STREAMING] Disable schema i...

2016-05-20 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/13238#discussion_r64127527 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -186,6 +187,14 @@ case class DataSource

[GitHub] spark pull request: [SPARK-15458][SQL][STREAMING] Disable schema i...

2016-05-20 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/13238#discussion_r64127555 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/FileStreamSourceSuite.scala --- @@ -140,6 +140,18 @@ class FileStreamSourceSuite

[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...

2016-05-05 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12928#discussion_r62169812 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -155,10 +155,10 @@ private[spark] class Client( // Get

[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...

2016-05-05 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/12928 [SPARK-15152][DOC][MINOR] Scaladoc and Code style Improvements ## What changes were proposed in this pull request? Minor doc and code style fixes ## How was this patch

[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...

2016-05-05 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12928#discussion_r62170253 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -168,7 +168,7 @@ private[spark] class Client( val appContext

[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...

2016-05-05 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12928#discussion_r62170912 --- Diff: core/src/main/scala/org/apache/spark/scheduler/Pool.scala --- @@ -98,13 +97,12 @@ private[spark] class Pool( } override

[GitHub] spark pull request: Minor typo fixes

2016-04-18 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12469#discussion_r60092677 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -453,8 +453,8 @@ class Dataset[T] private[sql]( * Returns true

[GitHub] spark pull request: Minor typo fixes

2016-04-18 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12469#discussion_r60092525 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala --- @@ -38,8 +38,6 @@ import org.apache.spark.sql.types

[GitHub] spark pull request: Minor typo fixes

2016-04-18 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12469#discussion_r60092825 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala --- @@ -46,8 +46,7 @@ import

[GitHub] spark pull request: Minor typo fixes

2016-04-18 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12469#discussion_r60092387 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala --- @@ -438,7 +438,6 @@ class LinearRegressionModel private[ml

[GitHub] spark pull request: [MINOR][ML] Use Datasets (to improve internal ...

2016-04-18 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/11915#issuecomment-211605171 Sorry, no. Got distracted and had no time for it. Let me work on it later today (it's 00:21 already). --- If your project is set up for it, you can reply

[GitHub] spark pull request: Minor typo fixes

2016-04-18 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/12469 Minor typo fixes ## What changes were proposed in this pull request? Minor typo fixes (too minor to deserve separate a JIRA) ## How was this patch tested? local

[GitHub] spark pull request: [MINOR][DOCS] Minor typo fixes

2016-04-20 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/12469#issuecomment-212316780 @srowen Mind reviewing and possibly merging afterwards? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [DOCS][MINOR] Accumulators

2016-04-21 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12569#discussion_r60577867 --- Diff: docs/programming-guide.md --- @@ -1328,12 +1328,18 @@ value of the broadcast variable (e.g. if the variable is shipped to a new node l

[GitHub] spark pull request: [MINOR][DOCS] Minor typo fixes

2016-04-21 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/12469#issuecomment-212920619 Done @srowen. Thanks a lot for your patience. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [DOCS][MINOR] Accumulators

2016-04-21 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/12569 [DOCS][MINOR] Accumulators ## What changes were proposed in this pull request? Added screenshot + minor fixes to improve reading ## How was this patch tested

[GitHub] spark pull request: [SPARK-15067] [YARN] YARN executors are launch...

2016-05-08 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/12985#issuecomment-217727591 What do you think about a unit test for this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-14938][ML] replace RDD.map with Dataset...

2016-04-28 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12718#discussion_r61494448 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/ChiSqSelector.scala --- @@ -79,11 +79,12 @@ final class ChiSqSelector(override val uid

[GitHub] spark pull request: [SPARK-14938][ML] replace RDD.map with Dataset...

2016-04-28 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215550190 Other than the few places where you could use symbols not string literals LGTM. Excellent job! Thanks. --- If your project is set up for it, you can reply

[GitHub] spark pull request #14302: [SPARK-16663][SQL] desc table should be consisten...

2016-07-22 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14302#discussion_r71885483 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -436,11 +436,13 @@ case class DescribeTableCommand(table

[GitHub] spark pull request #14309: [SPARK-11977][SQL] Support accessing a column con...

2016-07-22 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14309#discussion_r71888979 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -641,6 +641,10 @@ class DataFrameSuite extends QueryTest

[GitHub] spark pull request #14309: [SPARK-11977][SQL] Support accessing a column con...

2016-07-22 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14309#discussion_r7196 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -641,6 +641,10 @@ class DataFrameSuite extends QueryTest

[GitHub] spark pull request #14309: [SPARK-11977][SQL] Support accessing a column con...

2016-07-22 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14309#discussion_r7137 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -641,6 +641,10 @@ class DataFrameSuite extends QueryTest

[GitHub] spark pull request #14302: [SPARK-16663][SQL] desc table should be consisten...

2016-07-22 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14302#discussion_r71886372 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveOperatorQueryableSuite.scala --- @@ -41,13 +41,13 @@ class

[GitHub] spark issue #14307: [SPARK-16672][SQL] SQLBuilder should not raise exception...

2016-07-22 Thread jaceklaskowski
Github user jaceklaskowski commented on the issue: https://github.com/apache/spark/pull/14307 I didn't even know such a SQL query is possible in SQL :) Lots to learn still. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark issue #14305: Spark-16669:Adding partition prunning to Metastore stati...

2016-07-22 Thread jaceklaskowski
Github user jaceklaskowski commented on the issue: https://github.com/apache/spark/pull/14305 I think the PR is completely broken and should be recreated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #14292: [SPARK-14131][SQL[STREAMING] Improved fix for avo...

2016-07-25 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14292#discussion_r72148282 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/SQLTestUtils.scala --- @@ -247,6 +248,46 @@ private[sql] trait SQLTestUtils

[GitHub] spark pull request #14292: [SPARK-14131][SQL[STREAMING] Improved fix for avo...

2016-07-25 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14292#discussion_r72148009 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -269,19 +273,11 @@ class StreamExecution

[GitHub] spark pull request #14285: [SPARK-16649][SQL] Push partition predicates down...

2016-07-21 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14285#discussion_r71776630 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalog.scala --- @@ -167,6 +168,21 @@ abstract class

[GitHub] spark pull request #14304: [SPARK-16668][TEST] Test parquet reader for row g...

2016-07-21 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14304#discussion_r71773933 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetEncodingSuite.scala --- @@ -78,4 +78,29 @@ class

[GitHub] spark pull request #14285: [SPARK-16649][SQL] Push partition predicates down...

2016-07-21 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14285#discussion_r71776725 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -632,6 +632,23 @@ class SessionCatalog

[GitHub] spark pull request #14285: [SPARK-16649][SQL] Push partition predicates down...

2016-07-21 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14285#discussion_r71776877 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -632,6 +632,23 @@ class SessionCatalog

[GitHub] spark pull request #14304: [SPARK-16668][TEST] Test parquet reader for row g...

2016-07-21 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14304#discussion_r71774436 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetEncodingSuite.scala --- @@ -78,4 +78,29 @@ class

[GitHub] spark pull request #14295: [SPARK-16648][SQL] Overrides TreeNode.withNewChil...

2016-07-22 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14295#discussion_r71908714 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Last.scala --- @@ -42,6 +42,17 @@ case class Last(child

[GitHub] spark pull request #14327: [SPARK-16686][SQL] Project shouldn't be pushed do...

2016-07-24 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14327#discussion_r71992254 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -422,6 +422,32 @@ class DatasetSuite extends QueryTest

[GitHub] spark pull request #14327: [SPARK-16686][SQL] Project shouldn't be pushed do...

2016-07-24 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14327#discussion_r71992242 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -150,13 +150,20 @@ class SimpleTestOptimizer

[GitHub] spark pull request #14327: [SPARK-16686][SQL] Project shouldn't be pushed do...

2016-07-24 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14327#discussion_r71992270 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -422,6 +422,32 @@ class DatasetSuite extends QueryTest

[GitHub] spark pull request #14326: [SPARK-3181] [ML] Implement RobustRegression with...

2016-07-24 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14326#discussion_r71992474 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/RobustRegression.scala --- @@ -0,0 +1,466 @@ +/* + * Licensed to the Apache

<    1   2   3   4   5   6   >