[GitHub] spark pull request #18029: [SPARK-20168] [DStream] Add changes to use kinesi...

2017-12-24 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/18029#discussion_r158627994 --- Diff: external/kinesis-asl/src/test/java/org/apache/spark/streaming/kinesis/JavaKinesisInputDStreamBuilderSuite.java --- @@ -45,18 +44,90 @@ public

[GitHub] spark pull request #18029: [SPARK-20168] [DStream] Add changes to use kinesi...

2017-12-24 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/18029#discussion_r158627640 --- Diff: external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala --- @@ -56,12 +57,13 @@ import

[GitHub] spark pull request #18029: [SPARK-20168] [DStream] Add changes to use kinesi...

2017-12-24 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/18029#discussion_r158627744 --- Diff: external/kinesis-asl/src/main/java/org/apache/spark/streaming/kinesis/KinesisInitialPositions.java --- @@ -0,0 +1,91 @@ +/* + * Licensed

[GitHub] spark pull request #18029: [SPARK-20168] [DStream] Add changes to use kinesi...

2017-12-24 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/18029#discussion_r158627941 --- Diff: external/kinesis-asl/src/test/java/org/apache/spark/streaming/kinesis/JavaKinesisInputDStreamBuilderSuite.java --- @@ -45,18 +44,90 @@ public

[GitHub] spark issue #20076: [SPARK-21786][SQL] When acquiring 'compressionCodecClass...

2017-12-24 Thread fjh100456
Github user fjh100456 commented on the issue: https://github.com/apache/spark/pull/20076 cc @gatorsmile No orc configuration found in "sql-programming-guide.md", so I did not add the precedence description to `spark.sql.orc.compression.codec `. ---

[GitHub] spark issue #20076: [SPARK-21786][SQL] When acquiring 'compressionCodecClass...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20076 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20076: [SPARK-21786][SQL] When acquiring 'compressionCod...

2017-12-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20076#discussion_r158627363 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetOptions.scala --- @@ -42,8 +43,15 @@ private[parquet]

[GitHub] spark pull request #20076: [SPARK-21786][SQL] When acquiring 'compressionCod...

2017-12-24 Thread fjh100456
GitHub user fjh100456 opened a pull request: https://github.com/apache/spark/pull/20076 [SPARK-21786][SQL] When acquiring 'compressionCodecClassName' in 'ParquetOptions', `parquet.compression` needs to be considered. [SPARK-21786][SQL] When acquiring 'compressionCodecClassName' in

[GitHub] spark issue #19498: [SPARK-17756][PYTHON][STREAMING] Workaround to avoid ret...

2017-12-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19498 Will check and see if I could make a min fix soon. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20075: [SPARK-21208][R] Adds setLocalProperty in R

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20075 **[Test build #85369 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85369/testReport)** for PR 20075 at commit

[GitHub] spark issue #20075: [SPARK-21208][R] Adds setLocalProperty in R

2017-12-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20075 cc @felixcheung, could you take a look please? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20075: [SPARK-21208][R] Adds setLocalProperty in R

2017-12-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20075#discussion_r158626493 --- Diff: R/pkg/R/sparkR.R --- @@ -564,6 +564,23 @@ setJobDescription <- function(value) { invisible(callJMethod(sc, "setJobDescription", value))

[GitHub] spark pull request #20075: [SPARK-21208][R] Adds setLocalProperty in R

2017-12-24 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/20075 [SPARK-21208][R] Adds setLocalProperty in R ## What changes were proposed in this pull request? This PR adds `setLocalProperty` in R. ```R > df <- createDataFrame(iris)

[GitHub] spark issue #20043: [SPARK-22856][SQL] Add wrappers for codegen output and n...

2017-12-24 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20043 `StatementValue` means an output like `a + 1`. It is a java statement which doesn't rely on a local variable to hold the result. ---

[GitHub] spark pull request #19904: [SPARK-22707][ML] Optimize CrossValidator memory ...

2017-12-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19904 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20074: [SPARK-22874][PYSPARK][SQL][FOLLOW-UP] Modify error mess...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20074 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19904: [SPARK-22707][ML] Optimize CrossValidator memory occupat...

2017-12-24 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/19904 LGTM Sorry for the delay & thanks for the PR! Merging with master --- - To unsubscribe, e-mail:

[GitHub] spark issue #20074: [SPARK-22874][PYSPARK][SQL][FOLLOW-UP] Modify error mess...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20074 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85367/ Test PASSed. ---

[GitHub] spark issue #20074: [SPARK-22874][PYSPARK][SQL][FOLLOW-UP] Modify error mess...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20074 **[Test build #85367 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85367/testReport)** for PR 20074 at commit

[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...

2017-12-24 Thread aa8y
Github user aa8y commented on the issue: https://github.com/apache/spark/pull/20068 @HyukjinKwon I made code changes based on your suggestions. I also changed the tests to use the data mentioned in the ticket. However, you're right, the tests no longer pass. But that is because the

[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20068 **[Test build #85368 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85368/testReport)** for PR 20068 at commit

[GitHub] spark issue #20074: [SPARK-22874][PYSPARK][SQL][FOLLOW-UP] Modify error mess...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20074 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85366/ Test PASSed. ---

[GitHub] spark issue #20074: [SPARK-22874][PYSPARK][SQL][FOLLOW-UP] Modify error mess...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20074 **[Test build #85366 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85366/testReport)** for PR 20074 at commit

[GitHub] spark issue #20074: [SPARK-22874][PYSPARK][SQL][FOLLOW-UP] Modify error mess...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20074 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20074: [SPARK-22874][PYSPARK][SQL][FOLLOW-UP] Modify error mess...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20074 **[Test build #85367 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85367/testReport)** for PR 20074 at commit

[GitHub] spark issue #18754: [SPARK-21552][SQL] Add DecimalType support to ArrowWrite...

2017-12-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18754 Just took a quick look and looks fine to me too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20072: [SPARK-22790][SQL] add a configurable factor to describe...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20072 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85363/ Test PASSed. ---

[GitHub] spark issue #20072: [SPARK-22790][SQL] add a configurable factor to describe...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20072 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20074: [SPARK-22874][PYSPARK][SQL][FOLLOW-UP] Modify err...

2017-12-24 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20074#discussion_r158622968 --- Diff: python/pyspark/sql/utils.py --- @@ -118,7 +118,8 @@ def require_minimum_pandas_version(): from distutils.version import LooseVersion

[GitHub] spark issue #20072: [SPARK-22790][SQL] add a configurable factor to describe...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20072 **[Test build #85363 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85363/testReport)** for PR 20072 at commit

[GitHub] spark pull request #20074: [SPARK-22874][PYSPARK][SQL][FOLLOW-UP] Modify err...

2017-12-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20074#discussion_r158622598 --- Diff: python/pyspark/sql/utils.py --- @@ -118,7 +118,8 @@ def require_minimum_pandas_version(): from distutils.version import LooseVersion

[GitHub] spark issue #20074: [SPARK-22874][PYSPARK][SQL][FOLLOW-UP] Modify error mess...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20074 **[Test build #85366 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85366/testReport)** for PR 20074 at commit

[GitHub] spark pull request #20074: [SPARK-22874][PYSPARK][SQL][FOLLOW-UP] Modify err...

2017-12-24 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/20074 [SPARK-22874][PYSPARK][SQL][FOLLOW-UP] Modify error messages to show actual versions. ## What changes were proposed in this pull request? This is a follow-up pr of #20054 modifying error

[GitHub] spark issue #20043: [SPARK-22856][SQL] Add wrappers for codegen output and n...

2017-12-24 Thread gczsjdy
Github user gczsjdy commented on the issue: https://github.com/apache/spark/pull/20043 @viirya Thanks much. Actually local variable corresponds to `VariableValue` and `StatementValue`? IIUC `VariableValue` is value that depends on something else, but what is `StatementValue`? Maybe

[GitHub] spark issue #20073: [SPARK-22843][R] Adds localCheckpoint in R

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20073 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85364/ Test PASSed. ---

[GitHub] spark issue #20073: [SPARK-22843][R] Adds localCheckpoint in R

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20073 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20073: [SPARK-22843][R] Adds localCheckpoint in R

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20073 **[Test build #85364 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85364/testReport)** for PR 20073 at commit

[GitHub] spark issue #18754: [SPARK-21552][SQL] Add DecimalType support to ArrowWrite...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18754 **[Test build #85365 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85365/testReport)** for PR 18754 at commit

[GitHub] spark issue #19904: [SPARK-22707][ML] Optimize CrossValidator memory occupat...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19904 **[Test build #4024 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4024/testReport)** for PR 19904 at commit

[GitHub] spark issue #20073: [SPARK-22843][R] Adds localCheckpoint in R

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20073 **[Test build #85364 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85364/testReport)** for PR 20073 at commit

[GitHub] spark issue #20073: [SPARK-22843][R] Adds localCheckpoint in R

2017-12-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20073 cc @felixcheung, could you check if I understood your intention correctly? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20073: [SPARK-22843][R] Adds localCheckpoint in R

2017-12-24 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/20073 [SPARK-22843][R] Adds localCheckpoint in R ## What changes were proposed in this pull request? This PR proposes to add `localCheckpoint(..)` in R API. ```r df <-

[GitHub] spark pull request #18754: [SPARK-21552][SQL] Add DecimalType support to Arr...

2017-12-24 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18754#discussion_r158620106 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowWriter.scala --- @@ -214,6 +216,22 @@ private[arrow] class DoubleWriter(val

[GitHub] spark pull request #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator f...

2017-12-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19527#discussion_r158619213 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/OneHotEncoderEstimator.scala --- @@ -0,0 +1,479 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-12-24 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/19527 Agree on keeping the new OneHotEncoderEstimator as an alias for 3.0 On Fri, 1 Dec 2017 at 23:29, jkbradley wrote: > *@jkbradley* commented on this pull

[GitHub] spark issue #19904: [SPARK-22707][ML] Optimize CrossValidator memory occupat...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19904 **[Test build #4024 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4024/testReport)** for PR 19904 at commit

[GitHub] spark pull request #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator f...

2017-12-24 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19527#discussion_r158618991 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/OneHotEncoderEstimator.scala --- @@ -0,0 +1,479 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2017-12-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20023 Following ANSI SQL compliance sounds good to me. However, many details are vendor-specific. That means, the query results still varies even if we can be 100% ANSI SQL compliant. To

[GitHub] spark issue #19643: [SPARK-11421][CORE][PYTHON][R] Added ability for addJar ...

2017-12-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19643 Let me leave this closed now and will reopen when I am ready to proceed. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19643: [SPARK-11421][CORE][PYTHON][R] Added ability for ...

2017-12-24 Thread HyukjinKwon
Github user HyukjinKwon closed the pull request at: https://github.com/apache/spark/pull/19643 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20068: [SPARK-17916][SQL] Fix empty string being parsed ...

2017-12-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20068#discussion_r158616812 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1248,4 +1248,49 @@ class CSVSuite extends

[GitHub] spark pull request #20068: [SPARK-17916][SQL] Fix empty string being parsed ...

2017-12-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20068#discussion_r158616834 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1248,4 +1248,49 @@ class CSVSuite extends

[GitHub] spark pull request #20068: [SPARK-17916][SQL] Fix empty string being parsed ...

2017-12-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20068#discussion_r158616740 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVOptions.scala --- @@ -152,7 +152,7 @@ class CSVOptions(

[GitHub] spark pull request #20068: [SPARK-17916][SQL] Fix empty string being parsed ...

2017-12-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20068#discussion_r158616591 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1248,4 +1248,49 @@ class CSVSuite extends

[GitHub] spark pull request #20068: [SPARK-17916][SQL] Fix empty string being parsed ...

2017-12-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20068#discussion_r158617095 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1248,4 +1248,49 @@ class CSVSuite extends

[GitHub] spark pull request #20068: [SPARK-17916][SQL] Fix empty string being parsed ...

2017-12-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20068#discussion_r158616912 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1248,4 +1248,49 @@ class CSVSuite extends

[GitHub] spark pull request #20068: [SPARK-17916][SQL] Fix empty string being parsed ...

2017-12-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20068#discussion_r158616872 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1248,4 +1248,49 @@ class CSVSuite extends

[GitHub] spark issue #20072: [SPARK-22790][SQL] add a configurable factor to describe...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20072 **[Test build #85363 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85363/testReport)** for PR 20072 at commit

[GitHub] spark pull request #20072: [SPARK-22790][SQL] add a configurable factor to d...

2017-12-24 Thread CodingCat
GitHub user CodingCat opened a pull request: https://github.com/apache/spark/pull/20072 [SPARK-22790][SQL] add a configurable factor to describe HadoopFsRelation's size ## What changes were proposed in this pull request? as per discussion in

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-12-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19813 IMHO, in general, the output `ev.value` would be declared as local variable by parent as ``` s"""${ctx.javaType(dataType)} ${ev.value} = ${ctx.defaultValue(dataType)}; ``` Such as

[GitHub] spark issue #20067: [SPARK-22894][SQL] DateTimeOperations should accept SQL ...

2017-12-24 Thread gczsjdy
Github user gczsjdy commented on the issue: https://github.com/apache/spark/pull/20067 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator f...

2017-12-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19527#discussion_r158615298 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/OneHotEncoderEstimator.scala --- @@ -0,0 +1,479 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator f...

2017-12-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19527#discussion_r158615273 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/OneHotEncoderEstimator.scala --- @@ -0,0 +1,479 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19977: [SPARK-22771][SQL] Concatenate binary inputs into a bina...

2017-12-24 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19977 @gatorsmile @gatorsmile ping --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19675: [SPARK-14540][BUILD] Support Scala 2.12 closures and Jav...

2017-12-24 Thread jvican
Github user jvican commented on the issue: https://github.com/apache/spark/pull/19675 Is this issue partially or finally fixed? I could try to get this to the finish line for a nice start of the year. --- - To

[GitHub] spark pull request #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator f...

2017-12-24 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19527#discussion_r158610281 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/OneHotEncoderEstimator.scala --- @@ -0,0 +1,479 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19683 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19683 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85362/ Test FAILed. ---

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19683 **[Test build #85362 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85362/testReport)** for PR 19683 at commit

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19683 **[Test build #85362 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85362/testReport)** for PR 19683 at commit

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19683 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19683 **[Test build #85361 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85361/testReport)** for PR 19683 at commit

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19683 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85361/ Test FAILed. ---

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19683 **[Test build #85361 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85361/testReport)** for PR 19683 at commit

[GitHub] spark pull request #20002: [SPARK-22465][Core] Add a safety-check to RDD def...

2017-12-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20002 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20002: [SPARK-22465][Core] Add a safety-check to RDD defaultPar...

2017-12-24 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/20002 Merged, thanks for fixing this @sujithjay ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20068: [SPARK-17916][SQL] Fix empty string being parsed ...

2017-12-24 Thread aa8y
Github user aa8y commented on a diff in the pull request: https://github.com/apache/spark/pull/20068#discussion_r158606107 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVOptions.scala --- @@ -152,7 +152,7 @@ class CSVOptions(

[GitHub] spark issue #20002: [SPARK-22465][Core][WIP] Add a safety-check to RDD defau...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20002 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20002: [SPARK-22465][Core][WIP] Add a safety-check to RDD defau...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20002 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85360/ Test PASSed. ---

[GitHub] spark issue #20002: [SPARK-22465][Core][WIP] Add a safety-check to RDD defau...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20002 **[Test build #85360 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85360/testReport)** for PR 20002 at commit

[GitHub] spark issue #19929: [SPARK-22629][PYTHON] Add deterministic flag to pyspark ...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19929 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85359/ Test PASSed. ---

[GitHub] spark issue #19929: [SPARK-22629][PYTHON] Add deterministic flag to pyspark ...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19929 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19929: [SPARK-22629][PYTHON] Add deterministic flag to pyspark ...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19929 **[Test build #85359 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85359/testReport)** for PR 19929 at commit

[GitHub] spark issue #20002: [SPARK-22465][Core][WIP] Add a safety-check to RDD defau...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20002 **[Test build #85360 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85360/testReport)** for PR 20002 at commit

[GitHub] spark issue #20002: [SPARK-22465][Core][WIP] Add a safety-check to RDD defau...

2017-12-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20002 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20002: [SPARK-22465][Core][WIP] Add a safety-check to RDD defau...

2017-12-24 Thread sujithjay
Github user sujithjay commented on the issue: https://github.com/apache/spark/pull/20002 The failed unit test (in HistoryServerSuite.scala) seems unrelated to this PR. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19929: [SPARK-22629][PYTHON] Add deterministic flag to pyspark ...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19929 **[Test build #85359 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85359/testReport)** for PR 19929 at commit

[GitHub] spark issue #19929: [SPARK-22629][PYTHON] Add deterministic flag to pyspark ...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19929 **[Test build #85358 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85358/testReport)** for PR 19929 at commit

[GitHub] spark issue #19929: [SPARK-22629][PYTHON] Add deterministic flag to pyspark ...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19929 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19929: [SPARK-22629][PYTHON] Add deterministic flag to pyspark ...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19929 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85358/ Test FAILed. ---

[GitHub] spark issue #20071: SPARK-22896 Improvement in String interpolation | Graphx

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20071 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20071: SPARK-22896 Improvement in String interpolation |...

2017-12-24 Thread chetkhatri
GitHub user chetkhatri opened a pull request: https://github.com/apache/spark/pull/20071 SPARK-22896 Improvement in String interpolation | Graphx ## What changes were proposed in this pull request? * String interpolation in scala style corrected. ## How was this patch

[GitHub] spark issue #19929: [SPARK-22629][PYTHON] Add deterministic flag to pyspark ...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19929 **[Test build #85358 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85358/testReport)** for PR 19929 at commit

[GitHub] spark issue #20069: [SPARK-22895] [SQL] Push down the deterministic predicat...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20069 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85356/ Test PASSed. ---

[GitHub] spark issue #20069: [SPARK-22895] [SQL] Push down the deterministic predicat...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20069 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20069: [SPARK-22895] [SQL] Push down the deterministic predicat...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20069 **[Test build #85356 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85356/testReport)** for PR 20069 at commit

[GitHub] spark issue #20002: [SPARK-22465][Core][WIP] Add a safety-check to RDD defau...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20002 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20002: [SPARK-22465][Core][WIP] Add a safety-check to RDD defau...

2017-12-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20002 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85357/ Test FAILed. ---

[GitHub] spark issue #20002: [SPARK-22465][Core][WIP] Add a safety-check to RDD defau...

2017-12-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20002 **[Test build #85357 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85357/testReport)** for PR 20002 at commit

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2017-12-24 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20023 Thanks for your analysis @gatorsmile. Actually the rule you specified for Oracle is what it uses when casting, rather then when doing arithmetic operations. Yes DB2 has rather different rules to

  1   2   >