[GitHub] spark issue #23040: [SPARK-26068][Core]ChunkedByteBufferInputStream should h...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23040 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23040: [SPARK-26068][Core]ChunkedByteBufferInputStream should h...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23040 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98989/ Test PASSed. ---

[GitHub] spark issue #23040: [SPARK-26068][Core]ChunkedByteBufferInputStream should h...

2018-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23040 **[Test build #98989 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98989/testReport)** for PR 23040 at commit

[GitHub] spark issue #23025: [SPARK-26024][SQL]: Update documentation for repartition...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23025 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23025: [SPARK-26024][SQL]: Update documentation for repartition...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23025 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23025: [SPARK-26024][SQL]: Update documentation for repartition...

2018-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23025 **[Test build #98992 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98992/testReport)** for PR 23025 at commit

[GitHub] spark pull request #23025: [SPARK-26024][SQL]: Update documentation for repa...

2018-11-18 Thread JulienPeloton
Github user JulienPeloton commented on a diff in the pull request: https://github.com/apache/spark/pull/23025#discussion_r234512932 --- Diff: python/pyspark/sql/dataframe.py --- @@ -732,6 +732,11 @@ def repartitionByRange(self, numPartitions, *cols): At least one

[GitHub] spark pull request #23025: [SPARK-26024][SQL]: Update documentation for repa...

2018-11-18 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/23025#discussion_r234511357 --- Diff: python/pyspark/sql/dataframe.py --- @@ -732,6 +732,11 @@ def repartitionByRange(self, numPartitions, *cols): At least one partition-by

[GitHub] spark issue #23025: [SPARK-26024][SQL]: Update documentation for repartition...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23025 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23025: [SPARK-26024][SQL]: Update documentation for repartition...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23025 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23025: [SPARK-26024][SQL]: Update documentation for repa...

2018-11-18 Thread JulienPeloton
Github user JulienPeloton commented on a diff in the pull request: https://github.com/apache/spark/pull/23025#discussion_r234509708 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2789,6 +2789,12 @@ class Dataset[T] private[sql]( * When no

[GitHub] spark issue #23025: [SPARK-26024][SQL]: Update documentation for repartition...

2018-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23025 **[Test build #98991 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98991/testReport)** for PR 23025 at commit

[GitHub] spark pull request #23079: [SPARK-26107][SQL] Extend ReplaceNullWithFalseInP...

2018-11-18 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/23079#discussion_r234508866 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceNullWithFalseInPredicateSuite.scala --- @@ -298,6 +299,45 @@

[GitHub] spark pull request #23079: [SPARK-26107][SQL] Extend ReplaceNullWithFalseInP...

2018-11-18 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/23079#discussion_r234508561 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -767,6 +767,15 @@ object ReplaceNullWithFalse

[GitHub] spark issue #23082: [SPARK-26112][SQL] Update since versions of new built-in...

2018-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23082 **[Test build #98990 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98990/testReport)** for PR 23082 at commit

[GitHub] spark issue #23082: [SPARK-26112][SQL] Update since versions of new built-in...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23082 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23082: [SPARK-26112][SQL] Update since versions of new built-in...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23082 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23082: [SPARK-26112][SQL] Update since versions of new built-in...

2018-11-18 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/23082 cc @cloud-fan @gatorsmile @dongjoon-hyun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23082: [SPARK-26112][SQL] Update since versions of new b...

2018-11-18 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/23082 [SPARK-26112][SQL] Update since versions of new built-in functions. ## What changes were proposed in this pull request? The following 5 functions were removed from branch-2.4: -

[GitHub] spark issue #23045: [SPARK-26071][SQL] disallow map as map key

2018-11-18 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/23045 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #23045: [SPARK-26071][SQL] disallow map as map key

2018-11-18 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/23045#discussion_r234502854 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -521,13 +521,18 @@ case class

[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of non-struct type unde...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23054 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of non-struct type unde...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23054 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98988/ Test PASSed. ---

[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of non-struct type unde...

2018-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23054 **[Test build #98988 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98988/testReport)** for PR 23054 at commit

[GitHub] spark issue #23043: [SPARK-26021][SQL] replace minus zero with zero in Unsaf...

2018-11-18 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/23043 Do we need to consider `GenerateSafeProjection`, too? In other words, if the generated code or runtime does not use data in `Unsafe`, this `+0.0/-0.0` problem may still exist. Am I correct?

[GitHub] spark issue #23025: [SPARK-26024][SQL]: Update documentation for repartition...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23025 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98987/ Test PASSed. ---

[GitHub] spark issue #23025: [SPARK-26024][SQL]: Update documentation for repartition...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23025 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23025: [SPARK-26024][SQL]: Update documentation for repartition...

2018-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23025 **[Test build #98987 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98987/testReport)** for PR 23025 at commit

[GitHub] spark issue #23076: [SPARK-26103][SQL] Added maxDepth to limit the length of...

2018-11-18 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/23076 I'm seeing both sides of needs: while I think dumping full plan into file is a good feature for debugging specific issue, retaining full plans for representing them to UI page have been a

[GitHub] spark pull request #23045: [SPARK-26071][SQL] disallow map as map key

2018-11-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/23045#discussion_r234494542 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -521,13 +521,18 @@ case class

[GitHub] spark issue #23040: [SPARK-26068][Core]ChunkedByteBufferInputStream should h...

2018-11-18 Thread LinhongLiu
Github user LinhongLiu commented on the issue: https://github.com/apache/spark/pull/23040 cc @cloud-fan @srowen review is fixed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23040: [SPARK-26068][Core]ChunkedByteBufferInputStream should h...

2018-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23040 **[Test build #98989 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98989/testReport)** for PR 23040 at commit

[GitHub] spark issue #23058: [SPARK-25905][CORE] When getting a remote block, avoid f...

2018-11-18 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/23058 @attilapiros can you review this please? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23027: [SPARK-26049][SQL][TEST] FilterPushdownBenchmark ...

2018-11-18 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/23027#discussion_r234482766 --- Diff: sql/core/benchmarks/FilterPushdownBenchmark-results.txt --- @@ -2,669 +2,809 @@ Pushdown for many distinct value case

[GitHub] spark pull request #23045: [SPARK-26071][SQL] disallow map as map key

2018-11-18 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/23045#discussion_r234481249 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -521,13 +521,18 @@ case class

[GitHub] spark issue #23081: [SPARK-26109][WebUI]Duration in the task summary metrics...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23081 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98984/ Test FAILed. ---

[GitHub] spark issue #23081: [SPARK-26109][WebUI]Duration in the task summary metrics...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23081 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23081: [SPARK-26109][WebUI]Duration in the task summary metrics...

2018-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23081 **[Test build #98984 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98984/testReport)** for PR 23081 at commit

[GitHub] spark issue #21363: [SPARK-19228][SQL] Migrate on Java 8 time from FastDateF...

2018-11-18 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/21363 @MaxGekk Sorry for the late, something inserted in the my scheduler, I plan to start this PR in this weekend, if its too late please just take it, sorry for the late again. ---

[GitHub] spark issue #23043: [SPARK-26021][SQL] replace minus zero with zero in Unsaf...

2018-11-18 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/23043 Is it better to update this PR title now? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23054: [SPARK-26085][SQL] Key attribute of non-struct ty...

2018-11-18 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/23054#discussion_r234477289 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/KeyValueGroupedDataset.scala --- @@ -459,7 +460,11 @@ class KeyValueGroupedDataset[K, V]

[GitHub] spark issue #23043: [SPARK-26021][SQL] replace minus zero with zero in Unsaf...

2018-11-18 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/23043 @srowen #21794 is what I thought. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #23080: [SPARK-26108][SQL] Support custom lineSep in CSV datasou...

2018-11-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23080 Ah, also, `CsvParser.beginParsing` takes an additional argument `Charset`. It should rather be easily able to support encoding in `multiLine`. @MaxGekk, would you be able to find some time to

[GitHub] spark pull request #23054: [SPARK-26085][SQL] Key attribute of non-struct ty...

2018-11-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/23054#discussion_r234476607 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/KeyValueGroupedDataset.scala --- @@ -459,7 +460,11 @@ class KeyValueGroupedDataset[K, V]

[GitHub] spark pull request #23080: [SPARK-26108][SQL] Support custom lineSep in CSV ...

2018-11-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23080#discussion_r234476318 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/CSVOptions.scala --- @@ -192,6 +192,20 @@ class CSVOptions( */

[GitHub] spark pull request #23043: [SPARK-26021][SQL] replace minus zero with zero i...

2018-11-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/23043#discussion_r234476361 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala --- @@ -723,4 +723,32 @@ class DataFrameAggregateSuite extends

[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of non-struct type unde...

2018-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23054 **[Test build #98988 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98988/testReport)** for PR 23054 at commit

[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of non-struct type unde...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23054 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of non-struct type unde...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23054 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21888: [SPARK-24253][SQL][WIP] Implement DeleteFrom for v2 tabl...

2018-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21888 **[Test build #98986 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98986/testReport)** for PR 21888 at commit

[GitHub] spark issue #21888: [SPARK-24253][SQL][WIP] Implement DeleteFrom for v2 tabl...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21888 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98986/ Test FAILed. ---

[GitHub] spark issue #21888: [SPARK-24253][SQL][WIP] Implement DeleteFrom for v2 tabl...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21888 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #23043: [SPARK-26021][SQL] replace minus zero with zero i...

2018-11-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/23043#discussion_r234475978 --- Diff: common/unsafe/src/test/java/org/apache/spark/unsafe/PlatformUtilSuite.java --- @@ -157,4 +159,15 @@ public void heapMemoryReuse() {

[GitHub] spark pull request #23043: [SPARK-26021][SQL] replace minus zero with zero i...

2018-11-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/23043#discussion_r234476055 --- Diff: common/unsafe/src/test/java/org/apache/spark/unsafe/PlatformUtilSuite.java --- @@ -157,4 +159,15 @@ public void heapMemoryReuse() {

[GitHub] spark pull request #23043: [SPARK-26021][SQL] replace minus zero with zero i...

2018-11-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/23043#discussion_r234475858 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java --- @@ -120,6 +120,9 @@ public static float getFloat(Object object, long

[GitHub] spark issue #23025: [SPARK-26024][SQL]: Update documentation for repartition...

2018-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23025 **[Test build #98987 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98987/testReport)** for PR 23025 at commit

[GitHub] spark issue #23025: [SPARK-26024][SQL]: Update documentation for repartition...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23025 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23025: [SPARK-26024][SQL]: Update documentation for repartition...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23025 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23080: [SPARK-26108][SQL] Support custom lineSep in CSV ...

2018-11-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23080#discussion_r234475595 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/CSVOptions.scala --- @@ -192,6 +192,20 @@ class CSVOptions( */

[GitHub] spark pull request #23025: [SPARK-26024][SQL]: Update documentation for repa...

2018-11-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/23025#discussion_r234475550 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2789,6 +2789,12 @@ class Dataset[T] private[sql]( * When no explicit

[GitHub] spark pull request #23054: [SPARK-26085][SQL] Key attribute of primitive typ...

2018-11-18 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/23054#discussion_r234475488 --- Diff: docs/sql-migration-guide-upgrade.md --- @@ -17,6 +17,9 @@ displayTitle: Spark SQL Upgrading Guide - The `ADD JAR` command previously

[GitHub] spark issue #23025: [SPARK-26024][SQL]: Update documentation for repartition...

2018-11-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23025 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #23054: [SPARK-26085][SQL] Key attribute of primitive typ...

2018-11-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/23054#discussion_r234475321 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1594,6 +1594,15 @@ object SQLConf { "WHERE, which

[GitHub] spark pull request #23080: [SPARK-26108][SQL] Support custom lineSep in CSV ...

2018-11-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23080#discussion_r234475228 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/CSVOptions.scala --- @@ -192,6 +192,20 @@ class CSVOptions( */

[GitHub] spark pull request #23054: [SPARK-26085][SQL] Key attribute of primitive typ...

2018-11-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/23054#discussion_r234475156 --- Diff: docs/sql-migration-guide-upgrade.md --- @@ -17,6 +17,9 @@ displayTitle: Spark SQL Upgrading Guide - The `ADD JAR` command

[GitHub] spark issue #21888: [SPARK-24253][SQL][WIP] Implement DeleteFrom for v2 tabl...

2018-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21888 **[Test build #98986 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98986/testReport)** for PR 21888 at commit

[GitHub] spark pull request #23079: [SPARK-26107][SQL] Extend ReplaceNullWithFalseInP...

2018-11-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/23079#discussion_r234474562 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -767,6 +767,15 @@ object ReplaceNullWithFalse

[GitHub] spark pull request #23077: [SPARK-26105][PYTHON] Clean unittest2 imports up ...

2018-11-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/23077 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #23077: [SPARK-26105][PYTHON] Clean unittest2 imports up that we...

2018-11-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23077 Merged to master. Thanks for reviewing this, @BryanCutler and @srowen. --- - To unsubscribe, e-mail:

[GitHub] spark issue #23077: [SPARK-25344][PYTHON] Clean unittest2 imports up that we...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23077 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98985/ Test PASSed. ---

[GitHub] spark issue #23077: [SPARK-25344][PYTHON] Clean unittest2 imports up that we...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23077 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23077: [SPARK-25344][PYTHON] Clean unittest2 imports up that we...

2018-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23077 **[Test build #98985 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98985/testReport)** for PR 23077 at commit

[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of primitive type under...

2018-11-18 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/23054 For non-primitive types there is a struct named "key". --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #23077: [SPARK-25344][PYTHON] Clean unittest2 imports up that we...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23077 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23077: [SPARK-25344][PYTHON] Clean unittest2 imports up that we...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23077 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23077: [SPARK-25344][PYTHON] Clean unittest2 imports up that we...

2018-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23077 **[Test build #98985 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98985/testReport)** for PR 23077 at commit

[GitHub] spark issue #23077: [SPARK-25344][PYTHON] Clean unittest2 imports up that we...

2018-11-18 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/23077 Oh, I think the PR title should be SPARK-26105 too --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #23081: [SPARK-26109][WebUI]Duration in the task summary metrics...

2018-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23081 **[Test build #98984 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98984/testReport)** for PR 23081 at commit

[GitHub] spark issue #23081: [SPARK-26109][WebUI]Duration in the task summary metrics...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23081 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23081: [SPARK-26109][WebUI]Duration in the task summary metrics...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23081 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23077: [SPARK-25344][PYTHON] Clean unittest2 imports up that we...

2018-11-18 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/23077 >BTW, Bryan, do you have some time to work on the has_numpy stuff Yup, I can do that --- - To unsubscribe, e-mail:

[GitHub] spark issue #23077: [SPARK-25344][PYTHON] Clean unittest2 imports up that we...

2018-11-18 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/23077 Oops, actually I think there is one more here https://github.com/apache/spark/blob/master/python/pyspark/testing/mllibutils.py#L20 Other than that, looks good ---

[GitHub] spark pull request #23081: [SPARK-26109][WebUI]Duration in the task summary ...

2018-11-18 Thread shahidki31
GitHub user shahidki31 opened a pull request: https://github.com/apache/spark/pull/23081 [SPARK-26109][WebUI]Duration in the task summary metrics table and the task table are different ## What changes were proposed in this pull request? Task summary displays the summary of the

[GitHub] spark pull request #23079: [SPARK-26107][SQL] Extend ReplaceNullWithFalseInP...

2018-11-18 Thread aokolnychyi
Github user aokolnychyi commented on a diff in the pull request: https://github.com/apache/spark/pull/23079#discussion_r234467085 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceNullWithFalseInPredicateSuite.scala --- @@ -298,6 +299,45 @@

[GitHub] spark issue #23079: [SPARK-26107][SQL] Extend ReplaceNullWithFalseInPredicat...

2018-11-18 Thread aokolnychyi
Github user aokolnychyi commented on the issue: https://github.com/apache/spark/pull/23079 @rednaxelafx I am glad the rule gets more adoption. Renaming also makes sense to me. Shall we extend `ReplaceNullWithFalseEndToEndSuite` as well? ---

[GitHub] spark issue #23065: [SPARK-26090][CORE][SQL][ML] Resolve most miscellaneous ...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23065 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98983/ Test PASSed. ---

[GitHub] spark issue #23065: [SPARK-26090][CORE][SQL][ML] Resolve most miscellaneous ...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23065 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23065: [SPARK-26090][CORE][SQL][ML] Resolve most miscellaneous ...

2018-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23065 **[Test build #98983 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98983/testReport)** for PR 23065 at commit

[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of primitive type under...

2018-11-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/23054 BTW what does the non-primitive types look like? Do they get flattened, or is there a strict? --- - To unsubscribe, e-mail:

[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of primitive type under...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23054 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98981/ Test FAILed. ---

[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of primitive type under...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23054 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of primitive type under...

2018-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23054 **[Test build #98981 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98981/testReport)** for PR 23054 at commit

[GitHub] spark issue #23080: [SPARK-26108][SQL] Support custom lineSep in CSV datasou...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23080 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98982/ Test PASSed. ---

[GitHub] spark issue #23080: [SPARK-26108][SQL] Support custom lineSep in CSV datasou...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23080 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23080: [SPARK-26108][SQL] Support custom lineSep in CSV datasou...

2018-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23080 **[Test build #98982 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98982/testReport)** for PR 23080 at commit

[GitHub] spark issue #23075: [SPARK-26084][SQL] Fixes unresolved AggregateExpression....

2018-11-18 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/23075 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23075: [SPARK-26084][SQL] Fixes unresolved AggregateExpression....

2018-11-18 Thread ssimeonov
Github user ssimeonov commented on the issue: https://github.com/apache/spark/pull/23075 @MaxGekk done --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #23073: [SPARK-26104] [Hydrogen] expose pci info to task ...

2018-11-18 Thread chenqin
Github user chenqin commented on a diff in the pull request: https://github.com/apache/spark/pull/23073#discussion_r234453776 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/ExecutorData.scala --- @@ -27,12 +27,14 @@ import org.apache.spark.rpc.{RpcAddress,

[GitHub] spark issue #23065: [SPARK-26090][CORE][SQL][ML] Resolve most miscellaneous ...

2018-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23065 **[Test build #98983 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98983/testReport)** for PR 23065 at commit

[GitHub] spark issue #23065: [SPARK-26090][CORE][SQL][ML] Resolve most miscellaneous ...

2018-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23065 Test PASSed. Refer to this link for build results (access rights to CI server needed):

  1   2   >