[GitHub] spark issue #19570: [SPARK-22335][SQL] Clarify union behavior on Dataset of ...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19570 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19560: [SPARK-22334][SQL] Check table size from filesystem in c...

2017-10-24 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/19560 >My main concern is, we'd better not to put burden on Spark to deal with metastore failures I think this make sense. I was also thinking about this when proposing this pr. I do agree

[GitHub] spark issue #19570: [SPARK-22335][SQL] Clarify union behavior on Dataset of ...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19570 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83029/ Test PASSed. ---

[GitHub] spark issue #19570: [SPARK-22335][SQL] Clarify union behavior on Dataset of ...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19570 **[Test build #83029 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83029/testReport)** for PR 19570 at commit

[GitHub] spark pull request #19556: [SPARK-22328][Core] ClosureCleaner should not mis...

2017-10-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19556#discussion_r146760101 --- Diff: core/src/main/scala/org/apache/spark/util/ClosureCleaner.scala --- @@ -91,6 +91,50 @@ private[spark] object ClosureCleaner extends Logging {

[GitHub] spark pull request #19557: [SPARK-22281][SPARKR][WIP] Handle R method breaki...

2017-10-24 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/19557#discussion_r146760035 --- Diff: R/pkg/R/DataFrame.R --- @@ -3249,9 +3249,12 @@ setMethod("as.data.frame", #' @note attach since 1.6.0 setMethod("attach",

[GitHub] spark issue #19390: [SPARK-18935][MESOS] Fix dynamic reservations on mesos

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19390 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19390: [SPARK-18935][MESOS] Fix dynamic reservations on mesos

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19390 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83028/ Test PASSed. ---

[GitHub] spark issue #19390: [SPARK-18935][MESOS] Fix dynamic reservations on mesos

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19390 **[Test build #83028 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83028/testReport)** for PR 19390 at commit

[GitHub] spark issue #19550: [SPARK-22327][SPARKR][TEST][BACKPORT-2.0] check for vers...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19550 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19550: [SPARK-22327][SPARKR][TEST][BACKPORT-2.0] check for vers...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19550 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83033/ Test PASSed. ---

[GitHub] spark issue #19550: [SPARK-22327][SPARKR][TEST][BACKPORT-2.0] check for vers...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19550 **[Test build #83033 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83033/consoleFull)** for PR 19550 at commit

[GitHub] spark issue #19560: [SPARK-22334][SQL] Check table size from filesystem in c...

2017-10-24 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/19560 My main concern is, we'd better not to put burden on Spark to deal with metastore failures, because Spark doesn't have control on metastores. The system using Spark and metastore should be

[GitHub] spark pull request #19556: [SPARK-22328][Core] ClosureCleaner should not mis...

2017-10-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19556#discussion_r146759438 --- Diff: core/src/main/scala/org/apache/spark/util/ClosureCleaner.scala --- @@ -91,6 +91,50 @@ private[spark] object ClosureCleaner extends Logging {

[GitHub] spark pull request #19569: [SPARK-22348][SQL] The table cache providing Colu...

2017-10-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19569 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19569: [SPARK-22348][SQL] The table cache providing ColumnarBat...

2017-10-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19569 good catch! merging to master, thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19560: [SPARK-22334][SQL] Check table size from filesystem in c...

2017-10-24 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/19560 > Users always do not know there's error in stats. Isn't there any exceptions or error messages when updating table/stats fails? I suppose the system is able to know it through logging or

[GitHub] spark issue #19433: [SPARK-3162] [MLlib] Add local tree training for decisio...

2017-10-24 Thread smurching
Github user smurching commented on the issue: https://github.com/apache/spark/pull/19433 @WeichenXu123 Thanks for the comments! I'll respond inline: > In your doc, you said "Specifically, we only need to store sufficient stats for each bin of a single feature, as opposed to

[GitHub] spark pull request #19557: [SPARK-22281][SPARKR][WIP] Handle R method breaki...

2017-10-24 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/19557#discussion_r146757169 --- Diff: R/pkg/R/DataFrame.R --- @@ -3249,9 +3249,12 @@ setMethod("as.data.frame", #' @note attach since 1.6.0 setMethod("attach",

[GitHub] spark pull request #19557: [SPARK-22281][SPARKR][WIP] Handle R method breaki...

2017-10-24 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/19557#discussion_r146757193 --- Diff: R/pkg/R/DataFrame.R --- @@ -3249,9 +3249,12 @@ setMethod("as.data.frame", #' @note attach since 1.6.0 setMethod("attach",

[GitHub] spark pull request #19557: [SPARK-22281][SPARKR][WIP] Handle R method breaki...

2017-10-24 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/19557#discussion_r146757250 --- Diff: R/run-tests.sh --- @@ -38,6 +38,7 @@ FAILED=$((PIPESTATUS[0]||$FAILED)) NUM_CRAN_WARNING="$(grep -c WARNING$ $CRAN_CHECK_LOG_FILE)"

[GitHub] spark pull request #19568: SPARK-22345: Fix sort-merge joins with conditions...

2017-10-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19568#discussion_r146757237 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala --- @@ -585,21 +585,26 @@ case class SortMergeJoinExec(

[GitHub] spark pull request #19568: SPARK-22345: Fix sort-merge joins with conditions...

2017-10-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19568#discussion_r146756914 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala --- @@ -585,21 +585,26 @@ case class SortMergeJoinExec(

[GitHub] spark issue #19550: [SPARK-22327][SPARKR][TEST][BACKPORT-2.0] check for vers...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19550 **[Test build #83033 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83033/consoleFull)** for PR 19550 at commit

[GitHub] spark issue #19550: [SPARK-22327][SPARKR][TEST][BACKPORT-2.0] check for vers...

2017-10-24 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19550 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19568: SPARK-22345: Fix sort-merge joins with conditions...

2017-10-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19568#discussion_r146755690 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala --- @@ -615,6 +620,7 @@ case class SortMergeJoinExec(

[GitHub] spark issue #19569: [SPARK-22348][SQL] The table cache providing ColumnarBat...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19569 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83027/ Test PASSed. ---

[GitHub] spark issue #19569: [SPARK-22348][SQL] The table cache providing ColumnarBat...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19569 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19569: [SPARK-22348][SQL] The table cache providing ColumnarBat...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19569 **[Test build #83027 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83027/testReport)** for PR 19569 at commit

[GitHub] spark issue #19480: [SPARK-22226][SQL] splitExpression can create too many m...

2017-10-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19480 Sounds good to me. Sorry for being late since I was busy last week. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19571: [SPARK-15474][SQL] Write and read back non-emtpy ...

2017-10-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19571#discussion_r146752367 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala --- @@ -252,6 +253,13 @@ private[orc] class OrcOutputWriter(

[GitHub] spark pull request #19571: [SPARK-15474][SQL] Write and read back non-emtpy ...

2017-10-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19571#discussion_r146752170 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala --- @@ -252,6 +253,13 @@ private[orc] class OrcOutputWriter(

[GitHub] spark issue #19569: [SPARK-22348][SQL] The table cache providing ColumnarBat...

2017-10-24 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19569 also cc @cloud-fan for review. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19571: [SPARK-15474][SQL] Write and read back non-emtpy ...

2017-10-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19571#discussion_r146751242 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala --- @@ -39,4 +45,33 @@ private[sql] object

[GitHub] spark issue #19569: [SPARK-22348][SQL] The table cache providing ColumnarBat...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19569 **[Test build #83032 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83032/testReport)** for PR 19569 at commit

[GitHub] spark issue #19571: [SPARK-15474][SQL] Write and read back non-emtpy schema ...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19571 **[Test build #83030 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83030/testReport)** for PR 19571 at commit

[GitHub] spark issue #19569: [SPARK-22348][SQL] The table cache providing ColumnarBat...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19569 **[Test build #83031 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83031/testReport)** for PR 19569 at commit

[GitHub] spark pull request #19571: [SPARK-15474][SQL] Write and read back non-emtpy ...

2017-10-24 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/19571 [SPARK-15474][SQL] Write and read back non-emtpy schema with empty dataframe ## What changes were proposed in this pull request? Previously, ORC file format cannot write a correct

[GitHub] spark pull request #19569: [SPARK-22348][SQL] The table cache providing Colu...

2017-10-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19569#discussion_r146749535 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -201,35 +193,50 @@ case class

[GitHub] spark pull request #19569: [SPARK-22348][SQL] The table cache providing Colu...

2017-10-24 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/19569#discussion_r146748032 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -201,35 +193,50 @@ case class

[GitHub] spark pull request #18664: [SPARK-21375][PYSPARK][SQL] Add Date and Timestam...

2017-10-24 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18664#discussion_r146747982 --- Diff: python/pyspark/serializers.py --- @@ -224,7 +225,13 @@ def _create_batch(series): # If a nullable integer series has been promoted to

[GitHub] spark issue #19570: [SPARK-22335][SQL] Clarify union behavior on Dataset of ...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19570 **[Test build #83029 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83029/testReport)** for PR 19570 at commit

[GitHub] spark pull request #19570: [SPARK-22335][SQL] Clarify union behavior on Data...

2017-10-24 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/19570 [SPARK-22335][SQL] Clarify union behavior on Dataset of typed objects in the document ## What changes were proposed in this pull request? Seems that end users can be confused by the

[GitHub] spark issue #19560: [SPARK-22334][SQL] Check table size from filesystem in c...

2017-10-24 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/19560 @wzhfy Thanks for comment; I know your point. In my cluster, namenode is under heavy pressure. Errors in stats happen with big chance. Users always do not know there's error in stats.

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19568 Could you please change title from `SPARK-22345` to `[SPARK-22345]`? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19568: SPARK-22345: Fix sort-merge joins with conditions...

2017-10-24 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19568#discussion_r146743704 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala --- @@ -615,6 +620,7 @@ case class SortMergeJoinExec(

[GitHub] spark issue #19563: [SPARK-22284][SQL] Fix 64KB JVM bytecode limit problem i...

2017-10-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19563 ping @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19569: [SPARK-22348][SQL] The table cache providing ColumnarBat...

2017-10-24 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19569 @kiszk Thanks. I've roughly checked existing tests. Seems that there are related ones for pruning the table cache. Let me see if I can add one. ---

[GitHub] spark issue #19390: [SPARK-18935][MESOS] Fix dynamic reservations on mesos

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19390 **[Test build #83028 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83028/testReport)** for PR 19390 at commit

[GitHub] spark issue #19569: [SPARK-22348][SQL] The table cache providing ColumnarBat...

2017-10-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19569 Good catch, thank you. Would it be possible to add a test case for pruning with table cache? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19516: [SPARK-22277][ML]fix the bug of ChiSqSelector on ...

2017-10-24 Thread mpjlu
Github user mpjlu commented on a diff in the pull request: https://github.com/apache/spark/pull/19516#discussion_r146741139 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/ChiSqSelector.scala --- @@ -291,9 +291,13 @@ final class ChiSqSelectorModel private[ml] (

[GitHub] spark issue #19569: [SPARK-22348][SQL] The table cache providing ColumnarBat...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19569 **[Test build #83027 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83027/testReport)** for PR 19569 at commit

[GitHub] spark issue #19390: [SPARK-18935][MESOS] Fix dynamic reservations on mesos

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19390 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19390: [SPARK-18935][MESOS] Fix dynamic reservations on mesos

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19390 **[Test build #83026 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83026/testReport)** for PR 19390 at commit

[GitHub] spark issue #19390: [SPARK-18935][MESOS] Fix dynamic reservations on mesos

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19390 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83026/ Test FAILed. ---

[GitHub] spark pull request #19569: [SPARK-22348][SQL] The table cache providing Colu...

2017-10-24 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/19569 [SPARK-22348][SQL] The table cache providing ColumnarBatch should also do partition batch pruning ## What changes were proposed in this pull request? We enable table cache

[GitHub] spark issue #19390: [SPARK-18935][MESOS] Fix dynamic reservations on mesos

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19390 **[Test build #83026 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83026/testReport)** for PR 19390 at commit

[GitHub] spark issue #19390: [SPARK-18935][MESOS] Fix dynamic reservations on mesos

2017-10-24 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/19390 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19567: [SPARK-22291] Postgresql UUID[] to Cassandra: Conversion...

2017-10-24 Thread jmchung
Github user jmchung commented on the issue: https://github.com/apache/spark/pull/19567 Thanks @wangyum and @viirya, I'll add the corresponding tests in `PostgresIntegrationSuite`. To @viirya , I'm not sure if the other data types will work, will consider them into tests, thanks!

[GitHub] spark issue #19433: [SPARK-3162] [MLlib] Add local tree training for decisio...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19433 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19433: [SPARK-3162] [MLlib] Add local tree training for decisio...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19433 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83025/ Test PASSed. ---

[GitHub] spark issue #19433: [SPARK-3162] [MLlib] Add local tree training for decisio...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19433 **[Test build #83025 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83025/testReport)** for PR 19433 at commit

[GitHub] spark issue #19560: [SPARK-22334][SQL] Check table size from filesystem in c...

2017-10-24 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/19560 I wonder when this config should be used. If user knows there's some error in stats, why not just analyze the table (specify "noscan" if only size is needed)? This can fix the problem instead of

[GitHub] spark issue #19567: [SPARK-22291] Postgresql UUID[] to Cassandra: Conversion...

2017-10-24 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19567 Besides uuid, other PostgreSQL types such as "cidr", "inet", they are treated as StringType too, will they work? --- - To

[GitHub] spark issue #19567: [SPARK-22291] Postgresql UUID[] to Cassandra: Conversion...

2017-10-24 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19567 Seems that we don't even have test against uuid column, when you create test for uuid[], can you also create test for uuid? Thanks. ---

[GitHub] spark issue #19567: [SPARK-22291] Postgresql UUID[] to Cassandra: Conversion...

2017-10-24 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19567 @jmchung You can add unit test into `PostgresIntegrationSuite`. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19519: [SPARK-21840][core] Add trait that allows conf to...

2017-10-24 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19519#discussion_r146737263 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkApplication.scala --- @@ -0,0 +1,55 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-24 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/19568 @dongjoon-hyun, yes, I'm currently working on it. I just wanted to get the rest up. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19383: [SPARK-20643][core] Add listener implementation to colle...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19383 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83024/ Test PASSed. ---

[GitHub] spark issue #19383: [SPARK-20643][core] Add listener implementation to colle...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19383 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19383: [SPARK-20643][core] Add listener implementation to colle...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19383 **[Test build #83024 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83024/testReport)** for PR 19383 at commit

[GitHub] spark pull request #19433: [SPARK-3162] [MLlib] Add local tree training for ...

2017-10-24 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/19433#discussion_r146735946 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/LocalDecisionTree.scala --- @@ -0,0 +1,250 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19519: [SPARK-21840][core] Add trait that allows conf to...

2017-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19519#discussion_r146734568 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkApplication.scala --- @@ -0,0 +1,55 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #19390: [SPARK-18935][MESOS] Fix dynamic reservations on mesos

2017-10-24 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/19390 @susanxhuynh I will re-test and update the PR, the issue was at proto bufs, using reservation info without checking if the offer had that field anyway. ---

[GitHub] spark pull request #19519: [SPARK-21840][core] Add trait that allows conf to...

2017-10-24 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19519#discussion_r146734075 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkApplication.scala --- @@ -0,0 +1,55 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19558: [SPARK-22332][ML][TEST] Fix NaiveBayes unit test occasio...

2017-10-24 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19558 cc @jkbradley @MrBago --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19433: [SPARK-3162] [MLlib] Add local tree training for decisio...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19433 **[Test build #83025 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83025/testReport)** for PR 19433 at commit

[GitHub] spark pull request #19433: [SPARK-3162] [MLlib] Add local tree training for ...

2017-10-24 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19433#discussion_r146731101 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/TrainingInfo.scala --- @@ -0,0 +1,144 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19433: [SPARK-3162] [MLlib] Add local tree training for ...

2017-10-24 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19433#discussion_r146731083 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/LocalDecisionTree.scala --- @@ -0,0 +1,250 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19433: [SPARK-3162] [MLlib] Add local tree training for ...

2017-10-24 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19433#discussion_r146731070 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/AggUpdateUtils.scala --- @@ -0,0 +1,85 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19451: SPARK-22181 Adds ReplaceExceptWithNotFilter rule

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19451 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19433: [SPARK-3162] [MLlib] Add local tree training for ...

2017-10-24 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19433#discussion_r146731072 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/FeatureVector.scala --- @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19451: SPARK-22181 Adds ReplaceExceptWithNotFilter rule

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19451 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83022/ Test PASSed. ---

[GitHub] spark issue #19451: SPARK-22181 Adds ReplaceExceptWithNotFilter rule

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19451 **[Test build #83022 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83022/testReport)** for PR 19451 at commit

[GitHub] spark issue #19567: [SPARK-22291] Postgresql UUID[] to Cassandra: Conversion...

2017-10-24 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/19567 Please add tests to https://github.com/apache/spark/blob/master/external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/PostgresIntegrationSuite.scala ---

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19568 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19568 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83023/ Test FAILed. ---

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19568 **[Test build #83023 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83023/testReport)** for PR 19568 at commit

[GitHub] spark pull request #19568: SPARK-22345: Fix sort-merge joins with conditions...

2017-10-24 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19568#discussion_r146720363 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala --- @@ -615,6 +620,7 @@ case class SortMergeJoinExec(

[GitHub] spark issue #18527: [SPARK-21101][SQL] Catch IllegalStateException when CREA...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18527 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83021/ Test PASSed. ---

[GitHub] spark issue #18527: [SPARK-21101][SQL] Catch IllegalStateException when CREA...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18527 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18527: [SPARK-21101][SQL] Catch IllegalStateException when CREA...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18527 **[Test build #83021 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83021/testReport)** for PR 18527 at commit

[GitHub] spark issue #19217: [SPARK-21991][LAUNCHER] LauncherServer acceptConnections...

2017-10-24 Thread ash211
Github user ash211 commented on the issue: https://github.com/apache/spark/pull/19217 @nivox can you please update the PR title when you get the chance? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19383: [SPARK-20643][core] Add listener implementation to colle...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19383 **[Test build #83024 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83024/testReport)** for PR 19383 at commit

[GitHub] spark issue #17100: [SPARK-13947][SQL] The error message from using an inval...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17100 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83020/ Test PASSed. ---

[GitHub] spark issue #17100: [SPARK-13947][SQL] The error message from using an inval...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17100 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17100: [SPARK-13947][SQL] The error message from using an inval...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17100 **[Test build #83020 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83020/testReport)** for PR 17100 at commit

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r146706916 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala --- @@ -0,0 +1,690 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r146706701 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -0,0 +1,534 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r146705635 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -0,0 +1,534 @@ +/* + * Licensed to the Apache Software

  1   2   3   4   >