[GitHub] spark pull request #22566: [SPARK-25458][SQL] Support FOR ALL COLUMNS in ANA...

2018-09-27 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22566#discussion_r220985607 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala --- @@ -50,7 +52,26 @@ case class AnalyzeColumnComm

[GitHub] spark pull request #22165: [SPARK-25017][Core] Add test suite for BarrierCoo...

2018-09-27 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22165#discussion_r220986263 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -187,6 +191,12 @@ private[spark] class BarrierCoordinator( req

[GitHub] spark issue #22165: [SPARK-25017][Core] Add test suite for BarrierCoordinato...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22165 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3543/

[GitHub] spark issue #22165: [SPARK-25017][Core] Add test suite for BarrierCoordinato...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22165 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22165: [SPARK-25017][Core] Add test suite for BarrierCoordinato...

2018-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22165 **[Test build #96703 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96703/testReport)** for PR 22165 at commit [`fd4d150`](https://github.com/apache/spark/commit/fd

[GitHub] spark pull request #22566: [SPARK-25458][SQL] Support FOR ALL COLUMNS in ANA...

2018-09-27 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22566#discussion_r220986964 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala --- @@ -50,7 +52,26 @@ case class AnalyzeColumnComm

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-09-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22237 The problem here is the difference between JSON functionality and JSON datasource parsing. The JSON function was first added in a way of mimicking PERMISSIVE mode but not exactly same.

[GitHub] spark pull request #21157: [SPARK-22674][PYTHON] Removed the namedtuple pick...

2018-09-27 Thread superbobry
GitHub user superbobry reopened a pull request: https://github.com/apache/spark/pull/21157 [SPARK-22674][PYTHON] Removed the namedtuple pickling patch ## What changes were proposed in this pull request? This is a breaking change. Prior to this commit PySpark patched

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21157 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22038: [SPARK-25056][SQL] Unify the InConversion and Bin...

2018-09-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22038#discussion_r220989953 --- Diff: sql/core/src/test/resources/sql-tests/results/typeCoercion/native/inConversion.sql.out --- @@ -69,7 +69,7 @@ true -- !query 8 SELEC

[GitHub] spark issue #22473: [SPARK-25449][CORE] Heartbeat shouldn't include accumula...

2018-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22473 **[Test build #96704 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96704/testReport)** for PR 22473 at commit [`db14bd1`](https://github.com/apache/spark/commit/db

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21157 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22558: [SPARK-25546][core] Don't cache value of EVENT_LOG_CALLS...

2018-09-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22558 Merged to master/branch-2.4. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21157 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-09-27 Thread superbobry
Github user superbobry commented on the issue: https://github.com/apache/spark/pull/21157 Reopened and rebased to be merged into the 3.X branch. See discussion in #21180. --- - To unsubscribe, e-mail: reviews-unsub

[GitHub] spark issue #21180: [SPARK-22674][PYTHON] Disabled _hack_namedtuple for pick...

2018-09-27 Thread superbobry
Github user superbobry commented on the issue: https://github.com/apache/spark/pull/21180 @HyukjinKwon I've reopened the PR doing this and rebased it on the latest master, see #21157. --- - To unsubscribe, e-mail: r

[GitHub] spark pull request #21180: [SPARK-22674][PYTHON] Disabled _hack_namedtuple f...

2018-09-27 Thread superbobry
Github user superbobry closed the pull request at: https://github.com/apache/spark/pull/21180 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-09-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21157 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark pull request #22558: [SPARK-25546][core] Don't cache value of EVENT_LO...

2018-09-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22558 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21157 **[Test build #96706 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96706/testReport)** for PR 21157 at commit [`7f2ad87`](https://github.com/apache/spark/commit/7f

[GitHub] spark issue #22473: [SPARK-25449][CORE] Heartbeat shouldn't include accumula...

2018-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22473 **[Test build #96705 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96705/testReport)** for PR 22473 at commit [`e22ac78`](https://github.com/apache/spark/commit/e2

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-09-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21157 Woah. Okay. Let me add some guys interested in this again (@felixcheung looks already here) - @ueshin, @BryanCutler, @holdenk amd @JoshRosen Additionally @rxin too. Here's my understand

[GitHub] spark issue #22473: [SPARK-25449][CORE] Heartbeat shouldn't include accumula...

2018-09-27 Thread mukulmurthy
Github user mukulmurthy commented on the issue: https://github.com/apache/spark/pull/22473 Pushed updates. @zsxwing , I can add refactor Executor.scala so testing is easier, but I had hoped that doing it this way would allow more testing for reportHeartbeat in the future. We have pret

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-09-27 Thread superbobry
Github user superbobry commented on the issue: https://github.com/apache/spark/pull/21157 > So, real downside of removing this now is we disallow global scope namedtuple. Importable namedtuples and their subclasses could still be used inside an RDD. Only the namedtuples defin

[GitHub] spark issue #20405: [SPARK-23229][SQL] Dataset.hint should use planWithBarri...

2018-09-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20405 ping @jaceklaskowski --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #22144: [SPARK-24935][SQL] : Problem with Executing Hive UDF's f...

2018-09-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22144 hmm .. this is actually a regression and compatibility issue we should take a look into .. --- - To unsubscribe, e-mail: re

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22237 **[Test build #96693 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96693/testReport)** for PR 22237 at commit [`939e220`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3544/

[GitHub] spark pull request #22569: [SPARK-25542][SQL][Test] Move flaky test in OpenH...

2018-09-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22569#discussion_r220999634 --- Diff: core/src/test/scala/org/apache/spark/util/collection/OpenHashSetSuite.scala --- @@ -255,4 +255,17 @@ class OpenHashSetSuite extends SparkFun

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22237 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96693/ Test PASSed. ---

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22237 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22295 **[Test build #96707 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96707/testReport)** for PR 22295 at commit [`c966846`](https://github.com/apache/spark/commit/c9

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21157 **[Test build #96706 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96706/testReport)** for PR 21157 at commit [`7f2ad87`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21157 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96706/ Test FAILed. ---

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21157 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #22295: [SPARK-25255][PYTHON]Add getActiveSession to Spar...

2018-09-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22295#discussion_r221005236 --- Diff: python/pyspark/sql/session.py --- @@ -231,6 +231,7 @@ def __init__(self, sparkContext, jsparkSession=None): or SparkSessio

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22295 **[Test build #96707 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96707/testReport)** for PR 22295 at commit [`c966846`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96707/ Test FAILed. ---

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22531: [SPARK-25415][SQL][FOLLOW-UP] Add Locale.ROOT when toUpp...

2018-09-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22531 @srowen, WDYT about we add a rule and exclude them when only they are legitimate cases? I wrote a rule locally and these below are (almost) all legitimate cases. I was thinking we just

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-09-27 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/22237 I manually ran some queries. My point was about the value of the column `from_json`, `null` vs `Row(null, null, ...)`. Now I am +1 with the proposal of this PR, making the column v

[GitHub] spark issue #22570: [SPARK-25553][BUILD] Add EmptyInterpolatedStringChecker ...

2018-09-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22570 We had better update the code when we touch it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For addi

[GitHub] spark pull request #22571: [SPARK-25392][Spark Job History]Inconsistent beha...

2018-09-27 Thread sandeep-katta
GitHub user sandeep-katta opened a pull request: https://github.com/apache/spark/pull/22571 [SPARK-25392][Spark Job History]Inconsistent behaviour for pool details in spark web UI and history server page ## What changes were proposed in this pull request? 1. Added the

[GitHub] spark pull request #22429: [SPARK-25440][SQL] Dumping query execution info t...

2018-09-27 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22429#discussion_r221011834 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/package.scala --- @@ -167,6 +170,56 @@ package object util { builder.toStr

[GitHub] spark issue #22571: [SPARK-25392][Spark Job History]Inconsistent behaviour f...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22571 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22571: [SPARK-25392][Spark Job History]Inconsistent behaviour f...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22571 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22571: [SPARK-25392][Spark Job History]Inconsistent behaviour f...

2018-09-27 Thread sandeep-katta
Github user sandeep-katta commented on the issue: https://github.com/apache/spark/pull/22571 cc @vanzin @srowen ,Please review this improvement in History UI --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.ap

[GitHub] spark issue #22569: [SPARK-25542][SQL][Test] Move flaky test in OpenHashMapS...

2018-09-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22569 Also, please fix the tag in the PR title from `[SQL]` to `[CORE]`. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.

[GitHub] spark issue #22571: [SPARK-25392][Spark Job History]Inconsistent behaviour f...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22571 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18304: [SPARK-21098] Set lineseparator csv multiline and csv wr...

2018-09-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18304 Ping, @danielvdende . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: r

[GitHub] spark issue #18942: [BACKPORT-2.1][SPARK-19372][SQL] Fix throwing a Java exc...

2018-09-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18942 Ping, @poplav . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #22060: [DO NOT MERGE][TEST ONLY] Add once-policy rule check

2018-09-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22060 Ping, @maryannxue . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #22539: [SPARK-25517][SQL] Detect/Infer date type in CSV file

2018-09-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22539 Ping, @softmanu . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revie

[GitHub] spark issue #21517: Testing k8s change - please ignore (13)

2018-09-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21517 Ping, @ssuchter . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revie

[GitHub] spark pull request #18304: [SPARK-21098] Set lineseparator csv multiline and...

2018-09-27 Thread danielvdende
Github user danielvdende closed the pull request at: https://github.com/apache/spark/pull/18304 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.o

[GitHub] spark pull request #18942: [BACKPORT-2.1][SPARK-19372][SQL] Fix throwing a J...

2018-09-27 Thread poplav
Github user poplav closed the pull request at: https://github.com/apache/spark/pull/18942 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18942: [BACKPORT-2.1][SPARK-19372][SQL] Fix throwing a Java exc...

2018-09-27 Thread poplav
Github user poplav commented on the issue: https://github.com/apache/spark/pull/18942 Was away on vacation. Closing PR, thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22557: [SPARK-25535][core] Work around bad error handling in co...

2018-09-27 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22557 @squito --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.ap

[GitHub] spark pull request #22572: [SPARK-25521][SQL]Job id showing null in the logs...

2018-09-27 Thread sujith71955
GitHub user sujith71955 opened a pull request: https://github.com/apache/spark/pull/22572 [SPARK-25521][SQL]Job id showing null in the logs when insert into command Job is finished. ## What changes were proposed in this pull request? ``As part of insert command in FileFormatWr

[GitHub] spark issue #22572: [SPARK-25521][SQL]Job id showing null in the logs when i...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22572 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22572: [SPARK-25521][SQL]Job id showing null in the logs when i...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22572 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22572: [SPARK-25521][SQL]Job id showing null in the logs when i...

2018-09-27 Thread sujith71955
Github user sujith71955 commented on the issue: https://github.com/apache/spark/pull/22572 cc @cloud-fan @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: re

[GitHub] spark issue #22572: [SPARK-25521][SQL]Job id showing null in the logs when i...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22572 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22326: [SPARK-25314][SQL] Fix Python UDF accessing attributes f...

2018-09-27 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/22326 Thanks everyone for your review and advise. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additiona

[GitHub] spark pull request #22473: [SPARK-25449][CORE] Heartbeat shouldn't include a...

2018-09-27 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/22473#discussion_r221022651 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -609,13 +609,13 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with Loggi

[GitHub] spark pull request #22473: [SPARK-25449][CORE] Heartbeat shouldn't include a...

2018-09-27 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/22473#discussion_r221022783 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -609,13 +609,13 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with Loggi

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-09-27 Thread huaxingao
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/22295 I just saw this fix [SPARK-25525][SQL][PYSPARK] Do not update conf for existing SparkContext in SparkSession.getOrCreate. #22545 I will remove ```test_create_SparkContext_then_SparkSession```

[GitHub] spark issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaData...

2018-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22138 **[Test build #96695 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96695/testReport)** for PR 22138 at commit [`d3f097b`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaData...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22138 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaData...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22138 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96695/ Test PASSed. ---

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3545/

[GitHub] spark pull request #22473: [SPARK-25449][CORE] Heartbeat shouldn't include a...

2018-09-27 Thread mukulmurthy
Github user mukulmurthy commented on a diff in the pull request: https://github.com/apache/spark/pull/22473#discussion_r221026721 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -609,13 +609,13 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with L

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22295 **[Test build #96708 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96708/testReport)** for PR 22295 at commit [`3e11d0a`](https://github.com/apache/spark/commit/3e

[GitHub] spark pull request #22566: [SPARK-25458][SQL] Support FOR ALL COLUMNS in ANA...

2018-09-27 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22566#discussion_r221029986 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala --- @@ -50,7 +52,26 @@ case class AnalyzeColumnComm

[GitHub] spark issue #22569: [SPARK-25542][SQL][Test] Move flaky test in OpenHashMapS...

2018-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22569 **[Test build #96697 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96697/testReport)** for PR 22569 at commit [`39a77d6`](https://github.com/apache/spark/commit/3

[GitHub] spark pull request #22573: [SPARK-25558][SQL] Pushdown predicates for nested...

2018-09-27 Thread dbtsai
GitHub user dbtsai opened a pull request: https://github.com/apache/spark/pull/22573 [SPARK-25558][SQL] Pushdown predicates for nested fields in DataSource Strategy ## What changes were proposed in this pull request? This PR allows Spark to create predicates for nested fiel

[GitHub] spark issue #22569: [SPARK-25542][SQL][Test] Move flaky test in OpenHashMapS...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22569 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22569: [SPARK-25542][SQL][Test] Move flaky test in OpenHashMapS...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22569 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96697/ Test PASSed. ---

[GitHub] spark issue #22573: [SPARK-25558][SQL] Pushdown predicates for nested fields...

2018-09-27 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/22573 @gatorsmile @cloud-fan @dongjoon-hyun @viirya --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional c

[GitHub] spark issue #22573: [SPARK-25558][SQL] Pushdown predicates for nested fields...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22573 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3546/

[GitHub] spark issue #22573: [SPARK-25558][SQL] Pushdown predicates for nested fields...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22573 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22573: [SPARK-25558][SQL] Pushdown predicates for nested fields...

2018-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22573 **[Test build #96709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96709/testReport)** for PR 22573 at commit [`2f21842`](https://github.com/apache/spark/commit/2f

[GitHub] spark issue #18942: [BACKPORT-2.1][SPARK-19372][SQL] Fix throwing a Java exc...

2018-09-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18942 Thank you, @poplav ! :D --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18304: [SPARK-21098] Set lineseparator csv multiline and csv wr...

2018-09-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18304 Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@sp

[GitHub] spark issue #22531: [SPARK-25415][SQL][FOLLOW-UP] Add Locale.ROOT when toUpp...

2018-09-27 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22531 @HyukjinKwon I think it's OK to fix these instances, yes. We have to be careful to restrict it to cases where the locale definitely should not matter, like when comparing enum values and other progra

[GitHub] spark issue #22572: [SPARK-25521][SQL]Job id showing null in the logs when i...

2018-09-27 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22572 Is the value logged here always null? I am not sure if it's meaningful to log mapreduce.job.id, especially given its name. If there's no meaningful job ID here do we are about it at all? how abou

[GitHub] spark issue #22572: [SPARK-25521][SQL]Job id showing null in the logs when i...

2018-09-27 Thread sujith71955
Github user sujith71955 commented on the issue: https://github.com/apache/spark/pull/22572 > Is the value logged here always null? > I am not sure if it's meaningful to log mapreduce.job.id, especially given its name. If there's no meaningful job ID here do we are about it at all?

[GitHub] spark issue #22572: [SPARK-25521][SQL]Job id showing null in the logs when i...

2018-09-27 Thread sujith71955
Github user sujith71955 commented on the issue: https://github.com/apache/spark/pull/22572 cc @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22379: [SPARK-25393][SQL] Adding new function from_csv()

2018-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22379 **[Test build #96699 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96699/testReport)** for PR 22379 at commit [`4c2fcea`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #22571: [SPARK-25392][Spark Job History]Inconsistent behaviour f...

2018-09-27 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22571 I'm not sure I know enough to evaluate this. Is the solution to send all this info around, or just not link to a pool in this case? I don't know. For some reason the Github diff is highlighti

[GitHub] spark issue #22379: [SPARK-25393][SQL] Adding new function from_csv()

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22379 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22379: [SPARK-25393][SQL] Adding new function from_csv()

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22379 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96699/ Test PASSed. ---

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96708/ Test PASSed. ---

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22295 **[Test build #96708 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96708/testReport)** for PR 22295 at commit [`3e11d0a`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-09-27 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/21157 so this change would introduce a pretty big regression? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For addi

[GitHub] spark pull request #22379: [SPARK-25393][SQL] Adding new function from_csv()

2018-09-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22379#discussion_r221047590 --- Diff: R/pkg/R/functions.R --- @@ -2204,6 +2210,26 @@ setMethod("from_json", signature(x = "Column", schema = "characterOrstructType")

[GitHub] spark issue #22570: [SPARK-25553][BUILD] Add EmptyInterpolatedStringChecker ...

2018-09-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22570 **[Test build #96698 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96698/testReport)** for PR 22570 at commit [`68f83a3`](https://github.com/apache/spark/commit/6

<    1   2   3   4   5   6   7   >