[jira] [Commented] (SPARK-29301) Removing block is not reflected to the driver/executor's storage memory

2019-09-29 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940672#comment-16940672 ] Jungtaek Lim commented on SPARK-29301: -- Will raise a patch soon. > Removing block

[jira] [Created] (SPARK-29301) Removing block is not reflected to the driver/executor's storage memory

2019-09-29 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-29301: Summary: Removing block is not reflected to the driver/executor's storage memory Key: SPARK-29301 URL: https://issues.apache.org/jira/browse/SPARK-29301 Project: Spar

[jira] [Updated] (SPARK-29299) Intermittently getting "Cannot create the managed table error" while creating table from spark 2.4

2019-09-29 Thread Abhijeet (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhijeet updated SPARK-29299: - Summary: Intermittently getting "Cannot create the managed table error" while creating table from spark

[jira] [Updated] (SPARK-29299) Intermittently getting "Cannot create the managed table error" while creating table from spark 2.4

2019-09-29 Thread Abhijeet (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhijeet updated SPARK-29299: - Description: We are facing below error in spark 2.4 intermittently when saving the managed table from s

[jira] [Updated] (SPARK-29300) Compare `catalyst` and `avro` module benchmark in JDK8/11

2019-09-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29300: -- Component/s: (was: SQL) Tests > Compare `catalyst` and `avro` module benc

[jira] [Created] (SPARK-29300) Compare `catalyst` and `avro` module benchmark in JDK8/11

2019-09-29 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-29300: - Summary: Compare `catalyst` and `avro` module benchmark in JDK8/11 Key: SPARK-29300 URL: https://issues.apache.org/jira/browse/SPARK-29300 Project: Spark I

[jira] [Created] (SPARK-29299) Intermittently getting "Can not create the managed table error" while creating table from spark 2.4

2019-09-29 Thread Abhijeet (Jira)
Abhijeet created SPARK-29299: Summary: Intermittently getting "Can not create the managed table error" while creating table from spark 2.4 Key: SPARK-29299 URL: https://issues.apache.org/jira/browse/SPARK-29299

[jira] [Resolved] (SPARK-29297) Compare `core` and `mllib` module benchmark in JDK8/11

2019-09-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29297. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25969 [https://

[jira] [Assigned] (SPARK-29297) Compare `core` and `mllib` module benchmark in JDK8/11

2019-09-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29297: - Assignee: Dongjoon Hyun > Compare `core` and `mllib` module benchmark in JDK8/11 >

[jira] [Commented] (SPARK-27396) SPIP: Public APIs for extended Columnar Processing Support

2019-09-29 Thread Renjie Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940647#comment-16940647 ] Renjie Liu commented on SPARK-27396: [~revans2] Any update on this epic? > SPIP: Pu

[jira] [Commented] (SPARK-29298) Separate block manager heartbeat endpoint from driver endpoint

2019-09-29 Thread Lantao Jin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940644#comment-16940644 ] Lantao Jin commented on SPARK-29298: cc [~dongjoon] [~cloud_fan] [~hyukjin.kwon] [~v

[jira] [Created] (SPARK-29298) Separate block manager heartbeat endpoint from driver endpoint

2019-09-29 Thread Lantao Jin (Jira)
Lantao Jin created SPARK-29298: -- Summary: Separate block manager heartbeat endpoint from driver endpoint Key: SPARK-29298 URL: https://issues.apache.org/jira/browse/SPARK-29298 Project: Spark I

[jira] [Issue Comment Deleted] (SPARK-29186) SubqueryAlias name value is null in Spark 2.4.3 Logical plan.

2019-09-29 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-29186: -- Comment: was deleted (was: In spark-2.4 & and master branch.  RUN SQL  {code:java} create table custo

[jira] [Resolved] (SPARK-29186) SubqueryAlias name value is null in Spark 2.4.3 Logical plan.

2019-09-29 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-29186. - Resolution: Resolved > SubqueryAlias name value is null in Spark 2.4.3 Logical plan. > -

[jira] [Updated] (SPARK-29186) SubqueryAlias name value is null in Spark 2.4.3 Logical plan.

2019-09-29 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-29186: Fix Version/s: 3.0.0 2.4.5 > SubqueryAlias name value is null in Spark 2.4.3 Lo

[jira] [Commented] (SPARK-29288) Spark SQL add jar can't support HTTP path.

2019-09-29 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940605#comment-16940605 ] angerszhu commented on SPARK-29288: --- [~dongjoon] Thank your support. working on this.

[jira] [Issue Comment Deleted] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

2019-09-29 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29295: Comment: was deleted (was: I have tested in 2.3.1 branch, without SPARK-25271, it will always give duplic

[jira] [Updated] (SPARK-29297) Compare `core` and `mllib` module benchmark in JDK8/11

2019-09-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29297: -- Parent: SPARK-29194 Issue Type: Sub-task (was: Task) > Compare `core` and `mllib` mod

[jira] [Created] (SPARK-29297) Compare `core` and `mllib` module benchmark in JDK8/11

2019-09-29 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-29297: - Summary: Compare `core` and `mllib` module benchmark in JDK8/11 Key: SPARK-29297 URL: https://issues.apache.org/jira/browse/SPARK-29297 Project: Spark Issu

[jira] [Comment Edited] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

2019-09-29 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940600#comment-16940600 ] feiwang edited comment on SPARK-29295 at 9/30/19 1:52 AM: -- cc [

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

2019-09-29 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940600#comment-16940600 ] feiwang commented on SPARK-29295: - cc [~cloud_fan] > Duplicate result when dropping par

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

2019-09-29 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940599#comment-16940599 ] feiwang commented on SPARK-29295: - I have tested in 2.3.1 branch, without SPARK-25271, i

[jira] [Created] (SPARK-29296) Use scala-parallel-collections library in 2.13

2019-09-29 Thread Sean R. Owen (Jira)
Sean R. Owen created SPARK-29296: Summary: Use scala-parallel-collections library in 2.13 Key: SPARK-29296 URL: https://issues.apache.org/jira/browse/SPARK-29296 Project: Spark Issue Type: Su

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

2019-09-29 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940598#comment-16940598 ] feiwang commented on SPARK-29295: - When we set CONVERT_METASTORE_PARQUET=true, it will u

[jira] [Updated] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

2019-09-29 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29295: Description: When we drop a partition of a external table and then overwrite it, if we set CONVERT_METAST

[jira] [Created] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

2019-09-29 Thread feiwang (Jira)
feiwang created SPARK-29295: --- Summary: Duplicate result when dropping partition of an external table and then overwriting Key: SPARK-29295 URL: https://issues.apache.org/jira/browse/SPARK-29295 Project: Spa

[jira] [Updated] (SPARK-29289) Update scalatest, scalacheck, scopt, scala-parser-combinators, clapper classutil

2019-09-29 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-29289: - Description: We need to slightly update these dependencies for a version that works with 2.13.

[jira] [Updated] (SPARK-29289) Update scalatest, scalacheck, scopt, scala-parser-combinators, clapper classutil

2019-09-29 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-29289: - Description: We need to slightly update these dependencies for a version that works with 2.13.

[jira] [Created] (SPARK-29294) Update Kafka to a verison that supports Scala 2.13

2019-09-29 Thread Sean R. Owen (Jira)
Sean R. Owen created SPARK-29294: Summary: Update Kafka to a verison that supports Scala 2.13 Key: SPARK-29294 URL: https://issues.apache.org/jira/browse/SPARK-29294 Project: Spark Issue Type

[jira] [Resolved] (SPARK-29019) Improve tooltip information in JDBC/ODBC Server tab

2019-09-29 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-29019. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25723 [https://gi

[jira] [Assigned] (SPARK-29019) Improve tooltip information in JDBC/ODBC Server tab

2019-09-29 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-29019: Assignee: Pablo Langa Blanco > Improve tooltip information in JDBC/ODBC Server tab >

[jira] [Assigned] (SPARK-29282) Investigate the performance regression in `core` module on JDK11 benchmark

2019-09-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29282: - Assignee: Dongjoon Hyun > Investigate the performance regression in `core` module on JD

[jira] [Resolved] (SPARK-29282) Investigate the performance regression in `core` module on JDK11 benchmark

2019-09-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29282. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25966 [https://

[jira] [Updated] (SPARK-29282) Use the same VM configurations for test/benchmark

2019-09-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29282: -- Summary: Use the same VM configurations for test/benchmark (was: Investigate the performance

[jira] [Resolved] (SPARK-29247) HiveClientImpl may be log sensitive information

2019-09-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29247. --- Fix Version/s: 3.0.0 2.4.5 Resolution: Fixed Issue resolved by pul

[jira] [Assigned] (SPARK-29247) HiveClientImpl may be log sensitive information

2019-09-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29247: - Assignee: angerszhu > HiveClientImpl may be log sensitive information > ---

[jira] [Commented] (SPARK-29288) Spark SQL add jar can't support HTTP path.

2019-09-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940529#comment-16940529 ] Dongjoon Hyun commented on SPARK-29288: --- Thank you for pinging me. +1 for the feat

[jira] [Commented] (SPARK-29155) Support special date/timestamp values in the PostgreSQL dialect only

2019-09-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940527#comment-16940527 ] Dongjoon Hyun commented on SPARK-29155: --- Yep. Thank you for pinging me. > Support

[jira] [Commented] (SPARK-29241) Show alternative names in the function description

2019-09-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940528#comment-16940528 ] Dongjoon Hyun commented on SPARK-29241: --- +1 if this PR aims to do this automatical

[jira] [Resolved] (SPARK-29155) Support special date/timestamp values in the PostgreSQL dialect only

2019-09-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29155. --- Fix Version/s: 3.0.0 Assignee: Maxim Gekk Resolution: Fixed > Support specia

[jira] [Resolved] (SPARK-29115) Add benchmarks for make_date() and make_timestamp()

2019-09-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29115. --- Fix Version/s: 3.0.0 Assignee: Maxim Gekk Resolution: Fixed This is resolved

[jira] [Updated] (SPARK-29290) Update twitter chill for 2.13 when available, or remove it

2019-09-29 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-29290: - Description: Twitter's chill isn't cross-built for 2.13 (yet) and looks like it's waiting on se

[jira] [Resolved] (SPARK-29275) Update the SQL migration guide regarding special date/timestamp values

2019-09-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29275. --- Fix Version/s: 3.0.0 Resolution: Fixed > Update the SQL migration guide regarding spe

[jira] [Commented] (SPARK-29275) Update the SQL migration guide regarding special date/timestamp values

2019-09-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940523#comment-16940523 ] Dongjoon Hyun commented on SPARK-29275: --- This is resolved via https://github.com/a

[jira] [Commented] (SPARK-29275) Update the SQL migration guide regarding special date/timestamp values

2019-09-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940522#comment-16940522 ] Dongjoon Hyun commented on SPARK-29275: --- Sure! > Update the SQL migration guide r

[jira] [Assigned] (SPARK-29275) Update the SQL migration guide regarding special date/timestamp values

2019-09-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29275: - Assignee: Maxim Gekk > Update the SQL migration guide regarding special date/timestamp

[jira] [Created] (SPARK-29293) Update scalafmt for 2.13

2019-09-29 Thread Sean R. Owen (Jira)
Sean R. Owen created SPARK-29293: Summary: Update scalafmt for 2.13 Key: SPARK-29293 URL: https://issues.apache.org/jira/browse/SPARK-29293 Project: Spark Issue Type: Sub-task Compo

[jira] [Created] (SPARK-29292) Fix internal usages of mutable collection for Seq in 2.13

2019-09-29 Thread Sean R. Owen (Jira)
Sean R. Owen created SPARK-29292: Summary: Fix internal usages of mutable collection for Seq in 2.13 Key: SPARK-29292 URL: https://issues.apache.org/jira/browse/SPARK-29292 Project: Spark Iss

[jira] [Created] (SPARK-29291) Remove use of procedure-like method definition for 2.13

2019-09-29 Thread Sean R. Owen (Jira)
Sean R. Owen created SPARK-29291: Summary: Remove use of procedure-like method definition for 2.13 Key: SPARK-29291 URL: https://issues.apache.org/jira/browse/SPARK-29291 Project: Spark Issue

[jira] [Created] (SPARK-29290) Remove dependency on Twitter chill

2019-09-29 Thread Sean R. Owen (Jira)
Sean R. Owen created SPARK-29290: Summary: Remove dependency on Twitter chill Key: SPARK-29290 URL: https://issues.apache.org/jira/browse/SPARK-29290 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-29289) Update scalatest, scalacheck

2019-09-29 Thread Sean R. Owen (Jira)
Sean R. Owen created SPARK-29289: Summary: Update scalatest, scalacheck Key: SPARK-29289 URL: https://issues.apache.org/jira/browse/SPARK-29289 Project: Spark Issue Type: Sub-task C

[jira] [Resolved] (SPARK-28476) Support ALTER DATABASE SET LOCATION

2019-09-29 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-28476. - Fix Version/s: 3.0.0 Assignee: yuming.wang (was: Weichen Xu) Resolution: Fixed > Suppor

[jira] [Resolved] (SPARK-29172) Fix some exception issue of explain commands

2019-09-29 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-29172. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25848 [https://gi

[jira] [Assigned] (SPARK-29172) Fix some exception issue of explain commands

2019-09-29 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-29172: Assignee: Tomoko Komiyama > Fix some exception issue of explain commands > --

[jira] [Commented] (SPARK-29241) Show alternative names in the function description

2019-09-29 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940444#comment-16940444 ] Sean R. Owen commented on SPARK-29241: -- Sure, if it's relatively easy to do, I thin

[jira] [Commented] (SPARK-29275) Update the SQL migration guide regarding special date/timestamp values

2019-09-29 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940445#comment-16940445 ] Maxim Gekk commented on SPARK-29275: [~dongjoon] Can we resolve this? > Update the

[jira] [Resolved] (SPARK-24171) Update comments for non-deterministic functions

2019-09-29 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk resolved SPARK-24171. Fix Version/s: 2.4.0 Resolution: Done > Update comments for non-deterministic functions > -

[jira] [Resolved] (SPARK-24591) Number of cores and executors in the cluster

2019-09-29 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk resolved SPARK-24591. Resolution: Won't Fix > Number of cores and executors in the cluster > ---

[jira] [Resolved] (SPARK-25440) Dump query execution info to a file

2019-09-29 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk resolved SPARK-25440. Fix Version/s: 3.0.0 Resolution: Fixed > Dump query execution info to a file >

[jira] [Commented] (SPARK-29115) Add benchmarks for make_date() and make_timestamp()

2019-09-29 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940440#comment-16940440 ] Maxim Gekk commented on SPARK-29115: [~dongjoon]This can be resolved, I think. > Ad

[jira] [Commented] (SPARK-29155) Support special date/timestamp values in the PostgreSQL dialect only

2019-09-29 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940439#comment-16940439 ] Maxim Gekk commented on SPARK-29155: [~dongjoon]This can be resolved, right? > Supp

[jira] [Commented] (SPARK-29241) Show alternative names in the function description

2019-09-29 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940438#comment-16940438 ] Maxim Gekk commented on SPARK-29241: [~dongjoon] [~hyukjin.kwon] [~srowen] WDYT? Wil

[jira] [Commented] (SPARK-29222) Flaky test: pyspark.mllib.tests.test_streaming_algorithms.StreamingLinearRegressionWithTests.test_parameter_convergence

2019-09-29 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940434#comment-16940434 ] Jungtaek Lim commented on SPARK-29222: -- Did you see test failures consistently and

[jira] [Commented] (SPARK-29055) Memory leak in Spark

2019-09-29 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940429#comment-16940429 ] Jungtaek Lim commented on SPARK-29055: -- [~Geopap] Could you take memory dump for m

[jira] [Created] (SPARK-29288) Spark SQL add jar can't support HTTP path.

2019-09-29 Thread angerszhu (Jira)
angerszhu created SPARK-29288: - Summary: Spark SQL add jar can't support HTTP path. Key: SPARK-29288 URL: https://issues.apache.org/jira/browse/SPARK-29288 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-29055) Memory leak in Spark

2019-09-29 Thread George Papa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940419#comment-16940419 ] George Papa edited comment on SPARK-29055 at 9/29/19 1:49 PM:

[jira] [Commented] (SPARK-29055) Memory leak in Spark

2019-09-29 Thread George Papa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940419#comment-16940419 ] George Papa commented on SPARK-29055: - @tommy duan You have created the SPARK-2764

[jira] [Comment Edited] (SPARK-29055) Memory leak in Spark

2019-09-29 Thread George Papa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940417#comment-16940417 ] George Papa edited comment on SPARK-29055 at 9/29/19 1:47 PM:

[jira] [Commented] (SPARK-29055) Memory leak in Spark

2019-09-29 Thread George Papa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940417#comment-16940417 ] George Papa commented on SPARK-29055: - [~kabhwan] I have tested also with the lates

[jira] [Commented] (SPARK-18371) Spark Streaming backpressure bug - generates a batch with large number of records

2019-09-29 Thread Sebastian Arzt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940405#comment-16940405 ] Sebastian Arzt commented on SPARK-18371: [~rkarthikeyan] at a first glace I cann

[jira] [Commented] (SPARK-22536) VectorizedParquetRecordReader doesn't use Parquet's dictionary filtering feature

2019-09-29 Thread Gang Ma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940372#comment-16940372 ] Gang Ma commented on SPARK-22536: - [~hyukjin.kwon], please ignore it, just figure out th

[jira] [Commented] (SPARK-27884) Deprecate Python 2 support in Spark 3.0

2019-09-29 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940365#comment-16940365 ] Maciej Szymkiewicz commented on SPARK-27884: According to [related discussio

[jira] [Commented] (SPARK-29273) Spark peakExecutionMemory metrics is zero

2019-09-29 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940336#comment-16940336 ] angerszhu commented on SPARK-29273: --- We may always meet in jira and pr, hh > Spark pe

[jira] [Commented] (SPARK-22536) VectorizedParquetRecordReader doesn't use Parquet's dictionary filtering feature

2019-09-29 Thread Gang Ma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940330#comment-16940330 ] Gang Ma commented on SPARK-22536: - [~hyukjin.kwon] Why this one is resolved? > Vectoriz

[jira] [Created] (SPARK-29287) Executors should not receive any offers before they are actually constructed

2019-09-29 Thread Kent Yao (Jira)
Kent Yao created SPARK-29287: Summary: Executors should not receive any offers before they are actually constructed Key: SPARK-29287 URL: https://issues.apache.org/jira/browse/SPARK-29287 Project: Spark

[jira] [Created] (SPARK-29286) UnicodeDecodeError raised when running python tests on arm instance

2019-09-29 Thread huangtianhua (Jira)
huangtianhua created SPARK-29286: Summary: UnicodeDecodeError raised when running python tests on arm instance Key: SPARK-29286 URL: https://issues.apache.org/jira/browse/SPARK-29286 Project: Spark

[jira] [Commented] (SPARK-29222) Flaky test: pyspark.mllib.tests.test_streaming_algorithms.StreamingLinearRegressionWithTests.test_parameter_convergence

2019-09-29 Thread huangtianhua (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940282#comment-16940282 ] huangtianhua commented on SPARK-29222: -- About the issues happened in arm instance S