[jira] [Commented] (SPARK-25449) Don't send zero accumulators in heartbeats

2019-03-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791321#comment-16791321 ] Xiao Li commented on SPARK-25449: - This changed the unit of conf. > Don't send zero accumulators in

[jira] [Updated] (SPARK-25449) Don't send zero accumulators in heartbeats

2019-03-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25449: Labels: release-notes (was: ) > Don't send zero accumulators in heartbeats >

[jira] [Commented] (SPARK-27141) Use ConfigEntry for hardcoded configs Yarn

2019-03-12 Thread Sandeep Katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791284#comment-16791284 ] Sandeep Katta commented on SPARK-27141: --- I will be working on this PR > Use ConfigEntry for

[jira] [Assigned] (SPARK-27142) Provide REST API for SQL level information

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27142: Assignee: Apache Spark > Provide REST API for SQL level information >

[jira] [Commented] (SPARK-27143) Provide REST API for JDBC/ODBC level information

2019-03-12 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791272#comment-16791272 ] Ajith S commented on SPARK-27143: - ping [~srowen] [~cloud_fan] [~dongjoon]  Please suggest if this

[jira] [Commented] (SPARK-27143) Provide REST API for JDBC/ODBC level information

2019-03-12 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791270#comment-16791270 ] Ajith S commented on SPARK-27143: - I will be working on this > Provide REST API for JDBC/ODBC level

[jira] [Created] (SPARK-27143) Provide REST API for JDBC/ODBC level information

2019-03-12 Thread Ajith S (JIRA)
Ajith S created SPARK-27143: --- Summary: Provide REST API for JDBC/ODBC level information Key: SPARK-27143 URL: https://issues.apache.org/jira/browse/SPARK-27143 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-27142) Provide REST API for SQL level information

2019-03-12 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791269#comment-16791269 ] Ajith S commented on SPARK-27142: - I will be working on this > Provide REST API for SQL level

[jira] [Assigned] (SPARK-27142) Provide REST API for SQL level information

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27142: Assignee: (was: Apache Spark) > Provide REST API for SQL level information >

[jira] [Created] (SPARK-27142) Provide REST API for SQL level information

2019-03-12 Thread Ajith S (JIRA)
Ajith S created SPARK-27142: --- Summary: Provide REST API for SQL level information Key: SPARK-27142 URL: https://issues.apache.org/jira/browse/SPARK-27142 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-27141) Use ConfigEntry for hardcoded configs Yarn

2019-03-12 Thread wangjiaochun (JIRA)
wangjiaochun created SPARK-27141: Summary: Use ConfigEntry for hardcoded configs Yarn Key: SPARK-27141 URL: https://issues.apache.org/jira/browse/SPARK-27141 Project: Spark Issue Type:

[jira] [Updated] (SPARK-26936) On yarn-client mode, insert overwrite local directory can not create temporary path in local staging directory

2019-03-12 Thread jiaan.geng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-26936: --- Summary: On yarn-client mode, insert overwrite local directory can not create temporary path in

[jira] [Updated] (SPARK-26936) insert overwrite local directory can not create temporary path in local staging directory

2019-03-12 Thread jiaan.geng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-26936: --- Description: Let me introduce bug of  'insert overwrite local directory'. If I execute the SQL

[jira] [Assigned] (SPARK-27140) The feature is 'insert overwrite local directory' has an inconsistent behavior in different environment.

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27140: Assignee: (was: Apache Spark) > The feature is 'insert overwrite local directory'

[jira] [Assigned] (SPARK-27140) The feature is 'insert overwrite local directory' has an inconsistent behavior in different environment.

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27140: Assignee: Apache Spark > The feature is 'insert overwrite local directory' has an

[jira] [Commented] (SPARK-27140) The feature is 'insert overwrite local directory' has an inconsistent behavior in different environment.

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791236#comment-16791236 ] Apache Spark commented on SPARK-27140: -- User 'beliefer' has created a pull request for this issue:

[jira] [Updated] (SPARK-27140) The feature is 'insert overwrite local directory' has an inconsistent behavior in different environment.

2019-03-12 Thread jiaan.geng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-27140: --- Description: In local[*] mode, maropu give a test case as follows: {code:java} $ls /tmp/noexistdir

[jira] [Resolved] (SPARK-26976) Forbid reserved keywords as identifiers when ANSI mode is on

2019-03-12 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-26976. -- Resolution: Fixed Fix Version/s: 3.0.0 Resolved by 

[jira] [Created] (SPARK-27140) The feature is 'insert overwrite local directory' has an inconsistent behavior in different environment.

2019-03-12 Thread jiaan.geng (JIRA)
jiaan.geng created SPARK-27140: -- Summary: The feature is 'insert overwrite local directory' has an inconsistent behavior in different environment. Key: SPARK-27140 URL:

[jira] [Resolved] (SPARK-27137) Spark captured variable is null if the code is pasted via :paste

2019-03-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27137. -- Resolution: Cannot Reproduce Seems not failing in the current master: {code} scala> :paste

[jira] [Commented] (SPARK-27131) Merge funciton in QuantileSummaries

2019-03-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791217#comment-16791217 ] Hyukjin Kwon commented on SPARK-27131: -- Let's ask questions to mailing list. You could have a

[jira] [Resolved] (SPARK-27131) Merge funciton in QuantileSummaries

2019-03-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27131. -- Resolution: Invalid > Merge funciton in QuantileSummaries >

[jira] [Resolved] (SPARK-27045) SQL tab in UI shows actual SQL instead of callsite

2019-03-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27045. --- Resolution: Fixed Assignee: Ajith S Fix Version/s: 3.0.0 This is resolved

[jira] [Updated] (SPARK-27045) SQL tab in UI shows actual SQL instead of callsite in case of SparkSQLDriver

2019-03-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27045: -- Description: When we run sql in spark via SparkSQLDriver (thrift server, spark-sql), SQL

[jira] [Updated] (SPARK-27045) SQL tab in UI shows actual SQL instead of callsite in case of SparkSQLDriver

2019-03-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27045: -- Summary: SQL tab in UI shows actual SQL instead of callsite in case of SparkSQLDriver (was:

[jira] [Resolved] (SPARK-27130) Automatically select profile when executing sbt-checkstyle

2019-03-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27130. -- Resolution: Fixed Fix Version/s: 3.0.0 Fixed in

[jira] [Assigned] (SPARK-27130) Automatically select profile when executing sbt-checkstyle

2019-03-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-27130: Assignee: Hyukjin Kwon > Automatically select profile when executing sbt-checkstyle >

[jira] [Assigned] (SPARK-27130) Automatically select profile when executing sbt-checkstyle

2019-03-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-27130: Assignee: Yuming Wang (was: Hyukjin Kwon) > Automatically select profile when executing

[jira] [Commented] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791075#comment-16791075 ] Dongjoon Hyun commented on SPARK-27107: --- Thank you for confirmation, [~Dhruve Ashar]. > Spark SQL

[jira] [Resolved] (SPARK-27034) Nested schema pruning for ORC

2019-03-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27034. --- Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 3.0.0 This is

[jira] [Assigned] (SPARK-26176) Verify column name when creating table via `STORED AS`

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26176: Assignee: Apache Spark > Verify column name when creating table via `STORED AS` >

[jira] [Assigned] (SPARK-26176) Verify column name when creating table via `STORED AS`

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26176: Assignee: (was: Apache Spark) > Verify column name when creating table via `STORED

[jira] [Commented] (SPARK-26176) Verify column name when creating table via `STORED AS`

2019-03-12 Thread Sujith Chacko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791060#comment-16791060 ] Sujith Chacko commented on SPARK-26176: --- Issue is still happening with spark 2.4 latest version. I

[jira] [Assigned] (SPARK-27134) array_distinct function does not work correctly with columns containing array of array

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27134: Assignee: Apache Spark > array_distinct function does not work correctly with columns

[jira] [Resolved] (SPARK-27123) Improve CollapseProject to handle projects cross limit/repartition/sample

2019-03-12 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-27123. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24049

[jira] [Assigned] (SPARK-27134) array_distinct function does not work correctly with columns containing array of array

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27134: Assignee: (was: Apache Spark) > array_distinct function does not work correctly with

[jira] [Resolved] (SPARK-27139) NettyBlockTransferService does not abide by spark.blockManager.port config option

2019-03-12 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin resolved SPARK-27139. Resolution: Invalid And proper casing does the trick > NettyBlockTransferService does

[jira] [Updated] (SPARK-27139) NettyBlockTransferService does not abide by spark.blockManager.port config option

2019-03-12 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated SPARK-27139: --- Description: This is a regression from a fix in SPARK-4837 The NettyBlockTransferService

[jira] [Assigned] (SPARK-26927) Race condition may cause dynamic allocation not working

2019-03-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26927: -- Assignee: liupengcheng > Race condition may cause dynamic allocation not working >

[jira] [Resolved] (SPARK-26927) Race condition may cause dynamic allocation not working

2019-03-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26927. Resolution: Fixed Fix Version/s: 2.3.4 2.4.2

[jira] [Created] (SPARK-27139) NettyBlockTransferService does not abide by spark.blockManager.port config option

2019-03-12 Thread Bolke de Bruin (JIRA)
Bolke de Bruin created SPARK-27139: -- Summary: NettyBlockTransferService does not abide by spark.blockManager.port config option Key: SPARK-27139 URL: https://issues.apache.org/jira/browse/SPARK-27139

[jira] [Commented] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-12 Thread Dhruve Ashar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790911#comment-16790911 ] Dhruve Ashar commented on SPARK-27107: -- I verified the changes and we are no longer seeing the

[jira] [Commented] (SPARK-27087) Inability to access to column alias in pyspark

2019-03-12 Thread Thincrs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790929#comment-16790929 ] Thincrs commented on SPARK-27087: - A user of thincrs has selected this issue. Deadline: Tue, Mar 19,

[jira] [Updated] (SPARK-27138) Remove AdminUtils calls

2019-03-12 Thread Dylan Guedes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dylan Guedes updated SPARK-27138: - Description: KafkaTestUtils (from kafka010) currently uses AdminUtils to create and delete

[jira] [Commented] (SPARK-26251) isnan function not picking non-numeric values

2019-03-12 Thread Thincrs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790914#comment-16790914 ] Thincrs commented on SPARK-26251: - A user of thincrs has selected this issue. Deadline: Tue, Mar 19,

[jira] [Assigned] (SPARK-26089) Handle large corrupt shuffle blocks

2019-03-12 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid reassigned SPARK-26089: Assignee: Ankur Gupta > Handle large corrupt shuffle blocks >

[jira] [Resolved] (SPARK-26089) Handle large corrupt shuffle blocks

2019-03-12 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-26089. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23453

[jira] [Created] (SPARK-27138) Remove AdminUtils calls

2019-03-12 Thread Dylan Guedes (JIRA)
Dylan Guedes created SPARK-27138: Summary: Remove AdminUtils calls Key: SPARK-27138 URL: https://issues.apache.org/jira/browse/SPARK-27138 Project: Spark Issue Type: Task

[jira] [Assigned] (SPARK-27138) Remove AdminUtils calls

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27138: Assignee: (was: Apache Spark) > Remove AdminUtils calls > --- >

[jira] [Assigned] (SPARK-27138) Remove AdminUtils calls

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27138: Assignee: Apache Spark > Remove AdminUtils calls > --- > >

[jira] [Assigned] (SPARK-27010) find out the actual port number when hive.server2.thrift.port=0

2019-03-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27010: - Assignee: zuotingbing > find out the actual port number when hive.server2.thrift.port=0 >

[jira] [Resolved] (SPARK-27010) find out the actual port number when hive.server2.thrift.port=0

2019-03-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27010. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23917

[jira] [Resolved] (SPARK-27090) Removing old LEGACY_DRIVER_IDENTIFIER ("")

2019-03-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27090. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24026

[jira] [Assigned] (SPARK-27090) Removing old LEGACY_DRIVER_IDENTIFIER ("")

2019-03-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27090: - Assignee: Shivu Sondur > Removing old LEGACY_DRIVER_IDENTIFIER ("") >

[jira] [Assigned] (SPARK-23961) pyspark toLocalIterator throws an exception

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23961: Assignee: (was: Apache Spark) > pyspark toLocalIterator throws an exception >

[jira] [Assigned] (SPARK-23961) pyspark toLocalIterator throws an exception

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23961: Assignee: Apache Spark > pyspark toLocalIterator throws an exception >

[jira] [Created] (SPARK-27137) Spark captured variable is null if the code is pasted via :paste

2019-03-12 Thread Osira Ben (JIRA)
Osira Ben created SPARK-27137: - Summary: Spark captured variable is null if the code is pasted via :paste Key: SPARK-27137 URL: https://issues.apache.org/jira/browse/SPARK-27137 Project: Spark

[jira] [Updated] (SPARK-27137) Spark captured variable is null if the code is pasted via :paste

2019-03-12 Thread Osira Ben (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Osira Ben updated SPARK-27137: -- Description: If I execute this piece of code {code:java} val foo = "foo" def f(arg: Any): Unit = {

[jira] [Updated] (SPARK-27112) Spark Scheduler encounters two independent Deadlocks when trying to kill executors either due to dynamic allocation or blacklisting

2019-03-12 Thread Parth Gandhi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Gandhi updated SPARK-27112: - Description: Recently, a few spark users in the organization have reported that their jobs

[jira] [Commented] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790739#comment-16790739 ] Dongjoon Hyun commented on SPARK-27107: --- [~Dhruve Ashar]. Please vote on ORC RC1 after testing

[jira] [Commented] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790672#comment-16790672 ] Dongjoon Hyun commented on SPARK-27107: --- Yep. [~Dhruve Ashar]. I already tested and voted that.

[jira] [Comment Edited] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790672#comment-16790672 ] Dongjoon Hyun edited comment on SPARK-27107 at 3/12/19 3:47 PM: Yep. 

[jira] [Comment Edited] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790672#comment-16790672 ] Dongjoon Hyun edited comment on SPARK-27107 at 3/12/19 3:46 PM: Yep. 

[jira] [Assigned] (SPARK-27041) large partition data cause pyspark with python2.x oom

2019-03-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27041: - Assignee: David Yang > large partition data cause pyspark with python2.x oom >

[jira] [Resolved] (SPARK-27041) large partition data cause pyspark with python2.x oom

2019-03-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27041. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23954

[jira] [Resolved] (SPARK-27125) Add test suite for sql execution page

2019-03-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27125. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24052

[jira] [Assigned] (SPARK-27125) Add test suite for sql execution page

2019-03-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27125: - Assignee: shahid > Add test suite for sql execution page >

[jira] [Commented] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-12 Thread Dhruve Ashar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790610#comment-16790610 ] Dhruve Ashar commented on SPARK-27107: -- Update: The PR was merged in the orc repository. My

[jira] [Commented] (SPARK-23098) Migrate Kafka batch source to v2

2019-03-12 Thread Dylan Guedes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790585#comment-16790585 ] Dylan Guedes commented on SPARK-23098: -- Hi, I would like to work on this one. [~joseph.torres] do

[jira] [Comment Edited] (SPARK-23450) jars option in spark submit is documented in misleading way

2019-03-12 Thread Sujith Chacko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790558#comment-16790558 ] Sujith Chacko edited comment on SPARK-23450 at 3/12/19 1:43 PM: As per

[jira] [Commented] (SPARK-23450) jars option in spark submit is documented in misleading way

2019-03-12 Thread Sujith Chacko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790558#comment-16790558 ] Sujith Chacko commented on SPARK-23450: --- As pr my understanding the jar will be distributed to the

[jira] [Assigned] (SPARK-27136) Remove data source option check_files_exist

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27136: Assignee: Apache Spark > Remove data source option check_files_exist >

[jira] [Updated] (SPARK-27136) Remove data source option check_files_exist

2019-03-12 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-27136: --- Description: The data source option check_files_exist is introduced in In

[jira] [Assigned] (SPARK-27136) Remove data source option check_files_exist

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27136: Assignee: (was: Apache Spark) > Remove data source option check_files_exist >

[jira] [Updated] (SPARK-27136) Remove data source option check_files_exist

2019-03-12 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-27136: --- Description: The data source option check_files_exist is introduced in In

[jira] [Created] (SPARK-27136) Remove data source option check_files_exist

2019-03-12 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-27136: -- Summary: Remove data source option check_files_exist Key: SPARK-27136 URL: https://issues.apache.org/jira/browse/SPARK-27136 Project: Spark Issue Type:

[jira] [Updated] (SPARK-27135) Description column under jobs does not show complete text if there is overflow

2019-03-12 Thread Sandeep Katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Katta updated SPARK-27135: -- Summary: Description column under jobs does not show complete text if there is overflow

[jira] [Assigned] (SPARK-27105) Prevent exponential complexity in ORC `createFilter`

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27105: Assignee: Apache Spark > Prevent exponential complexity in ORC `createFilter` >

[jira] [Assigned] (SPARK-27105) Prevent exponential complexity in ORC `createFilter`

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27105: Assignee: (was: Apache Spark) > Prevent exponential complexity in ORC `createFilter`

[jira] [Commented] (SPARK-27135) Description column under jobs does not complete text if there is overflow

2019-03-12 Thread Sandeep Katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790440#comment-16790440 ] Sandeep Katta commented on SPARK-27135: --- On hover complete Query should be shown > Description

[jira] [Assigned] (SPARK-27135) Description column under jobs does not complete text if there is overflow

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27135: Assignee: Apache Spark > Description column under jobs does not complete text if there

[jira] [Assigned] (SPARK-27135) Description column under jobs does not complete text if there is overflow

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27135: Assignee: (was: Apache Spark) > Description column under jobs does not complete text

[jira] [Created] (SPARK-27135) Description column under jobs does not complete text if there is overflow

2019-03-12 Thread Sandeep Katta (JIRA)
Sandeep Katta created SPARK-27135: - Summary: Description column under jobs does not complete text if there is overflow Key: SPARK-27135 URL: https://issues.apache.org/jira/browse/SPARK-27135 Project:

[jira] [Updated] (SPARK-27135) Description column under jobs does not complete text if there is overflow

2019-03-12 Thread Sandeep Katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Katta updated SPARK-27135: -- Attachment: UIIssue.PNG > Description column under jobs does not complete text if there is

[jira] [Resolved] (SPARK-24486) Slow performance reading ArrayType columns

2019-03-12 Thread Luca Canali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali resolved SPARK-24486. - Resolution: Fixed Fix Version/s: 3.0.0 > Slow performance reading ArrayType columns >

[jira] [Commented] (SPARK-21385) hive-thriftserver register too many listener in listenerbus

2019-03-12 Thread Ekaterina Shurgalina (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790434#comment-16790434 ] Ekaterina Shurgalina commented on SPARK-21385: -- Hi [~honestman] ! Is this issue still

[jira] [Comment Edited] (SPARK-10746) count ( distinct columnref) over () returns wrong result set

2019-03-12 Thread Izek Greenfield (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790428#comment-16790428 ] Izek Greenfield edited comment on SPARK-10746 at 3/12/19 10:41 AM: ---

[jira] [Commented] (SPARK-10746) count ( distinct columnref) over () returns wrong result set

2019-03-12 Thread Izek Greenfield (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790428#comment-16790428 ] Izek Greenfield commented on SPARK-10746: - you can implement that by using: {code:java} // Some

[jira] [Created] (SPARK-27134) array_distinct function does not work correctly with columns containing array of array

2019-03-12 Thread Mike Trenaman (JIRA)
Mike Trenaman created SPARK-27134: - Summary: array_distinct function does not work correctly with columns containing array of array Key: SPARK-27134 URL: https://issues.apache.org/jira/browse/SPARK-27134

[jira] [Created] (SPARK-27132) Improve file source V2 framework

2019-03-12 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-27132: -- Summary: Improve file source V2 framework Key: SPARK-27132 URL: https://issues.apache.org/jira/browse/SPARK-27132 Project: Spark Issue Type: Task

[jira] [Updated] (SPARK-27133) Refactor the REST based spark app management API to follow the new interface

2019-03-12 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-27133: Summary: Refactor the REST based spark app management API to follow the new

[jira] [Updated] (SPARK-27133) Refactor the spark app management API based on REST to follow the new interface

2019-03-12 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-27133: Description: Based on the discussion

[jira] [Created] (SPARK-27133) Refactor the spark app management API based on REST to follow the new interface

2019-03-12 Thread Stavros Kontopoulos (JIRA)
Stavros Kontopoulos created SPARK-27133: --- Summary: Refactor the spark app management API based on REST to follow the new interface Key: SPARK-27133 URL: https://issues.apache.org/jira/browse/SPARK-27133

[jira] [Updated] (SPARK-27133) Refactor the spark app management API based on REST to follow the new interface

2019-03-12 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-27133: Description: Based on the discussion

[jira] [Assigned] (SPARK-27132) Improve file source V2 framework

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27132: Assignee: (was: Apache Spark) > Improve file source V2 framework >

[jira] [Assigned] (SPARK-27132) Improve file source V2 framework

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27132: Assignee: Apache Spark > Improve file source V2 framework >

[jira] [Commented] (SPARK-24486) Slow performance reading ArrayType columns

2019-03-12 Thread Luca Canali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790409#comment-16790409 ] Luca Canali commented on SPARK-24486: - Thanks [~yumwang] for looking at this. Indeed I confirm that

[jira] [Created] (SPARK-27131) Merge funciton in QuantileSummaries

2019-03-12 Thread Mingchao Tan (JIRA)
Mingchao Tan created SPARK-27131: Summary: Merge funciton in QuantileSummaries Key: SPARK-27131 URL: https://issues.apache.org/jira/browse/SPARK-27131 Project: Spark Issue Type: Question

[jira] [Assigned] (SPARK-27130) Automatically select profile when executing sbt-checkstyle

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27130: Assignee: Apache Spark > Automatically select profile when executing sbt-checkstyle >

[jira] [Assigned] (SPARK-27130) Automatically select profile when executing sbt-checkstyle

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27130: Assignee: (was: Apache Spark) > Automatically select profile when executing

  1   2   >