[jira] [Updated] (SPARK-27145) Close store after test, in the SQLAppStatusListenerSuite

2019-03-12 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-27145: --- Summary: Close store after test, in the SQLAppStatusListenerSuite (was: Close store after test, in SQLAppSt

[jira] [Created] (SPARK-27144) Explode with structType may throw NPE

2019-03-12 Thread Yoga (JIRA)
Yoga created SPARK-27144: Summary: Explode with structType may throw NPE Key: SPARK-27144 URL: https://issues.apache.org/jira/browse/SPARK-27144 Project: Spark Issue Type: Bug Components: S

[jira] [Assigned] (SPARK-27145) Close store after test, in the SQLAppStatusListenerSuite

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27145: Assignee: (was: Apache Spark) > Close store after test, in the SQLAppStatusListenerSu

[jira] [Updated] (SPARK-27144) Explode with structType may throw NPE when the first column's nullable is false while the second column's nullable is true

2019-03-12 Thread Yoga (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yoga updated SPARK-27144: - Summary: Explode with structType may throw NPE when the first column's nullable is false while the second column

[jira] [Comment Edited] (SPARK-27144) Explode with structType may throw NPE

2019-03-12 Thread Yoga (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791376#comment-16791376 ] Yoga edited comment on SPARK-27144 at 3/13/19 6:48 AM: --- I think th

[jira] [Comment Edited] (SPARK-27144) Explode with structType may throw NPE

2019-03-12 Thread Yoga (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791376#comment-16791376 ] Yoga edited comment on SPARK-27144 at 3/13/19 6:48 AM: --- I think th

[jira] [Created] (SPARK-27145) Close store after test, in SQLAppStatusListenerSuite

2019-03-12 Thread shahid (JIRA)
shahid created SPARK-27145: -- Summary: Close store after test, in SQLAppStatusListenerSuite Key: SPARK-27145 URL: https://issues.apache.org/jira/browse/SPARK-27145 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-27017) Creating orc table with special symbols in column name via spark.sql

2019-03-12 Thread Chakravarthi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791364#comment-16791364 ] Chakravarthi commented on SPARK-27017: -- Hi, [~uNxe] sorry for delay, please check t

[jira] [Commented] (SPARK-25449) Don't send zero accumulators in heartbeats

2019-03-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791321#comment-16791321 ] Xiao Li commented on SPARK-25449: - This changed the unit of conf. > Don't send zero ac

[jira] [Updated] (SPARK-25449) Don't send zero accumulators in heartbeats

2019-03-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25449: Labels: release-notes (was: ) > Don't send zero accumulators in heartbeats >

[jira] [Commented] (SPARK-27141) Use ConfigEntry for hardcoded configs Yarn

2019-03-12 Thread Sandeep Katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791284#comment-16791284 ] Sandeep Katta commented on SPARK-27141: --- I will be working on this PR > Use Confi

[jira] [Assigned] (SPARK-27142) Provide REST API for SQL level information

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27142: Assignee: Apache Spark > Provide REST API for SQL level information > ---

[jira] [Commented] (SPARK-27143) Provide REST API for JDBC/ODBC level information

2019-03-12 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791272#comment-16791272 ] Ajith S commented on SPARK-27143: - ping [~srowen] [~cloud_fan] [~dongjoon]  Please sugg

[jira] [Commented] (SPARK-27143) Provide REST API for JDBC/ODBC level information

2019-03-12 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791270#comment-16791270 ] Ajith S commented on SPARK-27143: - I will be working on this > Provide REST API for JDB

[jira] [Created] (SPARK-27143) Provide REST API for JDBC/ODBC level information

2019-03-12 Thread Ajith S (JIRA)
Ajith S created SPARK-27143: --- Summary: Provide REST API for JDBC/ODBC level information Key: SPARK-27143 URL: https://issues.apache.org/jira/browse/SPARK-27143 Project: Spark Issue Type: New Featur

[jira] [Commented] (SPARK-27142) Provide REST API for SQL level information

2019-03-12 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791269#comment-16791269 ] Ajith S commented on SPARK-27142: - I will be working on this > Provide REST API for SQL

[jira] [Assigned] (SPARK-27142) Provide REST API for SQL level information

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27142: Assignee: (was: Apache Spark) > Provide REST API for SQL level information >

[jira] [Created] (SPARK-27142) Provide REST API for SQL level information

2019-03-12 Thread Ajith S (JIRA)
Ajith S created SPARK-27142: --- Summary: Provide REST API for SQL level information Key: SPARK-27142 URL: https://issues.apache.org/jira/browse/SPARK-27142 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-27141) Use ConfigEntry for hardcoded configs Yarn

2019-03-12 Thread wangjiaochun (JIRA)
wangjiaochun created SPARK-27141: Summary: Use ConfigEntry for hardcoded configs Yarn Key: SPARK-27141 URL: https://issues.apache.org/jira/browse/SPARK-27141 Project: Spark Issue Type: Sub-ta

[jira] [Updated] (SPARK-26936) On yarn-client mode, insert overwrite local directory can not create temporary path in local staging directory

2019-03-12 Thread jiaan.geng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-26936: --- Summary: On yarn-client mode, insert overwrite local directory can not create temporary path in loca

[jira] [Updated] (SPARK-26936) insert overwrite local directory can not create temporary path in local staging directory

2019-03-12 Thread jiaan.geng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-26936: --- Description: Let me introduce bug of  'insert overwrite local directory'. If I execute the SQL ment

[jira] [Assigned] (SPARK-27140) The feature is 'insert overwrite local directory' has an inconsistent behavior in different environment.

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27140: Assignee: (was: Apache Spark) > The feature is 'insert overwrite local directory' has

[jira] [Assigned] (SPARK-27140) The feature is 'insert overwrite local directory' has an inconsistent behavior in different environment.

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27140: Assignee: Apache Spark > The feature is 'insert overwrite local directory' has an inconsi

[jira] [Commented] (SPARK-27140) The feature is 'insert overwrite local directory' has an inconsistent behavior in different environment.

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791236#comment-16791236 ] Apache Spark commented on SPARK-27140: -- User 'beliefer' has created a pull request

[jira] [Updated] (SPARK-27140) The feature is 'insert overwrite local directory' has an inconsistent behavior in different environment.

2019-03-12 Thread jiaan.geng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-27140: --- Description: In local[*] mode, maropu give a test case as follows: {code:java} $ls /tmp/noexistdir l

[jira] [Resolved] (SPARK-26976) Forbid reserved keywords as identifiers when ANSI mode is on

2019-03-12 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-26976. -- Resolution: Fixed Fix Version/s: 3.0.0 Resolved by https://github.com/apache/sp

[jira] [Created] (SPARK-27140) The feature is 'insert overwrite local directory' has an inconsistent behavior in different environment.

2019-03-12 Thread jiaan.geng (JIRA)
jiaan.geng created SPARK-27140: -- Summary: The feature is 'insert overwrite local directory' has an inconsistent behavior in different environment. Key: SPARK-27140 URL: https://issues.apache.org/jira/browse/SPARK-271

[jira] [Resolved] (SPARK-27137) Spark captured variable is null if the code is pasted via :paste

2019-03-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27137. -- Resolution: Cannot Reproduce Seems not failing in the current master: {code} scala> :paste //

[jira] [Commented] (SPARK-27131) Merge funciton in QuantileSummaries

2019-03-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791217#comment-16791217 ] Hyukjin Kwon commented on SPARK-27131: -- Let's ask questions to mailing list. You co

[jira] [Resolved] (SPARK-27131) Merge funciton in QuantileSummaries

2019-03-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27131. -- Resolution: Invalid > Merge funciton in QuantileSummaries > --

[jira] [Resolved] (SPARK-27045) SQL tab in UI shows actual SQL instead of callsite

2019-03-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27045. --- Resolution: Fixed Assignee: Ajith S Fix Version/s: 3.0.0 This is resolved vi

[jira] [Updated] (SPARK-27045) SQL tab in UI shows actual SQL instead of callsite in case of SparkSQLDriver

2019-03-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27045: -- Description: When we run sql in spark via SparkSQLDriver (thrift server, spark-sql), SQL stri

[jira] [Updated] (SPARK-27045) SQL tab in UI shows actual SQL instead of callsite in case of SparkSQLDriver

2019-03-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27045: -- Summary: SQL tab in UI shows actual SQL instead of callsite in case of SparkSQLDriver (was: S

[jira] [Resolved] (SPARK-27130) Automatically select profile when executing sbt-checkstyle

2019-03-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27130. -- Resolution: Fixed Fix Version/s: 3.0.0 Fixed in https://github.com/apache/spark/pull/24

[jira] [Assigned] (SPARK-27130) Automatically select profile when executing sbt-checkstyle

2019-03-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-27130: Assignee: Hyukjin Kwon > Automatically select profile when executing sbt-checkstyle > ---

[jira] [Assigned] (SPARK-27130) Automatically select profile when executing sbt-checkstyle

2019-03-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-27130: Assignee: Yuming Wang (was: Hyukjin Kwon) > Automatically select profile when executing

[jira] [Commented] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791075#comment-16791075 ] Dongjoon Hyun commented on SPARK-27107: --- Thank you for confirmation, [~Dhruve Asha

[jira] [Resolved] (SPARK-27034) Nested schema pruning for ORC

2019-03-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27034. --- Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 3.0.0 This is res

[jira] [Assigned] (SPARK-26176) Verify column name when creating table via `STORED AS`

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26176: Assignee: Apache Spark > Verify column name when creating table via `STORED AS` > ---

[jira] [Assigned] (SPARK-26176) Verify column name when creating table via `STORED AS`

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26176: Assignee: (was: Apache Spark) > Verify column name when creating table via `STORED AS

[jira] [Commented] (SPARK-26176) Verify column name when creating table via `STORED AS`

2019-03-12 Thread Sujith Chacko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791060#comment-16791060 ] Sujith Chacko commented on SPARK-26176: --- Issue is still happening with spark 2.4 l

[jira] [Assigned] (SPARK-27134) array_distinct function does not work correctly with columns containing array of array

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27134: Assignee: Apache Spark > array_distinct function does not work correctly with columns con

[jira] [Resolved] (SPARK-27123) Improve CollapseProject to handle projects cross limit/repartition/sample

2019-03-12 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-27123. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24049 [https://github.com/a

[jira] [Assigned] (SPARK-27134) array_distinct function does not work correctly with columns containing array of array

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27134: Assignee: (was: Apache Spark) > array_distinct function does not work correctly with

[jira] [Resolved] (SPARK-27139) NettyBlockTransferService does not abide by spark.blockManager.port config option

2019-03-12 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin resolved SPARK-27139. Resolution: Invalid And proper casing does the trick > NettyBlockTransferService does not

[jira] [Updated] (SPARK-27139) NettyBlockTransferService does not abide by spark.blockManager.port config option

2019-03-12 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated SPARK-27139: --- Description: This is a regression from a fix in SPARK-4837 The NettyBlockTransferService al

[jira] [Assigned] (SPARK-26927) Race condition may cause dynamic allocation not working

2019-03-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26927: -- Assignee: liupengcheng > Race condition may cause dynamic allocation not working > --

[jira] [Resolved] (SPARK-26927) Race condition may cause dynamic allocation not working

2019-03-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26927. Resolution: Fixed Fix Version/s: 2.3.4 2.4.2 3

[jira] [Created] (SPARK-27139) NettyBlockTransferService does not abide by spark.blockManager.port config option

2019-03-12 Thread Bolke de Bruin (JIRA)
Bolke de Bruin created SPARK-27139: -- Summary: NettyBlockTransferService does not abide by spark.blockManager.port config option Key: SPARK-27139 URL: https://issues.apache.org/jira/browse/SPARK-27139

[jira] [Commented] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-12 Thread Dhruve Ashar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790911#comment-16790911 ] Dhruve Ashar commented on SPARK-27107: -- I verified the changes and we are no longer

[jira] [Commented] (SPARK-27087) Inability to access to column alias in pyspark

2019-03-12 Thread Thincrs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790929#comment-16790929 ] Thincrs commented on SPARK-27087: - A user of thincrs has selected this issue. Deadline:

[jira] [Updated] (SPARK-27138) Remove AdminUtils calls

2019-03-12 Thread Dylan Guedes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dylan Guedes updated SPARK-27138: - Description: KafkaTestUtils (from kafka010) currently uses AdminUtils to create and delete topic

[jira] [Commented] (SPARK-26251) isnan function not picking non-numeric values

2019-03-12 Thread Thincrs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790914#comment-16790914 ] Thincrs commented on SPARK-26251: - A user of thincrs has selected this issue. Deadline:

[jira] [Assigned] (SPARK-26089) Handle large corrupt shuffle blocks

2019-03-12 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid reassigned SPARK-26089: Assignee: Ankur Gupta > Handle large corrupt shuffle blocks > ---

[jira] [Resolved] (SPARK-26089) Handle large corrupt shuffle blocks

2019-03-12 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-26089. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23453 [https://gi

[jira] [Created] (SPARK-27138) Remove AdminUtils calls

2019-03-12 Thread Dylan Guedes (JIRA)
Dylan Guedes created SPARK-27138: Summary: Remove AdminUtils calls Key: SPARK-27138 URL: https://issues.apache.org/jira/browse/SPARK-27138 Project: Spark Issue Type: Task Components

[jira] [Assigned] (SPARK-27138) Remove AdminUtils calls

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27138: Assignee: (was: Apache Spark) > Remove AdminUtils calls > --- > >

[jira] [Assigned] (SPARK-27138) Remove AdminUtils calls

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27138: Assignee: Apache Spark > Remove AdminUtils calls > --- > >

[jira] [Assigned] (SPARK-27010) find out the actual port number when hive.server2.thrift.port=0

2019-03-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27010: - Assignee: zuotingbing > find out the actual port number when hive.server2.thrift.port=0 > -

[jira] [Resolved] (SPARK-27010) find out the actual port number when hive.server2.thrift.port=0

2019-03-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27010. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23917 [https://github.c

[jira] [Resolved] (SPARK-27090) Removing old LEGACY_DRIVER_IDENTIFIER ("")

2019-03-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27090. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24026 [https://github.c

[jira] [Assigned] (SPARK-27090) Removing old LEGACY_DRIVER_IDENTIFIER ("")

2019-03-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27090: - Assignee: Shivu Sondur > Removing old LEGACY_DRIVER_IDENTIFIER ("") > -

[jira] [Assigned] (SPARK-23961) pyspark toLocalIterator throws an exception

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23961: Assignee: (was: Apache Spark) > pyspark toLocalIterator throws an exception > ---

[jira] [Assigned] (SPARK-23961) pyspark toLocalIterator throws an exception

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23961: Assignee: Apache Spark > pyspark toLocalIterator throws an exception > --

[jira] [Created] (SPARK-27137) Spark captured variable is null if the code is pasted via :paste

2019-03-12 Thread Osira Ben (JIRA)
Osira Ben created SPARK-27137: - Summary: Spark captured variable is null if the code is pasted via :paste Key: SPARK-27137 URL: https://issues.apache.org/jira/browse/SPARK-27137 Project: Spark I

[jira] [Updated] (SPARK-27137) Spark captured variable is null if the code is pasted via :paste

2019-03-12 Thread Osira Ben (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Osira Ben updated SPARK-27137: -- Description: If I execute this piece of code {code:java} val foo = "foo" def f(arg: Any): Unit = {

[jira] [Updated] (SPARK-27112) Spark Scheduler encounters two independent Deadlocks when trying to kill executors either due to dynamic allocation or blacklisting

2019-03-12 Thread Parth Gandhi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Gandhi updated SPARK-27112: - Description: Recently, a few spark users in the organization have reported that their jobs were

[jira] [Commented] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790739#comment-16790739 ] Dongjoon Hyun commented on SPARK-27107: --- [~Dhruve Ashar]. Please vote on ORC RC1 a

[jira] [Commented] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790672#comment-16790672 ] Dongjoon Hyun commented on SPARK-27107: --- Yep. [~Dhruve Ashar]. I already tested an

[jira] [Comment Edited] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790672#comment-16790672 ] Dongjoon Hyun edited comment on SPARK-27107 at 3/12/19 3:47 PM: --

[jira] [Comment Edited] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790672#comment-16790672 ] Dongjoon Hyun edited comment on SPARK-27107 at 3/12/19 3:46 PM: --

[jira] [Assigned] (SPARK-27041) large partition data cause pyspark with python2.x oom

2019-03-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27041: - Assignee: David Yang > large partition data cause pyspark with python2.x oom >

[jira] [Resolved] (SPARK-27041) large partition data cause pyspark with python2.x oom

2019-03-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27041. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23954 [https://github.c

[jira] [Resolved] (SPARK-27125) Add test suite for sql execution page

2019-03-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27125. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24052 [https://github.c

[jira] [Assigned] (SPARK-27125) Add test suite for sql execution page

2019-03-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27125: - Assignee: shahid > Add test suite for sql execution page >

[jira] [Commented] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-12 Thread Dhruve Ashar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790610#comment-16790610 ] Dhruve Ashar commented on SPARK-27107: -- Update: The PR was merged in the orc reposi

[jira] [Commented] (SPARK-23098) Migrate Kafka batch source to v2

2019-03-12 Thread Dylan Guedes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790585#comment-16790585 ] Dylan Guedes commented on SPARK-23098: -- Hi, I would like to work on this one. [~jos

[jira] [Comment Edited] (SPARK-23450) jars option in spark submit is documented in misleading way

2019-03-12 Thread Sujith Chacko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790558#comment-16790558 ] Sujith Chacko edited comment on SPARK-23450 at 3/12/19 1:43 PM: --

[jira] [Commented] (SPARK-23450) jars option in spark submit is documented in misleading way

2019-03-12 Thread Sujith Chacko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790558#comment-16790558 ] Sujith Chacko commented on SPARK-23450: --- As pr my understanding the jar will be di

[jira] [Assigned] (SPARK-27136) Remove data source option check_files_exist

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27136: Assignee: Apache Spark > Remove data source option check_files_exist > --

[jira] [Updated] (SPARK-27136) Remove data source option check_files_exist

2019-03-12 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-27136: --- Description: The data source option check_files_exist is introduced in In https://github.co

[jira] [Assigned] (SPARK-27136) Remove data source option check_files_exist

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27136: Assignee: (was: Apache Spark) > Remove data source option check_files_exist > ---

[jira] [Updated] (SPARK-27136) Remove data source option check_files_exist

2019-03-12 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-27136: --- Description: The data source option check_files_exist is introduced in In https://github.co

[jira] [Created] (SPARK-27136) Remove data source option check_files_exist

2019-03-12 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-27136: -- Summary: Remove data source option check_files_exist Key: SPARK-27136 URL: https://issues.apache.org/jira/browse/SPARK-27136 Project: Spark Issue Type: T

[jira] [Updated] (SPARK-27135) Description column under jobs does not show complete text if there is overflow

2019-03-12 Thread Sandeep Katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Katta updated SPARK-27135: -- Summary: Description column under jobs does not show complete text if there is overflow (was:

[jira] [Assigned] (SPARK-27105) Prevent exponential complexity in ORC `createFilter`

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27105: Assignee: Apache Spark > Prevent exponential complexity in ORC `createFilter` > ---

[jira] [Assigned] (SPARK-27105) Prevent exponential complexity in ORC `createFilter`

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27105: Assignee: (was: Apache Spark) > Prevent exponential complexity in ORC `createFilter`

[jira] [Commented] (SPARK-27135) Description column under jobs does not complete text if there is overflow

2019-03-12 Thread Sandeep Katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790440#comment-16790440 ] Sandeep Katta commented on SPARK-27135: --- On hover complete Query should be shown

[jira] [Assigned] (SPARK-27135) Description column under jobs does not complete text if there is overflow

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27135: Assignee: Apache Spark > Description column under jobs does not complete text if there is

[jira] [Assigned] (SPARK-27135) Description column under jobs does not complete text if there is overflow

2019-03-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27135: Assignee: (was: Apache Spark) > Description column under jobs does not complete text

[jira] [Created] (SPARK-27135) Description column under jobs does not complete text if there is overflow

2019-03-12 Thread Sandeep Katta (JIRA)
Sandeep Katta created SPARK-27135: - Summary: Description column under jobs does not complete text if there is overflow Key: SPARK-27135 URL: https://issues.apache.org/jira/browse/SPARK-27135 Project:

[jira] [Updated] (SPARK-27135) Description column under jobs does not complete text if there is overflow

2019-03-12 Thread Sandeep Katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Katta updated SPARK-27135: -- Attachment: UIIssue.PNG > Description column under jobs does not complete text if there is ove

[jira] [Resolved] (SPARK-24486) Slow performance reading ArrayType columns

2019-03-12 Thread Luca Canali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali resolved SPARK-24486. - Resolution: Fixed Fix Version/s: 3.0.0 > Slow performance reading ArrayType columns > ---

[jira] [Commented] (SPARK-21385) hive-thriftserver register too many listener in listenerbus

2019-03-12 Thread Ekaterina Shurgalina (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790434#comment-16790434 ] Ekaterina Shurgalina commented on SPARK-21385: -- Hi [~honestman] ! Is this

[jira] [Comment Edited] (SPARK-10746) count ( distinct columnref) over () returns wrong result set

2019-03-12 Thread Izek Greenfield (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790428#comment-16790428 ] Izek Greenfield edited comment on SPARK-10746 at 3/12/19 10:41 AM: ---

[jira] [Commented] (SPARK-10746) count ( distinct columnref) over () returns wrong result set

2019-03-12 Thread Izek Greenfield (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790428#comment-16790428 ] Izek Greenfield commented on SPARK-10746: - you can implement that by using: {co

[jira] [Created] (SPARK-27134) array_distinct function does not work correctly with columns containing array of array

2019-03-12 Thread Mike Trenaman (JIRA)
Mike Trenaman created SPARK-27134: - Summary: array_distinct function does not work correctly with columns containing array of array Key: SPARK-27134 URL: https://issues.apache.org/jira/browse/SPARK-27134

[jira] [Created] (SPARK-27132) Improve file source V2 framework

2019-03-12 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-27132: -- Summary: Improve file source V2 framework Key: SPARK-27132 URL: https://issues.apache.org/jira/browse/SPARK-27132 Project: Spark Issue Type: Task

[jira] [Updated] (SPARK-27133) Refactor the REST based spark app management API to follow the new interface

2019-03-12 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-27133: Summary: Refactor the REST based spark app management API to follow the new interf

[jira] [Updated] (SPARK-27133) Refactor the spark app management API based on REST to follow the new interface

2019-03-12 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-27133: Description: Based on the discussion [here|https://github.com/apache/spark/pull/23

  1   2   >