[jira] [Created] (SPARK-22744) Add a configuration to show the application submit hostname

Lantao Jin created SPARK-22744: -- Summary: Add a configuration to show the application submit hostname Key: SPARK-22744 URL: https://issues.apache.org/jira/browse/SPARK-22744 Project: Spark Issu

[jira] [Assigned] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-12297: Assignee: Imran Rashid > Add work-around for Parquet/Hive int96 timestamp bug. > -

[jira] [Resolved] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12297. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19769 [https://git

[jira] [Commented] (SPARK-22672) Refactor ORC Tests

[ https://issues.apache.org/jira/browse/SPARK-22672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16284511#comment-16284511 ] Apache Spark commented on SPARK-22672: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Updated] (SPARK-22743) Consolidate logic for handling spark.driver.memoryOverhead and spark.executor.memoryOverhead

[ https://issues.apache.org/jira/browse/SPARK-22743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yinan Li updated SPARK-22743: - Description: In https://github.com/apache/spark/pull/19717, we will combine {{spark.yarn.{driver|executor

[jira] [Updated] (SPARK-22743) Consolidate logic for handling spark.driver.memoryOverhead and spark.executor.memoryOverhead

[ https://issues.apache.org/jira/browse/SPARK-22743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yinan Li updated SPARK-22743: - Description: In https://github.com/apache/spark/pull/19717, we will combine {{spark.yarn.[driver|executor

[jira] [Created] (SPARK-22743) Consolidate logic for handling spark.driver.memoryOverhead and spark.executor.memoryOverhead

Yinan Li created SPARK-22743: Summary: Consolidate logic for handling spark.driver.memoryOverhead and spark.executor.memoryOverhead Key: SPARK-22743 URL: https://issues.apache.org/jira/browse/SPARK-22743

[jira] [Commented] (SPARK-18673) Dataframes doesn't work on Hadoop 3.x; Hive rejects Hadoop version

[ https://issues.apache.org/jira/browse/SPARK-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16284485#comment-16284485 ] Mingjie Tang commented on SPARK-18673: -- Hi Steve, the hive side is updated. https:/

[jira] [Commented] (SPARK-20555) Incorrect handling of Oracle's decimal types via JDBC

[ https://issues.apache.org/jira/browse/SPARK-20555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16284465#comment-16284465 ] Gabor Feher commented on SPARK-20555: - This issues was fixed, so in theory, you shoul

[jira] [Updated] (SPARK-22324) Upgrade Arrow to version 0.8.0

[ https://issues.apache.org/jira/browse/SPARK-22324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-22324: - Fix Version/s: 2.3.0 > Upgrade Arrow to version 0.8.0 > -- > >

[jira] [Resolved] (SPARK-22742) Spark2.x does not support read data from Hive 2.2 and 2.3

[ https://issues.apache.org/jira/browse/SPARK-22742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22742. --- Resolution: Won't Fix You didn't say what the error is, but guessing it's a duplicate of https://iss

[jira] [Created] (SPARK-22742) Spark2.x does not support read data from Hive 2.2 and 2.3

Zhongting Hu created SPARK-22742: Summary: Spark2.x does not support read data from Hive 2.2 and 2.3 Key: SPARK-22742 URL: https://issues.apache.org/jira/browse/SPARK-22742 Project: Spark Iss

[jira] [Commented] (SPARK-22279) Turn on spark.sql.hive.convertMetastoreOrc by default

[ https://issues.apache.org/jira/browse/SPARK-22279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16284336#comment-16284336 ] Apache Spark commented on SPARK-22279: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Assigned] (SPARK-21672) Remove SHS-specific application / attempt data structures

[ https://issues.apache.org/jira/browse/SPARK-21672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-21672: -- Assignee: Sandor Murakozi > Remove SHS-specific application / attempt data structures

[jira] [Resolved] (SPARK-21672) Remove SHS-specific application / attempt data structures

[ https://issues.apache.org/jira/browse/SPARK-21672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-21672. Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19920 [https:/

[jira] [Updated] (SPARK-22741) Add global aggregate for typed aggregation

[ https://issues.apache.org/jira/browse/SPARK-22741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruben Janssen updated SPARK-22741: -- Description: A global aggregate for typed aggregation is currently missing. When an aggregatio

[jira] [Created] (SPARK-22741) Add global aggregate for typed aggregation

Ruben Janssen created SPARK-22741: - Summary: Add global aggregate for typed aggregation Key: SPARK-22741 URL: https://issues.apache.org/jira/browse/SPARK-22741 Project: Spark Issue Type: New

[jira] [Resolved] (SPARK-16599) java.util.NoSuchElementException: None.get at at org.apache.spark.storage.BlockInfoManager.releaseAllLocksForTask(BlockInfoManager.scala:343)

[ https://issues.apache.org/jira/browse/SPARK-16599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16599. --- Resolution: Not A Problem OK. I added a minor improvement to the error handling, and haven't had con

[jira] [Commented] (SPARK-22629) incorrect handling of calls to random in UDFs

[ https://issues.apache.org/jira/browse/SPARK-22629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16284166#comment-16284166 ] Marco Gaido commented on SPARK-22629: - This problem is related to SPARK-20586. I crea

[jira] [Commented] (SPARK-22629) incorrect handling of calls to random in UDFs

[ https://issues.apache.org/jira/browse/SPARK-22629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16284164#comment-16284164 ] Apache Spark commented on SPARK-22629: -- User 'mgaido91' has created a pull request f

[jira] [Assigned] (SPARK-22629) incorrect handling of calls to random in UDFs

[ https://issues.apache.org/jira/browse/SPARK-22629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22629: Assignee: Apache Spark > incorrect handling of calls to random in UDFs > -

[jira] [Assigned] (SPARK-22629) incorrect handling of calls to random in UDFs

[ https://issues.apache.org/jira/browse/SPARK-22629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22629: Assignee: (was: Apache Spark) > incorrect handling of calls to random in UDFs > --

[jira] [Commented] (SPARK-16599) java.util.NoSuchElementException: None.get at at org.apache.spark.storage.BlockInfoManager.releaseAllLocksForTask(BlockInfoManager.scala:343)

[ https://issues.apache.org/jira/browse/SPARK-16599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16284163#comment-16284163 ] Alex Nastetsky commented on SPARK-16599: Ran into this while unit testing and can

[jira] [Updated] (SPARK-22346) Update VectorAssembler to work with Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-22346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-22346: -- Target Version/s: 2.3.0 > Update VectorAssembler to work with Structured Streaming > --

[jira] [Updated] (SPARK-22346) Update VectorAssembler to work with Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-22346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-22346: -- Shepherd: Joseph K. Bradley > Update VectorAssembler to work with Structured Streaming

[jira] [Commented] (SPARK-22740) [SQL][JDBC] Reserved SQL words are not escaped for table names

[ https://issues.apache.org/jira/browse/SPARK-22740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16284100#comment-16284100 ] Yuming Wang commented on SPARK-22740: - As Sean Owen said, pass {{`group`}} as the arg

[jira] [Commented] (SPARK-22740) [SQL][JDBC] Reserved SQL words are not escaped for table names

[ https://issues.apache.org/jira/browse/SPARK-22740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16284029#comment-16284029 ] Sean Owen commented on SPARK-22740: --- You would need to pass {{`group`}} as the argument

[jira] [Created] (SPARK-22740) [SQL][JDBC] Reserved SQL words are not escaped for table names

Nicolas Lalevée created SPARK-22740: --- Summary: [SQL][JDBC] Reserved SQL words are not escaped for table names Key: SPARK-22740 URL: https://issues.apache.org/jira/browse/SPARK-22740 Project: Spark

[jira] [Commented] (SPARK-22126) Fix model-specific optimization support for ML tuning

[ https://issues.apache.org/jira/browse/SPARK-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283982#comment-16283982 ] Tomas Nykodym commented on SPARK-22126: --- I think Weichen's API is more flexible and

[jira] [Created] (SPARK-22739) Additional Expression Support for Objects

Aleksander Eskilson created SPARK-22739: --- Summary: Additional Expression Support for Objects Key: SPARK-22739 URL: https://issues.apache.org/jira/browse/SPARK-22739 Project: Spark Issue

[jira] [Updated] (SPARK-22738) Spark YARN UI does not create fully qualified links for paging impacts use with Knox

[ https://issues.apache.org/jira/browse/SPARK-22738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Groves updated SPARK-22738: --- Summary: Spark YARN UI does not create fully qualified links for paging impacts use with Knox

[jira] [Updated] (SPARK-22738) Spark YARN UI does not create fully qualified links for paging

[ https://issues.apache.org/jira/browse/SPARK-22738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Groves updated SPARK-22738: --- Description: The paging links within the Spark (Yarn) UI are not created as fully qualified

[jira] [Created] (SPARK-22738) Spark YARN UI does not create fully qualified links for paging

Jeffrey Groves created SPARK-22738: -- Summary: Spark YARN UI does not create fully qualified links for paging Key: SPARK-22738 URL: https://issues.apache.org/jira/browse/SPARK-22738 Project: Spark

[jira] [Assigned] (SPARK-22655) Fail task instead of complete task silently in PythonRunner during shutdown

[ https://issues.apache.org/jira/browse/SPARK-22655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-22655: Assignee: Li Jin > Fail task instead of complete task silently in PythonRunner during shut

[jira] [Resolved] (SPARK-22655) Fail task instead of complete task silently in PythonRunner during shutdown

[ https://issues.apache.org/jira/browse/SPARK-22655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22655. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19852 [https://git

[jira] [Comment Edited] (SPARK-22267) Spark SQL incorrectly reads ORC file when column order is different

[ https://issues.apache.org/jira/browse/SPARK-22267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283410#comment-16283410 ] Dongjoon Hyun edited comment on SPARK-22267 at 12/8/17 11:29 AM: --

[jira] [Commented] (SPARK-22267) Spark SQL incorrectly reads ORC file when column order is different

[ https://issues.apache.org/jira/browse/SPARK-22267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283410#comment-16283410 ] Dongjoon Hyun commented on SPARK-22267: --- Hi, [~mpetruska] and [~cloud_fan]. After S

[jira] [Commented] (SPARK-22267) Spark SQL incorrectly reads ORC file when column order is different

[ https://issues.apache.org/jira/browse/SPARK-22267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283407#comment-16283407 ] Apache Spark commented on SPARK-22267: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Resolved] (SPARK-22279) Turn on spark.sql.hive.convertMetastoreOrc by default

[ https://issues.apache.org/jira/browse/SPARK-22279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-22279. --- Resolution: Fixed Fix Version/s: 2.3.0 > Turn on spark.sql.hive.convertMetastoreOrc by