[jira] [Created] (SPARK-23296) Diagnostics message for user code exceptions should include the stacktrace

Gera Shegalov created SPARK-23296: - Summary: Diagnostics message for user code exceptions should include the stacktrace Key: SPARK-23296 URL: https://issues.apache.org/jira/browse/SPARK-23296 Project:

[jira] [Commented] (SPARK-23296) Diagnostics message for user code exceptions should include the stacktrace

[ https://issues.apache.org/jira/browse/SPARK-23296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348168#comment-16348168 ] Apache Spark commented on SPARK-23296: -- User 'gerashegalov' has created a pull reque

[jira] [Assigned] (SPARK-23296) Diagnostics message for user code exceptions should include the stacktrace

[ https://issues.apache.org/jira/browse/SPARK-23296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23296: Assignee: (was: Apache Spark) > Diagnostics message for user code exceptions should in

[jira] [Assigned] (SPARK-23296) Diagnostics message for user code exceptions should include the stacktrace

[ https://issues.apache.org/jira/browse/SPARK-23296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23296: Assignee: Apache Spark > Diagnostics message for user code exceptions should include the s

[jira] [Commented] (SPARK-23280) add map type support to ColumnVector

[ https://issues.apache.org/jira/browse/SPARK-23280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348185#comment-16348185 ] Apache Spark commented on SPARK-23280: -- User 'ueshin' has created a pull request for

[jira] [Commented] (SPARK-23271) Parquet output contains only "_SUCCESS" file after empty DataFrame saving

[ https://issues.apache.org/jira/browse/SPARK-23271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348217#comment-16348217 ] Pavlo Z. commented on SPARK-23271: -- Yes, SPARK-15393 is similar, linked in JIRA to this

[jira] [Assigned] (SPARK-23200) Reset configuration when restarting from checkpoints

[ https://issues.apache.org/jira/browse/SPARK-23200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23200: Assignee: Apache Spark > Reset configuration when restarting from checkpoints > --

[jira] [Assigned] (SPARK-23200) Reset configuration when restarting from checkpoints

[ https://issues.apache.org/jira/browse/SPARK-23200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23200: Assignee: (was: Apache Spark) > Reset configuration when restarting from checkpoints >

[jira] [Commented] (SPARK-23290) inadvertent change in handling of DateType when converting to pandas dataframe

[ https://issues.apache.org/jira/browse/SPARK-23290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348223#comment-16348223 ] Nick Pentreath commented on SPARK-23290: cc [~bryanc] > inadvertent change in ha

[jira] [Updated] (SPARK-23297) Spark job is finished but the stage process is error

[ https://issues.apache.org/jira/browse/SPARK-23297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-23297: -- Description: I set the log level is WARN, and run spark job using spark-sql. My job is finished

[jira] [Created] (SPARK-23297) Spark job is finished but the stage process is error

KaiXinXIaoLei created SPARK-23297: - Summary: Spark job is finished but the stage process is error Key: SPARK-23297 URL: https://issues.apache.org/jira/browse/SPARK-23297 Project: Spark Issue

[jira] [Updated] (SPARK-23297) Spark job is finished but the stage process is error

[ https://issues.apache.org/jira/browse/SPARK-23297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-23297: -- Attachment: job finished but stage process is error.png > Spark job is finished but the stage p

[jira] [Resolved] (SPARK-23107) ML, Graph 2.3 QA: API: New Scala APIs, docs

[ https://issues.apache.org/jira/browse/SPARK-23107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-23107. Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20459 [https:/

[jira] [Resolved] (SPARK-23110) ML 2.3 QA: API: Java compatibility, docs

[ https://issues.apache.org/jira/browse/SPARK-23110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-23110. Resolution: Resolved Fix Version/s: 2.3.0 > ML 2.3 QA: API: Java compatibility, docs

[jira] [Resolved] (SPARK-23105) Spark MLlib, GraphX 2.3 QA umbrella

[ https://issues.apache.org/jira/browse/SPARK-23105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-23105. Resolution: Resolved Fix Version/s: 2.3.0 > Spark MLlib, GraphX 2.3 QA umbrella > --

[jira] [Commented] (SPARK-23266) Matrix Inversion on BlockMatrix

[ https://issues.apache.org/jira/browse/SPARK-23266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348310#comment-16348310 ] Chandan Misra commented on SPARK-23266: --- *How big is n typically for your use case?

[jira] [Commented] (SPARK-23297) Spark job is finished but the stage process is error

[ https://issues.apache.org/jira/browse/SPARK-23297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348332#comment-16348332 ] Yuming Wang commented on SPARK-23297: - It seems because {{SparkListenerTaskEnd}} even

[jira] [Created] (SPARK-23298) distinct.count on Dataset/DataFrame yields non-deterministic results

Mateusz Jukiewicz created SPARK-23298: - Summary: distinct.count on Dataset/DataFrame yields non-deterministic results Key: SPARK-23298 URL: https://issues.apache.org/jira/browse/SPARK-23298 Projec

[jira] [Commented] (SPARK-22751) Improve ML RandomForest shuffle performance

[ https://issues.apache.org/jira/browse/SPARK-22751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348430#comment-16348430 ] Apache Spark commented on SPARK-22751: -- User 'lucio-yz' has created a pull request f

[jira] [Assigned] (SPARK-22751) Improve ML RandomForest shuffle performance

[ https://issues.apache.org/jira/browse/SPARK-22751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22751: Assignee: Apache Spark > Improve ML RandomForest shuffle performance > ---

[jira] [Assigned] (SPARK-22751) Improve ML RandomForest shuffle performance

[ https://issues.apache.org/jira/browse/SPARK-22751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22751: Assignee: (was: Apache Spark) > Improve ML RandomForest shuffle performance >

[jira] [Created] (SPARK-23299) __repr__ broken for Rows instantiated with *args

Oli Hall created SPARK-23299: Summary: __repr__ broken for Rows instantiated with *args Key: SPARK-23299 URL: https://issues.apache.org/jira/browse/SPARK-23299 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-23202) Add new API in DataSourceWriter: onDataWriterCommit

[ https://issues.apache.org/jira/browse/SPARK-23202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-23202: --- Assignee: Gengliang Wang > Add new API in DataSourceWriter: onDataWriterCommit > ---

[jira] [Resolved] (SPARK-23202) Add new API in DataSourceWriter: onDataWriterCommit

[ https://issues.apache.org/jira/browse/SPARK-23202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23202. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20454 [https://githu

[jira] [Created] (SPARK-23300) Print out if NumPy, Pandas and PyArrow are installed or not in tests

Hyukjin Kwon created SPARK-23300: Summary: Print out if NumPy, Pandas and PyArrow are installed or not in tests Key: SPARK-23300 URL: https://issues.apache.org/jira/browse/SPARK-23300 Project: Spark

[jira] [Updated] (SPARK-23300) Print out if Pandas and PyArrow are installed or not in tests

[ https://issues.apache.org/jira/browse/SPARK-23300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23300: - Summary: Print out if Pandas and PyArrow are installed or not in tests (was: Print out if NumPy,

[jira] [Resolved] (SPARK-23289) OneForOneBlockFetcher.DownloadCallback.onData may write just a part of data

[ https://issues.apache.org/jira/browse/SPARK-23289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23289. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20461 [https://githu

[jira] [Commented] (SPARK-22575) Making Spark Thrift Server clean up its cache

[ https://issues.apache.org/jira/browse/SPARK-22575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348548#comment-16348548 ] Marco Gaido commented on SPARK-22575: - I am not able to reproduce the issue. May I as

[jira] [Assigned] (SPARK-23300) Print out if Pandas and PyArrow are installed or not in tests

[ https://issues.apache.org/jira/browse/SPARK-23300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23300: Assignee: (was: Apache Spark) > Print out if Pandas and PyArrow are installed or not i

[jira] [Comment Edited] (SPARK-22575) Making Spark Thrift Server clean up its cache

[ https://issues.apache.org/jira/browse/SPARK-22575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348548#comment-16348548 ] Marco Gaido edited comment on SPARK-22575 at 2/1/18 1:06 PM: -

[jira] [Commented] (SPARK-23300) Print out if Pandas and PyArrow are installed or not in tests

[ https://issues.apache.org/jira/browse/SPARK-23300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348547#comment-16348547 ] Apache Spark commented on SPARK-23300: -- User 'HyukjinKwon' has created a pull reques

[jira] [Assigned] (SPARK-23300) Print out if Pandas and PyArrow are installed or not in tests

[ https://issues.apache.org/jira/browse/SPARK-23300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23300: Assignee: Apache Spark > Print out if Pandas and PyArrow are installed or not in tests > -

[jira] [Commented] (SPARK-22575) Making Spark Thrift Server clean up its cache

[ https://issues.apache.org/jira/browse/SPARK-22575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348555#comment-16348555 ] Oz Ben-Ami commented on SPARK-22575: [~mgaido] The issue was apparently related to dy

[jira] [Updated] (SPARK-23295) Exclude Waring message when generating versions in make-distribution.sh

[ https://issues.apache.org/jira/browse/SPARK-23295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-23295: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) > Exclude Waring message when g

[jira] [Commented] (SPARK-22575) Making Spark Thrift Server clean up its cache

[ https://issues.apache.org/jira/browse/SPARK-22575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348603#comment-16348603 ] Marco Gaido commented on SPARK-22575: - Then the problem is likely that the executors

[jira] [Assigned] (SPARK-23235) Add executor Threaddump to api

[ https://issues.apache.org/jira/browse/SPARK-23235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23235: Assignee: Apache Spark > Add executor Threaddump to api > -- >

[jira] [Commented] (SPARK-23235) Add executor Threaddump to api

[ https://issues.apache.org/jira/browse/SPARK-23235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348616#comment-16348616 ] Apache Spark commented on SPARK-23235: -- User 'attilapiros' has created a pull reques

[jira] [Commented] (SPARK-22575) Making Spark Thrift Server clean up its cache

[ https://issues.apache.org/jira/browse/SPARK-22575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348617#comment-16348617 ] Oz Ben-Ami commented on SPARK-22575: That makes sense. Not sure why we only saw this

[jira] [Assigned] (SPARK-23235) Add executor Threaddump to api

[ https://issues.apache.org/jira/browse/SPARK-23235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23235: Assignee: (was: Apache Spark) > Add executor Threaddump to api > -

[jira] [Updated] (SPARK-23296) Diagnostics message for user code exceptions should include the stacktrace

[ https://issues.apache.org/jira/browse/SPARK-23296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-23296: -- Shepherd: (was: Sean Owen) > Diagnostics message for user code exceptions should include the stacktra

[jira] [Updated] (SPARK-23296) Diagnostics message for user code exceptions should include the stacktrace

[ https://issues.apache.org/jira/browse/SPARK-23296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-23296: -- Priority: Trivial (was: Major) > Diagnostics message for user code exceptions should include the stack

[jira] [Commented] (SPARK-22575) Making Spark Thrift Server clean up its cache

[ https://issues.apache.org/jira/browse/SPARK-22575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348622#comment-16348622 ] Marco Gaido commented on SPARK-22575: - I think STS is the only Spark application wher

[jira] [Created] (SPARK-23301) data source v2 column pruning with arbitrary expressions is broken

Wenchen Fan created SPARK-23301: --- Summary: data source v2 column pruning with arbitrary expressions is broken Key: SPARK-23301 URL: https://issues.apache.org/jira/browse/SPARK-23301 Project: Spark

[jira] [Assigned] (SPARK-23256) Add columnSchema method to PySpark image reader

[ https://issues.apache.org/jira/browse/SPARK-23256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23256: Assignee: Apache Spark > Add columnSchema method to PySpark image reader > ---

[jira] [Commented] (SPARK-23256) Add columnSchema method to PySpark image reader

[ https://issues.apache.org/jira/browse/SPARK-23256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348643#comment-16348643 ] Apache Spark commented on SPARK-23256: -- User 'HyukjinKwon' has created a pull reques

[jira] [Assigned] (SPARK-23256) Add columnSchema method to PySpark image reader

[ https://issues.apache.org/jira/browse/SPARK-23256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23256: Assignee: (was: Apache Spark) > Add columnSchema method to PySpark image reader >

[jira] [Assigned] (SPARK-23301) data source v2 column pruning with arbitrary expressions is broken

[ https://issues.apache.org/jira/browse/SPARK-23301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23301: Assignee: Apache Spark (was: Wenchen Fan) > data source v2 column pruning with arbitrary

[jira] [Assigned] (SPARK-23301) data source v2 column pruning with arbitrary expressions is broken

[ https://issues.apache.org/jira/browse/SPARK-23301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23301: Assignee: Wenchen Fan (was: Apache Spark) > data source v2 column pruning with arbitrary

[jira] [Commented] (SPARK-23301) data source v2 column pruning with arbitrary expressions is broken

[ https://issues.apache.org/jira/browse/SPARK-23301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348658#comment-16348658 ] Apache Spark commented on SPARK-23301: -- User 'cloud-fan' has created a pull request

[jira] [Created] (SPARK-23302) Refactor group aggregate pandas UDF to its own catalyst rules

Li Jin created SPARK-23302: -- Summary: Refactor group aggregate pandas UDF to its own catalyst rules Key: SPARK-23302 URL: https://issues.apache.org/jira/browse/SPARK-23302 Project: Spark Issue Type

[jira] [Created] (SPARK-23303) improve the explain result for data source v2 relations

Wenchen Fan created SPARK-23303: --- Summary: improve the explain result for data source v2 relations Key: SPARK-23303 URL: https://issues.apache.org/jira/browse/SPARK-23303 Project: Spark Issue T

[jira] [Assigned] (SPARK-23303) improve the explain result for data source v2 relations

[ https://issues.apache.org/jira/browse/SPARK-23303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23303: Assignee: Apache Spark (was: Wenchen Fan) > improve the explain result for data source v2

[jira] [Assigned] (SPARK-23303) improve the explain result for data source v2 relations

[ https://issues.apache.org/jira/browse/SPARK-23303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23303: Assignee: Wenchen Fan (was: Apache Spark) > improve the explain result for data source v2

[jira] [Commented] (SPARK-23303) improve the explain result for data source v2 relations

[ https://issues.apache.org/jira/browse/SPARK-23303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348737#comment-16348737 ] Apache Spark commented on SPARK-23303: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-23235) Add executor Threaddump to api

[ https://issues.apache.org/jira/browse/SPARK-23235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348758#comment-16348758 ] Imran Rashid commented on SPARK-23235: -- Just brainstorming I'd like to get some opin

[jira] [Commented] (SPARK-8835) Provide pluggable Congestion Strategies to deal with Streaming load

[ https://issues.apache.org/jira/browse/SPARK-8835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348886#comment-16348886 ] Apache Spark commented on SPARK-8835: - User 'EmergentOrder' has created a pull request

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348980#comment-16348980 ] Kazuaki Ishizaki commented on SPARK-18016: -- This issue has been fixed in Spark 2

[jira] [Commented] (SPARK-19275) Spark Streaming, Kafka receiver, "Failed to get records for ... after polling for 512"

[ https://issues.apache.org/jira/browse/SPARK-19275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348996#comment-16348996 ] Riccardo Vincelli commented on SPARK-19275: --- Hi, I would like to point out that

[jira] [Commented] (SPARK-23292) python tests related to pandas are skipped

[ https://issues.apache.org/jira/browse/SPARK-23292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349040#comment-16349040 ] Yin Huai commented on SPARK-23292: -- So, jenkins does have the right version of pandas an

[jira] [Created] (SPARK-23304) Spark SQL coalesce() against hive not working

Thomas Graves created SPARK-23304: - Summary: Spark SQL coalesce() against hive not working Key: SPARK-23304 URL: https://issues.apache.org/jira/browse/SPARK-23304 Project: Spark Issue Type: B

[jira] [Created] (SPARK-23305) Add `spark.sql.files.ignoreMissingFiles` for ORC data source

Dongjoon Hyun created SPARK-23305: - Summary: Add `spark.sql.files.ignoreMissingFiles` for ORC data source Key: SPARK-23305 URL: https://issues.apache.org/jira/browse/SPARK-23305 Project: Spark

[jira] [Updated] (SPARK-23305) Add `spark.sql.files.ignoreMissingFiles` test case for ORC

[ https://issues.apache.org/jira/browse/SPARK-23305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23305: -- Summary: Add `spark.sql.files.ignoreMissingFiles` test case for ORC (was: Add `spark.sql.files

[jira] [Updated] (SPARK-23305) Add `spark.sql.files.ignoreMissingFiles` test case for ORC data source

[ https://issues.apache.org/jira/browse/SPARK-23305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23305: -- Summary: Add `spark.sql.files.ignoreMissingFiles` test case for ORC data source (was: Add `spa

[jira] [Commented] (SPARK-23305) Add `spark.sql.files.ignoreMissingFiles` test case for ORC

[ https://issues.apache.org/jira/browse/SPARK-23305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349054#comment-16349054 ] Apache Spark commented on SPARK-23305: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Assigned] (SPARK-23305) Add `spark.sql.files.ignoreMissingFiles` test case for ORC

[ https://issues.apache.org/jira/browse/SPARK-23305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23305: Assignee: Apache Spark > Add `spark.sql.files.ignoreMissingFiles` test case for ORC >

[jira] [Assigned] (SPARK-23305) Add `spark.sql.files.ignoreMissingFiles` test case for ORC

[ https://issues.apache.org/jira/browse/SPARK-23305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23305: Assignee: (was: Apache Spark) > Add `spark.sql.files.ignoreMissingFiles` test case for

[jira] [Updated] (SPARK-23290) inadvertent change in handling of DateType when converting to pandas dataframe

[ https://issues.apache.org/jira/browse/SPARK-23290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-23290: Priority: Blocker (was: Major) > inadvertent change in handling of DateType when converting to pandas data

[jira] [Resolved] (SPARK-13983) HiveThriftServer2 can not get "--hiveconf" or ''--hivevar" variables since 1.6 version (both multi-session and single session)

[ https://issues.apache.org/jira/browse/SPARK-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-13983. - Resolution: Fixed Assignee: Yuming Wang (was: Cheng Lian) Fix Version/s: 2.3.0 > HiveThr

[jira] [Commented] (SPARK-15473) CSV fails to write and read back empty dataframe

[ https://issues.apache.org/jira/browse/SPARK-15473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349075#comment-16349075 ] Andrey Taptunov commented on SPARK-15473: - Just want to point out that example fr

[jira] [Commented] (SPARK-23107) ML, Graph 2.3 QA: API: New Scala APIs, docs

[ https://issues.apache.org/jira/browse/SPARK-23107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349090#comment-16349090 ] Sameer Agarwal commented on SPARK-23107: Thanks [~yanboliang], I'll cut the next

[jira] [Resolved] (SPARK-23301) data source v2 column pruning with arbitrary expressions is broken

[ https://issues.apache.org/jira/browse/SPARK-23301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23301. - Resolution: Fixed Fix Version/s: 2.3.0 > data source v2 column pruning with arbitrary expressions

[jira] [Assigned] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-23304: --- Assignee: Xiao Li > Spark SQL coalesce() against hive not working >

[jira] [Updated] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-23304: Target Version/s: 2.3.0 > Spark SQL coalesce() against hive not working > -

[jira] [Commented] (SPARK-23294) Spark Streaming + Rate source + Console Sink : Receiver MaxRate is violated

[ https://issues.apache.org/jira/browse/SPARK-23294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349111#comment-16349111 ] Ravinder Matte commented on SPARK-23294: [~maropu] Please do let me know if you n

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349147#comment-16349147 ] Sameer Agarwal commented on SPARK-23304: [~tgraves] just to rule out the obvious,

[jira] [Created] (SPARK-23306) Race condition in TaskMemoryManager

Zhan Zhang created SPARK-23306: -- Summary: Race condition in TaskMemoryManager Key: SPARK-23306 URL: https://issues.apache.org/jira/browse/SPARK-23306 Project: Spark Issue Type: Bug Com

[jira] [Commented] (SPARK-23306) Race condition in TaskMemoryManager

[ https://issues.apache.org/jira/browse/SPARK-23306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349203#comment-16349203 ] Apache Spark commented on SPARK-23306: -- User 'zhzhan' has created a pull request for

[jira] [Assigned] (SPARK-23306) Race condition in TaskMemoryManager

[ https://issues.apache.org/jira/browse/SPARK-23306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23306: Assignee: (was: Apache Spark) > Race condition in TaskMemoryManager >

[jira] [Assigned] (SPARK-23306) Race condition in TaskMemoryManager

[ https://issues.apache.org/jira/browse/SPARK-23306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23306: Assignee: Apache Spark > Race condition in TaskMemoryManager > ---

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349212#comment-16349212 ] Thomas Graves commented on SPARK-23304: --- yes there are difference in the # of parti

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349220#comment-16349220 ] Thomas Graves commented on SPARK-23304: --- If it helps , spark 2.3 # partitions is 31

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349227#comment-16349227 ] Thomas Graves commented on SPARK-23304: --- Ok, I just realized what you are getting a

[jira] [Updated] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-23304: -- Priority: Major (was: Blocker) > Spark SQL coalesce() against hive not working > -

[jira] [Updated] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-23304: -- Description: The query below seems to ignore the coalesce. This is running spark 2.2 or spark

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349270#comment-16349270 ] Thomas Graves commented on SPARK-23304: --- so with the new ORC code is there anyway t

[jira] [Created] (SPARK-23307) Spark UI should sort jobs/stages with the completed timestamp before cleaning up them

Shixiong Zhu created SPARK-23307: Summary: Spark UI should sort jobs/stages with the completed timestamp before cleaning up them Key: SPARK-23307 URL: https://issues.apache.org/jira/browse/SPARK-23307

[jira] [Commented] (SPARK-23307) Spark UI should sort jobs/stages with the completed timestamp before cleaning up them

[ https://issues.apache.org/jira/browse/SPARK-23307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349288#comment-16349288 ] Shixiong Zhu commented on SPARK-23307: -- cc [~vanzin] [~cloud_fan] > Spark UI should

[jira] [Updated] (SPARK-23307) Spark UI should sort jobs/stages with the completed timestamp before cleaning up them

[ https://issues.apache.org/jira/browse/SPARK-23307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23307: - Description: When you have a long running job, it may be deleted from UI quickly when it complet

[jira] [Assigned] (SPARK-23307) Spark UI should sort jobs/stages with the completed timestamp before cleaning up them

[ https://issues.apache.org/jira/browse/SPARK-23307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-23307: Assignee: Shixiong Zhu > Spark UI should sort jobs/stages with the completed timestamp bef

[jira] [Updated] (SPARK-23292) python tests related to pandas are skipped

[ https://issues.apache.org/jira/browse/SPARK-23292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-23292: - Priority: Critical (was: Blocker) > python tests related to pandas are skipped > ---

[jira] [Updated] (SPARK-23307) Spark UI should sort jobs/stages with the completed timestamp before cleaning up them

[ https://issues.apache.org/jira/browse/SPARK-23307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sameer Agarwal updated SPARK-23307: --- Target Version/s: 2.3.0 > Spark UI should sort jobs/stages with the completed timestamp befor

[jira] [Updated] (SPARK-23307) Spark UI should sort jobs/stages with the completed timestamp before cleaning up them

[ https://issues.apache.org/jira/browse/SPARK-23307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sameer Agarwal updated SPARK-23307: --- Priority: Blocker (was: Major) > Spark UI should sort jobs/stages with the completed timesta

[jira] [Commented] (SPARK-23307) Spark UI should sort jobs/stages with the completed timestamp before cleaning up them

[ https://issues.apache.org/jira/browse/SPARK-23307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349323#comment-16349323 ] Sameer Agarwal commented on SPARK-23307: Bumping this to a blocker for 2.3 > Spa

[jira] [Commented] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349325#comment-16349325 ] Yin Huai commented on SPARK-12297: -- [~zi] has this issue got resolved in Hive? I see HIV

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349339#comment-16349339 ] Xiao Li commented on SPARK-23304: - Hi, [~tgraves], could you change the two SQLConf `spa

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349340#comment-16349340 ] Xiao Li commented on SPARK-23304: - I do not think our native ORC reader respects `hive.ex

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349346#comment-16349346 ] Sameer Agarwal commented on SPARK-23304: Also, is there a JIRA/repro for the cach

[jira] [Commented] (SPARK-23284) Document several get API of ColumnVector's behavior when accessing null slot

[ https://issues.apache.org/jira/browse/SPARK-23284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349353#comment-16349353 ] Sameer Agarwal commented on SPARK-23284: Thanks, given the other open blockers, w

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349367#comment-16349367 ] Thomas Graves commented on SPARK-23304: --- ok I've attached 2 files one with spark 2.

[jira] [Updated] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-23304: -- Attachment: spark23_oldorc_explain.txt spark22_oldorc_explain.txt > Spark SQL c

  1   2   >