[jira] [Commented] (SPARK-11083) insert overwrite table failed when beeline reconnect

[ https://issues.apache.org/jira/browse/SPARK-11083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559297#comment-16559297 ] readme_kylin commented on SPARK-11083: -- is any one  working on this issue? spark 2.1.0 thrift

[jira] [Resolved] (SPARK-24929) Merge script swallow KeyboardInterrupt

[ https://issues.apache.org/jira/browse/SPARK-24929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24929. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21880

[jira] [Assigned] (SPARK-24929) Merge script swallow KeyboardInterrupt

[ https://issues.apache.org/jira/browse/SPARK-24929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24929: Assignee: Hyukjin Kwon > Merge script swallow KeyboardInterrupt >

[jira] [Resolved] (SPARK-24829) In Spark Thrift Server, CAST AS FLOAT inconsistent with spark-shell or spark-sql

[ https://issues.apache.org/jira/browse/SPARK-24829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24829. -- Resolution: Fixed Assignee: zuotingbing Fix Version/s: 2.4.0 Fixed in

[jira] [Created] (SPARK-24942) Improve cluster resource management with jobs containing barrier stage

Jiang Xingbo created SPARK-24942: Summary: Improve cluster resource management with jobs containing barrier stage Key: SPARK-24942 URL: https://issues.apache.org/jira/browse/SPARK-24942 Project:

[jira] [Created] (SPARK-24941) Add RDDBarrier.coalesce() function

Jiang Xingbo created SPARK-24941: Summary: Add RDDBarrier.coalesce() function Key: SPARK-24941 URL: https://issues.apache.org/jira/browse/SPARK-24941 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-24801) Empty byte[] arrays in spark.network.sasl.SaslEncryption$EncryptedMessage can waste a lot of memory

[ https://issues.apache.org/jira/browse/SPARK-24801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-24801. -- Resolution: Fixed Assignee: Misha Dmitriev Fix Version/s: 2.4.0 Fixed in

[jira] [Commented] (SPARK-24932) Allow update mode for streaming queries with join

[ https://issues.apache.org/jira/browse/SPARK-24932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559181#comment-16559181 ] Apache Spark commented on SPARK-24932: -- User 'fuyufjh' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24932) Allow update mode for streaming queries with join

[ https://issues.apache.org/jira/browse/SPARK-24932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24932: Assignee: (was: Apache Spark) > Allow update mode for streaming queries with join >

[jira] [Assigned] (SPARK-24932) Allow update mode for streaming queries with join

[ https://issues.apache.org/jira/browse/SPARK-24932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24932: Assignee: Apache Spark > Allow update mode for streaming queries with join >

[jira] [Updated] (SPARK-24932) Allow update mode for streaming queries with join

[ https://issues.apache.org/jira/browse/SPARK-24932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Fu updated SPARK-24932: Description: In issue SPARK-19140 we supported update output mode for non-aggregation streaming queries.

[jira] [Commented] (SPARK-4502) Spark SQL reads unneccesary nested fields from Parquet

[ https://issues.apache.org/jira/browse/SPARK-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559083#comment-16559083 ] Apache Spark commented on SPARK-4502: - User 'ajacques' has created a pull request for this issue:

[jira] [Updated] (SPARK-24940) Coalesce Hint for SQL Queries

[ https://issues.apache.org/jira/browse/SPARK-24940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated SPARK-24940: --- Summary: Coalesce Hint for SQL Queries (was: Coalesce Hint for SQL) > Coalesce Hint for SQL

[jira] [Updated] (SPARK-24940) Coalesce Hint for SQL

[ https://issues.apache.org/jira/browse/SPARK-24940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated SPARK-24940: --- Description: Many Spark SQL users in my company have asked for a way to control the number of

[jira] [Created] (SPARK-24940) Coalesce Hint for SQL

John Zhuge created SPARK-24940: -- Summary: Coalesce Hint for SQL Key: SPARK-24940 URL: https://issues.apache.org/jira/browse/SPARK-24940 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-24919) Scala linter rule for sparkContext.hadoopConfiguration

[ https://issues.apache.org/jira/browse/SPARK-24919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24919. - Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 2.4.0 > Scala linter rule for

[jira] [Commented] (SPARK-24253) DataSourceV2: Add DeleteSupport for delete and overwrite operations

[ https://issues.apache.org/jira/browse/SPARK-24253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558971#comment-16558971 ] Apache Spark commented on SPARK-24253: -- User 'rdblue' has created a pull request for this issue:

[jira] [Issue Comment Deleted] (SPARK-24937) Datasource partition table should load empty partitions

[ https://issues.apache.org/jira/browse/SPARK-24937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24937: Comment: was deleted (was: I'm working on.) > Datasource partition table should load empty

[jira] [Created] (SPARK-24939) Support YARN Shared Cache in Spark

Jonathan Bender created SPARK-24939: --- Summary: Support YARN Shared Cache in Spark Key: SPARK-24939 URL: https://issues.apache.org/jira/browse/SPARK-24939 Project: Spark Issue Type:

[jira] [Commented] (SPARK-24918) Executor Plugin API

[ https://issues.apache.org/jira/browse/SPARK-24918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558829#comment-16558829 ] Imran Rashid commented on SPARK-24918: -- The only thing I *really* needed was just to be able to

[jira] [Updated] (SPARK-24801) Empty byte[] arrays in spark.network.sasl.SaslEncryption$EncryptedMessage can waste a lot of memory

[ https://issues.apache.org/jira/browse/SPARK-24801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-24801: - Labels: memory-analysis (was: ) > Empty byte[] arrays in

[jira] [Commented] (SPARK-24938) Understand usage of netty's onheap memory use, even with offheap pools

[ https://issues.apache.org/jira/browse/SPARK-24938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558798#comment-16558798 ] Imran Rashid commented on SPARK-24938: -- This should be an easy change to make, its just a question

[jira] [Created] (SPARK-24938) Understand usage of netty's onheap memory use, even with offheap pools

Imran Rashid created SPARK-24938: Summary: Understand usage of netty's onheap memory use, even with offheap pools Key: SPARK-24938 URL: https://issues.apache.org/jira/browse/SPARK-24938 Project:

[jira] [Assigned] (SPARK-23633) Update Pandas UDFs section in sql-programming-guide

[ https://issues.apache.org/jira/browse/SPARK-23633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23633: Assignee: (was: Apache Spark) > Update Pandas UDFs section in sql-programming-guide

[jira] [Commented] (SPARK-23633) Update Pandas UDFs section in sql-programming-guide

[ https://issues.apache.org/jira/browse/SPARK-23633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558796#comment-16558796 ] Apache Spark commented on SPARK-23633: -- User 'icexelloss' has created a pull request for this

[jira] [Assigned] (SPARK-23633) Update Pandas UDFs section in sql-programming-guide

[ https://issues.apache.org/jira/browse/SPARK-23633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23633: Assignee: Apache Spark > Update Pandas UDFs section in sql-programming-guide >

[jira] [Resolved] (SPARK-24795) Implement barrier execution mode

[ https://issues.apache.org/jira/browse/SPARK-24795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24795. - Resolution: Fixed Assignee: Jiang Xingbo Fix Version/s: 2.4.0 > Implement barrier

[jira] [Resolved] (SPARK-14543) SQL/Hive insertInto has unexpected results

[ https://issues.apache.org/jira/browse/SPARK-14543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Blue resolved SPARK-14543. --- Resolution: Later This is addressed by SPARK-24251 for DataSourceV2 writers. > SQL/Hive insertInto

[jira] [Comment Edited] (SPARK-24882) separate responsibilities of the data source v2 read API

[ https://issues.apache.org/jira/browse/SPARK-24882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558666#comment-16558666 ] Ryan Blue edited comment on SPARK-24882 at 7/26/18 6:19 PM: [~cloud_fan],

[jira] [Comment Edited] (SPARK-24882) separate responsibilities of the data source v2 read API

[ https://issues.apache.org/jira/browse/SPARK-24882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558666#comment-16558666 ] Ryan Blue edited comment on SPARK-24882 at 7/26/18 6:18 PM: [~cloud_fan],

[jira] [Commented] (SPARK-21274) Implement EXCEPT ALL and INTERSECT ALL

[ https://issues.apache.org/jira/browse/SPARK-21274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558687#comment-16558687 ] Apache Spark commented on SPARK-21274: -- User 'dilipbiswal' has created a pull request for this

[jira] [Assigned] (SPARK-24926) Ensure numCores is used consistently in all netty configuration (driver and executors)

[ https://issues.apache.org/jira/browse/SPARK-24926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24926: Assignee: Apache Spark > Ensure numCores is used consistently in all netty configuration

[jira] [Assigned] (SPARK-24926) Ensure numCores is used consistently in all netty configuration (driver and executors)

[ https://issues.apache.org/jira/browse/SPARK-24926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24926: Assignee: (was: Apache Spark) > Ensure numCores is used consistently in all netty

[jira] [Commented] (SPARK-24926) Ensure numCores is used consistently in all netty configuration (driver and executors)

[ https://issues.apache.org/jira/browse/SPARK-24926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558688#comment-16558688 ] Apache Spark commented on SPARK-24926: -- User 'NiharS' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24927) The hadoop-provided profile doesn't play well with Snappy-compressed Parquet files

[ https://issues.apache.org/jira/browse/SPARK-24927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-24927: -- Assignee: Cheng Lian > The hadoop-provided profile doesn't play well with Snappy-compressed

[jira] [Commented] (SPARK-24882) separate responsibilities of the data source v2 read API

[ https://issues.apache.org/jira/browse/SPARK-24882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558666#comment-16558666 ] Ryan Blue commented on SPARK-24882: --- [~cloud_fan], I'm adding some suggestions here because comments

[jira] [Commented] (SPARK-23683) FileCommitProtocol.instantiate to require 3-arg constructor for dynamic partition overwrite

[ https://issues.apache.org/jira/browse/SPARK-23683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558660#comment-16558660 ] Steve Loughran commented on SPARK-23683: If it's a regression, you could argue for it >

[jira] [Updated] (SPARK-24927) The hadoop-provided profile doesn't play well with Snappy-compressed Parquet files

[ https://issues.apache.org/jira/browse/SPARK-24927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-24927: --- Description: Reproduction: {noformat} wget

[jira] [Updated] (SPARK-24934) Complex type and binary type in in-memory partition pruning does not work due to missing upper/lower bounds cases

[ https://issues.apache.org/jira/browse/SPARK-24934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24934: - Summary: Complex type and binary type in in-memory partition pruning does not work due to

[jira] [Assigned] (SPARK-24937) Datasource partition table should load empty partitions

[ https://issues.apache.org/jira/browse/SPARK-24937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24937: Assignee: (was: Apache Spark) > Datasource partition table should load empty

[jira] [Assigned] (SPARK-24937) Datasource partition table should load empty partitions

[ https://issues.apache.org/jira/browse/SPARK-24937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24937: Assignee: Apache Spark > Datasource partition table should load empty partitions >

[jira] [Commented] (SPARK-24937) Datasource partition table should load empty partitions

[ https://issues.apache.org/jira/browse/SPARK-24937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558438#comment-16558438 ] Apache Spark commented on SPARK-24937: -- User 'wangyum' has created a pull request for this issue:

[jira] [Updated] (SPARK-24937) Datasource partition table should load empty partitions

[ https://issues.apache.org/jira/browse/SPARK-24937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24937: Description: How to reproduce: {code:sql} spark-sql> CREATE TABLE tbl AS SELECT 1; spark-sql>

[jira] [Commented] (SPARK-24937) Datasource partition table should load empty partitions

[ https://issues.apache.org/jira/browse/SPARK-24937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558386#comment-16558386 ] Yuming Wang commented on SPARK-24937: - I'm working on. > Datasource partition table should load

[jira] [Created] (SPARK-24937) Datasource partition table should load empty partitions

Yuming Wang created SPARK-24937: --- Summary: Datasource partition table should load empty partitions Key: SPARK-24937 URL: https://issues.apache.org/jira/browse/SPARK-24937 Project: Spark Issue

[jira] [Updated] (SPARK-24937) Datasource partition table should load empty partitions

[ https://issues.apache.org/jira/browse/SPARK-24937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24937: Description: How to reproduce: {code:sql} spark-sql> CREATE TABLE tbl AS SELECT 1; 18/07/26

[jira] [Commented] (SPARK-24934) Should handle missing upper/lower bounds cases in in-memory partition pruning

[ https://issues.apache.org/jira/browse/SPARK-24934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558364#comment-16558364 ] Hyukjin Kwon commented on SPARK-24934: -- np! BTW, the workaround will be turning off

[jira] [Created] (SPARK-24936) Better error message when trying a shuffle fetch over 2 GB

Imran Rashid created SPARK-24936: Summary: Better error message when trying a shuffle fetch over 2 GB Key: SPARK-24936 URL: https://issues.apache.org/jira/browse/SPARK-24936 Project: Spark

[jira] [Commented] (SPARK-24934) Should handle missing upper/lower bounds cases in in-memory partition pruning

[ https://issues.apache.org/jira/browse/SPARK-24934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558360#comment-16558360 ] David Vogelbacher commented on SPARK-24934: --- Thanks for opening and making the pr

[jira] [Created] (SPARK-24935) Problem with Executing Hive UDF's from Spark 2.2 Onwards

Parth Gandhi created SPARK-24935: Summary: Problem with Executing Hive UDF's from Spark 2.2 Onwards Key: SPARK-24935 URL: https://issues.apache.org/jira/browse/SPARK-24935 Project: Spark

[jira] [Commented] (SPARK-24918) Executor Plugin API

[ https://issues.apache.org/jira/browse/SPARK-24918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558331#comment-16558331 ] Thomas Graves commented on SPARK-24918: --- I think this is a good idea. I thought I had seen a Jira

[jira] [Assigned] (SPARK-24934) Should handle missing upper/lower bounds cases in in-memory partition pruning

[ https://issues.apache.org/jira/browse/SPARK-24934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24934: Assignee: Apache Spark > Should handle missing upper/lower bounds cases in in-memory

[jira] [Assigned] (SPARK-24934) Should handle missing upper/lower bounds cases in in-memory partition pruning

[ https://issues.apache.org/jira/browse/SPARK-24934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24934: Assignee: (was: Apache Spark) > Should handle missing upper/lower bounds cases in

[jira] [Commented] (SPARK-24934) Should handle missing upper/lower bounds cases in in-memory partition pruning

[ https://issues.apache.org/jira/browse/SPARK-24934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558295#comment-16558295 ] Apache Spark commented on SPARK-24934: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Created] (SPARK-24934) Should handle missing upper/lower bounds cases in in-memory partition pruning

Hyukjin Kwon created SPARK-24934: Summary: Should handle missing upper/lower bounds cases in in-memory partition pruning Key: SPARK-24934 URL: https://issues.apache.org/jira/browse/SPARK-24934

[jira] [Commented] (SPARK-12911) Cacheing a dataframe causes array comparisons to fail (in filter / where) after 1.6

[ https://issues.apache.org/jira/browse/SPARK-12911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558291#comment-16558291 ] Hyukjin Kwon commented on SPARK-12911: -- I opened - SPARK-24934 > Cacheing a dataframe causes array

[jira] [Commented] (SPARK-24928) spark sql cross join running time too long

[ https://issues.apache.org/jira/browse/SPARK-24928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558287#comment-16558287 ] Marco Gaido commented on SPARK-24928: - The affected version is pretty old, can you check a newer

[jira] [Created] (SPARK-24933) SinkProgress should report written rows

Vaclav Kosar created SPARK-24933: Summary: SinkProgress should report written rows Key: SPARK-24933 URL: https://issues.apache.org/jira/browse/SPARK-24933 Project: Spark Issue Type:

[jira] [Created] (SPARK-24932) Allow update mode for streaming queries with join

Eric Fu created SPARK-24932: --- Summary: Allow update mode for streaming queries with join Key: SPARK-24932 URL: https://issues.apache.org/jira/browse/SPARK-24932 Project: Spark Issue Type:

[jira] [Updated] (SPARK-24931) CoarseGrainedExecutorBackend send wrong 'Reason' when executor exits which leading to job failed.

[ https://issues.apache.org/jira/browse/SPARK-24931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ice bai updated SPARK-24931: Priority: Major (was: Blocker) > CoarseGrainedExecutorBackend send wrong 'Reason' when executor exits

[jira] [Updated] (SPARK-24931) CoarseGrainedExecutorBackend send wrong 'Reason' when executor exits which leading to job failed.

[ https://issues.apache.org/jira/browse/SPARK-24931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ice bai updated SPARK-24931: Summary: CoarseGrainedExecutorBackend send wrong 'Reason' when executor exits which leading to job

[jira] [Created] (SPARK-24931) ExecutorBackend send wrong Reason when executor exits

ice bai created SPARK-24931: --- Summary: ExecutorBackend send wrong Reason when executor exits Key: SPARK-24931 URL: https://issues.apache.org/jira/browse/SPARK-24931 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-24527) select column alias should support quotation marks

[ https://issues.apache.org/jira/browse/SPARK-24527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ice bai updated SPARK-24527: Description: It will be failed when user use spark-sql or sql API to select come columns with quoted

[jira] [Updated] (SPARK-24647) Sink Should Return Writen Offsets For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaclav Kosar updated SPARK-24647: - Description: To be able to track data lineage for Structured Streaming (I intend to implement

[jira] [Updated] (SPARK-24647) Sink Should Return Writen Offsets For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaclav Kosar updated SPARK-24647: - Summary: Sink Should Return Writen Offsets For ProgressReporting (was: Sink Should Return

[jira] [Commented] (SPARK-24897) DAGScheduler should not unregisterMapOutput and increaseEpoch repeatedly for stage fetchFailed

[ https://issues.apache.org/jira/browse/SPARK-24897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558144#comment-16558144 ] liupengcheng commented on SPARK-24897: -- already fixed at 2.x > DAGScheduler should not

[jira] [Updated] (SPARK-24897) DAGScheduler should not unregisterMapOutput and increaseEpoch repeatedly for stage fetchFailed

[ https://issues.apache.org/jira/browse/SPARK-24897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liupengcheng updated SPARK-24897: - Affects Version/s: (was: 2.3.1) (was: 2.1.0)

[jira] [Resolved] (SPARK-24897) DAGScheduler should not unregisterMapOutput and increaseEpoch repeatedly for stage fetchFailed

[ https://issues.apache.org/jira/browse/SPARK-24897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liupengcheng resolved SPARK-24897. -- Resolution: Invalid > DAGScheduler should not unregisterMapOutput and increaseEpoch

[jira] [Updated] (SPARK-24930) Exception information is not accurate when using `LOAD DATA LOCAL INPATH`

[ https://issues.apache.org/jira/browse/SPARK-24930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaochen Ouyang updated SPARK-24930: Description: # root user create a test.txt file contains a record '123'  in /root/

[jira] [Updated] (SPARK-24930) Exception information is not accurate when using `LOAD DATA LOCAL INPATH`

[ https://issues.apache.org/jira/browse/SPARK-24930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaochen Ouyang updated SPARK-24930: Description: # root user create a test.txt file contains a record '123'  in /root/

[jira] [Assigned] (SPARK-24930) Exception information is not accurate when using `LOAD DATA LOCAL INPATH`

[ https://issues.apache.org/jira/browse/SPARK-24930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24930: Assignee: (was: Apache Spark) > Exception information is not accurate when using

[jira] [Commented] (SPARK-24930) Exception information is not accurate when using `LOAD DATA LOCAL INPATH`

[ https://issues.apache.org/jira/browse/SPARK-24930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558090#comment-16558090 ] Apache Spark commented on SPARK-24930: -- User 'ouyangxiaochen' has created a pull request for this

[jira] [Assigned] (SPARK-24930) Exception information is not accurate when using `LOAD DATA LOCAL INPATH`

[ https://issues.apache.org/jira/browse/SPARK-24930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24930: Assignee: Apache Spark > Exception information is not accurate when using `LOAD DATA

[jira] [Commented] (SPARK-12911) Cacheing a dataframe causes array comparisons to fail (in filter / where) after 1.6

[ https://issues.apache.org/jira/browse/SPARK-12911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558087#comment-16558087 ] Hyukjin Kwon commented on SPARK-12911: -- Looks indeed similar. Mind if I ask to open another JIRA

[jira] [Created] (SPARK-24930) Exception information is not accurate when using `LOAD DATA LOCAL INPATH`

Xiaochen Ouyang created SPARK-24930: --- Summary: Exception information is not accurate when using `LOAD DATA LOCAL INPATH` Key: SPARK-24930 URL: https://issues.apache.org/jira/browse/SPARK-24930

[jira] [Updated] (SPARK-24928) spark sql cross join running time too long

[ https://issues.apache.org/jira/browse/SPARK-24928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LIFULONG updated SPARK-24928: - Description: spark sql running time is too long while input left table and right table is small hdfs

[jira] [Commented] (SPARK-24929) Merge script swallow KeyboardInterrupt

[ https://issues.apache.org/jira/browse/SPARK-24929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558065#comment-16558065 ] Apache Spark commented on SPARK-24929: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-24929) Merge script swallow KeyboardInterrupt

[ https://issues.apache.org/jira/browse/SPARK-24929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24929: Assignee: (was: Apache Spark) > Merge script swallow KeyboardInterrupt >

[jira] [Assigned] (SPARK-24929) Merge script swallow KeyboardInterrupt

[ https://issues.apache.org/jira/browse/SPARK-24929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24929: Assignee: Apache Spark > Merge script swallow KeyboardInterrupt >

[jira] [Updated] (SPARK-24928) spark sql cross join running time too long

[ https://issues.apache.org/jira/browse/SPARK-24928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LIFULONG updated SPARK-24928: - Priority: Minor (was: Major) > spark sql cross join running time too long >

[jira] [Created] (SPARK-24929) Merge script swallow KeyboardInterrupt

Hyukjin Kwon created SPARK-24929: Summary: Merge script swallow KeyboardInterrupt Key: SPARK-24929 URL: https://issues.apache.org/jira/browse/SPARK-24929 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-24924) Add mapping for built-in Avro data source

[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24924. -- Resolution: Fixed Fixed in https://github.com/apache/spark/pull/21878 > Add mapping for

[jira] [Updated] (SPARK-24924) Add mapping for built-in Avro data source

[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24924: - Fix Version/s: 2.4.0 > Add mapping for built-in Avro data source >

[jira] [Assigned] (SPARK-24924) Add mapping for built-in Avro data source

[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24924: Assignee: Dongjoon Hyun > Add mapping for built-in Avro data source >

[jira] [Created] (SPARK-24928) spark sql cross join running time too long

LIFULONG created SPARK-24928: Summary: spark sql cross join running time too long Key: SPARK-24928 URL: https://issues.apache.org/jira/browse/SPARK-24928 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-21436) Take advantage of known partioner for distinct on RDDs

[ https://issues.apache.org/jira/browse/SPARK-21436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558013#comment-16558013 ] zhengruifeng edited comment on SPARK-21436 at 7/26/18 7:38 AM: --- [~holdenk]

[jira] [Comment Edited] (SPARK-21436) Take advantage of known partioner for distinct on RDDs

[ https://issues.apache.org/jira/browse/SPARK-21436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558013#comment-16558013 ] zhengruifeng edited comment on SPARK-21436 at 7/26/18 7:37 AM: --- [~holdenk]

[jira] [Commented] (SPARK-21436) Take advantage of known partioner for distinct on RDDs

[ https://issues.apache.org/jira/browse/SPARK-21436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558013#comment-16558013 ] zhengruifeng commented on SPARK-21436: -- [~holdenk] It looks like that \{distinct} already utilized

[jira] [Commented] (SPARK-23683) FileCommitProtocol.instantiate to require 3-arg constructor for dynamic partition overwrite

[ https://issues.apache.org/jira/browse/SPARK-23683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558009#comment-16558009 ] Felix Cheung commented on SPARK-23683: -- should this be ported back to branch 2.3 ? we ran into this

[jira] [Resolved] (SPARK-24878) Fix reverse function for array type of primitive type containing null.

[ https://issues.apache.org/jira/browse/SPARK-24878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24878. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21830

[jira] [Assigned] (SPARK-24927) The hadoop-provided profile doesn't play well with Snappy-compressed Parquet files

[ https://issues.apache.org/jira/browse/SPARK-24927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24927: Assignee: Apache Spark > The hadoop-provided profile doesn't play well with

[jira] [Commented] (SPARK-24927) The hadoop-provided profile doesn't play well with Snappy-compressed Parquet files

[ https://issues.apache.org/jira/browse/SPARK-24927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16557985#comment-16557985 ] Apache Spark commented on SPARK-24927: -- User 'liancheng' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24927) The hadoop-provided profile doesn't play well with Snappy-compressed Parquet files

[ https://issues.apache.org/jira/browse/SPARK-24927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24927: Assignee: (was: Apache Spark) > The hadoop-provided profile doesn't play well with

[jira] [Assigned] (SPARK-24878) Fix reverse function for array type of primitive type containing null.

[ https://issues.apache.org/jira/browse/SPARK-24878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-24878: --- Assignee: Takuya Ueshin > Fix reverse function for array type of primitive type containing

[jira] [Commented] (SPARK-24927) The hadoop-provided profile doesn't play well with Snappy-compressed Parquet files

[ https://issues.apache.org/jira/browse/SPARK-24927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16557603#comment-16557603 ] Cheng Lian commented on SPARK-24927: Downgraded from blocker to major, since it's not a regression.

[jira] [Updated] (SPARK-24927) The hadoop-provided profile doesn't play well with Snappy-compressed Parquet files

[ https://issues.apache.org/jira/browse/SPARK-24927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-24927: --- Priority: Major (was: Blocker) > The hadoop-provided profile doesn't play well with

[jira] [Commented] (SPARK-24927) The hadoop-provided profile doesn't play well with Snappy-compressed Parquet files

[ https://issues.apache.org/jira/browse/SPARK-24927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16556975#comment-16556975 ] Xiao Li commented on SPARK-24927: - cc [~jerryshao] > The hadoop-provided profile doesn't play well

[jira] [Created] (SPARK-24927) The hadoop-provided profile doesn't play well with Snappy-compressed Parquet files

Cheng Lian created SPARK-24927: -- Summary: The hadoop-provided profile doesn't play well with Snappy-compressed Parquet files Key: SPARK-24927 URL: https://issues.apache.org/jira/browse/SPARK-24927

[jira] [Commented] (SPARK-24926) Ensure numCores is used consistently in all netty configuration (driver and executors)

[ https://issues.apache.org/jira/browse/SPARK-24926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16556965#comment-16556965 ] Imran Rashid commented on SPARK-24926: -- I was talking to [~nsheth] about this, he's going to work

[jira] [Commented] (SPARK-24918) Executor Plugin API

[ https://issues.apache.org/jira/browse/SPARK-24918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16556964#comment-16556964 ] Imran Rashid commented on SPARK-24918: -- I have some changes with an initial draft of this, at

  1   2   >