[jira] [Updated] (SPARK-25193) insert overwrite doesn't throw exception when drop old data fails

[ https://issues.apache.org/jira/browse/SPARK-25193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chen xiao updated SPARK-25193: -- Affects Version/s: (was: 2.2.2) 2.2.0 > insert overwrite doesn't throw exce

[jira] [Updated] (SPARK-25193) insert overwrite doesn't throw exception when drop old data fails

[ https://issues.apache.org/jira/browse/SPARK-25193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chen xiao updated SPARK-25193: -- Description: dataframe.write.mode(SaveMode.Overwrite).insertInto(s"$databaseName.$tableName") Insert

[jira] [Created] (SPARK-25193) insert overwrite doesn't throw exception when drop old data fails

chen xiao created SPARK-25193: - Summary: insert overwrite doesn't throw exception when drop old data fails Key: SPARK-25193 URL: https://issues.apache.org/jira/browse/SPARK-25193 Project: Spark

[jira] [Resolved] (SPARK-25168) PlanTest.comparePlans may make a supplied resolved plan unresolved.

[ https://issues.apache.org/jira/browse/SPARK-25168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Biswal resolved SPARK-25168. -- Resolution: Won't Fix > PlanTest.comparePlans may make a supplied resolved plan unresolved. >

[jira] [Assigned] (SPARK-25093) CodeFormatter could avoid creating regex object again and again

[ https://issues.apache.org/jira/browse/SPARK-25093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25093: --- Assignee: Marco Gaido > CodeFormatter could avoid creating regex object again and again > -

[jira] [Resolved] (SPARK-25093) CodeFormatter could avoid creating regex object again and again

[ https://issues.apache.org/jira/browse/SPARK-25093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25093. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22135 [https://gith

[jira] [Created] (SPARK-25192) Remove SupportsPushdownCatalystFilter

Wenchen Fan created SPARK-25192: --- Summary: Remove SupportsPushdownCatalystFilter Key: SPARK-25192 URL: https://issues.apache.org/jira/browse/SPARK-25192 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-25191) continuous streaming should only create one ReadSupport instance per-query

Wenchen Fan created SPARK-25191: --- Summary: continuous streaming should only create one ReadSupport instance per-query Key: SPARK-25191 URL: https://issues.apache.org/jira/browse/SPARK-25191 Project: Spa

[jira] [Created] (SPARK-25190) Better operator pushdown API

Wenchen Fan created SPARK-25190: --- Summary: Better operator pushdown API Key: SPARK-25190 URL: https://issues.apache.org/jira/browse/SPARK-25190 Project: Spark Issue Type: Sub-task Com

[jira] [Created] (SPARK-25189) CustomMetrics should be created with ScanConfig

Wenchen Fan created SPARK-25189: --- Summary: CustomMetrics should be created with ScanConfig Key: SPARK-25189 URL: https://issues.apache.org/jira/browse/SPARK-25189 Project: Spark Issue Type: Sub

[jira] [Created] (SPARK-25188) Add WriteConfig

Wenchen Fan created SPARK-25188: --- Summary: Add WriteConfig Key: SPARK-25188 URL: https://issues.apache.org/jira/browse/SPARK-25188 Project: Spark Issue Type: Sub-task Components: SQL

[jira] [Created] (SPARK-25187) Revisit the life cycle of ReadSupport instances.

Wenchen Fan created SPARK-25187: --- Summary: Revisit the life cycle of ReadSupport instances. Key: SPARK-25187 URL: https://issues.apache.org/jira/browse/SPARK-25187 Project: Spark Issue Type: Su

[jira] [Created] (SPARK-25186) Stabilize Data Source V2 API

Wenchen Fan created SPARK-25186: --- Summary: Stabilize Data Source V2 API Key: SPARK-25186 URL: https://issues.apache.org/jira/browse/SPARK-25186 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-25132) Case-insensitive field resolution when reading from Parquet/ORC

[ https://issues.apache.org/jira/browse/SPARK-25132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588414#comment-16588414 ] Apache Spark commented on SPARK-25132: -- User 'seancxmao' has created a pull request

[jira] [Resolved] (SPARK-25155) Streaming from storage doesn't work when no directories exists

[ https://issues.apache.org/jira/browse/SPARK-25155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gil Vernik resolved SPARK-25155. Resolution: Cannot Reproduce > Streaming from storage doesn't work when no directories exists > --

[jira] [Commented] (SPARK-25155) Streaming from storage doesn't work when no directories exists

[ https://issues.apache.org/jira/browse/SPARK-25155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588412#comment-16588412 ] Gil Vernik commented on SPARK-25155: [~ste...@apache.org] thanks for the input. This

[jira] [Updated] (SPARK-25159) json schema inference should only trigger one job

[ https://issues.apache.org/jira/browse/SPARK-25159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25159: Fix Version/s: 2.4.0 > json schema inference should only trigger one job > ---

[jira] [Resolved] (SPARK-25159) json schema inference should only trigger one job

[ https://issues.apache.org/jira/browse/SPARK-25159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25159. - Resolution: Fixed Target Version/s: 2.4.0 > json schema inference should only trigger one job

[jira] [Resolved] (SPARK-25140) Add optional logging to UnsafeProjection.create when it falls back to interpreted mode

[ https://issues.apache.org/jira/browse/SPARK-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25140. - Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.4.0 > Add optional logging

[jira] [Created] (SPARK-25185) CBO rowcount statistics doesn't work for partitioned parquet external table

Amit created SPARK-25185: Summary: CBO rowcount statistics doesn't work for partitioned parquet external table Key: SPARK-25185 URL: https://issues.apache.org/jira/browse/SPARK-25185 Project: Spark

[jira] [Commented] (SPARK-24763) Remove redundant key data from value in streaming aggregation

[ https://issues.apache.org/jira/browse/SPARK-24763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588389#comment-16588389 ] Jungtaek Lim commented on SPARK-24763: -- [~tdas] Got it. Thanks for the input. > Re

[jira] [Assigned] (SPARK-25184) Flaky test: FlatMapGroupsWithState "streaming with processing time timeout"

[ https://issues.apache.org/jira/browse/SPARK-25184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25184: Assignee: (was: Apache Spark) > Flaky test: FlatMapGroupsWithState "streaming with pr

[jira] [Assigned] (SPARK-25184) Flaky test: FlatMapGroupsWithState "streaming with processing time timeout"

[ https://issues.apache.org/jira/browse/SPARK-25184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25184: Assignee: Apache Spark > Flaky test: FlatMapGroupsWithState "streaming with processing ti

[jira] [Commented] (SPARK-25184) Flaky test: FlatMapGroupsWithState "streaming with processing time timeout"

[ https://issues.apache.org/jira/browse/SPARK-25184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588385#comment-16588385 ] Apache Spark commented on SPARK-25184: -- User 'tdas' has created a pull request for

[jira] [Created] (SPARK-25184) Flaky test: FlatMapGroupsWithState "streaming with processing time timeout"

Tathagata Das created SPARK-25184: - Summary: Flaky test: FlatMapGroupsWithState "streaming with processing time timeout" Key: SPARK-25184 URL: https://issues.apache.org/jira/browse/SPARK-25184 Project

[jira] [Assigned] (SPARK-25163) Flaky test: o.a.s.util.collection.ExternalAppendOnlyMapSuite.spilling with compression

[ https://issues.apache.org/jira/browse/SPARK-25163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25163: Assignee: Apache Spark > Flaky test: o.a.s.util.collection.ExternalAppendOnlyMapSuite.spi

[jira] [Assigned] (SPARK-25163) Flaky test: o.a.s.util.collection.ExternalAppendOnlyMapSuite.spilling with compression

[ https://issues.apache.org/jira/browse/SPARK-25163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25163: Assignee: (was: Apache Spark) > Flaky test: o.a.s.util.collection.ExternalAppendOnlyM

[jira] [Commented] (SPARK-25163) Flaky test: o.a.s.util.collection.ExternalAppendOnlyMapSuite.spilling with compression

[ https://issues.apache.org/jira/browse/SPARK-25163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588296#comment-16588296 ] Apache Spark commented on SPARK-25163: -- User 'viirya' has created a pull request fo

[jira] [Assigned] (SPARK-25174) ApplicationMaster suspends when unregistering itself from RM with extreme large diagnostic message

[ https://issues.apache.org/jira/browse/SPARK-25174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25174: Assignee: Apache Spark > ApplicationMaster suspends when unregistering itself from RM wit

[jira] [Commented] (SPARK-25174) ApplicationMaster suspends when unregistering itself from RM with extreme large diagnostic message

[ https://issues.apache.org/jira/browse/SPARK-25174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588283#comment-16588283 ] Apache Spark commented on SPARK-25174: -- User 'yaooqinn' has created a pull request

[jira] [Assigned] (SPARK-25174) ApplicationMaster suspends when unregistering itself from RM with extreme large diagnostic message

[ https://issues.apache.org/jira/browse/SPARK-25174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25174: Assignee: (was: Apache Spark) > ApplicationMaster suspends when unregistering itself

[jira] [Commented] (SPARK-24763) Remove redundant key data from value in streaming aggregation

[ https://issues.apache.org/jira/browse/SPARK-24763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588234#comment-16588234 ] Tathagata Das commented on SPARK-24763: --- They will be. The merge script always put

[jira] [Commented] (SPARK-23131) Stackoverflow using ML and Kryo serializer

[ https://issues.apache.org/jira/browse/SPARK-23131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588232#comment-16588232 ] Apache Spark commented on SPARK-23131: -- User 'wangyum' has created a pull request f

[jira] [Assigned] (SPARK-25127) DataSourceV2: Remove SupportsPushDownCatalystFilters

[ https://issues.apache.org/jira/browse/SPARK-25127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-25127: --- Assignee: Reynold Xin > DataSourceV2: Remove SupportsPushDownCatalystFilters >

[jira] [Created] (SPARK-25183) Spark HiveServer2 registers shutdown hook with JVM, not ShutdownHookManager; race conditions can arise

Steve Loughran created SPARK-25183: -- Summary: Spark HiveServer2 registers shutdown hook with JVM, not ShutdownHookManager; race conditions can arise Key: SPARK-25183 URL: https://issues.apache.org/jira/browse/SPA

[jira] [Commented] (SPARK-25164) Parquet reader builds entire list of columns once for each column

[ https://issues.apache.org/jira/browse/SPARK-25164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588168#comment-16588168 ] Bruce Robbins commented on SPARK-25164: --- [~viirya] Sure. I will try to get somethi

[jira] [Commented] (SPARK-25119) stages in wrong order within job page DAG chart

[ https://issues.apache.org/jira/browse/SPARK-25119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588160#comment-16588160 ] Yunjian Zhang commented on SPARK-25119: --- create PR as below [https://github.com/a

[jira] [Comment Edited] (SPARK-25164) Parquet reader builds entire list of columns once for each column

[ https://issues.apache.org/jira/browse/SPARK-25164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588159#comment-16588159 ] Liang-Chi Hsieh edited comment on SPARK-25164 at 8/21/18 11:30 PM: ---

[jira] [Commented] (SPARK-25164) Parquet reader builds entire list of columns once for each column

[ https://issues.apache.org/jira/browse/SPARK-25164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588159#comment-16588159 ] Liang-Chi Hsieh commented on SPARK-25164: - This is easy and looks good to have.

[jira] [Assigned] (SPARK-25181) Block Manager master and slave thread pools are unbounded

[ https://issues.apache.org/jira/browse/SPARK-25181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25181: Assignee: (was: Apache Spark) > Block Manager master and slave thread pools are unbou

[jira] [Assigned] (SPARK-25181) Block Manager master and slave thread pools are unbounded

[ https://issues.apache.org/jira/browse/SPARK-25181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25181: Assignee: Apache Spark > Block Manager master and slave thread pools are unbounded >

[jira] [Commented] (SPARK-25181) Block Manager master and slave thread pools are unbounded

[ https://issues.apache.org/jira/browse/SPARK-25181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588154#comment-16588154 ] Apache Spark commented on SPARK-25181: -- User 'mukulmurthy' has created a pull reque

[jira] [Resolved] (SPARK-25182) Block Manager master and slave thread pools are unbounded

[ https://issues.apache.org/jira/browse/SPARK-25182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Murthy resolved SPARK-25182. -- Resolution: Duplicate Target Version/s: (was: 2.4.0) > Block Manager master and sl

[jira] [Created] (SPARK-25182) Block Manager master and slave thread pools are unbounded

Mukul Murthy created SPARK-25182: Summary: Block Manager master and slave thread pools are unbounded Key: SPARK-25182 URL: https://issues.apache.org/jira/browse/SPARK-25182 Project: Spark Iss

[jira] [Created] (SPARK-25181) Block Manager master and slave thread pools are unbounded

Mukul Murthy created SPARK-25181: Summary: Block Manager master and slave thread pools are unbounded Key: SPARK-25181 URL: https://issues.apache.org/jira/browse/SPARK-25181 Project: Spark Iss

[jira] [Commented] (SPARK-25114) RecordBinaryComparator may return wrong result when subtraction between two words is divisible by Integer.MAX_VALUE

[ https://issues.apache.org/jira/browse/SPARK-25114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588121#comment-16588121 ] Xiao Li commented on SPARK-25114: - Let us update the fix version after the fix of 2.2 is

[jira] [Resolved] (SPARK-25114) RecordBinaryComparator may return wrong result when subtraction between two words is divisible by Integer.MAX_VALUE

[ https://issues.apache.org/jira/browse/SPARK-25114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25114. - Resolution: Fixed Assignee: Jiang Xingbo Fix Version/s: 2.4.0 2.3.2 >

[jira] [Resolved] (SPARK-25095) Python support for BarrierTaskContext

[ https://issues.apache.org/jira/browse/SPARK-25095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25095. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22085 [https://

[jira] [Assigned] (SPARK-25095) Python support for BarrierTaskContext

[ https://issues.apache.org/jira/browse/SPARK-25095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-25095: - Assignee: Jiang Xingbo > Python support for BarrierTaskContext > --

[jira] [Commented] (SPARK-24564) Add test suite for RecordBinaryComparator

[ https://issues.apache.org/jira/browse/SPARK-24564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588114#comment-16588114 ] Apache Spark commented on SPARK-24564: -- User 'bersprockets' has created a pull requ

[jira] [Commented] (SPARK-25114) RecordBinaryComparator may return wrong result when subtraction between two words is divisible by Integer.MAX_VALUE

[ https://issues.apache.org/jira/browse/SPARK-25114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588115#comment-16588115 ] Apache Spark commented on SPARK-25114: -- User 'bersprockets' has created a pull requ

[jira] [Comment Edited] (SPARK-25168) PlanTest.comparePlans may make a supplied resolved plan unresolved.

[ https://issues.apache.org/jira/browse/SPARK-25168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587956#comment-16587956 ] Dilip Biswal edited comment on SPARK-25168 at 8/21/18 10:47 PM: --

[jira] [Commented] (SPARK-24763) Remove redundant key data from value in streaming aggregation

[ https://issues.apache.org/jira/browse/SPARK-24763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588101#comment-16588101 ] Jungtaek Lim commented on SPARK-24763: -- [~tdas] One question regarding fix version

[jira] [Resolved] (SPARK-24441) Expose total estimated size of states in HDFSBackedStateStoreProvider

[ https://issues.apache.org/jira/browse/SPARK-24441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-24441. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 21469 [https://

[jira] [Assigned] (SPARK-24441) Expose total estimated size of states in HDFSBackedStateStoreProvider

[ https://issues.apache.org/jira/browse/SPARK-24441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-24441: - Assignee: Jungtaek Lim > Expose total estimated size of states in HDFSBackedStateStoreP

[jira] [Assigned] (SPARK-24763) Remove redundant key data from value in streaming aggregation

[ https://issues.apache.org/jira/browse/SPARK-24763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-24763: - Assignee: Jungtaek Lim (was: Tathagata Das) > Remove redundant key data from value in

[jira] [Assigned] (SPARK-24763) Remove redundant key data from value in streaming aggregation

[ https://issues.apache.org/jira/browse/SPARK-24763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-24763: - Assignee: Tathagata Das > Remove redundant key data from value in streaming aggregation

[jira] [Resolved] (SPARK-24763) Remove redundant key data from value in streaming aggregation

[ https://issues.apache.org/jira/browse/SPARK-24763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-24763. --- Resolution: Done Fix Version/s: 3.0.0 2.4.0 > Remove redundant key

[jira] [Resolved] (SPARK-25149) Personalized PageRank raises an error if vertexIDs are > MaxInt

[ https://issues.apache.org/jira/browse/SPARK-25149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-25149. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22139 [

[jira] [Assigned] (SPARK-25149) Personalized PageRank raises an error if vertexIDs are > MaxInt

[ https://issues.apache.org/jira/browse/SPARK-25149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-25149: - Assignee: Bago Amirbekian > Personalized PageRank raises an error if vertexIDs

[jira] [Updated] (SPARK-25149) Personalized PageRank raises an error if vertexIDs are > MaxInt

[ https://issues.apache.org/jira/browse/SPARK-25149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-25149: -- Summary: Personalized PageRank raises an error if vertexIDs are > MaxInt (was: Person

[jira] [Updated] (SPARK-24307) Support sending messages over 2GB from memory

[ https://issues.apache.org/jira/browse/SPARK-24307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24307: Labels: release-notes (was: releasenotes) > Support sending messages over 2GB from memory > -

[jira] [Updated] (SPARK-24307) Support sending messages over 2GB from memory

[ https://issues.apache.org/jira/browse/SPARK-24307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24307: Labels: releasenotes (was: ) > Support sending messages over 2GB from memory > --

[jira] [Commented] (SPARK-25050) Handle more than two types in avro union types when writing avro files

[ https://issues.apache.org/jira/browse/SPARK-25050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588068#comment-16588068 ] DB Tsai commented on SPARK-25050: - We handle more than two types in avro when reading in

[jira] [Updated] (SPARK-25050) Handle more than two types in avro union types when writing avro files

[ https://issues.apache.org/jira/browse/SPARK-25050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-25050: Summary: Handle more than two types in avro union types when writing avro files (was: Handle more than tw

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588053#comment-16588053 ] Steve Loughran commented on SPARK-6305: --- 1. exclusion of log4j 1.x you can only sa

[jira] [Commented] (SPARK-25180) Spark standalone failure in Utils.doFetchFile() if nslookup of local hostname fails

[ https://issues.apache.org/jira/browse/SPARK-25180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588035#comment-16588035 ] Steve Loughran commented on SPARK-25180: FWIW, there was no in-progress data at

[jira] [Commented] (SPARK-25162) Kubernetes 'in-cluster' client mode and value of spark.driver.host

[ https://issues.apache.org/jira/browse/SPARK-25162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588014#comment-16588014 ] Yinan Li commented on SPARK-25162: -- We actually moved away from using the IP address of

[jira] [Commented] (SPARK-25180) Spark standalone failure in Utils.doFetchFile() if nslookup of local hostname fails

[ https://issues.apache.org/jira/browse/SPARK-25180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588012#comment-16588012 ] Steve Loughran commented on SPARK-25180: Netty converts the UnknownHostException

[jira] [Updated] (SPARK-25149) Personalized Page Rank raises an error if vertexIDs are > MaxInt

[ https://issues.apache.org/jira/browse/SPARK-25149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-25149: -- Shepherd: Joseph K. Bradley > Personalized Page Rank raises an error if vertexIDs are

[jira] [Commented] (SPARK-25180) Spark standalone failure in Utils.doFetchFile() if nslookup of local hostname fails

[ https://issues.apache.org/jira/browse/SPARK-25180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588005#comment-16588005 ] Steve Loughran commented on SPARK-25180: Stack {code} scala> text("hello all!")

[jira] [Commented] (SPARK-25180) Spark standalone failure in Utils.doFetchFile() if nslookup of local hostname fails

[ https://issues.apache.org/jira/browse/SPARK-25180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588001#comment-16588001 ] Steve Loughran commented on SPARK-25180: code snippet was some trivial CSV => OR

[jira] [Created] (SPARK-25180) Spark standalone failure in Utils.doFetchFile() if nslookup of local hostname fails

Steve Loughran created SPARK-25180: -- Summary: Spark standalone failure in Utils.doFetchFile() if nslookup of local hostname fails Key: SPARK-25180 URL: https://issues.apache.org/jira/browse/SPARK-25180

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587962#comment-16587962 ] Sean Owen commented on SPARK-6305: -- It was a mess. Excluding dependencies is an ongoing

[jira] [Commented] (SPARK-25168) PlanTest.comparePlans may make a supplied resolved plan unresolved.

[ https://issues.apache.org/jira/browse/SPARK-25168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587956#comment-16587956 ] Dilip Biswal commented on SPARK-25168: -- [~cloud_fan] OK.. Let me close this then si

[jira] [Commented] (SPARK-7768) Make user-defined type (UDT) API public

[ https://issues.apache.org/jira/browse/SPARK-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587928#comment-16587928 ] Alexander commented on SPARK-7768: -- It's been a while since this had any activity. What

[jira] [Commented] (SPARK-25178) Use dummy name for xxxHashMapGenerator key/value schema field

[ https://issues.apache.org/jira/browse/SPARK-25178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587897#comment-16587897 ] Kazuaki Ishizaki commented on SPARK-25178: -- [~rednaxelafx] Thank you for openin

[jira] [Assigned] (SPARK-25179) Document the features that require Pyarrow 0.10

[ https://issues.apache.org/jira/browse/SPARK-25179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-25179: --- Assignee: Bryan Cutler > Document the features that require Pyarrow 0.10 >

[jira] [Updated] (SPARK-25179) Document the features that require Pyarrow 0.10

[ https://issues.apache.org/jira/browse/SPARK-25179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25179: Description: binary type support requires pyarrow 0.10.0. (was: binary type support requires pyarrow 0.1

[jira] [Commented] (SPARK-25179) Document the features that require Pyarrow 0.10

[ https://issues.apache.org/jira/browse/SPARK-25179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587884#comment-16587884 ] Xiao Li commented on SPARK-25179: - Thanks! It is not urgent as long as it is documented

[jira] [Commented] (SPARK-25179) Document the features that require Pyarrow 0.10

[ https://issues.apache.org/jira/browse/SPARK-25179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587882#comment-16587882 ] Bryan Cutler commented on SPARK-25179: -- I can work on this, probably can't get to i

[jira] [Updated] (SPARK-25179) Document the features that require Pyarrow 0.10

[ https://issues.apache.org/jira/browse/SPARK-25179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-25179: - Description: binary type support requires pyarrow 0.10.0 > Document the features that require Py

[jira] [Commented] (SPARK-22779) ConfigEntry's default value should actually be a value

[ https://issues.apache.org/jira/browse/SPARK-22779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587869#comment-16587869 ] Apache Spark commented on SPARK-22779: -- User 'GregOwen' has created a pull request

[jira] [Resolved] (SPARK-6236) Support caching blocks larger than 2G

[ https://issues.apache.org/jira/browse/SPARK-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-6236. - Resolution: Fixed Fix Version/s: 2.4.0 As mentioned above, this was covered elsewhere, par

[jira] [Commented] (SPARK-24961) sort operation causes out of memory

[ https://issues.apache.org/jira/browse/SPARK-24961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587850#comment-16587850 ] Markus Breuer commented on SPARK-24961: --- Some notes to your last comment: Reducin

[jira] [Commented] (SPARK-25179) Document the features that require Pyarrow 0.10

[ https://issues.apache.org/jira/browse/SPARK-25179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587847#comment-16587847 ] Xiao Li commented on SPARK-25179: - cc [~bryanc] > Document the features that require Py

[jira] [Updated] (SPARK-25179) Document the features that require Pyarrow 0.10

[ https://issues.apache.org/jira/browse/SPARK-25179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25179: Issue Type: Documentation (was: Improvement) > Document the features that require Pyarrow 0.10 >

[jira] [Created] (SPARK-25179) Document the features that require Pyarrow 0.10

Xiao Li created SPARK-25179: --- Summary: Document the features that require Pyarrow 0.10 Key: SPARK-25179 URL: https://issues.apache.org/jira/browse/SPARK-25179 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-25178) Use dummy name for xxxHashMapGenerator key/value schema field

[ https://issues.apache.org/jira/browse/SPARK-25178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587841#comment-16587841 ] Xiao Li commented on SPARK-25178: - cc [~kiszk] Do you have a bandwidth to take this? >

[jira] [Resolved] (SPARK-24296) Support replicating blocks larger than 2 GB

[ https://issues.apache.org/jira/browse/SPARK-24296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24296. Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21451 [https:

[jira] [Assigned] (SPARK-24296) Support replicating blocks larger than 2 GB

[ https://issues.apache.org/jira/browse/SPARK-24296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24296: -- Assignee: Imran Rashid > Support replicating blocks larger than 2 GB > --

[jira] [Commented] (SPARK-25178) Use dummy name for xxxHashMapGenerator key/value schema field

[ https://issues.apache.org/jira/browse/SPARK-25178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587820#comment-16587820 ] Kris Mok commented on SPARK-25178: -- FYI: I've just come across this behavior and raised

[jira] [Created] (SPARK-25178) Use dummy name for xxxHashMapGenerator key/value schema field

Kris Mok created SPARK-25178: Summary: Use dummy name for xxxHashMapGenerator key/value schema field Key: SPARK-25178 URL: https://issues.apache.org/jira/browse/SPARK-25178 Project: Spark Issue

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587814#comment-16587814 ] Chris Martin commented on SPARK-6305: - [~srowen] I've taken a look at this and I'd l

[jira] [Commented] (SPARK-24335) Dataset.map schema not applied in some cases

[ https://issues.apache.org/jira/browse/SPARK-24335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587785#comment-16587785 ] Apache Spark commented on SPARK-24335: -- User 'redsanket' has created a pull request

[jira] [Commented] (SPARK-23874) Upgrade apache/arrow to 0.10.0

[ https://issues.apache.org/jira/browse/SPARK-23874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587770#comment-16587770 ] Bryan Cutler commented on SPARK-23874: -- Yes, I still would recommend users upgrade

[jira] [Resolved] (SPARK-25173) fail to pass kerberos authentification on executers

[ https://issues.apache.org/jira/browse/SPARK-25173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25173. Resolution: Duplicate > fail to pass kerberos authentification on executers >

[jira] [Resolved] (SPARK-25172) kererbos issue when use jdbc to connect hive server2 on executors

[ https://issues.apache.org/jira/browse/SPARK-25172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25172. Resolution: Invalid Please use the mailing list for questions: http://spark.apache.org/com

[jira] [Commented] (SPARK-23874) Upgrade apache/arrow to 0.10.0

[ https://issues.apache.org/jira/browse/SPARK-23874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587702#comment-16587702 ] Xiao Li commented on SPARK-23874: - Great! Then, users can keep using pyarrow 0.8 and get

[jira] [Assigned] (SPARK-25161) Fix several bugs in failure handling of barrier execution mode

[ https://issues.apache.org/jira/browse/SPARK-25161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-25161: - Assignee: Jiang Xingbo > Fix several bugs in failure handling of barrier execution mode

  1   2   >