[jira] [Updated] (SPARK-3873) Scala style: check import ordering

[ https://issues.apache.org/jira/browse/SPARK-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3873: --- Assignee: (was: Marcelo Vanzin) > Scala style: check import ordering > ---

[jira] [Commented] (SPARK-3963) Support getting task-scoped properties from TaskContext

[ https://issues.apache.org/jira/browse/SPARK-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173451#comment-14173451 ] Reynold Xin commented on SPARK-3963: Would it make sense to support arbitrary data typ

[jira] [Commented] (SPARK-3965) Spark assembly for hadoop2 contains avro-mapred for hadoop1

[ https://issues.apache.org/jira/browse/SPARK-3965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173445#comment-14173445 ] Apache Spark commented on SPARK-3965: - User 'dajac' has created a pull request for thi

[jira] [Issue Comment Deleted] (SPARK-3882) JobProgressListener gets permanently out of sync with long running job

[ https://issues.apache.org/jira/browse/SPARK-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Shepherd updated SPARK-3882: -- Comment: was deleted (was: This is also a serious memory leak that will cause long running drive

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173434#comment-14173434 ] Saisai Shao commented on SPARK-3948: Besides I think we can add configuration as [~jos

[jira] [Commented] (SPARK-3807) SparkSql does not work for tables created using custom serde

[ https://issues.apache.org/jira/browse/SPARK-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173430#comment-14173430 ] Apache Spark commented on SPARK-3807: - User 'adrian-wang' has created a pull request f

[jira] [Commented] (SPARK-3882) JobProgressListener gets permanently out of sync with long running job

[ https://issues.apache.org/jira/browse/SPARK-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173428#comment-14173428 ] Davis Shepherd commented on SPARK-3882: --- This is also a serious memory leak that wil

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173427#comment-14173427 ] Saisai Shao commented on SPARK-3948: Hi [~mridulm80], thanks a lot for your suggestion

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173413#comment-14173413 ] Mridul Muralidharan commented on SPARK-3948: Not exactly, what I was suggestin

[jira] [Commented] (SPARK-3885) Provide mechanism to remove accumulators once they are no longer used

[ https://issues.apache.org/jira/browse/SPARK-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173407#comment-14173407 ] Nathan Kronenfeld commented on SPARK-3885: -- I tried reusing accumulators and clea

[jira] [Updated] (SPARK-3963) Support getting task-scoped properties from TaskContext

[ https://issues.apache.org/jira/browse/SPARK-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3963: --- Description: This is a proposal for a minor feature. Given stabilization of the TaskContext A

[jira] [Commented] (SPARK-3963) Support getting task-scoped properties from TaskContext

[ https://issues.apache.org/jira/browse/SPARK-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173404#comment-14173404 ] Patrick Wendell commented on SPARK-3963: [~rxin] and [~adav] I'd be interested in

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173403#comment-14173403 ] Praveen Seluka commented on SPARK-3174: --- Thanks [~vanzin] for taking a look. Regardi

[jira] [Commented] (SPARK-3966) Fix nullabilities of Cast related to DateType.

[ https://issues.apache.org/jira/browse/SPARK-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173400#comment-14173400 ] Apache Spark commented on SPARK-3966: - User 'ueshin' has created a pull request for th

[jira] [Created] (SPARK-3966) Fix nullabilities of Cast related to DateType.

Takuya Ueshin created SPARK-3966: Summary: Fix nullabilities of Cast related to DateType. Key: SPARK-3966 URL: https://issues.apache.org/jira/browse/SPARK-3966 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-3965) Spark assembly for hadoop2 contains avro-mapred for hadoop1

David Jacot created SPARK-3965: -- Summary: Spark assembly for hadoop2 contains avro-mapred for hadoop1 Key: SPARK-3965 URL: https://issues.apache.org/jira/browse/SPARK-3965 Project: Spark Issue

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173353#comment-14173353 ] Saisai Shao commented on SPARK-3948: Thanks for your suggestions. I think it would be

[jira] [Created] (SPARK-3964) Python API for Hypothesis testing

Davies Liu created SPARK-3964: - Summary: Python API for Hypothesis testing Key: SPARK-3964 URL: https://issues.apache.org/jira/browse/SPARK-3964 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-3963) Support getting task-scoped properties from TaskContext

[ https://issues.apache.org/jira/browse/SPARK-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3963: --- Description: This is a proposal for a minor feature. Given stabilization of the TaskContext A

[jira] [Updated] (SPARK-3963) Support getting task-scoped properties from TaskContext

[ https://issues.apache.org/jira/browse/SPARK-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3963: --- Description: This is a proposal for a minor feature. Given stabilization of the TaskContext A

[jira] [Created] (SPARK-3963) Support getting task-scoped properties from TaskContext

Patrick Wendell created SPARK-3963: -- Summary: Support getting task-scoped properties from TaskContext Key: SPARK-3963 URL: https://issues.apache.org/jira/browse/SPARK-3963 Project: Spark Iss

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173300#comment-14173300 ] Patrick Wendell commented on SPARK-3431: [~srowen] - just wondering, is it trivial

[jira] [Commented] (SPARK-3951) Make the external-* jars fat jars

[ https://issues.apache.org/jira/browse/SPARK-3951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173295#comment-14173295 ] Hari Shreedharan commented on SPARK-3951: - That makes sense. But we should fix the

[jira] [Resolved] (SPARK-1561) sbt/sbt assembly generates too many local files

[ https://issues.apache.org/jira/browse/SPARK-1561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1561. Resolution: Not a Problem Does not seem to be an issue any more - resolving per comment from

[jira] [Updated] (SPARK-3962) Mark spark dependency as "provided" in external libraries

[ https://issues.apache.org/jira/browse/SPARK-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3962: --- Description: Right now there is not an easy way for users to link against the external stream

[jira] [Created] (SPARK-3962) Mark spark dependency as "provided" in external libraries

Patrick Wendell created SPARK-3962: -- Summary: Mark spark dependency as "provided" in external libraries Key: SPARK-3962 URL: https://issues.apache.org/jira/browse/SPARK-3962 Project: Spark I

[jira] [Commented] (SPARK-3951) Make the external-* jars fat jars

[ https://issues.apache.org/jira/browse/SPARK-3951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173289#comment-14173289 ] Patrick Wendell commented on SPARK-3951: I think the best solution here is actuall

[jira] [Updated] (SPARK-3954) Optimization to FileInputDStream

[ https://issues.apache.org/jira/browse/SPARK-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3954: --- Summary: Optimization to FileInputDStream (was: promote the speed of convert files to RDDS)

[jira] [Updated] (SPARK-3954) promote the speed of convert files to RDDS

[ https://issues.apache.org/jira/browse/SPARK-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3954: --- Component/s: (was: Input/Output) Streaming > promote the speed of conver

[jira] [Commented] (SPARK-1209) SparkHadoop{MapRed,MapReduce}Util should not use package org.apache.hadoop

[ https://issues.apache.org/jira/browse/SPARK-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173284#comment-14173284 ] Patrick Wendell commented on SPARK-1209: Got it! Yeah so anything that is internal

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173269#comment-14173269 ] Zhan Zhang commented on SPARK-2883: --- It is scala patch, and I am not very familiar with

[jira] [Updated] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-2883: -- Attachment: orc.diff Just for completion, I also attach the diff here besides spark-3720. > Spark Supp

[jira] [Updated] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-3720: -- Attachment: orc.diff This is the diff for orc file support. Because I also work on this item in paralle

[jira] [Commented] (SPARK-3958) Possible stream-corruption issues in TorrentBroadcast

[ https://issues.apache.org/jira/browse/SPARK-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173207#comment-14173207 ] Josh Rosen commented on SPARK-3958: --- Hi [~jerryshao], I don't have a reliable reproduct

[jira] [Commented] (SPARK-3958) Possible stream-corruption issues in TorrentBroadcast

[ https://issues.apache.org/jira/browse/SPARK-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173199#comment-14173199 ] Saisai Shao commented on SPARK-3958: Hi Josh, have you tried other compression like LZ

[jira] [Commented] (SPARK-3961) Python API for mllib.feature

[ https://issues.apache.org/jira/browse/SPARK-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173172#comment-14173172 ] Apache Spark commented on SPARK-3961: - User 'davies' has created a pull request for th

[jira] [Created] (SPARK-3961) Python API for mllib.feature

Davies Liu created SPARK-3961: - Summary: Python API for mllib.feature Key: SPARK-3961 URL: https://issues.apache.org/jira/browse/SPARK-3961 Project: Spark Issue Type: New Feature Repo

[jira] [Commented] (SPARK-3955) Different versions between jackson-mapper-asl and jackson-core-asl

[ https://issues.apache.org/jira/browse/SPARK-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173166#comment-14173166 ] Apache Spark commented on SPARK-3955: - User 'jongyoul' has created a pull request for

[jira] [Commented] (SPARK-3958) Possible stream-corruption issues in TorrentBroadcast

[ https://issues.apache.org/jira/browse/SPARK-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173154#comment-14173154 ] Josh Rosen commented on SPARK-3958: --- I think that I can safely rule out problems in Torr

[jira] [Updated] (SPARK-3958) Possible stream-corruption issues in TorrentBroadcast

[ https://issues.apache.org/jira/browse/SPARK-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3958: -- Affects Version/s: (was: 1.1.0) Removing 1.1.0 as an affected version for now, since the stacktrace

[jira] [Updated] (SPARK-3958) Possible stream-corruption issues in TorrentBroadcast

[ https://issues.apache.org/jira/browse/SPARK-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3958: -- Target Version/s: 1.2.0 (was: 1.1.1, 1.2.0) > Possible stream-corruption issues in TorrentBroadcast > -

[jira] [Commented] (SPARK-3937) Unsafe memory access inside of Snappy library

[ https://issues.apache.org/jira/browse/SPARK-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173025#comment-14173025 ] Josh Rosen commented on SPARK-3937: --- Another occurrence of this problem, running a recen

[jira] [Commented] (SPARK-3960) We can apply unary minus only to literal.

[ https://issues.apache.org/jira/browse/SPARK-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172986#comment-14172986 ] Apache Spark commented on SPARK-3960: - User 'sarutak' has created a pull request for t

[jira] [Commented] (SPARK-3959) SqlParser fails to parse literal -9223372036854775808 (Long.MinValue).

[ https://issues.apache.org/jira/browse/SPARK-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172985#comment-14172985 ] Apache Spark commented on SPARK-3959: - User 'sarutak' has created a pull request for t

[jira] [Commented] (SPARK-2681) Spark can hang when fetching shuffle blocks

[ https://issues.apache.org/jira/browse/SPARK-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172959#comment-14172959 ] Malte Buecken commented on SPARK-2681: -- I am still seeing this issue in 1.1.0 14/10/

[jira] [Commented] (SPARK-3958) Possible stream-corruption issues in TorrentBroadcast

[ https://issues.apache.org/jira/browse/SPARK-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172949#comment-14172949 ] Josh Rosen commented on SPARK-3958: --- Digging into this stacktrace in more detail: Snapp

[jira] [Updated] (SPARK-3960) We can apply unary minus only to literal.

[ https://issues.apache.org/jira/browse/SPARK-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3960: -- Summary: We can apply unary minus only to literal. (was: We cannot apply unary minus only to li

[jira] [Updated] (SPARK-3960) We cannot apply unary minus only to literal.

[ https://issues.apache.org/jira/browse/SPARK-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3960: -- Description: Because of the wrong syntax definition, we cannot apply unary minus only to litera

[jira] [Created] (SPARK-3960) We cannot apply unary minus only to literal.

Kousuke Saruta created SPARK-3960: - Summary: We cannot apply unary minus only to literal. Key: SPARK-3960 URL: https://issues.apache.org/jira/browse/SPARK-3960 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-3959) SqlParser fails to parse literal -9223372036854775808 (Long.MinValue).

Kousuke Saruta created SPARK-3959: - Summary: SqlParser fails to parse literal -9223372036854775808 (Long.MinValue). Key: SPARK-3959 URL: https://issues.apache.org/jira/browse/SPARK-3959 Project: Spark

[jira] [Updated] (SPARK-3630) Identify cause of Kryo+Snappy PARSING_ERROR

[ https://issues.apache.org/jira/browse/SPARK-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3630: -- Target Version/s: 1.1.1, 1.2.0 Affects Version/s: 1.2.0 Assignee: Josh Rosen > Identif

[jira] [Commented] (SPARK-3630) Identify cause of Kryo+Snappy PARSING_ERROR

[ https://issues.apache.org/jira/browse/SPARK-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172926#comment-14172926 ] Josh Rosen commented on SPARK-3630: --- I think that there may be multiple causes of these

[jira] [Created] (SPARK-3958) Possible stream-corruption issues in TorrentBroadcast

Josh Rosen created SPARK-3958: - Summary: Possible stream-corruption issues in TorrentBroadcast Key: SPARK-3958 URL: https://issues.apache.org/jira/browse/SPARK-3958 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172917#comment-14172917 ] Fi commented on SPARK-2883: --- This sounds great! Is this a scala only patch, or would it also so

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172895#comment-14172895 ] Josh Rosen commented on SPARK-3948: --- [~mridulm80], that explicit position-checking and m

[jira] [Created] (SPARK-3957) Broadcast variable memory usage not reflected in UI

Shivaram Venkataraman created SPARK-3957: Summary: Broadcast variable memory usage not reflected in UI Key: SPARK-3957 URL: https://issues.apache.org/jira/browse/SPARK-3957 Project: Spark

[jira] [Commented] (SPARK-3936) Incorrect result in GraphX BytecodeUtils with closures + class/object methods

[ https://issues.apache.org/jira/browse/SPARK-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172867#comment-14172867 ] Apache Spark commented on SPARK-3936: - User 'jegonzal' has created a pull request for

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172861#comment-14172861 ] Marcelo Vanzin commented on SPARK-3174: --- [~praveenseluka] I took a quick look at you

[jira] [Commented] (SPARK-3704) the types not match adding value form spark row to hive row in SparkSQLOperationManager

[ https://issues.apache.org/jira/browse/SPARK-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172850#comment-14172850 ] Michael Armbrust commented on SPARK-3704: - Yeah, this should be in 1.1.1. > the

[jira] [Updated] (SPARK-3956) Python API for Distributed Matrix

[ https://issues.apache.org/jira/browse/SPARK-3956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-3956: -- Priority: Minor (was: Blocker) > Python API for Distributed Matrix > -

[jira] [Commented] (SPARK-3704) the types not match adding value form spark row to hive row in SparkSQLOperationManager

[ https://issues.apache.org/jira/browse/SPARK-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172645#comment-14172645 ] JIng Wang commented on SPARK-3704: -- Will this fix be merged to branch-v1.1 for the next 1

[jira] [Created] (SPARK-3956) Python API for Distributed Matrix

Davies Liu created SPARK-3956: - Summary: Python API for Distributed Matrix Key: SPARK-3956 URL: https://issues.apache.org/jira/browse/SPARK-3956 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172585#comment-14172585 ] Mridul Muralidharan commented on SPARK-3948: [~jerryshao] great work ! I agree

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172582#comment-14172582 ] Josh Rosen commented on SPARK-3948: --- That kernel bug seems like it could explain why the

[jira] [Updated] (SPARK-1209) SparkHadoop{MapRed,MapReduce}Util should not use package org.apache.hadoop

[ https://issues.apache.org/jira/browse/SPARK-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-1209: - Summary: SparkHadoop{MapRed,MapReduce}Util should not use package org.apache.hadoop (was: SparkHadoopUtil

[jira] [Commented] (SPARK-1209) SparkHadoopUtil should not use package org.apache.hadoop

[ https://issues.apache.org/jira/browse/SPARK-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172571#comment-14172571 ] Sean Owen commented on SPARK-1209: -- ... and why wouldn't you, that's the title of the JIR

[jira] [Commented] (SPARK-3873) Scala style: check import ordering

[ https://issues.apache.org/jira/browse/SPARK-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172552#comment-14172552 ] Marcelo Vanzin commented on SPARK-3873: --- It doesn't seem like I can unassing bugs...

[jira] [Commented] (SPARK-1209) SparkHadoopUtil should not use package org.apache.hadoop

[ https://issues.apache.org/jira/browse/SPARK-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172496#comment-14172496 ] Patrick Wendell commented on SPARK-1209: Hey Sean the class I thought this pertain

[jira] [Commented] (SPARK-2429) Hierarchical Implementation of KMeans

[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172486#comment-14172486 ] RJ Nowling commented on SPARK-2429: --- Great to know! I'm glad that isn't a bottleneck. H

[jira] [Commented] (SPARK-1209) SparkHadoopUtil should not use package org.apache.hadoop

[ https://issues.apache.org/jira/browse/SPARK-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172479#comment-14172479 ] Apache Spark commented on SPARK-1209: - User 'srowen' has created a pull request for th

[jira] [Commented] (SPARK-3629) Improvements to YARN doc

[ https://issues.apache.org/jira/browse/SPARK-3629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172444#comment-14172444 ] Apache Spark commented on SPARK-3629: - User 'ssjssh' has created a pull request for th

[jira] [Commented] (SPARK-3629) Improvements to YARN doc

[ https://issues.apache.org/jira/browse/SPARK-3629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172416#comment-14172416 ] ssj commented on SPARK-3629: I have pushed an request on github about it. https://github.com/

[jira] [Commented] (SPARK-2429) Hierarchical Implementation of KMeans

[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172404#comment-14172404 ] Yu Ishikawa commented on SPARK-2429: I am improving the performance of my code. Theref

[jira] [Commented] (SPARK-2075) Anonymous classes are missing from Spark distribution

[ https://issues.apache.org/jira/browse/SPARK-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172394#comment-14172394 ] Iulian Dragos commented on SPARK-2075: -- The Scala compiler produces stable names for

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172389#comment-14172389 ] Saisai Shao commented on SPARK-3948: So probably a kernel bug in 2.6.32, as you can se

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172387#comment-14172387 ] Praveen Seluka commented on SPARK-3174: --- [~andrewor] - Can you also comment on the A

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172371#comment-14172371 ] Saisai Shao commented on SPARK-3948: Hi [~mridulm80], thanks a lot for your suggestion

[jira] [Commented] (SPARK-3939) NPE caused by SessionState.out not set in thriftserver2

[ https://issues.apache.org/jira/browse/SPARK-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172230#comment-14172230 ] Apache Spark commented on SPARK-3939: - User 'adrian-wang' has created a pull request f

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172226#comment-14172226 ] Mridul Muralidharan commented on SPARK-3948: Note, "t" is just some file with

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172225#comment-14172225 ] Mridul Muralidharan commented on SPARK-3948: That is weird, I tried a stripped

[jira] [Commented] (SPARK-3954) promote the speed of convert files to RDDS

[ https://issues.apache.org/jira/browse/SPARK-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172167#comment-14172167 ] Apache Spark commented on SPARK-3954: - User 'surq' has created a pull request for this

[jira] [Closed] (SPARK-3898) History Web UI display incorrectly.

[ https://issues.apache.org/jira/browse/SPARK-3898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zzc closed SPARK-3898. -- Resolution: Not a Problem Target Version/s: 1.2.0 > History Web UI display incorrectly. >

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172152#comment-14172152 ] Saisai Shao commented on SPARK-3948: Hi [~mridul], what I observed is that, after writ

[jira] [Commented] (SPARK-3955) Different versions between jackson-mapper-asl and jackson-core-asl

[ https://issues.apache.org/jira/browse/SPARK-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172148#comment-14172148 ] Sean Owen commented on SPARK-3955: -- Looks like Jackson is managed to version 1.8.8 for Av

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172134#comment-14172134 ] Mridul Muralidharan commented on SPARK-3948: [~jerryshao] Just to clarify, wha

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172117#comment-14172117 ] Saisai Shao commented on SPARK-3948: Hi Josh, I think for old code without append, th

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172116#comment-14172116 ] Mridul Muralidharan commented on SPARK-3948: [~joshrosen] Assuming there are n

[jira] [Created] (SPARK-3955) Different versions between jackson-mapper-asl and jackson-core-asl

Jongyoul Lee created SPARK-3955: --- Summary: Different versions between jackson-mapper-asl and jackson-core-asl Key: SPARK-3955 URL: https://issues.apache.org/jira/browse/SPARK-3955 Project: Spark

[jira] [Commented] (SPARK-3926) result of JavaRDD collectAsMap() is not serializable

[ https://issues.apache.org/jira/browse/SPARK-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172098#comment-14172098 ] Sean Owen commented on SPARK-3926: -- Nice one, yes it should probably be changed upstream.

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172080#comment-14172080 ] Josh Rosen commented on SPARK-3948: --- I'm surprised that the lack of {{append}} didn't ca

[jira] [Created] (SPARK-3954) promote the speed of convert files to RDDS

宿荣全 created SPARK-3954: -- Summary: promote the speed of convert files to RDDS Key: SPARK-3954 URL: https://issues.apache.org/jira/browse/SPARK-3954 Project: Spark Issue Type: Improvement Compo