[jira] [Commented] (SPARK-3404) SparkSubmitSuite fails with "spark-submit exits with code 1"

[ https://issues.apache.org/jira/browse/SPARK-3404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126698#comment-14126698 ] Apache Spark commented on SPARK-3404: - User 'srowen' has created a pull request for th

[jira] [Updated] (SPARK-3452) Maven build should skip publishing artifacts people shouldn't depend on

[ https://issues.apache.org/jira/browse/SPARK-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3452: --- Affects Version/s: 1.1.0 1.0.0 > Maven build should skip publishing art

[jira] [Created] (SPARK-3452) Maven build should skip publishing artifacts people shouldn't depend on

Patrick Wendell created SPARK-3452: -- Summary: Maven build should skip publishing artifacts people shouldn't depend on Key: SPARK-3452 URL: https://issues.apache.org/jira/browse/SPARK-3452 Project: Sp

[jira] [Updated] (SPARK-3452) Maven build should skip publishing artifacts people shouldn't depend on

[ https://issues.apache.org/jira/browse/SPARK-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3452: --- Priority: Critical (was: Major) > Maven build should skip publishing artifacts people shouldn

[jira] [Commented] (SPARK-2972) APPLICATION_COMPLETE not created in Python unless context explicitly stopped

[ https://issues.apache.org/jira/browse/SPARK-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126686#comment-14126686 ] Shay Rojansky commented on SPARK-2972: -- > you're right! imho, this means your program

[jira] [Commented] (SPARK-3294) Avoid boxing/unboxing when handling in-memory columnar storage

[ https://issues.apache.org/jira/browse/SPARK-3294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126592#comment-14126592 ] Apache Spark commented on SPARK-3294: - User 'liancheng' has created a pull request for

[jira] [Updated] (SPARK-2972) APPLICATION_COMPLETE not created in Python unless context explicitly stopped

[ https://issues.apache.org/jira/browse/SPARK-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2972: --- Priority: Critical (was: Major) > APPLICATION_COMPLETE not created in Python unless context e

[jira] [Updated] (SPARK-2972) APPLICATION_COMPLETE not created in Python unless context explicitly stopped

[ https://issues.apache.org/jira/browse/SPARK-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2972: --- Priority: Major (was: Critical) > APPLICATION_COMPLETE not created in Python unless context e

[jira] [Updated] (SPARK-3450) Enable specifiying the --jars CLI option multiple times

[ https://issues.apache.org/jira/browse/SPARK-3450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3450: --- Shepherd: (was: Marcelo Vanzin) > Enable specifiying the --jars CLI option multiple times >

[jira] [Updated] (SPARK-3451) spark-submit should support specifying glob wildcards in the --jars CLI option

[ https://issues.apache.org/jira/browse/SPARK-3451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3451: --- Shepherd: (was: Marcelo Vanzin) > spark-submit should support specifying glob wildcards in t

[jira] [Created] (SPARK-3451) spark-submit should support specifying glob wildcards in the --jars CLI option

wolfgang hoschek created SPARK-3451: --- Summary: spark-submit should support specifying glob wildcards in the --jars CLI option Key: SPARK-3451 URL: https://issues.apache.org/jira/browse/SPARK-3451 Pr

[jira] [Closed] (SPARK-2425) Standalone Master is too aggressive in removing Applications

[ https://issues.apache.org/jira/browse/SPARK-2425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-2425. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 > Standalone Master is too aggressive

[jira] [Created] (SPARK-3450) Enable specifiying the --jars CLI option multiple times

wolfgang hoschek created SPARK-3450: --- Summary: Enable specifiying the --jars CLI option multiple times Key: SPARK-3450 URL: https://issues.apache.org/jira/browse/SPARK-3450 Project: Spark I

[jira] [Commented] (SPARK-3449) Akka-based receiver can't find messages defined in uploaded jar

[ https://issues.apache.org/jira/browse/SPARK-3449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126513#comment-14126513 ] Anton B commented on SPARK-3449: Proposed fix (by Tathagata Das ): I am not sure if there

[jira] [Commented] (SPARK-3449) Akka-based receiver can't find messages defined in uploaded jar

[ https://issues.apache.org/jira/browse/SPARK-3449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126512#comment-14126512 ] Anton B commented on SPARK-3449: Workaround: adding jar with custom messages to "spark.exe

[jira] [Commented] (SPARK-3449) Akka-based receiver can't find messages defined in uploaded jar

[ https://issues.apache.org/jira/browse/SPARK-3449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126511#comment-14126511 ] Anton B commented on SPARK-3449: User-list message: http://mail-archives.apache.org/mod_m

[jira] [Updated] (SPARK-3449) Akka-based receiver can't find messages defined in uploaded jar

[ https://issues.apache.org/jira/browse/SPARK-3449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton B updated SPARK-3449: --- Attachment: customMessage.zip > Akka-based receiver can't find messages defined in uploaded jar >

[jira] [Created] (SPARK-3449) Akka-based receiver can't find messages defined in uploaded jar

Anton B created SPARK-3449: -- Summary: Akka-based receiver can't find messages defined in uploaded jar Key: SPARK-3449 URL: https://issues.apache.org/jira/browse/SPARK-3449 Project: Spark Issue Type

[jira] [Resolved] (SPARK-3329) HiveQuerySuite SET tests depend on map orderings

[ https://issues.apache.org/jira/browse/SPARK-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3329. - Resolution: Fixed Fix Version/s: 1.2.0 Assignee: William Benton > HiveQuer

[jira] [Closed] (SPARK-3394) TakeOrdered crashes when limit is 0

[ https://issues.apache.org/jira/browse/SPARK-3394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Liang closed SPARK-3394. - > TakeOrdered crashes when limit is 0 > --- > > Key: SPARK-339

[jira] [Closed] (SPARK-3349) Incorrect partitioning after LIMIT operator

[ https://issues.apache.org/jira/browse/SPARK-3349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Liang closed SPARK-3349. - > Incorrect partitioning after LIMIT operator > --- > >

[jira] [Resolved] (SPARK-3414) Case insensitivity breaks when unresolved relation contains attributes with uppercase letters in their names

[ https://issues.apache.org/jira/browse/SPARK-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3414. - Resolution: Fixed Fix Version/s: 1.2.0 Target Version/s: (was: 1.1.0) >

[jira] [Resolved] (SPARK-3423) Implement BETWEEN support for regular SQL parser

[ https://issues.apache.org/jira/browse/SPARK-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3423. - Resolution: Fixed Fix Version/s: 1.2.0 > Implement BETWEEN support for regular SQL

[jira] [Resolved] (SPARK-3443) Update the default values of some decision tree parameters

[ https://issues.apache.org/jira/browse/SPARK-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3443. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2322 [https://githu

[jira] [Closed] (SPARK-2122) Move aggregation into shuffle implementation

[ https://issues.apache.org/jira/browse/SPARK-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao closed SPARK-2122. -- Resolution: Duplicate > Move aggregation into shuffle implementation > -

[jira] [Commented] (SPARK-2122) Move aggregation into shuffle implementation

[ https://issues.apache.org/jira/browse/SPARK-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126471#comment-14126471 ] Saisai Shao commented on SPARK-2122: Yes, this is a duplicated ticket, it is fixed in

[jira] [Commented] (SPARK-3448) SpecificMutableRow.update doesn't check for null

[ https://issues.apache.org/jira/browse/SPARK-3448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126463#comment-14126463 ] Apache Spark commented on SPARK-3448: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-3441) Explain in docs that repartitionAndSortWithinPartitions enacts Hadoop style shuffle

[ https://issues.apache.org/jira/browse/SPARK-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126454#comment-14126454 ] Sandy Ryza commented on SPARK-3441: --- Right. It's not much work, but there are some ques

[jira] [Created] (SPARK-3448) SpecificMutableRow.update doesn't check for null

Cheng Lian created SPARK-3448: - Summary: SpecificMutableRow.update doesn't check for null Key: SPARK-3448 URL: https://issues.apache.org/jira/browse/SPARK-3448 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3447) Kryo NPE when serializing JListWrapper

[ https://issues.apache.org/jira/browse/SPARK-3447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126323#comment-14126323 ] Apache Spark commented on SPARK-3447: - User 'marmbrus' has created a pull request for

[jira] [Commented] (SPARK-3422) JavaAPISuite.getHadoopInputSplits isn't used anywhere

[ https://issues.apache.org/jira/browse/SPARK-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126324#comment-14126324 ] Apache Spark commented on SPARK-3422: - User 'sryza' has created a pull request for thi

[jira] [Commented] (SPARK-3441) Explain in docs that repartitionAndSortWithinPartitions enacts Hadoop style shuffle

[ https://issues.apache.org/jira/browse/SPARK-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126308#comment-14126308 ] Patrick Wendell commented on SPARK-3441: Hey [~sandyr] - what do you mean by "grou

[jira] [Updated] (SPARK-3160) Simplify DecisionTree data structure for training

[ https://issues.apache.org/jira/browse/SPARK-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3160: - Assignee: Joseph K. Bradley > Simplify DecisionTree data structure for training >

[jira] [Resolved] (SPARK-3349) Incorrect partitioning after LIMIT operator

[ https://issues.apache.org/jira/browse/SPARK-3349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3349. - Resolution: Fixed Fix Version/s: 1.2.0 > Incorrect partitioning after LIMIT operato

[jira] [Resolved] (SPARK-3019) Pluggable block transfer (data plane communication) interface

[ https://issues.apache.org/jira/browse/SPARK-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-3019. Resolution: Fixed Fix Version/s: 1.2.0 > Pluggable block transfer (data plane communication)

[jira] [Created] (SPARK-3447) Kryo NPE when serializing JListWrapper

Michael Armbrust created SPARK-3447: --- Summary: Kryo NPE when serializing JListWrapper Key: SPARK-3447 URL: https://issues.apache.org/jira/browse/SPARK-3447 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-3417) Use of old-style classes in pyspark

[ https://issues.apache.org/jira/browse/SPARK-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3417. - Resolution: Fixed Fix Version/s: 1.2.0 > Use of old-style classes in pyspark >

[jira] [Updated] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1239: - Target Version/s: 1.2.0 Affects Version/s: 1.1.0 1.0.2 Fix Version/s:

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126215#comment-14126215 ] Andrew Or commented on SPARK-1239: -- I have reassigned it to you Kostas. > Don't fetch al

[jira] [Updated] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1239: - Assignee: Kostas Sakellis > Don't fetch all map output statuses at each reducer during shuffles >

[jira] [Updated] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1239: - Assignee: (was: Andrew Or) > Don't fetch all map output statuses at each reducer during shuffles > ---

[jira] [Commented] (SPARK-3441) Explain in docs that repartitionAndSortWithinPartitions enacts Hadoop style shuffle

[ https://issues.apache.org/jira/browse/SPARK-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126192#comment-14126192 ] Sandy Ryza commented on SPARK-3441: --- bq. One case where you may not care about giving a

[jira] [Commented] (SPARK-3249) Fix links in ScalaDoc that cause warning messages in `sbt/sbt unidoc`

[ https://issues.apache.org/jira/browse/SPARK-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126191#comment-14126191 ] Xiangrui Meng commented on SPARK-3249: -- I think we should point to the one with the m

[jira] [Commented] (SPARK-3441) Explain in docs that repartitionAndSortWithinPartitions enacts Hadoop style shuffle

[ https://issues.apache.org/jira/browse/SPARK-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126175#comment-14126175 ] Matei Zaharia commented on SPARK-3441: -- I agree that we should have more of a doc her

[jira] [Created] (SPARK-3446) FutureAction should expose the job ID

Marcelo Vanzin created SPARK-3446: - Summary: FutureAction should expose the job ID Key: SPARK-3446 URL: https://issues.apache.org/jira/browse/SPARK-3446 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-3445) Deprecate and later remove YARN alpha support

Patrick Wendell created SPARK-3445: -- Summary: Deprecate and later remove YARN alpha support Key: SPARK-3445 URL: https://issues.apache.org/jira/browse/SPARK-3445 Project: Spark Issue Type: I

[jira] [Closed] (SPARK-3442) Create LengthBoundedInputStream

[ https://issues.apache.org/jira/browse/SPARK-3442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-3442. -- Resolution: Won't Fix Going to use Guava's ByteStreams.limit instead. > Create LengthBoundedInputStream

[jira] [Commented] (SPARK-3443) Update the default values of some decision tree parameters

[ https://issues.apache.org/jira/browse/SPARK-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125985#comment-14125985 ] Apache Spark commented on SPARK-3443: - User 'mengxr' has created a pull request for th

[jira] [Commented] (SPARK-3129) Prevent data loss in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125981#comment-14125981 ] Thomas Graves commented on SPARK-3129: -- yes that should be enough. > Prevent data lo

[jira] [Updated] (SPARK-3444) Provide a way to easily change the log level in the Spark shell while running

[ https://issues.apache.org/jira/browse/SPARK-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-3444: - Assignee: Holden Karau > Provide a way to easily change the log level in the Spark shell while run

[jira] [Updated] (SPARK-3444) Provide a way to easily change the log level in the Spark shell while running

[ https://issues.apache.org/jira/browse/SPARK-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-3444: - Assignee: Holden Karau (was: Holden Karau) > Provide a way to easily change the log level in the

[jira] [Created] (SPARK-3444) Provide a way to easily change the log level in the Spark shell while running

holdenk created SPARK-3444: -- Summary: Provide a way to easily change the log level in the Spark shell while running Key: SPARK-3444 URL: https://issues.apache.org/jira/browse/SPARK-3444 Project: Spark

[jira] [Commented] (SPARK-3272) Calculate prediction for nodes separately from calculating information gain for splits in decision tree

[ https://issues.apache.org/jira/browse/SPARK-3272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125969#comment-14125969 ] Joseph K. Bradley commented on SPARK-3272: -- Hi Qiping, Thanks for your patience;

[jira] [Commented] (SPARK-1087) Separate file for traceback and callsite related functions

[ https://issues.apache.org/jira/browse/SPARK-1087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125957#comment-14125957 ] Matthew Farrellee commented on SPARK-1087: -- [~jyotiska] please do! > Separate fi

[jira] [Commented] (SPARK-3442) Create LengthBoundedInputStream

[ https://issues.apache.org/jira/browse/SPARK-3442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125948#comment-14125948 ] Sean Owen commented on SPARK-3442: -- This exists in Guava as LimitInputStream until Guava

[jira] [Updated] (SPARK-3443) Update the default values of some decision tree parameters

[ https://issues.apache.org/jira/browse/SPARK-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3443: - Priority: Minor (was: Major) Target Version/s: 1.2.0 > Update the default values of s

[jira] [Created] (SPARK-3443) Update the default values of some decision tree parameters

Xiangrui Meng created SPARK-3443: Summary: Update the default values of some decision tree parameters Key: SPARK-3443 URL: https://issues.apache.org/jira/browse/SPARK-3443 Project: Spark Issu

[jira] [Commented] (SPARK-3441) Explain in docs that repartitionAndSortWithinPartitions enacts Hadoop style shuffle

[ https://issues.apache.org/jira/browse/SPARK-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125936#comment-14125936 ] Sandy Ryza commented on SPARK-3441: --- I'll add mention that this can be used to get Hadoo

[jira] [Updated] (SPARK-3442) Create LengthBoundedInputStream

[ https://issues.apache.org/jira/browse/SPARK-3442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3442: --- Description: To create a LengthBoundedInputStream, which is an InputStream decorator that limits the

[jira] [Created] (SPARK-3442) Create LengthBoundedInputStream

Reynold Xin created SPARK-3442: -- Summary: Create LengthBoundedInputStream Key: SPARK-3442 URL: https://issues.apache.org/jira/browse/SPARK-3442 Project: Spark Issue Type: Sub-task Re

[jira] [Commented] (SPARK-2972) APPLICATION_COMPLETE not created in Python unless context explicitly stopped

[ https://issues.apache.org/jira/browse/SPARK-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125938#comment-14125938 ] Matthew Farrellee commented on SPARK-2972: -- > Thanks for answering. I guess it's

[jira] [Comment Edited] (SPARK-3441) Explain in docs that repartitionAndSortWithinPartitions enacts Hadoop style shuffle

[ https://issues.apache.org/jira/browse/SPARK-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125936#comment-14125936 ] Sandy Ryza edited comment on SPARK-3441 at 9/8/14 7:09 PM: --- Beca

[jira] [Updated] (SPARK-3441) Explain in docs that repartitionAndSortWithinPartitions enacts Hadoop style shuffle

[ https://issues.apache.org/jira/browse/SPARK-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3441: --- Description: I think it would be good to say something like this in the doc for repartitionAn

[jira] [Created] (SPARK-3441) Explain in docs that repartitionAndSortWithinPartitions enacts Hadoop style shuffle

Patrick Wendell created SPARK-3441: -- Summary: Explain in docs that repartitionAndSortWithinPartitions enacts Hadoop style shuffle Key: SPARK-3441 URL: https://issues.apache.org/jira/browse/SPARK-3441

[jira] [Created] (SPARK-3440) HiveServer2 and CLI should retrieve Hive result set schema

Cheng Lian created SPARK-3440: - Summary: HiveServer2 and CLI should retrieve Hive result set schema Key: SPARK-3440 URL: https://issues.apache.org/jira/browse/SPARK-3440 Project: Spark Issue Type

[jira] [Updated] (SPARK-3129) Prevent data loss in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Shreedharan updated SPARK-3129: Attachment: SecurityFix.diff Looks like this should be enough, correct? Since the SecurityM

[jira] [Commented] (SPARK-2688) Need a way to run multiple data pipeline concurrently

[ https://issues.apache.org/jira/browse/SPARK-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125868#comment-14125868 ] Matei Zaharia commented on SPARK-2688: -- Just as a note, to launch multiple Spark acti

[jira] [Updated] (SPARK-2978) Provide an MR-style shuffle transformation

[ https://issues.apache.org/jira/browse/SPARK-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-2978: - Assignee: Sandy Ryza > Provide an MR-style shuffle transformation > --

[jira] [Resolved] (SPARK-2978) Provide an MR-style shuffle transformation

[ https://issues.apache.org/jira/browse/SPARK-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-2978. -- Resolution: Fixed Fix Version/s: 1.2.0 > Provide an MR-style shuffle transformation > ---

[jira] [Commented] (SPARK-3129) Prevent data loss in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125828#comment-14125828 ] Hari Shreedharan commented on SPARK-3129: - Correct me if I am wrong here, it looks

[jira] [Commented] (SPARK-3129) Prevent data loss in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125810#comment-14125810 ] Hari Shreedharan commented on SPARK-3129: - (I am not too familiar with how UGI get

[jira] [Commented] (SPARK-3129) Prevent data loss in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125790#comment-14125790 ] Hari Shreedharan commented on SPARK-3129: - [~tgraves] - It looks like the Security

[jira] [Commented] (SPARK-3129) Prevent data loss in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125784#comment-14125784 ] Hari Shreedharan commented on SPARK-3129: - Hi Saisai, You are correct that there

[jira] [Resolved] (SPARK-3337) Paranoid quoting in shell to allow install dirs with spaces within.

[ https://issues.apache.org/jira/browse/SPARK-3337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-3337. -- Resolution: Fixed Target Version/s: 1.1.1 > Paranoid quoting in shell to allow install dirs wi

[jira] [Updated] (SPARK-3337) Paranoid quoting in shell to allow install dirs with spaces within.

[ https://issues.apache.org/jira/browse/SPARK-3337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3337: - Fix Version/s: (was: 1.2.0) 1.1.1 > Paranoid quoting in shell to allow install dirs

[jira] [Resolved] (SPARK-3156) DecisionTree: Order categorical features adaptively

[ https://issues.apache.org/jira/browse/SPARK-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3156. -- Resolution: Fixed Fix Version/s: 1.2.0 > DecisionTree: Order categorical features adaptiv

[jira] [Resolved] (SPARK-3043) DecisionTree aggregation is inefficient

[ https://issues.apache.org/jira/browse/SPARK-3043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3043. -- Resolution: Fixed Fix Version/s: 1.2.0 Target Version/s: 1.2.0 (was: 1.1.0) >

[jira] [Resolved] (SPARK-3086) Use 1-indexing for decision tree nodes

[ https://issues.apache.org/jira/browse/SPARK-3086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3086. -- Resolution: Fixed Fix Version/s: 1.2.0 > Use 1-indexing for decision tree nodes > ---

[jira] [Resolved] (SPARK-675) Gateway JVM should ask for less than SPARK_MEM memory

[ https://issues.apache.org/jira/browse/SPARK-675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-675. -- Resolution: Invalid Thanks for the reminder. I'm going to close this since it only affected a very old

[jira] [Created] (SPARK-3439) Add Canopy Clustering Algorithm

Yu Ishikawa created SPARK-3439: -- Summary: Add Canopy Clustering Algorithm Key: SPARK-3439 URL: https://issues.apache.org/jira/browse/SPARK-3439 Project: Spark Issue Type: New Feature C

[jira] [Commented] (SPARK-3438) Adding support for accessing secured HDFS

[ https://issues.apache.org/jira/browse/SPARK-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125670#comment-14125670 ] Zhanfeng Huo commented on SPARK-3438: - This is a newest PR on master with commit 0d1c

[jira] [Updated] (SPARK-3438) Adding support for accessing secured HDFS

[ https://issues.apache.org/jira/browse/SPARK-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhanfeng Huo updated SPARK-3438: Description: Reading data from secure HDFS into spark is a usefull feature. (was: Reading data fro

[jira] [Updated] (SPARK-3438) Adding support for accessing secured HDFS

[ https://issues.apache.org/jira/browse/SPARK-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhanfeng Huo updated SPARK-3438: Description: Reading data from secure HDFS into spark is a usefull function > Adding support for acc

[jira] [Created] (SPARK-3438) Adding support for accessing secured HDFS

Zhanfeng Huo created SPARK-3438: --- Summary: Adding support for accessing secured HDFS Key: SPARK-3438 URL: https://issues.apache.org/jira/browse/SPARK-3438 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-3437) Adapt maven build to work without the need of hardcoding scala binary version in artifact id.

[ https://issues.apache.org/jira/browse/SPARK-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-3437: --- Component/s: (was: Spark Core) > Adapt maven build to work without the need of hardcoding

[jira] [Commented] (SPARK-3437) Adapt maven build to work without the need of hardcoding scala binary version in artifact id.

[ https://issues.apache.org/jira/browse/SPARK-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125355#comment-14125355 ] Apache Spark commented on SPARK-3437: - User 'ScrapCodes' has created a pull request fo

[jira] [Updated] (SPARK-3437) Adapt maven build to work without the need of hardcoding scala binary version in artifact id.

[ https://issues.apache.org/jira/browse/SPARK-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-3437: --- Summary: Adapt maven build to work without the need of hardcoding scala binary version in arti

[jira] [Assigned] (SPARK-3437) Adapt maven build to work without the need of hardcoding scala binary version in artifact id.

[ https://issues.apache.org/jira/browse/SPARK-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma reassigned SPARK-3437: -- Assignee: Prashant Sharma > Adapt maven build to work without the need of hardcoding sc

[jira] [Created] (SPARK-3437) Maven build to spport building without the need of hardcoding scala binary version in artifact id.

Prashant Sharma created SPARK-3437: -- Summary: Maven build to spport building without the need of hardcoding scala binary version in artifact id. Key: SPARK-3437 URL: https://issues.apache.org/jira/browse/SPARK-34

[jira] [Closed] (SPARK-2981) PartitionStrategy: VertexID hash overflow

[ https://issues.apache.org/jira/browse/SPARK-2981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave closed SPARK-2981. - Resolution: Fixed > PartitionStrategy: VertexID hash overflow > -

[jira] [Updated] (SPARK-2981) PartitionStrategy: VertexID hash overflow

[ https://issues.apache.org/jira/browse/SPARK-2981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-2981: -- Fix Version/s: (was: 1.1.1) 1.1.0 > PartitionStrategy: VertexID hash overflow > -

[jira] [Reopened] (SPARK-2981) PartitionStrategy: VertexID hash overflow

[ https://issues.apache.org/jira/browse/SPARK-2981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave reopened SPARK-2981: --- Reopening temporarily to correct the Fix Version. > PartitionStrategy: VertexID hash overflow > -

[jira] [Updated] (SPARK-3190) Creation of large graph(> 2.15 B nodes) seems to be broken:possible overflow somewhere

[ https://issues.apache.org/jira/browse/SPARK-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-3190: -- Fix Version/s: (was: 1.1.1) 1.1.0 > Creation of large graph(> 2.15 B nodes) seems

[jira] [Updated] (SPARK-3400) GraphX unit tests fail nondeterministically

[ https://issues.apache.org/jira/browse/SPARK-3400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-3400: -- Affects Version/s: 1.1.0 Fix Version/s: (was: 1.3.0) (was: 1.0.1)

[jira] [Created] (SPARK-3436) [MLlib]Streaming SVM

Liquan Pei created SPARK-3436: - Summary: [MLlib]Streaming SVM Key: SPARK-3436 URL: https://issues.apache.org/jira/browse/SPARK-3436 Project: Spark Issue Type: New Feature Components: M

[jira] [Created] (SPARK-3435) Distributed matrix multiplication

Xiangrui Meng created SPARK-3435: Summary: Distributed matrix multiplication Key: SPARK-3435 URL: https://issues.apache.org/jira/browse/SPARK-3435 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-3434) Distributed block matrix

Xiangrui Meng created SPARK-3434: Summary: Distributed block matrix Key: SPARK-3434 URL: https://issues.apache.org/jira/browse/SPARK-3434 Project: Spark Issue Type: New Feature Comp