[jira] [Commented] (SPARK-24387) Heartbeat-timeout executor is added back and used again

2018-06-11 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16509214#comment-16509214 ] Rui Li commented on SPARK-24387: Yes, blacklisting can be used to avoid the issue. But b

[jira] [Resolved] (SPARK-24502) flaky test: UnsafeRowSerializerSuite

2018-06-11 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24502. - Resolution: Fixed Fix Version/s: 2.3.2 2.4.0 Issue resolved by pull re

[jira] [Assigned] (SPARK-23596) Modify Dataset test harness to include interpreted execution

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23596: Assignee: Apache Spark > Modify Dataset test harness to include interpreted execution > -

[jira] [Commented] (SPARK-23596) Modify Dataset test harness to include interpreted execution

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16509194#comment-16509194 ] Apache Spark commented on SPARK-23596: -- User 'viirya' has created a pull request fo

[jira] [Assigned] (SPARK-23596) Modify Dataset test harness to include interpreted execution

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23596: Assignee: (was: Apache Spark) > Modify Dataset test harness to include interpreted ex

[jira] [Assigned] (SPARK-24526) Spaces in the build dir causes failures in the build/mvn script

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24526: Assignee: Apache Spark > Spaces in the build dir causes failures in the build/mvn script

[jira] [Assigned] (SPARK-24526) Spaces in the build dir causes failures in the build/mvn script

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24526: Assignee: (was: Apache Spark) > Spaces in the build dir causes failures in the build/

[jira] [Commented] (SPARK-24526) Spaces in the build dir causes failures in the build/mvn script

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16509184#comment-16509184 ] Apache Spark commented on SPARK-24526: -- User 'trystanleftwich' has created a pull r

[jira] [Created] (SPARK-24526) Spaces in the build dir causes failures in the build/mvn script

2018-06-11 Thread Trystan Leftwich (JIRA)
Trystan Leftwich created SPARK-24526: Summary: Spaces in the build dir causes failures in the build/mvn script Key: SPARK-24526 URL: https://issues.apache.org/jira/browse/SPARK-24526 Project: Spar

[jira] [Comment Edited] (SPARK-23596) Modify Dataset test harness to include interpreted execution

2018-06-11 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16509137#comment-16509137 ] Liang-Chi Hsieh edited comment on SPARK-23596 at 6/12/18 3:55 AM:

[jira] [Commented] (SPARK-23596) Modify Dataset test harness to include interpreted execution

2018-06-11 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16509137#comment-16509137 ] Liang-Chi Hsieh commented on SPARK-23596: - One concern I have right now is that

[jira] [Commented] (SPARK-24517) Bug in loading unstructured data

2018-06-11 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16509117#comment-16509117 ] Liang-Chi Hsieh commented on SPARK-24517: - Is it also happened when using built-

[jira] [Assigned] (SPARK-24195) sc.addFile for local:/ path is broken

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24195: Assignee: (was: Apache Spark) > sc.addFile for local:/ path is broken > -

[jira] [Commented] (SPARK-24195) sc.addFile for local:/ path is broken

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16509066#comment-16509066 ] Apache Spark commented on SPARK-24195: -- User 'xuanyuanking' has created a pull requ

[jira] [Assigned] (SPARK-24195) sc.addFile for local:/ path is broken

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24195: Assignee: Apache Spark > sc.addFile for local:/ path is broken >

[jira] [Resolved] (SPARK-24514) Exception while converting RDD to DataFrame

2018-06-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24514. -- Resolution: Invalid Reopen and go ahead for a PR if anyone is clear on this. It at least looks

[jira] [Assigned] (SPARK-23732) Broken link to scala source code in Spark Scala api Scaladoc

2018-06-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23732: Assignee: Marcelo Vanzin > Broken link to scala source code in Spark Scala api Scaladoc >

[jira] [Resolved] (SPARK-23732) Broken link to scala source code in Spark Scala api Scaladoc

2018-06-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23732. -- Resolution: Fixed Fix Version/s: 2.3.2 2.4.0 Issue resolved by pull

[jira] [Assigned] (SPARK-22144) ExchangeCoordinator will not combine the partitions of an 0 sized pre-shuffle

2018-06-11 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-22144: --- Assignee: Lijia Liu > ExchangeCoordinator will not combine the partitions of an 0 sized pre

[jira] [Resolved] (SPARK-22144) ExchangeCoordinator will not combine the partitions of an 0 sized pre-shuffle

2018-06-11 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22144. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 19364 [https://gith

[jira] [Updated] (SPARK-24523) InterruptedException when closing SparkContext

2018-06-11 Thread Umayr Hassan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umayr Hassan updated SPARK-24523: - Environment: EMR 5.14.0, S3/HDFS inputs and outputs       was: EMR 5.12.0, S3/HDFS inputs

[jira] [Commented] (SPARK-24516) PySpark Bindings for K8S - make Python 3 the default

2018-06-11 Thread Teng Peng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508945#comment-16508945 ] Teng Peng commented on SPARK-24516: --- +1. > PySpark Bindings for K8S - make Python 3 t

[jira] [Created] (SPARK-24525) Provide an option to limit MemorySink memory usage

2018-06-11 Thread Mukul Murthy (JIRA)
Mukul Murthy created SPARK-24525: Summary: Provide an option to limit MemorySink memory usage Key: SPARK-24525 URL: https://issues.apache.org/jira/browse/SPARK-24525 Project: Spark Issue Type

[jira] [Resolved] (SPARK-24134) A missing full-stop in doc "Tuning Spark"

2018-06-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24134. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21205 [https://github.c

[jira] [Resolved] (SPARK-24520) Double braces in link

2018-06-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24520. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21528 [https://github.c

[jira] [Commented] (SPARK-24387) Heartbeat-timeout executor is added back and used again

2018-06-11 Thread Jiang Xingbo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508832#comment-16508832 ] Jiang Xingbo commented on SPARK-24387: -- {quote}So I think there's a race condition

[jira] [Commented] (SPARK-24093) Make some fields of KafkaStreamWriter/InternalRowMicroBatchWriter visible to outside of the classes

2018-06-11 Thread Mingjie Tang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508814#comment-16508814 ] Mingjie Tang commented on SPARK-24093: -- The related topic could be viewed when user

[jira] [Assigned] (SPARK-24524) Improve aggregateMetrics: less memory usage and loops

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24524: Assignee: (was: Apache Spark) > Improve aggregateMetrics: less memory usage and loops

[jira] [Commented] (SPARK-24524) Improve aggregateMetrics: less memory usage and loops

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508811#comment-16508811 ] Apache Spark commented on SPARK-24524: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-24524) Improve aggregateMetrics: less memory usage and loops

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24524: Assignee: Apache Spark > Improve aggregateMetrics: less memory usage and loops >

[jira] [Commented] (SPARK-24093) Make some fields of KafkaStreamWriter/InternalRowMicroBatchWriter visible to outside of the classes

2018-06-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508808#comment-16508808 ] Sean Owen commented on SPARK-24093: --- What's the use case for exposing these? > Make s

[jira] [Created] (SPARK-24524) Improve aggregateMetrics: less memory usage and loops

2018-06-11 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-24524: -- Summary: Improve aggregateMetrics: less memory usage and loops Key: SPARK-24524 URL: https://issues.apache.org/jira/browse/SPARK-24524 Project: Spark Iss

[jira] [Assigned] (SPARK-24134) A missing full-stop in doc "Tuning Spark"

2018-06-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-24134: - Assignee: Xiaodong DENG Flags: (was: Patch) Priority: Trivial (was: Minor) Please

[jira] [Assigned] (SPARK-24520) Double braces in link

2018-06-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-24520: - Assignee: Fokko Driesprong Priority: Trivial (was: Major) > Double braces in link > --

[jira] [Created] (SPARK-24523) InterruptedException when closing SparkContext

2018-06-11 Thread Umayr Hassan (JIRA)
Umayr Hassan created SPARK-24523: Summary: InterruptedException when closing SparkContext Key: SPARK-24523 URL: https://issues.apache.org/jira/browse/SPARK-24523 Project: Spark Issue Type: Bu

[jira] [Commented] (SPARK-24522) Centralize code to deal with security-related HTTP features

2018-06-11 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508727#comment-16508727 ] Marcelo Vanzin commented on SPARK-24522: FYI I have a prototype for this that I'

[jira] [Created] (SPARK-24522) Centralize code to deal with security-related HTTP features

2018-06-11 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-24522: -- Summary: Centralize code to deal with security-related HTTP features Key: SPARK-24522 URL: https://issues.apache.org/jira/browse/SPARK-24522 Project: Spark

[jira] [Updated] (SPARK-24484) Power Iteration Clustering is giving incorrect clustering results when there are mutiple leading eigen values.

2018-06-11 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-24484: --- Priority: Major (was: Minor) > Power Iteration Clustering is giving incorrect clustering results when there

[jira] [Assigned] (SPARK-24521) Fix ineffective test in CachedTableSuite

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24521: Assignee: (was: Apache Spark) > Fix ineffective test in CachedTableSuite > --

[jira] [Assigned] (SPARK-24521) Fix ineffective test in CachedTableSuite

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24521: Assignee: Apache Spark > Fix ineffective test in CachedTableSuite > -

[jira] [Commented] (SPARK-24521) Fix ineffective test in CachedTableSuite

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508660#comment-16508660 ] Apache Spark commented on SPARK-24521: -- User 'icexelloss' has created a pull reques

[jira] [Created] (SPARK-24521) Fix ineffective test in CachedTableSuite

2018-06-11 Thread Li Jin (JIRA)
Li Jin created SPARK-24521: -- Summary: Fix ineffective test in CachedTableSuite Key: SPARK-24521 URL: https://issues.apache.org/jira/browse/SPARK-24521 Project: Spark Issue Type: Test Compo

[jira] [Resolved] (SPARK-19826) spark.ml Python API for PIC

2018-06-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-19826. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21513 [https://

[jira] [Commented] (SPARK-24495) SortMergeJoin with duplicate keys wrong results

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508450#comment-16508450 ] Apache Spark commented on SPARK-24495: -- User 'mgaido91' has created a pull request

[jira] [Assigned] (SPARK-24495) SortMergeJoin with duplicate keys wrong results

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24495: Assignee: Apache Spark > SortMergeJoin with duplicate keys wrong results > --

[jira] [Assigned] (SPARK-24495) SortMergeJoin with duplicate keys wrong results

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24495: Assignee: (was: Apache Spark) > SortMergeJoin with duplicate keys wrong results > ---

[jira] [Assigned] (SPARK-24520) Double braces in link

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24520: Assignee: (was: Apache Spark) > Double braces in link > - > >

[jira] [Assigned] (SPARK-24520) Double braces in link

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24520: Assignee: Apache Spark > Double braces in link > - > >

[jira] [Commented] (SPARK-24520) Double braces in link

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508429#comment-16508429 ] Apache Spark commented on SPARK-24520: -- User 'Fokko' has created a pull request for

[jira] [Created] (SPARK-24520) Double braces in link

2018-06-11 Thread Fokko Driesprong (JIRA)
Fokko Driesprong created SPARK-24520: Summary: Double braces in link Key: SPARK-24520 URL: https://issues.apache.org/jira/browse/SPARK-24520 Project: Spark Issue Type: Bug Compo

[jira] [Resolved] (SPARK-24511) Spark WebUI allows Weak TLS Protocols

2018-06-11 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24511. Resolution: Not A Problem The default in jdk8 is 1.2. If you configure your application wi

[jira] [Resolved] (SPARK-24508) Spark WebUIs [Security] - Inadequate Cache Directive Headers

2018-06-11 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24508. Resolution: Won't Fix > Spark WebUIs [Security] - Inadequate Cache Directive Headers > ---

[jira] [Reopened] (SPARK-24508) Spark WebUIs [Security] - Inadequate Cache Directive Headers

2018-06-11 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reopened SPARK-24508: > Spark WebUIs [Security] - Inadequate Cache Directive Headers > -

[jira] [Resolved] (SPARK-24508) Spark WebUIs [Security] - Inadequate Cache Directive Headers

2018-06-11 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24508. Resolution: Fixed The "impact" doesn't really explain anything that would make the cached

[jira] [Resolved] (SPARK-24509) Spark WebUI [security] - Web Server Version Disclosure

2018-06-11 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24509. Resolution: Won't Fix The version of the http server Spark uses isn't really a secret. >

[jira] [Resolved] (SPARK-24510) Spark WebUI filters use Basic Authentication [security]

2018-06-11 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24510. Resolution: Not A Bug Please read the documentation. Spark doesn't provide any authenticat

[jira] [Commented] (SPARK-24519) MapStatus has 2000 hardcoded

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508139#comment-16508139 ] Apache Spark commented on SPARK-24519: -- User 'hthuynh2' has created a pull request

[jira] [Assigned] (SPARK-24519) MapStatus has 2000 hardcoded

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24519: Assignee: (was: Apache Spark) > MapStatus has 2000 hardcoded > --

[jira] [Assigned] (SPARK-24519) MapStatus has 2000 hardcoded

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24519: Assignee: Apache Spark > MapStatus has 2000 hardcoded > > >

[jira] [Created] (SPARK-24519) MapStatus has 2000 hardcoded

2018-06-11 Thread Hieu Tri Huynh (JIRA)
Hieu Tri Huynh created SPARK-24519: -- Summary: MapStatus has 2000 hardcoded Key: SPARK-24519 URL: https://issues.apache.org/jira/browse/SPARK-24519 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-24492) Endless attempted task when TaskCommitDenied exception writing to S3A

2018-06-11 Thread Yu-Jhe Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508065#comment-16508065 ] Yu-Jhe Li commented on SPARK-24492: --- [~ste...@apache.org] thanks for your replay, do y

[jira] [Commented] (SPARK-24509) Spark WebUI [security] - Web Server Version Disclosure

2018-06-11 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508032#comment-16508032 ] Marco Gaido commented on SPARK-24509: - I see the point, but Spark is open source, so

[jira] [Comment Edited] (SPARK-24434) Support user-specified driver and executor pod templates

2018-06-11 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16507975#comment-16507975 ] Stavros Kontopoulos edited comment on SPARK-24434 at 6/11/18 1:02 PM:

[jira] [Comment Edited] (SPARK-24434) Support user-specified driver and executor pod templates

2018-06-11 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16507975#comment-16507975 ] Stavros Kontopoulos edited comment on SPARK-24434 at 6/11/18 12:50 PM: ---

[jira] [Commented] (SPARK-24510) Spark WebUI filters use Basic Authentication [security]

2018-06-11 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16507976#comment-16507976 ] Marco Gaido commented on SPARK-24510: - I am not sure this is a real issue. You can c

[jira] [Comment Edited] (SPARK-24434) Support user-specified driver and executor pod templates

2018-06-11 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16507975#comment-16507975 ] Stavros Kontopoulos edited comment on SPARK-24434 at 6/11/18 12:49 PM: ---

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

2018-06-11 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16507975#comment-16507975 ] Stavros Kontopoulos commented on SPARK-24434: - [~liyinan926] here is the des

[jira] [Commented] (SPARK-24474) Cores are left idle when there are a lot of stages to run

2018-06-11 Thread Al M (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16507965#comment-16507965 ] Al M commented on SPARK-24474: -- Also tried changing spark.scheduler.mode to "FIFO"; that di

[jira] [Updated] (SPARK-24518) Using Hadoop credential provider API to store password

2018-06-11 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24518: Description: Current Spark configs password in a plaintext way, like putting in the configuration

[jira] [Updated] (SPARK-24518) Using Hadoop credential provider API to store password

2018-06-11 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24518: Environment: (was: Current Spark configs password in a plaintext way, like putting in the conf

[jira] [Created] (SPARK-24518) Using Hadoop credential provider API to store password

2018-06-11 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-24518: --- Summary: Using Hadoop credential provider API to store password Key: SPARK-24518 URL: https://issues.apache.org/jira/browse/SPARK-24518 Project: Spark Issue Ty

[jira] [Created] (SPARK-24517) Bug in loading unstructured data

2018-06-11 Thread Moshe Israel (JIRA)
Moshe Israel created SPARK-24517: Summary: Bug in loading unstructured data Key: SPARK-24517 URL: https://issues.apache.org/jira/browse/SPARK-24517 Project: Spark Issue Type: Bug Co

[jira] [Created] (SPARK-24516) PySpark Bindings for K8S - make Python 3 the default

2018-06-11 Thread Ondrej Kokes (JIRA)
Ondrej Kokes created SPARK-24516: Summary: PySpark Bindings for K8S - make Python 3 the default Key: SPARK-24516 URL: https://issues.apache.org/jira/browse/SPARK-24516 Project: Spark Issue Ty

[jira] [Assigned] (SPARK-24515) No need to warning user when output commit coordination enabled

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24515: Assignee: Apache Spark > No need to warning user when output commit coordination enabled

[jira] [Assigned] (SPARK-24515) No need to warning user when output commit coordination enabled

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24515: Assignee: (was: Apache Spark) > No need to warning user when output commit coordinati

[jira] [Commented] (SPARK-24515) No need to warning user when output commit coordination enabled

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16507835#comment-16507835 ] Apache Spark commented on SPARK-24515: -- User 'caneGuy' has created a pull request f

[jira] [Updated] (SPARK-24515) No need to warning user when output commit coordination enabled

2018-06-11 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang updated SPARK-24515: - Description:   No need to warning user when output commit coordination enabled {code:java} // When spec

[jira] [Updated] (SPARK-24515) No need to warning user when output commit coordination enabled

2018-06-11 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang updated SPARK-24515: - Description:   {code:java}  {code} No need to warning user when output commit coordination enabled {cod

[jira] [Created] (SPARK-24515) No need to warning user when output commit coordination enabled

2018-06-11 Thread zhoukang (JIRA)
zhoukang created SPARK-24515: Summary: No need to warning user when output commit coordination enabled Key: SPARK-24515 URL: https://issues.apache.org/jira/browse/SPARK-24515 Project: Spark Issu

[jira] [Assigned] (SPARK-24500) UnsupportedOperationException when trying to execute Union plan with Stream of children

2018-06-11 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell reassigned SPARK-24500: - Assignee: Herman van Hovell > UnsupportedOperationException when trying to exec

[jira] [Assigned] (SPARK-24513) Attribute support in UnaryTransformer

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24513: Assignee: (was: Apache Spark) > Attribute support in UnaryTransformer > -

[jira] [Assigned] (SPARK-24513) Attribute support in UnaryTransformer

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24513: Assignee: Apache Spark > Attribute support in UnaryTransformer >

[jira] [Commented] (SPARK-24513) Attribute support in UnaryTransformer

2018-06-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16507825#comment-16507825 ] Apache Spark commented on SPARK-24513: -- User 'dongjinleekr' has created a pull requ

[jira] [Commented] (SPARK-24514) Exception while converting RDD to DataFrame

2018-06-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16507818#comment-16507818 ] Hyukjin Kwon commented on SPARK-24514: -- Please avoid setting target version which i

[jira] [Updated] (SPARK-24514) Exception while converting RDD to DataFrame

2018-06-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24514: - Target Version/s: (was: 2.3.0) > Exception while converting RDD to DataFrame > ---