[jira] [Commented] (SPARK-25102) Write Spark version to ORC/Parquet file metadata

2018-11-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673948#comment-16673948 ] Apache Spark commented on SPARK-25102: -- User 'dongjoon-hyun' has created a pull req

[jira] [Updated] (SPARK-25102) Write Spark version to ORC/Parquet file metadata

2018-11-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25102: -- Description: Currently, Spark writes Spark version number into Hive Table properties with `sp

[jira] [Updated] (SPARK-25102) Write Spark version to ORC/Parquet file metadata

2018-11-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25102: -- Description: Currently, Spark writes Spark version number into Hive Table properties with `sp

[jira] [Updated] (SPARK-25102) Write Spark version to ORC/Parquet file metadata

2018-11-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25102: -- Description: Currently, Spark writes Spark version number into Hive Table properties with `sp

[jira] [Updated] (SPARK-25102) Write Spark version to ORC/Parquet file metadata

2018-11-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25102: -- Description: Currently, Spark writes Spark version number into Hive Table properties with `or

[jira] [Updated] (SPARK-25102) Write Spark version to Parquet/ORC file metadata

2018-11-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25102: -- Description: Currently, Spark writes Spark version number into Hive Table properties with `or

[jira] [Updated] (SPARK-25102) Write Spark version to ORC/Parquet file metadata

2018-11-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25102: -- Summary: Write Spark version to ORC/Parquet file metadata (was: Write Spark version to Parque

[jira] [Updated] (SPARK-25102) Write Spark version to Parquet/ORC file metadata

2018-11-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25102: -- Summary: Write Spark version to Parquet/ORC file metadata (was: Write Spark version informati

[jira] [Updated] (SPARK-25102) Write Spark version to Parquet/ORC file metadata

2018-11-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25102: -- Priority: Major (was: Minor) > Write Spark version to Parquet/ORC file metadata > ---

[jira] [Updated] (SPARK-25102) Write Spark version to Parquet/ORC file metadata

2018-11-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25102: -- Affects Version/s: (was: 2.3.1) 3.0.0 > Write Spark version to Parq

[jira] [Resolved] (SPARK-25901) Barrier mode spawns a bunch of threads that get collected on gc

2018-11-02 Thread Xingbo Jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-25901. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22912 [https://gi

[jira] [Commented] (SPARK-24758) Create table wants to use /user/hive/warehouse in clean clone

2018-11-02 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673939#comment-16673939 ] Yuming Wang commented on SPARK-24758: - cc [~Qin Yao] > Create table wants to use /u

[jira] [Commented] (SPARK-24758) Create table wants to use /user/hive/warehouse in clean clone

2018-11-02 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673931#comment-16673931 ] Yuming Wang commented on SPARK-24758: - The reason is that  [SessionState.start(sessi

[jira] [Comment Edited] (SPARK-25925) Spark 2.3.1 retrieves all partitions from Hive Metastore by default

2018-11-02 Thread Alex Ivanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673842#comment-16673842 ] Alex Ivanov edited comment on SPARK-25925 at 11/3/18 12:05 AM: ---

[jira] [Commented] (SPARK-25925) Spark 2.3.1 retrieves all partitions from Hive Metastore by default

2018-11-02 Thread Alex Ivanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673842#comment-16673842 ] Alex Ivanov commented on SPARK-25925: - Thank you for the clarification, [~budde]. Th

[jira] [Updated] (SPARK-24417) Build and Run Spark on JDK11

2018-11-02 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-24417: Description: This is an umbrella JIRA for Apache Spark to support JDK11 As JDK8 is reaching EOL, and JDK9

[jira] [Updated] (SPARK-24417) Build and Run Spark on JDK11

2018-11-02 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-24417: Summary: Build and Run Spark on JDK11 (was: Build and Run Spark on JDK9+) > Build and Run Spark on JDK11

[jira] [Commented] (SPARK-24417) Build and Run Spark on JDK9+

2018-11-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673781#comment-16673781 ] Dongjoon Hyun commented on SPARK-24417: --- +1 for skipping 9 and 10. > Build and Ru

[jira] [Commented] (SPARK-24417) Build and Run Spark on JDK9+

2018-11-02 Thread Randall Whitman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673779#comment-16673779 ] Randall Whitman commented on SPARK-24417: - Agree - I'm interested in Java-11, no

[jira] [Commented] (SPARK-24417) Build and Run Spark on JDK9+

2018-11-02 Thread Cesar Delgado (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673778#comment-16673778 ] Cesar Delgado commented on SPARK-24417: --- Hadoop just went though the same process

[jira] [Commented] (SPARK-24417) Build and Run Spark on JDK9+

2018-11-02 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673769#comment-16673769 ] DB Tsai commented on SPARK-24417: - Since JDK 11 is LTS, should we skip 9 and 10? If no o

[jira] [Commented] (SPARK-25925) Spark 2.3.1 retrieves all partitions from Hive Metastore by default

2018-11-02 Thread Adam Budde (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673763#comment-16673763 ] Adam Budde commented on SPARK-25925: It's been a while since I've thought about all

[jira] [Comment Edited] (SPARK-25924) k8s integration tests on jenkins do not save log output

2018-11-02 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673753#comment-16673753 ] shane knapp edited comment on SPARK-25924 at 11/2/18 10:27 PM: ---

[jira] [Resolved] (SPARK-25924) k8s integration tests on jenkins do not save log output

2018-11-02 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shane knapp resolved SPARK-25924. - Resolution: Fixed > k8s integration tests on jenkins do not save log output > --

[jira] [Reopened] (SPARK-25924) k8s integration tests on jenkins do not save log output

2018-11-02 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shane knapp reopened SPARK-25924: - this is actually causing builds that don't test k8s to fail: https://amplab.cs.berkeley.edu/jenkins

[jira] [Assigned] (SPARK-25861) Remove unused refreshInterval parameter from the headerSparkPage method.

2018-11-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25861: - Assignee: shahid > Remove unused refreshInterval parameter from the headerSparkPage method. > -

[jira] [Resolved] (SPARK-25861) Remove unused refreshInterval parameter from the headerSparkPage method.

2018-11-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25861. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22864 [https://github.c

[jira] [Resolved] (SPARK-25924) k8s integration tests on jenkins do not save log output

2018-11-02 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shane knapp resolved SPARK-25924. - Resolution: Fixed > k8s integration tests on jenkins do not save log output > --

[jira] [Commented] (SPARK-25924) k8s integration tests on jenkins do not save log output

2018-11-02 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673711#comment-16673711 ] shane knapp commented on SPARK-25924: - ok done for real! > k8s integration tests on

[jira] [Commented] (SPARK-25590) kubernetes-model-2.0.0.jar masks default Spark logging config

2018-11-02 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673689#comment-16673689 ] Marcelo Vanzin commented on SPARK-25590: {noformat} $ jar tf assembly/target/sca

[jira] [Commented] (SPARK-25928) NoSuchMethodError net.jpountz.lz4.LZ4BlockInputStream.(Ljava/io/InputStream;Z)V

2018-11-02 Thread Jerry Chabot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673680#comment-16673680 ] Jerry Chabot commented on SPARK-25928: -- I noticed our uber jar has spark classes in

[jira] [Commented] (SPARK-25924) k8s integration tests on jenkins do not save log output

2018-11-02 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673674#comment-16673674 ] shane knapp commented on SPARK-25924: - alright, waiting for the next test to run bef

[jira] [Resolved] (SPARK-25875) Merge code to set up driver features for different languages

2018-11-02 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah resolved SPARK-25875. Resolution: Fixed > Merge code to set up driver features for different languages > ---

[jira] [Commented] (SPARK-25924) k8s integration tests on jenkins do not save log output

2018-11-02 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673655#comment-16673655 ] shane knapp commented on SPARK-25924: - ah, found the right log!  :) ./resource-mana

[jira] [Assigned] (SPARK-25827) Replicating a block > 2gb with encryption fails

2018-11-02 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25827: -- Assignee: Imran Rashid > Replicating a block > 2gb with encryption fails > --

[jira] [Resolved] (SPARK-25827) Replicating a block > 2gb with encryption fails

2018-11-02 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25827. Resolution: Fixed Fix Version/s: 2.4.1 3.0.0 Issue resolved by p

[jira] [Comment Edited] (SPARK-25924) k8s integration tests on jenkins do not save log output

2018-11-02 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673624#comment-16673624 ] shane knapp edited comment on SPARK-25924 at 11/2/18 8:19 PM:

[jira] [Commented] (SPARK-25924) k8s integration tests on jenkins do not save log output

2018-11-02 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673624#comment-16673624 ] shane knapp commented on SPARK-25924: - done.  i'll close this after we have a failin

[jira] [Commented] (SPARK-25924) k8s integration tests on jenkins do not save log output

2018-11-02 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673623#comment-16673623 ] shane knapp commented on SPARK-25924: - i will copy the setup from the regular pull r

[jira] [Assigned] (SPARK-25924) k8s integration tests on jenkins do not save log output

2018-11-02 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shane knapp reassigned SPARK-25924: --- Assignee: shane knapp > k8s integration tests on jenkins do not save log output > -

[jira] [Updated] (SPARK-25930) Fix scala version string detection when maven-help-plugin is not pre-installed

2018-11-02 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-25930: Description: This line [https://github.com/apache/spark/blob/c71db43e11fb90d66754

[jira] [Commented] (SPARK-25930) Fix scala version string detection when maven-help-plugin is not pre-installed

2018-11-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673604#comment-16673604 ] Apache Spark commented on SPARK-25930: -- User 'skonto' has created a pull request fo

[jira] [Assigned] (SPARK-25930) Fix scala version string detection when maven-help-plugin is not pre-installed

2018-11-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25930: Assignee: (was: Apache Spark) > Fix scala version string detection when maven-help-pl

[jira] [Commented] (SPARK-25930) Fix scala version string detection when maven-help-plugin is not pre-installed

2018-11-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673602#comment-16673602 ] Apache Spark commented on SPARK-25930: -- User 'skonto' has created a pull request fo

[jira] [Assigned] (SPARK-25930) Fix scala version string detection when maven-help-plugin is not pre-installed

2018-11-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25930: Assignee: Apache Spark > Fix scala version string detection when maven-help-plugin is not

[jira] [Updated] (SPARK-25930) Fix scala version string detection when maven-help-plugin is not pre-installed

2018-11-02 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-25930: Priority: Minor (was: Major) > Fix scala version string detection when maven-help

[jira] [Comment Edited] (SPARK-25079) [PYTHON] upgrade python 3.4 -> 3.5

2018-11-02 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673591#comment-16673591 ] shane knapp edited comment on SPARK-25079 at 11/2/18 7:29 PM:

[jira] [Comment Edited] (SPARK-25079) [PYTHON] upgrade python 3.4 -> 3.5

2018-11-02 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673591#comment-16673591 ] shane knapp edited comment on SPARK-25079 at 11/2/18 7:27 PM:

[jira] [Commented] (SPARK-25079) [PYTHON] upgrade python 3.4 -> 3.5

2018-11-02 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673591#comment-16673591 ] shane knapp commented on SPARK-25079: - i think we're ready to deploy python3.5 (whic

[jira] [Comment Edited] (SPARK-25079) [PYTHON] upgrade python 3.4 -> 3.5

2018-11-02 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673591#comment-16673591 ] shane knapp edited comment on SPARK-25079 at 11/2/18 7:26 PM:

[jira] [Commented] (SPARK-25930) Fix scala version string detection when maven-help-plugin is not pre-installed

2018-11-02 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673565#comment-16673565 ] Stavros Kontopoulos commented on SPARK-25930: - Will have a PR shortly.  > F

[jira] [Created] (SPARK-25930) Fix scala version string detection when maven-help-plugin is not pre-installed

2018-11-02 Thread Stavros Kontopoulos (JIRA)
Stavros Kontopoulos created SPARK-25930: --- Summary: Fix scala version string detection when maven-help-plugin is not pre-installed Key: SPARK-25930 URL: https://issues.apache.org/jira/browse/SPARK-25930

[jira] [Issue Comment Deleted] (SPARK-25079) [PYTHON] upgrade python 3.4 -> 3.5

2018-11-02 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shane knapp updated SPARK-25079: Comment: was deleted (was: question:  do we want to upgrade to 3.6 instead?) > [PYTHON] upgrade p

[jira] [Commented] (SPARK-24869) SaveIntoDataSourceCommand's input Dataset does not use Cached Data

2018-11-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673531#comment-16673531 ] Apache Spark commented on SPARK-24869: -- User 'wangyum' has created a pull request f

[jira] [Commented] (SPARK-24869) SaveIntoDataSourceCommand's input Dataset does not use Cached Data

2018-11-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673532#comment-16673532 ] Apache Spark commented on SPARK-24869: -- User 'wangyum' has created a pull request f

[jira] [Commented] (SPARK-25924) k8s integration tests on jenkins do not save log output

2018-11-02 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673431#comment-16673431 ] Marcelo Vanzin commented on SPARK-25924: [~shaneknapp], it seems that the ITs do

[jira] [Assigned] (SPARK-25909) Error in documentation: number of cluster managers

2018-11-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25909: - Assignee: James Lamb > Error in documentation: number of cluster managers > ---

[jira] [Resolved] (SPARK-25909) Error in documentation: number of cluster managers

2018-11-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25909. --- Resolution: Fixed Fix Version/s: 3.0.0 2.4.1 Issue resolved by pull reques

[jira] [Commented] (SPARK-25909) Error in documentation: number of cluster managers

2018-11-02 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673335#comment-16673335 ] Sean Owen commented on SPARK-25909: --- It's fine, but no need to make a Jira for trivial

[jira] [Assigned] (SPARK-25023) Clarify Spark security documentation

2018-11-02 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-25023: - Assignee: Thomas Graves > Clarify Spark security documentation > --

[jira] [Resolved] (SPARK-25023) Clarify Spark security documentation

2018-11-02 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-25023. --- Resolution: Fixed Fix Version/s: 3.0.0 2.4.1 > Clarify Spark secur

[jira] [Updated] (SPARK-25929) Support metrics with tags

2018-11-02 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated SPARK-25929: --- Description: For better integration with DBs that support tags/labels, e.g., InfluxDB, Prometheus,

[jira] [Created] (SPARK-25929) Support metrics with tags

2018-11-02 Thread John Zhuge (JIRA)
John Zhuge created SPARK-25929: -- Summary: Support metrics with tags Key: SPARK-25929 URL: https://issues.apache.org/jira/browse/SPARK-25929 Project: Spark Issue Type: Improvement Compo

[jira] [Created] (SPARK-25928) NoSuchMethodError net.jpountz.lz4.LZ4BlockInputStream.(Ljava/io/InputStream;Z)V

2018-11-02 Thread Jerry Chabot (JIRA)
Jerry Chabot created SPARK-25928: Summary: NoSuchMethodError net.jpountz.lz4.LZ4BlockInputStream.(Ljava/io/InputStream;Z)V Key: SPARK-25928 URL: https://issues.apache.org/jira/browse/SPARK-25928 Proje

[jira] [Updated] (SPARK-25504) [Spark Job History]If the number of tasks exceeds spark.ui.retainedTask value, then stage page of UI should display upto the value set in the retainedTask.

2018-11-02 Thread ABHISHEK KUMAR GUPTA (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ABHISHEK KUMAR GUPTA updated SPARK-25504: - Priority: Minor (was: Major) Issue Type: Documentation (was: Bug) Docume

[jira] [Resolved] (SPARK-25923) SparkR UT Failure (checking CRAN incoming feasibility)

2018-11-02 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25923. -- Resolution: Fixed Assignee: Liang-Chi Hsieh > SparkR UT Failure (checking CRAN incoming

[jira] [Commented] (SPARK-25923) SparkR UT Failure (checking CRAN incoming feasibility)

2018-11-02 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672893#comment-16672893 ] Hyukjin Kwon commented on SPARK-25923: -- Thanks. I'm leaving this resolved for now.

[jira] [Commented] (SPARK-25927) Fix number of partitions returned by outputPartitioning

2018-11-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672889#comment-16672889 ] Apache Spark commented on SPARK-25927: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-25927) Fix number of partitions returned by outputPartitioning

2018-11-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25927: Assignee: Apache Spark > Fix number of partitions returned by outputPartitioning > --

[jira] [Commented] (SPARK-25927) Fix number of partitions returned by outputPartitioning

2018-11-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672887#comment-16672887 ] Apache Spark commented on SPARK-25927: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-25927) Fix number of partitions returned by outputPartitioning

2018-11-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25927: Assignee: (was: Apache Spark) > Fix number of partitions returned by outputPartitioni

[jira] [Commented] (SPARK-25923) SparkR UT Failure (checking CRAN incoming feasibility)

2018-11-02 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672885#comment-16672885 ] Liang-Chi Hsieh commented on SPARK-25923: - CRAN sysadmin replied me it should be

[jira] [Comment Edited] (SPARK-25923) SparkR UT Failure (checking CRAN incoming feasibility)

2018-11-02 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672885#comment-16672885 ] Liang-Chi Hsieh edited comment on SPARK-25923 at 11/2/18 10:27 AM: ---

[jira] [Created] (SPARK-25927) Fix number of partitions returned by outputPartitioning

2018-11-02 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-25927: -- Summary: Fix number of partitions returned by outputPartitioning Key: SPARK-25927 URL: https://issues.apache.org/jira/browse/SPARK-25927 Project: Spark Issue Typ

[jira] [Commented] (SPARK-20568) Delete files after processing in structured streaming

2018-11-02 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672805#comment-16672805 ] Jungtaek Lim commented on SPARK-20568: -- [~zsxwing] I've thought about it a bit. I'm

[jira] [Commented] (SPARK-25926) Move config entries in core module to internal.config.

2018-11-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672771#comment-16672771 ] Apache Spark commented on SPARK-25926: -- User 'ueshin' has created a pull request fo

[jira] [Assigned] (SPARK-25926) Move config entries in core module to internal.config.

2018-11-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25926: Assignee: (was: Apache Spark) > Move config entries in core module to internal.config

[jira] [Assigned] (SPARK-25926) Move config entries in core module to internal.config.

2018-11-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25926: Assignee: Apache Spark > Move config entries in core module to internal.config. > ---

[jira] [Commented] (SPARK-25926) Move config entries in core module to internal.config.

2018-11-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672770#comment-16672770 ] Apache Spark commented on SPARK-25926: -- User 'ueshin' has created a pull request fo

[jira] [Created] (SPARK-25926) Move config entries in core module to internal.config.

2018-11-02 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-25926: - Summary: Move config entries in core module to internal.config. Key: SPARK-25926 URL: https://issues.apache.org/jira/browse/SPARK-25926 Project: Spark Issu