[jira] [Created] (SPARK-20676) Upload to PyPi

2017-05-08 Thread holdenk (JIRA)
holdenk created SPARK-20676: --- Summary: Upload to PyPi Key: SPARK-20676 URL: https://issues.apache.org/jira/browse/SPARK-20676 Project: Spark Issue Type: Sub-task Components: PySpark A

[jira] [Created] (SPARK-20675) Support Index to skip when retrieval disk structure in CoGroupedRDD

2017-05-08 Thread darion yaphet (JIRA)
darion yaphet created SPARK-20675: - Summary: Support Index to skip when retrieval disk structure in CoGroupedRDD Key: SPARK-20675 URL: https://issues.apache.org/jira/browse/SPARK-20675 Project: Spark

[jira] [Assigned] (SPARK-20674) Support registering UserDefinedFunction as named UDF

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20674: Assignee: Apache Spark (was: Reynold Xin) > Support registering UserDefinedFunction as na

[jira] [Assigned] (SPARK-20674) Support registering UserDefinedFunction as named UDF

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20674: Assignee: Reynold Xin (was: Apache Spark) > Support registering UserDefinedFunction as na

[jira] [Commented] (SPARK-20674) Support registering UserDefinedFunction as named UDF

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002142#comment-16002142 ] Apache Spark commented on SPARK-20674: -- User 'rxin' has created a pull request for t

[jira] [Created] (SPARK-20674) Support registering UserDefinedFunction as named UDF

2017-05-08 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-20674: --- Summary: Support registering UserDefinedFunction as named UDF Key: SPARK-20674 URL: https://issues.apache.org/jira/browse/SPARK-20674 Project: Spark Issue Type

[jira] [Assigned] (SPARK-20673) LDA `optimizer` do not really support case insensitive

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20673: Assignee: (was: Apache Spark) > LDA `optimizer` do not really support case insensitive

[jira] [Assigned] (SPARK-20673) LDA `optimizer` do not really support case insensitive

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20673: Assignee: Apache Spark > LDA `optimizer` do not really support case insensitive > ---

[jira] [Commented] (SPARK-20673) LDA `optimizer` do not really support case insensitive

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002130#comment-16002130 ] Apache Spark commented on SPARK-20673: -- User 'zhengruifeng' has created a pull reque

[jira] [Assigned] (SPARK-20672) Keep the `isStreaming` property in triggerLogicalPlan in Structured Streaming

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20672: Assignee: (was: Apache Spark) > Keep the `isStreaming` property in triggerLogicalPlan

[jira] [Assigned] (SPARK-20672) Keep the `isStreaming` property in triggerLogicalPlan in Structured Streaming

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20672: Assignee: Apache Spark > Keep the `isStreaming` property in triggerLogicalPlan in Structur

[jira] [Commented] (SPARK-20672) Keep the `isStreaming` property in triggerLogicalPlan in Structured Streaming

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002127#comment-16002127 ] Apache Spark commented on SPARK-20672: -- User 'uncleGen' has created a pull request f

[jira] [Created] (SPARK-20673) LDA `optimizer` do not really support case insensitive

2017-05-08 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-20673: Summary: LDA `optimizer` do not really support case insensitive Key: SPARK-20673 URL: https://issues.apache.org/jira/browse/SPARK-20673 Project: Spark Issue

[jira] [Commented] (SPARK-20608) Standby namenodes should be allowed to included in yarn.spark.access.namenodes to support HDFS HA

2017-05-08 Thread Yuechen Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002123#comment-16002123 ] Yuechen Chen commented on SPARK-20608: -- All said, I think it is unreasonable that sp

[jira] [Created] (SPARK-20672) Keep the `isStreaming` property in triggerLogicalPlan in Structured Streaming

2017-05-08 Thread Genmao Yu (JIRA)
Genmao Yu created SPARK-20672: - Summary: Keep the `isStreaming` property in triggerLogicalPlan in Structured Streaming Key: SPARK-20672 URL: https://issues.apache.org/jira/browse/SPARK-20672 Project: Spar

[jira] [Commented] (SPARK-20608) Standby namenodes should be allowed to included in yarn.spark.access.namenodes to support HDFS HA

2017-05-08 Thread Yuechen Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002119#comment-16002119 ] Yuechen Chen commented on SPARK-20608: -- Thanks [~liuml07], [~vanzin]. My colleague i

[jira] [Assigned] (SPARK-20670) Simplify FPGrowth transform

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20670: Assignee: (was: Apache Spark) > Simplify FPGrowth transform >

[jira] [Assigned] (SPARK-20670) Simplify FPGrowth transform

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20670: Assignee: Apache Spark > Simplify FPGrowth transform > --- > >

[jira] [Commented] (SPARK-20670) Simplify FPGrowth transform

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002098#comment-16002098 ] Apache Spark commented on SPARK-20670: -- User 'hhbyyh' has created a pull request for

[jira] [Created] (SPARK-20671) Processing muitple kafka topics with single spark streaming context hangs on batchSubmitted.

2017-05-08 Thread amit kumar (JIRA)
amit kumar created SPARK-20671: -- Summary: Processing muitple kafka topics with single spark streaming context hangs on batchSubmitted. Key: SPARK-20671 URL: https://issues.apache.org/jira/browse/SPARK-20671

[jira] [Assigned] (SPARK-20668) Modify ScalaUDF to handle nullability.

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20668: Assignee: Apache Spark > Modify ScalaUDF to handle nullability. >

[jira] [Assigned] (SPARK-20668) Modify ScalaUDF to handle nullability.

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20668: Assignee: (was: Apache Spark) > Modify ScalaUDF to handle nullability. > -

[jira] [Commented] (SPARK-20668) Modify ScalaUDF to handle nullability.

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002090#comment-16002090 ] Apache Spark commented on SPARK-20668: -- User 'ueshin' has created a pull request for

[jira] [Created] (SPARK-20670) Simplify FPGrowth transform

2017-05-08 Thread yuhao yang (JIRA)
yuhao yang created SPARK-20670: -- Summary: Simplify FPGrowth transform Key: SPARK-20670 URL: https://issues.apache.org/jira/browse/SPARK-20670 Project: Spark Issue Type: Improvement Com

[jira] [Assigned] (SPARK-20669) LogisticRegression family should be case insensitive

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20669: Assignee: Apache Spark > LogisticRegression family should be case insensitive > -

[jira] [Assigned] (SPARK-20669) LogisticRegression family should be case insensitive

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20669: Assignee: (was: Apache Spark) > LogisticRegression family should be case insensitive

[jira] [Commented] (SPARK-20669) LogisticRegression family should be case insensitive

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002086#comment-16002086 ] Apache Spark commented on SPARK-20669: -- User 'zhengruifeng' has created a pull reque

[jira] [Created] (SPARK-20669) LogisticRegression family should be case insensitive

2017-05-08 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-20669: Summary: LogisticRegression family should be case insensitive Key: SPARK-20669 URL: https://issues.apache.org/jira/browse/SPARK-20669 Project: Spark Issue T

[jira] [Commented] (SPARK-20661) SparkR tableNames() test fails

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002082#comment-16002082 ] Apache Spark commented on SPARK-20661: -- User 'felixcheung' has created a pull reques

[jira] [Created] (SPARK-20668) Modify ScalaUDF to handle nullability.

2017-05-08 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-20668: - Summary: Modify ScalaUDF to handle nullability. Key: SPARK-20668 URL: https://issues.apache.org/jira/browse/SPARK-20668 Project: Spark Issue Type: Improvem

[jira] [Assigned] (SPARK-20667) Cleanup the cataloged metadata after completing the package of sql/core and sql/hive

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20667: Assignee: Apache Spark (was: Xiao Li) > Cleanup the cataloged metadata after completing t

[jira] [Assigned] (SPARK-20667) Cleanup the cataloged metadata after completing the package of sql/core and sql/hive

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20667: Assignee: Xiao Li (was: Apache Spark) > Cleanup the cataloged metadata after completing t

[jira] [Commented] (SPARK-20667) Cleanup the cataloged metadata after completing the package of sql/core and sql/hive

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002035#comment-16002035 ] Apache Spark commented on SPARK-20667: -- User 'gatorsmile' has created a pull request

[jira] [Updated] (SPARK-20667) Cleanup the cataloged metadata after completing the package of sql/core and sql/hive

2017-05-08 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-20667: Description: So far, we do not drop all the cataloged tables after each package. Sometimes, we might hit st

[jira] [Updated] (SPARK-20667) Cleanup the cataloged metadata after completing the package of sql/core and sql/hive

2017-05-08 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-20667: Description: So far, we do not drop all the cataloged tables after each package. Sometimes, we might hit st

[jira] [Created] (SPARK-20667) Cleanup the cataloged metadata after completing the package of sql/core and sql/hive

2017-05-08 Thread Xiao Li (JIRA)
Xiao Li created SPARK-20667: --- Summary: Cleanup the cataloged metadata after completing the package of sql/core and sql/hive Key: SPARK-20667 URL: https://issues.apache.org/jira/browse/SPARK-20667 Project: S

[jira] [Updated] (SPARK-20666) Flaky test - SparkListenerBus randomly failing on Windows

2017-05-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20666: - Summary: Flaky test - SparkListenerBus randomly failing on Windows (was: Flaky test - random ml

[jira] [Updated] (SPARK-20666) Flaky test - SparkListenerBus randomly failing java.lang.IllegalAccessError on Windows

2017-05-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20666: - Summary: Flaky test - SparkListenerBus randomly failing java.lang.IllegalAccessError on Windows

[jira] [Updated] (SPARK-20666) Flaky test - SparkListenerBus randomly failing on Windows

2017-05-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20666: - Description: seeing quite a bit of this on AppVeyor, aka Windows only, always only when running

[jira] [Updated] (SPARK-20666) Flaky test - SparkListenerBus randomly failing on Windows

2017-05-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20666: - Component/s: Spark Core > Flaky test - SparkListenerBus randomly failing on Windows > ---

[jira] [Updated] (SPARK-20666) Flaky test - random ml test failure on Windows

2017-05-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20666: - Description: seeing quite a bit of this on AppVeyor, aka Windows only {code} Exception in thread

[jira] [Created] (SPARK-20666) Flaky test - random ml test failure on Windows

2017-05-08 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-20666: Summary: Flaky test - random ml test failure on Windows Key: SPARK-20666 URL: https://issues.apache.org/jira/browse/SPARK-20666 Project: Spark Issue Type: Bu

[jira] [Updated] (SPARK-20666) Flaky test - random ml test failure on Windows

2017-05-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20666: - Description: seeing quite a bit of this on AppVeyor, aka Windows only {code} java.lang.IllegalSt

[jira] [Commented] (SPARK-19581) running NaiveBayes model with 0 features can crash the executor with D rorreGEMV

2017-05-08 Thread 颜发才
[ https://issues.apache.org/jira/browse/SPARK-19581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001987#comment-16001987 ] Yan Facai (颜发才) commented on SPARK-19581: - [~barrybecker4] Could you give a sampl

[jira] [Assigned] (SPARK-7856) Scalable PCA implementation for tall and fat matrices

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7856: --- Assignee: Apache Spark > Scalable PCA implementation for tall and fat matrices >

[jira] [Assigned] (SPARK-7856) Scalable PCA implementation for tall and fat matrices

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7856: --- Assignee: (was: Apache Spark) > Scalable PCA implementation for tall and fat matrices > -

[jira] [Commented] (SPARK-7856) Scalable PCA implementation for tall and fat matrices

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001976#comment-16001976 ] Apache Spark commented on SPARK-7856: - User 'ghoto' has created a pull request for thi

[jira] [Assigned] (SPARK-20665) Spark-sql, "Bround" function return NULL

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20665: Assignee: (was: Apache Spark) > Spark-sql, "Bround" function return NULL > ---

[jira] [Assigned] (SPARK-20665) Spark-sql, "Bround" function return NULL

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20665: Assignee: Apache Spark > Spark-sql, "Bround" function return NULL > --

[jira] [Commented] (SPARK-20665) Spark-sql, "Bround" function return NULL

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001970#comment-16001970 ] Apache Spark commented on SPARK-20665: -- User '10110346' has created a pull request f

[jira] [Commented] (SPARK-20661) SparkR tableNames() test fails

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001966#comment-16001966 ] Apache Spark commented on SPARK-20661: -- User 'felixcheung' has created a pull reques

[jira] [Updated] (SPARK-20665) Spark-sql, "Bround" function return NULL

2017-05-08 Thread liuxian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-20665: Description: >select bround(12.3, 2); >NULL For this case, the expected result is 12.3, but it is null

[jira] [Updated] (SPARK-20665) Spark-sql, "Bround" function return NULL

2017-05-08 Thread liuxian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-20665: Description: >select bround(12.3, 2); >NULL For this case, we expected the result is 12.3, but it is null

[jira] [Created] (SPARK-20665) Spark-sql, "Bround" function return NULL

2017-05-08 Thread liuxian (JIRA)
liuxian created SPARK-20665: --- Summary: Spark-sql, "Bround" function return NULL Key: SPARK-20665 URL: https://issues.apache.org/jira/browse/SPARK-20665 Project: Spark Issue Type: Bug Comp

[jira] [Commented] (SPARK-18278) Support native submission of spark jobs to a kubernetes cluster

2017-05-08 Thread sandflee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001924#comment-16001924 ] sandflee commented on SPARK-18278: -- for a spark user, what benefits could we get? beside

[jira] [Commented] (SPARK-20658) spark.yarn.am.attemptFailuresValidityInterval doesn't seem to have an effect

2017-05-08 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001917#comment-16001917 ] Saisai Shao commented on SPARK-20658: - It is mainly depends on YARN to measure the fa

[jira] [Comment Edited] (SPARK-7856) Scalable PCA implementation for tall and fat matrices

2017-05-08 Thread Ignacio Bermudez Corrales (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001551#comment-16001551 ] Ignacio Bermudez Corrales edited comment on SPARK-7856 at 5/9/17 1:59 AM: --

[jira] [Comment Edited] (SPARK-7856) Scalable PCA implementation for tall and fat matrices

2017-05-08 Thread Ignacio Bermudez Corrales (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001551#comment-16001551 ] Ignacio Bermudez Corrales edited comment on SPARK-7856 at 5/9/17 2:00 AM: --

[jira] [Updated] (SPARK-20663) Data missing after insert overwrite table partition which is created on specific location

2017-05-08 Thread kobefeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kobefeng updated SPARK-20663: - Description: Use spark sql to create partition table first, and alter table by adding partition on speci

[jira] [Updated] (SPARK-20663) Data missing after insert overwrite table partition which is created on specific location

2017-05-08 Thread kobefeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kobefeng updated SPARK-20663: - Description: Use spark sql to create partition table first, and alter table by adding partition on speci

[jira] [Created] (SPARK-20664) Remove stale applications from SHS listing

2017-05-08 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-20664: -- Summary: Remove stale applications from SHS listing Key: SPARK-20664 URL: https://issues.apache.org/jira/browse/SPARK-20664 Project: Spark Issue Type: Su

[jira] [Updated] (SPARK-20663) Data missing after insert overwrite table partition which is created on specific location

2017-05-08 Thread kobefeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kobefeng updated SPARK-20663: - Description: Use spark sql to create partition table first, and alter table by adding partition on speci

[jira] [Updated] (SPARK-20663) Data missing after insert overwrite table partition which is created on specific location

2017-05-08 Thread kobefeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kobefeng updated SPARK-20663: - Priority: Major (was: Minor) > Data missing after insert overwrite table partition which is created on

[jira] [Updated] (SPARK-20663) Data missing after insert overwrite table partition which is created on specific location

2017-05-08 Thread kobefeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kobefeng updated SPARK-20663: - Affects Version/s: (was: 2.1.1) > Data missing after insert overwrite table partition which is create

[jira] [Updated] (SPARK-20663) Data missing after insert overwrite table partition which is created on specific location

2017-05-08 Thread kobefeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kobefeng updated SPARK-20663: - Labels: easyfix (was: ) > Data missing after insert overwrite table partition which is created on > spe

[jira] [Updated] (SPARK-20663) Data missing after insert overwrite table partition which is created on specific location

2017-05-08 Thread kobefeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kobefeng updated SPARK-20663: - Description: Use spark sql to create partition table first, and alter table by adding partition on speci

[jira] [Updated] (SPARK-20663) Data missing after insert overwrite table partition which is created on specific location

2017-05-08 Thread kobefeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kobefeng updated SPARK-20663: - Description: Use spark sql to create partition table first, and alter table by adding partition on speci

[jira] [Updated] (SPARK-20663) Data missing after insert overwrite table partition which is created on specific location

2017-05-08 Thread kobefeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kobefeng updated SPARK-20663: - Description: Use spark sql to create partition table first, and alter table by adding partition on speci

[jira] [Updated] (SPARK-20663) Data missing after insert overwrite table partition which is created on specific location

2017-05-08 Thread kobefeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kobefeng updated SPARK-20663: - Description: Use spark sql to create partition table first, and alter table by adding partition on speci

[jira] [Updated] (SPARK-20663) Data missing after insert overwrite table partition which is created on specific location

2017-05-08 Thread kobefeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kobefeng updated SPARK-20663: - Docs Text: (was: -- create partition table first $ hadoop fs -mkdir /user/kofeng/partitioned_table $ /a

[jira] [Created] (SPARK-20663) Data missing after insert overwrite table partition which is created on specific location

2017-05-08 Thread kobefeng (JIRA)
kobefeng created SPARK-20663: Summary: Data missing after insert overwrite table partition which is created on specific location Key: SPARK-20663 URL: https://issues.apache.org/jira/browse/SPARK-20663 Pro

[jira] [Assigned] (SPARK-20638) Optimize the CartesianRDD to reduce repeatedly data fetching

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20638: Assignee: (was: Apache Spark) > Optimize the CartesianRDD to reduce repeatedly data fe

[jira] [Assigned] (SPARK-20638) Optimize the CartesianRDD to reduce repeatedly data fetching

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20638: Assignee: Apache Spark > Optimize the CartesianRDD to reduce repeatedly data fetching > --

[jira] [Commented] (SPARK-20658) spark.yarn.am.attemptFailuresValidityInterval doesn't seem to have an effect

2017-05-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001811#comment-16001811 ] Marcelo Vanzin commented on SPARK-20658: Ok, so it's not an issue with old YARN j

[jira] [Commented] (SPARK-20658) spark.yarn.am.attemptFailuresValidityInterval doesn't seem to have an effect

2017-05-08 Thread Paul Jones (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001803#comment-16001803 ] Paul Jones commented on SPARK-20658: The jars are versioned 2.7.3. Finally finished

[jira] [Commented] (SPARK-20608) Standby namenodes should be allowed to included in yarn.spark.access.namenodes to support HDFS HA

2017-05-08 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001778#comment-16001778 ] Mingliang Liu commented on SPARK-20608: --- Quick question [~charliechen] : I think [~

[jira] [Reopened] (SPARK-1449) Please delete old releases from mirroring system

2017-05-08 Thread Sebb (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebb reopened SPARK-1449: - The mirror system includes the following: spark-1.6.2 spark-1.6.3 spark-2.0.1 spark-2.0.2 spark-2.1.0 spark-2.1.1 At

[jira] [Comment Edited] (SPARK-20600) KafkaRelation should be pretty printed in web UI (Details for Query)

2017-05-08 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001680#comment-16001680 ] Shixiong Zhu edited comment on SPARK-20600 at 5/8/17 10:38 PM:

[jira] [Commented] (SPARK-20600) KafkaRelation should be pretty printed in web UI (Details for Query)

2017-05-08 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001680#comment-16001680 ] Shixiong Zhu commented on SPARK-20600: -- [~jlaskowski] Hope you want do it soon. Then

[jira] [Commented] (SPARK-20658) spark.yarn.am.attemptFailuresValidityInterval doesn't seem to have an effect

2017-05-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001638#comment-16001638 ] Marcelo Vanzin commented on SPARK-20658: That's different... what version of Hado

[jira] [Comment Edited] (SPARK-20658) spark.yarn.am.attemptFailuresValidityInterval doesn't seem to have an effect

2017-05-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001638#comment-16001638 ] Marcelo Vanzin edited comment on SPARK-20658 at 5/8/17 10:02 PM: --

[jira] [Commented] (SPARK-19268) File does not exist: /tmp/temporary-157b89c1-27bb-49f3-a70c-ca1b75022b4d/state/0/2/1.delta

2017-05-08 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001631#comment-16001631 ] Shixiong Zhu commented on SPARK-19268: -- [~skrishna] could you provide your codes, or

[jira] [Commented] (SPARK-20658) spark.yarn.am.attemptFailuresValidityInterval doesn't seem to have an effect

2017-05-08 Thread Paul Jones (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001628#comment-16001628 ] Paul Jones commented on SPARK-20658: Ah... This is using Amazons version of Hadoop 2

[jira] [Commented] (SPARK-20658) spark.yarn.am.attemptFailuresValidityInterval doesn't seem to have an effect

2017-05-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001623#comment-16001623 ] Marcelo Vanzin commented on SPARK-20658: That does not say which package you used

[jira] [Assigned] (SPARK-20661) SparkR tableNames() test fails

2017-05-08 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai reassigned SPARK-20661: Assignee: Hossein Falaki > SparkR tableNames() test fails > -- > >

[jira] [Commented] (SPARK-20658) spark.yarn.am.attemptFailuresValidityInterval doesn't seem to have an effect

2017-05-08 Thread Paul Jones (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001612#comment-16001612 ] Paul Jones commented on SPARK-20658: {noformat} $ spark-submit --version version 2.1

[jira] [Resolved] (SPARK-20661) SparkR tableNames() test fails

2017-05-08 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-20661. -- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17903 [https://github.com/

[jira] [Assigned] (SPARK-20630) Thread Dump link available in Executors tab irrespective of spark.ui.threadDumpsEnabled

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20630: Assignee: (was: Apache Spark) > Thread Dump link available in Executors tab irrespecti

[jira] [Assigned] (SPARK-20630) Thread Dump link available in Executors tab irrespective of spark.ui.threadDumpsEnabled

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20630: Assignee: Apache Spark > Thread Dump link available in Executors tab irrespective of > sp

[jira] [Commented] (SPARK-20630) Thread Dump link available in Executors tab irrespective of spark.ui.threadDumpsEnabled

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001609#comment-16001609 ] Apache Spark commented on SPARK-20630: -- User 'ajbozarth' has created a pull request

[jira] [Commented] (SPARK-20580) Allow RDD cache with unserializable objects

2017-05-08 Thread Fernando Pereira (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001607#comment-16001607 ] Fernando Pereira commented on SPARK-20580: -- I understand that at some point it w

[jira] [Resolved] (SPARK-20605) Deprecate not used AM and executor port configuration

2017-05-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-20605. Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 2.3.0 > Deprecate no

[jira] [Created] (SPARK-20662) Block jobs that have greater than a configured number of tasks

2017-05-08 Thread Xuefu Zhang (JIRA)
Xuefu Zhang created SPARK-20662: --- Summary: Block jobs that have greater than a configured number of tasks Key: SPARK-20662 URL: https://issues.apache.org/jira/browse/SPARK-20662 Project: Spark

[jira] [Commented] (SPARK-20500) ML, Graph 2.2 QA: API: Binary incompatible changes

2017-05-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001564#comment-16001564 ] Joseph K. Bradley commented on SPARK-20500: --- I'll take this one. > ML, Graph 2

[jira] [Assigned] (SPARK-20500) ML, Graph 2.2 QA: API: Binary incompatible changes

2017-05-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-20500: - Assignee: Joseph K. Bradley > ML, Graph 2.2 QA: API: Binary incompatible changes

[jira] [Comment Edited] (SPARK-7856) Scalable PCA implementation for tall and fat matrices

2017-05-08 Thread Ignacio Bermudez Corrales (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001551#comment-16001551 ] Ignacio Bermudez Corrales edited comment on SPARK-7856 at 5/8/17 9:13 PM: --

[jira] [Commented] (SPARK-7856) Scalable PCA implementation for tall and fat matrices

2017-05-08 Thread Ignacio Bermudez Corrales (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001551#comment-16001551 ] Ignacio Bermudez Corrales commented on SPARK-7856: -- Apart from implementi

[jira] [Assigned] (SPARK-20661) SparkR tableNames() test fails

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20661: Assignee: Apache Spark > SparkR tableNames() test fails > -- >

[jira] [Assigned] (SPARK-20661) SparkR tableNames() test fails

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20661: Assignee: (was: Apache Spark) > SparkR tableNames() test fails > -

[jira] [Commented] (SPARK-20661) SparkR tableNames() test fails

2017-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001521#comment-16001521 ] Apache Spark commented on SPARK-20661: -- User 'falaki' has created a pull request for

  1   2   3   >