[jira] [Commented] (SPARK-19659) Fetch big blocks to disk when shuffle-read

[ https://issues.apache.org/jira/browse/SPARK-19659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025900#comment-16025900 ] Apache Spark commented on SPARK-19659: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-20877) Investigate if tests will time out on CRAN

[ https://issues.apache.org/jira/browse/SPARK-20877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025878#comment-16025878 ] Shivaram Venkataraman commented on SPARK-20877: --- I've been investigating th

[jira] [Commented] (SPARK-14174) Accelerate KMeans via Mini-Batch EM

[ https://issues.apache.org/jira/browse/SPARK-14174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025870#comment-16025870 ] zhengruifeng commented on SPARK-14174: -- [~mlnick] [~sethah] I am sorry to say that

[jira] [Comment Edited] (SPARK-20877) Investigate if tests will time out on CRAN

[ https://issues.apache.org/jira/browse/SPARK-20877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025861#comment-16025861 ] Felix Cheung edited comment on SPARK-20877 at 5/26/17 6:10 AM:

[jira] [Commented] (SPARK-20877) Investigate if tests will time out on CRAN

[ https://issues.apache.org/jira/browse/SPARK-20877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025861#comment-16025861 ] Felix Cheung commented on SPARK-20877: -- One run with skip_on_cran was 27min <7min -

[jira] [Comment Edited] (SPARK-20787) PySpark can't handle datetimes before 1900

[ https://issues.apache.org/jira/browse/SPARK-20787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025853#comment-16025853 ] Yan Facai (颜发才) edited comment on SPARK-20787 at 5/26/17 6:03 AM: -

[jira] [Created] (SPARK-20894) Error while checkpointing to HDFS (similar to JIRA SPARK-19268)

kant kodali created SPARK-20894: --- Summary: Error while checkpointing to HDFS (similar to JIRA SPARK-19268) Key: SPARK-20894 URL: https://issues.apache.org/jira/browse/SPARK-20894 Project: Spark

[jira] [Commented] (SPARK-20787) PySpark can't handle datetimes before 1900

[ https://issues.apache.org/jira/browse/SPARK-20787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025853#comment-16025853 ] Yan Facai (颜发才) commented on SPARK-20787: - It seems that the exception is raised

[jira] [Resolved] (SPARK-20849) Document R DecisionTree

[ https://issues.apache.org/jira/browse/SPARK-20849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20849. -- Resolution: Fixed Assignee: zhengruifeng Fix Version/s: 2.3.0 Targe

[jira] [Resolved] (SPARK-20392) Slow performance when calling fit on ML pipeline for dataset with many columns but few rows

[ https://issues.apache.org/jira/browse/SPARK-20392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20392. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 17770 [https://githu

[jira] [Assigned] (SPARK-20392) Slow performance when calling fit on ML pipeline for dataset with many columns but few rows

[ https://issues.apache.org/jira/browse/SPARK-20392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20392: --- Assignee: Liang-Chi Hsieh > Slow performance when calling fit on ML pipeline for dataset wit

[jira] [Commented] (SPARK-20893) Should we create a constructor for LabelsPoint which is using ml.linalg.Vectors?

[ https://issues.apache.org/jira/browse/SPARK-20893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025821#comment-16025821 ] Frank Sha commented on SPARK-20893: --- I agree that this is not a bug, just didn't find t

[jira] [Resolved] (SPARK-20893) Should we create a constructor for LabelsPoint which is using ml.linalg.Vectors?

[ https://issues.apache.org/jira/browse/SPARK-20893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20893. --- Resolution: Invalid Please read http://spark.apache.org/contributing.html first. This isn't a bug, i

[jira] [Created] (SPARK-20893) Should we create a constructor for LabelsPoint which is using ml.linalg.Vectors?

Frank Sha created SPARK-20893: - Summary: Should we create a constructor for LabelsPoint which is using ml.linalg.Vectors? Key: SPARK-20893 URL: https://issues.apache.org/jira/browse/SPARK-20893 Project: S

[jira] [Commented] (SPARK-20877) Investigate if tests will time out on CRAN

[ https://issues.apache.org/jira/browse/SPARK-20877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025800#comment-16025800 ] Felix Cheung commented on SPARK-20877: -- According to one run, in Jenkins, the build/

[jira] [Commented] (SPARK-16441) Spark application hang when dynamic allocation is enabled

[ https://issues.apache.org/jira/browse/SPARK-16441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025792#comment-16025792 ] Yash Sharma commented on SPARK-16441: - Hitting this issue as well, willing to help if

[jira] [Commented] (SPARK-19372) Code generation for Filter predicate including many OR conditions exceeds JVM method size limit

[ https://issues.apache.org/jira/browse/SPARK-19372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025789#comment-16025789 ] Kazuaki Ishizaki commented on SPARK-19372: -- I see. Let me create a PR for 2.2.0

[jira] [Commented] (SPARK-19372) Code generation for Filter predicate including many OR conditions exceeds JVM method size limit

[ https://issues.apache.org/jira/browse/SPARK-19372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025752#comment-16025752 ] Vish Persaud commented on SPARK-19372: -- +1 for backporting this to 2.2.0! > Code ge

[jira] [Commented] (SPARK-19372) Code generation for Filter predicate including many OR conditions exceeds JVM method size limit

[ https://issues.apache.org/jira/browse/SPARK-19372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025749#comment-16025749 ] Dongjoon Hyun commented on SPARK-19372: --- Hi, [~kiszk]. I met this failure also. Is

[jira] [Resolved] (SPARK-14659) OneHotEncoder support drop first category alphabetically in the encoded vector

[ https://issues.apache.org/jira/browse/SPARK-14659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-14659. - Resolution: Fixed Target Version/s: 2.3.0 > OneHotEncoder support drop first category a

[jira] [Commented] (SPARK-20775) from_json should also have an API where the schema is specified with a string

[ https://issues.apache.org/jira/browse/SPARK-20775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025707#comment-16025707 ] Wenchen Fan commented on SPARK-20775: - hi setjet can you provide your JIRA ID? I wann

[jira] [Resolved] (SPARK-20775) from_json should also have an API where the schema is specified with a string

[ https://issues.apache.org/jira/browse/SPARK-20775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20775. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18094 [https://githu

[jira] [Resolved] (SPARK-20888) Document HiveCaseSensitiveInferenceMode.INFER_AND_SAVE in Spark SQL 2.1 to 2.2 migration notes

[ https://issues.apache.org/jira/browse/SPARK-20888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20888. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 18112 [https://githu

[jira] [Assigned] (SPARK-20888) Document HiveCaseSensitiveInferenceMode.INFER_AND_SAVE in Spark SQL 2.1 to 2.2 migration notes

[ https://issues.apache.org/jira/browse/SPARK-20888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20888: --- Assignee: Michael Allman > Document HiveCaseSensitiveInferenceMode.INFER_AND_SAVE in Spark S

[jira] [Commented] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025588#comment-16025588 ] Shixiong Zhu commented on SPARK-20882: -- [~cenyuhai] Did you see this log "logger.err

[jira] [Commented] (SPARK-20892) Add SQL trunc function to SparkR

[ https://issues.apache.org/jira/browse/SPARK-20892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025583#comment-16025583 ] Apache Spark commented on SPARK-20892: -- User 'actuaryzhang' has created a pull reque

[jira] [Assigned] (SPARK-20892) Add SQL trunc function to SparkR

[ https://issues.apache.org/jira/browse/SPARK-20892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20892: Assignee: (was: Apache Spark) > Add SQL trunc function to SparkR > ---

[jira] [Assigned] (SPARK-20892) Add SQL trunc function to SparkR

[ https://issues.apache.org/jira/browse/SPARK-20892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20892: Assignee: Apache Spark > Add SQL trunc function to SparkR > --

[jira] [Created] (SPARK-20892) Add SQL trunc function to SparkR

Wayne Zhang created SPARK-20892: --- Summary: Add SQL trunc function to SparkR Key: SPARK-20892 URL: https://issues.apache.org/jira/browse/SPARK-20892 Project: Spark Issue Type: New Feature

[jira] [Comment Edited] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025581#comment-16025581 ] cen yuhai edited comment on SPARK-20882 at 5/26/17 12:08 AM: -

[jira] [Commented] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025581#comment-16025581 ] cen yuhai commented on SPARK-20882: --- I think the problem is when the connection to node

[jira] [Commented] (SPARK-20885) JDBC predicate pushdown uses hardcoded date format

[ https://issues.apache.org/jira/browse/SPARK-20885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025533#comment-16025533 ] Jia Li commented on SPARK-20885: I can help to take a look at this. > JDBC predicate pu

[jira] [Commented] (SPARK-20889) SparkR grouped documentation for Column methods

[ https://issues.apache.org/jira/browse/SPARK-20889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025526#comment-16025526 ] Apache Spark commented on SPARK-20889: -- User 'actuaryzhang' has created a pull reque

[jira] [Commented] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025506#comment-16025506 ] Shixiong Zhu commented on SPARK-20882: -- [~cenyuhai] Is it possible to reproduce it a

[jira] [Commented] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025496#comment-16025496 ] cen yuhai commented on SPARK-20882: --- Yes， I am using shuffle service > Executor is wai

[jira] [Comment Edited] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025496#comment-16025496 ] cen yuhai edited comment on SPARK-20882 at 5/25/17 10:38 PM: -

[jira] [Comment Edited] (SPARK-20891) Reduce duplicate code in typedaggregators.scala

[ https://issues.apache.org/jira/browse/SPARK-20891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025459#comment-16025459 ] Ruben Janssen edited comment on SPARK-20891 at 5/25/17 10:03 PM: --

[jira] [Commented] (SPARK-20891) Reduce duplicate code in typedaggregators.scala

[ https://issues.apache.org/jira/browse/SPARK-20891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025459#comment-16025459 ] Ruben Janssen commented on SPARK-20891: --- I thought it would be cleaner to have sepa

[jira] [Commented] (SPARK-20891) Reduce duplicate code in typedaggregators.scala

[ https://issues.apache.org/jira/browse/SPARK-20891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025445#comment-16025445 ] Sean Owen commented on SPARK-20891: --- How is this separate from the other two JIRAs? it

[jira] [Commented] (SPARK-20411) New features for expression.scalalang.typed

[ https://issues.apache.org/jira/browse/SPARK-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025416#comment-16025416 ] Ruben Janssen commented on SPARK-20411: --- Given that there is quite a large number o

[jira] [Comment Edited] (SPARK-20411) New features for expression.scalalang.typed

[ https://issues.apache.org/jira/browse/SPARK-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025416#comment-16025416 ] Ruben Janssen edited comment on SPARK-20411 at 5/25/17 9:19 PM: ---

[jira] [Commented] (SPARK-20891) Reduce duplicate code in typedaggregators.scala

[ https://issues.apache.org/jira/browse/SPARK-20891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025412#comment-16025412 ] Ruben Janssen commented on SPARK-20891: --- Working on this, waiting for SPARK-20890 t

[jira] [Created] (SPARK-20891) Reduce duplicate code in typedaggregators.scala

Ruben Janssen created SPARK-20891: - Summary: Reduce duplicate code in typedaggregators.scala Key: SPARK-20891 URL: https://issues.apache.org/jira/browse/SPARK-20891 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-20890) Add min and max functions for dataset aggregation

[ https://issues.apache.org/jira/browse/SPARK-20890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20890: Assignee: Apache Spark > Add min and max functions for dataset aggregation > -

[jira] [Assigned] (SPARK-20890) Add min and max functions for dataset aggregation

[ https://issues.apache.org/jira/browse/SPARK-20890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20890: Assignee: (was: Apache Spark) > Add min and max functions for dataset aggregation > --

[jira] [Commented] (SPARK-20890) Add min and max functions for dataset aggregation

[ https://issues.apache.org/jira/browse/SPARK-20890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025407#comment-16025407 ] Apache Spark commented on SPARK-20890: -- User 'setjet' has created a pull request for

[jira] [Created] (SPARK-20890) Add min and max functions for dataset aggregation

Ruben Janssen created SPARK-20890: - Summary: Add min and max functions for dataset aggregation Key: SPARK-20890 URL: https://issues.apache.org/jira/browse/SPARK-20890 Project: Spark Issue Typ

[jira] [Commented] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025377#comment-16025377 ] Shixiong Zhu commented on SPARK-20882: -- NVM. I saw NettyBlockTransferService in the

[jira] [Updated] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-20882: -- Description: This bug is like https://issues.apache.org/jira/browse/SPARK-19300. but I have updated my

[jira] [Commented] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025227#comment-16025227 ] Shixiong Zhu commented on SPARK-20882: -- [~cenyuhai] are you using shuffle service?

[jira] [Updated] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-20882: -- Description: This bug is like https://issues.apache.org/jira/browse/SPARK-19300. but I have updated my

[jira] [Assigned] (SPARK-20888) Document HiveCaseSensitiveInferenceMode.INFER_AND_SAVE in Spark SQL 2.1 to 2.2 migration notes

[ https://issues.apache.org/jira/browse/SPARK-20888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20888: Assignee: Apache Spark > Document HiveCaseSensitiveInferenceMode.INFER_AND_SAVE in Spark S

[jira] [Assigned] (SPARK-20888) Document HiveCaseSensitiveInferenceMode.INFER_AND_SAVE in Spark SQL 2.1 to 2.2 migration notes

[ https://issues.apache.org/jira/browse/SPARK-20888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20888: Assignee: (was: Apache Spark) > Document HiveCaseSensitiveInferenceMode.INFER_AND_SAVE

[jira] [Commented] (SPARK-20888) Document HiveCaseSensitiveInferenceMode.INFER_AND_SAVE in Spark SQL 2.1 to 2.2 migration notes

[ https://issues.apache.org/jira/browse/SPARK-20888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025112#comment-16025112 ] Apache Spark commented on SPARK-20888: -- User 'mallman' has created a pull request fo

[jira] [Resolved] (SPARK-20874) The "examples" project doesn't depend on Structured Streaming Kafka source

[ https://issues.apache.org/jira/browse/SPARK-20874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-20874. -- Resolution: Fixed > The "examples" project doesn't depend on Structured Streaming Kafka source

[jira] [Updated] (SPARK-20874) The "examples" project doesn't depend on Structured Streaming Kafka source

[ https://issues.apache.org/jira/browse/SPARK-20874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-20874: - Fix Version/s: 2.2.0 2.1.2 > The "examples" project doesn't depend on Structur

[jira] [Updated] (SPARK-20874) The "examples" project doesn't depend on Structured Streaming Kafka source

[ https://issues.apache.org/jira/browse/SPARK-20874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-20874: - Affects Version/s: 2.2.0 > The "examples" project doesn't depend on Structured Streaming Kafka so

[jira] [Updated] (SPARK-20889) SparkR grouped documentation for Column methods

[ https://issues.apache.org/jira/browse/SPARK-20889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20889: - Labels: documentation (was: ) > SparkR grouped documentation for Column methods > --

[jira] [Updated] (SPARK-20889) SparkR grouped documentation for Column methods

[ https://issues.apache.org/jira/browse/SPARK-20889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20889: - Issue Type: Documentation (was: Improvement) > SparkR grouped documentation for Column methods >

[jira] [Assigned] (SPARK-20889) SparkR grouped documentation for Column methods

[ https://issues.apache.org/jira/browse/SPARK-20889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20889: Assignee: Apache Spark > SparkR grouped documentation for Column methods > ---

[jira] [Assigned] (SPARK-20889) SparkR grouped documentation for Column methods

[ https://issues.apache.org/jira/browse/SPARK-20889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20889: Assignee: (was: Apache Spark) > SparkR grouped documentation for Column methods >

[jira] [Commented] (SPARK-20889) SparkR grouped documentation for Column methods

[ https://issues.apache.org/jira/browse/SPARK-20889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025062#comment-16025062 ] Apache Spark commented on SPARK-20889: -- User 'actuaryzhang' has created a pull reque

[jira] [Created] (SPARK-20889) SparkR grouped documentation for Column methods

Wayne Zhang created SPARK-20889: --- Summary: SparkR grouped documentation for Column methods Key: SPARK-20889 URL: https://issues.apache.org/jira/browse/SPARK-20889 Project: Spark Issue Type: Imp

[jira] [Commented] (SPARK-20888) Document HiveCaseSensitiveInferenceMode.INFER_AND_SAVE in Spark SQL 2.1 to 2.2 migration notes

[ https://issues.apache.org/jira/browse/SPARK-20888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025059#comment-16025059 ] Michael Allman commented on SPARK-20888: I will work on a PR for this and try to

[jira] [Created] (SPARK-20888) Document HiveCaseSensitiveInferenceMode.INFER_AND_SAVE in Spark SQL 2.1 to 2.2 migration notes

Michael Allman created SPARK-20888: -- Summary: Document HiveCaseSensitiveInferenceMode.INFER_AND_SAVE in Spark SQL 2.1 to 2.2 migration notes Key: SPARK-20888 URL: https://issues.apache.org/jira/browse/SPARK-20888

[jira] [Commented] (SPARK-20843) Cannot gracefully kill drivers which take longer than 10 seconds to die

[ https://issues.apache.org/jira/browse/SPARK-20843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025042#comment-16025042 ] Reynold Xin commented on SPARK-20843: - cc [~joshrosen] and [~marmbrus] > Cannot grac

[jira] [Commented] (SPARK-20843) Cannot gracefully kill drivers which take longer than 10 seconds to die

[ https://issues.apache.org/jira/browse/SPARK-20843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025036#comment-16025036 ] Michael Allman commented on SPARK-20843: [~rxin] I'd like to bump this to "Critic

[jira] [Assigned] (SPARK-20741) SparkSubmit does not clean up after uploading spark_libs to the distributed cache

[ https://issues.apache.org/jira/browse/SPARK-20741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-20741: - Assignee: Lior Regev Issue Type: Improvement (was: Bug) > SparkSubmit does not clean up a

[jira] [Resolved] (SPARK-20741) SparkSubmit does not clean up after uploading spark_libs to the distributed cache

[ https://issues.apache.org/jira/browse/SPARK-20741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20741. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17986 [https://github.co

[jira] [Assigned] (SPARK-20886) HadoopMapReduceCommitProtocol to fail with message if FileOutputCommitter.getWorkPath==null

[ https://issues.apache.org/jira/browse/SPARK-20886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20886: Assignee: Apache Spark > HadoopMapReduceCommitProtocol to fail with message if > FileOutp

[jira] [Assigned] (SPARK-20886) HadoopMapReduceCommitProtocol to fail with message if FileOutputCommitter.getWorkPath==null

[ https://issues.apache.org/jira/browse/SPARK-20886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20886: Assignee: (was: Apache Spark) > HadoopMapReduceCommitProtocol to fail with message if

[jira] [Commented] (SPARK-20886) HadoopMapReduceCommitProtocol to fail with message if FileOutputCommitter.getWorkPath==null

[ https://issues.apache.org/jira/browse/SPARK-20886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024914#comment-16024914 ] Apache Spark commented on SPARK-20886: -- User 'steveloughran' has created a pull requ

[jira] [Assigned] (SPARK-20887) support alternative keys in ConfigBuilder

[ https://issues.apache.org/jira/browse/SPARK-20887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20887: Assignee: Wenchen Fan (was: Apache Spark) > support alternative keys in ConfigBuilder > -

[jira] [Assigned] (SPARK-20887) support alternative keys in ConfigBuilder

[ https://issues.apache.org/jira/browse/SPARK-20887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20887: Assignee: Apache Spark (was: Wenchen Fan) > support alternative keys in ConfigBuilder > -

[jira] [Commented] (SPARK-20887) support alternative keys in ConfigBuilder

[ https://issues.apache.org/jira/browse/SPARK-20887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024912#comment-16024912 ] Apache Spark commented on SPARK-20887: -- User 'cloud-fan' has created a pull request

[jira] [Created] (SPARK-20887) support alternative keys in ConfigBuilder

Wenchen Fan created SPARK-20887: --- Summary: support alternative keys in ConfigBuilder Key: SPARK-20887 URL: https://issues.apache.org/jira/browse/SPARK-20887 Project: Spark Issue Type: Improveme

[jira] [Commented] (SPARK-20886) HadoopMapReduceCommitProtocol to fail with message if FileOutputCommitter.getWorkPath==null

[ https://issues.apache.org/jira/browse/SPARK-20886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024886#comment-16024886 ] Steve Loughran commented on SPARK-20886: Stack trace: after {code} 2017-05-25 16:

[jira] [Commented] (SPARK-20886) HadoopMapReduceCommitProtocol to fail with message if FileOutputCommitter.getWorkPath==null

[ https://issues.apache.org/jira/browse/SPARK-20886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024885#comment-16024885 ] Steve Loughran commented on SPARK-20886: Stack trace: before {code} Driver stackt

[jira] [Created] (SPARK-20886) HadoopMapReduceCommitProtocol to fail with message if FileOutputCommitter.getWorkPath==null

Steve Loughran created SPARK-20886: -- Summary: HadoopMapReduceCommitProtocol to fail with message if FileOutputCommitter.getWorkPath==null Key: SPARK-20886 URL: https://issues.apache.org/jira/browse/SPARK-20886

[jira] [Created] (SPARK-20885) JDBC predicate pushdown uses hardcoded date format

Peter Halverson created SPARK-20885: --- Summary: JDBC predicate pushdown uses hardcoded date format Key: SPARK-20885 URL: https://issues.apache.org/jira/browse/SPARK-20885 Project: Spark Issu

[jira] [Commented] (SPARK-20879) Spark SQL will read Date type column from avro file as Int

[ https://issues.apache.org/jira/browse/SPARK-20879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024806#comment-16024806 ] bing huang commented on SPARK-20879: Appreciate, thanks! > Spark SQL will read Date

[jira] [Commented] (SPARK-20879) Spark SQL will read Date type column from avro file as Int

[ https://issues.apache.org/jira/browse/SPARK-20879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024805#comment-16024805 ] Hyukjin Kwon commented on SPARK-20879: -- Not sure. Probably, https://github.com/datab

[jira] [Commented] (SPARK-20879) Spark SQL will read Date type column from avro file as Int

[ https://issues.apache.org/jira/browse/SPARK-20879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024799#comment-16024799 ] bing huang commented on SPARK-20879: Thanks for reply Is there any link to the issue

[jira] [Updated] (SPARK-20799) Unable to infer schema for ORC on S3N when secrets are in the URL

[ https://issues.apache.org/jira/browse/SPARK-20799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-20799: --- Priority: Minor (was: Major) > Unable to infer schema for ORC on S3N when secrets are in the

[jira] [Resolved] (SPARK-20768) PySpark FPGrowth does not expose numPartitions (expert) param

[ https://issues.apache.org/jira/browse/SPARK-20768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-20768. - Resolution: Fixed Fix Version/s: 2.2.0 > PySpark FPGrowth does not expose numPartitions (e

[jira] [Comment Edited] (SPARK-19293) Spark 2.1.x unstable with spark.speculation=true

[ https://issues.apache.org/jira/browse/SPARK-19293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024684#comment-16024684 ] Damian Momot edited comment on SPARK-19293 at 5/25/17 12:50 PM: ---

[jira] [Commented] (SPARK-18838) High latency of event processing for large jobs

[ https://issues.apache.org/jira/browse/SPARK-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024690#comment-16024690 ] Antoine PRANG commented on SPARK-18838: --- [~joshrosen] I uploaded the new timings an

[jira] [Comment Edited] (SPARK-19293) Spark 2.1.x unstable with spark.speculation=true

[ https://issues.apache.org/jira/browse/SPARK-19293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024684#comment-16024684 ] Damian Momot edited comment on SPARK-19293 at 5/25/17 12:45 PM: ---

[jira] [Comment Edited] (SPARK-19293) Spark 2.1.x unstable with spark.speculation=true

[ https://issues.apache.org/jira/browse/SPARK-19293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024684#comment-16024684 ] Damian Momot edited comment on SPARK-19293 at 5/25/17 12:44 PM: ---

[jira] [Commented] (SPARK-19293) Spark 2.1.x unstable with spark.speculation=true

[ https://issues.apache.org/jira/browse/SPARK-19293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024684#comment-16024684 ] Damian Momot commented on SPARK-19293: -- >From what I can see speculative tasks are f

[jira] [Updated] (SPARK-19293) Spark 2.1.x unstable with spark.speculation=true

[ https://issues.apache.org/jira/browse/SPARK-19293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Momot updated SPARK-19293: - Affects Version/s: 2.1.1 Component/s: Spark Core Summary: Spark 2.1.x unst

[jira] [Updated] (SPARK-18838) High latency of event processing for large jobs

[ https://issues.apache.org/jira/browse/SPARK-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antoine PRANG updated SPARK-18838: -- Attachment: perfResults.pdf last timings > High latency of event processing for large jobs >

[jira] [Updated] (SPARK-20881) Use Hive's stats in metastore when cbo is disabled

[ https://issues.apache.org/jira/browse/SPARK-20881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-20881: - Description: Currently statistics are generated by "analyze command" in Spark. However, when us

[jira] [Assigned] (SPARK-20884) Spark' masters will be both standby due to the bug of curator

[ https://issues.apache.org/jira/browse/SPARK-20884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20884: Assignee: Apache Spark > Spark' masters will be both standby due to the bug of curator >

[jira] [Assigned] (SPARK-20884) Spark' masters will be both standby due to the bug of curator

[ https://issues.apache.org/jira/browse/SPARK-20884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20884: Assignee: (was: Apache Spark) > Spark' masters will be both standby due to the bug of

[jira] [Commented] (SPARK-20884) Spark' masters will be both standby due to the bug of curator

[ https://issues.apache.org/jira/browse/SPARK-20884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024623#comment-16024623 ] Apache Spark commented on SPARK-20884: -- User 'liu-zhaokun' has created a pull reques

[jira] [Created] (SPARK-20884) Spark' masters will be both standby due to the bug of curator

liuzhaokun created SPARK-20884: -- Summary: Spark' masters will be both standby due to the bug of curator Key: SPARK-20884 URL: https://issues.apache.org/jira/browse/SPARK-20884 Project: Spark I

[jira] [Commented] (SPARK-19067) mapGroupsWithState - arbitrary stateful operations with Structured Streaming (similar to DStream.mapWithState)

[ https://issues.apache.org/jira/browse/SPARK-19067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024602#comment-16024602 ] Yuval Itzchakov commented on SPARK-19067: - [~marmbrus] [~tdas] Is there any reas

[jira] [Assigned] (SPARK-20376) Make StateStoreProvider plugable

[ https://issues.apache.org/jira/browse/SPARK-20376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20376: Assignee: Tathagata Das (was: Apache Spark) > Make StateStoreProvider plugable >

[jira] [Commented] (SPARK-20376) Make StateStoreProvider plugable

[ https://issues.apache.org/jira/browse/SPARK-20376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024564#comment-16024564 ] Apache Spark commented on SPARK-20376: -- User 'tdas' has created a pull request for t

  1   2   >