[jira] [Created] (SPARK-22085) When the applicationhas no core left,do not request more executors from the cluster manager in ExecutorAllocationManager

2017-09-21 Thread liuxian (JIRA)
liuxian created SPARK-22085: --- Summary: When the applicationhas no core left,do not request more executors from the cluster manager in ExecutorAllocationManager Key: SPARK-22085 URL: https://issues.apache.org/jira/brows

[jira] [Updated] (SPARK-22058) the BufferedInputStream will not be closed if an exception occurs

2017-09-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22058: -- Priority: Trivial (was: Minor) > the BufferedInputStream will not be closed if an exception occurs > -

[jira] [Commented] (SPARK-22085) When the applicationhas no core left,do not request more executors from the cluster manager in ExecutorAllocationManager

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174404#comment-16174404 ] Apache Spark commented on SPARK-22085: -- User '10110346' has created a pull request f

[jira] [Assigned] (SPARK-22085) When the applicationhas no core left,do not request more executors from the cluster manager in ExecutorAllocationManager

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22085: Assignee: (was: Apache Spark) > When the applicationhas no core left,do not request mo

[jira] [Assigned] (SPARK-22085) When the applicationhas no core left,do not request more executors from the cluster manager in ExecutorAllocationManager

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22085: Assignee: Apache Spark > When the applicationhas no core left,do not request more executor

[jira] [Commented] (SPARK-22081) Generalized Reduced Error Logistic Regression

2017-09-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174410#comment-16174410 ] Sean Owen commented on SPARK-22081: --- I don't believe this would be added by request. Yo

[jira] [Created] (SPARK-22086) Add expression description for CASE WHEN

2017-09-21 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-22086: --- Summary: Add expression description for CASE WHEN Key: SPARK-22086 URL: https://issues.apache.org/jira/browse/SPARK-22086 Project: Spark Issue Type: Do

[jira] [Updated] (SPARK-22086) Add expression description for CASE WHEN

2017-09-21 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-22086: Issue Type: Sub-task (was: Documentation) Parent: SPARK-21653 > Add expression des

[jira] [Assigned] (SPARK-22086) Add expression description for CASE WHEN

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22086: Assignee: (was: Apache Spark) > Add expression description for CASE WHEN > ---

[jira] [Assigned] (SPARK-22086) Add expression description for CASE WHEN

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22086: Assignee: Apache Spark > Add expression description for CASE WHEN > --

[jira] [Commented] (SPARK-22086) Add expression description for CASE WHEN

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174512#comment-16174512 ] Apache Spark commented on SPARK-22086: -- User 'viirya' has created a pull request for

[jira] [Commented] (SPARK-21653) Complement SQL expression document

2017-09-21 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174514#comment-16174514 ] Liang-Chi Hsieh commented on SPARK-21653: - I just go through all SQL expressions.

[jira] [Comment Edited] (SPARK-21653) Complement SQL expression document

2017-09-21 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174514#comment-16174514 ] Liang-Chi Hsieh edited comment on SPARK-21653 at 9/21/17 9:55 AM: -

[jira] [Created] (SPARK-22087) Clear remaining compile errors for 2.12; resolve most warnings

2017-09-21 Thread Sean Owen (JIRA)
Sean Owen created SPARK-22087: - Summary: Clear remaining compile errors for 2.12; resolve most warnings Key: SPARK-22087 URL: https://issues.apache.org/jira/browse/SPARK-22087 Project: Spark Iss

[jira] [Created] (SPARK-22088) Incorrect scalastyle comment causes wrong styles in stringExpressions

2017-09-21 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-22088: --- Summary: Incorrect scalastyle comment causes wrong styles in stringExpressions Key: SPARK-22088 URL: https://issues.apache.org/jira/browse/SPARK-22088 Project:

[jira] [Commented] (SPARK-22088) Incorrect scalastyle comment causes wrong styles in stringExpressions

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174531#comment-16174531 ] Apache Spark commented on SPARK-22088: -- User 'viirya' has created a pull request for

[jira] [Assigned] (SPARK-22088) Incorrect scalastyle comment causes wrong styles in stringExpressions

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22088: Assignee: (was: Apache Spark) > Incorrect scalastyle comment causes wrong styles in st

[jira] [Assigned] (SPARK-22088) Incorrect scalastyle comment causes wrong styles in stringExpressions

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22088: Assignee: Apache Spark > Incorrect scalastyle comment causes wrong styles in stringExpress

[jira] [Updated] (SPARK-22087) Clear remaining compile errors for 2.12; resolve most warnings

2017-09-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22087: -- Description: In the course of finishing SPARK-14650 I found there are still some compile problems and

[jira] [Updated] (SPARK-22061) Add pipeline model of SVM

2017-09-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22061: -- Target Version/s: (was: 2.3.0) Labels: (was: features) Fix Version/s: (was:

[jira] [Commented] (SPARK-21977) SinglePartition optimizations break certain Streaming Stateful Aggregation requirements

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174562#comment-16174562 ] Apache Spark commented on SPARK-21977: -- User 'srowen' has created a pull request for

[jira] [Commented] (SPARK-14650) Compile Spark REPL for Scala 2.12

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174571#comment-16174571 ] Apache Spark commented on SPARK-14650: -- User 'srowen' has created a pull request for

[jira] [Assigned] (SPARK-14650) Compile Spark REPL for Scala 2.12

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14650: Assignee: Apache Spark > Compile Spark REPL for Scala 2.12 > -

[jira] [Commented] (SPARK-22087) Clear remaining compile errors for 2.12; resolve most warnings

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174570#comment-16174570 ] Apache Spark commented on SPARK-22087: -- User 'srowen' has created a pull request for

[jira] [Assigned] (SPARK-22087) Clear remaining compile errors for 2.12; resolve most warnings

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22087: Assignee: Apache Spark (was: Sean Owen) > Clear remaining compile errors for 2.12; resolv

[jira] [Assigned] (SPARK-22087) Clear remaining compile errors for 2.12; resolve most warnings

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22087: Assignee: Sean Owen (was: Apache Spark) > Clear remaining compile errors for 2.12; resolv

[jira] [Assigned] (SPARK-14650) Compile Spark REPL for Scala 2.12

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14650: Assignee: (was: Apache Spark) > Compile Spark REPL for Scala 2.12 > --

[jira] [Resolved] (SPARK-21780) Simpler Dataset.sample API in R

2017-09-21 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21780. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19243 [https://git

[jira] [Assigned] (SPARK-21780) Simpler Dataset.sample API in R

2017-09-21 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-21780: Assignee: Hyukjin Kwon > Simpler Dataset.sample API in R > ---

[jira] [Resolved] (SPARK-21777) Simpler Dataset.sample API

2017-09-21 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21777. -- Resolution: Done > Simpler Dataset.sample API > -- > >

[jira] [Updated] (SPARK-21777) Simpler Dataset.sample API

2017-09-21 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-21777: - Fix Version/s: 2.3.0 > Simpler Dataset.sample API > -- > >

[jira] [Commented] (SPARK-21418) NoSuchElementException: None.get in DataSourceScanExec with sun.io.serialization.extendedDebugInfo=true

2017-09-21 Thread Artem Kupchinskiy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174642#comment-16174642 ] Artem Kupchinskiy commented on SPARK-21418: --- There is still a place in FileSour

[jira] [Commented] (SPARK-18727) Support schema evolution as new files are inserted into table

2017-09-21 Thread Serge Smertin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174660#comment-16174660 ] Serge Smertin commented on SPARK-18727: --- in one of the use-cases for project in [#c

[jira] [Comment Edited] (SPARK-18727) Support schema evolution as new files are inserted into table

2017-09-21 Thread Serge Smertin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174660#comment-16174660 ] Serge Smertin edited comment on SPARK-18727 at 9/21/17 12:31 PM: --

[jira] [Commented] (SPARK-19141) VectorAssembler metadata causing memory issues

2017-09-21 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174685#comment-16174685 ] Weichen Xu commented on SPARK-19141: Maybe we need design a sparse format of Attribut

[jira] [Commented] (SPARK-16845) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB

2017-09-21 Thread Marco Veluscek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174727#comment-16174727 ] Marco Veluscek commented on SPARK-16845: Hello, I have just encountered a simila

[jira] [Commented] (SPARK-22034) CrossValidator's training and testing set with different set of labels, resulting in encoder transform error

2017-09-21 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174729#comment-16174729 ] Weichen Xu commented on SPARK-22034: Do you mean a pipeline including stage VectorInd

[jira] [Created] (SPARK-22089) There is no need for fileStatusCache to invalidateAll when InMemoryFileIndex refresh

2017-09-21 Thread guichaoxian (JIRA)
guichaoxian created SPARK-22089: --- Summary: There is no need for fileStatusCache to invalidateAll when InMemoryFileIndex refresh Key: SPARK-22089 URL: https://issues.apache.org/jira/browse/SPARK-22089

[jira] [Created] (SPARK-22090) There is no need for fileStatusCache to invalidateAll when InMemoryFileIndex refresh

2017-09-21 Thread guichaoxian (JIRA)
guichaoxian created SPARK-22090: --- Summary: There is no need for fileStatusCache to invalidateAll when InMemoryFileIndex refresh Key: SPARK-22090 URL: https://issues.apache.org/jira/browse/SPARK-22090

[jira] [Created] (SPARK-22091) There is no need for fileStatusCache to invalidateAll when InMemoryFileIndex refresh

2017-09-21 Thread guichaoxian (JIRA)
guichaoxian created SPARK-22091: --- Summary: There is no need for fileStatusCache to invalidateAll when InMemoryFileIndex refresh Key: SPARK-22091 URL: https://issues.apache.org/jira/browse/SPARK-22091

[jira] [Resolved] (SPARK-22090) There is no need for fileStatusCache to invalidateAll when InMemoryFileIndex refresh

2017-09-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22090. --- Resolution: Duplicate > There is no need for fileStatusCache to invalidateAll when InMemoryFileIndex

[jira] [Reopened] (SPARK-17997) Aggregation function for counting distinct values for multiple intervals

2017-09-21 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-17997: - Assignee: Zhenhua Wang > Aggregation function for counting distinct values for multiple interva

[jira] [Resolved] (SPARK-17997) Aggregation function for counting distinct values for multiple intervals

2017-09-21 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-17997. - Resolution: Fixed > Aggregation function for counting distinct values for multiple intervals > --

[jira] [Assigned] (SPARK-22086) Add expression description for CASE WHEN

2017-09-21 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-22086: Assignee: Liang-Chi Hsieh > Add expression description for CASE WHEN > ---

[jira] [Resolved] (SPARK-22086) Add expression description for CASE WHEN

2017-09-21 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22086. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19304 [https://git

[jira] [Resolved] (SPARK-21653) Complement SQL expression document

2017-09-21 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh resolved SPARK-21653. - Resolution: Fixed > Complement SQL expression document >

[jira] [Closed] (SPARK-22090) There is no need for fileStatusCache to invalidateAll when InMemoryFileIndex refresh

2017-09-21 Thread guichaoxian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guichaoxian closed SPARK-22090. --- > There is no need for fileStatusCache to invalidateAll when InMemoryFileIndex > refresh > ---

[jira] [Closed] (SPARK-22091) There is no need for fileStatusCache to invalidateAll when InMemoryFileIndex refresh

2017-09-21 Thread guichaoxian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guichaoxian closed SPARK-22091. --- Resolution: Duplicate > There is no need for fileStatusCache to invalidateAll when InMemoryFileIndex

[jira] [Commented] (SPARK-21928) ML LogisticRegression training occasionally produces java.lang.ClassNotFoundException when attempting to load custom Kryo registrator class

2017-09-21 Thread John Brock (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175066#comment-16175066 ] John Brock commented on SPARK-21928: It does! I see this in the log right before an e

[jira] [Created] (SPARK-22092) Reallocation in OffHeapColumnVector.reserveInternal corrupts array data

2017-09-21 Thread Ala Luszczak (JIRA)
Ala Luszczak created SPARK-22092: Summary: Reallocation in OffHeapColumnVector.reserveInternal corrupts array data Key: SPARK-22092 URL: https://issues.apache.org/jira/browse/SPARK-22092 Project: Spar

[jira] [Assigned] (SPARK-22092) Reallocation in OffHeapColumnVector.reserveInternal corrupts array data

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22092: Assignee: (was: Apache Spark) > Reallocation in OffHeapColumnVector.reserveInternal co

[jira] [Assigned] (SPARK-22092) Reallocation in OffHeapColumnVector.reserveInternal corrupts array data

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22092: Assignee: Apache Spark > Reallocation in OffHeapColumnVector.reserveInternal corrupts arra

[jira] [Commented] (SPARK-22092) Reallocation in OffHeapColumnVector.reserveInternal corrupts array data

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175087#comment-16175087 ] Apache Spark commented on SPARK-22092: -- User 'ala' has created a pull request for th

[jira] [Updated] (SPARK-22083) When dropping multiple blocks to disk, Spark should release all locks on a failure

2017-09-21 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-22083: - Description: {{MemoryStore.evictBlocksToFreeSpace}} first [acquires writer locks on all the bloc

[jira] [Created] (SPARK-22093) UtilsSuite "resolveURIs with multiple paths" test always cancelled

2017-09-21 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-22093: -- Summary: UtilsSuite "resolveURIs with multiple paths" test always cancelled Key: SPARK-22093 URL: https://issues.apache.org/jira/browse/SPARK-22093 Project: Spark

[jira] [Resolved] (SPARK-22061) Add pipeline model of SVM

2017-09-21 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-22061. Resolution: Won't Fix > Add pipeline model of SVM > - > >

[jira] [Commented] (SPARK-22061) Add pipeline model of SVM

2017-09-21 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175124#comment-16175124 ] Nick Pentreath commented on SPARK-22061: Agreed, this already exists. I closed th

[jira] [Resolved] (SPARK-21928) ML LogisticRegression training occasionally produces java.lang.ClassNotFoundException when attempting to load custom Kryo registrator class

2017-09-21 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-21928. Resolution: Fixed Assignee: Imran Rashid Fix Version/s: 2.3.0

[jira] [Commented] (SPARK-21866) SPIP: Image support in Spark

2017-09-21 Thread Timothy Hunter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175158#comment-16175158 ] Timothy Hunter commented on SPARK-21866: Putting this code under {{org.apache.spa

[jira] [Assigned] (SPARK-19558) Provide a config option to attach QueryExecutionListener to SparkSession

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19558: Assignee: Apache Spark > Provide a config option to attach QueryExecutionListener to Spark

[jira] [Commented] (SPARK-19558) Provide a config option to attach QueryExecutionListener to SparkSession

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175173#comment-16175173 ] Apache Spark commented on SPARK-19558: -- User 'vanzin' has created a pull request for

[jira] [Assigned] (SPARK-19558) Provide a config option to attach QueryExecutionListener to SparkSession

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19558: Assignee: (was: Apache Spark) > Provide a config option to attach QueryExecutionListen

[jira] [Assigned] (SPARK-18136) Make PySpark pip install works on windows

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18136: Assignee: Apache Spark > Make PySpark pip install works on windows > -

[jira] [Assigned] (SPARK-18136) Make PySpark pip install works on windows

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18136: Assignee: (was: Apache Spark) > Make PySpark pip install works on windows > --

[jira] [Commented] (SPARK-18136) Make PySpark pip install works on windows

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175258#comment-16175258 ] Apache Spark commented on SPARK-18136: -- User 'jsnowacki' has created a pull request

[jira] [Commented] (SPARK-18136) Make PySpark pip install works on windows

2017-09-21 Thread Jakub Nowacki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175276#comment-16175276 ] Jakub Nowacki commented on SPARK-18136: --- [PR|https://github.com/apache/spark/pull/1

[jira] [Resolved] (SPARK-22088) Incorrect scalastyle comment causes wrong styles in stringExpressions

2017-09-21 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-22088. - Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.3.0 > Incorrect scalastyle c

[jira] [Resolved] (SPARK-22075) GBTs forgot to unpersist datasets cached by Checkpointer

2017-09-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22075. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19288 [https://github.co

[jira] [Assigned] (SPARK-22075) GBTs forgot to unpersist datasets cached by Checkpointer

2017-09-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-22075: - Assignee: zhengruifeng Priority: Minor (was: Major) Issue Type: Improvement (was: B

[jira] [Commented] (SPARK-18136) Make PySpark pip install works on windows

2017-09-21 Thread Jakub Nowacki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175303#comment-16175303 ] Jakub Nowacki commented on SPARK-18136: --- I've tried using Windows command {{mklink}

[jira] [Assigned] (SPARK-22009) Using treeAggregate improve some algs

2017-09-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-22009: - Assignee: zhengruifeng > Using treeAggregate improve some algs > ---

[jira] [Resolved] (SPARK-22009) Using treeAggregate improve some algs

2017-09-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22009. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19232 [https://github.co

[jira] [Assigned] (SPARK-22083) When dropping multiple blocks to disk, Spark should release all locks on a failure

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22083: Assignee: (was: Apache Spark) > When dropping multiple blocks to disk, Spark should re

[jira] [Assigned] (SPARK-22083) When dropping multiple blocks to disk, Spark should release all locks on a failure

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22083: Assignee: Apache Spark > When dropping multiple blocks to disk, Spark should release all l

[jira] [Commented] (SPARK-22083) When dropping multiple blocks to disk, Spark should release all locks on a failure

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175309#comment-16175309 ] Apache Spark commented on SPARK-22083: -- User 'squito' has created a pull request for

[jira] [Assigned] (SPARK-22072) Allow the same shell params to be used for all of the different steps in release-build

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22072: Assignee: Apache Spark (was: holdenk) > Allow the same shell params to be used for all of

[jira] [Assigned] (SPARK-22071) Improve release build scripts to check correct JAVA version is being used for build

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22071: Assignee: holdenk (was: Apache Spark) > Improve release build scripts to check correct JA

[jira] [Assigned] (SPARK-22072) Allow the same shell params to be used for all of the different steps in release-build

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22072: Assignee: holdenk (was: Apache Spark) > Allow the same shell params to be used for all of

[jira] [Commented] (SPARK-22071) Improve release build scripts to check correct JAVA version is being used for build

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175354#comment-16175354 ] Apache Spark commented on SPARK-22071: -- User 'holdenk' has created a pull request fo

[jira] [Commented] (SPARK-22072) Allow the same shell params to be used for all of the different steps in release-build

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175352#comment-16175352 ] Apache Spark commented on SPARK-22072: -- User 'holdenk' has created a pull request fo

[jira] [Assigned] (SPARK-22071) Improve release build scripts to check correct JAVA version is being used for build

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22071: Assignee: Apache Spark (was: holdenk) > Improve release build scripts to check correct JA

[jira] [Commented] (SPARK-21928) ML LogisticRegression training occasionally produces java.lang.ClassNotFoundException when attempting to load custom Kryo registrator class

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175385#comment-16175385 ] Apache Spark commented on SPARK-21928: -- User 'squito' has created a pull request for

[jira] [Updated] (SPARK-21928) ML LogisticRegression training occasionally produces java.lang.ClassNotFoundException when attempting to load custom Kryo registrator class

2017-09-21 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-21928: - Description: >From SPARK-13990 & SPARK-13926, Spark's SerializerManager has its own instance >of

[jira] [Updated] (SPARK-21928) ClassNotFoundException for custom Kryo registrator class during serde in netty threads

2017-09-21 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-21928: - Summary: ClassNotFoundException for custom Kryo registrator class during serde in netty threads

[jira] [Commented] (SPARK-21928) ClassNotFoundException for custom Kryo registrator class during serde in netty threads

2017-09-21 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175410#comment-16175410 ] Imran Rashid commented on SPARK-21928: -- thanks [~jbrock], thats great. I think this

[jira] [Updated] (SPARK-22083) When dropping multiple blocks to disk, Spark should release all locks on a failure

2017-09-21 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-22083: - Description: {{MemoryStore.evictBlocksToFreeSpace}} first [acquires writer locks on all the bloc

[jira] [Commented] (SPARK-21928) ClassNotFoundException for custom Kryo registrator class during serde in netty threads

2017-09-21 Thread John Brock (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175452#comment-16175452 ] John Brock commented on SPARK-21928: Excellent, thanks for looking into this. > Clas

[jira] [Commented] (SPARK-22077) RpcEndpointAddress fails to parse spark URL if it is an ipv6 address.

2017-09-21 Thread Eric Vandenberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175479#comment-16175479 ] Eric Vandenberg commented on SPARK-22077: - Yes, it worked when I overloaded with

[jira] [Commented] (SPARK-14236) UDAF does not use incomingSchema for update Method

2017-09-21 Thread Guilherme Braccialli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175541#comment-16175541 ] Guilherme Braccialli commented on SPARK-14236: -- +1 to implement this. as a

[jira] [Comment Edited] (SPARK-14236) UDAF does not use incomingSchema for update Method

2017-09-21 Thread Guilherme Braccialli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175541#comment-16175541 ] Guilherme Braccialli edited comment on SPARK-14236 at 9/21/17 10:29 PM: ---

[jira] [Updated] (SPARK-22060) CrossValidator/TrainValidationSplit parallelism param persist/load bug

2017-09-21 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-22060: -- Shepherd: Joseph K. Bradley > CrossValidator/TrainValidationSplit parallelism param per

[jira] [Comment Edited] (SPARK-14236) UDAF does not use incomingSchema for update Method

2017-09-21 Thread Guilherme Braccialli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175541#comment-16175541 ] Guilherme Braccialli edited comment on SPARK-14236 at 9/21/17 10:29 PM: ---

[jira] [Updated] (SPARK-22060) CrossValidator/TrainValidationSplit parallelism param persist/load bug

2017-09-21 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-22060: -- Target Version/s: 2.3.0 > CrossValidator/TrainValidationSplit parallelism param persist

[jira] [Assigned] (SPARK-22060) CrossValidator/TrainValidationSplit parallelism param persist/load bug

2017-09-21 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-22060: - Assignee: Weichen Xu > CrossValidator/TrainValidationSplit parallelism param per

[jira] [Resolved] (SPARK-22053) Implement stream-stream inner join in Append mode

2017-09-21 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-22053. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 19271 [https://g

[jira] [Updated] (SPARK-21928) ClassNotFoundException for custom Kryo registrator class during serde in netty threads

2017-09-21 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-21928: --- Fix Version/s: 2.1.2 > ClassNotFoundException for custom Kryo registrator class during serde

[jira] [Commented] (SPARK-19357) Parallel Model Evaluation for ML Tuning: Scala

2017-09-21 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175629#comment-16175629 ] Joseph K. Bradley commented on SPARK-19357: --- [~bryanc], [~nick.pentre...@gmail.

[jira] [Comment Edited] (SPARK-19357) Parallel Model Evaluation for ML Tuning: Scala

2017-09-21 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175629#comment-16175629 ] Joseph K. Bradley edited comment on SPARK-19357 at 9/21/17 11:14 PM: --

[jira] [Created] (SPARK-22094) processAllAvailable should not block forever when a query is stopped

2017-09-21 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-22094: Summary: processAllAvailable should not block forever when a query is stopped Key: SPARK-22094 URL: https://issues.apache.org/jira/browse/SPARK-22094 Project: Spark

[jira] [Assigned] (SPARK-22094) processAllAvailable should not block forever when a query is stopped

2017-09-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22094: Assignee: Shixiong Zhu (was: Apache Spark) > processAllAvailable should not block forever

  1   2   >