[jira] [Comment Edited] (SPARK-23716) Change SHA512 style in release artifacts to play nicely with shasum utility

[ https://issues.apache.org/jira/browse/SPARK-23716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412423#comment-16412423 ] Nicholas Chammas edited comment on SPARK-23716 at 3/24/18 5:13 AM: --- For

[jira] [Resolved] (SPARK-23716) Change SHA512 style in release artifacts to play nicely with shasum utility

[ https://issues.apache.org/jira/browse/SPARK-23716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas resolved SPARK-23716. -- Resolution: Won't Fix For my use case, there is no value in updating the Spark release

[jira] [Comment Edited] (SPARK-22876) spark.yarn.am.attemptFailuresValidityInterval does not work correctly

[ https://issues.apache.org/jira/browse/SPARK-22876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412379#comment-16412379 ] MIN-FU YANG edited comment on SPARK-22876 at 3/24/18 3:00 AM: -- I found that

[jira] [Commented] (SPARK-22876) spark.yarn.am.attemptFailuresValidityInterval does not work correctly

[ https://issues.apache.org/jira/browse/SPARK-22876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412379#comment-16412379 ] MIN-FU YANG commented on SPARK-22876: - I found that current Yarn implementation doesn't expose number

[jira] [Commented] (SPARK-14352) approxQuantile should support multi columns

[ https://issues.apache.org/jira/browse/SPARK-14352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412370#comment-16412370 ] Walt Elder commented on SPARK-14352: Seems like this should be marked closed as of 2.2, right? >

[jira] [Commented] (SPARK-23785) LauncherBackend doesn't check state of connection before setting state

[ https://issues.apache.org/jira/browse/SPARK-23785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412361#comment-16412361 ] Sahil Takiar commented on SPARK-23785: -- Updated the PR. {quote} in

[jira] [Commented] (SPARK-23788) Race condition in StreamingQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-23788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412360#comment-16412360 ] Apache Spark commented on SPARK-23788: -- User 'jose-torres' has created a pull request for this

[jira] [Assigned] (SPARK-23788) Race condition in StreamingQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-23788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23788: Assignee: Apache Spark > Race condition in StreamingQuerySuite >

[jira] [Assigned] (SPARK-23788) Race condition in StreamingQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-23788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23788: Assignee: (was: Apache Spark) > Race condition in StreamingQuerySuite >

[jira] [Created] (SPARK-23788) Race condition in StreamingQuerySuite

Jose Torres created SPARK-23788: --- Summary: Race condition in StreamingQuerySuite Key: SPARK-23788 URL: https://issues.apache.org/jira/browse/SPARK-23788 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-22876) spark.yarn.am.attemptFailuresValidityInterval does not work correctly

[ https://issues.apache.org/jira/browse/SPARK-22876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412228#comment-16412228 ] MIN-FU YANG commented on SPARK-22876: - Hi, I also encounter this problem. Please assign it to me, I

[jira] [Resolved] (SPARK-23615) Add maxDF Parameter to Python CountVectorizer

[ https://issues.apache.org/jira/browse/SPARK-23615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-23615. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20777

[jira] [Assigned] (SPARK-23615) Add maxDF Parameter to Python CountVectorizer

[ https://issues.apache.org/jira/browse/SPARK-23615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-23615: Assignee: Huaxin Gao > Add maxDF Parameter to Python CountVectorizer >

[jira] [Commented] (SPARK-22513) Provide build profile for hadoop 2.8

[ https://issues.apache.org/jira/browse/SPARK-22513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412218#comment-16412218 ] Nicholas Chammas commented on SPARK-22513: -- Fair enough. Just as an alternate confirmation,

[jira] [Assigned] (SPARK-23787) SparkSubmitSuite::"download remote resource if it is not supported by yarn" fails on Hadoop 2.9

[ https://issues.apache.org/jira/browse/SPARK-23787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23787: Assignee: Apache Spark > SparkSubmitSuite::"download remote resource if it is not

[jira] [Commented] (SPARK-23787) SparkSubmitSuite::"download remote resource if it is not supported by yarn" fails on Hadoop 2.9

[ https://issues.apache.org/jira/browse/SPARK-23787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412202#comment-16412202 ] Apache Spark commented on SPARK-23787: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23787) SparkSubmitSuite::"download remote resource if it is not supported by yarn" fails on Hadoop 2.9

[ https://issues.apache.org/jira/browse/SPARK-23787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23787: Assignee: (was: Apache Spark) > SparkSubmitSuite::"download remote resource if it is

[jira] [Updated] (SPARK-23787) SparkSubmitSuite::"download remote resource if it is not supported by yarn" fails on Hadoop 2.9

[ https://issues.apache.org/jira/browse/SPARK-23787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-23787: --- Summary: SparkSubmitSuite::"download remote resource if it is not supported by yarn" fails

[jira] [Created] (SPARK-23787) SparkSubmitSuite::"" fails on Hadoop 2.9

Marcelo Vanzin created SPARK-23787: -- Summary: SparkSubmitSuite::"" fails on Hadoop 2.9 Key: SPARK-23787 URL: https://issues.apache.org/jira/browse/SPARK-23787 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-23787) SparkSubmitSuite::"download list of files to local" fails on Hadoop 2.9

[ https://issues.apache.org/jira/browse/SPARK-23787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-23787: --- Summary: SparkSubmitSuite::"download list of files to local" fails on Hadoop 2.9 (was:

[jira] [Commented] (SPARK-23785) LauncherBackend doesn't check state of connection before setting state

[ https://issues.apache.org/jira/browse/SPARK-23785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412130#comment-16412130 ] Marcelo Vanzin commented on SPARK-23785: This is a little trickier than just the checks you have

[jira] [Assigned] (SPARK-23786) CSV schema validation - column names are not checked

[ https://issues.apache.org/jira/browse/SPARK-23786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23786: Assignee: Apache Spark > CSV schema validation - column names are not checked >

[jira] [Commented] (SPARK-23786) CSV schema validation - column names are not checked

[ https://issues.apache.org/jira/browse/SPARK-23786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412088#comment-16412088 ] Apache Spark commented on SPARK-23786: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23786) CSV schema validation - column names are not checked

[ https://issues.apache.org/jira/browse/SPARK-23786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23786: Assignee: (was: Apache Spark) > CSV schema validation - column names are not checked

[jira] [Created] (SPARK-23786) CSV schema validation - column names are not checked

Maxim Gekk created SPARK-23786: -- Summary: CSV schema validation - column names are not checked Key: SPARK-23786 URL: https://issues.apache.org/jira/browse/SPARK-23786 Project: Spark Issue Type:

[jira] [Commented] (SPARK-23785) LauncherBackend doesn't check state of connection before setting state

[ https://issues.apache.org/jira/browse/SPARK-23785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412042#comment-16412042 ] Sahil Takiar commented on SPARK-23785: -- [~vanzin] opened a PR that just checks if {{isConnected}} is

[jira] [Assigned] (SPARK-23785) LauncherBackend doesn't check state of connection before setting state

[ https://issues.apache.org/jira/browse/SPARK-23785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23785: Assignee: Apache Spark > LauncherBackend doesn't check state of connection before setting

[jira] [Commented] (SPARK-23785) LauncherBackend doesn't check state of connection before setting state

[ https://issues.apache.org/jira/browse/SPARK-23785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412004#comment-16412004 ] Apache Spark commented on SPARK-23785: -- User 'sahilTakiar' has created a pull request for this

[jira] [Commented] (SPARK-23772) Provide an option to ignore column of all null values or empty map/array during JSON schema inference

[ https://issues.apache.org/jira/browse/SPARK-23772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412007#comment-16412007 ] Reynold Xin commented on SPARK-23772: - This is a good change to do!   > Provide an option to ignore

[jira] [Updated] (SPARK-23654) Cut jets3t as a dependency of spark-core; exclude it from hadoop-cloud module as incompatible

[ https://issues.apache.org/jira/browse/SPARK-23654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-23654: --- Summary: Cut jets3t as a dependency of spark-core; exclude it from hadoop-cloud module as

[jira] [Assigned] (SPARK-23785) LauncherBackend doesn't check state of connection before setting state

[ https://issues.apache.org/jira/browse/SPARK-23785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23785: Assignee: (was: Apache Spark) > LauncherBackend doesn't check state of connection

[jira] [Created] (SPARK-23785) LauncherBackend doesn't check state of connection before setting state

Sahil Takiar created SPARK-23785: Summary: LauncherBackend doesn't check state of connection before setting state Key: SPARK-23785 URL: https://issues.apache.org/jira/browse/SPARK-23785 Project:

[jira] [Created] (SPARK-23784) Cannot use custom Aggregator with groupBy/agg

Joshua Howard created SPARK-23784: - Summary: Cannot use custom Aggregator with groupBy/agg Key: SPARK-23784 URL: https://issues.apache.org/jira/browse/SPARK-23784 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-23783) Add new generic export trait for ML pipelines

[ https://issues.apache.org/jira/browse/SPARK-23783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk resolved SPARK-23783. - Resolution: Fixed Fix Version/s: 2.4.0 > Add new generic export trait for ML pipelines >

[jira] [Assigned] (SPARK-23783) Add new generic export trait for ML pipelines

[ https://issues.apache.org/jira/browse/SPARK-23783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk reassigned SPARK-23783: --- Assignee: holdenk > Add new generic export trait for ML pipelines >

[jira] [Assigned] (SPARK-11239) PMML export for ML linear regression

[ https://issues.apache.org/jira/browse/SPARK-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk reassigned SPARK-11239: --- Assignee: holdenk > PMML export for ML linear regression > > >

[jira] [Resolved] (SPARK-11239) PMML export for ML linear regression

[ https://issues.apache.org/jira/browse/SPARK-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk resolved SPARK-11239. - Resolution: Fixed Fix Version/s: 2.4.0 > PMML export for ML linear regression >

[jira] [Commented] (SPARK-21834) Incorrect executor request in case of dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-21834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411897#comment-16411897 ] Imran Rashid commented on SPARK-21834: -- SPARK-23365 is basically a duplicate of this, though they

[jira] [Commented] (SPARK-23365) DynamicAllocation with failure in straggler task can lead to a hung spark job

[ https://issues.apache.org/jira/browse/SPARK-23365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411894#comment-16411894 ] Imran Rashid commented on SPARK-23365: -- This is mostly a duplicate of SPARK-21834, though I'm not

[jira] [Assigned] (SPARK-23783) Add new generic export trait for ML pipelines

[ https://issues.apache.org/jira/browse/SPARK-23783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23783: Assignee: Apache Spark > Add new generic export trait for ML pipelines >

[jira] [Updated] (SPARK-23365) DynamicAllocation with failure in straggler task can lead to a hung spark job

[ https://issues.apache.org/jira/browse/SPARK-23365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-23365: - Description: Dynamic Allocation can lead to a spark app getting stuck with 0 executors

[jira] [Assigned] (SPARK-23783) Add new generic export trait for ML pipelines

[ https://issues.apache.org/jira/browse/SPARK-23783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23783: Assignee: (was: Apache Spark) > Add new generic export trait for ML pipelines >

[jira] [Commented] (SPARK-23783) Add new generic export trait for ML pipelines

[ https://issues.apache.org/jira/browse/SPARK-23783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411891#comment-16411891 ] Apache Spark commented on SPARK-23783: -- User 'holdenk' has created a pull request for this issue:

[jira] [Created] (SPARK-23783) Add new generic export trait for ML pipelines

holdenk created SPARK-23783: --- Summary: Add new generic export trait for ML pipelines Key: SPARK-23783 URL: https://issues.apache.org/jira/browse/SPARK-23783 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-21685) Params isSet in scala Transformer triggered by _setDefault in pyspark

[ https://issues.apache.org/jira/browse/SPARK-21685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk reassigned SPARK-21685: --- Assignee: Bryan Cutler > Params isSet in scala Transformer triggered by _setDefault in pyspark >

[jira] [Resolved] (SPARK-21685) Params isSet in scala Transformer triggered by _setDefault in pyspark

[ https://issues.apache.org/jira/browse/SPARK-21685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk resolved SPARK-21685. - Resolution: Fixed Fix Version/s: 2.4.0 > Params isSet in scala Transformer triggered by

[jira] [Commented] (SPARK-23700) Cleanup unused imports

[ https://issues.apache.org/jira/browse/SPARK-23700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411860#comment-16411860 ] Apache Spark commented on SPARK-23700: -- User 'BryanCutler' has created a pull request for this

[jira] [Assigned] (SPARK-23700) Cleanup unused imports

[ https://issues.apache.org/jira/browse/SPARK-23700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23700: Assignee: Apache Spark > Cleanup unused imports > -- > >

[jira] [Assigned] (SPARK-23700) Cleanup unused imports

[ https://issues.apache.org/jira/browse/SPARK-23700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23700: Assignee: (was: Apache Spark) > Cleanup unused imports > -- > >

[jira] [Commented] (SPARK-23776) pyspark-sql tests should display build instructions when components are missing

[ https://issues.apache.org/jira/browse/SPARK-23776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411854#comment-16411854 ] Bruce Robbins commented on SPARK-23776: --- As it turns out, the building-spark page does have maven

[jira] [Commented] (SPARK-23782) SHS should not show applications to user without read permission

[ https://issues.apache.org/jira/browse/SPARK-23782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411851#comment-16411851 ] Marcelo Vanzin commented on SPARK-23782: More discussion at:

[jira] [Comment Edited] (SPARK-23782) SHS should not show applications to user without read permission

[ https://issues.apache.org/jira/browse/SPARK-23782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411843#comment-16411843 ] Marcelo Vanzin edited comment on SPARK-23782 at 3/23/18 6:23 PM: - bq.

[jira] [Commented] (SPARK-23782) SHS should not show applications to user without read permission

[ https://issues.apache.org/jira/browse/SPARK-23782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411843#comment-16411843 ] Marcelo Vanzin commented on SPARK-23782: bq. This seems a security hole to me What sensitive

[jira] [Commented] (SPARK-23782) SHS should not show applications to user without read permission

[ https://issues.apache.org/jira/browse/SPARK-23782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411836#comment-16411836 ] Marco Gaido commented on SPARK-23782: - [~vanzin] sorry but I have not been able to find any JIRA

[jira] [Assigned] (SPARK-23782) SHS should not show applications to user without read permission

[ https://issues.apache.org/jira/browse/SPARK-23782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23782: Assignee: Apache Spark > SHS should not show applications to user without read permission

[jira] [Commented] (SPARK-23782) SHS should not show applications to user without read permission

[ https://issues.apache.org/jira/browse/SPARK-23782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411824#comment-16411824 ] Apache Spark commented on SPARK-23782: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23782) SHS should not show applications to user without read permission

[ https://issues.apache.org/jira/browse/SPARK-23782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23782: Assignee: (was: Apache Spark) > SHS should not show applications to user without read

[jira] [Commented] (SPARK-23782) SHS should not show applications to user without read permission

[ https://issues.apache.org/jira/browse/SPARK-23782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411811#comment-16411811 ] Marcelo Vanzin commented on SPARK-23782: I'm pretty sure this was discussed before and the

[jira] [Created] (SPARK-23782) SHS should not show applications to user without read permission

Marco Gaido created SPARK-23782: --- Summary: SHS should not show applications to user without read permission Key: SPARK-23782 URL: https://issues.apache.org/jira/browse/SPARK-23782 Project: Spark

[jira] [Commented] (SPARK-22342) refactor schedulerDriver registration

[ https://issues.apache.org/jira/browse/SPARK-22342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411780#comment-16411780 ] Susan X. Huynh commented on SPARK-22342: The multiple re-registration issue can lead to

[jira] [Assigned] (SPARK-23759) Unable to bind Spark UI to specific host name / IP

[ https://issues.apache.org/jira/browse/SPARK-23759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23759: -- Assignee: Felix > Unable to bind Spark UI to specific host name / IP >

[jira] [Resolved] (SPARK-23759) Unable to bind Spark UI to specific host name / IP

[ https://issues.apache.org/jira/browse/SPARK-23759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23759. Resolution: Fixed Fix Version/s: 2.3.1 2.4.0

[jira] [Created] (SPARK-23781) Merge YARN and Mesos token renewal code

Marcelo Vanzin created SPARK-23781: -- Summary: Merge YARN and Mesos token renewal code Key: SPARK-23781 URL: https://issues.apache.org/jira/browse/SPARK-23781 Project: Spark Issue Type:

[jira] [Updated] (SPARK-23365) DynamicAllocation with failure in straggler task can lead to a hung spark job

[ https://issues.apache.org/jira/browse/SPARK-23365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-23365: - Description: Dynamic Allocation can lead to a spark app getting stuck with 0 executors

[jira] [Created] (SPARK-23780) Failed to use googleVis library with new SparkR

Ivan Dzikovsky created SPARK-23780: -- Summary: Failed to use googleVis library with new SparkR Key: SPARK-23780 URL: https://issues.apache.org/jira/browse/SPARK-23780 Project: Spark Issue

[jira] [Commented] (SPARK-23739) Spark structured streaming long running problem

[ https://issues.apache.org/jira/browse/SPARK-23739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411503#comment-16411503 ] Cody Koeninger commented on SPARK-23739: I meant the version of the org.apache.kafka

[jira] [Commented] (SPARK-23655) Add support for type aclitem (PostgresDialect)

[ https://issues.apache.org/jira/browse/SPARK-23655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411485#comment-16411485 ] Diego da Silva Colombo commented on SPARK-23655: [~maropu] I feel it too, it's look like

[jira] [Commented] (SPARK-22239) User-defined window functions with pandas udf

[ https://issues.apache.org/jira/browse/SPARK-22239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411465#comment-16411465 ] Li Jin commented on SPARK-22239: Yeah unbounded windows are really just "groupby" in this case. I need to

[jira] [Commented] (SPARK-23779) TaskMemoryManager and UnsafeSorter use MemoryBlock

[ https://issues.apache.org/jira/browse/SPARK-23779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411425#comment-16411425 ] Apache Spark commented on SPARK-23779: -- User 'kiszk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23779) TaskMemoryManager and UnsafeSorter use MemoryBlock

[ https://issues.apache.org/jira/browse/SPARK-23779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23779: Assignee: Apache Spark > TaskMemoryManager and UnsafeSorter use MemoryBlock >

[jira] [Assigned] (SPARK-23779) TaskMemoryManager and UnsafeSorter use MemoryBlock

[ https://issues.apache.org/jira/browse/SPARK-23779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23779: Assignee: (was: Apache Spark) > TaskMemoryManager and UnsafeSorter use MemoryBlock >

[jira] [Created] (SPARK-23779) TaskMemoryManager and UnsafeSorter use MemoryBlock

Kazuaki Ishizaki created SPARK-23779: Summary: TaskMemoryManager and UnsafeSorter use MemoryBlock Key: SPARK-23779 URL: https://issues.apache.org/jira/browse/SPARK-23779 Project: Spark

[jira] [Commented] (SPARK-23739) Spark structured streaming long running problem

[ https://issues.apache.org/jira/browse/SPARK-23739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411406#comment-16411406 ] Florencio commented on SPARK-23739: --- Thanks for the information. The kafka version is 

[jira] [Comment Edited] (SPARK-23650) Slow SparkR udf (dapply)

[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411246#comment-16411246 ] Deepansh edited comment on SPARK-23650 at 3/23/18 1:43 PM: --- R environment

[jira] [Commented] (SPARK-23739) Spark structured streaming long running problem

[ https://issues.apache.org/jira/browse/SPARK-23739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411396#comment-16411396 ] Cody Koeninger commented on SPARK-23739: What version of the org.apache.kafka artifact is in the

[jira] [Commented] (SPARK-23739) Spark structured streaming long running problem

[ https://issues.apache.org/jira/browse/SPARK-23739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411377#comment-16411377 ] Marco Gaido commented on SPARK-23739: - [~zsxwing] [~joseph.torres] [~c...@koeninger.org] I am not

[jira] [Updated] (SPARK-23778) SparkContext.emptyRDD confuses SparkContext.union

[ https://issues.apache.org/jira/browse/SPARK-23778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefano Pettini updated SPARK-23778: Attachment: as_it_should_be.png > SparkContext.emptyRDD confuses SparkContext.union >

[jira] [Created] (SPARK-23778) SparkContext.emptyRDD confuses SparkContext.union

Stefano Pettini created SPARK-23778: --- Summary: SparkContext.emptyRDD confuses SparkContext.union Key: SPARK-23778 URL: https://issues.apache.org/jira/browse/SPARK-23778 Project: Spark

[jira] [Updated] (SPARK-23778) SparkContext.emptyRDD confuses SparkContext.union

[ https://issues.apache.org/jira/browse/SPARK-23778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefano Pettini updated SPARK-23778: Attachment: partitioner_lost_and_unneeded_extra_stage.png > SparkContext.emptyRDD confuses

[jira] [Resolved] (SPARK-23769) Remove unnecessary scalastyle check disabling

[ https://issues.apache.org/jira/browse/SPARK-23769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23769. -- Resolution: Fixed Assignee: Riaas Mokiem Fix Version/s: 2.4.0

[jira] [Created] (SPARK-23777) Missing DAG arrows between stages

Stefano Pettini created SPARK-23777: --- Summary: Missing DAG arrows between stages Key: SPARK-23777 URL: https://issues.apache.org/jira/browse/SPARK-23777 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-23777) Missing DAG arrows between stages

[ https://issues.apache.org/jira/browse/SPARK-23777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefano Pettini updated SPARK-23777: Attachment: Screenshot-2018-3-23 RDDTestApp - Details for Job 0.png > Missing DAG arrows

[jira] [Updated] (SPARK-23650) Slow SparkR udf (dapply)

[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepansh updated SPARK-23650: - Attachment: packageReload.txt > Slow SparkR udf (dapply) > > >

[jira] [Commented] (SPARK-23650) Slow SparkR udf (dapply)

[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411246#comment-16411246 ] Deepansh commented on SPARK-23650: -- R environment inside the thread for applying UDF is not getting

[jira] [Commented] (SPARK-23685) Spark Structured Streaming Kafka 0.10 Consumer Can't Handle Non-consecutive Offsets (i.e. Log Compaction)

[ https://issues.apache.org/jira/browse/SPARK-23685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410939#comment-16410939 ] Gabor Somogyi commented on SPARK-23685: --- Jira assignment is not required. You can write a comment

[jira] [Commented] (SPARK-23734) InvalidSchemaException While Saving ALSModel

[ https://issues.apache.org/jira/browse/SPARK-23734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410914#comment-16410914 ] Liang-Chi Hsieh commented on SPARK-23734: - I use the latest master branch and can't reproduce the

[jira] [Resolved] (SPARK-22744) Cannot get the submit hostname of application

[ https://issues.apache.org/jira/browse/SPARK-22744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin resolved SPARK-22744. Resolution: Won't Fix Close it while found a work around way. > Cannot get the submit hostname of

[jira] [Commented] (SPARK-23497) Sparklyr Applications doesn't disconnect spark driver in client mode

[ https://issues.apache.org/jira/browse/SPARK-23497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410858#comment-16410858 ] Felix Cheung commented on SPARK-23497: -- you should probably follow up with sparklyr/rstudio on this.

[jira] [Commented] (SPARK-23650) Slow SparkR udf (dapply)

[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410856#comment-16410856 ] Felix Cheung commented on SPARK-23650: -- can you clarify where you see "R environment inside the

[jira] [Resolved] (SPARK-23361) Driver restart fails if it happens after 7 days from app submission

[ https://issues.apache.org/jira/browse/SPARK-23361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-23361. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20657

[jira] [Assigned] (SPARK-23361) Driver restart fails if it happens after 7 days from app submission

[ https://issues.apache.org/jira/browse/SPARK-23361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-23361: --- Assignee: Marcelo Vanzin > Driver restart fails if it happens after 7 days from app