[jira] [Updated] (SPARK-1359) SGD implementation is not efficient

[ https://issues.apache.org/jira/browse/SPARK-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-1359: --- Remaining Estimate: (was: 168h) Original Estimate: (was: 168h) > SGD

[jira] [Updated] (SPARK-1359) SGD implementation is not efficient

[ https://issues.apache.org/jira/browse/SPARK-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-1359: --- Remaining Estimate: 168h Original Estimate: 168h > SGD implementation is not efficient >

[jira] [Updated] (SPARK-1359) SGD implementation is not efficient

[ https://issues.apache.org/jira/browse/SPARK-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-1359: --- Remaining Estimate: (was: 504h) Original Estimate: (was: 504h) > SGD

[jira] [Updated] (SPARK-1359) SGD implementation is not efficient

[ https://issues.apache.org/jira/browse/SPARK-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-1359: --- Remaining Estimate: 504h Original Estimate: 504h > SGD implementation is not efficient >

[jira] [Commented] (SPARK-23598) WholeStageCodegen can lead to IllegalAccessError calling append for HashAggregateExec

[ https://issues.apache.org/jira/browse/SPARK-23598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414984#comment-16414984 ] Dongjoon Hyun commented on SPARK-23598: --- Thank you, [~hvanhovell] ! > WholeStageCodegen can lead

[jira] [Commented] (SPARK-19552) Upgrade Netty version to 4.1.x final

[ https://issues.apache.org/jira/browse/SPARK-19552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414831#comment-16414831 ] ASF GitHub Bot commented on SPARK-19552: Github user robertdale commented on the issue:

[jira] [Created] (SPARK-23797) SparkSQL performance on small TPCDS tables is very low when compared to Drill or Presto

Tin Vu created SPARK-23797: -- Summary: SparkSQL performance on small TPCDS tables is very low when compared to Drill or Presto Key: SPARK-23797 URL: https://issues.apache.org/jira/browse/SPARK-23797 Project:

[jira] [Commented] (SPARK-22839) Refactor Kubernetes code for configuring driver/executor pods to use consistent and cleaner abstraction

[ https://issues.apache.org/jira/browse/SPARK-22839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414739#comment-16414739 ] Apache Spark commented on SPARK-22839: -- User 'mccheah' has created a pull request for this issue:

[jira] [Commented] (SPARK-22839) Refactor Kubernetes code for configuring driver/executor pods to use consistent and cleaner abstraction

[ https://issues.apache.org/jira/browse/SPARK-22839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414738#comment-16414738 ] Matt Cheah commented on SPARK-22839: Design was proposed and agreed upon in

[jira] [Assigned] (SPARK-23776) pyspark-sql tests should display build instructions when components are missing

[ https://issues.apache.org/jira/browse/SPARK-23776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23776: Assignee: Apache Spark > pyspark-sql tests should display build instructions when

[jira] [Assigned] (SPARK-23776) pyspark-sql tests should display build instructions when components are missing

[ https://issues.apache.org/jira/browse/SPARK-23776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23776: Assignee: (was: Apache Spark) > pyspark-sql tests should display build instructions

[jira] [Commented] (SPARK-23776) pyspark-sql tests should display build instructions when components are missing

[ https://issues.apache.org/jira/browse/SPARK-23776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414718#comment-16414718 ] Apache Spark commented on SPARK-23776: -- User 'bersprockets' has created a pull request for this

[jira] [Resolved] (SPARK-23162) PySpark ML LinearRegressionSummary missing r2adj

[ https://issues.apache.org/jira/browse/SPARK-23162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-23162. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20842

[jira] [Assigned] (SPARK-23162) PySpark ML LinearRegressionSummary missing r2adj

[ https://issues.apache.org/jira/browse/SPARK-23162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-23162: Assignee: kevin yu > PySpark ML LinearRegressionSummary missing r2adj >

[jira] [Resolved] (SPARK-23572) Update security.md to cover new features

[ https://issues.apache.org/jira/browse/SPARK-23572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23572. Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20742

[jira] [Assigned] (SPARK-23572) Update security.md to cover new features

[ https://issues.apache.org/jira/browse/SPARK-23572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23572: -- Assignee: Marcelo Vanzin > Update security.md to cover new features >

[jira] [Updated] (SPARK-23736) Extension of the concat function to support array columns

[ https://issues.apache.org/jira/browse/SPARK-23736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marek Novotny updated SPARK-23736: -- Description: Extend the _concat_ function to also support array columns. Example:

[jira] [Updated] (SPARK-23736) Extension of the concat function to support array columns

[ https://issues.apache.org/jira/browse/SPARK-23736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marek Novotny updated SPARK-23736: -- Description: Extend the _concat_ function to also support array columns. Example:

[jira] [Assigned] (SPARK-23672) Document Support returning lists in Arrow UDFs

[ https://issues.apache.org/jira/browse/SPARK-23672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23672: Assignee: (was: Apache Spark) > Document Support returning lists in Arrow UDFs >

[jira] [Assigned] (SPARK-23672) Document Support returning lists in Arrow UDFs

[ https://issues.apache.org/jira/browse/SPARK-23672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23672: Assignee: Apache Spark > Document Support returning lists in Arrow UDFs >

[jira] [Commented] (SPARK-23672) Document Support returning lists in Arrow UDFs

[ https://issues.apache.org/jira/browse/SPARK-23672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414348#comment-16414348 ] Apache Spark commented on SPARK-23672: -- User 'holdenk' has created a pull request for this issue:

[jira] [Commented] (SPARK-11237) PMML export for ML KMeans

[ https://issues.apache.org/jira/browse/SPARK-11237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414344#comment-16414344 ] Apache Spark commented on SPARK-11237: -- User 'holdenk' has created a pull request for this issue:

[jira] [Commented] (SPARK-23739) Spark structured streaming long running problem

[ https://issues.apache.org/jira/browse/SPARK-23739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414326#comment-16414326 ] Cody Koeninger commented on SPARK-23739: Ok, the OutOfMemoryError is probably a separate and

[jira] [Updated] (SPARK-23599) The UUID() expression is too non-deterministic

[ https://issues.apache.org/jira/browse/SPARK-23599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-23599: -- Fix Version/s: 2.3.1 > The UUID() expression is too non-deterministic >

[jira] [Comment Edited] (SPARK-23790) proxy-user failed connecting to a kerberos configured metastore

[ https://issues.apache.org/jira/browse/SPARK-23790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414291#comment-16414291 ] Stavros Kontopoulos edited comment on SPARK-23790 at 3/26/18 6:29 PM:

[jira] [Comment Edited] (SPARK-23790) proxy-user failed connecting to a kerberos configured metastore

[ https://issues.apache.org/jira/browse/SPARK-23790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414291#comment-16414291 ] Stavros Kontopoulos edited comment on SPARK-23790 at 3/26/18 6:29 PM:

[jira] [Commented] (SPARK-23790) proxy-user failed connecting to a kerberos configured metastore

[ https://issues.apache.org/jira/browse/SPARK-23790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414291#comment-16414291 ] Stavros Kontopoulos commented on SPARK-23790: - Yes that is what I am saying. The initial fix

[jira] [Updated] (SPARK-23672) Document Support returning lists in Arrow UDFs

[ https://issues.apache.org/jira/browse/SPARK-23672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-23672: Description: Documenting the support for returning lists for individual inputs on non-grouped data inside

[jira] [Updated] (SPARK-23672) Document Support returning lists in Arrow UDFs

[ https://issues.apache.org/jira/browse/SPARK-23672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-23672: Summary: Document Support returning lists in Arrow UDFs (was: Support returning lists in Arrow UDFs) >

[jira] [Assigned] (SPARK-23561) make StreamWriter not a DataSourceWriter subclass

[ https://issues.apache.org/jira/browse/SPARK-23561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23561: Assignee: (was: Apache Spark) > make StreamWriter not a DataSourceWriter subclass >

[jira] [Assigned] (SPARK-23561) make StreamWriter not a DataSourceWriter subclass

[ https://issues.apache.org/jira/browse/SPARK-23561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23561: Assignee: Apache Spark > make StreamWriter not a DataSourceWriter subclass >

[jira] [Commented] (SPARK-23561) make StreamWriter not a DataSourceWriter subclass

[ https://issues.apache.org/jira/browse/SPARK-23561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414209#comment-16414209 ] Apache Spark commented on SPARK-23561: -- User 'jose-torres' has created a pull request for this

[jira] [Commented] (SPARK-23790) proxy-user failed connecting to a kerberos configured metastore

[ https://issues.apache.org/jira/browse/SPARK-23790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414208#comment-16414208 ] Marcelo Vanzin commented on SPARK-23790: BTW if what you're saying is that Yuming's fix also

[jira] [Commented] (SPARK-22513) Provide build profile for hadoop 2.8

[ https://issues.apache.org/jira/browse/SPARK-22513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414190#comment-16414190 ] Nicholas Chammas commented on SPARK-22513: -- Thanks for the breakdown. This will be handy for

[jira] [Updated] (SPARK-23672) Support returning lists in Arrow UDFs

[ https://issues.apache.org/jira/browse/SPARK-23672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-23672: Description: Consider to add support for returning lists for individual inputs on non-grouped data inside

[jira] [Commented] (SPARK-23790) proxy-user failed connecting to a kerberos configured metastore

[ https://issues.apache.org/jira/browse/SPARK-23790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414176#comment-16414176 ] Marcelo Vanzin commented on SPARK-23790: I haven't had the time to see exactly what spark-cli is

[jira] [Updated] (SPARK-23796) There's no API to change state RDD's name

[ https://issues.apache.org/jira/browse/SPARK-23796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] István Gansperger updated SPARK-23796: -- Description: I use a few {{mapWithState}} stream oparations in my application and at

[jira] [Created] (SPARK-23796) There's no API to change state RDD's name

István Gansperger created SPARK-23796: - Summary: There's no API to change state RDD's name Key: SPARK-23796 URL: https://issues.apache.org/jira/browse/SPARK-23796 Project: Spark Issue

[jira] [Resolved] (SPARK-23795) AbstractLauncher is not extendable

[ https://issues.apache.org/jira/browse/SPARK-23795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23795. Resolution: Not A Problem That class is not meant to be extended by outside libraries. >

[jira] [Commented] (SPARK-23737) Scala API documentation leads to nonexistent pages for sources

[ https://issues.apache.org/jira/browse/SPARK-23737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414111#comment-16414111 ] Alexander Bessonov commented on SPARK-23737: [~sameerag], Making a wild guess the username in

[jira] [Reopened] (SPARK-23737) Scala API documentation leads to nonexistent pages for sources

[ https://issues.apache.org/jira/browse/SPARK-23737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Bessonov reopened SPARK-23737: Okay. The bug isn't fixed and it affects everyone who wants to jump to the source

[jira] [Assigned] (SPARK-23795) AbstractLauncher is not extendable

[ https://issues.apache.org/jira/browse/SPARK-23795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23795: Assignee: (was: Apache Spark) > AbstractLauncher is not extendable >

[jira] [Assigned] (SPARK-23795) AbstractLauncher is not extendable

[ https://issues.apache.org/jira/browse/SPARK-23795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23795: Assignee: Apache Spark > AbstractLauncher is not extendable >

[jira] [Commented] (SPARK-23795) AbstractLauncher is not extendable

[ https://issues.apache.org/jira/browse/SPARK-23795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414092#comment-16414092 ] Apache Spark commented on SPARK-23795: -- User 'dansanduleac' has created a pull request for this

[jira] [Created] (SPARK-23795) AbstractLauncher is not extendable

Dan Sanduleac created SPARK-23795: - Summary: AbstractLauncher is not extendable Key: SPARK-23795 URL: https://issues.apache.org/jira/browse/SPARK-23795 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-23751) Kolmogorov-Smirnoff test Python API in pyspark.ml

[ https://issues.apache.org/jira/browse/SPARK-23751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413704#comment-16413704 ] Apache Spark commented on SPARK-23751: -- User 'WeichenXu123' has created a pull request for this

[jira] [Comment Edited] (SPARK-22513) Provide build profile for hadoop 2.8

[ https://issues.apache.org/jira/browse/SPARK-22513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413685#comment-16413685 ] Steve Loughran edited comment on SPARK-22513 at 3/26/18 11:13 AM: -- API

[jira] [Comment Edited] (SPARK-22513) Provide build profile for hadoop 2.8

[ https://issues.apache.org/jira/browse/SPARK-22513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413685#comment-16413685 ] Steve Loughran edited comment on SPARK-22513 at 3/26/18 11:10 AM: -- API

[jira] [Assigned] (SPARK-23751) Kolmogorov-Smirnoff test Python API in pyspark.ml

[ https://issues.apache.org/jira/browse/SPARK-23751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23751: Assignee: (was: Apache Spark) > Kolmogorov-Smirnoff test Python API in pyspark.ml >

[jira] [Assigned] (SPARK-23751) Kolmogorov-Smirnoff test Python API in pyspark.ml

[ https://issues.apache.org/jira/browse/SPARK-23751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23751: Assignee: Apache Spark > Kolmogorov-Smirnoff test Python API in pyspark.ml >

[jira] [Comment Edited] (SPARK-22513) Provide build profile for hadoop 2.8

[ https://issues.apache.org/jira/browse/SPARK-22513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413685#comment-16413685 ] Steve Loughran edited comment on SPARK-22513 at 3/26/18 11:10 AM: -- API

[jira] [Commented] (SPARK-22513) Provide build profile for hadoop 2.8

[ https://issues.apache.org/jira/browse/SPARK-22513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413685#comment-16413685 ] Steve Loughran commented on SPARK-22513: API wise, everything compiled against 2.6 should compile

[jira] [Comment Edited] (SPARK-22513) Provide build profile for hadoop 2.8

[ https://issues.apache.org/jira/browse/SPARK-22513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413685#comment-16413685 ] Steve Loughran edited comment on SPARK-22513 at 3/26/18 11:11 AM: -- API

[jira] [Created] (SPARK-23794) UUID() should be stateful

Herman van Hovell created SPARK-23794: - Summary: UUID() should be stateful Key: SPARK-23794 URL: https://issues.apache.org/jira/browse/SPARK-23794 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-23599) The UUID() expression is too non-deterministic

[ https://issues.apache.org/jira/browse/SPARK-23599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413620#comment-16413620 ] Apache Spark commented on SPARK-23599: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-23565) Improved error message for when the number of sources for a query changes

[ https://issues.apache.org/jira/browse/SPARK-23565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413570#comment-16413570 ] Roman Maier commented on SPARK-23565: - I can not continue to work on this task in the foreseeable

[jira] [Commented] (SPARK-23793) Handle database names in spark.udf.register()

[ https://issues.apache.org/jira/browse/SPARK-23793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413542#comment-16413542 ] Takeshi Yamamuro commented on SPARK-23793: -- [~smilegator] This behaviour is expected? > Handle

[jira] [Created] (SPARK-23793) Handle database names in spark.udf.register()

Takeshi Yamamuro created SPARK-23793: Summary: Handle database names in spark.udf.register() Key: SPARK-23793 URL: https://issues.apache.org/jira/browse/SPARK-23793 Project: Spark Issue

[jira] [Commented] (SPARK-23739) Spark structured streaming long running problem

[ https://issues.apache.org/jira/browse/SPARK-23739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413530#comment-16413530 ] Florencio commented on SPARK-23739: --- Thanks Cody. The kafka version is version=0.10.0.1 and I checked

[jira] [Commented] (SPARK-23770) Expose repartitionByRange in SparkR

[ https://issues.apache.org/jira/browse/SPARK-23770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413438#comment-16413438 ] Apache Spark commented on SPARK-23770: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-23770) Expose repartitionByRange in SparkR

[ https://issues.apache.org/jira/browse/SPARK-23770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23770: Assignee: Apache Spark > Expose repartitionByRange in SparkR >

[jira] [Assigned] (SPARK-23770) Expose repartitionByRange in SparkR

[ https://issues.apache.org/jira/browse/SPARK-23770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23770: Assignee: (was: Apache Spark) > Expose repartitionByRange in SparkR >

[jira] [Commented] (SPARK-22876) spark.yarn.am.attemptFailuresValidityInterval does not work correctly

[ https://issues.apache.org/jira/browse/SPARK-22876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413412#comment-16413412 ] Saisai Shao commented on SPARK-22876: - Yes, that's the key problem. Current YARN doesn't support

[jira] [Resolved] (SPARK-23787) SparkSubmitSuite::"download remote resource if it is not supported by yarn" fails on Hadoop 2.9

[ https://issues.apache.org/jira/browse/SPARK-23787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-23787. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20895

[jira] [Assigned] (SPARK-23787) SparkSubmitSuite::"download remote resource if it is not supported by yarn" fails on Hadoop 2.9

[ https://issues.apache.org/jira/browse/SPARK-23787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-23787: --- Assignee: Marcelo Vanzin > SparkSubmitSuite::"download remote resource if it is not