[jira] [Issue Comment Deleted] (SPARK-26183) ConcurrentModificationException when using Spark collectionAccumulator

2018-11-29 Thread Jairo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jairo updated SPARK-26183: -- Comment: was deleted (was: testing thincrs) > ConcurrentModificationException when using Spark

[jira] [Created] (SPARK-26219) Executor summary is not getting updated for failure jobs in history server UI

2018-11-29 Thread shahid (JIRA)
shahid created SPARK-26219: -- Summary: Executor summary is not getting updated for failure jobs in history server UI Key: SPARK-26219 URL: https://issues.apache.org/jira/browse/SPARK-26219 Project: Spark

[jira] [Updated] (SPARK-26219) Executor summary is not getting updated for failure jobs in history server UI

2018-11-29 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-26219: --- Attachment: Screenshot from 2018-11-29 22-13-34.png > Executor summary is not getting updated for failure

[jira] [Updated] (SPARK-26219) Executor summary is not getting updated for failure jobs in history server UI

2018-11-29 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-26219: --- Description: Test step to reproduce: {code:java} bin/spark-shell --master yarn --conf

[jira] [Assigned] (SPARK-26158) Enhance the accuracy of covariance in RowMatrix for DenseVector

2018-11-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26158: - Assignee: Liang Li > Enhance the accuracy of covariance in RowMatrix for DenseVector >

[jira] [Resolved] (SPARK-26158) Enhance the accuracy of covariance in RowMatrix for DenseVector

2018-11-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26158. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23126

[jira] [Commented] (SPARK-26177) Automated formatting for Scala code

2018-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703700#comment-16703700 ] Apache Spark commented on SPARK-26177: -- User 'koeninger' has created a pull request for this issue:

[jira] [Created] (SPARK-26222) Scan: track file listing time

2018-11-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26222: --- Summary: Scan: track file listing time Key: SPARK-26222 URL: https://issues.apache.org/jira/browse/SPARK-26222 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-25905) BlockManager should expose getRemoteManagedBuffer to avoid creating bytebuffers

2018-11-29 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-25905. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23058

[jira] [Assigned] (SPARK-26015) Include a USER directive in project provided Spark Dockerfiles

2018-11-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26015: -- Assignee: Rob Vesse > Include a USER directive in project provided Spark Dockerfiles

[jira] [Resolved] (SPARK-26015) Include a USER directive in project provided Spark Dockerfiles

2018-11-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26015. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23017

[jira] [Created] (SPARK-26221) Improve Spark SQL instrumentation and metrics

2018-11-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26221: --- Summary: Improve Spark SQL instrumentation and metrics Key: SPARK-26221 URL: https://issues.apache.org/jira/browse/SPARK-26221 Project: Spark Issue Type:

[jira] [Updated] (SPARK-26221) Improve Spark SQL instrumentation and metrics

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26221: Description: This is an umbrella ticket for various small improvements for better metrics and

[jira] [Commented] (SPARK-26183) ConcurrentModificationException when using Spark collectionAccumulator

2018-11-29 Thread Jairo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703534#comment-16703534 ] Jairo commented on SPARK-26183: --- .../ > ConcurrentModificationException when using Spark

[jira] [Issue Comment Deleted] (SPARK-26183) ConcurrentModificationException when using Spark collectionAccumulator

2018-11-29 Thread Jairo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jairo updated SPARK-26183: -- Comment: was deleted (was: ..///) > ConcurrentModificationException when using Spark collectionAccumulator >

[jira] [Issue Comment Deleted] (SPARK-26183) ConcurrentModificationException when using Spark collectionAccumulator

2018-11-29 Thread Jairo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jairo updated SPARK-26183: -- Comment: was deleted (was: .) > ConcurrentModificationException when using Spark collectionAccumulator >

[jira] [Issue Comment Deleted] (SPARK-26183) ConcurrentModificationException when using Spark collectionAccumulator

2018-11-29 Thread Jairo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jairo updated SPARK-26183: -- Comment: was deleted (was: .) > ConcurrentModificationException when using Spark collectionAccumulator >

[jira] [Issue Comment Deleted] (SPARK-26183) ConcurrentModificationException when using Spark collectionAccumulator

2018-11-29 Thread Jairo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jairo updated SPARK-26183: -- Comment: was deleted (was: .) > ConcurrentModificationException when using Spark collectionAccumulator >

[jira] [Issue Comment Deleted] (SPARK-26183) ConcurrentModificationException when using Spark collectionAccumulator

2018-11-29 Thread Jairo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jairo updated SPARK-26183: -- Comment: was deleted (was: .../) > ConcurrentModificationException when using Spark collectionAccumulator >

[jira] [Commented] (SPARK-26219) Executor summary is not getting updated for failure jobs in history server UI

2018-11-29 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703619#comment-16703619 ] shahid commented on SPARK-26219: I will raise a PR > Executor summary is not getting updated for

[jira] [Commented] (SPARK-26100) [History server ]Jobs table and Aggregate metrics table are showing lesser number of tasks

2018-11-29 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703693#comment-16703693 ] shahid commented on SPARK-26100: Sorry, I have wrongly linked the JIRA > [History server ]Jobs table

[jira] [Created] (SPARK-26223) Scan: track metastore operation time

2018-11-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26223: --- Summary: Scan: track metastore operation time Key: SPARK-26223 URL: https://issues.apache.org/jira/browse/SPARK-26223 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-26183) ConcurrentModificationException when using Spark collectionAccumulator

2018-11-29 Thread Jairo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703593#comment-16703593 ] Jairo commented on SPARK-26183: --- ../// > ConcurrentModificationException when using Spark

[jira] [Updated] (SPARK-26166) CrossValidator.fit() bug,training and validation dataset may overlap

2018-11-29 Thread Xinyong Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinyong Tian updated SPARK-26166: - Description: In the code pyspark.ml.tuning.CrossValidator.fit(), after adding random column df

[jira] [Updated] (SPARK-26166) CrossValidator.fit() bug,training and validation dataset may overlap

2018-11-29 Thread Xinyong Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinyong Tian updated SPARK-26166: - Description: In the code pyspark.ml.tuning.CrossValidator.fit(), after adding random column df

[jira] [Assigned] (SPARK-25905) BlockManager should expose getRemoteManagedBuffer to avoid creating bytebuffers

2018-11-29 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid reassigned SPARK-25905: Assignee: Wing Yew Poon > BlockManager should expose getRemoteManagedBuffer to avoid

[jira] [Resolved] (SPARK-26186) In progress applications with last updated time is lesser than the cleaning interval are getting removed during cleaning logs

2018-11-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26186. Resolution: Fixed Fix Version/s: 2.4.1 3.0.0 Issue resolved by

[jira] [Assigned] (SPARK-26186) In progress applications with last updated time is lesser than the cleaning interval are getting removed during cleaning logs

2018-11-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26186: -- Assignee: shahid > In progress applications with last updated time is lesser than

[jira] [Assigned] (SPARK-26184) Last updated time is not getting updated in the History Server UI

2018-11-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26184: -- Assignee: shahid > Last updated time is not getting updated in the History Server UI

[jira] [Resolved] (SPARK-26184) Last updated time is not getting updated in the History Server UI

2018-11-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26184. Resolution: Fixed Fix Version/s: 2.4.1 3.0.0 Issue resolved by

[jira] [Updated] (SPARK-26219) Executor summary is not getting updated for failure jobs in history server UI

2018-11-29 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-26219: --- Attachment: Screenshot from 2018-11-29 22-13-44.png > Executor summary is not getting updated for failure

[jira] [Created] (SPARK-26224) Results in stackOverFlowError when trying to add 3000 new columns using withColumn function of dataframe.

2018-11-29 Thread Dorjee Tsering (JIRA)
Dorjee Tsering created SPARK-26224: -- Summary: Results in stackOverFlowError when trying to add 3000 new columns using withColumn function of dataframe. Key: SPARK-26224 URL:

[jira] [Resolved] (SPARK-24498) Add JDK compiler for runtime codegen

2018-11-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-24498. -- Resolution: Won't Do > Add JDK compiler for runtime codegen >

[jira] [Commented] (SPARK-26219) Executor summary is not getting updated for failure jobs in history server UI

2018-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703694#comment-16703694 ] Apache Spark commented on SPARK-26219: -- User 'shahidki31' has created a pull request for this

[jira] [Commented] (SPARK-26219) Executor summary is not getting updated for failure jobs in history server UI

2018-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703696#comment-16703696 ] Apache Spark commented on SPARK-26219: -- User 'shahidki31' has created a pull request for this

[jira] [Assigned] (SPARK-26219) Executor summary is not getting updated for failure jobs in history server UI

2018-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26219: Assignee: Apache Spark > Executor summary is not getting updated for failure jobs in

[jira] [Resolved] (SPARK-26188) Spark 2.4.0 Partitioning behavior breaks backwards compatibility

2018-11-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26188. - Resolution: Fixed Fix Version/s: 2.4.1 3.0.0 Issue resolved by pull

[jira] [Commented] (SPARK-26183) ConcurrentModificationException when using Spark collectionAccumulator

2018-11-29 Thread Jairo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703596#comment-16703596 ] Jairo commented on SPARK-26183: --- testing thincrs > ConcurrentModificationException when using Spark

[jira] [Commented] (SPARK-26100) [History server ]Jobs table and Aggregate metrics table are showing lesser number of tasks

2018-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703631#comment-16703631 ] Apache Spark commented on SPARK-26100: -- User 'shahidki31' has created a pull request for this

[jira] [Updated] (SPARK-26220) Flaky Test: org.apache.spark.sql.kafka010.KafkaSourceStressSuite

2018-11-29 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-26220: -- Issue Type: Bug (was: New Feature) > Flaky Test:

[jira] [Created] (SPARK-26220) Flaky Test: org.apache.spark.sql.kafka010.KafkaSourceStressSuite

2018-11-29 Thread Gabor Somogyi (JIRA)
Gabor Somogyi created SPARK-26220: - Summary: Flaky Test: org.apache.spark.sql.kafka010.KafkaSourceStressSuite Key: SPARK-26220 URL: https://issues.apache.org/jira/browse/SPARK-26220 Project: Spark

[jira] [Updated] (SPARK-26220) Flaky Test: org.apache.spark.sql.kafka010.KafkaSourceStressSuite

2018-11-29 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-26220: -- Description: There was a kafka-clients version update lately and I've seen a test failure

[jira] [Commented] (SPARK-26183) ConcurrentModificationException when using Spark collectionAccumulator

2018-11-29 Thread Thincrs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703667#comment-16703667 ] Thincrs commented on SPARK-26183: - testing thincrs > ConcurrentModificationException when using Spark

[jira] [Assigned] (SPARK-26219) Executor summary is not getting updated for failure jobs in history server UI

2018-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26219: Assignee: (was: Apache Spark) > Executor summary is not getting updated for failure

[jira] [Assigned] (SPARK-26060) Track SparkConf entries and make SET command reject such entries.

2018-11-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26060: --- Assignee: Takuya Ueshin > Track SparkConf entries and make SET command reject such

[jira] [Resolved] (SPARK-26060) Track SparkConf entries and make SET command reject such entries.

2018-11-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26060. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23031

[jira] [Resolved] (SPARK-25977) Parsing decimals from CSV using locale

2018-11-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25977. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22979

[jira] [Assigned] (SPARK-25977) Parsing decimals from CSV using locale

2018-11-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25977: Assignee: Maxim Gekk > Parsing decimals from CSV using locale >

[jira] [Resolved] (SPARK-25446) Add schema_of_json() and schema_of_csv() to R

2018-11-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25446. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22939

[jira] [Commented] (SPARK-26227) from_[csv|json] should accept schema_of_[csv|json] in R API

2018-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704231#comment-16704231 ] Apache Spark commented on SPARK-26227: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-26188) Spark 2.4.0 Partitioning behavior breaks backwards compatibility

2018-11-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26188: --- Assignee: Gengliang Wang > Spark 2.4.0 Partitioning behavior breaks backwards

[jira] [Comment Edited] (SPARK-26200) Column values are incorrectly transposed when a field in a PySpark Row requires serialization

2018-11-29 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704092#comment-16704092 ] Bryan Cutler edited comment on SPARK-26200 at 11/30/18 12:56 AM: - I

[jira] [Commented] (SPARK-26200) Column values are incorrectly transposed when a field in a PySpark Row requires serialization

2018-11-29 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704092#comment-16704092 ] Bryan Cutler commented on SPARK-26200: -- I think this is a duplicate of

[jira] [Assigned] (SPARK-25501) Kafka delegation token support

2018-11-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25501: -- Assignee: Gabor Somogyi > Kafka delegation token support >

[jira] [Resolved] (SPARK-25501) Kafka delegation token support

2018-11-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25501. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22598

[jira] [Commented] (SPARK-26227) from_[csv|json] should accept schema_of_[csv|json] in R API

2018-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704230#comment-16704230 ] Apache Spark commented on SPARK-26227: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-26228) OOM issue encountered when computing Gramian matrix

2018-11-29 Thread Chen Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Lin updated SPARK-26228: - Description: /**  * Computes the Gramian matrix `A^T A`.  *  * @note This cannot be computed on

[jira] [Updated] (SPARK-26228) OOM issue encountered when computing Gramian matrix

2018-11-29 Thread Chen Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Lin updated SPARK-26228: - Description: /**  * Computes the Gramian matrix `A^T A`.  *  *@note This cannot be computed on

[jira] [Created] (SPARK-26228) OOM issue encountered when computing Gramian matrix

2018-11-29 Thread Chen Lin (JIRA)
Chen Lin created SPARK-26228: Summary: OOM issue encountered when computing Gramian matrix Key: SPARK-26228 URL: https://issues.apache.org/jira/browse/SPARK-26228 Project: Spark Issue Type:

[jira] [Created] (SPARK-26227) from_[csv|json] should accept schema_of_[csv|json] in R API

2018-11-29 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-26227: Summary: from_[csv|json] should accept schema_of_[csv|json] in R API Key: SPARK-26227 URL: https://issues.apache.org/jira/browse/SPARK-26227 Project: Spark

[jira] [Assigned] (SPARK-26227) from_[csv|json] should accept schema_of_[csv|json] in R API

2018-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26227: Assignee: Apache Spark > from_[csv|json] should accept schema_of_[csv|json] in R API >

[jira] [Assigned] (SPARK-26227) from_[csv|json] should accept schema_of_[csv|json] in R API

2018-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26227: Assignee: (was: Apache Spark) > from_[csv|json] should accept schema_of_[csv|json]

[jira] [Updated] (SPARK-26228) OOM issue encountered when computing Gramian matrix

2018-11-29 Thread Chen Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Lin updated SPARK-26228: - Description: {quote}/**  * Computes the Gramian matrix `A^T A`.  *  * @note This cannot be computed

[jira] [Updated] (SPARK-26228) OOM issue encountered when computing Gramian matrix

2018-11-29 Thread Chen Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Lin updated SPARK-26228: - Description: /**  * Computes the Gramian matrix `A^T A`.  *  * @note This cannot be computed on

[jira] [Commented] (SPARK-26212) Upgrade maven from 3.5.4 to 3.6.0

2018-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702841#comment-16702841 ] Apache Spark commented on SPARK-26212: -- User 'kiszk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26212) Upgrade maven from 3.5.4 to 3.6.0

2018-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26212: Assignee: (was: Apache Spark) > Upgrade maven from 3.5.4 to 3.6.0 >

[jira] [Assigned] (SPARK-26212) Upgrade maven from 3.5.4 to 3.6.0

2018-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26212: Assignee: Apache Spark > Upgrade maven from 3.5.4 to 3.6.0 >

[jira] [Commented] (SPARK-24498) Add JDK compiler for runtime codegen

2018-11-29 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702889#comment-16702889 ] Marco Gaido commented on SPARK-24498: - +1 for closing this. > Add JDK compiler for runtime codegen

[jira] [Created] (SPARK-26212) Upgrade maven from 3.5.4 to 3.6.0

2018-11-29 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-26212: Summary: Upgrade maven from 3.5.4 to 3.6.0 Key: SPARK-26212 URL: https://issues.apache.org/jira/browse/SPARK-26212 Project: Spark Issue Type:

[jira] [Updated] (SPARK-26129) Instrumentation for query planning time

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26129: Issue Type: Sub-task (was: New Feature) Parent: SPARK-26221 > Instrumentation for query

[jira] [Commented] (SPARK-26226) Update query tracker to report timeline for phases, rather than duration

2018-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704011#comment-16704011 ] Apache Spark commented on SPARK-26226: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26226) Update query tracker to report timeline for phases, rather than duration

2018-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26226: Assignee: Reynold Xin (was: Apache Spark) > Update query tracker to report timeline for

[jira] [Updated] (SPARK-26221) Improve Spark SQL instrumentation and metrics

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26221: Description: This is an umbrella ticket for various small improvements for better metrics and

[jira] [Assigned] (SPARK-26226) Update query tracker to report timeline for phases, rather than duration

2018-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26226: Assignee: Apache Spark (was: Reynold Xin) > Update query tracker to report timeline for

[jira] [Assigned] (SPARK-26226) Update query tracker to report timeline for phases, rather than duration

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-26226: --- Assignee: Reynold Xin > Update query tracker to report timeline for phases, rather than

[jira] [Commented] (SPARK-26226) Update query tracker to report timeline for phases, rather than duration

2018-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704010#comment-16704010 ] Apache Spark commented on SPARK-26226: -- User 'rxin' has created a pull request for this issue:

[jira] [Commented] (SPARK-26209) Allow for dataframe bucketization without Hive

2018-11-29 Thread Sam hendley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703824#comment-16703824 ] Sam hendley commented on SPARK-26209: - Seems like dataframe needs a side-channel for the

[jira] [Created] (SPARK-26225) Scan: track decoding time for row-based data sources

2018-11-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26225: --- Summary: Scan: track decoding time for row-based data sources Key: SPARK-26225 URL: https://issues.apache.org/jira/browse/SPARK-26225 Project: Spark Issue

[jira] [Updated] (SPARK-26223) Scan: track metastore operation time

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26223: Description: The Scan node should report how much time it spent in metastore operations. Similar

[jira] [Updated] (SPARK-26221) Improve Spark SQL instrumentation and metrics

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26221: Description: This is an umbrella ticket for various small improvements for better metrics and

[jira] [Updated] (SPARK-26221) Improve Spark SQL instrumentation and metrics

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26221: Description: This is an umbrella ticket for various small improvements for better metrics and

[jira] [Created] (SPARK-26226) Update query tracker to report timeline for phases, rather than duration

2018-11-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26226: --- Summary: Update query tracker to report timeline for phases, rather than duration Key: SPARK-26226 URL: https://issues.apache.org/jira/browse/SPARK-26226 Project:

[jira] [Updated] (SPARK-25708) HAVING without GROUP BY means global aggregate

2018-11-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-25708: Description: According to the SQL standard, when a query contains `HAVING`, it indicates an

[jira] [Commented] (SPARK-26206) Spark structured streaming with kafka integration fails in update mode

2018-11-29 Thread indraneel r (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704277#comment-16704277 ] indraneel r commented on SPARK-26206: - Will check on spark-shell but not sure if it will make any

[jira] [Commented] (SPARK-26182) Cost increases when optimizing scalaUDF

2018-11-29 Thread Jiayi Liao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704347#comment-16704347 ] Jiayi Liao commented on SPARK-26182: [~fanweiwen] We're doing some optimizations on our own

[jira] [Created] (SPARK-26213) Custom Receiver for Structured streaming

2018-11-29 Thread Aarthi (JIRA)
Aarthi created SPARK-26213: -- Summary: Custom Receiver for Structured streaming Key: SPARK-26213 URL: https://issues.apache.org/jira/browse/SPARK-26213 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-26214) Add "broadcast" method to DataFrame

2018-11-29 Thread Thomas Decaux (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Decaux updated SPARK-26214: -- Description: As discussed at

[jira] [Created] (SPARK-26215) define reserved keywords after SQL standard

2018-11-29 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-26215: --- Summary: define reserved keywords after SQL standard Key: SPARK-26215 URL: https://issues.apache.org/jira/browse/SPARK-26215 Project: Spark Issue Type:

[jira] [Commented] (SPARK-26215) define reserved keywords after SQL standard

2018-11-29 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703042#comment-16703042 ] Marco Gaido commented on SPARK-26215: - [~cloud_fan] thanks for pinging me. I agree on putting a

[jira] [Created] (SPARK-26214) Add "broadcast" method to DataFrame

2018-11-29 Thread Thomas Decaux (JIRA)
Thomas Decaux created SPARK-26214: - Summary: Add "broadcast" method to DataFrame Key: SPARK-26214 URL: https://issues.apache.org/jira/browse/SPARK-26214 Project: Spark Issue Type: Wish

[jira] [Commented] (SPARK-26215) define reserved keywords after SQL standard

2018-11-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703031#comment-16703031 ] Wenchen Fan commented on SPARK-26215: - cc [~maropu] [~LI,Xiao] [~viirya] [~mgaido] > define

[jira] [Commented] (SPARK-26214) Add "broadcast" method to DataFrame

2018-11-29 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702994#comment-16702994 ] Marco Gaido commented on SPARK-26214: - You can just use the {{broadcast}} function from

[jira] [Commented] (SPARK-26215) define reserved keywords after SQL standard

2018-11-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703035#comment-16703035 ] Liang-Chi Hsieh commented on SPARK-26215: - Thanks for pinging me. Is "In Spark SQL, we are too

[jira] [Updated] (SPARK-26173) Prior regularization for Logistic Regression

2018-11-29 Thread Facundo Bellosi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Facundo Bellosi updated SPARK-26173: Description: This feature enables Maximum A Posteriori (MAP) optimization for Logistic

[jira] [Updated] (SPARK-26213) Custom Receiver for Structured streaming

2018-11-29 Thread Aarthi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aarthi updated SPARK-26213: --- Component/s: (was: Spark Core) Structured Streaming > Custom Receiver for Structured

[jira] [Assigned] (SPARK-26177) Automated formatting for Scala code

2018-11-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26177: - Assignee: Cody Koeninger > Automated formatting for Scala code >

[jira] [Commented] (SPARK-26215) define reserved keywords after SQL standard

2018-11-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703233#comment-16703233 ] Wenchen Fan commented on SPARK-26215: - > Is "In Spark SQL, we are too tolerant about non-reserved

[jira] [Assigned] (SPARK-26163) Parsing decimals from JSON using locale

2018-11-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26163: --- Assignee: Maxim Gekk > Parsing decimals from JSON using locale >

[jira] [Updated] (SPARK-23179) Support option to throw exception if overflow occurs

2018-11-29 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-23179: Issue Type: Sub-task (was: Improvement) Parent: SPARK-26217 > Support option to throw

[jira] [Assigned] (SPARK-26218) Throw exception on overflow for integers

2018-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26218: Assignee: (was: Apache Spark) > Throw exception on overflow for integers >

  1   2   >