[jira] [Commented] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

2018-11-27 Thread Xianjin YE (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700054#comment-16700054 ] Xianjin YE commented on SPARK-26155: cc [~cloud_fan]. do you confirm this problem? A

[jira] [Commented] (SPARK-19256) Hive bucketing support

2018-11-27 Thread Shay Elbaz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700067#comment-16700067 ] Shay Elbaz commented on SPARK-19256: [~chengsu] this is great! If there is anything

[jira] [Commented] (SPARK-19256) Hive bucketing support

2018-11-27 Thread Xianjin YE (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700089#comment-16700089 ] Xianjin YE commented on SPARK-19256: [~chengsu] great news. I can do some review wor

[jira] [Commented] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

2018-11-27 Thread Yang Jie (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700130#comment-16700130 ] Yang Jie commented on SPARK-26155: -- May need [~Jk_Self] to provide more details to see

[jira] [Created] (SPARK-26182) Cost increases when optimizing scalaUDF

2018-11-27 Thread Jiayi Liao (JIRA)
Jiayi Liao created SPARK-26182: -- Summary: Cost increases when optimizing scalaUDF Key: SPARK-26182 URL: https://issues.apache.org/jira/browse/SPARK-26182 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-26178) Use java.time API for parsing timestamps and dates from CSV

2018-11-27 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700147#comment-16700147 ] Maxim Gekk commented on SPARK-26178: [~srowen] It is somehow related to the ticket b

[jira] [Commented] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

2018-11-27 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700213#comment-16700213 ] Liang-Chi Hsieh commented on SPARK-26155: - "Q19 analysis in Spark2.3 without L48

[jira] [Commented] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

2018-11-27 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700228#comment-16700228 ] Liang-Chi Hsieh commented on SPARK-26155: - Is there possibly any other thing aff

[jira] [Commented] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

2018-11-27 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700253#comment-16700253 ] Adrian Wang commented on SPARK-26155: - [~viirya] , thanks for your reply. [~Jk_Self]

[jira] [Comment Edited] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

2018-11-27 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700213#comment-16700213 ] Liang-Chi Hsieh edited comment on SPARK-26155 at 11/27/18 10:48 AM: --

[jira] [Assigned] (SPARK-26147) Python UDFs in join condition fail even when using columns from only one side of join

2018-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26147: Assignee: Apache Spark > Python UDFs in join condition fail even when using columns from

[jira] [Commented] (SPARK-26147) Python UDFs in join condition fail even when using columns from only one side of join

2018-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700274#comment-16700274 ] Apache Spark commented on SPARK-26147: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-26147) Python UDFs in join condition fail even when using columns from only one side of join

2018-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26147: Assignee: (was: Apache Spark) > Python UDFs in join condition fail even when using co

[jira] [Resolved] (SPARK-23356) Pushes Project to both sides of Union when expression is non-deterministic

2018-11-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23356. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23138 [https://gith

[jira] [Assigned] (SPARK-23356) Pushes Project to both sides of Union when expression is non-deterministic

2018-11-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-23356: --- Assignee: caoxuewen > Pushes Project to both sides of Union when expression is non-determin

[jira] [Comment Edited] (SPARK-23153) Support application dependencies in submission client's local file system

2018-11-27 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700356#comment-16700356 ] Stavros Kontopoulos edited comment on SPARK-23153 at 11/27/18 12:40 PM: --

[jira] [Commented] (SPARK-23153) Support application dependencies in submission client's local file system

2018-11-27 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700356#comment-16700356 ] Stavros Kontopoulos commented on SPARK-23153: - [~eje] [~rvesse] [~liyinan926

[jira] [Created] (SPARK-26183) ConcurrentModificationException when using Spark collectionAccumulator

2018-11-27 Thread Rob Dawson (JIRA)
Rob Dawson created SPARK-26183: -- Summary: ConcurrentModificationException when using Spark collectionAccumulator Key: SPARK-26183 URL: https://issues.apache.org/jira/browse/SPARK-26183 Project: Spark

[jira] [Commented] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

2018-11-27 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700414#comment-16700414 ] Liang-Chi Hsieh commented on SPARK-26155: - I'd also like to see more details abo

[jira] [Commented] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

2018-11-27 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700431#comment-16700431 ] Liang-Chi Hsieh commented on SPARK-26155: - Btw, have you observed the same thing

[jira] [Commented] (SPARK-26165) Date and Timestamp column expression is getting converted to string in less than/greater than filter query even though valid date/timestamp string literal is used in t

2018-11-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700441#comment-16700441 ] Sean Owen commented on SPARK-26165: --- OK this is probably a question for [~yhuai] or [~

[jira] [Commented] (SPARK-24063) Control maximum epoch backlog

2018-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700449#comment-16700449 ] Apache Spark commented on SPARK-24063: -- User 'gaborgsomogyi' has created a pull req

[jira] [Commented] (SPARK-26165) Date and Timestamp column expression is getting converted to string in less than/greater than filter query even though valid date/timestamp string literal is used in t

2018-11-27 Thread Sujith (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700459#comment-16700459 ] Sujith commented on SPARK-26165: [~srowen] i can also raise a PR for this issue so that

[jira] [Commented] (SPARK-26089) Handle large corrupt shuffle blocks

2018-11-27 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700493#comment-16700493 ] Thomas Graves commented on SPARK-26089: --- Yeah that seems to make sense and I would

[jira] [Comment Edited] (SPARK-25433) Add support for PEX in PySpark

2018-11-27 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-25433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700590#comment-16700590 ] Fabian Höring edited comment on SPARK-25433 at 11/27/18 3:31 PM: -

[jira] [Commented] (SPARK-25433) Add support for PEX in PySpark

2018-11-27 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-25433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700590#comment-16700590 ] Fabian Höring commented on SPARK-25433: --- {code} $ pex numpy -o myarchive.pex $ exp

[jira] [Resolved] (SPARK-25433) Add support for PEX in PySpark

2018-11-27 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-25433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabian Höring resolved SPARK-25433. --- Resolution: Not A Problem > Add support for PEX in PySpark > --

[jira] [Comment Edited] (SPARK-25433) Add support for PEX in PySpark

2018-11-27 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-25433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700590#comment-16700590 ] Fabian Höring edited comment on SPARK-25433 at 11/27/18 3:33 PM: -

[jira] [Commented] (SPARK-26183) ConcurrentModificationException when using Spark collectionAccumulator

2018-11-27 Thread Jairo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700700#comment-16700700 ] Jairo commented on SPARK-26183: --- . > ConcurrentModificationException when using Spark col

[jira] [Commented] (SPARK-26183) ConcurrentModificationException when using Spark collectionAccumulator

2018-11-27 Thread Jairo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700702#comment-16700702 ] Jairo commented on SPARK-26183: --- . > ConcurrentModificationException when using Spark col

[jira] [Resolved] (SPARK-26025) Docker image build on dev builds is slow

2018-11-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26025. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23019 [https:

[jira] [Assigned] (SPARK-26025) Docker image build on dev builds is slow

2018-11-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26025: -- Assignee: Marcelo Vanzin > Docker image build on dev builds is slow > ---

[jira] [Commented] (SPARK-21291) R bucketBy partitionBy API

2018-11-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700743#comment-16700743 ] Felix Cheung commented on SPARK-21291: -- I think we need to reopen this Jira since b

[jira] [Created] (SPARK-26184) Last updated time is not getting updated in the History Server UI

2018-11-27 Thread ABHISHEK KUMAR GUPTA (JIRA)
ABHISHEK KUMAR GUPTA created SPARK-26184: Summary: Last updated time is not getting updated in the History Server UI Key: SPARK-26184 URL: https://issues.apache.org/jira/browse/SPARK-26184 Pro

[jira] [Updated] (SPARK-26184) Last updated time is not getting updated in the History Server UI

2018-11-27 Thread ABHISHEK KUMAR GUPTA (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ABHISHEK KUMAR GUPTA updated SPARK-26184: - Description: For inprogress application, last updated time is not getting update

[jira] [Commented] (SPARK-26184) Last updated time is not getting updated in the History Server UI

2018-11-27 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700792#comment-16700792 ] shahid commented on SPARK-26184: I will raise a PR > Last updated time is not getting u

[jira] [Updated] (SPARK-26184) Last updated time is not getting updated in the History Server UI

2018-11-27 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-26184: --- Attachment: Screenshot from 2018-11-27 13-21-34.png > Last updated time is not getting updated in the Histor

[jira] [Updated] (SPARK-26184) Last updated time is not getting updated in the History Server UI

2018-11-27 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-26184: --- Attachment: screenshot-1.png > Last updated time is not getting updated in the History Server UI > -

[jira] [Created] (SPARK-26185) add weightCol in python MulticlassClassificationEvaluator

2018-11-27 Thread Huaxin Gao (JIRA)
Huaxin Gao created SPARK-26185: -- Summary: add weightCol in python MulticlassClassificationEvaluator Key: SPARK-26185 URL: https://issues.apache.org/jira/browse/SPARK-26185 Project: Spark Issue T

[jira] [Updated] (SPARK-26184) Last updated time is not getting updated in the History Server UI

2018-11-27 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-26184: --- Attachment: Screenshot from 2018-11-27 23-20-11.png > Last updated time is not getting updated in the Histor

[jira] [Updated] (SPARK-26184) Last updated time is not getting updated in the History Server UI

2018-11-27 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-26184: --- Description: For inprogress application, last updated time is not getting updated. !Screenshot from 2018-

[jira] [Updated] (SPARK-26184) Last updated time is not getting updated in the History Server UI

2018-11-27 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-26184: --- Attachment: (was: screenshot-1.png) > Last updated time is not getting updated in the History Server UI

[jira] [Updated] (SPARK-26184) Last updated time is not getting updated in the History Server UI

2018-11-27 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-26184: --- Description: For inprogress application, last updated time is not getting updated. !Screenshot from 2018

[jira] [Updated] (SPARK-26184) Last updated time is not getting updated in the History Server UI

2018-11-27 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-26184: --- Attachment: Screenshot from 2018-11-27 23-22-38.png > Last updated time is not getting updated in the Histor

[jira] [Commented] (SPARK-23410) Unable to read jsons in charset different from UTF-8

2018-11-27 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700799#comment-16700799 ] Maxim Gekk commented on SPARK-23410: > Even if lineSeps is set, it is still necessar

[jira] [Commented] (SPARK-26186) In progress applications with last updated time is lesser than the cleaning interval are getting removed during cleaning logs

2018-11-27 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700804#comment-16700804 ] shahid commented on SPARK-26186: I will raise a PR. > In progress applications with las

[jira] [Assigned] (SPARK-26185) add weightCol in python MulticlassClassificationEvaluator

2018-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26185: Assignee: (was: Apache Spark) > add weightCol in python MulticlassClassificationEvalu

[jira] [Updated] (SPARK-26185) add weightCol in python MulticlassClassificationEvaluator

2018-11-27 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-26185: --- Description: -https://issues.apache.org/jira/browse/SPARK-24101- added weightCol in MulticlassClassi

[jira] [Assigned] (SPARK-26185) add weightCol in python MulticlassClassificationEvaluator

2018-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26185: Assignee: Apache Spark > add weightCol in python MulticlassClassificationEvaluator >

[jira] [Updated] (SPARK-26185) add weightCol in python MulticlassClassificationEvaluator

2018-11-27 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-26185: --- Description: --https://issues.apache.org/jira/browse/SPARK-24101-- added weightCol in MulticlassClas

[jira] [Commented] (SPARK-26185) add weightCol in python MulticlassClassificationEvaluator

2018-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700806#comment-16700806 ] Apache Spark commented on SPARK-26185: -- User 'huaxingao' has created a pull request

[jira] [Updated] (SPARK-26184) Last updated time is not getting updated in the History Server UI

2018-11-27 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-26184: --- Description: For inprogress application, last updated time is not getting updated. !Screenshot from 2018-1

[jira] [Updated] (SPARK-26184) Last updated time is not getting updated in the History Server UI

2018-11-27 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-26184: --- Attachment: (was: Screenshot from 2018-11-27 13-21-34.png) > Last updated time is not getting updated in

[jira] [Created] (SPARK-26186) In progress applications with last updated time is lesser than the cleaning interval are getting removed during cleaning logs

2018-11-27 Thread shahid (JIRA)
shahid created SPARK-26186: -- Summary: In progress applications with last updated time is lesser than the cleaning interval are getting removed during cleaning logs Key: SPARK-26186 URL: https://issues.apache.org/jira/bro

[jira] [Updated] (SPARK-26185) add weightCol in python MulticlassClassificationEvaluator

2018-11-27 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-26185: --- Description: https://issues.apache.org/jira/browse/SPARK-24101 added weightCol in MulticlassClassifi

[jira] [Assigned] (SPARK-26186) In progress applications with last updated time is lesser than the cleaning interval are getting removed during cleaning logs

2018-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26186: Assignee: Apache Spark > In progress applications with last updated time is lesser than t

[jira] [Commented] (SPARK-26186) In progress applications with last updated time is lesser than the cleaning interval are getting removed during cleaning logs

2018-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700814#comment-16700814 ] Apache Spark commented on SPARK-26186: -- User 'shahidki31' has created a pull reques

[jira] [Assigned] (SPARK-26186) In progress applications with last updated time is lesser than the cleaning interval are getting removed during cleaning logs

2018-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26186: Assignee: (was: Apache Spark) > In progress applications with last updated time is le

[jira] [Assigned] (SPARK-26184) Last updated time is not getting updated in the History Server UI

2018-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26184: Assignee: (was: Apache Spark) > Last updated time is not getting updated in the Histo

[jira] [Commented] (SPARK-26184) Last updated time is not getting updated in the History Server UI

2018-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700816#comment-16700816 ] Apache Spark commented on SPARK-26184: -- User 'shahidki31' has created a pull reques

[jira] [Assigned] (SPARK-26184) Last updated time is not getting updated in the History Server UI

2018-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26184: Assignee: Apache Spark > Last updated time is not getting updated in the History Server U

[jira] [Commented] (SPARK-26184) Last updated time is not getting updated in the History Server UI

2018-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700815#comment-16700815 ] Apache Spark commented on SPARK-26184: -- User 'shahidki31' has created a pull reques

[jira] [Commented] (SPARK-26184) Last updated time is not getting updated in the History Server UI

2018-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700817#comment-16700817 ] Apache Spark commented on SPARK-26184: -- User 'shahidki31' has created a pull reques

[jira] [Commented] (SPARK-26186) In progress applications with last updated time is lesser than the cleaning interval are getting removed during cleaning logs

2018-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700813#comment-16700813 ] Apache Spark commented on SPARK-26186: -- User 'shahidki31' has created a pull reques

[jira] [Commented] (SPARK-21291) R bucketBy partitionBy API

2018-11-27 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700829#comment-16700829 ] Huaxin Gao commented on SPARK-21291: [~felixcheung] Is it OK with you if I modify th

[jira] [Commented] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

2018-11-27 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700836#comment-16700836 ] Adrian Wang commented on SPARK-26155: - [~Jk_Self] can you also test this on Spark 2.

[jira] [Created] (SPARK-26187) Stream-stream left outer join returns outer nulls for already matched rows

2018-11-27 Thread Pavel Chernikov (JIRA)
Pavel Chernikov created SPARK-26187: --- Summary: Stream-stream left outer join returns outer nulls for already matched rows Key: SPARK-26187 URL: https://issues.apache.org/jira/browse/SPARK-26187 Proj

[jira] [Updated] (SPARK-26187) Stream-stream left outer join returns outer nulls for already matched rows

2018-11-27 Thread Pavel Chernikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Chernikov updated SPARK-26187: Description: This is basically the same issue as SPARK-26154, but with slightly easier  re

[jira] [Created] (SPARK-26188) Spark 2.4.0 behavior breaks backwards compatibility

2018-11-27 Thread Damien Doucet-Girard (JIRA)
Damien Doucet-Girard created SPARK-26188: Summary: Spark 2.4.0 behavior breaks backwards compatibility Key: SPARK-26188 URL: https://issues.apache.org/jira/browse/SPARK-26188 Project: Spark

[jira] [Updated] (SPARK-26188) Spark 2.4.0 behavior breaks backwards compatibility

2018-11-27 Thread Damien Doucet-Girard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damien Doucet-Girard updated SPARK-26188: - Description: My team uses spark to partition and output parquet files to amazon

[jira] [Updated] (SPARK-26188) Spark 2.4.0 behavior breaks backwards compatibility

2018-11-27 Thread Damien Doucet-Girard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damien Doucet-Girard updated SPARK-26188: - Description: My team uses spark to partition and output parquet files to amazon

[jira] [Updated] (SPARK-26188) Spark 2.4.0 behavior breaks backwards compatibility

2018-11-27 Thread Damien Doucet-Girard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damien Doucet-Girard updated SPARK-26188: - Description: My team uses spark to partition and output parquet files to amazon

[jira] [Created] (SPARK-26189) Fix the doc of unionAll in SparkR

2018-11-27 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26189: --- Summary: Fix the doc of unionAll in SparkR Key: SPARK-26189 URL: https://issues.apache.org/jira/browse/SPARK-26189 Project: Spark Issue Type: Documentation C

[jira] [Updated] (SPARK-26189) Fix the doc of unionAll in SparkR

2018-11-27 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26189: Description: We should fix the doc of unionAll in SparkR. See the discussion: https://github.com/apache/sp

[jira] [Commented] (SPARK-26189) Fix the doc of unionAll in SparkR

2018-11-27 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700880#comment-16700880 ] Xiao Li commented on SPARK-26189: - cc [~huaxingao] > Fix the doc of unionAll in SparkR

[jira] [Updated] (SPARK-26188) Spark 2.4.0 behavior breaks backwards compatibility

2018-11-27 Thread Damien Doucet-Girard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damien Doucet-Girard updated SPARK-26188: - Description: My team uses spark to partition and output parquet files to amazon

[jira] [Updated] (SPARK-26188) Spark 2.4.0 behavior breaks backwards compatibility

2018-11-27 Thread Damien Doucet-Girard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damien Doucet-Girard updated SPARK-26188: - Description: My team uses spark to partition and output parquet files to amazon

[jira] [Updated] (SPARK-26188) Spark 2.4.0 Partitioning behavior breaks backwards compatibility

2018-11-27 Thread Damien Doucet-Girard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damien Doucet-Girard updated SPARK-26188: - Summary: Spark 2.4.0 Partitioning behavior breaks backwards compatibility (was:

[jira] [Created] (SPARK-26190) SparkLauncher: Allow users to set their own submitter script instead of hardcoded spark-submit

2018-11-27 Thread Gyanendra Dwivedi (JIRA)
Gyanendra Dwivedi created SPARK-26190: - Summary: SparkLauncher: Allow users to set their own submitter script instead of hardcoded spark-submit Key: SPARK-26190 URL: https://issues.apache.org/jira/browse/SPARK

[jira] [Updated] (SPARK-26188) Spark 2.4.0 Partitioning behavior breaks backwards compatibility

2018-11-27 Thread Damien Doucet-Girard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damien Doucet-Girard updated SPARK-26188: - Description: My team uses spark to partition and output parquet files to amazon

[jira] [Commented] (SPARK-26190) SparkLauncher: Allow users to set their own submitter script instead of hardcoded spark-submit

2018-11-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700890#comment-16700890 ] Marcelo Vanzin commented on SPARK-26190: Based solely on what you wrote here, I'

[jira] [Updated] (SPARK-26190) SparkLauncher: Allow users to set their own submitter script instead of hardcoded spark-submit

2018-11-27 Thread Gyanendra Dwivedi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gyanendra Dwivedi updated SPARK-26190: -- Description: Currently the script name is hard-coded in the 'createBuilder()' method o

[jira] [Comment Edited] (SPARK-26190) SparkLauncher: Allow users to set their own submitter script instead of hardcoded spark-submit

2018-11-27 Thread Gyanendra Dwivedi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700892#comment-16700892 ] Gyanendra Dwivedi edited comment on SPARK-26190 at 11/27/18 7:23 PM: -

[jira] [Commented] (SPARK-26190) SparkLauncher: Allow users to set their own submitter script instead of hardcoded spark-submit

2018-11-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700896#comment-16700896 ] Marcelo Vanzin commented on SPARK-26190: Just run your script without SparkLaunc

[jira] [Commented] (SPARK-26190) SparkLauncher: Allow users to set their own submitter script instead of hardcoded spark-submit

2018-11-27 Thread Gyanendra Dwivedi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700892#comment-16700892 ] Gyanendra Dwivedi commented on SPARK-26190: --- [~vanzin] I have a wrapper custom

[jira] [Commented] (SPARK-26190) SparkLauncher: Allow users to set their own submitter script instead of hardcoded spark-submit

2018-11-27 Thread Gyanendra Dwivedi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700909#comment-16700909 ] Gyanendra Dwivedi commented on SPARK-26190: --- [~vanzin] If it was so easy to ru

[jira] [Updated] (SPARK-26190) SparkLauncher: Allow users to set their own submitter script instead of hardcoded spark-submit

2018-11-27 Thread Gyanendra Dwivedi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gyanendra Dwivedi updated SPARK-26190: -- Description: Currently the script name is hard-coded in the 'createBuilder()' method o

[jira] [Comment Edited] (SPARK-26190) SparkLauncher: Allow users to set their own submitter script instead of hardcoded spark-submit

2018-11-27 Thread Gyanendra Dwivedi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700909#comment-16700909 ] Gyanendra Dwivedi edited comment on SPARK-26190 at 11/27/18 7:44 PM: -

[jira] [Comment Edited] (SPARK-26190) SparkLauncher: Allow users to set their own submitter script instead of hardcoded spark-submit

2018-11-27 Thread Gyanendra Dwivedi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700909#comment-16700909 ] Gyanendra Dwivedi edited comment on SPARK-26190 at 11/27/18 7:51 PM: -

[jira] [Updated] (SPARK-26190) SparkLauncher: Allow users to set their own submitter script instead of hardcoded spark-submit

2018-11-27 Thread Gyanendra Dwivedi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gyanendra Dwivedi updated SPARK-26190: -- Description: The improvement request is for improvement in the SparkLauncher class whi

[jira] [Commented] (SPARK-26190) SparkLauncher: Allow users to set their own submitter script instead of hardcoded spark-submit

2018-11-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700937#comment-16700937 ] Marcelo Vanzin commented on SPARK-26190: If you need to run things before spark-

[jira] [Updated] (SPARK-26189) Fix the doc of unionAll in SparkR

2018-11-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-26189: -- Priority: Minor (was: Major) > Fix the doc of unionAll in SparkR > -

[jira] [Created] (SPARK-26191) Control number of truncated fields

2018-11-27 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26191: -- Summary: Control number of truncated fields Key: SPARK-26191 URL: https://issues.apache.org/jira/browse/SPARK-26191 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-26190) SparkLauncher: Allow users to set their own submitter script instead of hardcoded spark-submit

2018-11-27 Thread Gyanendra Dwivedi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701136#comment-16701136 ] Gyanendra Dwivedi commented on SPARK-26190: --- [~vanzin] I am sorry I am not abl

[jira] [Created] (SPARK-26192) MesosClusterScheduler reads options from dispatcher conf instead of submission conf

2018-11-27 Thread Martin Loncaric (JIRA)
Martin Loncaric created SPARK-26192: --- Summary: MesosClusterScheduler reads options from dispatcher conf instead of submission conf Key: SPARK-26192 URL: https://issues.apache.org/jira/browse/SPARK-26192

[jira] [Assigned] (SPARK-26191) Control number of truncated fields

2018-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26191: Assignee: (was: Apache Spark) > Control number of truncated fields >

[jira] [Commented] (SPARK-26191) Control number of truncated fields

2018-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701139#comment-16701139 ] Apache Spark commented on SPARK-26191: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-26190) SparkLauncher: Allow users to set their own submitter script instead of hardcoded spark-submit

2018-11-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701140#comment-16701140 ] Marcelo Vanzin commented on SPARK-26190: bq. Can you give me one good reason tha

[jira] [Resolved] (SPARK-26190) SparkLauncher: Allow users to set their own submitter script instead of hardcoded spark-submit

2018-11-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26190. Resolution: Won't Fix I'm closing this for now until I see a better use case. Seems to be

[jira] [Assigned] (SPARK-26191) Control number of truncated fields

2018-11-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26191: Assignee: Apache Spark > Control number of truncated fields > ---

  1   2   >